Most Decisions to Approve or Reject New Drugs Based on Poor Clinical Data, Study Suggests

Medication displayed on shelves at the Monklands University Hospital during a visit of Scotland's irst minister (not pictured), as part of the launch of the Scottish Apprenticeship Week, in Airdrie, Lanarkshire, Scotland, on March 7, 2022. Andy Buchanan/WPA Pool/Getty Images

Researchers at the London School of Economics (LSE) have found that most decisions to approve or reject new drugs, medical procedures, and devices are not based on solid clinical data, with no improvement in the past 20 years.

The study, published on BMJ Open, examined 400 appraisals made by the National Institute of Health and Care Excellence (NICE) and the data used to inform them, finding that around two-thirds of decisions were not backed by good quality evidence.

Just 1 percent of cases were classed as having a “good” quality of evidence, with around a third considered to have an “acceptable” basis for approval or rejection. The study found the data was “consistently poor” with no improvement in its quality between 2000 and 2019.

NICE advises the NHS on both the clinical and the economic evidence for new drugs and medical devices through an independent body of specialists which make up its appraisal committee. The economic data can help give a cost-benefit analysis for new drugs and devices, taking into account quality of life.

The 400 appraisals included 384 drugs, 25 non-pharmaceutical products, 14 medical devices, six other therapies, and five surgical procedures.

The research was done in conjunction with Consilium Scientific, an independent non-profit organisation that aims to “improve clinical research through scientific rigour and transparency.”

“We found that the primary components of clinical evidence influencing NICE’s decision-making framework were of poor quality. It is essential to continue to generate robust clinical data for premarket and post-market introduction of medicines into clinical practice to ensure they deliver benefits to patients,” the study authors wrote.

Weak or insufficient evidence from poorly conducted clinical trials was often used because it was the only evidence available, researchers found.

But even when the trials had been done well, and the evidence was comprehensive, the comparators being used were often unsuitable for decision-making about whether the treatments should be rolled out, they said.

The study found that data used often did not reflect the UK population and routine treatment pathways available to patients, and that “indirect comparisons” were used in more than two-thirds of technology appraisals to establish the clinical effectiveness of treatments.

Evidence Bar ‘Continues to Be Lowered’

Quality of life data was often of poor or unacceptable quality and clarity in reporting methodology and details by both manufacturers and assessment bodies varied significantly, the researchers found.

“Since the evidence bar continues to be lowered, it is essential to have [health technology assessment] bodies and payers’ input to ensure that the generation of evidence submitted to NICE is strengthened.

“Numerous studies have raised alarm about lowering the bar for evidence requirements for regulatory approval of medicinal products, which leads to many expensive drugs with low or no efficacy entering the market,” researchers wrote.

Leeza Osipenko, founder of Consilium Scientific and coauthor of the study, told the “Trials with Maya Z“ podcast that there are many ”sub-optimal clinical trials providing sub-optimal data,“ which are submitted in order to get new drugs and treatments reviewed, and the goal of her organisation is to improve the integrity of these trials to bring about ”holistic change” to health care.

Vials containing biological samples are stored on ice to keep them fresh before being analysed to see how they are affected by chemotherapy drugs at the Cancer Research UK Cambridge Institute in Cambridge, England, on Dec. 9, 2014. Dan Kitwood/Getty Images/Cancer Research UK

‘Big Hype’ Around Cancer, Alzheimer’s Drugs

She said that at the moment, there is “big hype” around Alzheimer’s and cancer drugs, and how “families of these patients want them to have at least some hope to get that drug, and there’s a lot to be said about whether these drugs should be on the market and whether they are actually helping patients or not.”

She added that many stakeholders did not want to hear that clinical trials were poor quality “because this is not money making news. The money making news is, we have a new blockbuster, we have another [diabetes medication] Ozempic, where the future is. No one wants to hear that the trial was not good enough, that this drug is not as good as they thought it might be.

“It’s not good news for shareholders, it’s not good news for the [drug] company, it’s not good news for the regulators, especially if they approved the drug and some smart academic is ripping apart the trial behind it.”

Ms. Osipenko said that what makes clinical trials reliable—whether they demonstrate a treatment works or not—is “integrity and transparency,” and what breaks them is “academic ego and shareholder expectation.”

Jacoline Bouvy, programme director of medicines evaluation at NICE, said the researchers in the LSE study had not fully explained their methodology and disagreed with its conclusion that the body had lowered the bar for its evidence base.

She told Medscape News UK: “NICE technology appraisals are founded on principles of independence, transparency, and rigour that allow NICE to help practitioners and commissioners get the best care to patients fast, while ensuring value for the taxpayer.

“The NICE methods manual is clear that despite calling for evidence of the highest possible standard, limitations in the evidence available to inform an evaluation are likely and sometimes inevitable.

“NICE committees still have to make recommendations in such cases, and can do so when limitations in the evidence are fully described, the risk of bias is characterised, and the resulting uncertainty in the evidence is fully considered.”