Flawed COVID-19 Papers Cited an Average of 53 Times per Paper in Research and Media

An investigation reported that even after papers were retracted, they continued to be cited. ImageFlow/Shutterstock

An investigation into COVID-19 research found that among the more than 270,000 papers that have been published since the start of the pandemic, 212 retracted papers were cited 2,697 times, with a median of seven times and an average of 53 times per paper.

A retracted study linking the antimalarial drug Hydroxychloroquine to an increased risk of mortality and heart arrhythmia was the most cited paper with 1,360 citations at the time of data extraction.

Publishing processes were often compromised with COVID-19, according to the co-author of the investigation and director of Cochrane Australia Steve McDonald.

“We saw this push to get information out quickly, and with many more people doing and rapidly publishing COVID research, there’s been a spike in retractions,” senior research fellow McDonald said.

Eighteen percent of citations from retracted papers were critical and “may have directly impacted patient care,” the authors wrote in their paper (pdf).

Despite the retractions, the damage has been done as the research has already been cited by other researchers in the field, spawning more citations.

It had also been reported on in the media, changing the direction of policymaking, including social distancing measures, travel restrictions, and infection control measures which introduced a myriad of disruptions.

Healthcare worker Dante Hills (L) passes paperwork to a woman in a vehicle at a COVID-19 testing site outside of Marlins Park, in Miami, on July 27, 2020. Lynne Sladky/AP Photo

Retractions safeguard against error and misconduct, stopping research from impacting scientific ideas and clinical practice, and are crucial to preserving scientific integrity.

However, even high-profile medical journals became vulnerable to haste during the COVID-19 pandemic, the report found.

This comes after hundreds of COVID-19 papers have been removed due to compromising ethical standards, such as using fake or suspect patient data, and were either withdrawn by the prominent medical journals that published them or removed altogether.

READ MORE

• Over 300 COVID-19 Papers Withdrawn for Not Meeting Standards of Scientific Soundness

Alternative Treatment Soup

Evidence of research papers changing the trajectory of governmental decision-making can be found in the case of monoclonal antibodies, which triggered controversy after several scientists said certain brands of the key COVID-19 treatment would not work for the Omicron variant.

A few months after preprints written by those scientists were published, the monoclonal antibody “sotrovimab” lost Emergency Use Authorisation, causing policymakers to move on to COVID-19 drugs like remdesivir.

The U.S. Food and Drug Administration (FDA) later expanded remdesivir’s authorisation to outpatient treatment and pediatric patients.

Eventually, pandemic response critics put monoclonal antibodies into the alternative treatment group, a place where critics say is automatically stifled or publicly scrutinised as unsafe or ineffective.

A Regeneron monoclonal antibody infusion bag. Joe Cavaretta/South Florida Sun-Sentinel via AP

Another significant example of governments and the World Health Organisation acting on suspected fraudulent and unverifiable data is the hydroxychloroquine study.

Published in the Lancet on May 2020, the study concluded that the drugs hydroxychloroquine and chloroquine increased the chances of death from COVID-19 at a time when the drug was largely untested.

The authors of the study claimed to obtain medical records of nearly 100,000 patients from hundreds of hospitals on six continents, but more than 100 scientists analysed the findings and found major issues, including inadequate adjustment for variables, a lack of ethics review, and numbers that don’t appear to add up regarding patients in Australia and Africa.

The paper was retracted after two weeks, but it had already shaken the scientific world, prompting the World Health Organization and French authorities to suspend clinical trials testing hydroxychloroquine against COVID-19.

While some studies have shown patients experiencing heart problems when taking hydroxychloroquine or chloroquine, the drugs were approved decades ago and have been used historically by people against malaria and other ailments with little concern.

Why Did This Happen?

McDonald said that preprints—which allow authors to publish early versions of research papers before peer review or journal publications—resulted in dubious COVID-19 science, for academics were able to exploit loopholes in the process.

Further, retracted studies weren’t treated with due severity, McDonald said.

“In theory, when people cite retracted studies, they should be citing them in a critical way, alluding to the fact that these papers have been retracted because the research is unreliable,” he said.

“But what we found was that actually in a lot of these cases, even if the author team who cites the retracted paper were doing so long after the paper had been retracted, they weren’t citing it as a retraction.

“They were using it as evidence that ‘this particular intervention is effective’, or ‘there’s nothing wrong with that research’. So they were uncritically citing retracted papers.”

COVID-19 Research Volume Dwarf Other Pandemics

Different sources have stated that some 90,000 to 450,000 COVID-19 papers have been lodged online since the start of the pandemic, outstripping that of other pandemics “by orders of magnitude.”

One source said nearly 28,000 COVID-19 research papers were published in 2020, rising to nearly 68,000 in both 2021 and 2022, whereas another study quotes 404,541 papers from 2020 to 2022.

The Institute for Scientific Information examined the evolution of research across five pandemics—SARS, MERS, H1N1, Zika virus, and COVID-19.

They found that only H1N1 came close to COVID-19 in numbers, peaking at about 1,300 papers in 2011.

McDonald said the pandemic has exposed frailties in scientific publishing that should serve as a warning to the medical science community.

“Blindly citing papers—irrespective of where they’re published—without first assessing their reliability or retraction status can falsely elevate poor and possibly fraudulent research, potentially harming the very people the research should be helping,” he said.