TRUSTWORTHY RESEARCH

Post Publication Response Concerns

  • We flagged almost 900 clinical studies in women’s health with journal editors and publishes
  • Only 13% of flagged cases reached a decision within a year. 
  • Of 263 articles with a decision (many after more than a year), 227 (86%) were labelled as problematic [152 (58%) retracted; 75 (29%) EoC].
  • Our findings suggested that the current post publication response process of publishers and journals is inefficient and ineffective in assessing and removing untrustworthy data from the medical literature.

Randomized controlled trials (RCT) are considered the highest level of evidence to support medical treatments. While it has long been taken for granted on face value that these trials are based on reliable data, emerging evidence from recent years suggests that this is not always the case [1–4]. This is worrisome, as problematic trials could find their way into meta-analyses and medical guidelines. Ultimately, this could harm individual and public health.

While preventing integrity issues before publication is ideal, addressing concerns in already published studies is needed. The Committee on Publication Ethics (COPE), a charitable organization dedicated to promoting research integrity, has established guidelines for post-publication review, which COPE member journals are obligated to follow [5]. When potential data integrity issues arise, journals and publishers must investigate and issue a formal public notice if misconduct is confirmed [6]. This step is crucial in identifying, correcting, or retracting problematic data from the literature.

To evaluate the effectiveness and efficiency of this process, we studied how editors and publishers of women’s health journals responded to concerns about studies with potentially unreliable data post publication.

We evaluated the effectiveness and efficiency of post-publication review of such studies in women’s health on a sample of 891 potentially untrustworthy papers published in women’s health journals.  We wrote to the editors and publishers about potentially untrustworthy papers in women’s health and requested an investigation according to the procedure established by the Committee of Publication Ethics (COPE). We analysed outcomes such as study characteristics, investigation outcome classed as retraction, expression of concern (EoC), correction or no wrongdoing found, and time to decision. We also report the case completion rate per journal and publisher. 

Between 7th November 2017 and 30th April 2024, we contacted the editors and publishers of these 891 potentially untrustworthy papers published in 206 different journals. This sample primarily spans publications from the last three decades, with 60% of the papers published between 2010 and 2020. These papers were published in various countries worldwide, with the majority 789 of 891 (89%) originating from the Middle East.

At present, 263 (30%) of 891 papers received an outcome, with 227 (86%) labelled as problematic [152 (58%) retracted; 75 (29%) EoC]. Significant delays were observed in academic publishers’ responses to concerns about problematic studies. A decision was made in only less than a third of flagged cases, with a median time to decision of over three years (38 months). Only 13% of flagged cases reached a decision within a year.   

Elsevier, Taylor & Francis, Springer, and Wiley-Blackwell were the four publishers with the most cases raised, with case completion rates varying between 30% and 42%. Combined retraction and EoC rates for decisions taken were high, varying between (70/71, 99%) for Taylor & Francis and (68%) for Wiley-Blackwell. Publisher Karger had the highest case completion rate (7/9, 78%), with a median time of 7 months to resolve a case. Publisher Termedia did not resolve any of the 11 flagged cases.

From a region perspective and comparison, for European-based journals, 34% of cases were completed, with 59% resulting in retraction and 28% in an Expression of Concern (EoC). In US-based journals, the completion rate was 32%, with 52% of resolved cases leading to retraction and 32% to an EoC. Notably, none of the Middle East-based journals reached a decision on the 46 flagged cases.

Our findings suggested that the current post publication response process of publishers and journals is inefficient and ineffective in assessing and removing untrustworthy data from the medical literature. We have identified limitations in the current post-publication review system for problematic papers, including delays and a lack of follow-up on many investigations. Editors and publishers should take responsibility to collaborate more effectively and adhere to reasonable timelines, to enhance the post-publication review process and combat the growing prevalence of false data in scientific literature to protect patient safety

References

  1. J. B. Carlisle, “False Individual Patient Data and Zombie Randomised Controlled Trials Submitted to Anaesthesia,” Anaesthesia 76 (2021): 472–479.
  2. L. Chambers, C. Michener, and T. Falcone, “Plagiarism and Data Falsification Are the Most Common Reasons for Retracted Publications in Obstetrics and Gynaecology,” BJOG 126 (2019): 1134–1140.
  3. J. P. A. Ioannidis, “Hundreds of Thousands of Zombie Randomised Trials Circulate Among Us,” Anaesthesia 76 (2021): 444–447.
  4. L. C. Saiz, J. Erviti, and J. Garjón, “When Authors Lie, Readers Cry and Editors Sigh,” BMJ Evidence-Based Medicine 23 (2018): 92–95.
  5. Committee on Publication Ethics, “About COPE,” 2023, https://publicatio nethics.org/ about/ our-organisation.
  6. COPE Council, “COPE Flowcharts and Infographics—Handling of Post-Publication Critiques,” 2021, https://publicationethics.org/node/50816 .

Tags:

Date:

Up next:

Before:

Leave a comment