Fake Publications in Biomedical Science: Red-flagging Method Indicates Mass Production, 2023, Sabel et al

Fake Publications in Biomedical Science: Red-flagging Method Indicates Mass Production

Bernhard A. Sabel, Emely Knaack, View ORCID ProfileGerd Gigerenzer, Mirela Bilc

ABSTRACT

Background Integrity of academic publishing is increasingly undermined by fake science publications massively produced by commercial “editing services” (so-called “paper mills”). They use AI-supported, automated production techniques at scale and sell fake publications to students, scientists, and physicians under pressure to advance their careers. Because the scale of fake publications in biomedicine is unknown, we developed a simple method to red-flag them and estimate their number.

Methods To identify indicators able to red-flagged fake publications (RFPs), we sent questionnaires to authors. Based on author responses, three indicators were identified: “author’s private email”, “international co-author” and “hospital affiliation”. These were used to analyze 15,120 PubMed®-listed publications regarding date, journal, impact factor, and country of author and validated in a sample of 400 known fakes and 400 matched presumed non-fakes using classification (tallying) rules to red-flag potential fakes. For a subsample of 80 papers we used an additional indicator related to the percentage of RFP citations.

Results The classification rules using two (three) indicators had sensitivities of 86% (90%) and false alarm rates of 44% (37%). From 2010 to 2020 the RFP rate increased from 16% to 28%. Given the 1.3 million biomedical Scimago-listed publications in 2020, we estimate the scope of >300,000 RFPs annually. Countries with the highest RFP proportion are Russia, Turkey, China, Egypt, and India (39%-48%), with China, in absolute terms, as the largest contributor of all RFPs (55%).

Conclusions Potential fake publications can be red-flagged using simple-to-use, validated classification rules to earmark them for subsequent scrutiny. RFP rates are increasing, suggesting higher actual fake rates than previously reported. The scale and proliferation of fake publications in biomedicine can damage trust in science, endanger public health, and impact economic spending and security. Easy-to-apply fake detection methods, as proposed here, or more complex automated methods can help prevent further damage to the permanent scientific record and enable the retraction of fake publications at scale.

 

Carl Bergstrom has debunked this article and condemned Stuart Ritchie for his “racist” interpretation in this Mastodon thread: https://fediscience.org/@ct_bergstrom/110357259338364341

Stuart Ritchie does seem biased:
https://twitter.com/user/status/1656994679227838469

 

Some interesting points in that Mastodon thread, including that plagiarism detectors are easily triggered by stock phrases in results sections, so authors have to sacrifice intelligibility to meet automated acceptance criteria.

 

I like Bergstrom's collegiate approach to challenging bad science, nothing quite like rewriting Godwin's Law to read "racist", to ensure everyone is on the same page when it comes to the challenges of scientific publishing.

Spoiler

<snark

 

Hadn’t come across Bergstrom before, but he made his name with a manifesto entitled Calling Bullshit, so I guess he’s not that fussed about being invited to the genteeler academic gatherings where you can reliably expect cakes to be served and criticism to be tactful.

 

So this was a fake publication?