AI screen marks 250,000 cancer papers as suspicious
A BMJ study says a language model found paper mill-like writing patterns in a large set of cancer research papers published from 1999 to 2024.
By Priya Raghavan · Science Reporter
3 min read
A machine-learning screen of cancer literature has marked more than 250,000 papers as potentially linked to paper mills, according to a study published in The BMJ. The findings matter for cancer science because fabricated or low-quality studies can distort evidence used in trials, drug development and patient care, Queensland University of Technology researcher Adrian Barnett said.
The study reviewed 2.6 million cancer research papers published between 1999 and 2024, QUT said. Barnett, a professor in QUT’s School of Public Health and Social Work and the Australian Centre for Health Services and Innovation, led the work with international collaborators.
The researchers were looking for writing patterns similar to those found in papers that had already been retracted over suspected fabrication, according to QUT. They reported that more than 250,000 papers matched those patterns closely enough to warrant scrutiny.
How the screening worked
Paper mills sell authorship slots and, in some cases, entire manuscripts, Barnett said. QUT said such papers can include recycled text, awkward phrasing, made-up data or manipulated images.
Barnett said the operations are likely using template-based writing that large language models can detect by studying textual patterns. The research team trained a language model known as BERT to find recurring signals in papers associated with suspected paper mill output, according to QUT.
When the model was tested against verified examples, it identified suspicious papers correctly 91% of the time, QUT said. Barnett compared the system to a spam filter for science, designed to alert editors when a manuscript resembles work already linked to fraud concerns.
The researchers stressed that the tool does not prove misconduct by itself, according to QUT. A flagged paper should be treated as a warning sign for human review, rather than a finding that the authors fabricated results.
Growth across journals and topics
The BMJ study found that the share of flagged papers rose over two decades, from about 1% in the early 2000s to more than 16% at its 2022 peak, QUT said. The pattern appeared across thousands of journals, including titles from major publishers and journals with strong reputations.
QUT said suspicious papers were especially common in molecular cancer biology and early-stage laboratory research. The study also found higher rates of flagged work in research on gastric, liver, bone and lung cancers, according to the university.
- The analysis covered cancer papers published from 1999 through 2024.
- More than 250,000 papers showed text patterns similar to previously retracted work suspected of fabrication.
- Three scientific journals are testing the tool in editorial screening, according to QUT.
The journals using the system are testing it before manuscripts go to outside peer reviewers, QUT said. The aim is to help editors decide which papers need closer checks before they enter the formal review process.
The research team also plans to adapt the method for other scientific fields, according to QUT. The university said the model’s accuracy is expected to improve as more confirmed examples of paper mill activity become available.
Barnett said cancer research feeds into clinical trials, drug development and care decisions. If fabricated studies enter that evidence base, he said, they can mislead scientists and slow progress for patients.
This story draws on original reporting from ScienceDaily.