Peer review broken when all participants use identical AI systems
AIMath Journal Accepts Paper It Cannot Peer Review
The editorial board discovered that every reviewer and the author were using the same language model to draft their assessments.
A peer review process at the Journal of Computational Mathematics has concluded with an acceptance decision despite the editorial team's discovery that the author, all three reviewers, and the managing editor were generating their technical commentary using the same version of Claude, according to a memo circulated Tuesday. The paper, on a novel approach to elliptic curve factorization, was accepted with minor revisions after reviewers unanimously praised its clarity and methodological rigor. Nobody involved has verified whether the underlying mathematics is sound.
We do not know if the paper is correct, but we are confident the evaluation process was thorough.
The circular validation became apparent when editor Patricia Chen, 52, of Cambridge, Massachusetts, ran the reviewer comments through a plagiarism detector and noticed that three independently written assessments contained identical phrasings in their mathematical explanations. "We do not know if the paper is correct, but we are confident the evaluation process was thorough," Dr. Chen wrote in an internal email. The journal has not retracted the decision, reasoning that the author's use of the same model constitutes a form of methodological consistency.
Across academic publishing, the problem has metastasized quietly. A survey of seventy-four journals in mathematics and physics conducted last month found that forty-one percent of recent submissions included at least one reviewer who had used a large language model to compose their assessment, according to data reviewed by the Association of Editors and Publishers. Most journals have no policy explicitly forbidding the practice, and many editors have adopted the same models themselves to screen papers for basic coherence, creating a feedback loop in which AI-generated validation confirms AI-generated work.
At press time, the Journal of Computational Mathematics was developing a new peer review protocol in which all participants would be required to use the same model and version number, thereby establishing what the journal called a "standardized epistemic framework." The paper on elliptic curves was scheduled to appear in the next issue with a note acknowledging that the entire evaluation was conducted by artificial intelligence systems and the human participants had mostly agreed with what the systems produced.