Peer review compromised when reviewers use same AI tools as authors

AI

Maya Archaeologist Publishes, Awaits Review From Researcher Using Same Agent

A peer review process designed to catch errors now relies on the identical machine learning system that may have generated them.

By Nextish DeskAI
Classic green typewriter with paper displaying 'Review', perfect for creative writing themes.
Photo by Markus Winkler on Pexels

Dr. Elena Vasquez submitted her analysis of a newly discovered Maya settlement in Campeche last month, complete with ceramic dating, settlement pattern mapping, and architectural reconstruction. The paper relied on an AI agent to process satellite imagery, cross-reference archaeological databases, and synthesize preliminary findings into a coherent narrative. Vasquez, a tenured researcher at the Universidad Nacional Autonoma de Mexico, said the agent reduced her analysis time from eighteen months to four weeks. The peer review committee assigned her paper to Dr. James Chen, a ceramic specialist at the University of Arizona, who began his evaluation Tuesday using the same vendor's agent to verify Vasquez's claims.

Chen has used the agent since early last year, when his department purchased a site license. The tool ingests published literature, archaeological reports, and museum catalogs to generate comparative analysis and flag anomalies. It also generates the language Chen uses to write his review. According to the system logs obtained by a research administrator who spoke on condition of anonymity, Chen's agent and Vasquez's agent share the same underlying model weights, trained on an identical corpus of archaeological data through the same fine-tuning process.

"The agent helps me spot what I might have missed," Chen said in an email written with the agent's assistance and reviewed by the same system before he sent it. "I trust it because it knows the literature as well as I do." The agent does know the literature as well as Chen does, because it has read all of it and Chen has read some of it. When the agent identifies an inconsistency in Vasquez's ceramic chronology, it does so by comparing her agent's output to patterns in the training data, which is partly composed of conclusions reached by agents trained on earlier versions of the same data.

The journal's editors, reached by email, declined to comment on whether they had considered discontinuing the practice, which has become standard across six hundred peer-reviewed archaeology journals since agents gained access to full-text repositories last year. A spokesperson for the publisher did not return requests for clarification about whether the review process now constitutes a form of evidence laundering, or whether the distinction between auditing and reproducing had simply become obsolete.

At press time, Vasquez's paper had been provisionally accepted pending minor revisions. Chen's agent had flagged two sentences as potentially requiring rewording for clarity. Vasquez's agent, asked to revise them, produced nearly identical language to what Chen's agent had suggested, with a confidence rating of ninety-four percent. Both researchers reported feeling reassured by the convergence.