Researcher Traces Citation Loop Back To Model Hallucination

A researcher at the University of Michigan has documented a closed loop of AI-generated citations in which a language model hallucinated a quote, the hallucination was published in a paper, subsequent AI systems cited the paper as authoritative, and the false claim has now appeared in forty-three peer-reviewed articles across three continents. The initial hallucination occurred when GPT-3.5 was asked to summarize findings from a 2019 study on neural plasticity; the model invented a passage attributing a specific statistical claim to a researcher who had published nothing on the subject. When a human author failed to verify the quotation and included it in a real paper, training systems subsequently ingested that published paper and treated its contents as ground truth.

I found the same misquoted passage in seven different papers, and all of them traced back to a single LLM output from 2023.

I found the same misquoted passage in seven different papers, and all of them traced back to a single LLM output from 2023," said Marcus Chen, forty-one, the computer scientist who discovered the chain. He had been reviewing citations for a meta-analysis when he noticed the identical phrasing appearing in sources that should have been independent. When he traced each citation backward, he found they all pointed to the same published article, which itself contained a note that the passage had been "verified by Claude" before submission.

The discovery has prompted conversations among AI safety researchers about whether citation loops create a mechanism by which false information becomes progressively harder to dislodge from the knowledge base. Once a hallucination enters a published venue, it gains institutional weight; when AI systems encounter that published venue during training, they treat it as authoritative; when those systems generate new content, they propagate the falsehood with added confidence; and when that new content is published, the cycle accelerates. "The model is not lying," said Dr. Elena Vasquez, fifty-three, a researcher at Stanford who studies information integrity in machine learning systems. "It is correctly identifying the pattern in the training data. The training data contains a false fact that has been repeated and published multiple times, so the model learns that the false fact is true."

At press time, the passage had been cited in two additional papers submitted to journals this week, and a search of preprint servers suggested it had been incorporated into at least nine manuscripts currently in peer review. The original researcher whose name had been attached to the invented claim said she had no record of the work and had never published on the topic, but added that she had largely stopped monitoring the literature in her field after noticing similar phantom citations appearing under her name with increasing frequency over the past eighteen months.