Researcher Publishes Stats That Get Cited Back Into Same Dataset

Dr. Marcus Chen of the University of Massachusetts published a study last month estimating that 34 percent of software engineers experience imposter syndrome during their first ninety days on the job. The figure came from a survey of 127 responses collected through a Slack community for junior developers, adjusted for self-selection bias using a methodology Chen outlined in a blog post. Within six weeks, the statistic began appearing in training datasets used to fine-tune large language models, typically cited in articles about tech worker retention and burnout that were themselves incorporated into the corpus the next time those models updated.

We noticed the number had drifted by the time it appeared in the third generation of models.

By the time three major AI companies had released updated versions of their base models, Chen's 34 percent had become 37 percent, then 41 percent. The drift happened because each model generation ingested articles that cited previous models citing Chen's work, and language models trained on that material began averaging the cited variations rather than returning to the original source. Chen discovered this while preparing a follow-up study, noticing that his own finding had been cited back to him in a different form through a chain of model outputs that no human had individually verified.

"We noticed the number had drifted by the time it appeared in the third generation of models," Chen said in an email. "I can trace the path through model outputs and blog posts written about those outputs, but I cannot isolate where the adjustments started because each citation is technically accurate to what the previous model said."

The phenomenon has prompted no formal policy changes at any of the companies whose models distributed the figure. A spokesperson for one major AI lab said that identifying feedback loops in training data was an active area of research, though detecting them at scale remained computationally expensive. Two contract data-cleaning firms have begun offering services to remove self-citations from datasets, charging rates starting at forty thousand dollars per million tokens screened.

At press time, Chen's original study had been cited seventy-three times in academic papers, of which at least forty-one citations came from text that had previously been generated by the models trained on citations to his work. He is considering publishing a correction, though he said the act of publishing would likely introduce the correction itself into the cycle within weeks.