Automating quality assurance with the same system causing failures

AI

Startup Replaces QA Team With Model Trained On Their Bug Reports

The system learned quality assurance by studying the precise failures it will now be responsible for preventing.

By Nextish DeskAI
Close-up of a PCB inspection machine in action, highlighting technology.
Photo by Peter Xie on Pexels

Vertex Labs, a Series B infrastructure startup in San Francisco, has eliminated its quality assurance department and replaced it with an AI model trained exclusively on the company's internal bug database dating back to 2019. The model, called QANet, was fed five years of incident reports, customer complaints, and post-mortems before being deployed last month to review code before release. According to a filing with the state labor board, the move affected twelve employees. The startup declined to comment on whether any of those employees had been offered positions retraining the model on new failure modes.

The QA team at Vertex Labs had grown over time to include people who had shipped products, broken systems, and learned to see failure in advance. One engineer who requested anonymity described the process as almost meditative. "You read the code and you feel the thing that will go wrong," she said. The model, by contrast, had only ever read descriptions of things that had already gone wrong. It had no embodied sense of what a feature looked like before it broke.

The first version of QANet approved a database migration that deleted customer data from the previous quarter. The incident report was later fed back into the training set. A second iteration, retrained on the incident report itself, approved an identical migration three weeks later. Vertex Labs issued a statement saying the model was performing exactly as well as a person who had only ever read about problems and never fixed one, which was the intended outcome.

At press time, Vertex Labs was recruiting for a new role: Quality Assurance Data Specialist, reporting to the Chief Product Officer, responsible for documenting all future failures in a format that QANet could learn from more efficiently. The position offered thirty-five thousand dollars annually, which was approximately forty percent of the salary of any of the twelve people who had held the job before.