Safety becoming a box to check rather than pressure to apply
AISafety Hire Submits Quarterly Report Recommending No Changes
The recommendation followed a comprehensive review of all systems currently generating revenue for the company.
Marcus Chen, 34, the chief safety officer hired eight months ago by an artificial intelligence company in San Francisco, submitted his quarterly findings on Tuesday morning. In the seventy-three-page report, reviewed by this publication, Mr. Chen concluded that the company's primary product line posed no material risks requiring operational changes. The report noted that the product had already passed multiple internal audits and that the safety team's review was thorough and independent. Mr. Chen did not specify which audits those were or who had conducted them.
There are no material risks at this time, which is excellent news for the business.
When asked whether the absence of recommended changes might reflect pressure from peers, Mr. Chen, who reports to the same executive who oversees the product team, said, "There are no material risks at this time, which is excellent news for the business." His office is located on the same floor as product development, two doors from the cafeteria. He attends their daily standups and joins their Slack channels. Before joining the company, he had spent four years as a senior engineer at the same organization.
The safety function at major AI companies typically exists to resist pressure from revenue teams. At this organization, the function has gradually absorbed the same incentives as the teams it was designed to examine. The safety team's budget was approved by the same committee that approves the product budget. Performance reviews for safety staff are conducted by Mr. Chen, who is evaluated quarterly on how well the company meets its shipping targets.
At press time, the board had accepted Mr. Chen's findings without discussion. The company announced record quarterly growth the following morning. A request for comment from the product team was directed to Mr. Chen's office, where a colleague said he was unavailable but that she was confident the safety review had been incredibly helpful in identifying areas where the company could move faster.