News reaction: AI security accords cannot replace strong security hygiene

New AI security accords formalise cooperation between the U.S. government and frontier AI labs, but questions remain over whether they will address wider AI security weaknesses.

“The U.S. government and frontier labs have been coordinating around high-capability models for some time now (e.g. frontier security-capable models have required sign-off from the U.S. government for several months), and the accords provide another formalization around an arrangement that may largely go unchanged. However, many policy-level AI discussions confuse alignment training, the process in which an AI model is rendered less likely to misbehave, with proper AI system security, which is implemented in the controls around the model based on its implementation context (e.g. sandboxes, access controls, context window trust management, etc.). These accords are unlikely to affect the latter category in meaningful ways, especially since the industry has found it quite difficult to adapt to AI's security paradigm.

“Franky, I don't think another round of voluntary cooperation with the government or even regulatory requirements are going to solve the security hygiene problems that have been rampant with the use of AI within frontier labs and businesses alike (and I think it overstates the risk these labs actually pose to the security landscape). We should treat these issues like security problems, not AI problems, and enforce laws already on the books related to organizational security requirements. Businesses should stop relying on frontier labs to rescue them from ineffective security posture. Much of this ongoing discussion regarding U.S. involvement distracts from the fact that the number one contributor to an organization's security is that very organization, and continuing to turn to the labs to correct decisions made by an internal security team will only lead to new vulnerabilities later down the line.”

David Brauchler biography

David Brauchler is a Technical Director and AI/ML Security Lead at NCC Group in Dallas, specialising in offensive security, application testing and emerging technologies. He is also guest lecturer across several universities and holds a master’s degree in Security Engineering alongside OSCP certification. 

David has published research with BlackHat on securing IJTAG hardware standards and is regularly featured in leading global technology and cybersecurity publications for his expert insights. His technical strengths span web and mobile application testing, threat modelling, network penetration testing and advanced security research. Passionate about innovation, he closely follows developments in AI/ML, blockchain and cuttingedge computing hardware.