STANFORD, California—An Anthropic researcher recently resigned over concerns that leading artificial intelligence (AI) labs are racing toward model capabilities they cannot control, setting off a chorus of warnings from industry leaders for the AI race to slow. Those warnings follow a series of AI cybersecurity crises this summer involving agentic AI systems created by frontier labs.
Against that backdrop, the White House in August finalized a framework for testing frontier closed-source AI models for safety and cybersecurity risks. However, the criteria still have not been made public and were only shared directly with AI companies. The secrecy surrounding the framework has drawn criticism over transparency. More broadly, it raises a practical question of whether the government can provide sufficient oversight of frontier AI models.
What has been made public is that this cybersecurity framework applies only to closed-source models—models that do not release their underlying source code or parameters—and only to the most powerful of those models, which a June executive order preemptively refers to as “covered frontier models.” Though the criteria for what counts as a covered frontier model also have not been made public, they will presumably include OpenAI’s and Anthropic’s most advanced models. Perhaps most notable, the framework is voluntary: frontier AI labs can submit their models for review pre-release. The government will then have just thirty days to evaluate the model and determine whether it poses any safety or cybersecurity risks.
Although many questions remain about what exact safety and cybersecurity measures the frontier models will be tested for, the more critical question is who in the government will execute such evaluations and, more broadly, whether the government is capable of carrying out this kind of direct oversight within such a short timeframe. To the first question, the threshold for what models count as covered frontier models is supposed to be decided in collaboration between the National Security Agency, the National Cyber Director, the Office of Science and Technology Policy (OSTP), the Cybersecurity and Infrastructure Security Agency (CISA), and the Department of War. But there is not yet any public information about which subject-matter experts or offices will conduct the tests. In an emerging research field that faces technical challenges and verification limits itself, once a model is determined safe or unsafe, the credibility of that evaluation will depend on the transparency needed to verify findings and the existence of a robust technical AI workforce to conduct it in the first place.
Why direct government auditing and voluntary approaches fall short
While the White House’s AI cybersecurity framework is a first step toward federal testing, its current split between government oversight and a voluntary framework sets it up for failure. Both have shortcomings in feasibility and effectiveness, which are two key criteria identified by experts at the Human-Centered Artificial Intelligence Institute at Stanford University for assessing whether regulatory frameworks are aligned with their intended AI policy goals.
First, direct government auditing is currently infeasible because the government faces three technical capacity gaps: a technical talent gap, a software knowledge gap, and a difficulty in recruiting and retaining technical expertise. Those constraints are compounded by the concentration of frontier AI expertise and research in private industry. According to a 2023 Massachusetts Institute of Technology study, “Roughly 70% of individuals with a PhD in artificial intelligence get jobs in private industry, compared with 20% two decades ago.” In short, the federal government does not have the workforce needed to evaluate frontier AI systems independently, and if the government defers to industry expertise to help evaluate models, it could create conflicts of interest.
Second, voluntary frameworks would continue many of the self-regulatory and often inadequate approaches that have characterized much of the AI governance landscape to date. Companies have maintained internal trust and safety teams, created their own AI principles, founded their own AI safety organizations, joined coalitions, formed partnerships, and adopted voluntary commitments made in collaboration with government. However, these voluntary frameworks are not enforceable. Recent cybersecurity crises involving frontier AI models, including the Fable 5 shutdown and OpenAI and Anthropic AI agents hacking into external companies, reveal the limits of voluntary commitments alone. Additionally, a voluntary auditing approach, in which AI companies internally audit or choose auditors without centralized oversight, would risk auditee-auditor conflicts of interest and replicate the kind of structural failures that reforms in the financial auditing sector were designed to address.
The White House’s AI cybersecurity framework, as a combination of direct government evaluation and voluntary submission, risks being both opaque and ineffective by inheriting the drawbacks of each regulatory approach.
Creating an oversight board and auditing market
Federally mandated oversight is still needed, but it should be designed to promote transparency and independence. One option is an oversight board that regulates a third-party auditing market, an approach consistent with the Trump administration’s stated concern about avoiding overly burdensome regulation. Building on its AI policy efforts to date, the White House should next adopt a framework modeled after regulatory structures created in financial auditing.
A new oversight board for AI auditing should be loosely modeled after the Public Company Accounting Oversight Board (PCAOB), which was created by the Sarbanes-Oxley (SOX) Act of 2002 after a series of major corporate fraud and accounting scandals. The Artificial Intelligence Auditing Oversight Board (AIAOB), as it could be called, would balance governmental oversight with a third-party auditing market.
Just as the PCAOB is overseen by the Securities and Exchange Commission (SEC), the AIAOB would consist of board members appointed by the federal government, such as the Office of Science and Technology Policy. And just as the PCAOB registers public accounting firms and establishes auditing standards, the responsibilities of the AIAOB would be to:
- Register and license AI auditing firms that prepare audit reports for AI companies.
- Establish auditing and evaluation standards in collaboration with agencies like the National Institute of Standards and Technology.
- Inspect registered AI auditing firms’ audits.
- Investigate and discipline registered AI auditing firms for violations of specified laws, rules, or conduct standards to ensure independence.
Some industry leaders have already called for similar frameworks for standards bodies. Crucially, however, the AIAOB would be entirely self-funded by the license fees from AI auditing firms, which would eliminate its dependence on congressional appropriations or AI industry funding. This approach would ensure independence and incentivize a competitive AI auditing market. That market could also offer salaries that attract technical talent into AI safety at private AI auditing firms. Some AI evaluation nonprofits, for example, have already partnered with frontier companies to conduct risk assessments.