Anthropic announced that it has begun implementing an independent oversight model granting six employees of the organization METR access similar to that of company employees to the devices, office spaces, tools, and internal systems needed to assess the risks of advanced artificial intelligence models. The company says METR will publish the results of its assessments of risk levels, incidents, and practices, without Anthropic having veto power over publication.
The move follows Anthropic’s pledge last month to strengthen independent third-party assessment, and its signing of the “White House Agreement on Superintelligence,” which includes a commitment by companies to cooperate with independent external auditors or assessors. Over the past year, Anthropic worked with METR on several reviews of its risk reports, while also collaborating with SecureBio, Accenture, and Faculty, and holding initial discussions with other organizations, including Hugging Face.
What Does Employee-Like Access Mean?
According to Anthropic, employees are encouraged to share information with the METR team as they would with a colleague working on alignment risks. This includes access roughly equivalent to that available to internal risk assessment teams, except for specific categories such as customer data, information protected by attorney-client privilege, and certain information related to model architecture.
Anthropic cannot prevent METR from publishing its observations or assessments, but it may request that information falling within narrowly defined, predetermined categories be withheld. In return, METR may publicly describe the withholding process and state whether it considers it justified, without disclosing the withheld information. Participation is also limited to a small team working under enforceable agreements, with the rest of METR’s employees isolated from this task.
Three Levels of Independent Assessment
Anthropic is working to establish a three-tier model for partnerships with assessors:
- Level One: Limited-scope investigations, with Anthropic determining the level of access, and the output published only with its approval.
- Level Two: Broader access and the right to publish without prior approval, while Anthropic retains the right to request the removal of certain categories of information.
- Level Three: METR’s current arrangements, including employee-like access and publication rights similar to those under Level Two.
SecureBio is currently reviewing the chemical and biological weapons chapter in Anthropic’s August risk report, after completing a pilot review of an earlier report and publishing its results. The partnership with Accenture and Faculty includes assessing and testing models through red teaming, conducting alignment assessments, testing safeguards, and building joint capabilities.
Why Does This Change Matter?
Anthropic acknowledges that the company knows more about its systems and risks than an external observer can know, but it cannot publicly disclose every detail because of misuse risks or commercial sensitivity, including information about computing and training. The model therefore aims to bridge the gap between what independent assessors see and what the company knows internally, while giving assessors genuine latitude to publish their conclusions.
Nevertheless, the experiment remains limited to a small team and governed by access agreements and confidentiality controls, while some other partnerships are still at an early stage and have narrower publication rights. The model’s effectiveness will be tied to the assessors’ independence, the breadth of information available to them, and their practical ability to publish findings that are unfavorable to the company—points that Anthropic said it is still learning how to organize.