Anthropic partners with METR and SecureBio for risk assessment
by@CNBC
Summary
Anthropic announced on Thursday that it will collaborate with nonprofits METR and SecureBio to assess risks and safety practices within its operations, furthering its commitment to integrate independent evaluators. This move comes after CEO Dario Amodei's call last month to slow the pace of AI model development, emphasizing the need for third-party verification of safety adherence, a sentiment echoed by President Donald Trump in a recent voluntary accord encouraging such partnerships. Under a new tiered evaluation framework, METR will conduct in-depth risk investigations with third-tier access while SecureBio will operate at a first-tier level, reflecting the growing importance of independent assessments as AI safety concerns rise.
Analysis
METR: METR is a nonprofit organization specializing in model evaluation and threat research for AI systems. It has been onboarded by Anthropic as a third-tier evaluator, granting its employees access to the company's hardware, offices, and tools comparable to internal risk teams to analyze risks, incidents, and practices. METR's findings are intended for public publication. Anthropic: Anthropic is an artificial intelligence lab focused on developing advanced AI models with an emphasis on safety practices. The company recently announced partnerships with independent evaluators including METR and SecureBio to assess risks and verify adherence to safety standards. CEO Dario Amodei proposed slowing the pace of model development last month and signed a voluntary accord presented by President Donald Trump encouraging such external partnerships. SecureBio: SecureBio is a nonprofit organization working on biosecurity issues, including chemical and biological weapons risks. It is serving as a first-tier evaluator for Anthropic by reviewing the chemical and biological weapons section of the company's risk report from August. Future collaborations with broader access and publication rights may be possible. Dario Amodei: Dario Amodei is the CEO of Anthropic who has advocated for independent third-party evaluators to verify AI safety practices. He published a proposal last month to slow model development and signed a voluntary accord with President Donald Trump encouraging AI companies to partner with external auditors. Policy: President Donald Trump presented a voluntary accord in late September encouraging AI companies to partner with independent external auditors or evaluators. AI Safety: Independent evaluators have gained prominence in the AI industry as concerns mount over whether companies can adequately safeguard their models. Evaluation Framework: Anthropic is developing a tiered system for evaluators that provides varying levels of access, from narrowly scoped investigations to employee-like access with publication rights.
Categories
predictionsaitechmachine_learningpoliticsai_agents