Anthropic proves security properties in 123 open-source libraries with Claude

Summary

In a groundbreaking initiative, researchers have utilized the AI model Claude to achieve formal verification of security properties in over 123 real-world open-source software libraries, significantly speeding up the verification process that historically required years of expert effort. This project, conducted in collaboration with Theorem, represents a substantial shift in the scalability of formal verification, now allowing such proofs to be generated at the assembly level in just weeks. Moreover, the process has revealed latent bugs in mature and extensively tested libraries that traditional methods had overlooked, thereby enhancing the overall security of software development. The effort also demonstrates that sharing formal models and proof components across various projects can reduce repetitive work, suggesting a promising future for the integration of AI in securing software.

Analysis

Claude: Claude is Anthropic's frontier large language model designed for advanced reasoning and code-related tasks. In this news, it is the primary AI system used to autonomously generate formal proofs of security properties for over one hundred real-world open-source libraries at the RISC-V assembly level. The work highlights how Claude can also identify and patch long-standing bugs during the proof generation process. Theorem: Theorem is a collaborator focused on formal verification tools and semantics for software analysis. In this news, it partnered with Anthropic to develop and apply a pipeline that translates RISC-V binaries and Linux process models into mathematical specifications for Claude to prove. Theorem is also publishing a Claude skill and offering verification services for other open-source projects. Luna Tong: Luna Tong is a researcher contributing to AI-assisted formal verification efforts. In this news, she is listed as a co-author of the study demonstrating large-scale use of Claude to prove security properties in production open-source software. Ava Howell: Ava Howell is a researcher contributing to AI-assisted formal verification efforts. In this news, she is listed as a co-author of the study demonstrating large-scale use of Claude to prove security properties in production open-source software. Chris Rohlf: Chris Rohlf is a researcher contributing to AI-assisted formal verification efforts. In this news, he is listed as a co-author of the study demonstrating large-scale use of Claude to prove security properties in production open-source software. Jason Gross: Jason Gross is a researcher contributing to AI-assisted formal verification efforts. In this news, he is listed as a co-author of the study demonstrating large-scale use of Claude to prove security properties in production open-source software. Keane Lucas: Keane Lucas is a researcher contributing to AI-assisted formal verification efforts. In this news, he is listed as a co-author of the study demonstrating large-scale use of Claude to prove security properties in production open-source software. Sami Jawhar: Sami Jawhar is a researcher contributing to AI-assisted formal verification efforts. In this news, he is listed as a co-author of the study demonstrating large-scale use of Claude to prove security properties in production open-source software. Aengus Lynch: Aengus Lynch is a researcher contributing to AI-assisted formal verification efforts. In this news, he is listed as a co-author of the study demonstrating large-scale use of Claude to prove security properties in production open-source software. Luke Champine: Luke Champine is a researcher contributing to AI-assisted formal verification efforts. In this news, he is listed as a co-author of the study demonstrating large-scale use of Claude to prove security properties in production open-source software. Dino Distefano: Dino Distefano is a researcher contributing to AI-assisted formal verification efforts. In this news, he is listed as a co-author of the study demonstrating large-scale use of Claude to prove security properties in production open-source software. Nicholas Carlini: Nicholas Carlini is a researcher contributing to AI-assisted formal verification efforts. In this news, he is listed as a co-author of the study demonstrating large-scale use of Claude to prove security properties in production open-source software. Zygi Straznickas: Zygi Straznickas is a researcher contributing to AI-assisted formal verification efforts. In this news, he is listed as a co-author of the study demonstrating large-scale use of Claude to prove security properties in production open-source software. Evyatar Ben Asher: Evyatar Ben Asher is a researcher contributing to AI-assisted formal verification efforts. In this news, he is listed as a co-author of the study demonstrating large-scale use of Claude to prove security properties in production open-source software. Harshikaa Agrawal: Harshikaa Agrawal is a researcher contributing to AI-assisted formal verification efforts. In this news, she is listed as a co-author of the study demonstrating large-scale use of Claude to prove security properties in production open-source software. Rajashree Agrawal: Rajashree Agrawal is a researcher contributing to AI-assisted formal verification efforts. In this news, she is listed as a co-author of the study demonstrating large-scale use of Claude to prove security properties in production open-source software. Nickolai Zeldovich: Nickolai Zeldovich is a researcher contributing to AI-assisted formal verification efforts. In this news, he is listed as a co-author of the study demonstrating large-scale use of Claude to prove security properties in production open-source software. Proof Reusability: Shared formal models and proof components allow verification efforts across different programs to build on each other, reducing redundant work as more codebases are formalized. Bug Discovery Benefit: The process of attempting formal proofs with AI can uncover latent bugs in mature, heavily tested libraries that traditional testing methods have missed. AI Verification Scaling: Frontier AI models are enabling the generation of formal proofs for real-world software at assembly level across many open-source projects in weeks rather than years of expert effort.

Categories

predictionspredictions:techaimachine_learningtech
View Original Tweet