Google Deep Mind study shows AI agents solve 49 of 50 physics problems
Summary
A recent study conducted by Google Deep Mind alongside several top universities revealed that the leading AI agent, GPT-6-Astra in Codex, successfully solved 49 out of 50 unpublished theoretical physics problems, each representing about a day's work for an expert. This evaluation method marks a shift in assessing AI capabilities, moving beyond traditional textbook problems to more closely reflect the challenges faced by researchers. The findings show that AI models, including GPT-6-Astra and Opus 5.5 in Claude Code, maintained high accuracy regardless of web search capability, indicating they possess substantial knowledge rooted in existing scientific literature.
Analysis
GPT-6-Astra: GPT-6-Astra refers to an advanced AI agent system associated with Codex that excels at processing and solving intricate scientific tasks. In the reported study, it achieved top results on unpublished physics problems when given access to web search capabilities. Google Deep Mind: Google DeepMind is an AI research lab focused on advancing machine learning systems for scientific discovery and complex problem-solving. The organization led the development of a new paper evaluating AI agents on real-world theoretical physics challenges contributed by academic experts. Opus 5.5 in Claude Code: Opus 5.5 in Claude Code is a high-performance AI model variant from the Claude family designed for code-related and analytical reasoning. The evaluation showed it maintained strong accuracy on specialized physics problems even without external search tools, indicating deep internalized knowledge of the field. Model Robustness: Leading AI systems maintain high accuracy on theoretical physics tasks whether or not they use web search, pointing to substantial embedded knowledge of scientific literature. AI Evaluation Methods: Benchmarks are evolving to incorporate unpublished problems from active research to better reflect how AI agents perform on genuine scientific calculations.
Categories
aitechmachine_learning