CritPt v2 launches with revised physics challenges and code execution

CritPt: CritPt is a benchmark consisting of research-level physics challenges created by active physicists to evaluate AI reasoning on unpublished problems across multiple subfields. The project maintains and revises the dataset in response to community feedback, with v2 introducing agentic evaluation using code execution tools. It is directly relevant as the subject of the recent launch on Artificial Analysis, where the team revised all challenges for improved stability and accuracy. OpenAI: OpenAI develops advanced AI models including the GPT series used for complex reasoning tasks. Recent releases from the company are featured prominently in evaluations of physics problem-solving capabilities. This positions OpenAI models at the forefront of performance on the revised CritPt v2 benchmark. Haopeng: Haopeng is a researcher affiliated with UIUC involved in the CritPt physics benchmark development. Haopeng took part in the recent team effort to revise challenges and enhance the grader for v2. This collaboration ensures the benchmark remains relevant for frontier AI assessments. MiniHui Zhu: MiniHui Zhu is a researcher contributing to the CritPt project on physics benchmarks for AI evaluation. Zhu participated in the expert review and revisions for the v2 release alongside collaborators. This involvement helped refine challenges based on community input and grader improvements. Minyang Tian: Minyang Tian is a researcher who collaborates on the CritPt benchmark for testing AI on advanced physics tasks. Tian contributed to the v2 revision process with the broader team. The work supports ongoing updates to the benchmark's challenges and evaluation methods. Benchmark Update: CritPt v2 incorporates revisions to all challenges following expert and community feedback, enabling agentic model evaluation with code execution in a sandbox environment. AI Performance Trends: Recent model releases demonstrate strong results on research-level physics reasoning tasks within the updated CritPt framework.