Agent Arena updates Bash Recovery to Bash Reliability, ranks top models
by@arena
Summary
In the latest update for Agent Arena, the evolution of the Bash Recovery signal to Bash Reliability has refined how model performance is assessed, impacting the current leaderboard. Leading the rankings is Claude Opus 5.5 from @AnthropicAI, followed closely by OpenAI’s GPT-6.1 Sol. This new metric focuses solely on agent-caused errors, rewarding models for accurate first-time command use and reflecting real user interactions, as opposed to synthetic benchmarks. The top 14 slots on the leaderboard are held by proprietary models, with the best open-weights model, GLM 5.2, sitting in fifteenth place.
Analysis
Gemini: Gemini is Google DeepMind's Gemini 4 Argon (High) model. It occupies fifth place on the Agent Arena leaderboard. OpenAI: OpenAI creates frontier AI systems including the GPT model line. Its offerings GPT-6.1 Sol (Max) and GPT-6 Astra (Max) occupy second and fourth places on the updated Agent Arena leaderboard. GPT-6.1: GPT-6.1 is OpenAI's GPT-6.1 Sol (Max) model. It ranks second on the Agent Arena leaderboard after the Bash Reliability signal update. Anthropic: Anthropic is an AI research company that develops advanced language models, primarily the Claude series. In the news, it holds the top position on the Agent Arena leaderboard with Claude Opus 5.5 (Max) and third place with Claude Fable 5.1 (Max). Claude Opus: Claude Opus refers to Anthropic's Claude Opus 5.5 (Max) model. It currently leads the Agent Arena leaderboard based on real-user agentic task performance. GPT-6 Astra: GPT-6 Astra is OpenAI's GPT-6 Astra (Max) model. It ranks fourth on the Agent Arena leaderboard. Claude Fable: Claude Fable is Anthropic's Claude Fable 5.1 (Max) model. It holds third place on the Agent Arena leaderboard. Google DeepMind: Google DeepMind is the AI research organization behind the Gemini model family. Gemini 4 Argon (High) ranks fifth on the Agent Arena leaderboard for agentic capabilities. Evaluation Basis: Agent Arena derives all rankings from live user sessions on Arena.ai rather than synthetic benchmarks, incorporating explicit confirmations, sentiment labels, and steering effort. Signal Refinement: Bash Reliability replaces the prior Bash Recovery metric by evaluating only agent-caused errors across bash calls, crediting clean first-time command use. Leaderboard Composition: The top 14 positions on the Agent Arena leaderboard are held exclusively by proprietary models, with the highest open-weights model at fifteenth place.
Categories
aitechai_agentsmachine_learningvirtuals