Epoch's SpireClimb benchmark shows AI models winning 90% of runs at easiest difficulty in Slay the Spire

Summary

Recent developments in AI gaming have shown significant improvements as frontier models, like GPT-6.1 Sol and GPT-6 Astra, achieved win rates of over 90% on the easiest difficulty of "Slay the Spire" and around 20% on its hardest setting. This marks a notable advancement, as just six months ago, these models struggled to win even at the easiest level. The gaming community, dedicated to mastering such complex strategy games since the original game's release in 2017, has seen AI’s performance progress rapidly, positioning "Slay the Spire" as a valuable benchmark for evaluating AI capabilities in human-like contexts. Researchers highlight that while AI models are improving, they still have a long way to go before matching the win rates of top human players, who regularly achieve over 80% on the hardest difficulties.

Analysis

Epoch: Epoch is the research organization that developed and published the SpireClimb benchmark to evaluate frontier models on Slay the Spire games using a visual harness. It focuses on tracking AI progress through consistent, human-like interfaces on challenging tasks. The benchmark represents their effort to measure end-to-end computer-use and strategic capabilities in complex games. Ajeya Cotra: Ajeya Cotra is a researcher at METR who has incorporated Slay the Spire 2 win rates into forecasts of AI capabilities for 2026. Her work highlights the game's value as a testbed for strategic reasoning and long-horizon planning in AI systems. The new benchmark data offers fresh empirical input for such forecasting efforts. GPT-6 Astra: GPT-6 Astra is a frontier AI model that achieves competitive results on both Slay the Spire and its sequel in the visual benchmark setup. It shows notable capability on the sequel's hardest setting and provides baseline data for comparing model progress on familiar versus newer games. Its inclusion underscores the rapid iteration in leading AI systems tested on complex strategy tasks. GPT-6.1 Sol: GPT-6.1 Sol is a frontier AI model that leads performance on the original Slay the Spire at maximum difficulty in the SpireClimb benchmark. It demonstrates strong character-specific patterns mirroring top human playstyles while still trailing elite human win rates. The results position it as a key example of recent advances in AI game-playing and computer-use abilities. Inspect ReAct: Inspect ReAct is the agent framework used as the basis for the single-agent visual computer-use harness in the SpireClimb evaluation. It enables models to interact with the game interface, gather information, and make decisions in a manner similar to human play. The harness supports the benchmark's goal of assessing real-world applicability of model skills. Claude Fable 5: Claude Fable 5 is Anthropic's model released in June 2026, noted for improved performance on Slay the Spire compared to prior versions in reaching later game stages. It reflects ongoing advancements in frontier models' ability to handle strategic games with hidden information. The benchmark provides updated context on how such models fare under consistent visual computer-use evaluation. Slay the Spire: Slay the Spire is a single-player roguelike deck-building game released in 2017 that features randomness, hidden information, and deep strategic decision-making across multiple characters and difficulty levels. It has developed a dedicated community of players who compete on high difficulties and has attracted attention from AI researchers evaluating model capabilities. In this news, it serves as the primary benchmark for measuring rapid progress in frontier AI models playing through a visual computer-use interface. Claude Opus 4.8: Claude Opus 4.8 is an earlier Anthropic model referenced as a baseline showing lower Slay the Spire performance relative to Claude Fable 5. It illustrates the pace of recent model improvements in game-playing benchmarks. Its mention helps frame the current frontier results against prior capabilities. Slay the Spire 2: Slay the Spire 2 is the early-access sequel to the original game, considered easier overall by players due to the absence of an equivalent to the final Heart boss, and is used here to test model generalization to less familiar content. It features new characters and mechanics while maintaining the core strategic elements of deck-building and run-based play. The benchmark results highlight stronger model performance on its maximum difficulty compared to the original game. Research Focus: Organizations are prioritizing consistent visual benchmarks to better understand how models handle real-world task automation without custom text-based harnesses. AI Gaming Benchmarks: Frontier AI models are increasingly evaluated on complex strategy games like Slay the Spire to assess computer-use and generalization abilities in human-like conditions. Model Capability Trends: Recent releases from major labs show accelerating progress in AI performance on games with high decision complexity and visual interfaces.

Categories

machine_learningaiai_agentstech
View Original Tweet