Furniture Assembly Benchmark scores improve from 28% to 80% in 10 months

GPT-5.4: GPT-5.4 is a frontier AI model from OpenAI. It is tested on the Furniture Assembly Benchmark in the news, noted for failing to catch many real mistakes while performing differently from models that over-flag errors. GPT-6 Astra: GPT-6 Astra is an advanced AI model recognized for strong multimodal reasoning capabilities. In the news, it is highlighted as the top performer on the Furniture Assembly Benchmark, excelling in both accuracy and processing speed compared to other tested models. Gemini 3.1 Pro: Gemini 3.1 Pro is a frontier multimodal AI model developed by Google. In the news, it is evaluated on the Furniture Assembly Benchmark, where it tends to incorrectly flag correct builds as erroneous rather than missing actual mistakes. Furniture Assembly Benchmark: The Furniture Assembly Benchmark (FAB) is an evaluation framework that tests AI models on their ability to identify assembly errors in furniture using an instruction manual and a photograph of a half-completed build. It consists of 60 photos across three different furniture items. The benchmark is the central development in the news, which reports rapid progress in model performance and highlights differences in how leading models handle error detection versus false positives. AI Evaluation Trends: New benchmarks are emerging to assess AI performance on practical, real-world tasks that combine visual analysis with instructional understanding. Deployment Considerations: Both accuracy and processing speed are critical factors when evaluating AI models for potential practical applications involving visual and procedural reasoning. Model Behavior Differences: Frontier AI models display distinct patterns in error handling, with some favoring over-detection of issues and others under-detection when processing combined image and text inputs.