OpenAI's MentalHealthBench rates GPT-6 Astra at 57.3 for mental health conversations

GPT-4o: GPT-4o is an earlier OpenAI language model used as a baseline comparison in the MentalHealthBench evaluation. OpenAI: OpenAI is an AI research and deployment company known for developing advanced large language models and related systems. It created and released MentalHealthBench to demonstrate continued progress in its frontier models' handling of realistic mental health conversations. GPT-5.6 Sol: GPT-5.6 Sol is an OpenAI model variant employed to grade responses against the expert-written criteria in MentalHealthBench. GPT-6 Astra: GPT-6 Astra is a frontier language model developed by OpenAI. It is evaluated in the news on the new MentalHealthBench to illustrate improvements in managing nuanced mental health dialogues. MentalHealthBench: MentalHealthBench is an open evaluation benchmark focused on realistic, everyday, and ambiguous mental health conversations rather than just emergencies. OpenAI developed it in collaboration with mental health experts and is releasing it publicly for broader research use. Open Release: The benchmark is being released openly so researchers can examine the methods, run evaluations, and extend the work. Evaluation Process: Each synthetic conversation in the benchmark uses custom expert rubrics with review by at least three experts to ensure reliability of the criteria. Expert Involvement: OpenAI collaborated with more than 80 licensed psychologists and psychiatrists from 22 countries across 19 languages and nearly 20 subspecialties to build the benchmark.