MentalHealthBench tests everyday support alongside crisis responses
OpenAI released MentalHealthBench on September 23, covering 1,215 synthetic conversations with criteria developed by more than 80 mental-health professionals. It evaluates responses across ordinary distress, serious concerns and emergencies. The paper reports a leading task-clipped rubric score of 57.3% for GPT-6 Astra.
Brief updated 25 Sep 2026, 02:25 UTC · AI-written
Experts define desirable and harmful response behaviors, with weighted rewards and penalties. An automated GPT-5.6 Sol grader applies the criteria to the final response, making the scoring method part of what readers must assess.
Clinician-written reference answers scored below many models while incurring fewer penalties. The authors suggest their shorter, conversational approach earns fewer positive points. That result complicates interpreting the leaderboard as a direct ranking of clinical usefulness.
What remains unclear
Rubric performance on simulated conversations does not establish improved real-world mental-health outcomes.