OpenAI Announces MentalHealthBench to Measure Mental Health Conversations

OpenAI has introduced a new open benchmarking tool called MentalHealthBench, developed in collaboration with more than 80 licensed experts from 22 countries, to measure how artificial intelligence models respond to realistic mental health conversations.

Artificial Intelligence and Mental Health Communication

People turn to artificial intelligence for many different topics, such as managing challenging relationships, coping with daily stress, or making decisions in critical situations.

With more than a million people using ChatGPT every week, it is essential that models provide careful responses and prioritize safety.

Gaps in Evaluations

Current evaluations often focus only on emergency scenarios, leaving gaps in understanding performance across a broad spectrum.

It is of great importance to measure how well models align with expert guidance and how they behave beyond simply avoiding prohibited responses.

What is MentalHealthBench?

MentalHealthBench is a new open benchmarking tool that measures the performance of artificial intelligence systems in non-emergency, high-sensitivity, and emergency scenarios.

This tool evaluates core behaviors such as safety, seeking context, preserving user agency, and providing actionable guidance when appropriate.

Expert Collaboration and Future Goals

Developed in partnership with over 80 experts from 22 countries, this system has been made open access so researchers can conduct their own evaluations.

While ChatGPT is not a replacement for therapy or professional care, these efforts measure the development of models in showing empathy and directing users to real support resources.