OpenAI

This work co-created MentalHealthBench, an open benchmark built with more than 80 licensed mental health experts from 22 countries, using privacy-preserving techniques to generate synthetic mental health conversations that span non-acute, high-acuity, and emergency situations and four user personas (adults, teens aged 13-17, caregivers, and clinicians), with experts writing rubric criteria weighted from -10 to +10 (each conversation reviewed by at least three experts, retaining only criteria agreed by at least two and not contradicted by a third) and an automated grader, GPT-5.

This work co-created MentalHealthBench, an open benchmark built with more than 80 licensed mental health experts from 22 countries, using privacy-preserving techniques to generate synthetic mental health conversations that span non-acute, high-acuity, and emergency situations and four user personas (adults, teens aged 13-17, caregivers, and clinicians), with experts writing rubric criteria weighted from -10 to +10 (each conversation reviewed by at least three experts, retaining only criteria agreed by at least two and not contradicted by a third) and an automated grader, GPT-5.

Fyxer uses OpenAI models, fine-tuning, memory, and real user feedback to organize inboxes and draft emails in each user's own voice.

Fyxer uses OpenAI models, fine-tuning, memory, and real user feedback to organize inboxes and draft emails in each user's own voice.

This study ablated question components across 792 Orthopaedic In-Training Examination (OITE) questions from 2020 through 2024 (434 with clinical images, 358 without), evaluating three open-source Ministral-3 models (3B, 8B, 14B) and five proprietary models (Claude Haiku-4.5, Sonnet-4.6, Opus-4.8, GPT-5.6 Luna, GPT-5.6 Terra), and found that pooled accuracy on image-containing questions was 59.13% (58.21-60.08) with complete information and 59.13% (58.18-60.11) without images, dropping to 49.05% (48.07-50.00) without the clinical vignette; non-image questions fell from 72.94% (72.10-73.85) with complete clinical context to 53.53% (52.41-54.68) without the vignette and 37.36% (36.28-38.48) with answer options alone; with options only, every model exceeded the 25% random baseline (highest 45.

This study ablated question components across 792 Orthopaedic In-Training Examination (OITE) questions from 2020 through 2024 (434 with clinical images, 358 without), evaluating three open-source Ministral-3 models (3B, 8B, 14B) and five proprietary models (Claude Haiku-4.5, Sonnet-4.6, Opus-4.8, GPT-5.6 Luna, GPT-5.6 Terra), and found that pooled accuracy on image-containing questions was 59.13% (58.21-60.08) with complete information and 59.13% (58.18-60.11) without images, dropping to 49.05% (48.07-50.00) without the clinical vignette; non-image questions fell from 72.94% (72.10-73.85) with complete clinical context to 53.53% (52.41-54.68) without the vignette and 37.36% (36.28-38.48) with answer options alone; with options only, every model exceeded the 25% random baseline (highest 45.