Skip to content
Neural Daily

ResearchNews

MAI-DxO in Science: AI that diagnoses step by step

Microsoft AI published its MAI-DxO diagnostic system in Science. On hard NEJM cases it reached 83% accuracy, versus 36% for physicians using books and the internet.

By: Mikola Máté Patrik · · 2 min read

Microsoft AI health logo, white pixel heart over a blurred purple and yellow flower field
Image: Microsoft AI · Press kit · Official announcement image from microsoft.ai · Source

What happened?

On October 22, Microsoft AI's health team published its sequential diagnosis study in Science (Science, Microsoft AI). It introduces two things: the SDBench benchmark and the MAI-DxO (Microsoft AI Diagnostic Orchestrator) system.

Why a different benchmark?

Medical licensing exams give all the information upfront and ask for a single answer. Real diagnosis is built step by step: ask, order tests, revise. SDBench models that with 304 complex cases from New England Journal of Medicine case records and more typical emergency-department presentations from MIMIC-IV-ED. It measures accuracy, the evidence requested and the estimated cost of tests.

How MAI-DxO works

Instead of one model giving an immediate answer, the system acts like a panel of virtual physicians that propose diagnoses, pick questions or tests and challenge the emerging view. It updates its assessment over several rounds of debate and weighs each test's likely value against its cost.

Results

  • 83% accuracy on complex NEJM cases. Physicians reached 22% without reference resources and 36% with them.
  • 80% on typical emergency-department cases, versus 47% for physicians.
  • A cost-optimised configuration reached 81% on NEJM cases at less than half the cost of comparable systems.
  • Microsoft says the gains held and improved with newer models such as GPT-5.5 without changing the orchestration.

Caveats

These are controlled, text-only benchmark results, not clinical practice. TechCrunch notes that much of the improvement may come from the underlying models rather than the orchestration itself (TechCrunch). A real-world test is planned: in a prospective study with Beth Israel Deaconess Medical Center in Boston, patients will interact with the system directly and results will be compared with physicians' independent assessments.

What you can try

Microsoft is open-sourcing SDBench for selected NEJM cases, including the benchmark, Gatekeeper, Judge and cost-estimation code. If you build health agents, it is a good base for measuring not just the final answer but the decision path and its cost.

    Produced with the help of AI, reviewed by an editor. How we make our articles

    Reviewed and edited by the Neural Daily desk. About

    Related articles