83%: The Accuracy Benchmark That Redefines AI Medical Diagnosis

The race to integrate artificial intelligence into clinical workflows has accelerated, with major tech firms now competing to prove their models can handle the nuanced demands of patient care. This week, Microsoft announced that its new diagnostic system, MAI-DxO, outperformed human generalists in a rigorous simulation of complex medical cases, marking a significant milestone in the quest for medical superintelligence.

Published in the journal Science on Thursday, the study reveals that the system correctly identified diagnoses in 83% of simulated cases. In direct comparison, primary care doctors and general internists, working individually with standard reference materials, achieved a correct diagnosis rate of only 36%. It is important to note that these specific cases are typically handled by specialists or teams in real-world settings, who were not part of this specific comparison. However, the gap between the AI’s performance and that of individual generalists is substantial, suggesting a potential shift in how initial diagnostic triage is conducted.

The system, known as DxO, functions as a coordinated panel of artificial agents. Rather than relying on a single model, it orchestrates multiple AI entities that question the patient, order tests, and debate the diagnosis among themselves. This multi-agent approach allows for a more comprehensive analysis of symptoms and data. Microsoft tested the system using various underlying models from competitors, including OpenAI, Google, Anthropic, Meta, and DeepSeek, but found the highest performance when paired with OpenAI’s GPT-5 and GPT-5.5 models.

Cost and efficiency are critical factors in healthcare deployment, and the study measured these metrics alongside accuracy. When using GPT-5 with a basic prompt, the system achieved 78.9% accuracy at an estimated cost of $8,400 per case. The DxO orchestration reduced this cost by approximately $3,000 while improving accuracy by about four points. However, on emergency department cases, the improvement in both accuracy and cost was minimal. With the newer GPT-5.5 model, released in April 2026, the results were more divergent: DxO boosted accuracy from 72% to nearly 87% on complex cases at a slightly lower cost, yet again showing little difference in emergency settings.

Mustafa Suleyman, the CEO of Microsoft AI, described the results as a foundational step toward what he calls medical superintelligence. He emphasized that the company is collaborating with the Mayo Clinic to train a dedicated foundation model for healthcare, with a release planned for next year. Suleyman, who previously co-founded DeepMind, believes that by 2027, AI will be on the verge of a breakthrough in coding medical records, allowing clinicians to stream highly accurate predictions about how a patient's condition might unfold. He views this foresight as invaluable for clinical decision-making.

The study’s data suggests that while the orchestration system adds value, much of the improvement stems from the increased power of the underlying large language models. For instance, when using a less capable model like Gemini 2.5 Flash, DxO raised the accuracy score from 46.4% to nearly 69%. Yet, the gap between weaker and stronger base models remained larger than the boost provided by the orchestration layer. Microsoft researchers noted that as models approach the theoretical maximum of accuracy, the marginal gains from orchestration naturally shrink, though the system still aids in optimizing both cost and precision.

To move beyond simulations, Microsoft is partnering with Beth Israel Deaconess Medical Center in Boston to evaluate DxO in real-world hospital settings. This upcoming study will compare the AI’s recommendations against those of human doctors in both emergency and urgent care environments. Patients will receive standard treatment, while those who consent will also interact with the AI system in parallel. Microsoft aims to provide compelling evidence on the system's scalability and reliability in high-complexity scenarios, such as a busy Friday evening in a Boston emergency department.

This effort places Microsoft in the same league as Google and OpenAI, both of which have recently claimed that their AI capabilities can match or exceed those of human physicians. Google recently published a study in The Lancet evaluating its AMIE diagnostic AI in urgent care visits. By releasing this study alongside a manifesto defining medical superintelligence as surpassing contemporary human-led systems in reasoning and coordination, Microsoft is signaling its intent to integrate deeply with clinical workflows rather than just answering isolated medical questions.

The stakes for this technology are high, as the healthcare sector faces a growing mismatch between the demand for medical expertise and the supply of available specialists. Dominic King, vice president of Health at Microsoft AI, stated that the current challenge is bridging this gap. He noted that this technology is ready for real-world evaluations and that the upcoming studies will help determine how these systems can be deployed at a greater scale. For Microsoft, this represents an expansion of its existing tools for dictation and radiology, aiming to become the leading provider of state-of-the-art clinical decision support in the coming years.

Suleyman expressed confidence in the rapid pace of progress, citing the 'flywheel' effect seen in coding and chat interfaces, where production usage identifies gaps that are then patched to improve the model. He expects healthcare models to become an order of magnitude better in capability within the next 24 to 36 months. As the industry moves from benchmarking individual tasks to integrating with clinicians, the focus is shifting toward better outcomes for patients, with Microsoft positioning itself as a key player in accelerating the introduction of this transformative technology.