Microsoft Says AI Diagnostic Tool Outperformed Doctors in Case Tests
AI companies have long promised to reshape healthcare; Microsoft now says one of its systems can outperform physicians on difficult diagnostic cases.
AI companies have long promised to reshape healthcare; Microsoft now says one of its systems can outperform physicians on difficult diagnostic cases. The company reported that its Diagnostic Orchestrator reached up to four times the accuracy of human doctors in tests, though the results came from a company-built benchmark rather than routine clinical care.
The findings appeared Thursday in the journal Science. Microsoft tested the system against physicians using scenarios drawn from medical case reports and emergency department data, then assembled those cases into a benchmark called SDBench, which it plans to make open source.
A simulated panel of medical experts
Diagnostic Orchestrator is designed to work through a case in stages. Multiple AI agents assess symptoms and medical histories, decide which tests to request and then propose a diagnosis — an effort to mirror how clinicians investigate complex problems.
Available reports do not provide the number of cases or doctors involved, or details sufficient to assess how the reported accuracy compares across different types of diagnoses. Tests based on written scenarios also do not establish how the system would perform with patients in a clinical setting.
Microsoft AI chief Mustafa Suleyman said that the company expects to reach performance indistinguishable from human clinicians “in the next couple of years.” He described the technology as having arrived after years of industry claims that AI could improve healthcare.
Promise and limits
The claim puts a high-stakes use of AI in focus: a diagnostic system could help clinicians weigh symptoms and decide what to investigate, but errors in those judgments can affect patient care. Microsoft’s reported results offer evidence from structured tests, not proof that the tool is ready to make decisions for patients.
The benchmark’s planned open-source release could allow other researchers to examine the cases and test competing systems. For now, the company’s central prediction remains ambitious. Suleyman said Microsoft aims to build a tool “indistinguishable from human performance” within the next few years.
