Epic Sepsis Prediction Model Failed to Generalise, Missing Majority of Cases
Epic Systems / University of Michigan
What happened
A University of Michigan study found that Epic's commercially deployed Sepsis Prediction Model performed significantly worse than claimed when deployed in their health system. The model flagged only 7% of sepsis patients before clinical deterioration and generated so many false positives that clinicians began ignoring alerts.
Root cause
Model trained and validated on a non-representative population without external validation across diverse hospital demographics; no ongoing performance monitoring after deployment.
Architectural failure
Absent model drift monitoring and post-deployment performance benchmarking; training data governance did not account for demographic variation; no model rollback mechanism.
Outcome
Study published in JAMA Internal Medicine sparked widespread re-evaluation of AI clinical decision tools. Raised systemic concerns about commercial AI tools in healthcare.
Architectural Failure Patterns
These pattern categories on aipatterns.com.au describe the systemic failure modes this incident exhibited.
observability
model drift detection
aipatterns.com.au
observability
ai performance benchmarking
aipatterns.com.au
data
ai training data governance
aipatterns.com.au
data
data lineage for ai
aipatterns.com.au
model management
model rollback
aipatterns.com.au
governance
ai risk assessment framework
aipatterns.com.au
human in the loop
ai confidence threshold routing
aipatterns.com.au
Cite this incident
https://corporateai.com.au/incidents/epic-sepsis-model-failure-2021Quick facts
- Date
- 1 July 2021
- Organisation
- Epic Systems / University of Michigan
- Sector
- Healthcare
- Severity
- Critical
- Regulatory bodies
- FDA AI/ML-Based Software as a Medical Device (SaMD)21st Century Cures ActHIPAA
- Tags
- healthcare-aimodel-driftgeneralisationclinical-decision-supportpatient-safetyalert-fatigue
Related in Healthcare
View all Healthcare incidents →Explore failure patterns
aipatterns.com.au