IBM Watson for Oncology Recommended Unsafe Cancer Treatments
IBM Watson Health
What happened
Internal IBM documents leaked to STAT News revealed that IBM Watson for Oncology had recommended treatments that oncologists described as 'unsafe and incorrect'. The system generated advice based on hypothetical training cases rather than real patient data.
Root cause
Training methodology relied on synthetic hypothetical cases rather than real clinical trial data or patient outcomes; no validation against clinical trial evidence.
Architectural failure
Training data governance did not require evidence-based real-world data; no external clinical validation process; no model performance benchmarking against established guidelines.
Outcome
IBM Watson for Oncology lost multiple hospital contracts globally. IBM Watson Health division was eventually sold in 2022 for a fraction of its development cost.
Architectural Failure Patterns
These pattern categories on aipatterns.com.au describe the systemic failure modes this incident exhibited.
data
ai training data governance
aipatterns.com.au
observability
ai performance benchmarking
aipatterns.com.au
human in the loop
human approval gateway
aipatterns.com.au
governance
ai risk assessment framework
aipatterns.com.au
model management
model rollback
aipatterns.com.au
governance
ai ethics review board
aipatterns.com.au
Cite this incident
https://corporateai.com.au/incidents/ibm-watson-oncology-2017Quick facts
- Date
- 25 July 2017
- Organisation
- IBM Watson Health
- Sector
- Healthcare
- Severity
- Critical
- Regulatory bodies
- FDA SaMD Guidance21 CFR Part 820 (QSR)HIPAAICH E6 GCP Guidelines
- Tags
- healthcare-aiclinical-decision-supportunsafe-recommendationstraining-datapatient-safetyoncology
Related in Healthcare
View all Healthcare incidents →Explore failure patterns
aipatterns.com.au