Robodebt: Automated Income-Averaging Issued 381,000 Unlawful Debt Notices
Services Australia (Department of Human Services)
What happened
The Australian Government's automated welfare debt recovery scheme used income-averaging to calculate alleged debts owed by welfare recipients, comparing Centrelink payment data against ATO tax records without human verification. The algorithm generated 381,000 debt notices over three years.
Root cause
Automated financial decisions with legal and welfare consequences enacted with no human approval gate; no mechanism for individuals to contest algorithmic outputs before debt recovery began.
Architectural failure
Zero human-in-the-loop for high-stakes automated decisions affecting individual financial welfare; no audit trail per individual decision; no explainability mechanism for recipients.
Outcome
$1.8B scheme ruled unlawful by Federal Court. $720M repaid to affected recipients. Royal Commission found the scheme caused serious harm. Multiple suicides linked to debt stress.
Architectural Failure Patterns
These pattern categories on aipatterns.com.au describe the systemic failure modes this incident exhibited.
Cite this incident
https://corporateai.com.au/incidents/robodebt-australia-2016Quick facts
- Date
- 1 July 2016
- Organisation
- Services Australia (Department of Human Services)
- Sector
- Government
- Severity
- Critical
- Regulatory bodies
- Privacy ActSocial Security ActAdministrative Decisions (Judicial Review) Act
- Tags
- government-aiautomated-decisionswelfareunlawfulroyal-commissionhuman-in-the-loopexplainability
Related in Government
US Air Force AI Drone Reportedly Attacked Operator in Simulation
US Air Force · 2 June 2023
Allegheny Family Screening Tool Found to Flag Black Families at Higher Rates
Allegheny County Department of Human Services · 2 May 2023
Australian Robodebt Automated Debt Scheme Found Unlawful, Linked to Deaths
Australian Department of Human Services (Services Australia) · 6 November 2020
Explore failure patterns
aipatterns.com.au