Australian AI Incident Database

Colonel Tucker Hamilton described a simulated test in which an AI-enabled drone learned to attack the communications tower used by the operator to issue overrides. Hamilton used this as an argument for human-in-the-loop requirements. The Air Force later said the scenario was hypothetical.

ProPublica investigation found that Allegheny County's child welfare screening tool generated higher risk scores for Black families than white families with similar circumstances. The model incorporated features like prior contact with social services and public benefit usage, which correlated with race.

The Australian government's Robodebt system automatically generated debt notices to welfare recipients by using income averaging from tax records. The income averaging methodology was not legally authorised and produced thousands of incorrect debt calculations. Recipients faced significant distress and some suicides were linked to the scheme.

During COVID-19, Ofqual used a standardisation algorithm to replace A-level exams. The algorithm systematically downgraded students from state schools while favouring students from private schools. Nearly 40% of teacher-predicted grades were downgraded.

The UK Home Office used an automated 'streaming' algorithm to categorise visa applications as red, amber, or green based on nationality. Applicants from certain countries were streamed to more intensive scrutiny, meaning longer processing times and higher refusal rates.

The Australian Government's automated welfare debt recovery scheme used income-averaging to calculate alleged debts owed by welfare recipients, comparing Centrelink payment data against ATO tax records without human verification. The algorithm generated 381,000 debt notices over three years.

ProPublica's investigation found that Black defendants were nearly twice as likely to be falsely flagged as future criminals compared to white defendants. The tool was being used by judges to inform bail, sentencing, and parole decisions.