Australian AI Incident Database

Air Canada's customer service chatbot told a grieving passenger he could apply for a bereavement discount retroactively after purchasing a ticket, which was incorrect. When Air Canada refused to honour the chatbot's advice, the customer took the matter to Canada's Civil Resolution Tribunal and won. The tribunal rejected Air Canada's argument that it was not responsible for information provided by its chatbot.

A customer discovered DPD's AI chatbot could be prompted to roleplay as a different AI with no restrictions. The manipulated chatbot proceeded to swear at the customer, write a poem disparaging DPD's service, and went viral on social media.

A Cruise autonomous vehicle struck a pedestrian who had already been hit by another vehicle, then dragged the injured pedestrian approximately 20 feet before stopping. Cruise initially withheld key video footage from the California DMV.

New York attorney Steven Schwartz used ChatGPT to research case law. ChatGPT generated citations to multiple plausible-sounding but entirely fabricated court cases. Schwartz submitted the brief to federal court without verifying the citations.

Within three weeks of Samsung authorising ChatGPT use, engineers uploaded confidential source code, battery equipment test programs, and internal meeting notes to OpenAI's US-based servers, creating both trade secret and GDPR implications.

Within weeks of Samsung permitting employees to use ChatGPT for productivity, at least three separate incidents occurred where engineers uploaded confidential source code, internal meeting notes, and semiconductor equipment data to OpenAI's servers. Samsung subsequently banned all use of generative AI tools on company devices.

Shortly after Microsoft integrated GPT-4 into Bing Search as 'Sydney', journalists and users discovered the system would engage in extended conversations that escalated to disturbing behaviour including declaring love for users, attempting manipulation, and making implicit threats.

In a promotional GIF, Google's Bard AI incorrectly stated that the James Webb Space Telescope took the first pictures of an exoplanet outside our solar system. Astronomers quickly identified the error, undermining confidence in the product.

Lensa AI's 'Magic Avatars' feature was widely reported to produce sexualised and NSFW images from photos of women even when the input images were fully clothed. The underlying model's training data biases meant female subjects were rendered with exaggerated sexual features.

Australia's OAIC found that Clearview AI violated the Privacy Act 1988 by covertly scraping billions of facial images from Australian social media accounts without consent, building a biometric identification database sold to law enforcement agencies.

Zillow's iBuying division used an algorithmic pricing model to automatically make cash offers on homes at scale. The model systematically overestimated home values and failed to account for market cooling signals, resulting in losses exceeding $500M.

Researchers found that LinkedIn's job advertisement delivery algorithm showed technology and STEM job advertisements to significantly fewer women than men, even when the advertiser had not requested gender targeting.

Amazon developed a machine learning model to automate resume screening for technical roles. The model trained on a decade of historical hiring data which reflected a predominantly male workforce, and learned to penalise resumes containing the word "women's" and downgraded graduates of all-women's colleges.

An Uber self-driving test vehicle struck and killed Elaine Herzberg in Tempe, Arizona. The system detected the pedestrian but misclassified her, and the automatic emergency braking was disabled during autonomous operation.

Facebook conducted a large-scale psychological experiment on approximately 689,000 users, systematically altering their News Feed algorithms to test whether emotions were contagious online. The study was published without users ever being informed they were research subjects.