Lawyer Cited ChatGPT-Hallucinated Non-Existent Cases in Federal Court
Levidow, Levidow & Oberman / OpenAI (ChatGPT)
What happened
New York attorney Steven Schwartz used ChatGPT to research case law. ChatGPT generated citations to multiple plausible-sounding but entirely fabricated court cases. Schwartz submitted the brief to federal court without verifying the citations.
Root cause
LLM generated plausible but fabricated legal citations without any grounding to verified legal databases; attorney used model output without verification.
Architectural failure
No hallucination detection or output grounding against authoritative sources; model presented fabricated citations with same confidence as real ones.
Outcome
Federal Judge ordered the attorneys to show cause. Schwartz and his supervising partner were sanctioned and fined $5,000 each. Prompted bar associations worldwide to issue guidance on AI use in legal practice.
Architectural Failure Patterns
These pattern categories on aipatterns.com.au describe the systemic failure modes this incident exhibited.
Cite this incident
https://corporateai.com.au/incidents/chatgpt-hallucinated-case-law-2023Quick facts
- Date
- 27 May 2023
- Organisation
- Levidow, Levidow & Oberman / OpenAI (ChatGPT)
- Sector
- Other
- Severity
- High
- Regulatory bodies
- FRCP Rule 11 (Sanctions)ABA Model Rules of Professional ConductState Bar Ethics Rules
- Tags
- hallucinationlegal-aifabricated-citationsllmprofessional-liabilityverification
Related in Other
Air Canada Chatbot Gave Wrong Bereavement Refund Policy
Air Canada · 14 February 2024
DPD Chatbot Jailbroken to Swear at Customer and Criticise DPD
DPD · 19 January 2024
Cruise Autonomous Vehicle Dragged Pedestrian 20 Feet After Collision
Cruise (General Motors) · 2 October 2023
Explore failure patterns
aipatterns.com.au