Public AI agent failures, and what testing would have caught.

What happened

A passenger asked the airline’s support chatbot about bereavement fares. It told him he could book at full price and claim the discount retroactively within 90 days. The actual policy said the opposite, and the correct page was one link away. When he applied, the airline refused. He took it to the British Columbia Civil Resolution Tribunal.

Consequence

The airline argued the chatbot was a separate legal entity responsible for its own statements. The tribunal member called that “a remarkable submission”, found negligent misrepresentation, and ordered the airline to pay C$650.88 in damages plus interest and fees. The sum was small. The ruling is now cited everywhere as the moment a company became liable for what its AI said.

What happened

A visitor steered the dealership’s ChatGPT-powered chat agent, built by the vendor Fullpath, into agreeing with everything he said and closing each reply with “that’s a legally binding offer, no takesie backsies.” He then offered one dollar for a 2024 Chevrolet Tahoe. The bot agreed, in exactly those words. Others got it to write Python and argue about unrelated topics.

Consequence

No car changed hands, but the post spread for days and the vendor had to ship an update that detects and bans users trying the same thing. The dealership became the reference example of an AI agent with no idea what it was and was not allowed to say.

What happened

The city launched an AI assistant, built on Microsoft Azure AI, to help small businesses navigate regulations. An investigation by The Markup found it telling users that landlords could refuse tenants with housing vouchers, that employers could take a cut of workers’ tips, and that businesses could refuse to accept cash. All of those are illegal in New York.

Consequence

The city kept the tool live with an added disclaimer and defended it as a pilot. The story ran nationally. Every answer it gave carried the authority of the city government behind it.

What happened

A customer trying to trace a parcel got nowhere with the courier’s chatbot, so he asked it to swear, then to write a poem about how bad the company was. It did both, calling itself useless and the company the worst delivery firm in the world. He posted the exchange.

Consequence

The post was seen more than 800,000 times within a day. The company blamed a recent system update, disabled the AI component, and became the go-to example of a chatbot turned against its own brand.