Editorial
The Gist
- AI failures now have legal consequences. Cases like Air Canada’s chatbot ruling prove brands are liable for their AI’s misinformation in customer interactions.
- Emotional damage magnifies AI mistakes. When AI gives false or tone-deaf responses to frustrated customers, it destroys trust at critical moments.
- Human-AI partnerships outperform full automation. The best results come when AI supports—rather than replaces—agents, improving speed, clarity, and empathy.
- Training is AI’s safest proving ground. AI-driven simulations build skill and confidence without exposing customers to risk or reputational harm.
The numbers are sobering. Organizations invested $47 billion in AI initiatives during the first half of 2025, yet 89% of that spend delivered minimal returns. Most projects collapsed under the weight of compliance complexity, organizational chaos or the messy reality of actual customer interactions.
But here's the dangerous part: nowhere is the AI-as-hammer approach more tempting, or more destructive, than in customer service.
When AI Hallucinations Meet Real Customers
The consequences of deploying unready AI in customer-facing roles aren't theoretical. They're playing out in real time with measurable brand damage:
Air Canada's chatbot told a bereaved grandson he could retroactively claim bereavement fares after purchasing full-price tickets, information that was completely false. When the airline refused the refund, a tribunal ruled Air Canada liable for its chatbot's misinformation. The airline's defense? The chatbot was "a separate legal entity" responsible for its own actions.
The legal precedent is now set: you own your AI's mistakes.
AI Missteps That Shaped the Industry
Recent cases show the reputational and financial risks of untested AI systems in customer service.
| Case | Outcome |
|---|---|
| Air Canada Chatbot Incident | Tribunal ruled airline liable for false AI claims; established legal accountability for chatbot errors. |
| Cursor “Sam” Fabrication | Fictional policy spread online, triggering subscription cancellations and public backlash. |
| Retail Bank Misadvice Bot | Erroneously offered mortgage extensions, leading to regulatory review and customer compensation. |
More recently, Cursor's AI support agent "Sam" fabricated an entirely fictional policy, telling developers they were limited to one device per subscription due to "security features." The hallucination spread rapidly through developer communities, triggering subscription cancellations and a viral crisis before the company could intervene.
As former Google chief decision scientist Cassie Kozyrkov noted, "This mess could have been avoided if leaders understood that AI makes mistakes, AI can't take responsibility for those mistakes, and users hate being tricked by a machine posing as a human."
Related Article: Preventing AI Hallucinations in Customer Service: What CX Leaders Must Know
The Psychology of AI Betrayal
What makes these failures particularly damaging is the emotional dimension. When customers contact support, they're already in a vulnerable state: confused, frustrated or grieving. They're seeking human understanding and accurate solutions.
When an AI confidently delivers false information, it not only fails to solve the problem but also undermines customer trust. It betrays trust at the precise moment trust matters most. The consequences extend beyond individual interactions: Cursor saw users canceling subscriptions, while research shows customer support requires empathy, nuance and problem-solving that AI currently struggles to deliver.
The Seductive Trap of Cost Reduction in Customer Service
The appeal is obvious: replace expensive human agents with scalable AI, slash training costs and handle volume 24/7. But this thinking fundamentally misunderstands what customer service is.
Customer service isn't transactional information exchange; it's emotional labor. It's de-escalation. It's reading between the lines to understand what customers actually need versus what they're asking for. It's the ability to say, "I don't know, but let me find out" instead of confidently inventing policies.
The Right Way: AI as Agent Empowerment, Not Replacement
Here's where the conversation becomes more nuanced. AI has tremendous potential in customer service when deployed to enhance human judgment rather than replace it.
The most successful implementations utilize AI as an intelligent assistant that works alongside agents, rather than replacing them. Real-time conversation summaries allow agents to quickly grasp context during transfers or follow-ups. Multilingual translation breaks down language barriers while preserving the human connection. Recommended actions surface relevant knowledge articles and troubleshooting steps, reducing cognitive load. Suggested responses provide starting points that agents can personalize and refine based on the emotional context they perceive. AI-powered writing improvement helps agents communicate more clearly and professionally while maintaining their authentic voice.
The critical distinction? The agent remains in control. They evaluate AI suggestions through the lens of empathy, situational awareness and customer experience. They can override recommendations when human judgment reveals nuances the AI missed. This approach delivers measurable gains, including faster resolution times, improved consistency and reduced agent stress, without the brand risk associated with autonomous AI making consequential decisions.
Related Article: Your Contact Center Agents Don't Fear AI—They Fear Your Leadership
AI-Powered Training: Building Capability Without Risk
Beyond agent assistance, AI-powered role-play training creates safe environments where agents practice complex scenarios without risking real customer relationships.
AI-Driven Agent Training Benefits
Measured performance outcomes from AI-powered training platforms in customer service.
| Training Outcome | Measured Improvement |
|---|---|
| Training speed | 60–70% faster onboarding compared to traditional classroom methods |
| Customer satisfaction (CSAT) | 35% increase after six months of AI-assisted coaching |
| Error rate in live calls | 30% reduction in service errors and policy violations |
This approach delivers 60-70% faster training, 35% improvement in customer satisfaction, and 30% fewer errors in live interactions—without the brand risk of customer-facing hallucinations.
The Strategic Choice for AI in Customer Service
While competitors chase the mirage of fully automated support, savvy organizations recognize AI's true value proposition: enhancing human capability, not replacing human judgment.
Webinar
Nov
6
Fix the Content Bottleneck: Build a Better WebOps Strategy
Content stalled? Dev overloaded? You’re not the only one. Learn how streamlined WebOps bridges the publishing gap.
Register
Webinar
On demand
Beyond Storage: Smarter Content, Bigger Impact with DAM + AI
Discover how the DAM + AI duo makes content smarter, stronger and more accessible.
Watch Now
Webinar
On demand
Agentic AI Playbook: Real-World Customer Service Use Cases You Can Deploy Now
Boost self-service by 30% and slash call volume by 63% with agentic AI.
Watch Now
Webinar
On demand
CMS Briefing: A Live Look at What’s Next in AI-Driven Platforms
Learn how leading organizations are using AI‑driven tools to publish faster, personalize smarter and stay secure.
Watch Now
Webinar
On demand
Ready or Not: How Data-First Organizations Are Unlocking Agentforce Potential
Learn how to cut through the noise, activate Agentforce and build a Salesforce AI strategy that actually delivers.
Watch Now
Webinar
On demand
AI in Customer Service: Faster Resolutions, Happier Customers
Don’t let rising demand burn out your team. See how to build a smarter, more resilient support org.
Watch Now
Before deploying any customer-facing AI, ask three questions:
- What happens when this AI makes a mistake in front of our customers?
- Can we clearly disclose when customers are interacting with AI?
- Do we have human oversight for every consequential interaction?
If the answers make you uncomfortable, you're deploying AI in the wrong place.
The organizations winning with AI aren't using it as a hammer to automate everything. They're using it strategically, training better agents who deliver better experiences, preserving the human connection that defines exceptional service.
That's not just a better customer experience. It's better business.
Learn how you can join our contributor community.
