08-13-2026, 08:03 AM
Not every AI project is a win, and I see way more "here's how I crushed it" posts than honest post-mortems. So here's one I wish I'd read before I built this.
The setup: a mid-sized e-commerce client asked me to build an AI system that auto-responded to customer "where is my order" emails and refund requests. The goal was to cut support tickets by 60% and save on part-time support headcount. ~$2k/month for the build and monitoring.
What went right initially:
- Built it on their existing helpdesk + their actual order DB (not just RAG over docs)
- Did 3 weeks of shadow-mode testing before flipping it live
- Goldenset of 200 tickets was passing at 97%
- First month looked great: 58% of tickets auto-handled, customers happy per CSAT
What went wrong 6 weeks later:
- A shipping carrier's API changed their tracking-status labels. The AI interpreted the new "in transit — exception" code as "lost in transit" and started auto-approving refunds for orders that were simply delayed.
- Took the client 9 days to notice. In that window, the bot incorrectly refunded ~$30k worth of orders that did eventually arrive. Most customers were honest and told support; some weren't.
- Kill switch existed but nobody had been told how to trigger it, because I hadn't run a failure drill.
- The alerting was set up for errors, not for anomalies ("refund rate is 4x baseline" wasn't being monitored).
What I changed permanently after this:
- Anomaly detection on EVERY customer-facing AI's output metrics (refund rate, average refund size, response sentiment, escalation rate). Alerts fire on deviations, not just errors.
- Mandatory documented kill-switch *and* a walkthrough with whoever will be on call.
- "Change in external dependency" (API, carrier, model version) is now explicitly listed as a risk in every SOW.
- Hard monetary caps per action per day. The bot should not be allowed to issue more than $N in refunds per day without human approval, full stop.
- Shadow mode isn't enough. I now do a "canary week" where it acts on 10% of traffic, ramps to 50%, then 100%.
This single project probably cost me 2 months of net profit counting the discount I gave the client to make it right, but the lessons are now priced into every proposal I write.
What's your most expensive AI lesson? Sharing post-mortems openly is how the whole ecosystem gets better.
The setup: a mid-sized e-commerce client asked me to build an AI system that auto-responded to customer "where is my order" emails and refund requests. The goal was to cut support tickets by 60% and save on part-time support headcount. ~$2k/month for the build and monitoring.
What went right initially:
- Built it on their existing helpdesk + their actual order DB (not just RAG over docs)
- Did 3 weeks of shadow-mode testing before flipping it live
- Goldenset of 200 tickets was passing at 97%
- First month looked great: 58% of tickets auto-handled, customers happy per CSAT
What went wrong 6 weeks later:
- A shipping carrier's API changed their tracking-status labels. The AI interpreted the new "in transit — exception" code as "lost in transit" and started auto-approving refunds for orders that were simply delayed.
- Took the client 9 days to notice. In that window, the bot incorrectly refunded ~$30k worth of orders that did eventually arrive. Most customers were honest and told support; some weren't.
- Kill switch existed but nobody had been told how to trigger it, because I hadn't run a failure drill.
- The alerting was set up for errors, not for anomalies ("refund rate is 4x baseline" wasn't being monitored).
What I changed permanently after this:
- Anomaly detection on EVERY customer-facing AI's output metrics (refund rate, average refund size, response sentiment, escalation rate). Alerts fire on deviations, not just errors.
- Mandatory documented kill-switch *and* a walkthrough with whoever will be on call.
- "Change in external dependency" (API, carrier, model version) is now explicitly listed as a risk in every SOW.
- Hard monetary caps per action per day. The bot should not be allowed to issue more than $N in refunds per day without human approval, full stop.
- Shadow mode isn't enough. I now do a "canary week" where it acts on 10% of traffic, ramps to 50%, then 100%.
This single project probably cost me 2 months of net profit counting the discount I gave the client to make it right, but the lessons are now priced into every proposal I write.
What's your most expensive AI lesson? Sharing post-mortems openly is how the whole ecosystem gets better.
