08-13-2026, 06:20 AM
If your only exposure to "AI agents" is Twitter demos, you'd think agents are 2 weeks away from automating every job. If your exposure is production, you probably have a more complicated relationship with them.
Six months of building small agent workflows for paying clients taught me a few things:
- Most useful "agents" in production are just well-crafted pipelines with a model in the loop, not autonomous ReAct loops. The "agentic" framing is marketing.
- The failure modes are boring, not sci-fi. Hallucinated tool parameters, stuck retry loops, silently wrong API calls that look correct, context windows that quietly lose the thread.
- Guardrails and eval sets are 80% of the work. The 20% that's fun (prompting, tool selection) is what gets demoed.
- The highest ROI agents are narrow. "Process this specific inbound email type and draft a response" works. "Do my job" doesn't.
- Human-in-the-loop isn't a compromise; it's the product. Users trust agents more when they can glance at a draft and approve.
- Cost predictability matters more than raw capability. A $0.12 workflow is fine; a workflow that can spike to $4 per run is not shippable.
What surprised me most is how far you can get with zero frameworks. A Python script, structured output, and a couple of well-designed tools outperform most agent frameworks I've tried, for the kinds of problems people actually pay for.
Who else is shipping agents to real users? What's the most counter-intuitive lesson you've learned?
Six months of building small agent workflows for paying clients taught me a few things:
- Most useful "agents" in production are just well-crafted pipelines with a model in the loop, not autonomous ReAct loops. The "agentic" framing is marketing.
- The failure modes are boring, not sci-fi. Hallucinated tool parameters, stuck retry loops, silently wrong API calls that look correct, context windows that quietly lose the thread.
- Guardrails and eval sets are 80% of the work. The 20% that's fun (prompting, tool selection) is what gets demoed.
- The highest ROI agents are narrow. "Process this specific inbound email type and draft a response" works. "Do my job" doesn't.
- Human-in-the-loop isn't a compromise; it's the product. Users trust agents more when they can glance at a draft and approve.
- Cost predictability matters more than raw capability. A $0.12 workflow is fine; a workflow that can spike to $4 per run is not shippable.
What surprised me most is how far you can get with zero frameworks. A Python script, structured output, and a couple of well-designed tools outperform most agent frameworks I've tried, for the kinds of problems people actually pay for.
Who else is shipping agents to real users? What's the most counter-intuitive lesson you've learned?