The $400/Day AI Loop: How We Built a Circuit Breaker for LLM Costs
The $400/Day AI Loop: How We Built a Circuit Breaker for LLM Costs
A single stuck agent loop cost us $400 in one day. We built CostGuard so it never happens again.
The Wake-Up Call
We run 20+ AI agents on our Mac Mini infrastructure. One evening, a content repurposing agent got stuck in an infinite refinement loop — taking a 500-word blog post and "improving" it 47 times.
We discovered this from the AWS bill. Three days later.
That's the problem with AI agents in production: you don't see the costs until it's too late. LLM APIs charge per token, and a stuck agent burns through tokens faster than you'd believe.
What CostGuard Does
CostGuard is a real-time cost monitoring and circuit breaker system that sits between your application and 7 LLM providers:
- OpenAI (GPT-4o, GPT-4.1, o4-mini)
- Anthropic (Claude Sonnet 4, Claude Haiku 4.5, Claude Opus 4)
- Google (Gemini 2.5 Pro, Gemini 2.5 Flash)
- Cohere (Command R+, Embed)
- Meta (Llama 4 via Groq/Together)
- Mistral (Mistral Large, Codestral)
- DeepSeek (DeepSeek-V3, DeepSeek-R1)
Three Layers of Protection
Layer 1: Track Every Call Per-call monitoring by agent, model, project, and endpoint. You see exactly where your tokens are going in real-time.
Layer 2: Budget Alerts Set monthly budgets per agent or globally. Alerts fire at 80%, 90%, and 100% thresholds. Slack, email, or webhook notifications.
Layer 3: Circuit Breakers When a budget is exceeded, CostGuard trips a circuit breaker. The agent's API calls are blocked — but your service keeps running. No crashes, no cascading failures.
Zombie Agent Detection
This is the killer feature. CostGuard analyzes API call patterns to detect agents stuck in loops:
- Rapid repeated calls to the same endpoint
- Degrading output quality (the agent keeps "refining" into nonsense)
- Token usage far exceeding historical averages for that agent
When a zombie is detected, CostGuard sends an alert, shows you the offending prompt, and can auto-terminate the loop.
Integration: One Line of Code
# Before
response = openai.chat.completions.create(...)
# After — one line changed
response = costguard.wrap_openai().chat.completions.create(...)
Same for Anthropic:
response = costguard.wrap_anthropic().messages.create(...)
No refactoring. No architecture changes. The SDK intercepts API calls transparently.
The Dashboard
CostGuard runs as a FastAPI server with a React dashboard. You get:
- Real-time cost per agent, per model, per project
- Historical trends with 30-day projections
- Budget consumption gauges
- Zombie agent alerts
- Export to CSV for accounting
Results After One Month
In our first month running CostGuard across our 20-agent fleet:
- Caught 3 zombie agent loops (saved ~$280)
- Identified 2 agents with 3x higher cost than alternatives (switched to cheaper models)
- Reduced monthly LLM spend by 31% through better model selection
- Zero surprise AWS bills
Try It
CostGuard is live at costguard.onrender.com. The SDK is open source.
If you're running AI agents in production, you need a circuit breaker. Don't wait for the $400 day to prove it.