Every AI agent demo is cheap. A proof of concept runs on a laptop, costs a few dollars, and the invoice at the end of the week is small enough to ignore. Then it ships, real traffic arrives, and the true cost structure shows up — almost always in the wrong direction.

The uncomfortable truth is that per-token pricing is the easy part to reason about, and for many products it isn't even the largest line. What breaks budgets is everything you bolt on around the model to make its output reliable, observable, and safe.

The token bill is the cheap part. The expensive part is everything you attach to make the tokens trustworthy.

Field notes from an AI platform team

Why agent costs don't look like API costs

Traditional cloud workloads are roughly linear: more requests, more cost, in a straight line you can forecast. Agents break that intuition, because an agent rarely makes one call — it loops.

Context accumulation

Every step of an agent loop re-sends the growing conversation, the system prompt, and any retrieved context back into the model. In audits of production agent systems, that re-sent context routinely accounts for the majority of the bill. A single agent can burn far more tokens than the same task handled in one shot — often by more than an order of magnitude on long, multi-step runs.

Output is where it hurts

Output tokens typically cost several times more than input tokens. An agent that "thinks out loud" before every action multiplies that expense with no change to what the user actually sees.

The line items nobody budgets for

Here is where the gap between the plan and the invoice actually lives.

Cost line	Usually budgeted?	Where it hides
Model tokens	Yes	The invoice everyone reads
Retries & re-planning	Rarely	A stuck agent loops and burns tokens fast
Evaluation & QA	Rarely	You have to prove it works — continuously
Human-in-the-loop	Almost never	The 5% of cases that need a person
Observability & tracing	Sometimes	Logging every step of every run isn't free
Guardrails & safety	Rarely	Filters, red-teaming, policy checks
Model version churn	Almost never	Every model update re-opens your evals

The through-line: none of these appear on a vendor pricing page, and all of them scale with usage.

What you budget vs. what you pay

A rule of thumb from teams who have been through it:

For every dollar of model spend in year one, plan for two to four dollars of everything else — evaluation, observability, review, and the engineering time to keep it all running.

The costs you can actually control

The good news: most of the runaway is controllable once you can see it. The highest-leverage moves tend to be:

• Prompt caching — cached input is billed at a fraction of normal rates and can cut agent bills by half or more.
• Ruthless context management — retrieve the paragraph you need, not the forty pages around it.
• Deterministic checks around the model — validate before you retry, so one failure doesn't trigger an expensive re-run.
• Continuous evaluation — catch quality regressions before they reach users, and before they become a support queue.

The infrastructure all of this rides on:

Watch: where agentic workflows are heading

For the bigger picture on why agents loop the way they do — and why that shape drives cost — this talk is a solid primer:

https://www.youtube.com/watch?v=sal78ACtGTc

The takeaway

Budgeting an AI agent by its token price is like buying a car and pricing only the fuel. The model invoice is real, but it's the smallest and most visible part of a much larger bill. The teams that model the whole cost — retries, evals, review, observability, and the churn of a moving model landscape — are the ones whose agents stay in production. The rest quietly pull them after the first surprise invoice.