The most underrated metric in enterprise AI right now isn’t accuracy, isn’t latency, isn’t a benchmark number. It’s cost per completed task. The companies we watch pulling ahead in 2025 are not the ones with the biggest AI budgets — they are the ones whose unit economics make shipping a tenth feature just as affordable as shipping the first. The gap between those two postures is now measurable on a balance sheet.
Cost per task is a composite metric. It includes model calls, retrieval queries, guardrail evaluations, cached responses, failure retries, and the long tail of very hard tasks that burn tokens disproportionately. A team optimising one of these in isolation rarely moves the total; a team optimising the whole system end-to-end typically cuts cost per task by three-quarters in the first quarter they start paying attention.
Five techniques that actually move cost per task in production:
- Cascaded models with confidence-based escalation. Run a small cheap model first; escalate to a bigger one only when the small one reports low confidence or fails a validation check. Done well, this routes 70–90% of traffic to the cheap tier and delivers a quality indistinguishable from always-using-the-big-model.
- Prompt compression and context pruning. Most production prompts are carrying 30–50% payload they don’t need. Caching the unchanging system prompt, compressing retrieved context with a small extraction model, and removing redundant examples are all unglamorous ways to cut bill immediately.
- Aggressive response caching, scoped narrowly. Most AI systems see the same questions repeatedly. A properly-scoped semantic cache — careful with personalisation — eliminates a significant fraction of expensive model calls. The trick is choosing the scope: too broad and you cache wrong answers; too narrow and you cache nothing.
- Batching and async execution where the UI allows. Not every task needs a sub-second response. Background processing at off-peak times, using lower-priority API tiers, is a quiet ten-to-thirty percent saving that nobody sees except the finance team.
- Instrument the long tail ruthlessly. The 95th-percentile task in a typical AI workflow often costs 20× the median one. Dashboards that surface cost per request distribution — not just average — let you hunt down the handful of requests that are silently consuming most of the budget.
Model pricing has fallen by more than an order of magnitude since 2023, and it will fall again. The instinct that “we’ll wait for inference to get cheap” is not wrong, exactly — but teams that wait tend to also miss the compounding discipline that comes from treating cost as a product metric. The companies who optimise for unit economics while prices are still high ship more features when prices fall. The ones who didn’t just end up with a bigger version of the same inefficient system.
