Cost per successful task beats cost per request
Updated August 27, 2026 · first published August 27, 2026
Cost per request is useful, but it is not the unit that users buy. A request that fails, retries three times, or produces an unusable answer is not a successful unit of work. For production AI, the stronger metric is cost per successful task: all provider and tool costs divided by tasks that meet a defined quality threshold.
The metric
Use successful_task_cost = total attributable spend / successful tasks. Include input, output, cache, reasoning, tool, retry, and fallback costs. Define success before measuring it: a support answer accepted by the user, a classification above its confidence threshold, or a workflow completed without human repair.
Why request averages mislead
Request averages improve when you count cheap failed attempts. They also hide the long tail: a feature can have a low median request cost while its failed cases consume expensive reasoning calls and human review. Segment by feature, model, customer cohort, and outcome. Report p50 and p95 successful-task cost alongside success rate.
Instrument the denominator
Every model span needs a request ID and feature tag; the application must then emit an outcome event linked to that ID. Record success, failure reason, human repair, latency, and revenue or service value where available. Reconcile provider invoices to the numerator and sample completed tasks to validate the denominator.
Use it to choose an optimization
If request cost is high but success rate is high, routing or caching may help. If request cost is low but successful-task cost is high, quality is the problem: retries, weak models, missing context, or poor stopping rules. Optimize the metric that includes the failure cost, then verify quality does not regress.
Related
- LLM cost monitoring
- Cost per request as a product KPI
- LLM cost attribution
Related
Want this applied to your own LLM spend? FinOps LLM runs a free audit of your AI costs and shows where the savings are. Book free audit →