What is a practical AI agent workflow architecture?

Use an orchestrator with bounded steps, allowlisted tools, durable state and a shared budget ledger. Record every billable model and tool attempt under one workflow ID.

How do you stop parallel agents overspending?

Atomically reserve the next attempt against both workflow and owner budgets before dispatch. Include outstanding reservations in available balance and retain holds for unresolved usage.

What should an agent workflow log?

Log workflow and attempt IDs, ownership, model and tier, billed usage categories, tool fees, latency, outcome, price version and reconciliation status.

Does this policy template work directly in an agent SDK?

No. It is an application policy template. Map its fields to your orchestrator and provider settings, then test admission, retries, recovery and settlement.

How do you measure agent cost per successful task?

Divide total spend across all attempts, including failures and retries, by the number of tasks that pass the same acceptance check. Report completion rate alongside cost.

A practical AI agent workflow: architecture, budgets and logging

Updated October 22, 2026 · first published October 22, 2026

A practical AI agent workflow needs a bounded sequence of steps, a shared budget ledger and a record of every billable attempt. Start with one orchestrator and a small tool allowlist. Add parallel work or specialist agents only when a measured quality or latency gain justifies their cost.

This guide builds an invoice-variance triage workflow: read a billing summary, investigate a change, and return an evidence-linked explanation for a human reviewer. It includes a policy template, a logging example, retry rules and a worked cost ledger. The design and dollar figures are illustrative; they are not a claim that FinOps LLM ships this orchestration runtime.

What should the agent workflow architecture look like?

Billing summary + question
          |
Validate input and identify budget owner
          |
Orchestrator ---- shared budget ledger
          |
Plan -> allowlisted read tools -> draft -> review
          |                       |
          +---- attempt events ---+
                                  |
                    Check evidence and arithmetic
                                  |
                   Report or explicit stop reason

The orchestrator owns state transitions and stop conditions. The model proposes actions inside that boundary. Model and tool calls both pass through budget admission and event recording. The final report needs a deterministic check that its numeric totals match the billing summary and that its cited records exist.

Keep execution state and spend state durable. A queue retry must resume the existing workflow ID rather than create a new budget. Subagents, if enabled later, inherit that ID and the owner's budget instead of receiving fresh independent allowances.

Which steps belong in a multi-step agent workflow?

Step	Work	Exit condition
1. Validate	Check period, currency, tenant and required ownership tags	A valid input and a known price snapshot
2. Plan	Select a small set of questions about volume, rates or model mix	A structured plan within the step limit
3. Read evidence	Fetch billing aggregates and the relevant pricing snapshot	Bounded results with record IDs
4. Draft	Explain the variance using only the retrieved evidence	A report with amounts, sources and uncertainty
5. Review	Optionally use a model to check unsupported explanations; always verify totals in code	Accepted, rejected or checkpointed for a human

Use a fixed workflow when the steps are already known. A free-form planner adds value when evidence determines the next question; it should not repeat arithmetic a deterministic function can perform. Do not give this reporting agent tools that change billing accounts or payment settings.

What configuration can you reuse?

The following is valid JSON for an application policy template, not configuration accepted by a particular agent SDK. Replace the model labels, tool names and limits with your own approved choices, then implement each check in the orchestrator. These example limits are starting assumptions, not production recommendations.

{
  "workflow": "invoice-variance-triage-v1",
  "currency": "USD",
  "owner_tags_required": [
    "team",
    "feature",
    "tenant",
    "environment"
  ],
  "limits": {
    "max_workflow_cost_usd": 0.2,
    "max_attempt_reservation_usd": 0.05,
    "daily_owner_budget_usd": 50,
    "daily_reset_timezone": "UTC",
    "max_model_attempts": 6,
    "max_tool_attempts": 4,
    "max_parallel_tools": 2,
    "max_subagents": 0,
    "workflow_deadline_seconds": 120,
    "max_input_tokens_per_attempt": 12000,
    "max_output_tokens_per_attempt": 600,
    "max_tool_result_bytes": 8000
  },
  "retry": {
    "max_retries_per_step": 1,
    "retryable": [
      "rate_limited",
      "temporary_unavailable"
    ],
    "backoff_ms": 500,
    "jitter": true,
    "timeout_billing": "retain_reservation_until_reconciled"
  },
  "tools": {
    "allow": [
      "billing.read_summary",
      "pricing.read_snapshot"
    ],
    "writes_allowed": false
  },
  "routing": {
    "default_model": "approved-economy-model",
    "fallback_model": "approved-premium-model",
    "fallback_requires_new_reservation": true
  },
  "on_budget_denial": "checkpoint_and_return_budget_exceeded",
  "on_unknown_price": "stop_before_dispatch",
  "on_unknown_usage": "mark_pending_and_keep_reservation"
}

The six-attempt limit counts all model dispatches, including retries, fallbacks and review calls. The tool-attempt limit also includes retries. A per-step retry allowance cannot bypass those workflow-wide ceilings. Check input tokens before sending and apply the provider's output cap in the actual request.

An 8,000-byte tool-result limit is not an 8,000-token limit. Enforce both the result-size boundary and the model's input-token budget. If required evidence cannot fit, paginate or checkpoint; do not silently truncate the amounts needed to explain the variance.

How do you enforce a budget before parallel calls?

Checking yesterday's dashboard is too late. Admit each attempt against a shared ledger before dispatch. The same atomic operation must check both the workflow limit and the owner's daily limit:

1. Estimate the maximum permitted charge for the next attempt.
2. Atomically check spent + outstanding reservations + new reservation
   against BOTH the workflow and daily-owner budgets.
3. If either check fails, checkpoint and return budget_exceeded.
4. Otherwise create one reservation for the unique attempt ID.
5. Dispatch, with explicit input, output and time limits.
6. Settle the reservation with observed usage; retain unresolved holds.

For a model call, the reservation covers the allowed input, maximum billed output, service tier and any applicable cache-write or long-context rate. For a paid tool, include its maximum permitted fee or require a bounded quote. If you cannot bound the charge, stop or escalate instead of claiming a hard monetary cap.

Use a transaction or equivalent compare-and-set across the workflow and owner accounts. Two parallel workers must not each see the same unreserved balance. At $0.12 already spent, a $0.20 workflow has room for one $0.05 reservation, leaving $0.03; a second $0.05 attempt must wait or be denied.

If an attempt costs less than its reservation, settle the actual estimate and release only the unused balance. If it exceeds the reservation, record the full amount and stop further admissions. A timeout does not prove a request was free: retain its reservation as pending until provider usage or billing resolves it. Late usage belongs to the original attempt and budget period.

What should you log for every step?

Use a parent workflow trace and a child span or event for each attempt. OpenTelemetry's GenAI conventions provide model/provider and input/output usage attributes; the conventions remain in development. Keep a versioned mapping from provider responses to your accounting schema.

This example uses application-defined field names. It models a successful draft attempt with 6,000 ordinary input tokens and 600 billed output tokens at assumed rates of $2 and $10 per million:

{
  "event": "model_attempt_completed",
  "workflow_id": "variance-20261022-0042",
  "trace_id": "trace-0042",
  "step_id": "draft-report",
  "attempt_id": "draft-report-2",
  "parent_step_id": "triage",
  "team": "platform",
  "feature": "invoice-triage",
  "tenant": "tenant-042",
  "environment": "production",
  "provider": "example-provider",
  "requested_model": "approved-economy-model",
  "returned_model": "approved-economy-model",
  "service_tier": "standard",
  "provider_request_id": "provider-request-0042",
  "usage": {
    "input_total_tokens": 6000,
    "input_uncached_tokens": 6000,
    "input_cache_read_tokens": 0,
    "input_cache_write_tokens": 0,
    "output_billed_tokens": 600
  },
  "duration_ms": 3200,
  "status": "succeeded",
  "estimated_cost_usd": 0.018,
  "price_snapshot": "illustrative-usd-2026-10-22",
  "reconciliation_status": "estimated"
}

Also emit reservation-created, dispatch-started, attempt-failed, reservation-settled, budget-denied and workflow-finished events. Record a distinct outcome for a completed report that fails quality checks. Use attempt IDs to make duplicate event delivery idempotent; a queue redelivery must not add the same charge twice.

Retain provider usage or a restricted reference to it for reconciliation. A nested cached-token count may already be included in total input; bill its category once. The same applies to reasoning tokens included in output. Keep cache writes split by retention when the provider prices those categories differently. A tool that invokes a model needs its own attributed child usage, with a single accounting owner.

Routine telemetry should contain record IDs and bounded metadata rather than full invoices, prompts or customer identifiers. Store evidence separately under the workload's access and retention rules. Emit workflow IDs in traces or logs; avoid using a unique workflow ID as a metric label.

How should retries, fallbacks and automation behave?

Retry only eligible failures: bounded retries for temporary errors, with jitter and any provider retry-after instruction. Validation failures and missing access need an explicit outcome.
Reserve again before a retry: the first attempt's possible charge remains in the ledger. Retrying is another potential bill.
Handle uncertain dispatch: record a dispatch intent before the network call. If a worker crashes after sending, recover by request ID or supported provider idempotency rather than blindly replaying it.
Treat fallback as a new choice: verify the alternative model, price, context limits and remaining budget before dispatch.
Checkpoint on denial: preserve retrieved evidence and the next intended step. Return the exhausted limit and permitted recovery to the caller.

This example allows read-only tools. A workflow that writes to an external service needs an idempotency key or a recovery process for ambiguous results. A generic retry policy is not sufficient for an action that might already have happened.

What does a complete workflow cost?

The following ledger is synthetic. Model attempts use the same assumed $2 input/$10 output rates with no caching. The search tool has an assumed $0.002 external fee. Infrastructure and human review are excluded.

Attempt	Usage or fee	Cost
Plan	1,000 input + 100 output tokens	$0.003
Read evidence	One paid search, no nested model call	$0.002
Draft attempt 1: failed	2,000 input + 100 output tokens, usage confirmed	$0.005
Draft attempt 2: succeeded	6,000 input + 600 output tokens	$0.018
Model review	1,000 input + 100 output tokens	$0.003
Total	Four model attempts and one tool attempt	$0.031

The failed draft remains billable in this scenario. The deterministic arithmetic check adds no model tokens; its compute belongs in infrastructure allocation. Do not count both the $0.002 tool fee and an identical bundled fee on a second ledger line.

If 100 attempted workflows each cost this illustrative $0.031, total spend is $3.10. If 90 pass the acceptance check, cost per successful task is $3.10 / 90 = about $0.0344. Report the 90% completion rate alongside it. Comparing only the final successful model call would hide retries and failed workflows.

How do you test the workflow before automating it?

Replay a fixed task set: declare what a correct report must contain and verify amounts against trusted records.
Exercise the limits: an oversized prompt, repeated tool failure and exhausted owner budget must return clear outcomes without more dispatches.
Race parallel admissions: simultaneous reservations must not exceed either shared balance.
Simulate timeout and recovery: unresolved charges remain pending, and a repeated completion event settles only once.
Check price failures: an unknown model or stale price snapshot must not silently become a zero-cost request. Define the price freshness policy explicitly.
Reconcile a small pilot: match recorded requests and fees to provider totals, explain differences, and only then increase automation.

The operational dashboard should show cost per successful task, completion rate, p95 latency, retries per task, budget denials, pending charges and unattributed spend. Alert thresholds should come from the pilot's workload and quality expectations.

Where should you go next?

Use the MCP cost guide to isolate tool-related overhead, the Astra cost guide for a concrete provider pricing example, and agent spend attribution for the owner and tenant dimensions. Start with one bounded workflow and verify the ledger before adding more agents.

Sources and limitations

Design references checked 22 October 2026: MCP architecture, OpenTelemetry GenAI spans, and OpenAI usage and cache accounting. Budget admission, recovery policy and the configuration above are the article's proposed application design. They require implementation and testing in your own runtime.

Want this applied to your stack? Bring the provider bills, gateway logs, and top workflows; we will map the cost drivers and savings path. Book a free audit →

Back to finopsllm.com