Your streamed calls all show 0 tokens in the cost log. What is the most likely cause?

The request did not set stream_options.include_usage

The agent retries a step after broken JSON. What should the meter log?

Both attempts, with attempt numbers

Which calls belong in the critical path?

The calls the user waits for, not background calls

Where the Money and the Milliseconds Go

By Syncfusion®, Inc. · Last updated November 2, 2026

Add cost and latency logging to an AI agent in TypeScript or C#. Read usage from an OpenAI-compatible server, meter every streaming call and retry, and find the calls that make most of the bill.

What you will learn

Read prompt, completion and cached tokens from an OpenAI-compatible usage field
Price one call from its usage and a dated price table
Stream with include_usage and measure time to first token and total time
Meter every model call and every retry in one wrapper, with a request id
Roll records into one report per user action: cost, critical path and hotspots

Measure Before You Optimise

You own the order-support agent in Brightgrid Support. It answers questions like "Where is my order A-1042?".

Two messages arrive in the same week:

Finance: the AI bill is almost nine times the estimate.
Support: customers wait more than 13 seconds for an answer.

You cannot fix either one yet. You do not know which model calls cost the money or the time.

In this lesson, you will add cost and latency logging to the agent's code. Then you will read the trace your code produces and find the hotspots.

Press Play, or use Next to go one step at a time.

Each row: one model call the agent makes for one ticket.
The numbers: cost per 1,000 tickets, and seconds.
The last frame: the three calls that make most of the bill.

Simple lesson: one user action is usually many model calls. You need a record of every one.

A Real-World Example

The team priced the feature as one model call per ticket: the reply.

The cell below runs both numbers against the fake server. The estimate prices the reply call only. The metered number runs the real agent and logs every call.

Press Run. Then change perDay to your own traffic and run it again.

estimate: one reply call per ticket, per month.
metered: every call the agent really makes.
ratio: how far off the estimate is.

Simple lesson: an estimate that counts one call per user action is usually wrong for agents.

What You Will Build

In this lesson, you will learn:

what an OpenAI-compatible server returns in usage, and how to price it
why a streaming call has no usage unless you ask for it
how to measure time to first token and total time
how to log one cost record per call, retries included
how to roll records into a report per user action: cost, wait time and hotspots

You will finish three files in the course project, in TypeScript or C#. Pick the language in the top bar.

Cost record (src/cost/costRecord.ts, or Cost/CostRecord.cs): turns usage into a priced record.
Meter (src/cost/meter.ts, or Cost/Meter.cs): wraps every streaming call and logs a record.
Report (src/cost/report.ts, or Cost/Report.cs): rolls records into one report per user action.

Open the Code tab.

Left: the project files. The three cost files start with TODO lines.
The harness file: the fake server, the price table and the agent. It is read-only.
The two buttons: run the tests, or run the agent and print the trace.

The lab server is fake, and the calls are real code. Real servers such as Ollama, vLLM and most hosted APIs return the same OpenAI-compatible shape. In this lab, a fake client replays recorded replies on a fake clock. So your code gets the same numbers on every run.

Simple lesson: cost tracking is normal application code. It lives next to your model client.

Read the Usage Field

Every reply from an OpenAI-compatible server carries a usage object.

prompt_tokens = all the input tokens, cached ones included.
completion_tokens = the output tokens.
prompt_tokens_details.cached_tokens = the part of the input the server had cached. It is often missing when nothing was cached.

Run the cell. Look at the three numbers and the time.

Other APIs use other names. Anthropic's API, for example, uses input_tokens, output_tokens and cache_read_input_tokens. Map them to one record shape in one place.

Price One Call

The price list gives dollars per 1 million tokens, with a different price for each kind of token.

cost = new input × input price + cached input × cached price + output × output price, all divided by 1,000,000

Run the cell. Then change the step to "agent.pick_tool" and run it again.

input: almost the whole cost of this agent step.
output: small, even though output costs more per token.

Step through all three frames. The last frame previews what caching does to the same call. You will build it in Prompt Architecture and Caching.

Open the Prices tab. This is the PRICES table your code reads.

Simple lesson: price each kind of token at its own price, and fail loudly when a price is missing.

Stream, Time and Meter Every Call

Most chat features stream the reply, so the user sees words early. Streaming changes what you must measure.

Usage Arrives Last, Only If You Ask

With streaming, an OpenAI-compatible server usually sends usage in one extra chunk at the end. It only sends it when the request has stream_options: { include_usage: true }.

Run the cell.

First line: no include_usage, so the usage is missing. A cost log built on this would show zero.
Second line: the same call with include_usage. Now the usage arrives in the last chunk.

Time to First Token and Total Time

Time to first token (TTFT) = from sending the request to the first visible text. It grows with the input, because the model reads it all first.
Total time = from sending the request to the last chunk.

Streaming hides the writing time from the user. It does not hide TTFT, and it does not hide calls that run before the reply.

Step through all three frames.

Frame 1: one call, split into first token and writing.
Frame 2: the calls before the reply add up.
Frame 3: the background call costs money but no waiting.

One Wrapper for Every Call

Put the timing and the logging in one function that every model call goes through. Then no call is missed.

The wrapper must:

add include_usage to the request
note the time of the first text chunk and the last chunk
log a record for every attempt, because a failed attempt is billed too
carry a request id, so all calls for one user action can be added up later

A retry is a full extra call. Here, the agent's "read the order" step returns broken JSON, so the agent asks again. Both attempts read about 6,000 tokens. If your wrapper logs only the successful attempt, you hide one of your most expensive calls.

Simple lesson: one wrapper, every call, every attempt, with the request id.

Roll Up Each User Action

Single records are not the answer to "what does a ticket cost?". You need one report per user action.

The Multipliers

Agent loops: one call per step, and each step resends the instructions, tools and history.
Retries: a broken reply is asked for again.
Judges: a second model checks the first one.
Extra steps: classifiers, rewrites and summaries.

Step through all three frames.

Left: what the estimate priced.
Right: what one ticket really does.
Frame 3: the gap per month at three traffic levels.

Critical Path and Hotspots

Cost per action = the sum of all records with the same request id.
Critical path = the time the user waits. Add up only the calls the user waits for. Leave out background calls such as the CRM summary.
Hotspots = the few costliest calls. Fix these first.

Run the trace command, then open the Trace tab.

Each row: a record your meter logged.
Red rows: the hotspots your summarize found.
At the bottom: cost per 1,000 tickets, calls, retries and the user's wait.

Check it with your own model. Point the same wrapper at a local server such as Ollama. Its /v1/chat/completions endpoint returns the same usage shape. Compare its numbers with the lab's before you trust a saving.

Simple lesson: group records by request id. Cost per action and the critical path come from that group.

Try it on a real model. Your meter logs the usage numbers a real model returns. Run the ticket on a real model, and read the record it would log. Switch Model to Real in the lab's top bar. It runs an open model in your browser with WebLLM, and needs WebGPU. It is not graded.

Put It in Your App

Two token costs hide in agents. Tool definitions are input tokens, sent again on every agent step. Reasoning ("thinking") models bill hidden reasoning tokens as output. Your meter must count both: The Cost Equation shows how.

Where the records go depends on your stack.

Logs: one structured log line per call. Easy to add up per feature, user and day.
Traces: OpenTelemetry has standard attributes for model calls, such as gen_ai.usage.input_tokens and gen_ai.usage.output_tokens. Add cost and attempt as your own attributes.
Prices: keep them in one versioned file with the date they were checked. Prices change often, so never hard-code them in the call site.

For product managers: you do not need to write this code. Ask your team for three numbers per feature. All three come from these records:

cost per user action
the user's wait
the top 3 hotspots

Simple lesson: log every call once, in one place, with the request id and the price date.

Common mistakes

Mistake	Wrong idea	Correct idea	What goes wrong
Streaming without `include_usage`	"The server always sends usage."	Ask for it in `stream_options`.	Every streamed call is logged at $0.
Pricing `prompt_tokens` at the full input price	"Cached tokens are separate."	Subtract `cached_tokens`, then price it at the cached price.	Cached calls look more expensive than they are.
Logging only the successful attempt	"The retry does not count."	Log every attempt with its attempt number.	The costliest calls vanish from the report.
No request id	"We can add calls up by time."	Pass one request id through every call of an action.	You cannot say what one ticket costs.
Adding background calls to the wait	"Every call slows the user."	Add only calls on the critical path.	You optimise calls the user never waits for.
A missing price becomes zero	"Unknown models are rare."	Throw, or alert, when a model has no price.	A new model looks free in every report.

Check Your Understanding

Exercises

Build It in the Project

The other files of this lesson's project are optional practice. Finish them to run the lesson's script on your own code.

What to Remember

One user action usually makes many model calls. Log every one.
usage gives prompt, completion and cached tokens. Cached tokens are inside prompt_tokens.
With streaming, ask for include_usage, or the usage never arrives.
Time to first token grows with the input. Writing time grows with the output.
Log every attempt, with a request id, through one wrapper.
Cost per action and the critical path come from grouping records by request id.