Why compare with the median of the previous hours, not the mean?

The median is not pulled up by the spike itself

The median is faster to compute

The median is not pulled up by the spike itself

Where should the budget check run?

In the same wrapper as the meter, before every call

Budgets, Limits and Cost Alerts

By Syncfusion®, Inc. · Last updated November 2, 2026

Put limits around every model call: budget checks per user, feature and day, step, retry and cost caps for agent loops, and an alert on cost per request against a rolling median, in TypeScript or C#.

What you will learn

Check a budget before every model call, per user, per feature and per day
Cap an agent loop by steps, retries and cost, with a clear stop reason
Alert on cost per request against a rolling median

The Night an Agent Did Not Stop

At hour 50 of the cost log, cost per request jumps from about $0.0034 to $0.016. A prompt change made the agent call its tools again and again, never deciding it was done.

Nothing stopped it. The provider's monthly spending limit was far away, and nobody looked at the dashboard until the next morning.

Every lever so far made calls cheaper. This lesson puts limits around them, so one bug or one heavy user cannot run up the bill.

Press Play, or use Next to go one step at a time.

Frame 1: an agent loop with no stop rule.
Frame 2: the same loop with step, retry and cost caps.
Frame 3: the cost alert that fires within the first hour.

Simple lesson: the model decides when it is done, and your code decides when it must stop.

A Real-World Example

One customer wrote a script that sent a support question every few seconds. Each question was cheap. Together they used a fifth of the day's budget before lunch.

A per-user daily cap would have stopped the script after a few dollars, and left everyone else unaffected.

Simple lesson: caps belong to the user, the feature and the day.

What You Will Build

In this lesson, you will learn:

how to check a budget before every model call: per user, per feature and per day
how to cap an agent loop by steps, retries and cost
how to alert on cost per request with a rolling median

You will finish three files in the guards folder:

Budget guard (src/guards/budgetGuard.ts, or Guards/BudgetGuard.cs): allow or deny, with a reason.
Agent loop (src/guards/agentLoop.ts, or Guards/AgentLoop.cs): stop rules.
Cost alert (src/guards/costAlert.ts, or Guards/CostAlert.cs): the hours to page someone.

Open the Code tab. The harness has the limits, three agent behaviours (normal, runaway, flaky) and 72 hours of cost per request.

Simple lesson: guards are small, testable functions around the model call.

A Budget Check Before Every Call

Before each model call, estimate its cost from the prompt size and the price table. Then check three caps:

user daily: one user cannot spend more than a set amount a day
feature daily: one feature cannot use the whole budget
global daily: the whole product stops at a hard ceiling

After the call, add the real cost from the cost record.

Run the cell. A heavy user is denied at ticket 15, after about $0.048. Change userDailyUsd in the harness and run it again.

Provider spending limits are a last resort. They are usually per month and for the whole account. They stop everything at once. Your own caps work per user and per feature, and can show a friendly message.

Simple lesson: estimate before, record after, and say why a request was denied.

Stop Rules for Agent Loops

An agent loop needs three stop rules besides "done":

max steps: the loop never runs more than N model calls
max retries: broken replies are retried a few times, then the loop gives up
max cost: the run stops when it has spent its share

Return a clear stop reason, and log it. A rise in max_steps stops is an early sign of a prompt bug.

A retry limit of zero is not safe either. Without retries, one broken reply fails the user. Without a limit, a model that always breaks the format loops forever. Pick a small number, such as 2, and log every retry.

Simple lesson: every loop has a step cap, a retry cap and a cost cap.

Alert on Cost per Request

A daily total rises slowly, and traffic changes it too. Cost per request jumps the moment the agent misbehaves.

Compare each hour with the median of the 24 hours before it. Alert when it is more than 3 times higher.

The median ignores the spike itself. A mean would rise with the spike and hide it.

Run the cell. Try other factors and windows.

Factor 3, 24 hours: alerts at hours 50 to 53, the incident, and nothing else.
Factor 1.3: also alerts on a busy hour. Someone gets woken up for nothing.
Window 6 hours: the spike fills the window, so the last hour is missed.

Run npm run guards (or dotnet run -- guards). The normal agent finishes, the runaway agent stops at its cost cap, the flaky one stops at its retry limit, and the alert fires at hour 50.

Simple lesson: alert on cost per request, against a rolling median.

Rate Limits and Showback

Two more guardrails:

a token bucket per user: each user has a bucket of tokens that refills at a fixed rate. A burst uses the bucket, and a steady flood is slowed down. It protects your budget and your provider quota.
showback: a monthly report of AI cost per team and per feature, from the tags in your records. Teams that see their cost manage it.

Run the cell. The user gets 12 requests in each of the first two hours, then 6 in the third, when the bucket runs dry.

Simple lesson: limit by tokens per user, and show every team its own cost.

Try it on a real model. Run a real loop that never says done. The step cap and the cost cap stop it, and the tab shows the stop reason. Switch Model to Real in the lab's top bar. It runs an open model in your browser with WebLLM, and needs WebGPU. It is not graded.

Put It in Your App

Guard in one place. Call the budget check inside the same wrapper as the meter from Lesson 2, so no call skips it.
Store spend outside the process. A shared store such as Redis keeps the daily totals right across servers.
Tell the user. A denied request needs a clear message, and a way to reach a human.
Connect the alert. Send it to the same place as your other alerts, with a link to the cost records for that hour.

Unbounded consumption is a security risk too. Attackers can use your AI feature to burn your budget or slow it down for others. Per-user caps and step caps are part of your security controls, not only cost controls.

For product managers: the caps are product decisions. Decide the daily limit per plan, what the user sees when they hit it, and who gets the cost alert.

Simple lesson: guards live in the wrapper, totals live in a shared store, and denials explain themselves.

Common mistakes

Mistake	Wrong idea	Correct idea	What goes wrong
Only a provider spending limit	"The provider will stop us."	Add per-user and per-feature caps.	One user or bug stops the whole product.
Checking after the call	"We will see the cost later."	Check an estimate before the call.	The cap is passed on every request.
No step cap	"The agent will finish."	Cap steps, retries and cost.	A runaway loop all night.
Alerting on the mean	"Average cost is a fine baseline."	Use a rolling median.	The spike raises its own baseline.
Daily totals in memory	"One server is enough."	Keep totals in a shared store.	Caps reset on every deploy.

Check Your Understanding

Exercises

Build It in the Project

The other files of this lesson's project are optional practice. Finish them to run the lesson's script on your own code.

What to Remember

Check a budget before every call: per user, per feature and per day.
Record the real cost after the call.
Cap agent loops by steps, retries and cost, and log the stop reason.
Alert on cost per request against a rolling median.
Keep the guard in the same wrapper as the meter.

Next: Build vs Buy: Open-Weight, Self-Host or API. You will find the volume where running the model yourself starts to pay.