Observability, Cost and Reliability

By the Syncfusion DevTutorials team · Last updated October 27, 2026

Read a trace to see what every step of an agent costs in coins and time. Cut a bloated prompt, set a budget per message, add timeouts and retries, and send easy questions to a smaller model.

Observability, Cost and Reliability

A customer types, "Are you open on Sunday?"

Shop Helper answers in one line. The answer is right.

Then the bill comes in. That one line cost about 400 coins. A normal answer costs about 12.

Nobody noticed, because nobody was watching the steps.

This lesson shows how to see what every step costs. It also shows how to set limits before the bill gets big.

Simple lesson: watch every step's cost and time, and set limits before it gets expensive.

A Real-World Example

Think about a phone bill that is far too high.

• You look at the total. It only tells you something went wrong.
• You open the list of calls. One call lasted three hours.
• Now you know what to fix.

The list of calls is what helps, not the total.

An agent needs the same kind of list, one line for every step it takes.

Simple lesson: a total tells you there is a problem, and a list of steps tells you where.

What You Will Learn

In this lesson, you will learn:

1. How to read a trace and find the most expensive step.
2. Why pasting everything into every call costs so much.
3. How to set a budget for each message.
4. How timeouts and retry with backoff handle a flaky tool.
5. How to send simple questions to a smaller model, and check that it still works.

The 400-Coin Question

You met coins in Workflow or Agent?. One coin stands for 100 tokens the model reads or writes.

Real prices vary by provider and change often. Real providers usually charge more for the words a model writes than for the words it reads. Many also offer prompt caching, which makes resending the same start of a prompt cheaper. In this lab, one coin is a simple stand-in for both.

Trace = a record of every step in a run, with its cost and time.

Here is the trace for "Are you open on Sunday?" This helper pastes two things into every model call:

• the whole help centre, all 34 pages,
• the customer's full chat history, 8 past chats.

Press Play and watch the rows of the trace appear.

• Step 1: the model reads almost 20,000 tokens just to pick a tool.
• Step 2: the help page search costs no coins at all.
• Step 3: the same huge pile is pasted in again for the answer.
• The total: about 40,000 tokens, or 400 coins, and over 30 seconds.
• The last frame: a budget stops the run before the first call.

The answer was right. That is why nobody noticed. Tests that only check the answer will not catch a cost problem.

Simple lesson: a big prompt times several calls makes a huge bill, even for a tiny question.

Reading a Trace

A trace has one row per step. Each row says:

• Who did the step. It is the model, a tool, or the app waiting.
• What it did, in a few words.
• How many coins it cost.
• How many seconds it took.

To find a cost problem, find the row with the most coins. Then ask why it is so big.

Press Play and read each row as it appears.

• Model rows: they cost coins, because the model reads the whole chat each time.
• Tool rows: they cost no coins here, but they still take time.
• The glow: it marks the costliest row.
• The last call: it is the biggest, because the chat is longest by then.

Real tools often show traces as a list or a timeline. Many log tools and agent kits can record them for you. In this lab, a trace is a simple table.

Simple lesson: read the coins column to find the costly step, then ask why it is big.

Lean, Not Bloated

The fix for the 400-coin question is to send less.

The model does not need every help page in every call. It has a search tool, so it can fetch the one page it needs.

It also does not need every old chat. Only this chat matters here. State, Memory and Context Limits shows how to keep the useful parts.

Press Play and compare the two traces.

• Bloated: every page and every old chat, pasted into every call.
• Lean: only this chat and the tool list.
• The answers: both are right.
• The totals: the lean helper costs a tiny part of the bloated one.

Pasting a few short help pages is fine (see Workflow or Agent?). Pasting a whole help centre and every old chat into every call is what makes it expensive.

Notice the last frame. The lean helper is kept only because its answer is still right.

Simple lesson: send the model what this step needs, and let tools fetch the rest.

A Budget for Every Message

Mistakes like the bloated prompt will happen again. A budget catches them before the bill does.

You met the budget limit in The Agent Loop by Hand. It is the most coins one customer message may spend.

Set it from your test chats:

• Run every good test chat.
• Find the one that costs the most.
• Set the budget a little above it.

A good chat then always finishes. A runaway chat, like a loop stuck on a missing order, stops early.

The budget check runs before each call. The app can count the prompt before it sends it.

So a 20-coin budget would have stopped the 400-coin question before its first call. The prompt alone costs about 200 coins. The helper then sends a short honest message, or the app trims the prompt. That stop is a clue. It tells you to read the trace.

Simple lesson: set a budget per message just above your dearest good chat.

When a Tool Is Slow or Flaky

Cost is one problem. Failure is another.

The order lookup goes through a shipping service. Sometimes that service is busy. Sometimes it says nothing at all.

Timeout = the longest the app will wait for a tool before it stops waiting.

Retry with backoff = try again after a failure, waiting a bit longer each time.

The app has three choices when the service is busy:

• give up at once,
• hammer it: try again with no wait, many times,
• retry with backoff: wait 1 second, then 2, then 4, then stop.

Press Play and watch the ticks on each line.

• The yellow zone: the service is busy for its first 5 seconds.
• Give up: one quick failure, even though the service came back soon.
• Hammer: six tries bunched together, all inside the busy zone.
• Backoff: the tries spread out, and the last one lands after the service is back.

Hammering also adds load to a service that is already struggling. Real services may block you for it.

Backoff needs a stop, too. After the last wait, the app gives up and tells the customer the truth.

Now the second problem. The service hangs and never answers.

Press Play and watch the blue timeout line.

• No timeout: the customer waits 30 seconds for nothing.
• A 3-second timeout: the app stops waiting and takes charge again.
• Timeout plus backoff: the second try works, in under 5 seconds.

Safe to Retry?

Not every tool call is safe to repeat.

• A lookup is safe. Reading an order twice changes nothing.
• An action may not be. A refund or an email may have happened even if the answer timed out.

So before you retry an action, check first whether it already happened. Or send a request ID, so the service ignores repeats. Otherwise the customer may be refunded twice.

When the Model Provider Fails

The model service itself can fail, too. It can be busy (rate limits), slow or down.

• Use timeouts and backoff for model calls as well.
• Keep a fallback, such as a second model or an honest "please try again" message.
• Limit how many messages one user can send per minute, so one person cannot run up the bill.

Simple lesson: give every slow tool a timeout, and retry with longer and longer waits.

Small Model, Big Model

Not every question needs the biggest model.

Many providers offer a small model and a big one. A small model usually costs less and answers faster. It may fail harder questions.

In this lab, the small model costs a fifth as much. It follows two plain rules:

• With two jobs in one message, it does the first job and guesses the rest.
• It does not know a late parcel needs the late-parcel rule. It guesses that rule.

The big model solves all of these. It even notices a late parcel by itself.

Model routing = sending simple questions to a smaller, cheaper model, and hard ones to a bigger model.

A router rule picks the model before any model sees the message. Ours plays safe. Only help questions with one job, no order number and no late parcel go to the small model.

Anything with an order goes to the big model, even a simple lookup. The router cannot know if the order is late until after the lookup.

Press Play and watch the two doors and the scorecard.

• All big: every chat is solved, but simple ones pay big prices.
• All small: much cheaper, but two chats fail.
• Routed: simple questions go through the small door.
• The scorecard: the same chats solved, for fewer coins.

Cost & Model Selection, coming later, goes deeper into choosing models and prices.

Simple lesson: send simple questions to a small model, and prove it still works before you keep it.

Where This Is Used

Every agent in a real product needs these habits:

• Support helpers watch the cost per chat and set a budget.
• Coding helpers call slow tools, so they need timeouts.
• Busy apps route simple questions to small models to save money.

Many teams check their traces every week. A rising cost is often the first sign of a bug.

Simple lesson: watch traces all the time, not just when the bill shocks you.

In Code

Here is a trace, a budget, retries and a router in TypeScript. It uses the same Shop Helper names (lookup_order, order 1044).

Trace every step and check the budget

// Trace every step: tokens, coins and seconds. Check the budget before each model call.
interface TraceRow { step: string; tokens: number; coins: number; seconds: number }
interface StepResult<T> { value: T; tokens: number; seconds: number }

const BUDGET_COINS = 20; // per customer message
const rows: TraceRow[] = [];
let spent = 0;

function trace<T>(stepName: string, fn: () => StepResult<T>): T {
  const r = fn();
  const coins = r.tokens / 100; // 1 coin = 100 tokens
  rows.push({ step: stepName, tokens: r.tokens, coins, seconds: r.seconds });
  spent += coins;
  return r.value;
}

// The app counts the prompt before it sends it.
function budgetAllows(promptTokens: number): boolean {
  return spent + promptTokens / 100 <= BUDGET_COINS;
}

// Fake model and fake shop: fixed sizes and times, so the output never changes.
// They are plain functions here. A real app awaits them.
function fakeModel(promptTokens: number, reply: string): StepResult<string> {
  return { value: reply, tokens: promptTokens + 100, seconds: 1.5 };
}
function lookupOrder(id: string): StepResult<string> {
  return { value: `order ${id}: shipped, late, arrives Monday 3 Nov`, tokens: 0, seconds: 0.8 };
}

const tool = trace("model: pick a tool", () => fakeModel(500, "lookup_order"));
const facts = trace(`tool: ${tool} 1044`, () => lookupOrder("1044"));
trace("model: write the answer", () => fakeModel(700, `Sorry, ${facts}.`));

let top = rows[0];
for (const r of rows) if (r.coins > top.coins) top = r;
console.log("step                     coins  secs");
for (const r of rows) {
  const mark = r === top ? "  <- costliest" : "";
  console.log(`${r.step.padEnd(25)}${String(r.coins).padStart(5)}  ${r.seconds.toFixed(1)}${mark}`);
}
console.log(`total: ${spent} coins of ${BUDGET_COINS}`);

// A bloated prompt: the whole help centre pasted in again.
const bloated = 19800;
console.log(budgetAllows(bloated) ? "sent" : `budget stop: next call needs ${bloated / 100} coins, ${BUDGET_COINS - spent} left`);

Expected output:

step                     coins  secs
model: pick a tool           6  1.5
tool: lookup_order 1044      0  0.8
model: write the answer      8  1.5  <- costliest
total: 14 coins of 20
budget stop: next call needs 198 coins, 6 left

• trace(): wraps one step and writes a row with its tokens, coins and seconds.
• The table: finds the row with the most coins and marks it, so you know where to look first.
• budgetAllows(): runs before a model call. The bloated prompt is stopped before it is sent, because the app can count it first.

Retry with backoff, then route by a simple rule

// Retry with backoff, a max-attempts limit, and a small/big model router.
let clock = 0; // simulated seconds

function wait(seconds: number): void {
  clock += seconds; // a real app awaits a real sleep here
  console.log(`  waited ${seconds}s`);
}

function withRetry<T>(fn: () => T, maxAttempts: number, safeToRetry: boolean): T | null {
  let delay = 1;
  for (let attempt = 1; attempt <= maxAttempts; attempt++) {
    try {
      const value = fn();
      console.log(`  try ${attempt}: ok at ${clock}s`);
      return value;
    } catch (e) {
      console.log(`  try ${attempt}: ${e instanceof Error ? e.message : "failed"}`);
      if (!safeToRetry) break; // an action may have happened, so check first
      if (attempt < maxAttempts) { wait(delay); delay *= 2; }
    }
  }
  return null; // give up and tell the customer the truth
}

// Fake flaky shipping service: busy twice, then it works.
let shippingCalls = 0;
function lookupOrder(id: string): string {
  shippingCalls++;
  if (shippingCalls <= 2) throw new Error("shipping service busy");
  return `order ${id}: shipped`;
}
function deadService(): string { throw new Error("no answer"); }

console.log("lookup_order 1044 (safe to retry):");
console.log(`  -> ${withRetry(() => lookupOrder("1044"), 4, true)}`);
clock = 0;
console.log("service is down:");
console.log(`  -> ${withRetry(deadService, 4, true) ?? "gave up, tell the customer"}`);

// Router: one help question, no order number, no late parcel -> small model.
function route(message: string): "small" | "big" {
  const jobs = message.split(/[.?!]/).filter((s) => s.trim() !== "").length;
  const hasOrder = /\b\d{4}\b/.test(message);
  const late = /\blate\b/i.test(message);
  return jobs === 1 && !hasOrder && !late ? "small" : "big";
}
for (const m of ["Can I send back a jacket that does not fit?", "Where is order 1043? Please send it to 6 Elm Street."]) {
  console.log(`${route(m)}: ${m}`);
}

Expected output:

lookup_order 1044 (safe to retry):
  try 1: shipping service busy
  waited 1s
  try 2: shipping service busy
  waited 2s
  try 3: ok at 3s
  -> order 1044: shipped
service is down:
  try 1: no answer
  waited 1s
  try 2: no answer
  waited 2s
  try 3: no answer
  waited 4s
  try 4: no answer
  -> gave up, tell the customer
small: Can I send back a jacket that does not fit?
big: Where is order 1043? Please send it to 6 Elm Street.

• withRetry(): waits 1, 2, then 4 seconds, and stops after 4 tries. Then the helper tells the customer the truth.
• safeToRetry: only lookups are retried on their own. A refund or an email is never retried blindly, because it may already have happened.
• route(): picks the model before any model sees the message. Anything with an order number goes to the big model.

In a Real App

A real app sleeps for real and cancels a slow call. This sketch is simplified. ShopApi is a small interface you would connect to your own services, and names differ between providers and SDK versions.

interface ShopApi {
  lookupOrder(id: string, opts: { signal: AbortSignal }): Promise<unknown>;
  issueRefund(req: { orderId: string; amount: number; requestId: string }, opts: { signal: AbortSignal }): Promise<unknown>;
}
declare const shop: ShopApi;

const sleep = (ms: number) => new Promise<void>((resolve) => setTimeout(resolve, ms));

// Timeout: abort the call if it takes longer than `ms`.
async function withTimeout<T>(ms: number, fn: (signal: AbortSignal) => Promise<T>): Promise<T> {
  const controller = new AbortController();
  const timer = setTimeout(() => controller.abort(), ms);
  try {
    return await fn(controller.signal);
  } finally {
    clearTimeout(timer);
  }
}

async function retry<T>(fn: () => Promise<T>, maxAttempts = 4, baseMs = 1000): Promise<T> {
  for (let attempt = 1; ; attempt++) {
    try {
      return await fn();
    } catch (err) {
      if (attempt >= maxAttempts) throw err;
      await sleep(baseMs * 2 ** (attempt - 1)); // 1s, 2s, 4s
    }
  }
}

// A lookup is safe to retry.
const order = await retry(() => withTimeout(3000, (signal) => shop.lookupOrder("1044", { signal })));

// An action gets one request id, reused on every try, so the service can ignore repeats.
const requestId = crypto.randomUUID();
await retry(() =>
  withTimeout(3000, (signal) => shop.issueRefund({ orderId: "1044", amount: 30, requestId }, { signal })),
);

Retry only when the service supports a request id like this. Many payment services call it an idempotency key. Tracing libraries, such as OpenTelemetry, can record each step as a span with its time and token counts, so you do not need to build the table by hand.

Simple lesson: wrap every step in a trace, check the budget before each call, and retry only what is safe to repeat.

TL;DR

1. Trace = a record of every step in a run, with its cost and time.
2. Costly step = the row with the most coins. Find it first, then ask why.
3. Bloated prompt = pasting everything into every call. A big prompt times many calls is a huge bill.
4. Budget per message = set it just above your dearest good chat.
5. Timeout = the longest the app will wait for a tool.
6. Retry with backoff = try again after a failure, waiting a bit longer each time.
7. Model routing = simple questions to a small model, hard ones to a big model.
8. Measure the change = keep it only with the same success and a lower cost.

Common mistakes

Check Your Understanding

Exercises

What to Remember

• A trace shows every step with its coins and seconds. Read it, do not just read the total.
• Find the costliest row first, then ask why it is so big.
• Do not paste everything into every call. Let tools fetch what is needed.
• Set a budget per message, just above your dearest good chat.
• Give slow tools a timeout, and retry with backoff instead of hammering.
• Route simple questions to a small model, then measure: same success, lower cost.

Next: Framework Tour: The Same Agent, Five Ways, what agent frameworks do for you, and how to pick one.