The Token Diet

By Syncfusion®, Inc. · Last updated November 2, 2026

Cut the input tokens of every call without losing quality: select tools by intent and articles by score, trim chat history to a budget, and set max_tokens from real replies, in TypeScript or C#.

What you will learn

• Measure where a prompt's tokens go, section by section
• Select tools by intent and help articles by score
• Trim chat history to a token budget without gaps
• Set max_tokens from real replies and detect cut-off replies

Every Call Sends Too Much

Your Lesson 2 trace showed it: the agent steps read thousands of input tokens and write a few dozen. Input is the bill.

Open the prompt the agent builds for "Where is my order A-1042?":

• the system prompt
• all six tool schemas, even the refund and address tools
• the whole chat history
• six help articles from search, even the one scored 0.12
• the question: 40 tokens

Press Play, or use Next to go one step at a time.

• The bar: the prompt's tokens, coloured by section.
• Frame 2: only the tools and articles this question needs.
• Frame 3: only the recent chat turns.

Simple lesson: most of the prompt is not the question, and you pay for all of it on every call.

A Real-World Example

A team trims the prompt by hand: "remove old history, fewer articles". The bill drops. Two weeks later, support sees more wrong delivery dates.

The trimmed prompt dropped the article with delivery times. Nobody ran the eval.

Simple lesson: a token diet is code with tests, and the eval says when you cut too far.

What You Will Build

In this lesson, you will learn:

1. how to measure where a prompt's tokens go
2. how to select tools by intent and articles by score
3. how to trim chat history to a token budget without leaving gaps
4. how to set max_tokens from real replies, and detect a cut-off reply

You will finish three files in the prompt folder.

• Select (src/prompt/select.ts, or Prompt/Select.cs): tools for the intent, best articles.
• History (src/prompt/history.ts, or Prompt/History.cs): newest turns that fit a budget.
• Output limit (src/prompt/outputLimit.ts, or Prompt/OutputLimit.cs): max_tokens and truncation.

Open the Code tab. The harness has the tools, history and articles with their token counts. It also has qualityKept, which checks a prompt against what the eval says this question needs.

Simple lesson: the eval check sits next to the code that trims.

Measure Before You Cut

Run the cell.

• Help articles: about half the prompt.
• Tool schemas: almost a quarter.
• The question: under 1%.

Real token counts come from the model's tokenizer. Most SDKs and local servers can count tokens before you send. Your usage records from Lesson 2 tell you the real total after the call.

Simple lesson: cut the biggest sections first.

Select Tools and Articles

Tools by Intent

Every tool schema in the request is input tokens, on every agent step. Most questions need two or three tools.

Give each tool a list of intents. Send the tools for this intent, plus any tool marked for every intent, such as "escalate to a human".

Keep the tools in their original order. In Prompt Architecture and Caching, a stable order is what lets the cache hit.

Articles by Score

Search returns articles with a score. Keep the best few above a minimum score. A low-scoring article costs tokens and can confuse the model.

Simple lesson: send the tools and articles this question needs, not the ones you have.

Trim the History

Long chats grow every turn. Keep the newest turns that fit a token budget.

Two rules keep the meaning intact:

• No gaps. Stop at the first turn that does not fit. Skipping it and keeping an older one leaves a hole in the conversation.
• Keep the latest turn. It is usually what the user just asked about.

Run the cell. Then change the budgets.

• k=2, history 600: 4,400 tokens, and the eval passes.
• k=1: fewer tokens, and the eval fails: the delivery-times article is gone.
• history 400: one turn too few, and the eval fails.

Simple lesson: trim from the old end, and stop at the first turn that does not fit.

Cap the Output

Output tokens cost more per token, and every one adds writing time. Most APIs take a max_tokens limit.

Set it from data, not from a guess:

• take the completion_tokens of real replies for this step
• take the 95th percentile, so 95% of replies fit
• add headroom, then round up

A reply that hits the limit is cut off. OpenAI-compatible servers report this as finish_reason: "length". Anthropic's API reports stop_reason: "max_tokens". Log it, and retry with a higher limit or a shorter prompt.

Run npm run diet (or dotnet run -- diet). Input tokens go from 10,480 to 4,400, which is 58% less, and the eval still passes.

Simple lesson: set output limits from real replies, and treat a cut-off as an error.

Retrieve Less, Answer Short

Two more places hide tokens.

• Retrieval: a search returns the top articles, and every one goes into the prompt. Each extra article costs tokens on every call, and rarely adds the right answer.
• Output format: a reply in free text runs long. When code reads the reply, ask for JSON with a schema. It is shorter, and code can check it.

Run the cell. Going from top-2 to top-5 adds 1,140 tokens per call and finds the right article for 3 more questions out of 10. Check those 3 with the quality gate before you pay for them.

Simple lesson: choose top-k from labelled questions, and use structured output when code reads the reply.

Try it on a real model. Count real tokens, not estimates. Send the full prompt and the trimmed one to a real model, and compare input tokens and first-token time. Switch Model to Real in the lab's top bar. It runs an open model in your browser with WebLLM, and needs WebGPU. It is not graded.

Put It in Your App

• One prompt builder. Build every prompt in one place, with the sections as data. Then you can measure, trim and test it.
• Log section sizes. Add the token count per section to your cost record. A section that grows is a cost leak.
• Eval in CI. Run the eval suite when the prompt builder changes, and fail the build when quality drops.

Simple lesson: the prompt builder is code, so give it tests and an eval.

Common mistakes

Check Your Understanding

Exercises

Build It in the Project

The other files of this lesson's project are optional practice. Finish them to run the lesson's script on your own code.

What to Remember

• Input tokens are usually most of an agent's bill.
• Measure the prompt by section, and cut the biggest sections first.
• Select tools by intent and articles by score.
• Trim history from the oldest end, with no gaps, and keep the latest turn.
• Set max_tokens from the p95 of real replies, and log cut-off replies.
• Every cut must pass the eval.

Next: Prompt Architecture and Caching. You will reorder the prompt so the part you still send on every call is billed at a fraction of the price.