LLM APIs

By the Syncfusion DevTutorials team · Last updated October 20, 2026

How code talks to an LLM: the chat messages format, request settings, streaming replies, token usage and cost, errors and retries, with WebLLM and cloud services.

So far, you have used a model through the AI sandbox. Apps do the same thing in code: they send a request to a model and read the response. The code that does this is called an LLM API.

Good news: almost every LLM API today uses the same shape, first made popular by OpenAI's "chat completions". WebLLM, the library behind the sandbox, uses it too. Learn it once, and you can use most models.

The main idea: you send a list of messages plus a few settings, and you get back the reply plus how many tokens were used.

A Real-World Example

Think about ordering food through a delivery app.

You fill in an order form: what you want, any special notes, how spicy, and the address. You press Send. The kitchen sends back the food and a receipt. If the kitchen is busy, the app tells you to wait and tries again.

Calling an LLM works the same way.

• The order form is the request: messages and settings.
• Special notes are the system message.
• "How spicy" is a setting, like temperature.
• The food is the reply text.
• The receipt is the token usage, which decides the cost.
• "Busy, try again" is an error with a retry.

In simple words: an LLM API is an order form for text.

What You Will Learn

In this lesson, you will learn:

1. The parts of a chat completion request.
2. How to read the response and its token usage.
3. How to keep a conversation going.
4. How streaming sends the reply in pieces.
5. How to work out the cost of a request.
6. How to handle errors with retries.
7. How the same code works with WebLLM and with cloud services.

The Request

A chat completion request has two parts: the messages and the settings.

Build a request

const request = {
  model: "Qwen2.5-0.5B-Instruct-q4f16_1-MLC",
  messages: [
    { role: "system", content: "You are a support assistant. Answer in one sentence." },
    { role: "user", content: "How do I reset my password?" },
  ],
  temperature: 0.3,
  top_p: 0.9,
  max_tokens: 100,
};

console.log("Model:", request.model);
console.log("Messages:", request.messages.length);
console.log("Settings:", JSON.stringify({ temperature: request.temperature, top_p: request.top_p, max_tokens: request.max_tokens }));

Expected output:

Model: Qwen2.5-0.5B-Instruct-q4f16_1-MLC
Messages: 2
Settings: {"temperature":0.3,"top_p":0.9,"max_tokens":100}

You know every part from earlier lessons:

• model says which model to run.
• messages is the chat, with system, user, and assistant roles.
• temperature and top_p control sampling.
• max_tokens limits the length of the reply.

Other common settings are seed for repeatable output, stop for extra stop words, and response_format for JSON mode.

The Response

The response has the reply, the reason it stopped, and the token usage.

Read a response

const response = {
  choices: [
    {
      message: { role: "assistant", content: "Click 'Forgot password' on the sign-in page and follow the email link." },
      finish_reason: "stop",
    },
  ],
  usage: { prompt_tokens: 38, completion_tokens: 17, total_tokens: 55 },
};

const choice = response.choices[0];
console.log("Reply:", choice.message.content);
console.log("Stopped because:", choice.finish_reason);
console.log("Tokens used:", response.usage.total_tokens);

Expected output:

Reply: Click 'Forgot password' on the sign-in page and follow the email link.
Stopped because: stop
Tokens used: 55

• choices[0].message.content is the reply text.
• finish_reason is "stop" when the model chose its end token, and "length" when it hit max_tokens.
• usage counts the prompt tokens and the reply tokens.

In simple words: read the text from choices[0], and the cost from usage.

Keeping a Conversation Going

The model has no memory, as you saw in Generation Flow. To keep a chat going, your code adds the assistant's reply to the message list, then the next user message, and sends the whole list again.

Add each turn to the history

const messages = [{ role: "system", content: "Be brief." }];

function addTurn(question, reply) {
  messages.push({ role: "user", content: question });
  messages.push({ role: "assistant", content: reply });
}

addTurn("What is an LLM?", "A model that predicts text.");
addTurn("Name one.", "Qwen2.5.");

messages.push({ role: "user", content: "How big is it?" });
console.log("Messages sent on turn 3:", messages.length);
console.log(messages.map((m) => m.role).join(" -> "));

Expected output:

Messages sent on turn 3: 6
system -> user -> assistant -> user -> assistant -> user

This is also how the Clear chat button in the sandbox works: it simply empties this list.

Streaming

With streaming, the reply arrives in small pieces called chunks, one or a few tokens at a time. Your code joins them as they arrive and shows the text so far.

Join streamed chunks

const chunks = [
  { choices: [{ delta: { content: "Click" } }] },
  { choices: [{ delta: { content: " 'Forgot" } }] },
  { choices: [{ delta: { content: " password'" } }] },
  { choices: [{ delta: { content: " on the sign-in page." } }] },
  { choices: [{ delta: {}, finish_reason: "stop" }] },
];

let text = "";
for (const chunk of chunks) {
  const piece = chunk.choices[0].delta.content;
  if (piece) {
    text += piece;
    console.log("So far:", text);
  }
}

Expected output:

So far: Click
So far: Click 'Forgot
So far: Click 'Forgot password'
So far: Click 'Forgot password' on the sign-in page.

In a stream, each chunk has a delta, meaning "the new part". The last chunk has the finish_reason.

Cost From Usage

Paid services charge per token, usually with different prices for input and output. The usage numbers let you work out the cost of every request. The prices below are made up for the example.

Cost of one request and of a day

const usage = { prompt_tokens: 1200, completion_tokens: 300 };
const pricePerMillion = { input: 0.5, output: 1.5 };  // example prices in dollars

const cost =
  (usage.prompt_tokens / 1000000) * pricePerMillion.input +
  (usage.completion_tokens / 1000000) * pricePerMillion.output;

console.log("One request: $" + cost.toFixed(5));
console.log("5,000 requests a day: $" + (cost * 5000).toFixed(2));

Expected output:

One request: $0.00105
5,000 requests a day: $5.25

Models running in the browser, like the one in the sandbox, cost nothing per token. They use the reader's own computer instead.

Errors and Retries

Requests can fail. A cloud service may be busy, or rate-limit you for sending too many requests too fast. A browser model may fail to load. Good code retries a few times, waiting a little longer each time. This is called backoff.

Retry with growing waits

const outcomes = ["busy", "busy", "ok"];  // what the service answers on each try

function callWithRetry(maxTries) {
  let wait = 500;
  for (let attempt = 1; attempt <= maxTries; attempt++) {
    const result = outcomes[attempt - 1];
    if (result === "ok") return "Success on try " + attempt;
    console.log("Try " + attempt + ": " + result + ", waiting " + wait + " ms");
    wait *= 2;
  }
  return "Gave up after " + maxTries + " tries";
}

console.log(callWithRetry(4));

Expected output:

Try 1: busy, waiting 500 ms
Try 2: busy, waiting 1000 ms
Success on try 3

Retry only errors that may go away, like "busy" or a network drop. Do not retry a request that is wrong, like a missing message. It will fail every time.

The Real Calls

WebLLM, in the Browser

This is the code the AI sandbox runs, in short. The model downloads once and runs on the reader's graphics chip.

import { CreateMLCEngine } from "@mlc-ai/web-llm";

const engine = await CreateMLCEngine("Qwen2.5-0.5B-Instruct-q4f16_1-MLC", {
  initProgressCallback: (p) => console.log(p.text),
});

const reply = await engine.chat.completions.create({
  messages: [{ role: "user", content: "Say hello in French." }],
  temperature: 0,
});
console.log(reply.choices[0].message.content);

Streaming With WebLLM

const stream = await engine.chat.completions.create({
  messages: [{ role: "user", content: "Write a haiku about tea." }],
  stream: true,
});

let text = "";
for await (const chunk of stream) {
  text += chunk.choices[0]?.delta?.content ?? "";
}
console.log(text);

A Cloud Service

A cloud service uses the same request shape, sent over HTTP. The address and the model name depend on the provider.

const response = await fetch("https://api.example.com/v1/chat/completions", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    Authorization: "Bearer " + process.env.LLM_API_KEY,
  },
  body: JSON.stringify({
    model: "provider-model-name",
    messages: [{ role: "user", content: "Say hello in French." }],
    temperature: 0,
  }),
});
const data = await response.json();
console.log(data.choices[0].message.content);

Many providers follow this shape closely, but not all of them. Some use their own field names. Always check the provider's documentation.

Try It in the AI Sandbox

Press the button to send a message from the Chat tab. After the reply, look at the line under it: the token count and the speed come from the response's usage. Press Stop during a long reply to see an interrupted stream.

Common mistakes

Exercises

Each exercise ends with a value on the last line. Press Check to compare it with the expected answer.

What to Remember

• Most LLM APIs share the chat completions shape.
• A request has messages and settings: temperature, top_p, max_tokens, seed, response_format.
• The response has choices[0].message.content, a finish_reason, and usage.
• Send the whole history every turn. The model does not remember.
• Streaming sends deltas. Join them as they arrive.
• Work out cost from usage. Retry busy errors with backoff.
• Keep API keys on a server, never in the browser.

You have finished the LLM course. You now know how a language model reads text, predicts the next token, picks it, and how to use it safely from code. The Prompt Engineering course builds on all of this.