# Tracing with OTel GenAI Conventions

Course: [AI Evals & Observability](https://www.devtutorials.ai/learn/evals-observability) · Section: Observability and Production
Source: https://www.devtutorials.ai/learn/evals-observability/production/tracing

> A customer says Shop Helper refunded them without asking. The logs say "200 OK". Learn to record every model call, tool call and retrieval as spans with the standard gen_ai.* attributes, link chats with session ids, and keep personal data out.

## Tracing with OTel GenAI Conventions

Shop Helper is live. A customer writes in, upset.

"Your helper refunded me $88 without asking anyone. I only wanted to know my options."

The team opens the logs. Watch what they find. Then watch what a trace shows.

The log said "200 OK". That is all. It could not say which step decided. It could not say which tool ran.

The trace shows **5 spans**: the whole chat, two model calls and two tool calls. One of them is the refund.

**Simple lesson:** a log says that something happened. A trace shows how.

### A Real-World Example

Think about a parcel you ordered online.

"Delivered" tells you the end. The tracking page tells you the story: packed, left the warehouse, reached the depot, out for delivery.

If the parcel is late, the tracking page shows which step was slow.

A **trace** is the tracking page for one request to your AI feature.

**Simple lesson:** record each step, and you can find the step that went wrong.

## What You Will Learn

In this lesson, you will learn:

1. What a **trace** and a **span** are.
2. How to record model calls, tool calls and retrieval as spans.
3. The standard **gen_ai.*** attribute names, and why they matter.
4. How **session.id** links the turns of one chat.
5. How to keep personal data out of your traces.
6. How the same idea looks in TypeScript and in .NET.

## TL;DR

**Trace** = the record of one request, from start to end.

**Span** = one step inside a trace, with a name, a start time, a length and some attributes. Spans nest.

**OpenTelemetry** (OTel) = an open standard for traces. Its **GenAI conventions** name the attributes for AI calls, such as `gen_ai.request.model` and `gen_ai.usage.input_tokens`.

Use the standard names, and any backend can read your traces. Changing backend is then a settings change. It is not a rewrite.

## Cheat Sheet

Every model-call span should set these **6 attributes**:

| Attribute | What it holds | Example |
|---|---|---|
| `gen_ai.operation.name` | What kind of call | `chat` |
| `gen_ai.provider.name` | Who served it | your provider |
| `gen_ai.request.model` | The model you asked for | `model-s` |
| `gen_ai.usage.input_tokens` | Tokens the model read | `512` |
| `gen_ai.usage.output_tokens` | Tokens it wrote | `24` |
| `gen_ai.response.finish_reasons` | Why it stopped | `["stop"]` |

Other spans: tools use `gen_ai.operation.name = execute_tool` and `gen_ai.tool.name`. Searches use a retrieval span. The root span gets `session.id` and a hashed `user.id`.

**About these names**

The GenAI conventions are still being finished in 2026. A few names have changed before, for example `gen_ai.system` became `gen_ai.provider.name`. The course keeps the list in one file and checks it before each release. Check the OpenTelemetry docs before you rely on a name.

## Spans: One per Step

A Shop Helper chat has a few kinds of step:

- **The root span:** the whole chat turn.
- **Model calls:** each time the app sends text to the model.
- **Tool calls:** each time the app runs a tool, such as `lookup_order`.
- **Retrieval:** each time the app searches the help pages.

Each step becomes a span inside the root span. Here is a tiny tracer that prints the tree.

```ts runnable title="A tiny tracer and its span tree"
type Attr = string | number | string[];
type Span = { name: string; depth: number; attrs: Record<string, Attr> };
const spans: Span[] = [];
let depth = 0;

// A tiny tracer: each span records a name, its depth and its attributes.
function startSpan<T>(name: string, fn: (set: (k: string, v: Attr) => void) => T): T {
  const span: Span = { name, depth, attrs: {} };
  spans.push(span);
  depth++;
  const result = fn((k, v) => { span.attrs[k] = v; });
  depth--;
  return result;
}

startSpan("invoke_agent shop-helper", (set) => {
  set("session.id", "sess-418");
  startSpan("execute_tool lookup_order", (set) => {
    set("gen_ai.operation.name", "execute_tool");
    set("gen_ai.tool.name", "lookup_order");
  });
  startSpan("chat model-s", (set) => {
    set("gen_ai.operation.name", "chat");
    set("gen_ai.request.model", "model-s");
    set("gen_ai.usage.input_tokens", 512);
    set("gen_ai.usage.output_tokens", 24);
  });
});

for (const s of spans) {
  const attrs = Object.entries(s.attrs).map(([k, v]) => `${k}=${v}`).join(" ");
  console.log(`${"  ".repeat(s.depth)}${s.name}  ${attrs}`);
}
```

Expected output:

```text
invoke_agent shop-helper  session.id=sess-418
  execute_tool lookup_order  gen_ai.operation.name=execute_tool gen_ai.tool.name=lookup_order
  chat model-s  gen_ai.operation.name=chat gen_ai.request.model=model-s gen_ai.usage.input_tokens=512 gen_ai.usage.output_tokens=24
```

The two inner spans are indented. They sit inside the root span.

**Try this:** the chat span is missing two of the six required attributes. Add them, then run it.

**Simple lesson:** one span per step, nested inside the span for the whole request.

## The gen_ai.* Attributes

A span name like "chat model-s" is for people. Attributes are for tools.

**Attribute** = a key and a value stored on a span, such as `gen_ai.request.model = model-s`.

If every team picks its own names, every dashboard must be rebuilt. With the standard names, many backends can read the same spans. Langfuse, Arize Phoenix, Braintrust, LangSmith and Datadog all can.

The Help Centre feature has a retrieval step instead of tool calls. Its span shows which page was found.

**Simple lesson:** use the standard names, so your traces work in any backend.

## Sessions and Users

A chat has many turns. Each turn is its own trace.

**session.id** = an id shared by every turn of one conversation. Set it on the root span.

**user.id** = an id for the person, so you can see all their sessions. Use a hashed id, never their email.

## Keep Personal Data Out

Traces are read by many people, often in a third-party tool, and kept for weeks.

**PII** (personal data) = anything that can identify a person, such as an email, a phone number or a card number.

Two habits fix it. Store a hashed user id, not the email. Mask emails in any text before it is recorded.

```ts runnable title="Mask emails and hash the user id"
// Mask emails in any text before it goes into a span.
function redact(text: string): string {
  return text.replace(/[\w.+-]+@[\w-]+\.[\w.]+/g, "[email]");
}

// A stable id for a person that does not say who they are.
function userId(email: string): string {
  let h = 7;
  for (const ch of email) h = (h * 31 + ch.charCodeAt(0)) % 1000000;
  return `u-${h}`;
}

const prompt = "Customer asha.k@example.com asks: where is order 2047?";
console.log(redact(prompt));
console.log(userId("asha.k@example.com"));
console.log(userId("asha.k@example.com") === userId("asha.k@example.com"));
```

Expected output:

```text
Customer [email] asks: where is order 2047?
u-204780
true
```

The same email always gives the same id. So you can still group a person's chats.

**A real hash needs a secret.**

This tiny hash is for learning. In production, use a keyed hash (an HMAC with a secret key), so nobody can guess the email from the id.

## The Same Span in .NET

The GenAI names do not depend on the language. In .NET, a span is an `Activity`, and attributes are tags.

```csharp readonly title="Telemetry.cs: a model-call span in .NET"
using System.Diagnostics;

public static class Telemetry
{
    public static readonly ActivitySource Source = new("ShopHelper");

    public static Activity? StartChat(string model, string sessionId)
    {
        var activity = Source.StartActivity($"chat {model}", ActivityKind.Client);
        activity?.SetTag("gen_ai.operation.name", "chat");
        activity?.SetTag("gen_ai.provider.name", "your-provider");
        activity?.SetTag("gen_ai.request.model", model);
        activity?.SetTag("session.id", sessionId);
        return activity;
    }

    public static void EndChat(Activity? activity, int inputTokens, int outputTokens, string finishReason)
    {
        activity?.SetTag("gen_ai.usage.input_tokens", inputTokens);
        activity?.SetTag("gen_ai.usage.output_tokens", outputTokens);
        activity?.SetTag("gen_ai.response.finish_reasons", new[] { finishReason });
        activity?.Dispose();
    }
}
```

The names are the same as in TypeScript. That is the point of a standard.

## Where It Is Used

- **Debugging a complaint:** find the trace and see each step.
- **Cost:** add up token counts per feature, per customer or per day.
- **Speed:** find the slowest step.
- **Online evals:** attach scores to traces (a later lesson).
- **Turning failures into test cases:** a bad trace becomes a new eval case.

**Do's and Don'ts**

- **Do** make a span for every model call, tool call and retrieval.
- **Do** set the six required gen_ai.* attributes on model calls.
- **Do** set session.id and a hashed user.id on the root span.
- **Do** mask personal data before it is recorded.
- **Don't** invent your own names when a standard one exists.
- **Don't** put emails, phone numbers or card numbers in spans.
- **Don't** rely on logs alone for AI features.

## Common mistakes

| Mistake | Wrong idea | Correct idea | What goes wrong |
|---|---|---|---|
| Logs only | "The log says 200 OK, so it worked." | Trace every step as a span. | You cannot tell which step decided or failed. |
| Custom attribute names | "We will call it model_name." | Use the gen_ai.* names. | Every backend and dashboard has to be rebuilt. |
| No token counts | "Tokens do not matter for debugging." | Set input and output token usage. | You cannot see what each chat costs. |
| No session.id | "Each message stands alone." | Set session.id on every root span. | Follow-up turns make no sense on their own. |
| Emails in spans | "Traces are internal." | Hash user ids and mask text. | Personal data leaks to tools and people who should not see it. |

## Check Your Understanding

## Exercises

## What to Remember

- A **trace** records one request. A **span** records one step. Spans nest.
- Make a span for every model call, tool call and retrieval.
- Use the standard **gen_ai.*** names. Any backend can read them.
- Model-call spans need **6 attributes**: operation, provider, model, input tokens, output tokens and finish reasons.
- Set **session.id** and a hashed **user.id** on the root span.
- Keep emails, phone numbers and card numbers out of traces.

Next: **Cost, Latency and SLOs**, how to turn traces into numbers you can watch and promise.