# Feedback That Improves the Product

Course: [AI UX & Generative UI](https://www.devtutorials.ai/learn/ai-ux) · Section: Generative UI
Source: https://www.devtutorials.ai/learn/ai-ux/generative-ui/feedback-loop

> Capture thumbs, corrections and regenerate signals as structured events with a trace ID and output version, so they become test cases in your evals.

## What Product Feedback Is

**Feedback signal** = anything the user does that tells you how good an AI answer was.

In this lesson, you will capture thumbs, corrections and regenerate clicks as structured events, each tied to the exact answer it is about, so the evals team can turn them into test cases.

Users tell you about bad answers all the time. They press thumbs-down. They fix the draft before sending it. They press Regenerate. Most apps throw all of this away.

Press **Play**, or use **Next** to go one step at a time.

- **Frame 1:** a thumbs-down with a reason. The naive interface records nothing.
- **Frame 2:** the user fixes the draft. That fix is the right answer, and it is lost.
- **Frame 3:** a Regenerate click. A quiet "not good enough", also lost.

**Simple lesson:** every bad answer the user notices is a free lesson, if you keep it.

### A Real-World Example

Think about a restaurant's comment cards.

- "The soup was cold" on a card with no date and no table number helps nobody.
- "Table 12, Tuesday 7 pm, the soup was cold" lets the kitchen find out what went wrong that night.
- A customer who sends a dish back is giving feedback too, even without a card.

AI feedback works the same way:

- **Table 12, Tuesday 7 pm** = the trace ID and output version.
- **The card** = an explicit signal, like a thumbs-down.
- **Sending the dish back** = an implicit signal, like Regenerate.

Click the button. The Build tab plays the correction through the naive interface.

- **The screen:** the draft is edited, then sent.
- **Feedback captured:** nothing.
- **The checks:** the signal, its trace ID and the correction text.

**Simple lesson:** feedback you don't record cannot improve anything.

## What You Will Learn

In this lesson, you will learn:

1. Explicit and implicit feedback.
2. Why corrections are the most useful signal.
3. What every feedback event must carry.
4. How feedback becomes eval cases.

**About the evals in this lab**

- The **trace ID** is the ID of the agent run that made an answer. The Building AI Agents and Evals & Observability courses record runs this way.
- The lab's feedback events use the same fields as the Evals & Observability course's dataset.
- That course shows how to turn these rows into test cases and dashboards.

## Explicit and Implicit Signals

| Kind | Examples | Strength |
|---|---|---|
| Explicit | Thumbs up or down, a comment, an edit to the answer | Clear meaning, but few people give it |
| Implicit | Regenerate, copy, close without reading | Lots of it, but you have to infer the meaning |

### Corrections Are Gold

**Correction** = the user's own fix to an AI answer.

When the user edits a draft before sending it, the edited text is the answer they wanted. Save it word for word. It is a ready-made expected output for an eval.

### Ask One Short Why

A thumbs-down alone says "bad". One optional line ("wrong date", "too long") says what to fix. Keep it to one tap or one short field.

**Simple lesson:** record explicit signals, infer from implicit ones, and keep every correction.

## What Every Event Carries

A feedback event is only useful if you can find the answer it is about.

Step through both frames.

- **Frame 1:** the user's correction. Without a trace ID, nobody can match it to the prompt that made the mistake.
- **Frame 2:** the same thumbs-down. With the trace ID and output version, it points at one exact answer.

```ts
interface FeedbackEvent {
  trace_id: string;        // the agent run that made the answer
  output_version: number;  // which version, if the user pressed Retry
  signal: "thumbs-up" | "thumbs-down" | "correction" | "regenerate";
  comment?: string;
  correction?: string;     // the user's fixed text, word for word
}

navigator.sendBeacon("/api/feedback", JSON.stringify(event));
```

`sendBeacon` still delivers the event if the user closes the page right after.

**Feedback can hold private data.**

- A correction may contain a customer's name or address.
- Follow your company's rules for storing it.
- Tell users that feedback is used to improve the product.

**Simple lesson:** trace ID, output version, signal, and the correction if there is one.

## From a Thumbs-Down to a Better Release

Step through all five frames.

- **Frames 1 to 3:** the user reacts, the app records an event, the event becomes a dataset row.
- **Frame 4:** the evals team turns rows into test cases, such as "the date must match tracking".
- **Frame 5:** the next prompt or model change is tested against them.

**Easy way to remember:** **Who, which, what.** Which run (trace ID), which version, what the user said or fixed.

**Simple lesson:** the interface collects the evidence; the evals close the loop.

### Lab Exercise: Capture Feedback for the Evals

Your task: change the Lesson 11 settings in the Build tab so all three scenarios pass. Then press **Check all scenarios**.

Click the button and open the **Build** tab.

- **The three scenarios:** a thumbs-down, a correction and a regenerate.
- **Event for the Evals dataset:** shows the exact row your interface records.
- **To pass:** every signal is captured, with a trace ID and output version, and corrections keep the fixed text.

**Simple lesson:** capture every signal, tied to its answer.

### What Changes per Framework

| Option | Feedback |
|---|---|
| Hand-built React | A `FeedbackEvent` and `sendBeacon`, as above |
| Observability tools | Many tracing tools accept feedback scores tied to a trace ID |
| CopilotKit / Vercel AI SDK UI | Add your own thumbs and edit capture to each message |
| Syncfusion AI components | AI AssistView response toolbars can hold like and dislike buttons (optional, not graded) |

## Build It in React

In the **Code** tab, the `answer-done` event carries the run's `trace` and `version`. Keep them. Every feedback event you send must carry both:

```tsx
agent.feedback({ trace_id: run.trace, output_version: run.version, signal: "correction", correction: draft });
```

The lab acts as the user. It presses thumbs-down and types a reason, edits the answer and presses Save, or presses Regenerate. The checks read each event your component sends.

## TL;DR

1. **Feedback signal** = anything that tells you how good an answer was.
2. **Explicit** = thumbs, comments, edits. **Implicit** = regenerate, copy, close.
3. **Correction** = the user's fix. Save it word for word.
4. **Every event** = trace ID, output version, signal, and correction or comment.
5. **The loop** = event → dataset row → eval case → a better release.

## Common mistakes

| Mistake | Wrong idea | Correct idea | What goes wrong |
|---|---|---|---|
| Thumbs with no trace ID | "We know it was bad." | Attach the trace ID and version. | Nobody can find what went wrong. |
| Ignoring edits | "They fixed it, so it's fine." | Save the correction as the expected answer. | You lose the best label you will get. |
| Ignoring Regenerate | "No feedback was given." | Record it as a signal. | You miss most of the bad answers. |
| A long feedback form | "More detail is better." | One tap, one optional line. | Nobody fills it in. |
| Feedback nobody reads | "It's in the database." | Feed it into evals. | The same mistake ships again. |

## Check Your Understanding

## Exercises

## What to Remember

- Users give feedback all the time. Keep it.
- Corrections are the most useful signal. Save them word for word.
- Record implicit signals like Regenerate too.
- Every event carries the trace ID, the output version and the signal.
- Feedback improves the product only when it reaches the evals.

## Further Reading

- [People + AI Guidebook: Feedback and Control, by Google PAIR](https://pair.withgoogle.com/chapter/feedback-controls/): how to ask for feedback people will give.
- [Guidelines for Human-AI Interaction, by Microsoft Research](https://www.microsoft.com/en-us/research/project/guidelines-for-human-ai-interaction/): see "Encourage granular feedback".

Next: **Accessible AI UI**, where a streamed answer has to work for a screen-reader user, a keyboard-only user and someone who needs less motion.