Reading an AI Diff

By Syncfusion®, Inc. · Last updated

Read an AI change line by line. Match every claim in the agent's summary to the diff, spot weakened, deleted and skipped tests, catch scope creep, and turn each unbacked claim into a test.

What an AI Diff Is

When an agent changes code, it usually shows you two things:

  • a summary of what it says it did
  • a diff of what it really changed

The summary is a claim. The diff is the evidence.

In this lesson, you will read a diff line by line. You will check every claim in the summary against it.

Simple lesson: read the diff, not just the summary.

A Real-World Example

Think about a tailor who returns your shirt.

  • The tailor says: "I shortened the sleeves and fixed the button."
  • You check the sleeves. They are shorter. Good.
  • You check the button. It is still loose.
  • You also see the collar was changed. You never asked for that.

You only found this because you looked at the shirt, not just the words.

An AI diff works the same way:

  1. The summary is what the tailor says.
  2. The diff is the shirt itself.
  3. A claim with no line in the diff is the loose button.
  4. A change nobody asked for is the new collar.

What You Will Learn

In this lesson, you will learn:

  1. How to read a diff: added lines, removed lines and files.
  2. How to match each summary claim to a line in the diff.
  3. How to spot tests that were weakened, deleted or skipped.
  4. How to spot scope creep: changes nobody asked for.
  5. How to write a test for each claim, so it becomes evidence.

How to Read a Diff

Lines and Files

Diff = a list of the lines that changed between the old code and the new code.

  • A line starting with + (green) was added.
  • A line starting with − (red) was removed.
  • Lines with no mark did not change. They are there to give context.
  • A diff can touch many files. Each file has its own block.

The Change You Will Review

The task given to the agent was: "Make the cart total count the quantity of each item."

Here is what came back. Read the summary first, then every file.

Look closely:

  • src/cart.ts: item.price became item.price * item.qty. Also, the rounding line changed.
  • src/money.ts: this file changed too. The task never mentioned it.
  • tests/cart.test.ts: one check got weaker, and the paise test is gone.

Now run the agent's own tests.

Press ▶ Run tests.

  • The Tests tab: 2 passed. Green.
  • But: the diff showed problems the tests do not look at.

Simple lesson: a diff shows what really changed. Green tests only show what the tests looked at.

Match Every Claim to the Diff

The Claim Check

For each line of the summary, ask: which line in the diff makes this true?

Animation: three summary claims on the left connect to lines in the diff on the right. The third claim, rejects negative quantity, finds no line and turns red. Summary (claims) Diff (evidence)

Counts the quantity

Handles an empty cart

Rejects a negative quantity

+ total += price * qty

let total = 0; (loop)

(no line checks qty < 0)

? No line backs this claim

Two claims find a line in the diff. One finds nothing. That claim is not true yet.

Turn a Claim Into a Test

A claim with no line in the diff needs a test. The test will tell you if it is true.

Replace the TODO line with a real check:

expect(() => cartTotal([{ name: 'Pen', price: 10, qty: -1 }])).toThrow();

Press ▶ Run tests.

  • The Tests tab: red. "expected the function to throw, but it did not".
  • Why: the summary said "rejects a negative quantity". No code does that.
  • What it means: the claim was false. Now you have evidence, not a guess.

toThrow needs a function.

  • Write expect(() => cartTotal(items)).toThrow().
  • Not expect(cartTotal(items)).toThrow().
  • The arrow () => lets the test catch the error.

Simple lesson: every claim should point to a line in the diff, or to a test that proves it.

Watch the Test Files

Agents sometimes change the tests, not the code, to get green. Read every change in a test file with extra care.

A Weakened Check

Weakened check = a check changed so that more answers pass.

This lab uses the old cart code, which ignores the quantity.

Press ▶ Run tests.

  • The Tests tab: green. toBeGreaterThan(0) passes for 10, 30 or 9,999.
  • Now try: change it to expect(total).toBe(30); and run again. Red. The old code gives 10.

A Deleted Test

Deleted test = a test that was in the old file and is missing in the new one.

The diff removed the paise test. Paste it back, below the comment:

it('keeps paise: 2 × ₹10.25 is ₹20.50', () => {
  expect(cartTotal([{ name: 'Pen', price: 10.25, qty: 2 }])).toBe(20.5);
});

Press ▶ Run tests.

  • The Tests tab: red. "expected 20 to be 20.5".
  • Why: the agent changed the rounding to Math.floor(total). That drops the paise.
  • What it means: the deleted test was the one that would have caught it.

Three test changes that need a reason:

  • A check made weaker, like toBe(330) → toBeGreaterThan(0).
  • A test deleted.
  • A test skipped with .skip, .only or .todo.

If the summary does not explain the change, treat it as a red flag.

Simple lesson: when a diff touches a test, ask: did the test get stronger or weaker?

Scope Creep

Scope creep = changes the agent made that nobody asked for.

  • The task was about the cart.
  • The diff also changed formatRupees in src/money.ts.
  • The summary did not mention it.

Run the project's old test for that function against the new code.

Press ▶ Run tests.

  • The Tests tab: red. "expected "Rs 1500" to be "₹1500.00"".
  • Why: the agent rewrote a function that was not part of the task.
  • The risk: every bill and receipt on the site now looks different.

Quick scope check:

  • List the files the task should touch.
  • Compare with the files in the diff.
  • Any extra file needs a reason in the summary.

Simple lesson: a small task should give a small diff. Extra files are a warning sign.

A Review Routine

Use these five steps on every AI diff:

  1. Read the task. What should change?
  2. Read the summary. List every claim.
  3. Read the diff, file by file. Match each claim to a line.
  4. Read the test files twice. Look for weakened, deleted and skipped tests.
  5. Write a test for each claim with no line, and for each change nobody asked for.

Simple lesson: the routine turns "it looks fine" into "here is the evidence".

Use It at Work

  • See the size first: git diff --stat main.
  • Read the tests first: git diff main -- tests/.
  • Find removed checks fast: git diff main -- tests/ | grep "^-.*expect(".
  • Ask the agent about scope:
List every file you changed and why.
Did you change anything the ticket did not ask for? If yes, undo it or explain it.
  • Rule of thumb: a change you did not ask for needs its own test, or it goes.

TL;DR

  1. Summary = the agent's claims. Diff = the evidence.
  2. Diff = the lines added (+, green) and removed (−, red), file by file.
  3. Claim check = find the line in the diff that makes each claim true.
  4. Weakened check = more answers pass now, for example toBe(330) → toBeGreaterThan(0).
  5. Deleted or skipped test = a check that no longer runs.
  6. Scope creep = changes nobody asked for.
  7. No line for a claim? Write a test for it.

Common mistakes

MistakeWrong ideaCorrect ideaWhat goes wrong
Reading only the summary"The summary says it handles negatives."Find the line in the diff that does it.A false claim ships.
Skipping test files"Tests are not the real code."Agents may change tests to get green. Read them twice.A weakened test hides the bug.
Ignoring extra files"It changed one more file, no big deal."Every extra file needs a reason.Bills look different on the whole site.
Trusting a new test name"It says 'counts the quantity', so it does."Read the check inside, not the name.toBeGreaterThan(0) passes wrong totals.
Missing removed lines"I only looked at the green lines."Red lines matter too. A removed check is lost evidence.The paise test disappears unseen.

Check Your Understanding

Exercises

What to Remember

  • The summary is a claim. The diff is the evidence.
  • Match every claim to a line in the diff.
  • A claim with no line needs a test.
  • Read test files twice. Watch for weakened, deleted and skipped tests.
  • Extra files nobody asked for are scope creep.

Further Reading

Next: Spec First: Tests Before Code. You will write the tests before the agent writes the code, and make sure they fail first.