Done Is a Claim

By Syncfusion®, Inc. · Last updated November 13, 2026

Why an AI agent's "done" is a claim, not proof. Four ways a test stays green on wrong code, the one question that tells strong tests from weak ones, and the evidence ladder from weakest to strongest proof.

What "Done" Means

An AI coding agent often ends its work with a happy message.

For example:

• "Done! ✅"
• "All tests pass."
• "10% discount works."

That message is a claim. It is not proof.

In this course, you will learn to ask one question about every change:

What evidence says this really works?

Simple lesson: "done" is something the agent says. Evidence is something you can check.

A Real-World Example

Think about a smoke detector on the ceiling.

• Its small light blinks every minute.
• The blink tells you the battery is fine.
• But does it beep when there is smoke? The blink cannot tell you.
• The only real check: put smoke near it, and listen for the alarm.

A test that cannot fail is like that blinking light. It looks fine. It proves nothing.

Now meet an AI change.

• The agent added a discount function.
• It wrote a test.
• It says "Done! All tests pass."

Press ▶ Run tests.

• The Tests tab: the test passes. Green.
• The test file: it compares the answer with itself. It can never fail.
• Now try: change the test to .toBe(450) and run it again. It turns red. The code gives 490, not 450.
• The bug: the code takes ₹10 off, not 10%.

Simple lesson: the agent's test was green, and the code was still wrong.

What You Will Learn

In this lesson, you will learn:

1. Why "done" is a claim, not proof.
2. Four ways a test can be green while the code is wrong.
3. The one question that tells a strong test from a weak one.
4. The evidence ladder: eight kinds of proof, from weakest to strongest.
5. How to judge five real-looking AI changes yourself.

Claims and Evidence

Claim

Claim = something someone says is true.

Examples of claims:

• The agent writes "Fixed the bug."
• The summary says "Handles an empty cart."
• A teammate says "I tested it."

A claim may be true. You just cannot tell yet.

Evidence

Evidence = something you can run or read that shows the claim is true.

Examples of evidence:

• A test that fails on wrong code and passes on right code.
• A test that failed before the fix and passes after it.
• The app working when you click through it.

Oracle

Oracle = the part of a test that knows the right answer.

• In expect(applyDiscount(500, 10)).toBe(450), the oracle is 450.
• You work it out yourself: 10% of 500 is 50, and 500 minus 50 is 450.
• A test with a weak oracle cannot spot a wrong answer.

Simple lesson: a claim says it works. Evidence shows it works. The oracle is what makes evidence count.

Green Is Not Proof

A test can pass while the code is wrong. Here are four common ways. AI agents make all four.

A Test That Cannot Fail

You saw this one already.

• expect(applyDiscount(500, 10)).toBe(applyDiscount(500, 10))
• Both sides run the same code. They are always equal.
• So the test is green for right code and wrong code.

This is called a tautology: a check that is always true.

A Test With No Check

Press ▶ Run tests.

• The Tests tab: "1 passed (0 checks)".
• Why: the test calls isFreeDelivery(499) but never uses expect.
• The bug it hides: the rule says ₹499 or more is free. The code uses >, so ₹499 pays.
• Now try: add expect(isFreeDelivery(499)).toBe(true); inside the test and run it again.

A Test That Copies the Code

Press ▶ Run tests.

• The test: it works out expected with Math.floor, the same formula as the code.
• The problem: if the formula is wrong, the test is wrong in the same way.
• The truth: 2.678 should round to 2.68. Math.floor gives 2.67.
• Now try: replace expected with 2.68 and run it again.

A Skipped Test

Press ▶ Run tests.

• The Tests tab: "1 passed | 1 skipped".
• Why: it.skip tells the runner not to run the test at all.
• The bug it hides: the cart ignores the quantity. 3 pens at ₹10 should cost 30.
• Now try: change it.skip to it and run again.

A Test With Real Numbers

Now the same discount bug, with a strong test.

Press ▶ Run tests.

• The Tests tab: red. "expected 490 to be 450".
• Why it works: 450 was worked out by hand, not by the code.
• Now try: open src/discount.ts. Fix the code to Math.round(price * (100 - percent)) / 100. Run again. Green.

Here is the difference between a weak test and a strong test, side by side.

Simple lesson: green only means the tests found nothing. It does not mean there was nothing to find.

The Key Question

Would This Test Fail If the Code Were Wrong?

Ask this about every test. It is the most useful question in this course.

• If yes, the test is real evidence.
• If no, the test is only decoration.

You can answer it for real. Plant a bug in the code and run the tests. A good test turns red.

Plant Bugs and Watch

This lab hides five buggy versions of applyDiscount. Each one is a mistake an AI could make.

Press ▶ Run tests, then press 🐞 Plant bugs.

• The Bug check tab: the test catches only 1 of 5 bugs.
• Why: with ₹100 and 10%, "₹10 off" and "10% off" give the same answer, 90.
• Now try: add a second test with ₹500. Plant the bugs again. More are caught.
• The Evidence tab: each rung lights up when its evidence exists.

Simple lesson: a test is only as good as the bugs it can catch.

The Evidence Ladder

Some evidence is stronger than other evidence. The evidence ladder puts it in order, from weakest at the bottom to strongest at the top.

Click each rung to see what it proves and what it misses.

• Rungs 1 to 3 are where most AI changes stop.
• Rungs 4 to 8 are what this course teaches you to add.
• You do not always need all eight. A small text change needs less proof than a payment change.

Simple lesson: always know which rung your evidence is on. Climb higher when the change matters more.

Is AI-Written Code Worse?

What Studies Found

Recent research looked at tests written by AI agents.

• One 2026 study looked at about 86,000 test changes made by five coding agents. It found that about four in five had weak checks or no real checks.
• Another study of over a million commits found that agents often use more mocks than people do. A mock is a fake part that stands in for a real one. You learn about mocks in Integration and Contract Tests.
• Giving AI feedback from planted bugs made its tests much stronger in one study.

The Other Side

• A 2026 report found that recent tests written by one AI model were about as strong as tests written by people.
• So the lesson is not "AI tests are bad".
• The lesson is: measure every test the same way, no matter who wrote it.

Simple lesson: do not trust or distrust a test because of who wrote it. Check what it can catch.

Practice: Five Green AI Changes

An agent made five changes to a shop app. Every change came with tests, and every test is green.

Three of them are broken.

1. Press ▶ Run the agent's tests. All five turn green.
2. Open each change. Read the code and the test.
3. Pick Really works or Broken for each one.
4. Press 🔍 Reveal the truth. A hidden test with hand-worked numbers runs on each change.

Things to look for:

• Does the test compare the code with itself?
• Does the test copy the code's formula?
• Does it try the edge? For example, exactly ₹499.
• What evidence came with it? "Failed first" and "checked in the app" are higher rungs.

Simple lesson: green tests looked the same for all five. Only reading the tests showed the difference.

How This Course Works

Every lesson follows the same steps:

1. An AI change looks done.
2. Evidence shows it is not.
3. You build the check that catches it.

The lessons in this course:

• Done Is a Claim (this lesson)
• Reading an AI Diff
• Spec First: Tests Before Code
• Reviewing AI Code
• Auditing AI-Written Tests
• Property-Based Testing
• Integration and Contract Tests
• User-Flow Checks
• Verification Gates for Agents
• Characterization Tests
• Security and Dependency Checks
• Capstone: Your Verification Pack
• Four .NET lessons come in a later release.

At the end, you export a verification pack for your own project. It works with Claude Code, Codex and Cursor.

Simple lesson: each lesson adds one rung of evidence you can use on real AI changes.

Use It at Work

Do this on the next change an agent says is "done":

• Run the tests yourself. npx vitest run. Do not trust a summary.
• Look for skips. npx vitest run --reporter=verbose lists every test, skipped ones too.
• Ask the agent for proof:

Show me the exact test command you ran and its full output.
List the tests you added for this change, and what each one checks.

• Check the claim against the ladder: types clean, tests ran, new tests exist for the change, nothing skipped.

TL;DR

1. Claim = something the agent says, like "done".
2. Evidence = something you can run or read that shows it is true.
3. Oracle = the part of a test that knows the right answer. Work it out by hand.

4. Green is not proof. Four ways a test passes on wrong code:

   • It compares the code with itself.
   • It has no expect.
   • It copies the code's formula.
   • It is skipped.
5. The key question = would this test fail if the code were wrong?
6. Evidence ladder = eight kinds of proof, from "the agent says done" to "a human reviewed it".

Common mistakes

Check Your Understanding

Exercises

What to Remember

• "Done" is a claim. Ask what evidence says it works.
• A green test only means the tests found nothing.
• Work out expected values by hand. Never copy them from the code.
• Watch for tests that cannot fail, tests with no check, copied formulas and skipped tests.
• Ask about every test: would it fail if the code were wrong?
• Know which rung of the evidence ladder you are on.

Further Reading

• Vitest documentation: the test runner this course's lab copies.
• Stryker Mutator: a real tool that plants bugs in your code to test your tests. You use it in Auditing AI-Written Tests.
• fast-check: a tool that tries hundreds of random inputs. You use it in Property-Based Testing.

Next: Reading an AI Diff. You will check what an AI change really did, line by line, and catch claims its summary cannot back up.