A suite has 100% coverage and a 0% mutation score. What does that mean?

Every line ran, but no small change made a test fail

What does it mean when a mutant is "killed"?

At least one test failed on the mutant

Which change usually raises the mutation score most?

Use exact answers on both sides of each limit

A test mocks the tax function to return 18, then checks the total is 118. What is the problem?

The real tax code never runs, so its bug is hidden

When is a saved expected value dangerous?

When it was copied from code that was already wrong

Auditing AI-Written Tests

By Syncfusion®, Inc. · Last updated November 13, 2026

Measure AI-written tests for real. See why 100% coverage can hide weak tests, run mutation testing and watch mutants survive, then raise the score with exact checks, remove over-mocks and spot saved answers with the bug inside.

What Auditing Tests Means

Audit = a careful check of something that others will trust.

AI agents write a lot of tests. Many look complete. Some even run every line of the code. But do they catch bugs?

In this lesson, you will measure that for real:

with coverage, which tells you which lines ran
with mutation testing, which tells you which bugs your tests would catch

You will also spot three test smells that AI tests often have:

weak checks
over-mocking
saved answers with the bug inside

Simple lesson: a test suite is evidence only if it fails on wrong code. Mutation testing measures that.

A Real-World Example

Think about a security guard at a building gate.

The guard's report says: "I checked every person who came in." That is like coverage.
But did the guard look at their ID cards, or just wave at them?
The only way to know: send in a person with a fake ID. Does the guard stop them?

Mutation testing sends fake IDs to your tests. Each fake ID is a small bug.

Need a refresher? Unit, integration and end-to-end tests

Unit test = checks one small piece of code, like one function. Fast.
Integration test = checks that two or more pieces work together, like your code and a real API.
End-to-end test = checks a whole user journey in the running app, like "add to cart, then pay".

This lesson is about unit tests. Integration and Contract Tests and User-Flow Checks cover the other two.

Need a refresher? Mocks, stubs and fakes

Mock = a fake part that stands in for a real one, and records how it was called. In Vitest: vi.fn().
Stub = a fake part that only returns a fixed answer.
Fake = a simple working copy, like an in-memory list instead of a real database.

They are useful at the edges, like the network or the clock. They are risky when they replace the code you are testing.

What You Will Learn

In this lesson, you will learn:

Why 95% coverage can still mean weak tests.
What a mutant is, and what the mutation score means.
How to raise the mutation score with exact checks at the edges.
How over-mocking hides real bugs.
Why expected values copied from a run are dangerous.

Coverage Is Not Quality

What Coverage Measures

Coverage = the share of lines (or branches) that ran during the tests.

100% coverage means every line ran at least once.
It does not mean every line was checked.
A test with toBeDefined() runs the code, then checks almost nothing.

What a Mutant Is

Mutant = a copy of your code with one small change, made on purpose.

Examples of mutations:

>= becomes >
499 becomes 500
40 becomes 41
Math.round becomes Math.floor

If a test fails on a mutant, the mutant is killed. If every test still passes, the mutant survived.

Mutation score = killed mutants ÷ all mutants. Higher is better.

Animation: four mutants of the shipping code go into the weak test suite and all survive. The same four go into the strong suite and all are killed.

Weak suite: coverage 100% Strong suite: coverage 100%

>= → > 499 → 500 40 → 41 > 5 → >= 5 >= → > 499 → 500 40 → 41 > 5 → >= 5

survived survived survived survived killed killed killed killed Mutation score 0% Mutation score 100%

Same code, same coverage. Only the strong suite notices the changes.

Watch Mutants Survive

The agent wrote three tests for shippingFee. Every line of the code runs.

Press ▶ Run tests. Green. Now press 🧬 Run mutation.

The Mutants tab: most mutants survived. The score is low.
Why: toBeGreaterThan(0) and toBeDefined() accept almost any answer.
Click a line number: the editor jumps to the line that was changed.

Exact Answers at the Edges

Here are four better tests for the same code.

Press 🧬 Run mutation.

The Mutants tab: every mutant is killed.
Why: each test has an exact answer (toBe(40)), and they sit on the edges: ₹498 and ₹499, 5 kg and 6 kg.

Two moves raise the mutation score:

Replace loose checks (toBeGreaterThan, toBeDefined, toBeTruthy) with exact ones (toBe, toEqual).
Test both sides of every limit: ₹498 and ₹499, 5 kg and 6 kg.

This lab's mutation engine is a small teaching copy. Real projects use Stryker. Stryker makes many more kinds of mutants and can test only the lines that changed. The idea is the same.

Simple lesson: coverage says the code ran. Mutation score says the tests would notice if it were wrong.

Over-Mocking

Over-mocking = faking so much that the real code under test never runs.

Agents often mock more than people do. A mock that returns the expected answer turns any test green.

Press ▶ Run tests. Green.

The mock: vi.fn().mockReturnValue(18) gives 18, so the total is 118.
The real code: gstOn uses 12%, not 18%. It never ran.
Now try: add a test with no mock: expect(orderTotal(100)).toBe(118);. Run again. Red: 112.

Mock at the edges only.

Fine to mock: the network, the clock, random numbers, emails.
Not fine: the function you are testing, or plain maths it uses.
Ask: "Does the real code still run in this test?"

Simple lesson: if the mock gives the answer, the test only checks the mock.

Saved Answers With the Bug Inside

Sometimes the expected value is copied from a run of the code. A snapshot test saves output like this to a file, so later runs can compare against it.

That is fine when you have checked the saved output.
It is dangerous when the code was already wrong. The bug gets saved as the "right" answer.
People call this blessing a snapshot.

In this lab, the saved answer is written into the test as an object.

Press ▶ Run tests. Green.

Look closely: total: 100. With 18% GST on ₹100, the total should be ₹118.
Now try: change it to total: 118 and run again. Red. The code forgot to add the GST.

Simple lesson: check every saved answer by hand before you trust it.

Your Turn: Raise the Score

Press 🧬 Run mutation. Read which mutants survived.

For each survivor, add a test where that change gives a different answer.
Run mutation again. Aim for 80% or more.

Simple lesson: every surviving mutant points to a test you have not written yet.

TL;DR

Coverage = which lines ran. It does not say if they were checked.
Mutant = a copy of the code with one small change.
Killed = a test failed on the mutant. Survived = all tests still passed.
Mutation score = killed ÷ all mutants. Teams often ask for 80% on changed code.
Raise it with exact checks, on both sides of every limit.
Over-mocking = the mock gives the answer, so the real code never runs.
Blessed snapshot = a saved answer copied from buggy code.

Common mistakes

Mistake	Wrong idea	Correct idea	What goes wrong
Trusting coverage	"95% coverage means good tests."	Measure the mutation score too.	Bugs pass through lines that ran but were not checked.
Loose checks	"toBeGreaterThan(0) is safe."	Use the exact answer.	₹41 or ₹9999 also passes.
Mocking the unit	"Mock the tax to keep it simple."	Mock only the edges, like the network.	The 12% bug ships.
Copying output	"The code printed it, so it is the answer."	Work the answer out by hand.	The bug is saved inside the test.
Chasing 100%	"Every mutant must die."	Aim for a set threshold, like 80%.	Time is lost on mutants that change nothing.

Check Your Understanding

Exercises

What to Remember

Coverage tells you what ran. Mutation score tells you what was checked.
Each surviving mutant is a test you still need.
Use exact answers on both sides of every limit.
Mock only at the edges. The code under test must really run.
Check saved answers by hand before you trust them.