Characterization Tests

By Syncfusion®, Inc. · Last updated November 13, 2026

Before an AI refactor, pin what old code does today. Build a golden table of edge cases, compare old and new code on hundreds of random orders, keep quirks on purpose, and follow a safe order of steps for every refactor.

Old Code and AI Refactors

Most real projects have old code. It works, people depend on it, and nobody remembers every rule inside it.

Agents are good at refactoring: rewriting code so it is cleaner, without changing what it does. But "without changing what it does" is a claim. Old code often has odd rules that a clean rewrite quietly drops.

Characterization test = a test that records what the code does today, so you can tell if a change breaks it. Some people call these golden tests, because the saved answers are like a gold standard.

Simple lesson: before an AI refactor, pin down what the code does now.

A Real-World Example

Think about a family recipe for sambar that your grandmother made.

• Nobody wrote it down exactly.
• A cousin offers to "clean up" the recipe and write it neatly.
• The new version looks better. But the sambar tastes different.
• A pinch of something was in the old way of cooking, and it is missing now.

If someone had tasted and noted every detail of the old sambar first, you would know exactly what changed.

Characterization tests are that tasting note, for code.

What You Will Learn

In this lesson, you will learn:

1. What a characterization test is, and how it differs from a spec test.
2. How to pin the current behaviour with a table of saved answers.
3. How to compare old and new code on many random inputs.
4. Why quirks deserve a test, even when they look wrong.
5. A safe order of steps for any AI refactor.

Pin What the Code Does

Capture, Do Not Judge

In Spec First: Tests Before Code, the expected value came from the ticket.

Here, the expected value comes from running the current code. That sounds like the mistake from Auditing AI-Written Tests. The difference is the goal:

• Spec test: "Is the answer right?" The answer comes from the ticket.
• Characterization test: "Did the answer change?" The answer comes from today's code.

A Table of Saved Answers

Here is the old pricing code, with eight pinned cases.

Press ▶ Run tests. Green: 9 passed.

• Each row is one order and the answer the old code gives today.
• Odd values are fine: "two pens" costs ₹24.20, not ₹24.19. The old code rounds GST on each line.
• Choose cases at the edges: 9 and 10 items, paise, members, tiny orders, an empty order.

Simple lesson: a characterization test asks "did it change?", not "is it right?".

Catch the Refactor

The Pins Catch a Change

The agent rewrote price(). Its summary says "same rules, clearer code".

Press ▶ Run tests.

• The Tests tab: "pins: 10 items (bulk)" is red.
• Why: the new code gives the bulk discount at qty > 10. The old code used qty >= 10.
• Compare: open src/legacy-price.ts, a saved copy of the old code.

Old vs New on Many Inputs

A table only covers the cases you picked. You can also compare old and new code on hundreds of random orders. People call this differential testing: the old code becomes the oracle.

Press ▶ Run tests.

• The Tests tab: red, with a small counterexample of two lines.
• Why: the new code rounds GST once on the total. The old code rounds it on each line. For some orders, that is one paisa different.
• This mixes two lessons: a property test from Property-Based Testing, with the old code as the answer key.

Simple lesson: pinned cases catch known edges. Old-vs-new on random inputs catches the ones you did not think of.

Quirks Deserve a Test

Quirk = an odd rule in old code that looks like a mistake.

Example: the old code never charges less than ₹1 for a non-empty order.

• It looks strange. An agent may "clean it up".
• But maybe the payment gateway rejects amounts under ₹1. Then the quirk protects every tiny order.

Press ▶ Run tests.

• The Tests tab: red. "expected 0.24 to be 1".
• Why: the refactor dropped the minimum charge.
• The comment in the test explains why the quirk matters. That stops the next person from deleting it.

Simple lesson: pin the quirk, write down why it exists, and change it only on purpose.

A Safe Refactor

When everything matches, the refactor is safe to accept.

Press ▶ Run tests, then 🐞 Plant bugs.

• The Tests tab: green on 500 random orders.
• The Bug check tab: all five refactor mistakes are caught.

The safe order of steps:

1. Pick the code the agent will refactor.
2. Write characterization tests that pass on the current code.
3. Keep a copy of the old code for old-vs-new checks.
4. Let the agent refactor.
5. Run the pins and the old-vs-new check.
6. If anything is red, decide: is it a bug in the refactor, or a change someone wanted?

Simple lesson: pins first, refactor second, and every red result gets a decision.

TL;DR

1. Refactor = rewrite code to be cleaner without changing what it does.
2. Characterization test = records what the code does today. Also called a golden test.
3. Spec test asks "is it right?". Characterization test asks "did it change?".
4. Golden table = cases at the edges, with answers captured from the current code.
5. Differential testing = old and new code on many random inputs. The old code is the oracle.
6. Quirk = an odd rule. Pin it, explain it, and change it only on purpose.

Common mistakes

Check Your Understanding

Exercises

What to Remember

• Before an AI refactor, pin what the code does now.
• Characterization tests ask "did it change?", not "is it right?".
• Use a table of edge cases, and compare old vs new on random inputs.
• Pin quirks, write down why they exist, and change them only on purpose.
• Keep a copy of the old code until the new code is stable.

Further Reading

• Characterization test, Wikipedia: the idea and where it comes from.
• fast-check documentation: generators for the old-vs-new check.

Next: Security and Dependency Checks. You will find an injected vulnerability, a hardcoded secret and a made-up package name in AI changes.