Evaluator-Optimizer

By the Syncfusion DevTutorials team · Last updated October 20, 2026

A fixer and a checker go round until the work passes. Give the loop real exits for oscillation, no progress, a flat score and a budget, report a stop on a bound clearly, then export your first graph pack.

Watch a Fixer Go Back and Forth

A failing test checks how prices are rounded. A fixer agent changes the code. A checker runs the tests. If they fail, the fixer tries again. That is the evaluator-optimizer pattern, and it is the loop you built in the Loop course.

This fixer has one stop rule: 30 tries. Run it and read the tries.

Try 1 rounds half down: one test fails. Try 2 rounds half up: another test fails. Try 3 rounds half down again. It is going back and forth between two broken fixes, and it will do that 30 times. Then the result says "failed", and nobody can tell what happened.

In this lesson you give the loop exits that notice this early, and report it clearly.

TL;DR

• Evaluator-optimizer: a fixer makes a change, a checker scores it, and they go round until a clear pass condition is met.
• It works only when the pass condition is mechanical: a test command, a schema, a number.

• It fails when the exits are weak. Four exits catch the common ways it gets stuck:

  • oscillation: going back and forth, A → B → A;
  • no_progress: the same diff and the same error again;
  • flat_score: several tries with no real gain;
  • budget: a token limit for this node.
• Keep max_iterations as a small backstop, not the main exit.
• A stop on a bound is not a failure. Report it as "stopped on bound" and send it to a person.

A Real-World Example

Ana's team runs a fixer loop on flaky end-to-end tests. Every morning the bill shows one loop that ran 40 times overnight. Each try changed a wait time from 500 ms to 1000 ms and back.

The logs only said FAILED. A developer spent an hour each time working out that the code was fine and the loop was stuck.

They added two exits: stop if a try repeats a try from two steps before, and stop if the same error appears twice in a row. They also changed what the loop reports: "stopped on bound: oscillation, here are both diffs". The overnight cost dropped, and the morning message now says exactly where to look.

What You Will Learn

• When evaluator-optimizer helps, and what it needs
• Four exits: oscillation, no progress, flat score, budget
• Why the turn cap is a backstop, not a plan
• How to report a stop on a bound and escalate it
• How to export your graph as a graph pack for your tool

The Pattern

The loop has two roles. The optimizer (the fixer) changes the work. The evaluator (the checker) scores it against a pass condition. In this course, both live in one node: an agent node with an edge back to itself and an exit: block.

nodes:
  - { id: fix, kind: agent, on_bound: escalate, reads: [ticket, last_error], writes: [diff, last_error],
      exit: { checker: pnpm test, max_iterations: 10, oscillation: true, no_progress: 2, flat_score: 3, budget: 20000 } }
edges:
  - { from: fix, to: fix }

It is the right pattern when you can state the pass condition before you start, and a program can check it. "All tests pass" works. "Make the code nicer" does not.

Four Exits

Oscillation: A → B → A

The fixer swaps between two broken fixes. Each one fixes the other's test and breaks its own.

Spot oscillation

const tries = ["round half down", "round half up", "round half down", "round half up"];

function oscillates(list) {
  for (let i = 2; i < list.length; i++) {
    if (list[i] === list[i - 2] && list[i] !== list[i - 1]) return "oscillation at try " + (i + 1);
  }
  return "no oscillation";
}

console.log(oscillates(tries));

Expected output:

oscillation at try 3

No Progress: The Same Thing Again

The fixer makes the same change and gets the same error. Trying a third time will not help.

Spot no progress

const tries = [
  { diff: "add null check", error: "test_empty_cart" },
  { diff: "add null check", error: "test_empty_cart" },
  { diff: "add null check", error: "test_empty_cart" },
];

let same = 1;
for (let i = 1; i < tries.length; i++) {
  same = tries[i].diff === tries[i - 1].diff && tries[i].error === tries[i - 1].error ? same + 1 : 1;
  if (same >= 2) { console.log("no progress at try " + (i + 1)); break; }
}

Expected output:

no progress at try 2

Flat Score, Rising Cost

The sneakiest one. Every try is different, so it looks busy. But the score does not really improve, and each try reads more context and costs more.

Spot a flat score

const scores = [61, 62, 61, 62, 61];
const costs = [1500, 1900, 2300, 2700, 3100];
const FLAT = 3;

for (let i = FLAT; i < scores.length; i++) {
  const before = Math.max(...scores.slice(0, i - FLAT + 1));
  const recent = Math.max(...scores.slice(i - FLAT + 1, i + 1));
  if (recent <= before + 1) {
    const spent = costs.slice(0, i + 1).reduce((a, b) => a + b, 0);
    console.log("flat score at try " + (i + 1) + ", spent " + spent + " tokens");
    break;
  }
}

Expected output:

flat score at try 4, spent 8400 tokens

Budget

budget is a token limit for this node. It always stops a runaway, but often late. Run the same fixer with only a budget:

The budget worked, but only after paying for many useless tries. The other three exits stop it much earlier, because they notice why it is stuck. Keep the budget as a safety net under them.

Stopped on a Bound Is Not Failed

When an exit fires, the loop did its job: it noticed it was stuck and stopped. That is very different from "the code is broken" or "the tests are wrong".

Set on_bound: escalate on the fixer. Then the run is reported as "stopped on bound: oscillation" (or the exit that fired). A person gets the evidence: the last diffs, the errors and the score history.

Report the stop clearly

function report(stop, evidence) {
  if (stop === "pass") return "fixed";
  if (stop === "error") return "failed: " + evidence;
  return "stopped on bound: " + stop + " -> sent to a person with " + evidence;
}

console.log(report("pass", ""));
console.log(report("oscillation", "both diffs and both errors"));
console.log(report("error", "the test runner crashed"));

Expected output:

fixed
stopped on bound: oscillation -> sent to a person with both diffs and both errors
failed: the test runner crashed

All Exits Together

Here are the exits working together on one stuck run. The first one that fires stops the loop:

The first exit that fires

const tries = [
  { diff: "A", error: "e1", score: 60, tokens: 2500 },
  { diff: "B", error: "e2", score: 58, tokens: 2500 },
  { diff: "A", error: "e1", score: 60, tokens: 2500 },
];
const BUDGET = 20000, CAP = 10;

let spent = 0, stop = "";
for (let i = 0; i < tries.length && !stop; i++) {
  spent += tries[i].tokens;
  if (i >= 2 && tries[i].diff === tries[i - 2].diff && tries[i].diff !== tries[i - 1].diff) stop = "oscillation";
  else if (i > 0 && tries[i].diff === tries[i - 1].diff && tries[i].error === tries[i - 1].error) stop = "no_progress";
  else if (spent >= BUDGET) stop = "budget";
  else if (i + 1 >= CAP) stop = "max_iterations";
}
console.log("stopped on bound: " + stop + " after " + spent + " tokens");

Expected output:

stopped on bound: oscillation after 7500 tokens

Lab Exercise: Write the Exits

Open the graph file. Change the fix node so that:

1. Its exit: has oscillation: true, no_progress: 2, flat_score: 3 and a budget (for example 20000).
2. max_iterations is 12 or less.
3. It has on_bound: escalate.

Then press Check my graph. In each of the 3 versions the fixer gets stuck in a different way: oscillation, no progress, or a flat score with rising cost. Each run must stop under 10,000 tokens and be reported as "stopped on bound".

Your First Graph Pack

You now have every part of a working graph: nodes, a state contract, a handoff, a router, joins, caps and exits. Time to take one home.

Open the Export tab. It turns the graph file you are looking at into files your tool can follow:

• graph.md: the topology, written as rules the main agent follows;
• one subagent file per node, in your tool's format (.claude/agents/, .codex/agents/ or .cursor/agents/);
• state-contract.yaml: every field, its type and its one writer;
• a supervisor subagent that checks the state file between nodes and stops the graph if something drifts;
• GRAPH-README.md: an optional checklist for a first real run.

The smallest useful pack is two nodes: an executor that does the work round by round, and a supervisor that checks it from outside. The open-source graphkit skill uses the same shape. Each round, the executor writes what it did to a log file. A supervisor with a clean context reads that file, spots when the work goes off track, and writes fixes.

Common mistakes

Exercises

Each exercise ends with a value on the last line. Press Check to compare it with the expected answer.

What to Remember

• Evaluator-optimizer: a fixer and a checker go round against a mechanical pass condition.
• Four exits catch the common stuck states: oscillation, no progress, flat score, budget.
• The turn cap is a small backstop, not the plan.
• A stop on a bound is not a failure. Report it with evidence and escalate it.
• The Export tab turns any graph file into a graph pack for your tool.

Further Reading

• How to Implement Graph Engineering, by QA Wolf: evaluator-optimizer and the four missing exit conditions.
• Building Effective Agents, by Anthropic: the evaluator-optimizer workflow and when it fits.
• graphkit, by levi-qiao: a framework-free executor and supervisor graph, kept in Markdown files.
• The Stop Rules lesson in the Loop Engineering course: the stop rules this lesson builds on.

Next: Composing Patterns. Two audits feed a fixer with no verifier, router or gate, and you compose a review graph that meets a scorecard.