Advanced Planning

By the Syncfusion DevTutorials team · Last updated

How an agent compares several plans before choosing one: Tree of Thoughts, Monte Carlo Tree Search, and dynamic planning that revises the plan, changes priorities and adapts while it runs.

Think about playing chess.

A beginner often looks at only one move and plays it immediately. An experienced player thinks ahead: "What might happen if I move here? What if I choose a different move?" They compare multiple possible outcomes before making a decision.

Plans can also change during the game. If the other player makes an unexpected move, a good player adapts and creates a new plan.

Agents can work in the same way. They can compare several possible approaches before choosing one, and they can revise their plan when new information appears.

The main idea: Advanced planning means exploring multiple possible plans, evaluating them, and adapting when circumstances change.

A Real-World Example

The Brightgrid ticket page now has two issues in a single line of code. It displays "80000 row" instead of "80,000 rows". The number is missing commas, and the word is incorrect.

A simple agent might try one fix, run the tests, and then try another fix if needed. An agent that plans ahead first creates several possible solutions:

  • Path A: Add commas to the number.
  • Path B: Fix the word.
  • Path C: Fix both problems.

Before changing any code, the agent evaluates each option against the expected test results.

Only Path C passes all the tests, so the agent chooses Path C.

In simple words: Think of several possible solutions, evaluate them, and then choose the one that works best.

What You Will Learn

In this lesson, you will learn:

  1. How Tree of Thoughts evaluates multiple reasoning paths.
  2. How to score possible paths and continue exploring the most promising ones.
  3. How Monte Carlo Tree Search repeatedly tests options to discover the best choice.
  4. How an agent updates plans, changes priorities, and adapts while working.

Real Tree of Thoughts and Monte Carlo Tree Search usually require powerful models and many model calls. The small model running in your browser cannot perform them effectively, so this lesson uses plain JavaScript to simulate the scoring process. The concepts remain the same. The Agent Lab demonstrates Tree of Thoughts selecting a fix and an agent that updates its plan while running.

Tree of Thoughts

Tree of Thoughts is a planning technique where the agent creates several possible next steps, often called paths or thoughts. It evaluates each path and follows the one that appears most promising.

It is called a tree because each path can branch into additional paths, similar to branches growing from a tree.

Tree of Thoughts: multiple paths branch from the problem. Each path is evaluated against the tests, and the most promising path is selected.

Multiple Reasoning Paths

Multiple reasoning paths means the agent does not immediately act on its first idea. Instead, it generates several possible approaches before making a decision.

The first idea is not always the best one. Creating a few alternative options usually costs very little, while following the wrong solution can be expensive and time-consuming.

Path Evaluation

Path evaluation is the process of assigning a score to each possible path.

A good score should be based on something measurable, such as the number of tests that would pass.

The next example scores the three possible fixes using the two test cases from web/format.test.js.

Score three paths against the tests

const tests = [
  { input: 80000, wanted: "80,000 rows" },
  { input: 12, wanted: "12 rows" },
];

const paths = [
  { name: "Path A: add commas", run: (n) => n.toLocaleString("en-US") + " row" },
  { name: "Path B: fix the word", run: (n) => n + " rows" },
  { name: "Path C: commas and the word", run: (n) => n.toLocaleString("en-US") + " rows" },
];

for (const path of paths) {
  const score = tests.filter((t) => path.run(t.input) === t.wanted).length;
  console.log(path.name + ": " + score + " of " + tests.length);
}

Expected output:

Path A: add commas: 0 of 2
Path B: fix the word: 1 of 2
Path C: commas and the word: 2 of 2

Path B passes the small-number test because the value 12 does not need commas. Only Path C passes both tests, making it the only solution that fixes all the problems.

Now the agent can choose the best path with confidence.

Run the example in the Agent Lab and look at the reasoning shown in step 4.

Tree of Thoughts: Compare 3 Paths

Step 4 shows all three paths, their scores, and the selected choice. The agent picks Path C, changes the code only once, and passes all the tests on the first verification.

Search Expansion

Search expansion is the process of growing the tree by exploring additional steps from the most promising paths.

The agent takes its best paths so far and creates new possibilities from them.

The tree cannot keep growing every path forever because the number of possibilities increases very quickly. Instead, the agent keeps only the most promising paths and discards the rest.

This smaller set of selected paths is called a beam. The number of paths kept at each level is called the beam width.

Keep only the best 2 paths at each level

const level1 = [
  { path: "fix web/", score: 7 },
  { path: "fix api/", score: 4 },
  { path: "rewrite everything", score: 2 },
];

const beamWidth = 2;
const kept = [...level1].sort((a, b) => b.score - a.score).slice(0, beamWidth);
console.log("Kept: " + kept.map((p) => p.path).join(", "));

const level2 = kept.flatMap((p) => [
  { path: p.path + " > small change", score: p.score + 2 },
  { path: p.path + " > big change", score: p.score - 1 },
]);
const best = level2.reduce((a, b) => (b.score > a.score ? b : a));
console.log("Paths at level 2: " + level2.length);
console.log("Best: " + best.path + " (score " + best.score + ")");

Expected output:

Kept: fix web/, fix api/
Paths at level 2: 4
Best: fix web/ > small change (score 9)

"Rewrite everything" was removed at level 1, so the agent never spent time exploring that option further. By keeping only the best paths, a beam search keeps the planning process efficient and manageable.

Monte Carlo Tree Search

Monte Carlo Tree Search (MCTS) is a planning method used when the quality of an option cannot be determined immediately. Instead of calculating the best choice in advance, the agent tries different options many times and learns from the outcomes.

The term Monte Carlo refers to methods that use many random trials to estimate the best decision. The name comes from a famous casino city known for games of chance.

Decision Exploration

Exploration means testing options that the agent does not know much about.

Exploitation means choosing the option that has produced the best results so far.

MCTS balances both approaches. It usually prefers options with strong results, but it still occasionally explores other choices in case they turn out to be better than expected.

Utility Scoring

Utility is a numerical value that represents how useful an outcome is.

For example:

  • A successful fix might receive a utility score of 1.
  • A failed fix might receive a utility score of 0.

Utility can also include other factors, such as execution time or cost.

MCTS tracks the average utility of each option. As more trials are performed, these averages become more reliable.

Outcome Evaluation

Outcome evaluation means reviewing the collected results to identify which option performed best.

The agent looks at:

  • How many times each option was tested.
  • The average utility of each option.
  • Which option consistently produced the best outcomes.

In many cases, the option explored most often by the end of the search is the one selected.

The next example runs 60 rounds. Each option has a hidden probability of success that the agent does not know ahead of time.

To decide which option to explore next, the agent uses a common formula called UCB1. UCB1 gives additional priority to options that have been explored less often, helping balance exploration and exploitation.

Monte Carlo Tree Search learns which option is best

function random(seed) {
  return () => {
    seed = (seed * 1103515245 + 12345) % 2147483648;
    return seed / 2147483648;
  };
}
const roll = random(42);

const options = [
  { name: "small fix", chance: 0.7, tries: 0, wins: 0 },
  { name: "big rewrite", chance: 0.3, tries: 0, wins: 0 },
  { name: "change the test", chance: 0.1, tries: 0, wins: 0 },
];

for (let round = 1; round <= 60; round++) {
  const pick = options.reduce((best, o) => {
    const score = o.tries === 0 ? Infinity : o.wins / o.tries + Math.sqrt((2 * Math.log(round)) / o.tries);
    return score > best.score ? { o, score } : best;
  }, { o: null, score: -1 }).o;
  pick.tries++;
  if (roll() < pick.chance) pick.wins++;
}

for (const o of options) {
  console.log(o.name + ": tried " + o.tries + " times, won " + o.wins);
}

Expected output:

small fix: tried 40 times, won 27
big rewrite: tried 13 times, won 4
change the test: tried 7 times, won 0

Without being told which option was best, the agent gradually learned that the "small fix" produced the best results and selected it most often. It still tested the other options occasionally to make sure a better choice was not being missed.

This balance between trying new options (exploration) and using the best-known option (exploitation) is the key idea behind Monte Carlo Tree Search.

Easy Way to Remember: Tree of Thoughts compares ideas. Monte Carlo Tree Search tests ideas repeatedly. Dynamic planning updates the plan when new information appears.

Dynamic Planning

A plan is based on what the agent currently knows. Dynamic planning means updating that plan when new information shows that the original plan is no longer correct.

Plan Revision

Plan revision means changing the plan after something unexpected happens.

For example, an agent fixes the comma formatting issue and runs the tests. Instead of all tests passing, a new failure appears.

The original plan assumed the work would be finished after fixing the commas. The new information shows there is still another problem to solve, so the agent updates its plan and adds another step.

Dynamic Planning: A New Problem Appears

In step 4, a new test failure appears: "80,000 row". In step 5, the agent's reasoning begins with "Plan revised". The agent updates its plan, adds a step to fix the incorrect word, and the tests pass in step 6.

Goal Reprioritization

Goal reprioritization means changing the order of importance of tasks or goals.

For example, an agent may be working on a minor improvement when an urgent production issue is reported. In that situation, the urgent issue should become the highest priority and move ahead of the original task.

Dynamic planning allows the agent to adjust its priorities based on the latest information instead of following the original plan without change.

An urgent task moves to the front

let queue = [
  { task: "Fix the typo on the help page", priority: 3 },
  { task: "Add a dark mode button", priority: 2 },
];

queue.push({ task: "Exports fail for every customer", priority: 0 });
queue = queue.sort((a, b) => a.priority - b.priority);

queue.forEach((q, i) => console.log(i + 1 + ". " + q.task));

Expected output:

1. Exports fail for every customer
2. Add a dark mode button
3. Fix the typo on the help page

The new task arrived last, but it is now first in the queue. A priority of 0 means "handle this immediately."

Runtime Adaptation

Runtime adaptation means changing how the agent works while it is running.

For example, a tool might stop working unexpectedly. Instead of stopping completely, the agent can switch to another tool that performs the same job.

Switch to another tool when one is down

const tools = {
  run_tests: { up: false },
  start_tests: { up: true },
};

function pickTestTool() {
  if (tools.run_tests.up) return "run_tests";
  if (tools.start_tests.up) return "start_tests (background)";
  return "none: stop and report";
}

console.log("Plan step 'check the fix' will use: " + pickTestTool());
tools.start_tests.up = false;
console.log("If both are down: " + pickTestTool());

Expected output:

Plan step 'check the fix' will use: start_tests (background)
If both are down: none: stop and report

The plan step stays the same: "check the fix". Only the tool changes. When no tool can do the job, the agent stops and reports the issue instead of skipping the check.

Comparing multiple paths requires many model calls. Use Tree of Thoughts and MCTS only when making a wrong decision is expensive. For small issues, a simple ReAct approach is usually enough.

Common Mistakes

MistakeIncorrect UnderstandingCorrect UnderstandingWhat Can Go Wrong
Acting on the first idea"My first idea is good enough."Compare a few options for difficult problems before choosing one.The agent fixes only part of the problem.
Scoring based on intuition"This option sounds correct."Score options using measurable results.The agent chooses a confident but incorrect solution.
Expanding every path"More paths always lead to better answers."Keep only the strongest paths at each level.The search becomes slow and expensive.
Only exploiting known options"Always use the option that looks best so far."Occasionally explore other options as well.A better option may never be discovered.
Never changing the plan"Always stick to the original plan."Update the plan when new results reveal unexpected problems.The agent stops while another issue still exists.

Exercises

Each exercise ends with a value on the last line. Press Check to compare it with the expected answer.

What to Remember

  • Tree of Thoughts generates multiple possible paths and follows the one that performs best.
  • Score paths using something measurable, and keep only the most promising paths.
  • MCTS evaluates options through repeated trials and balances exploration with exploitation.
  • Dynamic planning updates the plan when new results show that the current plan needs to change.
  • Use advanced planning methods when the cost of making a wrong decision is high.

Next: Loop Mechanics, what happens inside one turn of the agent loop.