Generate Knowledge Prompting

By the Syncfusion DevTutorials team · Last updated

Ask for facts before the answer: the single-prompt and two-step versions, splitting facts from the answer in code, flagging facts to check, and how it differs from grounding.

Ask a model "Is it a good idea to water a cactus every day?" and it may answer quickly and badly. Now ask it to first "list 4 facts about how cacti store water and what harms them", and then answer the question using those facts. The answer is usually more careful and more correct.

This is called generate knowledge prompting. The model first writes down the facts it knows that matter for the question, and then uses them to answer. This brings useful knowledge to the front of the text, where it can guide the answer.

The main idea: ask for the facts first, and the answer second. The facts become context for the answer, and you can check them.

A Real-World Example

Think about a student answering an exam question about the water cycle.

  • A student in a hurry writes the answer straight away and forgets half the steps.
  • A careful student first writes the key words in the margin: evaporation, condensation, rain, rivers. Then they write the answer using that list.

The list doesn't add any new knowledge. It brings what the student already knows to the front, so nothing important is missed. Generate knowledge prompting does the same thing for a model.

What You Will Learn

In this lesson, you will learn:

  1. What generate knowledge prompting is and why it helps.
  2. The one-prompt version: facts first, then the answer.
  3. The two-step version: one prompt for the facts, and another for the answer.
  4. How to read and check the model's facts with code.
  5. How this is different from grounding with your own documents.
  6. When it helps and when it is risky.

Try the one-prompt version in the Compare tab of the Prompt sandbox. Then try the two-step version: copy the facts from one reply into a second prompt, in the Context part of the Builder tab.

Why Facts First Helps

Generate knowledge: in step 1, the model writes facts that matter for the question. In step 2, it answers using those facts. The facts give context for the answer, and they give you a place to catch mistakes.

A model writes one token at a time. When it answers straight away, it guesses the first words of the answer with only the question as context. When it writes the facts first:

  • The facts are now in the text, so the answer can build on them.
  • The model is less likely to jump to an answer that is popular but wrong.
  • You can see the facts, and spot a wrong one before you trust the answer.

This is close to chain of thought. The difference is what the model writes first: chain of thought writes steps, while generate knowledge writes facts.

The Single-Prompt Version

Ask for the facts and the answer in one reply, in two parts with labels.

Facts first, then the answer, in one prompt

function knowledgeFirst(question, n) {
  return [
    "Question: " + question,
    "First, under the heading Facts:, list " + n + " short facts that are relevant to the question, one per line starting with -.",
    "Then, under the heading Answer:, answer the question in at most 2 sentences, using only those facts.",
  ].join("\n");
}

console.log(knowledgeFirst("Is it a good idea to water a cactus every day?", 4));

Expected output:

Question: Is it a good idea to water a cactus every day?
First, under the heading Facts:, list 4 short facts that are relevant to the question, one per line starting with -.
Then, under the heading Answer:, answer the question in at most 2 sentences, using only those facts.

Compare it with asking the question directly:

Read the Facts and the Answer

The reply has two parts with labels, so code can separate them. The reply below is a sample written for this example.

Split a reply into facts and answer

function splitReply(reply) {
  const [factsPart, answerPart = ""] = reply.split(/\n?Answer:/i);
  const facts = factsPart
    .split("\n")
    .map((l) => l.trim())
    .filter((l) => l.startsWith("-"))
    .map((l) => l.slice(1).trim());
  return { facts, answer: answerPart.trim() };
}

const reply = "Facts:\n- Cacti store water in their stems.\n- They grow in dry places.\n- Too much water makes their roots rot.\n- Most need water every 1 to 3 weeks.\nAnswer: No. Daily watering can rot the roots; water every 1 to 3 weeks instead.";
const parts = splitReply(reply);

console.log(parts.facts.length + " facts");
parts.facts.forEach((f) => console.log("-", f));
console.log("Answer:", parts.answer);

Expected output:

4 facts
- Cacti store water in their stems.
- They grow in dry places.
- Too much water makes their roots rot.
- Most need water every 1 to 3 weeks.
Answer: No. Daily watering can rot the roots; water every 1 to 3 weeks instead.

Now each fact is its own string. You can show the facts to a person, check them, or pass them to a second prompt.

Easy Way to Remember: generate knowledge means "What do you know? Now answer." Use two parts with labels, so you can check the first part before you trust the second.

The Two-Step Version

You can also split the work into two prompts, one after the other:

  1. Prompt 1 asks only for the facts.
  2. Your code checks the facts, and can remove or fix some of them.
  3. Prompt 2 gives the facts as context and asks the question.

This needs two calls to the model, but you get a check in between. This matters when a wrong fact would lead to a wrong answer.

Check facts, then build the answer prompt

const generated = [
  "Cacti store water in their stems.",
  "Too much water makes their roots rot.",
  "Cacti need water every day in summer.",
  "Most need water every 1 to 3 weeks.",
];

// A person or a trusted source marked this fact as wrong.
const rejected = ["Cacti need water every day in summer."];
const kept = generated.filter((f) => !rejected.includes(f));

const step2 = "Answer using only these facts.\n<facts>\n" + kept.map((f) => "- " + f).join("\n") + "\n</facts>\nQuestion: Is it a good idea to water a cactus every day?";
console.log(step2);

Expected output:

Answer using only these facts.
<facts>
- Cacti store water in their stems.
- Too much water makes their roots rot.
- Most need water every 1 to 3 weeks.
</facts>
Question: Is it a good idea to water a cactus every day?

The wrong fact never reaches the answer prompt. Step 2 is now a grounded prompt, just like the ones in the Context and Grounding lesson.

Check Generated Facts

The model's facts come from its memory, so they can be wrong. Before you trust them, look for warning signs. The next box flags facts that have exact numbers, or strong words like "always" and "never". These are the facts most worth checking.

Flag facts that need checking

function needsCheck(fact) {
  const reasons = [];
  if (/\d/.test(fact)) reasons.push("has a number");
  if (/\b(always|never|all|every|only)\b/i.test(fact)) reasons.push("absolute word");
  if (/\b(19|20)\d{2}\b/.test(fact)) reasons.push("has a year");
  return reasons;
}

const facts = [
  "Cacti store water in their stems.",
  "Cacti never need water in winter.",
  "The saguaro cactus can grow over 12 metres tall.",
];
facts.forEach((f) => {
  const r = needsCheck(f);
  console.log(r.length ? "CHECK (" + r.join(", ") + "):" : "ok:", f);
});

Expected output:

ok: Cacti store water in their stems.
CHECK (absolute word): Cacti never need water in winter.
CHECK (has a number): The saguaro cactus can grow over 12 metres tall.

A simple check like this doesn't prove anything. It just tells a person where to look first.

Did the Answer Use the Facts?

The prompt says "using only those facts". You can roughly check whether the answer did this. The next box looks for the key words of each fact in the answer.

Check which facts the answer used

const facts = [
  "Cacti store water in their stems.",
  "Too much water makes their roots rot.",
  "Most need water every 1 to 3 weeks.",
];
const answer = "No. Daily watering can make the roots rot. Water every 1 to 3 weeks instead.";

const stop = ["their", "most", "make", "makes", "much", "water", "cacti", "need", "store"];
const answerWords = new Set(answer.toLowerCase().match(/[a-z0-9]+/g));

facts.forEach((f) => {
  const keys = f.toLowerCase().match(/[a-z0-9]+/g).filter((w) => w.length > 2 && !stop.includes(w));
  const hits = keys.filter((w) => answerWords.has(w));
  console.log(hits.length ? "used" : "not used", "|", f, "|", hits.join(" ") || "-");
});

Expected output:

not used | Cacti store water in their stems. | -
used | Too much water makes their roots rot. | roots rot
used | Most need water every 1 to 3 weeks. | every weeks

Two of the facts shaped the answer. The first fact wasn't needed. If an answer uses none of the facts, the model probably ignored them and guessed, so check that answer.

Generate Knowledge or Grounding?

Both methods put facts before the answer. The difference is where the facts come from.

Generate knowledgeGrounding
Where the facts come fromThe model's own memoryYour own trusted documents
Can the facts be wrong?Yes, they are guessesOnly if your documents are wrong
Does it work for private or new facts?NoYes
Best forQuestions about general knowledgeCompany data, recent information, and anything you must be able to trust

When you have trusted documents, use grounding. Use generate knowledge when the question is about general knowledge and you have no documents to give the model.

The model's facts are not sources. A model can write a fact that sounds sure but is wrong, and then build an answer on it that also sounds sure but is wrong. Never present the model's facts as checked facts. Never use this technique for private, recent, medical, legal, or money questions without a trusted source.

Common mistakes

MistakeWrong ideaCorrect ideaWhat goes wrong
Trusting the facts"It listed facts, so the answer must be correct."The facts are also guesses from the model's memory.A wrong fact leads to a wrong answer that sounds sure.
Reply without labels"I'll find the answer in the text."Use the headings Facts: and Answer:.Code can't separate the facts from the answer.
Too many facts"More facts mean a better answer."Ask for 3 to 5 facts that matter for the question.The replies get long and full of facts that barely matter.
Using it for private data"It can list facts about our product."It has never seen your data. Use grounding.The model makes up facts about your product.
No check between the steps"Using two steps is enough."Check or remove facts before step 2.Wrong facts go straight into the answer.
The answer ignores the facts"It will use the facts."Say "using only those facts".The answer slides back to a popular guess.

Exercises

In each exercise, the last line should give back a value. Press Check to compare your value with the expected answer.

What to Remember

  • Generate knowledge asks for the facts that matter first, and then an answer that uses them.
  • It works because the facts become context for the answer.
  • Use parts with labels (Facts: and Answer:) so code can split them.
  • The two-step version adds a check in between, so you can remove wrong facts.
  • The model's facts are guesses from its memory. Mark facts with numbers or words like "always" and "never" for checking.
  • When you have trusted documents, use grounding instead.

Next: Prompt Evaluation and Optimization, where you test prompts properly and use scores to improve them.