# Retrieval Patterns

Course: [Context Engineering](https://www.devtutorials.ai/learn/context-engineering) · Section: Design Patterns
Source: https://www.devtutorials.ai/learn/context-engineering/design-patterns/retrieval-patterns

> How to handle retrieved documents once they are in the context: relevance, not-enough-evidence, confidence, and metadata such as region, date, tier, role and clearance, plus conflict rules.

Imagine asking a friend to fetch "the cookbook" from your shelf. They come back with three books: a cookbook from 2010, a new cookbook, and a gardening book that was next to them. Before you cook, you check: which book is the right one? Is the recipe in it at all? And if the two cookbooks disagree, which one do you trust?

An AI app has the same problem. A search step finds documents and puts them in the context. Some are useful. Some are not. Some are old. Some disagree. The model must not trust them all blindly.

**The main idea:** check what retrieval brought back. Use the useful, current, allowed documents, and say so when they are not enough.

## A Real-World Example

Think about a new employee looking up the holiday policy on the company intranet.

- The search shows the 2023 policy and the 2026 policy. The employee must use the 2026 one.
- One result is the policy for the US office. The employee works in the EU office.
- One result is a managers-only guide. The employee should not be reading it at all.

A careful employee looks at the **date**, the **region**, and **who it is for** before trusting any page.

**In simple words:** a retrieved document needs a quick check before it is used: is it relevant, is it current, and is it meant for this user?

## What You Will Learn

In this lesson, you will learn:

1. Where retrieval fits in a context.
2. How to judge whether retrieved text is relevant.
3. What to do when the evidence is not enough.
4. How to judge confidence.
5. How to use metadata: regions, dates, tiers, roles, and clearance levels.
6. How to settle conflicts between documents.
7. How to combine both patterns into one assessment step.

The Builder has two new parts for this lesson: an **Old pricing document** that disagrees with the current one, and **Metadata rules** that say how to choose. Try them together.

## RAG From the Context View

**RAG** stands for **retrieval-augmented generation**. The app first **retrieves** documents that match the question. Then it adds them to the context, so the model can **generate** an answer from them.

The RAG course covers how the search itself works: chunks, embeddings, and ranking. This lesson looks at one question only: **once documents arrive in the context, how should they be handled?**

That gives two patterns:

- The **retrieval-aware pattern** checks the quality of what came back.
- The **metadata-aware pattern** checks the labels on what came back.

## Retrieval-Aware Pattern

Search is never perfect. It can return the wrong document, a half-right one, or nothing useful at all. The **retrieval-aware pattern** treats retrieved text as **possible** evidence, not as proven fact.

### Relevance Evaluation

**Relevance** is how well a document matches the question. **Relevance evaluation** means scoring each document and keeping only the ones that score well.

Search tools give each result a score, usually from 0 to 1. You can also check relevance yourself. The next box gives each chunk a simple score: the share of question words it contains.

```js runnable title="Keep the chunks that match the question"
const question = ["export", "limit", "free"];

const chunks = [
  { id: "doc-1", text: "Free plan export limit: 5,000 rows." },
  { id: "doc-2", text: "Themes can be changed in Grid settings." },
  { id: "doc-3", text: "The Pro plan has a higher export limit." },
];

function relevance(text) {
  const words = text.toLowerCase();
  return question.filter((w) => words.includes(w)).length / question.length;
}

for (const c of chunks) {
  const score = relevance(c.text);
  console.log(c.id + " score " + score.toFixed(2) + (score >= 0.6 ? " keep" : " drop"));
}
```

Expected output:

```text
doc-1 score 1.00 keep
doc-2 score 0.00 drop
doc-3 score 0.67 keep
```

The theme article was found by search, but it has nothing to do with the question. Dropping it removes a distraction. Real systems use better scores, but the idea is the same: **a cut-off line, below which a document does not go into the context**.

You can also ask the model to judge relevance itself: "Before answering, decide which documents are relevant. Ignore the others."

### Insufficiency Handling

Sometimes nothing relevant comes back. Or the relevant documents cover only part of the question. This is called **insufficient evidence**.

There are three good responses:

1. **Say so.** "The documents don't cover this." This is the refusal rule from Lesson 5.
2. **Search again**, with different words, or in another source.
3. **Answer the part you can**, and say clearly which part is missing.

The bad response is the common one: answer anyway, from memory.

### Confidence Evaluation

**Confidence** is how sure the system can be that the answer is right. It should come from the **evidence**, not from how sure the model sounds.

A simple rule: use the scores of the useful chunks.

```js runnable title="Judge the evidence before answering"
function assess(scores, threshold) {
  const good = scores.filter((s) => s >= threshold);
  if (good.length === 0) return "insufficient: say you don't know, or search again";
  const best = Math.max(...good);
  const confidence = best >= 0.85 ? "high" : best >= 0.6 ? "medium" : "low";
  return good.length + " useful chunk(s), confidence " + confidence;
}

console.log(assess([0.91, 0.34, 0.12], 0.6));
console.log(assess([0.64, 0.61], 0.6));
console.log(assess([0.31, 0.22, 0.18], 0.6));
```

Expected output:

```text
1 useful chunk(s), confidence high
2 useful chunk(s), confidence medium
insufficient: say you don't know, or search again
```

With medium confidence, the app can still answer, but add "please check" or a link to the source. With no useful chunks, it should not answer at all.

Try it with the real model. The question asks about something the help article does not cover.

A good reply says the documents don't cover it. If the model invents a number of days, that is the failure this pattern exists to stop.

**Easy Way to Remember:** retrieved text is a **suspect, not a witness**, until it passes the checks.

## Metadata-Aware Pattern

**Metadata** is the label on a document: where it came from, when, and who it is for. You met metadata headers in Lesson 1. The **metadata-aware pattern** uses those labels to decide which documents may be used for **this** user and **this** question.

The same sentence can be true in one place and false in another. "The export limit is 5,000 rows" may be true for the EU Free plan, and false for the US Free plan. Only the label tells them apart.

### Regions

Many rules differ by **region**: prices, taxes, laws, and features. A document labelled `region: US` should not answer a question from an EU customer.

### Effective Dates

An **effective date** is the day a document starts to apply. Some documents also have an end date.

Two traps:

- An **old** document that has been replaced.
- A **future** document, written early, that does not apply yet.

Compare each document's dates with today's date.

### Product Tiers

A **product tier** is a level of the product, such as Free, Pro, or Enterprise. Limits and features change between tiers. A Pro article must not answer a Free customer's question about their own limits.

### Roles

A **role** is the kind of user: customer, support agent, manager, or admin. Some documents are written for staff only. They may contain steps a customer must not see or use.

### Clearance Levels

A **clearance level** is how much a user is allowed to see, often a number such as 1 to 3. A document marked level 3 must not reach a user with level 1.

This is a **security** rule, not a quality rule. Filter by clearance **before** the documents go into the context. Don't trust the model to keep a secret it can see.

The next box filters five documents by all five labels at once.

```js runnable title="Keep only documents meant for this user, today"
const user = { region: "EU", tier: "Free", role: "customer", clearance: 1, today: "2026-10-09" };

const docs = [
  { id: "doc-1", region: "EU", tier: "Free", role: "customer", clearance: 1, effective: "2026-05-01", text: "Free plan export limit: 5,000 rows." },
  { id: "doc-2", region: "US", tier: "Free", role: "customer", clearance: 1, effective: "2026-05-01", text: "Free plan export limit: 10,000 rows." },
  { id: "doc-3", region: "EU", tier: "Pro", role: "customer", clearance: 1, effective: "2026-05-01", text: "Pro export limit: 1,000,000 rows." },
  { id: "doc-4", region: "EU", tier: "Free", role: "staff", clearance: 3, effective: "2026-05-01", text: "Staff note: limits can be raised by hand." },
  { id: "doc-5", region: "EU", tier: "Free", role: "customer", clearance: 1, effective: "2027-01-01", text: "From 2027 the Free limit is 8,000 rows." },
];

const allowed = docs.filter((d) =>
  d.region === user.region &&
  d.tier === user.tier &&
  d.role === user.role &&
  d.clearance <= user.clearance &&
  d.effective <= user.today,
);

console.log("Kept:", allowed.map((d) => d.id).join(", "));
console.log(allowed[0].text);
```

Expected output:

```text
Kept: doc-1
Free plan export limit: 5,000 rows.
```

Each of the other four documents is dropped for a different reason: wrong region, wrong tier, staff only, and not in effect yet. Every one of them was a real search result. Without the labels, any of them could have produced a wrong answer.

### Conflict Resolution

Sometimes two documents that both pass the filters still disagree. **Conflict resolution** is the rule that decides which one wins.

A common rule has two steps:

1. Prefer the **more trusted** source: official before community, community before archived.
2. If trust is equal, prefer the **newer** one.

```js runnable title="When documents disagree, trust first, then date"
const docs = [
  { id: "pricing-2024", updated: "2024-01-10", trust: "archived", text: "Pro costs $15 per developer per month." },
  { id: "pricing-2026", updated: "2026-03-12", trust: "official", text: "Pro costs $20 per developer per month." },
  { id: "forum-88", updated: "2026-09-30", trust: "community", text: "Pro is $10 if you ask nicely." },
];

const rank = { official: 3, community: 2, archived: 1 };

const winner = [...docs].sort((a, b) => rank[b.trust] - rank[a.trust] || b.updated.localeCompare(a.updated))[0];

console.log("Use:", winner.id);
console.log(winner.text);
```

Expected output:

```text
Use: pricing-2026
Pro costs $20 per developer per month.
```

The forum post is the newest, but it is not official. Trust wins over date. Write the same rule into the context, so the model uses it too.

Try it. Both contexts contain the current article and the old one. Only B has the metadata rules.

The right answer is $20, from the official 2026 article. Without the rules, the model has to guess between $15 and $20. With them, it has a clear way to choose, and it should name the source it used.

### Metadata Omissions

A **metadata omission** is a missing label. Real documents often have no region, no date, or no trust level.

Decide in advance what a missing label means. For example:

- No region: treat the document as general, and say so in the answer.
- No date: treat it as older than any dated document.
- No clearance level: treat it as the **highest** level, so it is hidden until someone labels it.

```js runnable title="A missing label needs a rule too"
const docs = [
  { id: "doc-1", region: "EU", text: "Free plan export limit: 5,000 rows." },
  { id: "doc-2", text: "Exports run in the background." },
];

for (const d of docs) {
  const note = d.region ? "region " + d.region : "no region label: treat as general, and say so";
  console.log(d.id + ": " + note);
}
```

Expected output:

```text
doc-1: region EU
doc-2: no region label: treat as general, and say so
```

The safest default depends on the label. For quality labels like region, a missing label can mean "general". For security labels like clearance, a missing label must mean "keep it hidden".

## Pattern Integration

The two patterns work best together, as one step between search and answer.

### Retrieval Assessment

**Retrieval assessment** is that step. For each retrieved document, the app asks, in order:

1. **Is this user allowed to see it?** Check clearance and role. If not, drop it.
2. **Does it apply?** Check region, tier, and dates. If not, drop it.
3. **Is it relevant?** Check the score. If it is too low, drop it.
4. **Do the survivors agree?** If not, use the conflict rule.
5. **Is it enough?** If not, refuse, search again, or answer only part.

| Step | Checks | Kind of check |
|---|---|---|
| Allowed | Clearance, role | Security: done in code, before the context |
| Applies | Region, tier, effective date | Correctness |
| Relevant | Relevance score | Quality |
| Agrees | Trust level, date | Conflict resolution |
| Enough | Number and strength of survivors | Confidence |

### Metadata Overrides

A **metadata override** is when a label beats a score. A document can be the best match for the question and still be the wrong one to use.

For example, the US export article may match "Free plan export limit" perfectly. But the user is in the EU. The region label **overrides** the relevance score, and the document is dropped.

The rule of thumb: **filters first, scores second.** A high score never lets a document past a security or applicability filter.

Never rely on the model to hide documents a user is not allowed to see. If restricted text is in the context, it can leak. Filter it out in code before the context is built.

## Common mistakes

| Mistake | Wrong idea | Correct idea | What goes wrong |
|---|---|---|---|
| Trusting every result | "Search found it, so it's relevant." | Score each result and drop weak ones. | Off-topic text distracts the model. |
| Ignoring dates | "Any version of the policy will do." | Use the document in effect today. | Old prices or limits are quoted. |
| Wrong region or tier | "The best match is the right one." | Labels override scores. | An EU customer gets US rules. |
| Model as gatekeeper | "I told the model not to reveal staff notes." | Remove restricted documents in code first. | Restricted text leaks into answers. |
| No conflict rule | "The model will pick the right one." | Trust first, then date. Write it down. | The answer flips between two prices. |
| Missing labels ignored | "No label means it applies everywhere." | Set a default for each missing label. | Unlabelled private text is treated as public. |

## Exercises

Each exercise ends with a value on the last line. Press **Check** to compare it with the expected answer.

## What to Remember

- **RAG** puts retrieved documents in the context. This lesson is about handling them once they arrive.
- **Relevance evaluation** drops weak results. **Insufficiency handling** refuses, searches again, or answers only part.
- **Confidence** should come from the evidence, not from the model's tone.
- **Metadata** labels decide whether a document applies: **region**, **effective date**, **tier**, **role**, and **clearance**.
- **Conflict resolution**: trust first, then date. Write the rule into the context.
- Decide what **missing labels** mean. For security labels, missing means hidden.
- **Filters first, scores second.** Clearance and role are checked in code, before the context is built.

Next: **Memory Patterns**, how to keep the important parts of a long conversation without sending all of it every time.