Top-K Retrieval

By the Syncfusion DevTutorials team · Last updated October 20, 2026

Turning a question into the right few chunks: score cutoffs, top-k selection, dynamic thresholds, the search flow and its latency, deduplication, out-of-distribution checks, fallbacks and corrective retrieval.

A shop assistant who always brings you exactly three items, whether or not the shop has what you asked for, is not very helpful. Ask for "blue running shoes, size 44" in a shop that only sells hats, and you still get three hats. A good assistant brings only things that really match, says "we do not have that" when nothing fits, and asks a colleague or checks the back room when the first look comes up short.

Retrieval is the same. Top-k search always returns k chunks. Your job is to keep the good ones, drop the noise, and know when to try again or say "I do not know".

The main idea: top-k retrieval turns the question into a vector, searches the index, ranks by distance, and fetches the chunks' text and metadata. Then quality control takes over: score cutoffs (fixed or dynamic) remove weak matches, deduplication removes copies, out-of-distribution checks catch questions far from your content, and fallbacks like corrective retrieval try again before giving up.

A Real-World Example

Brightgrid noticed something odd in its logs: for every question, the assistant received exactly three chunks, even for "What is the CEO's favourite colour?" The model got privacy, contact and safety articles, tried to be helpful, and wrote a confusing answer about data privacy.

The team added three rules:

1. A score cutoff. Chunks below a minimum similarity are dropped. If nothing is left, the app answers "I do not know based on the help center" without calling the model at all.
2. Deduplication. The two near-identical password articles no longer fill two of the three slots.
3. A corrective retry. If the best score is weak, the app expands the question with synonyms and searches once more before giving up. "My light has no internet" now finds the Wi-Fi article.

Unhelpful answers dropped by a third, and model costs fell too, because refused questions never reached the model.

In simple words: take only what really matches, and have a plan for when nothing does.

What You Will Learn

In this lesson, you will learn:

1. Retrieval thresholding: score cutoffs, top-k selection, noise filtering and dynamic thresholds.
2. The search execution flow: query vectorization, index search and ranking, payload extraction and latency tracking.
3. Retrieval quality control: out-of-distribution matches, duplicates, low-confidence results and fallback retrieval.

Retrieval Thresholding

Top-K Candidate Selection

k is how many chunks you keep. Small k (3–5) keeps prompts short and focused. Larger k (10–20) gives the model more to work with, but adds noise and cost. A common pattern is to retrieve more than you need, say 20, then cut and rerank (next chapter) down to the best 3–5.

Similarity Score Cutoffs

Top-k has a blind spot: it returns k results no matter what. A fixed minimum score removes chunks that are not really related:

Top-k alone vs top-k with a cutoff

// Top-k always returns k results, even when none are relevant. A score cutoff fixes that.
const results = {
  "How long is the warranty?": [
    { id: "warranty#1", score: 0.82 },
    { id: "warranty#3", score: 0.61 },
    { id: "safety#2", score: 0.34 },
  ],
  "What is the CEO's favourite colour?": [
    { id: "privacy#1", score: 0.21 },
    { id: "contact-support#1", score: 0.19 },
    { id: "safety#1", score: 0.17 },
  ],
};

const MIN_SCORE = 0.4;
for (const [q, hits] of Object.entries(results)) {
  const kept = hits.filter((h) => h.score >= MIN_SCORE);
  console.log(q);
  console.log("  top-3 only:   ", hits.map((h) => h.id).join(", "));
  console.log(
    "  with cutoff:  ",
    kept.map((h) => h.id).join(", ") || "(nothing: answer 'I do not know')",
  );
}

Expected output:

How long is the warranty?
  top-3 only:    warranty#1, warranty#3, safety#2
  with cutoff:   warranty#1, warranty#3
What is the CEO's favourite colour?
  top-3 only:    privacy#1, contact-support#1, safety#1
  with cutoff:   (nothing: answer 'I do not know')

Choose the cutoff by looking at your gold set: what scores do correct chunks usually get, and what do unrelated ones get? Score ranges differ between embedding models, so a cutoff must be chosen per model.

Filtering Noise Candidates

Noise is a chunk that matched for the wrong reason: a shared common word ("Brightgrid", "lamp"), a navigation line, or a vague, general paragraph. Cutoffs remove much of it. Clean ingestion (Chapter 2) and good chunking remove the rest at the source.

Dynamic Score Radius Thresholds

A fixed cutoff can be too strict for some questions and too loose for others. Dynamic thresholds adapt to each result list:

• Relative cutoff: keep chunks scoring at least, say, 80% of the best one.
• Largest gap ("elbow"): cut where scores suddenly drop between two neighbours.

Relative cutoff and largest gap

// Dynamic thresholds: compare scores with the best one, and cut at a big drop.
const hits = [
  { id: "brightgrid-plus#2", score: 0.81 },
  { id: "brightgrid-plus#1", score: 0.77 },
  { id: "schedules#1", score: 0.52 },
  { id: "energy-use#2", score: 0.5 },
  { id: "billing-questions#1", score: 0.47 },
];

// Rule 1: keep scores within 80% of the best.
const best = hits[0].score;
const relative = hits.filter((h) => h.score >= 0.8 * best);
console.log("within 80% of best:", relative.map((h) => h.id).join(", "));

// Rule 2: cut at the largest gap between neighbouring scores (the "elbow").
let cutAt = hits.length;
let biggest = 0;
for (let i = 1; i < hits.length; i++) {
  const gap = hits[i - 1].score - hits[i].score;
  if (gap > biggest) {
    biggest = gap;
    cutAt = i;
  }
}
console.log(
  `largest gap ${biggest.toFixed(2)} after rank ${cutAt}:`,
  hits
    .slice(0, cutAt)
    .map((h) => h.id)
    .join(", "),
);

Expected output:

within 80% of best: brightgrid-plus#2, brightgrid-plus#1
largest gap 0.25 after rank 2: brightgrid-plus#2, brightgrid-plus#1

Search Execution Flow

Query Vectorization

The cleaned question is turned into a vector with the same embedding model used for the chunks (Chapter 3). In keyword search, the "vector" is the list of query terms instead.

Index Search & Distance Ranking

The index returns the nearest chunk ids with their scores, best first. With flat search that is exact. With HNSW or IVF it is approximate (Lesson 10).

Candidate Payload Extraction

Indexes usually return only ids and scores. The next step fetches each chunk's payload: its text and metadata (title, section, dates, access tags). Fetch only the fields you need: the text for the prompt, metadata for filtering and citations.

Index Lookup Latency Tracking

Each step takes time. Measure it, so you know where to optimize. In most small systems, embedding the question is the slowest part of retrieval, and generation is the slowest part overall.

Steps and their latency budget

// The search flow, step by step, with a latency budget for each step.
const steps = [
  { step: "1 clean the query", ms: 0.2 },
  { step: "2 embed the query", ms: 12 },
  { step: "3 search the index", ms: 4 },
  { step: "4 rank by distance", ms: 0.3 },
  { step: "5 fetch payloads (text, metadata)", ms: 3 },
];
const budget = 50;
let total = 0;
for (const s of steps) {
  total += s.ms;
  console.log(`${s.step.padEnd(36)} ${String(s.ms).padStart(5)} ms`);
}
console.log(
  `${"total".padEnd(36)} ${total.toFixed(1).padStart(5)} ms of a ${budget} ms budget`,
);
const slowest = steps.reduce((a, b) => (b.ms > a.ms ? b : a));
console.log("slowest step:", slowest.step);

Expected output:

1 clean the query                      0.2 ms
2 embed the query                       12 ms
3 search the index                       4 ms
4 rank by distance                     0.3 ms
5 fetch payloads (text, metadata)        3 ms
total                                 19.5 ms of a 50 ms budget
slowest step: 2 embed the query

Retrieval Quality Control

Removing Out-of-Distribution Matches

An out-of-distribution question is far from everything in your corpus: "write a poem", "what's the weather", a question in another language your corpus does not cover. Its best score will be much lower than usual. Compare the best score with what normal questions get, and treat a very low best score as "no relevant content".

Deduplicating Identical Vectors

Near-identical chunks produce near-identical vectors and scores, so they arrive together and fill your top-k with one fact. Remove duplicates by comparing normalized text (or a hash of it), or by checking whether two vectors are almost identical, with a cosine above about 0.98.

Remove duplicates, flag far-away questions

// Quality control: remove duplicate chunks, and flag questions far from everything.
const hits = [
  {
    id: "reset-password#1",
    text: "Press Forgot password and enter your email.",
    score: 0.84,
  },
  {
    id: "forgot-password#1",
    text: "Press Forgot password and enter your email.",
    score: 0.84,
  },
  { id: "two-step-sign-in#2", text: "Use one of your 10 backup codes.", score: 0.55 },
];

const seen = new Set();
const unique = hits.filter((h) => {
  const key = h.text.toLowerCase().replace(/\s+/g, " ").trim();
  if (seen.has(key)) {
    return false;
  }
  seen.add(key);
  return true;
});
console.log("after dedup:", unique.map((h) => h.id).join(", "));

// Out-of-distribution: the best score is far below what normal questions get.
const typicalBest = 0.7;
for (const [q, bestScore] of [
  ["How do I reset my password?", 0.84],
  ["Write me a poem about the sea", 0.18],
]) {
  console.log(
    `${q.padEnd(30)} best ${bestScore} -> ${bestScore < typicalBest / 2 ? "out of distribution: do not answer from these" : "normal"}`,
  );
}

Expected output:

after dedup: reset-password#1, two-step-sign-in#2
How do I reset my password?    best 0.84 -> normal
Write me a poem about the sea  best 0.18 -> out of distribution: do not answer from these

Handling Low-Confidence Query Returns

When the best result is weak but not useless, you have options:

• Answer carefully, and say that the sources may not fully cover the question.
• Ask the user to clarify: "Do you mean the Lumo Lamp or the Glow Strip?"
• Retry with a better query (below).

Fallback Retrieval Mechanisms

A fallback is a second plan when the first search fails:

• Switch search type: if vector search is weak, try keyword search, or the other way round.
• Widen the scope: include outer-volume sources such as tickets, or remove an optional filter.
• Corrective retrieval: grade the results, and if they are weak, rewrite or expand the query and search again. You will learn more ways to rewrite queries in Lesson 14.
• Hand over to a person when nothing works.

Retry with an expanded query

// Fallbacks and corrective retrieval: check the results, and try again if they are weak.
const index = {
  "wifi-troubleshooting#2":
    "move the lamp closer to the router and use a 2.4 ghz wi-fi network",
  "hub-pairing#2": "hold the button for 5 seconds to pair a light with the hub",
};
const synonyms = { internet: ["wi-fi", "network", "router"], light: ["lamp"] };

function search(q) {
  const words = q.toLowerCase().match(/[a-z0-9.-]+/g);
  return Object.entries(index)
    .map(([id, t]) => ({
      id,
      score: words.filter((w) => t.includes(w)).length / words.length,
    }))
    .sort((a, b) => b.score - a.score)[0];
}

function retrieve(q) {
  let best = search(q);
  console.log(`try 1 "${q}" -> ${best.id} (${best.score.toFixed(2)})`);
  if (best.score >= 0.4) {
    return best.id;
  }
  // Corrective step: the result looks weak, so expand the query and search again.
  const expanded =
    q +
    " " +
    q
      .split(" ")
      .flatMap((w) => synonyms[w] ?? [])
      .join(" ");
  best = search(expanded);
  console.log(`try 2 "${expanded}" -> ${best.id} (${best.score.toFixed(2)})`);
  return best.score >= 0.4 ? best.id : "no good match: hand over to a person";
}

console.log("result:", retrieve("my light has no internet"));

Expected output:

try 1 "my light has no internet" -> hub-pairing#2 (0.20)
try 2 "my light has no internet lamp wi-fi network router" -> wifi-troubleshooting#2 (0.44)
result: wifi-troubleshooting#2

Compare a plain search with one that expands the query, in the lab:

Common mistakes

• No cutoff, so unrelated chunks reach the model and get "used".
• A cutoff borrowed from another model, which removes everything or nothing.
• Fetching full documents when only the chunk text and a few fields are needed.
• No refusal path when every score is low.
• Duplicates that make one fact look like strong agreement between sources.

Exercises

Each exercise ends with a value on the last line. Press Check to compare it with the expected answer.

What to Remember

• Top-k search: vectorize the query → search the index → rank by distance → fetch payloads. Track the latency of each step.
• Top-k always returns k. Add a minimum score, chosen per model from your gold set.
• Dynamic thresholds (relative to the best score, or the largest gap) adapt to each question.
• Deduplicate near-identical chunks, and treat unusually low best scores as out of distribution.
• Plan fallbacks: switch search type, widen the scope, corrective retrieval with an expanded query, or hand over to a person. Limit retries.

Next: Hybrid Search, combining keyword and vector search for the best of both.