Retrieval Engine in Retrieval-Augmented Generation

Lessons in this section

1. Top-K Retrieval

   Turning a question into the right few chunks: score cutoffs, top-k selection, dynamic thresholds, the search flow and its latency, deduplication, out-of-distribution checks, fallbacks and corrective retrieval.

2. Hybrid Search

   Combining keyword and vector search: TF-IDF and BM25, dense search, normalizing and weighting scores, Reciprocal Rank Fusion with k=60, alpha calibration, parallel execution, acronym boosts and exact-match overrides.

3. Metadata Filtering

   Restricting search with metadata: pre-retrieval hard filters, scoping and partitioning, tenant isolation, post-filtering, missing metadata, conflict rules, soft penalties, AND/OR expressions, bitmaps and cardinality.

4. Query Transformation

   Improving questions before search: disambiguation and paraphrasing, conversational rewriting, query routing, sub-queries, HyDE, context-aware expansion, vague queries and multi-query fusion with RRF.