Context Refinement in Retrieval-Augmented Generation

Lessons in this section

1. Reranking

   Two-stage retrieval and rerankers: high-recall first stage, high-precision second stage, pool sizing and latency, MMR diversity, cross-encoders vs bi-encoders, joint attention, reranker models and score normalization.

2. Context Packing

   Packing retrieved chunks into the prompt: token budgets, removing overlaps, lost-in-the-middle ordering, source labels and ids, chronological vs relevance order, source enclosures, metadata and visible system instructions.