Back to Posts

RAG Optimization Techniques

February 9, 2026

RAG LLM Machine Learning Vector Search Information Retrieval

A comprehensive guide to optimizing Retrieval-Augmented Generation (RAG) systems for production use.

Core Techniques

🪓 Chunking Strategy

Getting chunk size right is critical. Too small = loss of context. Too large = noisy retrieval.

Key Principles

Chunking Options

Important Considerations

🎯 Query Rewriting

User queries are often incomplete, especially in multi-turn conversations.

The Problem

Example conversation:

User: When was the last time John Doe bought something from us?
AI: John bought a Fruity Fedora on January 3, 2030.
User: How about Emily Doe?

The last query will fail retrieval—it lacks context. It should be rewritten to "When was the last time Emily Doe bought something from us?"

Solution

Important Warning

🔁 Reranking

The two-stage retrieval paradigm: cheap retrieval → expensive reranking.

Why Reranking?

Especially useful when you want to:

Common Approach: Hybrid Retrieval System

  1. First stage: Fetch candidates with a cheap retriever
    • Use BM25 + vector search to cast a wide net
    • Term-based retrieval catches exact keyword matches
    • Embedding-based retrieval handles semantic similarity
  2. Second stage: Rerank with a better model
    • Use cross-encoder models for precision
    • More computationally expensive but much more accurate
    • Only applied to the smaller candidate set

Time-Based Reranking

Documents can also be reranked based on time, giving higher weight to more recent items.

This is critical for time-sensitive applications:

Benefits

📍 Contextual Retrieval

This one's a game-changer. Chunks often lack context needed for accurate retrieval.

The Problem

A chunk about "Q3 revenue increased 15%" is useless without knowing:

Anthropic's Solution

  1. For each chunk, use an LLM to generate a brief context (50-100 tokens) explaining the chunk's relationship to the overall document
  2. Prepend this context to the chunk before embedding
  3. Now the chunk becomes self-contained and retrievable

Example Transformation

Before: "Q3 revenue increased 15%"

After: "This chunk is from Acme Corp's 2024 annual report, discussing the cloud services division. Q3 revenue increased 15%"

Anthropic's Prompt Template

<document>
{{WHOLE_DOCUMENT}}
</document>

Here is the chunk we want to situate within the whole document:
<chunk>
{{CHUNK_CONTENT}}
</chunk>

Please give a short succinct context to situate this chunk within
the overall document for the purposes of improving search retrieval.
Answer only with the succinct context and nothing else.

Additional Augmentation Strategies

You can also augment chunks with:

The Process

Document Corpus → Split into chunks →
Generate context for each chunk →
Prepend context to chunk →
Embed augmented chunks →
Store in vector database

Benefits

Contextual Retrieval Process
Figure: Contextual Retrieval Pipeline - Document chunks are augmented with context before embedding.

Combining Contextual Retrieval with Reranking

Contextual retrieval can be combined with reranking to further improve retrieval precision. After initial candidates are retrieved with contextual retrieval, they are reranked with a better model to improve precision.

Contextual Retrieval with Reranking
Figure: Contextual Retrieval architecture with an added Reranking stage by Anthropic.

Advanced Techniques

🔍 Hybrid Search

What it is

Combining dense (vector-based) retrieval with sparse (keyword-based) retrieval. This is arguably the most impactful optimization missing from basic RAG implementations.

How it works

Step-by-step process

  1. Chunk the knowledge base: Break down documents into smaller chunks (typically a few hundred tokens each)
  2. Create dual encodings: Generate both TF-IDF encodings and semantic embeddings for each chunk
  3. BM25 retrieval: Use BM25 algorithm to find top chunks based on exact keyword matches
  4. Vector retrieval: Use embeddings to find top chunks based on semantic similarity
  5. Rank fusion: Combine and deduplicate results from both methods using rank fusion techniques (e.g., Reciprocal Rank Fusion)
  6. Context augmentation: Add the top-K chunks to the prompt to generate the final response

Why it's critical

Almost every production RAG system uses hybrid search because:

Implementation

# Pseudocode
vector_results = vector_search(query_embedding, top_k=20)
bm25_results = bm25_search(query_text, top_k=20)

# Merge and rerank
combined_results = merge_with_weights(
    vector_results,
    bm25_results,
    vector_weight=0.7,
    bm25_weight=0.3
)

🏷️ Metadata Filtering

What it is

Pre-filtering documents by metadata attributes before performing vector search to narrow the search space.

How it works

Instead of searching your entire vector database, first filter by metadata:

Then perform vector search only on the filtered subset.

Example

Query: "What's our refund policy for enterprise customers?"

Without metadata filtering:

With metadata filtering:

Benefits

When to use

🌳 Parent-Child / Hierarchical Retrieval

What it is

Embed small chunks for precision, but return larger parent chunks to the LLM for better context.

The Problem

The Solution

  1. Create a hierarchy: large parent chunks and smaller child chunks
  2. Embed and index the small child chunks for retrieval
  3. When a child chunk is retrieved, return the parent chunk to the LLM

Example Structure

Document: "Product Manual"
├── Parent Chunk 1: "Installation Guide" (2000 tokens)
│   ├── Child Chunk 1.1: "Prerequisites" (200 tokens) ← embed this
│   ├── Child Chunk 1.2: "Step-by-step instructions" (200 tokens) ← embed this
│   └── Child Chunk 1.3: "Troubleshooting" (200 tokens) ← embed this
└── Parent Chunk 2: "Configuration Guide" (2000 tokens)
    ├── Child Chunk 2.1: "Basic settings" (200 tokens) ← embed this
    └── Child Chunk 2.2: "Advanced settings" (200 tokens) ← embed this

Retrieval Flow

  1. User query: "How do I fix installation errors?"
  2. Vector search finds Child Chunk 1.3: "Troubleshooting" (high precision)
  3. System returns Parent Chunk 1: "Installation Guide" to the LLM (full context)
  4. LLM has both the relevant section AND surrounding context

Benefits

🎭 HyDE (Hypothetical Document Embeddings)

What it is

Generate a hypothetical answer first, then embed that for retrieval instead of the original query. This bridges the query-document semantic gap.

The Problem

Questions and answers use different vocabulary and structure:

These have different semantic representations despite being related.

How HyDE works

  1. User submits query: "How do I fix CUDA out of memory errors?"
  2. Ask LLM to generate a hypothetical answer (even if it hallucinates): 
    "To fix CUDA OOM errors, you should reduce batch size,
use gradient accumulation, enable mixed precision training
with torch.cuda.amp, clear cache with torch.cuda.empty_cache()..."
  3. Embed this hypothetical answer
  4. Search using the hypothetical answer's embedding
  5. The real documentation is semantically closer to this "answer" than to the original question

Why it works

When to use

Caution

🔀 Multi-Query Retrieval

What it is

Generate multiple variations of the user's query and retrieve documents for each variation, then merge the results.

How it works

  1. Original query: "Best practices for API security"
  2. Generate variations:
    • "How to secure REST APIs"
    • "API authentication and authorization methods"
    • "Preventing API vulnerabilities"
    • "API security design patterns"
  3. Retrieve documents for each variation
  4. Merge and deduplicate results
  5. Optionally rerank the merged set

Difference from Query Rewriting

Benefits

Implementation Strategies

Trade-offs

✅ Self-RAG / Corrective RAG

What it is

The system evaluates whether retrieved documents are actually relevant to the query and can take corrective actions (re-retrieve, try different strategy, or fall back gracefully).

The Problem

Traditional RAG blindly trusts retrieved documents:

How Self-RAG works

1. Retrieve documents
2. LLM evaluates each document:
   - Is this relevant to the query?
   - Does this support or contradict other documents?
   - Is this information sufficient to answer?
3. Decision branches:
   a. All docs relevant → Proceed to generate answer
   b. Partially relevant → Retrieve more documents
   c. Not relevant → Try different retrieval strategy or web search
   d. No docs available → Answer from parametric knowledge or admit uncertainty

Example Flow

Query: "Latest features in Python 3.12"

Step 1: Initial retrieval returns docs about Python 3.10
Step 2: Self-RAG detects version mismatch
        "Query asks for 3.12, but retrieved docs are about 3.10"
Step 3: Corrective action: Trigger web search for current information
Step 4: Verify new results match query requirements
Step 5: Generate answer with appropriate sources

Corrective RAG Variant

Focuses specifically on correction strategies:

Implementation Considerations

Benefits

When to implement

🕸️ Graph RAG

What it is

Use knowledge graphs to capture entity relationships that vector search misses. Instead of treating documents as isolated chunks, build a graph of interconnected entities and relationships.

The Problem with Vector Search

Vector search finds semantically similar text, but it misses explicit relationships:

How Graph RAG works

  1. Entity Extraction: Extract entities (people, places, organizations, concepts) from documents
  2. Relationship Mapping: Identify relationships between entities
  3. Graph Construction: Build knowledge graph with entities as nodes and relationships as edges
  4. Hybrid Retrieval:
    • Use vector search for semantic similarity
    • Use graph traversal for relationship queries
    • Combine both for comprehensive retrieval

Example Graph Structure

[John] --works_on--> [Project X] --failed_in--> [2023]
[Project X] --belongs_to--> [Cloud Division]
[Cloud Division] --part_of--> [Acme Corp]

Query Examples

"What projects did John work on that failed?"

Benefits

Use Cases

Implementation

Challenges

When to use

🤖 Agentic RAG

What it is

Treat RAG as a multi-step decision-making process where an LLM agent decides how to retrieve information, rather than using a single fixed retrieval pipeline.

The Problem

Different queries need different retrieval strategies:

One-size-fits-all retrieval doesn't work for diverse queries.

How Agentic RAG works

  1. Query Analysis: Agent analyzes the user's query to understand intent
  2. Strategy Selection: Agent decides which tools/retrievers to use
  3. Multi-Step Retrieval: Agent can chain multiple retrievals
  4. Self-Evaluation: Agent evaluates if retrieved information is sufficient
  5. Adaptive Refinement: Agent can try different approaches if first attempt fails

Example Flow

Query: "Compare our Q3 revenue vs competitors in the cloud market"

Agent reasoning:
1. "Need internal Q3 revenue" → Query SQL database
2. "Need competitor revenue" → Web search
3. "Need cloud market context" → Vector search internal reports
4. Combine all three sources → Generate comparative analysis

Common Patterns

ReAct (Reasoning + Acting):

Thought: I need to find the user's order history
Action: query_database(user_id=123, table="orders")
Observation: Found 15 orders
Thought: User asked about recent orders, I should filter
Action: filter_orders(timeframe="last_30_days")
Observation: 3 orders in last 30 days
Thought: Now I can answer
Answer: You placed 3 orders in the last 30 days...

Tool Selection:

Implementation

Often built with frameworks like:

Benefits

Challenges

When to use

🎯 Fine-Tuning Embedding Models

What it is

Train domain-specific embedding models on your specialized corpus instead of using general-purpose embeddings.

The Problem

General-purpose embedding models (OpenAI, Cohere, etc.) are trained on broad internet data. They might not capture nuances in your specific domain:

How it works

  1. Collect training data:
    • Positive pairs: (query, relevant document)
    • Negative pairs: (query, irrelevant document)
    • Can generate synthetically using LLMs
  2. Fine-tune embedding model:
    • Start with pre-trained model (e.g., sentence-transformers)
    • Continue training on domain-specific pairs
    • Optimize for your retrieval task
  3. Deploy fine-tuned model:
    • Use for encoding both queries and documents
    • Store embeddings in vector database

Benefits

Trade-offs

When to use

Data Generation Strategy

Use LLMs to generate synthetic training data:

For each document chunk:
1. Generate 3-5 queries that this chunk should answer
2. Generate 3-5 queries that this chunk should NOT answer
3. Use as positive and negative training pairs

Quick Reference

Start Here (Highest ROI)

  1. Hybrid Search (dense + sparse retrieval)
  2. Contextual Retrieval (augment chunks with context)
  3. Reranking (two-stage retrieval)

Next Level

  1. Query Rewriting (handle multi-turn conversations)
  2. Metadata Filtering (narrow search space)
  3. Parent-Child Retrieval (precision + context)

Advanced (When Needed)

  1. HyDE (bridge query-document gap)
  2. Multi-Query Retrieval (comprehensive coverage)
  3. Self-RAG / Corrective RAG (robust production systems)
  4. Agentic RAG (complex multi-source queries)
  5. Graph RAG (relationship-heavy domains)
  6. Fine-tuned Embeddings (specialized domains)

Measuring Success

Always measure retrieval quality before optimizing generation:

Key Metrics

Remember: Bad inputs = bad outputs, no matter how good your LLM is.

Resources

Citation

If you found this guide helpful and would like to cite it:

Cited as:

Haseeb, Raja. (Feb 2026). "RAG Optimization Techniques". Personal Blog.

https://pytholic.github.io/posts/rag-optimization-techniques/

Or in BibTeX format:

@article{pytholic2026ragoptim,
    title   = "RAG Optimization Techniques",
    author  = "Haseeb, Raja",
    journal = "pytholic.github.io",
    year    = "2026",
    month   = "Feb",
    url     = "https://pytholic.github.io/posts/rag-optimization-techniques/"
}