Hybrid RAG pipelines combine keyword and semantic search — but merging their results without breaking both is harder than it sounds. This lesson teaches you Reciprocal Rank Fusion from the formula up through a complete, production-ready Python implementation, so you can build retrievers that outperform either approach alone.
You've built a RAG pipeline, tuned your embeddings, and you're getting decent results — but you keep noticing the same frustrating pattern. Queries that use exact product codes or proper nouns come back with semantically similar but wrong documents. Meanwhile, queries phrased in natural language sometimes miss highly relevant documents that don't happen to share many words with the query. You're caught between two retrieval paradigms that each have genuine strengths, and picking one means giving up the other.
This is the core problem that hybrid retrieval solves — and Reciprocal Rank Fusion (RRF) is the algorithm that makes hybrid retrieval actually work. Instead of choosing between BM25 keyword search and dense vector search, you run both and merge the results into a single, more reliable ranked list. RRF is elegant in its simplicity: it doesn't require you to tune score weights or normalize across incompatible score distributions. It just looks at where each document appeared in each ranked list and computes a single score from those positions. The result is consistently better than either retrieval method alone, which is why RRF shows up in production RAG systems at companies ranging from early-stage startups to enterprises running billions of documents.
By the end of this lesson, you'll be able to implement RRF from scratch, integrate it into a hybrid RAG pipeline, understand its failure modes, and make informed decisions about when to use it versus alternatives like learned sparse retrieval or weighted score fusion.
What you'll learn:
k mattersYou should be comfortable with:
Before we look at RRF, it's worth being precise about what you lose when you pick only one retrieval strategy.
Dense vector search converts both your query and documents into high-dimensional embeddings, then finds documents whose embeddings are geometrically close to the query embedding. It handles paraphrase and semantic drift beautifully — "cardiovascular event" and "heart attack" land near each other in embedding space. But it struggles with exact-match scenarios. If a user queries CVE-2024-3094, the embedding model doesn't have meaningful geometry to work with. The document containing that exact string might rank tenth behind documents that discuss software vulnerabilities in general.
BM25 keyword search rewards documents that contain the same terms as the query, weighted by how rare those terms are across the corpus. It's extremely reliable for exact lookups, product codes, named entities, and technical identifiers. Its failure mode is inverse: it's blind to meaning. A document about "myocardial infarction" scores zero against the query "heart attack" if those exact words don't appear together.
These failure modes are nearly complementary. That's not an accident — they're different approaches to the same retrieval problem, and their errors tend not to correlate. When you combine them intelligently, you get a retriever that handles both semantic similarity and lexical precision without sacrificing either.
The naive approach to combining them is to normalize the scores from each system and add them together. The problem is that BM25 scores and cosine similarity scores live in completely different distributions. A BM25 score of 12.4 against a cosine similarity of 0.87 doesn't mean anything coherent. You'd have to run experiments to calibrate the right weighting factor — and that factor would change with different corpora, different embedding models, and different query distributions. It's fragile in exactly the conditions where you need it to be robust.
RRF sidesteps all of this by ignoring raw scores entirely.
Reciprocal Rank Fusion was introduced by Cormack, Clarke, and Buettcher in a 2009 SIGIR paper. The formula is almost offensively simple:
RRF_score(d) = Σ 1 / (k + rank_i(d))
i∈retrievers
Where:
d is a documentrank_i(d) is the rank (position) of document d in the results from retriever ik is a constant, typically set to 60dThat's it. You find where each document appeared in each ranked list, compute the reciprocal of its rank plus the constant, and sum those values. Documents that appear at the top of multiple lists get the highest scores. Documents that appear in only one list, or only appear far down the ranking, get lower scores.
Let's trace through a concrete example. Suppose you have a technical documentation corpus and the query is "configure TLS certificate renewal for nginx." You run BM25 and a dense retriever and get these top-5 results:
BM25 results:
nginx-ssl-config.mdlets-encrypt-setup.mdnginx-performance-tuning.mdssl-cert-renewal-cron.mdhaproxy-tls-guide.mdDense vector results:
lets-encrypt-setup.mdssl-cert-renewal-cron.mdnginx-ssl-config.mdcertbot-automation.mdopenssl-reference.mdNow we compute RRF scores with k=60:
nginx-ssl-config.md: 1/(60+1) + 1/(60+3) = 0.01639 + 0.01587 = 0.03226
lets-encrypt-setup.md: 1/(60+2) + 1/(60+1) = 0.01613 + 0.01639 = 0.03252
ssl-cert-renewal-cron.md: 1/(60+4) + 1/(60+2) = 0.01563 + 0.01613 = 0.03175
certbot-automation.md: 0 + 1/(60+4) = 0 + 0.01563 = 0.01563
nginx-performance-tuning.md: 1/(60+3) + 0 = 0.01587 + 0 = 0.01587
The final ranking: lets-encrypt-setup.md (0.03252) → nginx-ssl-config.md (0.03226) → ssl-cert-renewal-cron.md (0.03175) → nginx-performance-tuning.md (0.01587) → certbot-automation.md (0.01563)
Notice what happened. lets-encrypt-setup.md was ranked #2 by BM25 and #1 by dense retrieval — it wins because both retrievers agree it's highly relevant. certbot-automation.md only appeared in one list, so despite being ranked 4th there, it ends up near the bottom of the fused list. nginx-performance-tuning.md appeared in BM25 but not in the dense results at all — suggesting it matched some keywords but wasn't semantically central, and RRF appropriately pushes it down.
The k=60 value comes from the original paper and has been validated empirically across many datasets. Its role is to dampen the advantage of very top-ranked documents. Without it (k=0), rank 1 would score 1.0 and rank 2 would score 0.5 — a 50% drop just from one position. With k=60, the drop from rank 1 to rank 2 is from 1/61 to 1/62, which is less than 2%.
This matters a lot in practice. If k is too small, you over-reward documents that happen to be ranked #1 in any single retriever, which makes RRF just as sensitive to the quirks of individual retrievers as those retrievers themselves. If k is too large, every document gets nearly the same score and the ranking becomes noise. The range k=10 to k=100 is reasonable; k=60 is a solid default you can use without tuning unless you have good reasons and evaluation data to support changing it.
Tip: In most production applications, you'll get 90% of the benefit from RRF with k=60 without any tuning. Reserve hyperparameter optimization for after you've built proper offline evaluation infrastructure — don't tune k based on vibes.
Let's build a clean, reusable implementation before we integrate it into a full pipeline.
from collections import defaultdict
from typing import Any
def reciprocal_rank_fusion(
ranked_lists: list[list[str]],
k: int = 60
) -> list[tuple[str, float]]:
"""
Merge multiple ranked lists into a single ranked list using RRF.
Args:
ranked_lists: A list of ranked lists, where each inner list
contains document IDs in ranked order (best first).
k: The rank constant. Default 60 follows the original paper.
Returns:
A list of (doc_id, rrf_score) tuples, sorted by score descending.
"""
scores: dict[str, float] = defaultdict(float)
for ranked_list in ranked_lists:
for rank, doc_id in enumerate(ranked_list, start=1):
scores[doc_id] += 1.0 / (k + rank)
return sorted(scores.items(), key=lambda x: x[1], reverse=True)
# Example usage
bm25_results = [
"nginx-ssl-config.md",
"lets-encrypt-setup.md",
"nginx-performance-tuning.md",
"ssl-cert-renewal-cron.md",
"haproxy-tls-guide.md",
]
dense_results = [
"lets-encrypt-setup.md",
"ssl-cert-renewal-cron.md",
"nginx-ssl-config.md",
"certbot-automation.md",
"openssl-reference.md",
]
fused = reciprocal_rank_fusion([bm25_results, dense_results])
for doc_id, score in fused:
print(f"{score:.5f} {doc_id}")
Output:
0.03252 lets-encrypt-setup.md
0.03226 nginx-ssl-config.md
0.03175 ssl-cert-renewal-cron.md
0.01587 nginx-performance-tuning.md
0.01563 certbot-automation.md
0.01639 haproxy-tls-guide.md
0.01563 openssl-reference.md
This implementation is intentionally document-ID-based. In real pipelines, you'll usually work with more complex objects — chunks with metadata, scores, source information. Here's a more production-ready version that carries the document payloads through the fusion process:
from collections import defaultdict
from dataclasses import dataclass, field
from typing import Any
@dataclass
class RetrievedChunk:
doc_id: str
content: str
metadata: dict[str, Any] = field(default_factory=dict)
original_scores: dict[str, float] = field(default_factory=dict) # retriever -> score
def rrf_merge(
retriever_results: dict[str, list[RetrievedChunk]],
k: int = 60,
top_n: int = 10
) -> list[tuple[RetrievedChunk, float]]:
"""
Merge results from multiple named retrievers using RRF.
Args:
retriever_results: Dict mapping retriever name to its ranked results.
k: Rank constant.
top_n: Number of results to return.
Returns:
Top-N (chunk, rrf_score) tuples sorted by RRF score descending.
"""
rrf_scores: dict[str, float] = defaultdict(float)
chunk_registry: dict[str, RetrievedChunk] = {}
for retriever_name, chunks in retriever_results.items():
for rank, chunk in enumerate(chunks, start=1):
rrf_scores[chunk.doc_id] += 1.0 / (k + rank)
# First time we see a doc_id, register it
if chunk.doc_id not in chunk_registry:
chunk_registry[chunk.doc_id] = chunk
# Track original scores from each retriever for debugging
chunk_registry[chunk.doc_id].original_scores[retriever_name] = \
chunk.original_scores.get(retriever_name, 0.0)
ranked = sorted(rrf_scores.items(), key=lambda x: x[1], reverse=True)
return [(chunk_registry[doc_id], score) for doc_id, score in ranked[:top_n]]
Warning: Notice we're using
doc_idto deduplicate across retrievers. If your BM25 index and your vector store use different identifiers for the same document chunks, RRF will treat them as separate documents and you'll get worse results than either retriever alone. Consistent chunk IDs across all retrieval systems is a prerequisite, not an afterthought.
Now let's put it together into a complete, working pipeline. We'll use Elasticsearch for BM25 (it has a built-in BM25 implementation) and FAISS for dense retrieval, with sentence-transformers for embeddings. This mirrors a common production setup.
First, install dependencies:
pip install elasticsearch faiss-cpu sentence-transformers langchain langchain-community openai python-dotenv
import os
import hashlib
import json
from elasticsearch import Elasticsearch
from sentence_transformers import SentenceTransformer
import faiss
import numpy as np
from dotenv import load_dotenv
load_dotenv()
# --- Configuration ---
EMBEDDING_MODEL = "sentence-transformers/all-mpnet-base-v2"
EMBEDDING_DIM = 768
ES_INDEX = "docs_bm25"
FAISS_INDEX_PATH = "docs_faiss.index"
CHUNK_METADATA_PATH = "chunk_metadata.json"
RRF_K = 60
TOP_K_PER_RETRIEVER = 20 # Fetch more than you need before fusion
TOP_N_AFTER_FUSION = 5 # Final context window size
# --- Initialize clients ---
es = Elasticsearch("http://localhost:9200")
embedder = SentenceTransformer(EMBEDDING_MODEL)
def chunk_id(content: str, source: str) -> str:
"""Deterministic chunk ID — same content always gets the same ID."""
return hashlib.sha256(f"{source}::{content}".encode()).hexdigest()[:16]
def index_documents(documents: list[dict]) -> tuple[faiss.Index, dict]:
"""
Index a list of documents into both Elasticsearch (BM25) and FAISS (dense).
Each document should have: {"content": str, "source": str, "metadata": dict}
Returns the FAISS index and a metadata registry mapping FAISS position -> chunk info.
"""
faiss_index = faiss.IndexFlatIP(EMBEDDING_DIM) # Inner product = cosine if normalized
chunk_registry = {} # Maps FAISS position (int) -> chunk metadata
# Create ES index if needed
if not es.indices.exists(index=ES_INDEX):
es.indices.create(index=ES_INDEX, body={
"mappings": {
"properties": {
"chunk_id": {"type": "keyword"},
"content": {"type": "text", "analyzer": "english"},
"source": {"type": "keyword"},
"metadata": {"type": "object"}
}
}
})
texts = [doc["content"] for doc in documents]
embeddings = embedder.encode(texts, normalize_embeddings=True, show_progress_bar=True)
for i, (doc, embedding) in enumerate(zip(documents, embeddings)):
cid = chunk_id(doc["content"], doc["source"])
# Index in Elasticsearch
es.index(index=ES_INDEX, id=cid, document={
"chunk_id": cid,
"content": doc["content"],
"source": doc["source"],
"metadata": doc.get("metadata", {})
})
# Index in FAISS — store the FAISS position -> chunk_id mapping
faiss_position = faiss_index.ntotal
faiss_index.add(embedding.reshape(1, -1))
chunk_registry[faiss_position] = {
"chunk_id": cid,
"content": doc["content"],
"source": doc["source"],
"metadata": doc.get("metadata", {})
}
# Save artifacts
faiss.write_index(faiss_index, FAISS_INDEX_PATH)
with open(CHUNK_METADATA_PATH, "w") as f:
json.dump(chunk_registry, f)
print(f"Indexed {len(documents)} chunks into ES and FAISS.")
return faiss_index, chunk_registry
def bm25_retrieve(query: str, top_k: int = 20) -> list[dict]:
"""Run BM25 retrieval against Elasticsearch."""
response = es.search(index=ES_INDEX, body={
"query": {"match": {"content": {"query": query, "operator": "or"}}},
"size": top_k
})
results = []
for hit in response["hits"]["hits"]:
results.append({
"chunk_id": hit["_source"]["chunk_id"],
"content": hit["_source"]["content"],
"source": hit["_source"]["source"],
"metadata": hit["_source"].get("metadata", {}),
"bm25_score": hit["_score"]
})
return results
def dense_retrieve(
query: str,
faiss_index: faiss.Index,
chunk_registry: dict,
top_k: int = 20
) -> list[dict]:
"""Run dense retrieval against FAISS."""
query_embedding = embedder.encode([query], normalize_embeddings=True)
scores, indices = faiss_index.search(query_embedding, top_k)
results = []
for score, idx in zip(scores[0], indices[0]):
if idx == -1: # FAISS returns -1 for unfilled slots
continue
chunk_info = chunk_registry[str(idx)] # JSON keys are strings
results.append({
"chunk_id": chunk_info["chunk_id"],
"content": chunk_info["content"],
"source": chunk_info["source"],
"metadata": chunk_info.get("metadata", {}),
"dense_score": float(score)
})
return results
def hybrid_retrieve(
query: str,
faiss_index: faiss.Index,
chunk_registry: dict,
top_k_per_retriever: int = 20,
top_n_final: int = 5,
k: int = 60
) -> list[dict]:
"""
Run BM25 and dense retrieval, merge with RRF, return top-N chunks.
"""
bm25_results = bm25_retrieve(query, top_k=top_k_per_retriever)
dense_results = dense_retrieve(query, faiss_index, chunk_registry, top_k=top_k_per_retriever)
# Build ranked lists (just chunk_ids, in order)
bm25_ranked = [r["chunk_id"] for r in bm25_results]
dense_ranked = [r["chunk_id"] for r in dense_results]
# Compute RRF scores
from collections import defaultdict
rrf_scores: dict[str, float] = defaultdict(float)
for rank, chunk_id in enumerate(bm25_ranked, start=1):
rrf_scores[chunk_id] += 1.0 / (k + rank)
for rank, chunk_id in enumerate(dense_ranked, start=1):
rrf_scores[chunk_id] += 1.0 / (k + rank)
# Build a registry of chunk content from both result sets
all_chunks = {r["chunk_id"]: r for r in bm25_results}
all_chunks.update({r["chunk_id"]: r for r in dense_results})
# Sort by RRF score and return top-N
ranked = sorted(rrf_scores.items(), key=lambda x: x[1], reverse=True)
final_results = []
for chunk_id, rrf_score in ranked[:top_n_final]:
chunk = all_chunks[chunk_id].copy()
chunk["rrf_score"] = rrf_score
final_results.append(chunk)
return final_results
from openai import OpenAI
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))
def build_context(retrieved_chunks: list[dict]) -> str:
"""Format retrieved chunks into a context string for the LLM."""
context_parts = []
for i, chunk in enumerate(retrieved_chunks, start=1):
source = chunk.get("source", "unknown")
content = chunk["content"]
context_parts.append(f"[Source {i}: {source}]\n{content}")
return "\n\n---\n\n".join(context_parts)
def rag_answer(
query: str,
faiss_index: faiss.Index,
chunk_registry: dict,
top_n: int = 5
) -> dict:
"""End-to-end RAG answer with hybrid retrieval."""
retrieved = hybrid_retrieve(
query=query,
faiss_index=faiss_index,
chunk_registry=chunk_registry,
top_n_final=top_n
)
context = build_context(retrieved)
system_prompt = (
"You are a technical assistant. Answer the user's question using only "
"the provided context. If the context doesn't contain enough information "
"to answer confidently, say so clearly."
)
user_message = f"""Context:
{context}
Question: {query}
Answer:"""
response = client.chat.completions.create(
model="gpt-4o",
messages=[
{"role": "system", "content": system_prompt},
{"role": "user", "content": user_message}
],
temperature=0.2
)
return {
"answer": response.choices[0].message.content,
"retrieved_chunks": retrieved,
"query": query
}
If you're already on Elasticsearch 8.8+, you don't need to implement RRF yourself. Elasticsearch has native RRF support in its hybrid search feature, which means the fusion happens at the search tier, not in your application layer. This has meaningful performance implications when you're dealing with large corpora.
def es_native_hybrid_search(
query: str,
query_vector: list[float],
index: str,
top_n: int = 5,
k: int = 60
) -> list[dict]:
"""
Use Elasticsearch's built-in RRF hybrid search.
Requires ES 8.8+ and a knn-enabled mapping.
"""
response = es.search(index=index, body={
"retriever": {
"rrf": {
"retrievers": [
{
"standard": {
"query": {
"match": {
"content": {"query": query}
}
}
}
},
{
"knn": {
"field": "embedding",
"query_vector": query_vector,
"num_candidates": 100
}
}
],
"rank_constant": k,
"rank_window_size": 100
}
},
"size": top_n
})
return [
{
"chunk_id": hit["_source"]["chunk_id"],
"content": hit["_source"]["content"],
"source": hit["_source"]["source"],
"rrf_score": hit["_score"]
}
for hit in response["hits"]["hits"]
]
This is cleaner and more efficient than the Python-layer approach, and it handles edge cases like partial result sets more gracefully. If your stack includes ES 8.8+, use this over rolling your own.
Tip: Weaviate, Qdrant, and OpenSearch also have hybrid search with RRF built in. Check your vector store's documentation before implementing fusion at the application layer — you may already have it available.
One underused feature of RRF is that it generalizes trivially to any number of ranked lists. In a complex production pipeline, you might run:
All four can be fused with RRF by simply summing over more lists. The formula doesn't change:
def multi_retriever_rrf(
retriever_outputs: dict[str, list[str]],
k: int = 60,
top_n: int = 10
) -> list[tuple[str, float]]:
"""
Generalized RRF for N retrievers.
Args:
retriever_outputs: Dict of retriever_name -> [doc_id, ...] ranked list
k: Rank constant
top_n: How many results to return
Returns:
List of (doc_id, rrf_score) sorted descending by score
"""
from collections import defaultdict
scores: dict[str, float] = defaultdict(float)
for retriever_name, ranked_list in retriever_outputs.items():
for rank, doc_id in enumerate(ranked_list, start=1):
scores[doc_id] += 1.0 / (k + rank)
ranked = sorted(scores.items(), key=lambda x: x[1], reverse=True)
return ranked[:top_n]
# Three-retriever example: BM25 + document-level dense + sentence-level dense
result = multi_retriever_rrf({
"bm25": ["doc-A", "doc-B", "doc-C", "doc-D"],
"dense_doc": ["doc-C", "doc-A", "doc-E", "doc-B"],
"dense_sentence": ["doc-A", "doc-C", "doc-F", "doc-E"]
})
for doc_id, score in result:
print(f"{score:.5f} {doc_id}")
Output:
0.04878 doc-A # Top of all three lists
0.04748 doc-C # Top in two lists
0.03191 doc-B # Appears in two lists at mid-rank
0.03145 doc-E # Appears in two lists at lower ranks
0.01587 doc-D # Only in BM25
0.01563 doc-F # Only in sentence-dense
doc-A is ranked #1 in two lists and #4 in none — it clearly wins. doc-C is #1 in dense-doc and #2 in sentence-dense but #3 in BM25, and it still comes in second. This is the behavior you want: consensus across retrievers is rewarded, outlier rankings are dampened.
Here's a realistic exercise you can run end-to-end. You'll build a hybrid RAG bot over a small technical documentation corpus and compare the retrieval quality of BM25-only, dense-only, and RRF-fused results.
The dataset: Use Python's built-in help() strings for stdlib modules — they're machine-readable, varied, and you definitely have them available.
import pydoc
import textwrap
# Build a small corpus from Python stdlib docs
MODULES = [
"os", "pathlib", "json", "csv", "datetime", "collections",
"itertools", "functools", "re", "urllib.request", "http.client",
"socket", "threading", "multiprocessing", "subprocess"
]
def get_module_docs(module_name: str) -> list[dict]:
"""Extract and chunk documentation for a Python module."""
try:
text = pydoc.render_doc(module_name, renderer=pydoc.plaintext)
except Exception:
return []
# Simple paragraph-based chunking
paragraphs = [p.strip() for p in text.split("\n\n") if len(p.strip()) > 100]
chunks = []
for i, para in enumerate(paragraphs):
chunks.append({
"content": textwrap.fill(para, width=500),
"source": f"python-stdlib/{module_name}",
"metadata": {"module": module_name, "chunk_index": i}
})
return chunks
# Generate the corpus
corpus = []
for module in MODULES:
corpus.extend(get_module_docs(module))
print(f"Generated {len(corpus)} chunks from {len(MODULES)} modules")
# Index the corpus
faiss_index, chunk_registry = index_documents(corpus)
# Test queries — designed to stress different retrieval modes
test_queries = [
"ThreadPoolExecutor concurrent futures", # Dense-friendly (paraphrase needed)
"os.path.join", # BM25-friendly (exact identifier)
"how to read a CSV file with headers", # Dense-friendly (natural language)
"json.JSONDecodeError exception handling", # Both (mixed)
"namedtuple _fields attribute", # BM25-friendly (exact API)
]
for query in test_queries:
print(f"\n{'='*60}")
print(f"Query: {query}")
print(f"{'='*60}")
bm25_top3 = [r["source"] for r in bm25_retrieve(query, top_k=3)]
dense_top3 = [r["source"] for r in dense_retrieve(query, faiss_index, chunk_registry, top_k=3)]
fused_top3 = [r["source"] for r in hybrid_retrieve(query, faiss_index, chunk_registry, top_n_final=3)]
print(f"BM25: {bm25_top3}")
print(f"Dense: {dense_top3}")
print(f"Fused: {fused_top3}")
What to look for: For the exact-identifier queries (os.path.join, namedtuple _fields), BM25 should return the right module at rank 1. Dense may or may not. For the natural language queries, Dense should win. For the mixed queries, watch how RRF handles the disagreement. If RRF consistently returns better or equal results across all five queries, you've validated the value of fusion on your specific corpus.
This is the most common production failure. If your BM25 index uses Elasticsearch document IDs (like "abc123") and your vector store uses its own internal IDs (like 1024), RRF sees every document as unique and just concatenates both ranked lists. Your "fusion" does nothing — you're returning 40 results and calling it 20.
Fix: Generate chunk IDs deterministically from content at indexing time (like the chunk_id() function above). Store this ID in both indexes. Use it as the deduplication key in your RRF implementation.
If you only ask each retriever for 5 results and then fuse them, you might miss highly relevant documents entirely — they might rank 8th in BM25 but 2nd in dense. RRF can only work with what you give it.
Fix: Fetch significantly more results than you'll ultimately use. A ratio of 4:1 is reasonable — if you want 5 chunks for context, fetch 20 from each retriever before fusing. The math in the formula accounts for documents that appear in only one list.
An RRF score of 0.032 doesn't mean a document is 96.8% irrelevant. RRF scores are ordinal, not probabilistic. They tell you relative ranking, not absolute relevance. Don't use them to filter results with a threshold like if rrf_score > 0.02: include.
Fix: Always use RRF for ranking, then take top-N by count. If you need a confidence signal, consider adding a cross-encoder re-ranker on top of the RRF output.
When a document only appears in BM25 but not in the dense results, it should get a lower score — not an error. Make sure your implementation handles the partial case correctly. The formula naturally handles this: documents in only one list only accumulate score from that one list.
Debug check: After fusion, log how many documents came exclusively from BM25, exclusively from dense, and from both. If you're seeing very few documents in both lists, your retrievers might be disagreeing more than expected — worth investigating your chunking strategy or embedding model.
def fusion_audit(bm25_ids: list[str], dense_ids: list[str]) -> dict:
"""Audit the overlap between retriever results."""
bm25_set = set(bm25_ids)
dense_set = set(dense_ids)
overlap = bm25_set & dense_set
bm25_only = bm25_set - dense_set
dense_only = dense_set - bm25_set
return {
"total_unique": len(bm25_set | dense_set),
"overlap_count": len(overlap),
"overlap_pct": len(overlap) / len(bm25_set | dense_set) * 100,
"bm25_only_count": len(bm25_only),
"dense_only_count": len(dense_only),
"overlap_ids": list(overlap)
}
If overlap_pct is consistently above 60-70%, your retrievers are largely agreeing and RRF adds less marginal value. If it's below 20%, they're retrieving almost entirely different document sets — that can be fine (it means high complementarity) but warrants investigation.
If you're using faiss.IndexFlatIP for cosine similarity (inner product on normalized vectors), but you're not normalizing your query embeddings at query time, you'll get subtly wrong rankings. The index embeddings might be normalized, but the query vector won't be.
# Wrong
query_embedding = embedder.encode([query])
# Right
query_embedding = embedder.encode([query], normalize_embeddings=True)
This won't cause an obvious error — it'll just silently degrade your dense retrieval quality.
RRF isn't always the right tool. Here's a framework for deciding:
Use RRF when:
Consider weighted score fusion when:
Consider learned sparse retrieval (SPLADE, etc.) instead of BM25 when:
Consider a cross-encoder re-ranker on top of RRF when:
RRF + cross-encoder re-ranking is one of the most effective general-purpose stacks for production RAG. RRF casts a wide, reliable net; the re-ranker tightens the precision at the top.
Reciprocal Rank Fusion is one of those rare techniques that earns its place in production systems not because it's clever, but because it's correct. It solves a real problem — merging incompatible ranked lists — with a formula that's transparent, parameter-free in practice, and empirically validated. By treating rank position as the common currency across all retrievers, it sidesteps the normalization mess that plagues score-based fusion.
Here's what you can do with what you've learned today:
fusion_audit() function to understand how complementary your retrievers actually arek=60 and leave it there until you have proper offline evaluation infrastructure to justify changing itNext steps to deepen your knowledge:
Build an offline evaluation harness — create a test set of queries with ground-truth relevant documents and measure MRR@5 and NDCG@10 for BM25-only, dense-only, and RRF-fused. The numbers will make the benefit concrete and tell you whether to invest in further tuning.
Add a cross-encoder re-ranker — after RRF returns your top-10 or top-20 results, run them through a cross-encoder/ms-marco-MiniLM-L-6-v2 model (available on HuggingFace) to get a final precision-optimized top-5.
Explore SPLADE — if your BM25 performance is limited by vocabulary mismatch in a domain-specific corpus, SPLADE is a learned sparse retriever that expands queries and documents semantically while maintaining sparse vector efficiency.
Implement query routing — for some applications, you can classify incoming queries and route exact-match-style queries directly to BM25 while semantic queries go to dense retrieval, using RRF only when you're uncertain. This reduces latency at the cost of some complexity.
The hybrid retrieval pattern — BM25 + dense + RRF — is the right default for serious RAG pipelines. Start here, measure your results, and optimize from evidence.