Parent Document Retrieval: Index Small Chunks, Return Rich Context for Better RAG*
⚡ Practitioner21 min readAug 3, 2026Updated Aug 29, 2026
Here's a scenario you've probably hit: you build a RAG pipeline, chunk your documents into 512-token pieces, index them, and everything looks fine in testing. Then a user asks a nuanced question that requires understanding a multi-paragraph argument — and the system returns three isolated sentence fragments from three different sections of the document. The LLM does its best, but without the surrounding context, the answer is vague, incomplete, or just wrong. You've indexed for precision but accidentally destroyed comprehension.
This is the core tension in RAG retrieval design. Small chunks give you precise semantic matching — a short, focused embedding represents a single idea well, and cosine similarity finds it reliably. But small chunks also strip away the context that gives meaning to that idea. The sentence "This limitation is addressed in the next section" means nothing on its own. The paragraph around it might be critical. Parent Document Retrieval (PDR) is the design pattern that resolves this tension: index the small chunks for search precision, but when a chunk matches, return its parent — the full section it came from — to the LLM.
By the end of this lesson, you'll have a working Parent Document Retrieval system built with LangChain, a solid mental model for why the two-layer storage architecture works the way it does, and enough understanding of the tradeoffs to decide when PDR is the right tool versus when it isn't.
What you'll learn:
ParentDocumentRetrieverYou should be comfortable with:
You'll need these installed:
pip install langchain langchain-openai langchain-chroma chromadb tiktoken
Let's make the problem concrete before we fix it. Suppose you're building a RAG system over a 200-page technical specification for a financial product — say, a bond prospectus. A compliance analyst asks: "What are the conditions under which the issuer can call the bond early?"
The answer lives in a section that spans roughly 600 words. It starts with a general statement of the call provision, then lists specific trigger conditions, then explains the notice period, then describes the redemption price formula. These four elements are semantically distinct enough that a naive chunker might put them in four different chunks.
When the analyst asks her question, the embedding for "call the bond early" might match the chunk about trigger conditions best. That chunk gets returned. The LLM gets 150 tokens of conditions with no preamble, no notice period, no price formula. The answer it generates is technically derived from the document but is functionally incomplete — and in a compliance context, that's dangerous.
The naive fix is to just use bigger chunks — 1500 or 2000 tokens. But now you've introduced a different problem: embeddings of long chunks are averaged representations of multiple ideas. A chunk about "call provisions, redemption procedures, and transfer restrictions" has an embedding that sits somewhere in the vector space between all three topics. When the analyst asks about call provisions, a more precise 150-token chunk about call provisions from a different document might outscore your 1500-token chunk even though the 1500-token chunk contains the right answer. Large chunks hurt retrieval precision. You're trading one problem for another.
Parent Document Retrieval says: don't choose. Index small, return large.
PDR relies on a clean architectural separation that's worth understanding deeply before you write any code.
The vector store holds embeddings of small child chunks — the units of semantic precision. These are what gets searched. A typical child chunk might be 200–400 tokens. Each child chunk has metadata that includes a reference (a key or ID) pointing back to its parent.
The document store (also called the docstore or parent store) is a simple key-value store that holds the full parent documents or parent sections. It doesn't need to support vector search. It just needs to be able to return a document given a key. In LangChain, this is an InMemoryStore by default, though you can back it with Redis, a database, or any persistent store for production.
The retrieval flow is:
What makes this elegant is that the vector store and document store have completely different jobs and can be optimized independently. Your vector store (Chroma, Pinecone, Weaviate, etc.) is tuned for fast approximate nearest neighbor search. Your document store is tuned for fast key-value lookup. Neither has to compromise for the other.
Key insight
The child chunks are search artifacts — they exist purely to improve retrieval precision. They are never shown to the LLM. Only the parents are.
Let's build this properly. We'll use a realistic scenario: a corpus of SEC 10-K filings that a financial analyst needs to query. We'll work with a subset for the example, but the architecture scales directly.
from langchain_community.document_loaders import TextLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_openai import OpenAIEmbeddings
from langchain_chroma import Chroma
from langchain.storage import InMemoryStore
from langchain.retrievers import ParentDocumentRetriever
# Load your documents — here we're loading from a directory of 10-K text files
# In production, you'd use PyPDFLoader, UnstructuredFileLoader, etc.
from langchain_community.document_loaders import DirectoryLoader
loader = DirectoryLoader(
"./sec_filings/",
glob="**/*.txt",
loader_cls=TextLoader,
loader_kwargs={"encoding": "utf-8"}
)
docs = loader.load()
print(f"Loaded {len(docs)} documents")
# Loaded 12 documents
This is the most consequential configuration decision in a PDR setup. You're defining two levels of granularity.
The child splitter creates the small, precise chunks that go into the vector store. These should be small enough that each chunk represents a single coherent idea.
The parent splitter creates the larger sections that get returned to the LLM. These should be large enough to contain enough context to actually answer questions.
# Child splitter: small chunks for precise retrieval
# 400 tokens is a good starting point for dense technical text
child_splitter = RecursiveCharacterTextSplitter(
chunk_size=400,
chunk_overlap=50,
length_function=len, # character count; use tiktoken for token count
separators=["\n\n", "\n", ". ", " ", ""]
)
# Parent splitter: larger sections for rich LLM context
# 1500 tokens gives the LLM a full section to work with
parent_splitter = RecursiveCharacterTextSplitter(
chunk_size=1500,
chunk_overlap=100,
separators=["\n\n", "\n", ". ", " ", ""]
)
Tip
For financial filings, legal documents, or academic papers — text where arguments build across paragraphs — err toward larger parent chunks (1500–2000 tokens). For FAQ documents, support tickets, or product descriptions where each entry is self-contained, smaller parents (600–800 tokens) are often sufficient.
# Vector store: Chroma backed by OpenAI embeddings
# In production, swap for Pinecone, Weaviate, or pgvector
embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
vectorstore = Chroma(
collection_name="sec_filings_child_chunks",
embedding_function=embeddings,
persist_directory="./chroma_db"
)
# Document store: holds the full parent sections
# InMemoryStore is fine for prototyping; use Redis or SQLite for production
docstore = InMemoryStore()
# Assemble the retriever
retriever = ParentDocumentRetriever(
vectorstore=vectorstore,
docstore=docstore,
child_splitter=child_splitter,
parent_splitter=parent_splitter,
search_kwargs={"k": 4} # retrieve top 4 child chunks, then fetch their parents
)
This is where both stores get populated. Watch what happens under the hood:
# This single call:
# 1. Splits docs into parent chunks using parent_splitter
# 2. Assigns each parent a UUID key and stores it in docstore
# 3. Splits each parent into child chunks using child_splitter
# 4. Tags each child chunk with its parent's UUID in metadata
# 5. Embeds all child chunks and stores them in vectorstore
retriever.add_documents(docs, ids=None)
print("Indexing complete.")
# Let's verify what's in each store
child_count = vectorstore._collection.count()
# Approximate parent count (InMemoryStore doesn't expose a direct count API)
parent_keys = list(docstore.yield_keys())
print(f"Child chunks in vector store: {child_count}")
print(f"Parent sections in document store: {len(parent_keys)}")
# Child chunks in vector store: 847
# Parent sections in document store: 214
Notice that 214 parents produced 847 children — roughly a 4:1 ratio, which is about right for a 400-token child chunk inside a 1500-token parent.
# Ask a question that requires multi-paragraph context
query = "What are the risk factors related to interest rate exposure for fixed-income securities?"
# This invoke() call:
# 1. Embeds the query
# 2. Finds top-4 child chunks in Chroma
# 3. Looks up their parent UUIDs
# 4. Returns the full parent documents from InMemoryStore
results = retriever.invoke(query)
print(f"Retrieved {len(results)} parent documents")
for i, doc in enumerate(results):
print(f"\n--- Parent Document {i+1} ---")
print(f"Source: {doc.metadata.get('source', 'unknown')}")
print(f"Length: {len(doc.page_content)} characters")
print(f"Preview: {doc.page_content[:200]}...")
You'll notice the retrieved documents are substantially longer than a standard 400-token chunk retrieval would give you. That's the point.
from langchain_openai import ChatOpenAI
from langchain.chains import create_retrieval_chain
from langchain.chains.combine_documents import create_stuff_documents_chain
from langchain_core.prompts import ChatPromptTemplate
llm = ChatOpenAI(model="gpt-4o", temperature=0)
system_prompt = """You are a financial analyst assistant specializing in SEC filings.
Answer questions based on the provided context from 10-K filings.
Be specific and cite relevant details from the documents.
If the context doesn't contain enough information to answer definitively, say so.
Context:
{context}"""
prompt = ChatPromptTemplate.from_messages([
("system", system_prompt),
("human", "{input}")
])
# create_stuff_documents_chain concatenates all retrieved docs into the context
question_answer_chain = create_stuff_documents_chain(llm, prompt)
rag_chain = create_retrieval_chain(retriever, question_answer_chain)
response = rag_chain.invoke({
"input": "What are the risk factors related to interest rate exposure for fixed-income securities?"
})
print(response["answer"])
There's actually a spectrum of PDR configurations worth knowing:
Configuration 1: Child chunks → Parent chunks (what we built above) Child splitter creates small search units. Parent splitter creates medium sections. Most common, most flexible.
Configuration 2: Child chunks → Full document
Skip the parent splitter entirely. When a child matches, return the entire source document. Use this when your documents are short enough to fit in context (a 2-page product spec, a 500-word policy document). In LangChain, you achieve this by omitting parent_splitter from the ParentDocumentRetriever constructor:
retriever_full_doc = ParentDocumentRetriever(
vectorstore=vectorstore_v2,
docstore=docstore_v2,
child_splitter=child_splitter,
# No parent_splitter — returns entire source document
search_kwargs={"k": 3}
)
Warning
Full document retrieval can blow up your context window if documents are long. A 50-page contract returned in full will consume 30,000+ tokens. This works for short documents; for long ones, stick with parent chunk retrieval.
Configuration 3: Sentence-level children → Paragraph parents Fine-grained for dense scientific or legal text where individual sentences carry specific meaning. Child chunks might be 100–150 tokens (1–2 sentences). Parents might be 600–800 tokens (a full paragraph or two). This gives maximum retrieval precision with enough context to understand the retrieved material.
sentence_splitter = RecursiveCharacterTextSplitter(
chunk_size=150,
chunk_overlap=20,
separators=[". ", "! ", "? ", "\n"]
)
paragraph_splitter = RecursiveCharacterTextSplitter(
chunk_size=700,
chunk_overlap=50,
separators=["\n\n", "\n"]
)
retriever_sentence = ParentDocumentRetriever(
vectorstore=vectorstore_v3,
docstore=docstore_v3,
child_splitter=sentence_splitter,
parent_splitter=paragraph_splitter,
search_kwargs={"k": 5}
)
InMemoryStore evaporates when your process restarts. For any production deployment, you need a persistent docstore. Here are two practical options.
import redis
from langchain.storage import RedisStore
redis_client = redis.Redis(host="localhost", port=6379, db=0)
persistent_docstore = RedisStore(client=redis_client, namespace="sec_filings_parents")
retriever_prod = ParentDocumentRetriever(
vectorstore=vectorstore,
docstore=persistent_docstore,
child_splitter=child_splitter,
parent_splitter=parent_splitter,
search_kwargs={"k": 4}
)
For lighter deployments where Redis is overkill, you can implement a simple SQLite-backed store:
import sqlite3
import json
from langchain_core.stores import BaseStore
from typing import Iterator, List, Optional, Sequence, Tuple
class SQLiteStore(BaseStore):
"""Lightweight persistent store backed by SQLite."""
def __init__(self, db_path: str, table_name: str = "docstore"):
self.db_path = db_path
self.table_name = table_name
self._init_db()
def _init_db(self):
conn = sqlite3.connect(self.db_path)
conn.execute(f"""
CREATE TABLE IF NOT EXISTS {self.table_name} (
key TEXT PRIMARY KEY,
value TEXT NOT NULL
)
""")
conn.commit()
conn.close()
def mget(self, keys: Sequence[str]) -> List[Optional[str]]:
conn = sqlite3.connect(self.db_path)
results = []
for key in keys:
row = conn.execute(
f"SELECT value FROM {self.table_name} WHERE key = ?", (key,)
).fetchone()
results.append(row[0] if row else None)
conn.close()
return results
def mset(self, key_value_pairs: Sequence[Tuple[str, str]]) -> None:
conn = sqlite3.connect(self.db_path)
conn.executemany(
f"INSERT OR REPLACE INTO {self.table_name} (key, value) VALUES (?, ?)",
key_value_pairs
)
conn.commit()
conn.close()
def mdelete(self, keys: Sequence[str]) -> None:
conn = sqlite3.connect(self.db_path)
conn.executemany(
f"DELETE FROM {self.table_name} WHERE key = ?",
[(k,) for k in keys]
)
conn.commit()
conn.close()
def yield_keys(self, prefix: Optional[str] = None) -> Iterator[str]:
conn = sqlite3.connect(self.db_path)
query = f"SELECT key FROM {self.table_name}"
if prefix:
query += f" WHERE key LIKE '{prefix}%'"
for row in conn.execute(query):
yield row[0]
conn.close()
# Use it
sqlite_docstore = SQLiteStore("./sec_filings_docstore.db")
retriever_sqlite = ParentDocumentRetriever(
vectorstore=vectorstore,
docstore=sqlite_docstore,
child_splitter=child_splitter,
parent_splitter=parent_splitter,
search_kwargs={"k": 4}
)
Tip
In production, keep your vector store and docstore in sync. If you re-index documents, clear both stores. Orphaned child chunks pointing to deleted parents are a subtle bug that produces empty retrievals with no error.
Getting the chunk sizes right is more art than science, but there are principled guidelines.
For the child chunk:
Your goal is a single coherent semantic unit. If you're using text-embedding-3-small, it handles up to 8191 tokens, but that's not the point — shorter embeddings are more precise. Test with 200–500 characters (roughly 50–125 tokens). The key question to ask: If this chunk came up in search results, would I be able to tell immediately whether it's relevant to my query? If the chunk mixes two topics, it's too large.
For the parent chunk: Your goal is enough context to answer questions that require understanding the section's argument. Think in terms of the natural structure of your documents. Is the document structured in 500-word subsections? Make your parent 500–600 tokens. Is it written in long flowing prose with extended arguments? Go up to 2000 tokens. Always consider your LLM's context window and the number of parents you'll retrieve — if you retrieve 4 parents at 2000 tokens each, that's 8000 tokens of context before you've even added the system prompt.
The ratio as a diagnostic: A parent:child token ratio of 3:1 to 6:1 is typical. If your ratio is 10:1 or higher, ask yourself whether the parent is genuinely necessary to answer questions or whether you're just stuffing the context window. If the ratio is 1.5:1, your child chunks might be too close in size to the parents to provide meaningful precision improvement.
The most common mistake practitioners make is assuming PDR is better without measuring it. Here's a quick evaluation framework.
For your specific document corpus, create 20–30 questions that require multi-paragraph context to answer correctly. Pair each question with a "gold standard" context — the section of the document that genuinely answers it.
eval_questions = [
{
"question": "What methodology does the company use to estimate allowances for credit losses?",
"gold_source": "2023_annual_report.txt",
"gold_section_keywords": ["allowance", "credit loss", "methodology", "discounted cash flow"]
},
{
"question": "Describe the company's policy for capitalizing software development costs.",
"gold_source": "2023_annual_report.txt",
"gold_section_keywords": ["capitalize", "software development", "internal use", "technological feasibility"]
},
# ... more questions
]
from langchain_chroma import Chroma
# Standard retriever: just returns chunks directly
standard_vectorstore = Chroma(
collection_name="standard_chunks",
embedding_function=embeddings
)
# Index with only one splitter (no PDR)
standard_texts = parent_splitter.split_documents(docs)
standard_vectorstore.add_documents(standard_texts)
standard_retriever = standard_vectorstore.as_retriever(search_kwargs={"k": 4})
def evaluate_retrieval(retriever, eval_questions):
results = []
for eq in eval_questions:
retrieved = retriever.invoke(eq["question"])
# Check if retrieved docs contain the gold section keywords
combined_text = " ".join([d.page_content.lower() for d in retrieved])
keyword_hits = sum(
1 for kw in eq["gold_section_keywords"]
if kw.lower() in combined_text
)
recall_score = keyword_hits / len(eq["gold_section_keywords"])
results.append({
"question": eq["question"],
"recall_score": recall_score,
"avg_doc_length": sum(len(d.page_content) for d in retrieved) / len(retrieved)
})
return results
standard_results = evaluate_retrieval(standard_retriever, eval_questions)
pdr_results = evaluate_retrieval(retriever, eval_questions)
avg_standard_recall = sum(r["recall_score"] for r in standard_results) / len(standard_results)
avg_pdr_recall = sum(r["recall_score"] for r in pdr_results) / len(pdr_results)
print(f"Standard retrieval avg recall: {avg_standard_recall:.2f}")
print(f"PDR avg recall: {avg_pdr_recall:.2f}")
print(f"Standard avg doc length: {sum(r['avg_doc_length'] for r in standard_results)/len(standard_results):.0f} chars")
print(f"PDR avg doc length: {sum(r['avg_doc_length'] for r in pdr_results)/len(pdr_results):.0f} chars")
You should see PDR with higher recall (more gold keywords present in retrieved context) and longer average document length. If PDR recall is not higher, your child chunk size might be too large (you're not getting precision improvements) or your parent chunk size might be misaligned with the document's natural structure.
Build a PDR-based research assistant over a real document corpus. Here's the challenge:
The scenario: Your team is doing due diligence on a merger. You have a folder of 8–10 documents: the target company's annual report, several analyst research reports, and their major vendor contracts (use publicly available PDFs or text files for this exercise — SEC EDGAR is a great source for real 10-K filings).
What to build:
Set up the two-layer architecture using Chroma for the vector store and your SQLite store from earlier for persistence. Index all documents with a 300-token child splitter and a 1200-token parent splitter.
Add source metadata filtering. Modify the retriever so analysts can specify {"source": "annual_report"} to restrict retrieval to a specific document. In LangChain, pass this as a filter to search_kwargs:
retriever_filtered = ParentDocumentRetriever(
vectorstore=vectorstore,
docstore=sqlite_docstore,
child_splitter=child_splitter,
parent_splitter=parent_splitter,
search_kwargs={
"k": 4,
"filter": {"source": "annual_report"} # Chroma metadata filter
}
)
Run these test queries and manually evaluate whether the retrieved context would allow an analyst to answer them completely:
Experiment with chunk sizes. Try child=200/parent=800, child=400/parent=1500, and child=600/parent=2000. For each configuration, run your test queries and note whether the retrieved context is more or less relevant and complete.
Add a deduplication step. When multiple child chunks from the same parent match, you should only return that parent once. Check whether LangChain's PDR handles this automatically (it does — verify by logging the parent UUIDs) and understand why it matters for context window efficiency.
If you forget to pass parent_splitter to ParentDocumentRetriever, it defaults to using the entire source document as the parent. For a 200-page filing, that's 150,000 tokens. Your LLM call will fail or cost a fortune.
Fix: Always explicitly define both splitters and double-check that parent chunk size > child chunk size by at least 3x.
You update your documents and re-run retriever.add_documents(). But the old child chunks in Chroma still exist and still point to old parent UUIDs. The docstore now has both old and new parents. Queries might retrieve old, stale content.
Fix: Before re-indexing, delete the Chroma collection and the docstore table/namespace, then rebuild.
# Clear and rebuild
vectorstore.delete_collection()
vectorstore = Chroma(
collection_name="sec_filings_child_chunks",
embedding_function=embeddings,
persist_directory="./chroma_db"
)
# Also clear docstore
for key in list(docstore.yield_keys()):
docstore.mdelete([key])
If your parent splitter uses "\n\n" as the primary separator but your documents use "\r\n\r\n" (Windows line endings), your parents will be the entire document. Check your documents' actual whitespace characters:
# Diagnose line endings
with open("./sec_filings/sample.txt", "rb") as f:
content = f.read(1000)
print(repr(content)) # Look for \r\n vs \n
Fix: Normalize line endings during loading:
loader = TextLoader("./sec_filings/sample.txt", encoding="utf-8")
doc = loader.load()[0]
doc.page_content = doc.page_content.replace("\r\n", "\n")
You set search_kwargs={"k": 3}, expecting 3 parent documents back. But two of the top-3 child chunks come from the same parent. After deduplication, you get 2 parents. If your questions need broad coverage, this is insufficient.
Fix: Increase k on the child retrieval side — k=6 or k=8 — knowing that deduplication will reduce it to something reasonable. The child retrieval is cheap; don't be stingy with k.
Four parent chunks at 1500 tokens each = 6000 tokens of context. Add a 500-token system prompt, a 50-token user query, and you're at 6550 tokens before the LLM generates a word. With a 4096-token model, this fails. With gpt-4o (128k context), it's fine. Know your model's limits.
Fix: Calculate your maximum context contribution: k × parent_chunk_tokens + system_prompt_tokens + query_tokens + expected_response_tokens ≤ model_context_window. Adjust k or parent size accordingly.
PDR is powerful but not always the right tool.
Use PDR when:
Consider alternatives when:
Compare against:
You've built a production-capable Parent Document Retrieval system. The core idea is simple but powerful: the unit of search and the unit of context don't have to be the same thing. By maintaining a two-store architecture — small child chunks in the vector store for precision, full parent sections in the document store for comprehension — you get the best of both worlds.
The most important decisions you make in a PDR system are your chunk sizes. Get those wrong and you're either searching imprecisely (children too large) or returning insufficient context (parents too small). The evaluation framework we built gives you a data-driven way to tune those parameters rather than guessing.
Where to go next:
Experiment with sentence-level children over a scientific paper corpus. The precision gains at sub-150-token child chunks can be dramatic for highly specific factual questions.
Add metadata-aware retrieval — combine PDR with document-level metadata filters so users can scope queries to date ranges, authors, or document types without re-indexing.
Explore EnsembleRetriever to combine your PDR with a BM25 lexical retriever for hybrid search. PDR handles semantic matching; BM25 catches exact terminology and acronyms that embeddings sometimes miss.
Build an evaluation pipeline with RAGAS — the recall and context precision metrics in RAGAS are exactly the right instruments to formally benchmark PDR against baseline retrieval at scale.
Move toward agentic retrieval — once you've mastered static retrieval patterns, look at how agents dynamically choose between PDR and other retrieval strategies based on the query type. A question about a specific number calls for small chunk retrieval; a question requiring synthesis calls for PDR.