Token limits aren't just about fitting content — they're about information density and attention mechanics. Learn systematic techniques for compressing instructions, schemas, examples, and context to maximize LLM output quality on complex enterprise tasks, including programmatic prompt assembly patterns for production systems.
You've got a 128K context window, a 47-page regulatory compliance document, a 200-row data schema, and a task that requires the model to reason across all of it simultaneously. You paste everything in and submit. The model wanders, misses key constraints, and produces something technically plausible but practically useless. You've hit not the hard limit — the model didn't reject your prompt — but the soft limit: the cognitive bandwidth of an LLM under full-context load.
This is one of the most underappreciated failure modes in enterprise AI work. Larger context windows have given practitioners a false sense of security. The assumption that "more context is always better" ignores a critical reality: LLMs don't process context uniformly. Token position, density, and structure all affect how much of your input actually influences the output. Stuffing a context window to capacity is not the same as giving the model the right information. Often, it's worse.
By the end of this lesson, you'll have a rigorous, practical toolkit for compressing information into prompts without losing the signal that drives quality outputs. You'll understand why compression works at the architectural level, not just how to do it mechanically.
What you'll learn:
You should be comfortable with the basics of prompt structure and how context windows work before diving in. If you need a refresher, start with Tokens, Context Windows, and Input Limits: What Data Professionals Need to Know Before Building AI Workflows and Prompt Engineering Fundamentals for Data Professionals. You should also have working familiarity with calling LLM APIs programmatically — we'll use Python examples throughout.
Most people approach prompt compression as an editing task: trim the verbose instructions, cut some boilerplate, shorten the examples. That thinking will take you about 20% of the way there. The deeper work requires understanding what's actually happening when the model reads your prompt.
Modern transformer-based LLMs process input through attention mechanisms that compute relationships between every token and every other token in the context. This is computationally expensive, but more importantly for our purposes, it means the model is not reading your prompt the way a human reads a document — linearly, with active comprehension of structure and intent. Instead, it's computing a weighted combination of all token relationships simultaneously.
Several empirical patterns have emerged from this architecture that directly affect prompt design:
The primacy-recency effect is real and dramatic. Tokens near the beginning and end of the context window receive disproportionately high attention weight compared to tokens in the middle. This is sometimes called the "lost in the middle" problem. In a 100K-token context, information placed between tokens 20K and 80K is statistically more likely to be underweighted in the model's output. For enterprise tasks where you're injecting large documents, this is catastrophic if your most critical constraints land in the middle.
Token density affects reasoning quality. When you express a concept in 200 tokens that could be expressed equally clearly in 40, you're not just wasting space — you're diluting the signal. The model has to spread attention across all 200 tokens, and the redundancy doesn't reinforce the concept as effectively as you might expect. Dense, precise language that encodes more meaning per token tends to produce better reasoning.
Structural tokens carry real semantic weight. Markdown headers, bullet points, XML tags, and other structural elements do more than organize the prompt for human readability. They create attention anchors that help the model segment and prioritize content. A flat wall of prose is objectively harder for an LLM to parse than the same content organized with clear structure.
Key insight
Prompt compression isn't just about fitting within a limit — it's about maximizing the signal-to-noise ratio of every token you spend. The goal is not brevity for its own sake; it's precision. Every token should be doing work.
If you want to understand more about the underlying mechanics, Understanding Large Language Models: How ChatGPT and Claude Actually Work covers the transformer architecture in accessible depth. That understanding is foundational to what follows.
The most immediately actionable category of compression is your instruction set. Enterprise prompts for complex tasks often suffer from what I call "legal brief syndrome": every edge case is enumerated, every assumption is explained, every constraint is stated in three different ways. This feels thorough. It's often counterproductive.
Here's a typical verbose instruction block for a financial data extraction task:
You are an AI assistant that specializes in extracting financial data from
quarterly earnings reports. Your job is to carefully read through the
provided text and find all mentions of revenue figures. When you find a
revenue figure, please make sure you record it accurately. You should also
look for gross profit figures. In addition to revenue and gross profit,
you should also look for EBITDA figures if they are present in the document.
Please note that some figures may be in millions and some may be in billions,
so pay careful attention to the units. If a figure appears multiple times,
please use the most recently stated figure. Please format your output as a
JSON object. The JSON object should have keys for revenue, gross_profit, and
ebitda. If a value is not found, please set it to null rather than omitting
the key. Please be careful to only extract figures that are explicitly stated
and not to calculate or infer figures that are not directly provided.
That's 178 tokens. Let's count what it actually says:
Here's the compressed version:
Extract financial metrics from the text below.
Output JSON with these exact keys (null if not found):
- revenue
- gross_profit
- ebitda
Rules:
- Preserve units exactly (millions/billions as stated)
- If a metric appears multiple times, use the final occurrence
- No inference or calculation — explicitly stated values only
That's 61 tokens — 66% reduction — and it's more parseable by both humans and models because the structure is cleaner. The model doesn't have to extract the rules from flowing prose; they're enumerated.
Run this process on every instruction block:
Warning
There is a real cost to over-compression. Instructions that are too terse can be ambiguous in ways that trigger the model's defaults, which may not match your intent. The target is precise brevity, not minimum tokens. If removing words makes a constraint ambiguous, keep them.
For conditional logic in instructions, a decision table is almost always more token-efficient than prose:
Prose version (94 tokens):
If the customer's account status is active and their order value is above
$500, apply the premium discount. If their account is active but the order
is $500 or below, apply the standard discount. If the account is inactive,
do not apply any discount regardless of order value.
Decision table version (38 tokens):
Discount logic:
| Status | Order > $500 | Discount |
|----------|-------------|----------|
| Active | Yes | Premium |
| Active | No | Standard |
| Inactive | Any | None |
The table encodes exactly the same information in less than half the tokens, and the model is considerably less likely to misparse conditional relationships expressed this way.
This is where enterprise AI projects live or die. You have a 200-page contract, a 500-row database schema, or a multi-year conversation history. The naive approach is to paste it all in. The expert approach is to understand what the model actually needs and inject only that.
Database schemas are a particularly common source of prompt bloat. A full DDL for a production database can run into tens of thousands of tokens. Here's a before/after:
Full DDL (fragment, ~800 tokens for this table alone):
CREATE TABLE customer_transactions (
transaction_id BIGINT NOT NULL AUTO_INCREMENT,
customer_id BIGINT NOT NULL,
transaction_date DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
transaction_type ENUM('purchase', 'refund', 'adjustment', 'reversal') NOT NULL,
amount DECIMAL(12, 2) NOT NULL,
currency_code CHAR(3) NOT NULL DEFAULT 'USD',
status ENUM('pending', 'completed', 'failed', 'cancelled') NOT NULL DEFAULT 'pending',
payment_method VARCHAR(50),
gateway_reference VARCHAR(100),
store_id INT,
associate_id INT,
notes TEXT,
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_at DATETIME ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (transaction_id),
INDEX idx_customer (customer_id),
INDEX idx_date (transaction_date),
INDEX idx_status (status),
CONSTRAINT fk_customer FOREIGN KEY (customer_id) REFERENCES customers(customer_id),
CONSTRAINT fk_store FOREIGN KEY (store_id) REFERENCES stores(store_id),
CONSTRAINT fk_associate FOREIGN KEY (associate_id) REFERENCES associates(associate_id)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
Compressed schema notation (~120 tokens):
TABLE: customer_transactions
KEY COLUMNS (for this task):
- transaction_id (PK, BIGINT)
- customer_id (FK→customers, BIGINT)
- transaction_date (DATETIME)
- transaction_type: purchase|refund|adjustment|reversal
- amount (DECIMAL 12,2)
- currency_code (CHAR 3, default USD)
- status: pending|completed|failed|cancelled
- store_id (FK→stores), associate_id (FK→associates)
OMITTED: gateway_reference, notes, timestamps (not needed for this task)
That's an 85% token reduction. The model has everything it needs to write correct SQL against this table. You've omitted the storage engine, charset, index definitions, and columns irrelevant to the task. Note the explicit OMITTED: line — this is important. It tells the model these fields exist but were intentionally excluded, which prevents it from assuming the table has fewer columns than it does, which could cause errors in wildcard queries.
Tip
Always note what you've omitted and why. "OMITTED: [field] (not relevant to this task)" uses a few tokens but prevents the model from making false assumptions about schema completeness. This is especially critical if the model might generate SELECT * queries.
For long documents, don't inject the raw document — inject a structured extraction of the relevant content. This sounds obvious but requires discipline because it means doing preprocessing work before your main prompt.
For a 47-page regulatory compliance document, the pattern looks like this:
# Step 1: Extract the relevant sections first
extraction_prompt = """
From this regulatory document, extract ONLY:
1. All specific numerical thresholds and their associated requirements
2. All deadline/timeline requirements
3. All entity definitions that apply to financial services companies
Format as:
THRESHOLDS: [list]
DEADLINES: [list]
DEFINITIONS: [list]
Document:
{document_text}
"""
# Step 2: Use the extracted summary in your main task prompt
main_prompt = """
Based on these regulatory requirements:
{extracted_summary}
Evaluate this proposed process:
{process_description}
Identify gaps and recommend remediations.
"""
This two-step approach is a form of prompt chaining — using one model call to pre-process context for the next. The first call produces a compressed, structured representation that can be 10-20x smaller than the source document while retaining the content relevant to your task. The second call then operates on high-density context rather than a sea of legalese.
For dynamically assembled prompts, you need a principled approach to selecting which context to include. Embedding-based relevance filtering is the standard approach at scale, but even without vector databases, you can apply these principles manually or with lightweight logic:
import anthropic
def compress_context_to_budget(context_chunks: list[dict],
query: str,
token_budget: int,
client: anthropic.Anthropic) -> str:
"""
Given a list of context chunks with pre-computed relevance scores,
greedily select chunks within a token budget.
"""
# Sort by relevance score descending
sorted_chunks = sorted(context_chunks,
key=lambda x: x['relevance_score'],
reverse=True)
selected = []
tokens_used = 0
for chunk in sorted_chunks:
chunk_tokens = chunk['token_count']
if tokens_used + chunk_tokens <= token_budget:
selected.append(chunk)
tokens_used += chunk_tokens
else:
# Try to fit a summarized version
if chunk['token_count'] > 200: # Only worth summarizing larger chunks
summary_response = client.messages.create(
model="claude-opus-4-5",
max_tokens=100,
messages=[{
"role": "user",
"content": f"Summarize this in 50 words, preserving all specific facts and figures:\n\n{chunk['text']}"
}]
)
summary = summary_response.content[0].text
summary_tokens = len(summary.split()) * 1.3 # rough token estimate
if tokens_used + summary_tokens <= token_budget:
selected.append({**chunk, 'text': summary, 'token_count': summary_tokens})
tokens_used += summary_tokens
# Reassemble in original document order (not relevance order) for coherence
selected.sort(key=lambda x: x['original_position'])
return "\n\n".join([c['text'] for c in selected])
This pattern — relevance-ranked selection with fallback summarization — is the building block of production RAG systems. For a deeper treatment of how this fits into larger architectures, see Retrieval-Augmented Generation in Practice: Building Knowledge-Grounded AI Pipelines for Enterprise Data Workflows.
Few-shot examples are one of the highest-leverage elements in any prompt, and also one of the most common sources of token waste. The research on few-shot prompting consistently shows that example quality matters far more than example quantity.
The most common mistake is including full, naturalistic examples that include lots of irrelevant surrounding context. You want examples that are maximally informative about the pattern you're demonstrating, with all irrelevant content stripped.
Bloated example (310 tokens):
Example 1:
Input: "We need to analyze Q3 performance across all our regions. The North
American division reported revenue of $24.7 million for the quarter ended
September 30, 2024. Our European operations saw revenue come in at €18.2
million. The Asia Pacific region achieved revenue of $31.1 million USD.
All figures are for the quarter ending September 30, 2024."
Output: {
"period": "Q3 2024",
"period_end": "2024-09-30",
"regions": {
"north_america": {"revenue": 24700000, "currency": "USD"},
"europe": {"revenue": 18200000, "currency": "EUR"},
"asia_pacific": {"revenue": 31100000, "currency": "USD"}
}
}
Compressed example (98 tokens):
Example:
Input: "Q3 ended Sep 30. NA revenue $24.7M. Europe €18.2M. APAC $31.1M USD."
Output: {
"period_end": "2024-09-30",
"regions": {
"NA": {"revenue": 24700000, "currency": "USD"},
"EU": {"revenue": 18200000, "currency": "EUR"},
"APAC": {"revenue": 31100000, "currency": "USD"}
}
}
The compressed example still demonstrates every critical pattern: currency normalization, unit conversion (millions to absolute), date formatting, and key structure. The bloated narrative around it adds nothing. The model doesn't need to see how the task appears in a real document — it needs to see the input-output transformation.
Rather than including multiple examples of the same pattern, use each example slot to demonstrate a distinct edge case. For an extraction task, you might have:
Four compact examples demonstrating four distinct behaviors is worth far more than four verbose examples of slightly different standard cases.
Key insight
Your examples are teaching the model a function, not familiarizing it with your writing style. Every example should cover unique behavioral territory. Redundant examples don't reinforce — they dilute the contrast between cases and waste tokens.
How you structure your prompt is itself a compression technique. Good structure lets you communicate hierarchy, priority, and relationships with formatting tokens rather than explanatory prose.
XML-style delimiters might seem like they'd add tokens (they do, slightly), but they earn their keep by eliminating the prose explanations that would otherwise be necessary to segment context:
Without structure (requires 30+ tokens of explanation per section):
Below is some background context about our company that you should keep in
mind. After that, I'll provide the specific task, and then some data. The
context section contains information about our business rules that should
govern your analysis...
[context]
Now here is the task you need to perform...
[task]
And here is the data to analyze...
[data]
With XML structure (self-explanatory):
<context>
[business rules - no explanation needed]
</context>
<task>
[task definition]
</task>
<data>
[data to analyze]
</data>
The XML version is cleaner and shorter, and studies on models like Claude specifically show that explicit XML delimiters significantly improve instruction-following accuracy because the model has clear boundaries between content types. You're trading a small number of tag tokens for a large reduction in prose explanation tokens.
You can encode priority without spending tokens explaining the hierarchy:
Verbose priority specification (~80 tokens):
The most important rule is that you must never include personally identifiable
information in your output. This is a hard requirement. The second most
important consideration is accuracy of the extracted figures. Style and
formatting are less important than accuracy, but still matter.
Compressed priority specification (~25 tokens):
RULES (in priority order):
1. [HARD] No PII in output
2. [HARD] Numerical accuracy
3. [SOFT] Formatting consistency
The brackets [HARD] and [SOFT] are semantic shorthands that most modern LLMs will correctly interpret as absolute vs. flexible requirements. You've saved 55 tokens and made the priority order explicit.
For production systems, manual compression isn't sustainable. You need programmatic systems that compress prompts at runtime based on available token budget and task requirements. This connects directly to the patterns described in Contextual Compression and Dynamic Prompt Assembly.
from dataclasses import dataclass
from typing import Optional
import tiktoken
@dataclass
class PromptComponent:
name: str
content: str
priority: int # Lower = higher priority (1 is critical)
compressible: bool # Can this section be summarized?
min_tokens: int # Minimum tokens needed to be useful
compressed_version: Optional[str] = None
class PromptBudgetAllocator:
def __init__(self, model: str = "gpt-4o",
total_budget: int = 8000,
output_reserve: int = 2000):
self.encoder = tiktoken.encoding_for_model(model)
self.available_budget = total_budget - output_reserve
def count_tokens(self, text: str) -> int:
return len(self.encoder.encode(text))
def assemble(self, components: list[PromptComponent]) -> str:
# Sort by priority
components.sort(key=lambda c: c.priority)
allocated = []
tokens_remaining = self.available_budget
for component in components:
full_tokens = self.count_tokens(component.content)
if full_tokens <= tokens_remaining:
# Fits fully — use it
allocated.append((component.name, component.content, 'full'))
tokens_remaining -= full_tokens
elif component.compressible and component.compressed_version:
compressed_tokens = self.count_tokens(component.compressed_version)
if compressed_tokens <= tokens_remaining:
allocated.append((component.name,
component.compressed_version,
'compressed'))
tokens_remaining -= compressed_tokens
elif compressed_tokens >= component.min_tokens:
# Can't fit even the compressed version
if component.priority <= 2:
# Critical component — truncate but include
truncated = self._truncate_to_budget(
component.compressed_version,
tokens_remaining
)
allocated.append((component.name, truncated, 'truncated'))
tokens_remaining = 0
# else: skip non-critical component
elif component.priority <= 2:
# Non-compressible critical component — truncate
truncated = self._truncate_to_budget(
component.content, tokens_remaining
)
allocated.append((component.name, truncated, 'truncated'))
tokens_remaining = 0
return self._render(allocated)
def _truncate_to_budget(self, text: str, budget: int) -> str:
tokens = self.encoder.encode(text)
return self.encoder.decode(tokens[:budget])
def _render(self, allocated: list) -> str:
sections = []
for name, content, status in allocated:
prefix = f"[{status.upper()}] " if status != 'full' else ""
sections.append(f"<{name}>\n{prefix}{content}\n</{name}>")
return "\n\n".join(sections)
This allocator handles the fundamental problem of priority-ordered prompt assembly with graceful degradation. Critical components are always included, even if truncated. Lower-priority components are dropped when the budget is exhausted. Compressed versions are used when available.
Warning
Truncation of context is a last resort, not a strategy. When the allocator signals that a critical component was truncated, that's a signal that your task needs to be decomposed into smaller sub-tasks, not that truncation is acceptable. Monitor for truncation events in production — they indicate architectural debt.
In production, you want to track how much compression you're applying over time, because drift in compression ratios can indicate that your context is growing in ways that will eventually break your budget:
import logging
from datetime import datetime
class CompressionTracker:
def __init__(self, alert_threshold: float = 0.6):
"""Alert when compressed size is < alert_threshold of original"""
self.alert_threshold = alert_threshold
self.log = logging.getLogger("compression_tracker")
def record(self, task_id: str, component: str,
original_tokens: int, final_tokens: int,
status: str):
ratio = final_tokens / original_tokens if original_tokens > 0 else 1.0
record = {
"timestamp": datetime.utcnow().isoformat(),
"task_id": task_id,
"component": component,
"original_tokens": original_tokens,
"final_tokens": final_tokens,
"compression_ratio": ratio,
"status": status
}
if ratio < self.alert_threshold:
self.log.warning(
f"Heavy compression on {component}: "
f"{original_tokens}→{final_tokens} tokens ({ratio:.1%}). "
f"Consider decomposing this task."
)
return record
Compression is a tool with real costs. Understanding when not to compress is as important as the compression techniques themselves.
When you strip instructions to their absolute minimum, you can introduce ambiguity that the model resolves using its defaults, which may not match your intent. Watch for this pattern:
Over-compressed (ambiguous):
Extract all dates. Format: ISO 8601.
The model must decide: all dates anywhere in the document? Only dates associated with events? What if a date appears in a URL or a file version number? What if the same event has multiple dates (filing date vs. effective date)? These edge cases need some specification. The compressed version needs a few more tokens:
Appropriately compressed (precise):
Extract all event dates (not dates in URLs, file versions, or boilerplate).
Format: YYYY-MM-DD. If an event has multiple dates, include all with labels.
That's 32 tokens instead of 12, but those 20 extra tokens eliminate an entire class of failure modes.
If your task has subtle boundary conditions, compressed examples may not preserve enough detail to teach those boundaries. For legal document review, for instance, whether a clause is "present" vs. "referenced" vs. "implied" may be a distinction that requires a full, realistic example to demonstrate correctly. In those cases, keep your examples full-fidelity.
When you summarize documents as a pre-processing step, you're making a lossy transformation. Summaries can miss the nuances that matter for edge cases. For tasks where rare edge cases have high stakes — compliance analysis, contract review, medical record extraction — test your summarization step explicitly against a held-out set of edge cases before relying on it in production. Connect this to your AI evaluation framework to make this systematic.
Tip
Build a "compression oracle" test: for any compression technique you apply, take a sample of 20-30 cases, run them through both the full-context and compressed-context versions, and measure output divergence. If more than 10-15% of cases produce meaningfully different outputs, the compression is losing information that matters. This isn't a one-time check — re-run it whenever your source data distribution changes.
Here's a realistic enterprise prompt for a financial reporting automation task. Your job is to apply the techniques from this lesson to reduce its token count by at least 50% while preserving all functional requirements.
You are an expert financial analyst AI assistant who specializes in
analyzing quarterly earnings reports for enterprise companies. You have
deep expertise in financial metrics, accounting standards, and business
performance analysis.
I need you to analyze the following quarterly earnings report excerpt and
extract key financial metrics. Please be very careful and thorough in your
analysis. You should look for the following financial metrics: total revenue,
net revenue (which is sometimes different from total revenue), cost of goods
sold (which may also be labeled as cost of revenue), gross profit, operating
expenses (which can include sales and marketing, research and development,
and general and administrative expenses), operating income or operating loss,
EBITDA (earnings before interest, taxes, depreciation, and amortization),
net income or net loss, earnings per share (both basic and diluted), and
cash and cash equivalents at period end.
When extracting these figures, please note that figures may be presented in
different units — some companies report in thousands, others in millions,
others in billions. Please convert all figures to absolute dollar values in
your output. For example, if the report says "$24.7 million," you should
output 24700000.
If a metric is not present in the provided text, please set it to null in
your output rather than trying to calculate it or make it up. This is very
important — only extract metrics that are explicitly stated.
Please format your output as a JSON object. The JSON should be clean and
valid. Use snake_case for all key names.
In addition to the metrics, please also extract the reporting period. This
should include the fiscal quarter (Q1, Q2, Q3, or Q4), the fiscal year, and
the period end date in ISO 8601 format (YYYY-MM-DD).
Please double-check your work before responding and make sure all numbers
are accurate.
Here is the earnings report text to analyze:
{earnings_text}
Work through it yourself first, then compare to this:
Extract financial metrics from the earnings text below.
METRICS TO EXTRACT (snake_case keys, null if not found, no inference):
- total_revenue, net_revenue, cost_of_goods_sold, gross_profit
- operating_expenses (sum of: sales_marketing, rd, gna)
- operating_income, ebitda, net_income
- eps_basic, eps_diluted
- cash_and_equivalents
PERIOD (required):
- fiscal_quarter: Q1|Q2|Q3|Q4
- fiscal_year: YYYY
- period_end_date: YYYY-MM-DD
RULES:
- Convert all figures to absolute values (e.g., "$24.7M" → 24700000)
- If metric appears multiple times, use the figure from the primary statements
- Output: valid JSON only, no commentary
<earnings_text>
{earnings_text}
</earnings_text>
The reduction: 640 → 195 tokens (70% reduction). What was preserved: every metric, the null-for-missing rule, the no-inference constraint, unit normalization, the output format requirement, and the period extraction. What was cut: the persona description, the repetitive explanations, the hedging language, and redundant restatements of requirements.
Note
The persona ("expert financial analyst AI assistant") was removed deliberately. For well-scoped extraction tasks with precise instructions, personas add tokens without improving output quality. Personas earn their keep in tasks requiring consistent tone or judgment style — not mechanical extraction. If you want to explore when personas do add value, Grounding AI Responses with Business Context covers that tradeoff in depth.
In sophisticated enterprise workflows, compression shouldn't be a one-time prompt-editing exercise — it should be a dedicated architectural layer. Here's how this looks in practice for a contract analysis pipeline:
class ContractAnalysisPipeline:
"""
Multi-stage pipeline with explicit compression between stages.
"""
def __init__(self, client, model="claude-opus-4-5"):
self.client = client
self.model = model
self.allocator = PromptBudgetAllocator(total_budget=8000, output_reserve=2000)
def stage_1_extract_structure(self, raw_contract: str) -> dict:
"""Convert raw contract text to structured section index."""
prompt = """Extract the structural outline of this contract.
Output JSON:
{
"parties": ["list of party names and roles"],
"sections": [{"title": "...", "key_obligations": ["..."], "has_numbers": bool}],
"effective_date": "YYYY-MM-DD or null",
"governing_law": "jurisdiction or null"
}
No other text. Contract:
""" + raw_contract[:4000] # First 4K tokens for structure
response = self.client.messages.create(
model=self.model,
max_tokens=1000,
messages=[{"role": "user", "content": prompt}]
)
return response.content[0].text
def stage_2_extract_obligations(self, structure: dict,
raw_contract: str,
focus_party: str) -> list:
"""Extract only the obligations for a specific party."""
# Build a targeted prompt using the structure from stage 1
relevant_sections = [
s for s in structure.get('sections', [])
if any(focus_party.lower() in ob.lower()
for ob in s.get('key_obligations', []))
]
section_titles = [s['title'] for s in relevant_sections]
prompt = f"""Extract all obligations for "{focus_party}" from this contract.
Focus sections (others may be skipped): {', '.join(section_titles)}
Output JSON array of obligations:
[{{"obligation": "...", "deadline": "...|ongoing|null",
"section": "...", "conditional": bool}}]
Contract:
{raw_contract}"""
response = self.client.messages.create(
model=self.model,
max_tokens=2000,
messages=[{"role": "user", "content": prompt}]
)
return response.content[0].text
def stage_3_risk_summary(self, obligations: list,
company_standards: str) -> str:
"""Produce compressed risk assessment from structured obligations."""
# At this point, obligations is already compressed — it's structured JSON
# from stage 2, not raw contract text. This is the compression dividend.
compressed_obligations = "\n".join([
f"- {ob['obligation']} (due: {ob['deadline']}, "
f"section: {ob['section']})"
for ob in obligations
])
prompt = f"""Compare these contractual obligations against company standards.
OBLIGATIONS:
{compressed_obligations}
STANDARDS:
{company_standards}
Output:
- GAPS: obligations that exceed standard risk tolerance
- ACCEPTABLE: obligations within standards
- AMBIGUOUS: obligations needing legal review
One bullet per item. Be specific about which standard each gap violates."""
response = self.client.messages.create(
model=self.model,
max_tokens=1500,
messages=[{"role": "user", "content": prompt}]
)
return response.content[0].text
Notice what's happening across stages: the raw contract (potentially 50K+ tokens) is progressively compressed into structured JSON, then into a filtered obligation list, then into a gap analysis. Each stage's input is the compressed output of the previous stage. The final risk assessment prompt is working with perhaps 2-3K tokens derived from a document that would never have fit in a single call — and the analysis is more focused and accurate than a naive full-document approach would produce.
For cost implications of this pattern — because those intermediate API calls add up — see Cost Optimization for AI API Usage: Managing Tokens, Model Tiers, and Caching Strategies.
The most dangerous pattern is optimizing your prompt for brevity and deploying it without regression testing. Compression changes the model's behavior in subtle ways. Always run your compressed prompt against a benchmark set of inputs with known correct outputs before deploying.
Fix: Maintain a golden test set for every production prompt. After any compression pass, measure recall, precision, and format compliance against the test set before shipping.
Practitioners often over-compress the examples (which are high-value) while leaving verbose persona descriptions and boilerplate headers intact (which are low-value). The compression effort should be proportional to the information density of each section.
Fix: Token-count each section of your prompt separately. Focus compression effort on the sections that consume the most tokens relative to the information they carry.
Writing a carefully ordered prompt and then pasting in a large document in the middle, burying your critical constraints in the exact zone where attention weight is lowest.
Fix: Structure your prompts so that critical instructions appear at the top (before any large context injection) and are reiterated — even briefly — immediately before the output request at the bottom. Use the primacy-recency effect deliberately.
Your task requirements evolve, your data distributions shift, and your models get updated. A compression strategy that works today may underperform in three months as the underlying model changes.
Fix: Make compression ratio and output quality two separate tracked metrics in your monitoring system. When output quality drops but compression ratios haven't changed, that's a model drift signal. When compression ratios change without an intentional prompt edit, investigate data distribution drift.
Multi-stage compression approaches require additional API calls. If you run a summarization pre-process on every document and each pre-process call costs $0.02, that adds up quickly at scale.
Fix: Cache aggressively. Summarization of static documents (contracts, policy documents, schemas) is a pure function — the same input always produces equivalent output. Cache these summaries and invalidate only when the source document changes.
Prompt compression is a discipline that sits at the intersection of linguistics, information theory, and LLM architecture. The practitioners who do it well aren't just editing prompts — they're thinking about signal density, attention mechanics, and system architecture simultaneously.
Here's the framework to take away:
The deepest insight to carry forward is this: compression forces you to understand your own task requirements with unusual precision. If you can't compress your instructions, it's often because you haven't fully specified what you actually want. The exercise of compression is, itself, a requirement-clarification exercise.
To extend this learning:
PromptBudgetAllocator pattern and integrate it into an existing workflow you ownThe context window is not a bucket. It's a bandwidth-constrained communication channel. Treat every token as a scarce resource, and your prompts — and your outputs — will be dramatically better for it.