Most agent frameworks conflate planning with execution — and that's why they fail on complex tasks. This lesson teaches you to build a plan-and-execute agent from scratch: a planner that generates structured, dependency-aware task plans, an executor that handles one focused step at a time, and a replanner that recovers gracefully from failure.

Imagine you've built a ReAct-style agent to help your data team answer complex analytical questions. It works beautifully on simple queries. Then someone asks it to "analyze Q3 sales performance, identify the top three underperforming regions, pull the relevant customer segment data, generate a comparison against Q2, and draft an executive summary with recommendations." Your agent starts executing, hits a tool call, gets distracted by an intermediate result, loops back on itself, runs out of context window, and returns something halfway useful after burning through 40,000 tokens and 90 seconds. Sound familiar?
The core problem is architectural: most agent frameworks conflate thinking about what to do with actually doing it. When planning and execution share the same inference loop, you get agents that are reactive instead of strategic. Each tool call response nudges the LLM's attention in a new direction, making it easy for long-horizon tasks to drift, stall, or hallucinate a completed plan that was never really followed. The plan-and-execute pattern breaks this coupling deliberately — you generate a full plan first, then execute each step with a focused executor that doesn't need to hold the entire task in working memory.
By the end of this lesson, you'll have a production-grade plan-and-execute agent you built yourself, and you'll understand the design decisions deeply enough to adapt it to your own domain. This isn't a framework tour; it's an architecture lesson.
What you'll learn:
You should be comfortable with:
You don't need LangGraph, LangChain, or any agent framework. We'll build from first principles using the OpenAI API and plain Python.
Before building the solution, let's precisely diagnose the problem. ReAct agents operate in a tight loop:
Thought → Action → Observation → Thought → Action → ...
Each iteration, the entire conversation history — thoughts, tool calls, observations — gets fed back into the LLM. This has two compounding failure modes on complex tasks.
Attention diffusion. LLMs are attention mechanisms. As the context grows, early instructions get diluted. An agent that starts with a 15-step goal will, by step 8, be primarily attending to the recent observations and tool outputs. The original goal becomes a faint signal. You end up with an agent that's highly reactive to what just happened rather than what it set out to accomplish.
No global task state. A ReAct agent has no explicit representation of "what's left to do." If step 4 fails, the agent has to infer from the conversation history that it needs to retry or pivot. This inference is unreliable, especially when the history contains confusing intermediate outputs. You get agents that silently skip steps or redo work already completed.
Context window pressure. A complex task with many tool calls rapidly fills the context window. When you're forced to truncate history, you lose the scaffolding the agent needs to reason coherently. Understanding how token limits shape system design makes it clear why this is a structural constraint, not just a quota problem.
The plan-and-execute architecture addresses all three failure modes with a single insight: planning is a different cognitive task than executing. A planner needs breadth — it must see the whole problem and generate a coherent sequence. An executor needs focus — it should do exactly one thing, with full attention on that one thing, and report results clearly. Keeping these concerns separate lets you optimize each independently.
Key insight
The plan-and-execute pattern isn't just a workflow pattern — it's a cognitive architecture choice. You're essentially creating two different agents with different system prompts, different context requirements, and different evaluation criteria. This separation is the whole point.
Here's the full system we'll implement:
User Request
│
▼
┌─────────────┐
│ PLANNER │ ← GPT-4 with structured output
│ (one shot) │ → Produces: Plan (list of Steps)
└─────────────┘
│
▼
┌──────────────────────────────┐
│ EXECUTOR LOOP │
│ for each step in plan: │
│ ┌───────────────────┐ │
│ │ Step Executor │ ←── Tool registry
│ │ (focused LLM) │ │
│ └───────────────────┘ │
│ │ │
│ Result Store │
│ (step outputs) │
│ │ │
│ ┌──────▼──────┐ │
│ │ Replanner │ ← optional│
│ │ (on fail) │ │
│ └─────────────┘ │
└──────────────────────────────┘
│
▼
┌──────────────┐
│ SYNTHESIZER │ ← Final LLM call with all step results
└──────────────┘
│
▼
Final Answer
Three distinct LLM calls, each with a focused job:
The replanner is optional but essential for production — it activates when a step fails or produces results that invalidate downstream assumptions.
The plan is the contract between the planner and the executor. It needs to be machine-readable, verifiable, and expressive enough to capture dependencies. We'll use Pydantic for schema enforcement.
from __future__ import annotations
from enum import Enum
from typing import Optional
from pydantic import BaseModel, Field
class StepStatus(str, Enum):
PENDING = "pending"
RUNNING = "running"
COMPLETED = "completed"
FAILED = "failed"
SKIPPED = "skipped"
class PlanStep(BaseModel):
step_id: str = Field(
description="Unique identifier like 'step_1', 'step_2'"
)
description: str = Field(
description="Human-readable description of what this step does"
)
tool_name: str = Field(
description="The name of the tool to invoke for this step"
)
tool_input_template: str = Field(
description=(
"The input to pass to the tool. May reference prior step outputs "
"using {step_1.result}, {step_2.result} syntax"
)
)
depends_on: list[str] = Field(
default_factory=list,
description="List of step_ids that must complete before this step runs"
)
is_critical: bool = Field(
default=True,
description="If True and this step fails, abort the plan. If False, continue."
)
status: StepStatus = Field(default=StepStatus.PENDING)
result: Optional[str] = Field(default=None)
error: Optional[str] = Field(default=None)
class ExecutionPlan(BaseModel):
task_description: str = Field(
description="The original user task being planned for"
)
steps: list[PlanStep] = Field(
description="Ordered list of steps to execute"
)
reasoning: str = Field(
description="Brief explanation of why this plan was constructed this way"
)
def get_step(self, step_id: str) -> Optional[PlanStep]:
return next((s for s in self.steps if s.step_id == step_id), None)
def get_ready_steps(self) -> list[PlanStep]:
"""Return steps that are PENDING and have all dependencies completed."""
ready = []
for step in self.steps:
if step.status != StepStatus.PENDING:
continue
deps_satisfied = all(
(dep_step := self.get_step(dep_id)) is not None
and dep_step.status == StepStatus.COMPLETED
for dep_id in step.depends_on
)
if deps_satisfied:
ready.append(step)
return ready
def is_complete(self) -> bool:
return all(
s.status in (StepStatus.COMPLETED, StepStatus.SKIPPED)
for s in self.steps
)
def has_failed(self) -> bool:
return any(
s.status == StepStatus.FAILED and s.is_critical
for s in self.steps
)
def get_completed_results(self) -> dict[str, str]:
return {
s.step_id: s.result
for s in self.steps
if s.status == StepStatus.COMPLETED and s.result
}
Notice the depends_on field. This is a DAG (directed acyclic graph) dependency model — it allows steps to run in parallel when their dependencies are satisfied, and it makes the execution order explicit and verifiable. The is_critical flag gives you a way to handle optional steps gracefully without aborting the entire plan.
The get_ready_steps() method is the execution engine's heartbeat — it answers "what can I do right now?" without requiring any LLM reasoning. This is intentional: dependency tracking is a pure programming concern, not an AI reasoning concern.
Tip
Always validate the plan for cycles before executing it. A planner that generates a cycle (step_1 depends on step_3, step_3 depends on step_1) will cause your executor to deadlock. Add a topological sort check in your planner output validation.
The planner's job is a single, focused LLM call. It sees the user's task, the available tools, and nothing else. Its output must be a valid ExecutionPlan.
import json
from openai import OpenAI
client = OpenAI()
PLANNER_SYSTEM_PROMPT = """You are a task planning agent. Your job is to decompose a complex user request into a precise, executable sequence of steps.
RULES:
1. Each step must use exactly one tool from the provided tool list.
2. Steps that depend on results from earlier steps must list those step IDs in `depends_on`.
3. Steps without dependencies can run in parallel — only add dependencies when genuinely required.
4. Keep steps atomic: one tool call, one clear output.
5. Set `is_critical: false` only for steps that are "nice to have" — the task can succeed without them.
6. Use the `tool_input_template` to express how prior results feed into this step, using {step_N.result} placeholders.
7. Your reasoning should explain the key sequencing decisions, not restate the steps.
You must respond with valid JSON matching the ExecutionPlan schema. Do not include prose outside the JSON."""
def build_tool_descriptions(tools: dict) -> str:
"""Format tool registry for inclusion in the planner prompt."""
lines = []
for name, tool in tools.items():
lines.append(f"- {name}: {tool['description']}")
if 'parameters' in tool:
lines.append(f" Parameters: {json.dumps(tool['parameters'])}")
return "\n".join(lines)
def create_plan(task: str, tools: dict) -> ExecutionPlan:
"""Call the planner LLM and return a validated ExecutionPlan."""
tool_descriptions = build_tool_descriptions(tools)
user_message = f"""Task: {task}
Available tools:
{tool_descriptions}
Generate an execution plan as JSON."""
response = client.chat.completions.create(
model="gpt-4o",
messages=[
{"role": "system", "content": PLANNER_SYSTEM_PROMPT},
{"role": "user", "content": user_message}
],
response_format={"type": "json_object"},
temperature=0.1, # Low temperature for deterministic planning
)
raw_json = response.choices[0].message.content
plan_data = json.loads(raw_json)
plan = ExecutionPlan(**plan_data)
# Validate no cycles
_validate_dag(plan)
return plan
def _validate_dag(plan: ExecutionPlan) -> None:
"""Topological sort to detect dependency cycles."""
step_ids = {s.step_id for s in plan.steps}
# Check all dependencies reference valid steps
for step in plan.steps:
for dep in step.depends_on:
if dep not in step_ids:
raise ValueError(
f"Step {step.step_id} depends on {dep}, which doesn't exist in the plan"
)
# Kahn's algorithm for cycle detection
in_degree = {s.step_id: 0 for s in plan.steps}
adjacency = {s.step_id: [] for s in plan.steps}
for step in plan.steps:
for dep in step.depends_on:
adjacency[dep].append(step.step_id)
in_degree[step.step_id] += 1
queue = [sid for sid, deg in in_degree.items() if deg == 0]
visited_count = 0
while queue:
node = queue.pop(0)
visited_count += 1
for neighbor in adjacency[node]:
in_degree[neighbor] -= 1
if in_degree[neighbor] == 0:
queue.append(neighbor)
if visited_count != len(plan.steps):
raise ValueError("Execution plan contains a dependency cycle. Plan is not executable.")
The key design choices here:
Temperature 0.1 for planning. You want deterministic, structured output from the planner. High temperature produces creative but inconsistent plans. This isn't the place for creativity — that's the executor's domain when it has to handle unexpected tool outputs.
Separate system prompt. The planner prompt contains no mention of how execution works, what the user experience looks like, or what happens after the plan. It's purely focused on decomposition. Mixing concerns into the planner prompt degrades plan quality.
Explicit tool descriptions. The planner needs to know what tools exist and what they do in order to assign them to steps. Don't pass the full tool schemas here — just names and descriptions. Full schemas go to the executor.
The executor is where the real work happens. Each call to the executor does exactly one thing: execute a single step by calling the appropriate tool with the appropriate inputs, with prior step results interpolated.
import re
from typing import Callable, Any
EXECUTOR_SYSTEM_PROMPT = """You are a precise task executor. You will be given a single step to execute and the results of any prior steps this step depends on.
Your job:
1. Call the specified tool with the correct input
2. The tool input may contain placeholders like {step_1.result} — these have already been filled in for you
3. Report the result clearly and concisely
4. If the tool fails, report the error accurately — do not guess or fabricate results
Execute exactly what is specified. Do not add extra tool calls or deviate from the step."""
def interpolate_template(template: str, completed_results: dict[str, str]) -> str:
"""Replace {step_N.result} placeholders with actual results from prior steps."""
def replacer(match):
step_id = match.group(1)
if step_id not in completed_results:
raise ValueError(
f"Template references {step_id} but that step has no result yet. "
f"This suggests a dependency was not declared."
)
return completed_results[step_id]
return re.sub(r'\{(step_\d+)\.result\}', replacer, template)
async def execute_step(
step: PlanStep,
plan: ExecutionPlan,
tools: dict,
tool_functions: dict[str, Callable]
) -> tuple[bool, str]:
"""
Execute a single plan step.
Returns (success: bool, result_or_error: str)
"""
# Interpolate any references to prior step results
completed_results = plan.get_completed_results()
try:
interpolated_input = interpolate_template(
step.tool_input_template,
completed_results
)
except ValueError as e:
return False, f"Template interpolation failed: {e}"
# Build the executor context
dep_context = ""
if step.depends_on:
dep_lines = []
for dep_id in step.depends_on:
dep_step = plan.get_step(dep_id)
if dep_step and dep_step.result:
dep_lines.append(f"{dep_id} result: {dep_step.result}")
dep_context = "\nPrior step results:\n" + "\n".join(dep_lines)
user_message = f"""Execute this step:
Step ID: {step.step_id}
Description: {step.description}
Tool to use: {step.tool_name}
Tool input: {interpolated_input}
{dep_context}
Call the tool now and report the result."""
# Build tool schema for this specific step (only the tool we need)
tool_schema = tools.get(step.tool_name, {})
openai_function = {
"type": "function",
"function": {
"name": step.tool_name,
"description": tool_schema.get("description", ""),
"parameters": tool_schema.get("parameters", {
"type": "object",
"properties": {
"input": {"type": "string"}
},
"required": ["input"]
})
}
}
messages = [
{"role": "system", "content": EXECUTOR_SYSTEM_PROMPT},
{"role": "user", "content": user_message}
]
# First LLM call: get tool call
response = client.chat.completions.create(
model="gpt-4o-mini", # Executor can use a cheaper model
messages=messages,
tools=[openai_function],
tool_choice={"type": "function", "function": {"name": step.tool_name}},
temperature=0,
)
tool_call = response.choices[0].message.tool_calls[0]
tool_args = json.loads(tool_call.function.arguments)
# Actually call the tool
if step.tool_name not in tool_functions:
return False, f"Tool '{step.tool_name}' not found in tool registry"
try:
tool_result = await tool_functions[step.tool_name](**tool_args)
tool_result_str = str(tool_result)
except Exception as e:
return False, f"Tool execution failed: {type(e).__name__}: {e}"
# Second LLM call: interpret result and produce clean output
messages.append(response.choices[0].message)
messages.append({
"role": "tool",
"tool_call_id": tool_call.id,
"content": tool_result_str
})
final_response = client.chat.completions.create(
model="gpt-4o-mini",
messages=messages,
temperature=0,
)
result_summary = final_response.choices[0].message.content
return True, result_summary
Notice two important choices here:
Use gpt-4o-mini for execution, gpt-4o for planning. The planner needs sophisticated reasoning to decompose the task correctly. The executor needs to do one focused thing. Using a smaller, faster, cheaper model for execution dramatically reduces cost and latency at scale while maintaining quality where it matters.
Force the tool call. We set tool_choice={"type": "function", "function": {"name": step.tool_name}}. This prevents the executor from deciding on its own to use a different tool or to not call a tool at all. The planner already decided which tool to use — the executor's job is to call it correctly.
Warning
Don't pass all available tools to the executor. Pass only the tool for the current step. When an executor sees 15 available tools, it may "helpfully" make multiple tool calls or choose a different tool it thinks is better. The planner's decision about which tool to use should be final.
Now we wire together the plan execution. Steps without dependencies can run concurrently:
import asyncio
async def execute_plan(
plan: ExecutionPlan,
tools: dict,
tool_functions: dict[str, Callable],
max_parallel: int = 3,
on_step_complete: Callable | None = None
) -> ExecutionPlan:
"""
Execute all steps in the plan, respecting dependencies and
running independent steps in parallel.
"""
while not plan.is_complete() and not plan.has_failed():
ready_steps = plan.get_ready_steps()
if not ready_steps:
# No ready steps but plan isn't complete: deadlock
# This shouldn't happen if DAG validation passed, but guard anyway
pending = [s for s in plan.steps if s.status == StepStatus.PENDING]
if pending:
raise RuntimeError(
f"Execution deadlocked: {len(pending)} steps pending but none are ready. "
f"Check dependency declarations."
)
break
# Cap parallelism to avoid overwhelming rate limits
batch = ready_steps[:max_parallel]
# Mark all batch steps as running
for step in batch:
step.status = StepStatus.RUNNING
# Execute batch in parallel
tasks = [
execute_step(step, plan, tools, tool_functions)
for step in batch
]
results = await asyncio.gather(*tasks, return_exceptions=True)
# Process results
for step, result in zip(batch, results):
if isinstance(result, Exception):
step.status = StepStatus.FAILED
step.error = str(result)
print(f"[FAILED] {step.step_id}: {step.error}")
else:
success, output = result
if success:
step.status = StepStatus.COMPLETED
step.result = output
print(f"[DONE] {step.step_id}: {output[:100]}...")
else:
step.status = StepStatus.FAILED
step.error = output
print(f"[FAILED] {step.step_id}: {output}")
# If non-critical, mark dependents as skipped
if not step.is_critical:
_cascade_skip(plan, step.step_id)
if on_step_complete:
on_step_complete(step)
return plan
def _cascade_skip(plan: ExecutionPlan, failed_step_id: str) -> None:
"""Mark all steps that depend on a failed non-critical step as SKIPPED."""
for step in plan.steps:
if failed_step_id in step.depends_on and step.status == StepStatus.PENDING:
step.status = StepStatus.SKIPPED
step.error = f"Skipped because dependency {failed_step_id} failed"
# Cascade skip further
_cascade_skip(plan, step.step_id)
The parallel execution approach here is worth unpacking. get_ready_steps() returns all steps whose dependencies are satisfied. We execute up to max_parallel of them concurrently using asyncio.gather. After each batch, we re-evaluate what's ready. This naturally handles fan-out (one step unlocking multiple independent steps) and fan-in (multiple steps that must all complete before the next one can run).
Tip
Set max_parallel based on your LLM provider's rate limits, not based on the number of steps. At three to five concurrent requests, you'll stay well clear of most rate limit windows. For tool calls that hit external APIs, you may need to reduce this further or add per-tool rate limiting.
A rigid plan is a fragile plan. Real data tasks often reveal information during execution that the planner couldn't have known upfront. Maybe step 2 discovers there's no Q2 data available, making the Q2-vs-Q3 comparison impossible. Maybe the sales data contains an unexpected field that opens up a better analysis angle.
The replanner handles this by examining the original task, the completed steps' results, and the remaining plan, then deciding whether to revise:
REPLANNER_SYSTEM_PROMPT = """You are a plan revision agent. A task is in progress and an issue has occurred or new information has emerged that may require revising the remaining plan.
You will receive:
1. The original task
2. The completed steps and their results
3. The step that failed or produced unexpected results
4. The remaining (not yet executed) steps
Your job: determine whether the remaining plan is still valid given what we now know.
Options:
- If the plan is still valid: return {"action": "continue", "revised_steps": null}
- If steps need modification: return {"action": "revise", "revised_steps": [...modified steps...]}
- If the task is now impossible: return {"action": "abort", "reason": "explanation"}
When revising, only modify the PENDING steps. Do not change completed steps."""
async def maybe_replan(
plan: ExecutionPlan,
failed_step: PlanStep,
tools: dict
) -> ExecutionPlan | None:
"""
Invoke the replanner when a critical step fails.
Returns a revised plan, or None if the task should abort.
"""
completed_summary = "\n".join([
f"{s.step_id} ({s.status}): {s.result or s.error}"
for s in plan.steps
if s.status in (StepStatus.COMPLETED, StepStatus.FAILED)
])
pending_steps = [s for s in plan.steps if s.status == StepStatus.PENDING]
pending_summary = json.dumps(
[s.model_dump(exclude={'status', 'result', 'error'}) for s in pending_steps],
indent=2
)
user_message = f"""Original task: {plan.task_description}
Completed steps:
{completed_summary}
Failed step: {failed_step.step_id}
Failure reason: {failed_step.error}
Remaining pending steps:
{pending_summary}
Available tools: {', '.join(tools.keys())}
Should we revise the remaining plan?"""
response = client.chat.completions.create(
model="gpt-4o",
messages=[
{"role": "system", "content": REPLANNER_SYSTEM_PROMPT},
{"role": "user", "content": user_message}
],
response_format={"type": "json_object"},
temperature=0.1,
)
decision = json.loads(response.choices[0].message.content)
if decision["action"] == "abort":
print(f"Replanner decided to abort: {decision.get('reason')}")
return None
if decision["action"] == "revise" and decision.get("revised_steps"):
# Replace pending steps with revised steps
completed_steps = [s for s in plan.steps if s.status != StepStatus.PENDING]
revised_pending = [PlanStep(**s) for s in decision["revised_steps"]]
plan.steps = completed_steps + revised_pending
_validate_dag(plan)
print(f"Plan revised: {len(revised_pending)} new steps replacing {len(pending_steps)} pending steps")
return plan
You'll notice the replanner uses gpt-4o rather than the cheaper model — this is deliberate. Replanning requires the same quality of strategic reasoning as the initial planning. This call is rare (only triggered on critical failures) and consequential, so cost is worth it.
The replanning pattern connects to what we explore in Agentic RAG: Building Self-Correcting Retrieval Pipelines — the core insight is the same: your system should be able to reflect on failure and adapt, rather than just error out.
Once all steps are complete (or the plan has executed as far as it can), the synthesizer produces the final answer. This is the only place in the system where the LLM sees all step outputs together.
SYNTHESIZER_SYSTEM_PROMPT = """You are a results synthesizer. You have been given the outputs of all steps that were executed as part of completing a user's task.
Your job: produce a coherent, accurate, and complete final response to the original user request.
IMPORTANT:
- Use only the information from the step results. Do not add information from your training data that wasn't surfaced in the execution.
- If some steps were skipped or failed, acknowledge the gap in your response rather than fabricating data.
- Format the response appropriately for the task (use markdown headers, tables, bullet points as needed).
- Be specific with numbers, names, and facts from the results."""
def synthesize_results(
task: str,
plan: ExecutionPlan
) -> str:
"""Generate the final user-facing response from all step results."""
step_results = []
for step in plan.steps:
if step.status == StepStatus.COMPLETED:
step_results.append(f"**{step.step_id}** - {step.description}:\n{step.result}")
elif step.status == StepStatus.FAILED:
step_results.append(f"**{step.step_id}** - {step.description}: FAILED - {step.error}")
elif step.status == StepStatus.SKIPPED:
step_results.append(f"**{step.step_id}** - {step.description}: SKIPPED (dependency failed)")
results_block = "\n\n".join(step_results)
response = client.chat.completions.create(
model="gpt-4o",
messages=[
{"role": "system", "content": SYNTHESIZER_SYSTEM_PROMPT},
{
"role": "user",
"content": f"Original task: {task}\n\nStep results:\n\n{results_block}\n\nPlease synthesize a final response."
}
],
temperature=0.3, # Slightly higher: we want coherent prose, not robotic output
)
return response.choices[0].message.content
Note
The synthesizer instruction to not use training data not surfaced during execution is crucial for grounding. Without it, the synthesizer will confidently fill in gaps from parametric memory — which is exactly what you're trying to avoid by running the retrieval steps in the first place. This connects to the broader principle of prompt engineering for RAG: ground the LLM's responses in retrieved context, not in what it already "knows."
Here's the top-level orchestrator that ties everything together:
async def run_plan_and_execute_agent(
task: str,
tools: dict,
tool_functions: dict[str, Callable],
enable_replanning: bool = True
) -> dict:
"""
Full plan-and-execute agent run.
Returns a dict with the final answer, the plan, and execution metadata.
"""
print(f"\n{'='*60}")
print(f"TASK: {task}")
print(f"{'='*60}\n")
# Phase 1: Planning
print("Phase 1: Planning...")
plan = create_plan(task, tools)
print(f"Generated {len(plan.steps)}-step plan:")
for step in plan.steps:
deps = f" (depends on: {', '.join(step.depends_on)})" if step.depends_on else ""
print(f" {step.step_id}: {step.description}{deps}")
print(f"\nPlanner reasoning: {plan.reasoning}\n")
# Phase 2: Execution
print("Phase 2: Executing...")
execution_complete = False
replan_attempts = 0
max_replan_attempts = 2
while not execution_complete:
plan = await execute_plan(plan, tools, tool_functions)
if plan.has_failed() and enable_replanning and replan_attempts < max_replan_attempts:
# Find the failed critical step
failed_step = next(
(s for s in plan.steps if s.status == StepStatus.FAILED and s.is_critical),
None
)
if failed_step:
print(f"\nCritical step failed: {failed_step.step_id}. Attempting replan...")
replan_attempts += 1
revised_plan = await maybe_replan(plan, failed_step, tools)
if revised_plan is None:
print("Replanner decided the task is unachievable. Proceeding to synthesis.")
execution_complete = True
else:
plan = revised_plan
print(f"Replan {replan_attempts} complete. Resuming execution.")
else:
execution_complete = True
else:
execution_complete = True
# Phase 3: Synthesis
print("\nPhase 3: Synthesizing results...")
final_answer = synthesize_results(task, plan)
# Compile metadata
completed = sum(1 for s in plan.steps if s.status == StepStatus.COMPLETED)
failed = sum(1 for s in plan.steps if s.status == StepStatus.FAILED)
skipped = sum(1 for s in plan.steps if s.status == StepStatus.SKIPPED)
return {
"task": task,
"answer": final_answer,
"plan": plan.model_dump(),
"metadata": {
"total_steps": len(plan.steps),
"completed": completed,
"failed": failed,
"skipped": skipped,
"replan_attempts": replan_attempts
}
}
This orchestrator caps replan attempts at two. After that, it proceeds to synthesis with whatever results are available. This prevents runaway replanning loops while still allowing recovery from transient failures.
For multi-agent orchestration scenarios, you'd extend this pattern so the executor for each step could itself be a plan-and-execute sub-agent — a recursive composition that handles tasks requiring nested decomposition.
Let's build a concrete example: a business intelligence agent that analyzes sales data.
# Tool definitions (what the planner sees)
TOOLS = {
"query_sales_database": {
"description": "Query the sales database with a SQL-like filter and return aggregated results",
"parameters": {
"type": "object",
"properties": {
"query": {"type": "string", "description": "Natural language query or SQL fragment"},
"time_period": {"type": "string", "description": "e.g., 'Q3 2024', '2024-07-01 to 2024-09-30'"},
"group_by": {"type": "string", "description": "Dimension to group by: region, product, segment"}
},
"required": ["query", "time_period"]
}
},
"retrieve_customer_segments": {
"description": "Retrieve customer segment composition and size for a given region",
"parameters": {
"type": "object",
"properties": {
"region": {"type": "string"},
"include_metrics": {"type": "boolean", "default": True}
},
"required": ["region"]
}
},
"calculate_variance": {
"description": "Calculate percentage variance between two numeric values",
"parameters": {
"type": "object",
"properties": {
"baseline": {"type": "number"},
"current": {"type": "number"},
"label": {"type": "string"}
},
"required": ["baseline", "current", "label"]
}
},
"generate_recommendation": {
"description": "Generate strategic recommendations based on performance data and customer insights",
"parameters": {
"type": "object",
"properties": {
"performance_data": {"type": "string"},
"customer_data": {"type": "string"},
"focus_area": {"type": "string"}
},
"required": ["performance_data"]
}
}
}
# Mock tool implementations (replace with real data connections)
async def query_sales_database(query: str, time_period: str, group_by: str = None) -> dict:
# Simulate database response
return {
"period": time_period,
"results": [
{"region": "Northeast", "revenue": 2_340_000, "units": 1_240},
{"region": "Southeast", "revenue": 1_890_000, "units": 980},
{"region": "Midwest", "revenue": 1_120_000, "units": 620},
{"region": "Southwest", "revenue": 2_780_000, "units": 1_480},
{"region": "West", "revenue": 3_210_000, "units": 1_720},
],
"target_revenue": 2_500_000,
"target_units": 1_300
}
async def retrieve_customer_segments(region: str, include_metrics: bool = True) -> dict:
segments = {
"Midwest": {
"enterprise": 0.15, "mid_market": 0.35, "smb": 0.50,
"avg_contract_value": 42_000, "churn_rate": 0.18
}
}
return segments.get(region, {"error": f"No data for region: {region}"})
async def calculate_variance(baseline: float, current: float, label: str) -> dict:
pct = ((current - baseline) / baseline) * 100
return {"label": label, "baseline": baseline, "current": current,
"variance_pct": round(pct, 2), "direction": "above" if pct > 0 else "below"}
async def generate_recommendation(
performance_data: str, customer_data: str = None, focus_area: str = None
) -> str:
# In production, this would call an LLM or a specialized model
return f"Based on the analysis: Focus on enterprise segment expansion in underperforming regions, targeting mid-market accounts where churn rate exceeds 15%. Consider territory restructuring for the Midwest region."
TOOL_FUNCTIONS = {
"query_sales_database": query_sales_database,
"retrieve_customer_segments": retrieve_customer_segments,
"calculate_variance": calculate_variance,
"generate_recommendation": generate_recommendation
}
import asyncio
async def main():
task = (
"Analyze Q3 2024 sales performance by region. "
"Identify the bottom two underperforming regions against target. "
"For the worst-performing region, pull customer segment breakdown. "
"Calculate the revenue variance against target for each underperforming region. "
"Generate strategic recommendations for Q4."
)
result = await run_plan_and_execute_agent(
task=task,
tools=TOOLS,
tool_functions=TOOL_FUNCTIONS,
enable_replanning=True
)
print("\n" + "="*60)
print("FINAL ANSWER:")
print("="*60)
print(result["answer"])
print(f"\nExecution: {result['metadata']['completed']}/{result['metadata']['total_steps']} steps completed")
asyncio.run(main())
Watch the execution output carefully. You should see:
Experiment by temporarily making query_sales_database raise an exception and observe the replanner in action. Then try making it a non-critical step and observe cascading skips.
Symptom: The planner creates 20 steps for what should be a 5-step task, with steps like "retrieve data" and "confirm data was retrieved" as separate steps.
Cause: Unclear task description or planner prompt that rewards comprehensiveness over efficiency.
Fix: Add an explicit instruction: "Prefer 3-7 steps for most tasks. If you find yourself creating more than 10 steps, reconsider whether steps can be combined." Also, your planner prompt should tell it that tool calls themselves are atomic — there's no need to add verification steps after each one.
Symptom: Step 6 receives 15,000 tokens of intermediate data because step 2's result was an entire database dump.
Cause: Tools that return raw data rather than summarized results, combined with templates that reference entire prior results.
Fix: Instruct the executor to summarize tool results before they go into the result store. Add a maximum result length (e.g., 2000 characters) with truncation and a note that truncation occurred. This is where contextual compression principles apply — compress intermediate results before passing them downstream.
Symptom: The executor makes multiple tool calls or uses a different tool than specified.
Cause: Some model versions handle tool_choice differently. Older GPT-4 versions sometimes ignore it.
Fix: Add validation after the LLM call: assert tool_call.function.name == step.tool_name. If it fails, retry once with a more explicit system prompt message: "You MUST call only the tool named [tool_name]. Do not call any other tool."
Symptom: The agent replans, fails again on the same step, replans again, and burns through tokens indefinitely.
Cause: The failure is caused by a systematic issue (tool is broken, data doesn't exist) that replanning can't fix.
Fix: Cap replan attempts (we did this with max_replan_attempts = 2). Also, pass the failure history to the replanner: "This step has already been replanned once and failed again. Consider whether this sub-goal is achievable with the available tools."
Symptom: The synthesizer's output includes specific numbers, names, or claims that never appeared in any step result.
Cause: When step results are sparse or the synthesizer prompt doesn't explicitly ground the response, the LLM fills in from training data.
Fix: Add this to the synthesizer prompt: "If you are about to include a specific fact (number, name, date, claim) in your response, verify that it appears in the step results above. If it doesn't appear, omit it and note the information was not available." This is the same grounding discipline required in evaluating RAG systems for faithfulness.
Warning
Never let the synthesizer see the original tool schemas or the tool function implementations. If it knows what the tools could return, it'll sometimes confabulate plausible-looking results that fit the tool's output format. Keep the synthesizer's context strictly limited to actual execution results.
Symptom: Intermittent failures where a step reports missing results from a dependency that shows as completed.
Cause: If you're using a shared mutable plan object across coroutines, parallel writes to step status can race.
Fix: Collect all parallel results before writing back to the plan object. The pattern in our execute_plan function already does this — asyncio.gather returns all results before we process any of them. If you're using threading instead of asyncio, add a lock around plan state mutations.
Each step should emit structured telemetry. Use OpenTelemetry spans or a lightweight logging approach:
import time
@dataclass
class StepTelemetry:
step_id: str
tool_name: str
start_time: float
end_time: float
success: bool
input_tokens: int
output_tokens: int
error: str | None = None
@property
def duration_ms(self) -> float:
return (self.end_time - self.start_time) * 1000
Track this for every step and you'll quickly learn which tools are your latency bottlenecks, which steps fail most often, and how plan complexity correlates with overall task success. For production monitoring patterns, this telemetry becomes the foundation of your improvement loop.
For recurring task types with similar structure, the plan itself can be cached. If you're running the same analytical workflow daily ("analyze yesterday's sales against 7-day average"), the plan structure is likely identical — only the tool inputs change. Cache the plan template and parameterize it, skipping the planner LLM call entirely.
The plan-and-execute agent as built has no memory across separate runs. For workflows where context from previous executions matters, add an episodic memory layer that stores plan summaries and key results. The tool memory vs. retrieval memory patterns apply directly here — retrieval memory lets the planner look up "what did we do last time for this type of task" before generating a new plan.
Plan-and-execute shines for:
It's overkill for:
For genuinely exploratory tasks where each result fundamentally changes what you'd do next, a ReAct loop is the right tool. The best production systems often combine both: a plan-and-execute outer loop for the high-level workflow, with ReAct mini-loops inside individual executor steps that need to be adaptive.
Key insight
Plan-and-execute is a reliability pattern, not a capability pattern. It doesn't make your agent smarter — it makes it more predictable, auditable, and debuggable. If your priority is raw capability on open-ended tasks, ReAct is more flexible. If your priority is reliability and observability on well-defined tasks, plan-and-execute wins.
You've now built a production-grade plan-and-execute agent from scratch. Let's consolidate what we covered:
The architectural insight is that planning and execution are cognitively different tasks. Keeping them separate allows you to optimize each independently, use different model sizes where appropriate, and build a system that's auditable at every step.
The key components are:
The reliability mechanisms that matter most in production:
Where to go next:
If you're building multi-step workflows that involve retrieval, the plan-and-execute architecture pairs naturally with building multi-step AI agents with planning and memory — that lesson extends the memory model in ways that make your agent genuinely stateful across runs.
For agents that need tools beyond simple function calls — web search, code execution, external APIs — see tool-augmented AI agents, which covers the tool integration patterns that the executor in this lesson calls into.
And if your tasks involve complex multi-hop reasoning over documents, consider extending the planner to generate retrieval steps that use Graph RAG for multi-hop queries — the structured plan is a natural fit for orchestrating knowledge graph traversal alongside conventional tool calls.
The plan-and-execute pattern is a foundation. Once you have it running reliably, you'll find yourself reaching for it again and again — because a system where you can inspect every decision and every intermediate result is a system you can actually trust in production.