Wicked Smart Data
LearnInsightsAboutContact
Sign InLet's Build
LearnInsightsAboutContact
Sign InLet's Build
Wicked Smart Data

Intelligence, automation, and expert execution — plus an elite library of free knowledge. We turn complexity into competitive advantage.

Start a conversation

Platform

  • Learning Paths
  • Insights
  • RSS Feed

Company

  • About
  • Contact
  • Work With Us

Legal

  • Privacy Policy
  • Terms of Service

© 2026 Wicked Smart Data. All rights reserved.

Intelligence · Automation · Advantage

All Insights
AI & Machine Learning

Implementing Multi-Agent Handoff Protocols: Designing Explicit Task Delegation, Context Transfer, and Result Aggregation Across Specialized LLM Agents in Production

Single-agent LLM systems hit their limits fast. Learn how to design production-grade multi-agent architectures with explicit task delegation contracts, surgical context transfer, and principled result aggregation — including full Python implementations and observability patterns.

🔥 Expert26 min readNov 18, 2026Updated Nov 18, 2026
Implementing Multi-Agent Handoff Protocols: Designing Explicit Task Delegation, Context Transfer, and Result Aggregation Across Specialized LLM Agents in Production
On this page
  • Introduction
  • Prerequisites
  • The Architecture: Orchestrator-Agent Topology
  • Designing the Task Delegation Contract
  • Building the Context Transfer Layer
  • Implementing the Agent Layer
  • The Orchestrator: Task Decomposition and Routing
  • Result Aggregation: Where the System Delivers Value
  • Putting It Together: End-to-End Pipeline
  • Observability: You Can't Debug What You Can't See
  • Hands-On Exercise
  • Common Mistakes & Troubleshooting
  • Mistake 1: Using prose context instead of structured context
  • Mistake 2: Circular dependency deadlock
  • Mistake 3: Over-relying on LLM task decomposition
  • Mistake 4: Failing to handle partial results in synthesis
  • Mistake 5: Not capping agent recursion
  • Troubleshooting: Schema validation failures
  • Troubleshooting: Context growing too large
  • Summary & Next Steps

Implementing Multi-Agent Handoff Protocols: Designing Explicit Task Delegation, Context Transfer, and Result Aggregation Across Specialized LLM Agents in Production

Introduction

Imagine you're building an AI-powered investment research platform. A user asks: "Should I rebalance my portfolio given the recent Fed rate decision, my current holdings, and my tax situation this year?" That question spans at least three distinct domains — macroeconomic analysis, portfolio optimization, and tax strategy — and no single LLM agent can handle all of it well. If you throw the whole thing at one GPT-4o instance with an enormous system prompt trying to be everything at once, you'll get mediocre output across all three dimensions and a context window that's hemorrhaging tokens and money.

The solution isn't a bigger model. It's a better architecture. Multi-agent systems let you decompose complex tasks across specialized agents, each optimized for a specific slice of the problem, and then stitch their outputs back together into a coherent result. But "just call multiple agents" is not a protocol — it's a wish. What separates a production multi-agent system from a research demo is the rigor of three things: how tasks get delegated (who decides what goes where, with what instructions), how context gets transferred (what information travels with the task, in what format, with what guarantees), and how results get aggregated (how partial outputs are combined, reconciled, and delivered as a unified response).

By the end of this lesson, you'll know how to implement all three layers with production-grade code. You'll understand the design trade-offs involved, avoid the classic failure modes that kill multi-agent systems in real deployments, and have a working reference architecture you can adapt immediately.

What you'll learn:

  • How to design an orchestrator-agent topology with explicit task delegation contracts
  • How to build context transfer envelopes that preserve state across agent boundaries without ballooning token costs
  • How to implement typed result aggregation with conflict resolution and confidence scoring
  • How to handle agent failures, partial results, and retry logic within a multi-agent pipeline
  • How to instrument the system for observability so you can actually debug it in production

Prerequisites

You should be comfortable with:

  • Making API calls to LLM providers programmatically (see Using the OpenAI API with Python if you need a refresher)
  • Structured output from LLMs — specifically enforcing JSON schemas (Structured Output: Getting JSON, Tables, and Code from LLMs)
  • Basic function calling and tool use patterns
  • Async Python (asyncio, aiohttp or similar)

You'll also benefit from familiarity with the agentic loop design patterns covered elsewhere in this learning path.


The Architecture: Orchestrator-Agent Topology

Before writing a line of code, you need a mental model that holds up under real complexity. The pattern that works in production is the orchestrator-agent topology: a central orchestrator that understands the overall task and delegates to specialist agents, each of which operates within a well-defined scope.

This is distinct from two other common patterns you'll encounter:

Fully sequential chains (Agent A → Agent B → Agent C) are simple but brittle. Each agent in the chain has to consume and re-emit the full context, which causes drift, token waste, and cascading failures. If Agent B fails, the whole chain stops.

Peer-to-peer meshes (any agent can call any other agent) are flexible but quickly become debugging nightmares. When something goes wrong, you have no clear accountability chain, and coordination overhead scales quadratically.

The orchestrator-agent topology hits the sweet spot: the orchestrator maintains the global task plan and state, while agents maintain only local task state. Agents don't know about each other. They receive a task envelope, do their job, and return a result envelope. The orchestrator does all the routing, context management, and aggregation.

User Request
    │
    ▼
┌─────────────────┐
│   Orchestrator  │  ← Owns the task plan, routes, aggregates
│   (LLM + logic) │
└────────┬────────┘
         │  Delegates tasks (with context envelopes)
    ┌────┴─────────┬─────────────┐
    ▼              ▼             ▼
┌────────┐   ┌─────────┐   ┌──────────┐
│Macro   │   │Portfolio│   │Tax       │
│Analysis│   │Agent    │   │Strategy  │
│Agent   │   │         │   │Agent     │
└────────┘   └─────────┘   └──────────┘
    │              │             │
    └──────────────┴─────────────┘
                   │
            Result Envelopes
                   │
                   ▼
            ┌─────────────┐
            │  Aggregator │  ← May be part of orchestrator
            └─────────────┘
                   │
                   ▼
            Final Response

Key insight

The orchestrator doesn't have to be an LLM — and often shouldn't be for the routing layer. Use an LLM where you need judgment (task decomposition, conflict resolution, final synthesis). Use deterministic code where you need reliability (routing, schema validation, retry logic). The best production systems mix both.


Designing the Task Delegation Contract

The most common failure in multi-agent systems is implicit contracts: the orchestrator sends a vague task description, the agent makes assumptions, and the result doesn't fit what the orchestrator expected. This is exactly the kind of problem that disappears in demos (where everything is cherry-picked) and destroys production systems.

The solution is to define a Task Envelope — a typed structure that carries everything an agent needs to execute its task, unambiguously. Here's the schema:

from pydantic import BaseModel, Field
from typing import Any, Optional, Literal
from datetime import datetime
import uuid

class TaskContext(BaseModel):
    """Minimal context slice relevant to this specific task."""
    user_id: str
    session_id: str
    original_request: str
    relevant_facts: dict[str, Any] = Field(default_factory=dict)
    upstream_results: dict[str, Any] = Field(default_factory=dict)
    constraints: list[str] = Field(default_factory=list)

class TaskEnvelope(BaseModel):
    """The contract between orchestrator and agent."""
    task_id: str = Field(default_factory=lambda: str(uuid.uuid4()))
    agent_role: str  # e.g., "macro_analysis", "portfolio_optimization"
    task_description: str  # Specific, scoped instruction
    context: TaskContext
    output_schema: dict[str, Any]  # JSON Schema the agent must conform to
    priority: Literal["low", "medium", "high", "critical"] = "medium"
    timeout_seconds: int = 30
    max_tokens: int = 1500
    created_at: datetime = Field(default_factory=datetime.utcnow)
    parent_task_id: Optional[str] = None  # For hierarchical task trees
    allow_tool_use: bool = True
    metadata: dict[str, Any] = Field(default_factory=dict)

The critical fields here are output_schema and task_description. The output schema tells the agent exactly what format its result must take — not vaguely, but as a concrete JSON Schema that you'll validate against. The task description is scoped, not open-ended. Compare these two:

Bad (implicit): "Analyze the macroeconomic situation."

Good (explicit): "Given the Fed rate decision described in context.relevant_facts['fed_announcement'], assess: (1) the likely impact on 10-year Treasury yields over the next 90 days, (2) implications for growth vs. value equity allocation, and (3) one specific risk factor that might invalidate this analysis. Output must conform to the provided schema."

The difference isn't just stylistic. The explicit version gives the agent a bounded task it can actually complete, produces output in a predictable shape, and surfaces uncertainty explicitly. The vague version gives you a fluent essay that's nearly impossible to programmatically aggregate with other results.


Building the Context Transfer Layer

Context transfer is where most multi-agent systems quietly leak both money and coherence. The naive approach is to pass the entire conversation history to every agent. That's expensive (each agent re-processes the full history), introduces irrelevant information that distracts the agent, and often leaks context from one domain into another where it doesn't belong.

The right approach is surgical context extraction: the orchestrator maintains the full context, but each agent receives only the slice it needs. This requires deliberate design.

class ContextManager:
    """
    Manages the global task context and produces minimal slices
    for individual agents.
    """
    
    def __init__(self, session_id: str, user_id: str):
        self.session_id = session_id
        self.user_id = user_id
        self._full_context: dict[str, Any] = {}
        self._completed_results: dict[str, "ResultEnvelope"] = {}
        self._fact_index: dict[str, str] = {}  # fact_key -> source_task_id
    
    def ingest_user_context(self, structured_data: dict[str, Any]) -> None:
        """Store structured facts extracted from user input."""
        self._full_context.update(structured_data)
        for key in structured_data:
            self._fact_index[key] = "user_input"
    
    def register_result(self, result: "ResultEnvelope") -> None:
        """Store a completed agent result and index its outputs."""
        self._completed_results[result.task_id] = result
        for key, value in result.extracted_facts.items():
            self._full_context[key] = value
            self._fact_index[key] = result.task_id
    
    def build_context_slice(
        self, 
        agent_role: str,
        required_facts: list[str],
        upstream_task_ids: list[str]
    ) -> TaskContext:
        """
        Build a minimal context envelope for a specific agent.
        Only includes facts the agent actually needs.
        """
        relevant_facts = {}
        for fact_key in required_facts:
            if fact_key in self._full_context:
                relevant_facts[fact_key] = self._full_context[fact_key]
        
        upstream_results = {}
        for task_id in upstream_task_ids:
            if task_id in self._completed_results:
                result = self._completed_results[task_id]
                # Pass only the structured output, not raw LLM text
                upstream_results[task_id] = {
                    "agent_role": result.agent_role,
                    "summary": result.summary,
                    "structured_output": result.structured_output,
                    "confidence": result.confidence_score
                }
        
        return TaskContext(
            user_id=self.user_id,
            session_id=self.session_id,
            original_request=self._full_context.get("original_request", ""),
            relevant_facts=relevant_facts,
            upstream_results=upstream_results,
            constraints=self._full_context.get("user_constraints", [])
        )

Notice that when we pass upstream results to an agent, we pass the structured output — not the raw LLM-generated text. This is a crucial discipline. If Agent B receives Agent A's result as a wall of narrative prose, it'll re-process it through its own lens and potentially contradict it without flagging the conflict. If it receives structured fields with typed values, it has something it can actually act on deterministically.

Warning

Be especially careful about context contamination when agents have overlapping domains. If your tax agent receives full portfolio optimization results, it may try to give portfolio advice instead of staying in its lane. Scope the context slice aggressively. It's better to give an agent slightly less context and have it ask for more (via a tool call back to the orchestrator) than to pollute its reasoning with irrelevant information.


Implementing the Agent Layer

Each specialized agent follows the same interface regardless of what it does internally. This is what makes the system composable:

from openai import AsyncOpenAI
import json
import asyncio
from typing import Optional

class ResultEnvelope(BaseModel):
    task_id: str
    agent_role: str
    status: Literal["success", "partial", "failed", "escalated"]
    structured_output: dict[str, Any]
    summary: str  # 1-2 sentence human-readable summary
    extracted_facts: dict[str, Any]  # Facts others can build on
    confidence_score: float  # 0.0-1.0
    warnings: list[str] = Field(default_factory=list)
    error_detail: Optional[str] = None
    input_tokens: int = 0
    output_tokens: int = 0
    latency_ms: float = 0.0
    created_at: datetime = Field(default_factory=datetime.utcnow)


class SpecialistAgent:
    """
    Base class for all specialist agents. Handles the
    protocol layer; subclasses implement domain logic.
    """
    
    def __init__(self, role: str, system_prompt: str, client: AsyncOpenAI):
        self.role = role
        self.system_prompt = system_prompt
        self.client = client
    
    async def execute(self, envelope: TaskEnvelope) -> ResultEnvelope:
        import time
        start = time.monotonic()
        
        try:
            result = await self._run_with_timeout(envelope)
            result.latency_ms = (time.monotonic() - start) * 1000
            return result
        except asyncio.TimeoutError:
            return ResultEnvelope(
                task_id=envelope.task_id,
                agent_role=self.role,
                status="failed",
                structured_output={},
                summary="Agent timed out.",
                extracted_facts={},
                confidence_score=0.0,
                error_detail=f"Exceeded {envelope.timeout_seconds}s timeout",
                latency_ms=(time.monotonic() - start) * 1000
            )
        except Exception as e:
            return ResultEnvelope(
                task_id=envelope.task_id,
                agent_role=self.role,
                status="failed",
                structured_output={},
                summary="Agent execution failed.",
                extracted_facts={},
                confidence_score=0.0,
                error_detail=str(e),
                latency_ms=(time.monotonic() - start) * 1000
            )
    
    async def _run_with_timeout(self, envelope: TaskEnvelope) -> ResultEnvelope:
        return await asyncio.wait_for(
            self._execute_core(envelope),
            timeout=envelope.timeout_seconds
        )
    
    async def _execute_core(self, envelope: TaskEnvelope) -> ResultEnvelope:
        """Core execution logic — override this in subclasses for custom behavior."""
        
        user_message = self._build_user_message(envelope)
        
        response = await self.client.chat.completions.create(
            model="gpt-4o",
            messages=[
                {"role": "system", "content": self.system_prompt},
                {"role": "user", "content": user_message}
            ],
            response_format={"type": "json_object"},
            max_tokens=envelope.max_tokens,
            temperature=0.2  # Low temperature for consistent structured output
        )
        
        raw_output = response.choices[0].message.content
        parsed = json.loads(raw_output)
        
        # Validate against expected schema
        self._validate_output(parsed, envelope.output_schema)
        
        return ResultEnvelope(
            task_id=envelope.task_id,
            agent_role=self.role,
            status="success",
            structured_output=parsed,
            summary=parsed.get("summary", ""),
            extracted_facts=parsed.get("extracted_facts", {}),
            confidence_score=parsed.get("confidence_score", 0.8),
            warnings=parsed.get("warnings", []),
            input_tokens=response.usage.prompt_tokens,
            output_tokens=response.usage.completion_tokens
        )
    
    def _build_user_message(self, envelope: TaskEnvelope) -> str:
        context_str = json.dumps(envelope.context.model_dump(), indent=2)
        schema_str = json.dumps(envelope.output_schema, indent=2)
        
        return f"""Task: {envelope.task_description}

Context:
{context_str}

Required Output Schema:
{schema_str}

IMPORTANT: Your entire response must be valid JSON conforming exactly to the schema above.
Include a "confidence_score" (0.0-1.0), a "summary" (1-2 sentences), 
"extracted_facts" (key facts others can build on), and "warnings" (any caveats).
"""
    
    def _validate_output(self, output: dict, schema: dict) -> None:
        """Validate output against JSON schema. Raises ValueError on failure."""
        import jsonschema
        try:
            jsonschema.validate(instance=output, schema=schema)
        except jsonschema.ValidationError as e:
            raise ValueError(f"Agent output failed schema validation: {e.message}")

Tip

Setting temperature to 0.1–0.2 for structured-output agents is not just a convention — it's a correctness strategy. Higher temperatures increase the chance the model will deviate from your JSON schema, invent fields, or add commentary outside the structure. Save the creativity for synthesis tasks where variation is an asset, not a liability.


The Orchestrator: Task Decomposition and Routing

The orchestrator has the hardest job in the system. It needs to:

  1. Decompose the user's request into agent-specific tasks
  2. Determine execution order (what can run in parallel, what has dependencies)
  3. Build the context slice for each task
  4. Handle failures and partial results gracefully
  5. Trigger aggregation when tasks complete

Here's a production-grade orchestrator implementation:

from dataclasses import dataclass, field
from enum import Enum
import asyncio

class ExecutionStrategy(Enum):
    PARALLEL = "parallel"
    SEQUENTIAL = "sequential"
    CONDITIONAL = "conditional"

@dataclass
class TaskNode:
    """Node in the task execution DAG."""
    envelope: TaskEnvelope
    agent: SpecialistAgent
    dependencies: list[str] = field(default_factory=list)  # task_ids this depends on
    strategy: ExecutionStrategy = ExecutionStrategy.PARALLEL


class Orchestrator:
    def __init__(self, client: AsyncOpenAI, context_manager: ContextManager):
        self.client = client
        self.context_manager = context_manager
        self.agents: dict[str, SpecialistAgent] = {}
        self.task_registry: dict[str, TaskNode] = {}
        self.results: dict[str, ResultEnvelope] = {}
    
    def register_agent(self, agent: SpecialistAgent) -> None:
        self.agents[agent.role] = agent
    
    async def decompose_request(self, user_request: str) -> list[dict]:
        """
        Use an LLM to decompose the user request into agent tasks.
        Returns a list of task specifications.
        """
        available_agents = list(self.agents.keys())
        
        decomposition_prompt = f"""You are a task routing system. 
Available specialist agents: {json.dumps(available_agents)}

User request: {user_request}

Decompose this request into specific tasks for the available agents.
Return a JSON array where each element has:
- "agent_role": one of the available agents
- "task_description": specific, scoped instruction for that agent  
- "required_facts": list of fact keys this agent needs from context
- "depends_on": list of agent_roles that must complete first (for sequential deps)
- "can_run_parallel": boolean

Only include agents that are genuinely needed for this request.
"""
        response = await self.client.chat.completions.create(
            model="gpt-4o",
            messages=[{"role": "user", "content": decomposition_prompt}],
            response_format={"type": "json_object"},
            temperature=0
        )
        
        result = json.loads(response.choices[0].message.content)
        return result.get("tasks", [])
    
    async def execute_plan(
        self, 
        task_specs: list[dict],
        user_context: dict[str, Any]
    ) -> dict[str, ResultEnvelope]:
        """
        Execute the task plan, respecting dependencies and maximizing parallelism.
        """
        self.context_manager.ingest_user_context(user_context)
        
        # Build task nodes
        task_nodes = self._build_task_dag(task_specs)
        
        # Execute in topological order, parallelizing where possible
        completed = set()
        all_results = {}
        
        while len(completed) < len(task_nodes):
            # Find tasks whose dependencies are all satisfied
            ready = [
                node for task_id, node in task_nodes.items()
                if task_id not in completed
                and all(dep in completed for dep in node.dependencies)
            ]
            
            if not ready:
                # Circular dependency or all remaining tasks have unfulfilled deps
                break
            
            # Execute ready tasks in parallel
            tasks = [self._execute_single_task(node) for node in ready]
            batch_results = await asyncio.gather(*tasks, return_exceptions=True)
            
            for node, result in zip(ready, batch_results):
                task_id = node.envelope.task_id
                
                if isinstance(result, Exception):
                    result = ResultEnvelope(
                        task_id=task_id,
                        agent_role=node.envelope.agent_role,
                        status="failed",
                        structured_output={},
                        summary="Unexpected error during execution.",
                        extracted_facts={},
                        confidence_score=0.0,
                        error_detail=str(result)
                    )
                
                all_results[task_id] = result
                self.context_manager.register_result(result)
                completed.add(task_id)
        
        return all_results
    
    async def _execute_single_task(self, node: TaskNode) -> ResultEnvelope:
        """Execute a single task node, building its context slice first."""
        # Rebuild context slice now that upstream results may be available
        upstream_task_ids = [
            tid for tid, n in self.task_registry.items()
            if n.envelope.agent_role in node.dependencies
        ]
        
        updated_context = self.context_manager.build_context_slice(
            agent_role=node.envelope.agent_role,
            required_facts=node.envelope.metadata.get("required_facts", []),
            upstream_task_ids=upstream_task_ids
        )
        
        # Update the envelope with fresh context
        updated_envelope = node.envelope.model_copy(
            update={"context": updated_context}
        )
        
        return await node.agent.execute(updated_envelope)
    
    def _build_task_dag(self, task_specs: list[dict]) -> dict[str, TaskNode]:
        """Build the task dependency graph from decomposition output."""
        nodes = {}
        role_to_task_id = {}
        
        for spec in task_specs:
            agent_role = spec["agent_role"]
            if agent_role not in self.agents:
                continue
            
            task_id = str(uuid.uuid4())
            role_to_task_id[agent_role] = task_id
            
            envelope = TaskEnvelope(
                task_id=task_id,
                agent_role=agent_role,
                task_description=spec["task_description"],
                context=TaskContext(
                    user_id=self.context_manager.user_id,
                    session_id=self.context_manager.session_id,
                    original_request=self.context_manager._full_context.get(
                        "original_request", ""
                    )
                ),
                output_schema=self._get_output_schema(agent_role),
                metadata={"required_facts": spec.get("required_facts", [])}
            )
            
            nodes[task_id] = TaskNode(
                envelope=envelope,
                agent=self.agents[agent_role],
                dependencies=[],  # Will resolve below
                strategy=(
                    ExecutionStrategy.PARALLEL 
                    if spec.get("can_run_parallel", True) 
                    else ExecutionStrategy.SEQUENTIAL
                )
            )
        
        # Resolve dependencies: convert role names to task IDs
        for spec in task_specs:
            agent_role = spec["agent_role"]
            if agent_role not in role_to_task_id:
                continue
            task_id = role_to_task_id[agent_role]
            nodes[task_id].dependencies = [
                role_to_task_id[dep_role]
                for dep_role in spec.get("depends_on", [])
                if dep_role in role_to_task_id
            ]
        
        self.task_registry = nodes
        return nodes
    
    def _get_output_schema(self, agent_role: str) -> dict:
        """Return the expected JSON schema for a given agent role."""
        # In production, these would be stored in a registry
        base_schema = {
            "type": "object",
            "required": ["summary", "confidence_score", "extracted_facts", "warnings"],
            "properties": {
                "summary": {"type": "string"},
                "confidence_score": {"type": "number", "minimum": 0, "maximum": 1},
                "extracted_facts": {"type": "object"},
                "warnings": {"type": "array", "items": {"type": "string"}}
            }
        }
        return base_schema

Result Aggregation: Where the System Delivers Value

Aggregation is the step that most tutorials skip. They show you how to run three agents, print their results, and call it done. But in a real application, the user asked one question and expects one answer — not three separate essays. Aggregation is where the multi-agent system either proves its worth or falls apart.

There are three distinct aggregation challenges you need to handle:

1. Conflict resolution: What happens when Agent A says "increase bond allocation" and Agent B says "decrease bond allocation"? You need to surface the conflict, not silently pick one.

2. Confidence weighting: A result with a 0.95 confidence score should contribute more to the final synthesis than one with 0.4. You need a principled weighting mechanism.

3. Partial failure handling: What if one of three agents failed? Can you still produce a useful result? Often yes — but you need to be explicit about what's missing.

class AggregationResult(BaseModel):
    final_response: str
    contributing_agents: list[str]
    failed_agents: list[str]
    conflicts_detected: list[dict[str, Any]]
    overall_confidence: float
    caveats: list[str]
    raw_agent_results: dict[str, Any]  # For debugging/audit


class ResultAggregator:
    
    def __init__(self, client: AsyncOpenAI):
        self.client = client
    
    def detect_conflicts(
        self, 
        results: dict[str, ResultEnvelope]
    ) -> list[dict]:
        """
        Heuristically detect potential conflicts across agent results.
        In production, this can be rule-based or LLM-assisted.
        """
        conflicts = []
        successful = {
            k: v for k, v in results.items() 
            if v.status in ("success", "partial")
        }
        
        # Look for explicit conflict markers in extracted facts
        all_facts = {}
        for task_id, result in successful.items():
            for fact_key, fact_value in result.extracted_facts.items():
                if fact_key in all_facts:
                    existing_value, existing_source = all_facts[fact_key]
                    if existing_value != fact_value:
                        conflicts.append({
                            "fact_key": fact_key,
                            "value_a": existing_value,
                            "source_a": existing_source,
                            "value_b": fact_value,
                            "source_b": task_id
                        })
                else:
                    all_facts[fact_key] = (fact_value, task_id)
        
        return conflicts
    
    def compute_overall_confidence(
        self, 
        results: dict[str, ResultEnvelope],
        total_expected: int
    ) -> float:
        """
        Weighted confidence score accounting for failures.
        """
        successful = [r for r in results.values() if r.status == "success"]
        partial = [r for r in results.values() if r.status == "partial"]
        
        if not successful and not partial:
            return 0.0
        
        # Weight: successes count fully, partials at 0.5
        weighted_confidence = sum(r.confidence_score for r in successful)
        weighted_confidence += sum(r.confidence_score * 0.5 for r in partial)
        
        # Penalize for missing agents
        completeness_factor = len(results) / total_expected
        
        denominator = len(successful) + len(partial) * 0.5
        if denominator == 0:
            return 0.0
        
        avg_confidence = weighted_confidence / denominator
        return round(avg_confidence * completeness_factor, 3)
    
    async def synthesize(
        self,
        results: dict[str, ResultEnvelope],
        original_request: str,
        total_expected_agents: int
    ) -> AggregationResult:
        """
        Use an LLM to synthesize agent results into a coherent final response.
        """
        successful_results = {
            k: v for k, v in results.items() 
            if v.status in ("success", "partial")
        }
        failed_results = {
            k: v for k, v in results.items() 
            if v.status in ("failed", "escalated")
        }
        
        conflicts = self.detect_conflicts(results)
        overall_confidence = self.compute_overall_confidence(
            results, total_expected_agents
        )
        
        # Build synthesis context (compact — only structured outputs)
        synthesis_context = {
            agent_id: {
                "role": result.agent_role,
                "summary": result.summary,
                "output": result.structured_output,
                "confidence": result.confidence_score,
                "warnings": result.warnings
            }
            for agent_id, result in successful_results.items()
        }
        
        failed_roles = [v.agent_role for v in failed_results.values()]
        
        synthesis_prompt = f"""You are synthesizing results from multiple specialist agents 
into a single coherent response for the user.

Original user request: {original_request}

Agent results:
{json.dumps(synthesis_context, indent=2)}

Detected conflicts: {json.dumps(conflicts, indent=2)}

Failed agents (data unavailable): {failed_roles}

Instructions:
1. Synthesize the agent results into a direct, actionable response to the user
2. Where agents agree, state the consensus confidently
3. Where conflicts exist, explicitly surface them and explain the trade-offs
4. If agents failed, acknowledge what information is missing and how it affects completeness
5. Do not add information not present in the agent results
6. Keep the response focused and practical

Respond in JSON with keys: "response", "caveats" (list of strings)"""

        llm_response = await self.client.chat.completions.create(
            model="gpt-4o",
            messages=[{"role": "user", "content": synthesis_prompt}],
            response_format={"type": "json_object"},
            temperature=0.3
        )
        
        synthesis = json.loads(llm_response.choices[0].message.content)
        
        return AggregationResult(
            final_response=synthesis["response"],
            contributing_agents=[r.agent_role for r in successful_results.values()],
            failed_agents=failed_roles,
            conflicts_detected=conflicts,
            overall_confidence=overall_confidence,
            caveats=synthesis.get("caveats", []),
            raw_agent_results={
                k: v.model_dump() for k, v in results.items()
            }
        )

Key insight

The synthesis LLM is acting as an editor and integrator, not as an analyst. Its job is to weave together structured inputs that other agents have already validated and reasoned over. This division of labor is what prevents the classic problem of a single "super-agent" drifting between roles and losing coherence.


Putting It Together: End-to-End Pipeline

Here's how the full system looks end-to-end for the investment research use case:

async def handle_investment_query(
    user_request: str,
    user_id: str,
    portfolio_data: dict,
    tax_data: dict
) -> AggregationResult:
    
    client = AsyncOpenAI()
    session_id = str(uuid.uuid4())
    
    # Initialize context manager
    ctx_manager = ContextManager(session_id=session_id, user_id=user_id)
    
    # Pre-load user context
    user_context = {
        "original_request": user_request,
        "portfolio": portfolio_data,
        "tax_situation": tax_data,
        "user_constraints": ["risk_tolerance: moderate", "investment_horizon: 5yr"]
    }
    
    # Create specialist agents
    macro_agent = SpecialistAgent(
        role="macro_analysis",
        system_prompt="""You are a macroeconomic analyst specializing in Fed policy 
        and its effects on asset classes. You reason from first principles, 
        cite specific data points, and always quantify uncertainty.""",
        client=client
    )
    
    portfolio_agent = SpecialistAgent(
        role="portfolio_optimization",
        system_prompt="""You are a portfolio analyst. You analyze allocation efficiency, 
        rebalancing triggers, and risk-adjusted return expectations. 
        You always work from provided data, not assumptions.""",
        client=client
    )
    
    tax_agent = SpecialistAgent(
        role="tax_strategy",
        system_prompt="""You are a tax strategy specialist focused on investment tax 
        efficiency. You identify tax-loss harvesting opportunities, wash-sale risks, 
        and capital gains optimization strategies. You flag when professional 
        tax advice is required.""",
        client=client
    )
    
    # Initialize orchestrator
    orchestrator = Orchestrator(client=client, context_manager=ctx_manager)
    orchestrator.register_agent(macro_agent)
    orchestrator.register_agent(portfolio_agent)
    orchestrator.register_agent(tax_agent)
    
    # Decompose and execute
    task_specs = await orchestrator.decompose_request(user_request)
    results = await orchestrator.execute_plan(task_specs, user_context)
    
    # Aggregate
    aggregator = ResultAggregator(client=client)
    final_result = await aggregator.synthesize(
        results=results,
        original_request=user_request,
        total_expected_agents=len(task_specs)
    )
    
    return final_result

Observability: You Can't Debug What You Can't See

A multi-agent system that you can't observe is a liability in production. When something goes wrong — and it will — you need to know which agent failed, what context it received, what it produced, how long it took, and how much it cost. Instrument everything from the start, not as an afterthought.

The key observability dimensions for multi-agent systems are:

Trace correlation: Every task needs a trace_id that links back to the originating user request. When you're looking at a weird result, you need to be able to pull all agent calls for that request in one query. The LLM observability patterns covered elsewhere in this path apply here, with the addition that you need to track the parent-child relationship between orchestrator calls and agent calls.

Cost attribution: Multi-agent systems can get expensive fast. Each agent call is a separate API call, and your orchestrator's decomposition + synthesis calls add overhead on top. Build token counting into your ResultEnvelope from day one (we did this above with input_tokens and output_tokens). Aggregate these per-request and alert when a request's total cost exceeds a threshold. For more on managing this systematically, see the guide on cost optimization strategies.

Latency breakdown: Log each agent's latency_ms and the orchestrator's overall latency separately. If you're seeing slow responses, you need to know whether it's one slow agent in a parallel batch (which you might be able to swap out) or the synthesis step (which might need a faster model).

def emit_telemetry(
    request_id: str,
    results: dict[str, ResultEnvelope],
    aggregation: AggregationResult,
    total_latency_ms: float
) -> None:
    """Emit structured telemetry for the full request."""
    
    total_input_tokens = sum(r.input_tokens for r in results.values())
    total_output_tokens = sum(r.output_tokens for r in results.values())
    
    telemetry = {
        "request_id": request_id,
        "timestamp": datetime.utcnow().isoformat(),
        "total_latency_ms": total_latency_ms,
        "total_input_tokens": total_input_tokens,
        "total_output_tokens": total_output_tokens,
        "agent_count": len(results),
        "success_count": sum(1 for r in results.values() if r.status == "success"),
        "failed_count": sum(1 for r in results.values() if r.status == "failed"),
        "overall_confidence": aggregation.overall_confidence,
        "conflicts_detected": len(aggregation.conflicts_detected),
        "per_agent": [
            {
                "agent_role": r.agent_role,
                "status": r.status,
                "latency_ms": r.latency_ms,
                "input_tokens": r.input_tokens,
                "output_tokens": r.output_tokens,
                "confidence": r.confidence_score
            }
            for r in results.values()
        ]
    }
    
    # In production: send to your observability platform (Langfuse, Datadog, etc.)
    print(json.dumps(telemetry, indent=2))

Note

If you're running this at scale, consider parallel LLM call patterns for the orchestration layer itself. When you have many concurrent user requests, you need async patterns that go beyond basic asyncio.gather — proper semaphores, rate limit backpressure, and connection pooling all matter.


Hands-On Exercise

Build a multi-agent system for the following scenario: a customer support escalation analyzer that handles incoming support tickets for a software product.

Your system should have:

  1. A sentiment & urgency agent that scores the ticket's emotional tone and business urgency
  2. A technical diagnosis agent that identifies the likely technical root cause based on described symptoms
  3. A resolution routing agent that recommends whether to auto-resolve, route to Tier 1, or escalate to engineering

Implementation requirements:

  1. Define TaskEnvelope and ResultEnvelope schemas with output schemas specific to each agent role.

  2. Implement the ContextManager with a build_context_slice method. The technical diagnosis agent should receive the full ticket text. The resolution routing agent should receive outputs from both upstream agents (sentiment/urgency AND technical diagnosis).

  3. Make sentiment/urgency and technical diagnosis run in parallel, then feed their results to the resolution routing agent sequentially.

  4. Implement conflict detection: if the urgency score is "critical" but the resolution router recommends "auto-resolve," flag this as a conflict requiring human review.

  5. Add telemetry emission that records per-agent token counts and latencies.

Stretch goal: Add a retry mechanism to the agent base class that retries once on schema validation failure, using the validation error message as additional context in the retry prompt.


Common Mistakes & Troubleshooting

Mistake 1: Using prose context instead of structured context

The most common issue: agents receive narrative summaries of upstream results rather than structured fields. This causes the downstream agent to re-interpret the prose and potentially introduce drift or hallucinations. Always extract and pass structured facts, not summaries.

Mistake 2: Circular dependency deadlock

If your dependency graph has a cycle (Agent A depends on B, B depends on A), asyncio.gather won't deadlock — it'll just never execute the cycle. Build cycle detection into _build_task_dag:

def _detect_cycles(self, nodes: dict[str, TaskNode]) -> bool:
    """DFS-based cycle detection in the task DAG."""
    visited = set()
    path = set()
    
    def dfs(task_id: str) -> bool:
        visited.add(task_id)
        path.add(task_id)
        for dep in nodes[task_id].dependencies:
            if dep not in visited:
                if dfs(dep):
                    return True
            elif dep in path:
                return True
        path.discard(task_id)
        return False
    
    for task_id in nodes:
        if task_id not in visited:
            if dfs(task_id):
                return True
    return False

Mistake 3: Over-relying on LLM task decomposition

The decomposition step is an LLM call with no ground truth. In production, you should validate decomposition outputs against a whitelist of known agent roles and expected task patterns. If the decomposition produces an unknown agent role, fail fast rather than silently skipping it.

Mistake 4: Failing to handle partial results in synthesis

If two of three agents succeed, you must decide: produce a partial answer with caveats, or block until all agents complete (possibly via retry)? The answer depends on your domain. For the investment research case, missing tax strategy is a material gap — you should surface it prominently. Build this logic into your aggregator explicitly.

Mistake 5: Not capping agent recursion

If an agent can itself spawn sub-agents (hierarchical multi-agent systems), you're one bug away from an infinite recursion loop that runs up a large API bill before hitting any timeout. Always maintain a depth counter in the task envelope and refuse to spawn sub-agents beyond a maximum depth (typically 2-3 levels).

Warning

Multi-agent systems can silently accumulate large token costs in production, especially if agents are verbose and pass each other's full outputs downstream. Build budget guardrails at the per-request level: estimate total tokens before dispatching the full plan, and refuse or downgrade the plan if it exceeds your budget threshold. This integrates naturally with the LLM router architecture pattern — cheaper agents for simpler subtasks.

Troubleshooting: Schema validation failures

When an agent consistently fails schema validation, the typical causes are:

  • The output schema is too strict (nullable fields not marked as optional)
  • The model is appending conversational text before or after the JSON
  • The task description is long enough to push the JSON instructions out of the model's attention window

Fix #1 and #2 through schema design and prompt hardening. Fix #3 by moving the JSON schema requirement to the system prompt rather than the user message.

Troubleshooting: Context growing too large

If your relevant_facts dict grows without bound as agents produce results, you'll eventually hit context window limits. Implement a pruning strategy: after each agent completes, extract only the specific fact keys the remaining agents need. Mark consumed facts so they can be pruned from the live context.


Summary & Next Steps

You now have a production-ready architecture for multi-agent handoff protocols. The core disciplines are:

  • Explicit contracts: TaskEnvelope and ResultEnvelope schemas with output schemas make the interface between orchestrator and agents deterministic and validatable
  • Surgical context transfer: ContextManager extracts minimal, relevant context slices rather than passing the full state to every agent
  • Dependency-aware execution: The DAG-based scheduler maximizes parallelism while respecting true sequential dependencies
  • Principled aggregation: Conflict detection, confidence weighting, and partial-failure handling in the aggregator turn multiple agent results into a single coherent output
  • Full observability: Per-agent token counts, latencies, and status codes give you the instrumentation to debug and optimize in production

The architecture presented here is intentionally framework-agnostic — it uses pure Python and the OpenAI SDK directly, so you can understand every moving part. If you're working with LangChain or LlamaIndex, the same conceptual layers apply; those frameworks provide some of the scaffolding but impose their own abstractions that you'll need to work around for fine-grained control.

Where to go next:

  • For agents that need persistent memory across sessions (not just within a single request), see the guide on structured agent memory
  • For securing your agents against adversarial inputs that try to hijack routing decisions, prompt injection defense is essential reading — multi-agent systems are especially vulnerable since a compromised agent can influence downstream agents through shared context
  • If your agents need access to external data sources, the production RAG system patterns integrate naturally with the context transfer layer designed here
  • For testing your multi-agent pipelines end-to-end before shipping, the comprehensive LLM testing and evaluation guide covers evaluation strategies specifically suited to multi-step, agentic systems

The hardest part of multi-agent systems isn't the code — it's the discipline of maintaining explicit contracts between components when LLMs make it tempting to just "prompt your way through" the coordination layer. The systems that hold up in production are the ones where every agent knows exactly what it's supposed to do, what it's supposed to return, and nothing more.

Work With Us

From insight to implementation

Reading is the start. When you're ready to build the data, automation, or AI systems behind it, our team turns strategy into shipped results.

Let's Build

Building with LLMs

Previous

Building a LangGraph Agent with Persistent State: Implementing Nodes, Edges, and Conditional Branching for Multi-Step AI Workflows

Related Insights

AI & Machine LearningExpert

Implementing LLM-Powered Document Routing: Classifying and Directing Incoming Files to Specialized Processing Pipelines Based on Content Type and Business Rules

25 min
AI & Machine LearningExpert

Implementing Plan-and-Execute Agents: Separating Task Planning from Execution to Build Reliable Long-Horizon LLM Workflows

31 min
AI & Machine LearningPractitioner

Building a LangGraph Agent with Persistent State: Implementing Nodes, Edges, and Conditional Branching for Multi-Step AI Workflows

22 min

On this page

  • Introduction
  • Prerequisites
  • The Architecture: Orchestrator-Agent Topology
  • Designing the Task Delegation Contract
  • Building the Context Transfer Layer
  • Implementing the Agent Layer
  • The Orchestrator: Task Decomposition and Routing
  • Result Aggregation: Where the System Delivers Value
  • Putting It Together: End-to-End Pipeline
  • Observability: You Can't Debug What You Can't See
  • Hands-On Exercise
  • Common Mistakes & Troubleshooting
  • Mistake 1: Using prose context instead of structured context
  • Mistake 2: Circular dependency deadlock
  • Mistake 3: Over-relying on LLM task decomposition
  • Mistake 4: Failing to handle partial results in synthesis
  • Mistake 5: Not capping agent recursion
  • Troubleshooting: Schema validation failures
  • Troubleshooting: Context growing too large
  • Summary & Next Steps