Translating Business Requirements into AI Task Specifications: A Practitioner's Framework for Scoping, Decomposing, and Documenting LLM Use Cases Before Building
Introduction
You've been handed a business requirement. Maybe it's a Jira ticket that says "use AI to improve our customer support process." Maybe it's a Slack message from your VP saying "can we automate the monthly narrative reports?" Maybe it's a sprawling Word document from a product manager describing a feature where the AI "just understands what users want and helps them." These are real requests that land on real practitioners' desks every week — and they are uniformly, spectacularly insufficient for building anything that actually works.
The gap between "a business need expressed in plain language" and "a working LLM-powered system" is where most AI projects silently fail. Not in the model selection. Not in the infrastructure. Not even in the prompts. They fail in the translation layer — the step where someone should have asked hard questions, drawn clear boundaries, defined what success looks like, and decomposed a fuzzy ambition into discrete, testable tasks before writing a single line of code or a single prompt.
By the end of this lesson, you'll have a complete practitioner's framework for doing exactly that translation work. We'll walk through how to interrogate a business requirement, scope an LLM use case honestly, decompose complex workflows into atomic AI tasks, and produce documentation that a developer, a stakeholder, and a future version of yourself can all use. This is the planning work that separates projects that ship and deliver value from the ones that quietly get shelved after three months of confusion.
What you'll learn:
- How to identify what a business requirement is actually asking for, versus what it says on the surface
- A structured scoping framework to determine whether a task is genuinely a good fit for an LLM
- How to decompose multi-step business workflows into discrete, specifiable AI subtasks
- How to write an AI task specification document that captures inputs, outputs, constraints, and success criteria
- How to anticipate failure modes and build them into your documentation before building begins
Prerequisites
This lesson assumes you're comfortable working with LLMs in a professional context and have at least experimented with prompt engineering. You should understand concepts like prompt chaining, context windows, and the basics of what LLMs are good and bad at. If you need a refresher on the underlying mechanics, Understanding Large Language Models: How ChatGPT and Claude Actually Work is worth a read first.
Why Most AI Projects Start Wrong
The typical pattern looks like this: a stakeholder identifies a pain point, someone in data or engineering gets tagged, there's a 30-minute meeting where everyone agrees AI could help, and then the technical person goes off to "figure it out." Two weeks later, they have a demo. The demo looks impressive. Then it hits production and immediately falls apart because nobody defined what "good" meant, what inputs the system would actually receive, what the system should do when it encounters edge cases, or how anyone will know if it's working.
This isn't a prompting problem or a model problem. It's a specification problem. And it's fixable with discipline applied at the start of a project — not retrofitted afterward.
The discipline we're talking about is borrowed from software engineering, modified for the specific quirks of probabilistic AI systems. In traditional software, a specification defines deterministic behavior: given input X, produce output Y. With LLMs, you're specifying a distribution of acceptable behaviors: given input X (which will vary in ways you can't fully anticipate), produce outputs that fall within a defined quality envelope, in a defined format, subject to a set of defined constraints.
That difference matters enormously for how you write specifications.
Key insight: A well-written AI task specification is not a prompt. It is the document that tells you what to build, why, and how you'll know it worked — before you start writing prompts at all.
Phase 1: Interrogating the Business Requirement
Before you can specify anything, you need to understand what you're actually being asked to do. This requires interrogating the original requirement systematically. Most business requirements contain four layers, and most practitioners only engage with the top one.
Layer 1: The Surface Request
This is what the requirement literally says. "Automate the monthly executive summary." "Use AI to triage support tickets." "Help sales reps write better proposals." These are starting points, not specifications.
Layer 2: The Underlying Problem
What pain is this solving? Who feels it and how often? What does the current process look like? A monthly executive summary might be painful because it takes a senior analyst two days to write, or because it's inconsistent across regions, or because it always arrives late. Each of those problems implies a different solution.
Ask: What happens today that shouldn't happen? What should happen that doesn't?
Layer 3: The Business Outcome
What will change in the business if this works? Not "we'll have AI," but what metric moves, what cost decreases, what quality improves, what speed increases? If you can't articulate the business outcome, you can't build evaluation criteria.
Ask: How will we know, six months from now, that this was worth building?
Layer 4: The Implicit Constraints
Every business requirement comes with constraints nobody thought to mention: regulatory requirements, brand voice guidelines, existing system integrations, SLA expectations, budget limits, technical infrastructure. These constraints will derail your build if you discover them in month two.
Ask: What can this system never do? What must it always do? What systems does it touch? Who will be accountable if it goes wrong?
The Interrogation Template
Use this set of questions in your requirements discovery sessions:
BUSINESS REQUIREMENT INTERROGATION TEMPLATE
1. SURFACE: What is being asked for, in one sentence?
2. PROBLEM:
- What current process does this replace or augment?
- Who is doing that process today, and how long does it take?
- What goes wrong with the current process?
3. OUTCOME:
- What specific metric should improve?
- By how much, over what time horizon?
- Who owns measuring this?
4. CONSTRAINTS:
- What data does this system have access to? What data is off-limits?
- What regulatory or compliance requirements apply?
- What tone, format, or brand requirements must outputs meet?
- What systems must this integrate with?
- What is the acceptable latency for a response?
- What happens when the AI is wrong?
5. VOLUME & VARIABILITY:
- How many requests per day/week/month?
- How variable are the inputs? (Highly structured vs. free-form)
- What are the extreme edge cases?
6. HUMAN-IN-THE-LOOP:
- Does a human review AI outputs before they're acted on?
- Who has authority to override or reject AI output?
- What escalation path exists for uncertain cases?
Tip: Run this interrogation as a structured meeting with both technical and non-technical stakeholders in the room. The best answers usually require at least two people — someone who knows the business deeply and someone who knows the data.
Phase 2: LLM Fit Assessment
Not every business problem is a good fit for an LLM. Before you go further, you need to honestly assess whether the task fits the tool. This is where premature enthusiasm does the most damage.
The Task Fit Matrix
Evaluate your task across five dimensions:
1. Language-Centrality
Is the core of the task manipulating, generating, interpreting, or transforming language? LLMs excel here. If the core task is numerical computation, database queries, or deterministic logic, an LLM is probably not your primary tool — though it might be useful at the edges.
Good fit: Summarizing call transcripts, classifying support tickets, drafting email responses
Poor fit: Calculating loan amortization schedules, running statistical regression, generating SQL queries from scratch without validation
2. Acceptable Output Variability
Can the system produce somewhat different outputs on similar inputs and still be correct? Or does the task require bit-for-bit reproducibility?
Good fit: Writing product descriptions (multiple valid phrasings exist)
Poor fit: Generating legal contract clauses that must match approved templates exactly
3. Ground Truth Availability
Do you have examples of what "good" looks like? Can you define and measure correctness? If you can't evaluate outputs, you can't improve the system.
Good fit: Classifying sentiment (you can label examples and measure accuracy)
Poor fit: Generating "creative" content where quality is entirely subjective
4. Error Tolerance
What happens when the system makes a mistake? Low-stakes errors (a slightly awkward email draft a human will edit anyway) are acceptable. High-stakes errors (misclassifying a fraud alert, generating incorrect medical information) require much more robust guardrails — and may require reconsidering the approach entirely.
5. Context Sufficiency
Does the LLM have access to the context it needs to do the task well? Or does it need domain knowledge, proprietary data, or real-time information that it doesn't have? This determines whether you need retrieval-augmented generation, fine-tuning, or just careful prompting.
Warning: A common mistake is building an LLM solution for a task the LLM fundamentally cannot do reliably — like precise numerical reasoning over large tables, or accurately recalling specific policy details from memory. Assess this honestly before committing to the approach.
Decision Output: Build, Don't Build, or Hybrid
After the fit assessment, you should reach one of three conclusions:
- Full LLM: The task is well-suited and you'll build primarily around an LLM
- Hybrid: The LLM handles language components while other systems handle logic, retrieval, or computation
- Not LLM: A different approach (traditional ML, rules-based logic, human process) is more appropriate
Document this decision and your reasoning. Future you will be grateful.
Phase 3: Task Decomposition
Assuming you've determined LLM is the right tool (or part of the right tool), the next step is decomposition. Real business workflows are rarely single-step AI tasks. They're pipelines, and confusing the pipeline for a single task is one of the most common specification mistakes.
The Decomposition Principle
An atomic AI task is one where:
- The input is well-defined and bounded
- The expected output is well-defined and bounded
- The transformation logic can be expressed in a single prompt (or a tightly scoped prompt chain)
- Success can be evaluated independently of the other tasks in the pipeline
If a task requires the AI to "understand the context, find the relevant information, synthesize a response, format it appropriately, and check it for compliance," that's five tasks masquerading as one.
A Worked Example: Customer Support Triage
Let's take a realistic scenario. Your company receives 2,000 support emails per day. The business requirement: "Use AI to route tickets to the right team and draft initial responses."
First pass (wrong): Build one AI that reads emails and handles everything.
Decomposition:
WORKFLOW: Support Email Processing Pipeline
Step 1: CLASSIFICATION
Input: Raw email text
Task: Classify into category (Billing / Technical / Account / Feedback / Abuse)
Output: Category label + confidence score
Human review trigger: confidence < 0.85
Step 2: INTENT EXTRACTION
Input: Raw email text + category label from Step 1
Task: Extract structured intent fields
Output: JSON with {primary_issue, urgency_indicator,
customer_sentiment, key_entities}
Step 3: ROUTING LOGIC
Input: Category label + intent JSON from Steps 1-2
Task: Deterministic rule-based routing (NOT an LLM task)
Output: Assigned team + priority level
Step 4: CONTEXT RETRIEVAL
Input: Customer ID + key_entities from Step 2
Task: Retrieve relevant account history, prior tickets,
product documentation (RAG step, NOT an LLM task)
Output: Retrieved context documents
Step 5: DRAFT GENERATION
Input: Original email + intent JSON + retrieved context +
routing info + team-specific tone guidelines
Task: Generate initial response draft
Output: Draft email in approved format
Step 6: COMPLIANCE CHECK
Input: Draft email from Step 5
Task: Flag any prohibited content, claims, or commitments
Output: Approved draft OR flagged draft with specific issues noted
Notice what happened here: the "one AI task" became six steps, two of which are deterministic logic (not LLM tasks at all) and one of which is retrieval. The LLM tasks are now clearly bounded: classification, intent extraction, draft generation, compliance check. Each can be prompted, evaluated, and improved independently.
Key insight: Decomposition often reveals that a significant portion of a seemingly "AI" workflow is actually deterministic logic or data retrieval that doesn't need an LLM at all. Keep those steps out of the LLM to reduce cost, latency, and failure modes.
Phase 4: Writing the AI Task Specification
Now you're ready to write a proper specification for each atomic AI task. This document is the artifact that bridges business requirements and technical implementation. It should be precise enough that a developer can build from it, specific enough that an evaluator can score outputs against it, and readable enough that a non-technical stakeholder can understand what the system will do.
The AI Task Specification Template
Here's the full template, followed by a worked example:
AI TASK SPECIFICATION
━━━━━━━━━━━━━━━━━━━━━━━━━━━━
TASK METADATA
━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Task ID: [unique identifier, e.g., SUPPORT-002]
Task Name: [descriptive name]
Parent Workflow: [which pipeline does this belong to]
Task Type: [Classification / Extraction / Generation /
Summarization / Comparison / Transformation]
Owner: [person responsible for this task's performance]
Version: [1.0]
Last Updated: [date]
━━━━━━━━━━━━━━━━━━━━━━━━━━━━
BUSINESS CONTEXT
━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Why this task exists:
[1-3 sentences connecting this task to the business need]
What this task replaces:
[Describe the current manual or automated process]
What happens if this task fails:
[Describe the failure impact — informs error handling priority]
━━━━━━━━━━━━━━━━━━━━━━━━━━━━
INPUT SPECIFICATION
━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Primary Input:
- Type: [text / structured data / document / multimodal]
- Source: [where does it come from]
- Format: [raw text / JSON / HTML / etc.]
- Expected length range: [e.g., 50–2000 tokens]
- Language: [English only / multilingual — specify]
Supplementary Inputs:
- [Any additional context, retrieved documents, metadata]
Input Variability:
- Common cases: [describe the typical input]
- Edge cases: [list 5-10 known edge cases]
- Out-of-scope inputs: [inputs the system should not process]
Example Inputs:
- [2-3 realistic examples spanning the range of variability]
━━━━━━━━━━━━━━━━━━━━━━━━━━━━
OUTPUT SPECIFICATION
━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Output Type: [label / structured JSON / prose / etc.]
Output Format: [specify schema, length limits, format requirements]
Output Schema (if structured):
{
"field_name": "type and description",
...
}
Quality Criteria:
- Accuracy: [how to measure correctness]
- Completeness: [what must always be present]
- Format compliance: [what structural requirements must be met]
- Tone/Style: [for generated text, describe voice and register]
What "good" looks like: [1-2 example outputs]
What "bad" looks like: [1-2 failure mode examples]
━━━━━━━━━━━━━━━━━━━━━━━━━━━━
CONSTRAINTS
━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Hard Constraints (non-negotiable):
- [Must never include X]
- [Must always include Y]
- [Max output length: Z tokens]
Soft Constraints (preferred but not absolute):
- [Should prefer X over Y when ambiguous]
Regulatory/Compliance Constraints:
- [GDPR, HIPAA, industry-specific rules that apply]
━━━━━━━━━━━━━━━━━━━━━━━━━━━━
SUCCESS CRITERIA
━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Minimum Viable Performance:
- [e.g., Classification accuracy ≥ 90% on held-out test set]
- [e.g., ≥ 85% of drafts accepted by agents without major edits]
Evaluation Method:
- How will you measure performance during development?
- How will you monitor performance in production?
- What is the review cadence?
Escalation Threshold:
- [When should this task escalate to human review?]
- [What signals indicate the system is degrading?]
━━━━━━━━━━━━━━━━━━━━━━━━━━━━
IMPLEMENTATION NOTES
━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Recommended Model Tier: [small/medium/large — and why]
Context Window Requirements: [estimated tokens needed]
Expected Latency: [acceptable response time]
Estimated Cost per Call: [token estimates × model pricing]
Dependencies: [which other tasks must run first]
Let me fill this in for the Intent Extraction step from our support email pipeline:
AI TASK SPECIFICATION
━━━━━━━━━━━━━━━━━━━━━━━━━━━━
TASK METADATA
━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Task ID: SUPPORT-002
Task Name: Customer Email Intent Extraction
Parent Workflow: Support Email Processing Pipeline
Task Type: Extraction (structured output from unstructured text)
Owner: Support Engineering Team
Version: 1.0
Last Updated: 2024-11-15
━━━━━━━━━━━━━━━━━━━━━━━━━━━━
BUSINESS CONTEXT
━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Why this task exists:
Support agents currently spend 4-6 minutes reading each email
to understand what the customer needs. Structured extraction
of intent fields speeds agent handling and enables automated
routing and prioritization.
What this task replaces:
Manual reading and mental parsing by Tier 1 support agents.
What happens if this task fails:
Tickets may be routed incorrectly or given wrong priority.
Medium-severity failure — a human will still process the
ticket, but with more friction.
━━━━━━━━━━━━━━━━━━━━━━━━━━━━
INPUT SPECIFICATION
━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Primary Input:
- Type: text
- Source: inbound customer email, raw text after HTML stripping
- Format: plain text string
- Expected length range: 20–800 tokens (99th percentile)
- Language: English only (non-English triggers separate routing)
Supplementary Inputs:
- Category label from SUPPORT-001 (classification task)
- Customer tier from CRM lookup (Free / Pro / Enterprise)
Input Variability:
- Common cases:
Customer describes a specific problem with product feature
Customer asks a billing question with account reference
- Edge cases:
Email is a reply mid-thread (contains quoted history)
Customer sends multiple unrelated issues in one email
Email contains only an image attachment (no text body)
Customer is non-native speaker with unconventional grammar
Email contains profanity or emotional distress signals
Automated email forwarded from another system
Email is clearly spam
- Out-of-scope inputs:
Emails not in English (handled by separate pipeline)
Internal team emails routed to support inbox
Example Inputs:
1. "Hi, I've been charged twice for my subscription this month.
My account is jane.smith@company.com and the charges
appeared on Nov 3 and Nov 7. Can you fix this?"
2. "The API is returning 429 errors on endpoint /v2/data whenever
we send more than 3 requests per second. This is breaking
our production pipeline. We're on the Enterprise plan."
3. "Thanks for the great product. Quick question — do you have
SSO support? We're evaluating whether to move from our
current tool."
━━━━━━━━━━━━━━━━━━━━━━━━━━━━
OUTPUT SPECIFICATION
━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Output Type: Structured JSON
Output Schema:
{
"primary_issue": "string (max 15 words, plain language)",
"issue_category_confirmed": "boolean",
"urgency": "low | medium | high | critical",
"urgency_reasoning": "string (max 20 words)",
"customer_sentiment": "frustrated | neutral | positive | distressed",
"key_entities": {
"account_identifiers": ["array of emails, IDs, account names"],
"product_features": ["array of mentioned features/endpoints"],
"financial_amounts": ["array of dollar amounts if mentioned"],
"dates_mentioned": ["array of dates if mentioned"]
},
"action_requested": "string (max 20 words, what customer wants done)",
"multi_issue": "boolean (true if email contains >1 distinct issue)",
"extraction_confidence": "high | medium | low"
}
Quality Criteria:
- Accuracy: primary_issue and action_requested should match
human agent's interpretation ≥ 92% of cases
- Completeness: all fields must be present; use null for
fields with no evidence in email, never omit keys
- Format compliance: must be valid parseable JSON, no markdown
wrapping, no prose outside the JSON object
- Urgency: "critical" reserved for system outages or data loss
affecting production systems
What "good" looks like:
For Example Input 1:
{
"primary_issue": "Double charge on subscription this month",
"issue_category_confirmed": true,
"urgency": "medium",
"urgency_reasoning": "Financial error but not system outage",
"customer_sentiment": "neutral",
"key_entities": {
"account_identifiers": ["jane.smith@company.com"],
"product_features": [],
"financial_amounts": [],
"dates_mentioned": ["Nov 3", "Nov 7"]
},
"action_requested": "Investigate and correct duplicate charges",
"multi_issue": false,
"extraction_confidence": "high"
}
What "bad" looks like:
- Wrapping JSON in markdown code fences (breaks parsing)
- Setting urgency="critical" for billing questions
- Omitting account_identifiers when email address is present
- primary_issue longer than 15 words
- Hallucinating entities not mentioned in the email
━━━━━━━━━━━━━━━━━━━━━━━━━━━━
CONSTRAINTS
━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Hard Constraints:
- Output MUST be valid JSON with all schema keys present
- Must not include any customer PII in urgency_reasoning
or primary_issue fields (GDPR compliance)
- Must not infer information not present in the email
Soft Constraints:
- Prefer conservative urgency ratings when ambiguous
- When extraction_confidence is "low", prefer null over
a low-confidence guess
Regulatory:
- GDPR Article 5: data minimization — do not extract or
repeat PII beyond key identifiers needed for routing
━━━━━━━━━━━━━━━━━━━━━━━━━━━━
SUCCESS CRITERIA
━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Minimum Viable Performance:
- ≥ 92% agreement with human agent on primary_issue
(evaluated on 200-email labeled test set)
- ≥ 98% valid JSON output (parseable without errors)
- ≥ 88% accuracy on urgency classification
- Zero instances of PII appearing in prohibited fields
Evaluation Method:
- Development: manual review of 200 labeled examples
across all edge case categories
- Production: weekly sample of 50 random tickets reviewed
by support lead; drift alert if accuracy drops below 88%
Escalation Threshold:
- extraction_confidence = "low" → route to human review queue
- JSON parse failure → log error, route to human, alert engineering
━━━━━━━━━━━━━━━━━━━━━━━━━━━━
IMPLEMENTATION NOTES
━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Recommended Model Tier: Medium (GPT-4o-mini or Claude Haiku)
Reasoning: task is well-defined extraction, not complex
reasoning; smaller model sufficient and reduces cost
Context Window Requirements: ~2,000 tokens
(system prompt + email + few-shot examples)
Expected Latency: < 2 seconds (acceptable for async processing)
Estimated Cost per Call: ~0.002 USD at medium tier × 2,000 calls/day
= ~$4/day operational cost
Dependencies: SUPPORT-001 must complete first (category label
used as supplementary input)
This is the level of specificity that makes implementation tractable. Notice that writing clear output requirements and examples is built directly into the specification — you're not leaving it to the prompt engineer to figure out what "good" means.
Phase 5: Documenting the Full Pipeline
Once you've written specs for each atomic task, you need one more artifact: a pipeline document that shows how the tasks connect. This is your architectural map.
Pipeline Document Structure
PIPELINE SPECIFICATION
Pipeline ID: SUPPORT-PIPELINE-001
Pipeline Name: Inbound Support Email Processing
Business Owner: VP Customer Success
Technical Owner: Support Engineering
TRIGGER: Inbound email arrives in support@company.com
FLOW:
[Email Received]
↓
[Step 1: SUPPORT-001 — Classification]
↓ Category + Confidence
[Gate: Confidence < 0.85?]
→ Yes: [Human Review Queue — END]
→ No: Continue
↓
[Step 2: SUPPORT-002 — Intent Extraction]
↓ Intent JSON
[Step 3: SUPPORT-003 — Routing (deterministic)]
↓ Team + Priority
[Step 4: Context Retrieval (RAG — not LLM)]
↓ Retrieved Documents
[Step 5: SUPPORT-005 — Draft Generation]
↓ Draft Response
[Step 6: SUPPORT-006 — Compliance Check]
↓
[Gate: Compliance issues found?]
→ Yes: [Flag for human review — END]
→ No: Continue
↓
[Draft delivered to agent queue]
FAILURE HANDLING:
- Any LLM task failure: log error, route to human queue, alert
- Latency > 10 seconds: timeout, route to human queue
- Daily error rate > 5%: pause pipeline, alert engineering
MONITORING:
- Dashboard: processing volume, error rates, routing accuracy
- Weekly: sample audit by support lead
- Monthly: full performance review against success criteria
Tip: Keep the pipeline document in a location accessible to both business stakeholders and technical teams — not buried in a dev repo. A shared Notion page or Confluence doc works well. The business owner should be able to read this without a technical translator.
Hands-On Exercise
Take the following business requirement and produce:
- A completed Interrogation Template (Phase 1)
- A Task Fit Assessment (Phase 2)
- A decomposed task list (Phase 3)
- A full AI Task Specification for one of the atomic tasks (Phase 4)
The Business Requirement:
"We want to use AI to help our analyst team produce the monthly regional performance reports faster. Right now it takes each analyst about 8 hours per report, and we have 12 regions. The reports go to regional VPs and need to be professional and consistent. They include financial performance vs. targets, key trend narratives, and a section on risks and opportunities."
Guidance for working through this:
Start with the interrogation. Ask yourself: what data feeds this process? What does "professional and consistent" mean precisely — is there a template? What does an analyst do during those 8 hours — are they pulling data, writing prose, or both? The data manipulation and chart creation are likely not LLM tasks. The narrative generation almost certainly is.
When you decompose, you'll likely find tasks like: (a) summarizing performance metrics into narrative language, (b) identifying trends from time-series data, (c) drafting risk/opportunity observations given a set of inputs, (d) checking narrative consistency across sections. Each of those is a different prompt with different inputs and different success criteria.
For the full task specification, pick the narrative generation task. Define the input format precisely — what structured data will the LLM receive? What's the expected output format? What tone guidelines apply (executive-facing, formal, concise)? What are the hard constraints (never make specific financial predictions, never contradict the raw data)?
Note: There is no single correct answer to this exercise. The value is in the rigor of your thinking, not in matching a specific output. If you're doing this in a team setting, compare your decompositions — the differences will reveal hidden assumptions about how the process works.
Common Mistakes & Troubleshooting
Mistake 1: Writing the Prompt Before the Specification
The most common mistake. People jump to prompt engineering before they've defined the problem, which means they iterate on prompts without a clear target to iterate toward. If you find yourself tweaking prompts and not knowing whether you're getting closer to "good," you skipped specification.
Fix: Write at least a minimal specification (inputs, outputs, success criteria) before writing your first prompt. The time investment pays for itself immediately.
Mistake 2: Treating a Pipeline as a Single Task
Specifying "the AI will process the customer email end-to-end" is not a specification — it's a wish. Single-task framing prevents you from identifying which steps can be evaluated, which steps need different models, and which steps aren't LLM tasks at all.
Fix: Decompose until every task has a single, evaluable transformation from input to output.
Mistake 3: Vague Success Criteria
"The output should be high quality" is not a success criterion. Neither is "stakeholders should be happy with it." Without measurable criteria, you can't evaluate whether what you built works, and you can't defend or improve it over time.
Fix: Every success criterion should be measurable. For generation tasks where human judgment is involved, define the rating mechanism (e.g., "rated by domain expert on 1-5 scale, must average ≥ 4.0") not just "it should be good."
Mistake 4: Ignoring Edge Cases in the Specification
Edge cases that aren't in the spec will become production bugs. Emails in unexpected formats, inputs that are too long, malicious or adversarial inputs, missing required fields — these will all happen in production.
Fix: Spend dedicated time brainstorming edge cases in your specification sessions. Aim for at least 10. Then define the system's expected behavior for each one. This connects directly to embedding appropriate guardrails in your implementation.
"Generate a summary" without format constraints will produce wildly inconsistent output lengths, structures, and styles across different inputs. This is especially painful for machine-readable outputs that feed downstream systems.
Fix: Define output format with the specificity of a schema. If it's prose, define length ranges, required sections, and tone. If it's structured data, define the exact schema and what happens when a field has no value. The structured output approaches covered in detail elsewhere should be specified at this stage, not discovered during implementation.
Mistake 6: Specification as a One-Time Document
Specifications get stale. The data changes, the business requirements evolve, the model gets updated. A spec that was written once and never revisited becomes a liability.
Fix: Version your specifications, assign an owner, and schedule reviews. Production monitoring should be explicitly connected to your success criteria — when metrics drift, revisit the spec first before adjusting prompts.
Warning: A specification that nobody maintains is worse than no specification at all, because it creates false confidence. If you can't commit to maintaining the document, keep it minimal and make sure the success criteria are prominently visible to everyone working on the system.
A Note on Ethical and Risk Dimensions
Specification work is also where you identify and document the ethical risks of a system. For any task that touches customer data, makes consequential decisions, or could produce harmful outputs, your spec should include a dedicated risk section. This isn't optional — it's part of professional practice.
Questions to address in the specification:
- What happens when this system is wrong, and who bears the consequence?
- Are there populations who might be systematically disadvantaged by this system?
- Is the human-in-the-loop design appropriate given the stakes?
- What data is being processed, and are customers aware of and consenting to that use?
This is documented in the specification because it's much cheaper to address at design time than after deployment. The broader ethical considerations for production AI systems are covered in depth in Building Ethical AI Systems: A Practitioner's Guide to Responsible Business Implementation — but the specification is where those considerations become operational.
Summary & Next Steps
The framework we've walked through has five phases:
- Interrogate the business requirement to get past the surface request to the real problem, outcome, and constraints
- Assess fit to determine whether an LLM is the right tool and what shape of solution is appropriate
- Decompose the workflow into atomic, evaluable tasks with clear inputs and outputs
- Specify each atomic task with enough rigor that implementation can proceed without ambiguity
- Document the full pipeline with flow, failure handling, and monitoring
This work happens before you write your first prompt. It takes time — usually two to four hours of structured work per significant use case. It pays back that investment many times over in reduced implementation churn, clearer stakeholder expectations, and systems that can actually be evaluated and improved.
Once your specifications are complete, you're ready to move into implementation. That means translating your input/output specifications into well-constructed system prompts, running your first prompt experiments against your defined success criteria, and building the evaluation harness that will tell you whether what you've built actually meets the bar.
For tasks that involve generating prose at scale, the specification-to-implementation bridge often runs through a prompt template — and how to build and manage reusable prompt templates across your team is the natural next stop. For tasks that involve complex multi-step reasoning, understanding how to structure prompt chaining across your decomposed tasks will be essential.
The specification is the foundation. Build it well, and everything that comes after is faster, cleaner, and more likely to actually work.
