Why Structured Output?
Why Structured Outputs Matter
The Problem: LLMs generate free-form text, but downstream code needs structured data -- JSON objects, typed fields, validated schemas. Parsing free text is fragile and error-prone.
The Solution: Structured output features force the LLM to produce valid JSON matching a specific schema, eliminating parse errors and ensuring type safety.
Real Impact: Structured outputs reduce parsing failures from 15-20% to near zero, making agents dramatically more reliable in production.
Real-World Analogy
Think of structured outputs like filling out a form vs. writing a letter:
- Free Text = Writing a letter -- creative but hard to process automatically
- JSON Mode = Filling out a form -- structured but flexible
- Schema Validation = A form with required fields and data types
- Pydantic Model = A digital form that auto-validates entries
- Retry on Failure = "Please re-fill this field, it was invalid"
Output Strategies
JSON Mode
Tell the API to produce valid JSON. Simple but no schema enforcement -- you get JSON but not necessarily the right shape.
Schema Enforcement
Provide a JSON Schema and the API guarantees output matches. Available in OpenAI and Anthropic APIs.
Pydantic Models
Define output structure as Python classes with type annotations. Auto-generates schema and validates output.
Retry Parsing
If output fails validation, send the error back to the LLM and ask it to fix the output. Usually succeeds on retry.
JSON Mode
from openai import OpenAI
from pydantic import BaseModel
from typing import List
class SearchResult(BaseModel):
title: str
url: str
relevance_score: float
summary: str
class SearchResponse(BaseModel):
query: str
results: List[SearchResult]
total_found: int
client = OpenAI()
response = client.beta.chat.completions.parse(
model="gpt-4o",
messages=[{
"role": "user",
"content": "Search for Python web frameworks"
}],
response_format=SearchResponse
)
# response.choices[0].message.parsed is a SearchResponse object
result = response.choices[0].message.parsed
print(result.results[0].title) # Typed access!Schema Validation
| Approach | Validation | Type Safety | Retry Built-in |
|---|---|---|---|
| JSON Mode | Valid JSON only | No | No |
| response_format | Schema-enforced | Yes | No |
| Pydantic + parse() | Full type validation | Yes | API-level |
| Instructor library | Full + retry logic | Yes | Yes |
Output Parsers
Popular Libraries
- Instructor: Pydantic-based structured outputs with retry logic for OpenAI/Anthropic
- LangChain OutputParsers: PydanticOutputParser, JsonOutputParser, StructuredOutputParser
- Outlines: Constrained generation for open-source models
- LMQL: Query language for LLMs with type constraints
Pydantic Models
| Feature | Description | Example |
|---|---|---|
| Type Hints | Python type annotations | name: str, age: int |
| Validators | Custom validation logic | @validator for range checks |
| Optional Fields | Fields that may be absent | nickname: Optional[str] |
| Nested Models | Complex object hierarchies | address: AddressModel |
| Enums | Constrained string values | status: Literal["active", "inactive"] |
Quick Reference
| Best Practice | Description | Why |
|---|---|---|
| Use Pydantic | Define output as typed models | Auto schema + validation |
| Add descriptions | Document each field | Helps LLM fill correctly |
| Set defaults | Provide fallback values | Handles missing fields gracefully |
| Retry on error | Send validation errors back | Self-correction usually works |
| Keep schemas simple | Avoid deeply nested structures | Reduces generation errors |