CrewAI & Multi-Agent Systems

CrewAI Overview

Why CrewAI Matters

The Problem: Single-agent systems struggle with complex tasks that require diverse expertise, different perspectives, or parallel workstreams.

The Solution: CrewAI lets you define teams of specialized AI agents with distinct roles, goals, and backstories that collaborate to complete complex tasks through delegation and coordination.

Real Impact: CrewAI enables building production-ready multi-agent teams that mirror real-world team structures, from research teams to content pipelines.

Real-World Analogy

Think of CrewAI as assembling a film production crew:

Agents = Crew members (Director, Writer, Editor, Researcher)
Roles = Job titles defining expertise and perspective
Tasks = Specific assignments each member must complete
Tools = Equipment each member uses (cameras, editing software)
Crew = The assembled team working together on the project

Core CrewAI Concepts

Agent Roles

Each agent has a role, goal, and backstory that shapes its behavior and expertise. Agents are specialists, not generalists.

Task Delegation

Tasks are assigned to specific agents and can include expected outputs, context from other tasks, and tool requirements.

Sequential Process

Tasks execute one after another, with each task output feeding into the next as context for downstream agents.

Hierarchical Process

A manager agent coordinates the crew, delegating tasks and reviewing outputs before passing work to the next stage.

Agents & Roles

CrewAI Multi-Agent Workflow

crewai_agents.py

from crewai import Agent, Task, Crew, Process
from crewai_tools import SerperDevTool, WebsiteSearchTool

# Define specialized agents
researcher = Agent(
    role="Senior Research Analyst",
    goal="Find comprehensive, accurate information",
    backstory="""You are an expert researcher with 15 years
    of experience in gathering and analyzing information.""",
    tools=[SerperDevTool(), WebsiteSearchTool()],
    verbose=True,
    allow_delegation=False,
)

writer = Agent(
    role="Technical Content Writer",
    goal="Create engaging, well-structured content",
    backstory="""You are a skilled technical writer who
    transforms complex research into clear content.""",
    verbose=True,
    allow_delegation=True,
)

editor = Agent(
    role="Senior Editor",
    goal="Ensure content is polished and accurate",
    backstory="""You are a meticulous editor who catches
    every error and ensures highest standards.""",
    verbose=True,
)

Tasks & Tools

crewai_tasks.py

# Define tasks with expected outputs
research_task = Task(
    description="""Research the latest developments in AI agents.
    Focus on frameworks, use cases, and adoption trends.""",
    expected_output="A detailed research report with findings",
    agent=researcher,
)

writing_task = Task(
    description="""Write a blog post about AI agents using
    the research provided.""",
    expected_output="A 1500-word blog post in markdown",
    agent=writer,
    context=[research_task],
)

editing_task = Task(
    description="""Review and edit the blog post for clarity,
    accuracy, and engagement.""",
    expected_output="Final edited blog post",
    agent=editor,
    context=[research_task, writing_task],
)

Crew Assembly

crewai_crew.py

# Assemble and run the crew
crew = Crew(
    agents=[researcher, writer, editor],
    tasks=[research_task, writing_task, editing_task],
    process=Process.sequential,
    verbose=True,
)

result = crew.kickoff()
print(result)

# Hierarchical process with manager
hierarchical_crew = Crew(
    agents=[researcher, writer, editor],
    tasks=[research_task, writing_task, editing_task],
    process=Process.hierarchical,
    manager_llm=ChatOpenAI(model="gpt-4"),
)

Process Types

Sequential vs Hierarchical

Sequential: Tasks execute in order, each output feeds the next
Hierarchical: A manager agent coordinates, delegates, and reviews
Delegation: Agents with allow_delegation=True can pass subtasks
Context: Use context parameter to pass outputs from prior tasks

Common Pitfall

Problem: Agents produce inconsistent outputs because backstories are too vague.

Solution: Write detailed, specific backstories and goals. Include constraints like "always cite sources" or "write in bullet points."

Quick Reference

Component	Description	Example
`Agent()`	Create an agent	`Agent(role="...", goal="...")`
`Task()`	Define a task	`Task(description="...", agent=a)`
`Crew()`	Assemble a crew	`Crew(agents=[...], tasks=[...])`
`.kickoff()`	Run the crew	`result = crew.kickoff()`
`Process.sequential`	Tasks run in order	`process=Process.sequential`
`Process.hierarchical`	Manager coordinates	`process=Process.hierarchical`
`context`	Pass task outputs	`context=[research_task]`