AutoGen & Conversational Agents

Hard 30 min read

AutoGen Overview

Why AutoGen Matters

The Problem: Building multi-agent systems that can hold natural conversations, execute code safely, and coordinate complex tasks requires significant infrastructure.

The Solution: AutoGen provides a conversation-driven framework where agents chat with each other to solve problems, with built-in code execution sandboxes and flexible conversation patterns.

Real Impact: Microsoft's AutoGen powers enterprise scenarios from automated code review to multi-step data analysis pipelines with human oversight.

Real-World Analogy

Think of AutoGen as a group video call with screen sharing:

  • AssistantAgent = The expert who proposes solutions and writes code
  • UserProxyAgent = The moderator who approves actions and runs code
  • GroupChatManager = The meeting facilitator deciding who speaks next
  • Code Execution = Screen sharing where everyone sees the code run
  • Termination = The meeting ends when the problem is solved

Core AutoGen Components

AssistantAgent

An LLM-powered agent that generates responses, writes code, and proposes solutions. Does not execute code itself.

UserProxyAgent

Acts as a human proxy that can execute code, provide feedback, and approve or reject assistant proposals.

GroupChatManager

Orchestrates multi-agent conversations by selecting which agent speaks next based on the conversation context.

Code Execution

Sandboxed environments (Docker or local) for safely executing code generated by agents during conversations.

Agent Types

AutoGen Conversation Flow
AssistantAgent LLM-powered UserProxyAgent Code executor 1. Propose solution + code 2. Execute code, return result GroupChatManager Selects next speaker Agent A Agent B Agent C

Conversations

autogen_basic.py
from autogen import AssistantAgent, UserProxyAgent

# Configure the LLM
llm_config = {
    "model": "gpt-4",
    "temperature": 0,
    "api_key": os.environ["OPENAI_API_KEY"],
}

# Create an assistant agent
assistant = AssistantAgent(
    name="coding_assistant",
    llm_config=llm_config,
    system_message="You are a helpful coding assistant. Write clean Python code.",
)

# Create a user proxy that can execute code
user_proxy = UserProxyAgent(
    name="user",
    human_input_mode="NEVER",  # ALWAYS, TERMINATE, or NEVER
    code_execution_config={
        "work_dir": "workspace",
        "use_docker": False,
    },
    max_consecutive_auto_reply=5,
)

# Start the conversation
user_proxy.initiate_chat(
    assistant,
    message="Write a Python function to calculate fibonacci numbers",
)

Group Chat

autogen_groupchat.py
from autogen import GroupChat, GroupChatManager

# Create specialized agents
planner = AssistantAgent(name="planner", llm_config=llm_config,
    system_message="You plan the approach to solve problems.")
coder = AssistantAgent(name="coder", llm_config=llm_config,
    system_message="You write clean, tested Python code.")
reviewer = AssistantAgent(name="reviewer", llm_config=llm_config,
    system_message="You review code for bugs and improvements.")

# Set up group chat
group_chat = GroupChat(
    agents=[user_proxy, planner, coder, reviewer],
    messages=[],
    max_round=12,
    speaker_selection_method="auto",
)

manager = GroupChatManager(
    groupchat=group_chat,
    llm_config=llm_config,
)

# Start group conversation
user_proxy.initiate_chat(
    manager,
    message="Build a REST API with Flask that has CRUD operations for a todo list",
)

Code Execution

Execution Modes

  • Local: Code runs directly on the host machine (use_docker=False)
  • Docker: Code runs in isolated containers for safety (use_docker=True)
  • human_input_mode="ALWAYS": Human approves every message
  • human_input_mode="TERMINATE": Human approves only when termination is suggested
  • human_input_mode="NEVER": Fully autonomous execution

Common Pitfall

Problem: Agents enter infinite conversation loops, generating excessive API calls.

Solution: Always set max_consecutive_auto_reply on UserProxyAgent and max_round on GroupChat. Add termination conditions in system messages like "Reply TERMINATE when the task is complete."

Quick Reference

ComponentDescriptionExample
AssistantAgentLLM-powered agentAssistantAgent(name="a", llm_config=cfg)
UserProxyAgentHuman proxy / executorUserProxyAgent(name="u", code_execution_config=...)
GroupChatMulti-agent conversationGroupChat(agents=[...], max_round=10)
GroupChatManagerManages group chatGroupChatManager(groupchat=gc)
initiate_chat()Start conversationproxy.initiate_chat(assistant, message="...")
human_input_modeControl human involvement"ALWAYS" | "TERMINATE" | "NEVER"
code_execution_configCode sandbox settings{"work_dir": "ws", "use_docker": True}