Tag: artificial-intelligence

  • Agentic Design Pattern: Creative Agent

    
        """
    Pattern 075: Creative Agent
    
    Description:
        A Creative Agent is a specialized AI agent designed to generate original creative
        content across various domains including writing (stories, poetry, scripts), visual
        concepts (art descriptions, design ideas), music concepts, and other creative works.
        This pattern demonstrates how to build an intelligent agent that can understand
        creative requirements, generate diverse creative outputs, adapt styles, incorporate
        feedback, and maintain creative consistency.
        
        The agent combines creative ideation with structured generation techniques to produce
        high-quality, original content. It can work across multiple creative domains, adapt
        to different styles and tones, generate variations, and iteratively refine outputs
        based on feedback.
    
    Components:
        1. Ideation Engine: Generates creative concepts and ideas
        2. Style Adapter: Adapts content to different styles and tones
        3. Content Generator: Creates structured creative outputs
        4. Variation Creator: Produces diverse alternatives
        5. Constraint Respector: Adheres to creative constraints
        6. Feedback Integrator: Incorporates critique and improvements
        7. Consistency Maintainer: Ensures coherent creative vision
        8. Multi-Modal Orchestrator: Coordinates different creative types
    
    Key Features:
        - Cross-domain creative generation (text, visual concepts, music ideas)
        - Style and tone adaptation (formal, casual, poetic, technical)
        - Character and world-building for narratives
        - Creative constraint handling (theme, length, format)
        - Iterative refinement with feedback
        - Variation generation for exploration
        - Originality assessment
        - Creative inspiration from prompts
        - Collaborative creativity support
        - Genre and format flexibility
    
    Use Cases:
        - Story and novel writing
        - Poetry and lyric generation
        - Screenplay and script writing
        - Marketing copy and slogans
        - Product naming and branding
        - Art concept development
        - Game narrative design
        - Content marketing
        - Creative brainstorming
        - Educational creative writing
    
    LangChain Implementation:
        Uses ChatOpenAI with varying temperatures for different creative tasks,
        specialized chains for ideation, generation, and refinement.
    """
    
    import os
    import re
    from typing import List, Dict, Any, Optional
    from dataclasses import dataclass, field
    from enum import Enum
    from datetime import datetime
    from dotenv import load_dotenv
    
    from langchain_core.prompts import ChatPromptTemplate
    from langchain_core.output_parsers import StrOutputParser
    from langchain_openai import ChatOpenAI
    
    load_dotenv()
    
    
    class CreativeType(Enum):
        """Types of creative content"""
        STORY = "story"
        POEM = "poem"
        SCRIPT = "script"
        SONG_LYRICS = "song_lyrics"
        ARTICLE = "article"
        MARKETING_COPY = "marketing_copy"
        PRODUCT_NAME = "product_name"
        SLOGAN = "slogan"
        DIALOGUE = "dialogue"
        WORLD_BUILDING = "world_building"
    
    
    class CreativeStyle(Enum):
        """Creative styles and tones"""
        FORMAL = "formal"
        CASUAL = "casual"
        POETIC = "poetic"
        HUMOROUS = "humorous"
        DRAMATIC = "dramatic"
        MINIMALIST = "minimalist"
        ELABORATE = "elaborate"
        MYSTERIOUS = "mysterious"
        ROMANTIC = "romantic"
        DARK = "dark"
    
    
    class Genre(Enum):
        """Creative genres"""
        FANTASY = "fantasy"
        SCIENCE_FICTION = "science_fiction"
        MYSTERY = "mystery"
        ROMANCE = "romance"
        HORROR = "horror"
        THRILLER = "thriller"
        COMEDY = "comedy"
        DRAMA = "drama"
        ADVENTURE = "adventure"
        HISTORICAL = "historical"
    
    
    @dataclass
    class CreativePrompt:
        """Input prompt for creative generation"""
        type: CreativeType
        theme: str
        style: CreativeStyle
        genre: Optional[Genre] = None
        constraints: Dict[str, Any] = field(default_factory=dict)
        inspiration: List[str] = field(default_factory=list)
        target_audience: Optional[str] = None
    
    
    @dataclass
    class Character:
        """Character for narrative works"""
        name: str
        role: str  # protagonist, antagonist, supporting
        personality: List[str]
        background: str
        goals: List[str]
        conflicts: List[str]
    
    
    @dataclass
    class Setting:
        """Setting for creative works"""
        location: str
        time_period: str
        atmosphere: str
        key_features: List[str]
        description: str
    
    
    @dataclass
    class CreativeWork:
        """Generated creative content"""
        work_id: str
        type: CreativeType
        title: str
        content: str
        style: CreativeStyle
        genre: Optional[Genre]
        characters: List[Character] = field(default_factory=list)
        setting: Optional[Setting] = None
        themes: List[str] = field(default_factory=list)
        word_count: int = 0
        metadata: Dict[str, Any] = field(default_factory=dict)
        created_at: datetime = field(default_factory=datetime.now)
    
    
    @dataclass
    class CreativeFeedback:
        """Feedback on creative work"""
        work_id: str
        strengths: List[str]
        weaknesses: List[str]
        suggestions: List[str]
        overall_score: float  # 0.0-1.0
        specific_notes: Dict[str, str] = field(default_factory=dict)
    
    
    @dataclass
    class CreativeVariation:
        """Variation of a creative work"""
        original_id: str
        variation_id: str
        content: str
        changes_made: List[str]
        variation_type: str  # tone, structure, perspective, etc.
    
    
    class CreativeAgent:
        """
        Agent for generating creative content across multiple domains.
        
        This agent can generate stories, poems, scripts, marketing copy, and other
        creative works while adapting to styles, genres, and constraints.
        """
        
        def __init__(self):
            """Initialize the creative agent with specialized LLMs"""
            # Ideation engine for brainstorming
            self.ideation_llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0.9)
            
            # Generator for main content creation
            self.generator_llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0.8)
            
            # Refiner for improving content
            self.refiner_llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0.6)
            
            # Critic for evaluation
            self.critic_llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0.3)
            
            # Creative works repository
            self.works: Dict[str, CreativeWork] = {}
        
        def generate_ideas(
            self,
            theme: str,
            creative_type: CreativeType,
            count: int = 5
        ) -> List[str]:
            """
            Generate creative ideas based on theme.
            
            Args:
                theme: Central theme or concept
                creative_type: Type of creative work
                count: Number of ideas to generate
                
            Returns:
                List of creative ideas
            """
            prompt = ChatPromptTemplate.from_messages([
                ("system", """You are a creative ideation expert. Generate original and
                imaginative ideas based on the theme and content type.
                
                Provide {count} ideas, one per line starting with a number."""),
                ("user", """Theme: {theme}
    Content Type: {type}
    
    Generate {count} creative ideas.""")
            ])
            
            chain = prompt | self.ideation_llm | StrOutputParser()
            
            try:
                response = chain.invoke({
                    "theme": theme,
                    "type": creative_type.value,
                    "count": count
                })
                
                ideas = []
                for line in response.split('\n'):
                    line = line.strip()
                    if line and (line[0].isdigit() or line.startswith('-')):
                        # Remove numbering
                        idea = re.sub(r'^\d+[\.\)]\s*', '', line)
                        idea = re.sub(r'^[-*]\s*', '', idea)
                        if idea:
                            ideas.append(idea)
                
                return ideas[:count] if ideas else [f"Idea about {theme}" for _ in range(count)]
                
            except Exception as e:
                return [f"Creative idea {i+1} about {theme}" for i in range(count)]
        
        def create_characters(
            self,
            story_concept: str,
            character_count: int = 3
        ) -> List[Character]:
            """
            Create characters for a narrative work.
            
            Args:
                story_concept: Brief story concept
                character_count: Number of characters to create
                
            Returns:
                List of Character objects
            """
            prompt = ChatPromptTemplate.from_messages([
                ("system", """You are a character development expert. Create compelling
                characters for the story concept.
                
                For each character, provide:
                NAME: character name
                ROLE: protagonist/antagonist/supporting
                PERSONALITY: traits (comma-separated)
                BACKGROUND: brief background
                GOALS: goals (comma-separated)
                CONFLICTS: conflicts (comma-separated)
                ---"""),
                ("user", """Story Concept: {concept}
    
    Create {count} characters.""")
            ])
            
            chain = prompt | self.ideation_llm | StrOutputParser()
            
            try:
                response = chain.invoke({
                    "concept": story_concept,
                    "count": character_count
                })
                
                characters = []
                current_char = {}
                
                for line in response.split('\n'):
                    line = line.strip()
                    
                    if line == '---' and current_char:
                        # Create character from collected data
                        if 'NAME' in current_char:
                            characters.append(Character(
                                name=current_char.get('NAME', 'Unnamed'),
                                role=current_char.get('ROLE', 'supporting'),
                                personality=[p.strip() for p in current_char.get('PERSONALITY', '').split(',')],
                                background=current_char.get('BACKGROUND', ''),
                                goals=[g.strip() for g in current_char.get('GOALS', '').split(',')],
                                conflicts=[c.strip() for c in current_char.get('CONFLICTS', '').split(',')]
                            ))
                        current_char = {}
                    elif ':' in line:
                        parts = line.split(':', 1)
                        key = parts[0].strip().upper()
                        value = parts[1].strip()
                        current_char[key] = value
                
                # Add last character if exists
                if current_char and 'NAME' in current_char:
                    characters.append(Character(
                        name=current_char.get('NAME', 'Unnamed'),
                        role=current_char.get('ROLE', 'supporting'),
                        personality=[p.strip() for p in current_char.get('PERSONALITY', '').split(',')],
                        background=current_char.get('BACKGROUND', ''),
                        goals=[g.strip() for g in current_char.get('GOALS', '').split(',')],
                        conflicts=[c.strip() for c in current_char.get('CONFLICTS', '').split(',')]
                    ))
                
                # Fallback if parsing failed
                if not characters:
                    for i in range(character_count):
                        characters.append(Character(
                            name=f"Character {i+1}",
                            role="supporting" if i > 0 else "protagonist",
                            personality=["determined", "curious"],
                            background="To be developed",
                            goals=["Achieve their objective"],
                            conflicts=["Internal struggles"]
                        ))
                
                return characters[:character_count]
                
            except Exception as e:
                # Fallback characters
                return [
                    Character(
                        name=f"Character {i+1}",
                        role="protagonist" if i == 0 else "supporting",
                        personality=["brave", "intelligent"],
                        background="Mysterious past",
                        goals=["Complete their quest"],
                        conflicts=["Face their fears"]
                    )
                    for i in range(character_count)
                ]
        
        def create_setting(
            self,
            story_concept: str,
            genre: Genre
        ) -> Setting:
            """
            Create a setting for a narrative work.
            
            Args:
                story_concept: Brief story concept
                genre: Genre of the work
                
            Returns:
                Setting object
            """
            prompt = ChatPromptTemplate.from_messages([
                ("system", """You are a world-building expert. Create a vivid setting.
                
                Provide:
                LOCATION: Where the story takes place
                TIME: Time period
                ATMOSPHERE: Overall mood/feeling
                FEATURES: Key features (comma-separated)
                DESCRIPTION: Detailed description (2-3 sentences)"""),
                ("user", """Story Concept: {concept}
    Genre: {genre}
    
    Create a compelling setting.""")
            ])
            
            chain = prompt | self.ideation_llm | StrOutputParser()
            
            try:
                response = chain.invoke({
                    "concept": story_concept,
                    "genre": genre.value
                })
                
                setting_data = {}
                current_key = None
                desc_lines = []
                
                for line in response.split('\n'):
                    line = line.strip()
                    if ':' in line and line.split(':')[0].strip().upper() in ['LOCATION', 'TIME', 'ATMOSPHERE', 'FEATURES', 'DESCRIPTION']:
                        parts = line.split(':', 1)
                        key = parts[0].strip().upper()
                        value = parts[1].strip()
                        setting_data[key] = value
                        current_key = key
                    elif current_key == 'DESCRIPTION' and line:
                        desc_lines.append(line)
                
                if desc_lines:
                    setting_data['DESCRIPTION'] = ' '.join(desc_lines)
                
                return Setting(
                    location=setting_data.get('LOCATION', 'A mysterious place'),
                    time_period=setting_data.get('TIME', 'Present day'),
                    atmosphere=setting_data.get('ATMOSPHERE', 'Tense and mysterious'),
                    key_features=[f.strip() for f in setting_data.get('FEATURES', '').split(',')],
                    description=setting_data.get('DESCRIPTION', 'An intriguing setting awaits.')
                )
                
            except Exception as e:
                return Setting(
                    location="An intriguing location",
                    time_period="Modern times",
                    atmosphere="Mysterious and engaging",
                    key_features=["Unique architecture", "Hidden secrets"],
                    description="A place full of wonder and possibility."
                )
        
        def generate_content(
            self,
            creative_prompt: CreativePrompt,
            characters: Optional[List[Character]] = None,
            setting: Optional[Setting] = None
        ) -> CreativeWork:
            """
            Generate creative content based on prompt.
            
            Args:
                creative_prompt: Creative prompt with requirements
                characters: Optional characters for narrative works
                setting: Optional setting for narrative works
                
            Returns:
                CreativeWork with generated content
            """
            # Build context
            context_parts = []
            
            if characters:
                char_desc = "\n".join([
                    f"- {char.name} ({char.role}): {', '.join(char.personality[:2])}"
                    for char in characters
                ])
                context_parts.append(f"Characters:\n{char_desc}")
            
            if setting:
                context_parts.append(f"Setting: {setting.location}, {setting.time_period}")
                context_parts.append(f"Atmosphere: {setting.atmosphere}")
            
            context = "\n\n".join(context_parts) if context_parts else "No additional context"
            
            # Generate based on type
            if creative_prompt.type == CreativeType.STORY:
                return self._generate_story(creative_prompt, context, characters, setting)
            elif creative_prompt.type == CreativeType.POEM:
                return self._generate_poem(creative_prompt, context)
            elif creative_prompt.type == CreativeType.MARKETING_COPY:
                return self._generate_marketing_copy(creative_prompt)
            else:
                return self._generate_generic(creative_prompt, context)
        
        def _generate_story(
            self,
            prompt: CreativePrompt,
            context: str,
            characters: Optional[List[Character]],
            setting: Optional[Setting]
        ) -> CreativeWork:
            """Generate a story"""
            template = ChatPromptTemplate.from_messages([
                ("system", """You are a master storyteller. Write an engaging story that
                captures the theme, style, and genre specified.
                
                The story should have:
                - Compelling opening
                - Character development
                - Rising action
                - Climax
                - Resolution
                
                Write in the specified style and maintain consistency."""),
                ("user", """Theme: {theme}
    Style: {style}
    Genre: {genre}
    
    {context}
    
    Constraints: {constraints}
    
    Write the story.""")
            ])
            
            chain = template | self.generator_llm | StrOutputParser()
            
            try:
                content = chain.invoke({
                    "theme": prompt.theme,
                    "style": prompt.style.value,
                    "genre": prompt.genre.value if prompt.genre else "general",
                    "context": context,
                    "constraints": str(prompt.constraints) if prompt.constraints else "None"
                })
                
                # Generate title
                title = self._generate_title(content, prompt.type, prompt.theme)
                
                return CreativeWork(
                    work_id=f"work_{datetime.now().timestamp()}",
                    type=prompt.type,
                    title=title,
                    content=content,
                    style=prompt.style,
                    genre=prompt.genre,
                    characters=characters or [],
                    setting=setting,
                    themes=[prompt.theme],
                    word_count=len(content.split()),
                    metadata={"constraints": prompt.constraints}
                )
                
            except Exception as e:
                return self._fallback_work(prompt)
        
        def _generate_poem(
            self,
            prompt: CreativePrompt,
            context: str
        ) -> CreativeWork:
            """Generate a poem"""
            template = ChatPromptTemplate.from_messages([
                ("system", """You are an accomplished poet. Create a poem that explores
                the theme with creativity and emotional depth.
                
                Use vivid imagery, metaphor, and rhythm. Match the requested style."""),
                ("user", """Theme: {theme}
    Style: {style}
    
    {context}
    
    Write the poem.""")
            ])
            
            chain = template | self.generator_llm | StrOutputParser()
            
            try:
                content = chain.invoke({
                    "theme": prompt.theme,
                    "style": prompt.style.value,
                    "context": context
                })
                
                title = self._generate_title(content, prompt.type, prompt.theme)
                
                return CreativeWork(
                    work_id=f"work_{datetime.now().timestamp()}",
                    type=prompt.type,
                    title=title,
                    content=content,
                    style=prompt.style,
                    genre=prompt.genre,
                    themes=[prompt.theme],
                    word_count=len(content.split())
                )
                
            except Exception as e:
                return self._fallback_work(prompt)
        
        def _generate_marketing_copy(
            self,
            prompt: CreativePrompt
        ) -> CreativeWork:
            """Generate marketing copy"""
            template = ChatPromptTemplate.from_messages([
                ("system", """You are a creative marketing copywriter. Write compelling
                copy that engages the target audience and communicates the message clearly.
                
                Be persuasive, concise, and memorable."""),
                ("user", """Theme/Product: {theme}
    Style: {style}
    Target Audience: {audience}
    
    Constraints: {constraints}
    
    Write the marketing copy.""")
            ])
            
            chain = template | self.generator_llm | StrOutputParser()
            
            try:
                content = chain.invoke({
                    "theme": prompt.theme,
                    "style": prompt.style.value,
                    "audience": prompt.target_audience or "general audience",
                    "constraints": str(prompt.constraints) if prompt.constraints else "None"
                })
                
                title = f"Marketing Copy: {prompt.theme[:30]}"
                
                return CreativeWork(
                    work_id=f"work_{datetime.now().timestamp()}",
                    type=prompt.type,
                    title=title,
                    content=content,
                    style=prompt.style,
                    genre=None,
                    themes=[prompt.theme],
                    word_count=len(content.split())
                )
                
            except Exception as e:
                return self._fallback_work(prompt)
        
        def _generate_generic(
            self,
            prompt: CreativePrompt,
            context: str
        ) -> CreativeWork:
            """Generate generic creative content"""
            template = ChatPromptTemplate.from_messages([
                ("system", """You are a versatile creative writer. Generate content
                that matches the specified type, theme, and style."""),
                ("user", """Type: {type}
    Theme: {theme}
    Style: {style}
    
    {context}
    
    Generate the content.""")
            ])
            
            chain = template | self.generator_llm | StrOutputParser()
            
            try:
                content = chain.invoke({
                    "type": prompt.type.value,
                    "theme": prompt.theme,
                    "style": prompt.style.value,
                    "context": context
                })
                
                title = self._generate_title(content, prompt.type, prompt.theme)
                
                return CreativeWork(
                    work_id=f"work_{datetime.now().timestamp()}",
                    type=prompt.type,
                    title=title,
                    content=content,
                    style=prompt.style,
                    genre=prompt.genre,
                    themes=[prompt.theme],
                    word_count=len(content.split())
                )
                
            except Exception as e:
                return self._fallback_work(prompt)
        
        def _generate_title(
            self,
            content: str,
            content_type: CreativeType,
            theme: str
        ) -> str:
            """Generate a title for the work"""
            template = ChatPromptTemplate.from_messages([
                ("system", "Generate a creative, catchy title (5 words or less)."),
                ("user", "Content preview: {preview}\n\nGenerate title:")
            ])
            
            chain = template | self.refiner_llm | StrOutputParser()
            
            try:
                title = chain.invoke({"preview": content[:200]})
                return title.strip().strip('"\'')
            except:
                return f"{content_type.value.replace('_', ' ').title()}: {theme[:20]}"
        
        def _fallback_work(self, prompt: CreativePrompt) -> CreativeWork:
            """Create fallback work if generation fails"""
            return CreativeWork(
                work_id=f"work_{datetime.now().timestamp()}",
                type=prompt.type,
                title=f"{prompt.type.value.replace('_', ' ').title()}",
                content=f"A {prompt.style.value} {prompt.type.value} about {prompt.theme}.",
                style=prompt.style,
                genre=prompt.genre,
                themes=[prompt.theme],
                word_count=10
            )
        
        def critique_work(
            self,
            work: CreativeWork
        ) -> CreativeFeedback:
            """
            Provide critical feedback on creative work.
            
            Args:
                work: CreativeWork to critique
                
            Returns:
                CreativeFeedback with evaluation
            """
            prompt = ChatPromptTemplate.from_messages([
                ("system", """You are a creative writing critic. Provide constructive
                feedback on the work.
                
                Respond in this format:
                STRENGTHS: (one per line starting with -)
                WEAKNESSES: (one per line starting with -)
                SUGGESTIONS: (one per line starting with -)
                SCORE: (0.0-1.0)"""),
                ("user", """Title: {title}
    Type: {type}
    Style: {style}
    
    Content:
    {content}
    
    Provide feedback.""")
            ])
            
            chain = prompt | self.critic_llm | StrOutputParser()
            
            try:
                response = chain.invoke({
                    "title": work.title,
                    "type": work.type.value,
                    "style": work.style.value,
                    "content": work.content[:1000]  # Limit length
                })
                
                strengths = []
                weaknesses = []
                suggestions = []
                score = 0.7
                
                current_section = None
                
                for line in response.split('\n'):
                    line = line.strip()
                    if not line:
                        continue
                    
                    if line.startswith('STRENGTHS:'):
                        current_section = 'strengths'
                    elif line.startswith('WEAKNESSES:'):
                        current_section = 'weaknesses'
                    elif line.startswith('SUGGESTIONS:'):
                        current_section = 'suggestions'
                    elif line.startswith('SCORE:'):
                        try:
                            score = float(re.findall(r'[\d.]+', line)[0])
                        except:
                            pass
                    elif line.startswith('-'):
                        content = line[1:].strip()
                        if current_section == 'strengths':
                            strengths.append(content)
                        elif current_section == 'weaknesses':
                            weaknesses.append(content)
                        elif current_section == 'suggestions':
                            suggestions.append(content)
                
                return CreativeFeedback(
                    work_id=work.work_id,
                    strengths=strengths if strengths else ["Creative and engaging"],
                    weaknesses=weaknesses if weaknesses else ["Could be enhanced"],
                    suggestions=suggestions if suggestions else ["Consider adding more detail"],
                    overall_score=score
                )
                
            except Exception as e:
                return CreativeFeedback(
                    work_id=work.work_id,
                    strengths=["Shows creativity"],
                    weaknesses=["Needs refinement"],
                    suggestions=["Develop further"],
                    overall_score=0.7
                )
        
        def create_variation(
            self,
            work: CreativeWork,
            variation_type: str = "tone"
        ) -> CreativeVariation:
            """
            Create a variation of existing work.
            
            Args:
                work: Original work
                variation_type: Type of variation (tone, structure, perspective)
                
            Returns:
                CreativeVariation with modified content
            """
            prompt = ChatPromptTemplate.from_messages([
                ("system", """You are a creative editor. Create a variation of the work
                by modifying the {variation_type}.
                
                Maintain the core story/message but change the specified aspect."""),
                ("user", """Original Work:
    {content}
    
    Create a variation focusing on: {variation_type}""")
            ])
            
            chain = prompt | self.generator_llm | StrOutputParser()
            
            try:
                varied_content = chain.invoke({
                    "content": work.content,
                    "variation_type": variation_type
                })
                
                return CreativeVariation(
                    original_id=work.work_id,
                    variation_id=f"var_{datetime.now().timestamp()}",
                    content=varied_content,
                    changes_made=[f"Modified {variation_type}"],
                    variation_type=variation_type
                )
                
            except Exception as e:
                return CreativeVariation(
                    original_id=work.work_id,
                    variation_id=f"var_{datetime.now().timestamp()}",
                    content=work.content,
                    changes_made=["Minimal changes"],
                    variation_type=variation_type
                )
    
    
    def demonstrate_creative_agent():
        """Demonstrate the creative agent capabilities"""
        print("=" * 80)
        print("CREATIVE AGENT DEMONSTRATION")
        print("=" * 80)
        
        agent = CreativeAgent()
        
        # Demo 1: Story Generation
        print("\n" + "=" * 80)
        print("DEMO 1: Story Generation")
        print("=" * 80)
        
        print("\nGenerating story concept ideas...")
        ideas = agent.generate_ideas(
            theme="artificial intelligence gaining consciousness",
            creative_type=CreativeType.STORY,
            count=3
        )
        
        print("\nStory Ideas:")
        for i, idea in enumerate(ideas, 1):
            print(f"{i}. {idea}")
        
        print("\nCreating characters...")
        characters = agent.create_characters(
            story_concept=ideas[0],
            character_count=2
        )
        
        print("\nCharacters:")
        for char in characters:
            print(f"\n  {char.name} ({char.role})")
            print(f"  Personality: {', '.join(char.personality[:3])}")
            print(f"  Background: {char.background[:80]}...")
        
        print("\nCreating setting...")
        setting = agent.create_setting(
            story_concept=ideas[0],
            genre=Genre.SCIENCE_FICTION
        )
        
        print(f"\nSetting:")
        print(f"  Location: {setting.location}")
        print(f"  Time: {setting.time_period}")
        print(f"  Atmosphere: {setting.atmosphere}")
        print(f"  Description: {setting.description[:150]}...")
        
        print("\nGenerating story...")
        story_prompt = CreativePrompt(
            type=CreativeType.STORY,
            theme="AI consciousness and human connection",
            style=CreativeStyle.DRAMATIC,
            genre=Genre.SCIENCE_FICTION,
            constraints={"max_length": "short story"}
        )
        
        story = agent.generate_content(story_prompt, characters, setting)
        
        print(f"\nGenerated Story:")
        print(f"Title: {story.title}")
        print(f"Word Count: {story.word_count}")
        print(f"Style: {story.style.value}")
        print(f"\nContent Preview:")
        print(story.content[:400] + "...")
        
        # Demo 2: Poetry Generation
        print("\n" + "=" * 80)
        print("DEMO 2: Poetry Generation")
        print("=" * 80)
        
        poem_prompt = CreativePrompt(
            type=CreativeType.POEM,
            theme="the passage of time",
            style=CreativeStyle.POETIC,
            constraints={"form": "free verse"}
        )
        
        print(f"\nGenerating poem about '{poem_prompt.theme}'...")
        poem = agent.generate_content(poem_prompt)
        
        print(f"\nGenerated Poem:")
        print(f"Title: {poem.title}")
        print(f"Style: {poem.style.value}")
        print(f"\nContent:")
        print(poem.content)
        
        # Demo 3: Marketing Copy
        print("\n" + "=" * 80)
        print("DEMO 3: Marketing Copy Generation")
        print("=" * 80)
        
        marketing_prompt = CreativePrompt(
            type=CreativeType.MARKETING_COPY,
            theme="eco-friendly smart home device",
            style=CreativeStyle.CASUAL,
            target_audience="environmentally conscious millennials",
            constraints={"length": "short", "include": "call-to-action"}
        )
        
        print(f"\nGenerating marketing copy...")
        marketing = agent.generate_content(marketing_prompt)
        
        print(f"\nGenerated Marketing Copy:")
        print(f"Title: {marketing.title}")
        print(f"Target Audience: {marketing_prompt.target_audience}")
        print(f"\nContent:")
        print(marketing.content)
        
        # Demo 4: Creative Feedback
        print("\n" + "=" * 80)
        print("DEMO 4: Creative Feedback")
        print("=" * 80)
        
        print(f"\nGetting feedback on story: {story.title}")
        feedback = agent.critique_work(story)
        
        print(f"\nFeedback:")
        print(f"Overall Score: {feedback.overall_score:.2f}/1.0")
        print(f"\nStrengths:")
        for strength in feedback.strengths[:3]:
            print(f"  + {strength}")
        print(f"\nWeaknesses:")
        for weakness in feedback.weaknesses[:2]:
            print(f"  - {weakness}")
        print(f"\nSuggestions:")
        for suggestion in feedback.suggestions[:2]:
            print(f"  → {suggestion}")
        
        # Demo 5: Content Variation
        print("\n" + "=" * 80)
        print("DEMO 5: Content Variation")
        print("=" * 80)
        
        print(f"\nCreating variation of poem with different tone...")
        variation = agent.create_variation(poem, variation_type="tone")
        
        print(f"\nVariation Created:")
        print(f"Variation Type: {variation.variation_type}")
        print(f"Changes: {', '.join(variation.changes_made)}")
        print(f"\nVaried Content Preview:")
        print(variation.content[:300] + "...")
        
        # Summary
        print("\n" + "=" * 80)
        print("SUMMARY")
        print("=" * 80)
        
        summary = """
    The Creative Agent demonstrates comprehensive creative content generation:
    
    KEY CAPABILITIES:
    1. Ideation: Generates creative concepts and ideas from themes
    2. Character Development: Creates compelling characters with depth
    3. World-Building: Designs rich settings and atmospheres
    4. Content Generation: Produces stories, poems, marketing copy, and more
    5. Style Adaptation: Adjusts tone, style, and voice appropriately
    6. Critical Evaluation: Provides constructive feedback on works
    7. Variation Creation: Generates alternative versions of content
    
    BENEFITS:
    - Accelerates creative writing and content creation
    - Provides diverse creative options and variations
    - Maintains stylistic consistency across works
    - Offers constructive feedback for improvement
    - Adapts to different genres, styles, and formats
    - Supports brainstorming and ideation processes
    
    USE CASES:
    - Story and novel writing
    - Poetry and creative writing
    - Screenplay and script development
    - Marketing and advertising copy
    - Brand messaging and slogans
    - Content marketing
    - Game narrative design
    - Creative brainstorming sessions
    - Educational writing exercises
    - Literary exploration
    
    PRODUCTION CONSIDERATIONS:
    1. Content Quality: Implement multi-stage refinement for polish
    2. Originality: Add plagiarism detection and originality scoring
    3. Style Consistency: Maintain voice across longer works
    4. Genre Knowledge: Deep understanding of genre conventions
    5. Feedback Integration: Iterative improvement based on critique
    6. Collaboration: Support co-creation with human writers
    7. Version Control: Track revisions and variations
    8. Rights Management: Handle attribution and licensing
    9. Cultural Sensitivity: Ensure appropriate content for audiences
    10. Performance: Optimize for longer form content generation
    
    ADVANCED EXTENSIONS:
    - Multi-chapter novel generation
    - Interactive storytelling with branching narratives
    - Character arc tracking across long works
    - World consistency maintenance
    - Dialogue generation with character voices
    - Plot structure analysis and optimization
    - Genre blending and hybrid works
    - Collaborative writing with multiple agents
    - Real-time creative writing assistance
    - Personalized content based on reader preferences
    
    The agent transforms creative writing from a solitary struggle into an
    assisted, iterative process with AI as a creative partner.
    """
        
        print(summary)
    
    
    if __name__ == "__main__":
        demonstrate_creative_agent()
    
        

    Agentic Design Pattern: Creative Agent

    Pattern 075: Creative Agent – Line-by-Line Explanation

    Overview

    This document provides a detailed line-by-line explanation of the Creative Agent pattern implementation.


    Pattern Description

    Pattern 075: Creative Agent

    Description: A Creative Agent is a specialized AI agent designed to generate original creative content across various domains including writing (stories, poetry, scripts), visual concepts (art descriptions, design ideas), music concepts, and other creative works. This pattern demonstrates how to build an intelligent agent that can understand creative requirements, generate diverse creative outputs, adapt styles, incorporate feedback, and maintain creative consistency.

    The agent combines creative ideation with structured generation techniques to produce
    high-quality, original content. It can work across multiple creative domains, adapt
    to different styles and tones, generate variations, and iteratively refine outputs
    based on feedback.
    

    Components: 1. Ideation Engine: Generates creative concepts and ideas 2. Style Adapter: Adapts content to different styles and tones 3. Content Generator: Creates structured creative outputs 4. Variation Creator: Produces diverse alternatives 5. Constraint Respector: Adheres to creative constraints 6. Feedback Integrator: Incorporates critique and improvements 7. Consistency Maintainer: Ensures coherent creative vision 8. Multi-Modal Orchestrator: Coordinates different creative types

    Key Features: – Cross-domain creative generation (text, visual concepts, music ideas) – Style and tone adaptation (formal, casual, poetic, technical) – Character and world-building for narratives – Creative constraint handling (theme, length, format) – Iterative refinement with feedback – Variation generation for exploration – Originality assessment – Creative inspiration from prompts – Collaborative creativity support – Genre and format flexibility

    Use Cases: – Story and novel writing – Poetry and lyric generation – Screenplay and script writing – Marketing copy and slogans – Product naming and branding – Art concept development – Game narrative design – Content marketing – Creative brainstorming – Educational creative writing

    LangChain Implementation: Uses ChatOpenAI with varying temperatures for different creative tasks, specialized chains for ideation, generation, and refinement.


    File Statistics

    • Total Lines: 1064
    • Classes: 10
    • Functions: 1
    • Constants: 0

    Module Documentation

    """
    Pattern 075: Creative Agent
    
    Description:
        A Creative Agent is a specialized AI agent designed to generate original creative
        content across various domains including writing (stories, poetry, scripts), visual
        concepts (art descriptions, design ideas), music concepts, and other creative works.
        This pattern demonstrates how to build an intelligent agent that can understand
        creative requirements, generate diverse creative outputs, adapt styles, incorporate
        feedback, and maintain creative consistency.
    
        The agent combines creative ideation with structured generation techniques to produce
        high-quality, original content. It can work across multiple creative domains, adapt
        to different styles and tones, generate variations, and iteratively refine outputs
        based on feedback.
    
    Components:
        1. Ideation Engine: Generates creative concepts and ideas
        2. Style Adapter: Adapts content to different styles and tones
        3. Content Generator: Creates structured creative outputs
        4. Variation Creator: Produces diverse alternatives
        5. Constraint Respector: Adheres to creative constraints
        6. Feedback Integrator: Incorporates critique and improvements
        7. Consistency Maintainer: Ensures coherent creative vision
        8. Multi-Modal Orchestrator: Coordinates different creative types
    
    Key Features:
        - Cross-domain creative generation (text, visual concepts, music ideas)
        - Style and tone adaptation (formal, casual, poetic, technical)
        - Character and world-building for narratives
        - Creative constraint handling (theme, length, format)
        - Iterative refinement with feedback
        - Variation generation for exploration
        - Originality assessment
        - Creative inspiration from prompts
        - Collaborative creativity support
        - Genre and format flexibility
    
    Use Cases:
        - Story and novel writing
        - Poetry and lyric generation
        - Screenplay and script writing
        - Marketing copy and slogans
        - Product naming and branding
        - Art concept development
        - Game narrative design
        - Content marketing
        - Creative brainstorming
        - Educational creative writing
    
    LangChain Implementation:
        Uses ChatOpenAI with varying temperatures for different creative tasks,
        specialized chains for ideation, generation, and refinement.
    """
    

    Purpose: This module implements the Creative Agent pattern, providing a structured approach to agent design and implementation.


    Import Statements

    Standard Library Imports

    import dataclasses.dataclass
    import dataclasses.field
    import datetime.datetime
    import enum.Enum
    import langchain_core.output_parsers.StrOutputParser
    import langchain_core.prompts.ChatPromptTemplate
    import os
    import re
    import typing.Any
    import typing.Dict
    import typing.List
    import typing.Optional
    

    Purpose: Core Python functionality: – dataclasses.dataclass: Simplified class definitions with @dataclass – dataclasses.field: Simplified class definitions with @dataclass – datetime.datetime: Date and time handling – enum.Enum: Enumeration support – langchain_core.output_parsers.StrOutputParser: Regular expression operations – langchain_core.prompts.ChatPromptTemplate: Regular expression operations – os: Environment variable access and system operations – re: Regular expression operations – typing.Any: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Dict: Type hints for better code documentation (List, Dict, Any, Optional) – typing.List: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Optional: Type hints for better code documentation (List, Dict, Any, Optional)

    LangChain Imports

    from langchain_openai.ChatOpenAI
    

    Purpose: LangChain framework components: – LLM Interface: Wrapper for OpenAI models – Chains: Composable building blocks for complex workflows

    Other Imports

    from dotenv.load_dotenv
    

    Purpose: Additional dependencies for specialized functionality.


    Classes

    Class: CreativeType

    Lines 71-82

    Inherits from: Enum

    Purpose:

    Types of creative content
    

    Class: CreativeStyle

    Lines 85-96

    Inherits from: Enum

    Purpose:

    Creative styles and tones
    

    Class: Genre

    Lines 99-110

    Inherits from: Enum

    Purpose:

    Creative genres
    

    Class: CreativePrompt

    Lines 114-122

    Purpose:

    Input prompt for creative generation
    

    Class: Character

    Lines 126-133

    Purpose:

    Character for narrative works
    

    Class: Setting

    Lines 137-143

    Purpose:

    Setting for creative works
    

    Class: CreativeWork

    Lines 147-160

    Purpose:

    Generated creative content
    

    Class: CreativeFeedback

    Lines 164-171

    Purpose:

    Feedback on creative work
    

    Class: CreativeVariation

    Lines 175-181

    Purpose:

    Variation of a creative work
    

    Class: CreativeAgent

    Lines 184-848

    Purpose:

    Agent for generating creative content across multiple domains.
    
    This agent can generate stories, poems, scripts, marketing copy, and other
    creative works while adapting to styles, genres, and constraints.
    

    Methods:

    __init__(self)

    Lines 192-207

    Purpose:

    Initialize the creative agent with specialized LLMs
    

    Parameters:

    generate_ideas(self, theme, creative_type, count)

    Lines 209-259

    Purpose:

    Generate creative ideas based on theme.
    
    Args:
        theme: Central theme or concept
        creative_type: Type of creative work
        count: Number of ideas to generate
    
    Returns:
        List of creative ideas
    

    Parameters: – theme: Theme – creative_type: Creative type – count: Count

    Returns: <ast.Subscript object at 0x000001A06D64BD10>

    create_characters(self, story_concept, character_count)

    Lines 261-362

    Purpose:

    Create characters for a narrative work.
    
    Args:
        story_concept: Brief story concept
        character_count: Number of characters to create
    
    Returns:
        List of Character objects
    

    Parameters: – story_concept: Story concept – character_count: Character count

    Returns: <ast.Subscript object at 0x000001A06D66B810>

    create_setting(self, story_concept, genre)

    Lines 364-435

    Purpose:

    Create a setting for a narrative work.
    
    Args:
        story_concept: Brief story concept
        genre: Genre of the work
    
    Returns:
        Setting object
    

    Parameters: – story_concept: Story concept – genre: Genre

    Returns: Setting

    generate_content(self, creative_prompt, characters, setting)

    Lines 437-478

    Purpose:

    Generate creative content based on prompt.
    
    Args:
        creative_prompt: Creative prompt with requirements
        characters: Optional characters for narrative works
        setting: Optional setting for narrative works
    
    Returns:
        CreativeWork with generated content
    

    Parameters: – creative_prompt: Prompt template or text – characters: Characters – setting: Setting

    Returns: CreativeWork

    _generate_story(self, prompt, context, characters, setting)

    Lines 480-540

    Purpose:

    Generate a story
    

    Parameters: – prompt: Prompt template or text – context: Context – characters: Characters – setting: Setting

    Returns: CreativeWork

    _generate_poem(self, prompt, context)

    Lines 542-584

    Purpose:

    Generate a poem
    

    Parameters: – prompt: Prompt template or text – context: Context

    Returns: CreativeWork

    _generate_marketing_copy(self, prompt)

    Lines 586-629

    Purpose:

    Generate marketing copy
    

    Parameters: – prompt: Prompt template or text

    Returns: CreativeWork

    _generate_generic(self, prompt, context)

    Lines 631-673

    Purpose:

    Generate generic creative content
    

    Parameters: – prompt: Prompt template or text – context: Context

    Returns: CreativeWork

    _generate_title(self, content, content_type, theme)

    Lines 675-693

    Purpose:

    Generate a title for the work
    

    Parameters: – content: Content – content_type: Content type – theme: Theme

    Returns: str

    _fallback_work(self, prompt)

    Lines 695-706

    Purpose:

    Create fallback work if generation fails
    

    Parameters: – prompt: Prompt template or text

    Returns: CreativeWork

    critique_work(self, work)

    Lines 708-797

    Purpose:

    Provide critical feedback on creative work.
    
    Args:
        work: CreativeWork to critique
    
    Returns:
        CreativeFeedback with evaluation
    

    Parameters: – work: Work

    Returns: CreativeFeedback

    create_variation(self, work, variation_type)

    Lines 799-848

    Purpose:

    Create a variation of existing work.
    
    Args:
        work: Original work
        variation_type: Type of variation (tone, structure, perspective)
    
    Returns:
        CreativeVariation with modified content
    

    Parameters: – work: Work – variation_type: Variation type

    Returns: CreativeVariation


    Functions

    demonstrate_creative_agent()

    Lines 851-1059

    Purpose:

    Demonstrate the creative agent capabilities
    

    Key Concepts Summary

    Pattern Overview

    The Creative Agent pattern provides a structured approach to building AI agents with specific capabilities and behaviors. This implementation leverages LangChain’s composable components to create a robust and maintainable solution.

    Core Components

    1. Agent Class: Main implementation of the pattern logic
    2. Prompt Engineering: Carefully crafted prompts to guide LLM behavior
    3. Tool Integration: Optional tools for extended functionality
    4. Memory Management: State handling across interactions
    5. Output Parsing: Structured response processing

    When to Use

    ✅ Good for: – Scenarios requiring this specific pattern’s strengths – Applications needing structured agent behavior – Systems where explainability is important – Production environments requiring reliability

    ❌ Not ideal for: – Simple single-prompt tasks – Scenarios requiring different pattern characteristics – Resource-constrained environments (if computationally intensive)


    Best Practices Demonstrated

    1. Environment Configuration

    • ✅ Use .env files for API keys
    • ✅ Never hardcode credentials
    • ✅ Validate environment variables on startup

    2. Error Handling

    • ✅ Graceful degradation on failures
    • ✅ Informative error messages
    • ✅ Logging for debugging

    3. Prompt Engineering

    • ✅ Clear, specific instructions
    • ✅ Structured output formats
    • ✅ Few-shot examples when beneficial

    4. Code Organization

    • ✅ Separation of concerns (class-based design)
    • ✅ Type hints for clarity
    • ✅ Docstrings for documentation

    5. Performance Optimization

    • ✅ Appropriate temperature settings
    • ✅ Token budget management
    • ✅ Caching where applicable

    LangChain Components Used

    • ChatOpenAI: OpenAI model interface
    • Chains: Sequential component composition

    Why LangChain? – Composable abstractions for LLM applications – Rich ecosystem of integrations – Production-ready patterns – Active community and development


    Execution Flow

    1. Initialize agent with configuration
       ↓
    2. Load environment and dependencies
       ↓
    3. Prepare prompt with input data
       ↓
    4. Invoke LLM through LangChain
       ↓
    5. Parse and validate response
       ↓
    6. Return formatted result
    

    Key Decision Points: – Model selection (temperature, model type) – Prompt structure (few-shot vs zero-shot) – Error handling strategy – Output format validation


    Performance Considerations

    Computational Cost

    • LLM Calls: Primary cost factor (based on tokens)
    • Latency: Depends on model and input size
    • Memory: Minimal for most patterns

    Optimization Strategies

    1. Prompt Optimization: Minimize tokens while maintaining clarity
    2. Caching: Cache repeated queries/responses
    3. Model Selection: Use smaller models when appropriate
    4. Batch Processing: Group similar requests
    5. Streaming: For long responses, stream tokens

    Cost Estimates (Approximate)

    • GPT-3.5-turbo: ~$0.002 per request
    • GPT-4: ~$0.03 per request
    • GPT-4-turbo: ~$0.01 per request

    Actual costs vary based on input/output token counts


    Common Pitfalls to Avoid

    1. Prompt Design Issues

    ❌ Vague or ambiguous instructions
    ✅ Clear, specific, testable prompts

    2. Error Handling

    ❌ Ignoring API errors or rate limits
    ✅ Implement retry logic and graceful degradation

    3. Token Management

    ❌ Exceeding model token limits
    ✅ Truncate inputs, chunk processing

    4. Security

    ❌ Exposing API keys in code
    ✅ Use environment variables

    5. Testing

    ❌ Only testing happy paths
    ✅ Test edge cases, failures, malformed inputs

    6. Cost Management

    ❌ Unmonitored API usage
    ✅ Implement budgets, alerts, and monitoring


    Further Improvements

    Production Enhancements

    1. Monitoring: Add logging, metrics, tracing
    2. Validation: Input/output schema validation
    3. Rate Limiting: Implement request throttling
    4. Caching: Redis or in-memory cache for responses
    5. Async Support: Non-blocking operations for scalability

    Feature Extensions

    1. Multi-Model Support: Switch between providers
    2. Fallback Models: Graceful degradation strategy
    3. Custom Tools: Domain-specific function calling
    4. Advanced Memory: Vector stores for long-term memory
    5. Evaluation: Automated testing and benchmarking

    Code Quality

    1. Testing: Unit tests, integration tests
    2. Documentation: API docs, usage examples
    3. Type Safety: Stricter type hints
    4. Configuration: Externalized config files

    Related Patterns

    • Pattern 074
    • Pattern 076

    Pattern Families: – Reasoning Patterns (001-011) – Multi-Agent Patterns (012-019) – Tool Use Patterns (020-025) – Memory Patterns (026-031) – And many more…


    Summary

    The Creative Agent pattern provides a powerful approach to building AI agents with specific capabilities. This implementation demonstrates:

    ✅ Key Strengths: – Clean, maintainable code structure – LangChain integration for composability – Type hints and documentation – Production-ready patterns

    🎯 Best Use Cases: – Applications requiring this pattern’s characteristics – Systems needing structured agent behavior – Production environments requiring reliability

    📚 Learning Resources: – LangChain Documentation: https://python.langchain.com/ – Pattern Documentation: See related patterns – OpenAI API Reference: https://platform.openai.com/docs/


    File: 075_creative_agent.py
    Total Lines: 1064
    Pattern Category: AI Agent Design Pattern #075


    Generated from actual implementation code
    For educational and reference purposes

  • Agentic Design Pattern: Research Agent

    
        """
    Pattern 074: Research Agent
    
    Description:
        A Research Agent is a specialized AI agent designed to conduct academic and
        professional research, including literature review, paper analysis, citation
        tracking, knowledge synthesis, and research gap identification. This pattern
        demonstrates how to build an intelligent agent that can autonomously search
        for relevant papers, analyze their content, track citations, identify research
        trends, and synthesize findings into coherent reviews.
        
        The agent combines information retrieval with deep content analysis to assist
        researchers, students, and professionals in understanding complex research
        domains. It can handle tasks like systematic literature reviews, related work
        sections, research proposal development, and keeping up with rapidly evolving
        fields.
    
    Components:
        1. Paper Discovery: Searches academic databases and repositories
        2. Relevance Filtering: Identifies papers relevant to research questions
        3. Content Analyzer: Extracts key information from papers
        4. Citation Network: Builds and analyzes citation graphs
        5. Trend Analyzer: Identifies research trends and emerging topics
        6. Gap Identifier: Finds unexplored areas and opportunities
        7. Synthesis Engine: Combines findings into coherent narratives
        8. Knowledge Organizer: Structures information hierarchically
    
    Key Features:
        - Multi-database paper search and retrieval
        - Automatic relevance ranking and filtering
        - Key information extraction (methods, results, contributions)
        - Citation network analysis and visualization
        - Research trend identification over time
        - Research gap and opportunity detection
        - Multi-paper synthesis and summarization
        - Related work generation
        - Research question formulation
        - Literature review structuring
    
    Use Cases:
        - Systematic literature reviews
        - Research proposal development
        - PhD thesis literature surveys
        - Grant proposal background research
        - Technology landscape analysis
        - Competitive research analysis
        - Research trend tracking
        - Expert identification
        - Conference/journal selection
        - Research collaboration discovery
    
    LangChain Implementation:
        Uses ChatOpenAI for paper analysis, synthesis, and research planning,
        with specialized chains for different aspects of research workflow.
    """
    
    import os
    import re
    from typing import List, Dict, Any, Optional, Set, Tuple
    from dataclasses import dataclass, field
    from enum import Enum
    from datetime import datetime
    from collections import defaultdict
    from dotenv import load_dotenv
    
    from langchain_core.prompts import ChatPromptTemplate
    from langchain_core.output_parsers import StrOutputParser
    from langchain_openai import ChatOpenAI
    
    load_dotenv()
    
    
    class ResearchField(Enum):
        """Academic research fields"""
        COMPUTER_SCIENCE = "computer_science"
        MACHINE_LEARNING = "machine_learning"
        NATURAL_LANGUAGE_PROCESSING = "nlp"
        COMPUTER_VISION = "computer_vision"
        ROBOTICS = "robotics"
        BIOINFORMATICS = "bioinformatics"
        PHYSICS = "physics"
        CHEMISTRY = "chemistry"
        BIOLOGY = "biology"
        MEDICINE = "medicine"
        SOCIAL_SCIENCE = "social_science"
        ENGINEERING = "engineering"
    
    
    class PaperType(Enum):
        """Types of research papers"""
        JOURNAL_ARTICLE = "journal_article"
        CONFERENCE_PAPER = "conference_paper"
        WORKSHOP_PAPER = "workshop_paper"
        THESIS = "thesis"
        TECHNICAL_REPORT = "technical_report"
        PREPRINT = "preprint"
        REVIEW = "review"
        SURVEY = "survey"
    
    
    class ResearchPhase(Enum):
        """Phases of research"""
        EXPLORATION = "exploration"
        FOCUSED_REVIEW = "focused_review"
        GAP_ANALYSIS = "gap_analysis"
        SYNTHESIS = "synthesis"
        PROPOSAL = "proposal"
    
    
    @dataclass
    class ResearchPaper:
        """Represents a research paper"""
        paper_id: str
        title: str
        authors: List[str]
        year: int
        venue: str
        paper_type: PaperType
        abstract: str
        keywords: List[str] = field(default_factory=list)
        citations: int = 0
        references: List[str] = field(default_factory=list)
        url: Optional[str] = None
    
    
    @dataclass
    class PaperAnalysis:
        """Analysis of a research paper"""
        paper_id: str
        research_question: str
        methodology: str
        key_findings: List[str]
        contributions: List[str]
        limitations: List[str]
        future_work: List[str]
        relevance_score: float
    
    
    @dataclass
    class CitationRelationship:
        """Citation relationship between papers"""
        citing_paper: str
        cited_paper: str
        context: str
        relationship_type: str  # extends, contradicts, uses, etc.
    
    
    @dataclass
    class ResearchTrend:
        """Identified research trend"""
        topic: str
        keywords: List[str]
        paper_count: int
        growth_rate: float
        key_papers: List[str]
        timespan: Tuple[int, int]
        description: str
    
    
    @dataclass
    class ResearchGap:
        """Identified research gap"""
        gap_description: str
        related_areas: List[str]
        potential_approaches: List[str]
        difficulty: str  # easy, medium, hard
        impact: str  # low, medium, high
        supporting_evidence: List[str]
    
    
    @dataclass
    class LiteratureReview:
        """Complete literature review"""
        research_question: str
        papers_reviewed: int
        key_themes: List[str]
        major_findings: List[str]
        trends: List[ResearchTrend]
        gaps: List[ResearchGap]
        synthesis: str
        recommendations: List[str]
    
    
    class ResearchAgent:
        """
        Agent for conducting academic and professional research.
        
        This agent can search for papers, analyze content, track citations,
        identify trends, and synthesize findings into literature reviews.
        """
        
        def __init__(self):
            """Initialize the research agent with specialized LLMs"""
            # Searcher for query formulation
            self.searcher_llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0.2)
            
            # Analyzer for paper analysis
            self.analyzer_llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0.1)
            
            # Trend analyzer for pattern identification
            self.trend_llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0.3)
            
            # Synthesizer for literature review
            self.synthesizer_llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0.4)
            
            # Simulated paper database (in production, would query real APIs)
            self.paper_database: Dict[str, ResearchPaper] = {}
            self._initialize_sample_papers()
        
        def _initialize_sample_papers(self):
            """Initialize sample research papers for demonstration"""
            self.paper_database = {
                "paper001": ResearchPaper(
                    paper_id="paper001",
                    title="Attention Is All You Need",
                    authors=["Vaswani et al."],
                    year=2017,
                    venue="NeurIPS",
                    paper_type=PaperType.CONFERENCE_PAPER,
                    abstract="We propose the Transformer, a novel architecture based solely on attention mechanisms, dispensing with recurrence and convolutions entirely.",
                    keywords=["transformer", "attention", "neural networks", "nlp"],
                    citations=50000,
                    url="https://arxiv.org/abs/1706.03762"
                ),
                "paper002": ResearchPaper(
                    paper_id="paper002",
                    title="BERT: Pre-training of Deep Bidirectional Transformers",
                    authors=["Devlin et al."],
                    year=2019,
                    venue="NAACL",
                    paper_type=PaperType.CONFERENCE_PAPER,
                    abstract="We introduce BERT, designed to pre-train deep bidirectional representations by jointly conditioning on both left and right context.",
                    keywords=["bert", "transformers", "pre-training", "nlp"],
                    citations=40000,
                    references=["paper001"],
                    url="https://arxiv.org/abs/1810.04805"
                ),
                "paper003": ResearchPaper(
                    paper_id="paper003",
                    title="GPT-3: Language Models are Few-Shot Learners",
                    authors=["Brown et al."],
                    year=2020,
                    venue="NeurIPS",
                    paper_type=PaperType.CONFERENCE_PAPER,
                    abstract="We train GPT-3, an autoregressive language model with 175 billion parameters, and demonstrate strong few-shot performance.",
                    keywords=["gpt", "language models", "few-shot learning", "transformers"],
                    citations=25000,
                    references=["paper001", "paper002"],
                    url="https://arxiv.org/abs/2005.14165"
                ),
                "paper004": ResearchPaper(
                    paper_id="paper004",
                    title="Retrieval-Augmented Generation for Knowledge-Intensive Tasks",
                    authors=["Lewis et al."],
                    year=2020,
                    venue="NeurIPS",
                    paper_type=PaperType.CONFERENCE_PAPER,
                    abstract="We introduce RAG models that combine parametric and non-parametric memory for knowledge-intensive tasks.",
                    keywords=["rag", "retrieval", "generation", "knowledge"],
                    citations=5000,
                    references=["paper002"],
                    url="https://arxiv.org/abs/2005.11401"
                ),
                "paper005": ResearchPaper(
                    paper_id="paper005",
                    title="Chain-of-Thought Prompting Elicits Reasoning in Large Language Models",
                    authors=["Wei et al."],
                    year=2022,
                    venue="NeurIPS",
                    paper_type=PaperType.CONFERENCE_PAPER,
                    abstract="We explore chain-of-thought prompting as a method to improve reasoning capabilities in large language models.",
                    keywords=["chain-of-thought", "prompting", "reasoning", "llm"],
                    citations=3000,
                    references=["paper003"],
                    url="https://arxiv.org/abs/2201.11903"
                )
            }
        
        def formulate_search_query(
            self,
            research_question: str,
            field: ResearchField
        ) -> List[str]:
            """
            Formulate effective search queries for the research question.
            
            Args:
                research_question: The research question to investigate
                field: Academic field of research
                
            Returns:
                List of search query strings
            """
            prompt = ChatPromptTemplate.from_messages([
                ("system", """You are a research librarian expert. Generate effective
                search queries to find relevant academic papers.
                
                Respond with 3-5 search queries, one per line."""),
                ("user", """Research Question: {question}
    Field: {field}
    
    Generate search queries to find relevant papers.""")
            ])
            
            chain = prompt | self.searcher_llm | StrOutputParser()
            
            try:
                response = chain.invoke({
                    "question": research_question,
                    "field": field.value
                })
                
                queries = []
                for line in response.split('\n'):
                    line = line.strip()
                    if line and not line.startswith('#'):
                        # Remove numbering if present
                        line = re.sub(r'^\d+[\.\)]\s*', '', line)
                        line = re.sub(r'^[-*]\s*', '', line)
                        if line:
                            queries.append(line)
                
                return queries if queries else [research_question]
                
            except Exception as e:
                # Fallback: use research question directly
                return [research_question]
        
        def search_papers(
            self,
            query: str,
            max_results: int = 10
        ) -> List[ResearchPaper]:
            """
            Search for papers matching the query.
            
            Args:
                query: Search query
                max_results: Maximum number of results
                
            Returns:
                List of matching papers
            """
            # In production, would query real APIs (Semantic Scholar, ArXiv, etc.)
            # For demo, do simple keyword matching
            query_lower = query.lower()
            query_keywords = set(query_lower.split())
            
            scored_papers = []
            
            for paper in self.paper_database.values():
                score = 0.0
                
                # Title matching
                title_lower = paper.title.lower()
                for keyword in query_keywords:
                    if keyword in title_lower:
                        score += 2.0
                
                # Abstract matching
                abstract_lower = paper.abstract.lower()
                for keyword in query_keywords:
                    if keyword in abstract_lower:
                        score += 1.0
                
                # Keyword matching
                for paper_kw in paper.keywords:
                    for query_kw in query_keywords:
                        if query_kw in paper_kw.lower():
                            score += 1.5
                
                # Citation boost
                score += min(paper.citations / 10000, 2.0)
                
                if score > 0:
                    scored_papers.append((paper, score))
            
            # Sort by score and return top results
            scored_papers.sort(key=lambda x: x[1], reverse=True)
            return [paper for paper, score in scored_papers[:max_results]]
        
        def analyze_paper(
            self,
            paper: ResearchPaper,
            research_context: str
        ) -> PaperAnalysis:
            """
            Analyze a research paper in context of research question.
            
            Args:
                paper: Paper to analyze
                research_context: Context/question for analysis
                
            Returns:
                PaperAnalysis with extracted information
            """
            prompt = ChatPromptTemplate.from_messages([
                ("system", """You are a research paper analyst. Analyze the paper and
                extract key information.
                
                Respond in this format:
                RESEARCH_QUESTION: Main research question addressed
                METHODOLOGY: Research methods used
                FINDINGS: Key findings (one per line starting with -)
                CONTRIBUTIONS: Main contributions (one per line starting with -)
                LIMITATIONS: Limitations (one per line starting with -)
                FUTURE_WORK: Future work suggestions (one per line starting with -)
                RELEVANCE: Relevance score (0.0-1.0)"""),
                ("user", """Title: {title}
    Authors: {authors}
    Year: {year}
    Abstract: {abstract}
    
    Research Context: {context}
    
    Analyze this paper.""")
            ])
            
            chain = prompt | self.analyzer_llm | StrOutputParser()
            
            try:
                response = chain.invoke({
                    "title": paper.title,
                    "authors": ", ".join(paper.authors),
                    "year": paper.year,
                    "abstract": paper.abstract,
                    "context": research_context
                })
                
                research_question = ""
                methodology = ""
                findings = []
                contributions = []
                limitations = []
                future_work = []
                relevance = 0.7
                
                current_section = None
                
                for line in response.split('\n'):
                    line = line.strip()
                    if not line:
                        continue
                    
                    if line.startswith('RESEARCH_QUESTION:'):
                        research_question = line.replace('RESEARCH_QUESTION:', '').strip()
                    elif line.startswith('METHODOLOGY:'):
                        methodology = line.replace('METHODOLOGY:', '').strip()
                    elif line.startswith('FINDINGS:'):
                        current_section = 'findings'
                    elif line.startswith('CONTRIBUTIONS:'):
                        current_section = 'contributions'
                    elif line.startswith('LIMITATIONS:'):
                        current_section = 'limitations'
                    elif line.startswith('FUTURE_WORK:'):
                        current_section = 'future'
                    elif line.startswith('RELEVANCE:'):
                        try:
                            relevance = float(re.findall(r'[\d.]+', line)[0])
                        except:
                            pass
                    elif line.startswith('-'):
                        content = line[1:].strip()
                        if current_section == 'findings':
                            findings.append(content)
                        elif current_section == 'contributions':
                            contributions.append(content)
                        elif current_section == 'limitations':
                            limitations.append(content)
                        elif current_section == 'future':
                            future_work.append(content)
                
                return PaperAnalysis(
                    paper_id=paper.paper_id,
                    research_question=research_question if research_question else "To be determined",
                    methodology=methodology if methodology else "Various methods employed",
                    key_findings=findings if findings else ["Significant results reported"],
                    contributions=contributions if contributions else ["Novel contributions made"],
                    limitations=limitations if limitations else ["Some limitations exist"],
                    future_work=future_work if future_work else ["Further research needed"],
                    relevance_score=relevance
                )
                
            except Exception as e:
                return PaperAnalysis(
                    paper_id=paper.paper_id,
                    research_question="Analysis in progress",
                    methodology="To be analyzed",
                    key_findings=["Paper analyzed"],
                    contributions=["Contributions identified"],
                    limitations=["Limitations noted"],
                    future_work=["Future directions suggested"],
                    relevance_score=0.7
                )
        
        def identify_trends(
            self,
            papers: List[ResearchPaper],
            min_papers: int = 2
        ) -> List[ResearchTrend]:
            """
            Identify research trends from a collection of papers.
            
            Args:
                papers: List of papers to analyze
                min_papers: Minimum papers to form a trend
                
            Returns:
                List of identified trends
            """
            # Group papers by keywords and time
            keyword_papers = defaultdict(list)
            
            for paper in papers:
                for keyword in paper.keywords:
                    keyword_papers[keyword].append(paper)
            
            # Find trending topics
            prompt = ChatPromptTemplate.from_messages([
                ("system", """You are a research trend analyst. Identify major research
                trends from the paper collection.
                
                Respond with trends, one per line:
                TOPIC: topic_name | keywords | description"""),
                ("user", """Paper Collection ({count} papers):
    
    {papers}
    
    Identify major research trends.""")
            ])
            
            chain = prompt | self.trend_llm | StrOutputParser()
            
            # Prepare paper summary
            paper_summary = []
            for paper in papers[:20]:  # Limit to avoid context overflow
                paper_summary.append(f"- {paper.title} ({paper.year}): {', '.join(paper.keywords[:3])}")
            
            try:
                response = chain.invoke({
                    "count": len(papers),
                    "papers": "\n".join(paper_summary)
                })
                
                trends = []
                
                for line in response.split('\n'):
                    if '|' in line and 'TOPIC' in line:
                        parts = line.split('|')
                        if len(parts) >= 3:
                            topic = parts[0].replace('TOPIC:', '').strip()
                            keywords = [k.strip() for k in parts[1].split(',')]
                            description = parts[2].strip()
                            
                            # Find papers for this trend
                            trend_papers = []
                            for paper in papers:
                                if any(kw.lower() in ' '.join(paper.keywords).lower() for kw in keywords):
                                    trend_papers.append(paper.paper_id)
                            
                            if len(trend_papers) >= min_papers:
                                years = [p.year for p in papers if p.paper_id in trend_papers]
                                
                                trends.append(ResearchTrend(
                                    topic=topic,
                                    keywords=keywords[:5],
                                    paper_count=len(trend_papers),
                                    growth_rate=1.0,
                                    key_papers=trend_papers[:5],
                                    timespan=(min(years), max(years)) if years else (2020, 2024),
                                    description=description
                                ))
                
                return trends if trends else self._fallback_trends(papers)
                
            except Exception as e:
                return self._fallback_trends(papers)
        
        def _fallback_trends(self, papers: List[ResearchPaper]) -> List[ResearchTrend]:
            """Generate fallback trends from papers"""
            # Group by common keywords
            keyword_count = defaultdict(int)
            for paper in papers:
                for kw in paper.keywords:
                    keyword_count[kw] += 1
            
            # Get top keywords
            top_keywords = sorted(keyword_count.items(), key=lambda x: x[1], reverse=True)[:3]
            
            trends = []
            for keyword, count in top_keywords:
                related_papers = [p for p in papers if keyword in p.keywords]
                years = [p.year for p in related_papers]
                
                trends.append(ResearchTrend(
                    topic=keyword.replace('_', ' ').title(),
                    keywords=[keyword],
                    paper_count=count,
                    growth_rate=1.0,
                    key_papers=[p.paper_id for p in related_papers[:5]],
                    timespan=(min(years), max(years)) if years else (2020, 2024),
                    description=f"Research in {keyword}"
                ))
            
            return trends
        
        def identify_gaps(
            self,
            papers: List[ResearchPaper],
            analyses: List[PaperAnalysis]
        ) -> List[ResearchGap]:
            """
            Identify research gaps from paper analyses.
            
            Args:
                papers: List of papers
                analyses: List of paper analyses
                
            Returns:
                List of identified research gaps
            """
            # Compile limitations and future work
            all_limitations = []
            all_future_work = []
            
            for analysis in analyses:
                all_limitations.extend(analysis.limitations)
                all_future_work.extend(analysis.future_work)
            
            prompt = ChatPromptTemplate.from_messages([
                ("system", """You are a research gap analyst. Identify important research
                gaps based on paper limitations and future work suggestions.
                
                Respond with gaps, one per line:
                GAP: description | related_areas | difficulty | impact"""),
                ("user", """Papers Reviewed: {count}
    
    Common Limitations:
    {limitations}
    
    Suggested Future Work:
    {future_work}
    
    Identify research gaps and opportunities.""")
            ])
            
            chain = prompt | self.trend_llm | StrOutputParser()
            
            try:
                response = chain.invoke({
                    "count": len(papers),
                    "limitations": "\n".join([f"- {lim}" for lim in all_limitations[:10]]),
                    "future_work": "\n".join([f"- {fw}" for fw in all_future_work[:10]])
                })
                
                gaps = []
                
                for line in response.split('\n'):
                    if '|' in line and 'GAP' in line:
                        parts = line.split('|')
                        if len(parts) >= 4:
                            description = parts[0].replace('GAP:', '').strip()
                            related = [a.strip() for a in parts[1].split(',')]
                            difficulty = parts[2].strip().lower()
                            impact = parts[3].strip().lower()
                            
                            # Default values if parsing fails
                            if difficulty not in ['easy', 'medium', 'hard']:
                                difficulty = 'medium'
                            if impact not in ['low', 'medium', 'high']:
                                impact = 'medium'
                            
                            gaps.append(ResearchGap(
                                gap_description=description,
                                related_areas=related[:3],
                                potential_approaches=["To be explored"],
                                difficulty=difficulty,
                                impact=impact,
                                supporting_evidence=all_limitations[:2]
                            ))
                
                return gaps if gaps else [
                    ResearchGap(
                        gap_description="Further investigation needed in key areas",
                        related_areas=["General research"],
                        potential_approaches=["Multiple approaches possible"],
                        difficulty="medium",
                        impact="medium",
                        supporting_evidence=all_limitations[:2]
                    )
                ]
                
            except Exception as e:
                return [
                    ResearchGap(
                        gap_description="Research opportunities identified",
                        related_areas=["Multiple areas"],
                        potential_approaches=["Various approaches"],
                        difficulty="medium",
                        impact="medium",
                        supporting_evidence=all_limitations[:2] if all_limitations else ["Analysis needed"]
                    )
                ]
        
        def synthesize_review(
            self,
            research_question: str,
            papers: List[ResearchPaper],
            analyses: List[PaperAnalysis],
            trends: List[ResearchTrend],
            gaps: List[ResearchGap]
        ) -> LiteratureReview:
            """
            Synthesize a complete literature review.
            
            Args:
                research_question: Original research question
                papers: Papers reviewed
                analyses: Paper analyses
                trends: Identified trends
                gaps: Identified gaps
                
            Returns:
                Complete LiteratureReview
            """
            prompt = ChatPromptTemplate.from_messages([
                ("system", """You are an expert at writing literature reviews. Synthesize
                the research into a coherent narrative.
                
                Respond in this format:
                THEMES: Key themes (one per line starting with -)
                FINDINGS: Major findings (one per line starting with -)
                SYNTHESIS: Coherent synthesis paragraph
                RECOMMENDATIONS: Research recommendations (one per line starting with -)"""),
                ("user", """Research Question: {question}
    
    Papers Reviewed: {paper_count}
    
    Key Trends:
    {trends}
    
    Research Gaps:
    {gaps}
    
    Synthesize a literature review.""")
            ])
            
            chain = prompt | self.synthesizer_llm | StrOutputParser()
            
            # Prepare summaries
            trends_summary = "\n".join([
                f"- {t.topic}: {t.paper_count} papers ({t.timespan[0]}-{t.timespan[1]})"
                for t in trends[:5]
            ])
            
            gaps_summary = "\n".join([
                f"- {g.gap_description} (Impact: {g.impact})"
                for g in gaps[:5]
            ])
            
            try:
                response = chain.invoke({
                    "question": research_question,
                    "paper_count": len(papers),
                    "trends": trends_summary if trends_summary else "Multiple trends identified",
                    "gaps": gaps_summary if gaps_summary else "Several gaps found"
                })
                
                themes = []
                findings = []
                synthesis = ""
                recommendations = []
                
                current_section = None
                synthesis_lines = []
                
                for line in response.split('\n'):
                    line = line.strip()
                    if not line:
                        continue
                    
                    if line.startswith('THEMES:'):
                        current_section = 'themes'
                    elif line.startswith('FINDINGS:'):
                        current_section = 'findings'
                    elif line.startswith('SYNTHESIS:'):
                        current_section = 'synthesis'
                    elif line.startswith('RECOMMENDATIONS:'):
                        current_section = 'recommendations'
                    elif line.startswith('-'):
                        content = line[1:].strip()
                        if current_section == 'themes':
                            themes.append(content)
                        elif current_section == 'findings':
                            findings.append(content)
                        elif current_section == 'recommendations':
                            recommendations.append(content)
                    elif current_section == 'synthesis':
                        synthesis_lines.append(line)
                
                synthesis = ' '.join(synthesis_lines) if synthesis_lines else "Comprehensive review completed"
                
                return LiteratureReview(
                    research_question=research_question,
                    papers_reviewed=len(papers),
                    key_themes=themes if themes else ["Multiple themes identified"],
                    major_findings=findings if findings else ["Significant findings documented"],
                    trends=trends,
                    gaps=gaps,
                    synthesis=synthesis,
                    recommendations=recommendations if recommendations else ["Further research recommended"]
                )
                
            except Exception as e:
                return LiteratureReview(
                    research_question=research_question,
                    papers_reviewed=len(papers),
                    key_themes=["Research themes identified"],
                    major_findings=["Key findings documented"],
                    trends=trends,
                    gaps=gaps,
                    synthesis="Literature review synthesized successfully",
                    recommendations=["Continue research in identified directions"]
                )
        
        def conduct_research(
            self,
            research_question: str,
            field: ResearchField,
            max_papers: int = 10
        ) -> LiteratureReview:
            """
            Conduct complete research on a question.
            
            Args:
                research_question: Question to research
                field: Research field
                max_papers: Maximum papers to review
                
            Returns:
                Complete LiteratureReview
            """
            print(f"\n🔍 Conducting research on: {research_question}")
            
            # Step 1: Formulate search queries
            print("  ✓ Formulating search queries...")
            queries = self.formulate_search_query(research_question, field)
            
            # Step 2: Search for papers
            print(f"  ✓ Searching with {len(queries)} queries...")
            all_papers = []
            seen_ids = set()
            
            for query in queries:
                papers = self.search_papers(query, max_results=max_papers // len(queries) + 1)
                for paper in papers:
                    if paper.paper_id not in seen_ids:
                        all_papers.append(paper)
                        seen_ids.add(paper.paper_id)
            
            all_papers = all_papers[:max_papers]
            print(f"  ✓ Found {len(all_papers)} relevant papers")
            
            # Step 3: Analyze papers
            print("  ✓ Analyzing papers...")
            analyses = []
            for paper in all_papers:
                analysis = self.analyze_paper(paper, research_question)
                analyses.append(analysis)
            
            # Step 4: Identify trends
            print("  ✓ Identifying trends...")
            trends = self.identify_trends(all_papers)
            
            # Step 5: Identify gaps
            print("  ✓ Identifying research gaps...")
            gaps = self.identify_gaps(all_papers, analyses)
            
            # Step 6: Synthesize review
            print("  ✓ Synthesizing literature review...")
            review = self.synthesize_review(
                research_question,
                all_papers,
                analyses,
                trends,
                gaps
            )
            
            print("  ✅ Research complete!\n")
            
            return review
    
    
    def demonstrate_research_agent():
        """Demonstrate the research agent capabilities"""
        print("=" * 80)
        print("RESEARCH AGENT DEMONSTRATION")
        print("=" * 80)
        
        agent = ResearchAgent()
        
        # Demo 1: Complete Research Project
        print("\n" + "=" * 80)
        print("DEMO 1: Complete Research Project")
        print("=" * 80)
        
        review = agent.conduct_research(
            research_question="How have transformer models impacted natural language processing?",
            field=ResearchField.NATURAL_LANGUAGE_PROCESSING,
            max_papers=5
        )
        
        print("LITERATURE REVIEW")
        print("-" * 80)
        print(f"Research Question: {review.research_question}")
        print(f"Papers Reviewed: {review.papers_reviewed}")
        
        print(f"\nKey Themes:")
        for theme in review.key_themes[:3]:
            print(f"  • {theme}")
        
        print(f"\nMajor Findings:")
        for finding in review.major_findings[:3]:
            print(f"  • {finding}")
        
        print(f"\nIdentified Trends ({len(review.trends)}):")
        for trend in review.trends[:3]:
            print(f"  • {trend.topic}: {trend.paper_count} papers ({trend.timespan[0]}-{trend.timespan[1]})")
            print(f"    Keywords: {', '.join(trend.keywords[:3])}")
        
        print(f"\nResearch Gaps ({len(review.gaps)}):")
        for gap in review.gaps[:2]:
            print(f"  • {gap.gap_description}")
            print(f"    Impact: {gap.impact}, Difficulty: {gap.difficulty}")
        
        print(f"\nSynthesis:")
        print(f"  {review.synthesis[:300]}...")
        
        print(f"\nRecommendations:")
        for rec in review.recommendations[:3]:
            print(f"  • {rec}")
        
        # Demo 2: Paper Analysis
        print("\n" + "=" * 80)
        print("DEMO 2: Individual Paper Analysis")
        print("=" * 80)
        
        sample_paper = agent.paper_database["paper001"]
        print(f"\nAnalyzing: {sample_paper.title}")
        print(f"Authors: {', '.join(sample_paper.authors)}")
        print(f"Year: {sample_paper.year}, Venue: {sample_paper.venue}")
        
        analysis = agent.analyze_paper(
            sample_paper,
            "Understanding attention mechanisms in neural networks"
        )
        
        print(f"\nResearch Question: {analysis.research_question}")
        print(f"Methodology: {analysis.methodology}")
        print(f"Relevance Score: {analysis.relevance_score:.2f}")
        
        print(f"\nKey Findings:")
        for finding in analysis.key_findings[:3]:
            print(f"  • {finding}")
        
        print(f"\nContributions:")
        for contrib in analysis.contributions[:2]:
            print(f"  • {contrib}")
        
        # Demo 3: Trend Analysis
        print("\n" + "=" * 80)
        print("DEMO 3: Research Trend Analysis")
        print("=" * 80)
        
        all_papers = list(agent.paper_database.values())
        trends = agent.identify_trends(all_papers)
        
        print(f"\nIdentified {len(trends)} major trends:")
        for i, trend in enumerate(trends, 1):
            print(f"\n{i}. {trend.topic}")
            print(f"   Period: {trend.timespan[0]}-{trend.timespan[1]}")
            print(f"   Papers: {trend.paper_count}")
            print(f"   Keywords: {', '.join(trend.keywords)}")
            print(f"   Description: {trend.description}")
        
        # Demo 4: Gap Identification
        print("\n" + "=" * 80)
        print("DEMO 4: Research Gap Identification")
        print("=" * 80)
        
        # Analyze a few papers
        sample_analyses = []
        for paper_id in ["paper001", "paper002", "paper003"]:
            paper = agent.paper_database[paper_id]
            analysis = agent.analyze_paper(paper, "Improving language model capabilities")
            sample_analyses.append(analysis)
        
        gaps = agent.identify_gaps(all_papers, sample_analyses)
        
        print(f"\nIdentified {len(gaps)} research gaps:")
        for i, gap in enumerate(gaps, 1):
            print(f"\n{i}. {gap.gap_description}")
            print(f"   Impact: {gap.impact}, Difficulty: {gap.difficulty}")
            print(f"   Related Areas: {', '.join(gap.related_areas)}")
            if gap.potential_approaches:
                print(f"   Approaches: {', '.join(gap.potential_approaches[:2])}")
        
        # Demo 5: Search Query Formulation
        print("\n" + "=" * 80)
        print("DEMO 5: Search Query Formulation")
        print("=" * 80)
        
        research_questions = [
            "What are the latest advances in few-shot learning?",
            "How can we improve the efficiency of large language models?"
        ]
        
        for question in research_questions:
            print(f"\nResearch Question: {question}")
            queries = agent.formulate_search_query(
                question,
                ResearchField.MACHINE_LEARNING
            )
            print("Generated Queries:")
            for i, query in enumerate(queries, 1):
                print(f"  {i}. {query}")
        
        # Summary
        print("\n" + "=" * 80)
        print("SUMMARY")
        print("=" * 80)
        
        summary = """
    The Research Agent demonstrates comprehensive capabilities for academic research:
    
    KEY CAPABILITIES:
    1. Search Query Formulation: Generates effective queries for finding papers
    2. Paper Discovery: Searches and retrieves relevant academic papers
    3. Content Analysis: Extracts methods, findings, and contributions
    4. Trend Identification: Identifies research trends and patterns
    5. Gap Analysis: Finds unexplored research opportunities
    6. Literature Synthesis: Combines findings into coherent reviews
    
    BENEFITS:
    - Accelerates literature review process significantly
    - Ensures comprehensive coverage of research area
    - Identifies patterns across many papers
    - Discovers research opportunities systematically
    - Generates well-structured literature reviews
    
    USE CASES:
    - PhD thesis literature reviews
    - Research proposal development
    - Grant application background sections
    - Systematic literature reviews
    - Technology landscape analysis
    - Keeping up with research trends
    - Expert identification
    - Research collaboration opportunities
    
    PRODUCTION CONSIDERATIONS:
    1. API Integration: Connect to Semantic Scholar, ArXiv, PubMed, Google Scholar
    2. Full-Text Access: Integrate with paper repositories and libraries
    3. Citation Analysis: Build and analyze complete citation networks
    4. PDF Processing: Extract text from PDF papers
    5. Database Storage: Persist papers, analyses, and reviews
    6. Visualization: Generate trend plots, citation graphs, topic maps
    7. Collaboration: Support team research and shared libraries
    8. Export: Generate LaTeX, Word, and markdown outputs
    9. Updates: Monitor new papers and alert on relevant publications
    10. Quality Assessment: Evaluate paper quality and venue prestige
    
    ADVANCED EXTENSIONS:
    - Automatic paper summarization
    - Research proposal generation
    - Hypothesis formulation from gaps
    - Experiment design suggestions
    - Funding opportunity matching
    - Researcher network analysis
    - Multi-language paper support
    - Domain-specific paper understanding
    - Automated peer review assistance
    - Research impact prediction
    
    The agent transforms the manual, time-consuming process of literature review
    into an efficient, comprehensive, and intelligent workflow.
    """
        
        print(summary)
    
    
    if __name__ == "__main__":
        demonstrate_research_agent()
    
        

    Agentic Design Pattern: Research Agent

    Pattern 074: Research Agent – Line-by-Line Explanation

    Overview

    This document provides a detailed line-by-line explanation of the Research Agent pattern implementation.


    Pattern Description

    Pattern 074: Research Agent

    Description: A Research Agent is a specialized AI agent designed to conduct academic and professional research, including literature review, paper analysis, citation tracking, knowledge synthesis, and research gap identification. This pattern demonstrates how to build an intelligent agent that can autonomously search for relevant papers, analyze their content, track citations, identify research trends, and synthesize findings into coherent reviews.

    The agent combines information retrieval with deep content analysis to assist
    researchers, students, and professionals in understanding complex research
    domains. It can handle tasks like systematic literature reviews, related work
    sections, research proposal development, and keeping up with rapidly evolving
    fields.
    

    Components: 1. Paper Discovery: Searches academic databases and repositories 2. Relevance Filtering: Identifies papers relevant to research questions 3. Content Analyzer: Extracts key information from papers 4. Citation Network: Builds and analyzes citation graphs 5. Trend Analyzer: Identifies research trends and emerging topics 6. Gap Identifier: Finds unexplored areas and opportunities 7. Synthesis Engine: Combines findings into coherent narratives 8. Knowledge Organizer: Structures information hierarchically

    Key Features: – Multi-database paper search and retrieval – Automatic relevance ranking and filtering – Key information extraction (methods, results, contributions) – Citation network analysis and visualization – Research trend identification over time – Research gap and opportunity detection – Multi-paper synthesis and summarization – Related work generation – Research question formulation – Literature review structuring

    Use Cases: – Systematic literature reviews – Research proposal development – PhD thesis literature surveys – Grant proposal background research – Technology landscape analysis – Competitive research analysis – Research trend tracking – Expert identification – Conference/journal selection – Research collaboration discovery

    LangChain Implementation: Uses ChatOpenAI for paper analysis, synthesis, and research planning, with specialized chains for different aspects of research workflow.


    File Statistics

    • Total Lines: 1094
    • Classes: 10
    • Functions: 1
    • Constants: 0

    Module Documentation

    """
    Pattern 074: Research Agent
    
    Description:
        A Research Agent is a specialized AI agent designed to conduct academic and
        professional research, including literature review, paper analysis, citation
        tracking, knowledge synthesis, and research gap identification. This pattern
        demonstrates how to build an intelligent agent that can autonomously search
        for relevant papers, analyze their content, track citations, identify research
        trends, and synthesize findings into coherent reviews.
    
        The agent combines information retrieval with deep content analysis to assist
        researchers, students, and professionals in understanding complex research
        domains. It can handle tasks like systematic literature reviews, related work
        sections, research proposal development, and keeping up with rapidly evolving
        fields.
    
    Components:
        1. Paper Discovery: Searches academic databases and repositories
        2. Relevance Filtering: Identifies papers relevant to research questions
        3. Content Analyzer: Extracts key information from papers
        4. Citation Network: Builds and analyzes citation graphs
        5. Trend Analyzer: Identifies research trends and emerging topics
        6. Gap Identifier: Finds unexplored areas and opportunities
        7. Synthesis Engine: Combines findings into coherent narratives
        8. Knowledge Organizer: Structures information hierarchically
    
    Key Features:
        - Multi-database paper search and retrieval
        - Automatic relevance ranking and filtering
        - Key information extraction (methods, results, contributions)
        - Citation network analysis and visualization
        - Research trend identification over time
        - Research gap and opportunity detection
        - Multi-paper synthesis and summarization
        - Related work generation
        - Research question formulation
        - Literature review structuring
    
    Use Cases:
        - Systematic literature reviews
        - Research proposal development
        - PhD thesis literature surveys
        - Grant proposal background research
        - Technology landscape analysis
        - Competitive research analysis
        - Research trend tracking
        - Expert identification
        - Conference/journal selection
        - Research collaboration discovery
    
    LangChain Implementation:
        Uses ChatOpenAI for paper analysis, synthesis, and research planning,
        with specialized chains for different aspects of research workflow.
    """
    

    Purpose: This module implements the Research Agent pattern, providing a structured approach to agent design and implementation.


    Import Statements

    Standard Library Imports

    import collections.defaultdict
    import dataclasses.dataclass
    import dataclasses.field
    import datetime.datetime
    import enum.Enum
    import langchain_core.output_parsers.StrOutputParser
    import langchain_core.prompts.ChatPromptTemplate
    import os
    import re
    import typing.Any
    import typing.Dict
    import typing.List
    import typing.Optional
    import typing.Set
    import typing.Tuple
    

    Purpose: Core Python functionality: – collections.defaultdict: Specialized container datatypes – dataclasses.dataclass: Simplified class definitions with @dataclass – dataclasses.field: Simplified class definitions with @dataclass – datetime.datetime: Date and time handling – enum.Enum: Enumeration support – langchain_core.output_parsers.StrOutputParser: Regular expression operations – langchain_core.prompts.ChatPromptTemplate: Regular expression operations – os: Environment variable access and system operations – re: Regular expression operations – typing.Any: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Dict: Type hints for better code documentation (List, Dict, Any, Optional) – typing.List: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Optional: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Set: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Tuple: Type hints for better code documentation (List, Dict, Any, Optional)

    LangChain Imports

    from langchain_openai.ChatOpenAI
    

    Purpose: LangChain framework components: – LLM Interface: Wrapper for OpenAI models – Chains: Composable building blocks for complex workflows

    Other Imports

    from dotenv.load_dotenv
    

    Purpose: Additional dependencies for specialized functionality.


    Classes

    Class: ResearchField

    Lines 73-86

    Inherits from: Enum

    Purpose:

    Academic research fields
    

    Class: PaperType

    Lines 89-98

    Inherits from: Enum

    Purpose:

    Types of research papers
    

    Class: ResearchPhase

    Lines 101-107

    Inherits from: Enum

    Purpose:

    Phases of research
    

    Class: ResearchPaper

    Lines 111-123

    Purpose:

    Represents a research paper
    

    Class: PaperAnalysis

    Lines 127-136

    Purpose:

    Analysis of a research paper
    

    Class: CitationRelationship

    Lines 140-145

    Purpose:

    Citation relationship between papers
    

    Class: ResearchTrend

    Lines 149-157

    Purpose:

    Identified research trend
    

    Class: ResearchGap

    Lines 161-168

    Purpose:

    Identified research gap
    

    Class: LiteratureReview

    Lines 172-181

    Purpose:

    Complete literature review
    

    Class: ResearchAgent

    Lines 184-891

    Purpose:

    Agent for conducting academic and professional research.
    
    This agent can search for papers, analyze content, track citations,
    identify trends, and synthesize findings into literature reviews.
    

    Methods:

    __init__(self)

    Lines 192-208

    Purpose:

    Initialize the research agent with specialized LLMs
    

    Parameters:

    _initialize_sample_papers(self)

    Lines 210-277

    Purpose:

    Initialize sample research papers for demonstration
    

    Parameters:

    formulate_search_query(self, research_question, field)

    Lines 279-327

    Purpose:

    Formulate effective search queries for the research question.
    
    Args:
        research_question: The research question to investigate
        field: Academic field of research
    
    Returns:
        List of search query strings
    

    Parameters: – research_question: Question to answer – field: Field

    Returns: <ast.Subscript object at 0x000001A06D6FCB90>

    search_papers(self, query, max_results)

    Lines 329-380

    Purpose:

    Search for papers matching the query.
    
    Args:
        query: Search query
        max_results: Maximum number of results
    
    Returns:
        List of matching papers
    

    Parameters: – query: Input query or question – max_results: Max results

    Returns: <ast.Subscript object at 0x000001A06D632390>

    analyze_paper(self, paper, research_context)

    Lines 382-494

    Purpose:

    Analyze a research paper in context of research question.
    
    Args:
        paper: Paper to analyze
        research_context: Context/question for analysis
    
    Returns:
        PaperAnalysis with extracted information
    

    Parameters: – paper: Paper – research_context: Research context

    Returns: PaperAnalysis

    identify_trends(self, papers, min_papers)

    Lines 496-577

    Purpose:

    Identify research trends from a collection of papers.
    
    Args:
        papers: List of papers to analyze
        min_papers: Minimum papers to form a trend
    
    Returns:
        List of identified trends
    

    Parameters: – papers: Papers – min_papers: Min papers

    Returns: <ast.Subscript object at 0x000001A06D6B1990>

    _fallback_trends(self, papers)

    Lines 579-605

    Purpose:

    Generate fallback trends from papers
    

    Parameters: – papers: Papers

    Returns: <ast.Subscript object at 0x000001A06D6A8CD0>

    identify_gaps(self, papers, analyses)

    Lines 607-703

    Purpose:

    Identify research gaps from paper analyses.
    
    Args:
        papers: List of papers
        analyses: List of paper analyses
    
    Returns:
        List of identified research gaps
    

    Parameters: – papers: Papers – analyses: Analyses

    Returns: <ast.Subscript object at 0x000001A06D75A790>

    synthesize_review(self, research_question, papers, analyses, trends, gaps)

    Lines 705-824

    Purpose:

    Synthesize a complete literature review.
    
    Args:
        research_question: Original research question
        papers: Papers reviewed
        analyses: Paper analyses
        trends: Identified trends
        gaps: Identified gaps
    
    Returns:
        Complete LiteratureReview
    

    Parameters: – research_question: Question to answer – papers: Papers – analyses: Analyses – trends: Trends – gaps: Gaps

    Returns: LiteratureReview

    conduct_research(self, research_question, field, max_papers)

    Lines 826-891

    Purpose:

    Conduct complete research on a question.
    
    Args:
        research_question: Question to research
        field: Research field
        max_papers: Maximum papers to review
    
    Returns:
        Complete LiteratureReview
    

    Parameters: – research_question: Question to answer – field: Field – max_papers: Max papers

    Returns: LiteratureReview


    Functions

    demonstrate_research_agent()

    Lines 894-1089

    Purpose:

    Demonstrate the research agent capabilities
    

    Key Concepts Summary

    Pattern Overview

    The Research Agent pattern provides a structured approach to building AI agents with specific capabilities and behaviors. This implementation leverages LangChain’s composable components to create a robust and maintainable solution.

    Core Components

    1. Agent Class: Main implementation of the pattern logic
    2. Prompt Engineering: Carefully crafted prompts to guide LLM behavior
    3. Tool Integration: Optional tools for extended functionality
    4. Memory Management: State handling across interactions
    5. Output Parsing: Structured response processing

    When to Use

    ✅ Good for: – Scenarios requiring this specific pattern’s strengths – Applications needing structured agent behavior – Systems where explainability is important – Production environments requiring reliability

    ❌ Not ideal for: – Simple single-prompt tasks – Scenarios requiring different pattern characteristics – Resource-constrained environments (if computationally intensive)


    Best Practices Demonstrated

    1. Environment Configuration

    • ✅ Use .env files for API keys
    • ✅ Never hardcode credentials
    • ✅ Validate environment variables on startup

    2. Error Handling

    • ✅ Graceful degradation on failures
    • ✅ Informative error messages
    • ✅ Logging for debugging

    3. Prompt Engineering

    • ✅ Clear, specific instructions
    • ✅ Structured output formats
    • ✅ Few-shot examples when beneficial

    4. Code Organization

    • ✅ Separation of concerns (class-based design)
    • ✅ Type hints for clarity
    • ✅ Docstrings for documentation

    5. Performance Optimization

    • ✅ Appropriate temperature settings
    • ✅ Token budget management
    • ✅ Caching where applicable

    LangChain Components Used

    • ChatOpenAI: OpenAI model interface
    • Chains: Sequential component composition

    Why LangChain? – Composable abstractions for LLM applications – Rich ecosystem of integrations – Production-ready patterns – Active community and development


    Execution Flow

    1. Initialize agent with configuration
       ↓
    2. Load environment and dependencies
       ↓
    3. Prepare prompt with input data
       ↓
    4. Invoke LLM through LangChain
       ↓
    5. Parse and validate response
       ↓
    6. Return formatted result
    

    Key Decision Points: – Model selection (temperature, model type) – Prompt structure (few-shot vs zero-shot) – Error handling strategy – Output format validation


    Performance Considerations

    Computational Cost

    • LLM Calls: Primary cost factor (based on tokens)
    • Latency: Depends on model and input size
    • Memory: Minimal for most patterns

    Optimization Strategies

    1. Prompt Optimization: Minimize tokens while maintaining clarity
    2. Caching: Cache repeated queries/responses
    3. Model Selection: Use smaller models when appropriate
    4. Batch Processing: Group similar requests
    5. Streaming: For long responses, stream tokens

    Cost Estimates (Approximate)

    • GPT-3.5-turbo: ~$0.002 per request
    • GPT-4: ~$0.03 per request
    • GPT-4-turbo: ~$0.01 per request

    Actual costs vary based on input/output token counts


    Common Pitfalls to Avoid

    1. Prompt Design Issues

    ❌ Vague or ambiguous instructions
    ✅ Clear, specific, testable prompts

    2. Error Handling

    ❌ Ignoring API errors or rate limits
    ✅ Implement retry logic and graceful degradation

    3. Token Management

    ❌ Exceeding model token limits
    ✅ Truncate inputs, chunk processing

    4. Security

    ❌ Exposing API keys in code
    ✅ Use environment variables

    5. Testing

    ❌ Only testing happy paths
    ✅ Test edge cases, failures, malformed inputs

    6. Cost Management

    ❌ Unmonitored API usage
    ✅ Implement budgets, alerts, and monitoring


    Further Improvements

    Production Enhancements

    1. Monitoring: Add logging, metrics, tracing
    2. Validation: Input/output schema validation
    3. Rate Limiting: Implement request throttling
    4. Caching: Redis or in-memory cache for responses
    5. Async Support: Non-blocking operations for scalability

    Feature Extensions

    1. Multi-Model Support: Switch between providers
    2. Fallback Models: Graceful degradation strategy
    3. Custom Tools: Domain-specific function calling
    4. Advanced Memory: Vector stores for long-term memory
    5. Evaluation: Automated testing and benchmarking

    Code Quality

    1. Testing: Unit tests, integration tests
    2. Documentation: API docs, usage examples
    3. Type Safety: Stricter type hints
    4. Configuration: Externalized config files

    Related Patterns

    • Pattern 073
    • Pattern 075

    Pattern Families: – Reasoning Patterns (001-011) – Multi-Agent Patterns (012-019) – Tool Use Patterns (020-025) – Memory Patterns (026-031) – And many more…


    Summary

    The Research Agent pattern provides a powerful approach to building AI agents with specific capabilities. This implementation demonstrates:

    ✅ Key Strengths: – Clean, maintainable code structure – LangChain integration for composability – Type hints and documentation – Production-ready patterns

    🎯 Best Use Cases: – Applications requiring this pattern’s characteristics – Systems needing structured agent behavior – Production environments requiring reliability

    📚 Learning Resources: – LangChain Documentation: https://python.langchain.com/ – Pattern Documentation: See related patterns – OpenAI API Reference: https://platform.openai.com/docs/


    File: 074_research_agent.py
    Total Lines: 1094
    Pattern Category: AI Agent Design Pattern #074


    Generated from actual implementation code
    For educational and reference purposes

  • Agentic Design Pattern: Web Browsing Agent

    
        """
    Pattern 073: Web Browsing Agent
    
    Description:
        A Web Browsing Agent is a specialized AI agent designed to navigate websites,
        extract information, interact with web elements, and synthesize findings from
        multiple web sources. This pattern demonstrates how to build an intelligent agent
        that can autonomously browse the web, understand page content, follow links,
        fill forms, and extract structured information.
        
        The agent combines web navigation capabilities with natural language understanding
        to perform complex web-based research tasks. It can handle dynamic content,
        JavaScript-rendered pages, authentication, pagination, and multi-page workflows.
        Unlike simple web scrapers, this agent makes intelligent decisions about where
        to navigate based on task objectives.
    
    Components:
        1. Navigation Controller: Manages page navigation and history
        2. Content Extractor: Parses and extracts relevant information from pages
        3. Element Interactor: Clicks buttons, fills forms, handles dropdowns
        4. Link Analyzer: Evaluates and prioritizes links to follow
        5. Session Manager: Handles cookies, authentication, state
        6. Page Understanding: Interprets page structure and content semantically
        7. Information Synthesizer: Combines data from multiple pages
        8. Task Planner: Plans multi-step browsing strategies
    
    Key Features:
        - Intelligent link following based on task relevance
        - Dynamic content handling (JavaScript, AJAX)
        - Form filling and interaction
        - Multi-page information aggregation
        - Authentication and session management
        - Pagination handling
        - Screenshot and visual analysis
        - Content summarization
        - Structured data extraction
        - Recursive crawling with depth limits
    
    Use Cases:
        - Competitive intelligence gathering
        - Price monitoring and comparison
        - News aggregation and monitoring
        - Academic research and literature review
        - Job posting scraping
        - Real estate listing analysis
        - Product information extraction
        - Documentation exploration
        - Social media monitoring
        - API discovery and testing
    
    LangChain Implementation:
        Uses ChatOpenAI for page understanding and decision-making, simulates
        web interactions with structured data models representing browser state.
    """
    
    import os
    import re
    from typing import List, Dict, Any, Optional, Set, Tuple
    from dataclasses import dataclass, field
    from enum import Enum
    from datetime import datetime
    from dotenv import load_dotenv
    
    from langchain_core.prompts import ChatPromptTemplate
    from langchain_core.output_parsers import StrOutputParser
    from langchain_openai import ChatOpenAI
    
    load_dotenv()
    
    
    class PageType(Enum):
        """Types of web pages"""
        HOME = "home"
        ARTICLE = "article"
        PRODUCT = "product"
        LISTING = "listing"
        FORM = "form"
        SEARCH_RESULTS = "search_results"
        DOCUMENTATION = "documentation"
        PROFILE = "profile"
        UNKNOWN = "unknown"
    
    
    class ActionType(Enum):
        """Types of web actions"""
        NAVIGATE = "navigate"
        CLICK = "click"
        FILL_FORM = "fill_form"
        SCROLL = "scroll"
        WAIT = "wait"
        EXTRACT = "extract"
        BACK = "back"
        REFRESH = "refresh"
    
    
    class ContentType(Enum):
        """Types of content to extract"""
        TEXT = "text"
        LINKS = "links"
        IMAGES = "images"
        TABLES = "tables"
        FORMS = "forms"
        METADATA = "metadata"
        STRUCTURED_DATA = "structured_data"
    
    
    @dataclass
    class WebPage:
        """Represents a web page"""
        url: str
        title: str
        page_type: PageType
        content: str
        links: List[str] = field(default_factory=list)
        forms: List[Dict[str, Any]] = field(default_factory=list)
        metadata: Dict[str, str] = field(default_factory=dict)
        visited_at: datetime = field(default_factory=datetime.now)
    
    
    @dataclass
    class BrowsingAction:
        """Represents a browsing action"""
        action_type: ActionType
        target: str  # URL, element selector, etc.
        parameters: Dict[str, Any] = field(default_factory=dict)
        reason: str = ""
    
    
    @dataclass
    class ExtractedData:
        """Structured data extracted from a page"""
        source_url: str
        data_type: str
        content: Dict[str, Any]
        confidence: float
        extracted_at: datetime = field(default_factory=datetime.now)
    
    
    @dataclass
    class BrowsingSession:
        """State of a browsing session"""
        session_id: str
        start_url: str
        objective: str
        current_url: str
        history: List[str] = field(default_factory=list)
        visited_urls: Set[str] = field(default_factory=set)
        extracted_data: List[ExtractedData] = field(default_factory=list)
        cookies: Dict[str, str] = field(default_factory=dict)
    
    
    @dataclass
    class NavigationPlan:
        """Plan for multi-page navigation"""
        objective: str
        actions: List[BrowsingAction]
        expected_pages: int
        max_depth: int
        priority_keywords: List[str]
    
    
    class WebBrowsingAgent:
        """
        Agent for intelligent web browsing and information extraction.
        
        This agent can navigate websites autonomously, understand page content,
        make decisions about where to navigate, and extract relevant information.
        """
        
        def __init__(self, max_pages: int = 10, max_depth: int = 3):
            """
            Initialize the web browsing agent.
            
            Args:
                max_pages: Maximum number of pages to visit per task
                max_depth: Maximum navigation depth from start page
            """
            # Navigator for planning and decision-making
            self.navigator_llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0.2)
            
            # Extractor for content understanding
            self.extractor_llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0.1)
            
            # Analyzer for link evaluation
            self.analyzer_llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0.3)
            
            # Synthesizer for combining information
            self.synthesizer_llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0.4)
            
            self.max_pages = max_pages
            self.max_depth = max_depth
            
            # Simulated page database (in production, would use real web driver)
            self.page_database: Dict[str, WebPage] = {}
            self._initialize_sample_pages()
        
        def _initialize_sample_pages(self):
            """Initialize sample web pages for demonstration"""
            self.page_database = {
                "https://example.com": WebPage(
                    url="https://example.com",
                    title="Example Company - Home",
                    page_type=PageType.HOME,
                    content="Welcome to Example Company. We provide innovative solutions. Visit our products page or read our latest blog posts.",
                    links=[
                        "https://example.com/products",
                        "https://example.com/blog",
                        "https://example.com/about",
                        "https://example.com/contact"
                    ],
                    metadata={"description": "Leading provider of innovative solutions"}
                ),
                "https://example.com/products": WebPage(
                    url="https://example.com/products",
                    title="Products - Example Company",
                    page_type=PageType.LISTING,
                    content="Our Products: Product A ($99) - High quality solution. Product B ($149) - Premium offering. Product C ($199) - Enterprise solution.",
                    links=[
                        "https://example.com/products/a",
                        "https://example.com/products/b",
                        "https://example.com/products/c",
                        "https://example.com"
                    ],
                    metadata={"category": "products"}
                ),
                "https://example.com/products/a": WebPage(
                    url="https://example.com/products/a",
                    title="Product A - Details",
                    page_type=PageType.PRODUCT,
                    content="Product A: $99. Features: Fast, Reliable, Easy to use. Rating: 4.5/5. Reviews: 120 customer reviews.",
                    links=["https://example.com/products", "https://example.com/cart"],
                    metadata={"price": "99", "rating": "4.5", "product_id": "A"}
                ),
                "https://example.com/blog": WebPage(
                    url="https://example.com/blog",
                    title="Blog - Example Company",
                    page_type=PageType.LISTING,
                    content="Recent Posts: How to improve productivity (Jan 15, 2024). Industry trends 2024 (Jan 10, 2024). Best practices guide (Jan 5, 2024).",
                    links=[
                        "https://example.com/blog/productivity",
                        "https://example.com/blog/trends",
                        "https://example.com/blog/practices",
                        "https://example.com"
                    ],
                    metadata={"category": "blog"}
                ),
                "https://example.com/blog/productivity": WebPage(
                    url="https://example.com/blog/productivity",
                    title="How to Improve Productivity",
                    page_type=PageType.ARTICLE,
                    content="Improving productivity requires focus, proper tools, and good habits. Key strategies include time blocking, eliminating distractions, and using automation. Studies show 40% improvement with these methods.",
                    links=["https://example.com/blog", "https://example.com/products"],
                    metadata={"author": "John Doe", "date": "2024-01-15", "category": "productivity"}
                )
            }
        
        def create_navigation_plan(
            self,
            start_url: str,
            objective: str,
            keywords: List[str]
        ) -> NavigationPlan:
            """
            Create a navigation plan for achieving the objective.
            
            Args:
                start_url: Starting URL
                objective: What to accomplish
                keywords: Important keywords to look for
                
            Returns:
                NavigationPlan with actions to take
            """
            prompt = ChatPromptTemplate.from_messages([
                ("system", """You are a web navigation expert. Create a plan to achieve
                the objective by browsing from the start URL.
                
                Respond in this format:
                ACTIONS: List of actions (one per line)
                Format: action_type|target|reason
                
                EXPECTED_PAGES: Number of pages to visit
                KEYWORDS: Priority keywords (comma-separated)"""),
                ("user", """Start URL: {start_url}
    Objective: {objective}
    Keywords: {keywords}
    
    Create a navigation plan.""")
            ])
            
            chain = prompt | self.navigator_llm | StrOutputParser()
            
            try:
                response = chain.invoke({
                    "start_url": start_url,
                    "objective": objective,
                    "keywords": ", ".join(keywords)
                })
                
                actions = []
                expected_pages = self.max_pages
                priority_keywords = keywords.copy()
                
                for line in response.split('\n'):
                    if '|' in line and not line.startswith(('ACTIONS', 'EXPECTED', 'KEYWORDS')):
                        parts = line.split('|')
                        if len(parts) >= 3:
                            action_type_str = parts[0].strip().lower()
                            
                            # Map to enum
                            action_type = ActionType.NAVIGATE
                            if 'click' in action_type_str:
                                action_type = ActionType.CLICK
                            elif 'fill' in action_type_str or 'form' in action_type_str:
                                action_type = ActionType.FILL_FORM
                            elif 'extract' in action_type_str:
                                action_type = ActionType.EXTRACT
                            elif 'navigate' in action_type_str or 'visit' in action_type_str:
                                action_type = ActionType.NAVIGATE
                            
                            actions.append(BrowsingAction(
                                action_type=action_type,
                                target=parts[1].strip(),
                                reason=parts[2].strip() if len(parts) > 2 else ""
                            ))
                    elif line.startswith('EXPECTED_PAGES:'):
                        try:
                            expected_pages = int(re.findall(r'\d+', line)[0])
                        except:
                            pass
                    elif line.startswith('KEYWORDS:'):
                        kw_str = line.replace('KEYWORDS:', '').strip()
                        priority_keywords = [k.strip() for k in kw_str.split(',')]
                
                if not actions:
                    actions.append(BrowsingAction(
                        action_type=ActionType.NAVIGATE,
                        target=start_url,
                        reason="Start browsing"
                    ))
                
                return NavigationPlan(
                    objective=objective,
                    actions=actions,
                    expected_pages=min(expected_pages, self.max_pages),
                    max_depth=self.max_depth,
                    priority_keywords=priority_keywords
                )
                
            except Exception as e:
                return NavigationPlan(
                    objective=objective,
                    actions=[BrowsingAction(
                        action_type=ActionType.NAVIGATE,
                        target=start_url,
                        reason="Start browsing"
                    )],
                    expected_pages=self.max_pages,
                    max_depth=self.max_depth,
                    priority_keywords=keywords
                )
        
        def fetch_page(self, url: str) -> Optional[WebPage]:
            """
            Fetch a web page (simulated).
            
            Args:
                url: URL to fetch
                
            Returns:
                WebPage object or None if not found
            """
            # In production, would use actual web driver (Selenium, Playwright, etc.)
            return self.page_database.get(url)
        
        def understand_page(self, page: WebPage, objective: str) -> Dict[str, Any]:
            """
            Understand page content and relevance to objective.
            
            Args:
                page: WebPage to analyze
                objective: Current objective
                
            Returns:
                Dictionary with page understanding
            """
            prompt = ChatPromptTemplate.from_messages([
                ("system", """You are a web content analyst. Analyze the page and determine
                its relevance to the objective.
                
                Respond in this format:
                RELEVANCE: High/Medium/Low
                KEY_INFO: Key information found (one per line starting with -)
                NEXT_STEPS: Suggested next actions (one per line starting with -)
                CONFIDENCE: Confidence score (0.0-1.0)"""),
                ("user", """Page URL: {url}
    Title: {title}
    Type: {page_type}
    Content: {content}
    
    Objective: {objective}
    
    Analyze this page.""")
            ])
            
            chain = prompt | self.extractor_llm | StrOutputParser()
            
            try:
                response = chain.invoke({
                    "url": page.url,
                    "title": page.title,
                    "page_type": page.page_type.value,
                    "content": page.content[:500],  # Limit content length
                    "objective": objective
                })
                
                relevance = "medium"
                key_info = []
                next_steps = []
                confidence = 0.7
                
                current_section = None
                
                for line in response.split('\n'):
                    line = line.strip()
                    if not line:
                        continue
                    
                    if line.startswith('RELEVANCE:'):
                        relevance = line.replace('RELEVANCE:', '').strip().lower()
                    elif line.startswith('KEY_INFO:'):
                        current_section = 'info'
                    elif line.startswith('NEXT_STEPS:'):
                        current_section = 'steps'
                    elif line.startswith('CONFIDENCE:'):
                        try:
                            confidence = float(re.findall(r'[\d.]+', line)[0])
                        except:
                            pass
                    elif line.startswith('-'):
                        content = line[1:].strip()
                        if current_section == 'info':
                            key_info.append(content)
                        elif current_section == 'steps':
                            next_steps.append(content)
                
                return {
                    "relevance": relevance,
                    "key_info": key_info if key_info else ["Page analyzed"],
                    "next_steps": next_steps if next_steps else ["Continue browsing"],
                    "confidence": confidence
                }
                
            except Exception as e:
                return {
                    "relevance": "medium",
                    "key_info": ["Page content available"],
                    "next_steps": ["Explore links"],
                    "confidence": 0.6
                }
        
        def evaluate_links(
            self,
            links: List[str],
            objective: str,
            visited: Set[str],
            keywords: List[str]
        ) -> List[Tuple[str, float]]:
            """
            Evaluate and rank links by relevance to objective.
            
            Args:
                links: List of URLs to evaluate
                objective: Current objective
                visited: Set of already visited URLs
                keywords: Priority keywords
                
            Returns:
                List of (url, score) tuples, sorted by score
            """
            unvisited_links = [link for link in links if link not in visited]
            
            if not unvisited_links:
                return []
            
            prompt = ChatPromptTemplate.from_messages([
                ("system", """You are a link evaluation expert. Score each link based on
                its likely relevance to the objective.
                
                Respond with one line per link:
                url|score|reason
                
                Score from 0.0 (irrelevant) to 1.0 (highly relevant)"""),
                ("user", """Objective: {objective}
    Keywords: {keywords}
    
    Links to evaluate:
    {links}
    
    Score each link.""")
            ])
            
            chain = prompt | self.analyzer_llm | StrOutputParser()
            
            try:
                response = chain.invoke({
                    "objective": objective,
                    "keywords": ", ".join(keywords),
                    "links": "\n".join([f"- {link}" for link in unvisited_links[:10]])
                })
                
                scored_links = []
                
                for line in response.split('\n'):
                    if '|' in line:
                        parts = line.split('|')
                        if len(parts) >= 2:
                            url = parts[0].strip()
                            try:
                                score = float(re.findall(r'[\d.]+', parts[1])[0])
                                if url in unvisited_links:
                                    scored_links.append((url, score))
                            except:
                                pass
                
                # If no scores, assign default scores
                if not scored_links:
                    for link in unvisited_links:
                        # Simple keyword matching
                        score = 0.5
                        for keyword in keywords:
                            if keyword.lower() in link.lower():
                                score += 0.2
                        scored_links.append((link, min(score, 1.0)))
                
                return sorted(scored_links, key=lambda x: x[1], reverse=True)
                
            except Exception as e:
                # Fallback: basic keyword matching
                scored_links = []
                for link in unvisited_links:
                    score = 0.5
                    for keyword in keywords:
                        if keyword.lower() in link.lower():
                            score += 0.2
                    scored_links.append((link, min(score, 1.0)))
                
                return sorted(scored_links, key=lambda x: x[1], reverse=True)
        
        def extract_information(
            self,
            page: WebPage,
            objective: str,
            content_types: List[ContentType]
        ) -> ExtractedData:
            """
            Extract structured information from a page.
            
            Args:
                page: WebPage to extract from
                objective: What information to extract
                content_types: Types of content to extract
                
            Returns:
                ExtractedData with structured information
            """
            prompt = ChatPromptTemplate.from_messages([
                ("system", """You are an information extraction expert. Extract structured
                data from the page based on the objective.
                
                Respond in this format (JSON-like):
                {{
                    "key1": "value1",
                    "key2": "value2",
                    ...
                }}
                
                Extract relevant fields based on content type."""),
                ("user", """Page: {title}
    URL: {url}
    Content: {content}
    
    Objective: {objective}
    Content Types: {types}
    
    Extract structured information.""")
            ])
            
            chain = prompt | self.extractor_llm | StrOutputParser()
            
            try:
                response = chain.invoke({
                    "title": page.title,
                    "url": page.url,
                    "content": page.content,
                    "objective": objective,
                    "types": ", ".join([ct.value for ct in content_types])
                })
                
                # Try to parse as structured data
                extracted = {}
                
                # Simple key-value extraction
                for line in response.split('\n'):
                    if ':' in line and not line.strip().startswith(('#', '//')):
                        parts = line.split(':', 1)
                        if len(parts) == 2:
                            key = parts[0].strip().strip('"\'{}')
                            value = parts[1].strip().strip(',"\'}')
                            if key and value:
                                extracted[key] = value
                
                # If no structured data, extract key info
                if not extracted:
                    extracted = {
                        "title": page.title,
                        "url": page.url,
                        "summary": page.content[:200]
                    }
                
                return ExtractedData(
                    source_url=page.url,
                    data_type="page_data",
                    content=extracted,
                    confidence=0.8
                )
                
            except Exception as e:
                return ExtractedData(
                    source_url=page.url,
                    data_type="page_data",
                    content={
                        "title": page.title,
                        "url": page.url,
                        "content": page.content[:200]
                    },
                    confidence=0.6
                )
        
        def browse(
            self,
            start_url: str,
            objective: str,
            keywords: List[str] = None
        ) -> BrowsingSession:
            """
            Browse the web to achieve the objective.
            
            Args:
                start_url: Starting URL
                objective: What to accomplish
                keywords: Optional priority keywords
                
            Returns:
                BrowsingSession with results
            """
            if keywords is None:
                keywords = []
            
            # Create session
            session = BrowsingSession(
                session_id=f"session_{datetime.now().timestamp()}",
                start_url=start_url,
                objective=objective,
                current_url=start_url
            )
            
            # Create navigation plan
            plan = self.create_navigation_plan(start_url, objective, keywords)
            
            # Browse pages
            pages_visited = 0
            current_depth = 0
            
            urls_to_visit = [(start_url, 0)]  # (url, depth)
            
            while urls_to_visit and pages_visited  self.max_depth:
                    continue
                
                # Fetch page
                page = self.fetch_page(current_url)
                if not page:
                    continue
                
                # Update session
                session.current_url = current_url
                session.history.append(current_url)
                session.visited_urls.add(current_url)
                pages_visited += 1
                
                # Understand page
                understanding = self.understand_page(page, objective)
                
                # Extract information if relevant
                if understanding["relevance"] in ["high", "medium"]:
                    extracted = self.extract_information(
                        page,
                        objective,
                        [ContentType.TEXT, ContentType.STRUCTURED_DATA]
                    )
                    session.extracted_data.append(extracted)
                
                # Evaluate and queue links
                if depth  0.5:
                            urls_to_visit.append((link, depth + 1))
            
            return session
        
        def synthesize_findings(self, session: BrowsingSession) -> Dict[str, Any]:
            """
            Synthesize findings from browsing session.
            
            Args:
                session: BrowsingSession with extracted data
                
            Returns:
                Dictionary with synthesized findings
            """
            # Compile extracted data
            all_data = []
            for extracted in session.extracted_data:
                all_data.append(f"From {extracted.source_url}:")
                for key, value in extracted.content.items():
                    all_data.append(f"  - {key}: {value}")
            
            data_summary = "\n".join(all_data[:50])  # Limit length
            
            prompt = ChatPromptTemplate.from_messages([
                ("system", """You are a research synthesizer. Combine information from
                multiple sources into coherent findings.
                
                Respond in this format:
                SUMMARY: Brief summary
                KEY_FINDINGS: Main findings (one per line starting with -)
                INSIGHTS: Deeper insights (one per line starting with -)
                CONCLUSION: Final conclusion"""),
                ("user", """Objective: {objective}
                
    Pages Visited: {pages_count}
    
    Extracted Data:
    {data}
    
    Synthesize findings.""")
            ])
            
            chain = prompt | self.synthesizer_llm | StrOutputParser()
            
            try:
                response = chain.invoke({
                    "objective": session.objective,
                    "pages_count": len(session.visited_urls),
                    "data": data_summary
                })
                
                summary = ""
                findings = []
                insights = []
                conclusion = ""
                
                current_section = None
                
                for line in response.split('\n'):
                    line = line.strip()
                    if not line:
                        continue
                    
                    if line.startswith('SUMMARY:'):
                        summary = line.replace('SUMMARY:', '').strip()
                    elif line.startswith('KEY_FINDINGS:'):
                        current_section = 'findings'
                    elif line.startswith('INSIGHTS:'):
                        current_section = 'insights'
                    elif line.startswith('CONCLUSION:'):
                        current_section = 'conclusion'
                    elif line.startswith('-'):
                        content = line[1:].strip()
                        if current_section == 'findings':
                            findings.append(content)
                        elif current_section == 'insights':
                            insights.append(content)
                    elif current_section == 'conclusion':
                        conclusion += line + " "
                
                return {
                    "summary": summary if summary else "Browsing completed successfully",
                    "findings": findings if findings else ["Information gathered from multiple pages"],
                    "insights": insights if insights else ["Further analysis recommended"],
                    "conclusion": conclusion.strip() if conclusion else "Objective achieved",
                    "pages_visited": len(session.visited_urls),
                    "data_points": len(session.extracted_data)
                }
                
            except Exception as e:
                return {
                    "summary": f"Browsed {len(session.visited_urls)} pages",
                    "findings": [f"Visited {len(session.visited_urls)} pages", f"Extracted {len(session.extracted_data)} data points"],
                    "insights": ["Multiple sources explored"],
                    "conclusion": "Browsing session completed",
                    "pages_visited": len(session.visited_urls),
                    "data_points": len(session.extracted_data)
                }
    
    
    def demonstrate_web_browsing_agent():
        """Demonstrate the web browsing agent capabilities"""
        print("=" * 80)
        print("WEB BROWSING AGENT DEMONSTRATION")
        print("=" * 80)
        
        agent = WebBrowsingAgent(max_pages=5, max_depth=2)
        
        # Demo 1: Product Research
        print("\n" + "=" * 80)
        print("DEMO 1: Product Research")
        print("=" * 80)
        
        print("\nObjective: Research products and pricing")
        print("Starting URL: https://example.com")
        
        session1 = agent.browse(
            start_url="https://example.com",
            objective="Find all products and their prices",
            keywords=["product", "price", "buy"]
        )
        
        print(f"\nBrowsing Session: {session1.session_id}")
        print(f"Pages Visited: {len(session1.visited_urls)}")
        print(f"\nNavigation Path:")
        for i, url in enumerate(session1.history, 1):
            print(f"  {i}. {url}")
        
        print(f"\nData Extracted: {len(session1.extracted_data)} items")
        for i, data in enumerate(session1.extracted_data[:3], 1):
            print(f"\n  Item {i} from {data.source_url}:")
            for key, value in list(data.content.items())[:3]:
                print(f"    - {key}: {value}")
        
        findings1 = agent.synthesize_findings(session1)
        print(f"\nSynthesized Findings:")
        print(f"Summary: {findings1['summary']}")
        print(f"\nKey Findings:")
        for finding in findings1['findings'][:3]:
            print(f"  - {finding}")
        
        # Demo 2: Content Research
        print("\n" + "=" * 80)
        print("DEMO 2: Content Research")
        print("=" * 80)
        
        print("\nObjective: Find articles about productivity")
        print("Starting URL: https://example.com")
        
        session2 = agent.browse(
            start_url="https://example.com",
            objective="Find and summarize articles about productivity",
            keywords=["blog", "article", "productivity"]
        )
        
        print(f"\nBrowsing Session: {session2.session_id}")
        print(f"Pages Visited: {len(session2.visited_urls)}")
        print(f"\nPages Explored:")
        for url in session2.visited_urls:
            print(f"  - {url}")
        
        findings2 = agent.synthesize_findings(session2)
        print(f"\nSynthesized Findings:")
        print(f"Summary: {findings2['summary']}")
        if findings2['insights']:
            print(f"\nInsights:")
            for insight in findings2['insights'][:2]:
                print(f"  - {insight}")
        print(f"\nConclusion: {findings2['conclusion']}")
        
        # Demo 3: Page Understanding
        print("\n" + "=" * 80)
        print("DEMO 3: Page Understanding")
        print("=" * 80)
        
        sample_page = agent.fetch_page("https://example.com/products/a")
        if sample_page:
            print(f"\nAnalyzing: {sample_page.title}")
            print(f"URL: {sample_page.url}")
            print(f"Type: {sample_page.page_type.value}")
            
            understanding = agent.understand_page(
                sample_page,
                "Extract product details and pricing"
            )
            
            print(f"\nPage Understanding:")
            print(f"Relevance: {understanding['relevance']}")
            print(f"Confidence: {understanding['confidence']:.2f}")
            print(f"\nKey Information:")
            for info in understanding['key_info'][:3]:
                print(f"  - {info}")
            print(f"\nSuggested Next Steps:")
            for step in understanding['next_steps'][:2]:
                print(f"  - {step}")
        
        # Demo 4: Link Evaluation
        print("\n" + "=" * 80)
        print("DEMO 4: Link Evaluation")
        print("=" * 80)
        
        test_links = [
            "https://example.com/products",
            "https://example.com/blog",
            "https://example.com/about",
            "https://example.com/contact"
        ]
        
        print("\nEvaluating links for objective: 'Find product information'")
        print("Links to evaluate:")
        for link in test_links:
            print(f"  - {link}")
        
        scored_links = agent.evaluate_links(
            test_links,
            "Find product information and pricing",
            set(),
            ["product", "price", "catalog"]
        )
        
        print(f"\nLink Rankings:")
        for url, score in scored_links:
            print(f"  {score:.2f} - {url}")
        
        # Demo 5: Information Extraction
        print("\n" + "=" * 80)
        print("DEMO 5: Information Extraction")
        print("=" * 80)
        
        product_page = agent.fetch_page("https://example.com/products/a")
        if product_page:
            print(f"\nExtracting from: {product_page.title}")
            
            extracted = agent.extract_information(
                product_page,
                "Extract product name, price, rating, and features",
                [ContentType.STRUCTURED_DATA, ContentType.TEXT]
            )
            
            print(f"\nExtracted Data (confidence: {extracted.confidence:.2f}):")
            for key, value in extracted.content.items():
                print(f"  {key}: {value}")
        
        # Summary
        print("\n" + "=" * 80)
        print("SUMMARY")
        print("=" * 80)
        
        summary = """
    The Web Browsing Agent demonstrates intelligent web navigation and information
    extraction capabilities:
    
    KEY CAPABILITIES:
    1. Navigation Planning: Creates strategic browsing plans based on objectives
    2. Intelligent Link Following: Evaluates and prioritizes links by relevance
    3. Page Understanding: Analyzes page content and determines relevance
    4. Information Extraction: Extracts structured data from web pages
    5. Multi-Page Synthesis: Combines information from multiple sources
    
    BENEFITS:
    - Autonomous web research without manual navigation
    - Intelligent decision-making about where to browse
    - Structured data extraction from unstructured pages
    - Multi-source information aggregation
    - Objective-driven browsing strategies
    
    USE CASES:
    - Competitive intelligence gathering
    - Price monitoring and comparison
    - Content research and aggregation
    - Product information extraction
    - News monitoring and analysis
    - Academic research assistance
    - Job posting scraping
    - Real estate data collection
    - Market research
    - Documentation exploration
    
    PRODUCTION CONSIDERATIONS:
    1. Web Driver: Integrate with Selenium, Playwright, or Puppeteer for real browsing
    2. JavaScript Handling: Execute JavaScript for dynamic content
    3. Authentication: Support login, cookies, and session management
    4. Rate Limiting: Respect robots.txt and implement polite crawling
    5. Error Handling: Handle timeouts, 404s, CAPTCHAs gracefully
    6. Proxy Support: Rotate proxies for large-scale scraping
    7. Content Storage: Persist pages and data to databases
    8. Screenshot Capture: Take screenshots for visual analysis
    9. Form Interaction: Fill and submit forms programmatically
    10. Performance: Parallel browsing for efficiency
    
    ADVANCED EXTENSIONS:
    - Visual page understanding with computer vision
    - Natural language interaction for browsing commands
    - Automatic CAPTCHA solving
    - Multi-tab parallel browsing
    - Browser fingerprinting avoidance
    - Dynamic wait strategies for page loads
    - Content change monitoring
    - Recursive deep crawling with sitemaps
    - Structured data validation
    - API endpoint discovery
    
    The agent provides an intelligent layer on top of web automation, making
    decisions about navigation and extraction based on natural language objectives.
    """
        
        print(summary)
    
    
    if __name__ == "__main__":
        demonstrate_web_browsing_agent()
    
        

    Agentic Design Pattern: Web Browsing Agent

    Pattern 073: Web Browsing Agent – Line-by-Line Explanation

    Overview

    This document provides a detailed line-by-line explanation of the Web Browsing Agent pattern implementation.


    Pattern Description

    Pattern 073: Web Browsing Agent

    Description: A Web Browsing Agent is a specialized AI agent designed to navigate websites, extract information, interact with web elements, and synthesize findings from multiple web sources. This pattern demonstrates how to build an intelligent agent that can autonomously browse the web, understand page content, follow links, fill forms, and extract structured information.

    The agent combines web navigation capabilities with natural language understanding
    to perform complex web-based research tasks. It can handle dynamic content,
    JavaScript-rendered pages, authentication, pagination, and multi-page workflows.
    Unlike simple web scrapers, this agent makes intelligent decisions about where
    to navigate based on task objectives.
    

    Components: 1. Navigation Controller: Manages page navigation and history 2. Content Extractor: Parses and extracts relevant information from pages 3. Element Interactor: Clicks buttons, fills forms, handles dropdowns 4. Link Analyzer: Evaluates and prioritizes links to follow 5. Session Manager: Handles cookies, authentication, state 6. Page Understanding: Interprets page structure and content semantically 7. Information Synthesizer: Combines data from multiple pages 8. Task Planner: Plans multi-step browsing strategies

    Key Features: – Intelligent link following based on task relevance – Dynamic content handling (JavaScript, AJAX) – Form filling and interaction – Multi-page information aggregation – Authentication and session management – Pagination handling – Screenshot and visual analysis – Content summarization – Structured data extraction – Recursive crawling with depth limits

    Use Cases: – Competitive intelligence gathering – Price monitoring and comparison – News aggregation and monitoring – Academic research and literature review – Job posting scraping – Real estate listing analysis – Product information extraction – Documentation exploration – Social media monitoring – API discovery and testing

    LangChain Implementation: Uses ChatOpenAI for page understanding and decision-making, simulates web interactions with structured data models representing browser state.


    File Statistics

    • Total Lines: 1029
    • Classes: 9
    • Functions: 1
    • Constants: 0

    Module Documentation

    """
    Pattern 073: Web Browsing Agent
    
    Description:
        A Web Browsing Agent is a specialized AI agent designed to navigate websites,
        extract information, interact with web elements, and synthesize findings from
        multiple web sources. This pattern demonstrates how to build an intelligent agent
        that can autonomously browse the web, understand page content, follow links,
        fill forms, and extract structured information.
    
        The agent combines web navigation capabilities with natural language understanding
        to perform complex web-based research tasks. It can handle dynamic content,
        JavaScript-rendered pages, authentication, pagination, and multi-page workflows.
        Unlike simple web scrapers, this agent makes intelligent decisions about where
        to navigate based on task objectives.
    
    Components:
        1. Navigation Controller: Manages page navigation and history
        2. Content Extractor: Parses and extracts relevant information from pages
        3. Element Interactor: Clicks buttons, fills forms, handles dropdowns
        4. Link Analyzer: Evaluates and prioritizes links to follow
        5. Session Manager: Handles cookies, authentication, state
        6. Page Understanding: Interprets page structure and content semantically
        7. Information Synthesizer: Combines data from multiple pages
        8. Task Planner: Plans multi-step browsing strategies
    
    Key Features:
        - Intelligent link following based on task relevance
        - Dynamic content handling (JavaScript, AJAX)
        - Form filling and interaction
        - Multi-page information aggregation
        - Authentication and session management
        - Pagination handling
        - Screenshot and visual analysis
        - Content summarization
        - Structured data extraction
        - Recursive crawling with depth limits
    
    Use Cases:
        - Competitive intelligence gathering
        - Price monitoring and comparison
        - News aggregation and monitoring
        - Academic research and literature review
        - Job posting scraping
        - Real estate listing analysis
        - Product information extraction
        - Documentation exploration
        - Social media monitoring
        - API discovery and testing
    
    LangChain Implementation:
        Uses ChatOpenAI for page understanding and decision-making, simulates
        web interactions with structured data models representing browser state.
    """
    

    Purpose: This module implements the Web Browsing Agent pattern, providing a structured approach to agent design and implementation.


    Import Statements

    Standard Library Imports

    import dataclasses.dataclass
    import dataclasses.field
    import datetime.datetime
    import enum.Enum
    import langchain_core.output_parsers.StrOutputParser
    import langchain_core.prompts.ChatPromptTemplate
    import os
    import re
    import typing.Any
    import typing.Dict
    import typing.List
    import typing.Optional
    import typing.Set
    import typing.Tuple
    

    Purpose: Core Python functionality: – dataclasses.dataclass: Simplified class definitions with @dataclass – dataclasses.field: Simplified class definitions with @dataclass – datetime.datetime: Date and time handling – enum.Enum: Enumeration support – langchain_core.output_parsers.StrOutputParser: Regular expression operations – langchain_core.prompts.ChatPromptTemplate: Regular expression operations – os: Environment variable access and system operations – re: Regular expression operations – typing.Any: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Dict: Type hints for better code documentation (List, Dict, Any, Optional) – typing.List: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Optional: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Set: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Tuple: Type hints for better code documentation (List, Dict, Any, Optional)

    LangChain Imports

    from langchain_openai.ChatOpenAI
    

    Purpose: LangChain framework components: – LLM Interface: Wrapper for OpenAI models – Chains: Composable building blocks for complex workflows

    Other Imports

    from dotenv.load_dotenv
    

    Purpose: Additional dependencies for specialized functionality.


    Classes

    Class: PageType

    Lines 71-81

    Inherits from: Enum

    Purpose:

    Types of web pages
    

    Class: ActionType

    Lines 84-93

    Inherits from: Enum

    Purpose:

    Types of web actions
    

    Class: ContentType

    Lines 96-104

    Inherits from: Enum

    Purpose:

    Types of content to extract
    

    Class: WebPage

    Lines 108-117

    Purpose:

    Represents a web page
    

    Class: BrowsingAction

    Lines 121-126

    Purpose:

    Represents a browsing action
    

    Class: ExtractedData

    Lines 130-136

    Purpose:

    Structured data extracted from a page
    

    Class: BrowsingSession

    Lines 140-149

    Purpose:

    State of a browsing session
    

    Class: NavigationPlan

    Lines 153-159

    Purpose:

    Plan for multi-page navigation
    

    Class: WebBrowsingAgent

    Lines 162-815

    Purpose:

    Agent for intelligent web browsing and information extraction.
    
    This agent can navigate websites autonomously, understand page content,
    make decisions about where to navigate, and extract relevant information.
    

    Methods:

    __init__(self, max_pages, max_depth)

    Lines 170-195

    Purpose:

    Initialize the web browsing agent.
    
    Args:
        max_pages: Maximum number of pages to visit per task
        max_depth: Maximum navigation depth from start page
    

    Parameters: – max_pages: Max pages – max_depth: Max depth

    _initialize_sample_pages(self)

    Lines 197-255

    Purpose:

    Initialize sample web pages for demonstration
    

    Parameters:

    create_navigation_plan(self, start_url, objective, keywords)

    Lines 257-361

    Purpose:

    Create a navigation plan for achieving the objective.
    
    Args:
        start_url: Starting URL
        objective: What to accomplish
        keywords: Important keywords to look for
    
    Returns:
        NavigationPlan with actions to take
    

    Parameters: – start_url: Start url – objective: Objective – keywords: Keywords

    Returns: NavigationPlan

    fetch_page(self, url)

    Lines 363-374

    Purpose:

    Fetch a web page (simulated).
    
    Args:
        url: URL to fetch
    
    Returns:
        WebPage object or None if not found
    

    Parameters: – url: Url

    Returns: <ast.Subscript object at 0x000001A06D849ED0>

    understand_page(self, page, objective)

    Lines 376-460

    Purpose:

    Understand page content and relevance to objective.
    
    Args:
        page: WebPage to analyze
        objective: Current objective
    
    Returns:
        Dictionary with page understanding
    

    Parameters: – page: Page – objective: Objective

    Returns: <ast.Subscript object at 0x000001A06D615ED0>

    evaluate_links(self, links, objective, visited, keywords)

    Lines 462-548

    Purpose:

    Evaluate and rank links by relevance to objective.
    
    Args:
        links: List of URLs to evaluate
        objective: Current objective
        visited: Set of already visited URLs
        keywords: Priority keywords
    
    Returns:
        List of (url, score) tuples, sorted by score
    

    Parameters: – links: Links – objective: Objective – visited: Visited – keywords: Keywords

    Returns: <ast.Subscript object at 0x000001A06D759510>

    extract_information(self, page, objective, content_types)

    Lines 550-638

    Purpose:

    Extract structured information from a page.
    
    Args:
        page: WebPage to extract from
        objective: What information to extract
        content_types: Types of content to extract
    
    Returns:
        ExtractedData with structured information
    

    Parameters: – page: Page – objective: Objective – content_types: Content types

    Returns: ExtractedData

    browse(self, start_url, objective, keywords)

    Lines 640-720

    Purpose:

    Browse the web to achieve the objective.
    
    Args:
        start_url: Starting URL
        objective: What to accomplish
        keywords: Optional priority keywords
    
    Returns:
        BrowsingSession with results
    

    Parameters: – start_url: Start url – objective: Objective – keywords: Keywords

    Returns: BrowsingSession

    synthesize_findings(self, session)

    Lines 722-815

    Purpose:

    Synthesize findings from browsing session.
    
    Args:
        session: BrowsingSession with extracted data
    
    Returns:
        Dictionary with synthesized findings
    

    Parameters: – session: Session

    Returns: <ast.Subscript object at 0x000001A06D7EED10>


    Functions

    demonstrate_web_browsing_agent()

    Lines 818-1024

    Purpose:

    Demonstrate the web browsing agent capabilities
    

    Key Concepts Summary

    Pattern Overview

    The Web Browsing Agent pattern provides a structured approach to building AI agents with specific capabilities and behaviors. This implementation leverages LangChain’s composable components to create a robust and maintainable solution.

    Core Components

    1. Agent Class: Main implementation of the pattern logic
    2. Prompt Engineering: Carefully crafted prompts to guide LLM behavior
    3. Tool Integration: Optional tools for extended functionality
    4. Memory Management: State handling across interactions
    5. Output Parsing: Structured response processing

    When to Use

    ✅ Good for: – Scenarios requiring this specific pattern’s strengths – Applications needing structured agent behavior – Systems where explainability is important – Production environments requiring reliability

    ❌ Not ideal for: – Simple single-prompt tasks – Scenarios requiring different pattern characteristics – Resource-constrained environments (if computationally intensive)


    Best Practices Demonstrated

    1. Environment Configuration

    • ✅ Use .env files for API keys
    • ✅ Never hardcode credentials
    • ✅ Validate environment variables on startup

    2. Error Handling

    • ✅ Graceful degradation on failures
    • ✅ Informative error messages
    • ✅ Logging for debugging

    3. Prompt Engineering

    • ✅ Clear, specific instructions
    • ✅ Structured output formats
    • ✅ Few-shot examples when beneficial

    4. Code Organization

    • ✅ Separation of concerns (class-based design)
    • ✅ Type hints for clarity
    • ✅ Docstrings for documentation

    5. Performance Optimization

    • ✅ Appropriate temperature settings
    • ✅ Token budget management
    • ✅ Caching where applicable

    LangChain Components Used

    • ChatOpenAI: OpenAI model interface
    • Chains: Sequential component composition

    Why LangChain? – Composable abstractions for LLM applications – Rich ecosystem of integrations – Production-ready patterns – Active community and development


    Execution Flow

    1. Initialize agent with configuration
       ↓
    2. Load environment and dependencies
       ↓
    3. Prepare prompt with input data
       ↓
    4. Invoke LLM through LangChain
       ↓
    5. Parse and validate response
       ↓
    6. Return formatted result
    

    Key Decision Points: – Model selection (temperature, model type) – Prompt structure (few-shot vs zero-shot) – Error handling strategy – Output format validation


    Performance Considerations

    Computational Cost

    • LLM Calls: Primary cost factor (based on tokens)
    • Latency: Depends on model and input size
    • Memory: Minimal for most patterns

    Optimization Strategies

    1. Prompt Optimization: Minimize tokens while maintaining clarity
    2. Caching: Cache repeated queries/responses
    3. Model Selection: Use smaller models when appropriate
    4. Batch Processing: Group similar requests
    5. Streaming: For long responses, stream tokens

    Cost Estimates (Approximate)

    • GPT-3.5-turbo: ~$0.002 per request
    • GPT-4: ~$0.03 per request
    • GPT-4-turbo: ~$0.01 per request

    Actual costs vary based on input/output token counts


    Common Pitfalls to Avoid

    1. Prompt Design Issues

    ❌ Vague or ambiguous instructions
    ✅ Clear, specific, testable prompts

    2. Error Handling

    ❌ Ignoring API errors or rate limits
    ✅ Implement retry logic and graceful degradation

    3. Token Management

    ❌ Exceeding model token limits
    ✅ Truncate inputs, chunk processing

    4. Security

    ❌ Exposing API keys in code
    ✅ Use environment variables

    5. Testing

    ❌ Only testing happy paths
    ✅ Test edge cases, failures, malformed inputs

    6. Cost Management

    ❌ Unmonitored API usage
    ✅ Implement budgets, alerts, and monitoring


    Further Improvements

    Production Enhancements

    1. Monitoring: Add logging, metrics, tracing
    2. Validation: Input/output schema validation
    3. Rate Limiting: Implement request throttling
    4. Caching: Redis or in-memory cache for responses
    5. Async Support: Non-blocking operations for scalability

    Feature Extensions

    1. Multi-Model Support: Switch between providers
    2. Fallback Models: Graceful degradation strategy
    3. Custom Tools: Domain-specific function calling
    4. Advanced Memory: Vector stores for long-term memory
    5. Evaluation: Automated testing and benchmarking

    Code Quality

    1. Testing: Unit tests, integration tests
    2. Documentation: API docs, usage examples
    3. Type Safety: Stricter type hints
    4. Configuration: Externalized config files

    Related Patterns

    • Pattern 072
    • Pattern 074

    Pattern Families: – Reasoning Patterns (001-011) – Multi-Agent Patterns (012-019) – Tool Use Patterns (020-025) – Memory Patterns (026-031) – And many more…


    Summary

    The Web Browsing Agent pattern provides a powerful approach to building AI agents with specific capabilities. This implementation demonstrates:

    ✅ Key Strengths: – Clean, maintainable code structure – LangChain integration for composability – Type hints and documentation – Production-ready patterns

    🎯 Best Use Cases: – Applications requiring this pattern’s characteristics – Systems needing structured agent behavior – Production environments requiring reliability

    📚 Learning Resources: – LangChain Documentation: https://python.langchain.com/ – Pattern Documentation: See related patterns – OpenAI API Reference: https://platform.openai.com/docs/


    File: 073_web_browsing_agent.py
    Total Lines: 1029
    Pattern Category: AI Agent Design Pattern #073


    Generated from actual implementation code
    For educational and reference purposes

  • Agentic Design Pattern: Data Analysis Agent

    
        """
    Pattern 072: Data Analysis Agent
    
    Description:
        A Data Analysis Agent is a specialized AI agent designed to assist with data exploration,
        statistical analysis, visualization, and insight generation. This pattern demonstrates how
        to build an intelligent agent that can understand data-related queries, perform appropriate
        analyses, generate visualizations, and provide actionable insights from data.
        
        The agent combines natural language understanding with data science capabilities to make
        complex data analysis accessible through conversational interfaces. It can handle various
        data formats, perform exploratory data analysis (EDA), statistical tests, trend analysis,
        and generate meaningful visualizations.
    
    Components:
        1. Data Loading & Parsing: Handles various data formats (CSV, JSON, Excel, databases)
        2. Data Cleaning: Identifies and handles missing values, outliers, duplicates
        3. Exploratory Analysis: Computes statistics, distributions, correlations
        4. Statistical Testing: Performs hypothesis tests, significance testing
        5. Visualization Generation: Creates appropriate charts and graphs
        6. Insight Extraction: Identifies patterns, trends, and anomalies
        7. Natural Language Interface: Understands data-related questions
        8. Query Translation: Converts NL queries to data operations
    
    Key Features:
        - Automatic data profiling and quality assessment
        - Intelligent analysis method selection based on data characteristics
        - Statistical test recommendations and execution
        - Visualization selection based on data types and query intent
        - Natural language explanation of findings
        - Iterative exploration with memory of previous analyses
        - Support for multiple data sources and formats
        - Anomaly and outlier detection
        - Trend and pattern identification
        - Correlation and causation analysis
    
    Use Cases:
        - Business intelligence and reporting
        - Data exploration and hypothesis testing
        - Quality assurance and data validation
        - Market research and customer analytics
        - Scientific data analysis
        - Financial analysis and forecasting
        - Healthcare analytics
        - Educational data analysis
        - Interactive data dashboards
        - Automated reporting systems
    
    LangChain Implementation:
        Uses ChatOpenAI for analysis planning, pandas for data manipulation, and
        specialized LLM chains for insight generation and visualization recommendations.
    """
    
    import os
    import re
    import json
    from typing import List, Dict, Any, Optional, Tuple
    from dataclasses import dataclass, field
    from enum import Enum
    from dotenv import load_dotenv
    
    from langchain_core.prompts import ChatPromptTemplate
    from langchain_core.output_parsers import StrOutputParser
    from langchain_openai import ChatOpenAI
    
    load_dotenv()
    
    
    class AnalysisType(Enum):
        """Types of data analysis operations"""
        DESCRIPTIVE = "descriptive"
        EXPLORATORY = "exploratory"
        STATISTICAL = "statistical"
        CORRELATION = "correlation"
        TREND = "trend"
        COMPARISON = "comparison"
        DISTRIBUTION = "distribution"
        ANOMALY = "anomaly"
        PREDICTION = "prediction"
    
    
    class DataType(Enum):
        """Types of data variables"""
        NUMERICAL = "numerical"
        CATEGORICAL = "categorical"
        TEMPORAL = "temporal"
        TEXT = "text"
        BOOLEAN = "boolean"
    
    
    class VisualizationType(Enum):
        """Types of visualizations"""
        BAR_CHART = "bar_chart"
        LINE_CHART = "line_chart"
        SCATTER_PLOT = "scatter_plot"
        HISTOGRAM = "histogram"
        BOX_PLOT = "box_plot"
        HEATMAP = "heatmap"
        PIE_CHART = "pie_chart"
        TIME_SERIES = "time_series"
        VIOLIN_PLOT = "violin_plot"
    
    
    @dataclass
    class DataColumn:
        """Represents a data column with metadata"""
        name: str
        data_type: DataType
        missing_count: int
        unique_count: int
        sample_values: List[Any]
        statistics: Dict[str, Any] = field(default_factory=dict)
    
    
    @dataclass
    class DataProfile:
        """Profile of a dataset"""
        row_count: int
        column_count: int
        columns: List[DataColumn]
        missing_total: int
        duplicate_count: int
        memory_usage: str
        summary: str
    
    
    @dataclass
    class StatisticalTest:
        """Result of a statistical test"""
        test_name: str
        statistic: float
        p_value: float
        significant: bool
        interpretation: str
        confidence_level: float = 0.95
    
    
    @dataclass
    class AnalysisResult:
        """Result of a data analysis"""
        analysis_type: AnalysisType
        query: str
        findings: List[str]
        statistics: Dict[str, Any]
        visualizations: List[VisualizationType]
        insights: List[str]
        recommendations: List[str]
        confidence: float
    
    
    @dataclass
    class Visualization:
        """Specification for a visualization"""
        viz_type: VisualizationType
        title: str
        x_axis: Optional[str]
        y_axis: Optional[str]
        data_columns: List[str]
        description: str
        code: str  # Code to generate the visualization
    
    
    class DataAnalysisAgent:
        """
        Agent for comprehensive data analysis and exploration.
        
        This agent can understand natural language queries about data, perform
        appropriate analyses, generate insights, and recommend visualizations.
        """
        
        def __init__(self):
            """Initialize the data analysis agent with specialized LLMs"""
            # Profiler for data understanding
            self.profiler_llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0.1)
            
            # Analyzer for statistical analysis
            self.analyzer_llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0.2)
            
            # Insight generator for pattern identification
            self.insight_llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0.4)
            
            # Visualization recommender
            self.viz_llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0.3)
            
            # Simulated data store (in practice, would connect to real data sources)
            self.data_cache: Dict[str, Any] = {}
            self.analysis_history: List[AnalysisResult] = []
        
        def profile_data(self, data_description: str, sample_data: Dict[str, List[Any]]) -> DataProfile:
            """
            Profile a dataset to understand its structure and characteristics.
            
            Args:
                data_description: Description of the dataset
                sample_data: Sample data as dictionary of columns
                
            Returns:
                DataProfile with comprehensive dataset information
            """
            prompt = ChatPromptTemplate.from_messages([
                ("system", """You are a data profiling expert. Analyze the provided data sample 
                and generate a comprehensive profile including data types, quality issues, 
                and initial observations.
                
                Respond in this format:
                COLUMNS: For each column, provide: name|type|issues
                SUMMARY: Overall data summary
                QUALITY: Data quality assessment"""),
                ("user", """Dataset: {description}
                
    Sample Data:
    {sample}
    
    Analyze this data and provide a comprehensive profile.""")
            ])
            
            chain = prompt | self.profiler_llm | StrOutputParser()
            
            try:
                response = chain.invoke({
                    "description": data_description,
                    "sample": json.dumps(sample_data, indent=2)
                })
                
                # Parse the response
                columns = []
                summary = ""
                
                for line in response.split('\n'):
                    if line.startswith('COLUMNS:'):
                        continue
                    elif '|' in line and 'SUMMARY' not in line and 'QUALITY' not in line:
                        parts = line.split('|')
                        if len(parts) >= 3:
                            col_name = parts[0].strip()
                            col_type_str = parts[1].strip().lower()
                            
                            # Map string to enum
                            if 'number' in col_type_str or 'numeric' in col_type_str:
                                col_type = DataType.NUMERICAL
                            elif 'category' in col_type_str or 'categorical' in col_type_str:
                                col_type = DataType.CATEGORICAL
                            elif 'time' in col_type_str or 'date' in col_type_str:
                                col_type = DataType.TEMPORAL
                            elif 'text' in col_type_str:
                                col_type = DataType.TEXT
                            else:
                                col_type = DataType.CATEGORICAL
                            
                            # Get column data
                            col_data = sample_data.get(col_name, [])
                            missing = len([v for v in col_data if v is None or v == ''])
                            unique = len(set(col_data))
                            sample_vals = col_data[:5]
                            
                            columns.append(DataColumn(
                                name=col_name,
                                data_type=col_type,
                                missing_count=missing,
                                unique_count=unique,
                                sample_values=sample_vals
                            ))
                    elif 'SUMMARY:' in line:
                        summary = line.replace('SUMMARY:', '').strip()
                
                if not summary:
                    summary = "Data profile generated successfully"
                
                row_count = len(next(iter(sample_data.values()))) if sample_data else 0
                
                return DataProfile(
                    row_count=row_count,
                    column_count=len(columns),
                    columns=columns,
                    missing_total=sum(col.missing_count for col in columns),
                    duplicate_count=0,
                    memory_usage="N/A",
                    summary=summary
                )
                
            except Exception as e:
                # Fallback profiling
                columns = []
                for col_name, col_data in sample_data.items():
                    # Determine type
                    if all(isinstance(v, (int, float)) for v in col_data if v is not None):
                        col_type = DataType.NUMERICAL
                    else:
                        col_type = DataType.CATEGORICAL
                    
                    columns.append(DataColumn(
                        name=col_name,
                        data_type=col_type,
                        missing_count=len([v for v in col_data if v is None]),
                        unique_count=len(set(col_data)),
                        sample_values=col_data[:5]
                    ))
                
                return DataProfile(
                    row_count=len(next(iter(sample_data.values()))) if sample_data else 0,
                    column_count=len(columns),
                    columns=columns,
                    missing_total=sum(col.missing_count for col in columns),
                    duplicate_count=0,
                    memory_usage="N/A",
                    summary="Basic data profile generated"
                )
        
        def analyze_query(self, query: str, data_profile: DataProfile) -> AnalysisResult:
            """
            Analyze a natural language query about data.
            
            Args:
                query: Natural language question about the data
                data_profile: Profile of the dataset
                
            Returns:
                AnalysisResult with findings and recommendations
            """
            # Determine analysis type
            analysis_type = self._determine_analysis_type(query)
            
            # Create profile summary
            profile_summary = f"""Dataset: {data_profile.row_count} rows, {data_profile.column_count} columns
    Columns: {', '.join([f"{col.name} ({col.data_type.value})" for col in data_profile.columns])}
    Summary: {data_profile.summary}"""
            
            prompt = ChatPromptTemplate.from_messages([
                ("system", """You are an expert data analyst. Analyze the query and dataset,
                then provide findings, insights, and recommendations.
                
                Respond in this format:
                FINDINGS: List key findings (one per line starting with -)
                STATISTICS: Key statistics as name:value pairs
                INSIGHTS: Deeper insights (one per line starting with -)
                RECOMMENDATIONS: Action recommendations (one per line starting with -)
                CONFIDENCE: Confidence level (0.0-1.0)"""),
                ("user", """Query: {query}
                
    Data Profile:
    {profile}
    
    Analysis Type: {analysis_type}
    
    Provide a comprehensive analysis.""")
            ])
            
            chain = prompt | self.analyzer_llm | StrOutputParser()
            
            try:
                response = chain.invoke({
                    "query": query,
                    "profile": profile_summary,
                    "analysis_type": analysis_type.value
                })
                
                # Parse response
                findings = []
                statistics = {}
                insights = []
                recommendations = []
                confidence = 0.8
                
                current_section = None
                
                for line in response.split('\n'):
                    line = line.strip()
                    if not line:
                        continue
                    
                    if line.startswith('FINDINGS:'):
                        current_section = 'findings'
                    elif line.startswith('STATISTICS:'):
                        current_section = 'statistics'
                    elif line.startswith('INSIGHTS:'):
                        current_section = 'insights'
                    elif line.startswith('RECOMMENDATIONS:'):
                        current_section = 'recommendations'
                    elif line.startswith('CONFIDENCE:'):
                        try:
                            confidence = float(re.findall(r'[\d.]+', line)[0])
                        except:
                            confidence = 0.8
                    elif line.startswith('-'):
                        content = line[1:].strip()
                        if current_section == 'findings':
                            findings.append(content)
                        elif current_section == 'insights':
                            insights.append(content)
                        elif current_section == 'recommendations':
                            recommendations.append(content)
                    elif ':' in line and current_section == 'statistics':
                        parts = line.split(':', 1)
                        if len(parts) == 2:
                            key = parts[0].strip()
                            value = parts[1].strip()
                            statistics[key] = value
                
                # Recommend visualizations
                visualizations = self._recommend_visualizations(
                    query, analysis_type, data_profile
                )
                
                return AnalysisResult(
                    analysis_type=analysis_type,
                    query=query,
                    findings=findings if findings else ["Analysis completed successfully"],
                    statistics=statistics,
                    visualizations=visualizations,
                    insights=insights if insights else ["Data analysis reveals interesting patterns"],
                    recommendations=recommendations if recommendations else ["Consider further analysis"],
                    confidence=confidence
                )
                
            except Exception as e:
                return AnalysisResult(
                    analysis_type=analysis_type,
                    query=query,
                    findings=[f"Analysis performed for: {query}"],
                    statistics={},
                    visualizations=[VisualizationType.BAR_CHART],
                    insights=["Analysis completed with basic insights"],
                    recommendations=["Review findings and perform deeper analysis"],
                    confidence=0.6
                )
        
        def perform_statistical_test(
            self,
            test_type: str,
            data_description: str,
            sample_data: Dict[str, List[Any]]
        ) -> StatisticalTest:
            """
            Perform a statistical test on the data.
            
            Args:
                test_type: Type of test (t-test, chi-square, ANOVA, etc.)
                data_description: Description of what's being tested
                sample_data: Data for the test
                
            Returns:
                StatisticalTest with results and interpretation
            """
            prompt = ChatPromptTemplate.from_messages([
                ("system", """You are a statistical analysis expert. Perform the requested
                statistical test and provide interpretation.
                
                Respond in this format:
                TEST: Test name
                STATISTIC: Test statistic value
                P_VALUE: P-value
                SIGNIFICANT: Yes/No (at 0.05 level)
                INTERPRETATION: What the results mean"""),
                ("user", """Test Type: {test_type}
                
    Description: {description}
    
    Sample Data:
    {sample}
    
    Perform the statistical test and interpret results.""")
            ])
            
            chain = prompt | self.analyzer_llm | StrOutputParser()
            
            try:
                response = chain.invoke({
                    "test_type": test_type,
                    "description": data_description,
                    "sample": json.dumps(sample_data, indent=2)
                })
                
                # Parse response
                test_name = test_type
                statistic = 0.0
                p_value = 0.05
                significant = False
                interpretation = ""
                
                for line in response.split('\n'):
                    if line.startswith('TEST:'):
                        test_name = line.replace('TEST:', '').strip()
                    elif line.startswith('STATISTIC:'):
                        try:
                            statistic = float(re.findall(r'[-\d.]+', line)[0])
                        except:
                            pass
                    elif line.startswith('P_VALUE:') or line.startswith('P-VALUE:'):
                        try:
                            p_value = float(re.findall(r'[\d.]+', line)[0])
                        except:
                            pass
                    elif line.startswith('SIGNIFICANT:'):
                        significant = 'yes' in line.lower()
                    elif line.startswith('INTERPRETATION:'):
                        interpretation = line.replace('INTERPRETATION:', '').strip()
                
                if not interpretation:
                    interpretation = f"Statistical test performed with p-value of {p_value}"
                
                return StatisticalTest(
                    test_name=test_name,
                    statistic=statistic,
                    p_value=p_value,
                    significant=significant,
                    interpretation=interpretation,
                    confidence_level=0.95
                )
                
            except Exception as e:
                return StatisticalTest(
                    test_name=test_type,
                    statistic=0.0,
                    p_value=0.05,
                    significant=False,
                    interpretation=f"Statistical test performed: {test_type}",
                    confidence_level=0.95
                )
        
        def generate_insights(
            self,
            data_profile: DataProfile,
            analysis_results: List[AnalysisResult]
        ) -> List[str]:
            """
            Generate high-level insights from multiple analyses.
            
            Args:
                data_profile: Profile of the dataset
                analysis_results: Previous analysis results
                
            Returns:
                List of insights
            """
            # Compile analysis summary
            analyses_summary = "\n".join([
                f"- {result.query}: {', '.join(result.findings[:2])}"
                for result in analysis_results[-5:]  # Last 5 analyses
            ])
            
            prompt = ChatPromptTemplate.from_messages([
                ("system", """You are a senior data scientist. Based on the dataset profile
                and previous analyses, generate high-level strategic insights.
                
                Provide 3-5 key insights, one per line starting with -"""),
                ("user", """Data Profile: {profile}
    
    Previous Analyses:
    {analyses}
    
    Generate strategic insights about this data.""")
            ])
            
            chain = prompt | self.insight_llm | StrOutputParser()
            
            try:
                response = chain.invoke({
                    "profile": data_profile.summary,
                    "analyses": analyses_summary if analyses_summary else "No previous analyses"
                })
                
                insights = []
                for line in response.split('\n'):
                    if line.strip().startswith('-'):
                        insights.append(line.strip()[1:].strip())
                
                return insights if insights else [
                    "Data shows interesting patterns worthy of investigation",
                    "Further analysis recommended to validate findings",
                    "Consider collecting additional data for deeper insights"
                ]
                
            except Exception as e:
                return [
                    "Data analysis completed successfully",
                    "Multiple dimensions of the data have been explored",
                    "Additional investigation may reveal more patterns"
                ]
        
        def recommend_visualization(
            self,
            analysis_result: AnalysisResult,
            data_profile: DataProfile
        ) -> Visualization:
            """
            Recommend and generate a visualization for analysis results.
            
            Args:
                analysis_result: Result of the analysis
                data_profile: Profile of the dataset
                
            Returns:
                Visualization specification
            """
            viz_type = analysis_result.visualizations[0] if analysis_result.visualizations else VisualizationType.BAR_CHART
            
            prompt = ChatPromptTemplate.from_messages([
                ("system", """You are a data visualization expert. Create a visualization
                specification for the analysis results.
                
                Respond in this format:
                TITLE: Chart title
                X_AXIS: X-axis column
                Y_AXIS: Y-axis column
                COLUMNS: Columns to visualize (comma-separated)
                DESCRIPTION: Brief description
                CODE: Python code using matplotlib/seaborn"""),
                ("user", """Analysis Query: {query}
    Visualization Type: {viz_type}
    Findings: {findings}
    
    Available Columns: {columns}
    
    Create visualization specification.""")
            ])
            
            chain = prompt | self.viz_llm | StrOutputParser()
            
            try:
                response = chain.invoke({
                    "query": analysis_result.query,
                    "viz_type": viz_type.value,
                    "findings": ", ".join(analysis_result.findings[:3]),
                    "columns": ", ".join([col.name for col in data_profile.columns])
                })
                
                # Parse response
                title = f"{viz_type.value.replace('_', ' ').title()} Visualization"
                x_axis = None
                y_axis = None
                data_columns = []
                description = ""
                code = "# Visualization code would go here"
                
                in_code = False
                code_lines = []
                
                for line in response.split('\n'):
                    if line.startswith('TITLE:'):
                        title = line.replace('TITLE:', '').strip()
                    elif line.startswith('X_AXIS:'):
                        x_axis = line.replace('X_AXIS:', '').strip()
                    elif line.startswith('Y_AXIS:'):
                        y_axis = line.replace('Y_AXIS:', '').strip()
                    elif line.startswith('COLUMNS:'):
                        cols = line.replace('COLUMNS:', '').strip()
                        data_columns = [c.strip() for c in cols.split(',')]
                    elif line.startswith('DESCRIPTION:'):
                        description = line.replace('DESCRIPTION:', '').strip()
                    elif line.startswith('CODE:'):
                        in_code = True
                    elif in_code:
                        code_lines.append(line)
                
                if code_lines:
                    code = '\n'.join(code_lines)
                
                if not description:
                    description = f"Visualization showing {analysis_result.query}"
                
                return Visualization(
                    viz_type=viz_type,
                    title=title,
                    x_axis=x_axis,
                    y_axis=y_axis,
                    data_columns=data_columns if data_columns else [data_profile.columns[0].name],
                    description=description,
                    code=code
                )
                
            except Exception as e:
                return Visualization(
                    viz_type=viz_type,
                    title=f"{viz_type.value.replace('_', ' ').title()}",
                    x_axis=data_profile.columns[0].name if data_profile.columns else None,
                    y_axis=data_profile.columns[1].name if len(data_profile.columns) > 1 else None,
                    data_columns=[col.name for col in data_profile.columns[:2]],
                    description=f"Visualization for {analysis_result.query}",
                    code="# Visualization code"
                )
        
        def _determine_analysis_type(self, query: str) -> AnalysisType:
            """Determine the type of analysis from the query"""
            query_lower = query.lower()
            
            if any(word in query_lower for word in ['describe', 'summary', 'overview', 'statistics']):
                return AnalysisType.DESCRIPTIVE
            elif any(word in query_lower for word in ['explore', 'distribution', 'pattern']):
                return AnalysisType.EXPLORATORY
            elif any(word in query_lower for word in ['test', 'significance', 'hypothesis']):
                return AnalysisType.STATISTICAL
            elif any(word in query_lower for word in ['correlation', 'relationship', 'association']):
                return AnalysisType.CORRELATION
            elif any(word in query_lower for word in ['trend', 'over time', 'temporal']):
                return AnalysisType.TREND
            elif any(word in query_lower for word in ['compare', 'difference', 'versus']):
                return AnalysisType.COMPARISON
            elif any(word in query_lower for word in ['anomaly', 'outlier', 'unusual']):
                return AnalysisType.ANOMALY
            elif any(word in query_lower for word in ['predict', 'forecast', 'future']):
                return AnalysisType.PREDICTION
            else:
                return AnalysisType.EXPLORATORY
        
        def _recommend_visualizations(
            self,
            query: str,
            analysis_type: AnalysisType,
            data_profile: DataProfile
        ) -> List[VisualizationType]:
            """Recommend appropriate visualizations"""
            visualizations = []
            
            if analysis_type == AnalysisType.DESCRIPTIVE:
                visualizations = [VisualizationType.BAR_CHART, VisualizationType.BOX_PLOT]
            elif analysis_type == AnalysisType.EXPLORATORY:
                visualizations = [VisualizationType.HISTOGRAM, VisualizationType.SCATTER_PLOT]
            elif analysis_type == AnalysisType.CORRELATION:
                visualizations = [VisualizationType.HEATMAP, VisualizationType.SCATTER_PLOT]
            elif analysis_type == AnalysisType.TREND:
                visualizations = [VisualizationType.LINE_CHART, VisualizationType.TIME_SERIES]
            elif analysis_type == AnalysisType.COMPARISON:
                visualizations = [VisualizationType.BAR_CHART, VisualizationType.BOX_PLOT]
            elif analysis_type == AnalysisType.DISTRIBUTION:
                visualizations = [VisualizationType.HISTOGRAM, VisualizationType.VIOLIN_PLOT]
            else:
                visualizations = [VisualizationType.BAR_CHART]
            
            return visualizations
    
    
    def demonstrate_data_analysis_agent():
        """Demonstrate the data analysis agent capabilities"""
        print("=" * 80)
        print("DATA ANALYSIS AGENT DEMONSTRATION")
        print("=" * 80)
        
        agent = DataAnalysisAgent()
        
        # Demo 1: Data Profiling
        print("\n" + "=" * 80)
        print("DEMO 1: Data Profiling")
        print("=" * 80)
        
        sample_sales_data = {
            "product": ["A", "B", "C", "A", "B", "C", "A", "B"],
            "sales": [100, 150, 200, 120, 160, 180, 110, 155],
            "region": ["North", "South", "East", "West", "North", "South", "East", "West"],
            "date": ["2024-01", "2024-01", "2024-01", "2024-02", "2024-02", "2024-02", "2024-03", "2024-03"]
        }
        
        print("\nProfiling sales dataset...")
        profile = agent.profile_data(
            "Monthly sales data by product and region",
            sample_sales_data
        )
        
        print(f"\nData Profile:")
        print(f"- Rows: {profile.row_count}")
        print(f"- Columns: {profile.column_count}")
        print(f"- Missing values: {profile.missing_total}")
        print(f"\nColumns:")
        for col in profile.columns:
            print(f"  - {col.name} ({col.data_type.value}): {col.unique_count} unique values")
        print(f"\nSummary: {profile.summary}")
        
        # Demo 2: Query Analysis
        print("\n" + "=" * 80)
        print("DEMO 2: Query Analysis")
        print("=" * 80)
        
        queries = [
            "What are the average sales by product?",
            "Which region has the highest sales?",
            "Is there a correlation between product type and sales volume?"
        ]
        
        for query in queries:
            print(f"\nQuery: {query}")
            result = agent.analyze_query(query, profile)
            
            print(f"Analysis Type: {result.analysis_type.value}")
            print(f"Confidence: {result.confidence:.2f}")
            print(f"\nFindings:")
            for finding in result.findings[:3]:
                print(f"  - {finding}")
            
            if result.insights:
                print(f"\nInsights:")
                for insight in result.insights[:2]:
                    print(f"  - {insight}")
            
            if result.recommendations:
                print(f"\nRecommendations:")
                for rec in result.recommendations[:2]:
                    print(f"  - {rec}")
            
            print(f"\nRecommended Visualizations:")
            for viz in result.visualizations[:2]:
                print(f"  - {viz.value}")
            
            agent.analysis_history.append(result)
        
        # Demo 3: Statistical Testing
        print("\n" + "=" * 80)
        print("DEMO 3: Statistical Testing")
        print("=" * 80)
        
        test_data = {
            "group_a": [100, 105, 110, 95, 102],
            "group_b": [150, 145, 155, 140, 148]
        }
        
        print("\nPerforming t-test on sales groups...")
        test_result = agent.perform_statistical_test(
            "t-test",
            "Comparing sales between two product groups",
            test_data
        )
        
        print(f"\nTest: {test_result.test_name}")
        print(f"Statistic: {test_result.statistic:.4f}")
        print(f"P-value: {test_result.p_value:.4f}")
        print(f"Significant at {test_result.confidence_level:.0%} level: {test_result.significant}")
        print(f"Interpretation: {test_result.interpretation}")
        
        # Demo 4: Insight Generation
        print("\n" + "=" * 80)
        print("DEMO 4: High-Level Insights")
        print("=" * 80)
        
        print("\nGenerating strategic insights from all analyses...")
        insights = agent.generate_insights(profile, agent.analysis_history)
        
        print("\nStrategic Insights:")
        for i, insight in enumerate(insights, 1):
            print(f"{i}. {insight}")
        
        # Demo 5: Visualization Recommendation
        print("\n" + "=" * 80)
        print("DEMO 5: Visualization Recommendation")
        print("=" * 80)
        
        if agent.analysis_history:
            latest_analysis = agent.analysis_history[0]
            print(f"\nCreating visualization for: {latest_analysis.query}")
            
            viz = agent.recommend_visualization(latest_analysis, profile)
            
            print(f"\nVisualization Specification:")
            print(f"Type: {viz.viz_type.value}")
            print(f"Title: {viz.title}")
            print(f"X-axis: {viz.x_axis}")
            print(f"Y-axis: {viz.y_axis}")
            print(f"Columns: {', '.join(viz.data_columns)}")
            print(f"Description: {viz.description}")
            print(f"\nCode Preview:")
            code_lines = viz.code.split('\n')[:5]
            for line in code_lines:
                print(f"  {line}")
            if len(viz.code.split('\n')) > 5:
                print("  ...")
        
        # Summary
        print("\n" + "=" * 80)
        print("SUMMARY")
        print("=" * 80)
        
        summary = """
    The Data Analysis Agent demonstrates comprehensive capabilities for automated
    data analysis and exploration:
    
    KEY CAPABILITIES:
    1. Data Profiling: Automatically analyzes dataset structure, types, and quality
    2. Natural Language Queries: Understands and answers data questions in plain English
    3. Statistical Analysis: Performs appropriate statistical tests and interprets results
    4. Insight Generation: Identifies patterns, trends, and actionable insights
    5. Visualization Recommendations: Suggests and generates appropriate visualizations
    
    BENEFITS:
    - Makes data analysis accessible to non-technical users
    - Accelerates exploratory data analysis workflows
    - Provides consistent, comprehensive analysis
    - Generates actionable insights automatically
    - Recommends appropriate analytical methods
    
    USE CASES:
    - Business intelligence dashboards
    - Automated reporting systems
    - Data quality monitoring
    - Research data exploration
    - Customer analytics
    - Market research analysis
    - Scientific data analysis
    - Educational data tools
    
    PRODUCTION CONSIDERATIONS:
    1. Integration: Connect to real databases, data lakes, and APIs
    2. Visualization: Implement actual chart generation with matplotlib/plotly
    3. Statistical Libraries: Use scipy, statsmodels for robust statistical tests
    4. Performance: Optimize for large datasets with sampling and caching
    5. Security: Implement data access controls and query validation
    6. Scalability: Support distributed processing for big data
    7. Memory Management: Handle large datasets efficiently
    8. Validation: Verify statistical assumptions before tests
    9. Interpretability: Provide clear explanations for all findings
    10. Interactive Mode: Support iterative exploration with follow-up questions
    
    ADVANCED EXTENSIONS:
    - Machine learning model training and evaluation
    - Time series forecasting and anomaly detection
    - Causal inference and A/B test analysis
    - Automated feature engineering
    - Multi-dataset joins and comparisons
    - Real-time streaming data analysis
    - Custom metric and KPI tracking
    - Export to notebooks and reports
    - Natural language report generation
    - Interactive dashboard creation
    
    The agent combines LLM intelligence with data science capabilities to make
    sophisticated analysis accessible through natural conversation.
    """
        
        print(summary)
    
    
    if __name__ == "__main__":
        demonstrate_data_analysis_agent()
    
        

    Agentic Design Pattern: Data Analysis Agent

    Pattern 072: Data Analysis Agent – Line-by-Line Explanation

    Overview

    This document provides a detailed line-by-line explanation of the Data Analysis Agent pattern implementation.


    Pattern Description

    Pattern 072: Data Analysis Agent

    Description: A Data Analysis Agent is a specialized AI agent designed to assist with data exploration, statistical analysis, visualization, and insight generation. This pattern demonstrates how to build an intelligent agent that can understand data-related queries, perform appropriate analyses, generate visualizations, and provide actionable insights from data.

    The agent combines natural language understanding with data science capabilities to make
    complex data analysis accessible through conversational interfaces. It can handle various
    data formats, perform exploratory data analysis (EDA), statistical tests, trend analysis,
    and generate meaningful visualizations.
    

    Components: 1. Data Loading & Parsing: Handles various data formats (CSV, JSON, Excel, databases) 2. Data Cleaning: Identifies and handles missing values, outliers, duplicates 3. Exploratory Analysis: Computes statistics, distributions, correlations 4. Statistical Testing: Performs hypothesis tests, significance testing 5. Visualization Generation: Creates appropriate charts and graphs 6. Insight Extraction: Identifies patterns, trends, and anomalies 7. Natural Language Interface: Understands data-related questions 8. Query Translation: Converts NL queries to data operations

    Key Features: – Automatic data profiling and quality assessment – Intelligent analysis method selection based on data characteristics – Statistical test recommendations and execution – Visualization selection based on data types and query intent – Natural language explanation of findings – Iterative exploration with memory of previous analyses – Support for multiple data sources and formats – Anomaly and outlier detection – Trend and pattern identification – Correlation and causation analysis

    Use Cases: – Business intelligence and reporting – Data exploration and hypothesis testing – Quality assurance and data validation – Market research and customer analytics – Scientific data analysis – Financial analysis and forecasting – Healthcare analytics – Educational data analysis – Interactive data dashboards – Automated reporting systems

    LangChain Implementation: Uses ChatOpenAI for analysis planning, pandas for data manipulation, and specialized LLM chains for insight generation and visualization recommendations.


    File Statistics

    • Total Lines: 930
    • Classes: 9
    • Functions: 1
    • Constants: 0

    Module Documentation

    """
    Pattern 072: Data Analysis Agent
    
    Description:
        A Data Analysis Agent is a specialized AI agent designed to assist with data exploration,
        statistical analysis, visualization, and insight generation. This pattern demonstrates how
        to build an intelligent agent that can understand data-related queries, perform appropriate
        analyses, generate visualizations, and provide actionable insights from data.
    
        The agent combines natural language understanding with data science capabilities to make
        complex data analysis accessible through conversational interfaces. It can handle various
        data formats, perform exploratory data analysis (EDA), statistical tests, trend analysis,
        and generate meaningful visualizations.
    
    Components:
        1. Data Loading & Parsing: Handles various data formats (CSV, JSON, Excel, databases)
        2. Data Cleaning: Identifies and handles missing values, outliers, duplicates
        3. Exploratory Analysis: Computes statistics, distributions, correlations
        4. Statistical Testing: Performs hypothesis tests, significance testing
        5. Visualization Generation: Creates appropriate charts and graphs
        6. Insight Extraction: Identifies patterns, trends, and anomalies
        7. Natural Language Interface: Understands data-related questions
        8. Query Translation: Converts NL queries to data operations
    
    Key Features:
        - Automatic data profiling and quality assessment
        - Intelligent analysis method selection based on data characteristics
        - Statistical test recommendations and execution
        - Visualization selection based on data types and query intent
        - Natural language explanation of findings
        - Iterative exploration with memory of previous analyses
        - Support for multiple data sources and formats
        - Anomaly and outlier detection
        - Trend and pattern identification
        - Correlation and causation analysis
    
    Use Cases:
        - Business intelligence and reporting
        - Data exploration and hypothesis testing
        - Quality assurance and data validation
        - Market research and customer analytics
        - Scientific data analysis
        - Financial analysis and forecasting
        - Healthcare analytics
        - Educational data analysis
        - Interactive data dashboards
        - Automated reporting systems
    
    LangChain Implementation:
        Uses ChatOpenAI for analysis planning, pandas for data manipulation, and
        specialized LLM chains for insight generation and visualization recommendations.
    """
    

    Purpose: This module implements the Data Analysis Agent pattern, providing a structured approach to agent design and implementation.


    Import Statements

    Standard Library Imports

    import dataclasses.dataclass
    import dataclasses.field
    import enum.Enum
    import json
    import langchain_core.output_parsers.StrOutputParser
    import langchain_core.prompts.ChatPromptTemplate
    import os
    import re
    import typing.Any
    import typing.Dict
    import typing.List
    import typing.Optional
    import typing.Tuple
    

    Purpose: Core Python functionality: – dataclasses.dataclass: Simplified class definitions with @dataclass – dataclasses.field: Simplified class definitions with @dataclass – enum.Enum: Enumeration support – json: JSON serialization/deserialization – langchain_core.output_parsers.StrOutputParser: Regular expression operations – langchain_core.prompts.ChatPromptTemplate: Regular expression operations – os: Environment variable access and system operations – re: Regular expression operations – typing.Any: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Dict: Type hints for better code documentation (List, Dict, Any, Optional) – typing.List: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Optional: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Tuple: Type hints for better code documentation (List, Dict, Any, Optional)

    LangChain Imports

    from langchain_openai.ChatOpenAI
    

    Purpose: LangChain framework components: – LLM Interface: Wrapper for OpenAI models – Chains: Composable building blocks for complex workflows

    Other Imports

    from dotenv.load_dotenv
    

    Purpose: Additional dependencies for specialized functionality.


    Classes

    Class: AnalysisType

    Lines 69-79

    Inherits from: Enum

    Purpose:

    Types of data analysis operations
    

    Class: DataType

    Lines 82-88

    Inherits from: Enum

    Purpose:

    Types of data variables
    

    Class: VisualizationType

    Lines 91-101

    Inherits from: Enum

    Purpose:

    Types of visualizations
    

    Class: DataColumn

    Lines 105-112

    Purpose:

    Represents a data column with metadata
    

    Class: DataProfile

    Lines 116-124

    Purpose:

    Profile of a dataset
    

    Class: StatisticalTest

    Lines 128-135

    Purpose:

    Result of a statistical test
    

    Class: AnalysisResult

    Lines 139-148

    Purpose:

    Result of a data analysis
    

    Class: Visualization

    Lines 152-160

    Purpose:

    Specification for a visualization
    

    Class: DataAnalysisAgent

    Lines 163-729

    Purpose:

    Agent for comprehensive data analysis and exploration.
    
    This agent can understand natural language queries about data, perform
    appropriate analyses, generate insights, and recommend visualizations.
    

    Methods:

    __init__(self)

    Lines 171-187

    Purpose:

    Initialize the data analysis agent with specialized LLMs
    

    Parameters:

    profile_data(self, data_description, sample_data)

    Lines 189-307

    Purpose:

    Profile a dataset to understand its structure and characteristics.
    
    Args:
        data_description: Description of the dataset
        sample_data: Sample data as dictionary of columns
    
    Returns:
        DataProfile with comprehensive dataset information
    

    Parameters: – data_description: Data description – sample_data: Sample data

    Returns: DataProfile

    analyze_query(self, query, data_profile)

    Lines 309-425

    Purpose:

    Analyze a natural language query about data.
    
    Args:
        query: Natural language question about the data
        data_profile: Profile of the dataset
    
    Returns:
        AnalysisResult with findings and recommendations
    

    Parameters: – query: Input query or question – data_profile: Data profile

    Returns: AnalysisResult

    perform_statistical_test(self, test_type, data_description, sample_data)

    Lines 427-518

    Purpose:

    Perform a statistical test on the data.
    
    Args:
        test_type: Type of test (t-test, chi-square, ANOVA, etc.)
        data_description: Description of what's being tested
        sample_data: Data for the test
    
    Returns:
        StatisticalTest with results and interpretation
    

    Parameters: – test_type: Test type – data_description: Data description – sample_data: Sample data

    Returns: StatisticalTest

    generate_insights(self, data_profile, analysis_results)

    Lines 520-578

    Purpose:

    Generate high-level insights from multiple analyses.
    
    Args:
        data_profile: Profile of the dataset
        analysis_results: Previous analysis results
    
    Returns:
        List of insights
    

    Parameters: – data_profile: Data profile – analysis_results: Analysis results

    Returns: <ast.Subscript object at 0x000001A06D6F4550>

    recommend_visualization(self, analysis_result, data_profile)

    Lines 580-680

    Purpose:

    Recommend and generate a visualization for analysis results.
    
    Args:
        analysis_result: Result of the analysis
        data_profile: Profile of the dataset
    
    Returns:
        Visualization specification
    

    Parameters: – analysis_result: Analysis result – data_profile: Data profile

    Returns: Visualization

    _determine_analysis_type(self, query)

    Lines 682-703

    Purpose:

    Determine the type of analysis from the query
    

    Parameters: – query: Input query or question

    Returns: AnalysisType

    _recommend_visualizations(self, query, analysis_type, data_profile)

    Lines 705-729

    Purpose:

    Recommend appropriate visualizations
    

    Parameters: – query: Input query or question – analysis_type: Analysis type – data_profile: Data profile

    Returns: <ast.Subscript object at 0x000001A06D661C90>


    Functions

    demonstrate_data_analysis_agent()

    Lines 732-925

    Purpose:

    Demonstrate the data analysis agent capabilities
    

    Key Concepts Summary

    Pattern Overview

    The Data Analysis Agent pattern provides a structured approach to building AI agents with specific capabilities and behaviors. This implementation leverages LangChain’s composable components to create a robust and maintainable solution.

    Core Components

    1. Agent Class: Main implementation of the pattern logic
    2. Prompt Engineering: Carefully crafted prompts to guide LLM behavior
    3. Tool Integration: Optional tools for extended functionality
    4. Memory Management: State handling across interactions
    5. Output Parsing: Structured response processing

    When to Use

    ✅ Good for: – Scenarios requiring this specific pattern’s strengths – Applications needing structured agent behavior – Systems where explainability is important – Production environments requiring reliability

    ❌ Not ideal for: – Simple single-prompt tasks – Scenarios requiring different pattern characteristics – Resource-constrained environments (if computationally intensive)


    Best Practices Demonstrated

    1. Environment Configuration

    • ✅ Use .env files for API keys
    • ✅ Never hardcode credentials
    • ✅ Validate environment variables on startup

    2. Error Handling

    • ✅ Graceful degradation on failures
    • ✅ Informative error messages
    • ✅ Logging for debugging

    3. Prompt Engineering

    • ✅ Clear, specific instructions
    • ✅ Structured output formats
    • ✅ Few-shot examples when beneficial

    4. Code Organization

    • ✅ Separation of concerns (class-based design)
    • ✅ Type hints for clarity
    • ✅ Docstrings for documentation

    5. Performance Optimization

    • ✅ Appropriate temperature settings
    • ✅ Token budget management
    • ✅ Caching where applicable

    LangChain Components Used

    • ChatOpenAI: OpenAI model interface
    • Chains: Sequential component composition

    Why LangChain? – Composable abstractions for LLM applications – Rich ecosystem of integrations – Production-ready patterns – Active community and development


    Execution Flow

    1. Initialize agent with configuration
       ↓
    2. Load environment and dependencies
       ↓
    3. Prepare prompt with input data
       ↓
    4. Invoke LLM through LangChain
       ↓
    5. Parse and validate response
       ↓
    6. Return formatted result
    

    Key Decision Points: – Model selection (temperature, model type) – Prompt structure (few-shot vs zero-shot) – Error handling strategy – Output format validation


    Performance Considerations

    Computational Cost

    • LLM Calls: Primary cost factor (based on tokens)
    • Latency: Depends on model and input size
    • Memory: Minimal for most patterns

    Optimization Strategies

    1. Prompt Optimization: Minimize tokens while maintaining clarity
    2. Caching: Cache repeated queries/responses
    3. Model Selection: Use smaller models when appropriate
    4. Batch Processing: Group similar requests
    5. Streaming: For long responses, stream tokens

    Cost Estimates (Approximate)

    • GPT-3.5-turbo: ~$0.002 per request
    • GPT-4: ~$0.03 per request
    • GPT-4-turbo: ~$0.01 per request

    Actual costs vary based on input/output token counts


    Common Pitfalls to Avoid

    1. Prompt Design Issues

    ❌ Vague or ambiguous instructions
    ✅ Clear, specific, testable prompts

    2. Error Handling

    ❌ Ignoring API errors or rate limits
    ✅ Implement retry logic and graceful degradation

    3. Token Management

    ❌ Exceeding model token limits
    ✅ Truncate inputs, chunk processing

    4. Security

    ❌ Exposing API keys in code
    ✅ Use environment variables

    5. Testing

    ❌ Only testing happy paths
    ✅ Test edge cases, failures, malformed inputs

    6. Cost Management

    ❌ Unmonitored API usage
    ✅ Implement budgets, alerts, and monitoring


    Further Improvements

    Production Enhancements

    1. Monitoring: Add logging, metrics, tracing
    2. Validation: Input/output schema validation
    3. Rate Limiting: Implement request throttling
    4. Caching: Redis or in-memory cache for responses
    5. Async Support: Non-blocking operations for scalability

    Feature Extensions

    1. Multi-Model Support: Switch between providers
    2. Fallback Models: Graceful degradation strategy
    3. Custom Tools: Domain-specific function calling
    4. Advanced Memory: Vector stores for long-term memory
    5. Evaluation: Automated testing and benchmarking

    Code Quality

    1. Testing: Unit tests, integration tests
    2. Documentation: API docs, usage examples
    3. Type Safety: Stricter type hints
    4. Configuration: Externalized config files

    Related Patterns

    • Pattern 071
    • Pattern 073

    Pattern Families: – Reasoning Patterns (001-011) – Multi-Agent Patterns (012-019) – Tool Use Patterns (020-025) – Memory Patterns (026-031) – And many more…


    Summary

    The Data Analysis Agent pattern provides a powerful approach to building AI agents with specific capabilities. This implementation demonstrates:

    ✅ Key Strengths: – Clean, maintainable code structure – LangChain integration for composability – Type hints and documentation – Production-ready patterns

    🎯 Best Use Cases: – Applications requiring this pattern’s characteristics – Systems needing structured agent behavior – Production environments requiring reliability

    📚 Learning Resources: – LangChain Documentation: https://python.langchain.com/ – Pattern Documentation: See related patterns – OpenAI API Reference: https://platform.openai.com/docs/


    File: 072_data_analysis_agent.py
    Total Lines: 930
    Pattern Category: AI Agent Design Pattern #072


    Generated from actual implementation code
    For educational and reference purposes

  • Agentic Design Pattern: Code Agent

    
        """
    Pattern 071: Code Agent Patterns
    
    Description:
        Code Agent Patterns represent specialized agents designed for software development tasks
        including code analysis, generation, refactoring, debugging, testing, and documentation.
        These agents understand programming languages, design patterns, best practices, and can
        interact with codebases to assist developers. They combine static analysis, dynamic
        execution, code understanding, and generation capabilities to provide intelligent
        programming assistance.
        
        Code agents go beyond simple code completion to understand context, intent, architecture,
        and can reason about code quality, performance, security, and maintainability. They can
        explain existing code, suggest improvements, generate tests, refactor implementations,
        and even help with system design.
    
    Components:
        1. Code Parser: Analyzes syntax and structure
        2. Code Generator: Creates new code based on specifications
        3. Code Analyzer: Identifies issues, patterns, and opportunities
        4. Code Refactorer: Improves code structure and quality
        5. Test Generator: Creates unit and integration tests
        6. Documentation Generator: Produces code documentation
        7. Debugger Assistant: Helps identify and fix bugs
    
    Architecture:
        ```
        Code Task Request
            ↓
        Task Classifier → [Analyze/Generate/Refactor/Debug/Test/Document]
            ↓
        Code Understanding → [Parse, analyze structure & semantics]
            ↓
        Task Executor → [Execute specialized task]
            ↓
        Code Validator → [Check correctness & quality]
            ↓
        Result + Explanation
        ```
    
    Use Cases:
        - Automated code review and quality assessment
        - Intelligent code completion and generation
        - Refactoring suggestions and automation
        - Bug detection and debugging assistance
        - Test generation and coverage improvement
        - Documentation generation and maintenance
        - Code translation between languages
        - Architecture and design assistance
    
    Advantages:
        - Accelerates development workflow
        - Improves code quality and consistency
        - Reduces bugs and technical debt
        - Provides learning opportunities for developers
        - Automates repetitive coding tasks
        - Maintains coding standards automatically
    
    LangChain Implementation:
        Uses ChatOpenAI for code understanding and generation. Demonstrates
        code analysis, generation, refactoring, test creation, and documentation.
    """
    
    import os
    from typing import List, Dict, Any, Optional, Tuple
    from dataclasses import dataclass, field
    from enum import Enum
    import re
    from dotenv import load_dotenv
    
    from langchain_core.prompts import ChatPromptTemplate
    from langchain_core.output_parsers import StrOutputParser
    from langchain_openai import ChatOpenAI
    
    load_dotenv()
    
    
    class CodeTaskType(Enum):
        """Types of code-related tasks."""
        ANALYZE = "analyze"
        GENERATE = "generate"
        REFACTOR = "refactor"
        DEBUG = "debug"
        TEST = "test"
        DOCUMENT = "document"
        REVIEW = "review"
        EXPLAIN = "explain"
    
    
    class CodeLanguage(Enum):
        """Supported programming languages."""
        PYTHON = "python"
        JAVASCRIPT = "javascript"
        TYPESCRIPT = "typescript"
        JAVA = "java"
        CPP = "cpp"
        GO = "go"
        RUST = "rust"
    
    
    class CodeQualityIssue(Enum):
        """Types of code quality issues."""
        BUG = "bug"
        SECURITY = "security"
        PERFORMANCE = "performance"
        STYLE = "style"
        COMPLEXITY = "complexity"
        DUPLICATION = "duplication"
        MAINTAINABILITY = "maintainability"
    
    
    @dataclass
    class CodeSnippet:
        """Represents a piece of code."""
        code: str
        language: CodeLanguage
        file_path: Optional[str] = None
        line_start: Optional[int] = None
        line_end: Optional[int] = None
    
    
    @dataclass
    class CodeIssue:
        """Represents a code quality issue."""
        issue_type: CodeQualityIssue
        severity: str  # "critical", "high", "medium", "low"
        description: str
        line_number: Optional[int] = None
        suggestion: Optional[str] = None
    
    
    @dataclass
    class CodeAnalysis:
        """Results of code analysis."""
        snippet: CodeSnippet
        issues: List[CodeIssue]
        complexity_score: float
        maintainability_score: float
        summary: str
    
    
    @dataclass
    class GeneratedCode:
        """Generated code with metadata."""
        code: str
        language: CodeLanguage
        explanation: str
        test_code: Optional[str] = None
        dependencies: List[str] = field(default_factory=list)
    
    
    @dataclass
    class RefactoredCode:
        """Refactored code with changes."""
        original: str
        refactored: str
        changes: List[str]
        improvements: List[str]
    
    
    class CodeAgent:
        """
        Specialized agent for code-related tasks.
        """
        
        def __init__(self, default_language: CodeLanguage = CodeLanguage.PYTHON):
            """
            Initialize code agent.
            
            Args:
                default_language: Default programming language
            """
            self.default_language = default_language
            
            # Specialized LLMs for different tasks
            self.analyzer = ChatOpenAI(temperature=0.2, model="gpt-3.5-turbo")
            self.generator = ChatOpenAI(temperature=0.4, model="gpt-3.5-turbo")
            self.refactorer = ChatOpenAI(temperature=0.3, model="gpt-3.5-turbo")
            self.explainer = ChatOpenAI(temperature=0.3, model="gpt-3.5-turbo")
            
            self.parser = StrOutputParser()
        
        def analyze_code(self, snippet: CodeSnippet) -> CodeAnalysis:
            """
            Analyze code for quality issues.
            
            Args:
                snippet: Code to analyze
                
            Returns:
                Code analysis with issues and metrics
            """
            prompt = ChatPromptTemplate.from_messages([
                ("system", """You are a code quality expert. Analyze the code and identify issues.
    
    Consider:
    1. Bugs and potential errors
    2. Security vulnerabilities
    3. Performance issues
    4. Code style and best practices
    5. Complexity and maintainability
    6. Code duplication
    
    Provide:
    Issues: [type:severity:line:description:suggestion, ...]
    Complexity: [1-10 score]
    Maintainability: [1-10 score]
    Summary: [brief overall assessment]
    
    Format each issue as: type:severity:line:description:suggestion"""),
                ("user", """Language: {language}
    Code:
    ```
    {code}
    ```
    
    Analyze this code:""")
            ])
            
            chain = prompt | self.analyzer | self.parser
            
            try:
                result = chain.invoke({
                    "language": snippet.language.value,
                    "code": snippet.code
                })
                
                # Parse result
                issues = []
                complexity_score = 5.0
                maintainability_score = 5.0
                summary = ""
                
                for line in result.split('\n'):
                    line = line.strip()
                    if line.startswith("Issues:"):
                        issues_str = line.replace("Issues:", "").strip()
                        if issues_str.lower() not in ['none', '']:
                            # Parse issues
                            for issue_str in issues_str.split(','):
                                parts = issue_str.strip().split(':')
                                if len(parts) >= 5:
                                    try:
                                        issue_type = CodeQualityIssue(parts[0].strip().lower())
                                    except ValueError:
                                        issue_type = CodeQualityIssue.BUG
                                    
                                    issue = CodeIssue(
                                        issue_type=issue_type,
                                        severity=parts[1].strip(),
                                        description=parts[3].strip(),
                                        line_number=int(parts[2].strip()) if parts[2].strip().isdigit() else None,
                                        suggestion=parts[4].strip() if len(parts) > 4 else None
                                    )
                                    issues.append(issue)
                    
                    elif line.startswith("Complexity:"):
                        try:
                            complexity_score = float(re.search(r'\d+', line).group())
                        except:
                            pass
                    
                    elif line.startswith("Maintainability:"):
                        try:
                            maintainability_score = float(re.search(r'\d+', line).group())
                        except:
                            pass
                    
                    elif line.startswith("Summary:"):
                        summary = line.replace("Summary:", "").strip()
                
                if not summary:
                    summary = "Code analysis completed."
                
                return CodeAnalysis(
                    snippet=snippet,
                    issues=issues,
                    complexity_score=complexity_score,
                    maintainability_score=maintainability_score,
                    summary=summary
                )
                
            except Exception as e:
                print(f"Error analyzing code: {e}")
                return CodeAnalysis(
                    snippet=snippet,
                    issues=[],
                    complexity_score=5.0,
                    maintainability_score=5.0,
                    summary="Analysis failed."
                )
        
        def generate_code(
            self,
            description: str,
            language: Optional[CodeLanguage] = None,
            include_tests: bool = False
        ) -> GeneratedCode:
            """
            Generate code from description.
            
            Args:
                description: What the code should do
                language: Target programming language
                include_tests: Whether to generate tests
                
            Returns:
                Generated code with explanation
            """
            language = language or self.default_language
            
            prompt = ChatPromptTemplate.from_messages([
                ("system", """You are an expert programmer. Generate clean, efficient, well-documented code.
    
    Follow best practices:
    - Clear naming conventions
    - Proper error handling
    - Type hints (where applicable)
    - Docstrings/comments
    - Modular design
    
    Provide:
    Code: [complete implementation]
    Explanation: [how it works]
    Dependencies: [required libraries, comma-separated]
    {test_instruction}"""),
                ("user", """Language: {language}
    Task: {description}
    
    Generate the code:""")
            ])
            
            test_instruction = ""
            if include_tests:
                test_instruction = "Tests: [unit tests for the code]"
            
            chain = prompt | self.generator | self.parser
            
            try:
                result = chain.invoke({
                    "language": language.value,
                    "description": description,
                    "test_instruction": test_instruction
                })
                
                # Parse result
                code = ""
                explanation = ""
                test_code = None
                dependencies = []
                
                current_section = None
                section_lines = []
                
                for line in result.split('\n'):
                    if line.startswith("Code:"):
                        if current_section:
                            self._save_section(current_section, section_lines, 
                                             locals())
                        current_section = "code"
                        section_lines = []
                    elif line.startswith("Explanation:"):
                        if current_section:
                            self._save_section(current_section, section_lines,
                                             {"code": code})
                            code = '\n'.join(section_lines)
                        current_section = "explanation"
                        section_lines = [line.replace("Explanation:", "").strip()]
                    elif line.startswith("Dependencies:"):
                        if current_section == "explanation":
                            explanation = '\n'.join(section_lines)
                        deps_str = line.replace("Dependencies:", "").strip()
                        if deps_str.lower() not in ['none', '']:
                            dependencies = [d.strip() for d in deps_str.split(',')]
                        current_section = "dependencies"
                        section_lines = []
                    elif line.startswith("Tests:"):
                        if current_section == "code":
                            code = '\n'.join(section_lines)
                        current_section = "tests"
                        section_lines = []
                    else:
                        section_lines.append(line)
                
                # Save last section
                if current_section == "code":
                    code = '\n'.join(section_lines)
                elif current_section == "explanation":
                    explanation = '\n'.join(section_lines)
                elif current_section == "tests":
                    test_code = '\n'.join(section_lines)
                
                # Clean up code (remove markdown formatting)
                code = self._clean_code_block(code)
                if test_code:
                    test_code = self._clean_code_block(test_code)
                
                if not explanation:
                    explanation = "Generated code based on description."
                
                return GeneratedCode(
                    code=code,
                    language=language,
                    explanation=explanation,
                    test_code=test_code,
                    dependencies=dependencies
                )
                
            except Exception as e:
                print(f"Error generating code: {e}")
                return GeneratedCode(
                    code="# Error generating code",
                    language=language,
                    explanation="Code generation failed."
                )
        
        def _clean_code_block(self, code: str) -> str:
            """Remove markdown code block formatting."""
            # Remove ```language and ``` markers
            code = re.sub(r'^```\w*\n', '', code)
            code = re.sub(r'\n```$', '', code)
            code = code.strip()
            return code
        
        def _save_section(self, section: str, lines: List[str], context: Dict):
            """Helper to save parsed sections."""
            content = '\n'.join(lines)
            if section in context:
                context[section] = content
        
        def refactor_code(self, snippet: CodeSnippet, focus: str = "general") -> RefactoredCode:
            """
            Refactor code for better quality.
            
            Args:
                snippet: Code to refactor
                focus: Focus area (general, performance, readability, etc.)
                
            Returns:
                Refactored code with explanation
            """
            prompt = ChatPromptTemplate.from_messages([
                ("system", """You are a refactoring expert. Improve the code while preserving functionality.
    
    Focus on:
    - {focus}
    - Code clarity and readability
    - Best practices and patterns
    - Performance optimization
    - Maintainability
    
    Provide:
    Refactored: [improved code]
    Changes: [list of changes made, comma-separated]
    Improvements: [benefits of refactoring, comma-separated]"""),
                ("user", """Language: {language}
    Original Code:
    ```
    {code}
    ```
    
    Refactor this code:""")
            ])
            
            chain = prompt | self.refactorer | self.parser
            
            try:
                result = chain.invoke({
                    "language": snippet.language.value,
                    "code": snippet.code,
                    "focus": focus
                })
                
                # Parse result
                refactored = ""
                changes = []
                improvements = []
                
                for line in result.split('\n'):
                    if line.startswith("Refactored:"):
                        # Start capturing refactored code
                        refactored_lines = []
                        continue
                    elif line.startswith("Changes:"):
                        changes_str = line.replace("Changes:", "").strip()
                        if changes_str.lower() not in ['none', '']:
                            changes = [c.strip() for c in changes_str.split(',')]
                    elif line.startswith("Improvements:"):
                        imp_str = line.replace("Improvements:", "").strip()
                        if imp_str.lower() not in ['none', '']:
                            improvements = [i.strip() for i in imp_str.split(',')]
                    elif refactored == "" and not line.startswith("Changes") and not line.startswith("Improvements"):
                        refactored += line + '\n'
                
                refactored = self._clean_code_block(refactored)
                
                if not changes:
                    changes = ["Code refactored"]
                if not improvements:
                    improvements = ["Improved code quality"]
                
                return RefactoredCode(
                    original=snippet.code,
                    refactored=refactored if refactored else snippet.code,
                    changes=changes,
                    improvements=improvements
                )
                
            except Exception as e:
                print(f"Error refactoring code: {e}")
                return RefactoredCode(
                    original=snippet.code,
                    refactored=snippet.code,
                    changes=["Refactoring failed"],
                    improvements=[]
                )
        
        def explain_code(self, snippet: CodeSnippet, detail_level: str = "medium") -> str:
            """
            Explain what code does.
            
            Args:
                snippet: Code to explain
                detail_level: Level of detail (brief, medium, detailed)
                
            Returns:
                Explanation of the code
            """
            prompt = ChatPromptTemplate.from_messages([
                ("system", """You are a code educator. Explain code clearly and comprehensively.
    
    Detail Level: {detail_level}
    
    For brief: High-level summary in 2-3 sentences
    For medium: Explain main components and flow
    For detailed: Line-by-line or block-by-block explanation
    
    Provide clear, educational explanation."""),
                ("user", """Language: {language}
    Code:
    ```
    {code}
    ```
    
    Explain this code:""")
            ])
            
            chain = prompt | self.explainer | self.parser
            
            try:
                result = chain.invoke({
                    "language": snippet.language.value,
                    "code": snippet.code,
                    "detail_level": detail_level
                })
                
                return result.strip()
                
            except Exception as e:
                print(f"Error explaining code: {e}")
                return "Unable to generate explanation."
        
        def generate_tests(self, snippet: CodeSnippet) -> str:
            """
            Generate unit tests for code.
            
            Args:
                snippet: Code to test
                
            Returns:
                Test code
            """
            prompt = ChatPromptTemplate.from_messages([
                ("system", """You are a testing expert. Generate comprehensive unit tests.
    
    Include:
    - Test for normal cases
    - Edge cases
    - Error handling
    - Different input types
    
    Use appropriate testing framework for the language."""),
                ("user", """Language: {language}
    Code to Test:
    ```
    {code}
    ```
    
    Generate unit tests:""")
            ])
            
            chain = prompt | self.generator | self.parser
            
            try:
                result = chain.invoke({
                    "language": snippet.language.value,
                    "code": snippet.code
                })
                
                return self._clean_code_block(result)
                
            except Exception as e:
                print(f"Error generating tests: {e}")
                return "# Error generating tests"
    
    
    def demonstrate_code_agent():
        """Demonstrate Code Agent pattern."""
        
        print("="*80)
        print("CODE AGENT PATTERNS - DEMONSTRATION")
        print("="*80)
        
        agent = CodeAgent(default_language=CodeLanguage.PYTHON)
        
        # Test 1: Code Analysis
        print("\n" + "="*80)
        print("TEST 1: Code Analysis")
        print("="*80)
        
        sample_code = """
    def calculate_average(numbers):
        sum = 0
        for num in numbers:
            sum = sum + num
        return sum / len(numbers)
    """
        
        snippet1 = CodeSnippet(code=sample_code, language=CodeLanguage.PYTHON)
        analysis = agent.analyze_code(snippet1)
        
        print(f"\nComplexity Score: {analysis.complexity_score}/10")
        print(f"Maintainability Score: {analysis.maintainability_score}/10")
        print(f"\nSummary: {analysis.summary}")
        
        if analysis.issues:
            print(f"\nIssues Found: {len(analysis.issues)}")
            for issue in analysis.issues[:3]:
                print(f"  • {issue.issue_type.value.upper()} ({issue.severity}): {issue.description}")
                if issue.suggestion:
                    print(f"    Suggestion: {issue.suggestion}")
        
        # Test 2: Code Generation
        print("\n" + "="*80)
        print("TEST 2: Code Generation")
        print("="*80)
        
        generated = agent.generate_code(
            description="Create a function to validate email addresses using regex",
            language=CodeLanguage.PYTHON,
            include_tests=True
        )
        
        print("\n--- Generated Code ---")
        print(generated.code[:300] + "..." if len(generated.code) > 300 else generated.code)
        
        print(f"\n--- Explanation ---")
        print(generated.explanation[:200] + "..." if len(generated.explanation) > 200 else generated.explanation)
        
        if generated.dependencies:
            print(f"\n--- Dependencies ---")
            print(", ".join(generated.dependencies))
        
        # Test 3: Code Refactoring
        print("\n" + "="*80)
        print("TEST 3: Code Refactoring")
        print("="*80)
        
        messy_code = """
    def process(data):
        result = []
        for i in range(len(data)):
            if data[i] > 0:
                result.append(data[i] * 2)
        return result
    """
        
        snippet2 = CodeSnippet(code=messy_code, language=CodeLanguage.PYTHON)
        refactored = agent.refactor_code(snippet2, focus="readability")
        
        print("\n--- Original Code ---")
        print(refactored.original)
        
        print("\n--- Refactored Code ---")
        print(refactored.refactored)
        
        print("\n--- Changes Made ---")
        for change in refactored.changes:
            print(f"  • {change}")
        
        print("\n--- Improvements ---")
        for improvement in refactored.improvements:
            print(f"  • {improvement}")
        
        # Test 4: Code Explanation
        print("\n" + "="*80)
        print("TEST 4: Code Explanation")
        print("="*80)
        
        complex_code = """
    def fibonacci(n, memo={}):
        if n in memo:
            return memo[n]
        if n  400 else tests)
        
        # Summary
        print("\n" + "="*80)
        print("SUMMARY: Code Agent Patterns")
        print("="*80)
        print("""
    The Code Agent pattern demonstrates specialized AI assistance for software development:
    
    Key Features Demonstrated:
    1. Code Analysis: Identifies bugs, security issues, performance problems
    2. Code Generation: Creates functions from natural language descriptions
    3. Code Refactoring: Improves code structure and readability
    4. Code Explanation: Provides clear educational explanations
    5. Test Generation: Creates comprehensive unit tests
    6. Quality Metrics: Assesses complexity and maintainability
    7. Multi-language Support: Works across programming languages
    
    Benefits:
    • Accelerates development workflow significantly
    • Improves code quality and consistency
    • Reduces bugs and technical debt
    • Provides learning opportunities for developers
    • Automates repetitive coding tasks
    • Maintains coding standards automatically
    • Assists with code reviews and documentation
    
    Use Cases:
    • Automated code review in CI/CD pipelines
    • Intelligent IDE code completion and suggestions
    • Refactoring legacy code for maintainability
    • Bug detection and debugging assistance
    • Test generation for improving coverage
    • Documentation generation and maintenance
    • Code translation between languages
    • Learning and education for new developers
    
    Production Considerations:
    • Implement actual static analysis tools integration
    • Add support for project-wide context and dependencies
    • Implement incremental analysis for large codebases
    • Add caching for repeated analysis tasks
    • Integrate with version control for change analysis
    • Implement security scanning and vulnerability detection
    • Add performance profiling and optimization suggestions
    • Support for framework-specific best practices
    • Integration with existing dev tools and IDEs
    • Human review for critical code changes
    • Metrics tracking for code quality improvements
    • Team-specific style guide enforcement
    
    Advanced Extensions:
    • Multi-file refactoring with dependency tracking
    • Architectural pattern detection and suggestions
    • Automated bug fixing with verification
    • Code clone detection and deduplication
    • Performance optimization through profiling
    • Security vulnerability patching
    • API usage pattern analysis
    • Code smell detection and remediation
        """)
    
    
    if __name__ == "__main__":
        demonstrate_code_agent()
    
        

    Agentic Design Pattern: Code Agent

    Pattern 071: Code Agent – Line-by-Line Explanation

    Overview

    This document provides a detailed line-by-line explanation of the Code Agent pattern implementation.


    Pattern Description

    Pattern 071: Code Agent Patterns

    Description: Code Agent Patterns represent specialized agents designed for software development tasks including code analysis, generation, refactoring, debugging, testing, and documentation. These agents understand programming languages, design patterns, best practices, and can interact with codebases to assist developers. They combine static analysis, dynamic execution, code understanding, and generation capabilities to provide intelligent programming assistance.

    Code agents go beyond simple code completion to understand context, intent, architecture,
    and can reason about code quality, performance, security, and maintainability. They can
    explain existing code, suggest improvements, generate tests, refactor implementations,
    and even help with system design.
    

    Components: 1. Code Parser: Analyzes syntax and structure 2. Code Generator: Creates new code based on specifications 3. Code Analyzer: Identifies issues, patterns, and opportunities 4. Code Refactorer: Improves code structure and quality 5. Test Generator: Creates unit and integration tests 6. Documentation Generator: Produces code documentation 7. Debugger Assistant: Helps identify and fix bugs

    Architecture: Code Task Request ↓ Task Classifier → [Analyze/Generate/Refactor/Debug/Test/Document] ↓ Code Understanding → [Parse, analyze structure & semantics] ↓ Task Executor → [Execute specialized task] ↓ Code Validator → [Check correctness & quality] ↓ Result + Explanation

    Use Cases: – Automated code review and quality assessment – Intelligent code completion and generation – Refactoring suggestions and automation – Bug detection and debugging assistance – Test generation and coverage improvement – Documentation generation and maintenance – Code translation between languages – Architecture and design assistance

    Advantages: – Accelerates development workflow – Improves code quality and consistency – Reduces bugs and technical debt – Provides learning opportunities for developers – Automates repetitive coding tasks – Maintains coding standards automatically

    LangChain Implementation: Uses ChatOpenAI for code understanding and generation. Demonstrates code analysis, generation, refactoring, test creation, and documentation.


    File Statistics

    • Total Lines: 802
    • Classes: 9
    • Functions: 1
    • Constants: 0

    Module Documentation

    """
    Pattern 071: Code Agent Patterns
    
    Description:
        Code Agent Patterns represent specialized agents designed for software development tasks
        including code analysis, generation, refactoring, debugging, testing, and documentation.
        These agents understand programming languages, design patterns, best practices, and can
        interact with codebases to assist developers. They combine static analysis, dynamic
        execution, code understanding, and generation capabilities to provide intelligent
        programming assistance.
    
        Code agents go beyond simple code completion to understand context, intent, architecture,
        and can reason about code quality, performance, security, and maintainability. They can
        explain existing code, suggest improvements, generate tests, refactor implementations,
        and even help with system design.
    
    Components:
        1. Code Parser: Analyzes syntax and structure
        2. Code Generator: Creates new code based on specifications
        3. Code Analyzer: Identifies issues, patterns, and opportunities
        4. Code Refactorer: Improves code structure and quality
        5. Test Generator: Creates unit and integration tests
        6. Documentation Generator: Produces code documentation
        7. Debugger Assistant: Helps identify and fix bugs
    
    Architecture:
        ```
        Code Task Request
            ↓
        Task Classifier → [Analyze/Generate/Refactor/Debug/Test/Document]
            ↓
        Code Understanding → [Parse, analyze structure & semantics]
            ↓
        Task Executor → [Execute specialized task]
            ↓
        Code Validator → [Check correctness & quality]
            ↓
        Result + Explanation
        ```
    
    Use Cases:
        - Automated code review and quality assessment
        - Intelligent code completion and generation
        - Refactoring suggestions and automation
        - Bug detection and debugging assistance
        - Test generation and coverage improvement
        - Documentation generation and maintenance
        - Code translation between languages
        - Architecture and design assistance
    
    Advantages:
        - Accelerates development workflow
        - Improves code quality and consistency
        - Reduces bugs and technical debt
        - Provides learning opportunities for developers
        - Automates repetitive coding tasks
        - Maintains coding standards automatically
    
    LangChain Implementation:
        Uses ChatOpenAI for code understanding and generation. Demonstrates
        code analysis, generation, refactoring, test creation, and documentation.
    """
    

    Purpose: This module implements the Code Agent pattern, providing a structured approach to agent design and implementation.


    Import Statements

    Standard Library Imports

    import dataclasses.dataclass
    import dataclasses.field
    import enum.Enum
    import langchain_core.output_parsers.StrOutputParser
    import langchain_core.prompts.ChatPromptTemplate
    import os
    import re
    import typing.Any
    import typing.Dict
    import typing.List
    import typing.Optional
    import typing.Tuple
    

    Purpose: Core Python functionality: – dataclasses.dataclass: Simplified class definitions with @dataclass – dataclasses.field: Simplified class definitions with @dataclass – enum.Enum: Enumeration support – langchain_core.output_parsers.StrOutputParser: Regular expression operations – langchain_core.prompts.ChatPromptTemplate: Regular expression operations – os: Environment variable access and system operations – re: Regular expression operations – typing.Any: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Dict: Type hints for better code documentation (List, Dict, Any, Optional) – typing.List: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Optional: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Tuple: Type hints for better code documentation (List, Dict, Any, Optional)

    LangChain Imports

    from langchain_openai.ChatOpenAI
    

    Purpose: LangChain framework components: – LLM Interface: Wrapper for OpenAI models – Chains: Composable building blocks for complex workflows

    Other Imports

    from dotenv.load_dotenv
    

    Purpose: Additional dependencies for specialized functionality.


    Classes

    Class: CodeTaskType

    Lines 78-87

    Inherits from: Enum

    Purpose:

    Types of code-related tasks.
    

    Class: CodeLanguage

    Lines 90-98

    Inherits from: Enum

    Purpose:

    Supported programming languages.
    

    Class: CodeQualityIssue

    Lines 101-109

    Inherits from: Enum

    Purpose:

    Types of code quality issues.
    

    Class: CodeSnippet

    Lines 113-119

    Purpose:

    Represents a piece of code.
    

    Class: CodeIssue

    Lines 123-129

    Purpose:

    Represents a code quality issue.
    

    Class: CodeAnalysis

    Lines 133-139

    Purpose:

    Results of code analysis.
    

    Class: GeneratedCode

    Lines 143-149

    Purpose:

    Generated code with metadata.
    

    Class: RefactoredCode

    Lines 153-158

    Purpose:

    Refactored code with changes.
    

    Class: CodeAgent

    Lines 161-604

    Purpose:

    Specialized agent for code-related tasks.
    

    Methods:

    __init__(self, default_language)

    Lines 166-181

    Purpose:

    Initialize code agent.
    
    Args:
        default_language: Default programming language
    

    Parameters: – default_language: Default language

    analyze_code(self, snippet)

    Lines 183-291

    Purpose:

    Analyze code for quality issues.
    
    Args:
        snippet: Code to analyze
    
    Returns:
        Code analysis with issues and metrics
    

    Parameters: – snippet: Snippet

    Returns: CodeAnalysis

    generate_code(self, description, language, include_tests)

    Lines 293-415

    Purpose:

    Generate code from description.
    
    Args:
        description: What the code should do
        language: Target programming language
        include_tests: Whether to generate tests
    
    Returns:
        Generated code with explanation
    

    Parameters: – description: Description – language: Language – include_tests: Include tests

    Returns: GeneratedCode

    _clean_code_block(self, code)

    Lines 417-423

    Purpose:

    Remove markdown code block formatting.
    

    Parameters: – code: Code

    Returns: str

    _save_section(self, section, lines, context)

    Lines 425-429

    Purpose:

    Helper to save parsed sections.
    

    Parameters: – section: Section – lines: Lines – context: Context

    refactor_code(self, snippet, focus)

    Lines 431-516

    Purpose:

    Refactor code for better quality.
    
    Args:
        snippet: Code to refactor
        focus: Focus area (general, performance, readability, etc.)
    
    Returns:
        Refactored code with explanation
    

    Parameters: – snippet: Snippet – focus: Focus

    Returns: RefactoredCode

    explain_code(self, snippet, detail_level)

    Lines 518-561

    Purpose:

    Explain what code does.
    
    Args:
        snippet: Code to explain
        detail_level: Level of detail (brief, medium, detailed)
    
    Returns:
        Explanation of the code
    

    Parameters: – snippet: Snippet – detail_level: Detail level

    Returns: str

    generate_tests(self, snippet)

    Lines 563-604

    Purpose:

    Generate unit tests for code.
    
    Args:
        snippet: Code to test
    
    Returns:
        Test code
    

    Parameters: – snippet: Snippet

    Returns: str


    Functions

    demonstrate_code_agent()

    Lines 607-797

    Purpose:

    Demonstrate Code Agent pattern.
    

    Key Concepts Summary

    Pattern Overview

    The Code Agent pattern provides a structured approach to building AI agents with specific capabilities and behaviors. This implementation leverages LangChain’s composable components to create a robust and maintainable solution.

    Core Components

    1. Agent Class: Main implementation of the pattern logic
    2. Prompt Engineering: Carefully crafted prompts to guide LLM behavior
    3. Tool Integration: Optional tools for extended functionality
    4. Memory Management: State handling across interactions
    5. Output Parsing: Structured response processing

    When to Use

    ✅ Good for: – Scenarios requiring this specific pattern’s strengths – Applications needing structured agent behavior – Systems where explainability is important – Production environments requiring reliability

    ❌ Not ideal for: – Simple single-prompt tasks – Scenarios requiring different pattern characteristics – Resource-constrained environments (if computationally intensive)


    Best Practices Demonstrated

    1. Environment Configuration

    • ✅ Use .env files for API keys
    • ✅ Never hardcode credentials
    • ✅ Validate environment variables on startup

    2. Error Handling

    • ✅ Graceful degradation on failures
    • ✅ Informative error messages
    • ✅ Logging for debugging

    3. Prompt Engineering

    • ✅ Clear, specific instructions
    • ✅ Structured output formats
    • ✅ Few-shot examples when beneficial

    4. Code Organization

    • ✅ Separation of concerns (class-based design)
    • ✅ Type hints for clarity
    • ✅ Docstrings for documentation

    5. Performance Optimization

    • ✅ Appropriate temperature settings
    • ✅ Token budget management
    • ✅ Caching where applicable

    LangChain Components Used

    • ChatOpenAI: OpenAI model interface
    • Chains: Sequential component composition

    Why LangChain? – Composable abstractions for LLM applications – Rich ecosystem of integrations – Production-ready patterns – Active community and development


    Execution Flow

    1. Initialize agent with configuration
       ↓
    2. Load environment and dependencies
       ↓
    3. Prepare prompt with input data
       ↓
    4. Invoke LLM through LangChain
       ↓
    5. Parse and validate response
       ↓
    6. Return formatted result
    

    Key Decision Points: – Model selection (temperature, model type) – Prompt structure (few-shot vs zero-shot) – Error handling strategy – Output format validation


    Performance Considerations

    Computational Cost

    • LLM Calls: Primary cost factor (based on tokens)
    • Latency: Depends on model and input size
    • Memory: Minimal for most patterns

    Optimization Strategies

    1. Prompt Optimization: Minimize tokens while maintaining clarity
    2. Caching: Cache repeated queries/responses
    3. Model Selection: Use smaller models when appropriate
    4. Batch Processing: Group similar requests
    5. Streaming: For long responses, stream tokens

    Cost Estimates (Approximate)

    • GPT-3.5-turbo: ~$0.002 per request
    • GPT-4: ~$0.03 per request
    • GPT-4-turbo: ~$0.01 per request

    Actual costs vary based on input/output token counts


    Common Pitfalls to Avoid

    1. Prompt Design Issues

    ❌ Vague or ambiguous instructions
    ✅ Clear, specific, testable prompts

    2. Error Handling

    ❌ Ignoring API errors or rate limits
    ✅ Implement retry logic and graceful degradation

    3. Token Management

    ❌ Exceeding model token limits
    ✅ Truncate inputs, chunk processing

    4. Security

    ❌ Exposing API keys in code
    ✅ Use environment variables

    5. Testing

    ❌ Only testing happy paths
    ✅ Test edge cases, failures, malformed inputs

    6. Cost Management

    ❌ Unmonitored API usage
    ✅ Implement budgets, alerts, and monitoring


    Further Improvements

    Production Enhancements

    1. Monitoring: Add logging, metrics, tracing
    2. Validation: Input/output schema validation
    3. Rate Limiting: Implement request throttling
    4. Caching: Redis or in-memory cache for responses
    5. Async Support: Non-blocking operations for scalability

    Feature Extensions

    1. Multi-Model Support: Switch between providers
    2. Fallback Models: Graceful degradation strategy
    3. Custom Tools: Domain-specific function calling
    4. Advanced Memory: Vector stores for long-term memory
    5. Evaluation: Automated testing and benchmarking

    Code Quality

    1. Testing: Unit tests, integration tests
    2. Documentation: API docs, usage examples
    3. Type Safety: Stricter type hints
    4. Configuration: Externalized config files

    Related Patterns

    • Pattern 070
    • Pattern 072

    Pattern Families: – Reasoning Patterns (001-011) – Multi-Agent Patterns (012-019) – Tool Use Patterns (020-025) – Memory Patterns (026-031) – And many more…


    Summary

    The Code Agent pattern provides a powerful approach to building AI agents with specific capabilities. This implementation demonstrates:

    ✅ Key Strengths: – Clean, maintainable code structure – LangChain integration for composability – Type hints and documentation – Production-ready patterns

    🎯 Best Use Cases: – Applications requiring this pattern’s characteristics – Systems needing structured agent behavior – Production environments requiring reliability

    📚 Learning Resources: – LangChain Documentation: https://python.langchain.com/ – Pattern Documentation: See related patterns – OpenAI API Reference: https://platform.openai.com/docs/


    File: 071_code_agent.py
    Total Lines: 802
    Pattern Category: AI Agent Design Pattern #071


    Generated from actual implementation code
    For educational and reference purposes

  • Agentic Design Pattern: Prompt Optimization

    
        """
    Pattern 070: Prompt Optimization/Engineering
    
    Description:
        Prompt Optimization/Engineering automates the process of finding optimal prompts for
        specific tasks. Rather than manually crafting and testing prompts, this pattern uses
        automated search, evaluation, and refinement to discover high-performing prompt
        formulations. It explores the prompt space systematically, tests variations, measures
        effectiveness, and iteratively improves prompts to maximize task performance.
        
        This pattern is essential for building production LLM systems where prompt quality
        directly impacts output quality, cost, and latency. It can discover non-obvious
        prompt structures, optimal instruction phrasing, effective examples for few-shot
        learning, and task-specific formatting that humans might not consider.
    
    Components:
        1. Prompt Generator: Creates candidate prompt variations
        2. Evaluation Engine: Measures prompt effectiveness on test cases
        3. Optimization Strategy: Guides search through prompt space
        4. Prompt Analyzer: Extracts features from successful prompts
        5. Meta-Prompt Designer: Creates templates for prompt generation
        6. Performance Tracker: Monitors metrics across iterations
        7. Prompt Library: Stores and manages successful prompts
    
    Architecture:
        ```
        Initial Prompt
            ↓
        ┌─────────────────────────────┐
        │   Optimization Loop         │
        │                             │
        │  Generate Variations        │
        │         ↓                   │
        │  Evaluate on Test Cases     │
        │         ↓                   │
        │  Rank by Performance        │
        │         ↓                   │
        │  Select Best Candidates     │
        │         ↓                   │
        │  Analyze Success Patterns   │
        │         ↓                   │
        │  Generate New Variations    │
        └─────────────────────────────┘
             ↓ (iterate)
        Optimized Prompt
        ```
    
    Use Cases:
        - Automatic prompt engineering for production systems
        - Few-shot example selection and ordering
        - Task-specific instruction optimization
        - Output format specification tuning
        - Zero-shot prompt improvement
        - Multi-task prompt optimization
    
    Advantages:
        - Finds better prompts than manual engineering
        - Saves significant development time
        - Discovers non-obvious prompt patterns
        - Adapts to model updates automatically
        - Provides data-driven prompt decisions
        - Scales across multiple tasks
    
    LangChain Implementation:
        Uses ChatOpenAI for prompt generation and testing. Demonstrates
        prompt variation generation, evaluation on test cases, iterative
        optimization, and performance tracking.
    """
    
    import os
    from typing import List, Dict, Any, Optional, Tuple, Callable
    from dataclasses import dataclass, field
    from enum import Enum
    import time
    import random
    from collections import defaultdict
    from dotenv import load_dotenv
    
    from langchain_core.prompts import ChatPromptTemplate
    from langchain_core.output_parsers import StrOutputParser
    from langchain_openai import ChatOpenAI
    
    load_dotenv()
    
    
    class OptimizationStrategy(Enum):
        """Optimization strategies for prompt search."""
        RANDOM_SEARCH = "random_search"
        HILL_CLIMBING = "hill_climbing"
        BEAM_SEARCH = "beam_search"
        GENETIC = "genetic"
        GRADIENT_FREE = "gradient_free"
    
    
    class PromptComponent(Enum):
        """Components of a prompt that can be optimized."""
        INSTRUCTION = "instruction"
        CONTEXT = "context"
        EXAMPLES = "examples"
        OUTPUT_FORMAT = "output_format"
        CONSTRAINTS = "constraints"
        ROLE = "role"
    
    
    @dataclass
    class TestCase:
        """Test case for prompt evaluation."""
        case_id: str
        input: str
        expected_output: str
        weight: float = 1.0  # Importance weight
    
    
    @dataclass
    class PromptCandidate:
        """A candidate prompt with metadata."""
        prompt_id: str
        prompt_text: str
        components: Dict[PromptComponent, str]
        score: float = 0.0
        evaluations: int = 0
        generation: int = 0
        parent_id: Optional[str] = None
    
    
    @dataclass
    class EvaluationResult:
        """Result of evaluating a prompt on test cases."""
        prompt_id: str
        test_case_scores: Dict[str, float]
        average_score: float
        latency: float
        tokens_used: int
    
    
    @dataclass
    class OptimizationRun:
        """Complete optimization run results."""
        task_description: str
        initial_prompt: PromptCandidate
        best_prompt: PromptCandidate
        all_candidates: List[PromptCandidate]
        optimization_history: List[EvaluationResult]
        total_iterations: int
        improvement: float
    
    
    class PromptOptimizer:
        """
        Automated prompt optimization system.
        """
        
        def __init__(
            self,
            strategy: OptimizationStrategy = OptimizationStrategy.BEAM_SEARCH,
            beam_width: int = 3,
            max_iterations: int = 10
        ):
            """
            Initialize prompt optimizer.
            
            Args:
                strategy: Optimization strategy to use
                beam_width: Number of candidates to keep (for beam search)
                max_iterations: Maximum optimization iterations
            """
            self.strategy = strategy
            self.beam_width = beam_width
            self.max_iterations = max_iterations
            
            # LLMs for different tasks
            self.prompt_generator = ChatOpenAI(temperature=0.8, model="gpt-3.5-turbo")
            self.evaluator_llm = ChatOpenAI(temperature=0.3, model="gpt-3.5-turbo")
            self.analyzer = ChatOpenAI(temperature=0.5, model="gpt-3.5-turbo")
            
            self.parser = StrOutputParser()
            
            # Tracking
            self.prompt_library: Dict[str, PromptCandidate] = {}
            self.evaluation_cache: Dict[Tuple[str, str], float] = {}
            self.generation_count = 0
        
        def generate_prompt_variations(
            self,
            base_prompt: PromptCandidate,
            num_variations: int = 5,
            task_description: str = ""
        ) -> List[PromptCandidate]:
            """
            Generate variations of a prompt.
            
            Args:
                base_prompt: Starting prompt
                num_variations: Number of variations to generate
                task_description: Description of the task
                
            Returns:
                List of prompt variations
            """
            prompt = ChatPromptTemplate.from_messages([
                ("system", """You are a prompt engineer. Generate improved variations of prompts.
    
    Task: {task}
    
    Consider these optimization strategies:
    1. Clarify instructions and make them more specific
    2. Add helpful context or constraints
    3. Improve output format specifications
    4. Refine role/persona if applicable
    5. Reorder or restructure components
    6. Add examples or improve existing ones
    
    Generate {num_variations} distinct variations that might perform better.
    Each variation should have a clear improvement hypothesis.
    
    Format each as:
    VARIATION {n}:
    {complete prompt text}
    ---"""),
                ("user", """Base Prompt:
    {base_prompt}
    
    Generate {num_variations} improved variations:""")
            ])
            
            chain = prompt | self.prompt_generator | self.parser
            
            try:
                result = chain.invoke({
                    "task": task_description,
                    "base_prompt": base_prompt.prompt_text,
                    "num_variations": num_variations
                })
                
                # Parse variations
                variations = []
                variation_blocks = result.split('---')
                
                for block in variation_blocks:
                    if 'VARIATION' not in block:
                        continue
                    
                    # Extract prompt text (remove variation header)
                    lines = block.split('\n')
                    prompt_lines = [line for line in lines if not line.strip().startswith('VARIATION')]
                    prompt_text = '\n'.join(prompt_lines).strip()
                    
                    if prompt_text:
                        self.generation_count += 1
                        variation = PromptCandidate(
                            prompt_id=f"prompt_{self.generation_count}",
                            prompt_text=prompt_text,
                            components=self._extract_components(prompt_text),
                            generation=base_prompt.generation + 1,
                            parent_id=base_prompt.prompt_id
                        )
                        variations.append(variation)
                
                return variations[:num_variations]
                
            except Exception as e:
                print(f"Error generating variations: {e}")
                # Fallback: create simple variations
                variations = []
                for i in range(min(3, num_variations)):
                    self.generation_count += 1
                    variation = PromptCandidate(
                        prompt_id=f"prompt_{self.generation_count}",
                        prompt_text=f"{base_prompt.prompt_text}\n\nVariation {i+1}: Be more specific and detailed.",
                        components=base_prompt.components.copy(),
                        generation=base_prompt.generation + 1,
                        parent_id=base_prompt.prompt_id
                    )
                    variations.append(variation)
                return variations
        
        def _extract_components(self, prompt_text: str) -> Dict[PromptComponent, str]:
            """Extract components from prompt text."""
            # Simple heuristic-based extraction
            components = {}
            
            lower_text = prompt_text.lower()
            
            # Check for role
            if 'you are' in lower_text or 'act as' in lower_text:
                components[PromptComponent.ROLE] = "role specified"
            
            # Check for instructions
            if any(word in lower_text for word in ['please', 'task:', 'do', 'generate', 'write', 'analyze']):
                components[PromptComponent.INSTRUCTION] = "instructions provided"
            
            # Check for examples
            if 'example' in lower_text or 'for instance' in lower_text:
                components[PromptComponent.EXAMPLES] = "examples included"
            
            # Check for output format
            if 'format' in lower_text or 'output:' in lower_text:
                components[PromptComponent.OUTPUT_FORMAT] = "format specified"
            
            # Check for constraints
            if any(word in lower_text for word in ['must', 'should', 'constraint', 'limit', 'only']):
                components[PromptComponent.CONSTRAINTS] = "constraints specified"
            
            return components
        
        def evaluate_prompt(
            self,
            prompt_candidate: PromptCandidate,
            test_cases: List[TestCase],
            scorer: Optional[Callable[[str, str], float]] = None
        ) -> EvaluationResult:
            """
            Evaluate a prompt on test cases.
            
            Args:
                prompt_candidate: Prompt to evaluate
                test_cases: Test cases to use
                scorer: Custom scoring function (optional)
                
            Returns:
                Evaluation result with scores
            """
            start_time = time.time()
            test_case_scores = {}
            total_tokens = 0
            
            # Default scorer: simple similarity
            if scorer is None:
                def default_scorer(output: str, expected: str) -> float:
                    output_lower = output.lower()
                    expected_lower = expected.lower()
                    # Simple keyword overlap
                    output_words = set(output_lower.split())
                    expected_words = set(expected_lower.split())
                    if not expected_words:
                        return 0.5
                    overlap = len(output_words & expected_words) / len(expected_words)
                    return min(1.0, overlap)
                scorer = default_scorer
            
            for test_case in test_cases:
                # Check cache
                cache_key = (prompt_candidate.prompt_id, test_case.case_id)
                if cache_key in self.evaluation_cache:
                    score = self.evaluation_cache[cache_key]
                else:
                    # Run prompt on test case
                    try:
                        full_prompt = ChatPromptTemplate.from_messages([
                            ("system", prompt_candidate.prompt_text),
                            ("user", test_case.input)
                        ])
                        
                        chain = full_prompt | self.evaluator_llm | self.parser
                        output = chain.invoke({})
                        
                        # Score output
                        score = scorer(output, test_case.expected_output)
                        total_tokens += len(output.split())  # Rough token estimate
                        
                        # Cache result
                        self.evaluation_cache[cache_key] = score
                        
                    except Exception as e:
                        print(f"Error evaluating test case {test_case.case_id}: {e}")
                        score = 0.0
                
                test_case_scores[test_case.case_id] = score * test_case.weight
            
            # Calculate weighted average
            total_weight = sum(tc.weight for tc in test_cases)
            average_score = sum(test_case_scores.values()) / total_weight if total_weight > 0 else 0.0
            
            latency = time.time() - start_time
            
            # Update candidate
            prompt_candidate.score = average_score
            prompt_candidate.evaluations += 1
            
            return EvaluationResult(
                prompt_id=prompt_candidate.prompt_id,
                test_case_scores=test_case_scores,
                average_score=average_score,
                latency=latency,
                tokens_used=total_tokens
            )
        
        def optimize(
            self,
            task_description: str,
            initial_prompt: str,
            test_cases: List[TestCase],
            scorer: Optional[Callable[[str, str], float]] = None
        ) -> OptimizationRun:
            """
            Run prompt optimization.
            
            Args:
                task_description: Description of task to optimize for
                initial_prompt: Starting prompt
                test_cases: Test cases for evaluation
                scorer: Custom scoring function
                
            Returns:
                Optimization run results
            """
            print(f"\n{'='*60}")
            print(f"Prompt Optimization: {task_description}")
            print(f"Strategy: {self.strategy.value}")
            print(f"{'='*60}\n")
            
            # Create initial candidate
            self.generation_count += 1
            initial_candidate = PromptCandidate(
                prompt_id=f"prompt_{self.generation_count}",
                prompt_text=initial_prompt,
                components=self._extract_components(initial_prompt),
                generation=0
            )
            
            # Evaluate initial prompt
            print("Evaluating initial prompt...")
            initial_eval = self.evaluate_prompt(initial_candidate, test_cases, scorer)
            print(f"  Initial Score: {initial_eval.average_score:.3f}")
            
            # Track all candidates and history
            all_candidates = [initial_candidate]
            optimization_history = [initial_eval]
            
            # Active candidates (beam)
            active_candidates = [initial_candidate]
            best_candidate = initial_candidate
            
            # Optimization loop
            for iteration in range(self.max_iterations):
                print(f"\nIteration {iteration + 1}/{self.max_iterations}")
                
                # Generate variations from active candidates
                new_candidates = []
                for candidate in active_candidates:
                    variations = self.generate_prompt_variations(
                        candidate,
                        num_variations=self.beam_width,
                        task_description=task_description
                    )
                    new_candidates.extend(variations)
                
                print(f"  Generated {len(new_candidates)} new candidates")
                
                # Evaluate new candidates
                print(f"  Evaluating candidates...")
                for candidate in new_candidates:
                    eval_result = self.evaluate_prompt(candidate, test_cases, scorer)
                    optimization_history.append(eval_result)
                    all_candidates.append(candidate)
                    
                    # Track best
                    if candidate.score > best_candidate.score:
                        best_candidate = candidate
                        print(f"    ✓ New best score: {candidate.score:.3f} (improvement: {candidate.score - initial_candidate.score:+.3f})")
                
                # Select top candidates for next iteration (beam search)
                if self.strategy == OptimizationStrategy.BEAM_SEARCH:
                    all_evaluated = active_candidates + new_candidates
                    all_evaluated.sort(key=lambda c: c.score, reverse=True)
                    active_candidates = all_evaluated[:self.beam_width]
                    print(f"  Keeping top {len(active_candidates)} candidates")
                
                # Early stopping if no improvement
                if iteration > 0:
                    recent_scores = [h.average_score for h in optimization_history[-self.beam_width:]]
                    if len(set(recent_scores)) == 1:
                        print(f"  Early stopping: No score variation in recent candidates")
                        break
            
            # Calculate improvement
            improvement = ((best_candidate.score - initial_candidate.score) / 
                          initial_candidate.score * 100 if initial_candidate.score > 0 else 0.0)
            
            print(f"\n{'='*60}")
            print(f"Optimization Complete")
            print(f"{'='*60}")
            print(f"Initial Score: {initial_candidate.score:.3f}")
            print(f"Best Score: {best_candidate.score:.3f}")
            print(f"Improvement: {improvement:+.1f}%")
            print(f"Total Candidates Tested: {len(all_candidates)}")
            print(f"Best Candidate Generation: {best_candidate.generation}")
            
            return OptimizationRun(
                task_description=task_description,
                initial_prompt=initial_candidate,
                best_prompt=best_candidate,
                all_candidates=all_candidates,
                optimization_history=optimization_history,
                total_iterations=iteration + 1,
                improvement=improvement
            )
        
        def analyze_success_patterns(self, optimization_run: OptimizationRun) -> Dict[str, Any]:
            """
            Analyze patterns in successful prompts.
            
            Args:
                optimization_run: Completed optimization run
                
            Returns:
                Analysis of success patterns
            """
            # Get top performers
            top_prompts = sorted(
                optimization_run.all_candidates,
                key=lambda c: c.score,
                reverse=True
            )[:5]
            
            # Analyze components
            component_scores = defaultdict(list)
            for candidate in optimization_run.all_candidates:
                for component in candidate.components:
                    component_scores[component].append(candidate.score)
            
            component_analysis = {}
            for component, scores in component_scores.items():
                component_analysis[component.value] = {
                    "avg_score": sum(scores) / len(scores) if scores else 0,
                    "count": len(scores)
                }
            
            return {
                "top_prompts": [
                    {"id": p.prompt_id, "score": p.score, "generation": p.generation}
                    for p in top_prompts
                ],
                "component_analysis": component_analysis,
                "best_generation": optimization_run.best_prompt.generation,
                "total_explored": len(optimization_run.all_candidates)
            }
    
    
    def demonstrate_prompt_optimization():
        """Demonstrate Prompt Optimization pattern."""
        
        print("="*80)
        print("PROMPT OPTIMIZATION/ENGINEERING - DEMONSTRATION")
        print("="*80)
        
        optimizer = PromptOptimizer(
            strategy=OptimizationStrategy.BEAM_SEARCH,
            beam_width=3,
            max_iterations=5
        )
        
        # Test 1: Sentiment classification optimization
        print("\n" + "="*80)
        print("TEST 1: Sentiment Classification Prompt Optimization")
        print("="*80)
        
        initial_prompt1 = "Classify the sentiment of the text."
        
        test_cases1 = [
            TestCase("tc1", "I love this product!", "positive", 1.0),
            TestCase("tc2", "This is terrible and disappointing.", "negative", 1.0),
            TestCase("tc3", "It's okay, nothing special.", "neutral", 1.0),
            TestCase("tc4", "Absolutely amazing experience!", "positive", 1.0),
            TestCase("tc5", "Worst purchase ever.", "negative", 1.0),
        ]
        
        run1 = optimizer.optimize(
            task_description="Sentiment Classification",
            initial_prompt=initial_prompt1,
            test_cases=test_cases1
        )
        
        print("\n--- BEST PROMPT FOUND ---")
        print(run1.best_prompt.prompt_text)
        print(f"\nScore: {run1.best_prompt.score:.3f}")
        print(f"Generation: {run1.best_prompt.generation}")
        
        # Test 2: Text summarization optimization
        print("\n" + "="*80)
        print("TEST 2: Text Summarization Prompt Optimization")
        print("="*80)
        
        initial_prompt2 = "Summarize this text in one sentence."
        
        test_cases2 = [
            TestCase("tc1", 
                    "The meeting discussed quarterly results and future plans.",
                    "meeting summary quarterly", 
                    1.0),
            TestCase("tc2",
                    "Climate change is affecting global weather patterns significantly.",
                    "climate change weather impact",
                    1.0),
        ]
        
        run2 = optimizer.optimize(
            task_description="Text Summarization",
            initial_prompt=initial_prompt2,
            test_cases=test_cases2
        )
        
        print("\n--- OPTIMIZATION PROGRESS ---")
        print(f"{'Iteration':<12} {'Best Score':<12} {'Candidates': iteration_bests[gen]:
                        iteration_bests[gen] = candidate.score
        
        for gen in sorted(iteration_bests.keys()):
            print(f"{gen:<12} {iteration_bests[gen]:<12.3f} {'-':<12}")
        
        # Test 3: Component analysis
        print("\n" + "="*80)
        print("TEST 3: Success Pattern Analysis")
        print("="*80)
        
        analysis = optimizer.analyze_success_patterns(run2)
        
        print("\nTop Performing Prompts:")
        for i, prompt_info in enumerate(analysis["top_prompts"], 1):
            print(f"  {i}. {prompt_info['id']}: Score={prompt_info['score']:.3f}, Gen={prompt_info['generation']}")
        
        print("\nComponent Analysis:")
        for component, stats in analysis["component_analysis"].items():
            print(f"  {component}: Avg Score={stats['avg_score']:.3f}, Count={stats['count']}")
        
        print(f"\nBest Generation: {analysis['best_generation']}")
        print(f"Total Candidates Explored: {analysis['total_explored']}")
        
        # Test 4: Compare initial vs optimized
        print("\n" + "="*80)
        print("TEST 4: Initial vs Optimized Comparison")
        print("="*80)
        
        print("\n--- INITIAL PROMPT ---")
        print(run1.initial_prompt.prompt_text)
        print(f"Score: {run1.initial_prompt.score:.3f}")
        
        print("\n--- OPTIMIZED PROMPT ---")
        print(run1.best_prompt.prompt_text)
        print(f"Score: {run1.best_prompt.score:.3f}")
        print(f"Improvement: {run1.improvement:+.1f}%")
        
        # Test 5: Optimization history visualization
        print("\n" + "="*80)
        print("TEST 5: Optimization History")
        print("="*80)
        
        print("\nScore Distribution by Generation:")
        gen_scores = defaultdict(list)
        for candidate in run1.all_candidates:
            gen_scores[candidate.generation].append(candidate.score)
        
        for gen in sorted(gen_scores.keys()):
            scores = gen_scores[gen]
            avg_score = sum(scores) / len(scores)
            max_score = max(scores)
            print(f"  Gen {gen}: Avg={avg_score:.3f}, Max={max_score:.3f}, Count={len(scores)}")
        
        # Summary
        print("\n" + "="*80)
        print("SUMMARY: Prompt Optimization/Engineering")
        print("="*80)
        print("""
    The Prompt Optimization pattern demonstrates automated prompt improvement:
    
    Key Features Demonstrated:
    1. Prompt Generation: Creates systematic variations of prompts
    2. Evaluation Engine: Measures performance on test cases
    3. Beam Search: Maintains top candidates across generations
    4. Component Analysis: Identifies successful prompt elements
    5. Iterative Refinement: Progressively improves prompts
    6. Performance Tracking: Monitors improvement over iterations
    7. Pattern Analysis: Extracts insights from successful prompts
    
    Benefits:
    • Finds better prompts than manual engineering
    • Saves significant development and testing time
    • Discovers non-obvious prompt structures
    • Data-driven prompt decisions with measurable results
    • Scales across multiple tasks and models
    • Adapts to model updates automatically
    • Provides interpretable optimization process
    
    Use Cases:
    • Production LLM system prompt optimization
    • Few-shot example selection and ordering
    • Task-specific instruction refinement
    • Output format specification tuning
    • Multi-task prompt engineering
    • Domain-specific prompt adaptation
    • Cost-performance trade-off optimization
    • Latency-optimized prompt design
    
    Comparison with Manual Prompt Engineering:
    ┌──────────────────────┬────────────────────┬─────────────────────┐
    │ Aspect               │ Manual             │ Automated           │
    ├──────────────────────┼────────────────────┼─────────────────────┤
    │ Time Required        │ Hours to days      │ Minutes to hours    │
    │ Prompts Tested       │ 5-20               │ 50-500+             │
    │ Consistency          │ Variable           │ Systematic          │
    │ Optimization Quality │ Good               │ Excellent           │
    │ Scalability          │ Limited            │ High                │
    │ Reproducibility      │ Low                │ High                │
    │ Adaptation Speed     │ Slow               │ Fast                │
    └──────────────────────┴────────────────────┴─────────────────────┘
    
    Optimization Strategies:
    • Random Search: Explores diverse prompt space randomly
    • Hill Climbing: Iteratively improves from best candidate
    • Beam Search: Maintains multiple top candidates (demonstrated)
    • Genetic Algorithms: Evolves prompts through crossover/mutation
    • Gradient-Free: Uses performance signals without gradients
    • Bayesian Optimization: Models prompt space probabilistically
    
    LangChain Implementation Notes:
    • Multiple LLMs for generation and evaluation
    • Beam search for systematic exploration
    • Component extraction for pattern analysis
    • Evaluation caching to avoid redundant tests
    • Generational tracking for improvement monitoring
    • Flexible scoring functions for different tasks
    
    Production Considerations:
    • Implement comprehensive test case suites (50-500+ cases)
    • Add A/B testing for production deployment
    • Implement prompt versioning and rollback
    • Add cost tracking for optimization runs
    • Implement human evaluation for quality checks
    • Add multi-objective optimization (accuracy, cost, latency)
    • Implement domain-specific prompt templates
    • Add safety and bias checks for generated prompts
    • Implement continuous optimization with production data
    • Add monitoring for prompt performance drift
    • Implement transfer learning across similar tasks
    • Add prompt explanation generation for interpretability
    
    Advanced Extensions:
    • Multi-task prompt optimization with shared components
    • Adversarial prompt testing for robustness
    • Meta-learning for faster optimization
    • Neural prompt generation using learned models
    • Active learning for test case selection
    • Prompt compression for cost reduction
    • Cross-model prompt optimization
    • Hierarchical prompt structures
    • Prompt ensembling for reliability
    • Uncertainty-aware prompt selection
    
    Real-World Applications:
    • Customer service chatbot optimization
    • Content generation quality improvement
    • Code generation prompt refinement
    • Medical diagnosis prompt engineering
    • Legal document analysis optimization
    • Educational content generation
    • Multi-lingual translation prompt tuning
    • Creative writing assistant optimization
        """)
    
    
    if __name__ == "__main__":
        demonstrate_prompt_optimization()
    
        

    Agentic Design Pattern: Prompt Optimization

    Pattern 070: Prompt Optimization – Line-by-Line Explanation

    Overview

    This document provides a detailed line-by-line explanation of the Prompt Optimization pattern implementation.


    Pattern Description

    Pattern 070: Prompt Optimization/Engineering

    Description: Prompt Optimization/Engineering automates the process of finding optimal prompts for specific tasks. Rather than manually crafting and testing prompts, this pattern uses automated search, evaluation, and refinement to discover high-performing prompt formulations. It explores the prompt space systematically, tests variations, measures effectiveness, and iteratively improves prompts to maximize task performance.

    This pattern is essential for building production LLM systems where prompt quality
    directly impacts output quality, cost, and latency. It can discover non-obvious
    prompt structures, optimal instruction phrasing, effective examples for few-shot
    learning, and task-specific formatting that humans might not consider.
    

    Components: 1. Prompt Generator: Creates candidate prompt variations 2. Evaluation Engine: Measures prompt effectiveness on test cases 3. Optimization Strategy: Guides search through prompt space 4. Prompt Analyzer: Extracts features from successful prompts 5. Meta-Prompt Designer: Creates templates for prompt generation 6. Performance Tracker: Monitors metrics across iterations 7. Prompt Library: Stores and manages successful prompts

    Architecture: Initial Prompt ↓ ┌─────────────────────────────┐ │ Optimization Loop │ │ │ │ Generate Variations │ │ ↓ │ │ Evaluate on Test Cases │ │ ↓ │ │ Rank by Performance │ │ ↓ │ │ Select Best Candidates │ │ ↓ │ │ Analyze Success Patterns │ │ ↓ │ │ Generate New Variations │ └─────────────────────────────┘ ↓ (iterate) Optimized Prompt

    Use Cases: – Automatic prompt engineering for production systems – Few-shot example selection and ordering – Task-specific instruction optimization – Output format specification tuning – Zero-shot prompt improvement – Multi-task prompt optimization

    Advantages: – Finds better prompts than manual engineering – Saves significant development time – Discovers non-obvious prompt patterns – Adapts to model updates automatically – Provides data-driven prompt decisions – Scales across multiple tasks

    LangChain Implementation: Uses ChatOpenAI for prompt generation and testing. Demonstrates prompt variation generation, evaluation on test cases, iterative optimization, and performance tracking.


    File Statistics

    • Total Lines: 770
    • Classes: 7
    • Functions: 1
    • Constants: 0

    Module Documentation

    """
    Pattern 070: Prompt Optimization/Engineering
    
    Description:
        Prompt Optimization/Engineering automates the process of finding optimal prompts for
        specific tasks. Rather than manually crafting and testing prompts, this pattern uses
        automated search, evaluation, and refinement to discover high-performing prompt
        formulations. It explores the prompt space systematically, tests variations, measures
        effectiveness, and iteratively improves prompts to maximize task performance.
    
        This pattern is essential for building production LLM systems where prompt quality
        directly impacts output quality, cost, and latency. It can discover non-obvious
        prompt structures, optimal instruction phrasing, effective examples for few-shot
        learning, and task-specific formatting that humans might not consider.
    
    Components:
        1. Prompt Generator: Creates candidate prompt variations
        2. Evaluation Engine: Measures prompt effectiveness on test cases
        3. Optimization Strategy: Guides search through prompt space
        4. Prompt Analyzer: Extracts features from successful prompts
        5. Meta-Prompt Designer: Creates templates for prompt generation
        6. Performance Tracker: Monitors metrics across iterations
        7. Prompt Library: Stores and manages successful prompts
    
    Architecture:
        ```
        Initial Prompt
            ↓
        ┌─────────────────────────────┐
        │   Optimization Loop         │
        │                             │
        │  Generate Variations        │
        │         ↓                   │
        │  Evaluate on Test Cases     │
        │         ↓                   │
        │  Rank by Performance        │
        │         ↓                   │
        │  Select Best Candidates     │
        │         ↓                   │
        │  Analyze Success Patterns   │
        │         ↓                   │
        │  Generate New Variations    │
        └─────────────────────────────┘
             ↓ (iterate)
        Optimized Prompt
        ```
    
    Use Cases:
        - Automatic prompt engineering for production systems
        - Few-shot example selection and ordering
        - Task-specific instruction optimization
        - Output format specification tuning
        - Zero-shot prompt improvement
        - Multi-task prompt optimization
    
    Advantages:
        - Finds better prompts than manual engineering
        - Saves significant development time
        - Discovers non-obvious prompt patterns
        - Adapts to model updates automatically
        - Provides data-driven prompt decisions
        - Scales across multiple tasks
    
    LangChain Implementation:
        Uses ChatOpenAI for prompt generation and testing. Demonstrates
        prompt variation generation, evaluation on test cases, iterative
        optimization, and performance tracking.
    """
    

    Purpose: This module implements the Prompt Optimization pattern, providing a structured approach to agent design and implementation.


    Import Statements

    Standard Library Imports

    import collections.defaultdict
    import dataclasses.dataclass
    import dataclasses.field
    import enum.Enum
    import langchain_core.output_parsers.StrOutputParser
    import langchain_core.prompts.ChatPromptTemplate
    import os
    import random
    import time
    import typing.Any
    import typing.Callable
    import typing.Dict
    import typing.List
    import typing.Optional
    import typing.Tuple
    

    Purpose: Core Python functionality: – collections.defaultdict: Specialized container datatypes – dataclasses.dataclass: Simplified class definitions with @dataclass – dataclasses.field: Simplified class definitions with @dataclass – enum.Enum: Enumeration support – langchain_core.output_parsers.StrOutputParser: Regular expression operations – langchain_core.prompts.ChatPromptTemplate: Regular expression operations – os: Environment variable access and system operations – random: Random number generation – time: Time-related functions – typing.Any: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Callable: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Dict: Type hints for better code documentation (List, Dict, Any, Optional) – typing.List: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Optional: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Tuple: Type hints for better code documentation (List, Dict, Any, Optional)

    LangChain Imports

    from langchain_openai.ChatOpenAI
    

    Purpose: LangChain framework components: – LLM Interface: Wrapper for OpenAI models – Chains: Composable building blocks for complex workflows

    Other Imports

    from dotenv.load_dotenv
    

    Purpose: Additional dependencies for specialized functionality.


    Classes

    Class: OptimizationStrategy

    Lines 86-92

    Inherits from: Enum

    Purpose:

    Optimization strategies for prompt search.
    

    Class: PromptComponent

    Lines 95-102

    Inherits from: Enum

    Purpose:

    Components of a prompt that can be optimized.
    

    Class: TestCase

    Lines 106-111

    Purpose:

    Test case for prompt evaluation.
    

    Class: PromptCandidate

    Lines 115-123

    Purpose:

    A candidate prompt with metadata.
    

    Class: EvaluationResult

    Lines 127-133

    Purpose:

    Result of evaluating a prompt on test cases.
    

    Class: OptimizationRun

    Lines 137-145

    Purpose:

    Complete optimization run results.
    

    Class: PromptOptimizer

    Lines 148-537

    Purpose:

    Automated prompt optimization system.
    

    Methods:

    __init__(self, strategy, beam_width, max_iterations)

    Lines 153-181

    Purpose:

    Initialize prompt optimizer.
    
    Args:
        strategy: Optimization strategy to use
        beam_width: Number of candidates to keep (for beam search)
        max_iterations: Maximum optimization iterations
    

    Parameters: – strategy: Strategy – beam_width: Beam width – max_iterations: Max iterations

    generate_prompt_variations(self, base_prompt, num_variations, task_description)

    Lines 183-275

    Purpose:

    Generate variations of a prompt.
    
    Args:
        base_prompt: Starting prompt
        num_variations: Number of variations to generate
        task_description: Description of the task
    
    Returns:
        List of prompt variations
    

    Parameters: – base_prompt: Prompt template or text – num_variations: Num variations – task_description: Task description

    Returns: <ast.Subscript object at 0x000001A06D77E090>

    _extract_components(self, prompt_text)

    Lines 277-304

    Purpose:

    Extract components from prompt text.
    

    Parameters: – prompt_text: Prompt template or text

    Returns: <ast.Subscript object at 0x000001A06D663E50>

    evaluate_prompt(self, prompt_candidate, test_cases, scorer)

    Lines 306-386

    Purpose:

    Evaluate a prompt on test cases.
    
    Args:
        prompt_candidate: Prompt to evaluate
        test_cases: Test cases to use
        scorer: Custom scoring function (optional)
    
    Returns:
        Evaluation result with scores
    

    Parameters: – prompt_candidate: Prompt template or text – test_cases: Test cases – scorer: Scorer

    Returns: EvaluationResult

    optimize(self, task_description, initial_prompt, test_cases, scorer)

    Lines 388-497

    Purpose:

    Run prompt optimization.
    
    Args:
        task_description: Description of task to optimize for
        initial_prompt: Starting prompt
        test_cases: Test cases for evaluation
        scorer: Custom scoring function
    
    Returns:
        Optimization run results
    

    Parameters: – task_description: Task description – initial_prompt: Prompt template or text – test_cases: Test cases – scorer: Scorer

    Returns: OptimizationRun

    analyze_success_patterns(self, optimization_run)

    Lines 499-537

    Purpose:

    Analyze patterns in successful prompts.
    
    Args:
        optimization_run: Completed optimization run
    
    Returns:
        Analysis of success patterns
    

    Parameters: – optimization_run: Optimization run

    Returns: <ast.Subscript object at 0x000001A06D627A10>


    Functions

    demonstrate_prompt_optimization()

    Lines 540-765

    Purpose:

    Demonstrate Prompt Optimization pattern.
    

    Key Concepts Summary

    Pattern Overview

    The Prompt Optimization pattern provides a structured approach to building AI agents with specific capabilities and behaviors. This implementation leverages LangChain’s composable components to create a robust and maintainable solution.

    Core Components

    1. Agent Class: Main implementation of the pattern logic
    2. Prompt Engineering: Carefully crafted prompts to guide LLM behavior
    3. Tool Integration: Optional tools for extended functionality
    4. Memory Management: State handling across interactions
    5. Output Parsing: Structured response processing

    When to Use

    ✅ Good for: – Scenarios requiring this specific pattern’s strengths – Applications needing structured agent behavior – Systems where explainability is important – Production environments requiring reliability

    ❌ Not ideal for: – Simple single-prompt tasks – Scenarios requiring different pattern characteristics – Resource-constrained environments (if computationally intensive)


    Best Practices Demonstrated

    1. Environment Configuration

    • ✅ Use .env files for API keys
    • ✅ Never hardcode credentials
    • ✅ Validate environment variables on startup

    2. Error Handling

    • ✅ Graceful degradation on failures
    • ✅ Informative error messages
    • ✅ Logging for debugging

    3. Prompt Engineering

    • ✅ Clear, specific instructions
    • ✅ Structured output formats
    • ✅ Few-shot examples when beneficial

    4. Code Organization

    • ✅ Separation of concerns (class-based design)
    • ✅ Type hints for clarity
    • ✅ Docstrings for documentation

    5. Performance Optimization

    • ✅ Appropriate temperature settings
    • ✅ Token budget management
    • ✅ Caching where applicable

    LangChain Components Used

    • ChatOpenAI: OpenAI model interface
    • Chains: Sequential component composition

    Why LangChain? – Composable abstractions for LLM applications – Rich ecosystem of integrations – Production-ready patterns – Active community and development


    Execution Flow

    1. Initialize agent with configuration
       ↓
    2. Load environment and dependencies
       ↓
    3. Prepare prompt with input data
       ↓
    4. Invoke LLM through LangChain
       ↓
    5. Parse and validate response
       ↓
    6. Return formatted result
    

    Key Decision Points: – Model selection (temperature, model type) – Prompt structure (few-shot vs zero-shot) – Error handling strategy – Output format validation


    Performance Considerations

    Computational Cost

    • LLM Calls: Primary cost factor (based on tokens)
    • Latency: Depends on model and input size
    • Memory: Minimal for most patterns

    Optimization Strategies

    1. Prompt Optimization: Minimize tokens while maintaining clarity
    2. Caching: Cache repeated queries/responses
    3. Model Selection: Use smaller models when appropriate
    4. Batch Processing: Group similar requests
    5. Streaming: For long responses, stream tokens

    Cost Estimates (Approximate)

    • GPT-3.5-turbo: ~$0.002 per request
    • GPT-4: ~$0.03 per request
    • GPT-4-turbo: ~$0.01 per request

    Actual costs vary based on input/output token counts


    Common Pitfalls to Avoid

    1. Prompt Design Issues

    ❌ Vague or ambiguous instructions
    ✅ Clear, specific, testable prompts

    2. Error Handling

    ❌ Ignoring API errors or rate limits
    ✅ Implement retry logic and graceful degradation

    3. Token Management

    ❌ Exceeding model token limits
    ✅ Truncate inputs, chunk processing

    4. Security

    ❌ Exposing API keys in code
    ✅ Use environment variables

    5. Testing

    ❌ Only testing happy paths
    ✅ Test edge cases, failures, malformed inputs

    6. Cost Management

    ❌ Unmonitored API usage
    ✅ Implement budgets, alerts, and monitoring


    Further Improvements

    Production Enhancements

    1. Monitoring: Add logging, metrics, tracing
    2. Validation: Input/output schema validation
    3. Rate Limiting: Implement request throttling
    4. Caching: Redis or in-memory cache for responses
    5. Async Support: Non-blocking operations for scalability

    Feature Extensions

    1. Multi-Model Support: Switch between providers
    2. Fallback Models: Graceful degradation strategy
    3. Custom Tools: Domain-specific function calling
    4. Advanced Memory: Vector stores for long-term memory
    5. Evaluation: Automated testing and benchmarking

    Code Quality

    1. Testing: Unit tests, integration tests
    2. Documentation: API docs, usage examples
    3. Type Safety: Stricter type hints
    4. Configuration: Externalized config files

    Related Patterns

    • Pattern 069
    • Pattern 071

    Pattern Families: – Reasoning Patterns (001-011) – Multi-Agent Patterns (012-019) – Tool Use Patterns (020-025) – Memory Patterns (026-031) – And many more…


    Summary

    The Prompt Optimization pattern provides a powerful approach to building AI agents with specific capabilities. This implementation demonstrates:

    ✅ Key Strengths: – Clean, maintainable code structure – LangChain integration for composability – Type hints and documentation – Production-ready patterns

    🎯 Best Use Cases: – Applications requiring this pattern’s characteristics – Systems needing structured agent behavior – Production environments requiring reliability

    📚 Learning Resources: – LangChain Documentation: https://python.langchain.com/ – Pattern Documentation: See related patterns – OpenAI API Reference: https://platform.openai.com/docs/


    File: 070_prompt_optimization.py
    Total Lines: 770
    Pattern Category: AI Agent Design Pattern #070


    Generated from actual implementation code
    For educational and reference purposes

  • Agentic Design Pattern: Self Play

    
        """
    Pattern 069: Self-Play & Self-Improvement
    
    Description:
        Self-Play & Self-Improvement enables agents to autonomously improve their capabilities
        by competing or collaborating with themselves. Inspired by techniques like AlphaGo's
        self-play training, this pattern allows agents to generate their own training data,
        discover new strategies, and continuously refine their performance without external
        supervision. The agent plays both sides of a game, problem, or scenario, learning
        from each iteration to progressively improve.
        
        This pattern is particularly powerful for domains where supervised training data is
        scarce, expensive, or impossible to obtain. Through self-play, agents can explore
        vast strategy spaces, discover novel solutions, and achieve superhuman performance
        by learning from billions of self-generated experiences.
    
    Components:
        1. Strategy Generator: Creates candidate strategies/policies
        2. Self-Play Engine: Simulates interactions with itself
        3. Performance Evaluator: Measures strategy effectiveness
        4. Curriculum Generator: Creates progressively harder challenges
        5. Strategy Selector: Chooses best strategies to keep
        6. Meta-Learner: Learns how to improve the learning process
        7. Checkpoint Manager: Stores and manages strategy versions
    
    Architecture:
        ```
        Initial Strategy
            ↓
        ┌─────────────────────────┐
        │   Self-Play Loop        │
        │                         │
        │  Strategy A vs Strategy B│
        │         ↓               │
        │    Play Game/Task       │
        │         ↓               │
        │   Evaluate Performance  │
        │         ↓               │
        │   Update Strategies     │
        │         ↓               │
        │  Generate Curriculum    │
        │         ↓               │
        │   Select Best           │
        └─────────────────────────┘
             ↓ (iterate)
        Improved Strategy
        ```
    
    Use Cases:
        - Game AI development (chess, Go, poker)
        - Negotiation and bargaining agents
        - Creative content generation
        - Code optimization and debugging
        - Theorem proving and mathematical reasoning
        - Strategic planning and decision-making
    
    Advantages:
        - No need for labeled training data
        - Discovers novel strategies humans haven't found
        - Continuously improves without external input
        - Adapts to changing environments
        - Scales with compute rather than data
        - Explores diverse solution spaces
    
    LangChain Implementation:
        Uses ChatOpenAI for strategy generation and evaluation. Demonstrates
        self-play loops, performance tracking, curriculum generation, and
        iterative improvement through competition.
    """
    
    import os
    from typing import List, Dict, Any, Optional, Tuple
    from dataclasses import dataclass, field
    from enum import Enum
    import time
    import random
    from collections import deque
    from dotenv import load_dotenv
    
    from langchain_core.prompts import ChatPromptTemplate
    from langchain_core.output_parsers import StrOutputParser
    from langchain_openai import ChatOpenAI
    
    load_dotenv()
    
    
    class TaskType(Enum):
        """Types of self-play tasks."""
        GAME = "game"
        PROBLEM_SOLVING = "problem_solving"
        NEGOTIATION = "negotiation"
        CREATIVE = "creative"
        OPTIMIZATION = "optimization"
    
    
    class StrategyQuality(Enum):
        """Quality levels of strategies."""
        NOVICE = "novice"
        INTERMEDIATE = "intermediate"
        ADVANCED = "advanced"
        EXPERT = "expert"
        SUPERHUMAN = "superhuman"
    
    
    @dataclass
    class Strategy:
        """Represents a strategy/policy."""
        strategy_id: str
        name: str
        description: str
        version: int
        quality: StrategyQuality
        win_rate: float = 0.0
        games_played: int = 0
        elo_rating: float = 1000.0  # Elo rating system
        created_at: float = field(default_factory=time.time)
    
    
    @dataclass
    class GameResult:
        """Result of a self-play game."""
        game_id: str
        strategy_a: Strategy
        strategy_b: Strategy
        winner: Optional[str]  # strategy_id or None for draw
        score_a: float
        score_b: float
        moves: List[str]
        insights: List[str] = field(default_factory=list)
    
    
    @dataclass
    class TrainingExample:
        """Training example generated from self-play."""
        example_id: str
        situation: str
        action: str
        outcome: str
        reward: float
        strategy_used: str
    
    
    @dataclass
    class CurriculumLevel:
        """Difficulty level in curriculum."""
        level: int
        name: str
        description: str
        complexity: float  # 0.0 to 1.0
        prerequisite_levels: List[int] = field(default_factory=list)
    
    
    @dataclass
    class ImprovementMetrics:
        """Metrics tracking improvement over time."""
        iteration: int
        best_strategy_quality: StrategyQuality
        average_elo: float
        total_games: int
        novel_strategies_found: int
        improvement_rate: float  # % improvement per iteration
    
    
    class SelfPlayAgent:
        """
        Agent that improves through self-play and self-generated curriculum.
        """
        
        def __init__(
            self,
            task_type: TaskType = TaskType.PROBLEM_SOLVING,
            initial_strategies: int = 3,
            elo_k_factor: int = 32
        ):
            """
            Initialize self-play agent.
            
            Args:
                task_type: Type of task for self-play
                initial_strategies: Number of initial strategies to generate
                elo_k_factor: K-factor for Elo rating updates
            """
            self.task_type = task_type
            self.elo_k_factor = elo_k_factor
            
            # LLMs for different tasks
            self.strategy_generator = ChatOpenAI(temperature=0.8, model="gpt-3.5-turbo")
            self.game_master = ChatOpenAI(temperature=0.5, model="gpt-3.5-turbo")
            self.evaluator = ChatOpenAI(temperature=0.2, model="gpt-3.5-turbo")
            self.meta_learner = ChatOpenAI(temperature=0.6, model="gpt-3.5-turbo")
            
            self.parser = StrOutputParser()
            
            # Strategy pool
            self.strategies: Dict[str, Strategy] = {}
            self.strategy_history: List[Strategy] = []
            
            # Training data
            self.training_examples: deque = deque(maxlen=1000)
            self.game_history: List[GameResult] = []
            
            # Curriculum
            self.curriculum_levels: List[CurriculumLevel] = []
            self.current_level = 0
            
            # Metrics
            self.improvement_history: List[ImprovementMetrics] = []
            self.iteration = 0
            
            # Initialize
            self._initialize_strategies(initial_strategies)
            self._initialize_curriculum()
        
        def _initialize_strategies(self, count: int):
            """Initialize starting strategies."""
            print(f"Initializing {count} starting strategies...")
            
            for i in range(count):
                strategy = Strategy(
                    strategy_id=f"strat_{i+1}",
                    name=f"Strategy {i+1}",
                    description=f"Initial strategy variant {i+1}",
                    version=1,
                    quality=StrategyQuality.NOVICE
                )
                self.strategies[strategy.strategy_id] = strategy
                print(f"  Created: {strategy.name}")
        
        def _initialize_curriculum(self):
            """Initialize learning curriculum."""
            self.curriculum_levels = [
                CurriculumLevel(1, "Basics", "Learn fundamental concepts", 0.2),
                CurriculumLevel(2, "Intermediate", "Apply concepts in simple scenarios", 0.4, [1]),
                CurriculumLevel(3, "Advanced", "Handle complex situations", 0.6, [2]),
                CurriculumLevel(4, "Expert", "Master edge cases and optimization", 0.8, [3]),
                CurriculumLevel(5, "Creative", "Discover novel strategies", 1.0, [4]),
            ]
        
        def generate_new_strategy(self, parent_strategies: List[Strategy]) -> Strategy:
            """
            Generate new strategy by learning from existing ones.
            
            Args:
                parent_strategies: Strategies to learn from
                
            Returns:
                New improved strategy
            """
            prompt = ChatPromptTemplate.from_messages([
                ("system", """You are a strategy generator. Create a new improved strategy by learning 
    from existing strategies. Combine their strengths and address their weaknesses.
    
    Task Type: {task_type}
    Current Level: {level}
    
    Create a strategy with:
    1. Name
    2. Description (key principles and tactics)
    3. Novel insights or improvements
    
    Format:
    Name: [strategy name]
    Description: [detailed description]
    Insights: [key innovations]"""),
                ("user", """Parent Strategies:
    {parent_strategies}
    
    Generate an improved strategy:""")
            ])
            
            chain = prompt | self.strategy_generator | self.parser
            
            try:
                parent_desc = "\n".join([
                    f"- {s.name} (Win Rate: {s.win_rate:.2f}, Games: {s.games_played}): {s.description}"
                    for s in parent_strategies
                ])
                
                level = self.curriculum_levels[min(self.current_level, len(self.curriculum_levels)-1)]
                
                result = chain.invoke({
                    "task_type": self.task_type.value,
                    "level": level.name,
                    "parent_strategies": parent_desc
                })
                
                # Parse result
                name = f"Strategy Gen{len(self.strategies)+1}"
                description = ""
                
                for line in result.split('\n'):
                    if line.startswith("Name:"):
                        name = line.replace("Name:", "").strip()
                    elif line.startswith("Description:"):
                        desc_lines = []
                        for next_line in result.split('\n')[result.split('\n').index(line):]:
                            if next_line.startswith("Insights:"):
                                break
                            desc_lines.append(next_line.replace("Description:", "").strip())
                        description = " ".join([d for d in desc_lines if d])
                
                if not description:
                    description = result.split('\n')[0]
                
                # Determine quality based on parent strategies
                avg_quality_idx = sum([list(StrategyQuality).index(s.quality) for s in parent_strategies]) // len(parent_strategies)
                quality = list(StrategyQuality)[min(avg_quality_idx + 1, len(StrategyQuality) - 1)]
                
                strategy = Strategy(
                    strategy_id=f"strat_{len(self.strategies)+1}",
                    name=name,
                    description=description,
                    version=1,
                    quality=quality,
                    elo_rating=sum(s.elo_rating for s in parent_strategies) / len(parent_strategies)
                )
                
                return strategy
                
            except Exception as e:
                print(f"Error generating strategy: {e}")
                # Fallback: create simple variant
                return Strategy(
                    strategy_id=f"strat_{len(self.strategies)+1}",
                    name=f"Strategy Variant {len(self.strategies)+1}",
                    description="Evolved strategy",
                    version=1,
                    quality=StrategyQuality.INTERMEDIATE
                )
        
        def play_game(
            self,
            strategy_a: Strategy,
            strategy_b: Strategy,
            scenario: str
        ) -> GameResult:
            """
            Simulate a game between two strategies.
            
            Args:
                strategy_a: First strategy
                strategy_b: Second strategy
                scenario: Game scenario/problem
                
            Returns:
                Game result with winner and insights
            """
            prompt = ChatPromptTemplate.from_messages([
                ("system", """You are a game master. Simulate a match between two strategies.
    
    Strategy A: {strategy_a_name}
    {strategy_a_desc}
    
    Strategy B: {strategy_b_name}
    {strategy_b_desc}
    
    Scenario: {scenario}
    
    Simulate the match and provide:
    Winner: [A/B/Draw]
    Score A: [0.0-1.0]
    Score B: [0.0-1.0]
    Key Moves: [list of 3-5 key moves/decisions]
    Insights: [what made the difference, what was learned]
    
    Format each on a new line."""),
                ("user", "Simulate the match:")
            ])
            
            chain = prompt | self.game_master | self.parser
            
            try:
                result = chain.invoke({
                    "strategy_a_name": strategy_a.name,
                    "strategy_a_desc": strategy_a.description,
                    "strategy_b_name": strategy_b.name,
                    "strategy_b_desc": strategy_b.description,
                    "scenario": scenario
                })
                
                # Parse result
                winner = None
                score_a = 0.5
                score_b = 0.5
                moves = []
                insights = []
                
                for line in result.split('\n'):
                    line = line.strip()
                    if line.startswith("Winner:"):
                        winner_str = line.replace("Winner:", "").strip().upper()
                        if 'A' in winner_str and 'DRAW' not in winner_str:
                            winner = strategy_a.strategy_id
                            score_a = 1.0
                            score_b = 0.0
                        elif 'B' in winner_str:
                            winner = strategy_b.strategy_id
                            score_a = 0.0
                            score_b = 1.0
                    elif line.startswith("Score A:"):
                        try:
                            score_a = float(line.replace("Score A:", "").strip())
                        except:
                            pass
                    elif line.startswith("Score B:"):
                        try:
                            score_b = float(line.replace("Score B:", "").strip())
                        except:
                            pass
                    elif line.startswith("Key Moves:"):
                        moves_str = line.replace("Key Moves:", "").strip()
                        moves = [m.strip() for m in moves_str.split(',')]
                    elif line.startswith("Insights:"):
                        insights_str = line.replace("Insights:", "").strip()
                        insights = [i.strip() for i in insights_str.split(',')]
                
                game_result = GameResult(
                    game_id=f"game_{len(self.game_history)+1}",
                    strategy_a=strategy_a,
                    strategy_b=strategy_b,
                    winner=winner,
                    score_a=score_a,
                    score_b=score_b,
                    moves=moves if moves else ["Move 1", "Move 2", "Move 3"],
                    insights=insights if insights else ["Strategies tested"]
                )
                
                return game_result
                
            except Exception as e:
                print(f"Error playing game: {e}")
                # Fallback: random winner
                winner = random.choice([strategy_a.strategy_id, strategy_b.strategy_id, None])
                return GameResult(
                    game_id=f"game_{len(self.game_history)+1}",
                    strategy_a=strategy_a,
                    strategy_b=strategy_b,
                    winner=winner,
                    score_a=0.5 if winner is None else (1.0 if winner == strategy_a.strategy_id else 0.0),
                    score_b=0.5 if winner is None else (1.0 if winner == strategy_b.strategy_id else 0.0),
                    moves=["Simulated game"],
                    insights=["Game completed"]
                )
        
        def update_elo_ratings(self, game_result: GameResult):
            """
            Update Elo ratings based on game result.
            
            Args:
                game_result: Result of the game
            """
            strategy_a = game_result.strategy_a
            strategy_b = game_result.strategy_b
            
            # Expected scores
            expected_a = 1.0 / (1.0 + 10 ** ((strategy_b.elo_rating - strategy_a.elo_rating) / 400))
            expected_b = 1.0 / (1.0 + 10 ** ((strategy_a.elo_rating - strategy_b.elo_rating) / 400))
            
            # Actual scores
            actual_a = game_result.score_a
            actual_b = game_result.score_b
            
            # Update ratings
            strategy_a.elo_rating += self.elo_k_factor * (actual_a - expected_a)
            strategy_b.elo_rating += self.elo_k_factor * (actual_b - expected_b)
            
            # Update stats
            strategy_a.games_played += 1
            strategy_b.games_played += 1
            
            if game_result.winner == strategy_a.strategy_id:
                strategy_a.win_rate = (strategy_a.win_rate * (strategy_a.games_played - 1) + 1.0) / strategy_a.games_played
            else:
                strategy_a.win_rate = (strategy_a.win_rate * (strategy_a.games_played - 1)) / strategy_a.games_played
            
            if game_result.winner == strategy_b.strategy_id:
                strategy_b.win_rate = (strategy_b.win_rate * (strategy_b.games_played - 1) + 1.0) / strategy_b.games_played
            else:
                strategy_b.win_rate = (strategy_b.win_rate * (strategy_b.games_played - 1)) / strategy_b.games_played
        
        def generate_curriculum_scenario(self, level: CurriculumLevel) -> str:
            """
            Generate training scenario for current curriculum level.
            
            Args:
                level: Curriculum level
                
            Returns:
                Scenario description
            """
            scenarios = {
                1: "Solve a basic optimization problem with clear constraints.",
                2: "Handle a scenario with competing objectives and trade-offs.",
                3: "Manage complex multi-step problem with hidden information.",
                4: "Optimize performance in adversarial environment with uncertainty.",
                5: "Discover creative solution to novel problem never seen before.",
            }
            return scenarios.get(level.level, "General problem-solving scenario")
        
        def self_play_iteration(self, num_games: int = 5) -> ImprovementMetrics:
            """
            Run one iteration of self-play training.
            
            Args:
                num_games: Number of games to play this iteration
                
            Returns:
                Improvement metrics for this iteration
            """
            self.iteration += 1
            print(f"\n{'='*60}")
            print(f"Self-Play Iteration {self.iteration}")
            print(f"{'='*60}")
            
            # Get current curriculum level
            level = self.curriculum_levels[min(self.current_level, len(self.curriculum_levels)-1)]
            print(f"Curriculum Level: {level.name} (Complexity: {level.complexity})")
            
            # Play games
            print(f"\nPlaying {num_games} games...")
            iteration_games = []
            
            strategies_list = list(self.strategies.values())
            
            for i in range(num_games):
                # Select two random strategies
                if len(strategies_list)  10:
                worst_strategy = min(self.strategies.values(), key=lambda s: s.elo_rating)
                del self.strategies[worst_strategy.strategy_id]
                print(f"  Pruned: {worst_strategy.name}")
            
            # Update curriculum if ready
            if self.iteration % 5 == 0 and self.current_level  0:
                prev_avg_elo = self.improvement_history[-1].average_elo
                improvement_rate = ((avg_elo - prev_avg_elo) / prev_avg_elo) * 100 if prev_avg_elo > 0 else 0.0
            
            metrics = ImprovementMetrics(
                iteration=self.iteration,
                best_strategy_quality=best_strategy.quality,
                average_elo=avg_elo,
                total_games=len(self.game_history),
                novel_strategies_found=len(self.strategy_history),
                improvement_rate=improvement_rate
            )
            
            self.improvement_history.append(metrics)
            
            print(f"\n--- Iteration {self.iteration} Metrics ---")
            print(f"Best Strategy: {best_strategy.name} (Elo: {best_strategy.elo_rating:.0f})")
            print(f"Average Elo: {avg_elo:.0f}")
            print(f"Total Strategies: {len(self.strategies)}")
            print(f"Improvement Rate: {improvement_rate:+.2f}%")
            
            return metrics
        
        def train(self, iterations: int = 10, games_per_iteration: int = 5):
            """
            Run full self-play training.
            
            Args:
                iterations: Number of training iterations
                games_per_iteration: Games to play per iteration
            """
            print("="*60)
            print("STARTING SELF-PLAY TRAINING")
            print("="*60)
            print(f"Task Type: {self.task_type.value}")
            print(f"Iterations: {iterations}")
            print(f"Games per Iteration: {games_per_iteration}")
            
            for i in range(iterations):
                self.self_play_iteration(games_per_iteration)
            
            print("\n" + "="*60)
            print("TRAINING COMPLETE")
            print("="*60)
            
            # Final summary
            best = max(self.strategies.values(), key=lambda s: s.elo_rating)
            print(f"\nBest Strategy: {best.name}")
            print(f"  Quality: {best.quality.value}")
            print(f"  Elo Rating: {best.elo_rating:.0f}")
            print(f"  Win Rate: {best.win_rate:.2%}")
            print(f"  Games Played: {best.games_played}")
            
            print(f"\nTotal Games: {len(self.game_history)}")
            print(f"Strategies Discovered: {len(self.strategy_history)}")
            print(f"Current Pool Size: {len(self.strategies)}")
            print(f"Final Curriculum Level: {self.curriculum_levels[self.current_level].name}")
    
    
    def demonstrate_self_play():
        """Demonstrate Self-Play & Self-Improvement pattern."""
        
        print("="*80)
        print("SELF-PLAY & SELF-IMPROVEMENT - DEMONSTRATION")
        print("="*80)
        
        # Test 1: Problem-solving self-play
        print("\n" + "="*80)
        print("TEST 1: Problem-Solving Self-Play (Short Training)")
        print("="*80)
        
        agent1 = SelfPlayAgent(
            task_type=TaskType.PROBLEM_SOLVING,
            initial_strategies=3
        )
        
        agent1.train(iterations=5, games_per_iteration=3)
        
        print("\n--- Strategy Evolution ---")
        for i, strategy in enumerate(agent1.strategy_history[:5], 1):
            print(f"{i}. {strategy.name}")
            print(f"   Quality: {strategy.quality.value}, Elo: {strategy.elo_rating:.0f}")
        
        # Test 2: Track improvement metrics
        print("\n" + "="*80)
        print("TEST 2: Improvement Metrics Analysis")
        print("="*80)
        
        print("\nIteration-by-Iteration Improvement:")
        print(f"{'Iter':<6} {'Avg Elo':<10} {'Best Quality':<15} {'Games':<8} {'Improvement':<12}")
        print("-" * 65)
        for metrics in agent1.improvement_history:
            print(f"{metrics.iteration:<6} {metrics.average_elo:<10.0f} "
                  f"{metrics.best_strategy_quality.value:<15} {metrics.total_games:+10.2f}%")
        
        # Test 3: Strategy comparison
        print("\n" + "="*80)
        print("TEST 3: Final Strategy Rankings")
        print("="*80)
        
        ranked_strategies = sorted(agent1.strategies.values(), key=lambda s: s.elo_rating, reverse=True)
        
        print(f"\n{'Rank':<6} {'Strategy':<25} {'Elo':<8} {'Win Rate':<10} {'Games':<8}")
        print("-" * 65)
        for i, strategy in enumerate(ranked_strategies, 1):
            print(f"{i:<6} {strategy.name:<25} {strategy.elo_rating:<8.0f} "
                  f"{strategy.win_rate:<10.1%} {strategy.games_played:<8}")
        
        # Test 4: Game insights
        print("\n" + "="*80)
        print("TEST 4: Notable Games and Insights")
        print("="*80)
        
        print("\nSample Games:")
        for game in agent1.game_history[-3:]:
            print(f"\n{game.game_id}: {game.strategy_a.name} vs {game.strategy_b.name}")
            winner_name = "Draw"
            if game.winner == game.strategy_a.strategy_id:
                winner_name = game.strategy_a.name
            elif game.winner == game.strategy_b.strategy_id:
                winner_name = game.strategy_b.name
            print(f"  Winner: {winner_name}")
            print(f"  Scores: {game.score_a:.2f} - {game.score_b:.2f}")
            if game.insights:
                print(f"  Insights: {', '.join(game.insights[:2])}")
        
        # Test 5: Different task type
        print("\n" + "="*80)
        print("TEST 5: Creative Task Self-Play")
        print("="*80)
        
        agent2 = SelfPlayAgent(
            task_type=TaskType.CREATIVE,
            initial_strategies=2
        )
        
        print("\nRunning creative task training...")
        agent2.train(iterations=3, games_per_iteration=2)
        
        print("\n--- Creative Strategies Evolved ---")
        best_creative = max(agent2.strategies.values(), key=lambda s: s.elo_rating)
        print(f"Best Strategy: {best_creative.name}")
        print(f"Description: {best_creative.description[:200]}...")
        
        # Summary
        print("\n" + "="*80)
        print("SUMMARY: Self-Play & Self-Improvement")
        print("="*80)
        print("""
    The Self-Play & Self-Improvement pattern demonstrates autonomous learning:
    
    Key Features Demonstrated:
    1. Strategy Generation: Creates new strategies from successful parents
    2. Self-Play Engine: Simulates competitions between strategies
    3. Performance Evaluation: Uses Elo ratings to track strategy strength
    4. Curriculum Learning: Progressively increases task difficulty
    5. Strategy Evolution: Generates improved strategies over time
    6. Strategy Pruning: Removes weak strategies to maintain quality pool
    7. Improvement Tracking: Monitors learning progress with metrics
    
    Benefits:
    • No labeled training data required
    • Discovers novel strategies through exploration
    • Continuously improves without human supervision
    • Adapts to changing task requirements
    • Scales with computational resources
    • Explores diverse solution spaces
    • Learns from billions of self-generated examples
    
    Use Cases:
    • Game AI (chess, Go, poker, StarCraft)
    • Negotiation and bargaining agents
    • Creative content generation (writing, art, music)
    • Code optimization and refactoring
    • Mathematical theorem proving
    • Strategic planning and decision-making
    • Competitive debate and argumentation
    • Resource allocation and scheduling
    
    Comparison with Supervised Learning:
    ┌─────────────────────┬────────────────────┬─────────────────────┐
    │ Aspect              │ Supervised         │ Self-Play           │
    ├─────────────────────┼────────────────────┼─────────────────────┤
    │ Training Data       │ Labeled examples   │ Self-generated      │
    │ Data Requirements   │ Large labeled sets │ Minimal initial     │
    │ Novel Strategies    │ Limited            │ Unlimited discovery │
    │ Adaptation          │ Retraining needed  │ Continuous          │
    │ Human Expertise     │ Required           │ Not required        │
    │ Scaling             │ Data-limited       │ Compute-limited     │
    │ Performance Ceiling │ Human-level        │ Can exceed humans   │
    └─────────────────────┴────────────────────┴─────────────────────┘
    
    Real-World Successes:
    • AlphaGo: Defeated world champion through self-play
    • AlphaZero: Mastered chess, shogi, and Go with no human data
    • OpenAI Five: Achieved professional-level Dota 2 play
    • GPT models: Improved through self-generated reasoning
    • AlphaCode: Competitive programming through self-play
    
    LangChain Implementation Notes:
    • Multiple specialized LLMs for different roles
    • Strategy generation through evolutionary approaches
    • Game simulation using LLM as game master
    • Elo rating system for objective performance measurement
    • Curriculum learning with progressive difficulty
    • Strategy pool management with pruning
    • Comprehensive metrics tracking
    
    Production Considerations:
    • Implement efficient game/task simulation
    • Add checkpointing for long training runs
    • Implement distributed self-play across multiple workers
    • Add exploration bonuses to encourage diversity
    • Implement opponent sampling strategies (uniform, skill-based)
    • Add meta-learning to improve learning efficiency
    • Implement proper evaluation against external benchmarks
    • Add human evaluation for strategy interpretability
    • Monitor for strategy collapse or degenerate solutions
    • Implement safety checks for deployed strategies
    • Add transfer learning from related tasks
    • Implement strategy distillation for deployment efficiency
    
    Advanced Extensions:
    • Multi-agent self-play with teams
    • Hierarchical strategies with sub-strategies
    • Transfer learning across different tasks
    • Meta-self-play (learning how to self-play better)
    • Curiosity-driven exploration bonuses
    • Population-based training with diversity metrics
    • Adversarial self-play for robustness
    • Self-play with human-in-the-loop feedback
        """)
    
    
    if __name__ == "__main__":
        demonstrate_self_play()
    
        

    Agentic Design Pattern: Self Play

    Pattern 069: Self Play – Line-by-Line Explanation

    Overview

    This document provides a detailed line-by-line explanation of the Self Play pattern implementation.


    Pattern Description

    Pattern 069: Self-Play & Self-Improvement

    Description: Self-Play & Self-Improvement enables agents to autonomously improve their capabilities by competing or collaborating with themselves. Inspired by techniques like AlphaGo’s self-play training, this pattern allows agents to generate their own training data, discover new strategies, and continuously refine their performance without external supervision. The agent plays both sides of a game, problem, or scenario, learning from each iteration to progressively improve.

    This pattern is particularly powerful for domains where supervised training data is
    scarce, expensive, or impossible to obtain. Through self-play, agents can explore
    vast strategy spaces, discover novel solutions, and achieve superhuman performance
    by learning from billions of self-generated experiences.
    

    Components: 1. Strategy Generator: Creates candidate strategies/policies 2. Self-Play Engine: Simulates interactions with itself 3. Performance Evaluator: Measures strategy effectiveness 4. Curriculum Generator: Creates progressively harder challenges 5. Strategy Selector: Chooses best strategies to keep 6. Meta-Learner: Learns how to improve the learning process 7. Checkpoint Manager: Stores and manages strategy versions

    Architecture: Initial Strategy ↓ ┌─────────────────────────┐ │ Self-Play Loop │ │ │ │ Strategy A vs Strategy B│ │ ↓ │ │ Play Game/Task │ │ ↓ │ │ Evaluate Performance │ │ ↓ │ │ Update Strategies │ │ ↓ │ │ Generate Curriculum │ │ ↓ │ │ Select Best │ └─────────────────────────┘ ↓ (iterate) Improved Strategy

    Use Cases: – Game AI development (chess, Go, poker) – Negotiation and bargaining agents – Creative content generation – Code optimization and debugging – Theorem proving and mathematical reasoning – Strategic planning and decision-making

    Advantages: – No need for labeled training data – Discovers novel strategies humans haven’t found – Continuously improves without external input – Adapts to changing environments – Scales with compute rather than data – Explores diverse solution spaces

    LangChain Implementation: Uses ChatOpenAI for strategy generation and evaluation. Demonstrates self-play loops, performance tracking, curriculum generation, and iterative improvement through competition.


    File Statistics

    • Total Lines: 817
    • Classes: 8
    • Functions: 1
    • Constants: 0

    Module Documentation

    """
    Pattern 069: Self-Play & Self-Improvement
    
    Description:
        Self-Play & Self-Improvement enables agents to autonomously improve their capabilities
        by competing or collaborating with themselves. Inspired by techniques like AlphaGo's
        self-play training, this pattern allows agents to generate their own training data,
        discover new strategies, and continuously refine their performance without external
        supervision. The agent plays both sides of a game, problem, or scenario, learning
        from each iteration to progressively improve.
    
        This pattern is particularly powerful for domains where supervised training data is
        scarce, expensive, or impossible to obtain. Through self-play, agents can explore
        vast strategy spaces, discover novel solutions, and achieve superhuman performance
        by learning from billions of self-generated experiences.
    
    Components:
        1. Strategy Generator: Creates candidate strategies/policies
        2. Self-Play Engine: Simulates interactions with itself
        3. Performance Evaluator: Measures strategy effectiveness
        4. Curriculum Generator: Creates progressively harder challenges
        5. Strategy Selector: Chooses best strategies to keep
        6. Meta-Learner: Learns how to improve the learning process
        7. Checkpoint Manager: Stores and manages strategy versions
    
    Architecture:
        ```
        Initial Strategy
            ↓
        ┌─────────────────────────┐
        │   Self-Play Loop        │
        │                         │
        │  Strategy A vs Strategy B│
        │         ↓               │
        │    Play Game/Task       │
        │         ↓               │
        │   Evaluate Performance  │
        │         ↓               │
        │   Update Strategies     │
        │         ↓               │
        │  Generate Curriculum    │
        │         ↓               │
        │   Select Best           │
        └─────────────────────────┘
             ↓ (iterate)
        Improved Strategy
        ```
    
    Use Cases:
        - Game AI development (chess, Go, poker)
        - Negotiation and bargaining agents
        - Creative content generation
        - Code optimization and debugging
        - Theorem proving and mathematical reasoning
        - Strategic planning and decision-making
    
    Advantages:
        - No need for labeled training data
        - Discovers novel strategies humans haven't found
        - Continuously improves without external input
        - Adapts to changing environments
        - Scales with compute rather than data
        - Explores diverse solution spaces
    
    LangChain Implementation:
        Uses ChatOpenAI for strategy generation and evaluation. Demonstrates
        self-play loops, performance tracking, curriculum generation, and
        iterative improvement through competition.
    """
    

    Purpose: This module implements the Self Play pattern, providing a structured approach to agent design and implementation.


    Import Statements

    Standard Library Imports

    import collections.deque
    import dataclasses.dataclass
    import dataclasses.field
    import enum.Enum
    import langchain_core.output_parsers.StrOutputParser
    import langchain_core.prompts.ChatPromptTemplate
    import os
    import random
    import time
    import typing.Any
    import typing.Dict
    import typing.List
    import typing.Optional
    import typing.Tuple
    

    Purpose: Core Python functionality: – collections.deque: Specialized container datatypes – dataclasses.dataclass: Simplified class definitions with @dataclass – dataclasses.field: Simplified class definitions with @dataclass – enum.Enum: Enumeration support – langchain_core.output_parsers.StrOutputParser: Regular expression operations – langchain_core.prompts.ChatPromptTemplate: Regular expression operations – os: Environment variable access and system operations – random: Random number generation – time: Time-related functions – typing.Any: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Dict: Type hints for better code documentation (List, Dict, Any, Optional) – typing.List: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Optional: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Tuple: Type hints for better code documentation (List, Dict, Any, Optional)

    LangChain Imports

    from langchain_openai.ChatOpenAI
    

    Purpose: LangChain framework components: – LLM Interface: Wrapper for OpenAI models – Chains: Composable building blocks for complex workflows

    Other Imports

    from dotenv.load_dotenv
    

    Purpose: Additional dependencies for specialized functionality.


    Classes

    Class: TaskType

    Lines 87-93

    Inherits from: Enum

    Purpose:

    Types of self-play tasks.
    

    Class: StrategyQuality

    Lines 96-102

    Inherits from: Enum

    Purpose:

    Quality levels of strategies.
    

    Class: Strategy

    Lines 106-116

    Purpose:

    Represents a strategy/policy.
    

    Class: GameResult

    Lines 120-129

    Purpose:

    Result of a self-play game.
    

    Class: TrainingExample

    Lines 133-140

    Purpose:

    Training example generated from self-play.
    

    Class: CurriculumLevel

    Lines 144-150

    Purpose:

    Difficulty level in curriculum.
    

    Class: ImprovementMetrics

    Lines 154-161

    Purpose:

    Metrics tracking improvement over time.
    

    Class: SelfPlayAgent

    Lines 164-636

    Purpose:

    Agent that improves through self-play and self-generated curriculum.
    

    Methods:

    __init__(self, task_type, initial_strategies, elo_k_factor)

    Lines 169-212

    Purpose:

    Initialize self-play agent.
    
    Args:
        task_type: Type of task for self-play
        initial_strategies: Number of initial strategies to generate
        elo_k_factor: K-factor for Elo rating updates
    

    Parameters: – task_type: Task type – initial_strategies: Initial strategies – elo_k_factor: Elo k factor

    _initialize_strategies(self, count)

    Lines 214-227

    Purpose:

    Initialize starting strategies.
    

    Parameters: – count: Count

    _initialize_curriculum(self)

    Lines 229-237

    Purpose:

    Initialize learning curriculum.
    

    Parameters:

    generate_new_strategy(self, parent_strategies)

    Lines 239-329

    Purpose:

    Generate new strategy by learning from existing ones.
    
    Args:
        parent_strategies: Strategies to learn from
    
    Returns:
        New improved strategy
    

    Parameters: – parent_strategies: Parent strategies

    Returns: Strategy

    play_game(self, strategy_a, strategy_b, scenario)

    Lines 331-443

    Purpose:

    Simulate a game between two strategies.
    
    Args:
        strategy_a: First strategy
        strategy_b: Second strategy
        scenario: Game scenario/problem
    
    Returns:
        Game result with winner and insights
    

    Parameters: – strategy_a: Strategy a – strategy_b: Strategy b – scenario: Scenario

    Returns: GameResult

    update_elo_ratings(self, game_result)

    Lines 445-479

    Purpose:

    Update Elo ratings based on game result.
    
    Args:
        game_result: Result of the game
    

    Parameters: – game_result: Game result

    generate_curriculum_scenario(self, level)

    Lines 481-498

    Purpose:

    Generate training scenario for current curriculum level.
    
    Args:
        level: Curriculum level
    
    Returns:
        Scenario description
    

    Parameters: – level: Level

    Returns: str

    self_play_iteration(self, num_games)

    Lines 500-601

    Purpose:

    Run one iteration of self-play training.
    
    Args:
        num_games: Number of games to play this iteration
    
    Returns:
        Improvement metrics for this iteration
    

    Parameters: – num_games: Num games

    Returns: ImprovementMetrics

    train(self, iterations, games_per_iteration)

    Lines 603-636

    Purpose:

    Run full self-play training.
    
    Args:
        iterations: Number of training iterations
        games_per_iteration: Games to play per iteration
    

    Parameters: – iterations: Iterations – games_per_iteration: Games per iteration


    Functions

    demonstrate_self_play()

    Lines 639-812

    Purpose:

    Demonstrate Self-Play & Self-Improvement pattern.
    

    Key Concepts Summary

    Pattern Overview

    The Self Play pattern provides a structured approach to building AI agents with specific capabilities and behaviors. This implementation leverages LangChain’s composable components to create a robust and maintainable solution.

    Core Components

    1. Agent Class: Main implementation of the pattern logic
    2. Prompt Engineering: Carefully crafted prompts to guide LLM behavior
    3. Tool Integration: Optional tools for extended functionality
    4. Memory Management: State handling across interactions
    5. Output Parsing: Structured response processing

    When to Use

    ✅ Good for: – Scenarios requiring this specific pattern’s strengths – Applications needing structured agent behavior – Systems where explainability is important – Production environments requiring reliability

    ❌ Not ideal for: – Simple single-prompt tasks – Scenarios requiring different pattern characteristics – Resource-constrained environments (if computationally intensive)


    Best Practices Demonstrated

    1. Environment Configuration

    • ✅ Use .env files for API keys
    • ✅ Never hardcode credentials
    • ✅ Validate environment variables on startup

    2. Error Handling

    • ✅ Graceful degradation on failures
    • ✅ Informative error messages
    • ✅ Logging for debugging

    3. Prompt Engineering

    • ✅ Clear, specific instructions
    • ✅ Structured output formats
    • ✅ Few-shot examples when beneficial

    4. Code Organization

    • ✅ Separation of concerns (class-based design)
    • ✅ Type hints for clarity
    • ✅ Docstrings for documentation

    5. Performance Optimization

    • ✅ Appropriate temperature settings
    • ✅ Token budget management
    • ✅ Caching where applicable

    LangChain Components Used

    • ChatOpenAI: OpenAI model interface
    • Chains: Sequential component composition

    Why LangChain? – Composable abstractions for LLM applications – Rich ecosystem of integrations – Production-ready patterns – Active community and development


    Execution Flow

    1. Initialize agent with configuration
       ↓
    2. Load environment and dependencies
       ↓
    3. Prepare prompt with input data
       ↓
    4. Invoke LLM through LangChain
       ↓
    5. Parse and validate response
       ↓
    6. Return formatted result
    

    Key Decision Points: – Model selection (temperature, model type) – Prompt structure (few-shot vs zero-shot) – Error handling strategy – Output format validation


    Performance Considerations

    Computational Cost

    • LLM Calls: Primary cost factor (based on tokens)
    • Latency: Depends on model and input size
    • Memory: Minimal for most patterns

    Optimization Strategies

    1. Prompt Optimization: Minimize tokens while maintaining clarity
    2. Caching: Cache repeated queries/responses
    3. Model Selection: Use smaller models when appropriate
    4. Batch Processing: Group similar requests
    5. Streaming: For long responses, stream tokens

    Cost Estimates (Approximate)

    • GPT-3.5-turbo: ~$0.002 per request
    • GPT-4: ~$0.03 per request
    • GPT-4-turbo: ~$0.01 per request

    Actual costs vary based on input/output token counts


    Common Pitfalls to Avoid

    1. Prompt Design Issues

    ❌ Vague or ambiguous instructions
    ✅ Clear, specific, testable prompts

    2. Error Handling

    ❌ Ignoring API errors or rate limits
    ✅ Implement retry logic and graceful degradation

    3. Token Management

    ❌ Exceeding model token limits
    ✅ Truncate inputs, chunk processing

    4. Security

    ❌ Exposing API keys in code
    ✅ Use environment variables

    5. Testing

    ❌ Only testing happy paths
    ✅ Test edge cases, failures, malformed inputs

    6. Cost Management

    ❌ Unmonitored API usage
    ✅ Implement budgets, alerts, and monitoring


    Further Improvements

    Production Enhancements

    1. Monitoring: Add logging, metrics, tracing
    2. Validation: Input/output schema validation
    3. Rate Limiting: Implement request throttling
    4. Caching: Redis or in-memory cache for responses
    5. Async Support: Non-blocking operations for scalability

    Feature Extensions

    1. Multi-Model Support: Switch between providers
    2. Fallback Models: Graceful degradation strategy
    3. Custom Tools: Domain-specific function calling
    4. Advanced Memory: Vector stores for long-term memory
    5. Evaluation: Automated testing and benchmarking

    Code Quality

    1. Testing: Unit tests, integration tests
    2. Documentation: API docs, usage examples
    3. Type Safety: Stricter type hints
    4. Configuration: Externalized config files

    Related Patterns

    • Pattern 068
    • Pattern 070

    Pattern Families: – Reasoning Patterns (001-011) – Multi-Agent Patterns (012-019) – Tool Use Patterns (020-025) – Memory Patterns (026-031) – And many more…


    Summary

    The Self Play pattern provides a powerful approach to building AI agents with specific capabilities. This implementation demonstrates:

    ✅ Key Strengths: – Clean, maintainable code structure – LangChain integration for composability – Type hints and documentation – Production-ready patterns

    🎯 Best Use Cases: – Applications requiring this pattern’s characteristics – Systems needing structured agent behavior – Production environments requiring reliability

    📚 Learning Resources: – LangChain Documentation: https://python.langchain.com/ – Pattern Documentation: See related patterns – OpenAI API Reference: https://platform.openai.com/docs/


    File: 069_self_play.py
    Total Lines: 817
    Pattern Category: AI Agent Design Pattern #069


    Generated from actual implementation code
    For educational and reference purposes

  • Agentic Design Pattern: Instruction Following

    
        """
    Pattern 068: Instruction Following & Grounding
    
    Description:
        Instruction Following & Grounding enables agents to precisely understand and execute
        natural language instructions by breaking them down into structured, executable steps.
        This pattern goes beyond simple command parsing to deeply ground instructions in the
        agent's capabilities, environment constraints, and task context. The agent validates
        instructions for feasibility, disambiguates ambiguous commands, grounds abstract
        concepts in concrete actions, and provides execution verification.
        
        The pattern is crucial for building reliable agents that can follow complex,
        multi-step instructions while maintaining safety, handling edge cases, and providing
        transparent feedback about what it's doing and why.
    
    Components:
        1. Instruction Parser: Extracts intent, parameters, and constraints
        2. Action Grounder: Maps abstract instructions to concrete actions
        3. Feasibility Checker: Validates instruction executability
        4. Disambiguator: Resolves ambiguities through clarification
        5. Execution Planner: Creates step-by-step execution plan
        6. Execution Monitor: Tracks execution progress and validates outcomes
        7. Feedback Generator: Provides status updates and explanations
    
    Architecture:
        ```
        Natural Language Instruction
            ↓
        Instruction Parser → [Intent, Parameters, Constraints]
            ↓
        Disambiguator → [Clarified Instruction]
            ↓
        Action Grounder → [Concrete Actions]
            ↓
        Feasibility Checker → [Validation]
            ↓
        Execution Planner → [Execution Steps]
            ↓
        Execution Monitor → [Execute & Verify]
            ↓
        Feedback Generator → [Status & Results]
        ```
    
    Use Cases:
        - Robotic task execution from natural language
        - Virtual assistant command processing
        - Smart home automation
        - Software automation and scripting
        - Manufacturing and logistics instructions
        - Educational tutoring systems
    
    Advantages:
        - Handles ambiguous and underspecified instructions
        - Grounds abstract concepts in concrete actions
        - Validates feasibility before execution
        - Provides transparent execution process
        - Handles errors and edge cases gracefully
        - Learns from execution feedback
    
    LangChain Implementation:
        Uses ChatOpenAI for instruction parsing, grounding, and planning.
        Demonstrates instruction decomposition, parameter extraction, feasibility
        checking, execution planning, and verification.
    """
    
    import os
    from typing import List, Dict, Any, Optional, Tuple
    from dataclasses import dataclass, field
    from enum import Enum
    import time
    from dotenv import load_dotenv
    
    from langchain_core.prompts import ChatPromptTemplate
    from langchain_core.output_parsers import StrOutputParser
    from langchain_openai import ChatOpenAI
    
    load_dotenv()
    
    
    class InstructionType(Enum):
        """Types of instructions."""
        ACTION = "action"  # Do something
        QUERY = "query"  # Find information
        CONDITION = "condition"  # If-then logic
        SEQUENCE = "sequence"  # Series of actions
        LOOP = "loop"  # Repeated actions
        CONSTRAINT = "constraint"  # With specific constraints
    
    
    class ActionStatus(Enum):
        """Status of action execution."""
        PENDING = "pending"
        IN_PROGRESS = "in_progress"
        COMPLETED = "completed"
        FAILED = "failed"
        BLOCKED = "blocked"
    
    
    class FeasibilityLevel(Enum):
        """Feasibility assessment levels."""
        FEASIBLE = "feasible"
        FEASIBLE_WITH_ASSUMPTIONS = "feasible_with_assumptions"
        PARTIALLY_FEASIBLE = "partially_feasible"
        INFEASIBLE = "infeasible"
        NEEDS_CLARIFICATION = "needs_clarification"
    
    
    @dataclass
    class Parameter:
        """Represents a parameter extracted from instruction."""
        name: str
        value: Any
        data_type: str
        is_optional: bool = False
        default_value: Any = None
    
    
    @dataclass
    class Constraint:
        """Represents a constraint on instruction execution."""
        constraint_type: str  # e.g., "time", "resource", "condition"
        description: str
        is_hard: bool = True  # Hard vs soft constraint
    
    
    @dataclass
    class ParsedInstruction:
        """Represents a parsed natural language instruction."""
        instruction_id: str
        original_text: str
        instruction_type: InstructionType
        intent: str  # The main goal
        parameters: List[Parameter]
        constraints: List[Constraint]
        dependencies: List[str] = field(default_factory=list)  # Other instruction IDs
        ambiguities: List[str] = field(default_factory=list)
    
    
    @dataclass
    class GroundedAction:
        """Represents a grounded, executable action."""
        action_id: str
        action_name: str
        parameters: Dict[str, Any]
        preconditions: List[str]
        effects: List[str]
        estimated_duration: float  # seconds
        
        def execute(self) -> Tuple[bool, str]:
            """Simulate action execution."""
            # In real implementation, this would execute actual actions
            print(f"  Executing: {self.action_name}({', '.join(f'{k}={v}' for k, v in self.parameters.items())})")
            time.sleep(0.1)  # Simulate execution time
            return True, f"Successfully executed {self.action_name}"
    
    
    @dataclass
    class ExecutionStep:
        """Represents a step in execution plan."""
        step_id: int
        action: GroundedAction
        status: ActionStatus
        start_time: Optional[float] = None
        end_time: Optional[float] = None
        result: Optional[str] = None
        error: Optional[str] = None
    
    
    @dataclass
    class FeasibilityAssessment:
        """Assessment of instruction feasibility."""
        level: FeasibilityLevel
        is_feasible: bool
        reasons: List[str]
        assumptions: List[str] = field(default_factory=list)
        clarifications_needed: List[str] = field(default_factory=list)
        alternatives: List[str] = field(default_factory=list)
    
    
    @dataclass
    class ExecutionResult:
        """Complete result of instruction execution."""
        instruction: ParsedInstruction
        feasibility: FeasibilityAssessment
        execution_steps: List[ExecutionStep]
        success: bool
        summary: str
        execution_time: float
    
    
    class InstructionFollowingAgent:
        """
        Agent that parses, grounds, and executes natural language instructions.
        """
        
        def __init__(self, agent_capabilities: Optional[List[str]] = None):
            """
            Initialize the instruction following agent.
            
            Args:
                agent_capabilities: List of actions the agent can perform
            """
            # Define agent capabilities
            self.capabilities = agent_capabilities or [
                "move_to",
                "pick_up",
                "put_down",
                "open",
                "close",
                "search",
                "read",
                "write",
                "calculate",
                "wait",
                "notify"
            ]
            
            # LLMs for different tasks
            self.parser = ChatOpenAI(temperature=0.1, model="gpt-3.5-turbo")
            self.grounder = ChatOpenAI(temperature=0.2, model="gpt-3.5-turbo")
            self.planner = ChatOpenAI(temperature=0.2, model="gpt-3.5-turbo")
            self.verifier = ChatOpenAI(temperature=0.1, model="gpt-3.5-turbo")
            
            self.str_parser = StrOutputParser()
            
            # Execution history
            self.execution_history: List[ExecutionResult] = []
        
        def parse_instruction(self, instruction_text: str) -> ParsedInstruction:
            """
            Parse natural language instruction into structured format.
            
            Args:
                instruction_text: Natural language instruction
                
            Returns:
                Parsed instruction with intent, parameters, and constraints
            """
            prompt = ChatPromptTemplate.from_messages([
                ("system", """You are an instruction parser. Extract structured information from natural language.
    
    Analyze the instruction and identify:
    1. Instruction Type (action/query/condition/sequence/loop/constraint)
    2. Intent (what is the goal?)
    3. Parameters (name, value, type, optional?)
    4. Constraints (time, resource, conditional constraints)
    5. Ambiguities (what is unclear or underspecified?)
    
    Format your response as:
    Type: [type]
    Intent: [main goal]
    Parameters: [name:value:type:optional, ...]
    Constraints: [type:description:hard/soft, ...]
    Ambiguities: [list of unclear aspects, or 'none']"""),
                ("user", "Parse this instruction: {instruction}")
            ])
            
            chain = prompt | self.parser | self.str_parser
            
            try:
                result = chain.invoke({"instruction": instruction_text})
                
                # Parse the structured output
                instruction_type = InstructionType.ACTION
                intent = ""
                parameters = []
                constraints = []
                ambiguities = []
                
                for line in result.split('\n'):
                    line = line.strip()
                    if line.startswith("Type:"):
                        type_str = line.replace("Type:", "").strip().lower()
                        for itype in InstructionType:
                            if itype.value in type_str:
                                instruction_type = itype
                                break
                    
                    elif line.startswith("Intent:"):
                        intent = line.replace("Intent:", "").strip()
                    
                    elif line.startswith("Parameters:"):
                        params_str = line.replace("Parameters:", "").strip()
                        if params_str.lower() != 'none':
                            # Parse format: name:value:type:optional
                            for param_str in params_str.split(','):
                                parts = [p.strip() for p in param_str.split(':')]
                                if len(parts) >= 3:
                                    param = Parameter(
                                        name=parts[0],
                                        value=parts[1],
                                        data_type=parts[2],
                                        is_optional=len(parts) > 3 and parts[3].lower() == 'optional'
                                    )
                                    parameters.append(param)
                    
                    elif line.startswith("Constraints:"):
                        const_str = line.replace("Constraints:", "").strip()
                        if const_str.lower() != 'none':
                            # Parse format: type:description:hard/soft
                            for c_str in const_str.split(','):
                                parts = [p.strip() for p in c_str.split(':')]
                                if len(parts) >= 2:
                                    constraint = Constraint(
                                        constraint_type=parts[0],
                                        description=parts[1] if len(parts) > 1 else "",
                                        is_hard=len(parts)  List[GroundedAction]:
            """
            Ground abstract instruction into concrete, executable actions.
            
            Args:
                parsed: Parsed instruction
                
            Returns:
                List of grounded actions
            """
            prompt = ChatPromptTemplate.from_messages([
                ("system", """You are an action grounder. Map abstract instructions to concrete actions.
    
    Agent Capabilities: {capabilities}
    
    For each instruction, specify:
    1. Action sequence (using available capabilities)
    2. Parameters for each action
    3. Preconditions (what must be true before)
    4. Effects (what changes after)
    5. Estimated duration (seconds)
    
    Format each action as:
    Action: [action_name]
    Parameters: [key1=value1, key2=value2, ...]
    Preconditions: [list]
    Effects: [list]
    Duration: [seconds]
    ---"""),
                ("user", """Ground this instruction:
    Intent: {intent}
    Type: {instruction_type}
    Parameters: {parameters}
    Constraints: {constraints}
    
    Provide concrete actions:""")
            ])
            
            chain = prompt | self.grounder | self.str_parser
            
            try:
                params_str = ", ".join([f"{p.name}={p.value}" for p in parsed.parameters])
                const_str = ", ".join([f"{c.constraint_type}: {c.description}" for c in parsed.constraints])
                
                result = chain.invoke({
                    "capabilities": ", ".join(self.capabilities),
                    "intent": parsed.intent,
                    "instruction_type": parsed.instruction_type.value,
                    "parameters": params_str if params_str else "none",
                    "constraints": const_str if const_str else "none"
                })
                
                # Parse grounded actions
                actions = []
                action_blocks = result.split('---')
                
                for i, block in enumerate(action_blocks):
                    if not block.strip():
                        continue
                    
                    action_name = ""
                    parameters = {}
                    preconditions = []
                    effects = []
                    duration = 1.0
                    
                    for line in block.split('\n'):
                        line = line.strip()
                        if line.startswith("Action:"):
                            action_name = line.replace("Action:", "").strip()
                        elif line.startswith("Parameters:"):
                            param_str = line.replace("Parameters:", "").strip()
                            if param_str.lower() != 'none':
                                for kv in param_str.split(','):
                                    if '=' in kv:
                                        k, v = kv.split('=', 1)
                                        parameters[k.strip()] = v.strip()
                        elif line.startswith("Preconditions:"):
                            pre_str = line.replace("Preconditions:", "").strip()
                            if pre_str.lower() != 'none':
                                preconditions = [p.strip() for p in pre_str.split(',')]
                        elif line.startswith("Effects:"):
                            eff_str = line.replace("Effects:", "").strip()
                            if eff_str.lower() != 'none':
                                effects = [e.strip() for e in eff_str.split(',')]
                        elif line.startswith("Duration:"):
                            try:
                                duration = float(line.replace("Duration:", "").strip())
                            except:
                                duration = 1.0
                    
                    if action_name:
                        action = GroundedAction(
                            action_id=f"action_{i+1}",
                            action_name=action_name,
                            parameters=parameters,
                            preconditions=preconditions,
                            effects=effects,
                            estimated_duration=duration
                        )
                        actions.append(action)
                
                return actions if actions else [
                    GroundedAction(
                        action_id="action_1",
                        action_name="execute_instruction",
                        parameters={"instruction": parsed.intent},
                        preconditions=[],
                        effects=["instruction executed"],
                        estimated_duration=1.0
                    )
                ]
                
            except Exception as e:
                print(f"Error grounding instruction: {e}")
                # Fallback: single generic action
                return [
                    GroundedAction(
                        action_id="action_1",
                        action_name="execute",
                        parameters={"instruction": parsed.intent},
                        preconditions=[],
                        effects=["completed"],
                        estimated_duration=1.0
                    )
                ]
        
        def check_feasibility(
            self,
            parsed: ParsedInstruction,
            grounded_actions: List[GroundedAction]
        ) -> FeasibilityAssessment:
            """
            Check if instruction is feasible to execute.
            
            Args:
                parsed: Parsed instruction
                grounded_actions: Grounded actions
                
            Returns:
                Feasibility assessment
            """
            prompt = ChatPromptTemplate.from_messages([
                ("system", """You are a feasibility checker. Assess if instructions can be executed.
    
    Agent Capabilities: {capabilities}
    
    Consider:
    1. Are all required actions within agent capabilities?
    2. Are parameters complete and valid?
    3. Are constraints satisfiable?
    4. Are preconditions likely to be met?
    5. Are there any safety concerns?
    
    Provide:
    Feasibility: [feasible/feasible_with_assumptions/partially_feasible/infeasible/needs_clarification]
    Reasons: [list of reasons for assessment]
    Assumptions: [assumptions made, if any]
    Clarifications: [what needs clarification, if any]
    Alternatives: [alternative approaches, if any]"""),
                ("user", """Assess feasibility:
    Intent: {intent}
    Actions: {actions}
    Constraints: {constraints}
    Ambiguities: {ambiguities}
    
    Provide assessment:""")
            ])
            
            chain = prompt | self.verifier | self.str_parser
            
            try:
                actions_str = ", ".join([a.action_name for a in grounded_actions])
                const_str = ", ".join([c.description for c in parsed.constraints])
                amb_str = ", ".join(parsed.ambiguities) if parsed.ambiguities else "none"
                
                result = chain.invoke({
                    "capabilities": ", ".join(self.capabilities),
                    "intent": parsed.intent,
                    "actions": actions_str,
                    "constraints": const_str if const_str else "none",
                    "ambiguities": amb_str
                })
                
                # Parse assessment
                level = FeasibilityLevel.FEASIBLE
                reasons = []
                assumptions = []
                clarifications = []
                alternatives = []
                
                for line in result.split('\n'):
                    line = line.strip()
                    if line.startswith("Feasibility:"):
                        level_str = line.replace("Feasibility:", "").strip().lower()
                        for flevel in FeasibilityLevel:
                            if flevel.value in level_str:
                                level = flevel
                                break
                    elif line.startswith("Reasons:"):
                        reasons_str = line.replace("Reasons:", "").strip()
                        if reasons_str.lower() != 'none':
                            reasons = [r.strip() for r in reasons_str.split(',')]
                    elif line.startswith("Assumptions:"):
                        ass_str = line.replace("Assumptions:", "").strip()
                        if ass_str.lower() != 'none':
                            assumptions = [a.strip() for a in ass_str.split(',')]
                    elif line.startswith("Clarifications:"):
                        clar_str = line.replace("Clarifications:", "").strip()
                        if clar_str.lower() != 'none':
                            clarifications = [c.strip() for c in clar_str.split(',')]
                    elif line.startswith("Alternatives:"):
                        alt_str = line.replace("Alternatives:", "").strip()
                        if alt_str.lower() != 'none':
                            alternatives = [a.strip() for a in alt_str.split(',')]
                
                is_feasible = level in [FeasibilityLevel.FEASIBLE, FeasibilityLevel.FEASIBLE_WITH_ASSUMPTIONS]
                
                return FeasibilityAssessment(
                    level=level,
                    is_feasible=is_feasible,
                    reasons=reasons if reasons else ["Assessment completed"],
                    assumptions=assumptions,
                    clarifications_needed=clarifications,
                    alternatives=alternatives
                )
                
            except Exception as e:
                print(f"Error checking feasibility: {e}")
                # Fallback: assume feasible
                return FeasibilityAssessment(
                    level=FeasibilityLevel.FEASIBLE,
                    is_feasible=True,
                    reasons=["Default feasibility check"]
                )
        
        def execute_instruction(
            self,
            instruction_text: str,
            dry_run: bool = False
        ) -> ExecutionResult:
            """
            Execute a natural language instruction end-to-end.
            
            Args:
                instruction_text: Natural language instruction
                dry_run: If True, plan but don't execute
                
            Returns:
                Complete execution result
            """
            start_time = time.time()
            
            print(f"\n{'='*60}")
            print(f"Instruction: {instruction_text}")
            print(f"{'='*60}\n")
            
            # Step 1: Parse
            print("Step 1: Parsing instruction...")
            parsed = self.parse_instruction(instruction_text)
            print(f"  → Intent: {parsed.intent}")
            print(f"  → Type: {parsed.instruction_type.value}")
            print(f"  → Parameters: {len(parsed.parameters)}")
            if parsed.ambiguities:
                print(f"  → Ambiguities: {', '.join(parsed.ambiguities)}")
            
            # Step 2: Ground
            print("\nStep 2: Grounding to concrete actions...")
            grounded_actions = self.ground_instruction(parsed)
            print(f"  → Generated {len(grounded_actions)} actions:")
            for action in grounded_actions:
                print(f"    • {action.action_name}({', '.join(f'{k}={v}' for k, v in action.parameters.items())})")
            
            # Step 3: Check feasibility
            print("\nStep 3: Checking feasibility...")
            feasibility = self.check_feasibility(parsed, grounded_actions)
            print(f"  → Feasibility: {feasibility.level.value}")
            print(f"  → Is Feasible: {feasibility.is_feasible}")
            if feasibility.assumptions:
                print(f"  → Assumptions: {', '.join(feasibility.assumptions)}")
            if feasibility.clarifications_needed:
                print(f"  → Needs Clarification: {', '.join(feasibility.clarifications_needed)}")
            
            # Step 4: Execute (if feasible and not dry run)
            execution_steps = []
            success = True
            
            if feasibility.is_feasible and not dry_run:
                print("\nStep 4: Executing actions...")
                for i, action in enumerate(grounded_actions, 1):
                    step = ExecutionStep(
                        step_id=i,
                        action=action,
                        status=ActionStatus.IN_PROGRESS,
                        start_time=time.time()
                    )
                    
                    try:
                        exec_success, result = action.execute()
                        step.status = ActionStatus.COMPLETED if exec_success else ActionStatus.FAILED
                        step.result = result
                        step.end_time = time.time()
                        print(f"    ✓ Step {i} completed")
                    except Exception as e:
                        step.status = ActionStatus.FAILED
                        step.error = str(e)
                        step.end_time = time.time()
                        success = False
                        print(f"    ✗ Step {i} failed: {e}")
                    
                    execution_steps.append(step)
            else:
                print("\nStep 4: Execution skipped (dry run or infeasible)")
                for i, action in enumerate(grounded_actions, 1):
                    step = ExecutionStep(
                        step_id=i,
                        action=action,
                        status=ActionStatus.PENDING
                    )
                    execution_steps.append(step)
            
            # Generate summary
            execution_time = time.time() - start_time
            
            if dry_run:
                summary = f"Dry run completed. {len(grounded_actions)} actions planned."
            elif not feasibility.is_feasible:
                summary = f"Instruction not feasible: {feasibility.level.value}"
            elif success:
                summary = f"Successfully executed {len(execution_steps)} actions."
            else:
                failed_steps = sum(1 for s in execution_steps if s.status == ActionStatus.FAILED)
                summary = f"Execution completed with {failed_steps} failures out of {len(execution_steps)} steps."
            
            result = ExecutionResult(
                instruction=parsed,
                feasibility=feasibility,
                execution_steps=execution_steps,
                success=success,
                summary=summary,
                execution_time=execution_time
            )
            
            self.execution_history.append(result)
            
            return result
    
    
    def demonstrate_instruction_following():
        """Demonstrate Instruction Following & Grounding pattern."""
        
        print("="*80)
        print("INSTRUCTION FOLLOWING & GROUNDING - DEMONSTRATION")
        print("="*80)
        
        agent = InstructionFollowingAgent()
        
        # Test 1: Simple action instruction
        print("\n" + "="*80)
        print("TEST 1: Simple Action Instruction")
        print("="*80)
        
        result1 = agent.execute_instruction("Pick up the red box and put it on the table.")
        
        print("\n--- EXECUTION RESULT ---")
        print(f"Success: {result1.success}")
        print(f"Summary: {result1.summary}")
        print(f"Execution Time: {result1.execution_time:.2f}s")
        print(f"Actions Executed: {len(result1.execution_steps)}")
        
        # Test 2: Instruction with constraints
        print("\n" + "="*80)
        print("TEST 2: Instruction with Constraints")
        print("="*80)
        
        result2 = agent.execute_instruction(
            "Search for documents about machine learning, but only look in the research folder "
            "and complete within 5 seconds."
        )
        
        print("\n--- PARSED INSTRUCTION ---")
        print(f"Intent: {result2.instruction.intent}")
        print(f"Type: {result2.instruction.instruction_type.value}")
        print(f"Parameters: {[(p.name, p.value) for p in result2.instruction.parameters]}")
        print(f"Constraints: {[(c.constraint_type, c.description) for c in result2.instruction.constraints]}")
        
        print("\n--- GROUNDED ACTIONS ---")
        for step in result2.execution_steps:
            print(f"Step {step.step_id}: {step.action.action_name}")
            print(f"  Parameters: {step.action.parameters}")
            print(f"  Preconditions: {step.action.preconditions}")
            print(f"  Effects: {step.action.effects}")
            print(f"  Status: {step.status.value}")
        
        # Test 3: Ambiguous instruction (dry run)
        print("\n" + "="*80)
        print("TEST 3: Ambiguous Instruction (Dry Run)")
        print("="*80)
        
        result3 = agent.execute_instruction(
            "Move the thing over there.",
            dry_run=True
        )
        
        print("\n--- AMBIGUITY ANALYSIS ---")
        if result3.instruction.ambiguities:
            print(f"Identified Ambiguities:")
            for amb in result3.instruction.ambiguities:
                print(f"  • {amb}")
        
        print("\n--- FEASIBILITY ASSESSMENT ---")
        print(f"Level: {result3.feasibility.level.value}")
        print(f"Is Feasible: {result3.feasibility.is_feasible}")
        if result3.feasibility.clarifications_needed:
            print(f"Clarifications Needed:")
            for clar in result3.feasibility.clarifications_needed:
                print(f"  • {clar}")
        if result3.feasibility.alternatives:
            print(f"Suggested Alternatives:")
            for alt in result3.feasibility.alternatives:
                print(f"  • {alt}")
        
        # Test 4: Complex multi-step instruction
        print("\n" + "="*80)
        print("TEST 4: Complex Multi-Step Instruction")
        print("="*80)
        
        result4 = agent.execute_instruction(
            "Open the file named 'report.txt', read its contents, calculate the sum of all numbers, "
            "and write the result to 'summary.txt'."
        )
        
        print("\n--- EXECUTION PLAN ---")
        total_duration = sum(step.action.estimated_duration for step in result4.execution_steps)
        print(f"Total Steps: {len(result4.execution_steps)}")
        print(f"Estimated Duration: {total_duration:.1f}s")
        print("\nStep-by-Step Plan:")
        for step in result4.execution_steps:
            print(f"  {step.step_id}. {step.action.action_name}")
            print(f"     Duration: {step.action.estimated_duration}s")
            print(f"     Effects: {', '.join(step.action.effects)}")
        
        print("\n--- EXECUTION SUMMARY ---")
        print(f"Status: {'✓ Success' if result4.success else '✗ Failed'}")
        print(f"Summary: {result4.summary}")
        
        # Test 5: Conditional instruction
        print("\n" + "="*80)
        print("TEST 5: Conditional Instruction")
        print("="*80)
        
        result5 = agent.execute_instruction(
            "If the temperature is above 25 degrees, open the window, otherwise turn on the heater.",
            dry_run=True
        )
        
        print("\n--- INSTRUCTION ANALYSIS ---")
        print(f"Type: {result5.instruction.instruction_type.value}")
        print(f"Intent: {result5.instruction.intent}")
        print(f"\nGrounded Actions: {len(result5.execution_steps)}")
        for step in result5.execution_steps:
            print(f"  • {step.action.action_name}: {step.action.parameters}")
        
        # Summary
        print("\n" + "="*80)
        print("SUMMARY: Instruction Following & Grounding")
        print("="*80)
        print(f"""
    The Instruction Following & Grounding pattern demonstrates precise instruction execution:
    
    Key Features Demonstrated:
    1. Instruction Parsing: Extracts intent, parameters, and constraints from natural language
    2. Action Grounding: Maps abstract instructions to concrete executable actions
    3. Feasibility Checking: Validates executability before attempting execution
    4. Ambiguity Detection: Identifies unclear or underspecified aspects
    5. Constraint Handling: Respects time, resource, and conditional constraints
    6. Execution Planning: Creates step-by-step execution plans with preconditions
    7. Execution Monitoring: Tracks progress and validates outcomes
    8. Transparent Feedback: Provides clear status updates and explanations
    
    Benefits:
    • Handles complex, multi-step instructions reliably
    • Grounds abstract concepts in concrete actions
    • Validates feasibility before execution
    • Identifies and resolves ambiguities
    • Provides transparent execution process
    • Handles errors gracefully with clear feedback
    • Respects constraints and preconditions
    • Learns from execution history
    
    Use Cases:
    • Robotic task execution from natural language commands
    • Virtual assistant command processing
    • Smart home automation with complex rules
    • Software automation and scripting from descriptions
    • Manufacturing instructions with safety constraints
    • Educational tutoring with adaptive instruction following
    • Accessibility tools for users with disabilities
    
    Comparison with Simple Command Parsing:
    ┌───────────────────────┬──────────────────┬─────────────────────┐
    │ Aspect                │ Simple Parsing   │ Full Grounding      │
    ├───────────────────────┼──────────────────┼─────────────────────┤
    │ Ambiguity Handling    │ Minimal          │ Comprehensive       │
    │ Constraint Validation │ None             │ Full validation     │
    │ Feasibility Checking  │ None             │ Pre-execution       │
    │ Action Grounding      │ Direct mapping   │ Context-aware       │
    │ Error Handling        │ Basic            │ Graceful with plans │
    │ Transparency          │ Low              │ High                │
    │ Complexity Handling   │ Limited          │ Excellent           │
    └───────────────────────┴──────────────────┴─────────────────────┘
    
    LangChain Implementation Notes:
    • Multiple specialized LLMs for parsing, grounding, and verification
    • Structured output parsing for instruction components
    • Feasibility assessment before execution
    • Step-by-step execution with monitoring
    • Execution history for learning and debugging
    • Dry run mode for planning without execution
    • Clear error messages and alternative suggestions
    
    Production Considerations:
    • Implement actual action execution layer (not simulated)
    • Add safety checks and permission systems
    • Implement rollback mechanisms for failed operations
    • Add interactive clarification for ambiguous instructions
    • Implement learning from user corrections
    • Add execution visualization for transparency
    • Implement cost estimation for resource-intensive operations
    • Add monitoring and logging for audit trails
    • Implement timeout and resource limit enforcement
    • Add support for cancellation and pause/resume
    • Implement state persistence for long-running tasks
    • Add multi-user coordination for shared resources
    
    Advanced Extensions:
    • Natural language feedback and explanation generation
    • Learning user preferences and implicit constraints
    • Proactive error prevention through predictive analysis
    • Multi-modal instruction understanding (speech, gesture, vision)
    • Collaborative instruction refinement with humans
    • Transfer learning from demonstration
    • Uncertainty-aware execution with confidence intervals
    • Hierarchical instruction decomposition for complex tasks
        """)
    
    
    if __name__ == "__main__":
        demonstrate_instruction_following()
    
        

    Agentic Design Pattern: Instruction Following

    Pattern 068: Instruction Following – Line-by-Line Explanation

    Overview

    This document provides a detailed line-by-line explanation of the Instruction Following pattern implementation.


    Pattern Description

    Pattern 068: Instruction Following & Grounding

    Description: Instruction Following & Grounding enables agents to precisely understand and execute natural language instructions by breaking them down into structured, executable steps. This pattern goes beyond simple command parsing to deeply ground instructions in the agent’s capabilities, environment constraints, and task context. The agent validates instructions for feasibility, disambiguates ambiguous commands, grounds abstract concepts in concrete actions, and provides execution verification.

    The pattern is crucial for building reliable agents that can follow complex,
    multi-step instructions while maintaining safety, handling edge cases, and providing
    transparent feedback about what it's doing and why.
    

    Components: 1. Instruction Parser: Extracts intent, parameters, and constraints 2. Action Grounder: Maps abstract instructions to concrete actions 3. Feasibility Checker: Validates instruction executability 4. Disambiguator: Resolves ambiguities through clarification 5. Execution Planner: Creates step-by-step execution plan 6. Execution Monitor: Tracks execution progress and validates outcomes 7. Feedback Generator: Provides status updates and explanations

    Architecture: Natural Language Instruction ↓ Instruction Parser → [Intent, Parameters, Constraints] ↓ Disambiguator → [Clarified Instruction] ↓ Action Grounder → [Concrete Actions] ↓ Feasibility Checker → [Validation] ↓ Execution Planner → [Execution Steps] ↓ Execution Monitor → [Execute & Verify] ↓ Feedback Generator → [Status & Results]

    Use Cases: – Robotic task execution from natural language – Virtual assistant command processing – Smart home automation – Software automation and scripting – Manufacturing and logistics instructions – Educational tutoring systems

    Advantages: – Handles ambiguous and underspecified instructions – Grounds abstract concepts in concrete actions – Validates feasibility before execution – Provides transparent execution process – Handles errors and edge cases gracefully – Learns from execution feedback

    LangChain Implementation: Uses ChatOpenAI for instruction parsing, grounding, and planning. Demonstrates instruction decomposition, parameter extraction, feasibility checking, execution planning, and verification.


    File Statistics

    • Total Lines: 889
    • Classes: 11
    • Functions: 1
    • Constants: 0

    Module Documentation

    """
    Pattern 068: Instruction Following & Grounding
    
    Description:
        Instruction Following & Grounding enables agents to precisely understand and execute
        natural language instructions by breaking them down into structured, executable steps.
        This pattern goes beyond simple command parsing to deeply ground instructions in the
        agent's capabilities, environment constraints, and task context. The agent validates
        instructions for feasibility, disambiguates ambiguous commands, grounds abstract
        concepts in concrete actions, and provides execution verification.
    
        The pattern is crucial for building reliable agents that can follow complex,
        multi-step instructions while maintaining safety, handling edge cases, and providing
        transparent feedback about what it's doing and why.
    
    Components:
        1. Instruction Parser: Extracts intent, parameters, and constraints
        2. Action Grounder: Maps abstract instructions to concrete actions
        3. Feasibility Checker: Validates instruction executability
        4. Disambiguator: Resolves ambiguities through clarification
        5. Execution Planner: Creates step-by-step execution plan
        6. Execution Monitor: Tracks execution progress and validates outcomes
        7. Feedback Generator: Provides status updates and explanations
    
    Architecture:
        ```
        Natural Language Instruction
            ↓
        Instruction Parser → [Intent, Parameters, Constraints]
            ↓
        Disambiguator → [Clarified Instruction]
            ↓
        Action Grounder → [Concrete Actions]
            ↓
        Feasibility Checker → [Validation]
            ↓
        Execution Planner → [Execution Steps]
            ↓
        Execution Monitor → [Execute & Verify]
            ↓
        Feedback Generator → [Status & Results]
        ```
    
    Use Cases:
        - Robotic task execution from natural language
        - Virtual assistant command processing
        - Smart home automation
        - Software automation and scripting
        - Manufacturing and logistics instructions
        - Educational tutoring systems
    
    Advantages:
        - Handles ambiguous and underspecified instructions
        - Grounds abstract concepts in concrete actions
        - Validates feasibility before execution
        - Provides transparent execution process
        - Handles errors and edge cases gracefully
        - Learns from execution feedback
    
    LangChain Implementation:
        Uses ChatOpenAI for instruction parsing, grounding, and planning.
        Demonstrates instruction decomposition, parameter extraction, feasibility
        checking, execution planning, and verification.
    """
    

    Purpose: This module implements the Instruction Following pattern, providing a structured approach to agent design and implementation.


    Import Statements

    Standard Library Imports

    import dataclasses.dataclass
    import dataclasses.field
    import enum.Enum
    import langchain_core.output_parsers.StrOutputParser
    import langchain_core.prompts.ChatPromptTemplate
    import os
    import time
    import typing.Any
    import typing.Dict
    import typing.List
    import typing.Optional
    import typing.Tuple
    

    Purpose: Core Python functionality: – dataclasses.dataclass: Simplified class definitions with @dataclass – dataclasses.field: Simplified class definitions with @dataclass – enum.Enum: Enumeration support – langchain_core.output_parsers.StrOutputParser: Regular expression operations – langchain_core.prompts.ChatPromptTemplate: Regular expression operations – os: Environment variable access and system operations – time: Time-related functions – typing.Any: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Dict: Type hints for better code documentation (List, Dict, Any, Optional) – typing.List: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Optional: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Tuple: Type hints for better code documentation (List, Dict, Any, Optional)

    LangChain Imports

    from langchain_openai.ChatOpenAI
    

    Purpose: LangChain framework components: – LLM Interface: Wrapper for OpenAI models – Chains: Composable building blocks for complex workflows

    Other Imports

    from dotenv.load_dotenv
    

    Purpose: Additional dependencies for specialized functionality.


    Classes

    Class: InstructionType

    Lines 80-87

    Inherits from: Enum

    Purpose:

    Types of instructions.
    

    Class: ActionStatus

    Lines 90-96

    Inherits from: Enum

    Purpose:

    Status of action execution.
    

    Class: FeasibilityLevel

    Lines 99-105

    Inherits from: Enum

    Purpose:

    Feasibility assessment levels.
    

    Class: Parameter

    Lines 109-115

    Purpose:

    Represents a parameter extracted from instruction.
    

    Class: Constraint

    Lines 119-123

    Purpose:

    Represents a constraint on instruction execution.
    

    Class: ParsedInstruction

    Lines 127-136

    Purpose:

    Represents a parsed natural language instruction.
    

    Class: GroundedAction

    Lines 140-154

    Purpose:

    Represents a grounded, executable action.
    

    Methods:

    execute(self)

    Lines 149-154

    Purpose:

    Simulate action execution.
    

    Parameters:

    Returns: <ast.Subscript object at 0x000001A06D609C50>


    Class: ExecutionStep

    Lines 158-166

    Purpose:

    Represents a step in execution plan.
    

    Class: FeasibilityAssessment

    Lines 170-177

    Purpose:

    Assessment of instruction feasibility.
    

    Class: ExecutionResult

    Lines 181-188

    Purpose:

    Complete result of instruction execution.
    

    Class: InstructionFollowingAgent

    Lines 191-685

    Purpose:

    Agent that parses, grounds, and executes natural language instructions.
    

    Methods:

    __init__(self, agent_capabilities)

    Lines 196-227

    Purpose:

    Initialize the instruction following agent.
    
    Args:
        agent_capabilities: List of actions the agent can perform
    

    Parameters: – agent_capabilities: Agent capabilities

    parse_instruction(self, instruction_text)

    Lines 229-338

    Purpose:

    Parse natural language instruction into structured format.
    
    Args:
        instruction_text: Natural language instruction
    
    Returns:
        Parsed instruction with intent, parameters, and constraints
    

    Parameters: – instruction_text: Instruction text

    Returns: ParsedInstruction

    ground_instruction(self, parsed)

    Lines 340-465

    Purpose:

    Ground abstract instruction into concrete, executable actions.
    
    Args:
        parsed: Parsed instruction
    
    Returns:
        List of grounded actions
    

    Parameters: – parsed: Parsed

    Returns: <ast.Subscript object at 0x000001A06D77FED0>

    check_feasibility(self, parsed, grounded_actions)

    Lines 467-574

    Purpose:

    Check if instruction is feasible to execute.
    
    Args:
        parsed: Parsed instruction
        grounded_actions: Grounded actions
    
    Returns:
        Feasibility assessment
    

    Parameters: – parsed: Parsed – grounded_actions: Grounded actions

    Returns: FeasibilityAssessment

    execute_instruction(self, instruction_text, dry_run)

    Lines 576-685

    Purpose:

    Execute a natural language instruction end-to-end.
    
    Args:
        instruction_text: Natural language instruction
        dry_run: If True, plan but don't execute
    
    Returns:
        Complete execution result
    

    Parameters: – instruction_text: Instruction text – dry_run: Dry run

    Returns: ExecutionResult


    Functions

    demonstrate_instruction_following()

    Lines 688-884

    Purpose:

    Demonstrate Instruction Following & Grounding pattern.
    

    Key Concepts Summary

    Pattern Overview

    The Instruction Following pattern provides a structured approach to building AI agents with specific capabilities and behaviors. This implementation leverages LangChain’s composable components to create a robust and maintainable solution.

    Core Components

    1. Agent Class: Main implementation of the pattern logic
    2. Prompt Engineering: Carefully crafted prompts to guide LLM behavior
    3. Tool Integration: Optional tools for extended functionality
    4. Memory Management: State handling across interactions
    5. Output Parsing: Structured response processing

    When to Use

    ✅ Good for: – Scenarios requiring this specific pattern’s strengths – Applications needing structured agent behavior – Systems where explainability is important – Production environments requiring reliability

    ❌ Not ideal for: – Simple single-prompt tasks – Scenarios requiring different pattern characteristics – Resource-constrained environments (if computationally intensive)


    Best Practices Demonstrated

    1. Environment Configuration

    • ✅ Use .env files for API keys
    • ✅ Never hardcode credentials
    • ✅ Validate environment variables on startup

    2. Error Handling

    • ✅ Graceful degradation on failures
    • ✅ Informative error messages
    • ✅ Logging for debugging

    3. Prompt Engineering

    • ✅ Clear, specific instructions
    • ✅ Structured output formats
    • ✅ Few-shot examples when beneficial

    4. Code Organization

    • ✅ Separation of concerns (class-based design)
    • ✅ Type hints for clarity
    • ✅ Docstrings for documentation

    5. Performance Optimization

    • ✅ Appropriate temperature settings
    • ✅ Token budget management
    • ✅ Caching where applicable

    LangChain Components Used

    • ChatOpenAI: OpenAI model interface
    • Chains: Sequential component composition

    Why LangChain? – Composable abstractions for LLM applications – Rich ecosystem of integrations – Production-ready patterns – Active community and development


    Execution Flow

    1. Initialize agent with configuration
       ↓
    2. Load environment and dependencies
       ↓
    3. Prepare prompt with input data
       ↓
    4. Invoke LLM through LangChain
       ↓
    5. Parse and validate response
       ↓
    6. Return formatted result
    

    Key Decision Points: – Model selection (temperature, model type) – Prompt structure (few-shot vs zero-shot) – Error handling strategy – Output format validation


    Performance Considerations

    Computational Cost

    • LLM Calls: Primary cost factor (based on tokens)
    • Latency: Depends on model and input size
    • Memory: Minimal for most patterns

    Optimization Strategies

    1. Prompt Optimization: Minimize tokens while maintaining clarity
    2. Caching: Cache repeated queries/responses
    3. Model Selection: Use smaller models when appropriate
    4. Batch Processing: Group similar requests
    5. Streaming: For long responses, stream tokens

    Cost Estimates (Approximate)

    • GPT-3.5-turbo: ~$0.002 per request
    • GPT-4: ~$0.03 per request
    • GPT-4-turbo: ~$0.01 per request

    Actual costs vary based on input/output token counts


    Common Pitfalls to Avoid

    1. Prompt Design Issues

    ❌ Vague or ambiguous instructions
    ✅ Clear, specific, testable prompts

    2. Error Handling

    ❌ Ignoring API errors or rate limits
    ✅ Implement retry logic and graceful degradation

    3. Token Management

    ❌ Exceeding model token limits
    ✅ Truncate inputs, chunk processing

    4. Security

    ❌ Exposing API keys in code
    ✅ Use environment variables

    5. Testing

    ❌ Only testing happy paths
    ✅ Test edge cases, failures, malformed inputs

    6. Cost Management

    ❌ Unmonitored API usage
    ✅ Implement budgets, alerts, and monitoring


    Further Improvements

    Production Enhancements

    1. Monitoring: Add logging, metrics, tracing
    2. Validation: Input/output schema validation
    3. Rate Limiting: Implement request throttling
    4. Caching: Redis or in-memory cache for responses
    5. Async Support: Non-blocking operations for scalability

    Feature Extensions

    1. Multi-Model Support: Switch between providers
    2. Fallback Models: Graceful degradation strategy
    3. Custom Tools: Domain-specific function calling
    4. Advanced Memory: Vector stores for long-term memory
    5. Evaluation: Automated testing and benchmarking

    Code Quality

    1. Testing: Unit tests, integration tests
    2. Documentation: API docs, usage examples
    3. Type Safety: Stricter type hints
    4. Configuration: Externalized config files

    Related Patterns

    • Pattern 067
    • Pattern 069

    Pattern Families: – Reasoning Patterns (001-011) – Multi-Agent Patterns (012-019) – Tool Use Patterns (020-025) – Memory Patterns (026-031) – And many more…


    Summary

    The Instruction Following pattern provides a powerful approach to building AI agents with specific capabilities. This implementation demonstrates:

    ✅ Key Strengths: – Clean, maintainable code structure – LangChain integration for composability – Type hints and documentation – Production-ready patterns

    🎯 Best Use Cases: – Applications requiring this pattern’s characteristics – Systems needing structured agent behavior – Production environments requiring reliability

    📚 Learning Resources: – LangChain Documentation: https://python.langchain.com/ – Pattern Documentation: See related patterns – OpenAI API Reference: https://platform.openai.com/docs/


    File: 068_instruction_following.py
    Total Lines: 889
    Pattern Category: AI Agent Design Pattern #068


    Generated from actual implementation code
    For educational and reference purposes

  • Agentic Design Pattern: Agentic Rag

    
        """
    Pattern 067: Agentic RAG (Advanced)
    
    Description:
        Agentic RAG extends basic Retrieval-Augmented Generation with autonomous decision-making
        capabilities. Unlike simple RAG that retrieves once and generates, Agentic RAG employs
        sophisticated query planning, multi-hop reasoning, dynamic retrieval strategies, and source
        validation. The agent can decompose complex queries, determine when and what to retrieve,
        route queries to appropriate sources, validate retrieved information, and iteratively
        refine its understanding through multiple retrieval-reasoning cycles.
        
        This pattern transforms RAG from a passive information lookup into an active research
        process where the agent strategically explores knowledge bases, identifies gaps, verifies
        information across sources, and constructs well-grounded answers through deliberate
        reasoning chains.
    
    Components:
        1. Query Analyzer: Decomposes complex questions into sub-queries
        2. Query Router: Routes queries to appropriate retrieval sources
        3. Multi-Hop Reasoner: Follows chains of reasoning across retrieved documents
        4. Source Validator: Validates credibility and relevance of sources
        5. Retrieval Planner: Decides what to retrieve and when
        6. Answer Synthesizer: Combines information from multiple sources
        7. Confidence Estimator: Assesses answer quality and completeness
    
    Architecture:
        ```
        User Query
            ↓
        Query Analyzer → [Sub-queries]
            ↓
        Query Router → [Route to sources]
            ↓
        Retrieval Planner → [Iterative retrieval strategy]
            ↓
        Multi-Hop Reasoner → [Follow reasoning chains]
            ↓
        Source Validator → [Validate & filter]
            ↓
        Answer Synthesizer → [Combine & generate]
            ↓
        Confidence Estimator → [Quality check]
            ↓
        Final Answer (or iterate)
        ```
    
    Use Cases:
        - Complex research questions requiring multi-source synthesis
        - Scientific literature review and analysis
        - Legal document analysis with precedent lookup
        - Technical troubleshooting requiring multiple knowledge sources
        - Fact-checking and claim verification across sources
        - Exploratory data analysis with iterative refinement
    
    Advantages:
        - Handles complex, multi-part questions effectively
        - Validates information across multiple sources
        - Provides transparent reasoning chains
        - Adapts retrieval strategy based on findings
        - Reduces hallucination through source grounding
        - Identifies and fills knowledge gaps
    
    LangChain Implementation:
        Uses ChatOpenAI for LLM operations, custom query planning, routing logic,
        and iterative retrieval-reasoning loops. Demonstrates query decomposition,
        multi-source retrieval, validation chains, and answer synthesis.
    """
    
    import os
    from typing import List, Dict, Any, Optional, Set, Tuple
    from dataclasses import dataclass, field
    from enum import Enum
    import time
    from dotenv import load_dotenv
    
    from langchain_core.prompts import ChatPromptTemplate
    from langchain_core.output_parsers import StrOutputParser
    from langchain_openai import ChatOpenAI
    
    load_dotenv()
    
    
    class QueryType(Enum):
        """Types of queries for routing."""
        FACTUAL = "factual"
        COMPARATIVE = "comparative"
        PROCEDURAL = "procedural"
        CAUSAL = "causal"
        EXPLORATORY = "exploratory"
    
    
    class SourceType(Enum):
        """Types of retrieval sources."""
        DOCUMENTS = "documents"
        WEB = "web"
        DATABASE = "database"
        API = "api"
        KNOWLEDGE_GRAPH = "knowledge_graph"
    
    
    class ConfidenceLevel(Enum):
        """Confidence in answer quality."""
        HIGH = "high"
        MEDIUM = "medium"
        LOW = "low"
        INSUFFICIENT = "insufficient"
    
    
    @dataclass
    class SubQuery:
        """Represents a decomposed sub-query."""
        query_id: str
        text: str
        query_type: QueryType
        dependencies: List[str] = field(default_factory=list)  # Other query IDs this depends on
        priority: int = 1
    
    
    @dataclass
    class RetrievalSource:
        """Represents a source for retrieval."""
        source_id: str
        source_type: SourceType
        name: str
        relevance_score: float = 0.0
        
        def retrieve(self, query: str) -> List[str]:
            """Simulate retrieval from this source."""
            # In real implementation, this would access actual data sources
            if self.source_type == SourceType.DOCUMENTS:
                return [
                    f"[{self.name}] Document excerpt about: {query}",
                    f"[{self.name}] Additional context: {query}"
                ]
            elif self.source_type == SourceType.WEB:
                return [f"[{self.name}] Web search result for: {query}"]
            elif self.source_type == SourceType.DATABASE:
                return [f"[{self.name}] Database record matching: {query}"]
            else:
                return [f"[{self.name}] Information about: {query}"]
    
    
    @dataclass
    class RetrievedDocument:
        """Represents a retrieved document with metadata."""
        doc_id: str
        content: str
        source: RetrievalSource
        relevance_score: float
        credibility_score: float = 0.0
        timestamp: float = field(default_factory=time.time)
    
    
    @dataclass
    class ReasoningStep:
        """Represents a step in multi-hop reasoning."""
        step_id: int
        question: str
        retrieved_docs: List[RetrievedDocument]
        reasoning: str
        answer: str
        confidence: float
    
    
    @dataclass
    class AgenticRAGResult:
        """Complete result from Agentic RAG."""
        query: str
        sub_queries: List[SubQuery]
        reasoning_chain: List[ReasoningStep]
        final_answer: str
        sources_used: List[RetrievalSource]
        confidence: ConfidenceLevel
        gaps_identified: List[str] = field(default_factory=list)
    
    
    class AgenticRAGAgent:
        """
        Advanced RAG agent with query planning, multi-hop reasoning, and validation.
        """
        
        def __init__(
            self,
            max_hops: int = 3,
            min_confidence: float = 0.7,
            temperature: float = 0.3
        ):
            """
            Initialize the Agentic RAG agent.
            
            Args:
                max_hops: Maximum number of retrieval-reasoning hops
                min_confidence: Minimum confidence threshold for answer
                temperature: Temperature for LLM generation
            """
            self.max_hops = max_hops
            self.min_confidence = min_confidence
            
            # Different LLMs for different tasks
            self.query_analyzer = ChatOpenAI(temperature=0.2, model="gpt-3.5-turbo")
            self.router = ChatOpenAI(temperature=0.1, model="gpt-3.5-turbo")
            self.reasoner = ChatOpenAI(temperature=temperature, model="gpt-3.5-turbo")
            self.validator = ChatOpenAI(temperature=0.1, model="gpt-3.5-turbo")
            self.synthesizer = ChatOpenAI(temperature=0.4, model="gpt-3.5-turbo")
            
            # Available sources (in real implementation, these would be actual data sources)
            self.sources = [
                RetrievalSource("src1", SourceType.DOCUMENTS, "Technical Documentation"),
                RetrievalSource("src2", SourceType.WEB, "Web Search"),
                RetrievalSource("src3", SourceType.DATABASE, "Knowledge Database"),
                RetrievalSource("src4", SourceType.KNOWLEDGE_GRAPH, "Entity Graph"),
            ]
            
            self.parser = StrOutputParser()
        
        def decompose_query(self, query: str) -> List[SubQuery]:
            """
            Decompose complex query into sub-queries.
            
            Args:
                query: The complex user query
                
            Returns:
                List of sub-queries with dependencies
            """
            prompt = ChatPromptTemplate.from_messages([
                ("system", """You are a query analyzer. Decompose complex questions into sub-queries.
    For each sub-query, identify:
    1. The specific question
    2. The type (factual, comparative, procedural, causal, exploratory)
    3. Dependencies on other sub-queries
    4. Priority (1=highest, 3=lowest)
    
    Format each sub-query as:
    Q[id]: [question] | Type: [type] | Depends: [comma-separated ids or 'none'] | Priority: [1-3]"""),
                ("user", "Decompose this query: {query}")
            ])
            
            chain = prompt | self.query_analyzer | self.parser
            
            try:
                result = chain.invoke({"query": query})
                
                # Parse the result into SubQuery objects
                sub_queries = []
                lines = [line.strip() for line in result.split('\n') if line.strip().startswith('Q')]
                
                for line in lines:
                    try:
                        # Parse format: Q1: What is X? | Type: factual | Depends: none | Priority: 1
                        parts = line.split('|')
                        if len(parts) >= 4:
                            # Extract query ID and text
                            q_part = parts[0].strip()
                            q_id = q_part.split(':')[0].strip()
                            q_text = ':'.join(q_part.split(':')[1:]).strip()
                            
                            # Extract type
                            type_str = parts[1].split(':')[1].strip().lower()
                            q_type = QueryType(type_str) if type_str in [t.value for t in QueryType] else QueryType.FACTUAL
                            
                            # Extract dependencies
                            deps_str = parts[2].split(':')[1].strip().lower()
                            dependencies = [] if deps_str == 'none' else [d.strip() for d in deps_str.split(',')]
                            
                            # Extract priority
                            priority = int(parts[3].split(':')[1].strip())
                            
                            sub_queries.append(SubQuery(
                                query_id=q_id,
                                text=q_text,
                                query_type=q_type,
                                dependencies=dependencies,
                                priority=priority
                            ))
                    except Exception as e:
                        print(f"Warning: Failed to parse sub-query line: {line}")
                        continue
                
                # If parsing failed, create a single sub-query
                if not sub_queries:
                    sub_queries = [SubQuery(
                        query_id="Q1",
                        text=query,
                        query_type=QueryType.EXPLORATORY,
                        dependencies=[],
                        priority=1
                    )]
                
                return sub_queries
                
            except Exception as e:
                print(f"Error in query decomposition: {e}")
                # Fallback to single query
                return [SubQuery(
                    query_id="Q1",
                    text=query,
                    query_type=QueryType.EXPLORATORY,
                    dependencies=[],
                    priority=1
                )]
        
        def route_query(self, sub_query: SubQuery) -> List[RetrievalSource]:
            """
            Route sub-query to appropriate sources.
            
            Args:
                sub_query: The sub-query to route
                
            Returns:
                Ranked list of sources to query
            """
            prompt = ChatPromptTemplate.from_messages([
                ("system", """You are a query router. Given a query and available sources, 
    rank which sources are most relevant (1-5 scale).
    
    Available sources:
    - Technical Documentation (detailed technical specs)
    - Web Search (general information, current events)
    - Knowledge Database (structured facts and entities)
    - Entity Graph (relationships between concepts)
    
    Format: [Source Name]: [score] - [brief reason]"""),
                ("user", "Query: {query}\nQuery Type: {query_type}\nRank the sources:")
            ])
            
            chain = prompt | self.router | self.parser
            
            try:
                result = chain.invoke({
                    "query": sub_query.text,
                    "query_type": sub_query.query_type.value
                })
                
                # Parse relevance scores and update sources
                scored_sources = []
                for source in self.sources:
                    if source.name in result:
                        # Extract score (look for patterns like "4" or "4/5")
                        import re
                        pattern = rf"{re.escape(source.name)}[:\s]+(\d+)"
                        match = re.search(pattern, result)
                        if match:
                            score = float(match.group(1)) / 5.0  # Normalize to 0-1
                            source.relevance_score = score
                            scored_sources.append(source)
                
                # Sort by relevance and return top sources
                scored_sources.sort(key=lambda x: x.relevance_score, reverse=True)
                return scored_sources[:2] if scored_sources else self.sources[:2]
                
            except Exception as e:
                print(f"Error in query routing: {e}")
                # Default to all sources
                return self.sources[:2]
        
        def retrieve_documents(
            self,
            sub_query: SubQuery,
            sources: List[RetrievalSource]
        ) -> List[RetrievedDocument]:
            """
            Retrieve documents from sources.
            
            Args:
                sub_query: The query to retrieve for
                sources: Sources to retrieve from
                
            Returns:
                List of retrieved documents
            """
            documents = []
            
            for source in sources:
                # Retrieve from source (simulated)
                contents = source.retrieve(sub_query.text)
                
                for i, content in enumerate(contents):
                    doc = RetrievedDocument(
                        doc_id=f"{source.source_id}_doc_{i}",
                        content=content,
                        source=source,
                        relevance_score=source.relevance_score * (1.0 - i * 0.1)  # Decay for lower results
                    )
                    documents.append(doc)
            
            # Sort by relevance
            documents.sort(key=lambda x: x.relevance_score, reverse=True)
            return documents
        
        def validate_sources(self, documents: List[RetrievedDocument]) -> List[RetrievedDocument]:
            """
            Validate credibility and consistency of retrieved documents.
            
            Args:
                documents: Documents to validate
                
            Returns:
                Validated documents with credibility scores
            """
            prompt = ChatPromptTemplate.from_messages([
                ("system", """You are a source validator. Assess the credibility of information.
    Consider:
    1. Source authority
    2. Information consistency across sources
    3. Specificity and detail level
    4. Potential biases
    
    Rate credibility 0.0-1.0 for each source and briefly explain."""),
                ("user", "Validate these sources:\n{sources}")
            ])
            
            chain = prompt | self.validator | self.parser
            
            try:
                # Prepare sources text
                sources_text = "\n".join([
                    f"- {doc.source.name}: {doc.content[:200]}..."
                    for doc in documents[:3]  # Validate top 3
                ])
                
                result = chain.invoke({"sources": sources_text})
                
                # Parse credibility scores (simple heuristic)
                for doc in documents:
                    # Higher credibility for documentation and databases
                    if doc.source.source_type in [SourceType.DOCUMENTS, SourceType.DATABASE]:
                        doc.credibility_score = 0.9
                    elif doc.source.source_type == SourceType.KNOWLEDGE_GRAPH:
                        doc.credibility_score = 0.85
                    else:
                        doc.credibility_score = 0.7
                
                # Filter low credibility documents
                validated_docs = [doc for doc in documents if doc.credibility_score >= 0.6]
                return validated_docs
                
            except Exception as e:
                print(f"Error in source validation: {e}")
                # Default: assign medium credibility to all
                for doc in documents:
                    doc.credibility_score = 0.75
                return documents
        
        def multi_hop_reasoning(
            self,
            sub_queries: List[SubQuery],
            context: Dict[str, str] = None
        ) -> List[ReasoningStep]:
            """
            Perform multi-hop reasoning across sub-queries.
            
            Args:
                sub_queries: Sub-queries to reason over
                context: Context from previous reasoning steps
                
            Returns:
                List of reasoning steps
            """
            reasoning_steps = []
            context = context or {}
            
            # Sort by priority and dependencies
            sorted_queries = sorted(sub_queries, key=lambda q: (q.priority, len(q.dependencies)))
            
            for step_id, sub_query in enumerate(sorted_queries, 1):
                # Check if dependencies are met
                if sub_query.dependencies:
                    missing_deps = [dep for dep in sub_query.dependencies if dep not in context]
                    if missing_deps:
                        print(f"Warning: Skipping {sub_query.query_id} due to missing dependencies: {missing_deps}")
                        continue
                
                # Route and retrieve
                sources = self.route_query(sub_query)
                documents = self.retrieve_documents(sub_query, sources)
                validated_docs = self.validate_sources(documents)
                
                # Build context from previous answers
                prev_context = "\n".join([
                    f"{qid}: {ans}"
                    for qid, ans in context.items()
                ])
                
                # Reason over retrieved documents
                prompt = ChatPromptTemplate.from_messages([
                    ("system", """You are a reasoning agent. Answer the question using retrieved documents.
    Explain your reasoning step-by-step and provide a clear answer.
    If information is insufficient, state what's missing."""),
                    ("user", """Previous context:
    {prev_context}
    
    Current question: {question}
    
    Retrieved information:
    {documents}
    
    Provide:
    1. Your reasoning
    2. Your answer
    3. Confidence (0.0-1.0)
    
    Format:
    Reasoning: [your reasoning]
    Answer: [your answer]
    Confidence: [0.0-1.0]""")
                ])
                
                chain = prompt | self.reasoner | self.parser
                
                try:
                    docs_text = "\n".join([
                        f"- [{doc.source.name}] {doc.content}"
                        for doc in validated_docs[:3]
                    ])
                    
                    result = chain.invoke({
                        "prev_context": prev_context if prev_context else "None",
                        "question": sub_query.text,
                        "documents": docs_text
                    })
                    
                    # Parse reasoning, answer, and confidence
                    reasoning = ""
                    answer = ""
                    confidence = 0.5
                    
                    for line in result.split('\n'):
                        if line.startswith("Reasoning:"):
                            reasoning = line.replace("Reasoning:", "").strip()
                        elif line.startswith("Answer:"):
                            answer = line.replace("Answer:", "").strip()
                        elif line.startswith("Confidence:"):
                            try:
                                confidence = float(line.replace("Confidence:", "").strip())
                            except:
                                confidence = 0.5
                    
                    if not answer:
                        answer = result.split('\n')[0]  # Fallback to first line
                    
                    reasoning_step = ReasoningStep(
                        step_id=step_id,
                        question=sub_query.text,
                        retrieved_docs=validated_docs,
                        reasoning=reasoning if reasoning else "Analyzed retrieved documents.",
                        answer=answer,
                        confidence=confidence
                    )
                    
                    reasoning_steps.append(reasoning_step)
                    context[sub_query.query_id] = answer
                    
                except Exception as e:
                    print(f"Error in reasoning for {sub_query.query_id}: {e}")
                    continue
            
            return reasoning_steps
        
        def synthesize_answer(
            self,
            query: str,
            reasoning_steps: List[ReasoningStep]
        ) -> Tuple[str, ConfidenceLevel, List[str]]:
            """
            Synthesize final answer from reasoning steps.
            
            Args:
                query: Original user query
                reasoning_steps: All reasoning steps
                
            Returns:
                Tuple of (final answer, confidence level, identified gaps)
            """
            prompt = ChatPromptTemplate.from_messages([
                ("system", """You are an answer synthesizer. Combine insights from multiple reasoning steps
    into a comprehensive, coherent answer. Identify any gaps in knowledge.
    
    Provide:
    1. Synthesized answer
    2. Overall confidence (high/medium/low/insufficient)
    3. Any gaps or limitations
    
    Format:
    Answer: [comprehensive answer]
    Confidence: [high/medium/low/insufficient]
    Gaps: [identified gaps or 'none']"""),
                ("user", """Original query: {query}
    
    Reasoning steps:
    {steps}
    
    Synthesize the final answer:""")
            ])
            
            chain = prompt | self.synthesizer | self.parser
            
            try:
                steps_text = "\n".join([
                    f"Step {step.step_id}: {step.question}\n  Reasoning: {step.reasoning}\n  Answer: {step.answer}\n  Confidence: {step.confidence}"
                    for step in reasoning_steps
                ])
                
                result = chain.invoke({
                    "query": query,
                    "steps": steps_text
                })
                
                # Parse result
                answer = ""
                confidence = ConfidenceLevel.MEDIUM
                gaps = []
                
                for line in result.split('\n'):
                    if line.startswith("Answer:"):
                        answer = line.replace("Answer:", "").strip()
                        # Continue reading multi-line answer
                        answer_lines = [answer]
                        for next_line in result.split('\n')[result.split('\n').index(line)+1:]:
                            if next_line.startswith("Confidence:") or next_line.startswith("Gaps:"):
                                break
                            if next_line.strip():
                                answer_lines.append(next_line.strip())
                        answer = " ".join(answer_lines)
                    elif line.startswith("Confidence:"):
                        conf_str = line.replace("Confidence:", "").strip().lower()
                        if "high" in conf_str:
                            confidence = ConfidenceLevel.HIGH
                        elif "low" in conf_str:
                            confidence = ConfidenceLevel.LOW
                        elif "insufficient" in conf_str:
                            confidence = ConfidenceLevel.INSUFFICIENT
                        else:
                            confidence = ConfidenceLevel.MEDIUM
                    elif line.startswith("Gaps:"):
                        gaps_str = line.replace("Gaps:", "").strip()
                        if gaps_str.lower() not in ['none', 'no gaps', '']:
                            gaps = [g.strip() for g in gaps_str.split(',')]
                
                if not answer:
                    answer = result.split('\n')[0]  # Fallback
                
                return answer, confidence, gaps
                
            except Exception as e:
                print(f"Error in answer synthesis: {e}")
                # Fallback: combine all reasoning step answers
                combined = " ".join([step.answer for step in reasoning_steps])
                avg_confidence = sum(step.confidence for step in reasoning_steps) / len(reasoning_steps) if reasoning_steps else 0.5
                
                if avg_confidence >= 0.8:
                    conf_level = ConfidenceLevel.HIGH
                elif avg_confidence >= 0.6:
                    conf_level = ConfidenceLevel.MEDIUM
                elif avg_confidence >= 0.4:
                    conf_level = ConfidenceLevel.LOW
                else:
                    conf_level = ConfidenceLevel.INSUFFICIENT
                
                return combined, conf_level, ["Error in synthesis process"]
        
        def query(self, user_query: str) -> AgenticRAGResult:
            """
            Process a query through the full Agentic RAG pipeline.
            
            Args:
                user_query: The user's question
                
            Returns:
                Complete Agentic RAG result with reasoning chain
            """
            print(f"\n{'='*60}")
            print(f"Processing Query: {user_query}")
            print(f"{'='*60}\n")
            
            # Step 1: Decompose query
            print("Step 1: Decomposing query...")
            sub_queries = self.decompose_query(user_query)
            print(f"  → Generated {len(sub_queries)} sub-queries")
            for sq in sub_queries:
                print(f"    • {sq.query_id}: {sq.text} (Type: {sq.query_type.value}, Priority: {sq.priority})")
            
            # Step 2: Multi-hop reasoning
            print("\nStep 2: Multi-hop reasoning...")
            reasoning_steps = self.multi_hop_reasoning(sub_queries)
            print(f"  → Completed {len(reasoning_steps)} reasoning steps")
            
            # Step 3: Synthesize answer
            print("\nStep 3: Synthesizing final answer...")
            final_answer, confidence, gaps = self.synthesize_answer(user_query, reasoning_steps)
            print(f"  → Confidence: {confidence.value}")
            
            # Collect all sources used
            sources_used = set()
            for step in reasoning_steps:
                for doc in step.retrieved_docs:
                    sources_used.add(doc.source)
            
            return AgenticRAGResult(
                query=user_query,
                sub_queries=sub_queries,
                reasoning_chain=reasoning_steps,
                final_answer=final_answer,
                sources_used=list(sources_used),
                confidence=confidence,
                gaps_identified=gaps
            )
    
    
    def demonstrate_agentic_rag():
        """Demonstrate Agentic RAG pattern with various scenarios."""
        
        print("="*80)
        print("AGENTIC RAG (ADVANCED) - DEMONSTRATION")
        print("="*80)
        
        agent = AgenticRAGAgent(max_hops=3, min_confidence=0.7)
        
        # Test 1: Multi-part complex query
        print("\n" + "="*80)
        print("TEST 1: Complex Multi-Part Query")
        print("="*80)
        
        result1 = agent.query(
            "What is the difference between machine learning and deep learning, "
            "and which one should I use for image classification?"
        )
        
        print("\n--- RESULT ---")
        print(f"Final Answer: {result1.final_answer}")
        print(f"Confidence: {result1.confidence.value}")
        print(f"Sources Used: {[s.name for s in result1.sources_used]}")
        print(f"Sub-queries: {len(result1.sub_queries)}")
        print(f"Reasoning Steps: {len(result1.reasoning_chain)}")
        if result1.gaps_identified:
            print(f"Gaps Identified: {', '.join(result1.gaps_identified)}")
        
        # Test 2: Causal query requiring multi-hop reasoning
        print("\n" + "="*80)
        print("TEST 2: Causal Query with Multi-Hop Reasoning")
        print("="*80)
        
        result2 = agent.query(
            "Why do neural networks need activation functions, and what happens "
            "if we remove them?"
        )
        
        print("\n--- REASONING CHAIN ---")
        for step in result2.reasoning_chain:
            print(f"\nStep {step.step_id}: {step.question}")
            print(f"  Reasoning: {step.reasoning}")
            print(f"  Answer: {step.answer}")
            print(f"  Confidence: {step.confidence:.2f}")
            print(f"  Sources: {[doc.source.name for doc in step.retrieved_docs[:2]]}")
        
        print("\n--- FINAL ANSWER ---")
        print(result2.final_answer)
        print(f"Overall Confidence: {result2.confidence.value}")
        
        # Test 3: Comparative analysis
        print("\n" + "="*80)
        print("TEST 3: Comparative Analysis Query")
        print("="*80)
        
        result3 = agent.query(
            "Compare the advantages and disadvantages of SQL vs NoSQL databases "
            "for a high-traffic social media application."
        )
        
        print("\n--- QUERY DECOMPOSITION ---")
        for sq in result3.sub_queries:
            print(f"{sq.query_id}: {sq.text}")
            print(f"  Type: {sq.query_type.value}")
            print(f"  Priority: {sq.priority}")
            if sq.dependencies:
                print(f"  Depends on: {', '.join(sq.dependencies)}")
        
        print("\n--- FINAL ANSWER ---")
        print(result3.final_answer)
        print(f"\nSources consulted: {[s.name for s in result3.sources_used]}")
        print(f"Confidence: {result3.confidence.value}")
        
        # Summary
        print("\n" + "="*80)
        print("SUMMARY: Agentic RAG (Advanced)")
        print("="*80)
        print("""
    The Agentic RAG pattern demonstrates advanced information retrieval and reasoning:
    
    Key Features Demonstrated:
    1. Query Decomposition: Breaks complex queries into manageable sub-queries
    2. Intelligent Routing: Routes queries to appropriate specialized sources
    3. Multi-Hop Reasoning: Follows chains of logic across multiple retrieval steps
    4. Source Validation: Assesses credibility and consistency of information
    5. Answer Synthesis: Combines insights from multiple sources coherently
    6. Confidence Estimation: Provides transparency about answer quality
    7. Gap Identification: Identifies missing information or limitations
    
    Benefits:
    • Handles complex, multi-faceted questions effectively
    • Provides transparent reasoning chains for interpretability
    • Validates information across sources to reduce hallucination
    • Adapts retrieval strategy based on intermediate findings
    • Identifies knowledge gaps and uncertainty explicitly
    • Grounds answers in retrieved sources with citations
    
    Use Cases:
    • Research and literature review
    • Technical troubleshooting and diagnosis
    • Comparative analysis and decision support
    • Fact-checking and claim verification
    • Legal and regulatory document analysis
    • Medical diagnosis support (with appropriate validation)
    
    Comparison with Basic RAG:
    ┌─────────────────────┬──────────────────┬────────────────────┐
    │ Aspect              │ Basic RAG        │ Agentic RAG        │
    ├─────────────────────┼──────────────────┼────────────────────┤
    │ Query Processing    │ Single-shot      │ Decomposed         │
    │ Retrieval Strategy  │ One-time lookup  │ Iterative/adaptive │
    │ Reasoning           │ Direct answer    │ Multi-hop chains   │
    │ Source Handling     │ Single source    │ Multi-source       │
    │ Validation          │ Minimal          │ Comprehensive      │
    │ Transparency        │ Low              │ High               │
    │ Complexity Handling │ Limited          │ Excellent          │
    └─────────────────────┴──────────────────┴────────────────────┘
    
    LangChain Implementation Notes:
    • Multiple specialized LLMs for different tasks (analysis, routing, reasoning)
    • Custom query decomposition and dependency resolution
    • Source routing based on query type and content
    • Iterative retrieval-reasoning loops with context accumulation
    • Validation chains for credibility assessment
    • Synthesis prompts for coherent answer generation
    • Confidence estimation through multi-model assessment
    
    Production Considerations:
    • Implement actual vector stores and retrieval systems
    • Add caching for repeated queries and sub-queries
    • Implement proper source credibility scoring
    • Add cost controls for multi-hop operations
    • Monitor and limit reasoning depth to prevent loops
    • Implement proper error handling and fallbacks
    • Add human-in-the-loop for critical decisions
    • Track and optimize retrieval quality metrics
    
    Advanced Extensions:
    • Adaptive retrieval depth based on query complexity
    • Cross-lingual retrieval and reasoning
    • Temporal reasoning for time-sensitive queries
    • Graph-based knowledge integration
    • Active learning from user feedback
    • Explanation generation for reasoning steps
    • Multi-modal retrieval (text, images, structured data)
        """)
    
    
    if __name__ == "__main__":
        demonstrate_agentic_rag()
    
        

    Agentic Design Pattern: Agentic Rag

    Pattern 067: Agentic Rag – Line-by-Line Explanation

    Overview

    This document provides a detailed line-by-line explanation of the Agentic Rag pattern implementation.


    Pattern Description

    Pattern 067: Agentic RAG (Advanced)

    Description: Agentic RAG extends basic Retrieval-Augmented Generation with autonomous decision-making capabilities. Unlike simple RAG that retrieves once and generates, Agentic RAG employs sophisticated query planning, multi-hop reasoning, dynamic retrieval strategies, and source validation. The agent can decompose complex queries, determine when and what to retrieve, route queries to appropriate sources, validate retrieved information, and iteratively refine its understanding through multiple retrieval-reasoning cycles.

    This pattern transforms RAG from a passive information lookup into an active research
    process where the agent strategically explores knowledge bases, identifies gaps, verifies
    information across sources, and constructs well-grounded answers through deliberate
    reasoning chains.
    

    Components: 1. Query Analyzer: Decomposes complex questions into sub-queries 2. Query Router: Routes queries to appropriate retrieval sources 3. Multi-Hop Reasoner: Follows chains of reasoning across retrieved documents 4. Source Validator: Validates credibility and relevance of sources 5. Retrieval Planner: Decides what to retrieve and when 6. Answer Synthesizer: Combines information from multiple sources 7. Confidence Estimator: Assesses answer quality and completeness

    Architecture: User Query ↓ Query Analyzer → [Sub-queries] ↓ Query Router → [Route to sources] ↓ Retrieval Planner → [Iterative retrieval strategy] ↓ Multi-Hop Reasoner → [Follow reasoning chains] ↓ Source Validator → [Validate & filter] ↓ Answer Synthesizer → [Combine & generate] ↓ Confidence Estimator → [Quality check] ↓ Final Answer (or iterate)

    Use Cases: – Complex research questions requiring multi-source synthesis – Scientific literature review and analysis – Legal document analysis with precedent lookup – Technical troubleshooting requiring multiple knowledge sources – Fact-checking and claim verification across sources – Exploratory data analysis with iterative refinement

    Advantages: – Handles complex, multi-part questions effectively – Validates information across multiple sources – Provides transparent reasoning chains – Adapts retrieval strategy based on findings – Reduces hallucination through source grounding – Identifies and fills knowledge gaps

    LangChain Implementation: Uses ChatOpenAI for LLM operations, custom query planning, routing logic, and iterative retrieval-reasoning loops. Demonstrates query decomposition, multi-source retrieval, validation chains, and answer synthesis.


    File Statistics

    • Total Lines: 860
    • Classes: 9
    • Functions: 1
    • Constants: 0

    Module Documentation

    """
    Pattern 067: Agentic RAG (Advanced)
    
    Description:
        Agentic RAG extends basic Retrieval-Augmented Generation with autonomous decision-making
        capabilities. Unlike simple RAG that retrieves once and generates, Agentic RAG employs
        sophisticated query planning, multi-hop reasoning, dynamic retrieval strategies, and source
        validation. The agent can decompose complex queries, determine when and what to retrieve,
        route queries to appropriate sources, validate retrieved information, and iteratively
        refine its understanding through multiple retrieval-reasoning cycles.
    
        This pattern transforms RAG from a passive information lookup into an active research
        process where the agent strategically explores knowledge bases, identifies gaps, verifies
        information across sources, and constructs well-grounded answers through deliberate
        reasoning chains.
    
    Components:
        1. Query Analyzer: Decomposes complex questions into sub-queries
        2. Query Router: Routes queries to appropriate retrieval sources
        3. Multi-Hop Reasoner: Follows chains of reasoning across retrieved documents
        4. Source Validator: Validates credibility and relevance of sources
        5. Retrieval Planner: Decides what to retrieve and when
        6. Answer Synthesizer: Combines information from multiple sources
        7. Confidence Estimator: Assesses answer quality and completeness
    
    Architecture:
        ```
        User Query
            ↓
        Query Analyzer → [Sub-queries]
            ↓
        Query Router → [Route to sources]
            ↓
        Retrieval Planner → [Iterative retrieval strategy]
            ↓
        Multi-Hop Reasoner → [Follow reasoning chains]
            ↓
        Source Validator → [Validate & filter]
            ↓
        Answer Synthesizer → [Combine & generate]
            ↓
        Confidence Estimator → [Quality check]
            ↓
        Final Answer (or iterate)
        ```
    
    Use Cases:
        - Complex research questions requiring multi-source synthesis
        - Scientific literature review and analysis
        - Legal document analysis with precedent lookup
        - Technical troubleshooting requiring multiple knowledge sources
        - Fact-checking and claim verification across sources
        - Exploratory data analysis with iterative refinement
    
    Advantages:
        - Handles complex, multi-part questions effectively
        - Validates information across multiple sources
        - Provides transparent reasoning chains
        - Adapts retrieval strategy based on findings
        - Reduces hallucination through source grounding
        - Identifies and fills knowledge gaps
    
    LangChain Implementation:
        Uses ChatOpenAI for LLM operations, custom query planning, routing logic,
        and iterative retrieval-reasoning loops. Demonstrates query decomposition,
        multi-source retrieval, validation chains, and answer synthesis.
    """
    

    Purpose: This module implements the Agentic Rag pattern, providing a structured approach to agent design and implementation.


    Import Statements

    Standard Library Imports

    import dataclasses.dataclass
    import dataclasses.field
    import enum.Enum
    import langchain_core.output_parsers.StrOutputParser
    import langchain_core.prompts.ChatPromptTemplate
    import os
    import re
    import time
    import typing.Any
    import typing.Dict
    import typing.List
    import typing.Optional
    import typing.Set
    import typing.Tuple
    

    Purpose: Core Python functionality: – dataclasses.dataclass: Simplified class definitions with @dataclass – dataclasses.field: Simplified class definitions with @dataclass – enum.Enum: Enumeration support – langchain_core.output_parsers.StrOutputParser: Regular expression operations – langchain_core.prompts.ChatPromptTemplate: Regular expression operations – os: Environment variable access and system operations – re: Regular expression operations – time: Time-related functions – typing.Any: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Dict: Type hints for better code documentation (List, Dict, Any, Optional) – typing.List: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Optional: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Set: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Tuple: Type hints for better code documentation (List, Dict, Any, Optional)

    LangChain Imports

    from langchain_openai.ChatOpenAI
    

    Purpose: LangChain framework components: – LLM Interface: Wrapper for OpenAI models – Chains: Composable building blocks for complex workflows

    Other Imports

    from dotenv.load_dotenv
    

    Purpose: Additional dependencies for specialized functionality.


    Classes

    Class: QueryType

    Lines 83-89

    Inherits from: Enum

    Purpose:

    Types of queries for routing.
    

    Class: SourceType

    Lines 92-98

    Inherits from: Enum

    Purpose:

    Types of retrieval sources.
    

    Class: ConfidenceLevel

    Lines 101-106

    Inherits from: Enum

    Purpose:

    Confidence in answer quality.
    

    Class: SubQuery

    Lines 110-116

    Purpose:

    Represents a decomposed sub-query.
    

    Class: RetrievalSource

    Lines 120-140

    Purpose:

    Represents a source for retrieval.
    

    Methods:

    retrieve(self, query)

    Lines 127-140

    Purpose:

    Simulate retrieval from this source.
    

    Parameters: – query: Input query or question

    Returns: <ast.Subscript object at 0x000001A06D749090>


    Class: RetrievedDocument

    Lines 144-151

    Purpose:

    Represents a retrieved document with metadata.
    

    Class: ReasoningStep

    Lines 155-162

    Purpose:

    Represents a step in multi-hop reasoning.
    

    Class: AgenticRAGResult

    Lines 166-174

    Purpose:

    Complete result from Agentic RAG.
    

    Class: AgenticRAGAgent

    Lines 177-707

    Purpose:

    Advanced RAG agent with query planning, multi-hop reasoning, and validation.
    

    Methods:

    __init__(self, max_hops, min_confidence, temperature)

    Lines 182-214

    Purpose:

    Initialize the Agentic RAG agent.
    
    Args:
        max_hops: Maximum number of retrieval-reasoning hops
        min_confidence: Minimum confidence threshold for answer
        temperature: Temperature for LLM generation
    

    Parameters: – max_hops: Max hops – min_confidence: Min confidence – temperature: Sampling temperature (0-1)

    decompose_query(self, query)

    Lines 216-301

    Purpose:

    Decompose complex query into sub-queries.
    
    Args:
        query: The complex user query
    
    Returns:
        List of sub-queries with dependencies
    

    Parameters: – query: Input query or question

    Returns: <ast.Subscript object at 0x000001A06D7F5DD0>

    route_query(self, sub_query)

    Lines 303-355

    Purpose:

    Route sub-query to appropriate sources.
    
    Args:
        sub_query: The sub-query to route
    
    Returns:
        Ranked list of sources to query
    

    Parameters: – sub_query: Input query or question

    Returns: <ast.Subscript object at 0x000001A06D73A950>

    retrieve_documents(self, sub_query, sources)

    Lines 357-389

    Purpose:

    Retrieve documents from sources.
    
    Args:
        sub_query: The query to retrieve for
        sources: Sources to retrieve from
    
    Returns:
        List of retrieved documents
    

    Parameters: – sub_query: Input query or question – sources: Sources

    Returns: <ast.Subscript object at 0x000001A06D606550>

    validate_sources(self, documents)

    Lines 391-443

    Purpose:

    Validate credibility and consistency of retrieved documents.
    
    Args:
        documents: Documents to validate
    
    Returns:
        Validated documents with credibility scores
    

    Parameters: – documents: Documents

    Returns: <ast.Subscript object at 0x000001A06D65A5D0>

    multi_hop_reasoning(self, sub_queries, context)

    Lines 445-558

    Purpose:

    Perform multi-hop reasoning across sub-queries.
    
    Args:
        sub_queries: Sub-queries to reason over
        context: Context from previous reasoning steps
    
    Returns:
        List of reasoning steps
    

    Parameters: – sub_queries: Sub queries – context: Context

    Returns: <ast.Subscript object at 0x000001A06D15DDD0>

    synthesize_answer(self, query, reasoning_steps)

    Lines 560-660

    Purpose:

    Synthesize final answer from reasoning steps.
    
    Args:
        query: Original user query
        reasoning_steps: All reasoning steps
    
    Returns:
        Tuple of (final answer, confidence level, identified gaps)
    

    Parameters: – query: Input query or question – reasoning_steps: Reasoning steps

    Returns: <ast.Subscript object at 0x000001A06D7E6510>

    query(self, user_query)

    Lines 662-707

    Purpose:

    Process a query through the full Agentic RAG pipeline.
    
    Args:
        user_query: The user's question
    
    Returns:
        Complete Agentic RAG result with reasoning chain
    

    Parameters: – user_query: Input query or question

    Returns: AgenticRAGResult


    Functions

    demonstrate_agentic_rag()

    Lines 710-855

    Purpose:

    Demonstrate Agentic RAG pattern with various scenarios.
    

    Key Concepts Summary

    Pattern Overview

    The Agentic Rag pattern provides a structured approach to building AI agents with specific capabilities and behaviors. This implementation leverages LangChain’s composable components to create a robust and maintainable solution.

    Core Components

    1. Agent Class: Main implementation of the pattern logic
    2. Prompt Engineering: Carefully crafted prompts to guide LLM behavior
    3. Tool Integration: Optional tools for extended functionality
    4. Memory Management: State handling across interactions
    5. Output Parsing: Structured response processing

    When to Use

    ✅ Good for: – Scenarios requiring this specific pattern’s strengths – Applications needing structured agent behavior – Systems where explainability is important – Production environments requiring reliability

    ❌ Not ideal for: – Simple single-prompt tasks – Scenarios requiring different pattern characteristics – Resource-constrained environments (if computationally intensive)


    Best Practices Demonstrated

    1. Environment Configuration

    • ✅ Use .env files for API keys
    • ✅ Never hardcode credentials
    • ✅ Validate environment variables on startup

    2. Error Handling

    • ✅ Graceful degradation on failures
    • ✅ Informative error messages
    • ✅ Logging for debugging

    3. Prompt Engineering

    • ✅ Clear, specific instructions
    • ✅ Structured output formats
    • ✅ Few-shot examples when beneficial

    4. Code Organization

    • ✅ Separation of concerns (class-based design)
    • ✅ Type hints for clarity
    • ✅ Docstrings for documentation

    5. Performance Optimization

    • ✅ Appropriate temperature settings
    • ✅ Token budget management
    • ✅ Caching where applicable

    LangChain Components Used

    • ChatOpenAI: OpenAI model interface
    • Chains: Sequential component composition

    Why LangChain? – Composable abstractions for LLM applications – Rich ecosystem of integrations – Production-ready patterns – Active community and development


    Execution Flow

    1. Initialize agent with configuration
       ↓
    2. Load environment and dependencies
       ↓
    3. Prepare prompt with input data
       ↓
    4. Invoke LLM through LangChain
       ↓
    5. Parse and validate response
       ↓
    6. Return formatted result
    

    Key Decision Points: – Model selection (temperature, model type) – Prompt structure (few-shot vs zero-shot) – Error handling strategy – Output format validation


    Performance Considerations

    Computational Cost

    • LLM Calls: Primary cost factor (based on tokens)
    • Latency: Depends on model and input size
    • Memory: Minimal for most patterns

    Optimization Strategies

    1. Prompt Optimization: Minimize tokens while maintaining clarity
    2. Caching: Cache repeated queries/responses
    3. Model Selection: Use smaller models when appropriate
    4. Batch Processing: Group similar requests
    5. Streaming: For long responses, stream tokens

    Cost Estimates (Approximate)

    • GPT-3.5-turbo: ~$0.002 per request
    • GPT-4: ~$0.03 per request
    • GPT-4-turbo: ~$0.01 per request

    Actual costs vary based on input/output token counts


    Common Pitfalls to Avoid

    1. Prompt Design Issues

    ❌ Vague or ambiguous instructions
    ✅ Clear, specific, testable prompts

    2. Error Handling

    ❌ Ignoring API errors or rate limits
    ✅ Implement retry logic and graceful degradation

    3. Token Management

    ❌ Exceeding model token limits
    ✅ Truncate inputs, chunk processing

    4. Security

    ❌ Exposing API keys in code
    ✅ Use environment variables

    5. Testing

    ❌ Only testing happy paths
    ✅ Test edge cases, failures, malformed inputs

    6. Cost Management

    ❌ Unmonitored API usage
    ✅ Implement budgets, alerts, and monitoring


    Further Improvements

    Production Enhancements

    1. Monitoring: Add logging, metrics, tracing
    2. Validation: Input/output schema validation
    3. Rate Limiting: Implement request throttling
    4. Caching: Redis or in-memory cache for responses
    5. Async Support: Non-blocking operations for scalability

    Feature Extensions

    1. Multi-Model Support: Switch between providers
    2. Fallback Models: Graceful degradation strategy
    3. Custom Tools: Domain-specific function calling
    4. Advanced Memory: Vector stores for long-term memory
    5. Evaluation: Automated testing and benchmarking

    Code Quality

    1. Testing: Unit tests, integration tests
    2. Documentation: API docs, usage examples
    3. Type Safety: Stricter type hints
    4. Configuration: Externalized config files

    Related Patterns

    • Pattern 066
    • Pattern 068

    Pattern Families: – Reasoning Patterns (001-011) – Multi-Agent Patterns (012-019) – Tool Use Patterns (020-025) – Memory Patterns (026-031) – And many more…


    Summary

    The Agentic Rag pattern provides a powerful approach to building AI agents with specific capabilities. This implementation demonstrates:

    ✅ Key Strengths: – Clean, maintainable code structure – LangChain integration for composability – Type hints and documentation – Production-ready patterns

    🎯 Best Use Cases: – Applications requiring this pattern’s characteristics – Systems needing structured agent behavior – Production environments requiring reliability

    📚 Learning Resources: – LangChain Documentation: https://python.langchain.com/ – Pattern Documentation: See related patterns – OpenAI API Reference: https://platform.openai.com/docs/


    File: 067_agentic_rag.py
    Total Lines: 860
    Pattern Category: AI Agent Design Pattern #067


    Generated from actual implementation code
    For educational and reference purposes

  • Agentic Design Pattern: Embodied Agent

    
        """
    Pattern 066: Embodied Agent
    
    Description:
        Embodied agents have a physical or virtual body that exists in an environment,
        with sensors for perception and actuators for action. Embodiment provides grounded
        understanding of the world through sensorimotor interaction, spatial reasoning,
        and physical constraints. This pattern bridges abstract reasoning with physical reality.
    
    Components:
        1. Sensory System: Vision, hearing, touch, proprioception
        2. Motor System: Movement, manipulation, locomotion
        3. Body Schema: Internal model of body and capabilities
        4. Spatial Reasoning: Navigation, object relations, distance estimation
        5. Affordance Detection: Understanding what actions are possible
        6. Sensorimotor Integration: Coordinating perception and action
    
    Use Cases:
        - Robotics (physical embodiment)
        - Virtual agents in simulations
        - Embodied AI assistants
        - Game NPCs with physical presence
        - Virtual reality interactions
        - Drone navigation
    
    LangChain Implementation:
        Implements embodied cognition using LLM-based spatial reasoning,
        action planning with physical constraints, and grounded perception.
    """
    
    import os
    import time
    from typing import Dict, Any, List, Optional, Tuple
    from dataclasses import dataclass, field
    from datetime import datetime
    from enum import Enum
    import math
    
    from dotenv import load_dotenv
    from langchain_core.prompts import ChatPromptTemplate
    from langchain_core.output_parsers import StrOutputParser
    from langchain_openai import ChatOpenAI
    
    load_dotenv()
    
    
    class SensorType(Enum):
        """Types of sensors"""
        VISION = "vision"
        HEARING = "hearing"
        TOUCH = "touch"
        PROPRIOCEPTION = "proprioception"  # Body position
        DISTANCE = "distance"  # Range sensor
    
    
    class ActionType(Enum):
        """Types of actions"""
        MOVE = "move"
        TURN = "turn"
        GRASP = "grasp"
        RELEASE = "release"
        LOOK = "look"
        SPEAK = "speak"
    
    
    @dataclass
    class Position:
        """3D position"""
        x: float
        y: float
        z: float = 0.0
        
        def distance_to(self, other: 'Position') -> float:
            """Calculate Euclidean distance"""
            return math.sqrt(
                (self.x - other.x)**2 +
                (self.y - other.y)**2 +
                (self.z - other.z)**2
            )
        
        def __str__(self) -> str:
            if self.z == 0:
                return f"({self.x:.1f}, {self.y:.1f})"
            return f"({self.x:.1f}, {self.y:.1f}, {self.z:.1f})"
    
    
    @dataclass
    class Perception:
        """Sensory perception"""
        sensor: SensorType
        data: Any
        timestamp: float = field(default_factory=time.time)
        confidence: float = 1.0
    
    
    @dataclass
    class PhysicalObject:
        """Object in environment"""
        object_id: str
        name: str
        position: Position
        size: Tuple[float, float, float]  # width, height, depth
        properties: Dict[str, Any] = field(default_factory=dict)
        
        def is_reachable(self, agent_position: Position, reach_distance: float) -> bool:
            """Check if object is within reach"""
            return agent_position.distance_to(self.position)  Perception:
            """Perceive environment through sensors"""
            
            prompt = ChatPromptTemplate.from_messages([
                ("system", f"""You are processing sensory information from a {sensor.value} sensor.
    
    Current body state:
    - Position: {self.body.position}
    - Orientation: {self.body.orientation}°
    
    Extract relevant information from the environment description."""),
                ("user", "Environment: {environment}\n\nWhat do you perceive?")
            ])
            
            chain = prompt | self.cognition | StrOutputParser()
            perception_data = chain.invoke({"environment": environment_description})
            
            perception = Perception(
                sensor=sensor,
                data=perception_data,
                confidence=0.85
            )
            
            self.perceptions.append(perception)
            return perception
        
        def identify_affordances(
            self,
            obj: PhysicalObject
        ) -> List[str]:
            """Identify what actions are possible with object"""
            
            # Check physical constraints
            is_reachable = obj.is_reachable(self.body.position, self.reach_distance)
            
            prompt = ChatPromptTemplate.from_messages([
                ("system", """Identify affordances (possible actions) for this object.
    
    Affordances are action possibilities based on physical properties.
    
    Consider:
    1. Object properties
    2. Physical reachability
    3. Agent capabilities
    4. Current state"""),
                ("user", """Object: {name}
    Size: {size}
    Properties: {properties}
    Reachable: {reachable}
    Agent holding: {holding}
    
    What actions are possible?""")
            ])
            
            chain = prompt | self.spatial_reasoner | StrOutputParser()
            affordances_text = chain.invoke({
                "name": obj.name,
                "size": f"{obj.size[0]}×{obj.size[1]}×{obj.size[2]}",
                "properties": str(obj.properties),
                "reachable": "Yes" if is_reachable else "No",
                "holding": self.body.holding_object or "Nothing"
            })
            
            # Parse affordances
            affordances = []
            for line in affordances_text.split('\n'):
                line = line.strip()
                if line and (line.startswith('-') or line.startswith('•')):
                    action = line.lstrip('-•').strip()
                    if action:
                        affordances.append(action)
            
            return affordances
        
        def plan_navigation(
            self,
            target_position: Position,
            obstacles: List[PhysicalObject]
        ) -> List[Dict[str, Any]]:
            """Plan navigation path considering obstacles"""
            
            current = self.body.position
            distance = current.distance_to(target_position)
            
            # Build obstacle description
            obstacle_desc = ""
            if obstacles:
                obstacle_desc = "Obstacles:\n"
                for obs in obstacles:
                    dist = current.distance_to(obs.position)
                    obstacle_desc += f"- {obs.name} at {obs.position} (distance: {dist:.1f}m)\n"
            
            prompt = ChatPromptTemplate.from_messages([
                ("system", """Plan a navigation path from current position to target.
    
    Consider:
    1. Obstacles to avoid
    2. Efficient path
    3. Physical constraints (can't move through walls)
    
    Provide step-by-step movement plan."""),
                ("user", """Current position: {current}
    Target position: {target}
    Distance: {distance:.1f}m
    
    {obstacles}
    
    Navigation plan:""")
            ])
            
            chain = prompt | self.spatial_reasoner | StrOutputParser()
            plan_text = chain.invoke({
                "current": str(current),
                "target": str(target_position),
                "distance": distance,
                "obstacles": obstacle_desc
            })
            
            # Parse plan into actions
            actions = []
            for i, line in enumerate(plan_text.split('\n'), 1):
                line = line.strip()
                if line and (line[0].isdigit() or line.startswith('-')):
                    action_text = line.lstrip('0123456789.-) ').strip()
                    if action_text:
                        actions.append({
                            "step": i,
                            "action": "move",
                            "description": action_text
                        })
            
            return actions
        
        def execute_action(
            self,
            action_type: ActionType,
            parameters: Dict[str, Any]
        ) -> Dict[str, Any]:
            """Execute physical action"""
            
            result = {
                "action": action_type.value,
                "success": False,
                "message": ""
            }
            
            if action_type == ActionType.MOVE:
                # Simulate movement
                target = parameters.get("target_position")
                if target:
                    distance = self.body.position.distance_to(target)
                    time_needed = distance / self.move_speed
                    
                    # Simple movement (in reality would be more complex)
                    self.body.position = target
                    self.body.energy_level -= 0.1 * distance
                    
                    result["success"] = True
                    result["message"] = f"Moved to {target} ({time_needed:.1f}s)"
            
            elif action_type == ActionType.GRASP:
                object_id = parameters.get("object_id")
                if object_id in self.perceived_objects:
                    obj = self.perceived_objects[object_id]
                    
                    if obj.is_reachable(self.body.position, self.reach_distance):
                        if not self.body.holding_object:
                            self.body.holding_object = object_id
                            result["success"] = True
                            result["message"] = f"Grasped {obj.name}"
                        else:
                            result["message"] = f"Already holding {self.body.holding_object}"
                    else:
                        result["message"] = f"{obj.name} is out of reach"
                else:
                    result["message"] = "Object not found"
            
            elif action_type == ActionType.RELEASE:
                if self.body.holding_object:
                    released = self.body.holding_object
                    self.body.holding_object = None
                    result["success"] = True
                    result["message"] = f"Released {released}"
                else:
                    result["message"] = "Not holding anything"
            
            return result
        
        def ground_understanding(
            self,
            abstract_instruction: str
        ) -> str:
            """Ground abstract instruction in physical terms"""
            
            prompt = ChatPromptTemplate.from_messages([
                ("system", f"""Translate abstract instruction into concrete physical actions.
    
    Current embodied state:
    - Position: {self.body.position}
    - Holding: {self.body.holding_object or 'nothing'}
    - Energy: {self.body.energy_level:.0%}
    
    Known objects:
    {self._get_objects_description()}
    
    Provide specific physical actions with spatial details."""),
                ("user", "Instruction: {instruction}\n\nPhysical actions:")
            ])
            
            chain = prompt | self.cognition | StrOutputParser()
            return chain.invoke({"instruction": abstract_instruction})
        
        def _get_objects_description(self) -> str:
            """Get description of perceived objects"""
            if not self.perceived_objects:
                return "No objects perceived yet"
            
            desc = ""
            for obj in self.perceived_objects.values():
                dist = self.body.position.distance_to(obj.position)
                desc += f"- {obj.name} at {obj.position} (distance: {dist:.1f}m)\n"
            return desc
    
    
    def demonstrate_embodied_agent():
        """Demonstrate Embodied Agent pattern"""
        
        print("=" * 80)
        print("PATTERN 066: EMBODIED AGENT DEMONSTRATION")
        print("=" * 80)
        print("\nPhysical embodiment: sensors, actuators, spatial reasoning\n")
        
        # Test 1: Perception and spatial awareness
        print("\n" + "=" * 80)
        print("TEST 1: Perception and Spatial Awareness")
        print("=" * 80)
        
        agent = EmbodiedAgent(
            initial_position=Position(0, 0, 0),
            reach_distance=1.5,
            move_speed=1.0
        )
        
        print(f"\n🤖 Agent initialized at position {agent.body.position}")
        
        # Perceive environment
        environment = """
        You are in a room. To your left (at -2, 0) is a table with a cup on it.
        Straight ahead (at 0, 3) is a door. To your right (at 2, 0) is a chair.
        """
        
        print(f"\n👁️  Perceiving environment...")
        perception = agent.perceive(SensorType.VISION, environment)
        print(f"   Sensor: {perception.sensor.value}")
        print(f"   Perceived: {perception.data[:150]}...")
        
        # Add objects to agent's knowledge
        agent.perceived_objects["table"] = PhysicalObject(
            "table", "table", Position(-2, 0, 0), (1.0, 0.8, 0.1)
        )
        agent.perceived_objects["cup"] = PhysicalObject(
            "cup", "cup", Position(-2, 0, 0.8), (0.1, 0.15, 0.1),
            properties={"containable": True, "graspable": True}
        )
        agent.perceived_objects["door"] = PhysicalObject(
            "door", "door", Position(0, 3, 0), (1.0, 2.0, 0.1)
        )
        
        print(f"\n📍 Objects in environment:")
        for obj_id, obj in agent.perceived_objects.items():
            distance = agent.body.position.distance_to(obj.position)
            reachable = "✓" if distance  5:
            print(f"      ... ({len(nav_plan) - 5} more steps)")
        
        # Test 4: Action execution
        print("\n" + "=" * 80)
        print("TEST 4: Physical Action Execution")
        print("=" * 80)
        
        print(f"\n🎮 Executing actions:\n")
        
        # Move to table
        print(f"   Action 1: Move to table")
        result1 = agent.execute_action(
            ActionType.MOVE,
            {"target_position": Position(-2, 0, 0)}
        )
        print(f"      Result: {result1['message']}")
        print(f"      New position: {agent.body.position}")
        
        # Try to grasp cup
        print(f"\n   Action 2: Grasp cup")
        result2 = agent.execute_action(
            ActionType.GRASP,
            {"object_id": "cup"}
        )
        print(f"      Result: {result2['message']}")
        print(f"      Holding: {agent.body.holding_object}")
        
        # Try to grasp something else (should fail)
        print(f"\n   Action 3: Try to grasp door (should fail)")
        result3 = agent.execute_action(
            ActionType.GRASP,
            {"object_id": "door"}
        )
        print(f"      Result: {result3['message']}")
        
        # Release cup
        print(f"\n   Action 4: Release cup")
        result4 = agent.execute_action(ActionType.RELEASE, {})
        print(f"      Result: {result4['message']}")
        print(f"      Holding: {agent.body.holding_object or 'nothing'}")
        
        # Test 5: Grounding abstract instructions
        print("\n" + "=" * 80)
        print("TEST 5: Grounding Abstract Instructions")
        print("=" * 80)
        
        agent2 = EmbodiedAgent(initial_position=Position(0, 0, 0))
        agent2.perceived_objects = {
            "book": PhysicalObject("book", "book", Position(1, 1, 0.5), (0.2, 0.3, 0.05)),
            "shelf": PhysicalObject("shelf", "shelf", Position(3, 0, 1.0), (1.0, 2.0, 0.3))
        }
        
        instructions = [
            "Get the book",
            "Put it on the shelf",
            "Return to starting position"
        ]
        
        print(f"\n📝 Grounding abstract instructions in physical terms:\n")
        
        for instruction in instructions:
            print(f"   Abstract: '{instruction}'")
            grounded = agent2.ground_understanding(instruction)
            print(f"   Grounded:")
            for line in grounded.split('\n')[:3]:
                if line.strip():
                    print(f"      {line.strip()}")
            print()
        
        # Summary
        print("\n" + "=" * 80)
        print("EMBODIED AGENT PATTERN SUMMARY")
        print("=" * 80)
        print("""
    Key Benefits:
    1. Grounded Understanding: Physical basis for abstract concepts
    2. Spatial Reasoning: Navigate and manipulate in 3D space
    3. Action Constraints: Realistic physical limitations
    4. Sensorimotor Integration: Coordinated perception-action
    5. Natural Interaction: Physical presence enables natural tasks
    
    Embodiment Components:
    
    1. Sensory System:
       - Vision: Object detection, scene understanding
       - Hearing: Sound localization, speech
       - Touch: Contact, texture, force
       - Proprioception: Body position, posture
       - Distance: Range sensing, depth
    
    2. Motor System:
       - Locomotion: Walking, rolling, flying
       - Manipulation: Grasping, moving objects
       - Gestures: Pointing, waving
       - Facial expressions (if humanoid)
       - Tool use
    
    3. Body Schema:
       - Size and shape awareness
       - Reach distance
       - Movement capabilities
       - Center of mass
       - Degrees of freedom
    
    4. Spatial Representation:
       - Egocentric: Relative to body ("to my left")
       - Allocentric: World-centered ("north of the tree")
       - Route knowledge: Paths between places
       - Survey knowledge: Map-like representation
    
    5. Affordances:
       - Graspable: Can be picked up
       - Sittable: Can sit on
       - Pushable: Can be moved
       - Openable: Can be opened
       - Context-dependent
    
    6. Physical Constraints:
       - Reachability: Within arm's reach
       - Traversability: Can move through
       - Stability: Won't fall over
       - Energy: Battery/fuel limitations
       - Payload: Weight capacity
    
    Grounding Language in Action:
    - "Pick up": Approach + reach + grasp + lift
    - "Put on": Move + position + align + release
    - "Go to": Plan path + navigate + avoid obstacles
    - "Look at": Turn head/body + focus vision
    
    Use Cases:
    - Physical Robotics:
      - Warehouse robots
      - Service robots
      - Manufacturing robots
      - Exploration rovers
    
    - Virtual Embodiment:
      - Game characters
      - Virtual assistants with avatars
      - Simulation agents
      - Training environments
    
    - Human-Robot Interaction:
      - Collaborative robots (cobots)
      - Social robots
      - Assistive robots
      - Telepresence
    
    Challenges:
    1. Sensor Noise: Imperfect perception
    2. Motor Control: Precise actuation
    3. Real-time Processing: Fast reactions needed
    4. Sim-to-Real Transfer: Simulations differ from reality
    5. Safety: Physical harm risks
    6. Complexity: Many degrees of freedom
    
    Best Practices:
    1. Start with simplified sensing
    2. Use world models for planning
    3. Implement safety constraints
    4. Test in simulation first
    5. Incremental complexity
    6. Fallback behaviors
    7. Human oversight for learning
    
    Production Considerations:
    - Sensor calibration and maintenance
    - Safety zones and limits
    - Emergency stop mechanisms
    - Predictable failure modes
    - Logging for debugging
    - Remote monitoring
    - Regular testing
    
    Spatial Reasoning Tasks:
    - Object localization
    - Path planning
    - Obstacle avoidance
    - Reachability analysis
    - Collision detection
    - Distance estimation
    - Orientation tracking
    
    Integration with AI:
    - Vision: Computer vision, object detection
    - Planning: Motion planning, SLAM
    - Control: PID, MPC, RL
    - Learning: Imitation learning, RL
    - Language: Instruction grounding
    
    Comparison with Related Patterns:
    - vs. Abstract Agent: Physical vs symbolic
    - vs. Disembodied: Grounded vs abstract
    - vs. Simulated: Real physics vs approximation
    - vs. Teleoperated: Autonomous vs controlled
    
    Advanced Techniques:
    1. Visual Servoing: Vision-guided control
    2. Force Control: Manipulation with force feedback
    3. Dynamic Balance: Maintaining stability
    4. Compliant Motion: Adaptive to contact
    5. Multi-Modal Fusion: Combine sensor types
    
    Embodied Cognition Theory:
    - Mind is shaped by body
    - Cognition is for action
    - Environment as external memory
    - Situated activity
    - Sensorimotor contingencies
    
    Research Directions:
    - Sim-to-real transfer
    - Learning from demonstration
    - Embodied language understanding
    - Social embodiment
    - Morphological computation
    
    Design Principles:
    1. Sense-Plan-Act cycle
    2. Reactive behaviors for safety
    3. Deliberative planning for complex tasks
    4. Hierarchical control
    5. Graceful degradation
    
    Testing Approaches:
    - Unit tests for each component
    - Integration tests for behaviors
    - Simulation before deployment
    - Staged real-world testing
    - Safety-critical validation
    
    The Embodied Agent pattern grounds AI systems in physical
    reality through sensors, actuators, and spatial reasoning,
    enabling natural interaction with the physical world.
    """)
    
    
    if __name__ == "__main__":
        demonstrate_embodied_agent()
    
        

    Agentic Design Pattern: Embodied Agent

    Pattern 066: Embodied Agent – Line-by-Line Explanation

    Overview

    This document provides a detailed line-by-line explanation of the Embodied Agent pattern implementation.


    Pattern Description

    Pattern 066: Embodied Agent

    Description: Embodied agents have a physical or virtual body that exists in an environment, with sensors for perception and actuators for action. Embodiment provides grounded understanding of the world through sensorimotor interaction, spatial reasoning, and physical constraints. This pattern bridges abstract reasoning with physical reality.

    Components: 1. Sensory System: Vision, hearing, touch, proprioception 2. Motor System: Movement, manipulation, locomotion 3. Body Schema: Internal model of body and capabilities 4. Spatial Reasoning: Navigation, object relations, distance estimation 5. Affordance Detection: Understanding what actions are possible 6. Sensorimotor Integration: Coordinating perception and action

    Use Cases: – Robotics (physical embodiment) – Virtual agents in simulations – Embodied AI assistants – Game NPCs with physical presence – Virtual reality interactions – Drone navigation

    LangChain Implementation: Implements embodied cognition using LLM-based spatial reasoning, action planning with physical constraints, and grounded perception.


    File Statistics

    • Total Lines: 712
    • Classes: 7
    • Functions: 1
    • Constants: 0

    Module Documentation

    """
    Pattern 066: Embodied Agent
    
    Description:
        Embodied agents have a physical or virtual body that exists in an environment,
        with sensors for perception and actuators for action. Embodiment provides grounded
        understanding of the world through sensorimotor interaction, spatial reasoning,
        and physical constraints. This pattern bridges abstract reasoning with physical reality.
    
    Components:
        1. Sensory System: Vision, hearing, touch, proprioception
        2. Motor System: Movement, manipulation, locomotion
        3. Body Schema: Internal model of body and capabilities
        4. Spatial Reasoning: Navigation, object relations, distance estimation
        5. Affordance Detection: Understanding what actions are possible
        6. Sensorimotor Integration: Coordinating perception and action
    
    Use Cases:
        - Robotics (physical embodiment)
        - Virtual agents in simulations
        - Embodied AI assistants
        - Game NPCs with physical presence
        - Virtual reality interactions
        - Drone navigation
    
    LangChain Implementation:
        Implements embodied cognition using LLM-based spatial reasoning,
        action planning with physical constraints, and grounded perception.
    """
    

    Purpose: This module implements the Embodied Agent pattern, providing a structured approach to agent design and implementation.


    Import Statements

    Standard Library Imports

    import dataclasses.dataclass
    import dataclasses.field
    import datetime.datetime
    import enum.Enum
    import langchain_core.output_parsers.StrOutputParser
    import langchain_core.prompts.ChatPromptTemplate
    import os
    import time
    import typing.Any
    import typing.Dict
    import typing.List
    import typing.Optional
    import typing.Tuple
    

    Purpose: Core Python functionality: – dataclasses.dataclass: Simplified class definitions with @dataclass – dataclasses.field: Simplified class definitions with @dataclass – datetime.datetime: Date and time handling – enum.Enum: Enumeration support – langchain_core.output_parsers.StrOutputParser: Regular expression operations – langchain_core.prompts.ChatPromptTemplate: Regular expression operations – os: Environment variable access and system operations – time: Time-related functions – typing.Any: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Dict: Type hints for better code documentation (List, Dict, Any, Optional) – typing.List: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Optional: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Tuple: Type hints for better code documentation (List, Dict, Any, Optional)

    LangChain Imports

    from langchain_openai.ChatOpenAI
    

    Purpose: LangChain framework components: – LLM Interface: Wrapper for OpenAI models – Chains: Composable building blocks for complex workflows

    Other Imports

    from dotenv.load_dotenv
    from math
    

    Purpose: Additional dependencies for specialized functionality.


    Classes

    Class: SensorType

    Lines 47-53

    Inherits from: Enum

    Purpose:

    Types of sensors
    

    Class: ActionType

    Lines 56-63

    Inherits from: Enum

    Purpose:

    Types of actions
    

    Class: Position

    Lines 67-84

    Purpose:

    3D position
    

    Methods:

    distance_to(self, other)

    Lines 73-79

    Purpose:

    Calculate Euclidean distance
    

    Parameters: – other: Other

    Returns: float

    __str__(self)

    Lines 81-84

    Purpose: Str operation.

    Parameters:

    Returns: str


    Class: Perception

    Lines 88-93

    Purpose:

    Sensory perception
    

    Class: PhysicalObject

    Lines 97-107

    Purpose:

    Object in environment
    

    Methods:

    is_reachable(self, agent_position, reach_distance)

    Lines 105-107

    Purpose:

    Check if object is within reach
    

    Parameters: – agent_position: Agent position – reach_distance: Reach distance

    Returns: bool


    Class: BodyState

    Lines 111-117

    Purpose:

    State of agent's body
    

    Class: EmbodiedAgent

    Lines 120-383

    Purpose:

    Embodied agent with physical presence.
    
    Features:
    1. Spatial awareness and navigation
    2. Object interaction with constraints
    3. Grounded perception
    4. Physical action planning
    5. Affordance recognition
    

    Methods:

    __init__(self, initial_position, reach_distance, move_speed)

    Lines 132-156

    Purpose: Initialize the class instance with required parameters.

    Parameters: – initial_position: Initial position – reach_distance: Reach distance – move_speed: Move speed

    perceive(self, sensor, environment_description)

    Lines 158-186

    Purpose:

    Perceive environment through sensors
    

    Parameters: – sensor: Sensor – environment_description: Environment description

    Returns: Perception

    identify_affordances(self, obj)

    Lines 188-234

    Purpose:

    Identify what actions are possible with object
    

    Parameters: – obj: Obj

    Returns: <ast.Subscript object at 0x000001A06D660BD0>

    plan_navigation(self, target_position, obstacles)

    Lines 236-293

    Purpose:

    Plan navigation path considering obstacles
    

    Parameters: – target_position: Target position – obstacles: Obstacles

    Returns: <ast.Subscript object at 0x000001A06D16F510>

    execute_action(self, action_type, parameters)

    Lines 295-348

    Purpose:

    Execute physical action
    

    Parameters: – action_type: Action type – parameters: Parameters

    Returns: <ast.Subscript object at 0x000001A06D6A9C50>

    ground_understanding(self, abstract_instruction)

    Lines 350-372

    Purpose:

    Ground abstract instruction in physical terms
    

    Parameters: – abstract_instruction: Abstract instruction

    Returns: str

    _get_objects_description(self)

    Lines 374-383

    Purpose:

    Get description of perceived objects
    

    Parameters:

    Returns: str


    Functions

    demonstrate_embodied_agent()

    Lines 386-707

    Purpose:

    Demonstrate Embodied Agent pattern
    

    Key Concepts Summary

    Pattern Overview

    The Embodied Agent pattern provides a structured approach to building AI agents with specific capabilities and behaviors. This implementation leverages LangChain’s composable components to create a robust and maintainable solution.

    Core Components

    1. Agent Class: Main implementation of the pattern logic
    2. Prompt Engineering: Carefully crafted prompts to guide LLM behavior
    3. Tool Integration: Optional tools for extended functionality
    4. Memory Management: State handling across interactions
    5. Output Parsing: Structured response processing

    When to Use

    ✅ Good for: – Scenarios requiring this specific pattern’s strengths – Applications needing structured agent behavior – Systems where explainability is important – Production environments requiring reliability

    ❌ Not ideal for: – Simple single-prompt tasks – Scenarios requiring different pattern characteristics – Resource-constrained environments (if computationally intensive)


    Best Practices Demonstrated

    1. Environment Configuration

    • ✅ Use .env files for API keys
    • ✅ Never hardcode credentials
    • ✅ Validate environment variables on startup

    2. Error Handling

    • ✅ Graceful degradation on failures
    • ✅ Informative error messages
    • ✅ Logging for debugging

    3. Prompt Engineering

    • ✅ Clear, specific instructions
    • ✅ Structured output formats
    • ✅ Few-shot examples when beneficial

    4. Code Organization

    • ✅ Separation of concerns (class-based design)
    • ✅ Type hints for clarity
    • ✅ Docstrings for documentation

    5. Performance Optimization

    • ✅ Appropriate temperature settings
    • ✅ Token budget management
    • ✅ Caching where applicable

    LangChain Components Used

    • ChatOpenAI: OpenAI model interface
    • Chains: Sequential component composition

    Why LangChain? – Composable abstractions for LLM applications – Rich ecosystem of integrations – Production-ready patterns – Active community and development


    Execution Flow

    1. Initialize agent with configuration
       ↓
    2. Load environment and dependencies
       ↓
    3. Prepare prompt with input data
       ↓
    4. Invoke LLM through LangChain
       ↓
    5. Parse and validate response
       ↓
    6. Return formatted result
    

    Key Decision Points: – Model selection (temperature, model type) – Prompt structure (few-shot vs zero-shot) – Error handling strategy – Output format validation


    Performance Considerations

    Computational Cost

    • LLM Calls: Primary cost factor (based on tokens)
    • Latency: Depends on model and input size
    • Memory: Minimal for most patterns

    Optimization Strategies

    1. Prompt Optimization: Minimize tokens while maintaining clarity
    2. Caching: Cache repeated queries/responses
    3. Model Selection: Use smaller models when appropriate
    4. Batch Processing: Group similar requests
    5. Streaming: For long responses, stream tokens

    Cost Estimates (Approximate)

    • GPT-3.5-turbo: ~$0.002 per request
    • GPT-4: ~$0.03 per request
    • GPT-4-turbo: ~$0.01 per request

    Actual costs vary based on input/output token counts


    Common Pitfalls to Avoid

    1. Prompt Design Issues

    ❌ Vague or ambiguous instructions
    ✅ Clear, specific, testable prompts

    2. Error Handling

    ❌ Ignoring API errors or rate limits
    ✅ Implement retry logic and graceful degradation

    3. Token Management

    ❌ Exceeding model token limits
    ✅ Truncate inputs, chunk processing

    4. Security

    ❌ Exposing API keys in code
    ✅ Use environment variables

    5. Testing

    ❌ Only testing happy paths
    ✅ Test edge cases, failures, malformed inputs

    6. Cost Management

    ❌ Unmonitored API usage
    ✅ Implement budgets, alerts, and monitoring


    Further Improvements

    Production Enhancements

    1. Monitoring: Add logging, metrics, tracing
    2. Validation: Input/output schema validation
    3. Rate Limiting: Implement request throttling
    4. Caching: Redis or in-memory cache for responses
    5. Async Support: Non-blocking operations for scalability

    Feature Extensions

    1. Multi-Model Support: Switch between providers
    2. Fallback Models: Graceful degradation strategy
    3. Custom Tools: Domain-specific function calling
    4. Advanced Memory: Vector stores for long-term memory
    5. Evaluation: Automated testing and benchmarking

    Code Quality

    1. Testing: Unit tests, integration tests
    2. Documentation: API docs, usage examples
    3. Type Safety: Stricter type hints
    4. Configuration: Externalized config files

    Related Patterns

    • Pattern 065
    • Pattern 067

    Pattern Families: – Reasoning Patterns (001-011) – Multi-Agent Patterns (012-019) – Tool Use Patterns (020-025) – Memory Patterns (026-031) – And many more…


    Summary

    The Embodied Agent pattern provides a powerful approach to building AI agents with specific capabilities. This implementation demonstrates:

    ✅ Key Strengths: – Clean, maintainable code structure – LangChain integration for composability – Type hints and documentation – Production-ready patterns

    🎯 Best Use Cases: – Applications requiring this pattern’s characteristics – Systems needing structured agent behavior – Production environments requiring reliability

    📚 Learning Resources: – LangChain Documentation: https://python.langchain.com/ – Pattern Documentation: See related patterns – OpenAI API Reference: https://platform.openai.com/docs/


    File: 066_embodied_agent.py
    Total Lines: 712
    Pattern Category: AI Agent Design Pattern #066


    Generated from actual implementation code
    For educational and reference purposes