"""
Pattern 098: Shared Context/Workspace

Description:
    Shared Context/Workspace provides a common space where multiple agents can read and write
    shared information, coordinate activities, and collaborate on tasks. This pattern enables
    agents to maintain shared state, exchange data without direct messaging, and build upon
    each other's work. The shared workspace acts as a collaborative knowledge base and
    coordination medium for multi-agent systems.

    Shared workspace features:
    - Common data structures accessible to all agents
    - Read/write operations with concurrency control
    - Version tracking and conflict resolution
    - Notifications on changes
    - Access control and permissions
    - Transaction support

Components:
    1. Workspace
       - Shared data store
       - Key-value storage
       - Document/object storage
       - Access control lists
       - Change notifications
       - Version history

    2. Context Manager
       - Manages workspace state
       - Handles concurrent access
       - Resolves conflicts
       - Maintains consistency
       - Provides transactions
       - Enforces policies

    3. Agent Interface
       - Read operations
       - Write operations
       - Subscribe to changes
       - Lock/unlock resources
       - Query workspace
       - Batch operations

    4. Synchronization
       - Optimistic locking
       - Pessimistic locking
       - Version control
       - Merge strategies
       - Conflict detection
       - Resolution policies

Use Cases:
    1. Collaborative Problem Solving
       - Shared scratchpad
       - Intermediate results
       - Partial solutions
       - Knowledge accumulation
       - Iterative refinement
       - Collective intelligence

    2. Multi-Agent Coordination
       - Task assignments
       - Status tracking
       - Resource allocation
       - Progress monitoring
       - Goal management
       - Constraint sharing

    3. Knowledge Sharing
       - Common knowledge base
       - Learned patterns
       - Best practices
       - Historical data
       - Shared context
       - Collective memory

    4. Workflow Management
       - Process state
       - Task dependencies
       - Input/output data
       - Checkpoint saving
       - Error recovery
       - Audit trails

LangChain Implementation:
    LangChain supports shared context through:
    - Memory objects shared across agents
    - Vector stores for shared knowledge
    - Custom state management
    - LangGraph for workflow state
    - Integration with databases

Key Features:
    1. Concurrent Access
       - Multiple readers
       - Controlled writers
       - Lock management
       - Deadlock prevention
       - Fair scheduling
       - Priority access

    2. Consistency Models
       - Strong consistency
       - Eventual consistency
       - Causal consistency
       - Read-your-writes
       - Monotonic reads
       - Application-specific

    3. Change Management
       - Version tracking
       - Change logs
       - Rollback capability
       - Diff computation
       - Merge operations
       - Conflict resolution

    4. Scalability
       - Partitioning
       - Replication
       - Caching
       - Lazy loading
       - Efficient indexing
       - Distributed storage

Best Practices:
    1. Data Organization
       - Clear naming conventions
       - Logical structure
       - Minimize shared state
       - Immutable where possible
       - Document schemas
       - Version data formats

    2. Concurrency Control
       - Use appropriate locking
       - Keep critical sections small
       - Avoid long-held locks
       - Handle deadlocks
       - Implement timeouts
       - Test concurrent scenarios

    3. Performance
       - Cache frequently read data
       - Batch operations
       - Async notifications
       - Index for queries
       - Minimize contention
       - Monitor bottlenecks

    4. Reliability
       - Persist important data
       - Backup regularly
       - Handle failures gracefully
       - Validate writes
       - Log operations
       - Support recovery

Trade-offs:
    Advantages:
    - Simple coordination
    - Shared visibility
    - Flexible collaboration
    - Natural data sharing
    - Easy debugging
    - Common patterns

    Disadvantages:
    - Potential bottleneck
    - Concurrency complexity
    - Consistency challenges
    - Scalability limits
    - Testing difficulty
    - State management overhead

Production Considerations:
    1. Storage Backend
       - Redis: Fast, in-memory
       - MongoDB: Document store
       - PostgreSQL: Relational, ACID
       - DynamoDB: Managed, scalable
       - Etcd: Distributed coordination
       - Memcached: Simple caching

    2. Consistency Strategy
       - ACID transactions for critical data
       - Eventual consistency for non-critical
       - CRDTs for conflict-free updates
       - Last-write-wins for simple cases
       - Application-level resolution
       - Version vectors for causality

    3. Performance Optimization
       - Read-through caching
       - Write-behind buffering
       - Optimistic locking
       - Sharding by key
       - Denormalization
       - Materialized views

    4. Monitoring
       - Access patterns
       - Lock contention
       - Query performance
       - Storage usage
       - Conflict rate
       - Error frequency

    5. Security
       - Authentication
       - Authorization (ACLs)
       - Encryption at rest
       - Audit logging
       - Input validation
       - Rate limiting
"""

import os
import json
import time
import threading
from typing import List, Dict, Any, Optional, Callable, Set
from dataclasses import dataclass, field
from datetime import datetime
from enum import Enum
from collections import defaultdict
from copy import deepcopy
from dotenv import load_dotenv

from langchain_core.prompts import ChatPromptTemplate
from langchain_openai import ChatOpenAI
from langchain_core.output_parsers import StrOutputParser

load_dotenv()


class AccessMode(Enum):
    """Access modes for workspace resources"""
    READ = "read"
    WRITE = "write"
    READ_WRITE = "read_write"


@dataclass
class WorkspaceEntry:
    """Entry in the shared workspace"""
    key: str
    value: Any
    version: int
    created_at: datetime
    updated_at: datetime
    created_by: str
    updated_by: str
    metadata: Dict[str, Any] = field(default_factory=dict)


@dataclass
class WorkspaceEvent:
    """Event notification for workspace changes"""
    event_type: str  # created, updated, deleted
    key: str
    value: Any
    agent_id: str
    timestamp: datetime


class SharedWorkspace:
    """
    Shared workspace for multi-agent collaboration.
    
    Provides concurrent access with locking, versioning, and change notifications.
    """
    
    def __init__(self):
        """Initialize shared workspace"""
        self.data: Dict[str, WorkspaceEntry] = {}
        self.locks: Dict[str, str] = {}  # key -> agent_id holding lock
        self.subscribers: Dict[str, Set[Callable]] = defaultdict(set)  # key -> callbacks
        self.history: List[WorkspaceEvent] = []
        self.lock = threading.RLock()  # Reentrant lock for thread safety
    
    def read(self, key: str, default: Any = None) -> Any:
        """
        Read value from workspace.
        
        Args:
            key: Key to read
            default: Default value if key doesn't exist
            
        Returns:
            Value or default
        """
        with self.lock:
            entry = self.data.get(key)
            if entry:
                return deepcopy(entry.value)
            return default
    
    def write(self, key: str, value: Any, agent_id: str, metadata: Optional[Dict[str, Any]] = None) -> bool:
        """
        Write value to workspace.
        
        Args:
            key: Key to write
            value: Value to write
            agent_id: Agent performing write
            metadata: Optional metadata
            
        Returns:
            True if successful
        """
        with self.lock:
            now = datetime.now()
            
            if key in self.data:
                # Update existing
                entry = self.data[key]
                entry.value = value
                entry.version += 1
                entry.updated_at = now
                entry.updated_by = agent_id
                if metadata:
                    entry.metadata.update(metadata)
                event_type = "updated"
            else:
                # Create new
                entry = WorkspaceEntry(
                    key=key,
                    value=value,
                    version=1,
                    created_at=now,
                    updated_at=now,
                    created_by=agent_id,
                    updated_by=agent_id,
                    metadata=metadata or {}
                )
                self.data[key] = entry
                event_type = "created"
            
            # Record event
            event = WorkspaceEvent(
                event_type=event_type,
                key=key,
                value=value,
                agent_id=agent_id,
                timestamp=now
            )
            self.history.append(event)
            
            # Notify subscribers
            self._notify_subscribers(key, event)
            
            return True
    
    def delete(self, key: str, agent_id: str) -> bool:
        """
        Delete value from workspace.
        
        Args:
            key: Key to delete
            agent_id: Agent performing deletion
            
        Returns:
            True if deleted, False if not found
        """
        with self.lock:
            if key in self.data:
                value = self.data[key].value
                del self.data[key]
                
                # Record event
                event = WorkspaceEvent(
                    event_type="deleted",
                    key=key,
                    value=value,
                    agent_id=agent_id,
                    timestamp=datetime.now()
                )
                self.history.append(event)
                
                # Notify subscribers
                self._notify_subscribers(key, event)
                
                return True
            return False
    
    def update(self, key: str, updater: Callable[[Any], Any], agent_id: str) -> bool:
        """
        Atomically update value using updater function.
        
        Args:
            key: Key to update
            updater: Function that takes current value and returns new value
            agent_id: Agent performing update
            
        Returns:
            True if successful
        """
        with self.lock:
            current = self.read(key)
            new_value = updater(current)
            return self.write(key, new_value, agent_id)
    
    def lock_key(self, key: str, agent_id: str, timeout: float = 5.0) -> bool:
        """
        Acquire exclusive lock on key.
        
        Args:
            key: Key to lock
            agent_id: Agent requesting lock
            timeout: Maximum time to wait for lock
            
        Returns:
            True if lock acquired
        """
        start_time = time.time()
        
        while time.time() - start_time  bool:
        """
        Release lock on key.
        
        Args:
            key: Key to unlock
            agent_id: Agent releasing lock
            
        Returns:
            True if unlocked
        """
        with self.lock:
            if key in self.locks and self.locks[key] == agent_id:
                del self.locks[key]
                return True
            return False
    
    def subscribe(self, key: str, callback: Callable[[WorkspaceEvent], None]):
        """
        Subscribe to changes on a key.
        
        Args:
            key: Key to watch
            callback: Function to call on changes
        """
        with self.lock:
            self.subscribers[key].add(callback)
    
    def unsubscribe(self, key: str, callback: Callable[[WorkspaceEvent], None]):
        """
        Unsubscribe from changes on a key.
        
        Args:
            key: Key to stop watching
            callback: Callback to remove
        """
        with self.lock:
            if key in self.subscribers:
                self.subscribers[key].discard(callback)
    
    def _notify_subscribers(self, key: str, event: WorkspaceEvent):
        """
        Notify subscribers of change.
        
        Args:
            key: Key that changed
            event: Event details
        """
        # Notify exact key subscribers
        for callback in self.subscribers.get(key, []):
            try:
                callback(event)
            except Exception as e:
                print(f"Error in subscriber callback: {e}")
        
        # Notify wildcard subscribers
        for callback in self.subscribers.get("*", []):
            try:
                callback(event)
            except Exception as e:
                print(f"Error in wildcard subscriber callback: {e}")
    
    def get_all_keys(self) -> List[str]:
        """Get all keys in workspace"""
        with self.lock:
            return list(self.data.keys())
    
    def get_info(self, key: str) -> Optional[Dict[str, Any]]:
        """
        Get metadata about a key.
        
        Args:
            key: Key to get info for
            
        Returns:
            Info dictionary or None
        """
        with self.lock:
            if key in self.data:
                entry = self.data[key]
                return {
                    'key': entry.key,
                    'version': entry.version,
                    'created_at': entry.created_at.isoformat(),
                    'updated_at': entry.updated_at.isoformat(),
                    'created_by': entry.created_by,
                    'updated_by': entry.updated_by,
                    'metadata': entry.metadata,
                    'is_locked': key in self.locks,
                    'locked_by': self.locks.get(key)
                }
            return None
    
    def get_history(self, key: Optional[str] = None, limit: int = 10) -> List[WorkspaceEvent]:
        """
        Get history of changes.
        
        Args:
            key: Filter by key (None for all)
            limit: Maximum events to return
            
        Returns:
            List of events
        """
        with self.lock:
            if key:
                events = [e for e in self.history if e.key == key]
            else:
                events = self.history
            return events[-limit:]
    
    def clear(self):
        """Clear all data from workspace"""
        with self.lock:
            self.data.clear()
            self.locks.clear()
            self.history.clear()


class CollaborativeAgent:
    """
    Agent that works with shared workspace.
    
    Demonstrates collaborative problem solving.
    """
    
    def __init__(self, agent_id: str, workspace: SharedWorkspace):
        """
        Initialize collaborative agent.
        
        Args:
            agent_id: Unique agent identifier
            workspace: Shared workspace
        """
        self.agent_id = agent_id
        self.workspace = workspace
        self.llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0.7)
    
    def contribute(self, key: str, contribution: Any):
        """
        Add contribution to workspace.
        
        Args:
            key: Key to write to
            contribution: Data to contribute
        """
        print(f"[{self.agent_id}] Contributing to '{key}'")
        self.workspace.write(key, contribution, self.agent_id)
    
    def read_contribution(self, key: str) -> Any:
        """
        Read contribution from workspace.
        
        Args:
            key: Key to read
            
        Returns:
            Value or None
        """
        value = self.workspace.read(key)
        if value is not None:
            print(f"[{self.agent_id}] Read '{key}': {value}")
        return value
    
    def append_to_list(self, key: str, item: Any):
        """
        Append item to a list in workspace.
        
        Args:
            key: Key of list
            item: Item to append
        """
        print(f"[{self.agent_id}] Appending to '{key}'")
        
        def updater(current):
            if current is None:
                return [item]
            elif isinstance(current, list):
                return current + [item]
            else:
                return [current, item]
        
        self.workspace.update(key, updater, self.agent_id)
    
    def increment_counter(self, key: str, amount: int = 1):
        """
        Atomically increment a counter.
        
        Args:
            key: Key of counter
            amount: Amount to increment
        """
        print(f"[{self.agent_id}] Incrementing '{key}' by {amount}")
        
        def updater(current):
            if current is None:
                return amount
            return current + amount
        
        self.workspace.update(key, updater, self.agent_id)
    
    def locked_operation(self, key: str, operation: Callable):
        """
        Perform operation with exclusive lock.
        
        Args:
            key: Key to lock
            operation: Operation to perform
        """
        if self.workspace.lock_key(key, self.agent_id):
            try:
                print(f"[{self.agent_id}] Acquired lock on '{key}'")
                operation()
            finally:
                self.workspace.unlock_key(key, self.agent_id)
                print(f"[{self.agent_id}] Released lock on '{key}'")
        else:
            print(f"[{self.agent_id}] Failed to acquire lock on '{key}'")


def demonstrate_shared_workspace():
    """Demonstrate shared context/workspace pattern"""
    print("=" * 80)
    print("SHARED CONTEXT/WORKSPACE DEMONSTRATION")
    print("=" * 80)
    
    workspace = SharedWorkspace()
    
    # Example 1: Basic Read/Write
    print("\n" + "=" * 80)
    print("EXAMPLE 1: Basic Read/Write Operations")
    print("=" * 80)
    
    agent1 = CollaborativeAgent("agent1", workspace)
    agent2 = CollaborativeAgent("agent2", workspace)
    
    print("\nAgent1 writes, Agent2 reads:")
    agent1.contribute("shared_data", {"status": "initialized", "count": 0})
    data = agent2.read_contribution("shared_data")
    
    # Example 2: Collaborative List Building
    print("\n" + "=" * 80)
    print("EXAMPLE 2: Collaborative List Building")
    print("=" * 80)
    
    agent3 = CollaborativeAgent("agent3", workspace)
    
    print("\nMultiple agents appending to shared list:")
    agent1.append_to_list("tasks", "Analyze data")
    agent2.append_to_list("tasks", "Generate report")
    agent3.append_to_list("tasks", "Send notifications")
    
    tasks = workspace.read("tasks")
    print(f"\nFinal task list: {tasks}")
    
    # Example 3: Atomic Counter Updates
    print("\n" + "=" * 80)
    print("EXAMPLE 3: Atomic Counter Updates")
    print("=" * 80)
    
    print("\nMultiple agents incrementing shared counter:")
    workspace.write("counter", 0, "system")
    
    agent1.increment_counter("counter", 5)
    agent2.increment_counter("counter", 3)
    agent3.increment_counter("counter", 7)
    
    final_count = workspace.read("counter")
    print(f"\nFinal counter value: {final_count}")
    
    # Example 4: Locking for Critical Sections
    print("\n" + "=" * 80)
    print("EXAMPLE 4: Exclusive Locking")
    print("=" * 80)
    
    workspace.write("resource", {"balance": 100}, "system")
    
    def withdraw_funds():
        resource = workspace.read("resource")
        if resource["balance"] >= 30:
            resource["balance"] -= 30
            workspace.write("resource", resource, agent1.agent_id)
            print(f"[{agent1.agent_id}] Withdrew 30, new balance: {resource['balance']}")
    
    print("\nAgent1 performing locked operation:")
    agent1.locked_operation("resource", withdraw_funds)
    
    # Example 5: Change Notifications
    print("\n" + "=" * 80)
    print("EXAMPLE 5: Change Notifications (Subscriptions)")
    print("=" * 80)
    
    def on_status_change(event: WorkspaceEvent):
        print(f"[Subscriber] '{event.key}' {event.event_type} by {event.agent_id}: {event.value}")
    
    print("\nSubscribing to 'system_status' changes:")
    workspace.subscribe("system_status", on_status_change)
    
    agent1.contribute("system_status", "starting")
    time.sleep(0.1)
    agent2.contribute("system_status", "running")
    time.sleep(0.1)
    agent3.contribute("system_status", "completed")
    time.sleep(0.1)
    
    # Example 6: Versioning and Metadata
    print("\n" + "=" * 80)
    print("EXAMPLE 6: Versioning and Metadata")
    print("=" * 80)
    
    print("\nTracking versions of shared document:")
    workspace.write("document", "Version 1 content", "agent1")
    print(f"Info: {workspace.get_info('document')}")
    
    time.sleep(0.1)
    workspace.write("document", "Version 2 content", "agent2", {"reviewed": True})
    print(f"Info: {workspace.get_info('document')}")
    
    time.sleep(0.1)
    workspace.write("document", "Version 3 content", "agent3")
    print(f"Info: {workspace.get_info('document')}")
    
    # Example 7: History Tracking
    print("\n" + "=" * 80)
    print("EXAMPLE 7: Change History")
    print("=" * 80)
    
    history = workspace.get_history("document")
    print(f"\nHistory for 'document' ({len(history)} events):")
    for event in history:
        print(f"  {event.timestamp.strftime('%H:%M:%S')} - {event.event_type} by {event.agent_id}")
    
    # Example 8: Workspace Overview
    print("\n" + "=" * 80)
    print("EXAMPLE 8: Workspace Overview")
    print("=" * 80)
    
    all_keys = workspace.get_all_keys()
    print(f"\nWorkspace contains {len(all_keys)} keys:")
    for key in all_keys:
        info = workspace.get_info(key)
        print(f"  - {key}: v{info['version']}, updated by {info['updated_by']}")
    
    # Summary
    print("\n" + "=" * 80)
    print("SHARED CONTEXT/WORKSPACE SUMMARY")
    print("=" * 80)
    print("""
Shared Workspace Benefits:
1. Centralized State: Single source of truth
2. Easy Coordination: Natural collaboration point
3. Flexible Access: Read/write as needed
4. Transparency: All agents see same data
5. Simple Model: Familiar data structure
6. Change Tracking: Version history

Key Components:
1. Workspace
   - Key-value storage
   - Thread-safe operations
   - Version tracking
   - Change history
   - Notification system
   - Lock management

2. Operations
   - read(key): Get value
   - write(key, value): Set value
   - update(key, updater): Atomic update
   - delete(key): Remove value
   - lock/unlock: Exclusive access
   - subscribe: Watch for changes

3. Concurrency Control
   - Thread-safe operations
   - Optimistic updates (default)
   - Pessimistic locking (when needed)
   - Atomic operations
   - Version tracking
   - Conflict detection

Use Cases:
1. Collaborative Problem Solving
   - Shared scratchpad
   - Incremental solutions
   - Knowledge accumulation
   - Team coordination

2. Workflow State
   - Task tracking
   - Progress monitoring
   - Status updates
   - Result sharing

3. Knowledge Base
   - Shared facts
   - Learned patterns
   - Best practices
   - Historical data

4. Resource Management
   - Resource allocation
   - Capacity tracking
   - Scheduling
   - Conflict resolution

Design Patterns:
1. Shared Scratchpad
   - Agents contribute ideas
   - Build on each other's work
   - Iterative refinement
   - Collective intelligence

2. Blackboard Pattern
   - Problem-solving workspace
   - Multiple experts contribute
   - Control component coordinates
   - Solution emerges

3. Task Board
   - Todo/In Progress/Done
   - Agents claim tasks
   - Update status
   - Track completion

4. Data Pipeline
   - Input → Transform → Output
   - Each agent processes
   - Results in workspace
   - Next agent consumes

Concurrency Strategies:
1. No Locking (Simple)
   - For non-critical data
   - Last write wins
   - Fast, simple
   - Risk of conflicts

2. Optimistic Locking
   - Read freely
   - Check version on write
   - Retry on conflict
   - Good for low contention

3. Pessimistic Locking
   - Lock before read/write
   - Hold during operation
   - Unlock after
   - Good for critical sections

4. Atomic Operations
   - Single operation
   - All or nothing
   - No intermediate state
   - Best for counters, lists

Best Practices:
1. Data Organization
   ✓ Logical key structure
   ✓ Minimize shared state
   ✓ Clear ownership
   ✓ Document schemas
   ✓ Version formats
   ✓ Clean up old data

2. Concurrency
   ✓ Keep locks short
   ✓ Use atomic operations
   ✓ Handle conflicts
   ✓ Avoid deadlocks
   ✓ Test concurrent access
   ✓ Monitor contention

3. Performance
   ✓ Cache frequently read
   ✓ Batch updates
   ✓ Index for queries
   ✓ Lazy loading
   ✓ Minimize writes
   ✓ Async notifications

4. Reliability
   ✓ Persist critical data
   ✓ Backup regularly
   ✓ Validate writes
   ✓ Handle failures
   ✓ Log operations
   ✓ Support rollback

Common Pitfalls:
❌ Too much shared state (tight coupling)
✓ Share only necessary data

❌ Long-held locks (deadlocks)
✓ Minimize critical sections

❌ No conflict handling
✓ Implement retry logic

❌ Missing validation
✓ Validate all writes

❌ No access control
✓ Implement permissions

Production Considerations:
1. Storage Backend
   - Redis: Fast, in-memory
   - MongoDB: Document store
   - PostgreSQL: ACID compliance
   - Memcached: Simple cache
   - DynamoDB: Scalable cloud
   - Etcd: Distributed config

2. Consistency Model
   - Strong: All see same value
   - Eventual: Converges over time
   - Causal: Cause before effect
   - Read-your-writes: See own changes
   - Choose based on needs

3. Scalability
   - Partition by key
   - Replicate for reads
   - Cache hot data
   - Shard large datasets
   - Horizontal scaling
   - Load balancing

4. Monitoring
   - Access patterns
   - Lock contention
   - Read/write ratios
   - Storage usage
   - Conflict rate
   - Performance metrics

When to Use:
✓ Multi-agent collaboration
✓ Shared state needed
✓ Flexible data sharing
✓ Simple coordination
✓ Transparent operations
✓ Natural fit for problem
✗ No shared state needed
✗ Message passing sufficient
✗ Ultra-high performance critical
✗ Complex distributed system

Alternatives:
- Message Passing: Decoupled, async
- Event Sourcing: Immutable events
- Tuple Spaces: Linda-style coordination
- Distributed Cache: Geographically distributed
- Blockchain: Immutable, consensus

ROI Analysis:
- Simplicity: High (natural pattern)
- Performance: Medium (can be bottleneck)
- Scalability: Medium (single point)
- Reliability: High (with proper backend)
- Maintainability: High (centralized state)
- Cost: Low (simple infrastructure)
""")
    
    print("\n" + "=" * 80)
    print("Pattern 098 (Shared Context/Workspace) demonstration complete!")
    print("=" * 80)


if __name__ == "__main__":
    demonstrate_shared_workspace()

    

Agentic Design Pattern: Shared Context

Pattern 098: Shared Context – Line-by-Line Explanation

Overview

This document provides a detailed line-by-line explanation of the Shared Context pattern implementation.

---

Pattern Description

Pattern 098: Shared Context/Workspace

Description: Shared Context/Workspace provides a common space where multiple agents can read and write shared information, coordinate activities, and collaborate on tasks. This pattern enables agents to maintain shared state, exchange data without direct messaging, and build upon each other’s work. The shared workspace acts as a collaborative knowledge base and coordination medium for multi-agent systems.

Shared workspace features:
- Common data structures accessible to all agents
- Read/write operations with concurrency control
- Version tracking and conflict resolution
- Notifications on changes
- Access control and permissions
- Transaction support

Components: 1. Workspace – Shared data store – Key-value storage – Document/object storage – Access control lists – Change notifications – Version history

2. Context Manager
   - Manages workspace state
   - Handles concurrent access
   - Resolves conflicts
   - Maintains consistency
   - Provides transactions
   - Enforces policies

3. Agent Interface
   - Read operations
   - Write operations
   - Subscribe to changes
   - Lock/unlock resources
   - Query workspace
   - Batch operations

4. Synchronization
   - Optimistic locking
   - Pessimistic locking
   - Version control
   - Merge strategies
   - Conflict detection
   - Resolution policies

Use Cases: 1. Collaborative Problem Solving – Shared scratchpad – Intermediate results – Partial solutions – Knowledge accumulation – Iterative refinement – Collective intelligence

2. Multi-Agent Coordination
   - Task assignments
   - Status tracking
   - Resource allocation
   - Progress monitoring
   - Goal management
   - Constraint sharing

3. Knowledge Sharing
   - Common knowledge base
   - Learned patterns
   - Best practices
   - Historical data
   - Shared context
   - Collective memory

4. Workflow Management
   - Process state
   - Task dependencies
   - Input/output data
   - Checkpoint saving
   - Error recovery
   - Audit trails

LangChain Implementation: LangChain supports shared context through: – Memory objects shared across agents – Vector stores for shared knowledge – Custom state management – LangGraph for workflow state – Integration with databases

Key Features: 1. Concurrent Access – Multiple readers – Controlled writers – Lock management – Deadlock prevention – Fair scheduling – Priority access

2. Consistency Models
   - Strong consistency
   - Eventual consistency
   - Causal consistency
   - Read-your-writes
   - Monotonic reads
   - Application-specific

3. Change Management
   - Version tracking
   - Change logs
   - Rollback capability
   - Diff computation
   - Merge operations
   - Conflict resolution

4. Scalability
   - Partitioning
   - Replication
   - Caching
   - Lazy loading
   - Efficient indexing
   - Distributed storage

Best Practices: 1. Data Organization – Clear naming conventions – Logical structure – Minimize shared state – Immutable where possible – Document schemas – Version data formats

2. Concurrency Control
   - Use appropriate locking
   - Keep critical sections small
   - Avoid long-held locks
   - Handle deadlocks
   - Implement timeouts
   - Test concurrent scenarios

3. Performance
   - Cache frequently read data
   - Batch operations
   - Async notifications
   - Index for queries
   - Minimize contention
   - Monitor bottlenecks

4. Reliability
   - Persist important data
   - Backup regularly
   - Handle failures gracefully
   - Validate writes
   - Log operations
   - Support recovery

Trade-offs: Advantages: – Simple coordination – Shared visibility – Flexible collaboration – Natural data sharing – Easy debugging – Common patterns

Disadvantages:
- Potential bottleneck
- Concurrency complexity
- Consistency challenges
- Scalability limits
- Testing difficulty
- State management overhead

Production Considerations: 1. Storage Backend – Redis: Fast, in-memory – MongoDB: Document store – PostgreSQL: Relational, ACID – DynamoDB: Managed, scalable – Etcd: Distributed coordination – Memcached: Simple caching

2. Consistency Strategy
   - ACID transactions for critical data
   - Eventual consistency for non-critical
   - CRDTs for conflict-free updates
   - Last-write-wins for simple cases
   - Application-level resolution
   - Version vectors for causality

3. Performance Optimization
   - Read-through caching
   - Write-behind buffering
   - Optimistic locking
   - Sharding by key
   - Denormalization
   - Materialized views

4. Monitoring
   - Access patterns
   - Lock contention
   - Query performance
   - Storage usage
   - Conflict rate
   - Error frequency

5. Security
   - Authentication
   - Authorization (ACLs)
   - Encryption at rest
   - Audit logging
   - Input validation
   - Rate limiting

---

File Statistics

• Total Lines: 999
• Classes: 5
• Functions: 1
• Constants: 0

---

Module Documentation

"""
Pattern 098: Shared Context/Workspace

Description:
    Shared Context/Workspace provides a common space where multiple agents can read and write
    shared information, coordinate activities, and collaborate on tasks. This pattern enables
    agents to maintain shared state, exchange data without direct messaging, and build upon
    each other's work. The shared workspace acts as a collaborative knowledge base and
    coordination medium for multi-agent systems.

    Shared workspace features:
    - Common data structures accessible to all agents
    - Read/write operations with concurrency control
    - Version tracking and conflict resolution
    - Notifications on changes
    - Access control and permissions
    - Transaction support

Components:
    1. Workspace
       - Shared data store
       - Key-value storage
       - Document/object storage
       - Access control lists
       - Change notifications
       - Version history

    2. Context Manager
       - Manages workspace state
       - Handles concurrent access
       - Resolves conflicts
       - Maintains consistency
       - Provides transactions
       - Enforces policies

    3. Agent Interface
       - Read operations
       - Write operations
       - Subscribe to changes
       - Lock/unlock resources
       - Query workspace
       - Batch operations

    4. Synchronization
       - Optimistic locking
       - Pessimistic locking
       - Version control
       - Merge strategies
       - Conflict detection
       - Resolution policies

Use Cases:
    1. Collaborative Problem Solving
       - Shared scratchpad
       - Intermediate results
       - Partial solutions
       - Knowledge accumulation
       - Iterative refinement
       - Collective intelligence

    2. Multi-Agent Coordination
       - Task assignments
       - Status tracking
       - Resource allocation
       - Progress monitoring
       - Goal management
       - Constraint sharing

    3. Knowledge Sharing
       - Common knowledge base
       - Learned patterns
       - Best practices
       - Historical data
       - Shared context
       - Collective memory

    4. Workflow Management
       - Process state
       - Task dependencies
       - Input/output data
       - Checkpoint saving
       - Error recovery
       - Audit trails

LangChain Implementation:
    LangChain supports shared context through:
    - Memory objects shared across agents
    - Vector stores for shared knowledge
    - Custom state management
    - LangGraph for workflow state
    - Integration with databases

Key Features:
    1. Concurrent Access
       - Multiple readers
       - Controlled writers
       - Lock management
       - Deadlock prevention
       - Fair scheduling
       - Priority access

    2. Consistency Models
       - Strong consistency
       - Eventual consistency
       - Causal consistency
       - Read-your-writes
       - Monotonic reads
       - Application-specific

    3. Change Management
       - Version tracking
       - Change logs
       - Rollback capability
       - Diff computation
       - Merge operations
       - Conflict resolution

    4. Scalability
       - Partitioning
       - Replication
       - Caching
       - Lazy loading
       - Efficient indexing
       - Distributed storage

Best Practices:
    1. Data Organization
       - Clear naming conventions
       - Logical structure
       - Minimize shared state
       - Immutable where possible
       - Document schemas
       - Version data formats

    2. Concurrency Control
       - Use appropriate locking
       - Keep critical sections small
       - Avoid long-held locks
       - Handle deadlocks
       - Implement timeouts
       - Test concurrent scenarios

    3. Performance
       - Cache frequently read data
       - Batch operations
       - Async notifications
       - Index for queries
       - Minimize contention
       - Monitor bottlenecks

    4. Reliability
       - Persist important data
       - Backup regularly
       - Handle failures gracefully
       - Validate writes
       - Log operations
       - Support recovery

Trade-offs:
    Advantages:
    - Simple coordination
    - Shared visibility
    - Flexible collaboration
    - Natural data sharing
    - Easy debugging
    - Common patterns

    Disadvantages:
    - Potential bottleneck
    - Concurrency complexity
    - Consistency challenges
    - Scalability limits
    - Testing difficulty
    - State management overhead

Production Considerations:
    1. Storage Backend
       - Redis: Fast, in-memory
       - MongoDB: Document store
       - PostgreSQL: Relational, ACID
       - DynamoDB: Managed, scalable
       - Etcd: Distributed coordination
       - Memcached: Simple caching

    2. Consistency Strategy
       - ACID transactions for critical data
       - Eventual consistency for non-critical
       - CRDTs for conflict-free updates
       - Last-write-wins for simple cases
       - Application-level resolution
       - Version vectors for causality

    3. Performance Optimization
       - Read-through caching
       - Write-behind buffering
       - Optimistic locking
       - Sharding by key
       - Denormalization
       - Materialized views

    4. Monitoring
       - Access patterns
       - Lock contention
       - Query performance
       - Storage usage
       - Conflict rate
       - Error frequency

    5. Security
       - Authentication
       - Authorization (ACLs)
       - Encryption at rest
       - Audit logging
       - Input validation
       - Rate limiting
"""

Purpose: This module implements the Shared Context pattern, providing a structured approach to agent design and implementation.

---

Import Statements

Standard Library Imports

import collections.defaultdict
import dataclasses.dataclass
import dataclasses.field
import datetime.datetime
import enum.Enum
import json
import langchain_core.output_parsers.StrOutputParser
import langchain_core.prompts.ChatPromptTemplate
import os
import threading
import time
import typing.Any
import typing.Callable
import typing.Dict
import typing.List
import typing.Optional
import typing.Set

Purpose: Core Python functionality: – collections.defaultdict: Specialized container datatypes – dataclasses.dataclass: Simplified class definitions with @dataclass – dataclasses.field: Simplified class definitions with @dataclass – datetime.datetime: Date and time handling – enum.Enum: Enumeration support – json: JSON serialization/deserialization – langchain_core.output_parsers.StrOutputParser: Regular expression operations – langchain_core.prompts.ChatPromptTemplate: Regular expression operations – os: Environment variable access and system operations – threading: Regular expression operations – time: Time-related functions – typing.Any: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Callable: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Dict: Type hints for better code documentation (List, Dict, Any, Optional) – typing.List: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Optional: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Set: Type hints for better code documentation (List, Dict, Any, Optional)

LangChain Imports

from langchain_openai.ChatOpenAI

Purpose: LangChain framework components: – LLM Interface: Wrapper for OpenAI models – Chains: Composable building blocks for complex workflows

Other Imports

from copy.deepcopy
from dotenv.load_dotenv

Purpose: Additional dependencies for specialized functionality.

---

Classes

Class: AccessMode

Lines 237-241

Inherits from: Enum

Purpose:

Access modes for workspace resources

---

Class: WorkspaceEntry

Lines 245-254

Purpose:

Entry in the shared workspace

---

Class: WorkspaceEvent

Lines 258-264

Purpose:

Event notification for workspace changes

---

Class: SharedWorkspace

Lines 267-543

Purpose:

Shared workspace for multi-agent collaboration.

Provides concurrent access with locking, versioning, and change notifications.

Methods:

__init__(self)

Lines 274-280

Purpose:

Initialize shared workspace

Parameters:

read(self, key, default)

Lines 282-297

Purpose:

Read value from workspace.

Args:
    key: Key to read
    default: Default value if key doesn't exist

Returns:
    Value or default

Parameters: – key: Key – default: Default

Returns: Any

write(self, key, value, agent_id, metadata)

Lines 299-353

Purpose:

Write value to workspace.

Args:
    key: Key to write
    value: Value to write
    agent_id: Agent performing write
    metadata: Optional metadata

Returns:
    True if successful

Parameters: – key: Key – value: Value – agent_id: Agent id – metadata: Metadata

Returns: bool

delete(self, key, agent_id)

Lines 355-385

Purpose:

Delete value from workspace.

Args:
    key: Key to delete
    agent_id: Agent performing deletion

Returns:
    True if deleted, False if not found

Parameters: – key: Key – agent_id: Agent id

Returns: bool

update(self, key, updater, agent_id)

Lines 387-402

Purpose:

Atomically update value using updater function.

Args:
    key: Key to update
    updater: Function that takes current value and returns new value
    agent_id: Agent performing update

Returns:
    True if successful

Parameters: – key: Key – updater: Updater – agent_id: Agent id

Returns: bool

lock_key(self, key, agent_id, timeout)

Lines 404-425

Purpose:

Acquire exclusive lock on key.

Args:
    key: Key to lock
    agent_id: Agent requesting lock
    timeout: Maximum time to wait for lock

Returns:
    True if lock acquired

Parameters: – key: Key – agent_id: Agent id – timeout: Timeout

Returns: bool

unlock_key(self, key, agent_id)

Lines 427-442

Purpose:

Release lock on key.

Args:
    key: Key to unlock
    agent_id: Agent releasing lock

Returns:
    True if unlocked

Parameters: – key: Key – agent_id: Agent id

Returns: bool

subscribe(self, key, callback)

Lines 444-453

Purpose:

Subscribe to changes on a key.

Args:
    key: Key to watch
    callback: Function to call on changes

Parameters: – key: Key – callback: Callback

unsubscribe(self, key, callback)

Lines 455-465

Purpose:

Unsubscribe from changes on a key.

Args:
    key: Key to stop watching
    callback: Callback to remove

Parameters: – key: Key – callback: Callback

_notify_subscribers(self, key, event)

Lines 467-487

Purpose:

Notify subscribers of change.

Args:
    key: Key that changed
    event: Event details

Parameters: – key: Key – event: Event

get_all_keys(self)

Lines 489-492

Purpose:

Get all keys in workspace

Parameters:

Returns: <ast.Subscript object at 0x000001A06D674890>

get_info(self, key)

Lines 494-518

Purpose:

Get metadata about a key.

Args:
    key: Key to get info for

Returns:
    Info dictionary or None

Parameters: – key: Key

Returns: <ast.Subscript object at 0x000001A06D88BDD0>

get_history(self, key, limit)

Lines 520-536

Purpose:

Get history of changes.

Args:
    key: Filter by key (None for all)
    limit: Maximum events to return

Returns:
    List of events

Parameters: – key: Key – limit: Limit

Returns: <ast.Subscript object at 0x000001A06D8886D0>

clear(self)

Lines 538-543

Purpose:

Clear all data from workspace

Parameters:

---

Class: CollaborativeAgent

Lines 546-644

Purpose:

Agent that works with shared workspace.

Demonstrates collaborative problem solving.

Methods:

__init__(self, agent_id, workspace)

Lines 553-563

Purpose:

Initialize collaborative agent.

Args:
    agent_id: Unique agent identifier
    workspace: Shared workspace

Parameters: – agent_id: Agent id – workspace: Workspace

contribute(self, key, contribution)

Lines 565-574

Purpose:

Add contribution to workspace.

Args:
    key: Key to write to
    contribution: Data to contribute

Parameters: – key: Key – contribution: Contribution

read_contribution(self, key)

Lines 576-589

Purpose:

Read contribution from workspace.

Args:
    key: Key to read

Returns:
    Value or None

Parameters: – key: Key

Returns: Any

append_to_list(self, key, item)

Lines 591-609

Purpose:

Append item to a list in workspace.

Args:
    key: Key of list
    item: Item to append

Parameters: – key: Key – item: Item

increment_counter(self, key, amount)

Lines 611-626

Purpose:

Atomically increment a counter.

Args:
    key: Key of counter
    amount: Amount to increment

Parameters: – key: Key – amount: Amount

locked_operation(self, key, operation)

Lines 628-644

Purpose:

Perform operation with exclusive lock.

Args:
    key: Key to lock
    operation: Operation to perform

Parameters: – key: Key – operation: Operation

---

Functions

demonstrate_shared_workspace()

Lines 647-994

Purpose:

Demonstrate shared context/workspace pattern

---

Key Concepts Summary

Pattern Overview

The Shared Context pattern provides a structured approach to building AI agents with specific capabilities and behaviors. This implementation leverages LangChain’s composable components to create a robust and maintainable solution.

Core Components

1. Agent Class: Main implementation of the pattern logic
2. Prompt Engineering: Carefully crafted prompts to guide LLM behavior
3. Tool Integration: Optional tools for extended functionality
4. Memory Management: State handling across interactions
5. Output Parsing: Structured response processing

When to Use

✅ Good for: – Scenarios requiring this specific pattern’s strengths – Applications needing structured agent behavior – Systems where explainability is important – Production environments requiring reliability

❌ Not ideal for: – Simple single-prompt tasks – Scenarios requiring different pattern characteristics – Resource-constrained environments (if computationally intensive)

---

Best Practices Demonstrated

1. Environment Configuration

• ✅ Use .env files for API keys
• ✅ Never hardcode credentials
• ✅ Validate environment variables on startup

2. Error Handling

• ✅ Graceful degradation on failures
• ✅ Informative error messages
• ✅ Logging for debugging

3. Prompt Engineering

• ✅ Clear, specific instructions
• ✅ Structured output formats
• ✅ Few-shot examples when beneficial

4. Code Organization

• ✅ Separation of concerns (class-based design)
• ✅ Type hints for clarity
• ✅ Docstrings for documentation

5. Performance Optimization

• ✅ Appropriate temperature settings
• ✅ Token budget management
• ✅ Caching where applicable

---

LangChain Components Used

• ChatOpenAI: OpenAI model interface
• Chains: Sequential component composition

Why LangChain? – Composable abstractions for LLM applications – Rich ecosystem of integrations – Production-ready patterns – Active community and development

---

Execution Flow

1. Initialize agent with configuration
   ↓
2. Load environment and dependencies
   ↓
3. Prepare prompt with input data
   ↓
4. Invoke LLM through LangChain
   ↓
5. Parse and validate response
   ↓
6. Return formatted result

Key Decision Points: – Model selection (temperature, model type) – Prompt structure (few-shot vs zero-shot) – Error handling strategy – Output format validation

---

Performance Considerations

Computational Cost

• LLM Calls: Primary cost factor (based on tokens)
• Latency: Depends on model and input size
• Memory: Minimal for most patterns

Optimization Strategies

1. Prompt Optimization: Minimize tokens while maintaining clarity
2. Caching: Cache repeated queries/responses
3. Model Selection: Use smaller models when appropriate
4. Batch Processing: Group similar requests
5. Streaming: For long responses, stream tokens

Cost Estimates (Approximate)

• GPT-3.5-turbo: ~$0.002 per request
• GPT-4: ~$0.03 per request
• GPT-4-turbo: ~$0.01 per request

Actual costs vary based on input/output token counts

---

Common Pitfalls to Avoid

1. Prompt Design Issues

❌ Vague or ambiguous instructions
✅ Clear, specific, testable prompts

2. Error Handling

❌ Ignoring API errors or rate limits
✅ Implement retry logic and graceful degradation

3. Token Management

❌ Exceeding model token limits
✅ Truncate inputs, chunk processing

4. Security

❌ Exposing API keys in code
✅ Use environment variables

5. Testing

❌ Only testing happy paths
✅ Test edge cases, failures, malformed inputs

6. Cost Management

❌ Unmonitored API usage
✅ Implement budgets, alerts, and monitoring

---

Further Improvements

Production Enhancements

1. Monitoring: Add logging, metrics, tracing
2. Validation: Input/output schema validation
3. Rate Limiting: Implement request throttling
4. Caching: Redis or in-memory cache for responses
5. Async Support: Non-blocking operations for scalability

Feature Extensions

1. Multi-Model Support: Switch between providers
2. Fallback Models: Graceful degradation strategy
3. Custom Tools: Domain-specific function calling
4. Advanced Memory: Vector stores for long-term memory
5. Evaluation: Automated testing and benchmarking

Code Quality

1. Testing: Unit tests, integration tests
2. Documentation: API docs, usage examples
3. Type Safety: Stricter type hints
4. Configuration: Externalized config files

---

Related Patterns

• Pattern 097
• Pattern 099

Pattern Families: – Reasoning Patterns (001-011) – Multi-Agent Patterns (012-019) – Tool Use Patterns (020-025) – Memory Patterns (026-031) – And many more…

---

Summary

The Shared Context pattern provides a powerful approach to building AI agents with specific capabilities. This implementation demonstrates:

✅ Key Strengths: – Clean, maintainable code structure – LangChain integration for composability – Type hints and documentation – Production-ready patterns

🎯 Best Use Cases: – Applications requiring this pattern’s characteristics – Systems needing structured agent behavior – Production environments requiring reliability

📚 Learning Resources: – LangChain Documentation: https://python.langchain.com/ – Pattern Documentation: See related patterns – OpenAI API Reference: https://platform.openai.com/docs/

---

File: 098_shared_context.py
Total Lines: 999
Pattern Category: AI Agent Design Pattern #098

---

Generated from actual implementation code
For educational and reference purposes

    """
Pattern 097: Message Passing

Description:
    Message Passing enables communication between agents through explicit message exchange.
    This pattern provides a structured way for agents to share information, coordinate actions,
    and collaborate on tasks without direct coupling. Messages are discrete units of information
    that flow between agents, enabling asynchronous, decoupled, and scalable multi-agent systems.

    Message passing supports:
    - Asynchronous communication
    - Loose coupling between agents
    - Point-to-point and broadcast messaging
    - Message queuing and buffering
    - Reliable delivery guarantees
    - Priority-based message handling

Components:
    1. Message
       - Sender identification
       - Recipient(s) specification
       - Message type/category
       - Payload (data content)
       - Metadata (timestamp, priority, etc.)
       - Reply-to information

    2. Message Bus/Broker
       - Routes messages between agents
       - Handles message queuing
       - Manages subscriptions
       - Ensures delivery
       - Provides message persistence
       - Supports patterns (pub/sub, request/reply)

    3. Agent Interface
       - Send messages
       - Receive messages
       - Subscribe to topics
       - Handle message callbacks
       - Acknowledge receipt
       - Reply to messages

    4. Message Protocol
       - Message format specification
       - Serialization/deserialization
       - Error handling conventions
       - Timeout management
       - Retry logic
       - Message versioning

Use Cases:
    1. Multi-Agent Coordination
       - Task delegation
       - Status updates
       - Resource sharing
       - Result aggregation
       - Workflow orchestration
       - Event notification

    2. Distributed Systems
       - Microservices communication
       - Remote procedure calls
       - Event-driven architectures
       - Stream processing
       - Job queues
       - Service integration

    3. Collaborative Problem Solving
       - Information sharing
       - Consensus building
       - Parallel processing
       - Load distribution
       - Fault tolerance
       - Scalability

    4. Real-Time Systems
       - Alert propagation
       - State synchronization
       - Live updates
       - Chat/messaging
       - Monitoring dashboards
       - Command execution

LangChain Implementation:
    LangChain supports message passing through:
    - Custom message passing classes
    - Callback handlers for events
    - LangGraph for state management
    - Integration with message brokers
    - Async message handling

Key Features:
    1. Asynchronous Delivery
       - Non-blocking sends
       - Background processing
       - Queue-based buffering
       - Fire-and-forget option
       - Guaranteed delivery
       - Order preservation

    2. Flexible Routing
       - Direct messaging (agent-to-agent)
       - Topic-based (publish/subscribe)
       - Broadcast (one-to-many)
       - Multicast (selected recipients)
       - Request/reply pattern
       - Router/filter chains

    3. Reliability Features
       - Acknowledgments
       - Retry mechanisms
       - Dead letter queues
       - Message persistence
       - Delivery guarantees
       - Duplicate detection

    4. Scalability
       - Horizontal scaling
       - Load balancing
       - Message partitioning
       - Connection pooling
       - Efficient serialization
       - Resource management

Best Practices:
    1. Message Design
       - Keep messages small and focused
       - Include all necessary context
       - Use standard formats (JSON, Protocol Buffers)
       - Version messages for evolution
       - Include correlation IDs
       - Document message schemas

    2. Error Handling
       - Implement retry logic with backoff
       - Use dead letter queues
       - Log failed messages
       - Monitor delivery rates
       - Timeout handling
       - Circuit breakers

    3. Performance
       - Batch messages when possible
       - Use async/await patterns
       - Implement connection pooling
       - Cache frequently used data
       - Monitor queue depths
       - Optimize serialization

    4. Security
       - Authenticate senders
       - Authorize operations
       - Encrypt sensitive data
       - Validate message content
       - Rate limiting
       - Audit trails

Trade-offs:
    Advantages:
    - Loose coupling
    - Asynchronous operation
    - Scalability
    - Fault tolerance
    - Flexibility
    - Technology independence

    Disadvantages:
    - Added complexity
    - Potential message loss
    - Ordering challenges
    - Debugging difficulty
    - Latency overhead
    - Infrastructure requirements

Production Considerations:
    1. Message Broker Selection
       - RabbitMQ: Feature-rich, reliable
       - Redis: Fast, simple pub/sub
       - Kafka: High throughput, persistent
       - AWS SQS/SNS: Managed, scalable
       - Google Pub/Sub: Global, managed
       - Azure Service Bus: Enterprise features

    2. Monitoring
       - Message throughput
       - Queue depths
       - Delivery latency
       - Error rates
       - Dead letter queue size
       - Consumer lag

    3. Scaling Strategy
       - Partition messages by key
       - Add consumer instances
       - Use competing consumers
       - Implement load shedding
       - Auto-scaling based on queue depth
       - Connection pooling

    4. Reliability
       - At-least-once delivery
       - Idempotent consumers
       - Duplicate detection
       - Message persistence
       - Automatic retries
       - Dead letter handling

    5. Operations
       - Message replay capability
       - Monitoring dashboards
       - Alert configuration
       - Log aggregation
       - Backup/recovery
       - Performance tuning
"""

import os
import json
import time
import uuid
from typing import List, Dict, Any, Optional, Callable
from dataclasses import dataclass, field, asdict
from datetime import datetime
from enum import Enum
from collections import defaultdict
from queue import Queue, Empty
from threading import Thread, Lock
from dotenv import load_dotenv

from langchain_core.prompts import ChatPromptTemplate
from langchain_openai import ChatOpenAI
from langchain_core.output_parsers import StrOutputParser

load_dotenv()


class MessageType(Enum):
    """Types of messages"""
    REQUEST = "request"
    RESPONSE = "response"
    NOTIFICATION = "notification"
    COMMAND = "command"
    EVENT = "event"


class MessagePriority(Enum):
    """Message priority levels"""
    LOW = 1
    NORMAL = 2
    HIGH = 3
    URGENT = 4


@dataclass
class Message:
    """
    A message passed between agents.
    
    Contains sender, recipient, type, payload, and metadata.
    """
    message_id: str
    sender: str
    recipient: str
    message_type: MessageType
    payload: Dict[str, Any]
    priority: MessagePriority = MessagePriority.NORMAL
    timestamp: datetime = field(default_factory=datetime.now)
    reply_to: Optional[str] = None
    correlation_id: Optional[str] = None
    metadata: Dict[str, Any] = field(default_factory=dict)
    
    def to_dict(self) -> Dict[str, Any]:
        """Convert message to dictionary"""
        data = asdict(self)
        data['message_type'] = self.message_type.value
        data['priority'] = self.priority.value
        data['timestamp'] = self.timestamp.isoformat()
        return data
    
    @classmethod
    def from_dict(cls, data: Dict[str, Any]) -> 'Message':
        """Create message from dictionary"""
        data['message_type'] = MessageType(data['message_type'])
        data['priority'] = MessagePriority(data['priority'])
        data['timestamp'] = datetime.fromisoformat(data['timestamp'])
        return cls(**data)


class MessageBus:
    """
    Central message bus for routing messages between agents.
    
    Supports point-to-point, publish/subscribe, and broadcast patterns.
    """
    
    def __init__(self):
        """Initialize message bus"""
        self.agents: Dict[str, 'Agent'] = {}
        self.subscriptions: Dict[str, List[str]] = defaultdict(list)  # topic -> agent_ids
        self.message_queue: Queue = Queue()
        self.message_history: List[Message] = []
        self.lock = Lock()
        self.running = False
        self.worker_thread: Optional[Thread] = None
    
    def register_agent(self, agent: 'Agent'):
        """
        Register an agent with the bus.
        
        Args:
            agent: Agent to register
        """
        with self.lock:
            self.agents[agent.agent_id] = agent
            print(f"[MessageBus] Registered agent: {agent.agent_id}")
    
    def unregister_agent(self, agent_id: str):
        """
        Unregister an agent.
        
        Args:
            agent_id: Agent identifier
        """
        with self.lock:
            if agent_id in self.agents:
                del self.agents[agent_id]
                # Remove from subscriptions
                for topic in list(self.subscriptions.keys()):
                    if agent_id in self.subscriptions[topic]:
                        self.subscriptions[topic].remove(agent_id)
                print(f"[MessageBus] Unregistered agent: {agent_id}")
    
    def subscribe(self, agent_id: str, topic: str):
        """
        Subscribe agent to a topic.
        
        Args:
            agent_id: Agent identifier
            topic: Topic to subscribe to
        """
        with self.lock:
            if agent_id not in self.subscriptions[topic]:
                self.subscriptions[topic].append(agent_id)
                print(f"[MessageBus] {agent_id} subscribed to '{topic}'")
    
    def unsubscribe(self, agent_id: str, topic: str):
        """
        Unsubscribe agent from a topic.
        
        Args:
            agent_id: Agent identifier
            topic: Topic to unsubscribe from
        """
        with self.lock:
            if agent_id in self.subscriptions[topic]:
                self.subscriptions[topic].remove(agent_id)
                print(f"[MessageBus] {agent_id} unsubscribed from '{topic}'")
    
    def send_message(self, message: Message):
        """
        Send a message through the bus.
        
        Args:
            message: Message to send
        """
        self.message_queue.put(message)
        with self.lock:
            self.message_history.append(message)
    
    def publish(self, topic: str, sender: str, payload: Dict[str, Any], priority: MessagePriority = MessagePriority.NORMAL):
        """
        Publish message to all subscribers of a topic.
        
        Args:
            topic: Topic to publish to
            sender: Sender agent ID
            payload: Message payload
            priority: Message priority
        """
        with self.lock:
            subscribers = self.subscriptions.get(topic, [])
        
        for recipient in subscribers:
            if recipient != sender:  # Don't send to self
                message = Message(
                    message_id=str(uuid.uuid4()),
                    sender=sender,
                    recipient=recipient,
                    message_type=MessageType.NOTIFICATION,
                    payload=payload,
                    priority=priority,
                    metadata={'topic': topic}
                )
                self.send_message(message)
    
    def broadcast(self, sender: str, payload: Dict[str, Any], priority: MessagePriority = MessagePriority.NORMAL):
        """
        Broadcast message to all agents.
        
        Args:
            sender: Sender agent ID
            payload: Message payload
            priority: Message priority
        """
        with self.lock:
            recipients = [aid for aid in self.agents.keys() if aid != sender]
        
        for recipient in recipients:
            message = Message(
                message_id=str(uuid.uuid4()),
                sender=sender,
                recipient=recipient,
                message_type=MessageType.NOTIFICATION,
                payload=payload,
                priority=priority,
                metadata={'broadcast': True}
            )
            self.send_message(message)
    
    def start(self):
        """Start processing messages"""
        if not self.running:
            self.running = True
            self.worker_thread = Thread(target=self._process_messages, daemon=True)
            self.worker_thread.start()
            print("[MessageBus] Started message processing")
    
    def stop(self):
        """Stop processing messages"""
        self.running = False
        if self.worker_thread:
            self.worker_thread.join(timeout=2)
        print("[MessageBus] Stopped message processing")
    
    def _process_messages(self):
        """Process messages from queue (runs in background thread)"""
        while self.running:
            try:
                message = self.message_queue.get(timeout=0.1)
                self._deliver_message(message)
            except Empty:
                continue
    
    def _deliver_message(self, message: Message):
        """
        Deliver message to recipient.
        
        Args:
            message: Message to deliver
        """
        with self.lock:
            recipient = self.agents.get(message.recipient)
        
        if recipient:
            recipient.receive_message(message)
        else:
            print(f"[MessageBus] Warning: Recipient '{message.recipient}' not found for message {message.message_id}")
    
    def get_message_history(self, agent_id: Optional[str] = None, limit: int = 10) -> List[Message]:
        """
        Get message history.
        
        Args:
            agent_id: Filter by agent (sender or recipient)
            limit: Maximum messages to return
            
        Returns:
            List of messages
        """
        with self.lock:
            if agent_id:
                filtered = [m for m in self.message_history 
                           if m.sender == agent_id or m.recipient == agent_id]
                return filtered[-limit:]
            else:
                return self.message_history[-limit:]


class Agent:
    """
    Base agent class with message passing capabilities.
    
    Agents can send, receive, and process messages.
    """
    
    def __init__(self, agent_id: str, message_bus: MessageBus):
        """
        Initialize agent.
        
        Args:
            agent_id: Unique agent identifier
            message_bus: Message bus for communication
        """
        self.agent_id = agent_id
        self.message_bus = message_bus
        self.inbox: Queue = Queue()
        self.message_handlers: Dict[MessageType, Callable] = {}
        self.running = False
        self.worker_thread: Optional[Thread] = None
        
        # Register with bus
        self.message_bus.register_agent(self)
    
    def send_message(self, recipient: str, message_type: MessageType, payload: Dict[str, Any], 
                    priority: MessagePriority = MessagePriority.NORMAL, reply_to: Optional[str] = None):
        """
        Send a message to another agent.
        
        Args:
            recipient: Recipient agent ID
            message_type: Type of message
            payload: Message payload
            priority: Message priority
            reply_to: Message ID this is replying to
        """
        message = Message(
            message_id=str(uuid.uuid4()),
            sender=self.agent_id,
            recipient=recipient,
            message_type=message_type,
            payload=payload,
            priority=priority,
            reply_to=reply_to
        )
        self.message_bus.send_message(message)
        print(f"[{self.agent_id}] Sent {message_type.value} to {recipient}")
    
    def receive_message(self, message: Message):
        """
        Receive a message (called by message bus).
        
        Args:
            message: Received message
        """
        self.inbox.put(message)
    
    def subscribe(self, topic: str):
        """
        Subscribe to a topic.
        
        Args:
            topic: Topic to subscribe to
        """
        self.message_bus.subscribe(self.agent_id, topic)
    
    def publish(self, topic: str, payload: Dict[str, Any], priority: MessagePriority = MessagePriority.NORMAL):
        """
        Publish to a topic.
        
        Args:
            topic: Topic to publish to
            payload: Message payload
            priority: Message priority
        """
        self.message_bus.publish(topic, self.agent_id, payload, priority)
    
    def broadcast(self, payload: Dict[str, Any], priority: MessagePriority = MessagePriority.NORMAL):
        """
        Broadcast message to all agents.
        
        Args:
            payload: Message payload
            priority: Message priority
        """
        self.message_bus.broadcast(self.agent_id, payload, priority)
    
    def register_handler(self, message_type: MessageType, handler: Callable[[Message], None]):
        """
        Register handler for message type.
        
        Args:
            message_type: Type of message
            handler: Handler function
        """
        self.message_handlers[message_type] = handler
    
    def start(self):
        """Start processing messages"""
        if not self.running:
            self.running = True
            self.worker_thread = Thread(target=self._process_inbox, daemon=True)
            self.worker_thread.start()
    
    def stop(self):
        """Stop processing messages"""
        self.running = False
        if self.worker_thread:
            self.worker_thread.join(timeout=2)
    
    def _process_inbox(self):
        """Process incoming messages (runs in background thread)"""
        while self.running:
            try:
                message = self.inbox.get(timeout=0.1)
                self._handle_message(message)
            except Empty:
                continue
    
    def _handle_message(self, message: Message):
        """
        Handle incoming message.
        
        Args:
            message: Message to handle
        """
        print(f"[{self.agent_id}] Received {message.message_type.value} from {message.sender}")
        
        # Call registered handler if exists
        handler = self.message_handlers.get(message.message_type)
        if handler:
            handler(message)
        else:
            self.handle_message(message)
    
    def handle_message(self, message: Message):
        """
        Default message handler (override in subclasses).
        
        Args:
            message: Message to handle
        """
        print(f"[{self.agent_id}] No handler for {message.message_type.value}")


class WorkerAgent(Agent):
    """
    Example worker agent that processes tasks.
    
    Demonstrates request/response pattern.
    """
    
    def __init__(self, agent_id: str, message_bus: MessageBus):
        """Initialize worker agent"""
        super().__init__(agent_id, message_bus)
        self.register_handler(MessageType.REQUEST, self.handle_request)
    
    def handle_request(self, message: Message):
        """
        Handle task request.
        
        Args:
            message: Request message
        """
        task = message.payload.get('task', '')
        print(f"[{self.agent_id}] Processing task: {task}")
        
        # Simulate work
        time.sleep(0.1)
        result = f"Completed: {task}"
        
        # Send response
        self.send_message(
            recipient=message.sender,
            message_type=MessageType.RESPONSE,
            payload={'result': result, 'task': task},
            reply_to=message.message_id
        )


class CoordinatorAgent(Agent):
    """
    Example coordinator agent that delegates work.
    
    Demonstrates task delegation pattern.
    """
    
    def __init__(self, agent_id: str, message_bus: MessageBus):
        """Initialize coordinator agent"""
        super().__init__(agent_id, message_bus)
        self.register_handler(MessageType.RESPONSE, self.handle_response)
        self.pending_tasks: Dict[str, str] = {}  # message_id -> task
        self.results: List[str] = []
    
    def delegate_task(self, worker_id: str, task: str):
        """
        Delegate task to worker.
        
        Args:
            worker_id: Worker agent ID
            task: Task to delegate
        """
        message_id = str(uuid.uuid4())
        self.pending_tasks[message_id] = task
        
        message = Message(
            message_id=message_id,
            sender=self.agent_id,
            recipient=worker_id,
            message_type=MessageType.REQUEST,
            payload={'task': task}
        )
        self.message_bus.send_message(message)
        print(f"[{self.agent_id}] Delegated '{task}' to {worker_id}")
    
    def handle_response(self, message: Message):
        """
        Handle response from worker.
        
        Args:
            message: Response message
        """
        if message.reply_to in self.pending_tasks:
            task = self.pending_tasks[message.reply_to]
            result = message.payload.get('result', '')
            self.results.append(result)
            print(f"[{self.agent_id}] Received result for '{task}': {result}")
            del self.pending_tasks[message.reply_to]


def demonstrate_message_passing():
    """Demonstrate message passing pattern"""
    print("=" * 80)
    print("MESSAGE PASSING DEMONSTRATION")
    print("=" * 80)
    
    # Example 1: Basic Point-to-Point Messaging
    print("\n" + "=" * 80)
    print("EXAMPLE 1: Point-to-Point Messaging")
    print("=" * 80)
    
    bus = MessageBus()
    bus.start()
    
    agent1 = Agent("agent1", bus)
    agent2 = Agent("agent2", bus)
    
    agent1.start()
    agent2.start()
    
    # Send message
    print("\nSending message from agent1 to agent2...")
    agent1.send_message(
        recipient="agent2",
        message_type=MessageType.NOTIFICATION,
        payload={"text": "Hello from agent1!"}
    )
    
    time.sleep(0.5)
    
    # Example 2: Request/Response Pattern
    print("\n" + "=" * 80)
    print("EXAMPLE 2: Request/Response Pattern with Workers")
    print("=" * 80)
    
    coordinator = CoordinatorAgent("coordinator", bus)
    worker1 = WorkerAgent("worker1", bus)
    worker2 = WorkerAgent("worker2", bus)
    
    coordinator.start()
    worker1.start()
    worker2.start()
    
    print("\nCoordinator delegating tasks to workers...")
    coordinator.delegate_task("worker1", "analyze_data")
    coordinator.delegate_task("worker2", "generate_report")
    coordinator.delegate_task("worker1", "send_notification")
    
    time.sleep(0.5)
    
    print(f"\nCoordinator collected {len(coordinator.results)} results:")
    for result in coordinator.results:
        print(f"  - {result}")
    
    # Example 3: Publish/Subscribe Pattern
    print("\n" + "=" * 80)
    print("EXAMPLE 3: Publish/Subscribe Pattern")
    print("=" * 80)
    
    subscriber1 = Agent("subscriber1", bus)
    subscriber2 = Agent("subscriber2", bus)
    subscriber3 = Agent("subscriber3", bus)
    publisher = Agent("publisher", bus)
    
    subscriber1.start()
    subscriber2.start()
    subscriber3.start()
    publisher.start()
    
    # Subscribe to topics
    print("\nSubscribing agents to topics...")
    subscriber1.subscribe("alerts")
    subscriber2.subscribe("alerts")
    subscriber2.subscribe("updates")
    subscriber3.subscribe("updates")
    
    time.sleep(0.2)
    
    # Publish messages
    print("\nPublishing messages to topics...")
    publisher.publish("alerts", {"level": "warning", "message": "High CPU usage"})
    publisher.publish("updates", {"version": "1.2.0", "changes": "Bug fixes"})
    
    time.sleep(0.5)
    
    # Example 4: Broadcast Messaging
    print("\n" + "=" * 80)
    print("EXAMPLE 4: Broadcast Messaging")
    print("=" * 80)
    
    print("\nBroadcasting message to all agents...")
    coordinator.broadcast({"announcement": "System maintenance in 10 minutes"})
    
    time.sleep(0.5)
    
    # Example 5: Priority Messaging
    print("\n" + "=" * 80)
    print("EXAMPLE 5: Priority-Based Messaging")
    print("=" * 80)
    
    print("\nSending messages with different priorities...")
    agent1.send_message(
        recipient="agent2",
        message_type=MessageType.NOTIFICATION,
        payload={"text": "Low priority message"},
        priority=MessagePriority.LOW
    )
    agent1.send_message(
        recipient="agent2",
        message_type=MessageType.COMMAND,
        payload={"action": "urgent_action"},
        priority=MessagePriority.URGENT
    )
    agent1.send_message(
        recipient="agent2",
        message_type=MessageType.NOTIFICATION,
        payload={"text": "Normal priority message"},
        priority=MessagePriority.NORMAL
    )
    
    time.sleep(0.5)
    
    # Example 6: Message History
    print("\n" + "=" * 80)
    print("EXAMPLE 6: Message History and Tracking")
    print("=" * 80)
    
    history = bus.get_message_history(limit=5)
    print(f"\nLast {len(history)} messages:")
    for msg in history:
        print(f"  {msg.timestamp.strftime('%H:%M:%S')} - {msg.sender} → {msg.recipient}: {msg.message_type.value}")
    
    # Example 7: Conversation Thread
    print("\n" + "=" * 80)
    print("EXAMPLE 7: Threaded Conversation (Reply Pattern)")
    print("=" * 80)
    
    print("\nAgent1 sends question, Agent2 replies...")
    
    # Send question
    question_msg_id = str(uuid.uuid4())
    question = Message(
        message_id=question_msg_id,
        sender="agent1",
        recipient="agent2",
        message_type=MessageType.REQUEST,
        payload={"question": "What is the status?"}
    )
    bus.send_message(question)
    
    time.sleep(0.2)
    
    # Send reply
    reply = Message(
        message_id=str(uuid.uuid4()),
        sender="agent2",
        recipient="agent1",
        message_type=MessageType.RESPONSE,
        payload={"answer": "All systems operational"},
        reply_to=question_msg_id
    )
    bus.send_message(reply)
    
    time.sleep(0.2)
    
    print("\nMessage thread:")
    print(f"  Q: {question.payload}")
    print(f"  A: {reply.payload} (reply_to: {reply.reply_to[:8]}...)")
    
    # Cleanup
    print("\n" + "=" * 80)
    print("Cleaning up...")
    print("=" * 80)
    
    agent1.stop()
    agent2.stop()
    coordinator.stop()
    worker1.stop()
    worker2.stop()
    subscriber1.stop()
    subscriber2.stop()
    subscriber3.stop()
    publisher.stop()
    bus.stop()
    
    # Summary
    print("\n" + "=" * 80)
    print("MESSAGE PASSING SUMMARY")
    print("=" * 80)
    print("""
Message Passing Benefits:
1. Loose Coupling: Agents don't need direct references
2. Asynchronous: Non-blocking communication
3. Scalable: Easy to add more agents
4. Flexible: Multiple communication patterns
5. Reliable: Message queuing and delivery guarantees
6. Debuggable: Message history and tracking

Key Components:
1. Message
   - Unique ID for tracking
   - Sender and recipient
   - Type (request, response, notification, etc.)
   - Payload (actual data)
   - Priority for ordering
   - Metadata (timestamp, correlation ID, etc.)

2. Message Bus
   - Central routing system
   - Message queue for buffering
   - Subscription management
   - Delivery guarantees
   - History tracking
   - Background processing

3. Agent Interface
   - Send messages (point-to-point)
   - Publish to topics (one-to-many)
   - Broadcast (to all)
   - Subscribe to topics
   - Register message handlers
   - Process inbox

Communication Patterns:
1. Point-to-Point
   - Direct agent-to-agent
   - Request/response
   - Guaranteed delivery
   - Use: Direct commands, queries

2. Publish/Subscribe
   - Topic-based
   - One-to-many
   - Subscribers receive all messages
   - Use: Events, notifications, updates

3. Broadcast
   - One-to-all
   - Reaches all registered agents
   - Use: System-wide announcements

4. Request/Reply
   - Synchronous-style communication
   - Reply linked to request
   - Correlation tracking
   - Use: RPC-style calls, queries

Message Types:
- REQUEST: Ask for action/information
- RESPONSE: Reply to request
- NOTIFICATION: Inform about event
- COMMAND: Direct agent to act
- EVENT: Something happened

Priority Levels:
- LOW: Background tasks
- NORMAL: Standard messages
- HIGH: Important messages
- URGENT: Critical, process first

Best Practices:
1. Message Design
   ✓ Keep messages small (< 1MB)
   ✓ Include correlation IDs
   ✓ Version message schemas
   ✓ Use standard formats (JSON)
   ✓ Include timestamps
   ✓ Document message types

2. Error Handling
   ✓ Implement retry with backoff
   ✓ Use dead letter queues
   ✓ Set timeouts
   ✓ Log failures
   ✓ Monitor delivery rates
   ✓ Handle duplicates

3. Performance
   ✓ Batch messages when possible
   ✓ Use async processing
   ✓ Implement backpressure
   ✓ Monitor queue depth
   ✓ Optimize serialization
   ✓ Connection pooling

4. Reliability
   ✓ Acknowledge receipt
   ✓ Persist critical messages
   ✓ Duplicate detection
   ✓ Ordered delivery (if needed)
   ✓ At-least-once semantics
   ✓ Idempotent handlers

Production Considerations:
1. Message Broker
   - RabbitMQ: Feature-rich, AMQP
   - Kafka: High throughput, log-based
   - Redis: Fast, simple pub/sub
   - AWS SQS: Managed, scalable
   - Google Pub/Sub: Global scale
   - NATS: Lightweight, fast

2. Delivery Guarantees
   - At-most-once: Fast, may lose messages
   - At-least-once: Reliable, may duplicate
   - Exactly-once: Most reliable, complex

3. Ordering
   - FIFO: First in, first out
   - Priority: Higher priority first
   - No guarantee: Fastest
   - Partitioned: Order within partition

4. Scaling
   - Horizontal: Add more consumers
   - Partitioning: Split by key
   - Load balancing: Distribute evenly
   - Competing consumers: Multiple workers
   - Auto-scaling: Dynamic capacity

Common Patterns:
1. Task Queue
   - Coordinator sends tasks
   - Workers pull and process
   - Results sent back
   - Use: Distributed processing

2. Event Sourcing
   - All changes as events
   - Agents subscribe to events
   - Rebuild state from events
   - Use: Audit trails, CQRS

3. Command Query Separation
   - Commands change state
   - Queries read state
   - Different channels
   - Use: Clean architecture

4. Saga Pattern
   - Long-running transactions
   - Compensating actions
   - Message coordination
   - Use: Distributed transactions

Monitoring Metrics:
- Messages sent/received per second
- Queue depth (backlog)
- Processing latency
- Error rate
- Dead letter queue size
- Consumer lag
- Message size distribution

When to Use:
✓ Multi-agent systems
✓ Distributed architectures
✓ Asynchronous processing
✓ Event-driven systems
✓ Microservices
✓ Scalable workflows
✗ Simple single-agent systems
✗ Synchronous-only requirements
✗ Ultra-low latency critical

Alternatives:
- Direct method calls (simpler, coupled)
- Shared memory (faster, requires locks)
- REST APIs (standard, synchronous)
- gRPC (fast, bidirectional)
- WebSockets (real-time, persistent)

ROI Analysis:
- Development time: +20% (added complexity)
- Scalability: +200-500% (horizontal scaling)
- Reliability: +50% (fault tolerance)
- Maintenance: +15% (debugging complexity)
- Performance: Variable (latency vs throughput)
- Cost: Infrastructure required
""")
    
    print("\n" + "=" * 80)
    print("Pattern 097 (Message Passing) demonstration complete!")
    print("=" * 80)


if __name__ == "__main__":
    demonstrate_message_passing()

    

Agentic Design Pattern: Message Passing

Pattern 097: Message Passing – Line-by-Line Explanation

Overview

This document provides a detailed line-by-line explanation of the Message Passing pattern implementation.

---

Pattern Description

Pattern 097: Message Passing

Description: Message Passing enables communication between agents through explicit message exchange. This pattern provides a structured way for agents to share information, coordinate actions, and collaborate on tasks without direct coupling. Messages are discrete units of information that flow between agents, enabling asynchronous, decoupled, and scalable multi-agent systems.

Message passing supports:
- Asynchronous communication
- Loose coupling between agents
- Point-to-point and broadcast messaging
- Message queuing and buffering
- Reliable delivery guarantees
- Priority-based message handling

Components: 1. Message – Sender identification – Recipient(s) specification – Message type/category – Payload (data content) – Metadata (timestamp, priority, etc.) – Reply-to information

2. Message Bus/Broker
   - Routes messages between agents
   - Handles message queuing
   - Manages subscriptions
   - Ensures delivery
   - Provides message persistence
   - Supports patterns (pub/sub, request/reply)

3. Agent Interface
   - Send messages
   - Receive messages
   - Subscribe to topics
   - Handle message callbacks
   - Acknowledge receipt
   - Reply to messages

4. Message Protocol
   - Message format specification
   - Serialization/deserialization
   - Error handling conventions
   - Timeout management
   - Retry logic
   - Message versioning

Use Cases: 1. Multi-Agent Coordination – Task delegation – Status updates – Resource sharing – Result aggregation – Workflow orchestration – Event notification

2. Distributed Systems
   - Microservices communication
   - Remote procedure calls
   - Event-driven architectures
   - Stream processing
   - Job queues
   - Service integration

3. Collaborative Problem Solving
   - Information sharing
   - Consensus building
   - Parallel processing
   - Load distribution
   - Fault tolerance
   - Scalability

4. Real-Time Systems
   - Alert propagation
   - State synchronization
   - Live updates
   - Chat/messaging
   - Monitoring dashboards
   - Command execution

LangChain Implementation: LangChain supports message passing through: – Custom message passing classes – Callback handlers for events – LangGraph for state management – Integration with message brokers – Async message handling

Key Features: 1. Asynchronous Delivery – Non-blocking sends – Background processing – Queue-based buffering – Fire-and-forget option – Guaranteed delivery – Order preservation

2. Flexible Routing
   - Direct messaging (agent-to-agent)
   - Topic-based (publish/subscribe)
   - Broadcast (one-to-many)
   - Multicast (selected recipients)
   - Request/reply pattern
   - Router/filter chains

3. Reliability Features
   - Acknowledgments
   - Retry mechanisms
   - Dead letter queues
   - Message persistence
   - Delivery guarantees
   - Duplicate detection

4. Scalability
   - Horizontal scaling
   - Load balancing
   - Message partitioning
   - Connection pooling
   - Efficient serialization
   - Resource management

Best Practices: 1. Message Design – Keep messages small and focused – Include all necessary context – Use standard formats (JSON, Protocol Buffers) – Version messages for evolution – Include correlation IDs – Document message schemas

2. Error Handling
   - Implement retry logic with backoff
   - Use dead letter queues
   - Log failed messages
   - Monitor delivery rates
   - Timeout handling
   - Circuit breakers

3. Performance
   - Batch messages when possible
   - Use async/await patterns
   - Implement connection pooling
   - Cache frequently used data
   - Monitor queue depths
   - Optimize serialization

4. Security
   - Authenticate senders
   - Authorize operations
   - Encrypt sensitive data
   - Validate message content
   - Rate limiting
   - Audit trails

Trade-offs: Advantages: – Loose coupling – Asynchronous operation – Scalability – Fault tolerance – Flexibility – Technology independence

Disadvantages:
- Added complexity
- Potential message loss
- Ordering challenges
- Debugging difficulty
- Latency overhead
- Infrastructure requirements

Production Considerations: 1. Message Broker Selection – RabbitMQ: Feature-rich, reliable – Redis: Fast, simple pub/sub – Kafka: High throughput, persistent – AWS SQS/SNS: Managed, scalable – Google Pub/Sub: Global, managed – Azure Service Bus: Enterprise features

2. Monitoring
   - Message throughput
   - Queue depths
   - Delivery latency
   - Error rates
   - Dead letter queue size
   - Consumer lag

3. Scaling Strategy
   - Partition messages by key
   - Add consumer instances
   - Use competing consumers
   - Implement load shedding
   - Auto-scaling based on queue depth
   - Connection pooling

4. Reliability
   - At-least-once delivery
   - Idempotent consumers
   - Duplicate detection
   - Message persistence
   - Automatic retries
   - Dead letter handling

5. Operations
   - Message replay capability
   - Monitoring dashboards
   - Alert configuration
   - Log aggregation
   - Backup/recovery
   - Performance tuning

---

File Statistics

• Total Lines: 1098
• Classes: 7
• Functions: 1
• Constants: 0

---

Module Documentation

"""
Pattern 097: Message Passing

Description:
    Message Passing enables communication between agents through explicit message exchange.
    This pattern provides a structured way for agents to share information, coordinate actions,
    and collaborate on tasks without direct coupling. Messages are discrete units of information
    that flow between agents, enabling asynchronous, decoupled, and scalable multi-agent systems.

    Message passing supports:
    - Asynchronous communication
    - Loose coupling between agents
    - Point-to-point and broadcast messaging
    - Message queuing and buffering
    - Reliable delivery guarantees
    - Priority-based message handling

Components:
    1. Message
       - Sender identification
       - Recipient(s) specification
       - Message type/category
       - Payload (data content)
       - Metadata (timestamp, priority, etc.)
       - Reply-to information

    2. Message Bus/Broker
       - Routes messages between agents
       - Handles message queuing
       - Manages subscriptions
       - Ensures delivery
       - Provides message persistence
       - Supports patterns (pub/sub, request/reply)

    3. Agent Interface
       - Send messages
       - Receive messages
       - Subscribe to topics
       - Handle message callbacks
       - Acknowledge receipt
       - Reply to messages

    4. Message Protocol
       - Message format specification
       - Serialization/deserialization
       - Error handling conventions
       - Timeout management
       - Retry logic
       - Message versioning

Use Cases:
    1. Multi-Agent Coordination
       - Task delegation
       - Status updates
       - Resource sharing
       - Result aggregation
       - Workflow orchestration
       - Event notification

    2. Distributed Systems
       - Microservices communication
       - Remote procedure calls
       - Event-driven architectures
       - Stream processing
       - Job queues
       - Service integration

    3. Collaborative Problem Solving
       - Information sharing
       - Consensus building
       - Parallel processing
       - Load distribution
       - Fault tolerance
       - Scalability

    4. Real-Time Systems
       - Alert propagation
       - State synchronization
       - Live updates
       - Chat/messaging
       - Monitoring dashboards
       - Command execution

LangChain Implementation:
    LangChain supports message passing through:
    - Custom message passing classes
    - Callback handlers for events
    - LangGraph for state management
    - Integration with message brokers
    - Async message handling

Key Features:
    1. Asynchronous Delivery
       - Non-blocking sends
       - Background processing
       - Queue-based buffering
       - Fire-and-forget option
       - Guaranteed delivery
       - Order preservation

    2. Flexible Routing
       - Direct messaging (agent-to-agent)
       - Topic-based (publish/subscribe)
       - Broadcast (one-to-many)
       - Multicast (selected recipients)
       - Request/reply pattern
       - Router/filter chains

    3. Reliability Features
       - Acknowledgments
       - Retry mechanisms
       - Dead letter queues
       - Message persistence
       - Delivery guarantees
       - Duplicate detection

    4. Scalability
       - Horizontal scaling
       - Load balancing
       - Message partitioning
       - Connection pooling
       - Efficient serialization
       - Resource management

Best Practices:
    1. Message Design
       - Keep messages small and focused
       - Include all necessary context
       - Use standard formats (JSON, Protocol Buffers)
       - Version messages for evolution
       - Include correlation IDs
       - Document message schemas

    2. Error Handling
       - Implement retry logic with backoff
       - Use dead letter queues
       - Log failed messages
       - Monitor delivery rates
       - Timeout handling
       - Circuit breakers

    3. Performance
       - Batch messages when possible
       - Use async/await patterns
       - Implement connection pooling
       - Cache frequently used data
       - Monitor queue depths
       - Optimize serialization

    4. Security
       - Authenticate senders
       - Authorize operations
       - Encrypt sensitive data
       - Validate message content
       - Rate limiting
       - Audit trails

Trade-offs:
    Advantages:
    - Loose coupling
    - Asynchronous operation
    - Scalability
    - Fault tolerance
    - Flexibility
    - Technology independence

    Disadvantages:
    - Added complexity
    - Potential message loss
    - Ordering challenges
    - Debugging difficulty
    - Latency overhead
    - Infrastructure requirements

Production Considerations:
    1. Message Broker Selection
       - RabbitMQ: Feature-rich, reliable
       - Redis: Fast, simple pub/sub
       - Kafka: High throughput, persistent
       - AWS SQS/SNS: Managed, scalable
       - Google Pub/Sub: Global, managed
       - Azure Service Bus: Enterprise features

    2. Monitoring
       - Message throughput
       - Queue depths
       - Delivery latency
       - Error rates
       - Dead letter queue size
       - Consumer lag

    3. Scaling Strategy
       - Partition messages by key
       - Add consumer instances
       - Use competing consumers
       - Implement load shedding
       - Auto-scaling based on queue depth
       - Connection pooling

    4. Reliability
       - At-least-once delivery
       - Idempotent consumers
       - Duplicate detection
       - Message persistence
       - Automatic retries
       - Dead letter handling

    5. Operations
       - Message replay capability
       - Monitoring dashboards
       - Alert configuration
       - Log aggregation
       - Backup/recovery
       - Performance tuning
"""

Purpose: This module implements the Message Passing pattern, providing a structured approach to agent design and implementation.

---

Import Statements

Standard Library Imports

import collections.defaultdict
import dataclasses.asdict
import dataclasses.dataclass
import dataclasses.field
import datetime.datetime
import enum.Enum
import json
import langchain_core.output_parsers.StrOutputParser
import langchain_core.prompts.ChatPromptTemplate
import os
import threading.Lock
import threading.Thread
import time
import typing.Any
import typing.Callable
import typing.Dict
import typing.List
import typing.Optional

Purpose: Core Python functionality: – collections.defaultdict: Specialized container datatypes – dataclasses.asdict: Simplified class definitions with @dataclass – dataclasses.dataclass: Simplified class definitions with @dataclass – dataclasses.field: Simplified class definitions with @dataclass – datetime.datetime: Date and time handling – enum.Enum: Enumeration support – json: JSON serialization/deserialization – langchain_core.output_parsers.StrOutputParser: Regular expression operations – langchain_core.prompts.ChatPromptTemplate: Regular expression operations – os: Environment variable access and system operations – threading.Lock: Regular expression operations – threading.Thread: Regular expression operations – time: Time-related functions – typing.Any: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Callable: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Dict: Type hints for better code documentation (List, Dict, Any, Optional) – typing.List: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Optional: Type hints for better code documentation (List, Dict, Any, Optional)

LangChain Imports

from langchain_openai.ChatOpenAI

Purpose: LangChain framework components: – LLM Interface: Wrapper for OpenAI models – Chains: Composable building blocks for complex workflows

Other Imports

from dotenv.load_dotenv
from queue.Empty
from queue.Queue
from uuid

Purpose: Additional dependencies for specialized functionality.

---

Classes

Class: MessageType

Lines 237-243

Inherits from: Enum

Purpose:

Types of messages

---

Class: MessagePriority

Lines 246-251

Inherits from: Enum

Purpose:

Message priority levels

---

Class: Message

Lines 255-286

Purpose:

A message passed between agents.

Contains sender, recipient, type, payload, and metadata.

Methods:

to_dict(self)

Lines 272-278

Purpose:

Convert message to dictionary

Parameters:

Returns: <ast.Subscript object at 0x000001A06D6F7590>

from_dict(cls, data)

Lines 281-286

Decorators: classmethod

Purpose:

Create message from dictionary

Parameters: – cls: Cls – data: Data

Returns: Message

---

Class: MessageBus

Lines 289-476

Purpose:

Central message bus for routing messages between agents.

Supports point-to-point, publish/subscribe, and broadcast patterns.

Methods:

__init__(self)

Lines 296-304

Purpose:

Initialize message bus

Parameters:

register_agent(self, agent)

Lines 306-315

Purpose:

Register an agent with the bus.

Args:
    agent: Agent to register

Parameters: – agent: Agent

unregister_agent(self, agent_id)

Lines 317-331

Purpose:

Unregister an agent.

Args:
    agent_id: Agent identifier

Parameters: – agent_id: Agent id

subscribe(self, agent_id, topic)

Lines 333-344

Purpose:

Subscribe agent to a topic.

Args:
    agent_id: Agent identifier
    topic: Topic to subscribe to

Parameters: – agent_id: Agent id – topic: Topic

unsubscribe(self, agent_id, topic)

Lines 346-357

Purpose:

Unsubscribe agent from a topic.

Args:
    agent_id: Agent identifier
    topic: Topic to unsubscribe from

Parameters: – agent_id: Agent id – topic: Topic

send_message(self, message)

Lines 359-368

Purpose:

Send a message through the bus.

Args:
    message: Message to send

Parameters: – message: Message

publish(self, topic, sender, payload, priority)

Lines 370-394

Purpose:

Publish message to all subscribers of a topic.

Args:
    topic: Topic to publish to
    sender: Sender agent ID
    payload: Message payload
    priority: Message priority

Parameters: – topic: Topic – sender: Sender – payload: Payload – priority: Priority

broadcast(self, sender, payload, priority)

Lines 396-418

Purpose:

Broadcast message to all agents.

Args:
    sender: Sender agent ID
    payload: Message payload
    priority: Message priority

Parameters: – sender: Sender – payload: Payload – priority: Priority

start(self)

Lines 420-426

Purpose:

Start processing messages

Parameters:

stop(self)

Lines 428-433

Purpose:

Stop processing messages

Parameters:

_process_messages(self)

Lines 435-442

Purpose:

Process messages from queue (runs in background thread)

Parameters:

_deliver_message(self, message)

Lines 444-457

Purpose:

Deliver message to recipient.

Args:
    message: Message to deliver

Parameters: – message: Message

get_message_history(self, agent_id, limit)

Lines 459-476

Purpose:

Get message history.

Args:
    agent_id: Filter by agent (sender or recipient)
    limit: Maximum messages to return

Returns:
    List of messages

Parameters: – agent_id: Agent id – limit: Limit

Returns: <ast.Subscript object at 0x000001A06D7BB390>

---

Class: Agent

Lines 479-622

Purpose:

Base agent class with message passing capabilities.

Agents can send, receive, and process messages.

Methods:

__init__(self, agent_id, message_bus)

Lines 486-502

Purpose:

Initialize agent.

Args:
    agent_id: Unique agent identifier
    message_bus: Message bus for communication

Parameters: – agent_id: Agent id – message_bus: Message bus

send_message(self, recipient, message_type, payload, priority, reply_to)

Lines 504-526

Purpose:

Send a message to another agent.

Args:
    recipient: Recipient agent ID
    message_type: Type of message
    payload: Message payload
    priority: Message priority
    reply_to: Message ID this is replying to

Parameters: – recipient: Recipient – message_type: Message type – payload: Payload – priority: Priority – reply_to: Reply to

receive_message(self, message)

Lines 528-535

Purpose:

Receive a message (called by message bus).

Args:
    message: Received message

Parameters: – message: Message

subscribe(self, topic)

Lines 537-544

Purpose:

Subscribe to a topic.

Args:
    topic: Topic to subscribe to

Parameters: – topic: Topic

publish(self, topic, payload, priority)

Lines 546-555

Purpose:

Publish to a topic.

Args:
    topic: Topic to publish to
    payload: Message payload
    priority: Message priority

Parameters: – topic: Topic – payload: Payload – priority: Priority

broadcast(self, payload, priority)

Lines 557-565

Purpose:

Broadcast message to all agents.

Args:
    payload: Message payload
    priority: Message priority

Parameters: – payload: Payload – priority: Priority

register_handler(self, message_type, handler)

Lines 567-575

Purpose:

Register handler for message type.

Args:
    message_type: Type of message
    handler: Handler function

Parameters: – message_type: Message type – handler: Handler

start(self)

Lines 577-582

Purpose:

Start processing messages

Parameters:

stop(self)

Lines 584-588

Purpose:

Stop processing messages

Parameters:

_process_inbox(self)

Lines 590-597

Purpose:

Process incoming messages (runs in background thread)

Parameters:

_handle_message(self, message)

Lines 599-613

Purpose:

Handle incoming message.

Args:
    message: Message to handle

Parameters: – message: Message

handle_message(self, message)

Lines 615-622

Purpose:

Default message handler (override in subclasses).

Args:
    message: Message to handle

Parameters: – message: Message

---

Class: WorkerAgent

Lines 625-657

Inherits from: Agent

Purpose:

Example worker agent that processes tasks.

Demonstrates request/response pattern.

Methods:

__init__(self, agent_id, message_bus)

Lines 632-635

Purpose:

Initialize worker agent

Parameters: – agent_id: Agent id – message_bus: Message bus

handle_request(self, message)

Lines 637-657

Purpose:

Handle task request.

Args:
    message: Request message

Parameters: – message: Message

---

Class: CoordinatorAgent

Lines 660-707

Inherits from: Agent

Purpose:

Example coordinator agent that delegates work.

Demonstrates task delegation pattern.

Methods:

__init__(self, agent_id, message_bus)

Lines 667-672

Purpose:

Initialize coordinator agent

Parameters: – agent_id: Agent id – message_bus: Message bus

delegate_task(self, worker_id, task)

Lines 674-693

Purpose:

Delegate task to worker.

Args:
    worker_id: Worker agent ID
    task: Task to delegate

Parameters: – worker_id: Worker id – task: Task

handle_response(self, message)

Lines 695-707

Purpose:

Handle response from worker.

Args:
    message: Response message

Parameters: – message: Message

---

Functions

demonstrate_message_passing()

Lines 710-1093

Purpose:

Demonstrate message passing pattern

---

Key Concepts Summary

Pattern Overview

The Message Passing pattern provides a structured approach to building AI agents with specific capabilities and behaviors. This implementation leverages LangChain’s composable components to create a robust and maintainable solution.

Core Components

1. Agent Class: Main implementation of the pattern logic
2. Prompt Engineering: Carefully crafted prompts to guide LLM behavior
3. Tool Integration: Optional tools for extended functionality
4. Memory Management: State handling across interactions
5. Output Parsing: Structured response processing

When to Use

✅ Good for: – Scenarios requiring this specific pattern’s strengths – Applications needing structured agent behavior – Systems where explainability is important – Production environments requiring reliability

❌ Not ideal for: – Simple single-prompt tasks – Scenarios requiring different pattern characteristics – Resource-constrained environments (if computationally intensive)

---

Best Practices Demonstrated

1. Environment Configuration

• ✅ Use .env files for API keys
• ✅ Never hardcode credentials
• ✅ Validate environment variables on startup

2. Error Handling

• ✅ Graceful degradation on failures
• ✅ Informative error messages
• ✅ Logging for debugging

3. Prompt Engineering

• ✅ Clear, specific instructions
• ✅ Structured output formats
• ✅ Few-shot examples when beneficial

4. Code Organization

• ✅ Separation of concerns (class-based design)
• ✅ Type hints for clarity
• ✅ Docstrings for documentation

5. Performance Optimization

• ✅ Appropriate temperature settings
• ✅ Token budget management
• ✅ Caching where applicable

---

LangChain Components Used

• ChatOpenAI: OpenAI model interface
• Chains: Sequential component composition

Why LangChain? – Composable abstractions for LLM applications – Rich ecosystem of integrations – Production-ready patterns – Active community and development

---

Execution Flow

1. Initialize agent with configuration
   ↓
2. Load environment and dependencies
   ↓
3. Prepare prompt with input data
   ↓
4. Invoke LLM through LangChain
   ↓
5. Parse and validate response
   ↓
6. Return formatted result

Key Decision Points: – Model selection (temperature, model type) – Prompt structure (few-shot vs zero-shot) – Error handling strategy – Output format validation

---

Performance Considerations

Computational Cost

• LLM Calls: Primary cost factor (based on tokens)
• Latency: Depends on model and input size
• Memory: Minimal for most patterns

Optimization Strategies

1. Prompt Optimization: Minimize tokens while maintaining clarity
2. Caching: Cache repeated queries/responses
3. Model Selection: Use smaller models when appropriate
4. Batch Processing: Group similar requests
5. Streaming: For long responses, stream tokens

Cost Estimates (Approximate)

• GPT-3.5-turbo: ~$0.002 per request
• GPT-4: ~$0.03 per request
• GPT-4-turbo: ~$0.01 per request

Actual costs vary based on input/output token counts

---

Common Pitfalls to Avoid

1. Prompt Design Issues

❌ Vague or ambiguous instructions
✅ Clear, specific, testable prompts

2. Error Handling

❌ Ignoring API errors or rate limits
✅ Implement retry logic and graceful degradation

3. Token Management

❌ Exceeding model token limits
✅ Truncate inputs, chunk processing

4. Security

❌ Exposing API keys in code
✅ Use environment variables

5. Testing

❌ Only testing happy paths
✅ Test edge cases, failures, malformed inputs

6. Cost Management

❌ Unmonitored API usage
✅ Implement budgets, alerts, and monitoring

---

Further Improvements

Production Enhancements

1. Monitoring: Add logging, metrics, tracing
2. Validation: Input/output schema validation
3. Rate Limiting: Implement request throttling
4. Caching: Redis or in-memory cache for responses
5. Async Support: Non-blocking operations for scalability

Feature Extensions

1. Multi-Model Support: Switch between providers
2. Fallback Models: Graceful degradation strategy
3. Custom Tools: Domain-specific function calling
4. Advanced Memory: Vector stores for long-term memory
5. Evaluation: Automated testing and benchmarking

Code Quality

1. Testing: Unit tests, integration tests
2. Documentation: API docs, usage examples
3. Type Safety: Stricter type hints
4. Configuration: Externalized config files

---

Related Patterns

• Pattern 096
• Pattern 098

Pattern Families: – Reasoning Patterns (001-011) – Multi-Agent Patterns (012-019) – Tool Use Patterns (020-025) – Memory Patterns (026-031) – And many more…

---

Summary

The Message Passing pattern provides a powerful approach to building AI agents with specific capabilities. This implementation demonstrates:

✅ Key Strengths: – Clean, maintainable code structure – LangChain integration for composability – Type hints and documentation – Production-ready patterns

🎯 Best Use Cases: – Applications requiring this pattern’s characteristics – Systems needing structured agent behavior – Production environments requiring reliability

📚 Learning Resources: – LangChain Documentation: https://python.langchain.com/ – Pattern Documentation: See related patterns – OpenAI API Reference: https://platform.openai.com/docs/

---

File: 097_message_passing.py
Total Lines: 1098
Pattern Category: AI Agent Design Pattern #097

---

Generated from actual implementation code
For educational and reference purposes

    """
Pattern 096: Performance Profiling

Description:
    Performance Profiling systematically measures and analyzes agent execution to identify
    bottlenecks, optimize resource usage, and improve response times. This pattern provides
    detailed insights into where time and resources are spent, enabling data-driven
    optimization decisions. Profiling is essential for production systems where performance,
    cost, and user experience are critical.

    Key metrics profiled:
    - Execution time (total, per step, per component)
    - LLM token usage (input, output, total)
    - API call counts and latency
    - Memory usage and allocation patterns
    - Cache hit/miss rates
    - Concurrent operation efficiency
    - Cost per operation

Components:
    1. Profiler
       - Time measurement (wall clock, CPU)
       - Memory tracking
       - Token counting
       - API call monitoring
       - Resource utilization
       - Custom metric collection

    2. Metrics Collector
       - Aggregates measurements
       - Calculates statistics
       - Tracks trends over time
       - Identifies outliers
       - Correlates metrics
       - Exports data

    3. Analyzer
       - Identifies bottlenecks
       - Compares components
       - Detects regressions
       - Suggests optimizations
       - Benchmarks performance
       - Generates reports

    4. Visualizer
       - Flame graphs
       - Timeline charts
       - Breakdown pie charts
       - Trend graphs
       - Comparison tables
       - Heatmaps

Use Cases:
    1. Performance Optimization
       - Identify slow components
       - Optimize hot paths
       - Reduce latency
       - Improve throughput
       - Minimize resource usage
       - Cost reduction

    2. Capacity Planning
       - Understand resource needs
       - Plan scaling requirements
       - Budget forecasting
       - Load handling capacity
       - Infrastructure sizing
       - Cost projections

    3. Regression Detection
       - Catch performance degradation
       - Validate optimizations
       - Monitor after deployments
       - Track trends over time
       - Alert on anomalies
       - Automated testing

    4. Debugging & Root Cause
       - Understand slow requests
       - Identify failure points
       - Analyze edge cases
       - Reproduce issues
       - Validate fixes
       - Production troubleshooting

    5. SLA Compliance
       - Monitor latency targets
       - Track availability
       - Measure throughput
       - Ensure quality of service
       - Report on SLAs
       - Alert on violations

LangChain Implementation:
    LangChain supports profiling through:
    - Callback handlers for timing
    - Token usage tracking
    - LangSmith for comprehensive monitoring
    - Custom profiling wrappers
    - Integration with monitoring tools

Key Features:
    1. Low Overhead
       - Minimal performance impact (< 2%)
       - Efficient data collection
       - Sampling strategies
       - Async operations
       - Batch processing

    2. Comprehensive Coverage
       - End-to-end timing
       - Per-component breakdown
       - LLM call analysis
       - Tool execution timing
       - Memory profiling
       - Network latency

    3. Actionable Insights
       - Clear bottleneck identification
       - Optimization suggestions
       - Cost breakdowns
       - Performance comparisons
       - Trend analysis
       - Automated alerts

    4. Production Ready
       - Always-on capability
       - Sampling support
       - Secure data handling
       - Scalable architecture
       - Integration friendly
       - Export options

Best Practices:
    1. What to Profile
       - Total execution time
       - LLM calls (time, tokens, cost)
       - Tool executions
       - Retrieval operations
       - Parsing/processing time
       - Network calls
       - Database queries
       - Cache operations

    2. Metric Collection
       - Start time, end time, duration
       - Token counts (input, output, total)
       - API call counts
       - Memory usage (peak, average)
       - Error rates
       - Retry counts
       - Cache hit rates

    3. Analysis Techniques
       - Identify top time consumers
       - Calculate percentiles (p50, p95, p99)
       - Detect outliers
       - Compare before/after
       - Trend analysis
       - Cost attribution

    4. Optimization Priorities
       - Focus on hot paths (80/20 rule)
       - Optimize high-frequency calls
       - Reduce expensive operations
       - Improve cache usage
       - Batch when possible
       - Parallel execution

Trade-offs:
    Advantages:
    - Identifies bottlenecks
    - Data-driven optimization
    - Cost reduction
    - Improved user experience
    - Regression detection
    - Capacity planning

    Disadvantages:
    - Small overhead (1-2%)
    - Implementation complexity
    - Storage requirements
    - Analysis time needed
    - May expose limitations
    - Maintenance burden

Production Considerations:
    1. Sampling Strategy
       - Development: 100% profiling
       - Staging: 50-100%
       - Production: 1-10% baseline
       - Slow requests: Always profile
       - Adaptive sampling

    2. Storage & Retention
       - Hot data: 7 days (full detail)
       - Warm data: 30 days (aggregated)
       - Cold data: 90+ days (summary)
       - Compression for old data
       - Archival policies

    3. Performance Impact
       - Target:  str:
        """
        Start profiling an execution.
        
        Args:
            execution_id: Unique execution identifier
            metadata: Optional metadata
            
        Returns:
            Execution ID
        """
        profile = ExecutionProfile(
            execution_id=execution_id,
            start_time=datetime.now(),
            metadata=metadata or {}
        )
        self.current_profile = profile
        self.profiles.append(profile)
        return execution_id
    
    def end_execution(self):
        """End profiling current execution"""
        if self.current_profile:
            self.current_profile.finalize()
            self.current_profile = None
    
    def start_component(self, component_name: str, metadata: Optional[Dict[str, Any]] = None) -> ComponentProfile:
        """
        Start profiling a component.
        
        Args:
            component_name: Component name
            metadata: Optional metadata
            
        Returns:
            Component profile
        """
        component = ComponentProfile(
            component_name=component_name,
            start_time=datetime.now(),
            metadata=metadata or {}
        )
        
        if self.current_profile:
            self.current_profile.add_component(component)
        
        self.component_stack.append(component)
        return component
    
    def end_component(self):
        """End profiling current component"""
        if self.component_stack:
            component = self.component_stack.pop()
            component.finalize()
    
    def add_metric(self, metric: ProfileMetric):
        """Add metric to current component"""
        if self.component_stack:
            self.component_stack[-1].add_metric(metric)
    
    def get_profile(self, execution_id: str) -> Optional[ExecutionProfile]:
        """Get profile by execution ID"""
        for profile in self.profiles:
            if profile.execution_id == execution_id:
                return profile
        return None
    
    def clear_profiles(self):
        """Clear all profiles"""
        self.profiles.clear()
        self.current_profile = None
        self.component_stack.clear()


class ProfileAnalyzer:
    """
    Analyzes profiling data to identify bottlenecks and trends.
    
    Provides insights and optimization recommendations.
    """
    
    def __init__(self):
        """Initialize analyzer"""
        pass
    
    def analyze_execution(self, profile: ExecutionProfile) -> Dict[str, Any]:
        """
        Analyze execution profile.
        
        Args:
            profile: Execution profile
            
        Returns:
            Analysis results
        """
        if not profile.components:
            return {
                "total_duration_ms": profile.total_duration_ms or 0,
                "component_count": 0,
                "components": []
            }
        
        # Component timing breakdown
        components_analysis = []
        total_component_time = 0.0
        
        for component in profile.components:
            if component.duration_ms:
                total_component_time += component.duration_ms
                
                components_analysis.append({
                    "name": component.component_name,
                    "duration_ms": component.duration_ms,
                    "percentage": 0.0,  # Will calculate after
                    "metric_count": len(component.metrics)
                })
        
        # Calculate percentages
        if total_component_time > 0:
            for comp in components_analysis:
                comp["percentage"] = (comp["duration_ms"] / total_component_time) * 100
        
        # Sort by duration
        components_analysis.sort(key=lambda c: c["duration_ms"], reverse=True)
        
        return {
            "execution_id": profile.execution_id,
            "total_duration_ms": profile.total_duration_ms or 0,
            "component_count": len(profile.components),
            "total_component_time_ms": total_component_time,
            "components": components_analysis,
            "overhead_ms": (profile.total_duration_ms or 0) - total_component_time
        }
    
    def identify_bottlenecks(self, profile: ExecutionProfile, threshold_pct: float = 20.0) -> List[Dict[str, Any]]:
        """
        Identify bottleneck components.
        
        Args:
            profile: Execution profile
            threshold_pct: Threshold percentage for bottleneck
            
        Returns:
            List of bottlenecks
        """
        analysis = self.analyze_execution(profile)
        
        bottlenecks = []
        for component in analysis["components"]:
            if component["percentage"] >= threshold_pct:
                bottlenecks.append({
                    "component": component["name"],
                    "duration_ms": component["duration_ms"],
                    "percentage": component["percentage"],
                    "severity": "high" if component["percentage"] >= 40 else "medium"
                })
        
        return bottlenecks
    
    def calculate_statistics(self, profiles: List[ExecutionProfile]) -> Dict[str, Any]:
        """
        Calculate statistics across multiple profiles.
        
        Args:
            profiles: List of execution profiles
            
        Returns:
            Statistical summary
        """
        if not profiles:
            return {}
        
        durations = [p.total_duration_ms for p in profiles if p.total_duration_ms]
        
        if not durations:
            return {}
        
        durations.sort()
        count = len(durations)
        
        return {
            "count": count,
            "min_ms": durations[0],
            "max_ms": durations[-1],
            "mean_ms": sum(durations) / count,
            "median_ms": durations[count // 2],
            "p95_ms": durations[int(count * 0.95)] if count > 1 else durations[0],
            "p99_ms": durations[int(count * 0.99)] if count > 1 else durations[0]
        }
    
    def compare_profiles(self, profile1: ExecutionProfile, profile2: ExecutionProfile) -> Dict[str, Any]:
        """
        Compare two profiles.
        
        Args:
            profile1: First profile (baseline)
            profile2: Second profile (current)
            
        Returns:
            Comparison results
        """
        analysis1 = self.analyze_execution(profile1)
        analysis2 = self.analyze_execution(profile2)
        
        duration_change = analysis2["total_duration_ms"] - analysis1["total_duration_ms"]
        duration_change_pct = (duration_change / analysis1["total_duration_ms"]) * 100 if analysis1["total_duration_ms"] > 0 else 0
        
        return {
            "baseline_duration_ms": analysis1["total_duration_ms"],
            "current_duration_ms": analysis2["total_duration_ms"],
            "change_ms": duration_change,
            "change_percentage": duration_change_pct,
            "is_regression": duration_change_pct > 10,  # > 10% slower
            "is_improvement": duration_change_pct  10% faster
        }
    
    def generate_recommendations(self, profile: ExecutionProfile) -> List[str]:
        """
        Generate optimization recommendations.
        
        Args:
            profile: Execution profile
            
        Returns:
            List of recommendations
        """
        recommendations = []
        bottlenecks = self.identify_bottlenecks(profile)
        
        for bottleneck in bottlenecks:
            component = bottleneck["component"]
            
            if "llm" in component.lower():
                recommendations.append(
                    f"Consider caching LLM responses for {component} (takes {bottleneck['percentage']:.1f}% of time)"
                )
                recommendations.append(
                    f"Try using a faster model for {component} if high accuracy isn't critical"
                )
            elif "retrieval" in component.lower() or "search" in component.lower():
                recommendations.append(
                    f"Optimize retrieval in {component} - consider better indexing or caching"
                )
            elif "tool" in component.lower():
                recommendations.append(
                    f"Tool execution in {component} is slow - consider async execution or caching"
                )
            else:
                recommendations.append(
                    f"Investigate {component} - it accounts for {bottleneck['percentage']:.1f}% of execution time"
                )
        
        if not bottlenecks:
            recommendations.append("No significant bottlenecks detected - performance is well-balanced")
        
        return recommendations


class ProfileVisualizer:
    """
    Visualizes profiling data.
    
    Creates text-based visualizations for analysis.
    """
    
    def __init__(self):
        """Initialize visualizer"""
        pass
    
    def visualize_breakdown(self, analysis: Dict[str, Any]) -> str:
        """
        Visualize time breakdown by component.
        
        Args:
            analysis: Analysis results
            
        Returns:
            Text visualization
        """
        lines = []
        lines.append("=" * 80)
        lines.append(f"EXECUTION BREAKDOWN: {analysis['execution_id']}")
        lines.append("=" * 80)
        lines.append(f"Total Duration: {analysis['total_duration_ms']:.2f}ms")
        lines.append(f"Components: {analysis['component_count']}")
        lines.append("")
        
        if analysis["components"]:
            lines.append("Time by Component:")
            lines.append("")
            
            for component in analysis["components"]:
                # Create bar
                bar_length = int((component["percentage"] / 100) * 40)
                bar = "█" * bar_length
                
                lines.append(f"{component['name']:30s} {bar} {component['percentage']:5.1f}% ({component['duration_ms']:7.2f}ms)")
        
        return "\n".join(lines)
    
    def visualize_statistics(self, stats: Dict[str, Any]) -> str:
        """
        Visualize statistics.
        
        Args:
            stats: Statistics dictionary
            
        Returns:
            Text visualization
        """
        lines = []
        lines.append("=" * 80)
        lines.append("PERFORMANCE STATISTICS")
        lines.append("=" * 80)
        lines.append(f"Sample Size: {stats.get('count', 0)} executions")
        lines.append("")
        lines.append(f"Min:     {stats.get('min_ms', 0):8.2f}ms")
        lines.append(f"Median:  {stats.get('median_ms', 0):8.2f}ms")
        lines.append(f"Mean:    {stats.get('mean_ms', 0):8.2f}ms")
        lines.append(f"P95:     {stats.get('p95_ms', 0):8.2f}ms")
        lines.append(f"P99:     {stats.get('p99_ms', 0):8.2f}ms")
        lines.append(f"Max:     {stats.get('max_ms', 0):8.2f}ms")
        
        return "\n".join(lines)


class ProfiledAgent:
    """
    Example agent with comprehensive profiling.
    
    Demonstrates performance profiling in practice.
    """
    
    def __init__(self, profiler: Profiler):
        """
        Initialize profiled agent.
        
        Args:
            profiler: Profiler instance
        """
        self.profiler = profiler
        self.llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0.7)
    
    def process_query(self, query: str, execution_id: str) -> str:
        """
        Process query with profiling.
        
        Args:
            query: User query
            execution_id: Execution identifier
            
        Returns:
            Response
        """
        # Start execution profiling
        self.profiler.start_execution(execution_id, {"query": query})
        
        try:
            # Step 1: Query analysis
            self.profiler.start_component("query_analysis")
            time.sleep(0.05)  # Simulate processing
            query_type = "question" if "?" in query else "statement"
            self.profiler.end_component()
            
            # Step 2: LLM call
            self.profiler.start_component("llm_generation")
            start_time = time.time()
            response = self._call_llm(query)
            llm_duration = (time.time() - start_time) * 1000
            
            # Add metrics
            self.profiler.add_metric(ProfileMetric(
                name="llm_tokens_input",
                metric_type=MetricType.TOKENS,
                value=len(query.split()) * 1.3,  # Approximate
                unit="tokens",
                timestamp=datetime.now()
            ))
            self.profiler.add_metric(ProfileMetric(
                name="llm_tokens_output",
                metric_type=MetricType.TOKENS,
                value=len(response.split()) * 1.3,  # Approximate
                unit="tokens",
                timestamp=datetime.now()
            ))
            
            self.profiler.end_component()
            
            # Step 3: Post-processing
            self.profiler.start_component("post_processing")
            time.sleep(0.02)  # Simulate processing
            processed_response = response.strip()
            self.profiler.end_component()
            
            # End execution profiling
            self.profiler.end_execution()
            
            return processed_response
            
        except Exception as e:
            self.profiler.end_execution()
            raise
    
    def _call_llm(self, query: str) -> str:
        """Call LLM"""
        prompt = ChatPromptTemplate.from_template("Answer concisely: {query}")
        chain = prompt | self.llm | StrOutputParser()
        return chain.invoke({"query": query})


def demonstrate_performance_profiling():
    """Demonstrate performance profiling"""
    print("=" * 80)
    print("PERFORMANCE PROFILING DEMONSTRATION")
    print("=" * 80)
    
    profiler = Profiler()
    analyzer = ProfileAnalyzer()
    visualizer = ProfileVisualizer()
    
    # Example 1: Basic Profiling
    print("\n" + "=" * 80)
    print("EXAMPLE 1: Basic Component Profiling")
    print("=" * 80)
    
    execution_id = "exec_001"
    profiler.start_execution(execution_id, {"user": "demo"})
    
    # Simulate components
    profiler.start_component("initialization")
    time.sleep(0.05)
    profiler.end_component()
    
    profiler.start_component("data_loading")
    time.sleep(0.1)
    profiler.add_metric(ProfileMetric(
        name="records_loaded",
        metric_type=MetricType.COUNT,
        value=100,
        unit="records",
        timestamp=datetime.now()
    ))
    profiler.end_component()
    
    profiler.start_component("processing")
    time.sleep(0.15)
    profiler.end_component()
    
    profiler.start_component("output_formatting")
    time.sleep(0.03)
    profiler.end_component()
    
    profiler.end_execution()
    
    # Analyze
    profile = profiler.get_profile(execution_id)
    if profile:
        analysis = analyzer.analyze_execution(profile)
        print("\n" + visualizer.visualize_breakdown(analysis))
    
    # Example 2: Bottleneck Identification
    print("\n" + "=" * 80)
    print("EXAMPLE 2: Bottleneck Identification")
    print("=" * 80)
    
    if profile:
        bottlenecks = analyzer.identify_bottlenecks(profile, threshold_pct=15.0)
        
        print(f"\nIdentified {len(bottlenecks)} bottleneck(s):\n")
        for bottleneck in bottlenecks:
            print(f"Component: {bottleneck['component']}")
            print(f"  Duration: {bottleneck['duration_ms']:.2f}ms")
            print(f"  Percentage: {bottleneck['percentage']:.1f}%")
            print(f"  Severity: {bottleneck['severity']}")
            print()
    
    # Example 3: Recommendations
    print("\n" + "=" * 80)
    print("EXAMPLE 3: Optimization Recommendations")
    print("=" * 80)
    
    if profile:
        recommendations = analyzer.generate_recommendations(profile)
        
        print("\nRecommendations:")
        for i, rec in enumerate(recommendations, 1):
            print(f"{i}. {rec}")
    
    # Example 4: Profiled Agent Execution
    print("\n" + "=" * 80)
    print("EXAMPLE 4: Full Agent Execution Profiling")
    print("=" * 80)
    
    profiler.clear_profiles()
    agent = ProfiledAgent(profiler)
    
    print("\nExecuting agent with profiling...")
    response = agent.process_query("What is artificial intelligence?", "exec_002")
    print(f"\nResponse: {response[:100]}...")
    
    # Analyze agent execution
    agent_profile = profiler.get_profile("exec_002")
    if agent_profile:
        agent_analysis = analyzer.analyze_execution(agent_profile)
        print("\n" + visualizer.visualize_breakdown(agent_analysis))
    
    # Example 5: Statistical Analysis
    print("\n" + "=" * 80)
    print("EXAMPLE 5: Statistical Analysis Across Multiple Runs")
    print("=" * 80)
    
    # Simulate multiple executions
    print("\nRunning 10 executions...")
    for i in range(10):
        agent.process_query(f"Query {i}", f"exec_batch_{i}")
    
    # Calculate statistics
    all_profiles = profiler.profiles
    stats = analyzer.calculate_statistics(all_profiles)
    
    print("\n" + visualizer.visualize_statistics(stats))
    
    # Example 6: Performance Comparison
    print("\n" + "=" * 80)
    print("EXAMPLE 6: Before/After Comparison")
    print("=" * 80)
    
    # Get two profiles to compare
    if len(all_profiles) >= 2:
        baseline = all_profiles[0]
        current = all_profiles[-1]
        
        comparison = analyzer.compare_profiles(baseline, current)
        
        print("\nPerformance Comparison:")
        print(f"Baseline: {comparison['baseline_duration_ms']:.2f}ms")
        print(f"Current:  {comparison['current_duration_ms']:.2f}ms")
        print(f"Change:   {comparison['change_ms']:+.2f}ms ({comparison['change_percentage']:+.1f}%)")
        
        if comparison['is_regression']:
            print("⚠️  REGRESSION DETECTED - Performance degraded")
        elif comparison['is_improvement']:
            print("✓ IMPROVEMENT - Performance improved")
        else:
            print("→ STABLE - No significant change")
    
    # Example 7: Token Usage Analysis
    print("\n" + "=" * 80)
    print("EXAMPLE 7: Token Usage and Cost Analysis")
    print("=" * 80)
    
    if agent_profile:
        total_input_tokens = 0
        total_output_tokens = 0
        
        for component in agent_profile.components:
            for metric in component.metrics:
                if "input" in metric.name:
                    total_input_tokens += metric.value
                elif "output" in metric.name:
                    total_output_tokens += metric.value
        
        total_tokens = total_input_tokens + total_output_tokens
        
        # Approximate costs (GPT-3.5-turbo pricing)
        input_cost = (total_input_tokens / 1000) * 0.0015
        output_cost = (total_output_tokens / 1000) * 0.002
        total_cost = input_cost + output_cost
        
        print("\nToken Usage:")
        print(f"Input tokens:  {total_input_tokens:.0f}")
        print(f"Output tokens: {total_output_tokens:.0f}")
        print(f"Total tokens:  {total_tokens:.0f}")
        print(f"\nEstimated Cost:")
        print(f"Input cost:    ${input_cost:.6f}")
        print(f"Output cost:   ${output_cost:.6f}")
        print(f"Total cost:    ${total_cost:.6f}")
    
    # Summary
    print("\n" + "=" * 80)
    print("PERFORMANCE PROFILING SUMMARY")
    print("=" * 80)
    print("""
Performance Profiling Benefits:
1. Identify Bottlenecks: Find slow components
2. Data-Driven Optimization: Focus efforts where they matter
3. Cost Control: Track and optimize API usage
4. Regression Detection: Catch performance degradation
5. Capacity Planning: Understand resource needs
6. SLA Compliance: Monitor latency targets

Key Metrics to Track:
1. Execution Time
   - Total duration (wall clock)
   - Per-component breakdown
   - CPU time vs wall time
   - Percentiles (p50, p95, p99)
   - Min/max/mean/median

2. Token Usage
   - Input tokens per call
   - Output tokens per call
   - Total tokens per execution
   - Cost per execution
   - Cost trends over time

3. API Calls
   - Call counts by endpoint
   - Latency per call
   - Error rates
   - Retry counts
   - Rate limit hits

4. Memory Usage
   - Peak memory
   - Average memory
   - Memory leaks detection
   - Allocation patterns
   - Cache memory

5. Cache Performance
   - Hit rate
   - Miss rate
   - Cache size
   - Eviction rate
   - Time saved

6. Throughput
   - Requests per second
   - Concurrent operations
   - Queue lengths
   - Success rate
   - Error rate

Profiling Workflow:
```
1. Instrument Code
   ├── Add profiling wrappers
   ├── Track entry/exit points
   ├── Collect metrics
   └── Handle errors

2. Execute & Collect
   ├── Run profiled code
   ├── Gather measurements
   ├── Store profile data
   └── Tag with metadata

3. Analyze Data
   ├── Calculate statistics
   ├── Identify bottlenecks
   ├── Compare versions
   └── Detect trends

4. Generate Insights
   ├── Bottleneck ranking
   ├── Optimization suggestions
   ├── Cost breakdown
   └── Performance report

5. Optimize & Repeat
   ├── Implement improvements
   ├── Profile again
   ├── Validate changes
   └── Monitor trends
```

Analysis Techniques:
1. Time Breakdown
   - Which component takes most time?
   - What percentage per component?
   - Sequential vs parallel?
   - Overhead calculation

2. Statistical Analysis
   - Mean, median, mode
   - Standard deviation
   - Percentiles (p50, p95, p99)
   - Outlier detection
   - Trend analysis

3. Comparison Analysis
   - Before vs after optimization
   - Version A vs version B
   - Expected vs actual
   - Production vs development
   - Baseline tracking

4. Cost Analysis
   - Cost per execution
   - Cost by component
   - Cost trends
   - Budget projection
   - ROI of optimizations

Optimization Strategies:
1. Caching (Highest Impact)
   - Cache LLM responses
   - Cache retrieval results
   - Cache computations
   - Set appropriate TTL
   - Invalidation strategy

2. Model Selection
   - Use faster models when possible
   - GPT-3.5 vs GPT-4 trade-off
   - Smaller models for simple tasks
   - Local models for common patterns
   - Model routing by complexity

3. Parallel Execution
   - Independent operations in parallel
   - Batch API calls
   - Async processing
   - Resource pooling
   - Load balancing

4. Prompt Optimization
   - Shorter prompts (less tokens)
   - Clear instructions (less retries)
   - Better examples (higher success)
   - Output format optimization
   - Token budget management

5. Retrieval Optimization
   - Better indexing
   - Smaller chunks
   - Top-k optimization
   - Embedding cache
   - Pre-computed similarities

6. Algorithm Improvements
   - More efficient logic
   - Early termination
   - Lazy evaluation
   - Result reuse
   - Approximations when appropriate

Production Profiling:
1. Sampling Strategy
   - Baseline: 1-5% always on
   - Errors: 100% profiling
   - Slow requests: 100% profiling
   - Adaptive: Increase on issues
   - Feature flags for control

2. Overhead Management
   - Target:  threshold
   - Error rate spike
   - Cost > budget
   - Regression detected
   - Resource exhaustion

5. Integration
   - Export to monitoring (DataDog, New Relic)
   - Dashboard visualization (Grafana)
   - CI/CD integration
   - Automated regression tests
   - Alert systems

Common Bottlenecks:
1. LLM Calls (Most Common)
   - Solution: Caching, faster models, parallel calls
   - Impact: 50-80% of total time

2. Retrieval/Search
   - Solution: Better indexing, caching, smaller top-k
   - Impact: 10-30% of total time

3. Tool Execution
   - Solution: Async execution, caching, timeout optimization
   - Impact: 5-20% of total time

4. Parsing/Processing
   - Solution: Efficient algorithms, caching, lazy evaluation
   - Impact: 5-15% of total time

5. Network Latency
   - Solution: Regional deployment, CDN, connection pooling
   - Impact: Variable

Profiling Tools:
- LangSmith: Native LangChain profiling
- cProfile: Python profiling
- memory_profiler: Memory tracking
- py-spy: Sampling profiler
- OpenTelemetry: Distributed tracing
- Custom: Roll your own

Best Practices:
1. ✓ Profile early and often
2. ✓ Focus on hot paths (80/20 rule)
3. ✓ Measure before optimizing
4. ✓ Track trends over time
5. ✓ Set performance budgets
6. ✓ Automate profiling in CI/CD
7. ✓ Profile production (with sampling)
8. ✓ Document optimizations
9. ✗ Don't optimize prematurely
10. ✗ Don't profile everything (overhead)

Metrics Dashboard Example:
```
=== Agent Performance Dashboard ===

Latency:
  P50: 1,250ms  ████████████████████ (Good)
  P95: 2,800ms  ████████████████████████████ (Warning)
  P99: 5,200ms  ████████████████████████████████████ (Alert)

Cost (last 24h):
  Total: $12.45
  Per request: $0.0025
  Trend: ↓ 15% (Improving)

Bottlenecks:
  1. LLM calls:    65% of time
  2. Retrieval:    20% of time
  3. Processing:   10% of time
  4. Other:         5% of time

Cache Performance:
  Hit rate:  78%  (Good)
  Saved:     $3.20 today
  Avg speedup: 850ms per hit
```

When to Profile:
✓ Development (optimize hot paths)
✓ Pre-production (validate performance)
✓ Production (monitor continuously)
✓ After changes (detect regressions)
✓ On errors (understand failures)
✓ Performance issues (debug slowness)
✗ Every single request (too much overhead)

ROI Analysis:
- Optimization time: 1-2 days
- Performance gain: 30-50% faster
- Cost reduction: 20-40%
- User satisfaction: +15-25%
- Engineering efficiency: +30%
- Total ROI: Very high
""")
    
    print("\n" + "=" * 80)
    print("Pattern 096 (Performance Profiling) demonstration complete!")
    print("=" * 80)


if __name__ == "__main__":
    demonstrate_performance_profiling()

    

Agentic Design Pattern: Performance Profiling

Pattern 096: Performance Profiling – Line-by-Line Explanation

Overview

This document provides a detailed line-by-line explanation of the Performance Profiling pattern implementation.

---

Pattern Description

Pattern 096: Performance Profiling

Description: Performance Profiling systematically measures and analyzes agent execution to identify bottlenecks, optimize resource usage, and improve response times. This pattern provides detailed insights into where time and resources are spent, enabling data-driven optimization decisions. Profiling is essential for production systems where performance, cost, and user experience are critical.

Key metrics profiled:
- Execution time (total, per step, per component)
- LLM token usage (input, output, total)
- API call counts and latency
- Memory usage and allocation patterns
- Cache hit/miss rates
- Concurrent operation efficiency
- Cost per operation

Components: 1. Profiler – Time measurement (wall clock, CPU) – Memory tracking – Token counting – API call monitoring – Resource utilization – Custom metric collection

2. Metrics Collector
   - Aggregates measurements
   - Calculates statistics
   - Tracks trends over time
   - Identifies outliers
   - Correlates metrics
   - Exports data

3. Analyzer
   - Identifies bottlenecks
   - Compares components
   - Detects regressions
   - Suggests optimizations
   - Benchmarks performance
   - Generates reports

4. Visualizer
   - Flame graphs
   - Timeline charts
   - Breakdown pie charts
   - Trend graphs
   - Comparison tables
   - Heatmaps

Use Cases: 1. Performance Optimization – Identify slow components – Optimize hot paths – Reduce latency – Improve throughput – Minimize resource usage – Cost reduction

2. Capacity Planning
   - Understand resource needs
   - Plan scaling requirements
   - Budget forecasting
   - Load handling capacity
   - Infrastructure sizing
   - Cost projections

3. Regression Detection
   - Catch performance degradation
   - Validate optimizations
   - Monitor after deployments
   - Track trends over time
   - Alert on anomalies
   - Automated testing

4. Debugging & Root Cause
   - Understand slow requests
   - Identify failure points
   - Analyze edge cases
   - Reproduce issues
   - Validate fixes
   - Production troubleshooting

5. SLA Compliance
   - Monitor latency targets
   - Track availability
   - Measure throughput
   - Ensure quality of service
   - Report on SLAs
   - Alert on violations

LangChain Implementation: LangChain supports profiling through: – Callback handlers for timing – Token usage tracking – LangSmith for comprehensive monitoring – Custom profiling wrappers – Integration with monitoring tools

Key Features: 1. Low Overhead – Minimal performance impact (< 2%) – Efficient data collection – Sampling strategies – Async operations – Batch processing

2. Comprehensive Coverage
   - End-to-end timing
   - Per-component breakdown
   - LLM call analysis
   - Tool execution timing
   - Memory profiling
   - Network latency

3. Actionable Insights
   - Clear bottleneck identification
   - Optimization suggestions
   - Cost breakdowns
   - Performance comparisons
   - Trend analysis
   - Automated alerts

4. Production Ready
   - Always-on capability
   - Sampling support
   - Secure data handling
   - Scalable architecture
   - Integration friendly
   - Export options

Best Practices: 1. What to Profile – Total execution time – LLM calls (time, tokens, cost) – Tool executions – Retrieval operations – Parsing/processing time – Network calls – Database queries – Cache operations

2. Metric Collection
   - Start time, end time, duration
   - Token counts (input, output, total)
   - API call counts
   - Memory usage (peak, average)
   - Error rates
   - Retry counts
   - Cache hit rates

3. Analysis Techniques
   - Identify top time consumers
   - Calculate percentiles (p50, p95, p99)
   - Detect outliers
   - Compare before/after
   - Trend analysis
   - Cost attribution

4. Optimization Priorities
   - Focus on hot paths (80/20 rule)
   - Optimize high-frequency calls
   - Reduce expensive operations
   - Improve cache usage
   - Batch when possible
   - Parallel execution

Trade-offs: Advantages: – Identifies bottlenecks – Data-driven optimization – Cost reduction – Improved user experience – Regression detection – Capacity planning

Disadvantages:
- Small overhead (1-2%)
- Implementation complexity
- Storage requirements
- Analysis time needed
- May expose limitations
- Maintenance burden

Production Considerations: 1. Sampling Strategy – Development: 100% profiling – Staging: 50-100% – Production: 1-10% baseline – Slow requests: Always profile – Adaptive sampling

2. Storage & Retention
   - Hot data: 7 days (full detail)
   - Warm data: 30 days (aggregated)
   - Cold data: 90+ days (summary)
   - Compression for old data
   - Archival policies

3. Performance Impact
   - Target: < 2% overhead
   - Measure actual impact
   - Use efficient collectors
   - Async writes
   - Batch operations

4. Privacy & Security
   - Redact sensitive data
   - Secure storage
   - Access controls
   - Retention limits
   - Compliance adherence

5. Integration
   - Export to monitoring systems
   - Alerting integration
   - Dashboard visualization
   - API for analysis
   - CI/CD integration

---

File Statistics

• Total Lines: 1182
• Classes: 8
• Functions: 1
• Constants: 0

---

Module Documentation

"""
Pattern 096: Performance Profiling

Description:
    Performance Profiling systematically measures and analyzes agent execution to identify
    bottlenecks, optimize resource usage, and improve response times. This pattern provides
    detailed insights into where time and resources are spent, enabling data-driven
    optimization decisions. Profiling is essential for production systems where performance,
    cost, and user experience are critical.

    Key metrics profiled:
    - Execution time (total, per step, per component)
    - LLM token usage (input, output, total)
    - API call counts and latency
    - Memory usage and allocation patterns
    - Cache hit/miss rates
    - Concurrent operation efficiency
    - Cost per operation

Components:
    1. Profiler
       - Time measurement (wall clock, CPU)
       - Memory tracking
       - Token counting
       - API call monitoring
       - Resource utilization
       - Custom metric collection

    2. Metrics Collector
       - Aggregates measurements
       - Calculates statistics
       - Tracks trends over time
       - Identifies outliers
       - Correlates metrics
       - Exports data

    3. Analyzer
       - Identifies bottlenecks
       - Compares components
       - Detects regressions
       - Suggests optimizations
       - Benchmarks performance
       - Generates reports

    4. Visualizer
       - Flame graphs
       - Timeline charts
       - Breakdown pie charts
       - Trend graphs
       - Comparison tables
       - Heatmaps

Use Cases:
    1. Performance Optimization
       - Identify slow components
       - Optimize hot paths
       - Reduce latency
       - Improve throughput
       - Minimize resource usage
       - Cost reduction

    2. Capacity Planning
       - Understand resource needs
       - Plan scaling requirements
       - Budget forecasting
       - Load handling capacity
       - Infrastructure sizing
       - Cost projections

    3. Regression Detection
       - Catch performance degradation
       - Validate optimizations
       - Monitor after deployments
       - Track trends over time
       - Alert on anomalies
       - Automated testing

    4. Debugging & Root Cause
       - Understand slow requests
       - Identify failure points
       - Analyze edge cases
       - Reproduce issues
       - Validate fixes
       - Production troubleshooting

    5. SLA Compliance
       - Monitor latency targets
       - Track availability
       - Measure throughput
       - Ensure quality of service
       - Report on SLAs
       - Alert on violations

LangChain Implementation:
    LangChain supports profiling through:
    - Callback handlers for timing
    - Token usage tracking
    - LangSmith for comprehensive monitoring
    - Custom profiling wrappers
    - Integration with monitoring tools

Key Features:
    1. Low Overhead
       - Minimal performance impact (< 2%)
       - Efficient data collection
       - Sampling strategies
       - Async operations
       - Batch processing

    2. Comprehensive Coverage
       - End-to-end timing
       - Per-component breakdown
       - LLM call analysis
       - Tool execution timing
       - Memory profiling
       - Network latency

    3. Actionable Insights
       - Clear bottleneck identification
       - Optimization suggestions
       - Cost breakdowns
       - Performance comparisons
       - Trend analysis
       - Automated alerts

    4. Production Ready
       - Always-on capability
       - Sampling support
       - Secure data handling
       - Scalable architecture
       - Integration friendly
       - Export options

Best Practices:
    1. What to Profile
       - Total execution time
       - LLM calls (time, tokens, cost)
       - Tool executions
       - Retrieval operations
       - Parsing/processing time
       - Network calls
       - Database queries
       - Cache operations

    2. Metric Collection
       - Start time, end time, duration
       - Token counts (input, output, total)
       - API call counts
       - Memory usage (peak, average)
       - Error rates
       - Retry counts
       - Cache hit rates

    3. Analysis Techniques
       - Identify top time consumers
       - Calculate percentiles (p50, p95, p99)
       - Detect outliers
       - Compare before/after
       - Trend analysis
       - Cost attribution

    4. Optimization Priorities
       - Focus on hot paths (80/20 rule)
       - Optimize high-frequency calls
       - Reduce expensive operations
       - Improve cache usage
       - Batch when possible
       - Parallel execution

Trade-offs:
    Advantages:
    - Identifies bottlenecks
    - Data-driven optimization
    - Cost reduction
    - Improved user experience
    - Regression detection
    - Capacity planning

    Disadvantages:
    - Small overhead (1-2%)
    - Implementation complexity
    - Storage requirements
    - Analysis time needed
    - May expose limitations
    - Maintenance burden

Production Considerations:
    1. Sampling Strategy
       - Development: 100% profiling
       - Staging: 50-100%
       - Production: 1-10% baseline
       - Slow requests: Always profile
       - Adaptive sampling

    2. Storage & Retention
       - Hot data: 7 days (full detail)
       - Warm data: 30 days (aggregated)
       - Cold data: 90+ days (summary)
       - Compression for old data
       - Archival policies

    3. Performance Impact
       - Target: < 2% overhead
       - Measure actual impact
       - Use efficient collectors
       - Async writes
       - Batch operations

    4. Privacy & Security
       - Redact sensitive data
       - Secure storage
       - Access controls
       - Retention limits
       - Compliance adherence

    5. Integration
       - Export to monitoring systems
       - Alerting integration
       - Dashboard visualization
       - API for analysis
       - CI/CD integration
"""

Purpose: This module implements the Performance Profiling pattern, providing a structured approach to agent design and implementation.

---

Import Statements

Standard Library Imports

import collections.defaultdict
import dataclasses.dataclass
import dataclasses.field
import datetime.datetime
import enum.Enum
import json
import langchain_core.output_parsers.StrOutputParser
import langchain_core.prompts.ChatPromptTemplate
import os
import time
import typing.Any
import typing.Callable
import typing.Dict
import typing.List
import typing.Optional

Purpose: Core Python functionality: – collections.defaultdict: Specialized container datatypes – dataclasses.dataclass: Simplified class definitions with @dataclass – dataclasses.field: Simplified class definitions with @dataclass – datetime.datetime: Date and time handling – enum.Enum: Enumeration support – json: JSON serialization/deserialization – langchain_core.output_parsers.StrOutputParser: Regular expression operations – langchain_core.prompts.ChatPromptTemplate: Regular expression operations – os: Environment variable access and system operations – time: Time-related functions – typing.Any: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Callable: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Dict: Type hints for better code documentation (List, Dict, Any, Optional) – typing.List: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Optional: Type hints for better code documentation (List, Dict, Any, Optional)

LangChain Imports

from langchain_openai.ChatOpenAI

Purpose: LangChain framework components: – LLM Interface: Wrapper for OpenAI models – Chains: Composable building blocks for complex workflows

Other Imports

from dotenv.load_dotenv

Purpose: Additional dependencies for specialized functionality.

---

Classes

Class: MetricType

Lines 241-247

Inherits from: Enum

Purpose:

Types of metrics to track

---

Class: ProfileMetric

Lines 251-258

Purpose:

A single profiling metric

---

Class: ComponentProfile

Lines 262-279

Purpose:

Profile data for a component

Methods:

add_metric(self, metric)

Lines 271-273

Purpose:

Add a metric to profile

Parameters: – metric: Metric

finalize(self)

Lines 275-279

Purpose:

Finalize profile with end time

Parameters:

---

Class: ExecutionProfile

Lines 283-300

Purpose:

Complete execution profile

Methods:

add_component(self, component)

Lines 292-294

Purpose:

Add component profile

Parameters: – component: Component

finalize(self)

Lines 296-300

Purpose:

Finalize execution profile

Parameters:

---

Class: Profiler

Lines 303-387

Purpose:

Performance profiler for agent execution.

Tracks timing, token usage, costs, and other metrics.

Methods:

__init__(self)

Lines 310-314

Purpose:

Initialize profiler

Parameters:

start_execution(self, execution_id, metadata)

Lines 316-334

Purpose:

Start profiling an execution.

Args:
    execution_id: Unique execution identifier
    metadata: Optional metadata

Returns:
    Execution ID

Parameters: – execution_id: Execution id – metadata: Metadata

Returns: str

end_execution(self)

Lines 336-340

Purpose:

End profiling current execution

Parameters:

start_component(self, component_name, metadata)

Lines 342-363

Purpose:

Start profiling a component.

Args:
    component_name: Component name
    metadata: Optional metadata

Returns:
    Component profile

Parameters: – component_name: Component name – metadata: Metadata

Returns: ComponentProfile

end_component(self)

Lines 365-369

Purpose:

End profiling current component

Parameters:

add_metric(self, metric)

Lines 371-374

Purpose:

Add metric to current component

Parameters: – metric: Metric

get_profile(self, execution_id)

Lines 376-381

Purpose:

Get profile by execution ID

Parameters: – execution_id: Execution id

Returns: <ast.Subscript object at 0x000001A06D700D10>

clear_profiles(self)

Lines 383-387

Purpose:

Clear all profiles

Parameters:

---

Class: ProfileAnalyzer

Lines 390-571

Purpose:

Analyzes profiling data to identify bottlenecks and trends.

Provides insights and optimization recommendations.

Methods:

__init__(self)

Lines 397-399

Purpose:

Initialize analyzer

Parameters:

analyze_execution(self, profile)

Lines 401-448

Purpose:

Analyze execution profile.

Args:
    profile: Execution profile

Returns:
    Analysis results

Parameters: – profile: Profile

Returns: <ast.Subscript object at 0x000001A06D7184D0>

identify_bottlenecks(self, profile, threshold_pct)

Lines 450-473

Purpose:

Identify bottleneck components.

Args:
    profile: Execution profile
    threshold_pct: Threshold percentage for bottleneck

Returns:
    List of bottlenecks

Parameters: – profile: Profile – threshold_pct: Threshold pct

Returns: <ast.Subscript object at 0x000001A06D80F5D0>

calculate_statistics(self, profiles)

Lines 475-504

Purpose:

Calculate statistics across multiple profiles.

Args:
    profiles: List of execution profiles

Returns:
    Statistical summary

Parameters: – profiles: Profiles

Returns: <ast.Subscript object at 0x000001A06D855750>

compare_profiles(self, profile1, profile2)

Lines 506-530

Purpose:

Compare two profiles.

Args:
    profile1: First profile (baseline)
    profile2: Second profile (current)

Returns:
    Comparison results

Parameters: – profile1: Profile1 – profile2: Profile2

Returns: <ast.Subscript object at 0x000001A06D83CA90>

generate_recommendations(self, profile)

Lines 532-571

Purpose:

Generate optimization recommendations.

Args:
    profile: Execution profile

Returns:
    List of recommendations

Parameters: – profile: Profile

Returns: <ast.Subscript object at 0x000001A06D676F10>

---

Class: ProfileVisualizer

Lines 574-639

Purpose:

Visualizes profiling data.

Creates text-based visualizations for analysis.

Methods:

__init__(self)

Lines 581-583

Purpose:

Initialize visualizer

Parameters:

visualize_breakdown(self, analysis)

Lines 585-614

Purpose:

Visualize time breakdown by component.

Args:
    analysis: Analysis results

Returns:
    Text visualization

Parameters: – analysis: Analysis

Returns: str

visualize_statistics(self, stats)

Lines 616-639

Purpose:

Visualize statistics.

Args:
    stats: Statistics dictionary

Returns:
    Text visualization

Parameters: – stats: Stats

Returns: str

---

Class: ProfiledAgent

Lines 642-723

Purpose:

Example agent with comprehensive profiling.

Demonstrates performance profiling in practice.

Methods:

__init__(self, profiler)

Lines 649-657

Purpose:

Initialize profiled agent.

Args:
    profiler: Profiler instance

Parameters: – profiler: Profiler

process_query(self, query, execution_id)

Lines 659-717

Purpose:

Process query with profiling.

Args:
    query: User query
    execution_id: Execution identifier

Returns:
    Response

Parameters: – query: Input query or question – execution_id: Execution id

Returns: str

_call_llm(self, query)

Lines 719-723

Purpose:

Call LLM

Parameters: – query: Input query or question

Returns: str

---

Functions

demonstrate_performance_profiling()

Lines 726-1177

Purpose:

Demonstrate performance profiling

---

Key Concepts Summary

Pattern Overview

The Performance Profiling pattern provides a structured approach to building AI agents with specific capabilities and behaviors. This implementation leverages LangChain’s composable components to create a robust and maintainable solution.

Core Components

1. Agent Class: Main implementation of the pattern logic
2. Prompt Engineering: Carefully crafted prompts to guide LLM behavior
3. Tool Integration: Optional tools for extended functionality
4. Memory Management: State handling across interactions
5. Output Parsing: Structured response processing

When to Use

✅ Good for: – Scenarios requiring this specific pattern’s strengths – Applications needing structured agent behavior – Systems where explainability is important – Production environments requiring reliability

❌ Not ideal for: – Simple single-prompt tasks – Scenarios requiring different pattern characteristics – Resource-constrained environments (if computationally intensive)

---

Best Practices Demonstrated

1. Environment Configuration

• ✅ Use .env files for API keys
• ✅ Never hardcode credentials
• ✅ Validate environment variables on startup

2. Error Handling

• ✅ Graceful degradation on failures
• ✅ Informative error messages
• ✅ Logging for debugging

3. Prompt Engineering

• ✅ Clear, specific instructions
• ✅ Structured output formats
• ✅ Few-shot examples when beneficial

4. Code Organization

• ✅ Separation of concerns (class-based design)
• ✅ Type hints for clarity
• ✅ Docstrings for documentation

5. Performance Optimization

• ✅ Appropriate temperature settings
• ✅ Token budget management
• ✅ Caching where applicable

---

LangChain Components Used

• ChatOpenAI: OpenAI model interface
• Chains: Sequential component composition

Why LangChain? – Composable abstractions for LLM applications – Rich ecosystem of integrations – Production-ready patterns – Active community and development

---

Execution Flow

1. Initialize agent with configuration
   ↓
2. Load environment and dependencies
   ↓
3. Prepare prompt with input data
   ↓
4. Invoke LLM through LangChain
   ↓
5. Parse and validate response
   ↓
6. Return formatted result

Key Decision Points: – Model selection (temperature, model type) – Prompt structure (few-shot vs zero-shot) – Error handling strategy – Output format validation

---

Performance Considerations

Computational Cost

• LLM Calls: Primary cost factor (based on tokens)
• Latency: Depends on model and input size
• Memory: Minimal for most patterns

Optimization Strategies

1. Prompt Optimization: Minimize tokens while maintaining clarity
2. Caching: Cache repeated queries/responses
3. Model Selection: Use smaller models when appropriate
4. Batch Processing: Group similar requests
5. Streaming: For long responses, stream tokens

Cost Estimates (Approximate)

• GPT-3.5-turbo: ~$0.002 per request
• GPT-4: ~$0.03 per request
• GPT-4-turbo: ~$0.01 per request

Actual costs vary based on input/output token counts

---

Common Pitfalls to Avoid

1. Prompt Design Issues

❌ Vague or ambiguous instructions
✅ Clear, specific, testable prompts

2. Error Handling

❌ Ignoring API errors or rate limits
✅ Implement retry logic and graceful degradation

3. Token Management

❌ Exceeding model token limits
✅ Truncate inputs, chunk processing

4. Security

❌ Exposing API keys in code
✅ Use environment variables

5. Testing

❌ Only testing happy paths
✅ Test edge cases, failures, malformed inputs

6. Cost Management

❌ Unmonitored API usage
✅ Implement budgets, alerts, and monitoring

---

Further Improvements

Production Enhancements

1. Monitoring: Add logging, metrics, tracing
2. Validation: Input/output schema validation
3. Rate Limiting: Implement request throttling
4. Caching: Redis or in-memory cache for responses
5. Async Support: Non-blocking operations for scalability

Feature Extensions

1. Multi-Model Support: Switch between providers
2. Fallback Models: Graceful degradation strategy
3. Custom Tools: Domain-specific function calling
4. Advanced Memory: Vector stores for long-term memory
5. Evaluation: Automated testing and benchmarking

Code Quality

1. Testing: Unit tests, integration tests
2. Documentation: API docs, usage examples
3. Type Safety: Stricter type hints
4. Configuration: Externalized config files

---

Related Patterns

• Pattern 095
• Pattern 097

Pattern Families: – Reasoning Patterns (001-011) – Multi-Agent Patterns (012-019) – Tool Use Patterns (020-025) – Memory Patterns (026-031) – And many more…

---

Summary

The Performance Profiling pattern provides a powerful approach to building AI agents with specific capabilities. This implementation demonstrates:

✅ Key Strengths: – Clean, maintainable code structure – LangChain integration for composability – Type hints and documentation – Production-ready patterns

🎯 Best Use Cases: – Applications requiring this pattern’s characteristics – Systems needing structured agent behavior – Production environments requiring reliability

📚 Learning Resources: – LangChain Documentation: https://python.langchain.com/ – Pattern Documentation: See related patterns – OpenAI API Reference: https://platform.openai.com/docs/

---

File: 096_performance_profiling.py
Total Lines: 1182
Pattern Category: AI Agent Design Pattern #096

---

Generated from actual implementation code
For educational and reference purposes

    """
Pattern 095: Explanation Generation

Description:
    Explanation Generation creates human-readable explanations for agent decisions,
    reasoning processes, and actions. This pattern is crucial for transparency,
    debugging, user trust, and regulatory compliance. Good explanations help users
    understand why an agent made specific decisions, what information influenced
    those decisions, and what alternatives were considered.

    Explanation types include:
    - Decision explanations (why this choice?)
    - Reasoning chain visualization (how did we get here?)
    - Confidence rationales (why this certainty?)
    - Alternative options (what else was considered?)
    - Error explanations (what went wrong?)
    - Action justifications (why this action?)

Components:
    1. Explanation Generator
       - Natural language generation
       - Template-based explanations
       - Context-aware formatting
       - Audience adaptation
       - Multiple explanation levels

    2. Reasoning Extractor
       - Captures decision points
       - Tracks information flow
       - Identifies key factors
       - Documents alternatives
       - Maintains causal chains

    3. Visualization Builder
       - Text-based visualizations
       - Tree/graph representations
       - Timeline views
       - Step-by-step breakdowns
       - Interactive explorations

    4. Context Enricher
       - Adds relevant background
       - Includes supporting evidence
       - References sources
       - Provides examples
       - Links to documentation

Use Cases:
    1. User-Facing Applications
       - Explain recommendations
       - Justify decisions
       - Build user trust
       - Enable user feedback
       - Support learning

    2. Debugging & Development
       - Understand agent behavior
       - Identify issues
       - Validate reasoning
       - Test edge cases
       - Documentation generation

    3. Compliance & Auditing
       - Regulatory requirements
       - Audit trails
       - Accountability documentation
       - Risk assessment
       - Bias detection

    4. Education & Training
       - Teaching materials
       - Example generation
       - Best practice documentation
       - User onboarding
       - Knowledge transfer

    5. Research & Analysis
       - Understand model behavior
       - Compare approaches
       - Identify patterns
       - Publication materials
       - Reproducibility

LangChain Implementation:
    LangChain supports explanation through:
    - Chain breakdown and visualization
    - Intermediate step logging
    - Prompt templates with reasoning
    - Custom explanation chains
    - Callback handlers for tracking

Key Features:
    1. Multiple Explanation Levels
       - High-level summary (executives)
       - Medium detail (general users)
       - Technical depth (developers)
       - Expert analysis (researchers)
       - Adaptive to audience

    2. Rich Context
       - Decision factors highlighted
       - Supporting evidence provided
       - Alternatives mentioned
       - Confidence indicated
       - Sources cited

    3. Clear Structure
       - Logical flow
       - Easy to follow
       - Visual aids
       - Key points emphasized
       - Actionable insights

    4. Interactive Exploration
       - Drill-down capability
       - Follow reasoning paths
       - Explore alternatives
       - Ask follow-ups
       - Customize detail level

Best Practices:
    1. Explanation Content
       - Start with main decision/conclusion
       - Explain key factors (2-5 most important)
       - Show reasoning steps
       - Mention alternatives considered
       - Include confidence/uncertainty
       - Cite sources/evidence
       - Use concrete examples

    2. Language & Style
       - Clear, simple language
       - Avoid jargon (or define it)
       - Active voice preferred
       - Concrete over abstract
       - Structured formatting
       - Consistent terminology

    3. Audience Adaptation
       - Technical users: More details, technical terms
       - Business users: Focus on impact, ROI
       - General users: Simple language, analogies
       - Experts: Assumptions, methodology
       - Regulators: Compliance focus

    4. Visual Aids
       - Decision trees for branching logic
       - Flowcharts for processes
       - Tables for comparisons
       - Timelines for sequences
       - Graphs for relationships

Trade-offs:
    Advantages:
    - Increased transparency
    - Better user trust
    - Easier debugging
    - Compliance support
    - Knowledge sharing
    - Educational value

    Disadvantages:
    - Additional computation
    - Complexity in generation
    - May reveal limitations
    - Risk of over-explanation
    - Maintenance burden
    - Translation challenges

Production Considerations:
    1. Performance Impact
       - Cache common explanations
       - Generate asynchronously
       - Provide summary first
       - Lazy load details
       - Optimize templates

    2. Quality Assurance
       - Validate explanations
       - Test with users
       - Check for hallucinations
       - Verify accuracy
       - Regular reviews

    3. Privacy & Security
       - Redact sensitive info
       - Control detail level
       - Access restrictions
       - Audit explanation access
       - Secure storage

    4. Localization
       - Multiple languages
       - Cultural adaptation
       - Units and formats
       - Examples relevance
       - Local regulations

    5. Maintenance
       - Update with model changes
       - Refresh examples
       - Improve based on feedback
       - Version explanations
       - Document changes
"""

import os
from typing import List, Dict, Any, Optional
from dataclasses import dataclass
from enum import Enum
from dotenv import load_dotenv

from langchain_core.prompts import ChatPromptTemplate
from langchain_openai import ChatOpenAI
from langchain_core.output_parsers import StrOutputParser

load_dotenv()


class ExplanationLevel(Enum):
    """Level of detail in explanation"""
    SUMMARY = "summary"  # High-level overview
    STANDARD = "standard"  # Balanced detail
    DETAILED = "detailed"  # Comprehensive explanation
    TECHNICAL = "technical"  # For experts


class AudienceType(Enum):
    """Type of audience for explanation"""
    GENERAL = "general"  # General public
    BUSINESS = "business"  # Business users
    TECHNICAL = "technical"  # Developers
    EXPERT = "expert"  # Domain experts
    REGULATOR = "regulator"  # Compliance/audit


@dataclass
class DecisionFactor:
    """A factor that influenced a decision"""
    name: str
    description: str
    weight: float  # 0-1, importance
    evidence: str
    impact: str  # positive, negative, neutral


@dataclass
class Alternative:
    """An alternative option that was considered"""
    option: str
    description: str
    pros: List[str]
    cons: List[str]
    score: float
    not_chosen_reason: str


@dataclass
class Explanation:
    """Complete explanation of a decision"""
    decision: str
    summary: str
    reasoning_steps: List[str]
    key_factors: List[DecisionFactor]
    alternatives: List[Alternative]
    confidence: float
    sources: List[str]
    timestamp: str
    metadata: Dict[str, Any]


class ExplanationGenerator:
    """
    Generates human-readable explanations for agent decisions.
    
    Supports multiple explanation levels and audience types.
    """
    
    def __init__(self, model: str = "gpt-3.5-turbo"):
        """
        Initialize explanation generator.
        
        Args:
            model: LLM model to use for generation
        """
        self.llm = ChatOpenAI(model=model, temperature=0.3)
    
    def generate_simple_explanation(
        self,
        decision: str,
        context: str,
        reasoning: str
    ) -> str:
        """
        Generate a simple, clear explanation.
        
        Args:
            decision: The decision made
            context: Context/input
            reasoning: Basic reasoning
            
        Returns:
            Simple explanation
        """
        prompt = ChatPromptTemplate.from_template(
            """Generate a clear, simple explanation for this decision.

Decision: {decision}
Context: {context}
Reasoning: {reasoning}

Write a 2-3 sentence explanation that:
1. States what was decided
2. Explains why in simple terms
3. Is easy for anyone to understand

Explanation:"""
        )
        
        chain = prompt | self.llm | StrOutputParser()
        
        return chain.invoke({
            "decision": decision,
            "context": context,
            "reasoning": reasoning
        })
    
    def generate_detailed_explanation(
        self,
        explanation_data: Explanation,
        level: ExplanationLevel = ExplanationLevel.STANDARD,
        audience: AudienceType = AudienceType.GENERAL
    ) -> str:
        """
        Generate detailed explanation from structured data.
        
        Args:
            explanation_data: Structured explanation data
            level: Level of detail
            audience: Target audience
            
        Returns:
            Formatted explanation
        """
        sections = []
        
        # 1. Summary
        sections.append(f"# Decision Explanation\n")
        sections.append(f"**Decision:** {explanation_data.decision}\n")
        sections.append(f"**Summary:** {explanation_data.summary}\n")
        sections.append(f"**Confidence:** {explanation_data.confidence:.0%}\n")
        
        # 2. Key Factors (always include)
        if explanation_data.key_factors:
            sections.append("\n## Key Factors\n")
            for factor in sorted(explanation_data.key_factors, key=lambda f: f.weight, reverse=True):
                sections.append(f"### {factor.name} (Weight: {factor.weight:.0%})")
                sections.append(f"- {factor.description}")
                sections.append(f"- Evidence: {factor.evidence}")
                sections.append(f"- Impact: {factor.impact}\n")
        
        # 3. Reasoning Steps (standard and above)
        if level in [ExplanationLevel.STANDARD, ExplanationLevel.DETAILED, ExplanationLevel.TECHNICAL]:
            if explanation_data.reasoning_steps:
                sections.append("\n## Reasoning Process\n")
                for i, step in enumerate(explanation_data.reasoning_steps, 1):
                    sections.append(f"{i}. {step}")
        
        # 4. Alternatives (detailed and above)
        if level in [ExplanationLevel.DETAILED, ExplanationLevel.TECHNICAL]:
            if explanation_data.alternatives:
                sections.append("\n\n## Alternatives Considered\n")
                for alt in explanation_data.alternatives:
                    sections.append(f"\n### Option: {alt.option} (Score: {alt.score:.2f})")
                    sections.append(f"{alt.description}\n")
                    sections.append("**Pros:**")
                    for pro in alt.pros:
                        sections.append(f"- {pro}")
                    sections.append("\n**Cons:**")
                    for con in alt.cons:
                        sections.append(f"- {con}")
                    sections.append(f"\n**Why not chosen:** {alt.not_chosen_reason}\n")
        
        # 5. Sources (technical)
        if level == ExplanationLevel.TECHNICAL:
            if explanation_data.sources:
                sections.append("\n## Sources & References\n")
                for i, source in enumerate(explanation_data.sources, 1):
                    sections.append(f"{i}. {source}")
        
        # 6. Metadata (technical)
        if level == ExplanationLevel.TECHNICAL:
            sections.append(f"\n\n## Technical Details\n")
            sections.append(f"- Timestamp: {explanation_data.timestamp}")
            for key, value in explanation_data.metadata.items():
                sections.append(f"- {key}: {value}")
        
        return "\n".join(sections)
    
    def generate_comparison_explanation(
        self,
        options: List[Dict[str, Any]],
        chosen: str,
        criteria: List[str]
    ) -> str:
        """
        Generate explanation comparing multiple options.
        
        Args:
            options: List of options with scores
            chosen: The chosen option
            criteria: Evaluation criteria
            
        Returns:
            Comparison explanation
        """
        prompt = ChatPromptTemplate.from_template(
            """Generate a clear explanation comparing these options.

Options evaluated:
{options}

Criteria used:
{criteria}

Chosen option: {chosen}

Provide a 3-4 paragraph explanation that:
1. Summarizes what was compared
2. Explains the key differences
3. Justifies why {chosen} was selected
4. Mentions any close alternatives

Explanation:"""
        )
        
        options_text = "\n".join([f"- {opt['name']}: {opt.get('description', '')}" for opt in options])
        criteria_text = "\n".join([f"- {c}" for c in criteria])
        
        chain = prompt | self.llm | StrOutputParser()
        
        return chain.invoke({
            "options": options_text,
            "criteria": criteria_text,
            "chosen": chosen
        })
    
    def explain_error(
        self,
        error_type: str,
        error_message: str,
        context: str,
        suggestions: List[str]
    ) -> str:
        """
        Generate explanation for an error.
        
        Args:
            error_type: Type of error
            error_message: Error message
            context: Context where error occurred
            suggestions: Suggested fixes
            
        Returns:
            Error explanation
        """
        sections = []
        
        sections.append(f"# Error Explanation\n")
        sections.append(f"**Error Type:** {error_type}\n")
        sections.append(f"**What Happened:**")
        sections.append(f"{error_message}\n")
        
        sections.append(f"\n**Context:**")
        sections.append(f"{context}\n")
        
        if suggestions:
            sections.append(f"\n**How to Fix:**")
            for i, suggestion in enumerate(suggestions, 1):
                sections.append(f"{i}. {suggestion}")
        
        return "\n".join(sections)
    
    def generate_step_by_step(
        self,
        task: str,
        steps: List[Dict[str, Any]]
    ) -> str:
        """
        Generate step-by-step explanation.
        
        Args:
            task: Overall task
            steps: List of steps with details
            
        Returns:
            Step-by-step explanation
        """
        sections = []
        
        sections.append(f"# How I Completed: {task}\n")
        sections.append("Here's what I did, step by step:\n")
        
        for i, step in enumerate(steps, 1):
            sections.append(f"\n## Step {i}: {step.get('name', f'Step {i}')}")
            sections.append(f"{step.get('description', '')}")
            
            if 'input' in step:
                sections.append(f"\n**Input:** {step['input']}")
            
            if 'action' in step:
                sections.append(f"**Action:** {step['action']}")
            
            if 'output' in step:
                sections.append(f"**Output:** {step['output']}")
            
            if 'reasoning' in step:
                sections.append(f"**Why:** {step['reasoning']}")
            
            sections.append("")  # Blank line
        
        return "\n".join(sections)


class ReasoningVisualizer:
    """
    Visualizes reasoning chains and decision trees.
    
    Creates text-based visualizations for better understanding.
    """
    
    def __init__(self):
        """Initialize visualizer"""
        pass
    
    def visualize_decision_tree(
        self,
        root: Dict[str, Any],
        indent: int = 0
    ) -> str:
        """
        Visualize decision tree.
        
        Args:
            root: Root node of tree
            indent: Current indentation level
            
        Returns:
            Tree visualization
        """
        lines = []
        prefix = "  " * indent
        
        # Node
        lines.append(f"{prefix}├─ {root.get('question', 'Decision')}")
        
        # Answer/Decision
        if 'answer' in root:
            lines.append(f"{prefix}│  → {root['answer']}")
        
        # Children
        for child in root.get('children', []):
            lines.extend(self.visualize_decision_tree(child, indent + 1).split('\n'))
        
        return '\n'.join(lines)
    
    def visualize_reasoning_chain(
        self,
        steps: List[str],
        show_numbers: bool = True
    ) -> str:
        """
        Visualize reasoning chain.
        
        Args:
            steps: List of reasoning steps
            show_numbers: Whether to show step numbers
            
        Returns:
            Chain visualization
        """
        lines = []
        
        for i, step in enumerate(steps, 1):
            if show_numbers:
                lines.append(f"Step {i}: {step}")
            else:
                lines.append(f"→ {step}")
            
            # Arrow to next step (except last)
            if i  str:
        """
        Visualize decision factors.
        
        Args:
            factors: List of decision factors
            
        Returns:
            Factor visualization
        """
        lines = []
        lines.append("Decision Factors (by importance):\n")
        
        # Sort by weight
        sorted_factors = sorted(factors, key=lambda f: f.weight, reverse=True)
        
        # Find max weight for scaling
        max_weight = max(f.weight for f in sorted_factors) if sorted_factors else 1.0
        
        for factor in sorted_factors:
            # Create bar
            bar_length = int((factor.weight / max_weight) * 30)
            bar = "█" * bar_length
            
            # Format
            lines.append(f"{factor.name:20s} {bar} {factor.weight:.0%}")
            lines.append(f"{'':20s} {factor.description}")
            lines.append("")
        
        return '\n'.join(lines)


def demonstrate_explanation_generation():
    """Demonstrate explanation generation patterns"""
    print("=" * 80)
    print("EXPLANATION GENERATION DEMONSTRATION")
    print("=" * 80)
    
    generator = ExplanationGenerator()
    visualizer = ReasoningVisualizer()
    
    # Example 1: Simple Explanation
    print("\n" + "=" * 80)
    print("EXAMPLE 1: Simple Decision Explanation")
    print("=" * 80)
    
    print("\nGenerating simple explanation...")
    simple_explanation = generator.generate_simple_explanation(
        decision="Recommended Product A",
        context="User looking for a laptop for video editing",
        reasoning="Product A has better GPU and RAM for video editing tasks"
    )
    
    print("\n" + simple_explanation)
    
    # Example 2: Detailed Explanation with Factors
    print("\n" + "=" * 80)
    print("EXAMPLE 2: Detailed Explanation with Multiple Factors")
    print("=" * 80)
    
    # Create structured explanation data
    factors = [
        DecisionFactor(
            name="Performance",
            description="GPU and RAM capacity for video editing",
            weight=0.35,
            evidence="16GB RAM, RTX 3060 GPU",
            impact="positive"
        ),
        DecisionFactor(
            name="Price",
            description="Within user's budget",
            weight=0.25,
            evidence="$1,200 (budget: $1,500)",
            impact="positive"
        ),
        DecisionFactor(
            name="User Reviews",
            description="High satisfaction among video editors",
            weight=0.20,
            evidence="4.5/5 stars, 500+ reviews",
            impact="positive"
        ),
        DecisionFactor(
            name="Battery Life",
            description="Adequate for mobile work",
            weight=0.15,
            evidence="6 hours under load",
            impact="neutral"
        ),
        DecisionFactor(
            name="Weight",
            description="Slightly heavy for portability",
            weight=0.05,
            evidence="5.5 lbs",
            impact="negative"
        )
    ]
    
    alternatives = [
        Alternative(
            option="Product B",
            description="Lighter laptop with good specs",
            pros=["More portable (3.5 lbs)", "Better battery life (8 hours)"],
            cons=["Weaker GPU (GTX 1660)", "More expensive ($1,600)"],
            score=0.75,
            not_chosen_reason="GPU performance insufficient for video editing"
        ),
        Alternative(
            option="Product C",
            description="Budget-friendly option",
            pros=["Lowest price ($900)", "Good battery (7 hours)"],
            cons=["Much weaker GPU", "Only 8GB RAM"],
            score=0.55,
            not_chosen_reason="Performance too limited for professional video editing"
        )
    ]
    
    explanation_data = Explanation(
        decision="Recommend Product A (XPS 15)",
        summary="Product A provides the best balance of performance and value for video editing within budget.",
        reasoning_steps=[
            "Identified video editing as primary use case requiring strong GPU",
            "Filtered products by GPU capability (minimum RTX 3050)",
            "Checked RAM requirements (minimum 16GB for smooth editing)",
            "Compared prices within $1,500 budget",
            "Reviewed user feedback from video editors",
            "Evaluated trade-offs (weight vs performance)"
        ],
        key_factors=factors,
        alternatives=alternatives,
        confidence=0.85,
        sources=[
            "Product specifications database",
            "User review aggregator",
            "Video editing hardware requirements guide"
        ],
        timestamp="2025-11-01T10:30:00Z",
        metadata={
            "model": "gpt-3.5-turbo",
            "execution_time_ms": 1250,
            "user_id": "user_123"
        }
    )
    
    # Generate different explanation levels
    print("\n--- SUMMARY LEVEL ---")
    summary = generator.generate_detailed_explanation(
        explanation_data,
        level=ExplanationLevel.SUMMARY,
        audience=AudienceType.GENERAL
    )
    print(summary)
    
    print("\n\n--- STANDARD LEVEL ---")
    standard = generator.generate_detailed_explanation(
        explanation_data,
        level=ExplanationLevel.STANDARD,
        audience=AudienceType.GENERAL
    )
    print(standard)
    
    # Example 3: Factor Visualization
    print("\n" + "=" * 80)
    print("EXAMPLE 3: Visual Factor Analysis")
    print("=" * 80)
    
    print("\n" + visualizer.visualize_factors(factors))
    
    # Example 4: Reasoning Chain Visualization
    print("\n" + "=" * 80)
    print("EXAMPLE 4: Reasoning Chain Visualization")
    print("=" * 80)
    
    print("\n" + visualizer.visualize_reasoning_chain(explanation_data.reasoning_steps))
    
    # Example 5: Comparison Explanation
    print("\n" + "=" * 80)
    print("EXAMPLE 5: Comparison Explanation")
    print("=" * 80)
    
    print("\nGenerating comparison explanation...")
    comparison = generator.generate_comparison_explanation(
        options=[
            {"name": "Product A", "description": "High performance, moderate price"},
            {"name": "Product B", "description": "Portable, higher price"},
            {"name": "Product C", "description": "Budget option, lower specs"}
        ],
        chosen="Product A",
        criteria=["GPU Performance", "RAM", "Price", "User Reviews", "Portability"]
    )
    print("\n" + comparison)
    
    # Example 6: Error Explanation
    print("\n" + "=" * 80)
    print("EXAMPLE 6: Error Explanation")
    print("=" * 80)
    
    error_explanation = generator.explain_error(
        error_type="Insufficient Information",
        error_message="Cannot make a recommendation without knowing your budget.",
        context="You asked for laptop recommendations for video editing but didn't specify a budget. Budget is a critical factor as video editing laptops range from $800 to $3000+.",
        suggestions=[
            "Please specify your maximum budget",
            "Tell me if you have flexibility in budget if needed for performance",
            "Mention if you're open to refurbished/used options for better value"
        ]
    )
    print("\n" + error_explanation)
    
    # Example 7: Step-by-Step Explanation
    print("\n" + "=" * 80)
    print("EXAMPLE 7: Step-by-Step Process Explanation")
    print("=" * 80)
    
    steps = [
        {
            "name": "Understand Requirements",
            "description": "Analyzed user's need for video editing laptop",
            "input": "User query about laptop for video editing",
            "action": "Extracted key requirement: video editing",
            "output": "Requirement profile created",
            "reasoning": "Video editing requires strong GPU and RAM"
        },
        {
            "name": "Filter by Specs",
            "description": "Filtered products meeting minimum specs",
            "input": "Full product catalog (500+ items)",
            "action": "Applied filters: GPU >= RTX 3050, RAM >= 16GB",
            "output": "15 candidate products",
            "reasoning": "These specs ensure smooth 1080p video editing"
        },
        {
            "name": "Budget Check",
            "description": "Checked prices against budget",
            "input": "15 candidate products, budget $1,500",
            "action": "Filtered by price <= $1,500",
            "output": "5 products within budget",
            "reasoning": "Must stay within user's financial constraints"
        },
        {
            "name": "Review Analysis",
            "description": "Analyzed user reviews and ratings",
            "input": "5 products with reviews",
            "action": "Calculated weighted rating score",
            "output": "Rankings by user satisfaction",
            "reasoning": "Real user experience is crucial indicator"
        },
        {
            "name": "Final Selection",
            "description": "Selected best overall option",
            "input": "Ranked products with all factors",
            "action": "Applied multi-criteria scoring",
            "output": "Product A recommended",
            "reasoning": "Best balance of performance, price, and reviews"
        }
    ]
    
    step_by_step = generator.generate_step_by_step(
        task="Find the Best Video Editing Laptop",
        steps=steps
    )
    print("\n" + step_by_step)
    
    # Summary
    print("\n" + "=" * 80)
    print("EXPLANATION GENERATION SUMMARY")
    print("=" * 80)
    print("""
Explanation Generation Benefits:
1. Transparency: Users understand agent decisions
2. Trust: Clear reasoning builds confidence
3. Debugging: Easier to identify issues
4. Compliance: Documentation for audits
5. Learning: Users learn from explanations
6. Feedback: Better user feedback on decisions

Key Components:
1. Explanation Generator
   - Natural language generation
   - Multiple detail levels
   - Audience adaptation
   - Template-based formatting
   - Context-aware content

2. Reasoning Extractor
   - Captures decision points
   - Tracks information flow
   - Identifies key factors
   - Documents alternatives
   - Maintains causal chains

3. Visualization Builder
   - Text-based visualizations
   - Decision trees
   - Factor importance charts
   - Reasoning chain flows
   - Step-by-step breakdowns

4. Context Enricher
   - Adds background info
   - Includes evidence
   - Cites sources
   - Provides examples
   - Links documentation

Explanation Levels:
1. Summary (1-2 sentences)
   - Main decision
   - Top reason
   - Confidence
   - Best for: Quick overview, dashboards

2. Standard (2-3 paragraphs)
   - Decision + rationale
   - Key factors (2-3)
   - Basic reasoning
   - Best for: General users, reports

3. Detailed (1-2 pages)
   - Full reasoning process
   - All factors with weights
   - Alternatives considered
   - Step-by-step logic
   - Best for: Stakeholders, debugging

4. Technical (2+ pages)
   - Complete technical details
   - Model parameters
   - Data sources
   - Methodology
   - Metadata
   - Best for: Developers, researchers

Audience Types:
1. General Public
   - Simple language
   - Avoid jargon
   - Use analogies
   - Focus on outcomes
   - Concrete examples

2. Business Users
   - Impact on metrics
   - ROI considerations
   - Risk assessment
   - Strategic alignment
   - Action items

3. Technical Users
   - Implementation details
   - Algorithms used
   - Data processing
   - Performance metrics
   - Code references

4. Domain Experts
   - Assumptions made
   - Methodology choices
   - Limitations
   - Alternative approaches
   - Research references

5. Regulators/Auditors
   - Compliance evidence
   - Decision documentation
   - Bias checks
   - Accountability trails
   - Policy adherence

Explanation Structure:
```
1. Decision/Conclusion
   - What was decided
   - High-level summary
   - Confidence level

2. Key Factors (Top 3-5)
   - Factor name
   - Description
   - Evidence/data
   - Impact (positive/negative)
   - Weight/importance

3. Reasoning Process
   - Step 1: [What, Why]
   - Step 2: [What, Why]
   - ...
   - Step N: [What, Why]

4. Alternatives Considered
   - Option A: [Pros, Cons, Score]
   - Option B: [Pros, Cons, Score]
   - Why not chosen

5. Supporting Information
   - Data sources
   - References
   - Related decisions
   - Confidence factors
   - Limitations/caveats
```

Best Practices:
1. Start with Conclusion
   - Lead with the decision
   - Then explain why
   - Don't bury the lead

2. Prioritize Information
   - Most important factors first
   - Top 3-5 factors usually sufficient
   - Details available on request

3. Use Clear Language
   - Short sentences
   - Active voice
   - Concrete terms
   - Define acronyms
   - Consistent terminology

4. Provide Context
   - Background information
   - Constraints considered
   - Assumptions made
   - Scope boundaries

5. Show Alternatives
   - What else was considered
   - Why not chosen
   - Trade-offs made
   - Close calls noted

6. Indicate Uncertainty
   - Confidence levels
   - Areas of uncertainty
   - Data limitations
   - Assumptions that could change

7. Support with Evidence
   - Cite data sources
   - Show calculations
   - Reference policies
   - Link to documentation

Visualization Types:
1. Decision Trees
   - Show branching logic
   - Visualize conditions
   - Trace decision path
   - Highlight chosen branch

2. Factor Charts
   - Bar charts for importance
   - Show relative weights
   - Compare factors
   - Visual hierarchy

3. Reasoning Chains
   - Sequential flow
   - Step-by-step logic
   - Input → Process → Output
   - Causal relationships

4. Comparison Tables
   - Side-by-side options
   - Feature comparisons
   - Score breakdowns
   - Highlight differences

5. Timelines
   - Sequential events
   - Process flow
   - Before/after
   - Milestone tracking

Common Pitfalls:
❌ Too much detail upfront
✓ Start simple, offer details

❌ Technical jargon
✓ Plain language, define terms

❌ No structure
✓ Clear sections, headings

❌ Missing alternatives
✓ Show what else was considered

❌ No confidence indication
✓ State uncertainty levels

❌ Unsupported claims
✓ Cite evidence, sources

Production Tips:
1. Cache common explanations
2. Generate asynchronously
3. Provide summary first, details on request
4. A/B test explanation formats
5. Collect user feedback on clarity
6. Update with model improvements
7. Version explanations with model versions
8. Monitor for hallucinations in explanations

Integration Examples:
- Chatbots: Explain recommendations
- Search: Why these results?
- E-commerce: Product recommendations
- Healthcare: Diagnosis support
- Finance: Investment advice
- Legal: Case analysis
- Education: Tutoring feedback
- Customer service: Solution rationale

Metrics to Track:
- User satisfaction with explanations
- Time to understand
- Follow-up questions needed
- Trust indicators
- Compliance audit pass rate
- Support ticket reduction
- User engagement with details

When to Use:
✓ User-facing decisions
✓ High-stakes choices
✓ Compliance requirements
✓ Learning applications
✓ Debugging & development
✓ Research & analysis
✗ Simple, obvious decisions (maybe)
✗ Real-time critical systems (cache)
✗ Privacy-sensitive contexts (redact)

ROI Analysis:
- User trust: 30-50% increase
- Support tickets: 20-40% reduction
- Debugging time: 40-60% faster
- Compliance: Audit-ready documentation
- User satisfaction: 15-30% improvement
- Cost: Low (mostly template-based)
""")
    
    print("\n" + "=" * 80)
    print("Pattern 095 (Explanation Generation) demonstration complete!")
    print("=" * 80)


if __name__ == "__main__":
    demonstrate_explanation_generation()

    

Agentic Design Pattern: Explanation Generation

Pattern 095: Explanation Generation – Line-by-Line Explanation

Overview

This document provides a detailed line-by-line explanation of the Explanation Generation pattern implementation.

---

Pattern Description

Pattern 095: Explanation Generation

Description: Explanation Generation creates human-readable explanations for agent decisions, reasoning processes, and actions. This pattern is crucial for transparency, debugging, user trust, and regulatory compliance. Good explanations help users understand why an agent made specific decisions, what information influenced those decisions, and what alternatives were considered.

Explanation types include:
- Decision explanations (why this choice?)
- Reasoning chain visualization (how did we get here?)
- Confidence rationales (why this certainty?)
- Alternative options (what else was considered?)
- Error explanations (what went wrong?)
- Action justifications (why this action?)

Components: 1. Explanation Generator – Natural language generation – Template-based explanations – Context-aware formatting – Audience adaptation – Multiple explanation levels

2. Reasoning Extractor
   - Captures decision points
   - Tracks information flow
   - Identifies key factors
   - Documents alternatives
   - Maintains causal chains

3. Visualization Builder
   - Text-based visualizations
   - Tree/graph representations
   - Timeline views
   - Step-by-step breakdowns
   - Interactive explorations

4. Context Enricher
   - Adds relevant background
   - Includes supporting evidence
   - References sources
   - Provides examples
   - Links to documentation

Use Cases: 1. User-Facing Applications – Explain recommendations – Justify decisions – Build user trust – Enable user feedback – Support learning

2. Debugging & Development
   - Understand agent behavior
   - Identify issues
   - Validate reasoning
   - Test edge cases
   - Documentation generation

3. Compliance & Auditing
   - Regulatory requirements
   - Audit trails
   - Accountability documentation
   - Risk assessment
   - Bias detection

4. Education & Training
   - Teaching materials
   - Example generation
   - Best practice documentation
   - User onboarding
   - Knowledge transfer

5. Research & Analysis
   - Understand model behavior
   - Compare approaches
   - Identify patterns
   - Publication materials
   - Reproducibility

LangChain Implementation: LangChain supports explanation through: – Chain breakdown and visualization – Intermediate step logging – Prompt templates with reasoning – Custom explanation chains – Callback handlers for tracking

Key Features: 1. Multiple Explanation Levels – High-level summary (executives) – Medium detail (general users) – Technical depth (developers) – Expert analysis (researchers) – Adaptive to audience

2. Rich Context
   - Decision factors highlighted
   - Supporting evidence provided
   - Alternatives mentioned
   - Confidence indicated
   - Sources cited

3. Clear Structure
   - Logical flow
   - Easy to follow
   - Visual aids
   - Key points emphasized
   - Actionable insights

4. Interactive Exploration
   - Drill-down capability
   - Follow reasoning paths
   - Explore alternatives
   - Ask follow-ups
   - Customize detail level

Best Practices: 1. Explanation Content – Start with main decision/conclusion – Explain key factors (2-5 most important) – Show reasoning steps – Mention alternatives considered – Include confidence/uncertainty – Cite sources/evidence – Use concrete examples

2. Language & Style
   - Clear, simple language
   - Avoid jargon (or define it)
   - Active voice preferred
   - Concrete over abstract
   - Structured formatting
   - Consistent terminology

3. Audience Adaptation
   - Technical users: More details, technical terms
   - Business users: Focus on impact, ROI
   - General users: Simple language, analogies
   - Experts: Assumptions, methodology
   - Regulators: Compliance focus

4. Visual Aids
   - Decision trees for branching logic
   - Flowcharts for processes
   - Tables for comparisons
   - Timelines for sequences
   - Graphs for relationships

Trade-offs: Advantages: – Increased transparency – Better user trust – Easier debugging – Compliance support – Knowledge sharing – Educational value

Disadvantages:
- Additional computation
- Complexity in generation
- May reveal limitations
- Risk of over-explanation
- Maintenance burden
- Translation challenges

Production Considerations: 1. Performance Impact – Cache common explanations – Generate asynchronously – Provide summary first – Lazy load details – Optimize templates

2. Quality Assurance
   - Validate explanations
   - Test with users
   - Check for hallucinations
   - Verify accuracy
   - Regular reviews

3. Privacy & Security
   - Redact sensitive info
   - Control detail level
   - Access restrictions
   - Audit explanation access
   - Secure storage

4. Localization
   - Multiple languages
   - Cultural adaptation
   - Units and formats
   - Examples relevance
   - Local regulations

5. Maintenance
   - Update with model changes
   - Refresh examples
   - Improve based on feedback
   - Version explanations
   - Document changes

---

File Statistics

• Total Lines: 1150
• Classes: 7
• Functions: 1
• Constants: 0

---

Module Documentation

"""
Pattern 095: Explanation Generation

Description:
    Explanation Generation creates human-readable explanations for agent decisions,
    reasoning processes, and actions. This pattern is crucial for transparency,
    debugging, user trust, and regulatory compliance. Good explanations help users
    understand why an agent made specific decisions, what information influenced
    those decisions, and what alternatives were considered.

    Explanation types include:
    - Decision explanations (why this choice?)
    - Reasoning chain visualization (how did we get here?)
    - Confidence rationales (why this certainty?)
    - Alternative options (what else was considered?)
    - Error explanations (what went wrong?)
    - Action justifications (why this action?)

Components:
    1. Explanation Generator
       - Natural language generation
       - Template-based explanations
       - Context-aware formatting
       - Audience adaptation
       - Multiple explanation levels

    2. Reasoning Extractor
       - Captures decision points
       - Tracks information flow
       - Identifies key factors
       - Documents alternatives
       - Maintains causal chains

    3. Visualization Builder
       - Text-based visualizations
       - Tree/graph representations
       - Timeline views
       - Step-by-step breakdowns
       - Interactive explorations

    4. Context Enricher
       - Adds relevant background
       - Includes supporting evidence
       - References sources
       - Provides examples
       - Links to documentation

Use Cases:
    1. User-Facing Applications
       - Explain recommendations
       - Justify decisions
       - Build user trust
       - Enable user feedback
       - Support learning

    2. Debugging & Development
       - Understand agent behavior
       - Identify issues
       - Validate reasoning
       - Test edge cases
       - Documentation generation

    3. Compliance & Auditing
       - Regulatory requirements
       - Audit trails
       - Accountability documentation
       - Risk assessment
       - Bias detection

    4. Education & Training
       - Teaching materials
       - Example generation
       - Best practice documentation
       - User onboarding
       - Knowledge transfer

    5. Research & Analysis
       - Understand model behavior
       - Compare approaches
       - Identify patterns
       - Publication materials
       - Reproducibility

LangChain Implementation:
    LangChain supports explanation through:
    - Chain breakdown and visualization
    - Intermediate step logging
    - Prompt templates with reasoning
    - Custom explanation chains
    - Callback handlers for tracking

Key Features:
    1. Multiple Explanation Levels
       - High-level summary (executives)
       - Medium detail (general users)
       - Technical depth (developers)
       - Expert analysis (researchers)
       - Adaptive to audience

    2. Rich Context
       - Decision factors highlighted
       - Supporting evidence provided
       - Alternatives mentioned
       - Confidence indicated
       - Sources cited

    3. Clear Structure
       - Logical flow
       - Easy to follow
       - Visual aids
       - Key points emphasized
       - Actionable insights

    4. Interactive Exploration
       - Drill-down capability
       - Follow reasoning paths
       - Explore alternatives
       - Ask follow-ups
       - Customize detail level

Best Practices:
    1. Explanation Content
       - Start with main decision/conclusion
       - Explain key factors (2-5 most important)
       - Show reasoning steps
       - Mention alternatives considered
       - Include confidence/uncertainty
       - Cite sources/evidence
       - Use concrete examples

    2. Language & Style
       - Clear, simple language
       - Avoid jargon (or define it)
       - Active voice preferred
       - Concrete over abstract
       - Structured formatting
       - Consistent terminology

    3. Audience Adaptation
       - Technical users: More details, technical terms
       - Business users: Focus on impact, ROI
       - General users: Simple language, analogies
       - Experts: Assumptions, methodology
       - Regulators: Compliance focus

    4. Visual Aids
       - Decision trees for branching logic
       - Flowcharts for processes
       - Tables for comparisons
       - Timelines for sequences
       - Graphs for relationships

Trade-offs:
    Advantages:
    - Increased transparency
    - Better user trust
    - Easier debugging
    - Compliance support
    - Knowledge sharing
    - Educational value

    Disadvantages:
    - Additional computation
    - Complexity in generation
    - May reveal limitations
    - Risk of over-explanation
    - Maintenance burden
    - Translation challenges

Production Considerations:
    1. Performance Impact
       - Cache common explanations
       - Generate asynchronously
       - Provide summary first
       - Lazy load details
       - Optimize templates

    2. Quality Assurance
       - Validate explanations
       - Test with users
       - Check for hallucinations
       - Verify accuracy
       - Regular reviews

    3. Privacy & Security
       - Redact sensitive info
       - Control detail level
       - Access restrictions
       - Audit explanation access
       - Secure storage

    4. Localization
       - Multiple languages
       - Cultural adaptation
       - Units and formats
       - Examples relevance
       - Local regulations

    5. Maintenance
       - Update with model changes
       - Refresh examples
       - Improve based on feedback
       - Version explanations
       - Document changes
"""

Purpose: This module implements the Explanation Generation pattern, providing a structured approach to agent design and implementation.

---

Import Statements

Standard Library Imports

import dataclasses.dataclass
import enum.Enum
import langchain_core.output_parsers.StrOutputParser
import langchain_core.prompts.ChatPromptTemplate
import os
import typing.Any
import typing.Dict
import typing.List
import typing.Optional

Purpose: Core Python functionality: – dataclasses.dataclass: Simplified class definitions with @dataclass – enum.Enum: Enumeration support – langchain_core.output_parsers.StrOutputParser: Regular expression operations – langchain_core.prompts.ChatPromptTemplate: Regular expression operations – os: Environment variable access and system operations – typing.Any: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Dict: Type hints for better code documentation (List, Dict, Any, Optional) – typing.List: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Optional: Type hints for better code documentation (List, Dict, Any, Optional)

LangChain Imports

from langchain_openai.ChatOpenAI

Purpose: LangChain framework components: – LLM Interface: Wrapper for OpenAI models – Chains: Composable building blocks for complex workflows

Other Imports

from dotenv.load_dotenv

Purpose: Additional dependencies for specialized functionality.

---

Classes

Class: ExplanationLevel

Lines 220-225

Inherits from: Enum

Purpose:

Level of detail in explanation

---

Class: AudienceType

Lines 228-234

Inherits from: Enum

Purpose:

Type of audience for explanation

---

Class: DecisionFactor

Lines 238-244

Purpose:

A factor that influenced a decision

---

Class: Alternative

Lines 248-255

Purpose:

An alternative option that was considered

---

Class: Explanation

Lines 259-269

Purpose:

Complete explanation of a decision

---

Class: ExplanationGenerator

Lines 272-522

Purpose:

Generates human-readable explanations for agent decisions.

Supports multiple explanation levels and audience types.

Methods:

__init__(self, model)

Lines 279-286

Purpose:

Initialize explanation generator.

Args:
    model: LLM model to use for generation

Parameters: – model: Language model instance

generate_simple_explanation(self, decision, context, reasoning)

Lines 288-326

Purpose:

Generate a simple, clear explanation.

Args:
    decision: The decision made
    context: Context/input
    reasoning: Basic reasoning

Returns:
    Simple explanation

Parameters: – decision: Decision – context: Context – reasoning: Reasoning

Returns: str

generate_detailed_explanation(self, explanation_data, level, audience)

Lines 328-398

Purpose:

Generate detailed explanation from structured data.

Args:
    explanation_data: Structured explanation data
    level: Level of detail
    audience: Target audience

Returns:
    Formatted explanation

Parameters: – explanation_data: Explanation data – level: Level – audience: Audience

Returns: str

generate_comparison_explanation(self, options, chosen, criteria)

Lines 400-446

Purpose:

Generate explanation comparing multiple options.

Args:
    options: List of options with scores
    chosen: The chosen option
    criteria: Evaluation criteria

Returns:
    Comparison explanation

Parameters: – options: Options – chosen: Chosen – criteria: Criteria

Returns: str

explain_error(self, error_type, error_message, context, suggestions)

Lines 448-482

Purpose:

Generate explanation for an error.

Args:
    error_type: Type of error
    error_message: Error message
    context: Context where error occurred
    suggestions: Suggested fixes

Returns:
    Error explanation

Parameters: – error_type: Error type – error_message: Error message – context: Context – suggestions: Suggestions

Returns: str

generate_step_by_step(self, task, steps)

Lines 484-522

Purpose:

Generate step-by-step explanation.

Args:
    task: Overall task
    steps: List of steps with details

Returns:
    Step-by-step explanation

Parameters: – task: Task – steps: Steps

Returns: str

---

Class: ReasoningVisualizer

Lines 525-628

Purpose:

Visualizes reasoning chains and decision trees.

Creates text-based visualizations for better understanding.

Methods:

__init__(self)

Lines 532-534

Purpose:

Initialize visualizer

Parameters:

visualize_decision_tree(self, root, indent)

Lines 536-565

Purpose:

Visualize decision tree.

Args:
    root: Root node of tree
    indent: Current indentation level

Returns:
    Tree visualization

Parameters: – root: Root – indent: Indent

Returns: str

visualize_reasoning_chain(self, steps, show_numbers)

Lines 567-594

Purpose:

Visualize reasoning chain.

Args:
    steps: List of reasoning steps
    show_numbers: Whether to show step numbers

Returns:
    Chain visualization

Parameters: – steps: Steps – show_numbers: Show numbers

Returns: str

visualize_factors(self, factors)

Lines 596-628

Purpose:

Visualize decision factors.

Args:
    factors: List of decision factors

Returns:
    Factor visualization

Parameters: – factors: Factors

Returns: str

---

Functions

demonstrate_explanation_generation()

Lines 631-1145

Purpose:

Demonstrate explanation generation patterns

---

Key Concepts Summary

Pattern Overview

The Explanation Generation pattern provides a structured approach to building AI agents with specific capabilities and behaviors. This implementation leverages LangChain’s composable components to create a robust and maintainable solution.

Core Components

1. Agent Class: Main implementation of the pattern logic
2. Prompt Engineering: Carefully crafted prompts to guide LLM behavior
3. Tool Integration: Optional tools for extended functionality
4. Memory Management: State handling across interactions
5. Output Parsing: Structured response processing

When to Use

✅ Good for: – Scenarios requiring this specific pattern’s strengths – Applications needing structured agent behavior – Systems where explainability is important – Production environments requiring reliability

❌ Not ideal for: – Simple single-prompt tasks – Scenarios requiring different pattern characteristics – Resource-constrained environments (if computationally intensive)

---

Best Practices Demonstrated

1. Environment Configuration

• ✅ Use .env files for API keys
• ✅ Never hardcode credentials
• ✅ Validate environment variables on startup

2. Error Handling

• ✅ Graceful degradation on failures
• ✅ Informative error messages
• ✅ Logging for debugging

3. Prompt Engineering

• ✅ Clear, specific instructions
• ✅ Structured output formats
• ✅ Few-shot examples when beneficial

4. Code Organization

• ✅ Separation of concerns (class-based design)
• ✅ Type hints for clarity
• ✅ Docstrings for documentation

5. Performance Optimization

• ✅ Appropriate temperature settings
• ✅ Token budget management
• ✅ Caching where applicable

---

LangChain Components Used

• ChatOpenAI: OpenAI model interface
• Chains: Sequential component composition

Why LangChain? – Composable abstractions for LLM applications – Rich ecosystem of integrations – Production-ready patterns – Active community and development

---

Execution Flow

1. Initialize agent with configuration
   ↓
2. Load environment and dependencies
   ↓
3. Prepare prompt with input data
   ↓
4. Invoke LLM through LangChain
   ↓
5. Parse and validate response
   ↓
6. Return formatted result

Key Decision Points: – Model selection (temperature, model type) – Prompt structure (few-shot vs zero-shot) – Error handling strategy – Output format validation

---

Performance Considerations

Computational Cost

• LLM Calls: Primary cost factor (based on tokens)
• Latency: Depends on model and input size
• Memory: Minimal for most patterns

Optimization Strategies

1. Prompt Optimization: Minimize tokens while maintaining clarity
2. Caching: Cache repeated queries/responses
3. Model Selection: Use smaller models when appropriate
4. Batch Processing: Group similar requests
5. Streaming: For long responses, stream tokens

Cost Estimates (Approximate)

• GPT-3.5-turbo: ~$0.002 per request
• GPT-4: ~$0.03 per request
• GPT-4-turbo: ~$0.01 per request

Actual costs vary based on input/output token counts

---

Common Pitfalls to Avoid

1. Prompt Design Issues

❌ Vague or ambiguous instructions
✅ Clear, specific, testable prompts

2. Error Handling

❌ Ignoring API errors or rate limits
✅ Implement retry logic and graceful degradation

3. Token Management

❌ Exceeding model token limits
✅ Truncate inputs, chunk processing

4. Security

❌ Exposing API keys in code
✅ Use environment variables

5. Testing

❌ Only testing happy paths
✅ Test edge cases, failures, malformed inputs

6. Cost Management

❌ Unmonitored API usage
✅ Implement budgets, alerts, and monitoring

---

Further Improvements

Production Enhancements

1. Monitoring: Add logging, metrics, tracing
2. Validation: Input/output schema validation
3. Rate Limiting: Implement request throttling
4. Caching: Redis or in-memory cache for responses
5. Async Support: Non-blocking operations for scalability

Feature Extensions

1. Multi-Model Support: Switch between providers
2. Fallback Models: Graceful degradation strategy
3. Custom Tools: Domain-specific function calling
4. Advanced Memory: Vector stores for long-term memory
5. Evaluation: Automated testing and benchmarking

Code Quality

1. Testing: Unit tests, integration tests
2. Documentation: API docs, usage examples
3. Type Safety: Stricter type hints
4. Configuration: Externalized config files

---

Related Patterns

• Pattern 094
• Pattern 096

Pattern Families: – Reasoning Patterns (001-011) – Multi-Agent Patterns (012-019) – Tool Use Patterns (020-025) – Memory Patterns (026-031) – And many more…

---

Summary

The Explanation Generation pattern provides a powerful approach to building AI agents with specific capabilities. This implementation demonstrates:

✅ Key Strengths: – Clean, maintainable code structure – LangChain integration for composability – Type hints and documentation – Production-ready patterns

🎯 Best Use Cases: – Applications requiring this pattern’s characteristics – Systems needing structured agent behavior – Production environments requiring reliability

📚 Learning Resources: – LangChain Documentation: https://python.langchain.com/ – Pattern Documentation: See related patterns – OpenAI API Reference: https://platform.openai.com/docs/

---

File: 095_explanation_generation.py
Total Lines: 1150
Pattern Category: AI Agent Design Pattern #095

---

Generated from actual implementation code
For educational and reference purposes

    """
Pattern 094: Trace/Lineage Tracking

Description:
    Trace and Lineage Tracking captures the complete execution path of an agent, including
    all decisions, actions, intermediate states, and data transformations. This pattern is
    essential for debugging complex agent behaviors, understanding how decisions were made,
    ensuring compliance, and optimizing performance. Comprehensive tracing enables
    reproducibility, auditing, and continuous improvement of AI systems.

    Trace tracking provides:
    - Complete execution history
    - Decision point visibility
    - Data lineage and provenance
    - Performance metrics per step
    - Error context and stack traces
    - Reproducibility information

Components:
    1. Trace Capture
       - Execution events (start, end, error)
       - Decision points and branches
       - Input/output at each step
       - Timestamps and durations
       - Resource consumption
       - Agent state snapshots

    2. Trace Storage
       - Structured trace format
       - Hierarchical relationships
       - Searchable metadata
       - Efficient retrieval
       - Long-term retention

    3. Trace Analysis
       - Path visualization
       - Performance analysis
       - Bottleneck identification
       - Error pattern detection
       - Comparison tools

    4. Trace Replay
       - Reproduce executions
       - Debug specific paths
       - Test alternative branches
       - Validate fixes
       - Training data generation

Use Cases:
    1. Debugging
       - Understand unexpected behavior
       - Reproduce production issues
       - Identify error causes
       - Validate fixes
       - Root cause analysis

    2. Performance Optimization
       - Identify bottlenecks
       - Measure step durations
       - Analyze resource usage
       - Compare implementations
       - Optimize hot paths

    3. Compliance & Auditing
       - Document decision making
       - Prove regulatory compliance
       - Audit trails for sensitive operations
       - Accountability tracking
       - Legal requirements

    4. Quality Assurance
       - Validate agent behavior
       - Ensure determinism
       - Track test coverage
       - Regression detection
       - Change impact analysis

    5. Agent Development
       - Understand agent flow
       - Optimize prompts
       - Refine decision logic
       - Test edge cases
       - Documentation generation

LangChain Implementation:
    LangChain supports tracing through:
    - LangSmith for comprehensive tracing
    - Custom callback handlers
    - OpenTelemetry integration
    - Structured logging
    - Trace context propagation

Key Features:
    1. Comprehensive Capture
       - Every agent step traced
       - Full context preservation
       - Hierarchical relationships
       - Performance metrics
       - Error details

    2. Structured Format
       - Consistent trace structure
       - Searchable attributes
       - Machine-readable format
       - Human-readable display
       - Standard schemas

    3. Powerful Analysis
       - Path visualization
       - Query capabilities
       - Statistical analysis
       - Comparison tools
       - Pattern detection

    4. Production Ready
       - Low overhead ( 1 year
       - Compression for old traces
       - Automatic cleanup

    3. Performance Impact
       - Target:  Dict[str, Any]:
        """Convert to dictionary"""
        data = asdict(self)
        data['event_type'] = self.event_type.value
        data['timestamp'] = self.timestamp.isoformat()
        return data


@dataclass
class Trace:
    """Complete trace of an agent execution"""
    trace_id: str
    start_time: datetime
    end_time: Optional[datetime] = None
    events: List[TraceEvent] = field(default_factory=list)
    status: str = "running"  # running, completed, failed
    metadata: Dict[str, Any] = field(default_factory=dict)
    
    @property
    def duration_ms(self) -> float:
        """Calculate total duration"""
        if self.end_time:
            return (self.end_time - self.start_time).total_seconds() * 1000
        return 0.0
    
    def to_dict(self) -> Dict[str, Any]:
        """Convert to dictionary"""
        return {
            'trace_id': self.trace_id,
            'start_time': self.start_time.isoformat(),
            'end_time': self.end_time.isoformat() if self.end_time else None,
            'duration_ms': self.duration_ms,
            'status': self.status,
            'event_count': len(self.events),
            'metadata': self.metadata,
            'events': [e.to_dict() for e in self.events]
        }


class TraceCollector:
    """
    Collects and manages traces for agent executions.
    
    Captures events, maintains relationships, and provides query capabilities.
    """
    
    def __init__(self):
        """Initialize trace collector"""
        self.traces: Dict[str, Trace] = {}
        self.current_trace_id: Optional[str] = None
        self.event_stack: List[str] = []  # Stack of event IDs for hierarchy
    
    def start_trace(self, metadata: Optional[Dict[str, Any]] = None) -> str:
        """
        Start a new trace.
        
        Args:
            metadata: Optional trace metadata
            
        Returns:
            Trace ID
        """
        trace_id = str(uuid.uuid4())
        trace = Trace(
            trace_id=trace_id,
            start_time=datetime.now(),
            metadata=metadata or {}
        )
        self.traces[trace_id] = trace
        self.current_trace_id = trace_id
        return trace_id
    
    def end_trace(self, trace_id: str, status: str = "completed"):
        """
        End a trace.
        
        Args:
            trace_id: Trace identifier
            status: Final status (completed, failed)
        """
        if trace_id in self.traces:
            trace = self.traces[trace_id]
            trace.end_time = datetime.now()
            trace.status = status
    
    def log_event(
        self,
        event_type: TraceEventType,
        name: str,
        input_data: Optional[Dict[str, Any]] = None,
        output_data: Optional[Dict[str, Any]] = None,
        metadata: Optional[Dict[str, Any]] = None,
        error: Optional[str] = None,
        duration_ms: Optional[float] = None
    ) -> str:
        """
        Log a trace event.
        
        Args:
            event_type: Type of event
            name: Event name
            input_data: Input data
            output_data: Output data
            metadata: Additional metadata
            error: Error message if applicable
            duration_ms: Event duration
            
        Returns:
            Event ID
        """
        if not self.current_trace_id:
            return ""
        
        event_id = str(uuid.uuid4())
        parent_id = self.event_stack[-1] if self.event_stack else None
        
        event = TraceEvent(
            event_id=event_id,
            trace_id=self.current_trace_id,
            parent_id=parent_id,
            event_type=event_type,
            timestamp=datetime.now(),
            name=name,
            input_data=input_data or {},
            output_data=output_data or {},
            metadata=metadata or {},
            error=error,
            duration_ms=duration_ms
        )
        
        trace = self.traces[self.current_trace_id]
        trace.events.append(event)
        
        return event_id
    
    def start_span(self, name: str, event_type: TraceEventType) -> str:
        """
        Start a span (nested event).
        
        Args:
            name: Span name
            event_type: Event type
            
        Returns:
            Event ID
        """
        event_id = self.log_event(
            event_type=event_type,
            name=name,
            metadata={'span_start': True}
        )
        self.event_stack.append(event_id)
        return event_id
    
    def end_span(self, event_id: str, output_data: Optional[Dict[str, Any]] = None):
        """
        End a span.
        
        Args:
            event_id: Event ID to end
            output_data: Output data
        """
        if self.event_stack and self.event_stack[-1] == event_id:
            self.event_stack.pop()
            
            # Update event with output and duration
            if self.current_trace_id:
                trace = self.traces[self.current_trace_id]
                for event in trace.events:
                    if event.event_id == event_id:
                        if output_data:
                            event.output_data = output_data
                        # Calculate duration from event timestamp to now
                        event.duration_ms = (datetime.now() - event.timestamp).total_seconds() * 1000
                        break
    
    def get_trace(self, trace_id: str) -> Optional[Trace]:
        """Get trace by ID"""
        return self.traces.get(trace_id)
    
    def get_all_traces(self) -> List[Trace]:
        """Get all traces"""
        return list(self.traces.values())
    
    def clear_traces(self):
        """Clear all traces"""
        self.traces.clear()
        self.current_trace_id = None
        self.event_stack.clear()


class TraceAnalyzer:
    """
    Analyzes traces to extract insights and metrics.
    
    Provides visualization, statistics, and pattern detection.
    """
    
    def __init__(self):
        """Initialize analyzer"""
        pass
    
    def get_execution_path(self, trace: Trace) -> List[str]:
        """
        Get execution path as list of event names.
        
        Args:
            trace: Trace to analyze
            
        Returns:
            List of event names in order
        """
        return [event.name for event in trace.events]
    
    def get_performance_stats(self, trace: Trace) -> Dict[str, Any]:
        """
        Calculate performance statistics.
        
        Args:
            trace: Trace to analyze
            
        Returns:
            Performance statistics
        """
        events_with_duration = [e for e in trace.events if e.duration_ms]
        
        if not events_with_duration:
            return {
                'total_duration_ms': trace.duration_ms,
                'event_count': len(trace.events),
                'avg_event_duration_ms': 0,
                'max_event_duration_ms': 0,
                'min_event_duration_ms': 0
            }
        
        durations = [e.duration_ms for e in events_with_duration]
        
        return {
            'total_duration_ms': trace.duration_ms,
            'event_count': len(trace.events),
            'events_with_timing': len(events_with_duration),
            'avg_event_duration_ms': sum(durations) / len(durations),
            'max_event_duration_ms': max(durations),
            'min_event_duration_ms': min(durations),
            'total_event_time_ms': sum(durations)
        }
    
    def get_event_type_stats(self, trace: Trace) -> Dict[str, int]:
        """
        Count events by type.
        
        Args:
            trace: Trace to analyze
            
        Returns:
            Event counts by type
        """
        stats = {}
        for event in trace.events:
            event_type = event.event_type.value
            stats[event_type] = stats.get(event_type, 0) + 1
        return stats
    
    def get_error_summary(self, trace: Trace) -> List[Dict[str, Any]]:
        """
        Get summary of errors in trace.
        
        Args:
            trace: Trace to analyze
            
        Returns:
            List of error details
        """
        errors = []
        for event in trace.events:
            if event.error:
                errors.append({
                    'event_id': event.event_id,
                    'event_type': event.event_type.value,
                    'name': event.name,
                    'error': event.error,
                    'timestamp': event.timestamp.isoformat()
                })
        return errors
    
    def visualize_trace(self, trace: Trace) -> str:
        """
        Create text-based visualization of trace.
        
        Args:
            trace: Trace to visualize
            
        Returns:
            Formatted string visualization
        """
        lines = []
        lines.append(f"Trace: {trace.trace_id}")
        lines.append(f"Status: {trace.status}")
        lines.append(f"Duration: {trace.duration_ms:.2f}ms")
        lines.append(f"Events: {len(trace.events)}")
        lines.append("")
        lines.append("Execution Path:")
        
        # Build hierarchy
        event_map = {e.event_id: e for e in trace.events}
        root_events = [e for e in trace.events if e.parent_id is None]
        
        def format_event(event: TraceEvent, indent: int = 0) -> List[str]:
            """Format event with indentation"""
            result = []
            prefix = "  " * indent
            duration = f"({event.duration_ms:.1f}ms)" if event.duration_ms else ""
            error_mark = " ❌" if event.error else ""
            
            result.append(f"{prefix}├─ {event.event_type.value}: {event.name} {duration}{error_mark}")
            
            # Find children
            children = [e for e in trace.events if e.parent_id == event.event_id]
            for child in children:
                result.extend(format_event(child, indent + 1))
            
            return result
        
        for root in root_events:
            lines.extend(format_event(root))
        
        return "\n".join(lines)


class TracedAgent:
    """
    Example agent with comprehensive tracing.
    
    Demonstrates trace collection for all agent operations.
    """
    
    def __init__(self, collector: TraceCollector):
        """
        Initialize traced agent.
        
        Args:
            collector: Trace collector instance
        """
        self.collector = collector
        self.llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0.7)
    
    def process_query(self, query: str, metadata: Optional[Dict[str, Any]] = None) -> str:
        """
        Process query with full tracing.
        
        Args:
            query: User query
            metadata: Optional metadata
            
        Returns:
            Response
        """
        # Start trace
        trace_id = self.collector.start_trace(metadata={'query': query, **(metadata or {})})
        
        try:
            # Log agent start
            agent_span = self.collector.start_span("process_query", TraceEventType.AGENT_START)
            
            # Step 1: Analyze query
            analysis_span = self.collector.start_span("analyze_query", TraceEventType.DECISION)
            query_type = self._analyze_query(query)
            self.collector.end_span(analysis_span, {'query_type': query_type})
            
            # Step 2: Generate response
            llm_span = self.collector.start_span("llm_call", TraceEventType.LLM_START)
            response = self._call_llm(query)
            self.collector.end_span(llm_span, {'response': response[:100]})
            
            # End agent span
            self.collector.end_span(agent_span, {'response': response})
            
            # End trace successfully
            self.collector.end_trace(trace_id, "completed")
            
            return response
            
        except Exception as e:
            # Log error
            self.collector.log_event(
                event_type=TraceEventType.ERROR,
                name="agent_error",
                error=str(e),
                metadata={'exception_type': type(e).__name__}
            )
            
            # End trace with failure
            self.collector.end_trace(trace_id, "failed")
            
            raise
    
    def _analyze_query(self, query: str) -> str:
        """Analyze query type"""
        time.sleep(0.05)  # Simulate processing
        if '?' in query:
            return "question"
        else:
            return "statement"
    
    def _call_llm(self, query: str) -> str:
        """Call LLM"""
        prompt = ChatPromptTemplate.from_template("Answer concisely: {query}")
        chain = prompt | self.llm | StrOutputParser()
        return chain.invoke({"query": query})


def demonstrate_trace_tracking():
    """Demonstrate trace and lineage tracking"""
    print("=" * 80)
    print("TRACE/LINEAGE TRACKING DEMONSTRATION")
    print("=" * 80)
    
    # Example 1: Basic Trace Collection
    print("\n" + "=" * 80)
    print("EXAMPLE 1: Basic Trace Collection")
    print("=" * 80)
    
    collector = TraceCollector()
    
    # Start a trace
    trace_id = collector.start_trace(metadata={'user': 'demo_user', 'session': 'session_001'})
    print(f"\nStarted trace: {trace_id[:8]}...")
    
    # Log some events
    collector.log_event(
        event_type=TraceEventType.AGENT_START,
        name="agent_init",
        metadata={'agent_type': 'demo'}
    )
    
    collector.log_event(
        event_type=TraceEventType.LLM_START,
        name="llm_call_1",
        input_data={'prompt': 'What is AI?'},
        duration_ms=1250.5
    )
    
    collector.log_event(
        event_type=TraceEventType.LLM_END,
        name="llm_call_1",
        output_data={'response': 'AI is...'},
        duration_ms=1250.5
    )
    
    collector.end_trace(trace_id, "completed")
    
    trace = collector.get_trace(trace_id)
    print(f"\nTrace completed:")
    print(f"  Events: {len(trace.events)}")
    print(f"  Duration: {trace.duration_ms:.2f}ms")
    print(f"  Status: {trace.status}")
    
    # Example 2: Hierarchical Tracing
    print("\n" + "=" * 80)
    print("EXAMPLE 2: Hierarchical Tracing with Spans")
    print("=" * 80)
    
    collector.clear_traces()
    trace_id = collector.start_trace()
    
    # Parent span
    parent_span = collector.start_span("main_task", TraceEventType.AGENT_START)
    time.sleep(0.1)
    
    # Child span 1
    child_span1 = collector.start_span("subtask_1", TraceEventType.TOOL_START)
    time.sleep(0.05)
    collector.end_span(child_span1, {'result': 'success'})
    
    # Child span 2
    child_span2 = collector.start_span("subtask_2", TraceEventType.TOOL_START)
    time.sleep(0.03)
    collector.end_span(child_span2, {'result': 'success'})
    
    collector.end_span(parent_span, {'status': 'completed'})
    collector.end_trace(trace_id, "completed")
    
    trace = collector.get_trace(trace_id)
    print(f"\nHierarchical trace with {len(trace.events)} events:")
    for event in trace.events:
        indent = "  " if event.parent_id else ""
        duration = f" ({event.duration_ms:.1f}ms)" if event.duration_ms else ""
        print(f"{indent}{event.event_type.value}: {event.name}{duration}")
    
    # Example 3: Traced Agent Execution
    print("\n" + "=" * 80)
    print("EXAMPLE 3: Full Agent Execution with Tracing")
    print("=" * 80)
    
    collector.clear_traces()
    agent = TracedAgent(collector)
    
    print("\nExecuting query with tracing...")
    query = "What is machine learning?"
    response = agent.process_query(query, metadata={'priority': 'high'})
    
    print(f"\nResponse: {response[:100]}...")
    
    # Get the trace
    traces = collector.get_all_traces()
    if traces:
        trace = traces[0]
        print(f"\nTrace captured:")
        print(f"  Trace ID: {trace.trace_id[:8]}...")
        print(f"  Events: {len(trace.events)}")
        print(f"  Duration: {trace.duration_ms:.2f}ms")
        print(f"  Status: {trace.status}")
    
    # Example 4: Trace Analysis
    print("\n" + "=" * 80)
    print("EXAMPLE 4: Trace Analysis and Insights")
    print("=" * 80)
    
    analyzer = TraceAnalyzer()
    
    if traces:
        trace = traces[0]
        
        # Performance stats
        print("\nPerformance Statistics:")
        stats = analyzer.get_performance_stats(trace)
        for key, value in stats.items():
            if isinstance(value, float):
                print(f"  {key}: {value:.2f}")
            else:
                print(f"  {key}: {value}")
        
        # Event type stats
        print("\nEvent Type Distribution:")
        type_stats = analyzer.get_event_type_stats(trace)
        for event_type, count in sorted(type_stats.items()):
            print(f"  {event_type}: {count}")
        
        # Execution path
        print("\nExecution Path:")
        path = analyzer.get_execution_path(trace)
        for i, step in enumerate(path, 1):
            print(f"  {i}. {step}")
    
    # Example 5: Trace Visualization
    print("\n" + "=" * 80)
    print("EXAMPLE 5: Trace Visualization")
    print("=" * 80)
    
    if traces:
        visualization = analyzer.visualize_trace(traces[0])
        print("\n" + visualization)
    
    # Example 6: Error Tracing
    print("\n" + "=" * 80)
    print("EXAMPLE 6: Error Tracing and Debugging")
    print("=" * 80)
    
    collector.clear_traces()
    trace_id = collector.start_trace(metadata={'test': 'error_handling'})
    
    # Log successful operation
    collector.log_event(
        event_type=TraceEventType.AGENT_START,
        name="operation_start"
    )
    
    # Log error
    collector.log_event(
        event_type=TraceEventType.ERROR,
        name="operation_failed",
        error="Simulated error: Connection timeout",
        metadata={'error_code': 'TIMEOUT', 'retry_count': 3}
    )
    
    collector.end_trace(trace_id, "failed")
    
    trace = collector.get_trace(trace_id)
    errors = analyzer.get_error_summary(trace)
    
    print(f"\nTrace with errors:")
    print(f"  Status: {trace.status}")
    print(f"  Error count: {len(errors)}")
    
    if errors:
        print("\nError Details:")
        for error in errors:
            print(f"  Event: {error['name']}")
            print(f"  Type: {error['event_type']}")
            print(f"  Error: {error['error']}")
            print(f"  Time: {error['timestamp']}")
    
    # Example 7: Trace Export and Storage
    print("\n" + "=" * 80)
    print("EXAMPLE 7: Trace Export for Storage/Analysis")
    print("=" * 80)
    
    if traces:
        trace = traces[0]
        trace_dict = trace.to_dict()
        
        print("\nTrace exported to dictionary:")
        print(f"  Trace ID: {trace_dict['trace_id'][:8]}...")
        print(f"  Duration: {trace_dict['duration_ms']:.2f}ms")
        print(f"  Event count: {trace_dict['event_count']}")
        print(f"  Status: {trace_dict['status']}")
        
        # Simulate JSON export
        trace_json = json.dumps(trace_dict, indent=2, default=str)
        print(f"\nJSON size: {len(trace_json)} bytes")
        print(f"First 200 chars:\n{trace_json[:200]}...")
    
    # Summary
    print("\n" + "=" * 80)
    print("TRACE/LINEAGE TRACKING SUMMARY")
    print("=" * 80)
    print("""
Trace Tracking Benefits:
1. Complete Visibility: See every step of agent execution
2. Debugging Power: Reproduce and understand issues
3. Performance Insights: Identify bottlenecks
4. Compliance: Document decision-making
5. Reproducibility: Replay executions exactly
6. Optimization: Data-driven improvements

Key Components:
1. Trace Events
   - Agent lifecycle (start, end)
   - LLM calls (input, output, tokens, cost)
   - Tool invocations (name, args, results)
   - Decision points (conditions, branches)
   - State changes (before, after)
   - Errors (type, message, context)

2. Event Hierarchy
   - Parent-child relationships
   - Nested spans for complex operations
   - Call stack preservation
   - Context propagation
   - Timing at each level

3. Trace Metadata
   - User/session information
   - Request context
   - Environment details
   - Custom attributes
   - Tags for filtering

4. Analysis Capabilities
   - Performance profiling
   - Path visualization
   - Error pattern detection
   - Statistical analysis
   - Comparison tools

What to Trace:
1. Essential (Always)
   - All LLM calls
   - Tool invocations
   - Agent start/end
   - Errors and exceptions
   - Final results

2. Important (Most cases)
   - Decision points
   - State changes
   - Performance metrics
   - Resource usage
   - Intermediate results

3. Detailed (Development/Debug)
   - Full context
   - Internal calculations
   - Validation steps
   - Cache hits/misses
   - Detailed timing

Trace Structure:
```
Trace
├── trace_id (unique identifier)
├── start_time
├── end_time
├── status (running, completed, failed)
├── metadata (user, session, etc.)
└── events []
    ├── Event 1
    │   ├── event_id
    │   ├── parent_id
    │   ├── event_type
    │   ├── timestamp
    │   ├── duration_ms
    │   ├── input_data
    │   ├── output_data
    │   └── metadata
    └── Event 2
        └── ...
```

Best Practices:
1. Trace Organization
   - Unique trace IDs (UUID v4)
   - Clear event names
   - Consistent structure
   - Rich metadata
   - Logical grouping

2. Performance
   - Async trace writes
   - Batch uploads
   - Sampling in production
   - Compression for storage
   - Resource limits

3. Privacy
   - PII redaction
   - Sensitive data filtering
   - Access controls
   - Encryption at rest/transit
   - Retention policies

4. Storage
   - Hot: 7-30 days (fast access)
   - Warm: 31-90 days (slower)
   - Cold: 91-365 days (archive)
   - Automatic cleanup
   - Efficient formats

Production Deployment:
1. Sampling Strategy
   - Development: 100%
   - Staging: 20-50%
   - Production: 1-10% baseline
   - Errors: Always 100%
   - Important users: Higher rate

2. Performance Impact
   - Target: < 5% overhead
   - Measure actual impact
   - Optimize hot paths
   - Circuit breaker
   - Degradation handling

3. Integration
   - LangSmith for LangChain
   - OpenTelemetry standard
   - Custom backends
   - Log aggregation systems
   - APM tools integration

4. Analysis Workflow
   - Real-time monitoring
   - Periodic analysis
   - Alert on patterns
   - Dashboard visualization
   - Export for deep analysis

When to Use Tracing:
✓ All production AI systems
✓ Complex multi-step agents
✓ Debugging difficult issues
✓ Performance optimization
✓ Compliance requirements
✓ Quality assurance
✗ Simple single-shot calls (maybe)
✗ Privacy-critical scenarios (with redaction)

Common Use Cases:
1. Debugging
   - Why did agent fail?
   - What was the decision path?
   - What inputs caused issue?
   - How to reproduce?

2. Optimization
   - Which step is slowest?
   - Where are bottlenecks?
   - Cost breakdown by component
   - Cache effectiveness

3. Monitoring
   - Error rate trends
   - Performance degradation
   - Usage patterns
   - Resource consumption

4. Compliance
   - Decision documentation
   - Audit trails
   - Regulatory requirements
   - Accountability

Tools & Services:
- LangSmith: Native LangChain tracing
- OpenTelemetry: Standard observability
- Jaeger: Distributed tracing
- DataDog: APM and tracing
- Custom: Roll your own

ROI Analysis:
- Debug time: 50-90% reduction
- Issue resolution: 2-10x faster
- Performance gains: 20-50%
- Compliance: Critical for regulated industries
- Cost: Low overhead, high value
""")
    
    print("\n" + "=" * 80)
    print("Pattern 094 (Trace/Lineage Tracking) demonstration complete!")
    print("=" * 80)


if __name__ == "__main__":
    demonstrate_trace_tracking()

    

Agentic Design Pattern: Trace Tracking

Pattern 094: Trace Tracking – Line-by-Line Explanation

Overview

This document provides a detailed line-by-line explanation of the Trace Tracking pattern implementation.

---

Pattern Description

Pattern 094: Trace/Lineage Tracking

Description: Trace and Lineage Tracking captures the complete execution path of an agent, including all decisions, actions, intermediate states, and data transformations. This pattern is essential for debugging complex agent behaviors, understanding how decisions were made, ensuring compliance, and optimizing performance. Comprehensive tracing enables reproducibility, auditing, and continuous improvement of AI systems.

Trace tracking provides:
- Complete execution history
- Decision point visibility
- Data lineage and provenance
- Performance metrics per step
- Error context and stack traces
- Reproducibility information

Components: 1. Trace Capture – Execution events (start, end, error) – Decision points and branches – Input/output at each step – Timestamps and durations – Resource consumption – Agent state snapshots

2. Trace Storage
   - Structured trace format
   - Hierarchical relationships
   - Searchable metadata
   - Efficient retrieval
   - Long-term retention

3. Trace Analysis
   - Path visualization
   - Performance analysis
   - Bottleneck identification
   - Error pattern detection
   - Comparison tools

4. Trace Replay
   - Reproduce executions
   - Debug specific paths
   - Test alternative branches
   - Validate fixes
   - Training data generation

Use Cases: 1. Debugging – Understand unexpected behavior – Reproduce production issues – Identify error causes – Validate fixes – Root cause analysis

2. Performance Optimization
   - Identify bottlenecks
   - Measure step durations
   - Analyze resource usage
   - Compare implementations
   - Optimize hot paths

3. Compliance & Auditing
   - Document decision making
   - Prove regulatory compliance
   - Audit trails for sensitive operations
   - Accountability tracking
   - Legal requirements

4. Quality Assurance
   - Validate agent behavior
   - Ensure determinism
   - Track test coverage
   - Regression detection
   - Change impact analysis

5. Agent Development
   - Understand agent flow
   - Optimize prompts
   - Refine decision logic
   - Test edge cases
   - Documentation generation

LangChain Implementation: LangChain supports tracing through: – LangSmith for comprehensive tracing – Custom callback handlers – OpenTelemetry integration – Structured logging – Trace context propagation

Key Features: 1. Comprehensive Capture – Every agent step traced – Full context preservation – Hierarchical relationships – Performance metrics – Error details

2. Structured Format
   - Consistent trace structure
   - Searchable attributes
   - Machine-readable format
   - Human-readable display
   - Standard schemas

3. Powerful Analysis
   - Path visualization
   - Query capabilities
   - Statistical analysis
   - Comparison tools
   - Pattern detection

4. Production Ready
   - Low overhead (< 5% impact)
   - Efficient storage
   - Privacy controls
   - Sampling strategies
   - Scalable architecture

Best Practices: 1. What to Trace – All LLM calls (input, output, model, tokens) – Tool invocations (name, args, results) – Decision points (conditions, chosen branch) – State changes (before, after) – Errors (type, message, stack) – Performance (start, end, duration)

2. Trace Organization
   - Clear trace IDs (unique, sortable)
   - Parent-child relationships
   - Logical grouping (by session, user, task)
   - Consistent naming
   - Rich metadata

3. Performance Considerations
   - Sampling in production (1-10%)
   - Async trace writes
   - Compression for storage
   - Automatic rotation
   - Resource limits

4. Privacy & Security
   - PII redaction
   - Access controls
   - Encryption at rest
   - Retention policies
   - Compliance adherence

Trade-offs: Advantages: – Complete execution visibility – Powerful debugging capabilities – Compliance documentation – Performance insights – Reproducible executions – Historical analysis

Disadvantages:
- Storage overhead
- Performance impact (small)
- Complexity in setup
- Privacy concerns
- Analysis tools needed
- Maintenance overhead

Production Considerations: 1. Sampling Strategy – 100% for development – 10-20% for staging – 1-5% for production (baseline) – 100% for errors – Adaptive based on load

2. Storage Management
   - Hot storage: 7-30 days
   - Cold storage: 90-365 days
   - Archive: > 1 year
   - Compression for old traces
   - Automatic cleanup

3. Performance Impact
   - Target: < 5% overhead
   - Async writes preferred
   - Batch uploads
   - Local buffering
   - Circuit breaker

4. Privacy Compliance
   - Automatic PII detection
   - Redaction policies
   - Consent tracking
   - Data retention limits
   - Right to deletion

5. Analysis Tools
   - Web UI for browsing
   - Query interface
   - Visualization tools
   - Alert integration
   - Export capabilities

---

File Statistics

• Total Lines: 1067
• Classes: 6
• Functions: 1
• Constants: 0

---

Module Documentation

"""
Pattern 094: Trace/Lineage Tracking

Description:
    Trace and Lineage Tracking captures the complete execution path of an agent, including
    all decisions, actions, intermediate states, and data transformations. This pattern is
    essential for debugging complex agent behaviors, understanding how decisions were made,
    ensuring compliance, and optimizing performance. Comprehensive tracing enables
    reproducibility, auditing, and continuous improvement of AI systems.

    Trace tracking provides:
    - Complete execution history
    - Decision point visibility
    - Data lineage and provenance
    - Performance metrics per step
    - Error context and stack traces
    - Reproducibility information

Components:
    1. Trace Capture
       - Execution events (start, end, error)
       - Decision points and branches
       - Input/output at each step
       - Timestamps and durations
       - Resource consumption
       - Agent state snapshots

    2. Trace Storage
       - Structured trace format
       - Hierarchical relationships
       - Searchable metadata
       - Efficient retrieval
       - Long-term retention

    3. Trace Analysis
       - Path visualization
       - Performance analysis
       - Bottleneck identification
       - Error pattern detection
       - Comparison tools

    4. Trace Replay
       - Reproduce executions
       - Debug specific paths
       - Test alternative branches
       - Validate fixes
       - Training data generation

Use Cases:
    1. Debugging
       - Understand unexpected behavior
       - Reproduce production issues
       - Identify error causes
       - Validate fixes
       - Root cause analysis

    2. Performance Optimization
       - Identify bottlenecks
       - Measure step durations
       - Analyze resource usage
       - Compare implementations
       - Optimize hot paths

    3. Compliance & Auditing
       - Document decision making
       - Prove regulatory compliance
       - Audit trails for sensitive operations
       - Accountability tracking
       - Legal requirements

    4. Quality Assurance
       - Validate agent behavior
       - Ensure determinism
       - Track test coverage
       - Regression detection
       - Change impact analysis

    5. Agent Development
       - Understand agent flow
       - Optimize prompts
       - Refine decision logic
       - Test edge cases
       - Documentation generation

LangChain Implementation:
    LangChain supports tracing through:
    - LangSmith for comprehensive tracing
    - Custom callback handlers
    - OpenTelemetry integration
    - Structured logging
    - Trace context propagation

Key Features:
    1. Comprehensive Capture
       - Every agent step traced
       - Full context preservation
       - Hierarchical relationships
       - Performance metrics
       - Error details

    2. Structured Format
       - Consistent trace structure
       - Searchable attributes
       - Machine-readable format
       - Human-readable display
       - Standard schemas

    3. Powerful Analysis
       - Path visualization
       - Query capabilities
       - Statistical analysis
       - Comparison tools
       - Pattern detection

    4. Production Ready
       - Low overhead (< 5% impact)
       - Efficient storage
       - Privacy controls
       - Sampling strategies
       - Scalable architecture

Best Practices:
    1. What to Trace
       - All LLM calls (input, output, model, tokens)
       - Tool invocations (name, args, results)
       - Decision points (conditions, chosen branch)
       - State changes (before, after)
       - Errors (type, message, stack)
       - Performance (start, end, duration)

    2. Trace Organization
       - Clear trace IDs (unique, sortable)
       - Parent-child relationships
       - Logical grouping (by session, user, task)
       - Consistent naming
       - Rich metadata

    3. Performance Considerations
       - Sampling in production (1-10%)
       - Async trace writes
       - Compression for storage
       - Automatic rotation
       - Resource limits

    4. Privacy & Security
       - PII redaction
       - Access controls
       - Encryption at rest
       - Retention policies
       - Compliance adherence

Trade-offs:
    Advantages:
    - Complete execution visibility
    - Powerful debugging capabilities
    - Compliance documentation
    - Performance insights
    - Reproducible executions
    - Historical analysis

    Disadvantages:
    - Storage overhead
    - Performance impact (small)
    - Complexity in setup
    - Privacy concerns
    - Analysis tools needed
    - Maintenance overhead

Production Considerations:
    1. Sampling Strategy
       - 100% for development
       - 10-20% for staging
       - 1-5% for production (baseline)
       - 100% for errors
       - Adaptive based on load

    2. Storage Management
       - Hot storage: 7-30 days
       - Cold storage: 90-365 days
       - Archive: > 1 year
       - Compression for old traces
       - Automatic cleanup

    3. Performance Impact
       - Target: < 5% overhead
       - Async writes preferred
       - Batch uploads
       - Local buffering
       - Circuit breaker

    4. Privacy Compliance
       - Automatic PII detection
       - Redaction policies
       - Consent tracking
       - Data retention limits
       - Right to deletion

    5. Analysis Tools
       - Web UI for browsing
       - Query interface
       - Visualization tools
       - Alert integration
       - Export capabilities
"""

Purpose: This module implements the Trace Tracking pattern, providing a structured approach to agent design and implementation.

---

Import Statements

Standard Library Imports

import dataclasses.asdict
import dataclasses.dataclass
import dataclasses.field
import datetime.datetime
import enum.Enum
import json
import langchain_core.output_parsers.StrOutputParser
import langchain_core.prompts.ChatPromptTemplate
import os
import time
import typing.Any
import typing.Dict
import typing.List
import typing.Optional

Purpose: Core Python functionality: – dataclasses.asdict: Simplified class definitions with @dataclass – dataclasses.dataclass: Simplified class definitions with @dataclass – dataclasses.field: Simplified class definitions with @dataclass – datetime.datetime: Date and time handling – enum.Enum: Enumeration support – json: JSON serialization/deserialization – langchain_core.output_parsers.StrOutputParser: Regular expression operations – langchain_core.prompts.ChatPromptTemplate: Regular expression operations – os: Environment variable access and system operations – time: Time-related functions – typing.Any: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Dict: Type hints for better code documentation (List, Dict, Any, Optional) – typing.List: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Optional: Type hints for better code documentation (List, Dict, Any, Optional)

LangChain Imports

from langchain_openai.ChatOpenAI

Purpose: LangChain framework components: – LLM Interface: Wrapper for OpenAI models – Chains: Composable building blocks for complex workflows

Other Imports

from dotenv.load_dotenv
from uuid

Purpose: Additional dependencies for specialized functionality.

---

Classes

Class: TraceEventType

Lines 223-233

Inherits from: Enum

Purpose:

Types of trace events

---

Class: TraceEvent

Lines 237-258

Purpose:

Represents a single event in a trace

Methods:

to_dict(self)

Lines 253-258

Purpose:

Convert to dictionary

Parameters:

Returns: <ast.Subscript object at 0x000001A06D624550>

---

Class: Trace

Lines 262-289

Purpose:

Complete trace of an agent execution

Methods:

duration_ms(self)

Lines 272-276

Decorators: property

Purpose:

Calculate total duration

Parameters:

Returns: float

to_dict(self)

Lines 278-289

Purpose:

Convert to dictionary

Parameters:

Returns: <ast.Subscript object at 0x000001A06D645990>

---

Class: TraceCollector

Lines 292-441

Purpose:

Collects and manages traces for agent executions.

Captures events, maintains relationships, and provides query capabilities.

Methods:

__init__(self)

Lines 299-303

Purpose:

Initialize trace collector

Parameters:

start_trace(self, metadata)

Lines 305-323

Purpose:

Start a new trace.

Args:
    metadata: Optional trace metadata

Returns:
    Trace ID

Parameters: – metadata: Metadata

Returns: str

end_trace(self, trace_id, status)

Lines 325-336

Purpose:

End a trace.

Args:
    trace_id: Trace identifier
    status: Final status (completed, failed)

Parameters: – trace_id: Trace id – status: Status

log_event(self, event_type, name, input_data, output_data, metadata, error, duration_ms)

Lines 338-386

Purpose:

Log a trace event.

Args:
    event_type: Type of event
    name: Event name
    input_data: Input data
    output_data: Output data
    metadata: Additional metadata
    error: Error message if applicable
    duration_ms: Event duration

Returns:
    Event ID

Parameters: – event_type: Event type – name: Name – input_data: Input data – output_data: Output data – metadata: Metadata – error: Error – duration_ms: Duration ms

Returns: str

start_span(self, name, event_type)

Lines 388-405

Purpose:

Start a span (nested event).

Args:
    name: Span name
    event_type: Event type

Returns:
    Event ID

Parameters: – name: Name – event_type: Event type

Returns: str

end_span(self, event_id, output_data)

Lines 407-427

Purpose:

End a span.

Args:
    event_id: Event ID to end
    output_data: Output data

Parameters: – event_id: Event id – output_data: Output data

get_trace(self, trace_id)

Lines 429-431

Purpose:

Get trace by ID

Parameters: – trace_id: Trace id

Returns: <ast.Subscript object at 0x000001A06D7C9750>

get_all_traces(self)

Lines 433-435

Purpose:

Get all traces

Parameters:

Returns: <ast.Subscript object at 0x000001A06D7C85D0>

clear_traces(self)

Lines 437-441

Purpose:

Clear all traces

Parameters:

---

Class: TraceAnalyzer

Lines 444-579

Purpose:

Analyzes traces to extract insights and metrics.

Provides visualization, statistics, and pattern detection.

Methods:

__init__(self)

Lines 451-453

Purpose:

Initialize analyzer

Parameters:

get_execution_path(self, trace)

Lines 455-465

Purpose:

Get execution path as list of event names.

Args:
    trace: Trace to analyze

Returns:
    List of event names in order

Parameters: – trace: Trace

Returns: <ast.Subscript object at 0x000001A06D660DD0>

get_performance_stats(self, trace)

Lines 467-498

Purpose:

Calculate performance statistics.

Args:
    trace: Trace to analyze

Returns:
    Performance statistics

Parameters: – trace: Trace

Returns: <ast.Subscript object at 0x000001A06D888850>

get_event_type_stats(self, trace)

Lines 500-514

Purpose:

Count events by type.

Args:
    trace: Trace to analyze

Returns:
    Event counts by type

Parameters: – trace: Trace

Returns: <ast.Subscript object at 0x000001A06D88B6D0>

get_error_summary(self, trace)

Lines 516-536

Purpose:

Get summary of errors in trace.

Args:
    trace: Trace to analyze

Returns:
    List of error details

Parameters: – trace: Trace

Returns: <ast.Subscript object at 0x000001A06D75B110>

visualize_trace(self, trace)

Lines 538-579

Purpose:

Create text-based visualization of trace.

Args:
    trace: Trace to visualize

Returns:
    Formatted string visualization

Parameters: – trace: Trace

Returns: str

---

Class: TracedAgent

Lines 582-661

Purpose:

Example agent with comprehensive tracing.

Demonstrates trace collection for all agent operations.

Methods:

__init__(self, collector)

Lines 589-597

Purpose:

Initialize traced agent.

Args:
    collector: Trace collector instance

Parameters: – collector: Collector

process_query(self, query, metadata)

Lines 599-647

Purpose:

Process query with full tracing.

Args:
    query: User query
    metadata: Optional metadata

Returns:
    Response

Parameters: – query: Input query or question – metadata: Metadata

Returns: str

_analyze_query(self, query)

Lines 649-655

Purpose:

Analyze query type

Parameters: – query: Input query or question

Returns: str

_call_llm(self, query)

Lines 657-661

Purpose:

Call LLM

Parameters: – query: Input query or question

Returns: str

---

Functions

demonstrate_trace_tracking()

Lines 664-1062

Purpose:

Demonstrate trace and lineage tracking

---

Key Concepts Summary

Pattern Overview

The Trace Tracking pattern provides a structured approach to building AI agents with specific capabilities and behaviors. This implementation leverages LangChain’s composable components to create a robust and maintainable solution.

Core Components

1. Agent Class: Main implementation of the pattern logic
2. Prompt Engineering: Carefully crafted prompts to guide LLM behavior
3. Tool Integration: Optional tools for extended functionality
4. Memory Management: State handling across interactions
5. Output Parsing: Structured response processing

When to Use

✅ Good for: – Scenarios requiring this specific pattern’s strengths – Applications needing structured agent behavior – Systems where explainability is important – Production environments requiring reliability

❌ Not ideal for: – Simple single-prompt tasks – Scenarios requiring different pattern characteristics – Resource-constrained environments (if computationally intensive)

---

Best Practices Demonstrated

1. Environment Configuration

• ✅ Use .env files for API keys
• ✅ Never hardcode credentials
• ✅ Validate environment variables on startup

2. Error Handling

• ✅ Graceful degradation on failures
• ✅ Informative error messages
• ✅ Logging for debugging

3. Prompt Engineering

• ✅ Clear, specific instructions
• ✅ Structured output formats
• ✅ Few-shot examples when beneficial

4. Code Organization

• ✅ Separation of concerns (class-based design)
• ✅ Type hints for clarity
• ✅ Docstrings for documentation

5. Performance Optimization

• ✅ Appropriate temperature settings
• ✅ Token budget management
• ✅ Caching where applicable

---

LangChain Components Used

• ChatOpenAI: OpenAI model interface
• Chains: Sequential component composition

Why LangChain? – Composable abstractions for LLM applications – Rich ecosystem of integrations – Production-ready patterns – Active community and development

---

Execution Flow

1. Initialize agent with configuration
   ↓
2. Load environment and dependencies
   ↓
3. Prepare prompt with input data
   ↓
4. Invoke LLM through LangChain
   ↓
5. Parse and validate response
   ↓
6. Return formatted result

Key Decision Points: – Model selection (temperature, model type) – Prompt structure (few-shot vs zero-shot) – Error handling strategy – Output format validation

---

Performance Considerations

Computational Cost

• LLM Calls: Primary cost factor (based on tokens)
• Latency: Depends on model and input size
• Memory: Minimal for most patterns

Optimization Strategies

1. Prompt Optimization: Minimize tokens while maintaining clarity
2. Caching: Cache repeated queries/responses
3. Model Selection: Use smaller models when appropriate
4. Batch Processing: Group similar requests
5. Streaming: For long responses, stream tokens

Cost Estimates (Approximate)

• GPT-3.5-turbo: ~$0.002 per request
• GPT-4: ~$0.03 per request
• GPT-4-turbo: ~$0.01 per request

Actual costs vary based on input/output token counts

---

Common Pitfalls to Avoid

1. Prompt Design Issues

❌ Vague or ambiguous instructions
✅ Clear, specific, testable prompts

2. Error Handling

❌ Ignoring API errors or rate limits
✅ Implement retry logic and graceful degradation

3. Token Management

❌ Exceeding model token limits
✅ Truncate inputs, chunk processing

4. Security

❌ Exposing API keys in code
✅ Use environment variables

5. Testing

❌ Only testing happy paths
✅ Test edge cases, failures, malformed inputs

6. Cost Management

❌ Unmonitored API usage
✅ Implement budgets, alerts, and monitoring

---

Further Improvements

Production Enhancements

1. Monitoring: Add logging, metrics, tracing
2. Validation: Input/output schema validation
3. Rate Limiting: Implement request throttling
4. Caching: Redis or in-memory cache for responses
5. Async Support: Non-blocking operations for scalability

Feature Extensions

1. Multi-Model Support: Switch between providers
2. Fallback Models: Graceful degradation strategy
3. Custom Tools: Domain-specific function calling
4. Advanced Memory: Vector stores for long-term memory
5. Evaluation: Automated testing and benchmarking

Code Quality

1. Testing: Unit tests, integration tests
2. Documentation: API docs, usage examples
3. Type Safety: Stricter type hints
4. Configuration: Externalized config files

---

Related Patterns

• Pattern 093
• Pattern 095

Pattern Families: – Reasoning Patterns (001-011) – Multi-Agent Patterns (012-019) – Tool Use Patterns (020-025) – Memory Patterns (026-031) – And many more…

---

Summary

The Trace Tracking pattern provides a powerful approach to building AI agents with specific capabilities. This implementation demonstrates:

✅ Key Strengths: – Clean, maintainable code structure – LangChain integration for composability – Type hints and documentation – Production-ready patterns

🎯 Best Use Cases: – Applications requiring this pattern’s characteristics – Systems needing structured agent behavior – Production environments requiring reliability

📚 Learning Resources: – LangChain Documentation: https://python.langchain.com/ – Pattern Documentation: See related patterns – OpenAI API Reference: https://platform.openai.com/docs/

---

File: 094_trace_tracking.py
Total Lines: 1067
Pattern Category: AI Agent Design Pattern #094

---

Generated from actual implementation code
For educational and reference purposes

    """
Pattern 093: A/B Testing Pattern

Description:
    A/B Testing (also called split testing) compares two or more variants of an agent,
    prompt, model, or configuration to determine which performs better. This pattern is
    essential for data-driven optimization of AI systems, enabling evidence-based decisions
    about changes rather than relying on intuition. A/B testing helps identify improvements,
    validate hypotheses, and continuously optimize agent performance in production.

    A/B testing enables:
    - Objective comparison of variants
    - Statistical significance validation
    - Risk mitigation through gradual rollout
    - Continuous improvement culture
    - Data-driven decision making
    - ROI measurement for changes

Components:
    1. Variant Definition
       - Control (baseline/current version)
       - Treatment (new version/s)
       - Configuration differences
       - Hypothesis statement
       - Success metrics

    2. Traffic Splitting
       - Random assignment
       - Consistent user experience
       - Traffic allocation ratios
       - Segment-based splitting
       - Gradual rollout

    3. Metrics Collection
       - Primary metrics (key objectives)
       - Secondary metrics (side effects)
       - Guardrail metrics (safety)
       - Performance metrics (cost, latency)
       - User satisfaction metrics

    4. Statistical Analysis
       - Sample size calculation
       - Significance testing
       - Confidence intervals
       - Effect size measurement
       - Winner determination

Use Cases:
    1. Prompt Optimization
       - Compare prompt variations
       - Test different instructions
       - Optimize few-shot examples
       - Refine system messages
       - Improve output formatting

    2. Model Selection
       - Compare different models (GPT-4 vs GPT-3.5)
       - Test model parameters (temperature, etc.)
       - Evaluate fine-tuned models
       - Assess cost/performance trade-offs
       - Validate model upgrades

    3. Feature Testing
       - New capabilities evaluation
       - Tool addition impact
       - Memory strategy comparison
       - Retrieval method testing
       - Chain architecture optimization

    4. Configuration Tuning
       - Response length optimization
       - Timeout adjustment
       - Retry logic tuning
       - Cache strategy comparison
       - Rate limit optimization

    5. User Experience
       - Response style testing
       - Persona variations
       - Output format preferences
       - Interaction flow optimization
       - Error message improvements

LangChain Implementation:
    LangChain supports A/B testing through:
    - Multiple chain configurations
    - Custom callbacks for tracking
    - LangSmith for experiment tracking
    - Flexible chain composition
    - Variant switching logic

Key Features:
    1. Experiment Management
       - Multiple variants support
       - Traffic allocation control
       - Consistent assignment
       - Experiment lifecycle
       - Result tracking

    2. Metrics Framework
       - Flexible metric definitions
       - Automatic collection
       - Real-time monitoring
       - Historical comparison
       - Statistical analysis

    3. Safety Mechanisms
       - Guardrail metrics
       - Auto-stop on degradation
       - Gradual rollout
       - Easy rollback
       - Error rate monitoring

    4. Analysis Tools
       - Statistical significance
       - Confidence intervals
       - Visualization support
       - Winner determination
       - ROI calculation

Best Practices:
    1. Experiment Design
       - Clear hypothesis
       - Single variable change
       - Appropriate metrics
       - Sufficient sample size
       - Realistic duration

    2. Traffic Allocation
       - Start small (5-10% treatment)
       - Increase gradually
       - Monitor continuously
       - Keep control stable
       - Random assignment

    3. Metric Selection
       - Primary metric (main goal)
       - Secondary metrics (side effects)
       - Guardrails (safety limits)
       - Leading indicators
       - Lagging indicators

    4. Analysis Rigor
       - Wait for significance
       - Check for novelty effects
       - Consider practical significance
       - Review all metrics
       - Document decisions

Trade-offs:
    Advantages:
    - Objective decision making
    - Risk reduction
    - Continuous optimization
    - Evidence-based improvements
    - Stakeholder confidence
    - Measurable impact

    Disadvantages:
    - Requires traffic volume
    - Time to statistical significance
    - Complexity overhead
    - May slow innovation
    - Requires discipline
    - Can be misinterpreted

Production Considerations:
    1. Sample Size
       - Calculate minimum required
       - Consider baseline metrics
       - Factor in expected lift
       - Account for variance
       - Plan for duration

    2. Experiment Duration
       - Minimum: 1-2 weeks
       - Cover business cycles
       - Avoid holidays/events
       - Monitor stability
       - Check for time effects

    3. Monitoring
       - Real-time dashboards
       - Alert on anomalies
       - Track guardrails
       - Monitor costs
       - Review regularly

    4. Decision Criteria
       - Statistical significance (p  values
    
    def get_mean(self, metric_name: str) -> float:
        """Calculate mean for a metric"""
        values = self.metrics.get(metric_name, [])
        return sum(values) / len(values) if values else 0.0
    
    def get_std(self, metric_name: str) -> float:
        """Calculate standard deviation for a metric"""
        values = self.metrics.get(metric_name, [])
        if len(values)  Variant:
        """
        Select variant for a request.
        
        Args:
            user_id: Optional user ID for consistent assignment
            
        Returns:
            Selected variant
        """
        # Use user_id for consistent assignment if provided
        if user_id:
            # Simple hash-based assignment for consistency
            hash_value = hash(user_id) % 1000 / 1000.0
        else:
            hash_value = random.random()
        
        # Select variant based on traffic allocation
        cumulative = 0.0
        for variant in self.variants:
            if variant.status != VariantStatus.ACTIVE:
                continue
            
            cumulative += variant.traffic_allocation
            if hash_value  VariantMetrics:
        """Get metrics for a variant"""
        return self.variant_metrics.get(variant_id)
    
    def check_guardrails(self) -> Dict[str, bool]:
        """
        Check if guardrail metrics are within thresholds.
        
        Returns:
            Dictionary of guardrail checks
        """
        results = {}
        
        for metric_name, metric in self.metrics.items():
            if not metric.is_guardrail:
                continue
            
            for variant in self.variants:
                if variant.is_control:
                    continue
                
                variant_metrics = self.variant_metrics[variant.id]
                mean_value = variant_metrics.get_mean(metric_name)
                
                if metric.threshold is not None:
                    if metric.higher_is_better:
                        passed = mean_value >= metric.threshold
                    else:
                        passed = mean_value  float:
        """
        Calculate t-statistic for two samples.
        
        Returns:
            T-statistic
        """
        if n1 < 2 or n2  float:
        """
        Calculate p-value from t-statistic (simplified).
        
        This is a simplified approximation for demonstration.
        Production code should use scipy.stats.
        
        Returns:
            P-value
        """
        # Simplified: convert to z-score approximation
        # For large samples, t-distribution ≈ normal distribution
        if df  2.0 else 0.5
        else:
            # Normal approximation for large samples
            return 0.01 if abs(t_stat) > 2.58 else 0.05 if abs(t_stat) > 1.96 else 0.5
    
    def compare_variants(
        self,
        experiment: ABExperiment,
        control_id: str,
        treatment_id: str,
        metric_name: str
    ) -> ABTestResult:
        """
        Compare two variants on a metric.
        
        Args:
            experiment: A/B experiment
            control_id: Control variant ID
            treatment_id: Treatment variant ID
            metric_name: Metric to compare
            
        Returns:
            Test result
        """
        control_metrics = experiment.get_variant_metrics(control_id)
        treatment_metrics = experiment.get_variant_metrics(treatment_id)
        
        control_mean = control_metrics.get_mean(metric_name)
        control_std = control_metrics.get_std(metric_name)
        control_n = control_metrics.sample_size
        
        treatment_mean = treatment_metrics.get_mean(metric_name)
        treatment_std = treatment_metrics.get_std(metric_name)
        treatment_n = treatment_metrics.sample_size
        
        # Calculate lift
        if control_mean != 0:
            lift = ((treatment_mean - control_mean) / control_mean) * 100
        else:
            lift = 0.0
        
        # Calculate t-statistic
        t_stat = self.calculate_t_statistic(
            treatment_mean, treatment_std, treatment_n,
            control_mean, control_std, control_n
        )
        
        # Calculate p-value
        df = control_n + treatment_n - 2
        p_value = self.calculate_p_value(t_stat, df)
        
        # Determine significance
        is_significant = p_value  control_mean else control_id
            else:
                winner = treatment_id if treatment_mean  0 else "↓"
            print(f"   {result.metric_name}: {direction} {abs(result.lift):.1f}%")
    
    print("\n" + "=" * 80)
    print("RECOMMENDATION:")
    print("=" * 80)
    
    if primary_result.is_significant and all_passed and primary_result.lift > 0:
        print("""
✓ DEPLOY TREATMENT

Justification:
- Primary metric shows significant improvement
- All guardrails passed
- No significant degradation in secondary metrics
- Safe to rollout to 100% of traffic

Next Steps:
1. Increase treatment traffic to 100%
2. Monitor for 1 week
3. Validate sustained improvement
4. Document learnings
""")
    else:
        print("""
✗ KEEP CONTROL

Justification:
- Treatment did not show sufficient improvement
- OR guardrail metrics were violated
- OR improvement not statistically significant

Next Steps:
1. Analyze why treatment underperformed
2. Iterate on design
3. Run new experiment
4. Consider alternative approaches
""")
    
    # Example 7: Gradual Rollout Simulation
    print("\n" + "=" * 80)
    print("EXAMPLE 7: Gradual Rollout Strategy")
    print("=" * 80)
    
    print("\nRecommended rollout plan:\n")
    
    rollout_stages = [
        {"traffic": 5, "duration": "2 days", "action": "Initial validation"},
        {"traffic": 10, "duration": "3 days", "action": "Monitor stability"},
        {"traffic": 25, "duration": "1 week", "action": "Expand cautiously"},
        {"traffic": 50, "duration": "1 week", "action": "Half traffic"},
        {"traffic": 100, "duration": "Ongoing", "action": "Full rollout"}
    ]
    
    for stage in rollout_stages:
        print(f"Stage {stage['traffic']}%:")
        print(f"  Duration: {stage['duration']}")
        print(f"  Action: {stage['action']}")
        print(f"  If issues detected: Rollback to previous stage")
        print()
    
    # Summary
    print("\n" + "=" * 80)
    print("A/B TESTING SUMMARY")
    print("=" * 80)
    print("""
A/B Testing Benefits:
1. Objective Decision Making: Data over intuition
2. Risk Mitigation: Gradual rollout limits exposure
3. Continuous Improvement: Systematic optimization
4. Evidence-Based: Statistical confidence
5. Stakeholder Alignment: Clear success criteria
6. ROI Measurement: Quantifiable impact

Key Components:
1. Variants
   - Control: Current/baseline version
   - Treatment: New version to test
   - Clear differences between variants
   - Consistent user experience

2. Metrics
   - Primary: Main success metric
   - Secondary: Side effect monitoring
   - Guardrails: Safety boundaries
   - Leading & lagging indicators

3. Traffic Splitting
   - Random assignment for unbiased results
   - Consistent per-user (same user = same variant)
   - Flexible allocation (50/50, 90/10, etc.)
   - Gradual rollout capability

4. Statistical Analysis
   - Significance testing (p-value < 0.05)
   - Effect size (practical significance)
   - Confidence intervals
   - Sample size validation

Best Practices:
1. Experiment Design
   - Clear hypothesis statement
   - Single variable change
   - Measurable success criteria
   - Appropriate sample size
   - Realistic timeline

2. Metric Selection
   - One primary metric (main goal)
   - Multiple secondary metrics
   - Guardrail metrics (boundaries)
   - Balance short/long term
   - Track user sentiment

3. Running Experiments
   - Minimum 1-2 weeks duration
   - Wait for statistical significance
   - Monitor guardrails continuously
   - Check for novelty effects
   - Consider seasonality

4. Making Decisions
   - Require statistical significance (p < 0.05)
   - Check practical significance (meaningful lift)
   - Validate all guardrails passed
   - Review all secondary metrics
   - Document reasoning

5. Rollout Strategy
   - Start small (5-10% traffic)
   - Increase gradually
   - Monitor at each stage
   - Easy rollback mechanism
   - Full rollout only after validation

Common Pitfalls to Avoid:
1. Stopping too early (not enough samples)
2. Multiple testing without correction
3. Ignoring guardrail metrics
4. Changing experiment mid-flight
5. Cherry-picking metrics
6. Insufficient traffic volume
7. Not considering novelty effects
8. Forgetting about opportunity cost

When to Use A/B Testing:
✓ Production systems with traffic
✓ Uncertain about impact
✓ Multiple competing approaches
✓ Optimization opportunities
✓ Risk-averse changes
✓ Stakeholder buy-in needed
✗ Low traffic applications
✗ Obviously correct changes
✗ Emergency fixes
✗ Insufficient time

Sample Size Guidelines:
- Minimum 100 samples per variant
- More for smaller expected differences
- Calculator: n = (Z * σ / E)²
  where Z = z-score (1.96 for 95% confidence)
        σ = standard deviation
        E = margin of error

Statistical Significance:
- Standard: p-value < 0.05 (95% confidence)
- Conservative: p-value < 0.01 (99% confidence)
- Always report effect size, not just p-value
- Consider practical significance

Production Tips:
- Integrate with feature flags
- Real-time monitoring dashboards
- Automated guardrail alerts
- Easy rollback capability
- Experiment registry/documentation
- Post-experiment analysis
- Share learnings across team
- Iterate based on results
""")
    
    print("\n" + "=" * 80)
    print("Pattern 093 (A/B Testing) demonstration complete!")
    print("=" * 80)


if __name__ == "__main__":
    demonstrate_ab_testing()

    

Agentic Design Pattern: Ab Testing

Pattern 093: Ab Testing – Line-by-Line Explanation

Overview

This document provides a detailed line-by-line explanation of the Ab Testing pattern implementation.

---

Pattern Description

Pattern 093: A/B Testing Pattern

Description: A/B Testing (also called split testing) compares two or more variants of an agent, prompt, model, or configuration to determine which performs better. This pattern is essential for data-driven optimization of AI systems, enabling evidence-based decisions about changes rather than relying on intuition. A/B testing helps identify improvements, validate hypotheses, and continuously optimize agent performance in production.

A/B testing enables:
- Objective comparison of variants
- Statistical significance validation
- Risk mitigation through gradual rollout
- Continuous improvement culture
- Data-driven decision making
- ROI measurement for changes

Components: 1. Variant Definition – Control (baseline/current version) – Treatment (new version/s) – Configuration differences – Hypothesis statement – Success metrics

2. Traffic Splitting
   - Random assignment
   - Consistent user experience
   - Traffic allocation ratios
   - Segment-based splitting
   - Gradual rollout

3. Metrics Collection
   - Primary metrics (key objectives)
   - Secondary metrics (side effects)
   - Guardrail metrics (safety)
   - Performance metrics (cost, latency)
   - User satisfaction metrics

4. Statistical Analysis
   - Sample size calculation
   - Significance testing
   - Confidence intervals
   - Effect size measurement
   - Winner determination

Use Cases: 1. Prompt Optimization – Compare prompt variations – Test different instructions – Optimize few-shot examples – Refine system messages – Improve output formatting

2. Model Selection
   - Compare different models (GPT-4 vs GPT-3.5)
   - Test model parameters (temperature, etc.)
   - Evaluate fine-tuned models
   - Assess cost/performance trade-offs
   - Validate model upgrades

3. Feature Testing
   - New capabilities evaluation
   - Tool addition impact
   - Memory strategy comparison
   - Retrieval method testing
   - Chain architecture optimization

4. Configuration Tuning
   - Response length optimization
   - Timeout adjustment
   - Retry logic tuning
   - Cache strategy comparison
   - Rate limit optimization

5. User Experience
   - Response style testing
   - Persona variations
   - Output format preferences
   - Interaction flow optimization
   - Error message improvements

LangChain Implementation: LangChain supports A/B testing through: – Multiple chain configurations – Custom callbacks for tracking – LangSmith for experiment tracking – Flexible chain composition – Variant switching logic

Key Features: 1. Experiment Management – Multiple variants support – Traffic allocation control – Consistent assignment – Experiment lifecycle – Result tracking

2. Metrics Framework
   - Flexible metric definitions
   - Automatic collection
   - Real-time monitoring
   - Historical comparison
   - Statistical analysis

3. Safety Mechanisms
   - Guardrail metrics
   - Auto-stop on degradation
   - Gradual rollout
   - Easy rollback
   - Error rate monitoring

4. Analysis Tools
   - Statistical significance
   - Confidence intervals
   - Visualization support
   - Winner determination
   - ROI calculation

Best Practices: 1. Experiment Design – Clear hypothesis – Single variable change – Appropriate metrics – Sufficient sample size – Realistic duration

2. Traffic Allocation
   - Start small (5-10% treatment)
   - Increase gradually
   - Monitor continuously
   - Keep control stable
   - Random assignment

3. Metric Selection
   - Primary metric (main goal)
   - Secondary metrics (side effects)
   - Guardrails (safety limits)
   - Leading indicators
   - Lagging indicators

4. Analysis Rigor
   - Wait for significance
   - Check for novelty effects
   - Consider practical significance
   - Review all metrics
   - Document decisions

Trade-offs: Advantages: – Objective decision making – Risk reduction – Continuous optimization – Evidence-based improvements – Stakeholder confidence – Measurable impact

Disadvantages:
- Requires traffic volume
- Time to statistical significance
- Complexity overhead
- May slow innovation
- Requires discipline
- Can be misinterpreted

Production Considerations: 1. Sample Size – Calculate minimum required – Consider baseline metrics – Factor in expected lift – Account for variance – Plan for duration

2. Experiment Duration
   - Minimum: 1-2 weeks
   - Cover business cycles
   - Avoid holidays/events
   - Monitor stability
   - Check for time effects

3. Monitoring
   - Real-time dashboards
   - Alert on anomalies
   - Track guardrails
   - Monitor costs
   - Review regularly

4. Decision Criteria
   - Statistical significance (p < 0.05)
   - Practical significance (effect size)
   - Cost implications
   - All metrics green
   - Business alignment

5. Rollout Strategy
   - Start with 5-10%
   - Monitor for issues
   - Increase to 50%
   - Full rollout if successful
   - Easy rollback plan

---

File Statistics

• Total Lines: 969
• Classes: 7
• Functions: 1
• Constants: 0

---

Module Documentation

"""
Pattern 093: A/B Testing Pattern

Description:
    A/B Testing (also called split testing) compares two or more variants of an agent,
    prompt, model, or configuration to determine which performs better. This pattern is
    essential for data-driven optimization of AI systems, enabling evidence-based decisions
    about changes rather than relying on intuition. A/B testing helps identify improvements,
    validate hypotheses, and continuously optimize agent performance in production.

    A/B testing enables:
    - Objective comparison of variants
    - Statistical significance validation
    - Risk mitigation through gradual rollout
    - Continuous improvement culture
    - Data-driven decision making
    - ROI measurement for changes

Components:
    1. Variant Definition
       - Control (baseline/current version)
       - Treatment (new version/s)
       - Configuration differences
       - Hypothesis statement
       - Success metrics

    2. Traffic Splitting
       - Random assignment
       - Consistent user experience
       - Traffic allocation ratios
       - Segment-based splitting
       - Gradual rollout

    3. Metrics Collection
       - Primary metrics (key objectives)
       - Secondary metrics (side effects)
       - Guardrail metrics (safety)
       - Performance metrics (cost, latency)
       - User satisfaction metrics

    4. Statistical Analysis
       - Sample size calculation
       - Significance testing
       - Confidence intervals
       - Effect size measurement
       - Winner determination

Use Cases:
    1. Prompt Optimization
       - Compare prompt variations
       - Test different instructions
       - Optimize few-shot examples
       - Refine system messages
       - Improve output formatting

    2. Model Selection
       - Compare different models (GPT-4 vs GPT-3.5)
       - Test model parameters (temperature, etc.)
       - Evaluate fine-tuned models
       - Assess cost/performance trade-offs
       - Validate model upgrades

    3. Feature Testing
       - New capabilities evaluation
       - Tool addition impact
       - Memory strategy comparison
       - Retrieval method testing
       - Chain architecture optimization

    4. Configuration Tuning
       - Response length optimization
       - Timeout adjustment
       - Retry logic tuning
       - Cache strategy comparison
       - Rate limit optimization

    5. User Experience
       - Response style testing
       - Persona variations
       - Output format preferences
       - Interaction flow optimization
       - Error message improvements

LangChain Implementation:
    LangChain supports A/B testing through:
    - Multiple chain configurations
    - Custom callbacks for tracking
    - LangSmith for experiment tracking
    - Flexible chain composition
    - Variant switching logic

Key Features:
    1. Experiment Management
       - Multiple variants support
       - Traffic allocation control
       - Consistent assignment
       - Experiment lifecycle
       - Result tracking

    2. Metrics Framework
       - Flexible metric definitions
       - Automatic collection
       - Real-time monitoring
       - Historical comparison
       - Statistical analysis

    3. Safety Mechanisms
       - Guardrail metrics
       - Auto-stop on degradation
       - Gradual rollout
       - Easy rollback
       - Error rate monitoring

    4. Analysis Tools
       - Statistical significance
       - Confidence intervals
       - Visualization support
       - Winner determination
       - ROI calculation

Best Practices:
    1. Experiment Design
       - Clear hypothesis
       - Single variable change
       - Appropriate metrics
       - Sufficient sample size
       - Realistic duration

    2. Traffic Allocation
       - Start small (5-10% treatment)
       - Increase gradually
       - Monitor continuously
       - Keep control stable
       - Random assignment

    3. Metric Selection
       - Primary metric (main goal)
       - Secondary metrics (side effects)
       - Guardrails (safety limits)
       - Leading indicators
       - Lagging indicators

    4. Analysis Rigor
       - Wait for significance
       - Check for novelty effects
       - Consider practical significance
       - Review all metrics
       - Document decisions

Trade-offs:
    Advantages:
    - Objective decision making
    - Risk reduction
    - Continuous optimization
    - Evidence-based improvements
    - Stakeholder confidence
    - Measurable impact

    Disadvantages:
    - Requires traffic volume
    - Time to statistical significance
    - Complexity overhead
    - May slow innovation
    - Requires discipline
    - Can be misinterpreted

Production Considerations:
    1. Sample Size
       - Calculate minimum required
       - Consider baseline metrics
       - Factor in expected lift
       - Account for variance
       - Plan for duration

    2. Experiment Duration
       - Minimum: 1-2 weeks
       - Cover business cycles
       - Avoid holidays/events
       - Monitor stability
       - Check for time effects

    3. Monitoring
       - Real-time dashboards
       - Alert on anomalies
       - Track guardrails
       - Monitor costs
       - Review regularly

    4. Decision Criteria
       - Statistical significance (p < 0.05)
       - Practical significance (effect size)
       - Cost implications
       - All metrics green
       - Business alignment

    5. Rollout Strategy
       - Start with 5-10%
       - Monitor for issues
       - Increase to 50%
       - Full rollout if successful
       - Easy rollback plan
"""

Purpose: This module implements the Ab Testing pattern, providing a structured approach to agent design and implementation.

---

Import Statements

Standard Library Imports

import dataclasses.dataclass
import dataclasses.field
import datetime.datetime
import enum.Enum
import langchain_core.output_parsers.StrOutputParser
import langchain_core.prompts.ChatPromptTemplate
import os
import random
import time
import typing.Any
import typing.Callable
import typing.Dict
import typing.List
import typing.Optional

Purpose: Core Python functionality: – dataclasses.dataclass: Simplified class definitions with @dataclass – dataclasses.field: Simplified class definitions with @dataclass – datetime.datetime: Date and time handling – enum.Enum: Enumeration support – langchain_core.output_parsers.StrOutputParser: Regular expression operations – langchain_core.prompts.ChatPromptTemplate: Regular expression operations – os: Environment variable access and system operations – random: Random number generation – time: Time-related functions – typing.Any: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Callable: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Dict: Type hints for better code documentation (List, Dict, Any, Optional) – typing.List: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Optional: Type hints for better code documentation (List, Dict, Any, Optional)

LangChain Imports

from langchain_openai.ChatOpenAI

Purpose: LangChain framework components: – LLM Interface: Wrapper for OpenAI models – Chains: Composable building blocks for complex workflows

Other Imports

from dotenv.load_dotenv
from math

Purpose: Additional dependencies for specialized functionality.

---

Classes

Class: VariantStatus

Lines 221-226

Inherits from: Enum

Purpose:

Variant status in experiment

---

Class: Variant

Lines 230-238

Purpose:

Represents an experiment variant

---

Class: Metric

Lines 242-249

Purpose:

Represents a metric to track

---

Class: VariantMetrics

Lines 253-272

Purpose:

Metrics for a variant

Methods:

get_mean(self, metric_name)

Lines 259-262

Purpose:

Calculate mean for a metric

Parameters: – metric_name: Metric name

Returns: float

get_std(self, metric_name)

Lines 264-272

Purpose:

Calculate standard deviation for a metric

Parameters: – metric_name: Metric name

Returns: float

---

Class: ABTestResult

Lines 276-288

Purpose:

Result of A/B test analysis

---

Class: ABExperiment

Lines 291-418

Purpose:

A/B test experiment manager.

Manages variants, traffic allocation, and metrics collection.

Methods:

__init__(self, experiment_id, name, description, metrics)

Lines 298-320

Purpose:

Initialize A/B experiment.

Args:
    experiment_id: Unique experiment identifier
    name: Experiment name
    description: Experiment description
    metrics: Metrics to track

Parameters: – experiment_id: Experiment id – name: Name – description: Description – metrics: Metrics

add_variant(self, variant)

Lines 322-334

Purpose:

Add a variant to the experiment.

Args:
    variant: Variant to add

Parameters: – variant: Variant

select_variant(self, user_id)

Lines 336-364

Purpose:

Select variant for a request.

Args:
    user_id: Optional user ID for consistent assignment

Returns:
    Selected variant

Parameters: – user_id: User id

Returns: Variant

record_metric(self, variant_id, metric_name, value)

Lines 366-384

Purpose:

Record a metric value for a variant.

Args:
    variant_id: Variant identifier
    metric_name: Metric name
    value: Metric value

Parameters: – variant_id: Variant id – metric_name: Metric name – value: Value

get_variant_metrics(self, variant_id)

Lines 386-388

Purpose:

Get metrics for a variant

Parameters: – variant_id: Variant id

Returns: VariantMetrics

check_guardrails(self)

Lines 390-418

Purpose:

Check if guardrail metrics are within thresholds.

Returns:
    Dictionary of guardrail checks

Parameters:

Returns: <ast.Subscript object at 0x000001A06D7BA250>

---

Class: ABTestAnalyzer

Lines 421-555

Purpose:

Analyzes A/B test results and determines winners.

Performs statistical tests and calculates significance.

Methods:

__init__(self, confidence_level)

Lines 428-436

Purpose:

Initialize analyzer.

Args:
    confidence_level: Confidence level for significance (default 95%)

Parameters: – confidence_level: Confidence level

calculate_t_statistic(self, mean1, std1, n1, mean2, std2, n2)

Lines 438-463

Purpose:

Calculate t-statistic for two samples.

Returns:
    T-statistic

Parameters: – mean1: Mean1 – std1: Std1 – n1: N1 – mean2: Mean2 – std2: Std2 – n2: N2

Returns: float

calculate_p_value(self, t_stat, df)

Lines 465-482

Purpose:

Calculate p-value from t-statistic (simplified).

This is a simplified approximation for demonstration.
Production code should use scipy.stats.

Returns:
    P-value

Parameters: – t_stat: T stat – df: Df

Returns: float

compare_variants(self, experiment, control_id, treatment_id, metric_name)

Lines 484-555

Purpose:

Compare two variants on a metric.

Args:
    experiment: A/B experiment
    control_id: Control variant ID
    treatment_id: Treatment variant ID
    metric_name: Metric to compare

Returns:
    Test result

Parameters: – experiment: Experiment – control_id: Control id – treatment_id: Treatment id – metric_name: Metric name

Returns: ABTestResult

---

Functions

demonstrate_ab_testing()

Lines 558-964

Purpose:

Demonstrate A/B testing pattern

---

Key Concepts Summary

Pattern Overview

The Ab Testing pattern provides a structured approach to building AI agents with specific capabilities and behaviors. This implementation leverages LangChain’s composable components to create a robust and maintainable solution.

Core Components

1. Agent Class: Main implementation of the pattern logic
2. Prompt Engineering: Carefully crafted prompts to guide LLM behavior
3. Tool Integration: Optional tools for extended functionality
4. Memory Management: State handling across interactions
5. Output Parsing: Structured response processing

When to Use

✅ Good for: – Scenarios requiring this specific pattern’s strengths – Applications needing structured agent behavior – Systems where explainability is important – Production environments requiring reliability

❌ Not ideal for: – Simple single-prompt tasks – Scenarios requiring different pattern characteristics – Resource-constrained environments (if computationally intensive)

---

Best Practices Demonstrated

1. Environment Configuration

• ✅ Use .env files for API keys
• ✅ Never hardcode credentials
• ✅ Validate environment variables on startup

2. Error Handling

• ✅ Graceful degradation on failures
• ✅ Informative error messages
• ✅ Logging for debugging

3. Prompt Engineering

• ✅ Clear, specific instructions
• ✅ Structured output formats
• ✅ Few-shot examples when beneficial

4. Code Organization

• ✅ Separation of concerns (class-based design)
• ✅ Type hints for clarity
• ✅ Docstrings for documentation

5. Performance Optimization

• ✅ Appropriate temperature settings
• ✅ Token budget management
• ✅ Caching where applicable

---

LangChain Components Used

• ChatOpenAI: OpenAI model interface
• Chains: Sequential component composition

Why LangChain? – Composable abstractions for LLM applications – Rich ecosystem of integrations – Production-ready patterns – Active community and development

---

Execution Flow

1. Initialize agent with configuration
   ↓
2. Load environment and dependencies
   ↓
3. Prepare prompt with input data
   ↓
4. Invoke LLM through LangChain
   ↓
5. Parse and validate response
   ↓
6. Return formatted result

Key Decision Points: – Model selection (temperature, model type) – Prompt structure (few-shot vs zero-shot) – Error handling strategy – Output format validation

---

Performance Considerations

Computational Cost

• LLM Calls: Primary cost factor (based on tokens)
• Latency: Depends on model and input size
• Memory: Minimal for most patterns

Optimization Strategies

1. Prompt Optimization: Minimize tokens while maintaining clarity
2. Caching: Cache repeated queries/responses
3. Model Selection: Use smaller models when appropriate
4. Batch Processing: Group similar requests
5. Streaming: For long responses, stream tokens

Cost Estimates (Approximate)

• GPT-3.5-turbo: ~$0.002 per request
• GPT-4: ~$0.03 per request
• GPT-4-turbo: ~$0.01 per request

Actual costs vary based on input/output token counts

---

Common Pitfalls to Avoid

1. Prompt Design Issues

❌ Vague or ambiguous instructions
✅ Clear, specific, testable prompts

2. Error Handling

❌ Ignoring API errors or rate limits
✅ Implement retry logic and graceful degradation

3. Token Management

❌ Exceeding model token limits
✅ Truncate inputs, chunk processing

4. Security

❌ Exposing API keys in code
✅ Use environment variables

5. Testing

❌ Only testing happy paths
✅ Test edge cases, failures, malformed inputs

6. Cost Management

❌ Unmonitored API usage
✅ Implement budgets, alerts, and monitoring

---

Further Improvements

Production Enhancements

1. Monitoring: Add logging, metrics, tracing
2. Validation: Input/output schema validation
3. Rate Limiting: Implement request throttling
4. Caching: Redis or in-memory cache for responses
5. Async Support: Non-blocking operations for scalability

Feature Extensions

1. Multi-Model Support: Switch between providers
2. Fallback Models: Graceful degradation strategy
3. Custom Tools: Domain-specific function calling
4. Advanced Memory: Vector stores for long-term memory
5. Evaluation: Automated testing and benchmarking

Code Quality

1. Testing: Unit tests, integration tests
2. Documentation: API docs, usage examples
3. Type Safety: Stricter type hints
4. Configuration: Externalized config files

---

Related Patterns

• Pattern 092
• Pattern 094

Pattern Families: – Reasoning Patterns (001-011) – Multi-Agent Patterns (012-019) – Tool Use Patterns (020-025) – Memory Patterns (026-031) – And many more…

---

Summary

The Ab Testing pattern provides a powerful approach to building AI agents with specific capabilities. This implementation demonstrates:

✅ Key Strengths: – Clean, maintainable code structure – LangChain integration for composability – Type hints and documentation – Production-ready patterns

🎯 Best Use Cases: – Applications requiring this pattern’s characteristics – Systems needing structured agent behavior – Production environments requiring reliability

📚 Learning Resources: – LangChain Documentation: https://python.langchain.com/ – Pattern Documentation: See related patterns – OpenAI API Reference: https://platform.openai.com/docs/

---

File: 093_ab_testing.py
Total Lines: 969
Pattern Category: AI Agent Design Pattern #093

---

Generated from actual implementation code
For educational and reference purposes

    """
Pattern 092: Simulation Testing

Description:
    Simulation Testing creates controlled, reproducible environments to test agent behavior
    under various scenarios, conditions, and edge cases. Unlike production testing,
    simulations allow you to test dangerous, expensive, or rare scenarios safely and
    repeatedly. This pattern is essential for validating agent behavior before deployment
    and for stress testing system limits.

    Simulation testing enables:
    - Safe testing of dangerous scenarios
    - Reproducible test conditions
    - Stress and load testing
    - Edge case exploration
    - Multi-agent interaction testing
    - Cost-effective experimentation

Components:
    1. Simulation Environment
       - Mock external services (APIs, databases)
       - Configurable conditions (delays, errors, loads)
       - State management and reset
       - Recording and replay capabilities
       - Deterministic execution

    2. Scenario Definition
       - Initial state setup
       - Event sequences
       - Expected outcomes
       - Success criteria
       - Failure conditions

    3. Test Harness
       - Scenario execution engine
       - Result validation
       - Metrics collection
       - Logging and tracing
       - Report generation

    4. Mock Components
       - LLM response mocking
       - Tool/API simulation
       - User interaction simulation
       - External system simulation
       - Network condition simulation

Use Cases:
    1. Safety Testing
       - Dangerous scenarios without risk
       - Edge case handling
       - Failure mode analysis
       - Recovery testing
       - Adversarial inputs

    2. Performance Testing
       - Load testing (many concurrent requests)
       - Stress testing (beyond capacity)
       - Endurance testing (long duration)
       - Spike testing (sudden load changes)
       - Scalability validation

    3. Integration Testing
       - Multi-component interaction
       - External service integration
       - Data flow validation
       - Error propagation
       - State consistency

    4. Regression Testing
       - Validate fixes don't break features
       - Ensure consistent behavior
       - Track performance changes
       - Verify improvements
       - Document behavior changes

    5. Exploratory Testing
       - Discover edge cases
       - Test rare scenarios
       - Validate assumptions
       - Find unexpected behaviors
       - Stress system boundaries

LangChain Implementation:
    LangChain supports simulation through:
    - Mock LLM implementations
    - Custom callbacks for control
    - Test chain compositions
    - Fake embeddings for testing
    - Memory simulation

Key Features:
    1. Controllable Environment
       - Deterministic responses
       - Configurable delays
       - Error injection
       - State manipulation
       - Reproducible conditions

    2. Comprehensive Scenarios
       - Normal operations
       - Error conditions
       - Edge cases
       - Load patterns
       - Attack scenarios

    3. Detailed Observation
       - Action logging
       - State tracking
       - Performance metrics
       - Error recording
       - Decision paths

    4. Flexible Validation
       - Multiple success criteria
       - Partial success handling
       - Performance thresholds
       - State assertions
       - Behavior verification

Best Practices:
    1. Scenario Design
       - Start simple, add complexity
       - Cover happy path first
       - Add error scenarios
       - Include edge cases
       - Test boundaries

    2. Mock Fidelity
       - Balance realism vs simplicity
       - Accurate error simulation
       - Realistic timing
       - State consistency
       - Clear documentation

    3. Test Independence
       - Each test isolated
       - Clean state between tests
       - No shared dependencies
       - Deterministic execution
       - Parallel-safe

    4. Performance
       - Fast execution ( str:
        """
        Mock invoke method.
        
        Args:
            input: Input string
            
        Returns:
            Mock response
        """
        self.call_count += 1
        self.call_history.append({
            "input": input,
            "timestamp": datetime.now()
        })
        
        # Find matching response
        response = self.responses.get(input, self.default_response)
        
        # Simulate delay
        if response.delay > 0:
            time.sleep(response.delay)
        
        # Simulate error
        if response.should_error:
            raise Exception(response.error_message or "Mock error")
        
        return response.content
    
    def add_response(self, input: str, response: MockResponse):
        """Add a response pattern"""
        self.responses[input] = response
    
    def reset(self):
        """Reset mock state"""
        self.call_count = 0
        self.call_history.clear()


class MockTool:
    """
    Mock tool for testing agent tool use.
    
    Simulates external tools without actual execution.
    """
    
    def __init__(
        self,
        name: str,
        behavior: Callable[[str], str],
        should_fail: bool = False,
        failure_rate: float = 0.0
    ):
        """
        Initialize mock tool.
        
        Args:
            name: Tool name
            behavior: Function defining tool behavior
            should_fail: Whether tool should always fail
            failure_rate: Probability of failure (0-1)
        """
        self.name = name
        self.behavior = behavior
        self.should_fail = should_fail
        self.failure_rate = failure_rate
        self.call_count = 0
    
    def execute(self, input: str) -> str:
        """
        Execute tool with mock behavior.
        
        Args:
            input: Tool input
            
        Returns:
            Tool output
        """
        self.call_count += 1
        
        # Simulate failure
        if self.should_fail or random.random()  Any:
        """Get environment state"""
        return self.state.get(key)
    
    def log_event(self, event_type: str, details: Dict[str, Any]):
        """Log an event"""
        self.event_log.append({
            "timestamp": datetime.now(),
            "type": event_type,
            "details": details
        })
    
    def reset(self):
        """Reset environment to initial state"""
        self.mock_llm.reset()
        for tool in self.mock_tools.values():
            tool.call_count = 0
        self.state.clear()
        self.event_log.clear()


class SimulationTester:
    """
    Executes and validates simulation tests.
    
    Runs scenarios in controlled environment and collects results.
    """
    
    def __init__(self, environment: SimulationEnvironment):
        """
        Initialize tester.
        
        Args:
            environment: Simulation environment
        """
        self.environment = environment
    
    def run_scenario(self, scenario: SimulationScenario) -> SimulationResult:
        """
        Run a simulation scenario.
        
        Args:
            scenario: Scenario to run
            
        Returns:
            Simulation result
        """
        # Reset environment
        self.environment.reset()
        
        # Setup initial state
        for key, value in scenario.initial_state.items():
            self.environment.set_state(key, value)
        
        start_time = time.time()
        observations = []
        errors = []
        events_executed = 0
        
        try:
            # Execute events
            for event in scenario.events:
                event_type = event.get("type")
                
                if event_type == "llm_call":
                    input_text = event.get("input", "")
                    try:
                        response = self.environment.mock_llm.invoke(input_text)
                        observations.append(f"LLM called: {input_text[:50]}...")
                        observations.append(f"LLM response: {response[:50]}...")
                        events_executed += 1
                    except Exception as e:
                        errors.append(f"LLM error: {str(e)}")
                
                elif event_type == "tool_call":
                    tool_name = event.get("tool")
                    tool_input = event.get("input", "")
                    
                    if tool_name in self.environment.mock_tools:
                        try:
                            result = self.environment.mock_tools[tool_name].execute(tool_input)
                            observations.append(f"Tool {tool_name} called")
                            observations.append(f"Tool result: {result[:50]}...")
                            events_executed += 1
                        except Exception as e:
                            errors.append(f"Tool error: {str(e)}")
                
                elif event_type == "state_update":
                    key = event.get("key")
                    value = event.get("value")
                    self.environment.set_state(key, value)
                    observations.append(f"State updated: {key}={value}")
                    events_executed += 1
                
                # Log event
                self.environment.log_event(event_type, event)
            
            # Evaluate success criteria
            success = scenario.success_criteria(self.environment.state)
            
            # Determine outcome
            if errors:
                outcome = OutcomeType.ERROR
            elif success:
                outcome = OutcomeType.SUCCESS
            elif events_executed  List[SimulationResult]:
        """
        Run multiple scenarios.
        
        Args:
            scenarios: List of scenarios
            
        Returns:
            List of results
        """
        results = []
        for scenario in scenarios:
            result = self.run_scenario(scenario)
            results.append(result)
        return results


def demonstrate_simulation_testing():
    """Demonstrate simulation testing"""
    print("=" * 80)
    print("SIMULATION TESTING DEMONSTRATION")
    print("=" * 80)
    
    # Example 1: Setup Simulation Environment
    print("\n" + "=" * 80)
    print("EXAMPLE 1: Creating Simulation Environment")
    print("=" * 80)
    
    env = SimulationEnvironment()
    
    # Configure mock LLM responses
    env.setup_mock_llm({
        "What is 2+2?": MockResponse(content="4", delay=0.1),
        "What is the capital of France?": MockResponse(content="Paris", delay=0.1),
        "Error test": MockResponse(
            content="",
            should_error=True,
            error_message="Simulated LLM error"
        )
    })
    
    # Add mock tools
    calculator = MockTool(
        name="calculator",
        behavior=lambda x: str(eval(x)),  # Simple eval for demo
        failure_rate=0.0
    )
    env.add_mock_tool(calculator)
    
    search = MockTool(
        name="search",
        behavior=lambda x: f"Search results for: {x}",
        failure_rate=0.0
    )
    env.add_mock_tool(search)
    
    print("\nSimulation environment configured:")
    print(f"  Mock LLM: {len(env.mock_llm.responses)} responses")
    print(f"  Mock Tools: {len(env.mock_tools)} tools")
    print(f"    - calculator")
    print(f"    - search")
    
    # Example 2: Normal Operation Scenario
    print("\n" + "=" * 80)
    print("EXAMPLE 2: Normal Operation Scenario")
    print("=" * 80)
    
    normal_scenario = SimulationScenario(
        id="scenario_001",
        name="Normal Q&A Flow",
        description="Test normal question answering",
        scenario_type=ScenarioType.NORMAL,
        initial_state={"question_count": 0},
        events=[
            {"type": "llm_call", "input": "What is 2+2?"},
            {"type": "state_update", "key": "question_count", "value": 1},
            {"type": "llm_call", "input": "What is the capital of France?"},
            {"type": "state_update", "key": "question_count", "value": 2}
        ],
        success_criteria=lambda state: state.get("question_count", 0) == 2
    )
    
    tester = SimulationTester(env)
    result = tester.run_scenario(normal_scenario)
    
    print(f"\nScenario: {normal_scenario.name}")
    print(f"Type: {normal_scenario.scenario_type.value}")
    print(f"Outcome: {result.outcome.value}")
    print(f"Duration: {result.duration:.3f}s")
    print(f"Events executed: {result.events_executed}/{len(normal_scenario.events)}")
    print(f"Success criteria met: {result.success_criteria_met}")
    print(f"\nMetrics:")
    for key, value in result.metrics.items():
        print(f"  {key}: {value}")
    
    # Example 3: Error Handling Scenario
    print("\n" + "=" * 80)
    print("EXAMPLE 3: Error Handling Scenario")
    print("=" * 80)
    
    error_scenario = SimulationScenario(
        id="scenario_002",
        name="LLM Error Recovery",
        description="Test error handling when LLM fails",
        scenario_type=ScenarioType.ERROR,
        initial_state={"error_handled": False},
        events=[
            {"type": "llm_call", "input": "Error test"},
            {"type": "state_update", "key": "error_handled", "value": True}
        ],
        success_criteria=lambda state: state.get("error_handled", False)
    )
    
    error_result = tester.run_scenario(error_scenario)
    
    print(f"\nScenario: {error_scenario.name}")
    print(f"Outcome: {error_result.outcome.value}")
    print(f"Errors encountered: {len(error_result.errors)}")
    for error in error_result.errors:
        print(f"  - {error}")
    
    # Example 4: Tool Usage Scenario
    print("\n" + "=" * 80)
    print("EXAMPLE 4: Tool Usage Scenario")
    print("=" * 80)
    
    tool_scenario = SimulationScenario(
        id="scenario_003",
        name="Calculator Tool Usage",
        description="Test agent using calculator tool",
        scenario_type=ScenarioType.NORMAL,
        initial_state={"calculations_done": 0},
        events=[
            {"type": "tool_call", "tool": "calculator", "input": "10 + 5"},
            {"type": "state_update", "key": "calculations_done", "value": 1},
            {"type": "tool_call", "tool": "calculator", "input": "20 * 3"},
            {"type": "state_update", "key": "calculations_done", "value": 2}
        ],
        success_criteria=lambda state: state.get("calculations_done", 0) == 2
    )
    
    tool_result = tester.run_scenario(tool_scenario)
    
    print(f"\nScenario: {tool_scenario.name}")
    print(f"Outcome: {tool_result.outcome.value}")
    print(f"Tool calls: {tool_result.metrics['tool_calls']}")
    print(f"Success: {tool_result.success_criteria_met}")
    
    print("\nObservations:")
    for obs in tool_result.observations:
        print(f"  {obs}")
    
    # Example 5: Edge Case Scenario
    print("\n" + "=" * 80)
    print("EXAMPLE 5: Edge Case Scenario")
    print("=" * 80)
    
    edge_scenario = SimulationScenario(
        id="scenario_004",
        name="Empty Input Handling",
        description="Test handling of empty inputs",
        scenario_type=ScenarioType.EDGE_CASE,
        initial_state={"edge_case_handled": False},
        events=[
            {"type": "llm_call", "input": ""},
            {"type": "state_update", "key": "edge_case_handled", "value": True}
        ],
        success_criteria=lambda state: state.get("edge_case_handled", False)
    )
    
    edge_result = tester.run_scenario(edge_scenario)
    
    print(f"\nScenario: {edge_scenario.name}")
    print(f"Type: {edge_scenario.scenario_type.value}")
    print(f"Outcome: {edge_result.outcome.value}")
    print(f"Success: {edge_result.success_criteria_met}")
    
    # Example 6: Load Testing Simulation
    print("\n" + "=" * 80)
    print("EXAMPLE 6: Load Testing Simulation")
    print("=" * 80)
    
    print("\nSimulating high load scenario...")
    
    load_scenarios = []
    for i in range(10):
        scenario = SimulationScenario(
            id=f"load_test_{i}",
            name=f"Load Test {i}",
            description="Concurrent request simulation",
            scenario_type=ScenarioType.LOAD,
            initial_state={},
            events=[
                {"type": "llm_call", "input": f"Query {i}"},
            ],
            success_criteria=lambda state: True
        )
        load_scenarios.append(scenario)
    
    start_time = time.time()
    load_results = tester.run_scenarios(load_scenarios)
    total_time = time.time() - start_time
    
    successful = sum(1 for r in load_results if r.outcome == OutcomeType.SUCCESS)
    
    print(f"\nLoad test completed:")
    print(f"  Total scenarios: {len(load_scenarios)}")
    print(f"  Successful: {successful}")
    print(f"  Failed: {len(load_scenarios) - successful}")
    print(f"  Total time: {total_time:.2f}s")
    print(f"  Avg time per scenario: {total_time/len(load_scenarios):.3f}s")
    
    # Example 7: Comprehensive Test Suite
    print("\n" + "=" * 80)
    print("EXAMPLE 7: Running Comprehensive Test Suite")
    print("=" * 80)
    
    test_suite = [normal_scenario, error_scenario, tool_scenario, edge_scenario]
    
    print(f"\nRunning {len(test_suite)} scenarios...\n")
    
    suite_results = tester.run_scenarios(test_suite)
    
    # Analyze results
    by_outcome = {}
    for result in suite_results:
        outcome = result.outcome.value
        by_outcome[outcome] = by_outcome.get(outcome, 0) + 1
    
    print("Test Suite Results:")
    print(f"  Total: {len(suite_results)}")
    for outcome, count in by_outcome.items():
        print(f"  {outcome.upper()}: {count}")
    
    success_rate = (by_outcome.get("success", 0) / len(suite_results)) * 100
    print(f"\nSuccess Rate: {success_rate:.1f}%")
    
    print("\nDetailed Results:")
    for result in suite_results:
        status = "✓" if result.outcome == OutcomeType.SUCCESS else "✗"
        print(f"  {status} {result.scenario_id}: {result.outcome.value}")
        print(f"     Duration: {result.duration:.3f}s")
        print(f"     Events: {result.events_executed}")
        if result.errors:
            print(f"     Errors: {len(result.errors)}")
    
    # Summary
    print("\n" + "=" * 80)
    print("SIMULATION TESTING SUMMARY")
    print("=" * 80)
    print("""
Simulation Testing Benefits:
1. Safety: Test dangerous scenarios without risk
2. Reproducibility: Consistent test conditions
3. Speed: Fast execution without API calls
4. Cost: No real API costs during testing
5. Control: Complete control over conditions
6. Coverage: Test rare and edge cases easily

Simulation Components:
1. Mock LLM
   - Pre-configured responses
   - Error simulation
   - Delay simulation
   - Call tracking
   - Deterministic behavior

2. Mock Tools
   - Simulated tool behavior
   - Failure injection
   - Performance simulation
   - Call counting
   - State management

3. Simulation Environment
   - State management
   - Event logging
   - Component coordination
   - Reset capabilities
   - Observation collection

Scenario Types:
1. Normal Operation
   - Happy path testing
   - Expected workflows
   - Common use cases
   - Standard behavior
   - Success criteria validation

2. Error Handling
   - LLM failures
   - Tool failures
   - Network errors
   - Timeout scenarios
   - Recovery mechanisms

3. Edge Cases
   - Boundary conditions
   - Empty inputs
   - Large inputs
   - Unusual patterns
   - Corner cases

4. Load Testing
   - Concurrent requests
   - High volume
   - Resource limits
   - Performance degradation
   - Scalability validation

5. Adversarial
   - Malicious inputs
   - Attack scenarios
   - Security testing
   - Abuse detection
   - Safety boundaries

Best Practices:
1. Scenario Design
   - Clear objectives
   - Measurable outcomes
   - Realistic conditions
   - Comprehensive coverage
   - Maintainable tests

2. Mock Fidelity
   - Accurate behavior
   - Realistic timing
   - Error patterns
   - State consistency
   - Documentation

3. Test Organization
   - Logical grouping
   - Clear naming
   - Independent tests
   - Reusable components
   - Version control

4. Continuous Testing
   - CI/CD integration
   - Automated runs
   - Fast feedback
   - Regression detection
   - Quality gates

When to Use Simulation:
✓ Development phase testing
✓ Dangerous scenario validation
✓ Rare case exploration
✓ Cost-sensitive testing
✓ Performance testing
✓ Edge case discovery
✗ Production validation (use real testing)
✗ API behavior verification (use integration tests)

Limitations:
- Mock/reality gap
- May miss real-world issues
- Requires maintenance
- Setup complexity
- False confidence risk

Complementary Approaches:
1. Simulation (Development)
   - Fast feedback
   - Safe exploration
   - Cost-effective

2. Integration Tests (Pre-production)
   - Real API calls
   - Actual behavior
   - Environment validation

3. Shadow Testing (Production)
   - Real traffic
   - No user impact
   - Behavior comparison

4. A/B Testing (Production)
   - User feedback
   - Real performance
   - Gradual rollout

Production Tips:
- Use simulation for development
- Combine with integration tests
- Validate mocks against reality
- Update scenarios regularly
- Monitor real vs simulated gaps
- Document assumptions
- Review test coverage
- Fast test execution (< 5s per scenario)
- Parallel execution for large suites
- Clear failure messages
""")
    
    print("\n" + "=" * 80)
    print("Pattern 092 (Simulation Testing) demonstration complete!")
    print("=" * 80)


if __name__ == "__main__":
    demonstrate_simulation_testing()

    

Agentic Design Pattern: Simulation Testing

Pattern 092: Simulation Testing – Line-by-Line Explanation

Overview

This document provides a detailed line-by-line explanation of the Simulation Testing pattern implementation.

---

Pattern Description

Pattern 092: Simulation Testing

Description: Simulation Testing creates controlled, reproducible environments to test agent behavior under various scenarios, conditions, and edge cases. Unlike production testing, simulations allow you to test dangerous, expensive, or rare scenarios safely and repeatedly. This pattern is essential for validating agent behavior before deployment and for stress testing system limits.

Simulation testing enables:
- Safe testing of dangerous scenarios
- Reproducible test conditions
- Stress and load testing
- Edge case exploration
- Multi-agent interaction testing
- Cost-effective experimentation

Components: 1. Simulation Environment – Mock external services (APIs, databases) – Configurable conditions (delays, errors, loads) – State management and reset – Recording and replay capabilities – Deterministic execution

2. Scenario Definition
   - Initial state setup
   - Event sequences
   - Expected outcomes
   - Success criteria
   - Failure conditions

3. Test Harness
   - Scenario execution engine
   - Result validation
   - Metrics collection
   - Logging and tracing
   - Report generation

4. Mock Components
   - LLM response mocking
   - Tool/API simulation
   - User interaction simulation
   - External system simulation
   - Network condition simulation

Use Cases: 1. Safety Testing – Dangerous scenarios without risk – Edge case handling – Failure mode analysis – Recovery testing – Adversarial inputs

2. Performance Testing
   - Load testing (many concurrent requests)
   - Stress testing (beyond capacity)
   - Endurance testing (long duration)
   - Spike testing (sudden load changes)
   - Scalability validation

3. Integration Testing
   - Multi-component interaction
   - External service integration
   - Data flow validation
   - Error propagation
   - State consistency

4. Regression Testing
   - Validate fixes don't break features
   - Ensure consistent behavior
   - Track performance changes
   - Verify improvements
   - Document behavior changes

5. Exploratory Testing
   - Discover edge cases
   - Test rare scenarios
   - Validate assumptions
   - Find unexpected behaviors
   - Stress system boundaries

LangChain Implementation: LangChain supports simulation through: – Mock LLM implementations – Custom callbacks for control – Test chain compositions – Fake embeddings for testing – Memory simulation

Key Features: 1. Controllable Environment – Deterministic responses – Configurable delays – Error injection – State manipulation – Reproducible conditions

2. Comprehensive Scenarios
   - Normal operations
   - Error conditions
   - Edge cases
   - Load patterns
   - Attack scenarios

3. Detailed Observation
   - Action logging
   - State tracking
   - Performance metrics
   - Error recording
   - Decision paths

4. Flexible Validation
   - Multiple success criteria
   - Partial success handling
   - Performance thresholds
   - State assertions
   - Behavior verification

Best Practices: 1. Scenario Design – Start simple, add complexity – Cover happy path first – Add error scenarios – Include edge cases – Test boundaries

2. Mock Fidelity
   - Balance realism vs simplicity
   - Accurate error simulation
   - Realistic timing
   - State consistency
   - Clear documentation

3. Test Independence
   - Each test isolated
   - Clean state between tests
   - No shared dependencies
   - Deterministic execution
   - Parallel-safe

4. Performance
   - Fast execution (< 1s per test)
   - Efficient mocking
   - Minimal overhead
   - Parallel execution
   - Resource cleanup

Trade-offs: Advantages: – Safe testing of dangerous scenarios – Reproducible conditions – Cost-effective (no real API calls) – Fast execution – Comprehensive coverage – Early issue detection

Disadvantages:
- Mock/reality gap
- Maintenance overhead
- Complexity in setup
- May miss real-world issues
- Requires good scenario design
- Can give false confidence

Production Considerations: 1. Test Coverage – Normal operation paths – Common error scenarios – Known edge cases – Performance limits – Security boundaries

2. Realism Balance
   - Accurate enough for confidence
   - Simple enough to maintain
   - Fast enough for CI/CD
   - Flexible for experimentation
   - Close to production behavior

3. Maintenance
   - Update with code changes
   - Sync with production behavior
   - Regular review
   - Refactor as needed
   - Document assumptions

4. Integration with CI/CD
   - Automated test runs
   - Fast feedback loops
   - Quality gates
   - Regression prevention
   - Performance baselines

5. Hybrid Approach
   - Simulation for development
   - Integration tests for validation
   - Shadow testing in production
   - A/B testing for changes
   - Monitoring in production

---

File Statistics

• Total Lines: 948
• Classes: 9
• Functions: 1
• Constants: 0

---

Module Documentation

"""
Pattern 092: Simulation Testing

Description:
    Simulation Testing creates controlled, reproducible environments to test agent behavior
    under various scenarios, conditions, and edge cases. Unlike production testing,
    simulations allow you to test dangerous, expensive, or rare scenarios safely and
    repeatedly. This pattern is essential for validating agent behavior before deployment
    and for stress testing system limits.

    Simulation testing enables:
    - Safe testing of dangerous scenarios
    - Reproducible test conditions
    - Stress and load testing
    - Edge case exploration
    - Multi-agent interaction testing
    - Cost-effective experimentation

Components:
    1. Simulation Environment
       - Mock external services (APIs, databases)
       - Configurable conditions (delays, errors, loads)
       - State management and reset
       - Recording and replay capabilities
       - Deterministic execution

    2. Scenario Definition
       - Initial state setup
       - Event sequences
       - Expected outcomes
       - Success criteria
       - Failure conditions

    3. Test Harness
       - Scenario execution engine
       - Result validation
       - Metrics collection
       - Logging and tracing
       - Report generation

    4. Mock Components
       - LLM response mocking
       - Tool/API simulation
       - User interaction simulation
       - External system simulation
       - Network condition simulation

Use Cases:
    1. Safety Testing
       - Dangerous scenarios without risk
       - Edge case handling
       - Failure mode analysis
       - Recovery testing
       - Adversarial inputs

    2. Performance Testing
       - Load testing (many concurrent requests)
       - Stress testing (beyond capacity)
       - Endurance testing (long duration)
       - Spike testing (sudden load changes)
       - Scalability validation

    3. Integration Testing
       - Multi-component interaction
       - External service integration
       - Data flow validation
       - Error propagation
       - State consistency

    4. Regression Testing
       - Validate fixes don't break features
       - Ensure consistent behavior
       - Track performance changes
       - Verify improvements
       - Document behavior changes

    5. Exploratory Testing
       - Discover edge cases
       - Test rare scenarios
       - Validate assumptions
       - Find unexpected behaviors
       - Stress system boundaries

LangChain Implementation:
    LangChain supports simulation through:
    - Mock LLM implementations
    - Custom callbacks for control
    - Test chain compositions
    - Fake embeddings for testing
    - Memory simulation

Key Features:
    1. Controllable Environment
       - Deterministic responses
       - Configurable delays
       - Error injection
       - State manipulation
       - Reproducible conditions

    2. Comprehensive Scenarios
       - Normal operations
       - Error conditions
       - Edge cases
       - Load patterns
       - Attack scenarios

    3. Detailed Observation
       - Action logging
       - State tracking
       - Performance metrics
       - Error recording
       - Decision paths

    4. Flexible Validation
       - Multiple success criteria
       - Partial success handling
       - Performance thresholds
       - State assertions
       - Behavior verification

Best Practices:
    1. Scenario Design
       - Start simple, add complexity
       - Cover happy path first
       - Add error scenarios
       - Include edge cases
       - Test boundaries

    2. Mock Fidelity
       - Balance realism vs simplicity
       - Accurate error simulation
       - Realistic timing
       - State consistency
       - Clear documentation

    3. Test Independence
       - Each test isolated
       - Clean state between tests
       - No shared dependencies
       - Deterministic execution
       - Parallel-safe

    4. Performance
       - Fast execution (< 1s per test)
       - Efficient mocking
       - Minimal overhead
       - Parallel execution
       - Resource cleanup

Trade-offs:
    Advantages:
    - Safe testing of dangerous scenarios
    - Reproducible conditions
    - Cost-effective (no real API calls)
    - Fast execution
    - Comprehensive coverage
    - Early issue detection

    Disadvantages:
    - Mock/reality gap
    - Maintenance overhead
    - Complexity in setup
    - May miss real-world issues
    - Requires good scenario design
    - Can give false confidence

Production Considerations:
    1. Test Coverage
       - Normal operation paths
       - Common error scenarios
       - Known edge cases
       - Performance limits
       - Security boundaries

    2. Realism Balance
       - Accurate enough for confidence
       - Simple enough to maintain
       - Fast enough for CI/CD
       - Flexible for experimentation
       - Close to production behavior

    3. Maintenance
       - Update with code changes
       - Sync with production behavior
       - Regular review
       - Refactor as needed
       - Document assumptions

    4. Integration with CI/CD
       - Automated test runs
       - Fast feedback loops
       - Quality gates
       - Regression prevention
       - Performance baselines

    5. Hybrid Approach
       - Simulation for development
       - Integration tests for validation
       - Shadow testing in production
       - A/B testing for changes
       - Monitoring in production
"""

Purpose: This module implements the Simulation Testing pattern, providing a structured approach to agent design and implementation.

---

Import Statements

Standard Library Imports

import dataclasses.dataclass
import dataclasses.field
import datetime.datetime
import enum.Enum
import langchain_core.output_parsers.StrOutputParser
import langchain_core.prompts.ChatPromptTemplate
import os
import random
import time
import typing.Any
import typing.Callable
import typing.Dict
import typing.List
import typing.Optional

Purpose: Core Python functionality: – dataclasses.dataclass: Simplified class definitions with @dataclass – dataclasses.field: Simplified class definitions with @dataclass – datetime.datetime: Date and time handling – enum.Enum: Enumeration support – langchain_core.output_parsers.StrOutputParser: Regular expression operations – langchain_core.prompts.ChatPromptTemplate: Regular expression operations – os: Environment variable access and system operations – random: Random number generation – time: Time-related functions – typing.Any: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Callable: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Dict: Type hints for better code documentation (List, Dict, Any, Optional) – typing.List: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Optional: Type hints for better code documentation (List, Dict, Any, Optional)

Other Imports

from dotenv.load_dotenv

Purpose: Additional dependencies for specialized functionality.

---

Classes

Class: ScenarioType

Lines 219-225

Inherits from: Enum

Purpose:

Types of test scenarios

---

Class: OutcomeType

Lines 228-233

Inherits from: Enum

Purpose:

Test outcome types

---

Class: MockResponse

Lines 237-242

Purpose:

Represents a mock LLM response

---

Class: SimulationScenario

Lines 246-255

Purpose:

Defines a simulation test scenario

---

Class: SimulationResult

Lines 259-268

Purpose:

Result of a simulation test

---

Class: MockLLM

Lines 271-329

Purpose:

Mock LLM for testing that returns pre-configured responses.

Allows testing without making actual API calls.

Methods:

__init__(self, responses)

Lines 278-291

Purpose:

Initialize mock LLM.

Args:
    responses: Dictionary mapping inputs to mock responses

Parameters: – responses: Responses

invoke(self, input)

Lines 293-320

Purpose:

Mock invoke method.

Args:
    input: Input string

Returns:
    Mock response

Parameters: – input: Input data

Returns: str

add_response(self, input, response)

Lines 322-324

Purpose:

Add a response pattern

Parameters: – input: Input data – response: Response

reset(self)

Lines 326-329

Purpose:

Reset mock state

Parameters:

---

Class: MockTool

Lines 332-377

Purpose:

Mock tool for testing agent tool use.

Simulates external tools without actual execution.

Methods:

__init__(self, name, behavior, should_fail, failure_rate)

Lines 339-359

Purpose:

Initialize mock tool.

Args:
    name: Tool name
    behavior: Function defining tool behavior
    should_fail: Whether tool should always fail
    failure_rate: Probability of failure (0-1)

Parameters: – name: Name – behavior: Behavior – should_fail: Should fail – failure_rate: Failure rate

execute(self, input)

Lines 361-377

Purpose:

Execute tool with mock behavior.

Args:
    input: Tool input

Returns:
    Tool output

Parameters: – input: Input data

Returns: str

---

Class: SimulationEnvironment

Lines 380-424

Purpose:

Controlled environment for simulation testing.

Provides mock components and state management.

Methods:

__init__(self)

Lines 387-392

Purpose:

Initialize simulation environment

Parameters:

setup_mock_llm(self, responses)

Lines 394-396

Purpose:

Configure mock LLM responses

Parameters: – responses: Responses

add_mock_tool(self, tool)

Lines 398-400

Purpose:

Add a mock tool

Parameters: – tool: Tool

set_state(self, key, value)

Lines 402-404

Purpose:

Set environment state

Parameters: – key: Key – value: Value

get_state(self, key)

Lines 406-408

Purpose:

Get environment state

Parameters: – key: Key

Returns: Any

log_event(self, event_type, details)

Lines 410-416

Purpose:

Log an event

Parameters: – event_type: Event type – details: Details

reset(self)

Lines 418-424

Purpose:

Reset environment to initial state

Parameters:

---

Class: SimulationTester

Lines 427-558

Purpose:

Executes and validates simulation tests.

Runs scenarios in controlled environment and collects results.

Methods:

__init__(self, environment)

Lines 434-441

Purpose:

Initialize tester.

Args:
    environment: Simulation environment

Parameters: – environment: Environment

run_scenario(self, scenario)

Lines 443-539

Purpose:

Run a simulation scenario.

Args:
    scenario: Scenario to run

Returns:
    Simulation result

Parameters: – scenario: Scenario

Returns: SimulationResult

run_scenarios(self, scenarios)

Lines 541-558

Purpose:

Run multiple scenarios.

Args:
    scenarios: List of scenarios

Returns:
    List of results

Parameters: – scenarios: Scenarios

Returns: <ast.Subscript object at 0x000001A06D60A650>

---

Functions

demonstrate_simulation_testing()

Lines 561-943

Purpose:

Demonstrate simulation testing

---

Key Concepts Summary

Pattern Overview

The Simulation Testing pattern provides a structured approach to building AI agents with specific capabilities and behaviors. This implementation leverages LangChain’s composable components to create a robust and maintainable solution.

Core Components

1. Agent Class: Main implementation of the pattern logic
2. Prompt Engineering: Carefully crafted prompts to guide LLM behavior
3. Tool Integration: Optional tools for extended functionality
4. Memory Management: State handling across interactions
5. Output Parsing: Structured response processing

When to Use

✅ Good for: – Scenarios requiring this specific pattern’s strengths – Applications needing structured agent behavior – Systems where explainability is important – Production environments requiring reliability

❌ Not ideal for: – Simple single-prompt tasks – Scenarios requiring different pattern characteristics – Resource-constrained environments (if computationally intensive)

---

Best Practices Demonstrated

1. Environment Configuration

• ✅ Use .env files for API keys
• ✅ Never hardcode credentials
• ✅ Validate environment variables on startup

2. Error Handling

• ✅ Graceful degradation on failures
• ✅ Informative error messages
• ✅ Logging for debugging

3. Prompt Engineering

• ✅ Clear, specific instructions
• ✅ Structured output formats
• ✅ Few-shot examples when beneficial

4. Code Organization

• ✅ Separation of concerns (class-based design)
• ✅ Type hints for clarity
• ✅ Docstrings for documentation

5. Performance Optimization

• ✅ Appropriate temperature settings
• ✅ Token budget management
• ✅ Caching where applicable

---

LangChain Components Used

• Core LangChain framework components

Why LangChain? – Composable abstractions for LLM applications – Rich ecosystem of integrations – Production-ready patterns – Active community and development

---

Execution Flow

1. Initialize agent with configuration
   ↓
2. Load environment and dependencies
   ↓
3. Prepare prompt with input data
   ↓
4. Invoke LLM through LangChain
   ↓
5. Parse and validate response
   ↓
6. Return formatted result

Key Decision Points: – Model selection (temperature, model type) – Prompt structure (few-shot vs zero-shot) – Error handling strategy – Output format validation

---

Performance Considerations

Computational Cost

• LLM Calls: Primary cost factor (based on tokens)
• Latency: Depends on model and input size
• Memory: Minimal for most patterns

Optimization Strategies

1. Prompt Optimization: Minimize tokens while maintaining clarity
2. Caching: Cache repeated queries/responses
3. Model Selection: Use smaller models when appropriate
4. Batch Processing: Group similar requests
5. Streaming: For long responses, stream tokens

Cost Estimates (Approximate)

• GPT-3.5-turbo: ~$0.002 per request
• GPT-4: ~$0.03 per request
• GPT-4-turbo: ~$0.01 per request

Actual costs vary based on input/output token counts

---

Common Pitfalls to Avoid

1. Prompt Design Issues

❌ Vague or ambiguous instructions
✅ Clear, specific, testable prompts

2. Error Handling

❌ Ignoring API errors or rate limits
✅ Implement retry logic and graceful degradation

3. Token Management

❌ Exceeding model token limits
✅ Truncate inputs, chunk processing

4. Security

❌ Exposing API keys in code
✅ Use environment variables

5. Testing

❌ Only testing happy paths
✅ Test edge cases, failures, malformed inputs

6. Cost Management

❌ Unmonitored API usage
✅ Implement budgets, alerts, and monitoring

---

Further Improvements

Production Enhancements

1. Monitoring: Add logging, metrics, tracing
2. Validation: Input/output schema validation
3. Rate Limiting: Implement request throttling
4. Caching: Redis or in-memory cache for responses
5. Async Support: Non-blocking operations for scalability

Feature Extensions

1. Multi-Model Support: Switch between providers
2. Fallback Models: Graceful degradation strategy
3. Custom Tools: Domain-specific function calling
4. Advanced Memory: Vector stores for long-term memory
5. Evaluation: Automated testing and benchmarking

Code Quality

1. Testing: Unit tests, integration tests
2. Documentation: API docs, usage examples
3. Type Safety: Stricter type hints
4. Configuration: Externalized config files

---

Related Patterns

• Pattern 091
• Pattern 093

Pattern Families: – Reasoning Patterns (001-011) – Multi-Agent Patterns (012-019) – Tool Use Patterns (020-025) – Memory Patterns (026-031) – And many more…

---

Summary

The Simulation Testing pattern provides a powerful approach to building AI agents with specific capabilities. This implementation demonstrates:

✅ Key Strengths: – Clean, maintainable code structure – LangChain integration for composability – Type hints and documentation – Production-ready patterns

🎯 Best Use Cases: – Applications requiring this pattern’s characteristics – Systems needing structured agent behavior – Production environments requiring reliability

📚 Learning Resources: – LangChain Documentation: https://python.langchain.com/ – Pattern Documentation: See related patterns – OpenAI API Reference: https://platform.openai.com/docs/

---

File: 092_simulation_testing.py
Total Lines: 948
Pattern Category: AI Agent Design Pattern #092

---

Generated from actual implementation code
For educational and reference purposes

    """
Pattern 091: Golden Dataset Testing

Description:
    Golden Dataset Testing involves using curated, high-quality test datasets with known
    expected outputs to systematically evaluate agent performance. These "golden" examples
    serve as ground truth for validating agent behavior, detecting regressions, and
    measuring improvements across different versions or configurations. This pattern is
    essential for maintaining quality in production AI systems.

    Golden datasets provide:
    - Reproducible test cases for consistent evaluation
    - Regression detection across code changes
    - Performance benchmarking and comparison
    - Quality assurance for production deployments
    - Documentation of expected behavior
    - Training data for evaluation models

Components:
    1. Golden Dataset Structure
       - Input examples (queries, prompts, contexts)
       - Expected outputs (answers, actions, decisions)
       - Metadata (difficulty, category, source)
       - Evaluation criteria (metrics, thresholds)
       - Test case descriptions

    2. Test Categories
       - Functional tests (correctness)
       - Edge cases (boundary conditions)
       - Error handling (invalid inputs)
       - Performance tests (speed, cost)
       - Regression tests (historical issues)

    3. Evaluation Metrics
       - Exact match accuracy
       - Semantic similarity
       - BLEU/ROUGE scores (text generation)
       - F1 score (classification)
       - Task-specific metrics

    4. Test Management
       - Dataset versioning
       - Test case prioritization
       - Result tracking and reporting
       - Historical comparison
       - Failure analysis

Use Cases:
    1. Regression Testing
       - Detect performance degradation
       - Validate bug fixes
       - Ensure consistent behavior
       - Track improvements over time
       - Compare model versions

    2. Quality Assurance
       - Pre-deployment validation
       - Continuous integration testing
       - Release quality gates
       - Performance monitoring
       - Compliance verification

    3. Model Evaluation
       - Compare different models
       - Evaluate prompt variations
       - Test configuration changes
       - Measure improvements
       - Benchmark performance

    4. Development Workflow
       - TDD (Test-Driven Development)
       - Rapid iteration feedback
       - Feature validation
       - Debug assistance
       - Documentation

    5. Production Monitoring
       - Ongoing quality checks
       - Canary testing
       - Shadow mode validation
       - A/B test evaluation
       - Incident investigation

LangChain Implementation:
    LangChain supports testing through:
    - Custom evaluation chains
    - Comparison evaluators
    - String distance metrics
    - LangSmith for test tracking
    - Dataset management utilities

Key Features:
    1. Comprehensive Test Coverage
       - Multiple difficulty levels
       - Diverse test categories
       - Edge case coverage
       - Error scenarios
       - Performance tests

    2. Automated Evaluation
       - Exact match checking
       - Semantic similarity (embeddings)
       - Custom scoring functions
       - Multi-metric evaluation
       - Threshold-based pass/fail

    3. Result Analysis
       - Detailed failure reports
       - Performance statistics
       - Trend analysis
       - Comparison views
       - Root cause identification

    4. Dataset Management
       - Easy addition of test cases
       - Categorization and tagging
       - Version control
       - Import/export capabilities
       - Collaborative curation

Best Practices:
    1. Dataset Quality
       - Curate carefully (quality over quantity)
       - Include diverse examples
       - Cover edge cases
       - Regular updates
       - Community validation

    2. Test Organization
       - Clear categorization
       - Descriptive naming
       - Priority levels
       - Dependency tracking
       - Logical grouping

    3. Evaluation Strategy
       - Multiple metrics
       - Appropriate thresholds
       - Context-aware scoring
       - Human validation for edge cases
       - Continuous calibration

    4. Maintenance
       - Regular review and updates
       - Remove obsolete tests
       - Add new failure cases
       - Update expected outputs
       - Document changes

Trade-offs:
    Advantages:
    - Reproducible testing
    - Early issue detection
    - Objective quality measurement
    - Regression prevention
    - Development confidence
    - Clear success criteria

    Disadvantages:
    - Dataset curation effort
    - Maintenance overhead
    - May not cover all scenarios
    - Can become outdated
    - Evaluation complexity
    - False positive/negative risks

Production Considerations:
    1. Dataset Coverage
       - Representative samples
       - Stratified by difficulty
       - Cover common patterns
       - Include rare cases
       - Balance across categories

    2. Evaluation Pipeline
       - Automated test runs
       - CI/CD integration
       - Fast feedback loops
       - Parallel execution
       - Result aggregation

    3. Metrics and Thresholds
       - Task-appropriate metrics
       - Calibrated thresholds
       - Multiple evaluation criteria
       - Weighted scoring
       - Context-specific rules

    4. Monitoring and Alerting
       - Track pass rates over time
       - Alert on regressions
       - Performance trends
       - Failure patterns
       - Quality dashboards

    5. Continuous Improvement
       - Add failing production cases
       - Update based on feedback
       - Refine evaluation criteria
       - Expand coverage
       - Community contributions
"""

import os
import json
from typing import List, Dict, Any, Optional, Callable
from dataclasses import dataclass, field
from enum import Enum
from datetime import datetime
from dotenv import load_dotenv

from langchain_core.prompts import ChatPromptTemplate
from langchain_openai import ChatOpenAI, OpenAIEmbeddings
from langchain_core.output_parsers import StrOutputParser

load_dotenv()


class TestCategory(Enum):
    """Test case categories"""
    FUNCTIONAL = "functional"
    EDGE_CASE = "edge_case"
    ERROR_HANDLING = "error_handling"
    PERFORMANCE = "performance"
    REGRESSION = "regression"


class Difficulty(Enum):
    """Test difficulty levels"""
    EASY = "easy"
    MEDIUM = "medium"
    HARD = "hard"


@dataclass
class GoldenTestCase:
    """Represents a single golden test case"""
    id: str
    input: str
    expected_output: str
    category: TestCategory
    difficulty: Difficulty
    description: str
    metadata: Dict[str, Any] = field(default_factory=dict)
    tags: List[str] = field(default_factory=list)


@dataclass
class TestResult:
    """Result of a single test case execution"""
    test_id: str
    passed: bool
    actual_output: str
    expected_output: str
    score: float
    execution_time: float
    error: Optional[str] = None
    details: Dict[str, Any] = field(default_factory=dict)


@dataclass
class TestSuiteResults:
    """Aggregated results for a test suite"""
    total_tests: int
    passed: int
    failed: int
    pass_rate: float
    average_score: float
    total_time: float
    results: List[TestResult]
    timestamp: datetime = field(default_factory=datetime.now)
    
    def summary(self) -> Dict[str, Any]:
        """Get summary statistics"""
        return {
            "total_tests": self.total_tests,
            "passed": self.passed,
            "failed": self.failed,
            "pass_rate": f"{self.pass_rate:.2f}%",
            "average_score": f"{self.average_score:.3f}",
            "total_time": f"{self.total_time:.2f}s",
            "timestamp": self.timestamp.isoformat()
        }


class GoldenDataset:
    """
    Manages a golden dataset of test cases.
    
    Provides methods for adding, retrieving, and organizing test cases.
    """
    
    def __init__(self):
        """Initialize golden dataset"""
        self.test_cases: List[GoldenTestCase] = []
    
    def add_test_case(
        self,
        id: str,
        input: str,
        expected_output: str,
        category: TestCategory,
        difficulty: Difficulty,
        description: str,
        metadata: Optional[Dict[str, Any]] = None,
        tags: Optional[List[str]] = None
    ):
        """
        Add a test case to the dataset.
        
        Args:
            id: Unique test case identifier
            input: Test input
            expected_output: Expected output
            category: Test category
            difficulty: Difficulty level
            description: Test description
            metadata: Additional metadata
            tags: Test tags
        """
        test_case = GoldenTestCase(
            id=id,
            input=input,
            expected_output=expected_output,
            category=category,
            difficulty=difficulty,
            description=description,
            metadata=metadata or {},
            tags=tags or []
        )
        self.test_cases.append(test_case)
    
    def get_by_category(self, category: TestCategory) -> List[GoldenTestCase]:
        """Get test cases by category"""
        return [tc for tc in self.test_cases if tc.category == category]
    
    def get_by_difficulty(self, difficulty: Difficulty) -> List[GoldenTestCase]:
        """Get test cases by difficulty"""
        return [tc for tc in self.test_cases if tc.difficulty == difficulty]
    
    def get_by_tag(self, tag: str) -> List[GoldenTestCase]:
        """Get test cases by tag"""
        return [tc for tc in self.test_cases if tag in tc.tags]
    
    def get_all(self) -> List[GoldenTestCase]:
        """Get all test cases"""
        return self.test_cases
    
    def size(self) -> int:
        """Get dataset size"""
        return len(self.test_cases)


class Evaluator:
    """
    Evaluates agent outputs against expected outputs.
    
    Supports multiple evaluation strategies and metrics.
    """
    
    def __init__(self):
        """Initialize evaluator"""
        self.embeddings = OpenAIEmbeddings()
    
    def exact_match(self, actual: str, expected: str) -> float:
        """
        Exact match evaluation.
        
        Returns:
            1.0 if exact match, 0.0 otherwise
        """
        return 1.0 if actual.strip().lower() == expected.strip().lower() else 0.0
    
    def contains_match(self, actual: str, expected: str) -> float:
        """
        Check if expected output is contained in actual output.
        
        Returns:
            1.0 if expected is in actual, 0.0 otherwise
        """
        return 1.0 if expected.lower() in actual.lower() else 0.0
    
    def semantic_similarity(self, actual: str, expected: str) -> float:
        """
        Compute semantic similarity using embeddings.
        
        Returns:
            Similarity score (0-1)
        """
        # Simple word overlap similarity (production would use actual embeddings)
        actual_words = set(actual.lower().split())
        expected_words = set(expected.lower().split())
        
        if not actual_words or not expected_words:
            return 0.0
        
        intersection = actual_words.intersection(expected_words)
        union = actual_words.union(expected_words)
        
        return len(intersection) / len(union)
    
    def fuzzy_match(self, actual: str, expected: str, threshold: float = 0.8) -> float:
        """
        Fuzzy string matching with threshold.
        
        Args:
            actual: Actual output
            expected: Expected output
            threshold: Similarity threshold
            
        Returns:
            1.0 if similarity >= threshold, else similarity score
        """
        similarity = self.semantic_similarity(actual, expected)
        return 1.0 if similarity >= threshold else similarity
    
    def evaluate(
        self,
        actual: str,
        expected: str,
        method: str = "semantic"
    ) -> float:
        """
        Evaluate using specified method.
        
        Args:
            actual: Actual output
            expected: Expected output
            method: Evaluation method (exact, contains, semantic, fuzzy)
            
        Returns:
            Score (0-1)
        """
        if method == "exact":
            return self.exact_match(actual, expected)
        elif method == "contains":
            return self.contains_match(actual, expected)
        elif method == "semantic":
            return self.semantic_similarity(actual, expected)
        elif method == "fuzzy":
            return self.fuzzy_match(actual, expected)
        else:
            return self.semantic_similarity(actual, expected)


class GoldenDatasetTester:
    """
    Tests an agent using a golden dataset.
    
    Runs test cases, evaluates results, and generates reports.
    """
    
    def __init__(
        self,
        agent_fn: Callable[[str], str],
        evaluator: Evaluator,
        pass_threshold: float = 0.8
    ):
        """
        Initialize tester.
        
        Args:
            agent_fn: Function that takes input and returns output
            evaluator: Evaluator instance
            pass_threshold: Minimum score to pass
        """
        self.agent_fn = agent_fn
        self.evaluator = evaluator
        self.pass_threshold = pass_threshold
    
    def run_test_case(
        self,
        test_case: GoldenTestCase,
        eval_method: str = "semantic"
    ) -> TestResult:
        """
        Run a single test case.
        
        Args:
            test_case: Test case to run
            eval_method: Evaluation method
            
        Returns:
            Test result
        """
        import time
        
        try:
            start_time = time.time()
            actual_output = self.agent_fn(test_case.input)
            execution_time = time.time() - start_time
            
            score = self.evaluator.evaluate(
                actual_output,
                test_case.expected_output,
                method=eval_method
            )
            
            passed = score >= self.pass_threshold
            
            return TestResult(
                test_id=test_case.id,
                passed=passed,
                actual_output=actual_output,
                expected_output=test_case.expected_output,
                score=score,
                execution_time=execution_time,
                details={
                    "category": test_case.category.value,
                    "difficulty": test_case.difficulty.value,
                    "eval_method": eval_method
                }
            )
            
        except Exception as e:
            return TestResult(
                test_id=test_case.id,
                passed=False,
                actual_output="",
                expected_output=test_case.expected_output,
                score=0.0,
                execution_time=0.0,
                error=str(e)
            )
    
    def run_test_suite(
        self,
        dataset: GoldenDataset,
        eval_method: str = "semantic"
    ) -> TestSuiteResults:
        """
        Run entire test suite.
        
        Args:
            dataset: Golden dataset
            eval_method: Evaluation method
            
        Returns:
            Test suite results
        """
        results = []
        
        for test_case in dataset.get_all():
            result = self.run_test_case(test_case, eval_method)
            results.append(result)
        
        passed = sum(1 for r in results if r.passed)
        failed = len(results) - passed
        pass_rate = (passed / len(results) * 100) if results else 0.0
        average_score = sum(r.score for r in results) / len(results) if results else 0.0
        total_time = sum(r.execution_time for r in results)
        
        return TestSuiteResults(
            total_tests=len(results),
            passed=passed,
            failed=failed,
            pass_rate=pass_rate,
            average_score=average_score,
            total_time=total_time,
            results=results
        )
    
    def generate_report(self, results: TestSuiteResults) -> str:
        """
        Generate detailed test report.
        
        Args:
            results: Test suite results
            
        Returns:
            Formatted report
        """
        report = []
        report.append("=" * 80)
        report.append("GOLDEN DATASET TEST REPORT")
        report.append("=" * 80)
        report.append(f"Timestamp: {results.timestamp}")
        report.append(f"Total Tests: {results.total_tests}")
        report.append(f"Passed: {results.passed}")
        report.append(f"Failed: {results.failed}")
        report.append(f"Pass Rate: {results.pass_rate:.2f}%")
        report.append(f"Average Score: {results.average_score:.3f}")
        report.append(f"Total Time: {results.total_time:.2f}s")
        report.append("")
        
        if results.failed > 0:
            report.append("FAILED TESTS:")
            report.append("-" * 80)
            for result in results.results:
                if not result.passed:
                    report.append(f"\nTest ID: {result.test_id}")
                    report.append(f"Score: {result.score:.3f}")
                    report.append(f"Expected: {result.expected_output[:100]}...")
                    report.append(f"Actual: {result.actual_output[:100]}...")
                    if result.error:
                        report.append(f"Error: {result.error}")
        
        report.append("\n" + "=" * 80)
        
        return "\n".join(report)


def demonstrate_golden_dataset_testing():
    """Demonstrate golden dataset testing"""
    print("=" * 80)
    print("GOLDEN DATASET TESTING DEMONSTRATION")
    print("=" * 80)
    
    # Example 1: Create Golden Dataset
    print("\n" + "=" * 80)
    print("EXAMPLE 1: Creating a Golden Dataset")
    print("=" * 80)
    
    dataset = GoldenDataset()
    
    # Add test cases
    dataset.add_test_case(
        id="test_001",
        input="What is 2 + 2?",
        expected_output="4",
        category=TestCategory.FUNCTIONAL,
        difficulty=Difficulty.EASY,
        description="Basic arithmetic",
        tags=["math", "basic"]
    )
    
    dataset.add_test_case(
        id="test_002",
        input="What is the capital of France?",
        expected_output="Paris",
        category=TestCategory.FUNCTIONAL,
        difficulty=Difficulty.EASY,
        description="Geography question",
        tags=["geography", "factual"]
    )
    
    dataset.add_test_case(
        id="test_003",
        input="Explain quantum computing in simple terms",
        expected_output="Quantum computing uses quantum mechanics principles like superposition and entanglement",
        category=TestCategory.FUNCTIONAL,
        difficulty=Difficulty.HARD,
        description="Complex explanation",
        tags=["science", "explanation"]
    )
    
    dataset.add_test_case(
        id="test_004",
        input="",
        expected_output="I need a question to answer",
        category=TestCategory.EDGE_CASE,
        difficulty=Difficulty.MEDIUM,
        description="Empty input handling",
        tags=["edge_case", "validation"]
    )
    
    print(f"\nCreated golden dataset with {dataset.size()} test cases")
    print("\nTest cases by category:")
    for category in TestCategory:
        cases = dataset.get_by_category(category)
        print(f"  {category.value}: {len(cases)} tests")
    
    print("\nTest cases by difficulty:")
    for difficulty in Difficulty:
        cases = dataset.get_by_difficulty(difficulty)
        print(f"  {difficulty.value}: {len(cases)} tests")
    
    # Example 2: Simple Agent for Testing
    print("\n" + "=" * 80)
    print("EXAMPLE 2: Creating Test Agent")
    print("=" * 80)
    
    llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0.3)
    
    def simple_agent(input: str) -> str:
        """Simple agent for testing"""
        if not input.strip():
            return "I need a question to answer"
        
        prompt = ChatPromptTemplate.from_template(
            "Answer the following question concisely:\n\n{question}"
        )
        chain = prompt | llm | StrOutputParser()
        return chain.invoke({"question": input})
    
    print("\nAgent created: simple_agent")
    print("Agent behavior: Answers questions concisely")
    
    # Example 3: Evaluation Methods
    print("\n" + "=" * 80)
    print("EXAMPLE 3: Testing Different Evaluation Methods")
    print("=" * 80)
    
    evaluator = Evaluator()
    
    test_actual = "The capital of France is Paris"
    test_expected = "Paris"
    
    print(f"\nActual: '{test_actual}'")
    print(f"Expected: '{test_expected}'\n")
    
    methods = ["exact", "contains", "semantic", "fuzzy"]
    for method in methods:
        score = evaluator.evaluate(test_actual, test_expected, method=method)
        print(f"{method.upper()} match: {score:.3f}")
    
    # Example 4: Running Test Suite
    print("\n" + "=" * 80)
    print("EXAMPLE 4: Running Complete Test Suite")
    print("=" * 80)
    
    tester = GoldenDatasetTester(
        agent_fn=simple_agent,
        evaluator=evaluator,
        pass_threshold=0.7
    )
    
    print("\nRunning test suite...")
    print(f"Evaluation method: semantic similarity")
    print(f"Pass threshold: 0.7\n")
    
    results = tester.run_test_suite(dataset, eval_method="semantic")
    
    print(f"Results:")
    print(f"  Total: {results.total_tests}")
    print(f"  Passed: {results.passed}")
    print(f"  Failed: {results.failed}")
    print(f"  Pass Rate: {results.pass_rate:.2f}%")
    print(f"  Avg Score: {results.average_score:.3f}")
    print(f"  Total Time: {results.total_time:.2f}s")
    
    # Example 5: Detailed Results Analysis
    print("\n" + "=" * 80)
    print("EXAMPLE 5: Detailed Results Analysis")
    print("=" * 80)
    
    print("\nIndividual Test Results:\n")
    for result in results.results:
        status = "✓ PASS" if result.passed else "✗ FAIL"
        print(f"{status} - {result.test_id}")
        print(f"  Score: {result.score:.3f}")
        print(f"  Time: {result.execution_time:.3f}s")
        print(f"  Category: {result.details.get('category', 'N/A')}")
        print(f"  Difficulty: {result.details.get('difficulty', 'N/A')}")
        if not result.passed:
            print(f"  Expected: {result.expected_output[:50]}...")
            print(f"  Actual: {result.actual_output[:50]}...")
        print()
    
    # Example 6: Category-Based Analysis
    print("\n" + "=" * 80)
    print("EXAMPLE 6: Performance by Category")
    print("=" * 80)
    
    category_stats = {}
    for result in results.results:
        category = result.details.get('category', 'unknown')
        if category not in category_stats:
            category_stats[category] = {"total": 0, "passed": 0, "scores": []}
        
        category_stats[category]["total"] += 1
        if result.passed:
            category_stats[category]["passed"] += 1
        category_stats[category]["scores"].append(result.score)
    
    print("\nPerformance by Category:\n")
    for category, stats in category_stats.items():
        pass_rate = (stats["passed"] / stats["total"] * 100) if stats["total"] > 0 else 0
        avg_score = sum(stats["scores"]) / len(stats["scores"]) if stats["scores"] else 0
        
        print(f"{category.upper()}:")
        print(f"  Tests: {stats['total']}")
        print(f"  Passed: {stats['passed']}/{stats['total']}")
        print(f"  Pass Rate: {pass_rate:.2f}%")
        print(f"  Avg Score: {avg_score:.3f}")
        print()
    
    # Example 7: Regression Testing
    print("\n" + "=" * 80)
    print("EXAMPLE 7: Regression Testing Simulation")
    print("=" * 80)
    
    print("\nSimulating version comparison...")
    
    # Simulate baseline results
    baseline_pass_rate = 85.0
    current_pass_rate = results.pass_rate
    
    print(f"\nBaseline (v1.0): {baseline_pass_rate:.2f}% pass rate")
    print(f"Current (v2.0): {current_pass_rate:.2f}% pass rate")
    
    if current_pass_rate >= baseline_pass_rate:
        print("\n✓ No regression detected - Quality maintained or improved!")
    else:
        regression = baseline_pass_rate - current_pass_rate
        print(f"\n✗ REGRESSION DETECTED: {regression:.2f}% decrease in pass rate")
        print("   Action: Investigate failing tests before deployment")
    
    # Example 8: Test Report Generation
    print("\n" + "=" * 80)
    print("EXAMPLE 8: Generating Test Report")
    print("=" * 80)
    
    report = tester.generate_report(results)
    print("\n" + report)
    
    # Summary
    print("\n" + "=" * 80)
    print("GOLDEN DATASET TESTING SUMMARY")
    print("=" * 80)
    print("""
Golden Dataset Testing Benefits:
1. Reproducibility: Consistent test results across runs
2. Regression Detection: Catch performance degradation early
3. Quality Assurance: Objective measurement of agent quality
4. Development Confidence: Safe to refactor and improve
5. Documentation: Tests document expected behavior
6. Benchmarking: Compare different configurations

Dataset Best Practices:
1. Quality Over Quantity
   - Carefully curate test cases
   - Focus on important scenarios
   - Include edge cases
   - Regular review and updates

2. Organization
   - Clear categories (functional, edge case, etc.)
   - Difficulty levels (easy, medium, hard)
   - Descriptive tags
   - Comprehensive metadata

3. Coverage
   - Common use cases
   - Edge cases and boundaries
   - Error scenarios
   - Performance critical paths
   - Regression cases from production

4. Maintenance
   - Add failing production cases
   - Update expected outputs when needed
   - Remove obsolete tests
   - Version control dataset
   - Document changes

Evaluation Methods:
1. Exact Match
   - Strict equality check
   - Best for: Factual answers, specific outputs
   - Limitation: Too strict for natural language

2. Contains Match
   - Check if expected is in actual
   - Best for: Key information presence
   - Limitation: Doesn't verify quality

3. Semantic Similarity
   - Meaning-based comparison
   - Best for: Natural language responses
   - Limitation: Computationally expensive

4. Fuzzy Match
   - Similarity with threshold
   - Best for: Near-match acceptance
   - Limitation: Threshold tuning needed

Test Suite Structure:
1. Core Functionality (60%)
   - Essential features
   - Common use cases
   - Critical paths

2. Edge Cases (20%)
   - Boundary conditions
   - Unusual inputs
   - Corner cases

3. Error Handling (10%)
   - Invalid inputs
   - System errors
   - Recovery scenarios

4. Performance (10%)
   - Speed benchmarks
   - Resource usage
   - Scalability tests

Continuous Integration:
1. Automated Test Runs
   - Run on every commit
   - Pre-deployment validation
   - Scheduled regression tests
   - Performance monitoring

2. Quality Gates
   - Minimum pass rate (e.g., 95%)
   - Maximum regression (e.g., 5%)
   - Performance thresholds
   - Coverage requirements

3. Reporting
   - Dashboard visualization
   - Trend analysis
   - Failure notifications
   - Historical comparison

When to Use Golden Dataset Testing:
✓ Production AI systems
✓ Iterative development
✓ Multiple team members
✓ Complex agent behaviors
✓ Compliance requirements
✓ Performance monitoring
✗ One-off scripts
✗ Exploratory prototypes (initially)

Production Tips:
- Start small, grow dataset organically
- Add production failures to dataset
- Regular dataset review sessions
- Version control your datasets
- Share datasets across team
- Integrate with CI/CD pipeline
- Monitor test execution time
- Keep tests fast (< 5 min for suite)
- Parallel execution for large datasets
- Alert on regressions immediately
""")
    
    print("\n" + "=" * 80)
    print("Pattern 091 (Golden Dataset Testing) demonstration complete!")
    print("=" * 80)


if __name__ == "__main__":
    demonstrate_golden_dataset_testing()

    

Agentic Design Pattern: Golden Dataset Testing

Pattern 091: Golden Dataset Testing – Line-by-Line Explanation

Overview

This document provides a detailed line-by-line explanation of the Golden Dataset Testing pattern implementation.

---

Pattern Description

Pattern 091: Golden Dataset Testing

Description: Golden Dataset Testing involves using curated, high-quality test datasets with known expected outputs to systematically evaluate agent performance. These “golden” examples serve as ground truth for validating agent behavior, detecting regressions, and measuring improvements across different versions or configurations. This pattern is essential for maintaining quality in production AI systems.

Golden datasets provide:
- Reproducible test cases for consistent evaluation
- Regression detection across code changes
- Performance benchmarking and comparison
- Quality assurance for production deployments
- Documentation of expected behavior
- Training data for evaluation models

Components: 1. Golden Dataset Structure – Input examples (queries, prompts, contexts) – Expected outputs (answers, actions, decisions) – Metadata (difficulty, category, source) – Evaluation criteria (metrics, thresholds) – Test case descriptions

2. Test Categories
   - Functional tests (correctness)
   - Edge cases (boundary conditions)
   - Error handling (invalid inputs)
   - Performance tests (speed, cost)
   - Regression tests (historical issues)

3. Evaluation Metrics
   - Exact match accuracy
   - Semantic similarity
   - BLEU/ROUGE scores (text generation)
   - F1 score (classification)
   - Task-specific metrics

4. Test Management
   - Dataset versioning
   - Test case prioritization
   - Result tracking and reporting
   - Historical comparison
   - Failure analysis

Use Cases: 1. Regression Testing – Detect performance degradation – Validate bug fixes – Ensure consistent behavior – Track improvements over time – Compare model versions

2. Quality Assurance
   - Pre-deployment validation
   - Continuous integration testing
   - Release quality gates
   - Performance monitoring
   - Compliance verification

3. Model Evaluation
   - Compare different models
   - Evaluate prompt variations
   - Test configuration changes
   - Measure improvements
   - Benchmark performance

4. Development Workflow
   - TDD (Test-Driven Development)
   - Rapid iteration feedback
   - Feature validation
   - Debug assistance
   - Documentation

5. Production Monitoring
   - Ongoing quality checks
   - Canary testing
   - Shadow mode validation
   - A/B test evaluation
   - Incident investigation

LangChain Implementation: LangChain supports testing through: – Custom evaluation chains – Comparison evaluators – String distance metrics – LangSmith for test tracking – Dataset management utilities

Key Features: 1. Comprehensive Test Coverage – Multiple difficulty levels – Diverse test categories – Edge case coverage – Error scenarios – Performance tests

2. Automated Evaluation
   - Exact match checking
   - Semantic similarity (embeddings)
   - Custom scoring functions
   - Multi-metric evaluation
   - Threshold-based pass/fail

3. Result Analysis
   - Detailed failure reports
   - Performance statistics
   - Trend analysis
   - Comparison views
   - Root cause identification

4. Dataset Management
   - Easy addition of test cases
   - Categorization and tagging
   - Version control
   - Import/export capabilities
   - Collaborative curation

Best Practices: 1. Dataset Quality – Curate carefully (quality over quantity) – Include diverse examples – Cover edge cases – Regular updates – Community validation

2. Test Organization
   - Clear categorization
   - Descriptive naming
   - Priority levels
   - Dependency tracking
   - Logical grouping

3. Evaluation Strategy
   - Multiple metrics
   - Appropriate thresholds
   - Context-aware scoring
   - Human validation for edge cases
   - Continuous calibration

4. Maintenance
   - Regular review and updates
   - Remove obsolete tests
   - Add new failure cases
   - Update expected outputs
   - Document changes

Trade-offs: Advantages: – Reproducible testing – Early issue detection – Objective quality measurement – Regression prevention – Development confidence – Clear success criteria

Disadvantages:
- Dataset curation effort
- Maintenance overhead
- May not cover all scenarios
- Can become outdated
- Evaluation complexity
- False positive/negative risks

Production Considerations: 1. Dataset Coverage – Representative samples – Stratified by difficulty – Cover common patterns – Include rare cases – Balance across categories

2. Evaluation Pipeline
   - Automated test runs
   - CI/CD integration
   - Fast feedback loops
   - Parallel execution
   - Result aggregation

3. Metrics and Thresholds
   - Task-appropriate metrics
   - Calibrated thresholds
   - Multiple evaluation criteria
   - Weighted scoring
   - Context-specific rules

4. Monitoring and Alerting
   - Track pass rates over time
   - Alert on regressions
   - Performance trends
   - Failure patterns
   - Quality dashboards

5. Continuous Improvement
   - Add failing production cases
   - Update based on feedback
   - Refine evaluation criteria
   - Expand coverage
   - Community contributions

---

File Statistics

• Total Lines: 938
• Classes: 8
• Functions: 1
• Constants: 0

---

Module Documentation

"""
Pattern 091: Golden Dataset Testing

Description:
    Golden Dataset Testing involves using curated, high-quality test datasets with known
    expected outputs to systematically evaluate agent performance. These "golden" examples
    serve as ground truth for validating agent behavior, detecting regressions, and
    measuring improvements across different versions or configurations. This pattern is
    essential for maintaining quality in production AI systems.

    Golden datasets provide:
    - Reproducible test cases for consistent evaluation
    - Regression detection across code changes
    - Performance benchmarking and comparison
    - Quality assurance for production deployments
    - Documentation of expected behavior
    - Training data for evaluation models

Components:
    1. Golden Dataset Structure
       - Input examples (queries, prompts, contexts)
       - Expected outputs (answers, actions, decisions)
       - Metadata (difficulty, category, source)
       - Evaluation criteria (metrics, thresholds)
       - Test case descriptions

    2. Test Categories
       - Functional tests (correctness)
       - Edge cases (boundary conditions)
       - Error handling (invalid inputs)
       - Performance tests (speed, cost)
       - Regression tests (historical issues)

    3. Evaluation Metrics
       - Exact match accuracy
       - Semantic similarity
       - BLEU/ROUGE scores (text generation)
       - F1 score (classification)
       - Task-specific metrics

    4. Test Management
       - Dataset versioning
       - Test case prioritization
       - Result tracking and reporting
       - Historical comparison
       - Failure analysis

Use Cases:
    1. Regression Testing
       - Detect performance degradation
       - Validate bug fixes
       - Ensure consistent behavior
       - Track improvements over time
       - Compare model versions

    2. Quality Assurance
       - Pre-deployment validation
       - Continuous integration testing
       - Release quality gates
       - Performance monitoring
       - Compliance verification

    3. Model Evaluation
       - Compare different models
       - Evaluate prompt variations
       - Test configuration changes
       - Measure improvements
       - Benchmark performance

    4. Development Workflow
       - TDD (Test-Driven Development)
       - Rapid iteration feedback
       - Feature validation
       - Debug assistance
       - Documentation

    5. Production Monitoring
       - Ongoing quality checks
       - Canary testing
       - Shadow mode validation
       - A/B test evaluation
       - Incident investigation

LangChain Implementation:
    LangChain supports testing through:
    - Custom evaluation chains
    - Comparison evaluators
    - String distance metrics
    - LangSmith for test tracking
    - Dataset management utilities

Key Features:
    1. Comprehensive Test Coverage
       - Multiple difficulty levels
       - Diverse test categories
       - Edge case coverage
       - Error scenarios
       - Performance tests

    2. Automated Evaluation
       - Exact match checking
       - Semantic similarity (embeddings)
       - Custom scoring functions
       - Multi-metric evaluation
       - Threshold-based pass/fail

    3. Result Analysis
       - Detailed failure reports
       - Performance statistics
       - Trend analysis
       - Comparison views
       - Root cause identification

    4. Dataset Management
       - Easy addition of test cases
       - Categorization and tagging
       - Version control
       - Import/export capabilities
       - Collaborative curation

Best Practices:
    1. Dataset Quality
       - Curate carefully (quality over quantity)
       - Include diverse examples
       - Cover edge cases
       - Regular updates
       - Community validation

    2. Test Organization
       - Clear categorization
       - Descriptive naming
       - Priority levels
       - Dependency tracking
       - Logical grouping

    3. Evaluation Strategy
       - Multiple metrics
       - Appropriate thresholds
       - Context-aware scoring
       - Human validation for edge cases
       - Continuous calibration

    4. Maintenance
       - Regular review and updates
       - Remove obsolete tests
       - Add new failure cases
       - Update expected outputs
       - Document changes

Trade-offs:
    Advantages:
    - Reproducible testing
    - Early issue detection
    - Objective quality measurement
    - Regression prevention
    - Development confidence
    - Clear success criteria

    Disadvantages:
    - Dataset curation effort
    - Maintenance overhead
    - May not cover all scenarios
    - Can become outdated
    - Evaluation complexity
    - False positive/negative risks

Production Considerations:
    1. Dataset Coverage
       - Representative samples
       - Stratified by difficulty
       - Cover common patterns
       - Include rare cases
       - Balance across categories

    2. Evaluation Pipeline
       - Automated test runs
       - CI/CD integration
       - Fast feedback loops
       - Parallel execution
       - Result aggregation

    3. Metrics and Thresholds
       - Task-appropriate metrics
       - Calibrated thresholds
       - Multiple evaluation criteria
       - Weighted scoring
       - Context-specific rules

    4. Monitoring and Alerting
       - Track pass rates over time
       - Alert on regressions
       - Performance trends
       - Failure patterns
       - Quality dashboards

    5. Continuous Improvement
       - Add failing production cases
       - Update based on feedback
       - Refine evaluation criteria
       - Expand coverage
       - Community contributions
"""

Purpose: This module implements the Golden Dataset Testing pattern, providing a structured approach to agent design and implementation.

---

Import Statements

Standard Library Imports

import dataclasses.dataclass
import dataclasses.field
import datetime.datetime
import enum.Enum
import json
import langchain_core.output_parsers.StrOutputParser
import langchain_core.prompts.ChatPromptTemplate
import os
import time
import typing.Any
import typing.Callable
import typing.Dict
import typing.List
import typing.Optional

Purpose: Core Python functionality: – dataclasses.dataclass: Simplified class definitions with @dataclass – dataclasses.field: Simplified class definitions with @dataclass – datetime.datetime: Date and time handling – enum.Enum: Enumeration support – json: JSON serialization/deserialization – langchain_core.output_parsers.StrOutputParser: Regular expression operations – langchain_core.prompts.ChatPromptTemplate: Regular expression operations – os: Environment variable access and system operations – time: Time-related functions – typing.Any: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Callable: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Dict: Type hints for better code documentation (List, Dict, Any, Optional) – typing.List: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Optional: Type hints for better code documentation (List, Dict, Any, Optional)

LangChain Imports

from langchain_openai.ChatOpenAI
from langchain_openai.OpenAIEmbeddings

Purpose: LangChain framework components: – LLM Interface: Wrapper for OpenAI models – Chains: Composable building blocks for complex workflows

Other Imports

from dotenv.load_dotenv

Purpose: Additional dependencies for specialized functionality.

---

Classes

Class: TestCategory

Lines 219-225

Inherits from: Enum

Purpose:

Test case categories

---

Class: Difficulty

Lines 228-232

Inherits from: Enum

Purpose:

Test difficulty levels

---

Class: GoldenTestCase

Lines 236-245

Purpose:

Represents a single golden test case

---

Class: TestResult

Lines 249-258

Purpose:

Result of a single test case execution

---

Class: TestSuiteResults

Lines 262-283

Purpose:

Aggregated results for a test suite

Methods:

summary(self)

Lines 273-283

Purpose:

Get summary statistics

Parameters:

Returns: <ast.Subscript object at 0x000001A06D855410>

---

Class: GoldenDataset

Lines 286-351

Purpose:

Manages a golden dataset of test cases.

Provides methods for adding, retrieving, and organizing test cases.

Methods:

__init__(self)

Lines 293-295

Purpose:

Initialize golden dataset

Parameters:

add_test_case(self, id, input, expected_output, category, difficulty, description, metadata, tags)

Lines 297-331

Purpose:

Add a test case to the dataset.

Args:
    id: Unique test case identifier
    input: Test input
    expected_output: Expected output
    category: Test category
    difficulty: Difficulty level
    description: Test description
    metadata: Additional metadata
    tags: Test tags

Parameters: – id: Id – input: Input data – expected_output: Expected output – category: Category – difficulty: Difficulty – description: Description – metadata: Metadata – tags: Tags

get_by_category(self, category)

Lines 333-335

Purpose:

Get test cases by category

Parameters: – category: Category

Returns: <ast.Subscript object at 0x000001A06E094450>

get_by_difficulty(self, difficulty)

Lines 337-339

Purpose:

Get test cases by difficulty

Parameters: – difficulty: Difficulty

Returns: <ast.Subscript object at 0x000001A06D65BE50>

get_by_tag(self, tag)

Lines 341-343

Purpose:

Get test cases by tag

Parameters: – tag: Tag

Returns: <ast.Subscript object at 0x000001A06D65A850>

get_all(self)

Lines 345-347

Purpose:

Get all test cases

Parameters:

Returns: <ast.Subscript object at 0x000001A06D65B2D0>

size(self)

Lines 349-351

Purpose:

Get dataset size

Parameters:

Returns: int

---

Class: Evaluator

Lines 354-443

Purpose:

Evaluates agent outputs against expected outputs.

Supports multiple evaluation strategies and metrics.

Methods:

__init__(self)

Lines 361-363

Purpose:

Initialize evaluator

Parameters:

exact_match(self, actual, expected)

Lines 365-372

Purpose:

Exact match evaluation.

Returns:
    1.0 if exact match, 0.0 otherwise

Parameters: – actual: Actual – expected: Expected

Returns: float

contains_match(self, actual, expected)

Lines 374-381

Purpose:

Check if expected output is contained in actual output.

Returns:
    1.0 if expected is in actual, 0.0 otherwise

Parameters: – actual: Actual – expected: Expected

Returns: float

semantic_similarity(self, actual, expected)

Lines 383-400

Purpose:

Compute semantic similarity using embeddings.

Returns:
    Similarity score (0-1)

Parameters: – actual: Actual – expected: Expected

Returns: float

fuzzy_match(self, actual, expected, threshold)

Lines 402-415

Purpose:

Fuzzy string matching with threshold.

Args:
    actual: Actual output
    expected: Expected output
    threshold: Similarity threshold

Returns:
    1.0 if similarity >= threshold, else similarity score

Parameters: – actual: Actual – expected: Expected – threshold: Threshold

Returns: float

evaluate(self, actual, expected, method)

Lines 417-443

Purpose:

Evaluate using specified method.

Args:
    actual: Actual output
    expected: Expected output
    method: Evaluation method (exact, contains, semantic, fuzzy)

Returns:
    Score (0-1)

Parameters: – actual: Actual – expected: Expected – method: Method

Returns: float

---

Class: GoldenDatasetTester

Lines 446-600

Purpose:

Tests an agent using a golden dataset.

Runs test cases, evaluates results, and generates reports.

Methods:

__init__(self, agent_fn, evaluator, pass_threshold)

Lines 453-469

Purpose:

Initialize tester.

Args:
    agent_fn: Function that takes input and returns output
    evaluator: Evaluator instance
    pass_threshold: Minimum score to pass

Parameters: – agent_fn: Agent fn – evaluator: Evaluator – pass_threshold: Pass threshold

run_test_case(self, test_case, eval_method)

Lines 471-524

Purpose:

Run a single test case.

Args:
    test_case: Test case to run
    eval_method: Evaluation method

Returns:
    Test result

Parameters: – test_case: Test case – eval_method: Eval method

Returns: TestResult

run_test_suite(self, dataset, eval_method)

Lines 526-561

Purpose:

Run entire test suite.

Args:
    dataset: Golden dataset
    eval_method: Evaluation method

Returns:
    Test suite results

Parameters: – dataset: Dataset – eval_method: Eval method

Returns: TestSuiteResults

generate_report(self, results)

Lines 563-600

Purpose:

Generate detailed test report.

Args:
    results: Test suite results

Returns:
    Formatted report

Parameters: – results: Results

Returns: str

---

Functions

demonstrate_golden_dataset_testing()

Lines 603-933

Purpose:

Demonstrate golden dataset testing

---

Key Concepts Summary

Pattern Overview

The Golden Dataset Testing pattern provides a structured approach to building AI agents with specific capabilities and behaviors. This implementation leverages LangChain’s composable components to create a robust and maintainable solution.

Core Components

1. Agent Class: Main implementation of the pattern logic
2. Prompt Engineering: Carefully crafted prompts to guide LLM behavior
3. Tool Integration: Optional tools for extended functionality
4. Memory Management: State handling across interactions
5. Output Parsing: Structured response processing

When to Use

✅ Good for: – Scenarios requiring this specific pattern’s strengths – Applications needing structured agent behavior – Systems where explainability is important – Production environments requiring reliability

❌ Not ideal for: – Simple single-prompt tasks – Scenarios requiring different pattern characteristics – Resource-constrained environments (if computationally intensive)

---

Best Practices Demonstrated

1. Environment Configuration

• ✅ Use .env files for API keys
• ✅ Never hardcode credentials
• ✅ Validate environment variables on startup

2. Error Handling

• ✅ Graceful degradation on failures
• ✅ Informative error messages
• ✅ Logging for debugging

3. Prompt Engineering

• ✅ Clear, specific instructions
• ✅ Structured output formats
• ✅ Few-shot examples when beneficial

4. Code Organization

• ✅ Separation of concerns (class-based design)
• ✅ Type hints for clarity
• ✅ Docstrings for documentation

5. Performance Optimization

• ✅ Appropriate temperature settings
• ✅ Token budget management
• ✅ Caching where applicable

---

LangChain Components Used

• ChatOpenAI: OpenAI model interface
• Chains: Sequential component composition

Why LangChain? – Composable abstractions for LLM applications – Rich ecosystem of integrations – Production-ready patterns – Active community and development

---

Execution Flow

1. Initialize agent with configuration
   ↓
2. Load environment and dependencies
   ↓
3. Prepare prompt with input data
   ↓
4. Invoke LLM through LangChain
   ↓
5. Parse and validate response
   ↓
6. Return formatted result

Key Decision Points: – Model selection (temperature, model type) – Prompt structure (few-shot vs zero-shot) – Error handling strategy – Output format validation

---

Performance Considerations

Computational Cost

• LLM Calls: Primary cost factor (based on tokens)
• Latency: Depends on model and input size
• Memory: Minimal for most patterns

Optimization Strategies

1. Prompt Optimization: Minimize tokens while maintaining clarity
2. Caching: Cache repeated queries/responses
3. Model Selection: Use smaller models when appropriate
4. Batch Processing: Group similar requests
5. Streaming: For long responses, stream tokens

Cost Estimates (Approximate)

• GPT-3.5-turbo: ~$0.002 per request
• GPT-4: ~$0.03 per request
• GPT-4-turbo: ~$0.01 per request

Actual costs vary based on input/output token counts

---

Common Pitfalls to Avoid

1. Prompt Design Issues

❌ Vague or ambiguous instructions
✅ Clear, specific, testable prompts

2. Error Handling

❌ Ignoring API errors or rate limits
✅ Implement retry logic and graceful degradation

3. Token Management

❌ Exceeding model token limits
✅ Truncate inputs, chunk processing

4. Security

❌ Exposing API keys in code
✅ Use environment variables

5. Testing

❌ Only testing happy paths
✅ Test edge cases, failures, malformed inputs

6. Cost Management

❌ Unmonitored API usage
✅ Implement budgets, alerts, and monitoring

---

Further Improvements

Production Enhancements

1. Monitoring: Add logging, metrics, tracing
2. Validation: Input/output schema validation
3. Rate Limiting: Implement request throttling
4. Caching: Redis or in-memory cache for responses
5. Async Support: Non-blocking operations for scalability

Feature Extensions

1. Multi-Model Support: Switch between providers
2. Fallback Models: Graceful degradation strategy
3. Custom Tools: Domain-specific function calling
4. Advanced Memory: Vector stores for long-term memory
5. Evaluation: Automated testing and benchmarking

Code Quality

1. Testing: Unit tests, integration tests
2. Documentation: API docs, usage examples
3. Type Safety: Stricter type hints
4. Configuration: Externalized config files

---

Related Patterns

• Pattern 090
• Pattern 092

Pattern Families: – Reasoning Patterns (001-011) – Multi-Agent Patterns (012-019) – Tool Use Patterns (020-025) – Memory Patterns (026-031) – And many more…

---

Summary

The Golden Dataset Testing pattern provides a powerful approach to building AI agents with specific capabilities. This implementation demonstrates:

✅ Key Strengths: – Clean, maintainable code structure – LangChain integration for composability – Type hints and documentation – Production-ready patterns

🎯 Best Use Cases: – Applications requiring this pattern’s characteristics – Systems needing structured agent behavior – Production environments requiring reliability

📚 Learning Resources: – LangChain Documentation: https://python.langchain.com/ – Pattern Documentation: See related patterns – OpenAI API Reference: https://platform.openai.com/docs/

---

File: 091_golden_dataset_testing.py
Total Lines: 938
Pattern Category: AI Agent Design Pattern #091

---

Generated from actual implementation code
For educational and reference purposes

    """
Pattern 090: Load Balancing Pattern

Description:
    The Load Balancing Pattern distributes requests across multiple LLM endpoints or
    instances to achieve high availability, improved performance, and fault tolerance.
    In production LLM applications, load balancing is critical for handling high traffic
    volumes, preventing single points of failure, and ensuring consistent service
    quality even when individual endpoints fail or become overloaded.

    Load balancing strategies enable:
    - High availability through redundancy
    - Improved performance via parallel processing
    - Cost optimization through provider selection
    - Rate limit management across multiple accounts
    - Geographic distribution for lower latency
    - Failover and disaster recovery

Components:
    1. Load Balancing Strategies
       - Round Robin: Distribute equally across endpoints
       - Weighted Round Robin: Priority-based distribution
       - Least Loaded: Send to endpoint with lowest load
       - Random: Random endpoint selection
       - Geographic: Route based on location
       - Cost-Based: Choose cheapest available endpoint

    2. Health Checking
       - Endpoint availability monitoring
       - Response time tracking
       - Error rate monitoring
       - Periodic health probes
       - Automatic endpoint removal/recovery

    3. Request Management
       - Request queuing and buffering
       - Retry logic with backoff
       - Circuit breaker pattern
       - Request timeout handling
       - Priority-based routing

    4. Failover Mechanisms
       - Automatic endpoint switching
       - Degraded mode operation
       - Fallback to alternative providers
       - Request replay on failure
       - State preservation

Use Cases:
    1. High-Traffic Applications
       - Chatbots serving thousands of users
       - API services with SLA requirements
       - Real-time processing pipelines
       - Multi-tenant platforms
       - Peak load handling

    2. Multi-Provider Deployments
       - OpenAI + Anthropic + Google
       - Primary and backup providers
       - Cost optimization strategies
       - Feature-specific routing
       - Geographic compliance

    3. Rate Limit Management
       - Distribute across multiple API keys
       - Prevent rate limit exhaustion
       - Smooth request distribution
       - Burst handling
       - Cost control

    4. Disaster Recovery
       - Automatic failover
       - Multi-region deployment
       - Zero-downtime updates
       - Service degradation handling
       - Business continuity

    5. Performance Optimization
       - Parallel request processing
       - Load-based routing
       - Latency optimization
       - Resource utilization
       - Capacity planning

LangChain Implementation:
    LangChain doesn't have built-in load balancing, but you can implement
    it using:
    - Custom LLM wrappers
    - Callback handlers for monitoring
    - Retry logic with different models
    - Custom routing logic
    - Health check mechanisms

Key Features:
    1. Endpoint Management
       - Dynamic endpoint registration
       - Health status tracking
       - Automatic recovery
       - Configuration updates
       - Version management

    2. Intelligent Routing
       - Strategy selection
       - Real-time decision making
       - Performance-based routing
       - Cost optimization
       - SLA enforcement

    3. Fault Tolerance
       - Automatic retry with exponential backoff
       - Circuit breaker to prevent cascading failures
       - Graceful degradation
       - Error handling and recovery
       - Request preservation

    4. Monitoring
       - Request distribution metrics
       - Endpoint health status
       - Response time tracking
       - Error rate monitoring
       - Cost tracking per endpoint

Best Practices:
    1. Health Monitoring
       - Regular health checks (every 30-60s)
       - Multiple health metrics (latency, errors, availability)
       - Automatic endpoint recovery
       - Alert on degraded performance
       - Log all health transitions

    2. Load Distribution
       - Consider endpoint capacity
       - Account for different model speeds
       - Balance cost vs performance
       - Avoid overloading any single endpoint
       - Reserve capacity for spikes

    3. Failover Strategy
       - Fast failure detection (< 5s)
       - Quick failover ( float:
        """Calculate success rate"""
        if self.total_requests == 0:
            return 0.0
        return (self.successful_requests / self.total_requests) * 100
    
    @property
    def average_response_time(self) -> float:
        """Calculate average response time"""
        if self.successful_requests == 0:
            return 0.0
        return self.total_response_time / self.successful_requests


@dataclass
class LoadBalancerStats:
    """Overall load balancer statistics"""
    total_requests: int = 0
    successful_requests: int = 0
    failed_requests: int = 0
    failovers: int = 0
    total_response_time: float = 0.0
    endpoint_metrics: Dict[str, EndpointMetrics] = field(default_factory=dict)


class LLMEndpoint:
    """
    Represents a single LLM endpoint with health tracking.
    """
    
    def __init__(self, config: EndpointConfig):
        """
        Initialize endpoint.
        
        Args:
            config: Endpoint configuration
        """
        self.config = config
        self.metrics = EndpointMetrics()
        self.llm: Optional[ChatOpenAI] = None
        self.last_health_check: Optional[datetime] = None
        self.request_times: deque = deque(maxlen=100)  # Track recent request times
        
        self._initialize_llm()
    
    def _initialize_llm(self):
        """Initialize the LLM client"""
        try:
            # Use provided API key or default from environment
            api_key = self.config.api_key or os.getenv("OPENAI_API_KEY")
            
            self.llm = ChatOpenAI(
                model=self.config.model,
                openai_api_key=api_key,
                timeout=self.config.timeout,
                temperature=0.7
            )
            self.metrics.status = EndpointStatus.HEALTHY
        except Exception as e:
            print(f"Error initializing endpoint {self.config.id}: {e}")
            self.metrics.status = EndpointStatus.UNHEALTHY
            self.metrics.last_error = str(e)
            self.metrics.last_error_time = datetime.now()
    
    def is_available(self) -> bool:
        """Check if endpoint is available for requests"""
        return (
            self.metrics.status in [EndpointStatus.HEALTHY, EndpointStatus.DEGRADED]
            and self.llm is not None
            and self.metrics.active_requests  EndpointStatus:
        """
        Perform health check on endpoint.
        
        Returns:
            Current health status
        """
        try:
            if self.llm is None:
                self._initialize_llm()
            
            # Simple health check - try a minimal request
            start_time = time.time()
            self.llm.invoke("test")
            response_time = time.time() - start_time
            
            # Update status based on response time and error rate
            if response_time  95:
                self.metrics.status = EndpointStatus.HEALTHY
            elif response_time  80:
                self.metrics.status = EndpointStatus.DEGRADED
            else:
                self.metrics.status = EndpointStatus.UNHEALTHY
            
            self.last_health_check = datetime.now()
            return self.metrics.status
            
        except Exception as e:
            self.metrics.status = EndpointStatus.UNHEALTHY
            self.metrics.last_error = str(e)
            self.metrics.last_error_time = datetime.now()
            return EndpointStatus.UNHEALTHY
    
    def generate(self, prompt: str) -> Dict[str, Any]:
        """
        Generate response using this endpoint.
        
        Args:
            prompt: Input prompt
            
        Returns:
            Response dictionary
        """
        self.metrics.total_requests += 1
        self.metrics.active_requests += 1
        
        try:
            start_time = time.time()
            response = self.llm.invoke(prompt)
            response_time = time.time() - start_time
            
            self.metrics.successful_requests += 1
            self.metrics.total_response_time += response_time
            self.metrics.active_requests -= 1
            self.request_times.append(response_time)
            
            return {
                "response": response.content,
                "endpoint_id": self.config.id,
                "response_time": response_time,
                "success": True
            }
            
        except Exception as e:
            self.metrics.failed_requests += 1
            self.metrics.active_requests -= 1
            self.metrics.last_error = str(e)
            self.metrics.last_error_time = datetime.now()
            
            # Mark as unhealthy after repeated failures
            if self.metrics.success_rate  Optional[LLMEndpoint]:
        """Select endpoint using round-robin strategy"""
        attempts = 0
        while attempts  Optional[LLMEndpoint]:
        """Select endpoint using weighted round-robin strategy"""
        # Create weighted list
        weighted_endpoints = []
        for endpoint in self.endpoints:
            if endpoint.is_available():
                weighted_endpoints.extend([endpoint] * endpoint.config.weight)
        
        if not weighted_endpoints:
            return None
        
        # Round-robin through weighted list
        endpoint = weighted_endpoints[self.current_endpoint_index % len(weighted_endpoints)]
        self.current_endpoint_index += 1
        return endpoint
    
    def _select_endpoint_least_loaded(self) -> Optional[LLMEndpoint]:
        """Select endpoint with least active requests"""
        available = [e for e in self.endpoints if e.is_available()]
        if not available:
            return None
        
        return min(available, key=lambda e: e.metrics.active_requests)
    
    def _select_endpoint_random(self) -> Optional[LLMEndpoint]:
        """Select random available endpoint"""
        available = [e for e in self.endpoints if e.is_available()]
        if not available:
            return None
        
        return random.choice(available)
    
    def _select_endpoint_fastest(self) -> Optional[LLMEndpoint]:
        """Select endpoint with fastest average response time"""
        available = [e for e in self.endpoints if e.is_available()]
        if not available:
            return None
        
        # Sort by average response time (endpoints with no history get high priority)
        return min(
            available,
            key=lambda e: e.metrics.average_response_time if e.metrics.successful_requests > 0 else 0.0
        )
    
    def _select_endpoint(self) -> Optional[LLMEndpoint]:
        """
        Select endpoint based on strategy.
        
        Returns:
            Selected endpoint or None if none available
        """
        if self.strategy == LoadBalancingStrategy.ROUND_ROBIN:
            return self._select_endpoint_round_robin()
        elif self.strategy == LoadBalancingStrategy.WEIGHTED_ROUND_ROBIN:
            return self._select_endpoint_weighted()
        elif self.strategy == LoadBalancingStrategy.LEAST_LOADED:
            return self._select_endpoint_least_loaded()
        elif self.strategy == LoadBalancingStrategy.RANDOM:
            return self._select_endpoint_random()
        elif self.strategy == LoadBalancingStrategy.FASTEST_RESPONSE:
            return self._select_endpoint_fastest()
        else:
            return self._select_endpoint_round_robin()
    
    def _perform_health_checks(self):
        """Perform health checks on all endpoints if needed"""
        now = datetime.now()
        if (now - self.last_health_check).seconds  Dict[str, Any]:
        """
        Generate response with load balancing and failover.
        
        Args:
            prompt: Input prompt
            max_retries: Maximum retry attempts
            
        Returns:
            Response dictionary
        """
        self.stats.total_requests += 1
        
        # Perform health checks if needed
        self._perform_health_checks()
        
        attempts = 0
        last_error = None
        
        while attempts  Dict[str, Any]:
        """Get load balancer statistics"""
        return {
            "total_requests": self.stats.total_requests,
            "successful_requests": self.stats.successful_requests,
            "failed_requests": self.stats.failed_requests,
            "failovers": self.stats.failovers,
            "success_rate": (
                (self.stats.successful_requests / self.stats.total_requests * 100)
                if self.stats.total_requests > 0 else 0.0
            ),
            "average_response_time": (
                self.stats.total_response_time / self.stats.successful_requests
                if self.stats.successful_requests > 0 else 0.0
            ),
            "endpoints": [
                {
                    "id": e.config.id,
                    "model": e.config.model,
                    "status": e.metrics.status.value,
                    "total_requests": e.metrics.total_requests,
                    "success_rate": f"{e.metrics.success_rate:.2f}%",
                    "avg_response_time": f"{e.metrics.average_response_time:.3f}s",
                    "active_requests": e.metrics.active_requests
                }
                for e in self.endpoints
            ]
        }


def demonstrate_load_balancing():
    """Demonstrate load balancing patterns"""
    print("=" * 80)
    print("LOAD BALANCING PATTERN DEMONSTRATION")
    print("=" * 80)
    
    # Example 1: Basic Round-Robin Load Balancing
    print("\n" + "=" * 80)
    print("EXAMPLE 1: Round-Robin Load Balancing")
    print("=" * 80)
    
    # Configure multiple endpoints (simulating multiple API keys or models)
    endpoints = [
        EndpointConfig(id="endpoint-1", model="gpt-3.5-turbo", weight=1),
        EndpointConfig(id="endpoint-2", model="gpt-3.5-turbo", weight=1),
        EndpointConfig(id="endpoint-3", model="gpt-3.5-turbo", weight=1),
    ]
    
    lb = LoadBalancer(endpoints, strategy=LoadBalancingStrategy.ROUND_ROBIN)
    
    print("\nConfigured 3 endpoints with round-robin strategy\n")
    print("Sending 6 requests to observe distribution:\n")
    
    queries = [
        "What is 2+2?",
        "What is Python?",
        "What is AI?",
        "What is ML?",
        "What is DL?",
        "What is NLP?"
    ]
    
    for i, query in enumerate(queries, 1):
        print(f"Request {i}: {query}")
        result = lb.generate(query)
        
        if result["success"]:
            print(f"  ✓ Endpoint: {result['endpoint_id']}")
            print(f"  Response time: {result['response_time']:.3f}s")
            print(f"  Response: {result['response'][:60]}...")
        else:
            print(f"  ✗ Error: {result['error']}")
        print()
    
    stats = lb.get_stats()
    print("Load Distribution:")
    for endpoint in stats["endpoints"]:
        print(f"  {endpoint['id']}: {endpoint['total_requests']} requests")
    
    # Example 2: Weighted Load Balancing
    print("\n" + "=" * 80)
    print("EXAMPLE 2: Weighted Load Balancing")
    print("=" * 80)
    
    weighted_endpoints = [
        EndpointConfig(id="premium-endpoint", model="gpt-4", weight=3),  # Gets more traffic
        EndpointConfig(id="standard-endpoint", model="gpt-3.5-turbo", weight=1),
    ]
    
    weighted_lb = LoadBalancer(
        weighted_endpoints,
        strategy=LoadBalancingStrategy.WEIGHTED_ROUND_ROBIN
    )
    
    print("\nConfigured 2 endpoints:")
    print("  - Premium (GPT-4): weight=3")
    print("  - Standard (GPT-3.5): weight=1")
    print("\nSending 8 requests:\n")
    
    for i in range(8):
        result = weighted_lb.generate(f"Request {i+1}")
        if result["success"]:
            print(f"Request {i+1} → {result['endpoint_id']}")
    
    print("\nExpected distribution: ~6 to premium, ~2 to standard")
    stats = weighted_lb.get_stats()
    for endpoint in stats["endpoints"]:
        print(f"  {endpoint['id']}: {endpoint['total_requests']} requests")
    
    # Example 3: Least-Loaded Strategy
    print("\n" + "=" * 80)
    print("EXAMPLE 3: Least-Loaded Strategy")
    print("=" * 80)
    
    ll_endpoints = [
        EndpointConfig(id="endpoint-A", model="gpt-3.5-turbo"),
        EndpointConfig(id="endpoint-B", model="gpt-3.5-turbo"),
    ]
    
    ll_lb = LoadBalancer(ll_endpoints, strategy=LoadBalancingStrategy.LEAST_LOADED)
    
    print("\nLeast-loaded strategy routes to endpoint with fewest active requests")
    print("This optimizes for load distribution in real-time\n")
    
    for i in range(4):
        result = ll_lb.generate(f"Query {i+1}")
        if result["success"]:
            print(f"Query {i+1} → {result['endpoint_id']}")
    
    # Example 4: Failover Handling
    print("\n" + "=" * 80)
    print("EXAMPLE 4: Automatic Failover")
    print("=" * 80)
    
    print("\nSimulating endpoint failure scenario:")
    print("  - 3 endpoints configured")
    print("  - Mark one as unhealthy")
    print("  - Load balancer automatically routes around it\n")
    
    failover_endpoints = [
        EndpointConfig(id="healthy-1", model="gpt-3.5-turbo"),
        EndpointConfig(id="unhealthy", model="gpt-3.5-turbo"),
        EndpointConfig(id="healthy-2", model="gpt-3.5-turbo"),
    ]
    
    fo_lb = LoadBalancer(failover_endpoints, strategy=LoadBalancingStrategy.ROUND_ROBIN)
    
    # Simulate endpoint failure
    fo_lb.endpoints[1].metrics.status = EndpointStatus.UNHEALTHY
    print("Marked 'unhealthy' endpoint as UNHEALTHY\n")
    
    print("Sending 4 requests:\n")
    for i in range(4):
        result = fo_lb.generate(f"Query {i+1}")
        if result["success"]:
            print(f"Query {i+1} → {result['endpoint_id']} ✓")
    
    print("\nOnly healthy endpoints received requests!")
    
    # Example 5: Performance Comparison
    print("\n" + "=" * 80)
    print("EXAMPLE 5: Strategy Performance Comparison")
    print("=" * 80)
    
    strategies = [
        LoadBalancingStrategy.ROUND_ROBIN,
        LoadBalancingStrategy.RANDOM,
        LoadBalancingStrategy.LEAST_LOADED,
    ]
    
    print("\nComparing different load balancing strategies:\n")
    
    for strategy in strategies:
        test_endpoints = [
            EndpointConfig(id=f"ep-{i}", model="gpt-3.5-turbo")
            for i in range(3)
        ]
        
        test_lb = LoadBalancer(test_endpoints, strategy=strategy)
        
        # Send some requests
        for i in range(6):
            test_lb.generate(f"Test query {i}")
        
        stats = test_lb.get_stats()
        print(f"{strategy.value.upper()}:")
        print(f"  Success Rate: {stats['success_rate']:.2f}%")
        print(f"  Total Requests: {stats['total_requests']}")
        for ep in stats["endpoints"]:
            print(f"    {ep['id']}: {ep['total_requests']} requests")
        print()
    
    # Example 6: Monitoring and Statistics
    print("\n" + "=" * 80)
    print("EXAMPLE 6: Comprehensive Statistics")
    print("=" * 80)
    
    monitor_endpoints = [
        EndpointConfig(id="prod-1", model="gpt-3.5-turbo"),
        EndpointConfig(id="prod-2", model="gpt-3.5-turbo"),
    ]
    
    monitor_lb = LoadBalancer(monitor_endpoints, strategy=LoadBalancingStrategy.ROUND_ROBIN)
    
    print("\nSimulating production workload...\n")
    
    # Simulate various scenarios
    for i in range(10):
        result = monitor_lb.generate(f"Production query {i+1}")
    
    stats = monitor_lb.get_stats()
    
    print("Load Balancer Statistics:")
    print(f"  Total Requests: {stats['total_requests']}")
    print(f"  Successful: {stats['successful_requests']}")
    print(f"  Failed: {stats['failed_requests']}")
    print(f"  Success Rate: {stats['success_rate']:.2f}%")
    print(f"  Failovers: {stats['failovers']}")
    print(f"  Avg Response Time: {stats['average_response_time']:.3f}s")
    
    print("\nPer-Endpoint Metrics:")
    for ep in stats["endpoints"]:
        print(f"\n  {ep['id']} ({ep['model']}):")
        print(f"    Status: {ep['status']}")
        print(f"    Requests: {ep['total_requests']}")
        print(f"    Success Rate: {ep['success_rate']}")
        print(f"    Avg Response Time: {ep['avg_response_time']}")
        print(f"    Active Requests: {ep['active_requests']}")
    
    # Summary
    print("\n" + "=" * 80)
    print("LOAD BALANCING PATTERN SUMMARY")
    print("=" * 80)
    print("""
Load Balancing Benefits:
1. High Availability: 99.9%+ uptime through redundancy
2. Performance: Better throughput via parallel processing
3. Fault Tolerance: Automatic failover on endpoint failure
4. Scalability: Easy to add more endpoints
5. Cost Optimization: Route based on cost/performance
6. Rate Limit Management: Distribute across API keys

Load Balancing Strategies:
1. Round Robin
   - Equal distribution across endpoints
   - Simple and predictable
   - Good for homogeneous endpoints
   - Use when: All endpoints have similar capacity

2. Weighted Round Robin
   - Priority-based distribution
   - Higher weight = more requests
   - Good for heterogeneous endpoints
   - Use when: Different endpoint capacities

3. Least Loaded
   - Route to least busy endpoint
   - Optimizes for current load
   - Dynamic adjustment
   - Use when: Variable request durations

4. Random
   - Random endpoint selection
   - Stateless and simple
   - Good load distribution over time
   - Use when: Simplicity preferred

5. Fastest Response
   - Route to historically fastest endpoint
   - Performance-optimized
   - Adaptive to endpoint characteristics
   - Use when: Latency is critical

Health Checking:
1. Active Health Checks
   - Regular test requests
   - Verify endpoint availability
   - Monitor response times
   - Interval: 30-60 seconds

2. Passive Health Monitoring
   - Track request success/failure
   - Monitor error rates
   - Calculate response times
   - Continuous monitoring

3. Status Levels
   - HEALTHY:  95% success
   - DEGRADED:  80% success
   - UNHEALTHY: Slow or 1000 req/hr)
✓ SLA requirements (>99% uptime)
✓ Rate limit concerns
✓ Multi-provider deployments
✓ Geographic distribution needed
✗ Low-traffic applications
✗ Single endpoint sufficient
✗ Cost constraints (added complexity)

ROI Analysis:
- Uptime improvement: 99% → 99.9% (10x fewer outages)
- Performance: 2-3x throughput capacity
- User satisfaction: Reduced error rates
- Cost: +20% complexity, but worth it for scale
- Break-even: ~500 requests/hour
""")
    
    print("\n" + "=" * 80)
    print("Pattern 090 (Load Balancing) demonstration complete!")
    print("=" * 80)


if __name__ == "__main__":
    demonstrate_load_balancing()

    

Agentic Design Pattern: Load Balancing

Pattern 090: Load Balancing – Line-by-Line Explanation

Overview

This document provides a detailed line-by-line explanation of the Load Balancing pattern implementation.

---

Pattern Description

Pattern 090: Load Balancing Pattern

Description: The Load Balancing Pattern distributes requests across multiple LLM endpoints or instances to achieve high availability, improved performance, and fault tolerance. In production LLM applications, load balancing is critical for handling high traffic volumes, preventing single points of failure, and ensuring consistent service quality even when individual endpoints fail or become overloaded.

Load balancing strategies enable:
- High availability through redundancy
- Improved performance via parallel processing
- Cost optimization through provider selection
- Rate limit management across multiple accounts
- Geographic distribution for lower latency
- Failover and disaster recovery

Components: 1. Load Balancing Strategies – Round Robin: Distribute equally across endpoints – Weighted Round Robin: Priority-based distribution – Least Loaded: Send to endpoint with lowest load – Random: Random endpoint selection – Geographic: Route based on location – Cost-Based: Choose cheapest available endpoint

2. Health Checking
   - Endpoint availability monitoring
   - Response time tracking
   - Error rate monitoring
   - Periodic health probes
   - Automatic endpoint removal/recovery

3. Request Management
   - Request queuing and buffering
   - Retry logic with backoff
   - Circuit breaker pattern
   - Request timeout handling
   - Priority-based routing

4. Failover Mechanisms
   - Automatic endpoint switching
   - Degraded mode operation
   - Fallback to alternative providers
   - Request replay on failure
   - State preservation

Use Cases: 1. High-Traffic Applications – Chatbots serving thousands of users – API services with SLA requirements – Real-time processing pipelines – Multi-tenant platforms – Peak load handling

2. Multi-Provider Deployments
   - OpenAI + Anthropic + Google
   - Primary and backup providers
   - Cost optimization strategies
   - Feature-specific routing
   - Geographic compliance

3. Rate Limit Management
   - Distribute across multiple API keys
   - Prevent rate limit exhaustion
   - Smooth request distribution
   - Burst handling
   - Cost control

4. Disaster Recovery
   - Automatic failover
   - Multi-region deployment
   - Zero-downtime updates
   - Service degradation handling
   - Business continuity

5. Performance Optimization
   - Parallel request processing
   - Load-based routing
   - Latency optimization
   - Resource utilization
   - Capacity planning

LangChain Implementation: LangChain doesn’t have built-in load balancing, but you can implement it using: – Custom LLM wrappers – Callback handlers for monitoring – Retry logic with different models – Custom routing logic – Health check mechanisms

Key Features: 1. Endpoint Management – Dynamic endpoint registration – Health status tracking – Automatic recovery – Configuration updates – Version management

2. Intelligent Routing
   - Strategy selection
   - Real-time decision making
   - Performance-based routing
   - Cost optimization
   - SLA enforcement

3. Fault Tolerance
   - Automatic retry with exponential backoff
   - Circuit breaker to prevent cascading failures
   - Graceful degradation
   - Error handling and recovery
   - Request preservation

4. Monitoring
   - Request distribution metrics
   - Endpoint health status
   - Response time tracking
   - Error rate monitoring
   - Cost tracking per endpoint

Best Practices: 1. Health Monitoring – Regular health checks (every 30-60s) – Multiple health metrics (latency, errors, availability) – Automatic endpoint recovery – Alert on degraded performance – Log all health transitions

2. Load Distribution
   - Consider endpoint capacity
   - Account for different model speeds
   - Balance cost vs performance
   - Avoid overloading any single endpoint
   - Reserve capacity for spikes

3. Failover Strategy
   - Fast failure detection (< 5s)
   - Quick failover (< 1s)
   - Preserve request context
   - Transparent to users
   - Log all failovers

4. Request Management
   - Set appropriate timeouts
   - Implement retry with backoff
   - Queue management for spikes
   - Priority handling
   - Request deduplication

Trade-offs: Advantages: – High availability (99.9%+ uptime) – Better performance through parallelization – Fault tolerance and resilience – Flexible scaling – Cost optimization opportunities – Geographic distribution

Disadvantages:
- Increased complexity
- Additional monitoring requirements
- Potential consistency issues
- Higher operational costs
- Configuration management overhead
- Testing complexity

Production Considerations: 1. Endpoint Configuration – Multiple providers (OpenAI, Anthropic, etc.) – Multiple API keys per provider – Geographic distribution – Capacity planning – Cost monitoring

2. Monitoring and Alerting
   - Real-time health dashboards
   - Performance metrics
   - Error tracking
   - Cost tracking
   - SLA compliance monitoring

3. Security
   - Secure credential management
   - Rate limiting per client
   - Access control
   - Audit logging
   - Encryption in transit

4. Scaling
   - Horizontal scaling capability
   - Auto-scaling based on load
   - Capacity planning
   - Performance testing
   - Load testing

5. Disaster Recovery
   - Multi-region deployment
   - Backup endpoints
   - Failover procedures
   - Data consistency
   - Recovery time objectives (RTO)

---

File Statistics

• Total Lines: 988
• Classes: 7
• Functions: 1
• Constants: 0

---

Module Documentation

"""
Pattern 090: Load Balancing Pattern

Description:
    The Load Balancing Pattern distributes requests across multiple LLM endpoints or
    instances to achieve high availability, improved performance, and fault tolerance.
    In production LLM applications, load balancing is critical for handling high traffic
    volumes, preventing single points of failure, and ensuring consistent service
    quality even when individual endpoints fail or become overloaded.

    Load balancing strategies enable:
    - High availability through redundancy
    - Improved performance via parallel processing
    - Cost optimization through provider selection
    - Rate limit management across multiple accounts
    - Geographic distribution for lower latency
    - Failover and disaster recovery

Components:
    1. Load Balancing Strategies
       - Round Robin: Distribute equally across endpoints
       - Weighted Round Robin: Priority-based distribution
       - Least Loaded: Send to endpoint with lowest load
       - Random: Random endpoint selection
       - Geographic: Route based on location
       - Cost-Based: Choose cheapest available endpoint

    2. Health Checking
       - Endpoint availability monitoring
       - Response time tracking
       - Error rate monitoring
       - Periodic health probes
       - Automatic endpoint removal/recovery

    3. Request Management
       - Request queuing and buffering
       - Retry logic with backoff
       - Circuit breaker pattern
       - Request timeout handling
       - Priority-based routing

    4. Failover Mechanisms
       - Automatic endpoint switching
       - Degraded mode operation
       - Fallback to alternative providers
       - Request replay on failure
       - State preservation

Use Cases:
    1. High-Traffic Applications
       - Chatbots serving thousands of users
       - API services with SLA requirements
       - Real-time processing pipelines
       - Multi-tenant platforms
       - Peak load handling

    2. Multi-Provider Deployments
       - OpenAI + Anthropic + Google
       - Primary and backup providers
       - Cost optimization strategies
       - Feature-specific routing
       - Geographic compliance

    3. Rate Limit Management
       - Distribute across multiple API keys
       - Prevent rate limit exhaustion
       - Smooth request distribution
       - Burst handling
       - Cost control

    4. Disaster Recovery
       - Automatic failover
       - Multi-region deployment
       - Zero-downtime updates
       - Service degradation handling
       - Business continuity

    5. Performance Optimization
       - Parallel request processing
       - Load-based routing
       - Latency optimization
       - Resource utilization
       - Capacity planning

LangChain Implementation:
    LangChain doesn't have built-in load balancing, but you can implement
    it using:
    - Custom LLM wrappers
    - Callback handlers for monitoring
    - Retry logic with different models
    - Custom routing logic
    - Health check mechanisms

Key Features:
    1. Endpoint Management
       - Dynamic endpoint registration
       - Health status tracking
       - Automatic recovery
       - Configuration updates
       - Version management

    2. Intelligent Routing
       - Strategy selection
       - Real-time decision making
       - Performance-based routing
       - Cost optimization
       - SLA enforcement

    3. Fault Tolerance
       - Automatic retry with exponential backoff
       - Circuit breaker to prevent cascading failures
       - Graceful degradation
       - Error handling and recovery
       - Request preservation

    4. Monitoring
       - Request distribution metrics
       - Endpoint health status
       - Response time tracking
       - Error rate monitoring
       - Cost tracking per endpoint

Best Practices:
    1. Health Monitoring
       - Regular health checks (every 30-60s)
       - Multiple health metrics (latency, errors, availability)
       - Automatic endpoint recovery
       - Alert on degraded performance
       - Log all health transitions

    2. Load Distribution
       - Consider endpoint capacity
       - Account for different model speeds
       - Balance cost vs performance
       - Avoid overloading any single endpoint
       - Reserve capacity for spikes

    3. Failover Strategy
       - Fast failure detection (< 5s)
       - Quick failover (< 1s)
       - Preserve request context
       - Transparent to users
       - Log all failovers

    4. Request Management
       - Set appropriate timeouts
       - Implement retry with backoff
       - Queue management for spikes
       - Priority handling
       - Request deduplication

Trade-offs:
    Advantages:
    - High availability (99.9%+ uptime)
    - Better performance through parallelization
    - Fault tolerance and resilience
    - Flexible scaling
    - Cost optimization opportunities
    - Geographic distribution

    Disadvantages:
    - Increased complexity
    - Additional monitoring requirements
    - Potential consistency issues
    - Higher operational costs
    - Configuration management overhead
    - Testing complexity

Production Considerations:
    1. Endpoint Configuration
       - Multiple providers (OpenAI, Anthropic, etc.)
       - Multiple API keys per provider
       - Geographic distribution
       - Capacity planning
       - Cost monitoring

    2. Monitoring and Alerting
       - Real-time health dashboards
       - Performance metrics
       - Error tracking
       - Cost tracking
       - SLA compliance monitoring

    3. Security
       - Secure credential management
       - Rate limiting per client
       - Access control
       - Audit logging
       - Encryption in transit

    4. Scaling
       - Horizontal scaling capability
       - Auto-scaling based on load
       - Capacity planning
       - Performance testing
       - Load testing

    5. Disaster Recovery
       - Multi-region deployment
       - Backup endpoints
       - Failover procedures
       - Data consistency
       - Recovery time objectives (RTO)
"""

Purpose: This module implements the Load Balancing pattern, providing a structured approach to agent design and implementation.

---

Import Statements

Standard Library Imports

import collections.deque
import dataclasses.dataclass
import dataclasses.field
import datetime.datetime
import datetime.timedelta
import enum.Enum
import langchain_core.prompts.ChatPromptTemplate
import os
import random
import time
import typing.Any
import typing.Callable
import typing.Dict
import typing.List
import typing.Optional

Purpose: Core Python functionality: – collections.deque: Specialized container datatypes – dataclasses.dataclass: Simplified class definitions with @dataclass – dataclasses.field: Simplified class definitions with @dataclass – datetime.datetime: Date and time handling – datetime.timedelta: Date and time handling – enum.Enum: Enumeration support – langchain_core.prompts.ChatPromptTemplate: Regular expression operations – os: Environment variable access and system operations – random: Random number generation – time: Time-related functions – typing.Any: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Callable: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Dict: Type hints for better code documentation (List, Dict, Any, Optional) – typing.List: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Optional: Type hints for better code documentation (List, Dict, Any, Optional)

LangChain Imports

from langchain_openai.ChatOpenAI

Purpose: LangChain framework components: – LLM Interface: Wrapper for OpenAI models – Chains: Composable building blocks for complex workflows

Other Imports

from dotenv.load_dotenv

Purpose: Additional dependencies for specialized functionality.

---

Classes

Class: LoadBalancingStrategy

Lines 222-228

Inherits from: Enum

Purpose:

Load balancing strategies

---

Class: EndpointStatus

Lines 231-236

Inherits from: Enum

Purpose:

Endpoint health status

---

Class: EndpointConfig

Lines 240-248

Purpose:

Configuration for an LLM endpoint

---

Class: EndpointMetrics

Lines 252-275

Purpose:

Metrics for an endpoint

Methods:

success_rate(self)

Lines 264-268

Decorators: property

Purpose:

Calculate success rate

Parameters:

Returns: float

average_response_time(self)

Lines 271-275

Decorators: property

Purpose:

Calculate average response time

Parameters:

Returns: float

---

Class: LoadBalancerStats

Lines 279-286

Purpose:

Overall load balancer statistics

---

Class: LLMEndpoint

Lines 289-414

Purpose:

Represents a single LLM endpoint with health tracking.

Methods:

__init__(self, config)

Lines 294-307

Purpose:

Initialize endpoint.

Args:
    config: Endpoint configuration

Parameters: – config: Config

_initialize_llm(self)

Lines 309-326

Purpose:

Initialize the LLM client

Parameters:

is_available(self)

Lines 328-334

Purpose:

Check if endpoint is available for requests

Parameters:

Returns: bool

check_health(self)

Lines 336-367

Purpose:

Perform health check on endpoint.

Returns:
    Current health status

Parameters:

Returns: EndpointStatus

generate(self, prompt)

Lines 369-414

Purpose:

Generate response using this endpoint.

Args:
    prompt: Input prompt

Returns:
    Response dictionary

Parameters: – prompt: Prompt template or text

Returns: <ast.Subscript object at 0x000001A06D7523D0>

---

Class: LoadBalancer

Lines 417-622

Purpose:

Load balancer for distributing requests across multiple LLM endpoints.

Supports multiple load balancing strategies and automatic failover.

Methods:

__init__(self, endpoints, strategy, health_check_interval)

Lines 424-449

Purpose:

Initialize load balancer.

Args:
    endpoints: List of endpoint configurations
    strategy: Load balancing strategy
    health_check_interval: Seconds between health checks

Parameters: – endpoints: Endpoints – strategy: Strategy – health_check_interval: Health check interval

_select_endpoint_round_robin(self)

Lines 451-463

Purpose:

Select endpoint using round-robin strategy

Parameters:

Returns: <ast.Subscript object at 0x000001A06D8881D0>

_select_endpoint_weighted(self)

Lines 465-479

Purpose:

Select endpoint using weighted round-robin strategy

Parameters:

Returns: <ast.Subscript object at 0x000001A06D88B410>

_select_endpoint_least_loaded(self)

Lines 481-487

Purpose:

Select endpoint with least active requests

Parameters:

Returns: <ast.Subscript object at 0x000001A06D75B250>

_select_endpoint_random(self)

Lines 489-495

Purpose:

Select random available endpoint

Parameters:

Returns: <ast.Subscript object at 0x000001A06D75A790>

_select_endpoint_fastest(self)

Lines 497-507

Purpose:

Select endpoint with fastest average response time

Parameters:

Returns: <ast.Subscript object at 0x000001A06D80CF90>

_select_endpoint(self)

Lines 509-527

Purpose:

Select endpoint based on strategy.

Returns:
    Selected endpoint or None if none available

Parameters:

Returns: <ast.Subscript object at 0x000001A06D157C90>

_perform_health_checks(self)

Lines 529-538

Purpose:

Perform health checks on all endpoints if needed

Parameters:

generate(self, prompt, max_retries)

Lines 540-593

Purpose:

Generate response with load balancing and failover.

Args:
    prompt: Input prompt
    max_retries: Maximum retry attempts

Returns:
    Response dictionary

Parameters: – prompt: Prompt template or text – max_retries: Max retries

Returns: <ast.Subscript object at 0x000001A06D83E910>

get_stats(self)

Lines 595-622

Purpose:

Get load balancer statistics

Parameters:

Returns: <ast.Subscript object at 0x000001A06D850690>

---

Functions

demonstrate_load_balancing()

Lines 625-983

Purpose:

Demonstrate load balancing patterns

---

Key Concepts Summary

Pattern Overview

The Load Balancing pattern provides a structured approach to building AI agents with specific capabilities and behaviors. This implementation leverages LangChain’s composable components to create a robust and maintainable solution.

Core Components

1. Agent Class: Main implementation of the pattern logic
2. Prompt Engineering: Carefully crafted prompts to guide LLM behavior
3. Tool Integration: Optional tools for extended functionality
4. Memory Management: State handling across interactions
5. Output Parsing: Structured response processing

When to Use

✅ Good for: – Scenarios requiring this specific pattern’s strengths – Applications needing structured agent behavior – Systems where explainability is important – Production environments requiring reliability

❌ Not ideal for: – Simple single-prompt tasks – Scenarios requiring different pattern characteristics – Resource-constrained environments (if computationally intensive)

---

Best Practices Demonstrated

1. Environment Configuration

• ✅ Use .env files for API keys
• ✅ Never hardcode credentials
• ✅ Validate environment variables on startup

2. Error Handling

• ✅ Graceful degradation on failures
• ✅ Informative error messages
• ✅ Logging for debugging

3. Prompt Engineering

• ✅ Clear, specific instructions
• ✅ Structured output formats
• ✅ Few-shot examples when beneficial

4. Code Organization

• ✅ Separation of concerns (class-based design)
• ✅ Type hints for clarity
• ✅ Docstrings for documentation

5. Performance Optimization

• ✅ Appropriate temperature settings
• ✅ Token budget management
• ✅ Caching where applicable

---

LangChain Components Used

• ChatOpenAI: OpenAI model interface
• Chains: Sequential component composition

Why LangChain? – Composable abstractions for LLM applications – Rich ecosystem of integrations – Production-ready patterns – Active community and development

---

Execution Flow

1. Initialize agent with configuration
   ↓
2. Load environment and dependencies
   ↓
3. Prepare prompt with input data
   ↓
4. Invoke LLM through LangChain
   ↓
5. Parse and validate response
   ↓
6. Return formatted result

Key Decision Points: – Model selection (temperature, model type) – Prompt structure (few-shot vs zero-shot) – Error handling strategy – Output format validation

---

Performance Considerations

Computational Cost

• LLM Calls: Primary cost factor (based on tokens)
• Latency: Depends on model and input size
• Memory: Minimal for most patterns

Optimization Strategies

1. Prompt Optimization: Minimize tokens while maintaining clarity
2. Caching: Cache repeated queries/responses
3. Model Selection: Use smaller models when appropriate
4. Batch Processing: Group similar requests
5. Streaming: For long responses, stream tokens

Cost Estimates (Approximate)

• GPT-3.5-turbo: ~$0.002 per request
• GPT-4: ~$0.03 per request
• GPT-4-turbo: ~$0.01 per request

Actual costs vary based on input/output token counts

---

Common Pitfalls to Avoid

1. Prompt Design Issues

❌ Vague or ambiguous instructions
✅ Clear, specific, testable prompts

2. Error Handling

❌ Ignoring API errors or rate limits
✅ Implement retry logic and graceful degradation

3. Token Management

❌ Exceeding model token limits
✅ Truncate inputs, chunk processing

4. Security

❌ Exposing API keys in code
✅ Use environment variables

5. Testing

❌ Only testing happy paths
✅ Test edge cases, failures, malformed inputs

6. Cost Management

❌ Unmonitored API usage
✅ Implement budgets, alerts, and monitoring

---

Further Improvements

Production Enhancements

1. Monitoring: Add logging, metrics, tracing
2. Validation: Input/output schema validation
3. Rate Limiting: Implement request throttling
4. Caching: Redis or in-memory cache for responses
5. Async Support: Non-blocking operations for scalability

Feature Extensions

1. Multi-Model Support: Switch between providers
2. Fallback Models: Graceful degradation strategy
3. Custom Tools: Domain-specific function calling
4. Advanced Memory: Vector stores for long-term memory
5. Evaluation: Automated testing and benchmarking

Code Quality

1. Testing: Unit tests, integration tests
2. Documentation: API docs, usage examples
3. Type Safety: Stricter type hints
4. Configuration: Externalized config files

---

Related Patterns

• Pattern 089
• Pattern 091

Pattern Families: – Reasoning Patterns (001-011) – Multi-Agent Patterns (012-019) – Tool Use Patterns (020-025) – Memory Patterns (026-031) – And many more…

---

Summary

The Load Balancing pattern provides a powerful approach to building AI agents with specific capabilities. This implementation demonstrates:

✅ Key Strengths: – Clean, maintainable code structure – LangChain integration for composability – Type hints and documentation – Production-ready patterns

🎯 Best Use Cases: – Applications requiring this pattern’s characteristics – Systems needing structured agent behavior – Production environments requiring reliability

📚 Learning Resources: – LangChain Documentation: https://python.langchain.com/ – Pattern Documentation: See related patterns – OpenAI API Reference: https://platform.openai.com/docs/

---

File: 090_load_balancing.py
Total Lines: 988
Pattern Category: AI Agent Design Pattern #090

---

Generated from actual implementation code
For educational and reference purposes

    """
Pattern 089: Caching Patterns

Description:
    The Caching Patterns involve storing and reusing previously computed results to
    improve performance, reduce latency, and minimize costs. In the context of LLM
    applications, caching is critical for optimizing response times and reducing API
    calls for repeated or similar queries. Effective caching strategies can
    dramatically improve user experience while significantly cutting operational costs.

    Caching in LLM applications targets:
    - Complete responses (exact match caching)
    - Embeddings (vector representations)
    - Prompt templates (reusable structures)
    - Intermediate results (multi-step workflows)
    - API responses (rate limit optimization)

Components:
    1. Cache Types
       - In-memory cache (fast, volatile)
       - Disk cache (persistent, slower)
       - Distributed cache (Redis, Memcached)
       - Database cache (structured, queryable)
       - CDN cache (geographic distribution)

    2. Cache Keys
       - Exact match (full prompt hash)
       - Semantic match (embedding similarity)
       - Parametric keys (template + parameters)
       - Hierarchical keys (nested cache structures)
       - Time-based keys (temporal validity)

    3. Cache Policies
       - TTL (Time To Live) - expire after time
       - LRU (Least Recently Used) - evict old entries
       - LFU (Least Frequently Used) - evict unpopular
       - Size-based - limit total cache size
       - Custom policies - application-specific

    4. Invalidation Strategies
       - Time-based expiration
       - Event-triggered invalidation
       - Manual invalidation
       - Cascade invalidation (dependencies)
       - Version-based invalidation

Use Cases:
    1. Repeated Queries
       - FAQ responses
       - Common questions
       - Popular content
       - Standard procedures
       - Frequently accessed data

    2. Expensive Operations
       - Complex reasoning chains
       - Multi-step workflows
       - Large context processing
       - Batch operations
       - Resource-intensive computations

    3. Rate Limit Optimization
       - Reduce API calls
       - Stay within quotas
       - Handle traffic spikes
       - Cost optimization
       - Performance consistency

    4. Embeddings and Vectors
       - Document embeddings
       - Query embeddings
       - Semantic search optimization
       - Similarity computations
       - Vector database queries

    5. Multi-User Applications
       - Shared responses
       - Common workflows
       - Popular queries
       - Template responses
       - Collaborative features

LangChain Implementation:
    LangChain supports caching through:
    - InMemoryCache for simple caching
    - SQLiteCache for persistent caching
    - Custom cache implementations
    - Semantic similarity caching
    - Embedding caching

Key Features:
    1. Multiple Cache Layers
       - L1: In-memory (fastest)
       - L2: Local disk (persistent)
       - L3: Distributed cache (shared)
       - L4: Database (long-term)

    2. Intelligent Matching
       - Exact match for identical queries
       - Semantic match for similar queries
       - Fuzzy match with thresholds
       - Parametric templates

    3. Performance Optimization
       - Fast lookup ( 70%)
       - Measure latency improvements
       - Calculate cost savings
       - Identify cache misses patterns

    4. Handle Cache Misses
       - Graceful fallback to computation
       - Transparent to users
       - Background cache warming
       - Predictive pre-caching

Trade-offs:
    Advantages:
    - Dramatically improved response times
    - Significant cost reduction
    - Better user experience
    - Reduced API rate limiting
    - Lower resource consumption
    - Improved scalability

    Disadvantages:
    - Stale data risk
    - Memory/storage overhead
    - Cache consistency challenges
    - Complexity in invalidation
    - Initial cache warming time
    - Debugging complexity

Production Considerations:
    1. Cache Size Management
       - Monitor memory usage
       - Implement eviction policies
       - Set size limits
       - Regular cleanup
       - Compression strategies

    2. Cache Consistency
       - Version tracking
       - Update propagation
       - Invalidation timing
       - Distributed consistency
       - Race condition handling

    3. Security
       - Access control for cached data
       - Encryption for sensitive content
       - Cache poisoning prevention
       - User isolation in multi-tenant
       - Audit trails

    4. Monitoring
       - Hit/miss rates
       - Cache size trends
       - Eviction frequency
       - Cost savings
       - Performance impact

    5. Disaster Recovery
       - Cache rebuild procedures
       - Backup strategies
       - Failover mechanisms
       - Data loss handling
       - Recovery time objectives
"""

import os
import time
import hashlib
import json
from typing import List, Dict, Any, Optional, Callable
from dataclasses import dataclass, field
from enum import Enum
from datetime import datetime, timedelta
from collections import OrderedDict
from dotenv import load_dotenv

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_openai import ChatOpenAI, OpenAIEmbeddings
from langchain_core.caches import InMemoryCache
from langchain.globals import set_llm_cache

load_dotenv()


class CachePolicy(Enum):
    """Cache eviction policies"""
    LRU = "lru"  # Least Recently Used
    LFU = "lfu"  # Least Frequently Used
    FIFO = "fifo"  # First In First Out
    TTL = "ttl"  # Time To Live


@dataclass
class CacheEntry:
    """Represents a cached entry"""
    key: str
    value: Any
    created_at: datetime
    last_accessed: datetime
    access_count: int = 0
    ttl: Optional[int] = None  # TTL in seconds
    
    def is_expired(self) -> bool:
        """Check if entry is expired"""
        if self.ttl is None:
            return False
        age = (datetime.now() - self.created_at).total_seconds()
        return age > self.ttl
    
    def touch(self):
        """Update access information"""
        self.last_accessed = datetime.now()
        self.access_count += 1


@dataclass
class CacheStats:
    """Cache performance statistics"""
    hits: int = 0
    misses: int = 0
    evictions: int = 0
    total_requests: int = 0
    
    @property
    def hit_rate(self) -> float:
        """Calculate cache hit rate"""
        if self.total_requests == 0:
            return 0.0
        return (self.hits / self.total_requests) * 100
    
    @property
    def miss_rate(self) -> float:
        """Calculate cache miss rate"""
        return 100.0 - self.hit_rate


class SimpleCache:
    """
    Simple in-memory cache with multiple eviction policies.
    
    This cache supports:
    1. Multiple eviction policies (LRU, LFU, FIFO, TTL)
    2. Size limits
    3. TTL per entry
    4. Performance statistics
    """
    
    def __init__(
        self,
        max_size: int = 1000,
        policy: CachePolicy = CachePolicy.LRU,
        default_ttl: Optional[int] = None
    ):
        """
        Initialize cache.
        
        Args:
            max_size: Maximum number of entries
            policy: Eviction policy
            default_ttl: Default TTL in seconds
        """
        self.max_size = max_size
        self.policy = policy
        self.default_ttl = default_ttl
        self.cache: OrderedDict[str, CacheEntry] = OrderedDict()
        self.stats = CacheStats()
    
    def _make_key(self, key: Any) -> str:
        """Create cache key from input"""
        if isinstance(key, str):
            return hashlib.md5(key.encode()).hexdigest()
        return hashlib.md5(json.dumps(key, sort_keys=True).encode()).hexdigest()
    
    def _evict(self):
        """Evict entries based on policy"""
        if len(self.cache)  Optional[Any]:
        """
        Get value from cache.
        
        Args:
            key: Cache key
            
        Returns:
            Cached value or None
        """
        self.stats.total_requests += 1
        cache_key = self._make_key(key)
        
        # Remove expired entries
        self._remove_expired()
        
        if cache_key in self.cache:
            entry = self.cache[cache_key]
            
            if entry.is_expired():
                del self.cache[cache_key]
                self.stats.misses += 1
                return None
            
            entry.touch()
            
            # Move to end for LRU
            if self.policy == CachePolicy.LRU:
                self.cache.move_to_end(cache_key)
            
            self.stats.hits += 1
            return entry.value
        
        self.stats.misses += 1
        return None
    
    def set(self, key: Any, value: Any, ttl: Optional[int] = None):
        """
        Set value in cache.
        
        Args:
            key: Cache key
            value: Value to cache
            ttl: Optional TTL in seconds
        """
        cache_key = self._make_key(key)
        
        # Evict if necessary
        if cache_key not in self.cache:
            self._evict()
        
        entry = CacheEntry(
            key=cache_key,
            value=value,
            created_at=datetime.now(),
            last_accessed=datetime.now(),
            ttl=ttl or self.default_ttl
        )
        
        self.cache[cache_key] = entry
        
        # Move to end for LRU
        if self.policy == CachePolicy.LRU:
            self.cache.move_to_end(cache_key)
    
    def clear(self):
        """Clear all cache entries"""
        self.cache.clear()
        self.stats = CacheStats()
    
    def size(self) -> int:
        """Get current cache size"""
        return len(self.cache)
    
    def get_stats(self) -> Dict[str, Any]:
        """Get cache statistics"""
        return {
            "hits": self.stats.hits,
            "misses": self.stats.misses,
            "evictions": self.stats.evictions,
            "total_requests": self.stats.total_requests,
            "hit_rate": f"{self.stats.hit_rate:.2f}%",
            "miss_rate": f"{self.stats.miss_rate:.2f}%",
            "current_size": len(self.cache),
            "max_size": self.max_size
        }


class SemanticCache:
    """
    Semantic cache using embeddings for similarity matching.
    
    This cache finds similar queries using vector similarity,
    allowing cache hits for semantically similar but not identical queries.
    """
    
    def __init__(
        self,
        similarity_threshold: float = 0.95,
        max_size: int = 500
    ):
        """
        Initialize semantic cache.
        
        Args:
            similarity_threshold: Minimum similarity for cache hit
            max_size: Maximum cache entries
        """
        self.similarity_threshold = similarity_threshold
        self.max_size = max_size
        self.embeddings = OpenAIEmbeddings()
        self.cache: List[Dict[str, Any]] = []
        self.stats = CacheStats()
    
    def _compute_similarity(self, query1: str, query2: str) -> float:
        """
        Compute semantic similarity between queries.
        
        Args:
            query1: First query
            query2: Second query
            
        Returns:
            Similarity score (0-1)
        """
        # Simple implementation - in production use precomputed embeddings
        # This is for demonstration only
        query1_lower = query1.lower()
        query2_lower = query2.lower()
        
        # Simple word overlap similarity
        words1 = set(query1_lower.split())
        words2 = set(query2_lower.split())
        
        if not words1 or not words2:
            return 0.0
        
        intersection = words1.intersection(words2)
        union = words1.union(words2)
        
        return len(intersection) / len(union)
    
    def get(self, query: str) -> Optional[Any]:
        """
        Get value for semantically similar query.
        
        Args:
            query: Query string
            
        Returns:
            Cached value or None
        """
        self.stats.total_requests += 1
        
        # Find most similar cached query
        best_match = None
        best_similarity = 0.0
        
        for entry in self.cache:
            similarity = self._compute_similarity(query, entry["query"])
            if similarity > best_similarity:
                best_similarity = similarity
                best_match = entry
        
        if best_match and best_similarity >= self.similarity_threshold:
            self.stats.hits += 1
            # Update access time
            best_match["last_accessed"] = datetime.now()
            return best_match["value"]
        
        self.stats.misses += 1
        return None
    
    def set(self, query: str, value: Any):
        """
        Cache value for query.
        
        Args:
            query: Query string
            value: Value to cache
        """
        # Evict oldest if at capacity
        if len(self.cache) >= self.max_size:
            self.cache.pop(0)
            self.stats.evictions += 1
        
        self.cache.append({
            "query": query,
            "value": value,
            "created_at": datetime.now(),
            "last_accessed": datetime.now()
        })
    
    def clear(self):
        """Clear cache"""
        self.cache.clear()
        self.stats = CacheStats()
    
    def get_stats(self) -> Dict[str, Any]:
        """Get cache statistics"""
        return {
            "hits": self.stats.hits,
            "misses": self.stats.misses,
            "evictions": self.stats.evictions,
            "total_requests": self.stats.total_requests,
            "hit_rate": f"{self.stats.hit_rate:.2f}%",
            "current_size": len(self.cache),
            "max_size": self.max_size,
            "similarity_threshold": self.similarity_threshold
        }


class CachedLLMAgent:
    """
    LLM agent with caching support.
    
    This agent automatically caches responses and retrieves them
    for repeated or similar queries.
    """
    
    def __init__(
        self,
        cache: SimpleCache,
        model: str = "gpt-3.5-turbo"
    ):
        """
        Initialize cached agent.
        
        Args:
            cache: Cache instance to use
            model: LLM model name
        """
        self.cache = cache
        self.llm = ChatOpenAI(model=model, temperature=0.7)
    
    def generate(self, prompt: str) -> Dict[str, Any]:
        """
        Generate response with caching.
        
        Args:
            prompt: Input prompt
            
        Returns:
            Dictionary with response and cache info
        """
        # Check cache first
        cached_response = self.cache.get(prompt)
        
        if cached_response is not None:
            return {
                "response": cached_response,
                "from_cache": True,
                "cache_stats": self.cache.get_stats()
            }
        
        # Generate new response
        start_time = time.time()
        response = self.llm.invoke(prompt)
        generation_time = time.time() - start_time
        
        # Cache the response
        self.cache.set(prompt, response.content)
        
        return {
            "response": response.content,
            "from_cache": False,
            "generation_time": generation_time,
            "cache_stats": self.cache.get_stats()
        }


def demonstrate_caching_patterns():
    """Demonstrate caching patterns"""
    print("=" * 80)
    print("CACHING PATTERNS DEMONSTRATION")
    print("=" * 80)
    
    # Example 1: Basic LRU Cache
    print("\n" + "=" * 80)
    print("EXAMPLE 1: Basic LRU Cache")
    print("=" * 80)
    
    cache = SimpleCache(max_size=5, policy=CachePolicy.LRU)
    agent = CachedLLMAgent(cache)
    
    print("\nMaking requests (first time - cache misses):\n")
    
    queries = [
        "What is Python?",
        "What is JavaScript?",
        "What is Python?",  # Repeat
        "What is Java?",
        "What is Python?",  # Repeat again
    ]
    
    for i, query in enumerate(queries, 1):
        print(f"Request {i}: {query}")
        result = agent.generate(query)
        
        cache_status = "HIT ✓" if result["from_cache"] else "MISS ✗"
        print(f"  Status: {cache_status}")
        
        if not result["from_cache"]:
            print(f"  Generation time: {result['generation_time']:.2f}s")
        print(f"  Response: {result['response'][:80]}...")
        print()
    
    stats = cache.get_stats()
    print("Cache Statistics:")
    print(f"  Hit Rate: {stats['hit_rate']}")
    print(f"  Total Requests: {stats['total_requests']}")
    print(f"  Hits: {stats['hits']}")
    print(f"  Misses: {stats['misses']}")
    
    # Example 2: TTL-Based Caching
    print("\n" + "=" * 80)
    print("EXAMPLE 2: TTL-Based Caching (Time Expiration)")
    print("=" * 80)
    
    ttl_cache = SimpleCache(max_size=100, default_ttl=5)  # 5 second TTL
    
    print("\nCaching value with 5-second TTL...")
    ttl_cache.set("test_key", "test_value", ttl=5)
    
    print("Immediate retrieval:")
    value = ttl_cache.get("test_key")
    print(f"  Result: {'HIT' if value else 'MISS'} - {value}")
    
    print("\nWaiting 6 seconds for expiration...")
    time.sleep(6)
    
    print("Retrieval after TTL:")
    value = ttl_cache.get("test_key")
    print(f"  Result: {'HIT' if value else 'MISS'} - {value}")
    
    # Example 3: Cache Eviction Policies
    print("\n" + "=" * 80)
    print("EXAMPLE 3: Cache Eviction Policies Comparison")
    print("=" * 80)
    
    policies = [CachePolicy.LRU, CachePolicy.LFU, CachePolicy.FIFO]
    
    for policy in policies:
        print(f"\n{policy.value.upper()} Policy:")
        test_cache = SimpleCache(max_size=3, policy=policy)
        
        # Add 5 items to cache of size 3
        for i in range(5):
            key = f"item_{i}"
            test_cache.set(key, f"value_{i}")
            print(f"  Added {key}, cache size: {test_cache.size()}")
        
        stats = test_cache.get_stats()
        print(f"  Evictions: {stats['evictions']}")
    
    # Example 4: Semantic Cache
    print("\n" + "=" * 80)
    print("EXAMPLE 4: Semantic Cache (Similar Query Matching)")
    print("=" * 80)
    
    semantic_cache = SemanticCache(similarity_threshold=0.7, max_size=100)
    
    # Cache a response
    original_query = "What is machine learning?"
    original_response = "Machine learning is a subset of AI that enables systems to learn from data."
    semantic_cache.set(original_query, original_response)
    
    print(f"\nCached query: '{original_query}'")
    print(f"Response: {original_response}\n")
    
    # Try similar queries
    similar_queries = [
        "What is machine learning?",  # Exact match
        "What's machine learning?",   # Slight variation
        "Tell me about machine learning",  # Similar
        "Explain artificial intelligence",  # Different topic
    ]
    
    for query in similar_queries:
        result = semantic_cache.get(query)
        status = "HIT ✓" if result else "MISS ✗"
        print(f"Query: '{query}'")
        print(f"  Status: {status}")
        if result:
            print(f"  Retrieved: {result}")
        print()
    
    # Example 5: Multi-Layer Cache
    print("\n" + "=" * 80)
    print("EXAMPLE 5: Multi-Layer Cache Strategy")
    print("=" * 80)
    
    # L1: Small, fast, in-memory cache
    l1_cache = SimpleCache(max_size=10, policy=CachePolicy.LRU)
    
    # L2: Larger, still in-memory
    l2_cache = SimpleCache(max_size=100, policy=CachePolicy.LRU)
    
    print("\nMulti-layer cache simulation:")
    print("  L1: Small (10 items), fast")
    print("  L2: Large (100 items), moderate\n")
    
    def multi_layer_get(key: str) -> Optional[str]:
        """Get from multi-layer cache"""
        # Try L1 first
        value = l1_cache.get(key)
        if value:
            print(f"  L1 HIT for '{key}'")
            return value
        
        # Try L2
        value = l2_cache.get(key)
        if value:
            print(f"  L2 HIT for '{key}'")
            # Promote to L1
            l1_cache.set(key, value)
            return value
        
        print(f"  MISS for '{key}'")
        return None
    
    # Simulate access pattern
    l1_cache.set("frequently_accessed", "hot_data")
    l2_cache.set("occasionally_accessed", "warm_data")
    
    print("Access patterns:")
    multi_layer_get("frequently_accessed")
    multi_layer_get("occasionally_accessed")
    multi_layer_get("never_cached")
    
    # Example 6: Cache Performance Metrics
    print("\n" + "=" * 80)
    print("EXAMPLE 6: Cache Performance Analysis")
    print("=" * 80)
    
    perf_cache = SimpleCache(max_size=50, policy=CachePolicy.LRU)
    perf_agent = CachedLLMAgent(perf_cache)
    
    print("\nSimulating workload with repeated queries...\n")
    
    workload = [
        "What is AI?",
        "What is ML?",
        "What is AI?",  # Repeat
        "What is DL?",
        "What is AI?",  # Repeat
        "What is ML?",  # Repeat
        "What is NLP?",
        "What is AI?",  # Repeat
    ]
    
    total_time_with_cache = 0
    total_time_without_cache = 0
    
    for query in workload:
        result = perf_agent.generate(query)
        
        if result["from_cache"]:
            total_time_with_cache += 0.001  # Assume 1ms for cache hit
        else:
            gen_time = result.get("generation_time", 1.0)
            total_time_with_cache += gen_time
            total_time_without_cache += gen_time
    
    # Estimate time without cache (all queries would need generation)
    total_time_without_cache = len(workload) * 1.0  # Assume 1s per generation
    
    print(f"Performance Analysis:")
    print(f"  Total Queries: {len(workload)}")
    print(f"  Time WITH cache: {total_time_with_cache:.2f}s")
    print(f"  Time WITHOUT cache (estimated): {total_time_without_cache:.2f}s")
    print(f"  Speedup: {total_time_without_cache/total_time_with_cache:.1f}x")
    print(f"  Time Saved: {total_time_without_cache - total_time_with_cache:.2f}s")
    
    final_stats = perf_cache.get_stats()
    print(f"\nFinal Cache Statistics:")
    print(f"  Hit Rate: {final_stats['hit_rate']}")
    print(f"  Hits: {final_stats['hits']}")
    print(f"  Misses: {final_stats['misses']}")
    
    # Summary
    print("\n" + "=" * 80)
    print("CACHING PATTERNS SUMMARY")
    print("=" * 80)
    print("""
Caching Pattern Benefits:
1. Performance: Dramatically faster response times (10-1000x)
2. Cost Reduction: Fewer API calls = lower costs
3. Scalability: Handle more requests with same resources
4. Reliability: Reduce dependency on external services
5. User Experience: Instant responses for cached queries

Cache Types Demonstrated:
1. Simple Cache: Basic in-memory caching with eviction policies
2. TTL Cache: Time-based expiration for freshness
3. Semantic Cache: Similarity-based matching for related queries
4. Multi-Layer Cache: Hierarchical caching strategy
5. Performance-Optimized: Focus on hit rates and speedup

Eviction Policies:
1. LRU (Least Recently Used): Remove oldest accessed items
2. LFU (Least Frequently Used): Remove least popular items
3. FIFO (First In First Out): Remove oldest items
4. TTL (Time To Live): Expire after time period

Best Practices:
1. Cache Strategy Selection:
   - Exact match: Simple, fast, for identical queries
   - Semantic: Better coverage, for similar queries
   - TTL: For time-sensitive data
   - Multi-layer: For diverse access patterns

2. TTL Configuration:
   - Short (minutes): Dynamic, frequently changing data
   - Medium (hours): Semi-static content
   - Long (days): Stable, rarely changing data
   - No TTL: Static reference data

3. Size Management:
   - Monitor memory usage
   - Set appropriate limits
   - Implement eviction policies
   - Regular cleanup

4. Performance Monitoring:
   - Target >70% hit rate for good performance
   - Track cache size trends
   - Monitor eviction frequency
   - Measure latency improvements

When to Use Caching:
✓ Repeated identical queries
✓ Similar semantic queries
✓ Expensive computations
✓ Rate-limited APIs
✓ Stable, unchanging data
✗ Real-time data requirements
✗ Highly personalized responses
✗ Security-sensitive information
✗ Rapidly changing data

Production Considerations:
- Distributed caching (Redis) for multi-instance deployments
- Cache warming strategies for cold starts
- Monitoring and alerting on hit rates
- Regular cache analysis and optimization
- Security: encrypt sensitive cached data
- Compliance: respect data retention policies
- Backup and recovery procedures
- Cache versioning for updates

Cost-Benefit Analysis:
- Typical API call: $0.001-0.01 per request
- Cache hit cost: ~$0.000001 per request
- Potential savings: 90-99% cost reduction
- Performance gain: 10-1000x faster responses
- ROI: Immediate for high-volume applications
""")
    
    print("\n" + "=" * 80)
    print("Pattern 089 (Caching Patterns) demonstration complete!")
    print("=" * 80)


if __name__ == "__main__":
    demonstrate_caching_patterns()

    

Agentic Design Pattern: Caching Patterns

Pattern 089: Caching Patterns – Line-by-Line Explanation

Overview

This document provides a detailed line-by-line explanation of the Caching Patterns pattern implementation.

---

Pattern Description

Pattern 089: Caching Patterns

Description: The Caching Patterns involve storing and reusing previously computed results to improve performance, reduce latency, and minimize costs. In the context of LLM applications, caching is critical for optimizing response times and reducing API calls for repeated or similar queries. Effective caching strategies can dramatically improve user experience while significantly cutting operational costs.

Caching in LLM applications targets:
- Complete responses (exact match caching)
- Embeddings (vector representations)
- Prompt templates (reusable structures)
- Intermediate results (multi-step workflows)
- API responses (rate limit optimization)

Components: 1. Cache Types – In-memory cache (fast, volatile) – Disk cache (persistent, slower) – Distributed cache (Redis, Memcached) – Database cache (structured, queryable) – CDN cache (geographic distribution)

2. Cache Keys
   - Exact match (full prompt hash)
   - Semantic match (embedding similarity)
   - Parametric keys (template + parameters)
   - Hierarchical keys (nested cache structures)
   - Time-based keys (temporal validity)

3. Cache Policies
   - TTL (Time To Live) - expire after time
   - LRU (Least Recently Used) - evict old entries
   - LFU (Least Frequently Used) - evict unpopular
   - Size-based - limit total cache size
   - Custom policies - application-specific

4. Invalidation Strategies
   - Time-based expiration
   - Event-triggered invalidation
   - Manual invalidation
   - Cascade invalidation (dependencies)
   - Version-based invalidation

Use Cases: 1. Repeated Queries – FAQ responses – Common questions – Popular content – Standard procedures – Frequently accessed data

2. Expensive Operations
   - Complex reasoning chains
   - Multi-step workflows
   - Large context processing
   - Batch operations
   - Resource-intensive computations

3. Rate Limit Optimization
   - Reduce API calls
   - Stay within quotas
   - Handle traffic spikes
   - Cost optimization
   - Performance consistency

4. Embeddings and Vectors
   - Document embeddings
   - Query embeddings
   - Semantic search optimization
   - Similarity computations
   - Vector database queries

5. Multi-User Applications
   - Shared responses
   - Common workflows
   - Popular queries
   - Template responses
   - Collaborative features

LangChain Implementation: LangChain supports caching through: – InMemoryCache for simple caching – SQLiteCache for persistent caching – Custom cache implementations – Semantic similarity caching – Embedding caching

Key Features: 1. Multiple Cache Layers – L1: In-memory (fastest) – L2: Local disk (persistent) – L3: Distributed cache (shared) – L4: Database (long-term)

2. Intelligent Matching
   - Exact match for identical queries
   - Semantic match for similar queries
   - Fuzzy match with thresholds
   - Parametric templates

3. Performance Optimization
   - Fast lookup (< 1ms for in-memory)
   - Efficient serialization
   - Compression for large entries
   - Lazy loading

4. Observability
   - Hit rate monitoring
   - Miss analysis
   - Performance metrics
   - Cost savings tracking

Best Practices: 1. Cache Appropriately – Cache stable, reusable results – Don’t cache rapidly changing data – Consider data sensitivity – Balance freshness vs performance

2. Set Proper TTLs
   - Shorter for dynamic content
   - Longer for stable content
   - Consider update frequency
   - Account for user expectations

3. Monitor Performance
   - Track hit rates (target > 70%)
   - Measure latency improvements
   - Calculate cost savings
   - Identify cache misses patterns

4. Handle Cache Misses
   - Graceful fallback to computation
   - Transparent to users
   - Background cache warming
   - Predictive pre-caching

Trade-offs: Advantages: – Dramatically improved response times – Significant cost reduction – Better user experience – Reduced API rate limiting – Lower resource consumption – Improved scalability

Disadvantages:
- Stale data risk
- Memory/storage overhead
- Cache consistency challenges
- Complexity in invalidation
- Initial cache warming time
- Debugging complexity

Production Considerations: 1. Cache Size Management – Monitor memory usage – Implement eviction policies – Set size limits – Regular cleanup – Compression strategies

2. Cache Consistency
   - Version tracking
   - Update propagation
   - Invalidation timing
   - Distributed consistency
   - Race condition handling

3. Security
   - Access control for cached data
   - Encryption for sensitive content
   - Cache poisoning prevention
   - User isolation in multi-tenant
   - Audit trails

4. Monitoring
   - Hit/miss rates
   - Cache size trends
   - Eviction frequency
   - Cost savings
   - Performance impact

5. Disaster Recovery
   - Cache rebuild procedures
   - Backup strategies
   - Failover mechanisms
   - Data loss handling
   - Recovery time objectives

---

File Statistics

• Total Lines: 897
• Classes: 6
• Functions: 1
• Constants: 0

---

Module Documentation

"""
Pattern 089: Caching Patterns

Description:
    The Caching Patterns involve storing and reusing previously computed results to
    improve performance, reduce latency, and minimize costs. In the context of LLM
    applications, caching is critical for optimizing response times and reducing API
    calls for repeated or similar queries. Effective caching strategies can
    dramatically improve user experience while significantly cutting operational costs.

    Caching in LLM applications targets:
    - Complete responses (exact match caching)
    - Embeddings (vector representations)
    - Prompt templates (reusable structures)
    - Intermediate results (multi-step workflows)
    - API responses (rate limit optimization)

Components:
    1. Cache Types
       - In-memory cache (fast, volatile)
       - Disk cache (persistent, slower)
       - Distributed cache (Redis, Memcached)
       - Database cache (structured, queryable)
       - CDN cache (geographic distribution)

    2. Cache Keys
       - Exact match (full prompt hash)
       - Semantic match (embedding similarity)
       - Parametric keys (template + parameters)
       - Hierarchical keys (nested cache structures)
       - Time-based keys (temporal validity)

    3. Cache Policies
       - TTL (Time To Live) - expire after time
       - LRU (Least Recently Used) - evict old entries
       - LFU (Least Frequently Used) - evict unpopular
       - Size-based - limit total cache size
       - Custom policies - application-specific

    4. Invalidation Strategies
       - Time-based expiration
       - Event-triggered invalidation
       - Manual invalidation
       - Cascade invalidation (dependencies)
       - Version-based invalidation

Use Cases:
    1. Repeated Queries
       - FAQ responses
       - Common questions
       - Popular content
       - Standard procedures
       - Frequently accessed data

    2. Expensive Operations
       - Complex reasoning chains
       - Multi-step workflows
       - Large context processing
       - Batch operations
       - Resource-intensive computations

    3. Rate Limit Optimization
       - Reduce API calls
       - Stay within quotas
       - Handle traffic spikes
       - Cost optimization
       - Performance consistency

    4. Embeddings and Vectors
       - Document embeddings
       - Query embeddings
       - Semantic search optimization
       - Similarity computations
       - Vector database queries

    5. Multi-User Applications
       - Shared responses
       - Common workflows
       - Popular queries
       - Template responses
       - Collaborative features

LangChain Implementation:
    LangChain supports caching through:
    - InMemoryCache for simple caching
    - SQLiteCache for persistent caching
    - Custom cache implementations
    - Semantic similarity caching
    - Embedding caching

Key Features:
    1. Multiple Cache Layers
       - L1: In-memory (fastest)
       - L2: Local disk (persistent)
       - L3: Distributed cache (shared)
       - L4: Database (long-term)

    2. Intelligent Matching
       - Exact match for identical queries
       - Semantic match for similar queries
       - Fuzzy match with thresholds
       - Parametric templates

    3. Performance Optimization
       - Fast lookup (< 1ms for in-memory)
       - Efficient serialization
       - Compression for large entries
       - Lazy loading

    4. Observability
       - Hit rate monitoring
       - Miss analysis
       - Performance metrics
       - Cost savings tracking

Best Practices:
    1. Cache Appropriately
       - Cache stable, reusable results
       - Don't cache rapidly changing data
       - Consider data sensitivity
       - Balance freshness vs performance

    2. Set Proper TTLs
       - Shorter for dynamic content
       - Longer for stable content
       - Consider update frequency
       - Account for user expectations

    3. Monitor Performance
       - Track hit rates (target > 70%)
       - Measure latency improvements
       - Calculate cost savings
       - Identify cache misses patterns

    4. Handle Cache Misses
       - Graceful fallback to computation
       - Transparent to users
       - Background cache warming
       - Predictive pre-caching

Trade-offs:
    Advantages:
    - Dramatically improved response times
    - Significant cost reduction
    - Better user experience
    - Reduced API rate limiting
    - Lower resource consumption
    - Improved scalability

    Disadvantages:
    - Stale data risk
    - Memory/storage overhead
    - Cache consistency challenges
    - Complexity in invalidation
    - Initial cache warming time
    - Debugging complexity

Production Considerations:
    1. Cache Size Management
       - Monitor memory usage
       - Implement eviction policies
       - Set size limits
       - Regular cleanup
       - Compression strategies

    2. Cache Consistency
       - Version tracking
       - Update propagation
       - Invalidation timing
       - Distributed consistency
       - Race condition handling

    3. Security
       - Access control for cached data
       - Encryption for sensitive content
       - Cache poisoning prevention
       - User isolation in multi-tenant
       - Audit trails

    4. Monitoring
       - Hit/miss rates
       - Cache size trends
       - Eviction frequency
       - Cost savings
       - Performance impact

    5. Disaster Recovery
       - Cache rebuild procedures
       - Backup strategies
       - Failover mechanisms
       - Data loss handling
       - Recovery time objectives
"""

Purpose: This module implements the Caching Patterns pattern, providing a structured approach to agent design and implementation.

---

Import Statements

Standard Library Imports

import collections.OrderedDict
import dataclasses.dataclass
import dataclasses.field
import datetime.datetime
import datetime.timedelta
import enum.Enum
import json
import langchain_core.caches.InMemoryCache
import langchain_core.output_parsers.StrOutputParser
import langchain_core.prompts.ChatPromptTemplate
import os
import time
import typing.Any
import typing.Callable
import typing.Dict
import typing.List
import typing.Optional

Purpose: Core Python functionality: – collections.OrderedDict: Regular expression operations – dataclasses.dataclass: Simplified class definitions with @dataclass – dataclasses.field: Simplified class definitions with @dataclass – datetime.datetime: Date and time handling – datetime.timedelta: Date and time handling – enum.Enum: Enumeration support – json: JSON serialization/deserialization – langchain_core.caches.InMemoryCache: Regular expression operations – langchain_core.output_parsers.StrOutputParser: Regular expression operations – langchain_core.prompts.ChatPromptTemplate: Regular expression operations – os: Environment variable access and system operations – time: Time-related functions – typing.Any: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Callable: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Dict: Type hints for better code documentation (List, Dict, Any, Optional) – typing.List: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Optional: Type hints for better code documentation (List, Dict, Any, Optional)

LangChain Imports

from langchain.globals.set_llm_cache
from langchain_openai.ChatOpenAI
from langchain_openai.OpenAIEmbeddings

Purpose: LangChain framework components: – LLM Interface: Wrapper for OpenAI models – Chains: Composable building blocks for complex workflows

Other Imports

from dotenv.load_dotenv
from hashlib

Purpose: Additional dependencies for specialized functionality.

---

Classes

Class: CachePolicy

Lines 215-220

Inherits from: Enum

Purpose:

Cache eviction policies

---

Class: CacheEntry

Lines 224-243

Purpose:

Represents a cached entry

Methods:

is_expired(self)

Lines 233-238

Purpose:

Check if entry is expired

Parameters:

Returns: bool

touch(self)

Lines 240-243

Purpose:

Update access information

Parameters:

---

Class: CacheStats

Lines 247-264

Purpose:

Cache performance statistics

Methods:

hit_rate(self)

Lines 255-259

Decorators: property

Purpose:

Calculate cache hit rate

Parameters:

Returns: float

miss_rate(self)

Lines 262-264

Decorators: property

Purpose:

Calculate cache miss rate

Parameters:

Returns: float

---

Class: SimpleCache

Lines 267-416

Purpose:

Simple in-memory cache with multiple eviction policies.

This cache supports:
1. Multiple eviction policies (LRU, LFU, FIFO, TTL)
2. Size limits
3. TTL per entry
4. Performance statistics

Methods:

__init__(self, max_size, policy, default_ttl)

Lines 278-296

Purpose:

Initialize cache.

Args:
    max_size: Maximum number of entries
    policy: Eviction policy
    default_ttl: Default TTL in seconds

Parameters: – max_size: Max size – policy: Policy – default_ttl: Default ttl

_make_key(self, key)

Lines 298-302

Purpose:

Create cache key from input

Parameters: – key: Key

Returns: str

_evict(self)

Lines 304-320

Purpose:

Evict entries based on policy

Parameters:

_remove_expired(self)

Lines 322-329

Purpose:

Remove expired entries

Parameters:

get(self, key)

Lines 331-365

Purpose:

Get value from cache.

Args:
    key: Cache key

Returns:
    Cached value or None

Parameters: – key: Key

Returns: <ast.Subscript object at 0x000001A06D62AED0>

set(self, key, value, ttl)

Lines 367-394

Purpose:

Set value in cache.

Args:
    key: Cache key
    value: Value to cache
    ttl: Optional TTL in seconds

Parameters: – key: Key – value: Value – ttl: Ttl

clear(self)

Lines 396-399

Purpose:

Clear all cache entries

Parameters:

size(self)

Lines 401-403

Purpose:

Get current cache size

Parameters:

Returns: int

get_stats(self)

Lines 405-416

Purpose:

Get cache statistics

Parameters:

Returns: <ast.Subscript object at 0x000001A06D74A3D0>

---

Class: SemanticCache

Lines 419-540

Purpose:

Semantic cache using embeddings for similarity matching.

This cache finds similar queries using vector similarity,
allowing cache hits for semantically similar but not identical queries.

Methods:

__init__(self, similarity_threshold, max_size)

Lines 427-443

Purpose:

Initialize semantic cache.

Args:
    similarity_threshold: Minimum similarity for cache hit
    max_size: Maximum cache entries

Parameters: – similarity_threshold: Similarity threshold – max_size: Max size

_compute_similarity(self, query1, query2)

Lines 445-471

Purpose:

Compute semantic similarity between queries.

Args:
    query1: First query
    query2: Second query

Returns:
    Similarity score (0-1)

Parameters: – query1: Input query or question – query2: Input query or question

Returns: float

get(self, query)

Lines 473-502

Purpose:

Get value for semantically similar query.

Args:
    query: Query string

Returns:
    Cached value or None

Parameters: – query: Input query or question

Returns: <ast.Subscript object at 0x000001A06D6A1F50>

set(self, query, value)

Lines 504-522

Purpose:

Cache value for query.

Args:
    query: Query string
    value: Value to cache

Parameters: – query: Input query or question – value: Value

clear(self)

Lines 524-527

Purpose:

Clear cache

Parameters:

get_stats(self)

Lines 529-540

Purpose:

Get cache statistics

Parameters:

Returns: <ast.Subscript object at 0x000001A06D7FC110>

---

Class: CachedLLMAgent

Lines 543-599

Purpose:

LLM agent with caching support.

This agent automatically caches responses and retrieves them
for repeated or similar queries.

Methods:

__init__(self, cache, model)

Lines 551-564

Purpose:

Initialize cached agent.

Args:
    cache: Cache instance to use
    model: LLM model name

Parameters: – cache: Cache – model: Language model instance

generate(self, prompt)

Lines 566-599

Purpose:

Generate response with caching.

Args:
    prompt: Input prompt

Returns:
    Dictionary with response and cache info

Parameters: – prompt: Prompt template or text

Returns: <ast.Subscript object at 0x000001A06D7C1A90>

---

Functions

demonstrate_caching_patterns()

Lines 602-892

Purpose:

Demonstrate caching patterns

---

Key Concepts Summary

Pattern Overview

The Caching Patterns pattern provides a structured approach to building AI agents with specific capabilities and behaviors. This implementation leverages LangChain’s composable components to create a robust and maintainable solution.

Core Components

1. Agent Class: Main implementation of the pattern logic
2. Prompt Engineering: Carefully crafted prompts to guide LLM behavior
3. Tool Integration: Optional tools for extended functionality
4. Memory Management: State handling across interactions
5. Output Parsing: Structured response processing

When to Use

✅ Good for: – Scenarios requiring this specific pattern’s strengths – Applications needing structured agent behavior – Systems where explainability is important – Production environments requiring reliability

❌ Not ideal for: – Simple single-prompt tasks – Scenarios requiring different pattern characteristics – Resource-constrained environments (if computationally intensive)

---

Best Practices Demonstrated

1. Environment Configuration

• ✅ Use .env files for API keys
• ✅ Never hardcode credentials
• ✅ Validate environment variables on startup

2. Error Handling

• ✅ Graceful degradation on failures
• ✅ Informative error messages
• ✅ Logging for debugging

3. Prompt Engineering

• ✅ Clear, specific instructions
• ✅ Structured output formats
• ✅ Few-shot examples when beneficial

4. Code Organization

• ✅ Separation of concerns (class-based design)
• ✅ Type hints for clarity
• ✅ Docstrings for documentation

5. Performance Optimization

• ✅ Appropriate temperature settings
• ✅ Token budget management
• ✅ Caching where applicable

---

LangChain Components Used

• ChatOpenAI: OpenAI model interface
• Chains: Sequential component composition

Why LangChain? – Composable abstractions for LLM applications – Rich ecosystem of integrations – Production-ready patterns – Active community and development

---

Execution Flow

1. Initialize agent with configuration
   ↓
2. Load environment and dependencies
   ↓
3. Prepare prompt with input data
   ↓
4. Invoke LLM through LangChain
   ↓
5. Parse and validate response
   ↓
6. Return formatted result

Key Decision Points: – Model selection (temperature, model type) – Prompt structure (few-shot vs zero-shot) – Error handling strategy – Output format validation

---

Performance Considerations

Computational Cost

• LLM Calls: Primary cost factor (based on tokens)
• Latency: Depends on model and input size
• Memory: Minimal for most patterns

Optimization Strategies

1. Prompt Optimization: Minimize tokens while maintaining clarity
2. Caching: Cache repeated queries/responses
3. Model Selection: Use smaller models when appropriate
4. Batch Processing: Group similar requests
5. Streaming: For long responses, stream tokens

Cost Estimates (Approximate)

• GPT-3.5-turbo: ~$0.002 per request
• GPT-4: ~$0.03 per request
• GPT-4-turbo: ~$0.01 per request

Actual costs vary based on input/output token counts

---

Common Pitfalls to Avoid

1. Prompt Design Issues

❌ Vague or ambiguous instructions
✅ Clear, specific, testable prompts

2. Error Handling

❌ Ignoring API errors or rate limits
✅ Implement retry logic and graceful degradation

3. Token Management

❌ Exceeding model token limits
✅ Truncate inputs, chunk processing

4. Security

❌ Exposing API keys in code
✅ Use environment variables

5. Testing

❌ Only testing happy paths
✅ Test edge cases, failures, malformed inputs

6. Cost Management

❌ Unmonitored API usage
✅ Implement budgets, alerts, and monitoring

---

Further Improvements

Production Enhancements

1. Monitoring: Add logging, metrics, tracing
2. Validation: Input/output schema validation
3. Rate Limiting: Implement request throttling
4. Caching: Redis or in-memory cache for responses
5. Async Support: Non-blocking operations for scalability

Feature Extensions

1. Multi-Model Support: Switch between providers
2. Fallback Models: Graceful degradation strategy
3. Custom Tools: Domain-specific function calling
4. Advanced Memory: Vector stores for long-term memory
5. Evaluation: Automated testing and benchmarking

Code Quality

1. Testing: Unit tests, integration tests
2. Documentation: API docs, usage examples
3. Type Safety: Stricter type hints
4. Configuration: Externalized config files

---

Related Patterns

• Pattern 088
• Pattern 090

Pattern Families: – Reasoning Patterns (001-011) – Multi-Agent Patterns (012-019) – Tool Use Patterns (020-025) – Memory Patterns (026-031) – And many more…

---

Summary

The Caching Patterns pattern provides a powerful approach to building AI agents with specific capabilities. This implementation demonstrates:

✅ Key Strengths: – Clean, maintainable code structure – LangChain integration for composability – Type hints and documentation – Production-ready patterns

🎯 Best Use Cases: – Applications requiring this pattern’s characteristics – Systems needing structured agent behavior – Production environments requiring reliability

📚 Learning Resources: – LangChain Documentation: https://python.langchain.com/ – Pattern Documentation: See related patterns – OpenAI API Reference: https://platform.openai.com/docs/

---

File: 089_caching_patterns.py
Total Lines: 897
Pattern Category: AI Agent Design Pattern #089

---

Generated from actual implementation code
For educational and reference purposes