"""
Pattern 090: Load Balancing Pattern

Description:
    The Load Balancing Pattern distributes requests across multiple LLM endpoints or
    instances to achieve high availability, improved performance, and fault tolerance.
    In production LLM applications, load balancing is critical for handling high traffic
    volumes, preventing single points of failure, and ensuring consistent service
    quality even when individual endpoints fail or become overloaded.

    Load balancing strategies enable:
    - High availability through redundancy
    - Improved performance via parallel processing
    - Cost optimization through provider selection
    - Rate limit management across multiple accounts
    - Geographic distribution for lower latency
    - Failover and disaster recovery

Components:
    1. Load Balancing Strategies
       - Round Robin: Distribute equally across endpoints
       - Weighted Round Robin: Priority-based distribution
       - Least Loaded: Send to endpoint with lowest load
       - Random: Random endpoint selection
       - Geographic: Route based on location
       - Cost-Based: Choose cheapest available endpoint

    2. Health Checking
       - Endpoint availability monitoring
       - Response time tracking
       - Error rate monitoring
       - Periodic health probes
       - Automatic endpoint removal/recovery

    3. Request Management
       - Request queuing and buffering
       - Retry logic with backoff
       - Circuit breaker pattern
       - Request timeout handling
       - Priority-based routing

    4. Failover Mechanisms
       - Automatic endpoint switching
       - Degraded mode operation
       - Fallback to alternative providers
       - Request replay on failure
       - State preservation

Use Cases:
    1. High-Traffic Applications
       - Chatbots serving thousands of users
       - API services with SLA requirements
       - Real-time processing pipelines
       - Multi-tenant platforms
       - Peak load handling

    2. Multi-Provider Deployments
       - OpenAI + Anthropic + Google
       - Primary and backup providers
       - Cost optimization strategies
       - Feature-specific routing
       - Geographic compliance

    3. Rate Limit Management
       - Distribute across multiple API keys
       - Prevent rate limit exhaustion
       - Smooth request distribution
       - Burst handling
       - Cost control

    4. Disaster Recovery
       - Automatic failover
       - Multi-region deployment
       - Zero-downtime updates
       - Service degradation handling
       - Business continuity

    5. Performance Optimization
       - Parallel request processing
       - Load-based routing
       - Latency optimization
       - Resource utilization
       - Capacity planning

LangChain Implementation:
    LangChain doesn't have built-in load balancing, but you can implement
    it using:
    - Custom LLM wrappers
    - Callback handlers for monitoring
    - Retry logic with different models
    - Custom routing logic
    - Health check mechanisms

Key Features:
    1. Endpoint Management
       - Dynamic endpoint registration
       - Health status tracking
       - Automatic recovery
       - Configuration updates
       - Version management

    2. Intelligent Routing
       - Strategy selection
       - Real-time decision making
       - Performance-based routing
       - Cost optimization
       - SLA enforcement

    3. Fault Tolerance
       - Automatic retry with exponential backoff
       - Circuit breaker to prevent cascading failures
       - Graceful degradation
       - Error handling and recovery
       - Request preservation

    4. Monitoring
       - Request distribution metrics
       - Endpoint health status
       - Response time tracking
       - Error rate monitoring
       - Cost tracking per endpoint

Best Practices:
    1. Health Monitoring
       - Regular health checks (every 30-60s)
       - Multiple health metrics (latency, errors, availability)
       - Automatic endpoint recovery
       - Alert on degraded performance
       - Log all health transitions

    2. Load Distribution
       - Consider endpoint capacity
       - Account for different model speeds
       - Balance cost vs performance
       - Avoid overloading any single endpoint
       - Reserve capacity for spikes

    3. Failover Strategy
       - Fast failure detection (< 5s)
       - Quick failover ( float:
        """Calculate success rate"""
        if self.total_requests == 0:
            return 0.0
        return (self.successful_requests / self.total_requests) * 100
    
    @property
    def average_response_time(self) -> float:
        """Calculate average response time"""
        if self.successful_requests == 0:
            return 0.0
        return self.total_response_time / self.successful_requests


@dataclass
class LoadBalancerStats:
    """Overall load balancer statistics"""
    total_requests: int = 0
    successful_requests: int = 0
    failed_requests: int = 0
    failovers: int = 0
    total_response_time: float = 0.0
    endpoint_metrics: Dict[str, EndpointMetrics] = field(default_factory=dict)


class LLMEndpoint:
    """
    Represents a single LLM endpoint with health tracking.
    """
    
    def __init__(self, config: EndpointConfig):
        """
        Initialize endpoint.
        
        Args:
            config: Endpoint configuration
        """
        self.config = config
        self.metrics = EndpointMetrics()
        self.llm: Optional[ChatOpenAI] = None
        self.last_health_check: Optional[datetime] = None
        self.request_times: deque = deque(maxlen=100)  # Track recent request times
        
        self._initialize_llm()
    
    def _initialize_llm(self):
        """Initialize the LLM client"""
        try:
            # Use provided API key or default from environment
            api_key = self.config.api_key or os.getenv("OPENAI_API_KEY")
            
            self.llm = ChatOpenAI(
                model=self.config.model,
                openai_api_key=api_key,
                timeout=self.config.timeout,
                temperature=0.7
            )
            self.metrics.status = EndpointStatus.HEALTHY
        except Exception as e:
            print(f"Error initializing endpoint {self.config.id}: {e}")
            self.metrics.status = EndpointStatus.UNHEALTHY
            self.metrics.last_error = str(e)
            self.metrics.last_error_time = datetime.now()
    
    def is_available(self) -> bool:
        """Check if endpoint is available for requests"""
        return (
            self.metrics.status in [EndpointStatus.HEALTHY, EndpointStatus.DEGRADED]
            and self.llm is not None
            and self.metrics.active_requests  EndpointStatus:
        """
        Perform health check on endpoint.
        
        Returns:
            Current health status
        """
        try:
            if self.llm is None:
                self._initialize_llm()
            
            # Simple health check - try a minimal request
            start_time = time.time()
            self.llm.invoke("test")
            response_time = time.time() - start_time
            
            # Update status based on response time and error rate
            if response_time  95:
                self.metrics.status = EndpointStatus.HEALTHY
            elif response_time  80:
                self.metrics.status = EndpointStatus.DEGRADED
            else:
                self.metrics.status = EndpointStatus.UNHEALTHY
            
            self.last_health_check = datetime.now()
            return self.metrics.status
            
        except Exception as e:
            self.metrics.status = EndpointStatus.UNHEALTHY
            self.metrics.last_error = str(e)
            self.metrics.last_error_time = datetime.now()
            return EndpointStatus.UNHEALTHY
    
    def generate(self, prompt: str) -> Dict[str, Any]:
        """
        Generate response using this endpoint.
        
        Args:
            prompt: Input prompt
            
        Returns:
            Response dictionary
        """
        self.metrics.total_requests += 1
        self.metrics.active_requests += 1
        
        try:
            start_time = time.time()
            response = self.llm.invoke(prompt)
            response_time = time.time() - start_time
            
            self.metrics.successful_requests += 1
            self.metrics.total_response_time += response_time
            self.metrics.active_requests -= 1
            self.request_times.append(response_time)
            
            return {
                "response": response.content,
                "endpoint_id": self.config.id,
                "response_time": response_time,
                "success": True
            }
            
        except Exception as e:
            self.metrics.failed_requests += 1
            self.metrics.active_requests -= 1
            self.metrics.last_error = str(e)
            self.metrics.last_error_time = datetime.now()
            
            # Mark as unhealthy after repeated failures
            if self.metrics.success_rate  Optional[LLMEndpoint]:
        """Select endpoint using round-robin strategy"""
        attempts = 0
        while attempts  Optional[LLMEndpoint]:
        """Select endpoint using weighted round-robin strategy"""
        # Create weighted list
        weighted_endpoints = []
        for endpoint in self.endpoints:
            if endpoint.is_available():
                weighted_endpoints.extend([endpoint] * endpoint.config.weight)
        
        if not weighted_endpoints:
            return None
        
        # Round-robin through weighted list
        endpoint = weighted_endpoints[self.current_endpoint_index % len(weighted_endpoints)]
        self.current_endpoint_index += 1
        return endpoint
    
    def _select_endpoint_least_loaded(self) -> Optional[LLMEndpoint]:
        """Select endpoint with least active requests"""
        available = [e for e in self.endpoints if e.is_available()]
        if not available:
            return None
        
        return min(available, key=lambda e: e.metrics.active_requests)
    
    def _select_endpoint_random(self) -> Optional[LLMEndpoint]:
        """Select random available endpoint"""
        available = [e for e in self.endpoints if e.is_available()]
        if not available:
            return None
        
        return random.choice(available)
    
    def _select_endpoint_fastest(self) -> Optional[LLMEndpoint]:
        """Select endpoint with fastest average response time"""
        available = [e for e in self.endpoints if e.is_available()]
        if not available:
            return None
        
        # Sort by average response time (endpoints with no history get high priority)
        return min(
            available,
            key=lambda e: e.metrics.average_response_time if e.metrics.successful_requests > 0 else 0.0
        )
    
    def _select_endpoint(self) -> Optional[LLMEndpoint]:
        """
        Select endpoint based on strategy.
        
        Returns:
            Selected endpoint or None if none available
        """
        if self.strategy == LoadBalancingStrategy.ROUND_ROBIN:
            return self._select_endpoint_round_robin()
        elif self.strategy == LoadBalancingStrategy.WEIGHTED_ROUND_ROBIN:
            return self._select_endpoint_weighted()
        elif self.strategy == LoadBalancingStrategy.LEAST_LOADED:
            return self._select_endpoint_least_loaded()
        elif self.strategy == LoadBalancingStrategy.RANDOM:
            return self._select_endpoint_random()
        elif self.strategy == LoadBalancingStrategy.FASTEST_RESPONSE:
            return self._select_endpoint_fastest()
        else:
            return self._select_endpoint_round_robin()
    
    def _perform_health_checks(self):
        """Perform health checks on all endpoints if needed"""
        now = datetime.now()
        if (now - self.last_health_check).seconds  Dict[str, Any]:
        """
        Generate response with load balancing and failover.
        
        Args:
            prompt: Input prompt
            max_retries: Maximum retry attempts
            
        Returns:
            Response dictionary
        """
        self.stats.total_requests += 1
        
        # Perform health checks if needed
        self._perform_health_checks()
        
        attempts = 0
        last_error = None
        
        while attempts  Dict[str, Any]:
        """Get load balancer statistics"""
        return {
            "total_requests": self.stats.total_requests,
            "successful_requests": self.stats.successful_requests,
            "failed_requests": self.stats.failed_requests,
            "failovers": self.stats.failovers,
            "success_rate": (
                (self.stats.successful_requests / self.stats.total_requests * 100)
                if self.stats.total_requests > 0 else 0.0
            ),
            "average_response_time": (
                self.stats.total_response_time / self.stats.successful_requests
                if self.stats.successful_requests > 0 else 0.0
            ),
            "endpoints": [
                {
                    "id": e.config.id,
                    "model": e.config.model,
                    "status": e.metrics.status.value,
                    "total_requests": e.metrics.total_requests,
                    "success_rate": f"{e.metrics.success_rate:.2f}%",
                    "avg_response_time": f"{e.metrics.average_response_time:.3f}s",
                    "active_requests": e.metrics.active_requests
                }
                for e in self.endpoints
            ]
        }


def demonstrate_load_balancing():
    """Demonstrate load balancing patterns"""
    print("=" * 80)
    print("LOAD BALANCING PATTERN DEMONSTRATION")
    print("=" * 80)
    
    # Example 1: Basic Round-Robin Load Balancing
    print("\n" + "=" * 80)
    print("EXAMPLE 1: Round-Robin Load Balancing")
    print("=" * 80)
    
    # Configure multiple endpoints (simulating multiple API keys or models)
    endpoints = [
        EndpointConfig(id="endpoint-1", model="gpt-3.5-turbo", weight=1),
        EndpointConfig(id="endpoint-2", model="gpt-3.5-turbo", weight=1),
        EndpointConfig(id="endpoint-3", model="gpt-3.5-turbo", weight=1),
    ]
    
    lb = LoadBalancer(endpoints, strategy=LoadBalancingStrategy.ROUND_ROBIN)
    
    print("\nConfigured 3 endpoints with round-robin strategy\n")
    print("Sending 6 requests to observe distribution:\n")
    
    queries = [
        "What is 2+2?",
        "What is Python?",
        "What is AI?",
        "What is ML?",
        "What is DL?",
        "What is NLP?"
    ]
    
    for i, query in enumerate(queries, 1):
        print(f"Request {i}: {query}")
        result = lb.generate(query)
        
        if result["success"]:
            print(f"  ✓ Endpoint: {result['endpoint_id']}")
            print(f"  Response time: {result['response_time']:.3f}s")
            print(f"  Response: {result['response'][:60]}...")
        else:
            print(f"  ✗ Error: {result['error']}")
        print()
    
    stats = lb.get_stats()
    print("Load Distribution:")
    for endpoint in stats["endpoints"]:
        print(f"  {endpoint['id']}: {endpoint['total_requests']} requests")
    
    # Example 2: Weighted Load Balancing
    print("\n" + "=" * 80)
    print("EXAMPLE 2: Weighted Load Balancing")
    print("=" * 80)
    
    weighted_endpoints = [
        EndpointConfig(id="premium-endpoint", model="gpt-4", weight=3),  # Gets more traffic
        EndpointConfig(id="standard-endpoint", model="gpt-3.5-turbo", weight=1),
    ]
    
    weighted_lb = LoadBalancer(
        weighted_endpoints,
        strategy=LoadBalancingStrategy.WEIGHTED_ROUND_ROBIN
    )
    
    print("\nConfigured 2 endpoints:")
    print("  - Premium (GPT-4): weight=3")
    print("  - Standard (GPT-3.5): weight=1")
    print("\nSending 8 requests:\n")
    
    for i in range(8):
        result = weighted_lb.generate(f"Request {i+1}")
        if result["success"]:
            print(f"Request {i+1} → {result['endpoint_id']}")
    
    print("\nExpected distribution: ~6 to premium, ~2 to standard")
    stats = weighted_lb.get_stats()
    for endpoint in stats["endpoints"]:
        print(f"  {endpoint['id']}: {endpoint['total_requests']} requests")
    
    # Example 3: Least-Loaded Strategy
    print("\n" + "=" * 80)
    print("EXAMPLE 3: Least-Loaded Strategy")
    print("=" * 80)
    
    ll_endpoints = [
        EndpointConfig(id="endpoint-A", model="gpt-3.5-turbo"),
        EndpointConfig(id="endpoint-B", model="gpt-3.5-turbo"),
    ]
    
    ll_lb = LoadBalancer(ll_endpoints, strategy=LoadBalancingStrategy.LEAST_LOADED)
    
    print("\nLeast-loaded strategy routes to endpoint with fewest active requests")
    print("This optimizes for load distribution in real-time\n")
    
    for i in range(4):
        result = ll_lb.generate(f"Query {i+1}")
        if result["success"]:
            print(f"Query {i+1} → {result['endpoint_id']}")
    
    # Example 4: Failover Handling
    print("\n" + "=" * 80)
    print("EXAMPLE 4: Automatic Failover")
    print("=" * 80)
    
    print("\nSimulating endpoint failure scenario:")
    print("  - 3 endpoints configured")
    print("  - Mark one as unhealthy")
    print("  - Load balancer automatically routes around it\n")
    
    failover_endpoints = [
        EndpointConfig(id="healthy-1", model="gpt-3.5-turbo"),
        EndpointConfig(id="unhealthy", model="gpt-3.5-turbo"),
        EndpointConfig(id="healthy-2", model="gpt-3.5-turbo"),
    ]
    
    fo_lb = LoadBalancer(failover_endpoints, strategy=LoadBalancingStrategy.ROUND_ROBIN)
    
    # Simulate endpoint failure
    fo_lb.endpoints[1].metrics.status = EndpointStatus.UNHEALTHY
    print("Marked 'unhealthy' endpoint as UNHEALTHY\n")
    
    print("Sending 4 requests:\n")
    for i in range(4):
        result = fo_lb.generate(f"Query {i+1}")
        if result["success"]:
            print(f"Query {i+1} → {result['endpoint_id']} ✓")
    
    print("\nOnly healthy endpoints received requests!")
    
    # Example 5: Performance Comparison
    print("\n" + "=" * 80)
    print("EXAMPLE 5: Strategy Performance Comparison")
    print("=" * 80)
    
    strategies = [
        LoadBalancingStrategy.ROUND_ROBIN,
        LoadBalancingStrategy.RANDOM,
        LoadBalancingStrategy.LEAST_LOADED,
    ]
    
    print("\nComparing different load balancing strategies:\n")
    
    for strategy in strategies:
        test_endpoints = [
            EndpointConfig(id=f"ep-{i}", model="gpt-3.5-turbo")
            for i in range(3)
        ]
        
        test_lb = LoadBalancer(test_endpoints, strategy=strategy)
        
        # Send some requests
        for i in range(6):
            test_lb.generate(f"Test query {i}")
        
        stats = test_lb.get_stats()
        print(f"{strategy.value.upper()}:")
        print(f"  Success Rate: {stats['success_rate']:.2f}%")
        print(f"  Total Requests: {stats['total_requests']}")
        for ep in stats["endpoints"]:
            print(f"    {ep['id']}: {ep['total_requests']} requests")
        print()
    
    # Example 6: Monitoring and Statistics
    print("\n" + "=" * 80)
    print("EXAMPLE 6: Comprehensive Statistics")
    print("=" * 80)
    
    monitor_endpoints = [
        EndpointConfig(id="prod-1", model="gpt-3.5-turbo"),
        EndpointConfig(id="prod-2", model="gpt-3.5-turbo"),
    ]
    
    monitor_lb = LoadBalancer(monitor_endpoints, strategy=LoadBalancingStrategy.ROUND_ROBIN)
    
    print("\nSimulating production workload...\n")
    
    # Simulate various scenarios
    for i in range(10):
        result = monitor_lb.generate(f"Production query {i+1}")
    
    stats = monitor_lb.get_stats()
    
    print("Load Balancer Statistics:")
    print(f"  Total Requests: {stats['total_requests']}")
    print(f"  Successful: {stats['successful_requests']}")
    print(f"  Failed: {stats['failed_requests']}")
    print(f"  Success Rate: {stats['success_rate']:.2f}%")
    print(f"  Failovers: {stats['failovers']}")
    print(f"  Avg Response Time: {stats['average_response_time']:.3f}s")
    
    print("\nPer-Endpoint Metrics:")
    for ep in stats["endpoints"]:
        print(f"\n  {ep['id']} ({ep['model']}):")
        print(f"    Status: {ep['status']}")
        print(f"    Requests: {ep['total_requests']}")
        print(f"    Success Rate: {ep['success_rate']}")
        print(f"    Avg Response Time: {ep['avg_response_time']}")
        print(f"    Active Requests: {ep['active_requests']}")
    
    # Summary
    print("\n" + "=" * 80)
    print("LOAD BALANCING PATTERN SUMMARY")
    print("=" * 80)
    print("""
Load Balancing Benefits:
1. High Availability: 99.9%+ uptime through redundancy
2. Performance: Better throughput via parallel processing
3. Fault Tolerance: Automatic failover on endpoint failure
4. Scalability: Easy to add more endpoints
5. Cost Optimization: Route based on cost/performance
6. Rate Limit Management: Distribute across API keys

Load Balancing Strategies:
1. Round Robin
   - Equal distribution across endpoints
   - Simple and predictable
   - Good for homogeneous endpoints
   - Use when: All endpoints have similar capacity

2. Weighted Round Robin
   - Priority-based distribution
   - Higher weight = more requests
   - Good for heterogeneous endpoints
   - Use when: Different endpoint capacities

3. Least Loaded
   - Route to least busy endpoint
   - Optimizes for current load
   - Dynamic adjustment
   - Use when: Variable request durations

4. Random
   - Random endpoint selection
   - Stateless and simple
   - Good load distribution over time
   - Use when: Simplicity preferred

5. Fastest Response
   - Route to historically fastest endpoint
   - Performance-optimized
   - Adaptive to endpoint characteristics
   - Use when: Latency is critical

Health Checking:
1. Active Health Checks
   - Regular test requests
   - Verify endpoint availability
   - Monitor response times
   - Interval: 30-60 seconds

2. Passive Health Monitoring
   - Track request success/failure
   - Monitor error rates
   - Calculate response times
   - Continuous monitoring

3. Status Levels
   - HEALTHY:  95% success
   - DEGRADED:  80% success
   - UNHEALTHY: Slow or 1000 req/hr)
✓ SLA requirements (>99% uptime)
✓ Rate limit concerns
✓ Multi-provider deployments
✓ Geographic distribution needed
✗ Low-traffic applications
✗ Single endpoint sufficient
✗ Cost constraints (added complexity)

ROI Analysis:
- Uptime improvement: 99% → 99.9% (10x fewer outages)
- Performance: 2-3x throughput capacity
- User satisfaction: Reduced error rates
- Cost: +20% complexity, but worth it for scale
- Break-even: ~500 requests/hour
""")
    
    print("\n" + "=" * 80)
    print("Pattern 090 (Load Balancing) demonstration complete!")
    print("=" * 80)


if __name__ == "__main__":
    demonstrate_load_balancing()

    

Agentic Design Pattern: Load Balancing

Pattern 090: Load Balancing – Line-by-Line Explanation

Overview

This document provides a detailed line-by-line explanation of the Load Balancing pattern implementation.

---

Pattern Description

Pattern 090: Load Balancing Pattern

Description: The Load Balancing Pattern distributes requests across multiple LLM endpoints or instances to achieve high availability, improved performance, and fault tolerance. In production LLM applications, load balancing is critical for handling high traffic volumes, preventing single points of failure, and ensuring consistent service quality even when individual endpoints fail or become overloaded.

Load balancing strategies enable:
- High availability through redundancy
- Improved performance via parallel processing
- Cost optimization through provider selection
- Rate limit management across multiple accounts
- Geographic distribution for lower latency
- Failover and disaster recovery

Components: 1. Load Balancing Strategies – Round Robin: Distribute equally across endpoints – Weighted Round Robin: Priority-based distribution – Least Loaded: Send to endpoint with lowest load – Random: Random endpoint selection – Geographic: Route based on location – Cost-Based: Choose cheapest available endpoint

2. Health Checking
   - Endpoint availability monitoring
   - Response time tracking
   - Error rate monitoring
   - Periodic health probes
   - Automatic endpoint removal/recovery

3. Request Management
   - Request queuing and buffering
   - Retry logic with backoff
   - Circuit breaker pattern
   - Request timeout handling
   - Priority-based routing

4. Failover Mechanisms
   - Automatic endpoint switching
   - Degraded mode operation
   - Fallback to alternative providers
   - Request replay on failure
   - State preservation

Use Cases: 1. High-Traffic Applications – Chatbots serving thousands of users – API services with SLA requirements – Real-time processing pipelines – Multi-tenant platforms – Peak load handling

2. Multi-Provider Deployments
   - OpenAI + Anthropic + Google
   - Primary and backup providers
   - Cost optimization strategies
   - Feature-specific routing
   - Geographic compliance

3. Rate Limit Management
   - Distribute across multiple API keys
   - Prevent rate limit exhaustion
   - Smooth request distribution
   - Burst handling
   - Cost control

4. Disaster Recovery
   - Automatic failover
   - Multi-region deployment
   - Zero-downtime updates
   - Service degradation handling
   - Business continuity

5. Performance Optimization
   - Parallel request processing
   - Load-based routing
   - Latency optimization
   - Resource utilization
   - Capacity planning

LangChain Implementation: LangChain doesn’t have built-in load balancing, but you can implement it using: – Custom LLM wrappers – Callback handlers for monitoring – Retry logic with different models – Custom routing logic – Health check mechanisms

Key Features: 1. Endpoint Management – Dynamic endpoint registration – Health status tracking – Automatic recovery – Configuration updates – Version management

2. Intelligent Routing
   - Strategy selection
   - Real-time decision making
   - Performance-based routing
   - Cost optimization
   - SLA enforcement

3. Fault Tolerance
   - Automatic retry with exponential backoff
   - Circuit breaker to prevent cascading failures
   - Graceful degradation
   - Error handling and recovery
   - Request preservation

4. Monitoring
   - Request distribution metrics
   - Endpoint health status
   - Response time tracking
   - Error rate monitoring
   - Cost tracking per endpoint

Best Practices: 1. Health Monitoring – Regular health checks (every 30-60s) – Multiple health metrics (latency, errors, availability) – Automatic endpoint recovery – Alert on degraded performance – Log all health transitions

2. Load Distribution
   - Consider endpoint capacity
   - Account for different model speeds
   - Balance cost vs performance
   - Avoid overloading any single endpoint
   - Reserve capacity for spikes

3. Failover Strategy
   - Fast failure detection (< 5s)
   - Quick failover (< 1s)
   - Preserve request context
   - Transparent to users
   - Log all failovers

4. Request Management
   - Set appropriate timeouts
   - Implement retry with backoff
   - Queue management for spikes
   - Priority handling
   - Request deduplication

Trade-offs: Advantages: – High availability (99.9%+ uptime) – Better performance through parallelization – Fault tolerance and resilience – Flexible scaling – Cost optimization opportunities – Geographic distribution

Disadvantages:
- Increased complexity
- Additional monitoring requirements
- Potential consistency issues
- Higher operational costs
- Configuration management overhead
- Testing complexity

Production Considerations: 1. Endpoint Configuration – Multiple providers (OpenAI, Anthropic, etc.) – Multiple API keys per provider – Geographic distribution – Capacity planning – Cost monitoring

2. Monitoring and Alerting
   - Real-time health dashboards
   - Performance metrics
   - Error tracking
   - Cost tracking
   - SLA compliance monitoring

3. Security
   - Secure credential management
   - Rate limiting per client
   - Access control
   - Audit logging
   - Encryption in transit

4. Scaling
   - Horizontal scaling capability
   - Auto-scaling based on load
   - Capacity planning
   - Performance testing
   - Load testing

5. Disaster Recovery
   - Multi-region deployment
   - Backup endpoints
   - Failover procedures
   - Data consistency
   - Recovery time objectives (RTO)

---

File Statistics

• Total Lines: 988
• Classes: 7
• Functions: 1
• Constants: 0

---

Module Documentation

"""
Pattern 090: Load Balancing Pattern

Description:
    The Load Balancing Pattern distributes requests across multiple LLM endpoints or
    instances to achieve high availability, improved performance, and fault tolerance.
    In production LLM applications, load balancing is critical for handling high traffic
    volumes, preventing single points of failure, and ensuring consistent service
    quality even when individual endpoints fail or become overloaded.

    Load balancing strategies enable:
    - High availability through redundancy
    - Improved performance via parallel processing
    - Cost optimization through provider selection
    - Rate limit management across multiple accounts
    - Geographic distribution for lower latency
    - Failover and disaster recovery

Components:
    1. Load Balancing Strategies
       - Round Robin: Distribute equally across endpoints
       - Weighted Round Robin: Priority-based distribution
       - Least Loaded: Send to endpoint with lowest load
       - Random: Random endpoint selection
       - Geographic: Route based on location
       - Cost-Based: Choose cheapest available endpoint

    2. Health Checking
       - Endpoint availability monitoring
       - Response time tracking
       - Error rate monitoring
       - Periodic health probes
       - Automatic endpoint removal/recovery

    3. Request Management
       - Request queuing and buffering
       - Retry logic with backoff
       - Circuit breaker pattern
       - Request timeout handling
       - Priority-based routing

    4. Failover Mechanisms
       - Automatic endpoint switching
       - Degraded mode operation
       - Fallback to alternative providers
       - Request replay on failure
       - State preservation

Use Cases:
    1. High-Traffic Applications
       - Chatbots serving thousands of users
       - API services with SLA requirements
       - Real-time processing pipelines
       - Multi-tenant platforms
       - Peak load handling

    2. Multi-Provider Deployments
       - OpenAI + Anthropic + Google
       - Primary and backup providers
       - Cost optimization strategies
       - Feature-specific routing
       - Geographic compliance

    3. Rate Limit Management
       - Distribute across multiple API keys
       - Prevent rate limit exhaustion
       - Smooth request distribution
       - Burst handling
       - Cost control

    4. Disaster Recovery
       - Automatic failover
       - Multi-region deployment
       - Zero-downtime updates
       - Service degradation handling
       - Business continuity

    5. Performance Optimization
       - Parallel request processing
       - Load-based routing
       - Latency optimization
       - Resource utilization
       - Capacity planning

LangChain Implementation:
    LangChain doesn't have built-in load balancing, but you can implement
    it using:
    - Custom LLM wrappers
    - Callback handlers for monitoring
    - Retry logic with different models
    - Custom routing logic
    - Health check mechanisms

Key Features:
    1. Endpoint Management
       - Dynamic endpoint registration
       - Health status tracking
       - Automatic recovery
       - Configuration updates
       - Version management

    2. Intelligent Routing
       - Strategy selection
       - Real-time decision making
       - Performance-based routing
       - Cost optimization
       - SLA enforcement

    3. Fault Tolerance
       - Automatic retry with exponential backoff
       - Circuit breaker to prevent cascading failures
       - Graceful degradation
       - Error handling and recovery
       - Request preservation

    4. Monitoring
       - Request distribution metrics
       - Endpoint health status
       - Response time tracking
       - Error rate monitoring
       - Cost tracking per endpoint

Best Practices:
    1. Health Monitoring
       - Regular health checks (every 30-60s)
       - Multiple health metrics (latency, errors, availability)
       - Automatic endpoint recovery
       - Alert on degraded performance
       - Log all health transitions

    2. Load Distribution
       - Consider endpoint capacity
       - Account for different model speeds
       - Balance cost vs performance
       - Avoid overloading any single endpoint
       - Reserve capacity for spikes

    3. Failover Strategy
       - Fast failure detection (< 5s)
       - Quick failover (< 1s)
       - Preserve request context
       - Transparent to users
       - Log all failovers

    4. Request Management
       - Set appropriate timeouts
       - Implement retry with backoff
       - Queue management for spikes
       - Priority handling
       - Request deduplication

Trade-offs:
    Advantages:
    - High availability (99.9%+ uptime)
    - Better performance through parallelization
    - Fault tolerance and resilience
    - Flexible scaling
    - Cost optimization opportunities
    - Geographic distribution

    Disadvantages:
    - Increased complexity
    - Additional monitoring requirements
    - Potential consistency issues
    - Higher operational costs
    - Configuration management overhead
    - Testing complexity

Production Considerations:
    1. Endpoint Configuration
       - Multiple providers (OpenAI, Anthropic, etc.)
       - Multiple API keys per provider
       - Geographic distribution
       - Capacity planning
       - Cost monitoring

    2. Monitoring and Alerting
       - Real-time health dashboards
       - Performance metrics
       - Error tracking
       - Cost tracking
       - SLA compliance monitoring

    3. Security
       - Secure credential management
       - Rate limiting per client
       - Access control
       - Audit logging
       - Encryption in transit

    4. Scaling
       - Horizontal scaling capability
       - Auto-scaling based on load
       - Capacity planning
       - Performance testing
       - Load testing

    5. Disaster Recovery
       - Multi-region deployment
       - Backup endpoints
       - Failover procedures
       - Data consistency
       - Recovery time objectives (RTO)
"""

Purpose: This module implements the Load Balancing pattern, providing a structured approach to agent design and implementation.

---

Import Statements

Standard Library Imports

import collections.deque
import dataclasses.dataclass
import dataclasses.field
import datetime.datetime
import datetime.timedelta
import enum.Enum
import langchain_core.prompts.ChatPromptTemplate
import os
import random
import time
import typing.Any
import typing.Callable
import typing.Dict
import typing.List
import typing.Optional

Purpose: Core Python functionality: – collections.deque: Specialized container datatypes – dataclasses.dataclass: Simplified class definitions with @dataclass – dataclasses.field: Simplified class definitions with @dataclass – datetime.datetime: Date and time handling – datetime.timedelta: Date and time handling – enum.Enum: Enumeration support – langchain_core.prompts.ChatPromptTemplate: Regular expression operations – os: Environment variable access and system operations – random: Random number generation – time: Time-related functions – typing.Any: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Callable: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Dict: Type hints for better code documentation (List, Dict, Any, Optional) – typing.List: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Optional: Type hints for better code documentation (List, Dict, Any, Optional)

LangChain Imports

from langchain_openai.ChatOpenAI

Purpose: LangChain framework components: – LLM Interface: Wrapper for OpenAI models – Chains: Composable building blocks for complex workflows

Other Imports

from dotenv.load_dotenv

Purpose: Additional dependencies for specialized functionality.

---

Classes

Class: LoadBalancingStrategy

Lines 222-228

Inherits from: Enum

Purpose:

Load balancing strategies

---

Class: EndpointStatus

Lines 231-236

Inherits from: Enum

Purpose:

Endpoint health status

---

Class: EndpointConfig

Lines 240-248

Purpose:

Configuration for an LLM endpoint

---

Class: EndpointMetrics

Lines 252-275

Purpose:

Metrics for an endpoint

Methods:

success_rate(self)

Lines 264-268

Decorators: property

Purpose:

Calculate success rate

Parameters:

Returns: float

average_response_time(self)

Lines 271-275

Decorators: property

Purpose:

Calculate average response time

Parameters:

Returns: float

---

Class: LoadBalancerStats

Lines 279-286

Purpose:

Overall load balancer statistics

---

Class: LLMEndpoint

Lines 289-414

Purpose:

Represents a single LLM endpoint with health tracking.

Methods:

__init__(self, config)

Lines 294-307

Purpose:

Initialize endpoint.

Args:
    config: Endpoint configuration

Parameters: – config: Config

_initialize_llm(self)

Lines 309-326

Purpose:

Initialize the LLM client

Parameters:

is_available(self)

Lines 328-334

Purpose:

Check if endpoint is available for requests

Parameters:

Returns: bool

check_health(self)

Lines 336-367

Purpose:

Perform health check on endpoint.

Returns:
    Current health status

Parameters:

Returns: EndpointStatus

generate(self, prompt)

Lines 369-414

Purpose:

Generate response using this endpoint.

Args:
    prompt: Input prompt

Returns:
    Response dictionary

Parameters: – prompt: Prompt template or text

Returns: <ast.Subscript object at 0x000001A06D7523D0>

---

Class: LoadBalancer

Lines 417-622

Purpose:

Load balancer for distributing requests across multiple LLM endpoints.

Supports multiple load balancing strategies and automatic failover.

Methods:

__init__(self, endpoints, strategy, health_check_interval)

Lines 424-449

Purpose:

Initialize load balancer.

Args:
    endpoints: List of endpoint configurations
    strategy: Load balancing strategy
    health_check_interval: Seconds between health checks

Parameters: – endpoints: Endpoints – strategy: Strategy – health_check_interval: Health check interval

_select_endpoint_round_robin(self)

Lines 451-463

Purpose:

Select endpoint using round-robin strategy

Parameters:

Returns: <ast.Subscript object at 0x000001A06D8881D0>

_select_endpoint_weighted(self)

Lines 465-479

Purpose:

Select endpoint using weighted round-robin strategy

Parameters:

Returns: <ast.Subscript object at 0x000001A06D88B410>

_select_endpoint_least_loaded(self)

Lines 481-487

Purpose:

Select endpoint with least active requests

Parameters:

Returns: <ast.Subscript object at 0x000001A06D75B250>

_select_endpoint_random(self)

Lines 489-495

Purpose:

Select random available endpoint

Parameters:

Returns: <ast.Subscript object at 0x000001A06D75A790>

_select_endpoint_fastest(self)

Lines 497-507

Purpose:

Select endpoint with fastest average response time

Parameters:

Returns: <ast.Subscript object at 0x000001A06D80CF90>

_select_endpoint(self)

Lines 509-527

Purpose:

Select endpoint based on strategy.

Returns:
    Selected endpoint or None if none available

Parameters:

Returns: <ast.Subscript object at 0x000001A06D157C90>

_perform_health_checks(self)

Lines 529-538

Purpose:

Perform health checks on all endpoints if needed

Parameters:

generate(self, prompt, max_retries)

Lines 540-593

Purpose:

Generate response with load balancing and failover.

Args:
    prompt: Input prompt
    max_retries: Maximum retry attempts

Returns:
    Response dictionary

Parameters: – prompt: Prompt template or text – max_retries: Max retries

Returns: <ast.Subscript object at 0x000001A06D83E910>

get_stats(self)

Lines 595-622

Purpose:

Get load balancer statistics

Parameters:

Returns: <ast.Subscript object at 0x000001A06D850690>

---

Functions

demonstrate_load_balancing()

Lines 625-983

Purpose:

Demonstrate load balancing patterns

---

Key Concepts Summary

Pattern Overview

The Load Balancing pattern provides a structured approach to building AI agents with specific capabilities and behaviors. This implementation leverages LangChain’s composable components to create a robust and maintainable solution.

Core Components

1. Agent Class: Main implementation of the pattern logic
2. Prompt Engineering: Carefully crafted prompts to guide LLM behavior
3. Tool Integration: Optional tools for extended functionality
4. Memory Management: State handling across interactions
5. Output Parsing: Structured response processing

When to Use

✅ Good for: – Scenarios requiring this specific pattern’s strengths – Applications needing structured agent behavior – Systems where explainability is important – Production environments requiring reliability

❌ Not ideal for: – Simple single-prompt tasks – Scenarios requiring different pattern characteristics – Resource-constrained environments (if computationally intensive)

---

Best Practices Demonstrated

1. Environment Configuration

• ✅ Use .env files for API keys
• ✅ Never hardcode credentials
• ✅ Validate environment variables on startup

2. Error Handling

• ✅ Graceful degradation on failures
• ✅ Informative error messages
• ✅ Logging for debugging

3. Prompt Engineering

• ✅ Clear, specific instructions
• ✅ Structured output formats
• ✅ Few-shot examples when beneficial

4. Code Organization

• ✅ Separation of concerns (class-based design)
• ✅ Type hints for clarity
• ✅ Docstrings for documentation

5. Performance Optimization

• ✅ Appropriate temperature settings
• ✅ Token budget management
• ✅ Caching where applicable

---

LangChain Components Used

• ChatOpenAI: OpenAI model interface
• Chains: Sequential component composition

Why LangChain? – Composable abstractions for LLM applications – Rich ecosystem of integrations – Production-ready patterns – Active community and development

---

Execution Flow

1. Initialize agent with configuration
   ↓
2. Load environment and dependencies
   ↓
3. Prepare prompt with input data
   ↓
4. Invoke LLM through LangChain
   ↓
5. Parse and validate response
   ↓
6. Return formatted result

Key Decision Points: – Model selection (temperature, model type) – Prompt structure (few-shot vs zero-shot) – Error handling strategy – Output format validation

---

Performance Considerations

Computational Cost

• LLM Calls: Primary cost factor (based on tokens)
• Latency: Depends on model and input size
• Memory: Minimal for most patterns

Optimization Strategies

1. Prompt Optimization: Minimize tokens while maintaining clarity
2. Caching: Cache repeated queries/responses
3. Model Selection: Use smaller models when appropriate
4. Batch Processing: Group similar requests
5. Streaming: For long responses, stream tokens

Cost Estimates (Approximate)

• GPT-3.5-turbo: ~$0.002 per request
• GPT-4: ~$0.03 per request
• GPT-4-turbo: ~$0.01 per request

Actual costs vary based on input/output token counts

---

Common Pitfalls to Avoid

1. Prompt Design Issues

❌ Vague or ambiguous instructions
✅ Clear, specific, testable prompts

2. Error Handling

❌ Ignoring API errors or rate limits
✅ Implement retry logic and graceful degradation

3. Token Management

❌ Exceeding model token limits
✅ Truncate inputs, chunk processing

4. Security

❌ Exposing API keys in code
✅ Use environment variables

5. Testing

❌ Only testing happy paths
✅ Test edge cases, failures, malformed inputs

6. Cost Management

❌ Unmonitored API usage
✅ Implement budgets, alerts, and monitoring

---

Further Improvements

Production Enhancements

1. Monitoring: Add logging, metrics, tracing
2. Validation: Input/output schema validation
3. Rate Limiting: Implement request throttling
4. Caching: Redis or in-memory cache for responses
5. Async Support: Non-blocking operations for scalability

Feature Extensions

1. Multi-Model Support: Switch between providers
2. Fallback Models: Graceful degradation strategy
3. Custom Tools: Domain-specific function calling
4. Advanced Memory: Vector stores for long-term memory
5. Evaluation: Automated testing and benchmarking

Code Quality

1. Testing: Unit tests, integration tests
2. Documentation: API docs, usage examples
3. Type Safety: Stricter type hints
4. Configuration: Externalized config files

---

Related Patterns

• Pattern 089
• Pattern 091

Pattern Families: – Reasoning Patterns (001-011) – Multi-Agent Patterns (012-019) – Tool Use Patterns (020-025) – Memory Patterns (026-031) – And many more…

---

Summary

The Load Balancing pattern provides a powerful approach to building AI agents with specific capabilities. This implementation demonstrates:

✅ Key Strengths: – Clean, maintainable code structure – LangChain integration for composability – Type hints and documentation – Production-ready patterns

🎯 Best Use Cases: – Applications requiring this pattern’s characteristics – Systems needing structured agent behavior – Production environments requiring reliability

📚 Learning Resources: – LangChain Documentation: https://python.langchain.com/ – Pattern Documentation: See related patterns – OpenAI API Reference: https://platform.openai.com/docs/

---

File: 090_load_balancing.py
Total Lines: 988
Pattern Category: AI Agent Design Pattern #090

---

Generated from actual implementation code
For educational and reference purposes

    """
Pattern 089: Caching Patterns

Description:
    The Caching Patterns involve storing and reusing previously computed results to
    improve performance, reduce latency, and minimize costs. In the context of LLM
    applications, caching is critical for optimizing response times and reducing API
    calls for repeated or similar queries. Effective caching strategies can
    dramatically improve user experience while significantly cutting operational costs.

    Caching in LLM applications targets:
    - Complete responses (exact match caching)
    - Embeddings (vector representations)
    - Prompt templates (reusable structures)
    - Intermediate results (multi-step workflows)
    - API responses (rate limit optimization)

Components:
    1. Cache Types
       - In-memory cache (fast, volatile)
       - Disk cache (persistent, slower)
       - Distributed cache (Redis, Memcached)
       - Database cache (structured, queryable)
       - CDN cache (geographic distribution)

    2. Cache Keys
       - Exact match (full prompt hash)
       - Semantic match (embedding similarity)
       - Parametric keys (template + parameters)
       - Hierarchical keys (nested cache structures)
       - Time-based keys (temporal validity)

    3. Cache Policies
       - TTL (Time To Live) - expire after time
       - LRU (Least Recently Used) - evict old entries
       - LFU (Least Frequently Used) - evict unpopular
       - Size-based - limit total cache size
       - Custom policies - application-specific

    4. Invalidation Strategies
       - Time-based expiration
       - Event-triggered invalidation
       - Manual invalidation
       - Cascade invalidation (dependencies)
       - Version-based invalidation

Use Cases:
    1. Repeated Queries
       - FAQ responses
       - Common questions
       - Popular content
       - Standard procedures
       - Frequently accessed data

    2. Expensive Operations
       - Complex reasoning chains
       - Multi-step workflows
       - Large context processing
       - Batch operations
       - Resource-intensive computations

    3. Rate Limit Optimization
       - Reduce API calls
       - Stay within quotas
       - Handle traffic spikes
       - Cost optimization
       - Performance consistency

    4. Embeddings and Vectors
       - Document embeddings
       - Query embeddings
       - Semantic search optimization
       - Similarity computations
       - Vector database queries

    5. Multi-User Applications
       - Shared responses
       - Common workflows
       - Popular queries
       - Template responses
       - Collaborative features

LangChain Implementation:
    LangChain supports caching through:
    - InMemoryCache for simple caching
    - SQLiteCache for persistent caching
    - Custom cache implementations
    - Semantic similarity caching
    - Embedding caching

Key Features:
    1. Multiple Cache Layers
       - L1: In-memory (fastest)
       - L2: Local disk (persistent)
       - L3: Distributed cache (shared)
       - L4: Database (long-term)

    2. Intelligent Matching
       - Exact match for identical queries
       - Semantic match for similar queries
       - Fuzzy match with thresholds
       - Parametric templates

    3. Performance Optimization
       - Fast lookup ( 70%)
       - Measure latency improvements
       - Calculate cost savings
       - Identify cache misses patterns

    4. Handle Cache Misses
       - Graceful fallback to computation
       - Transparent to users
       - Background cache warming
       - Predictive pre-caching

Trade-offs:
    Advantages:
    - Dramatically improved response times
    - Significant cost reduction
    - Better user experience
    - Reduced API rate limiting
    - Lower resource consumption
    - Improved scalability

    Disadvantages:
    - Stale data risk
    - Memory/storage overhead
    - Cache consistency challenges
    - Complexity in invalidation
    - Initial cache warming time
    - Debugging complexity

Production Considerations:
    1. Cache Size Management
       - Monitor memory usage
       - Implement eviction policies
       - Set size limits
       - Regular cleanup
       - Compression strategies

    2. Cache Consistency
       - Version tracking
       - Update propagation
       - Invalidation timing
       - Distributed consistency
       - Race condition handling

    3. Security
       - Access control for cached data
       - Encryption for sensitive content
       - Cache poisoning prevention
       - User isolation in multi-tenant
       - Audit trails

    4. Monitoring
       - Hit/miss rates
       - Cache size trends
       - Eviction frequency
       - Cost savings
       - Performance impact

    5. Disaster Recovery
       - Cache rebuild procedures
       - Backup strategies
       - Failover mechanisms
       - Data loss handling
       - Recovery time objectives
"""

import os
import time
import hashlib
import json
from typing import List, Dict, Any, Optional, Callable
from dataclasses import dataclass, field
from enum import Enum
from datetime import datetime, timedelta
from collections import OrderedDict
from dotenv import load_dotenv

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_openai import ChatOpenAI, OpenAIEmbeddings
from langchain_core.caches import InMemoryCache
from langchain.globals import set_llm_cache

load_dotenv()


class CachePolicy(Enum):
    """Cache eviction policies"""
    LRU = "lru"  # Least Recently Used
    LFU = "lfu"  # Least Frequently Used
    FIFO = "fifo"  # First In First Out
    TTL = "ttl"  # Time To Live


@dataclass
class CacheEntry:
    """Represents a cached entry"""
    key: str
    value: Any
    created_at: datetime
    last_accessed: datetime
    access_count: int = 0
    ttl: Optional[int] = None  # TTL in seconds
    
    def is_expired(self) -> bool:
        """Check if entry is expired"""
        if self.ttl is None:
            return False
        age = (datetime.now() - self.created_at).total_seconds()
        return age > self.ttl
    
    def touch(self):
        """Update access information"""
        self.last_accessed = datetime.now()
        self.access_count += 1


@dataclass
class CacheStats:
    """Cache performance statistics"""
    hits: int = 0
    misses: int = 0
    evictions: int = 0
    total_requests: int = 0
    
    @property
    def hit_rate(self) -> float:
        """Calculate cache hit rate"""
        if self.total_requests == 0:
            return 0.0
        return (self.hits / self.total_requests) * 100
    
    @property
    def miss_rate(self) -> float:
        """Calculate cache miss rate"""
        return 100.0 - self.hit_rate


class SimpleCache:
    """
    Simple in-memory cache with multiple eviction policies.
    
    This cache supports:
    1. Multiple eviction policies (LRU, LFU, FIFO, TTL)
    2. Size limits
    3. TTL per entry
    4. Performance statistics
    """
    
    def __init__(
        self,
        max_size: int = 1000,
        policy: CachePolicy = CachePolicy.LRU,
        default_ttl: Optional[int] = None
    ):
        """
        Initialize cache.
        
        Args:
            max_size: Maximum number of entries
            policy: Eviction policy
            default_ttl: Default TTL in seconds
        """
        self.max_size = max_size
        self.policy = policy
        self.default_ttl = default_ttl
        self.cache: OrderedDict[str, CacheEntry] = OrderedDict()
        self.stats = CacheStats()
    
    def _make_key(self, key: Any) -> str:
        """Create cache key from input"""
        if isinstance(key, str):
            return hashlib.md5(key.encode()).hexdigest()
        return hashlib.md5(json.dumps(key, sort_keys=True).encode()).hexdigest()
    
    def _evict(self):
        """Evict entries based on policy"""
        if len(self.cache)  Optional[Any]:
        """
        Get value from cache.
        
        Args:
            key: Cache key
            
        Returns:
            Cached value or None
        """
        self.stats.total_requests += 1
        cache_key = self._make_key(key)
        
        # Remove expired entries
        self._remove_expired()
        
        if cache_key in self.cache:
            entry = self.cache[cache_key]
            
            if entry.is_expired():
                del self.cache[cache_key]
                self.stats.misses += 1
                return None
            
            entry.touch()
            
            # Move to end for LRU
            if self.policy == CachePolicy.LRU:
                self.cache.move_to_end(cache_key)
            
            self.stats.hits += 1
            return entry.value
        
        self.stats.misses += 1
        return None
    
    def set(self, key: Any, value: Any, ttl: Optional[int] = None):
        """
        Set value in cache.
        
        Args:
            key: Cache key
            value: Value to cache
            ttl: Optional TTL in seconds
        """
        cache_key = self._make_key(key)
        
        # Evict if necessary
        if cache_key not in self.cache:
            self._evict()
        
        entry = CacheEntry(
            key=cache_key,
            value=value,
            created_at=datetime.now(),
            last_accessed=datetime.now(),
            ttl=ttl or self.default_ttl
        )
        
        self.cache[cache_key] = entry
        
        # Move to end for LRU
        if self.policy == CachePolicy.LRU:
            self.cache.move_to_end(cache_key)
    
    def clear(self):
        """Clear all cache entries"""
        self.cache.clear()
        self.stats = CacheStats()
    
    def size(self) -> int:
        """Get current cache size"""
        return len(self.cache)
    
    def get_stats(self) -> Dict[str, Any]:
        """Get cache statistics"""
        return {
            "hits": self.stats.hits,
            "misses": self.stats.misses,
            "evictions": self.stats.evictions,
            "total_requests": self.stats.total_requests,
            "hit_rate": f"{self.stats.hit_rate:.2f}%",
            "miss_rate": f"{self.stats.miss_rate:.2f}%",
            "current_size": len(self.cache),
            "max_size": self.max_size
        }


class SemanticCache:
    """
    Semantic cache using embeddings for similarity matching.
    
    This cache finds similar queries using vector similarity,
    allowing cache hits for semantically similar but not identical queries.
    """
    
    def __init__(
        self,
        similarity_threshold: float = 0.95,
        max_size: int = 500
    ):
        """
        Initialize semantic cache.
        
        Args:
            similarity_threshold: Minimum similarity for cache hit
            max_size: Maximum cache entries
        """
        self.similarity_threshold = similarity_threshold
        self.max_size = max_size
        self.embeddings = OpenAIEmbeddings()
        self.cache: List[Dict[str, Any]] = []
        self.stats = CacheStats()
    
    def _compute_similarity(self, query1: str, query2: str) -> float:
        """
        Compute semantic similarity between queries.
        
        Args:
            query1: First query
            query2: Second query
            
        Returns:
            Similarity score (0-1)
        """
        # Simple implementation - in production use precomputed embeddings
        # This is for demonstration only
        query1_lower = query1.lower()
        query2_lower = query2.lower()
        
        # Simple word overlap similarity
        words1 = set(query1_lower.split())
        words2 = set(query2_lower.split())
        
        if not words1 or not words2:
            return 0.0
        
        intersection = words1.intersection(words2)
        union = words1.union(words2)
        
        return len(intersection) / len(union)
    
    def get(self, query: str) -> Optional[Any]:
        """
        Get value for semantically similar query.
        
        Args:
            query: Query string
            
        Returns:
            Cached value or None
        """
        self.stats.total_requests += 1
        
        # Find most similar cached query
        best_match = None
        best_similarity = 0.0
        
        for entry in self.cache:
            similarity = self._compute_similarity(query, entry["query"])
            if similarity > best_similarity:
                best_similarity = similarity
                best_match = entry
        
        if best_match and best_similarity >= self.similarity_threshold:
            self.stats.hits += 1
            # Update access time
            best_match["last_accessed"] = datetime.now()
            return best_match["value"]
        
        self.stats.misses += 1
        return None
    
    def set(self, query: str, value: Any):
        """
        Cache value for query.
        
        Args:
            query: Query string
            value: Value to cache
        """
        # Evict oldest if at capacity
        if len(self.cache) >= self.max_size:
            self.cache.pop(0)
            self.stats.evictions += 1
        
        self.cache.append({
            "query": query,
            "value": value,
            "created_at": datetime.now(),
            "last_accessed": datetime.now()
        })
    
    def clear(self):
        """Clear cache"""
        self.cache.clear()
        self.stats = CacheStats()
    
    def get_stats(self) -> Dict[str, Any]:
        """Get cache statistics"""
        return {
            "hits": self.stats.hits,
            "misses": self.stats.misses,
            "evictions": self.stats.evictions,
            "total_requests": self.stats.total_requests,
            "hit_rate": f"{self.stats.hit_rate:.2f}%",
            "current_size": len(self.cache),
            "max_size": self.max_size,
            "similarity_threshold": self.similarity_threshold
        }


class CachedLLMAgent:
    """
    LLM agent with caching support.
    
    This agent automatically caches responses and retrieves them
    for repeated or similar queries.
    """
    
    def __init__(
        self,
        cache: SimpleCache,
        model: str = "gpt-3.5-turbo"
    ):
        """
        Initialize cached agent.
        
        Args:
            cache: Cache instance to use
            model: LLM model name
        """
        self.cache = cache
        self.llm = ChatOpenAI(model=model, temperature=0.7)
    
    def generate(self, prompt: str) -> Dict[str, Any]:
        """
        Generate response with caching.
        
        Args:
            prompt: Input prompt
            
        Returns:
            Dictionary with response and cache info
        """
        # Check cache first
        cached_response = self.cache.get(prompt)
        
        if cached_response is not None:
            return {
                "response": cached_response,
                "from_cache": True,
                "cache_stats": self.cache.get_stats()
            }
        
        # Generate new response
        start_time = time.time()
        response = self.llm.invoke(prompt)
        generation_time = time.time() - start_time
        
        # Cache the response
        self.cache.set(prompt, response.content)
        
        return {
            "response": response.content,
            "from_cache": False,
            "generation_time": generation_time,
            "cache_stats": self.cache.get_stats()
        }


def demonstrate_caching_patterns():
    """Demonstrate caching patterns"""
    print("=" * 80)
    print("CACHING PATTERNS DEMONSTRATION")
    print("=" * 80)
    
    # Example 1: Basic LRU Cache
    print("\n" + "=" * 80)
    print("EXAMPLE 1: Basic LRU Cache")
    print("=" * 80)
    
    cache = SimpleCache(max_size=5, policy=CachePolicy.LRU)
    agent = CachedLLMAgent(cache)
    
    print("\nMaking requests (first time - cache misses):\n")
    
    queries = [
        "What is Python?",
        "What is JavaScript?",
        "What is Python?",  # Repeat
        "What is Java?",
        "What is Python?",  # Repeat again
    ]
    
    for i, query in enumerate(queries, 1):
        print(f"Request {i}: {query}")
        result = agent.generate(query)
        
        cache_status = "HIT ✓" if result["from_cache"] else "MISS ✗"
        print(f"  Status: {cache_status}")
        
        if not result["from_cache"]:
            print(f"  Generation time: {result['generation_time']:.2f}s")
        print(f"  Response: {result['response'][:80]}...")
        print()
    
    stats = cache.get_stats()
    print("Cache Statistics:")
    print(f"  Hit Rate: {stats['hit_rate']}")
    print(f"  Total Requests: {stats['total_requests']}")
    print(f"  Hits: {stats['hits']}")
    print(f"  Misses: {stats['misses']}")
    
    # Example 2: TTL-Based Caching
    print("\n" + "=" * 80)
    print("EXAMPLE 2: TTL-Based Caching (Time Expiration)")
    print("=" * 80)
    
    ttl_cache = SimpleCache(max_size=100, default_ttl=5)  # 5 second TTL
    
    print("\nCaching value with 5-second TTL...")
    ttl_cache.set("test_key", "test_value", ttl=5)
    
    print("Immediate retrieval:")
    value = ttl_cache.get("test_key")
    print(f"  Result: {'HIT' if value else 'MISS'} - {value}")
    
    print("\nWaiting 6 seconds for expiration...")
    time.sleep(6)
    
    print("Retrieval after TTL:")
    value = ttl_cache.get("test_key")
    print(f"  Result: {'HIT' if value else 'MISS'} - {value}")
    
    # Example 3: Cache Eviction Policies
    print("\n" + "=" * 80)
    print("EXAMPLE 3: Cache Eviction Policies Comparison")
    print("=" * 80)
    
    policies = [CachePolicy.LRU, CachePolicy.LFU, CachePolicy.FIFO]
    
    for policy in policies:
        print(f"\n{policy.value.upper()} Policy:")
        test_cache = SimpleCache(max_size=3, policy=policy)
        
        # Add 5 items to cache of size 3
        for i in range(5):
            key = f"item_{i}"
            test_cache.set(key, f"value_{i}")
            print(f"  Added {key}, cache size: {test_cache.size()}")
        
        stats = test_cache.get_stats()
        print(f"  Evictions: {stats['evictions']}")
    
    # Example 4: Semantic Cache
    print("\n" + "=" * 80)
    print("EXAMPLE 4: Semantic Cache (Similar Query Matching)")
    print("=" * 80)
    
    semantic_cache = SemanticCache(similarity_threshold=0.7, max_size=100)
    
    # Cache a response
    original_query = "What is machine learning?"
    original_response = "Machine learning is a subset of AI that enables systems to learn from data."
    semantic_cache.set(original_query, original_response)
    
    print(f"\nCached query: '{original_query}'")
    print(f"Response: {original_response}\n")
    
    # Try similar queries
    similar_queries = [
        "What is machine learning?",  # Exact match
        "What's machine learning?",   # Slight variation
        "Tell me about machine learning",  # Similar
        "Explain artificial intelligence",  # Different topic
    ]
    
    for query in similar_queries:
        result = semantic_cache.get(query)
        status = "HIT ✓" if result else "MISS ✗"
        print(f"Query: '{query}'")
        print(f"  Status: {status}")
        if result:
            print(f"  Retrieved: {result}")
        print()
    
    # Example 5: Multi-Layer Cache
    print("\n" + "=" * 80)
    print("EXAMPLE 5: Multi-Layer Cache Strategy")
    print("=" * 80)
    
    # L1: Small, fast, in-memory cache
    l1_cache = SimpleCache(max_size=10, policy=CachePolicy.LRU)
    
    # L2: Larger, still in-memory
    l2_cache = SimpleCache(max_size=100, policy=CachePolicy.LRU)
    
    print("\nMulti-layer cache simulation:")
    print("  L1: Small (10 items), fast")
    print("  L2: Large (100 items), moderate\n")
    
    def multi_layer_get(key: str) -> Optional[str]:
        """Get from multi-layer cache"""
        # Try L1 first
        value = l1_cache.get(key)
        if value:
            print(f"  L1 HIT for '{key}'")
            return value
        
        # Try L2
        value = l2_cache.get(key)
        if value:
            print(f"  L2 HIT for '{key}'")
            # Promote to L1
            l1_cache.set(key, value)
            return value
        
        print(f"  MISS for '{key}'")
        return None
    
    # Simulate access pattern
    l1_cache.set("frequently_accessed", "hot_data")
    l2_cache.set("occasionally_accessed", "warm_data")
    
    print("Access patterns:")
    multi_layer_get("frequently_accessed")
    multi_layer_get("occasionally_accessed")
    multi_layer_get("never_cached")
    
    # Example 6: Cache Performance Metrics
    print("\n" + "=" * 80)
    print("EXAMPLE 6: Cache Performance Analysis")
    print("=" * 80)
    
    perf_cache = SimpleCache(max_size=50, policy=CachePolicy.LRU)
    perf_agent = CachedLLMAgent(perf_cache)
    
    print("\nSimulating workload with repeated queries...\n")
    
    workload = [
        "What is AI?",
        "What is ML?",
        "What is AI?",  # Repeat
        "What is DL?",
        "What is AI?",  # Repeat
        "What is ML?",  # Repeat
        "What is NLP?",
        "What is AI?",  # Repeat
    ]
    
    total_time_with_cache = 0
    total_time_without_cache = 0
    
    for query in workload:
        result = perf_agent.generate(query)
        
        if result["from_cache"]:
            total_time_with_cache += 0.001  # Assume 1ms for cache hit
        else:
            gen_time = result.get("generation_time", 1.0)
            total_time_with_cache += gen_time
            total_time_without_cache += gen_time
    
    # Estimate time without cache (all queries would need generation)
    total_time_without_cache = len(workload) * 1.0  # Assume 1s per generation
    
    print(f"Performance Analysis:")
    print(f"  Total Queries: {len(workload)}")
    print(f"  Time WITH cache: {total_time_with_cache:.2f}s")
    print(f"  Time WITHOUT cache (estimated): {total_time_without_cache:.2f}s")
    print(f"  Speedup: {total_time_without_cache/total_time_with_cache:.1f}x")
    print(f"  Time Saved: {total_time_without_cache - total_time_with_cache:.2f}s")
    
    final_stats = perf_cache.get_stats()
    print(f"\nFinal Cache Statistics:")
    print(f"  Hit Rate: {final_stats['hit_rate']}")
    print(f"  Hits: {final_stats['hits']}")
    print(f"  Misses: {final_stats['misses']}")
    
    # Summary
    print("\n" + "=" * 80)
    print("CACHING PATTERNS SUMMARY")
    print("=" * 80)
    print("""
Caching Pattern Benefits:
1. Performance: Dramatically faster response times (10-1000x)
2. Cost Reduction: Fewer API calls = lower costs
3. Scalability: Handle more requests with same resources
4. Reliability: Reduce dependency on external services
5. User Experience: Instant responses for cached queries

Cache Types Demonstrated:
1. Simple Cache: Basic in-memory caching with eviction policies
2. TTL Cache: Time-based expiration for freshness
3. Semantic Cache: Similarity-based matching for related queries
4. Multi-Layer Cache: Hierarchical caching strategy
5. Performance-Optimized: Focus on hit rates and speedup

Eviction Policies:
1. LRU (Least Recently Used): Remove oldest accessed items
2. LFU (Least Frequently Used): Remove least popular items
3. FIFO (First In First Out): Remove oldest items
4. TTL (Time To Live): Expire after time period

Best Practices:
1. Cache Strategy Selection:
   - Exact match: Simple, fast, for identical queries
   - Semantic: Better coverage, for similar queries
   - TTL: For time-sensitive data
   - Multi-layer: For diverse access patterns

2. TTL Configuration:
   - Short (minutes): Dynamic, frequently changing data
   - Medium (hours): Semi-static content
   - Long (days): Stable, rarely changing data
   - No TTL: Static reference data

3. Size Management:
   - Monitor memory usage
   - Set appropriate limits
   - Implement eviction policies
   - Regular cleanup

4. Performance Monitoring:
   - Target >70% hit rate for good performance
   - Track cache size trends
   - Monitor eviction frequency
   - Measure latency improvements

When to Use Caching:
✓ Repeated identical queries
✓ Similar semantic queries
✓ Expensive computations
✓ Rate-limited APIs
✓ Stable, unchanging data
✗ Real-time data requirements
✗ Highly personalized responses
✗ Security-sensitive information
✗ Rapidly changing data

Production Considerations:
- Distributed caching (Redis) for multi-instance deployments
- Cache warming strategies for cold starts
- Monitoring and alerting on hit rates
- Regular cache analysis and optimization
- Security: encrypt sensitive cached data
- Compliance: respect data retention policies
- Backup and recovery procedures
- Cache versioning for updates

Cost-Benefit Analysis:
- Typical API call: $0.001-0.01 per request
- Cache hit cost: ~$0.000001 per request
- Potential savings: 90-99% cost reduction
- Performance gain: 10-1000x faster responses
- ROI: Immediate for high-volume applications
""")
    
    print("\n" + "=" * 80)
    print("Pattern 089 (Caching Patterns) demonstration complete!")
    print("=" * 80)


if __name__ == "__main__":
    demonstrate_caching_patterns()

    

Agentic Design Pattern: Caching Patterns

Pattern 089: Caching Patterns – Line-by-Line Explanation

Overview

This document provides a detailed line-by-line explanation of the Caching Patterns pattern implementation.

---

Pattern Description

Pattern 089: Caching Patterns

Description: The Caching Patterns involve storing and reusing previously computed results to improve performance, reduce latency, and minimize costs. In the context of LLM applications, caching is critical for optimizing response times and reducing API calls for repeated or similar queries. Effective caching strategies can dramatically improve user experience while significantly cutting operational costs.

Caching in LLM applications targets:
- Complete responses (exact match caching)
- Embeddings (vector representations)
- Prompt templates (reusable structures)
- Intermediate results (multi-step workflows)
- API responses (rate limit optimization)

Components: 1. Cache Types – In-memory cache (fast, volatile) – Disk cache (persistent, slower) – Distributed cache (Redis, Memcached) – Database cache (structured, queryable) – CDN cache (geographic distribution)

2. Cache Keys
   - Exact match (full prompt hash)
   - Semantic match (embedding similarity)
   - Parametric keys (template + parameters)
   - Hierarchical keys (nested cache structures)
   - Time-based keys (temporal validity)

3. Cache Policies
   - TTL (Time To Live) - expire after time
   - LRU (Least Recently Used) - evict old entries
   - LFU (Least Frequently Used) - evict unpopular
   - Size-based - limit total cache size
   - Custom policies - application-specific

4. Invalidation Strategies
   - Time-based expiration
   - Event-triggered invalidation
   - Manual invalidation
   - Cascade invalidation (dependencies)
   - Version-based invalidation

Use Cases: 1. Repeated Queries – FAQ responses – Common questions – Popular content – Standard procedures – Frequently accessed data

2. Expensive Operations
   - Complex reasoning chains
   - Multi-step workflows
   - Large context processing
   - Batch operations
   - Resource-intensive computations

3. Rate Limit Optimization
   - Reduce API calls
   - Stay within quotas
   - Handle traffic spikes
   - Cost optimization
   - Performance consistency

4. Embeddings and Vectors
   - Document embeddings
   - Query embeddings
   - Semantic search optimization
   - Similarity computations
   - Vector database queries

5. Multi-User Applications
   - Shared responses
   - Common workflows
   - Popular queries
   - Template responses
   - Collaborative features

LangChain Implementation: LangChain supports caching through: – InMemoryCache for simple caching – SQLiteCache for persistent caching – Custom cache implementations – Semantic similarity caching – Embedding caching

Key Features: 1. Multiple Cache Layers – L1: In-memory (fastest) – L2: Local disk (persistent) – L3: Distributed cache (shared) – L4: Database (long-term)

2. Intelligent Matching
   - Exact match for identical queries
   - Semantic match for similar queries
   - Fuzzy match with thresholds
   - Parametric templates

3. Performance Optimization
   - Fast lookup (< 1ms for in-memory)
   - Efficient serialization
   - Compression for large entries
   - Lazy loading

4. Observability
   - Hit rate monitoring
   - Miss analysis
   - Performance metrics
   - Cost savings tracking

Best Practices: 1. Cache Appropriately – Cache stable, reusable results – Don’t cache rapidly changing data – Consider data sensitivity – Balance freshness vs performance

2. Set Proper TTLs
   - Shorter for dynamic content
   - Longer for stable content
   - Consider update frequency
   - Account for user expectations

3. Monitor Performance
   - Track hit rates (target > 70%)
   - Measure latency improvements
   - Calculate cost savings
   - Identify cache misses patterns

4. Handle Cache Misses
   - Graceful fallback to computation
   - Transparent to users
   - Background cache warming
   - Predictive pre-caching

Trade-offs: Advantages: – Dramatically improved response times – Significant cost reduction – Better user experience – Reduced API rate limiting – Lower resource consumption – Improved scalability

Disadvantages:
- Stale data risk
- Memory/storage overhead
- Cache consistency challenges
- Complexity in invalidation
- Initial cache warming time
- Debugging complexity

Production Considerations: 1. Cache Size Management – Monitor memory usage – Implement eviction policies – Set size limits – Regular cleanup – Compression strategies

2. Cache Consistency
   - Version tracking
   - Update propagation
   - Invalidation timing
   - Distributed consistency
   - Race condition handling

3. Security
   - Access control for cached data
   - Encryption for sensitive content
   - Cache poisoning prevention
   - User isolation in multi-tenant
   - Audit trails

4. Monitoring
   - Hit/miss rates
   - Cache size trends
   - Eviction frequency
   - Cost savings
   - Performance impact

5. Disaster Recovery
   - Cache rebuild procedures
   - Backup strategies
   - Failover mechanisms
   - Data loss handling
   - Recovery time objectives

---

File Statistics

• Total Lines: 897
• Classes: 6
• Functions: 1
• Constants: 0

---

Module Documentation

"""
Pattern 089: Caching Patterns

Description:
    The Caching Patterns involve storing and reusing previously computed results to
    improve performance, reduce latency, and minimize costs. In the context of LLM
    applications, caching is critical for optimizing response times and reducing API
    calls for repeated or similar queries. Effective caching strategies can
    dramatically improve user experience while significantly cutting operational costs.

    Caching in LLM applications targets:
    - Complete responses (exact match caching)
    - Embeddings (vector representations)
    - Prompt templates (reusable structures)
    - Intermediate results (multi-step workflows)
    - API responses (rate limit optimization)

Components:
    1. Cache Types
       - In-memory cache (fast, volatile)
       - Disk cache (persistent, slower)
       - Distributed cache (Redis, Memcached)
       - Database cache (structured, queryable)
       - CDN cache (geographic distribution)

    2. Cache Keys
       - Exact match (full prompt hash)
       - Semantic match (embedding similarity)
       - Parametric keys (template + parameters)
       - Hierarchical keys (nested cache structures)
       - Time-based keys (temporal validity)

    3. Cache Policies
       - TTL (Time To Live) - expire after time
       - LRU (Least Recently Used) - evict old entries
       - LFU (Least Frequently Used) - evict unpopular
       - Size-based - limit total cache size
       - Custom policies - application-specific

    4. Invalidation Strategies
       - Time-based expiration
       - Event-triggered invalidation
       - Manual invalidation
       - Cascade invalidation (dependencies)
       - Version-based invalidation

Use Cases:
    1. Repeated Queries
       - FAQ responses
       - Common questions
       - Popular content
       - Standard procedures
       - Frequently accessed data

    2. Expensive Operations
       - Complex reasoning chains
       - Multi-step workflows
       - Large context processing
       - Batch operations
       - Resource-intensive computations

    3. Rate Limit Optimization
       - Reduce API calls
       - Stay within quotas
       - Handle traffic spikes
       - Cost optimization
       - Performance consistency

    4. Embeddings and Vectors
       - Document embeddings
       - Query embeddings
       - Semantic search optimization
       - Similarity computations
       - Vector database queries

    5. Multi-User Applications
       - Shared responses
       - Common workflows
       - Popular queries
       - Template responses
       - Collaborative features

LangChain Implementation:
    LangChain supports caching through:
    - InMemoryCache for simple caching
    - SQLiteCache for persistent caching
    - Custom cache implementations
    - Semantic similarity caching
    - Embedding caching

Key Features:
    1. Multiple Cache Layers
       - L1: In-memory (fastest)
       - L2: Local disk (persistent)
       - L3: Distributed cache (shared)
       - L4: Database (long-term)

    2. Intelligent Matching
       - Exact match for identical queries
       - Semantic match for similar queries
       - Fuzzy match with thresholds
       - Parametric templates

    3. Performance Optimization
       - Fast lookup (< 1ms for in-memory)
       - Efficient serialization
       - Compression for large entries
       - Lazy loading

    4. Observability
       - Hit rate monitoring
       - Miss analysis
       - Performance metrics
       - Cost savings tracking

Best Practices:
    1. Cache Appropriately
       - Cache stable, reusable results
       - Don't cache rapidly changing data
       - Consider data sensitivity
       - Balance freshness vs performance

    2. Set Proper TTLs
       - Shorter for dynamic content
       - Longer for stable content
       - Consider update frequency
       - Account for user expectations

    3. Monitor Performance
       - Track hit rates (target > 70%)
       - Measure latency improvements
       - Calculate cost savings
       - Identify cache misses patterns

    4. Handle Cache Misses
       - Graceful fallback to computation
       - Transparent to users
       - Background cache warming
       - Predictive pre-caching

Trade-offs:
    Advantages:
    - Dramatically improved response times
    - Significant cost reduction
    - Better user experience
    - Reduced API rate limiting
    - Lower resource consumption
    - Improved scalability

    Disadvantages:
    - Stale data risk
    - Memory/storage overhead
    - Cache consistency challenges
    - Complexity in invalidation
    - Initial cache warming time
    - Debugging complexity

Production Considerations:
    1. Cache Size Management
       - Monitor memory usage
       - Implement eviction policies
       - Set size limits
       - Regular cleanup
       - Compression strategies

    2. Cache Consistency
       - Version tracking
       - Update propagation
       - Invalidation timing
       - Distributed consistency
       - Race condition handling

    3. Security
       - Access control for cached data
       - Encryption for sensitive content
       - Cache poisoning prevention
       - User isolation in multi-tenant
       - Audit trails

    4. Monitoring
       - Hit/miss rates
       - Cache size trends
       - Eviction frequency
       - Cost savings
       - Performance impact

    5. Disaster Recovery
       - Cache rebuild procedures
       - Backup strategies
       - Failover mechanisms
       - Data loss handling
       - Recovery time objectives
"""

Purpose: This module implements the Caching Patterns pattern, providing a structured approach to agent design and implementation.

---

Import Statements

Standard Library Imports

import collections.OrderedDict
import dataclasses.dataclass
import dataclasses.field
import datetime.datetime
import datetime.timedelta
import enum.Enum
import json
import langchain_core.caches.InMemoryCache
import langchain_core.output_parsers.StrOutputParser
import langchain_core.prompts.ChatPromptTemplate
import os
import time
import typing.Any
import typing.Callable
import typing.Dict
import typing.List
import typing.Optional

Purpose: Core Python functionality: – collections.OrderedDict: Regular expression operations – dataclasses.dataclass: Simplified class definitions with @dataclass – dataclasses.field: Simplified class definitions with @dataclass – datetime.datetime: Date and time handling – datetime.timedelta: Date and time handling – enum.Enum: Enumeration support – json: JSON serialization/deserialization – langchain_core.caches.InMemoryCache: Regular expression operations – langchain_core.output_parsers.StrOutputParser: Regular expression operations – langchain_core.prompts.ChatPromptTemplate: Regular expression operations – os: Environment variable access and system operations – time: Time-related functions – typing.Any: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Callable: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Dict: Type hints for better code documentation (List, Dict, Any, Optional) – typing.List: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Optional: Type hints for better code documentation (List, Dict, Any, Optional)

LangChain Imports

from langchain.globals.set_llm_cache
from langchain_openai.ChatOpenAI
from langchain_openai.OpenAIEmbeddings

Purpose: LangChain framework components: – LLM Interface: Wrapper for OpenAI models – Chains: Composable building blocks for complex workflows

Other Imports

from dotenv.load_dotenv
from hashlib

Purpose: Additional dependencies for specialized functionality.

---

Classes

Class: CachePolicy

Lines 215-220

Inherits from: Enum

Purpose:

Cache eviction policies

---

Class: CacheEntry

Lines 224-243

Purpose:

Represents a cached entry

Methods:

is_expired(self)

Lines 233-238

Purpose:

Check if entry is expired

Parameters:

Returns: bool

touch(self)

Lines 240-243

Purpose:

Update access information

Parameters:

---

Class: CacheStats

Lines 247-264

Purpose:

Cache performance statistics

Methods:

hit_rate(self)

Lines 255-259

Decorators: property

Purpose:

Calculate cache hit rate

Parameters:

Returns: float

miss_rate(self)

Lines 262-264

Decorators: property

Purpose:

Calculate cache miss rate

Parameters:

Returns: float

---

Class: SimpleCache

Lines 267-416

Purpose:

Simple in-memory cache with multiple eviction policies.

This cache supports:
1. Multiple eviction policies (LRU, LFU, FIFO, TTL)
2. Size limits
3. TTL per entry
4. Performance statistics

Methods:

__init__(self, max_size, policy, default_ttl)

Lines 278-296

Purpose:

Initialize cache.

Args:
    max_size: Maximum number of entries
    policy: Eviction policy
    default_ttl: Default TTL in seconds

Parameters: – max_size: Max size – policy: Policy – default_ttl: Default ttl

_make_key(self, key)

Lines 298-302

Purpose:

Create cache key from input

Parameters: – key: Key

Returns: str

_evict(self)

Lines 304-320

Purpose:

Evict entries based on policy

Parameters:

_remove_expired(self)

Lines 322-329

Purpose:

Remove expired entries

Parameters:

get(self, key)

Lines 331-365

Purpose:

Get value from cache.

Args:
    key: Cache key

Returns:
    Cached value or None

Parameters: – key: Key

Returns: <ast.Subscript object at 0x000001A06D62AED0>

set(self, key, value, ttl)

Lines 367-394

Purpose:

Set value in cache.

Args:
    key: Cache key
    value: Value to cache
    ttl: Optional TTL in seconds

Parameters: – key: Key – value: Value – ttl: Ttl

clear(self)

Lines 396-399

Purpose:

Clear all cache entries

Parameters:

size(self)

Lines 401-403

Purpose:

Get current cache size

Parameters:

Returns: int

get_stats(self)

Lines 405-416

Purpose:

Get cache statistics

Parameters:

Returns: <ast.Subscript object at 0x000001A06D74A3D0>

---

Class: SemanticCache

Lines 419-540

Purpose:

Semantic cache using embeddings for similarity matching.

This cache finds similar queries using vector similarity,
allowing cache hits for semantically similar but not identical queries.

Methods:

__init__(self, similarity_threshold, max_size)

Lines 427-443

Purpose:

Initialize semantic cache.

Args:
    similarity_threshold: Minimum similarity for cache hit
    max_size: Maximum cache entries

Parameters: – similarity_threshold: Similarity threshold – max_size: Max size

_compute_similarity(self, query1, query2)

Lines 445-471

Purpose:

Compute semantic similarity between queries.

Args:
    query1: First query
    query2: Second query

Returns:
    Similarity score (0-1)

Parameters: – query1: Input query or question – query2: Input query or question

Returns: float

get(self, query)

Lines 473-502

Purpose:

Get value for semantically similar query.

Args:
    query: Query string

Returns:
    Cached value or None

Parameters: – query: Input query or question

Returns: <ast.Subscript object at 0x000001A06D6A1F50>

set(self, query, value)

Lines 504-522

Purpose:

Cache value for query.

Args:
    query: Query string
    value: Value to cache

Parameters: – query: Input query or question – value: Value

clear(self)

Lines 524-527

Purpose:

Clear cache

Parameters:

get_stats(self)

Lines 529-540

Purpose:

Get cache statistics

Parameters:

Returns: <ast.Subscript object at 0x000001A06D7FC110>

---

Class: CachedLLMAgent

Lines 543-599

Purpose:

LLM agent with caching support.

This agent automatically caches responses and retrieves them
for repeated or similar queries.

Methods:

__init__(self, cache, model)

Lines 551-564

Purpose:

Initialize cached agent.

Args:
    cache: Cache instance to use
    model: LLM model name

Parameters: – cache: Cache – model: Language model instance

generate(self, prompt)

Lines 566-599

Purpose:

Generate response with caching.

Args:
    prompt: Input prompt

Returns:
    Dictionary with response and cache info

Parameters: – prompt: Prompt template or text

Returns: <ast.Subscript object at 0x000001A06D7C1A90>

---

Functions

demonstrate_caching_patterns()

Lines 602-892

Purpose:

Demonstrate caching patterns

---

Key Concepts Summary

Pattern Overview

The Caching Patterns pattern provides a structured approach to building AI agents with specific capabilities and behaviors. This implementation leverages LangChain’s composable components to create a robust and maintainable solution.

Core Components

1. Agent Class: Main implementation of the pattern logic
2. Prompt Engineering: Carefully crafted prompts to guide LLM behavior
3. Tool Integration: Optional tools for extended functionality
4. Memory Management: State handling across interactions
5. Output Parsing: Structured response processing

When to Use

✅ Good for: – Scenarios requiring this specific pattern’s strengths – Applications needing structured agent behavior – Systems where explainability is important – Production environments requiring reliability

❌ Not ideal for: – Simple single-prompt tasks – Scenarios requiring different pattern characteristics – Resource-constrained environments (if computationally intensive)

---

Best Practices Demonstrated

1. Environment Configuration

• ✅ Use .env files for API keys
• ✅ Never hardcode credentials
• ✅ Validate environment variables on startup

2. Error Handling

• ✅ Graceful degradation on failures
• ✅ Informative error messages
• ✅ Logging for debugging

3. Prompt Engineering

• ✅ Clear, specific instructions
• ✅ Structured output formats
• ✅ Few-shot examples when beneficial

4. Code Organization

• ✅ Separation of concerns (class-based design)
• ✅ Type hints for clarity
• ✅ Docstrings for documentation

5. Performance Optimization

• ✅ Appropriate temperature settings
• ✅ Token budget management
• ✅ Caching where applicable

---

LangChain Components Used

• ChatOpenAI: OpenAI model interface
• Chains: Sequential component composition

Why LangChain? – Composable abstractions for LLM applications – Rich ecosystem of integrations – Production-ready patterns – Active community and development

---

Execution Flow

1. Initialize agent with configuration
   ↓
2. Load environment and dependencies
   ↓
3. Prepare prompt with input data
   ↓
4. Invoke LLM through LangChain
   ↓
5. Parse and validate response
   ↓
6. Return formatted result

Key Decision Points: – Model selection (temperature, model type) – Prompt structure (few-shot vs zero-shot) – Error handling strategy – Output format validation

---

Performance Considerations

Computational Cost

• LLM Calls: Primary cost factor (based on tokens)
• Latency: Depends on model and input size
• Memory: Minimal for most patterns

Optimization Strategies

1. Prompt Optimization: Minimize tokens while maintaining clarity
2. Caching: Cache repeated queries/responses
3. Model Selection: Use smaller models when appropriate
4. Batch Processing: Group similar requests
5. Streaming: For long responses, stream tokens

Cost Estimates (Approximate)

• GPT-3.5-turbo: ~$0.002 per request
• GPT-4: ~$0.03 per request
• GPT-4-turbo: ~$0.01 per request

Actual costs vary based on input/output token counts

---

Common Pitfalls to Avoid

1. Prompt Design Issues

❌ Vague or ambiguous instructions
✅ Clear, specific, testable prompts

2. Error Handling

❌ Ignoring API errors or rate limits
✅ Implement retry logic and graceful degradation

3. Token Management

❌ Exceeding model token limits
✅ Truncate inputs, chunk processing

4. Security

❌ Exposing API keys in code
✅ Use environment variables

5. Testing

❌ Only testing happy paths
✅ Test edge cases, failures, malformed inputs

6. Cost Management

❌ Unmonitored API usage
✅ Implement budgets, alerts, and monitoring

---

Further Improvements

Production Enhancements

1. Monitoring: Add logging, metrics, tracing
2. Validation: Input/output schema validation
3. Rate Limiting: Implement request throttling
4. Caching: Redis or in-memory cache for responses
5. Async Support: Non-blocking operations for scalability

Feature Extensions

1. Multi-Model Support: Switch between providers
2. Fallback Models: Graceful degradation strategy
3. Custom Tools: Domain-specific function calling
4. Advanced Memory: Vector stores for long-term memory
5. Evaluation: Automated testing and benchmarking

Code Quality

1. Testing: Unit tests, integration tests
2. Documentation: API docs, usage examples
3. Type Safety: Stricter type hints
4. Configuration: Externalized config files

---

Related Patterns

• Pattern 088
• Pattern 090

Pattern Families: – Reasoning Patterns (001-011) – Multi-Agent Patterns (012-019) – Tool Use Patterns (020-025) – Memory Patterns (026-031) – And many more…

---

Summary

The Caching Patterns pattern provides a powerful approach to building AI agents with specific capabilities. This implementation demonstrates:

✅ Key Strengths: – Clean, maintainable code structure – LangChain integration for composability – Type hints and documentation – Production-ready patterns

🎯 Best Use Cases: – Applications requiring this pattern’s characteristics – Systems needing structured agent behavior – Production environments requiring reliability

📚 Learning Resources: – LangChain Documentation: https://python.langchain.com/ – Pattern Documentation: See related patterns – OpenAI API Reference: https://platform.openai.com/docs/

---

File: 089_caching_patterns.py
Total Lines: 897
Pattern Category: AI Agent Design Pattern #089

---

Generated from actual implementation code
For educational and reference purposes

    """
Pattern 088: Token Budget Management

Description:
    The Token Budget Management pattern monitors and controls token usage across LLM
    interactions to optimize costs, prevent budget overruns, and ensure efficient
    resource utilization. This pattern is critical for production deployments where
    token costs can escalate quickly, especially with high-volume applications or
    complex multi-step workflows.

    Token budget management involves:
    - Real-time token usage tracking
    - Cost estimation and forecasting
    - Budget enforcement and alerts
    - Optimization strategies for token efficiency
    - Usage analytics and reporting

Components:
    1. Token Counting
       - Input token counting (prompt tokens)
       - Output token counting (completion tokens)
       - Total token tracking per request
       - Cumulative usage tracking
       - Model-specific token limits

    2. Cost Calculation
       - Model pricing tables (per-token costs)
       - Request cost estimation
       - Cumulative cost tracking
       - Budget vs actual comparison
       - Cost forecasting

    3. Budget Enforcement
       - Hard limits (stop when exceeded)
       - Soft limits (warnings when approaching)
       - Rate-based budgets (tokens per hour/day)
       - User-specific quotas
       - Priority-based allocation

    4. Optimization Strategies
       - Prompt compression
       - Response length limiting
       - Model selection (cost vs quality)
       - Caching frequently used responses
       - Batching requests

Use Cases:
    1. Cost Control
       - Prevent unexpected charges
       - Stay within allocated budgets
       - Cost attribution per user/project
       - Predictable monthly spending
       - Resource optimization

    2. Multi-Tenant Applications
       - Per-user token quotas
       - Fair resource allocation
       - Usage-based billing
       - Tier-based limits (free/premium)
       - Abuse prevention

    3. Development and Testing
       - Sandbox environments with limits
       - Cost-effective experimentation
       - Testing budget allocation
       - Development vs production budgets
       - CI/CD pipeline limits

    4. High-Volume Applications
       - Aggregate usage tracking
       - Peak usage management
       - Resource planning
       - Scaling decisions
       - Cost optimization at scale

    5. Enterprise Deployments
       - Department-level budgets
       - Project-based allocation
       - Chargeback mechanisms
       - Compliance and auditing
       - Executive reporting

LangChain Implementation:
    LangChain supports token budget management through:
    - Callback handlers for token tracking
    - get_openai_callback for cost monitoring
    - Custom token counters
    - Usage analytics integration
    - Budget enforcement wrappers

Key Features:
    1. Real-Time Tracking
       - Immediate token counting
       - Running total updates
       - Cost calculation on-the-fly
       - Budget threshold monitoring
       - Alert triggering

    2. Multi-Model Support
       - Different pricing for different models
       - Model-specific token limits
       - Automatic model selection
       - Cost-aware routing
       - Fallback to cheaper models

    3. Granular Control
       - Request-level limits
       - Session-level budgets
       - User-level quotas
       - Project-level allocations
       - Global organization limits

    4. Analytics and Reporting
       - Usage trends over time
       - Cost breakdown by model
       - Per-user consumption
       - Peak usage identification
       - Cost forecasting

Best Practices:
    1. Set Appropriate Budgets
       - Based on historical data
       - Include buffer for peaks
       - Align with business value
       - Regular review and adjustment
       - Clear escalation procedures

    2. Implement Progressive Enforcement
       - Soft warnings at 70-80%
       - Stricter controls at 90%
       - Hard stops at 100%
       - Grace period for critical operations
       - Override mechanisms for emergencies

    3. Optimize Token Usage
       - Compress verbose prompts
       - Limit response lengths
       - Use cheaper models when sufficient
       - Cache common queries
       - Batch similar requests

    4. Monitor and Alert
       - Real-time dashboards
       - Automated alerts
       - Anomaly detection
       - Trend analysis
       - Regular reporting

Trade-offs:
    Advantages:
    - Predictable costs
    - Prevents budget overruns
    - Enables cost optimization
    - Fair resource allocation
    - Usage visibility and control
    - Informed scaling decisions

    Disadvantages:
    - Implementation complexity
    - Tracking overhead (minimal)
    - May limit functionality at budget
    - Requires ongoing management
    - False positives in alerting
    - User friction from limits

Production Considerations:
    1. Accuracy
       - Precise token counting
       - Up-to-date pricing information
       - Correct model identification
       - Cache hit accounting
       - Network overhead handling

    2. Performance
       - Minimal tracking overhead
       - Efficient storage of usage data
       - Fast budget checks
       - Optimized queries for analytics
       - Scalable architecture

    3. User Experience
       - Clear budget status visibility
       - Helpful error messages
       - Grace period handling
       - Budget increase requests
       - Usage optimization tips

    4. Maintenance
       - Regular pricing updates
       - Budget policy reviews
       - Alert threshold tuning
       - Anomaly investigation
       - Capacity planning

    5. Security
       - Access control for budget data
       - Secure token counting
       - Audit trail for usage
       - Fraud detection
       - Quota manipulation prevention
"""

import os
import time
from typing import List, Dict, Any, Optional, Callable
from dataclasses import dataclass, field
from enum import Enum
from datetime import datetime, timedelta
from collections import defaultdict
from dotenv import load_dotenv

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_openai import ChatOpenAI
from langchain.callbacks import get_openai_callback

load_dotenv()


class BudgetStatus(Enum):
    """Budget status levels"""
    NORMAL = "normal"
    WARNING = "warning"
    CRITICAL = "critical"
    EXCEEDED = "exceeded"


class ModelTier(Enum):
    """Model cost tiers"""
    CHEAP = "cheap"
    STANDARD = "standard"
    PREMIUM = "premium"


@dataclass
class ModelPricing:
    """Pricing information for a model"""
    model_name: str
    input_cost_per_1k: float  # Cost per 1000 input tokens
    output_cost_per_1k: float  # Cost per 1000 output tokens
    tier: ModelTier


@dataclass
class TokenUsage:
    """Token usage for a single request"""
    prompt_tokens: int
    completion_tokens: int
    total_tokens: int
    cost: float
    model: str
    timestamp: datetime = field(default_factory=datetime.now)


@dataclass
class BudgetConfig:
    """Configuration for token budget management"""
    total_budget: float  # Total budget in USD
    warning_threshold: float = 0.8  # Warning at 80%
    critical_threshold: float = 0.9  # Critical at 90%
    enforce_hard_limit: bool = True
    default_model: str = "gpt-3.5-turbo"
    allow_model_downgrade: bool = True


class TokenBudgetManager:
    """
    Manages token budgets and costs across LLM interactions.
    
    This manager:
    1. Tracks token usage in real-time
    2. Calculates costs based on model pricing
    3. Enforces budget limits
    4. Provides usage analytics
    """
    
    # Model pricing (as of 2025)
    MODEL_PRICING = {
        "gpt-3.5-turbo": ModelPricing("gpt-3.5-turbo", 0.0005, 0.0015, ModelTier.CHEAP),
        "gpt-4": ModelPricing("gpt-4", 0.03, 0.06, ModelTier.PREMIUM),
        "gpt-4-turbo": ModelPricing("gpt-4-turbo", 0.01, 0.03, ModelTier.STANDARD),
    }
    
    def __init__(self, config: BudgetConfig):
        """
        Initialize token budget manager.
        
        Args:
            config: Budget configuration
        """
        self.config = config
        self.total_spent = 0.0
        self.usage_history: List[TokenUsage] = []
        self.requests_count = 0
        self.model_usage: Dict[str, int] = defaultdict(int)
        self.alert_callbacks: List[Callable] = []
    
    def add_alert_callback(self, callback: Callable[[BudgetStatus, Dict], None]):
        """Add callback for budget alerts"""
        self.alert_callbacks.append(callback)
    
    def _trigger_alerts(self, status: BudgetStatus, details: Dict[str, Any]):
        """Trigger all registered alert callbacks"""
        for callback in self.alert_callbacks:
            try:
                callback(status, details)
            except Exception as e:
                print(f"Alert callback error: {e}")
    
    def get_status(self) -> BudgetStatus:
        """Get current budget status"""
        usage_ratio = self.total_spent / self.config.total_budget
        
        if usage_ratio >= 1.0:
            return BudgetStatus.EXCEEDED
        elif usage_ratio >= self.config.critical_threshold:
            return BudgetStatus.CRITICAL
        elif usage_ratio >= self.config.warning_threshold:
            return BudgetStatus.WARNING
        else:
            return BudgetStatus.NORMAL
    
    def can_proceed(self, estimated_cost: float = 0.0) -> tuple[bool, str]:
        """
        Check if request can proceed within budget.
        
        Args:
            estimated_cost: Estimated cost of request
            
        Returns:
            Tuple of (can_proceed, reason)
        """
        status = self.get_status()
        
        if status == BudgetStatus.EXCEEDED:
            if self.config.enforce_hard_limit:
                return False, "Budget exceeded. Request denied."
            else:
                return True, "Budget exceeded but enforcement disabled."
        
        projected_cost = self.total_spent + estimated_cost
        if projected_cost > self.config.total_budget:
            if self.config.enforce_hard_limit:
                return False, f"Request would exceed budget. Projected: ${projected_cost:.4f}"
            
        if status == BudgetStatus.CRITICAL:
            return True, "Budget critical. Proceed with caution."
        
        return True, "OK"
    
    def calculate_cost(
        self,
        prompt_tokens: int,
        completion_tokens: int,
        model: str
    ) -> float:
        """
        Calculate cost for token usage.
        
        Args:
            prompt_tokens: Number of prompt tokens
            completion_tokens: Number of completion tokens
            model: Model name
            
        Returns:
            Cost in USD
        """
        pricing = self.MODEL_PRICING.get(model)
        if not pricing:
            # Default to gpt-3.5-turbo pricing
            pricing = self.MODEL_PRICING["gpt-3.5-turbo"]
        
        input_cost = (prompt_tokens / 1000) * pricing.input_cost_per_1k
        output_cost = (completion_tokens / 1000) * pricing.output_cost_per_1k
        
        return input_cost + output_cost
    
    def record_usage(
        self,
        prompt_tokens: int,
        completion_tokens: int,
        model: str
    ) -> TokenUsage:
        """
        Record token usage and update budget.
        
        Args:
            prompt_tokens: Number of prompt tokens
            completion_tokens: Number of completion tokens
            model: Model name
            
        Returns:
            TokenUsage object
        """
        total_tokens = prompt_tokens + completion_tokens
        cost = self.calculate_cost(prompt_tokens, completion_tokens, model)
        
        usage = TokenUsage(
            prompt_tokens=prompt_tokens,
            completion_tokens=completion_tokens,
            total_tokens=total_tokens,
            cost=cost,
            model=model
        )
        
        # Update tracking
        self.total_spent += cost
        self.usage_history.append(usage)
        self.requests_count += 1
        self.model_usage[model] += total_tokens
        
        # Check status and trigger alerts if needed
        status = self.get_status()
        if status in [BudgetStatus.WARNING, BudgetStatus.CRITICAL, BudgetStatus.EXCEEDED]:
            self._trigger_alerts(status, {
                "total_spent": self.total_spent,
                "budget": self.config.total_budget,
                "usage_ratio": self.total_spent / self.config.total_budget,
                "latest_usage": usage
            })
        
        return usage
    
    def get_remaining_budget(self) -> float:
        """Get remaining budget"""
        return max(0, self.config.total_budget - self.total_spent)
    
    def get_usage_summary(self) -> Dict[str, Any]:
        """Get comprehensive usage summary"""
        status = self.get_status()
        
        return {
            "total_budget": self.config.total_budget,
            "total_spent": self.total_spent,
            "remaining": self.get_remaining_budget(),
            "usage_percentage": (self.total_spent / self.config.total_budget) * 100,
            "status": status.value,
            "total_requests": self.requests_count,
            "total_tokens": sum(u.total_tokens for u in self.usage_history),
            "model_breakdown": dict(self.model_usage),
            "average_cost_per_request": self.total_spent / max(1, self.requests_count)
        }
    
    def suggest_cheaper_model(self, current_model: str) -> Optional[str]:
        """Suggest a cheaper model alternative"""
        if not self.config.allow_model_downgrade:
            return None
        
        current_pricing = self.MODEL_PRICING.get(current_model)
        if not current_pricing:
            return None
        
        # Find cheaper alternatives
        cheaper_models = [
            model for model, pricing in self.MODEL_PRICING.items()
            if pricing.tier.value  Dict[str, Any]:
        """
        Generate response with budget tracking.
        
        Args:
            prompt: Input prompt
            max_tokens: Maximum tokens for response
            
        Returns:
            Dictionary with response and usage info
        """
        # Check budget before proceeding
        can_proceed, reason = self.budget_manager.can_proceed()
        if not can_proceed:
            return {
                "success": False,
                "error": reason,
                "response": None,
                "usage": None
            }
        
        # Generate with token tracking
        with get_openai_callback() as cb:
            try:
                response = self.llm.invoke(prompt)
                
                # Record usage
                usage = self.budget_manager.record_usage(
                    prompt_tokens=cb.prompt_tokens,
                    completion_tokens=cb.completion_tokens,
                    model=self.model
                )
                
                return {
                    "success": True,
                    "response": response.content,
                    "usage": {
                        "prompt_tokens": usage.prompt_tokens,
                        "completion_tokens": usage.completion_tokens,
                        "total_tokens": usage.total_tokens,
                        "cost": usage.cost
                    },
                    "budget_status": self.budget_manager.get_status().value
                }
                
            except Exception as e:
                return {
                    "success": False,
                    "error": str(e),
                    "response": None,
                    "usage": None
                }
    
    def generate_with_limit(
        self,
        prompt: str,
        max_cost: float
    ) -> Dict[str, Any]:
        """
        Generate response with per-request cost limit.
        
        Args:
            prompt: Input prompt
            max_cost: Maximum cost for this request
            
        Returns:
            Dictionary with response and usage info
        """
        # Estimate and check
        can_proceed, reason = self.budget_manager.can_proceed(max_cost)
        if not can_proceed:
            return {
                "success": False,
                "error": reason,
                "response": None
            }
        
        return self.generate(prompt)
    
    def batch_generate(
        self,
        prompts: List[str],
        max_total_cost: Optional[float] = None
    ) -> List[Dict[str, Any]]:
        """
        Generate responses for multiple prompts with budget tracking.
        
        Args:
            prompts: List of prompts
            max_total_cost: Optional cost limit for batch
            
        Returns:
            List of results
        """
        results = []
        batch_cost = 0.0
        
        for prompt in prompts:
            # Check batch cost limit
            if max_total_cost and batch_cost >= max_total_cost:
                results.append({
                    "success": False,
                    "error": "Batch cost limit reached",
                    "response": None
                })
                continue
            
            result = self.generate(prompt)
            results.append(result)
            
            if result["success"] and result["usage"]:
                batch_cost += result["usage"]["cost"]
        
        return results


def demonstrate_token_budget_management():
    """Demonstrate token budget management patterns"""
    print("=" * 80)
    print("TOKEN BUDGET MANAGEMENT PATTERN DEMONSTRATION")
    print("=" * 80)
    
    # Example 1: Basic Budget Tracking
    print("\n" + "=" * 80)
    print("EXAMPLE 1: Basic Budget Tracking")
    print("=" * 80)
    
    config = BudgetConfig(
        total_budget=1.00,  # $1.00 budget
        warning_threshold=0.7,
        critical_threshold=0.9,
        enforce_hard_limit=False  # Allow going over for demo
    )
    
    budget_manager = TokenBudgetManager(config)
    agent = BudgetAwareLLMAgent(budget_manager)
    
    print(f"\nInitial Budget: ${config.total_budget}")
    print("\nGenerating responses...\n")
    
    prompts = [
        "What is machine learning?",
        "Explain neural networks briefly.",
        "What is deep learning?"
    ]
    
    for i, prompt in enumerate(prompts, 1):
        print(f"Request {i}: {prompt}")
        result = agent.generate(prompt)
        
        if result["success"]:
            print(f"Response: {result['response'][:100]}...")
            print(f"Tokens: {result['usage']['total_tokens']}")
            print(f"Cost: ${result['usage']['cost']:.6f}")
            print(f"Status: {result['budget_status']}")
        else:
            print(f"Error: {result['error']}")
        print()
    
    # Show summary
    summary = budget_manager.get_usage_summary()
    print("\nBudget Summary:")
    print(f"  Total Spent: ${summary['total_spent']:.6f}")
    print(f"  Remaining: ${summary['remaining']:.6f}")
    print(f"  Usage: {summary['usage_percentage']:.1f}%")
    print(f"  Status: {summary['status']}")
    print(f"  Total Requests: {summary['total_requests']}")
    print(f"  Total Tokens: {summary['total_tokens']}")
    print(f"  Avg Cost/Request: ${summary['average_cost_per_request']:.6f}")
    
    # Example 2: Budget Alerts
    print("\n" + "=" * 80)
    print("EXAMPLE 2: Budget Alerts and Thresholds")
    print("=" * 80)
    
    def budget_alert_handler(status: BudgetStatus, details: Dict):
        """Custom alert handler"""
        print(f"\n🚨 BUDGET ALERT: {status.value.upper()}")
        print(f"   Spent: ${details['total_spent']:.6f}")
        print(f"   Budget: ${details['budget']:.6f}")
        print(f"   Usage: {details['usage_ratio']*100:.1f}%")
    
    config2 = BudgetConfig(
        total_budget=0.10,  # Small budget to trigger alerts
        warning_threshold=0.5,
        critical_threshold=0.8,
        enforce_hard_limit=False
    )
    
    budget_manager2 = TokenBudgetManager(config2)
    budget_manager2.add_alert_callback(budget_alert_handler)
    agent2 = BudgetAwareLLMAgent(budget_manager2)
    
    print(f"\nBudget: ${config2.total_budget}")
    print("Generating responses to trigger alerts...\n")
    
    test_prompts = [
        "Explain quantum computing in detail.",
        "What are the applications of AI?",
        "Describe blockchain technology."
    ]
    
    for prompt in test_prompts:
        result = agent2.generate(prompt)
        if result["success"]:
            print(f"✓ Generated response (${result['usage']['cost']:.6f})")
    
    # Example 3: Cost Estimation and Limits
    print("\n" + "=" * 80)
    print("EXAMPLE 3: Per-Request Cost Limits")
    print("=" * 80)
    
    config3 = BudgetConfig(total_budget=0.50, enforce_hard_limit=True)
    budget_manager3 = TokenBudgetManager(config3)
    agent3 = BudgetAwareLLMAgent(budget_manager3)
    
    print("\nTrying request with cost limit...\n")
    
    result = agent3.generate_with_limit(
        "Write a short poem about AI",
        max_cost=0.01
    )
    
    if result["success"]:
        print(f"Response: {result['response']}")
        print(f"Cost: ${result['usage']['cost']:.6f}")
    else:
        print(f"Request blocked: {result['error']}")
    
    # Example 4: Model Cost Comparison
    print("\n" + "=" * 80)
    print("EXAMPLE 4: Model Cost Comparison")
    print("=" * 80)
    
    print("\nModel Pricing:")
    for model_name, pricing in TokenBudgetManager.MODEL_PRICING.items():
        print(f"\n{model_name} ({pricing.tier.value}):")
        print(f"  Input: ${pricing.input_cost_per_1k}/1K tokens")
        print(f"  Output: ${pricing.output_cost_per_1k}/1K tokens")
    
    # Calculate example costs
    print("\n\nExample cost for 1000 input + 500 output tokens:")
    for model_name in TokenBudgetManager.MODEL_PRICING.keys():
        cost = budget_manager3.calculate_cost(1000, 500, model_name)
        print(f"  {model_name}: ${cost:.6f}")
    
    # Example 5: Batch Processing with Budget
    print("\n" + "=" * 80)
    print("EXAMPLE 5: Batch Processing with Budget Control")
    print("=" * 80)
    
    config5 = BudgetConfig(total_budget=0.50)
    budget_manager5 = TokenBudgetManager(config5)
    agent5 = BudgetAwareLLMAgent(budget_manager5, model="gpt-3.5-turbo")
    
    batch_prompts = [
        "What is Python?",
        "What is JavaScript?",
        "What is Java?",
        "What is C++?",
        "What is Ruby?"
    ]
    
    print(f"\nProcessing {len(batch_prompts)} prompts with batch cost limit...")
    print(f"Batch cost limit: $0.05\n")
    
    results = agent5.batch_generate(batch_prompts, max_total_cost=0.05)
    
    successful = sum(1 for r in results if r["success"])
    total_cost = sum(r["usage"]["cost"] for r in results if r["success"] and r["usage"])
    
    print(f"\nBatch Results:")
    print(f"  Successful: {successful}/{len(batch_prompts)}")
    print(f"  Total Cost: ${total_cost:.6f}")
    print(f"  Average Cost: ${total_cost/max(1, successful):.6f}")
    
    # Summary
    print("\n" + "=" * 80)
    print("TOKEN BUDGET MANAGEMENT SUMMARY")
    print("=" * 80)
    print("""
Token Budget Management Benefits:
1. Cost Control: Prevent unexpected charges and budget overruns
2. Visibility: Real-time tracking of token usage and costs
3. Optimization: Identify opportunities to reduce token consumption
4. Accountability: Per-user or per-project cost attribution
5. Planning: Forecast future costs based on usage trends
6. Alerts: Proactive notifications when approaching limits

Key Components Demonstrated:
1. Real-Time Tracking: Automatic token and cost monitoring
2. Budget Thresholds: Warning, critical, and exceeded states
3. Cost Calculation: Model-specific pricing and estimates
4. Enforcement: Hard and soft budget limits
5. Alerts: Custom callback handlers for notifications
6. Analytics: Usage summaries and breakdowns

Budget Management Strategies:
1. Progressive Thresholds:
   - Normal:  100% of budget

2. Cost Optimization:
   - Use cheaper models when quality allows
   - Compress verbose prompts
   - Limit response lengths
   - Cache common queries
   - Batch similar requests

3. Monitoring:
   - Real-time dashboards
   - Usage trend analysis
   - Anomaly detection
   - Regular reporting
   - Forecasting

Best Practices:
1. Set realistic budgets based on historical data
2. Implement progressive enforcement (soft → hard limits)
3. Monitor usage patterns and trends
4. Optimize prompts to reduce token consumption
5. Use alerts to catch issues early
6. Regular budget reviews and adjustments
7. Cost attribution by user/project/feature
8. Document budget policies clearly

Model Selection for Cost:
- gpt-3.5-turbo: Cheapest, good for simple tasks
- gpt-4-turbo: Balanced cost/quality
- gpt-4: Premium, best quality but expensive

Production Considerations:
- Accurate token counting (model-specific)
- Up-to-date pricing information
- Efficient usage data storage
- Scalable tracking architecture
- User-friendly budget notifications
- Grace periods for critical operations
- Emergency override procedures
- Regular auditing and reconciliation

Common Use Cases:
- SaaS applications with freemium tiers
- Multi-tenant platforms
- Development/testing environments
- High-volume production systems
- Enterprise cost centers
- Research project budgets
- Educational platforms with quotas
""")
    
    print("\n" + "=" * 80)
    print("Pattern 088 (Token Budget Management) demonstration complete!")
    print("=" * 80)


if __name__ == "__main__":
    demonstrate_token_budget_management()

    

Agentic Design Pattern: Token Budget

Pattern 088: Token Budget – Line-by-Line Explanation

Overview

This document provides a detailed line-by-line explanation of the Token Budget pattern implementation.

---

Pattern Description

Pattern 088: Token Budget Management

Description: The Token Budget Management pattern monitors and controls token usage across LLM interactions to optimize costs, prevent budget overruns, and ensure efficient resource utilization. This pattern is critical for production deployments where token costs can escalate quickly, especially with high-volume applications or complex multi-step workflows.

Token budget management involves:
- Real-time token usage tracking
- Cost estimation and forecasting
- Budget enforcement and alerts
- Optimization strategies for token efficiency
- Usage analytics and reporting

Components: 1. Token Counting – Input token counting (prompt tokens) – Output token counting (completion tokens) – Total token tracking per request – Cumulative usage tracking – Model-specific token limits

2. Cost Calculation
   - Model pricing tables (per-token costs)
   - Request cost estimation
   - Cumulative cost tracking
   - Budget vs actual comparison
   - Cost forecasting

3. Budget Enforcement
   - Hard limits (stop when exceeded)
   - Soft limits (warnings when approaching)
   - Rate-based budgets (tokens per hour/day)
   - User-specific quotas
   - Priority-based allocation

4. Optimization Strategies
   - Prompt compression
   - Response length limiting
   - Model selection (cost vs quality)
   - Caching frequently used responses
   - Batching requests

Use Cases: 1. Cost Control – Prevent unexpected charges – Stay within allocated budgets – Cost attribution per user/project – Predictable monthly spending – Resource optimization

2. Multi-Tenant Applications
   - Per-user token quotas
   - Fair resource allocation
   - Usage-based billing
   - Tier-based limits (free/premium)
   - Abuse prevention

3. Development and Testing
   - Sandbox environments with limits
   - Cost-effective experimentation
   - Testing budget allocation
   - Development vs production budgets
   - CI/CD pipeline limits

4. High-Volume Applications
   - Aggregate usage tracking
   - Peak usage management
   - Resource planning
   - Scaling decisions
   - Cost optimization at scale

5. Enterprise Deployments
   - Department-level budgets
   - Project-based allocation
   - Chargeback mechanisms
   - Compliance and auditing
   - Executive reporting

LangChain Implementation: LangChain supports token budget management through: – Callback handlers for token tracking – get_openai_callback for cost monitoring – Custom token counters – Usage analytics integration – Budget enforcement wrappers

Key Features: 1. Real-Time Tracking – Immediate token counting – Running total updates – Cost calculation on-the-fly – Budget threshold monitoring – Alert triggering

2. Multi-Model Support
   - Different pricing for different models
   - Model-specific token limits
   - Automatic model selection
   - Cost-aware routing
   - Fallback to cheaper models

3. Granular Control
   - Request-level limits
   - Session-level budgets
   - User-level quotas
   - Project-level allocations
   - Global organization limits

4. Analytics and Reporting
   - Usage trends over time
   - Cost breakdown by model
   - Per-user consumption
   - Peak usage identification
   - Cost forecasting

Best Practices: 1. Set Appropriate Budgets – Based on historical data – Include buffer for peaks – Align with business value – Regular review and adjustment – Clear escalation procedures

2. Implement Progressive Enforcement
   - Soft warnings at 70-80%
   - Stricter controls at 90%
   - Hard stops at 100%
   - Grace period for critical operations
   - Override mechanisms for emergencies

3. Optimize Token Usage
   - Compress verbose prompts
   - Limit response lengths
   - Use cheaper models when sufficient
   - Cache common queries
   - Batch similar requests

4. Monitor and Alert
   - Real-time dashboards
   - Automated alerts
   - Anomaly detection
   - Trend analysis
   - Regular reporting

Trade-offs: Advantages: – Predictable costs – Prevents budget overruns – Enables cost optimization – Fair resource allocation – Usage visibility and control – Informed scaling decisions

Disadvantages:
- Implementation complexity
- Tracking overhead (minimal)
- May limit functionality at budget
- Requires ongoing management
- False positives in alerting
- User friction from limits

Production Considerations: 1. Accuracy – Precise token counting – Up-to-date pricing information – Correct model identification – Cache hit accounting – Network overhead handling

2. Performance
   - Minimal tracking overhead
   - Efficient storage of usage data
   - Fast budget checks
   - Optimized queries for analytics
   - Scalable architecture

3. User Experience
   - Clear budget status visibility
   - Helpful error messages
   - Grace period handling
   - Budget increase requests
   - Usage optimization tips

4. Maintenance
   - Regular pricing updates
   - Budget policy reviews
   - Alert threshold tuning
   - Anomaly investigation
   - Capacity planning

5. Security
   - Access control for budget data
   - Secure token counting
   - Audit trail for usage
   - Fraud detection
   - Quota manipulation prevention

---

File Statistics

• Total Lines: 863
• Classes: 7
• Functions: 1
• Constants: 0

---

Module Documentation

"""
Pattern 088: Token Budget Management

Description:
    The Token Budget Management pattern monitors and controls token usage across LLM
    interactions to optimize costs, prevent budget overruns, and ensure efficient
    resource utilization. This pattern is critical for production deployments where
    token costs can escalate quickly, especially with high-volume applications or
    complex multi-step workflows.

    Token budget management involves:
    - Real-time token usage tracking
    - Cost estimation and forecasting
    - Budget enforcement and alerts
    - Optimization strategies for token efficiency
    - Usage analytics and reporting

Components:
    1. Token Counting
       - Input token counting (prompt tokens)
       - Output token counting (completion tokens)
       - Total token tracking per request
       - Cumulative usage tracking
       - Model-specific token limits

    2. Cost Calculation
       - Model pricing tables (per-token costs)
       - Request cost estimation
       - Cumulative cost tracking
       - Budget vs actual comparison
       - Cost forecasting

    3. Budget Enforcement
       - Hard limits (stop when exceeded)
       - Soft limits (warnings when approaching)
       - Rate-based budgets (tokens per hour/day)
       - User-specific quotas
       - Priority-based allocation

    4. Optimization Strategies
       - Prompt compression
       - Response length limiting
       - Model selection (cost vs quality)
       - Caching frequently used responses
       - Batching requests

Use Cases:
    1. Cost Control
       - Prevent unexpected charges
       - Stay within allocated budgets
       - Cost attribution per user/project
       - Predictable monthly spending
       - Resource optimization

    2. Multi-Tenant Applications
       - Per-user token quotas
       - Fair resource allocation
       - Usage-based billing
       - Tier-based limits (free/premium)
       - Abuse prevention

    3. Development and Testing
       - Sandbox environments with limits
       - Cost-effective experimentation
       - Testing budget allocation
       - Development vs production budgets
       - CI/CD pipeline limits

    4. High-Volume Applications
       - Aggregate usage tracking
       - Peak usage management
       - Resource planning
       - Scaling decisions
       - Cost optimization at scale

    5. Enterprise Deployments
       - Department-level budgets
       - Project-based allocation
       - Chargeback mechanisms
       - Compliance and auditing
       - Executive reporting

LangChain Implementation:
    LangChain supports token budget management through:
    - Callback handlers for token tracking
    - get_openai_callback for cost monitoring
    - Custom token counters
    - Usage analytics integration
    - Budget enforcement wrappers

Key Features:
    1. Real-Time Tracking
       - Immediate token counting
       - Running total updates
       - Cost calculation on-the-fly
       - Budget threshold monitoring
       - Alert triggering

    2. Multi-Model Support
       - Different pricing for different models
       - Model-specific token limits
       - Automatic model selection
       - Cost-aware routing
       - Fallback to cheaper models

    3. Granular Control
       - Request-level limits
       - Session-level budgets
       - User-level quotas
       - Project-level allocations
       - Global organization limits

    4. Analytics and Reporting
       - Usage trends over time
       - Cost breakdown by model
       - Per-user consumption
       - Peak usage identification
       - Cost forecasting

Best Practices:
    1. Set Appropriate Budgets
       - Based on historical data
       - Include buffer for peaks
       - Align with business value
       - Regular review and adjustment
       - Clear escalation procedures

    2. Implement Progressive Enforcement
       - Soft warnings at 70-80%
       - Stricter controls at 90%
       - Hard stops at 100%
       - Grace period for critical operations
       - Override mechanisms for emergencies

    3. Optimize Token Usage
       - Compress verbose prompts
       - Limit response lengths
       - Use cheaper models when sufficient
       - Cache common queries
       - Batch similar requests

    4. Monitor and Alert
       - Real-time dashboards
       - Automated alerts
       - Anomaly detection
       - Trend analysis
       - Regular reporting

Trade-offs:
    Advantages:
    - Predictable costs
    - Prevents budget overruns
    - Enables cost optimization
    - Fair resource allocation
    - Usage visibility and control
    - Informed scaling decisions

    Disadvantages:
    - Implementation complexity
    - Tracking overhead (minimal)
    - May limit functionality at budget
    - Requires ongoing management
    - False positives in alerting
    - User friction from limits

Production Considerations:
    1. Accuracy
       - Precise token counting
       - Up-to-date pricing information
       - Correct model identification
       - Cache hit accounting
       - Network overhead handling

    2. Performance
       - Minimal tracking overhead
       - Efficient storage of usage data
       - Fast budget checks
       - Optimized queries for analytics
       - Scalable architecture

    3. User Experience
       - Clear budget status visibility
       - Helpful error messages
       - Grace period handling
       - Budget increase requests
       - Usage optimization tips

    4. Maintenance
       - Regular pricing updates
       - Budget policy reviews
       - Alert threshold tuning
       - Anomaly investigation
       - Capacity planning

    5. Security
       - Access control for budget data
       - Secure token counting
       - Audit trail for usage
       - Fraud detection
       - Quota manipulation prevention
"""

Purpose: This module implements the Token Budget pattern, providing a structured approach to agent design and implementation.

---

Import Statements

Standard Library Imports

import collections.defaultdict
import dataclasses.dataclass
import dataclasses.field
import datetime.datetime
import datetime.timedelta
import enum.Enum
import langchain_core.output_parsers.StrOutputParser
import langchain_core.prompts.ChatPromptTemplate
import os
import time
import typing.Any
import typing.Callable
import typing.Dict
import typing.List
import typing.Optional

Purpose: Core Python functionality: – collections.defaultdict: Specialized container datatypes – dataclasses.dataclass: Simplified class definitions with @dataclass – dataclasses.field: Simplified class definitions with @dataclass – datetime.datetime: Date and time handling – datetime.timedelta: Date and time handling – enum.Enum: Enumeration support – langchain_core.output_parsers.StrOutputParser: Regular expression operations – langchain_core.prompts.ChatPromptTemplate: Regular expression operations – os: Environment variable access and system operations – time: Time-related functions – typing.Any: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Callable: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Dict: Type hints for better code documentation (List, Dict, Any, Optional) – typing.List: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Optional: Type hints for better code documentation (List, Dict, Any, Optional)

LangChain Imports

from langchain.callbacks.get_openai_callback
from langchain_openai.ChatOpenAI

Purpose: LangChain framework components: – LLM Interface: Wrapper for OpenAI models – Chains: Composable building blocks for complex workflows

Other Imports

from dotenv.load_dotenv

Purpose: Additional dependencies for specialized functionality.

---

Classes

Class: BudgetStatus

Lines 220-225

Inherits from: Enum

Purpose:

Budget status levels

---

Class: ModelTier

Lines 228-232

Inherits from: Enum

Purpose:

Model cost tiers

---

Class: ModelPricing

Lines 236-241

Purpose:

Pricing information for a model

---

Class: TokenUsage

Lines 245-252

Purpose:

Token usage for a single request

---

Class: BudgetConfig

Lines 256-263

Purpose:

Configuration for token budget management

---

Class: TokenBudgetManager

Lines 266-469

Purpose:

Manages token budgets and costs across LLM interactions.

This manager:
1. Tracks token usage in real-time
2. Calculates costs based on model pricing
3. Enforces budget limits
4. Provides usage analytics

Methods:

__init__(self, config)

Lines 284-296

Purpose:

Initialize token budget manager.

Args:
    config: Budget configuration

Parameters: – config: Config

add_alert_callback(self, callback)

Lines 298-300

Purpose:

Add callback for budget alerts

Parameters: – callback: Callback

_trigger_alerts(self, status, details)

Lines 302-308

Purpose:

Trigger all registered alert callbacks

Parameters: – status: Status – details: Details

get_status(self)

Lines 310-321

Purpose:

Get current budget status

Parameters:

Returns: BudgetStatus

can_proceed(self, estimated_cost)

Lines 323-349

Purpose:

Check if request can proceed within budget.

Args:
    estimated_cost: Estimated cost of request

Returns:
    Tuple of (can_proceed, reason)

Parameters: – estimated_cost: Estimated cost

Returns: <ast.Subscript object at 0x000001A06D71B2D0>

calculate_cost(self, prompt_tokens, completion_tokens, model)

Lines 351-376

Purpose:

Calculate cost for token usage.

Args:
    prompt_tokens: Number of prompt tokens
    completion_tokens: Number of completion tokens
    model: Model name

Returns:
    Cost in USD

Parameters: – prompt_tokens: Prompt template or text – completion_tokens: Completion tokens – model: Language model instance

Returns: float

record_usage(self, prompt_tokens, completion_tokens, model)

Lines 378-422

Purpose:

Record token usage and update budget.

Args:
    prompt_tokens: Number of prompt tokens
    completion_tokens: Number of completion tokens
    model: Model name

Returns:
    TokenUsage object

Parameters: – prompt_tokens: Prompt template or text – completion_tokens: Completion tokens – model: Language model instance

Returns: TokenUsage

get_remaining_budget(self)

Lines 424-426

Purpose:

Get remaining budget

Parameters:

Returns: float

get_usage_summary(self)

Lines 428-442

Purpose:

Get comprehensive usage summary

Parameters:

Returns: <ast.Subscript object at 0x000001A06D857210>

suggest_cheaper_model(self, current_model)

Lines 444-459

Purpose:

Suggest a cheaper model alternative

Parameters: – current_model: Language model instance

Returns: <ast.Subscript object at 0x000001A06D8505D0>

reset_budget(self, new_budget)

Lines 461-469

Purpose:

Reset budget and usage tracking

Parameters: – new_budget: New budget

---

Class: BudgetAwareLLMAgent

Lines 472-616

Purpose:

LLM agent with token budget awareness.

This agent:
1. Tracks token usage automatically
2. Enforces budget limits
3. Optimizes token usage
4. Provides cost visibility

Methods:

__init__(self, budget_manager, model)

Lines 483-497

Purpose:

Initialize budget-aware agent.

Args:
    budget_manager: Token budget manager
    model: Model to use (defaults to config default)

Parameters: – budget_manager: Budget manager – model: Language model instance

generate(self, prompt, max_tokens)

Lines 499-554

Purpose:

Generate response with budget tracking.

Args:
    prompt: Input prompt
    max_tokens: Maximum tokens for response

Returns:
    Dictionary with response and usage info

Parameters: – prompt: Prompt template or text – max_tokens: Maximum tokens to generate

Returns: <ast.Subscript object at 0x000001A06D83EF90>

generate_with_limit(self, prompt, max_cost)

Lines 556-580

Purpose:

Generate response with per-request cost limit.

Args:
    prompt: Input prompt
    max_cost: Maximum cost for this request

Returns:
    Dictionary with response and usage info

Parameters: – prompt: Prompt template or text – max_cost: Max cost

Returns: <ast.Subscript object at 0x000001A06D88B2D0>

batch_generate(self, prompts, max_total_cost)

Lines 582-616

Purpose:

Generate responses for multiple prompts with budget tracking.

Args:
    prompts: List of prompts
    max_total_cost: Optional cost limit for batch

Returns:
    List of results

Parameters: – prompts: Prompt template or text – max_total_cost: Max total cost

Returns: <ast.Subscript object at 0x000001A06D625290>

---

Functions

demonstrate_token_budget_management()

Lines 619-858

Purpose:

Demonstrate token budget management patterns

---

Key Concepts Summary

Pattern Overview

The Token Budget pattern provides a structured approach to building AI agents with specific capabilities and behaviors. This implementation leverages LangChain’s composable components to create a robust and maintainable solution.

Core Components

1. Agent Class: Main implementation of the pattern logic
2. Prompt Engineering: Carefully crafted prompts to guide LLM behavior
3. Tool Integration: Optional tools for extended functionality
4. Memory Management: State handling across interactions
5. Output Parsing: Structured response processing

When to Use

✅ Good for: – Scenarios requiring this specific pattern’s strengths – Applications needing structured agent behavior – Systems where explainability is important – Production environments requiring reliability

❌ Not ideal for: – Simple single-prompt tasks – Scenarios requiring different pattern characteristics – Resource-constrained environments (if computationally intensive)

---

Best Practices Demonstrated

1. Environment Configuration

• ✅ Use .env files for API keys
• ✅ Never hardcode credentials
• ✅ Validate environment variables on startup

2. Error Handling

• ✅ Graceful degradation on failures
• ✅ Informative error messages
• ✅ Logging for debugging

3. Prompt Engineering

• ✅ Clear, specific instructions
• ✅ Structured output formats
• ✅ Few-shot examples when beneficial

4. Code Organization

• ✅ Separation of concerns (class-based design)
• ✅ Type hints for clarity
• ✅ Docstrings for documentation

5. Performance Optimization

• ✅ Appropriate temperature settings
• ✅ Token budget management
• ✅ Caching where applicable

---

LangChain Components Used

• ChatOpenAI: OpenAI model interface
• Chains: Sequential component composition

Why LangChain? – Composable abstractions for LLM applications – Rich ecosystem of integrations – Production-ready patterns – Active community and development

---

Execution Flow

1. Initialize agent with configuration
   ↓
2. Load environment and dependencies
   ↓
3. Prepare prompt with input data
   ↓
4. Invoke LLM through LangChain
   ↓
5. Parse and validate response
   ↓
6. Return formatted result

Key Decision Points: – Model selection (temperature, model type) – Prompt structure (few-shot vs zero-shot) – Error handling strategy – Output format validation

---

Performance Considerations

Computational Cost

• LLM Calls: Primary cost factor (based on tokens)
• Latency: Depends on model and input size
• Memory: Minimal for most patterns

Optimization Strategies

1. Prompt Optimization: Minimize tokens while maintaining clarity
2. Caching: Cache repeated queries/responses
3. Model Selection: Use smaller models when appropriate
4. Batch Processing: Group similar requests
5. Streaming: For long responses, stream tokens

Cost Estimates (Approximate)

• GPT-3.5-turbo: ~$0.002 per request
• GPT-4: ~$0.03 per request
• GPT-4-turbo: ~$0.01 per request

Actual costs vary based on input/output token counts

---

Common Pitfalls to Avoid

1. Prompt Design Issues

❌ Vague or ambiguous instructions
✅ Clear, specific, testable prompts

2. Error Handling

❌ Ignoring API errors or rate limits
✅ Implement retry logic and graceful degradation

3. Token Management

❌ Exceeding model token limits
✅ Truncate inputs, chunk processing

4. Security

❌ Exposing API keys in code
✅ Use environment variables

5. Testing

❌ Only testing happy paths
✅ Test edge cases, failures, malformed inputs

6. Cost Management

❌ Unmonitored API usage
✅ Implement budgets, alerts, and monitoring

---

Further Improvements

Production Enhancements

1. Monitoring: Add logging, metrics, tracing
2. Validation: Input/output schema validation
3. Rate Limiting: Implement request throttling
4. Caching: Redis or in-memory cache for responses
5. Async Support: Non-blocking operations for scalability

Feature Extensions

1. Multi-Model Support: Switch between providers
2. Fallback Models: Graceful degradation strategy
3. Custom Tools: Domain-specific function calling
4. Advanced Memory: Vector stores for long-term memory
5. Evaluation: Automated testing and benchmarking

Code Quality

1. Testing: Unit tests, integration tests
2. Documentation: API docs, usage examples
3. Type Safety: Stricter type hints
4. Configuration: Externalized config files

---

Related Patterns

• Pattern 087
• Pattern 089

Pattern Families: – Reasoning Patterns (001-011) – Multi-Agent Patterns (012-019) – Tool Use Patterns (020-025) – Memory Patterns (026-031) – And many more…

---

Summary

The Token Budget pattern provides a powerful approach to building AI agents with specific capabilities. This implementation demonstrates:

✅ Key Strengths: – Clean, maintainable code structure – LangChain integration for composability – Type hints and documentation – Production-ready patterns

🎯 Best Use Cases: – Applications requiring this pattern’s characteristics – Systems needing structured agent behavior – Production environments requiring reliability

📚 Learning Resources: – LangChain Documentation: https://python.langchain.com/ – Pattern Documentation: See related patterns – OpenAI API Reference: https://platform.openai.com/docs/

---

File: 088_token_budget.py
Total Lines: 863
Pattern Category: AI Agent Design Pattern #088

---

Generated from actual implementation code
For educational and reference purposes

    """
Pattern 087: Constraint Specification Pattern

Description:
    The Constraint Specification pattern explicitly defines boundaries, rules, and
    limitations that the LLM must adhere to when generating responses. This pattern
    is crucial for controlling LLM behavior, ensuring compliance with requirements,
    and preventing undesired outputs. Constraints can be content-based, format-based,
    style-based, or domain-specific, and they guide the model toward appropriate
    responses while filtering out inappropriate ones.

    Constraint specification is essential for:
    - Safety and compliance requirements
    - Brand voice and style consistency
    - Content appropriateness filtering
    - Domain-specific requirements
    - Length and complexity control
    - Factual accuracy boundaries

Components:
    1. Constraint Types
       - Content constraints (what to include/exclude)
       - Format constraints (structure and presentation)
       - Style constraints (tone, formality, vocabulary)
       - Length constraints (word/character limits)
       - Domain constraints (topic boundaries)
       - Safety constraints (harmful content prevention)

    2. Constraint Expression
       - Explicit rules ("must", "must not")
       - Positive constraints (requirements)
       - Negative constraints (prohibitions)
       - Conditional constraints (if-then rules)
       - Priority levels (strict vs flexible)

    3. Enforcement Strategies
       - Pre-generation (prompt engineering)
       - Post-generation (validation and filtering)
       - Iterative refinement (generate-validate-refine)
       - Guardrails (safety layers)
       - Fallback handling (when constraints violated)

    4. Validation Methods
       - Rule-based checking
       - Pattern matching
       - Content analysis
       - Length verification
       - Format validation
       - Safety scoring

Use Cases:
    1. Content Safety
       - Prevent harmful content generation
       - Filter inappropriate language
       - Block sensitive topics
       - Ensure age-appropriate content
       - Compliance with platform policies

    2. Brand Consistency
       - Maintain brand voice
       - Use approved terminology
       - Follow style guidelines
       - Consistent tone across content
       - Corporate communication standards

    3. Domain Expertise
       - Stay within knowledge boundaries
       - Acknowledge uncertainty
       - Avoid medical/legal claims
       - Respect professional standards
       - Domain-specific terminology

    4. Format and Structure
       - Length limitations (tweets, abstracts)
       - Structural requirements (sections, headings)
       - Citation requirements
       - Formatting rules (markdown, HTML)
       - Template compliance

    5. Legal and Compliance
       - GDPR compliance (no PII without consent)
       - HIPAA compliance (no PHI exposure)
       - Copyright respect
       - Regulatory requirements
       - Terms of service adherence

LangChain Implementation:
    LangChain supports constraint specification through:
    - System messages with explicit rules
    - Constitutional AI principles
    - Output validators and filters
    - Custom guardrails
    - Retry mechanisms with corrections

Key Features:
    1. Multi-Level Constraints
       - Hard constraints (must be satisfied)
       - Soft constraints (preferred but flexible)
       - Conditional constraints (context-dependent)
       - Hierarchical constraints (priority ordering)

    2. Constraint Composition
       - Multiple constraints simultaneously
       - Constraint conflicts resolution
       - Priority-based enforcement
       - Constraint inheritance

    3. Validation and Feedback
       - Immediate validation
       - Detailed violation reporting
       - Correction suggestions
       - Iterative improvement

    4. Dynamic Constraints
       - Context-aware constraints
       - User-specific rules
       - Time-based constraints
       - Adaptive boundaries

Best Practices:
    1. Clarity and Specificity
       - State constraints explicitly
       - Provide examples of violations
       - Clarify edge cases
       - Use concrete language

    2. Prioritization
       - Identify critical constraints
       - Allow flexibility where possible
       - Balance constraints with quality
       - Avoid over-constraining

    3. Testing
       - Test constraint enforcement
       - Try boundary cases
       - Verify violation detection
       - Check false positives

    4. Documentation
       - Document all constraints
       - Explain rationale
       - Provide violation examples
       - Maintain constraint catalog

Trade-offs:
    Advantages:
    - Better control over outputs
    - Improved safety and compliance
    - Consistent brand voice
    - Reduced inappropriate content
    - Clear expectations
    - Easier validation

    Disadvantages:
    - May limit creativity
    - Can reduce output quality
    - Increases complexity
    - Requires careful design
    - May need frequent updates
    - Potential over-constraint

Production Considerations:
    1. Constraint Management
       - Centralized constraint definitions
       - Version control for constraints
       - Easy updates and modifications
       - Constraint testing framework
       - Documentation and examples

    2. Performance Impact
       - Validation overhead
       - Retry costs for violations
       - Caching validated outputs
       - Efficient constraint checking
       - Batch validation

    3. User Experience
       - Clear violation messages
       - Suggestion for corrections
       - Graceful degradation
       - Transparency about constraints
       - User constraint customization

    4. Monitoring
       - Track constraint violations
       - Analyze violation patterns
       - Measure enforcement effectiveness
       - Identify constraint conflicts
       - User feedback collection

    5. Evolution
       - Regular constraint reviews
       - Adapt to new requirements
       - Learn from violations
       - Community feedback
       - A/B test constraint changes
"""

import os
import re
from typing import List, Dict, Any, Optional, Callable
from dataclasses import dataclass
from enum import Enum
from dotenv import load_dotenv

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_openai import ChatOpenAI

load_dotenv()


class ConstraintType(Enum):
    """Types of constraints"""
    CONTENT = "content"
    LENGTH = "length"
    STYLE = "style"
    FORMAT = "format"
    SAFETY = "safety"
    DOMAIN = "domain"


class ConstraintSeverity(Enum):
    """Severity levels for constraints"""
    STRICT = "strict"  # Must be satisfied
    MODERATE = "moderate"  # Should be satisfied
    FLEXIBLE = "flexible"  # Nice to have


@dataclass
class Constraint:
    """Represents a single constraint"""
    name: str
    description: str
    constraint_type: ConstraintType
    severity: ConstraintSeverity
    validation_func: Optional[Callable[[str], bool]] = None
    violation_message: str = "Constraint violated"


@dataclass
class ConstraintConfig:
    """Configuration for constraint-based generation"""
    max_retries: int = 3
    temperature: float = 0.5
    model_name: str = "gpt-3.5-turbo"
    fail_on_violation: bool = False


class ConstraintSpecificationAgent:
    """
    Agent that enforces constraints on LLM outputs.
    
    This agent demonstrates:
    1. Defining multiple types of constraints
    2. Enforcing constraints during generation
    3. Validating outputs against constraints
    4. Handling constraint violations
    """
    
    def __init__(self, config: Optional[ConstraintConfig] = None):
        """
        Initialize constraint specification agent.
        
        Args:
            config: Configuration for constraint handling
        """
        self.config = config or ConstraintConfig()
        self.llm = ChatOpenAI(
            model=self.config.model_name,
            temperature=self.config.temperature
        )
        self.constraints: List[Constraint] = []
    
    def add_constraint(self, constraint: Constraint):
        """Add a constraint to the agent"""
        self.constraints.append(constraint)
    
    def clear_constraints(self):
        """Remove all constraints"""
        self.constraints = []
    
    def _build_constraint_instructions(self) -> str:
        """Build constraint instructions for prompt"""
        if not self.constraints:
            return ""
        
        instructions = ["You must follow these constraints:"]
        
        for i, constraint in enumerate(self.constraints, 1):
            severity_label = {
                ConstraintSeverity.STRICT: "MUST",
                ConstraintSeverity.MODERATE: "SHOULD",
                ConstraintSeverity.FLEXIBLE: "IDEALLY"
            }[constraint.severity]
            
            instructions.append(f"{i}. {severity_label}: {constraint.description}")
        
        return "\n".join(instructions)
    
    def _validate_constraints(self, text: str) -> Dict[str, Any]:
        """
        Validate text against all constraints.
        
        Args:
            text: Text to validate
            
        Returns:
            Validation results
        """
        violations = []
        
        for constraint in self.constraints:
            if constraint.validation_func:
                if not constraint.validation_func(text):
                    violations.append({
                        "constraint": constraint.name,
                        "severity": constraint.severity.value,
                        "message": constraint.violation_message
                    })
        
        return {
            "valid": len(violations) == 0,
            "violations": violations,
            "num_violations": len(violations)
        }
    
    def generate_with_constraints(
        self,
        prompt: str,
        task_description: str = "Generate a response"
    ) -> Dict[str, Any]:
        """
        Generate text while enforcing constraints.
        
        Args:
            prompt: User prompt
            task_description: Description of task
            
        Returns:
            Generation result with validation info
        """
        constraint_instructions = self._build_constraint_instructions()
        
        template = ChatPromptTemplate.from_messages([
            ("system", """{task_description}

{constraints}

Ensure your response satisfies all constraints."""),
            ("human", "{prompt}")
        ])
        
        chain = template | self.llm | StrOutputParser()
        
        for attempt in range(self.config.max_retries):
            # Generate
            result = chain.invoke({
                "task_description": task_description,
                "constraints": constraint_instructions,
                "prompt": prompt
            })
            
            # Validate
            validation = self._validate_constraints(result)
            
            if validation["valid"]:
                return {
                    "success": True,
                    "text": result,
                    "attempts": attempt + 1,
                    "validation": validation
                }
            
            # If not last attempt, provide feedback for retry
            if attempt  str:
        """
        Generate text with length constraints.
        
        Args:
            prompt: User prompt
            max_words: Maximum word count
            min_words: Optional minimum word count
            
        Returns:
            Generated text
        """
        self.clear_constraints()
        
        # Add length constraint
        def validate_length(text: str) -> bool:
            word_count = len(text.split())
            if min_words and word_count  max_words:
                return False
            return True
        
        length_desc = f"between {min_words} and {max_words} words" if min_words else f"at most {max_words} words"
        
        self.add_constraint(Constraint(
            name="length",
            description=f"Response must be {length_desc}",
            constraint_type=ConstraintType.LENGTH,
            severity=ConstraintSeverity.STRICT,
            validation_func=validate_length,
            violation_message=f"Length constraint violated (required: {length_desc})"
        ))
        
        result = self.generate_with_constraints(prompt)
        return result["text"] if result["success"] else result.get("error", "Generation failed")
    
    def generate_with_style_constraint(
        self,
        prompt: str,
        style: str = "professional"
    ) -> str:
        """
        Generate text with style constraints.
        
        Args:
            prompt: User prompt
            style: Desired style (professional, casual, formal)
            
        Returns:
            Generated text
        """
        self.clear_constraints()
        
        style_guidelines = {
            "professional": "Use professional language, avoid slang, maintain objective tone",
            "casual": "Use conversational language, contractions are fine, friendly tone",
            "formal": "Use formal language, no contractions, academic or business tone",
            "humorous": "Use humor and wit, light-hearted tone, creative wordplay"
        }
        
        guideline = style_guidelines.get(style.lower(), style_guidelines["professional"])
        
        self.add_constraint(Constraint(
            name="style",
            description=guideline,
            constraint_type=ConstraintType.STYLE,
            severity=ConstraintSeverity.MODERATE
        ))
        
        result = self.generate_with_constraints(
            prompt,
            task_description=f"Generate a {style} response"
        )
        
        return result["text"] if result["success"] else result.get("text", "")
    
    def generate_with_content_constraints(
        self,
        prompt: str,
        must_include: Optional[List[str]] = None,
        must_exclude: Optional[List[str]] = None
    ) -> str:
        """
        Generate text with content inclusion/exclusion constraints.
        
        Args:
            prompt: User prompt
            must_include: Terms that must appear
            must_exclude: Terms that must not appear
            
        Returns:
            Generated text
        """
        self.clear_constraints()
        
        # Inclusion constraint
        if must_include:
            def validate_inclusion(text: str) -> bool:
                text_lower = text.lower()
                return all(term.lower() in text_lower for term in must_include)
            
            self.add_constraint(Constraint(
                name="content_inclusion",
                description=f"Must include these terms: {', '.join(must_include)}",
                constraint_type=ConstraintType.CONTENT,
                severity=ConstraintSeverity.STRICT,
                validation_func=validate_inclusion,
                violation_message=f"Required terms missing: {', '.join(must_include)}"
            ))
        
        # Exclusion constraint
        if must_exclude:
            def validate_exclusion(text: str) -> bool:
                text_lower = text.lower()
                return not any(term.lower() in text_lower for term in must_exclude)
            
            self.add_constraint(Constraint(
                name="content_exclusion",
                description=f"Must NOT include these terms: {', '.join(must_exclude)}",
                constraint_type=ConstraintType.CONTENT,
                severity=ConstraintSeverity.STRICT,
                validation_func=validate_exclusion,
                violation_message=f"Prohibited terms found: {', '.join(must_exclude)}"
            ))
        
        result = self.generate_with_constraints(prompt)
        return result["text"] if result["success"] else result.get("text", "")
    
    def generate_with_format_constraint(
        self,
        prompt: str,
        format_type: str = "bullet_points"
    ) -> str:
        """
        Generate text with format constraints.
        
        Args:
            prompt: User prompt
            format_type: Format type (bullet_points, numbered_list, paragraphs)
            
        Returns:
            Generated text
        """
        self.clear_constraints()
        
        format_descriptions = {
            "bullet_points": "Use bullet points (- or •) for each item. No numbered lists.",
            "numbered_list": "Use numbered list format (1. 2. 3. etc.). No bullet points.",
            "paragraphs": "Write in paragraph form with clear topic sentences. No lists.",
            "single_sentence": "Provide answer as a single, concise sentence."
        }
        
        description = format_descriptions.get(format_type, format_descriptions["bullet_points"])
        
        self.add_constraint(Constraint(
            name="format",
            description=description,
            constraint_type=ConstraintType.FORMAT,
            severity=ConstraintSeverity.STRICT
        ))
        
        result = self.generate_with_constraints(
            prompt,
            task_description=f"Generate response in {format_type} format"
        )
        
        return result["text"] if result["success"] else result.get("text", "")
    
    def generate_with_safety_constraints(
        self,
        prompt: str
    ) -> str:
        """
        Generate text with safety constraints.
        
        Args:
            prompt: User prompt
            
        Returns:
            Generated text
        """
        self.clear_constraints()
        
        safety_rules = [
            "No harmful, hateful, or violent content",
            "No personal attacks or discrimination",
            "No illegal activities or advice",
            "No explicit or inappropriate content",
            "Respectful and inclusive language"
        ]
        
        for rule in safety_rules:
            self.add_constraint(Constraint(
                name=f"safety_{safety_rules.index(rule)}",
                description=rule,
                constraint_type=ConstraintType.SAFETY,
                severity=ConstraintSeverity.STRICT
            ))
        
        result = self.generate_with_constraints(
            prompt,
            task_description="Generate a safe and appropriate response"
        )
        
        return result["text"] if result["success"] else "Unable to generate safe content for this prompt."


def demonstrate_constraint_specification():
    """Demonstrate constraint specification patterns"""
    print("=" * 80)
    print("CONSTRAINT SPECIFICATION PATTERN DEMONSTRATION")
    print("=" * 80)
    
    agent = ConstraintSpecificationAgent()
    
    # Example 1: Length Constraints
    print("\n" + "=" * 80)
    print("EXAMPLE 1: Length Constraints")
    print("=" * 80)
    
    prompt = "Explain artificial intelligence"
    print(f"\nPrompt: {prompt}")
    print("\nConstraint: Maximum 50 words\n")
    
    short_response = agent.generate_with_length_constraint(prompt, max_words=50)
    print(f"Response ({len(short_response.split())} words):")
    print(short_response)
    
    print("\nConstraint: 80-120 words\n")
    medium_response = agent.generate_with_length_constraint(prompt, max_words=120, min_words=80)
    print(f"Response ({len(medium_response.split())} words):")
    print(medium_response)
    
    # Example 2: Style Constraints
    print("\n" + "=" * 80)
    print("EXAMPLE 2: Style Constraints")
    print("=" * 80)
    
    prompt = "Describe the benefits of exercise"
    print(f"\nPrompt: {prompt}\n")
    
    styles = ["professional", "casual", "formal"]
    for style in styles:
        print(f"\n{style.upper()} STYLE:")
        print("-" * 40)
        response = agent.generate_with_style_constraint(prompt, style=style)
        print(response)
    
    # Example 3: Content Inclusion/Exclusion
    print("\n" + "=" * 80)
    print("EXAMPLE 3: Content Constraints (Must Include/Exclude)")
    print("=" * 80)
    
    prompt = "Write about healthy eating habits"
    must_include = ["vegetables", "protein", "hydration"]
    must_exclude = ["diet pills", "supplements"]
    
    print(f"\nPrompt: {prompt}")
    print(f"Must include: {must_include}")
    print(f"Must exclude: {must_exclude}\n")
    
    response = agent.generate_with_content_constraints(
        prompt,
        must_include=must_include,
        must_exclude=must_exclude
    )
    print("Response:")
    print(response)
    
    # Example 4: Format Constraints
    print("\n" + "=" * 80)
    print("EXAMPLE 4: Format Constraints")
    print("=" * 80)
    
    prompt = "List the steps to start a business"
    print(f"\nPrompt: {prompt}\n")
    
    formats = ["bullet_points", "numbered_list", "paragraphs"]
    for fmt in formats:
        print(f"\n{fmt.upper().replace('_', ' ')} FORMAT:")
        print("-" * 40)
        response = agent.generate_with_format_constraint(prompt, format_type=fmt)
        print(response)
    
    # Example 5: Multiple Constraints
    print("\n" + "=" * 80)
    print("EXAMPLE 5: Multiple Simultaneous Constraints")
    print("=" * 80)
    
    agent.clear_constraints()
    
    # Add multiple constraints
    agent.add_constraint(Constraint(
        name="length",
        description="Response must be 60-80 words",
        constraint_type=ConstraintType.LENGTH,
        severity=ConstraintSeverity.STRICT,
        validation_func=lambda t: 60 <= len(t.split()) <= 80,
        violation_message="Length must be 60-80 words"
    ))
    
    agent.add_constraint(Constraint(
        name="style",
        description="Use professional, objective tone",
        constraint_type=ConstraintType.STYLE,
        severity=ConstraintSeverity.MODERATE
    ))
    
    agent.add_constraint(Constraint(
        name="content",
        description="Must mention 'innovation' and 'technology'",
        constraint_type=ConstraintType.CONTENT,
        severity=ConstraintSeverity.STRICT,
        validation_func=lambda t: "innovation" in t.lower() and "technology" in t.lower(),
        violation_message="Must include 'innovation' and 'technology'"
    ))
    
    prompt = "Describe the future of work"
    print(f"\nPrompt: {prompt}")
    print("\nConstraints:")
    print("1. Length: 60-80 words")
    print("2. Style: Professional, objective")
    print("3. Content: Must mention 'innovation' and 'technology'\n")
    
    result = agent.generate_with_constraints(prompt)
    print(f"Success: {result['success']}")
    print(f"Attempts: {result['attempts']}")
    print(f"\nResponse ({len(result['text'].split())} words):")
    print(result['text'])
    
    # Example 6: Safety Constraints
    print("\n" + "=" * 80)
    print("EXAMPLE 6: Safety Constraints")
    print("=" * 80)
    
    safe_prompt = "Give advice on managing workplace conflicts"
    print(f"\nPrompt: {safe_prompt}")
    print("\nApplying safety constraints...\n")
    
    response = agent.generate_with_safety_constraints(safe_prompt)
    print("Response:")
    print(response)
    
    # Summary
    print("\n" + "=" * 80)
    print("CONSTRAINT SPECIFICATION SUMMARY")
    print("=" * 80)
    print("""
Constraint Specification Pattern Benefits:
1. Control: Precise control over LLM outputs
2. Compliance: Ensures regulatory and policy adherence
3. Consistency: Maintains brand voice and standards
4. Safety: Prevents inappropriate content
5. Quality: Enforces quality standards
6. Integration: Easier to integrate with systems

Constraint Types Demonstrated:
1. Length Constraints: Word/character limits
2. Style Constraints: Tone, formality, voice
3. Content Constraints: Required/prohibited terms
4. Format Constraints: Structure and presentation
5. Safety Constraints: Harmful content prevention
6. Multiple Constraints: Combined requirements

Constraint Severity Levels:
- STRICT: Must be satisfied (violations cause failure)
- MODERATE: Should be satisfied (violations logged)
- FLEXIBLE: Nice to have (soft preferences)

Enforcement Strategies:
1. Pre-generation: Include constraints in prompt
2. Validation: Check outputs against rules
3. Iteration: Retry with feedback on violations
4. Fallback: Alternative approach when constraints can't be met

Best Practices:
1. Be explicit and specific with constraints
2. Prioritize constraints (strict vs flexible)
3. Provide examples of acceptable outputs
4. Test constraint enforcement thoroughly
5. Handle violations gracefully
6. Monitor constraint effectiveness
7. Update constraints based on feedback

Common Use Cases:
- Content moderation (safety)
- Brand consistency (style)
- Legal compliance (domain rules)
- Format standardization (structure)
- Length requirements (summaries, tweets)
- Quality control (standards)

Production Considerations:
- Balance constraints with creativity
- Allow appropriate flexibility
- Provide clear violation messages
- Implement retry mechanisms
- Monitor constraint violation rates
- A/B test constraint effectiveness
- Document all constraints clearly
- Version control constraint rules

Constraint Design Tips:
1. Start with essential constraints only
2. Add constraints incrementally
3. Test against diverse inputs
4. Avoid over-constraining
5. Make constraints measurable
6. Provide violation feedback
7. Allow for constraint evolution

When to Use Constraints:
- Safety-critical applications
- Regulated industries
- Brand-sensitive content
- System integration requirements
- Quality assurance needs
- Compliance requirements

Trade-offs:
✅ Better control and safety
✅ Consistent outputs
✅ Easier validation
❌ May limit creativity
❌ Requires careful design
❌ Can be overly restrictive
""")
    
    print("\n" + "=" * 80)
    print("Pattern 087 (Constraint Specification) demonstration complete!")
    print("=" * 80)


if __name__ == "__main__":
    demonstrate_constraint_specification()

    

Agentic Design Pattern: Constraint Specification

Pattern 087: Constraint Specification – Line-by-Line Explanation

Overview

This document provides a detailed line-by-line explanation of the Constraint Specification pattern implementation.

---

Pattern Description

Pattern 087: Constraint Specification Pattern

Description: The Constraint Specification pattern explicitly defines boundaries, rules, and limitations that the LLM must adhere to when generating responses. This pattern is crucial for controlling LLM behavior, ensuring compliance with requirements, and preventing undesired outputs. Constraints can be content-based, format-based, style-based, or domain-specific, and they guide the model toward appropriate responses while filtering out inappropriate ones.

Constraint specification is essential for:
- Safety and compliance requirements
- Brand voice and style consistency
- Content appropriateness filtering
- Domain-specific requirements
- Length and complexity control
- Factual accuracy boundaries

Components: 1. Constraint Types – Content constraints (what to include/exclude) – Format constraints (structure and presentation) – Style constraints (tone, formality, vocabulary) – Length constraints (word/character limits) – Domain constraints (topic boundaries) – Safety constraints (harmful content prevention)

2. Constraint Expression
   - Explicit rules ("must", "must not")
   - Positive constraints (requirements)
   - Negative constraints (prohibitions)
   - Conditional constraints (if-then rules)
   - Priority levels (strict vs flexible)

3. Enforcement Strategies
   - Pre-generation (prompt engineering)
   - Post-generation (validation and filtering)
   - Iterative refinement (generate-validate-refine)
   - Guardrails (safety layers)
   - Fallback handling (when constraints violated)

4. Validation Methods
   - Rule-based checking
   - Pattern matching
   - Content analysis
   - Length verification
   - Format validation
   - Safety scoring

Use Cases: 1. Content Safety – Prevent harmful content generation – Filter inappropriate language – Block sensitive topics – Ensure age-appropriate content – Compliance with platform policies

2. Brand Consistency
   - Maintain brand voice
   - Use approved terminology
   - Follow style guidelines
   - Consistent tone across content
   - Corporate communication standards

3. Domain Expertise
   - Stay within knowledge boundaries
   - Acknowledge uncertainty
   - Avoid medical/legal claims
   - Respect professional standards
   - Domain-specific terminology

4. Format and Structure
   - Length limitations (tweets, abstracts)
   - Structural requirements (sections, headings)
   - Citation requirements
   - Formatting rules (markdown, HTML)
   - Template compliance

5. Legal and Compliance
   - GDPR compliance (no PII without consent)
   - HIPAA compliance (no PHI exposure)
   - Copyright respect
   - Regulatory requirements
   - Terms of service adherence

LangChain Implementation: LangChain supports constraint specification through: – System messages with explicit rules – Constitutional AI principles – Output validators and filters – Custom guardrails – Retry mechanisms with corrections

Key Features: 1. Multi-Level Constraints – Hard constraints (must be satisfied) – Soft constraints (preferred but flexible) – Conditional constraints (context-dependent) – Hierarchical constraints (priority ordering)

2. Constraint Composition
   - Multiple constraints simultaneously
   - Constraint conflicts resolution
   - Priority-based enforcement
   - Constraint inheritance

3. Validation and Feedback
   - Immediate validation
   - Detailed violation reporting
   - Correction suggestions
   - Iterative improvement

4. Dynamic Constraints
   - Context-aware constraints
   - User-specific rules
   - Time-based constraints
   - Adaptive boundaries

Best Practices: 1. Clarity and Specificity – State constraints explicitly – Provide examples of violations – Clarify edge cases – Use concrete language

2. Prioritization
   - Identify critical constraints
   - Allow flexibility where possible
   - Balance constraints with quality
   - Avoid over-constraining

3. Testing
   - Test constraint enforcement
   - Try boundary cases
   - Verify violation detection
   - Check false positives

4. Documentation
   - Document all constraints
   - Explain rationale
   - Provide violation examples
   - Maintain constraint catalog

Trade-offs: Advantages: – Better control over outputs – Improved safety and compliance – Consistent brand voice – Reduced inappropriate content – Clear expectations – Easier validation

Disadvantages:
- May limit creativity
- Can reduce output quality
- Increases complexity
- Requires careful design
- May need frequent updates
- Potential over-constraint

Production Considerations: 1. Constraint Management – Centralized constraint definitions – Version control for constraints – Easy updates and modifications – Constraint testing framework – Documentation and examples

2. Performance Impact
   - Validation overhead
   - Retry costs for violations
   - Caching validated outputs
   - Efficient constraint checking
   - Batch validation

3. User Experience
   - Clear violation messages
   - Suggestion for corrections
   - Graceful degradation
   - Transparency about constraints
   - User constraint customization

4. Monitoring
   - Track constraint violations
   - Analyze violation patterns
   - Measure enforcement effectiveness
   - Identify constraint conflicts
   - User feedback collection

5. Evolution
   - Regular constraint reviews
   - Adapt to new requirements
   - Learn from violations
   - Community feedback
   - A/B test constraint changes

---

File Statistics

• Total Lines: 835
• Classes: 5
• Functions: 1
• Constants: 0

---

Module Documentation

"""
Pattern 087: Constraint Specification Pattern

Description:
    The Constraint Specification pattern explicitly defines boundaries, rules, and
    limitations that the LLM must adhere to when generating responses. This pattern
    is crucial for controlling LLM behavior, ensuring compliance with requirements,
    and preventing undesired outputs. Constraints can be content-based, format-based,
    style-based, or domain-specific, and they guide the model toward appropriate
    responses while filtering out inappropriate ones.

    Constraint specification is essential for:
    - Safety and compliance requirements
    - Brand voice and style consistency
    - Content appropriateness filtering
    - Domain-specific requirements
    - Length and complexity control
    - Factual accuracy boundaries

Components:
    1. Constraint Types
       - Content constraints (what to include/exclude)
       - Format constraints (structure and presentation)
       - Style constraints (tone, formality, vocabulary)
       - Length constraints (word/character limits)
       - Domain constraints (topic boundaries)
       - Safety constraints (harmful content prevention)

    2. Constraint Expression
       - Explicit rules ("must", "must not")
       - Positive constraints (requirements)
       - Negative constraints (prohibitions)
       - Conditional constraints (if-then rules)
       - Priority levels (strict vs flexible)

    3. Enforcement Strategies
       - Pre-generation (prompt engineering)
       - Post-generation (validation and filtering)
       - Iterative refinement (generate-validate-refine)
       - Guardrails (safety layers)
       - Fallback handling (when constraints violated)

    4. Validation Methods
       - Rule-based checking
       - Pattern matching
       - Content analysis
       - Length verification
       - Format validation
       - Safety scoring

Use Cases:
    1. Content Safety
       - Prevent harmful content generation
       - Filter inappropriate language
       - Block sensitive topics
       - Ensure age-appropriate content
       - Compliance with platform policies

    2. Brand Consistency
       - Maintain brand voice
       - Use approved terminology
       - Follow style guidelines
       - Consistent tone across content
       - Corporate communication standards

    3. Domain Expertise
       - Stay within knowledge boundaries
       - Acknowledge uncertainty
       - Avoid medical/legal claims
       - Respect professional standards
       - Domain-specific terminology

    4. Format and Structure
       - Length limitations (tweets, abstracts)
       - Structural requirements (sections, headings)
       - Citation requirements
       - Formatting rules (markdown, HTML)
       - Template compliance

    5. Legal and Compliance
       - GDPR compliance (no PII without consent)
       - HIPAA compliance (no PHI exposure)
       - Copyright respect
       - Regulatory requirements
       - Terms of service adherence

LangChain Implementation:
    LangChain supports constraint specification through:
    - System messages with explicit rules
    - Constitutional AI principles
    - Output validators and filters
    - Custom guardrails
    - Retry mechanisms with corrections

Key Features:
    1. Multi-Level Constraints
       - Hard constraints (must be satisfied)
       - Soft constraints (preferred but flexible)
       - Conditional constraints (context-dependent)
       - Hierarchical constraints (priority ordering)

    2. Constraint Composition
       - Multiple constraints simultaneously
       - Constraint conflicts resolution
       - Priority-based enforcement
       - Constraint inheritance

    3. Validation and Feedback
       - Immediate validation
       - Detailed violation reporting
       - Correction suggestions
       - Iterative improvement

    4. Dynamic Constraints
       - Context-aware constraints
       - User-specific rules
       - Time-based constraints
       - Adaptive boundaries

Best Practices:
    1. Clarity and Specificity
       - State constraints explicitly
       - Provide examples of violations
       - Clarify edge cases
       - Use concrete language

    2. Prioritization
       - Identify critical constraints
       - Allow flexibility where possible
       - Balance constraints with quality
       - Avoid over-constraining

    3. Testing
       - Test constraint enforcement
       - Try boundary cases
       - Verify violation detection
       - Check false positives

    4. Documentation
       - Document all constraints
       - Explain rationale
       - Provide violation examples
       - Maintain constraint catalog

Trade-offs:
    Advantages:
    - Better control over outputs
    - Improved safety and compliance
    - Consistent brand voice
    - Reduced inappropriate content
    - Clear expectations
    - Easier validation

    Disadvantages:
    - May limit creativity
    - Can reduce output quality
    - Increases complexity
    - Requires careful design
    - May need frequent updates
    - Potential over-constraint

Production Considerations:
    1. Constraint Management
       - Centralized constraint definitions
       - Version control for constraints
       - Easy updates and modifications
       - Constraint testing framework
       - Documentation and examples

    2. Performance Impact
       - Validation overhead
       - Retry costs for violations
       - Caching validated outputs
       - Efficient constraint checking
       - Batch validation

    3. User Experience
       - Clear violation messages
       - Suggestion for corrections
       - Graceful degradation
       - Transparency about constraints
       - User constraint customization

    4. Monitoring
       - Track constraint violations
       - Analyze violation patterns
       - Measure enforcement effectiveness
       - Identify constraint conflicts
       - User feedback collection

    5. Evolution
       - Regular constraint reviews
       - Adapt to new requirements
       - Learn from violations
       - Community feedback
       - A/B test constraint changes
"""

Purpose: This module implements the Constraint Specification pattern, providing a structured approach to agent design and implementation.

---

Import Statements

Standard Library Imports

import dataclasses.dataclass
import enum.Enum
import langchain_core.output_parsers.StrOutputParser
import langchain_core.prompts.ChatPromptTemplate
import os
import re
import typing.Any
import typing.Callable
import typing.Dict
import typing.List
import typing.Optional

Purpose: Core Python functionality: – dataclasses.dataclass: Simplified class definitions with @dataclass – enum.Enum: Enumeration support – langchain_core.output_parsers.StrOutputParser: Regular expression operations – langchain_core.prompts.ChatPromptTemplate: Regular expression operations – os: Environment variable access and system operations – re: Regular expression operations – typing.Any: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Callable: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Dict: Type hints for better code documentation (List, Dict, Any, Optional) – typing.List: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Optional: Type hints for better code documentation (List, Dict, Any, Optional)

LangChain Imports

from langchain_openai.ChatOpenAI

Purpose: LangChain framework components: – LLM Interface: Wrapper for OpenAI models – Chains: Composable building blocks for complex workflows

Other Imports

from dotenv.load_dotenv

Purpose: Additional dependencies for specialized functionality.

---

Classes

Class: ConstraintType

Lines 213-220

Inherits from: Enum

Purpose:

Types of constraints

---

Class: ConstraintSeverity

Lines 223-227

Inherits from: Enum

Purpose:

Severity levels for constraints

---

Class: Constraint

Lines 231-238

Purpose:

Represents a single constraint

---

Class: ConstraintConfig

Lines 242-247

Purpose:

Configuration for constraint-based generation

---

Class: ConstraintSpecificationAgent

Lines 250-604

Purpose:

Agent that enforces constraints on LLM outputs.

This agent demonstrates:
1. Defining multiple types of constraints
2. Enforcing constraints during generation
3. Validating outputs against constraints
4. Handling constraint violations

Methods:

__init__(self, config)

Lines 261-273

Purpose:

Initialize constraint specification agent.

Args:
    config: Configuration for constraint handling

Parameters: – config: Config

add_constraint(self, constraint)

Lines 275-277

Purpose:

Add a constraint to the agent

Parameters: – constraint: Constraint

clear_constraints(self)

Lines 279-281

Purpose:

Remove all constraints

Parameters:

_build_constraint_instructions(self)

Lines 283-299

Purpose:

Build constraint instructions for prompt

Parameters:

Returns: str

_validate_constraints(self, text)

Lines 301-326

Purpose:

Validate text against all constraints.

Args:
    text: Text to validate

Returns:
    Validation results

Parameters: – text: Text

Returns: <ast.Subscript object at 0x000001A06D7F4B90>

generate_with_constraints(self, prompt, task_description)

Lines 328-392

Purpose:

Generate text while enforcing constraints.

Args:
    prompt: User prompt
    task_description: Description of task

Returns:
    Generation result with validation info

Parameters: – prompt: Prompt template or text – task_description: Task description

Returns: <ast.Subscript object at 0x000001A06D849590>

generate_with_length_constraint(self, prompt, max_words, min_words)

Lines 394-434

Purpose:

Generate text with length constraints.

Args:
    prompt: User prompt
    max_words: Maximum word count
    min_words: Optional minimum word count

Returns:
    Generated text

Parameters: – prompt: Prompt template or text – max_words: Max words – min_words: Min words

Returns: str

generate_with_style_constraint(self, prompt, style)

Lines 436-474

Purpose:

Generate text with style constraints.

Args:
    prompt: User prompt
    style: Desired style (professional, casual, formal)

Returns:
    Generated text

Parameters: – prompt: Prompt template or text – style: Style

Returns: str

generate_with_content_constraints(self, prompt, must_include, must_exclude)

Lines 476-526

Purpose:

Generate text with content inclusion/exclusion constraints.

Args:
    prompt: User prompt
    must_include: Terms that must appear
    must_exclude: Terms that must not appear

Returns:
    Generated text

Parameters: – prompt: Prompt template or text – must_include: Must include – must_exclude: Must exclude

Returns: str

generate_with_format_constraint(self, prompt, format_type)

Lines 528-566

Purpose:

Generate text with format constraints.

Args:
    prompt: User prompt
    format_type: Format type (bullet_points, numbered_list, paragraphs)

Returns:
    Generated text

Parameters: – prompt: Prompt template or text – format_type: Format type

Returns: str

generate_with_safety_constraints(self, prompt)

Lines 568-604

Purpose:

Generate text with safety constraints.

Args:
    prompt: User prompt

Returns:
    Generated text

Parameters: – prompt: Prompt template or text

Returns: str

---

Functions

demonstrate_constraint_specification()

Lines 607-830

Purpose:

Demonstrate constraint specification patterns

---

Key Concepts Summary

Pattern Overview

The Constraint Specification pattern provides a structured approach to building AI agents with specific capabilities and behaviors. This implementation leverages LangChain’s composable components to create a robust and maintainable solution.

Core Components

1. Agent Class: Main implementation of the pattern logic
2. Prompt Engineering: Carefully crafted prompts to guide LLM behavior
3. Tool Integration: Optional tools for extended functionality
4. Memory Management: State handling across interactions
5. Output Parsing: Structured response processing

When to Use

✅ Good for: – Scenarios requiring this specific pattern’s strengths – Applications needing structured agent behavior – Systems where explainability is important – Production environments requiring reliability

❌ Not ideal for: – Simple single-prompt tasks – Scenarios requiring different pattern characteristics – Resource-constrained environments (if computationally intensive)

---

Best Practices Demonstrated

1. Environment Configuration

• ✅ Use .env files for API keys
• ✅ Never hardcode credentials
• ✅ Validate environment variables on startup

2. Error Handling

• ✅ Graceful degradation on failures
• ✅ Informative error messages
• ✅ Logging for debugging

3. Prompt Engineering

• ✅ Clear, specific instructions
• ✅ Structured output formats
• ✅ Few-shot examples when beneficial

4. Code Organization

• ✅ Separation of concerns (class-based design)
• ✅ Type hints for clarity
• ✅ Docstrings for documentation

5. Performance Optimization

• ✅ Appropriate temperature settings
• ✅ Token budget management
• ✅ Caching where applicable

---

LangChain Components Used

• ChatOpenAI: OpenAI model interface
• Chains: Sequential component composition

Why LangChain? – Composable abstractions for LLM applications – Rich ecosystem of integrations – Production-ready patterns – Active community and development

---

Execution Flow

1. Initialize agent with configuration
   ↓
2. Load environment and dependencies
   ↓
3. Prepare prompt with input data
   ↓
4. Invoke LLM through LangChain
   ↓
5. Parse and validate response
   ↓
6. Return formatted result

Key Decision Points: – Model selection (temperature, model type) – Prompt structure (few-shot vs zero-shot) – Error handling strategy – Output format validation

---

Performance Considerations

Computational Cost

• LLM Calls: Primary cost factor (based on tokens)
• Latency: Depends on model and input size
• Memory: Minimal for most patterns

Optimization Strategies

1. Prompt Optimization: Minimize tokens while maintaining clarity
2. Caching: Cache repeated queries/responses
3. Model Selection: Use smaller models when appropriate
4. Batch Processing: Group similar requests
5. Streaming: For long responses, stream tokens

Cost Estimates (Approximate)

• GPT-3.5-turbo: ~$0.002 per request
• GPT-4: ~$0.03 per request
• GPT-4-turbo: ~$0.01 per request

Actual costs vary based on input/output token counts

---

Common Pitfalls to Avoid

1. Prompt Design Issues

❌ Vague or ambiguous instructions
✅ Clear, specific, testable prompts

2. Error Handling

❌ Ignoring API errors or rate limits
✅ Implement retry logic and graceful degradation

3. Token Management

❌ Exceeding model token limits
✅ Truncate inputs, chunk processing

4. Security

❌ Exposing API keys in code
✅ Use environment variables

5. Testing

❌ Only testing happy paths
✅ Test edge cases, failures, malformed inputs

6. Cost Management

❌ Unmonitored API usage
✅ Implement budgets, alerts, and monitoring

---

Further Improvements

Production Enhancements

1. Monitoring: Add logging, metrics, tracing
2. Validation: Input/output schema validation
3. Rate Limiting: Implement request throttling
4. Caching: Redis or in-memory cache for responses
5. Async Support: Non-blocking operations for scalability

Feature Extensions

1. Multi-Model Support: Switch between providers
2. Fallback Models: Graceful degradation strategy
3. Custom Tools: Domain-specific function calling
4. Advanced Memory: Vector stores for long-term memory
5. Evaluation: Automated testing and benchmarking

Code Quality

1. Testing: Unit tests, integration tests
2. Documentation: API docs, usage examples
3. Type Safety: Stricter type hints
4. Configuration: Externalized config files

---

Related Patterns

• Pattern 086
• Pattern 088

Pattern Families: – Reasoning Patterns (001-011) – Multi-Agent Patterns (012-019) – Tool Use Patterns (020-025) – Memory Patterns (026-031) – And many more…

---

Summary

The Constraint Specification pattern provides a powerful approach to building AI agents with specific capabilities. This implementation demonstrates:

✅ Key Strengths: – Clean, maintainable code structure – LangChain integration for composability – Type hints and documentation – Production-ready patterns

🎯 Best Use Cases: – Applications requiring this pattern’s characteristics – Systems needing structured agent behavior – Production environments requiring reliability

📚 Learning Resources: – LangChain Documentation: https://python.langchain.com/ – Pattern Documentation: See related patterns – OpenAI API Reference: https://platform.openai.com/docs/

---

File: 087_constraint_specification.py
Total Lines: 835
Pattern Category: AI Agent Design Pattern #087

---

Generated from actual implementation code
For educational and reference purposes

    """
Pattern 086: Output Format Specification Pattern

Description:
    The Output Format Specification pattern explicitly defines the structure, format,
    and schema of the LLM's response. This pattern is crucial for integrating LLM
    outputs with downstream systems, APIs, and applications that require consistent,
    parseable, and structured data. By specifying the exact format upfront, this
    pattern eliminates ambiguity and ensures reliable programmatic processing.

    Format specification is essential for:
    - API integration and data exchange
    - Structured data extraction
    - Multi-step workflows with defined interfaces
    - Database storage and retrieval
    - User interface rendering
    - Automated testing and validation

Components:
    1. Format Type
       - JSON (JavaScript Object Notation)
       - XML (Extensible Markup Language)
       - YAML (YAML Ain't Markup Language)
       - CSV (Comma-Separated Values)
       - Markdown (formatted text)
       - Custom formats (domain-specific)

    2. Schema Definition
       - Field names and types
       - Required vs optional fields
       - Value constraints and ranges
       - Nested structures
       - Array/list specifications

    3. Format Enforcement
       - Clear examples in prompts
       - Schema validation
       - Error handling for format violations
       - Retry mechanisms
       - Fallback strategies

    4. Parsing and Validation
       - Automated parsing
       - Schema compliance checking
       - Type validation
       - Constraint verification
       - Error reporting

Use Cases:
    1. Data Extraction
       - Entity extraction from text → JSON
       - Information retrieval → structured records
       - Document parsing → database entries
       - Web scraping → CSV data
       - Form filling → key-value pairs

    2. API Integration
       - API request generation → JSON payloads
       - Response formatting → standardized structures
       - Webhook payloads → event schemas
       - Configuration files → YAML/JSON
       - Data exchange → XML documents

    3. Report Generation
       - Business reports → Markdown tables
       - Data summaries → formatted text
       - Analytics → JSON metrics
       - Documentation → structured sections
       - Dashboards → visualization specs

    4. Code Generation
       - Function signatures → specific syntax
       - Configuration files → language-specific format
       - Test cases → structured test specs
       - API documentation → OpenAPI/Swagger
       - Database schemas → SQL/DDL

    5. Content Creation
       - Blog posts → Markdown with frontmatter
       - Product descriptions → structured fields
       - Email templates → HTML with placeholders
       - Social media → platform-specific formats
       - Translations → parallel text structures

LangChain Implementation:
    LangChain provides excellent format control through:
    - StructuredOutputParser for schema-based parsing
    - JsonOutputParser for JSON responses
    - PydanticOutputParser for type-safe outputs
    - Custom output parsers for specific formats
    - Output fixing parser for auto-correction

Key Features:
    1. Type Safety
       - Pydantic models for validation
       - Automatic type checking
       - Type coercion when possible
       - Clear error messages

    2. Schema Validation
       - JSON Schema compliance
       - Custom validation rules
       - Nested object validation
       - Array element validation

    3. Error Recovery
       - Automatic retry with corrections
       - Format fixing parsers
       - Fallback to alternative formats
       - Graceful degradation

    4. Multiple Format Support
       - JSON (most common)
       - XML for legacy systems
       - CSV for tabular data
       - Markdown for documents
       - Custom formats as needed

Best Practices:
    1. Clear Format Specification
       - Provide explicit format examples
       - Use schema definitions when possible
       - Show complete structure (not partial)
       - Include edge cases in examples

    2. Validation Strategy
       - Validate immediately after generation
       - Provide clear error messages
       - Implement retry logic
       - Log format violations

    3. Error Handling
       - Graceful fallbacks
       - User-friendly error messages
       - Automatic format correction when possible
       - Manual intervention for critical failures

    4. Documentation
       - Document expected formats
       - Provide format examples
       - Explain field meanings
       - Note format versions

Trade-offs:
    Advantages:
    - Reliable programmatic processing
    - Easy integration with systems
    - Reduced parsing errors
    - Type safety and validation
    - Better error detection
    - Automated testing possible

    Disadvantages:
    - More rigid (less flexibility)
    - Format violations require handling
    - Increases prompt complexity
    - May constrain natural language quality
    - Requires careful schema design
    - Parsing overhead

Production Considerations:
    1. Schema Evolution
       - Version format schemas
       - Backward compatibility
       - Migration strategies
       - Deprecation policies

    2. Performance
       - Parsing efficiency
       - Validation overhead
       - Caching strategies
       - Batch processing

    3. Reliability
       - Retry logic for format errors
       - Format fixing mechanisms
       - Fallback formats
       - Monitoring and alerting

    4. Maintenance
       - Schema documentation
       - Format validation tests
       - Breaking change detection
       - Client SDK generation

    5. Security
       - Input sanitization
       - Injection prevention
       - Size limits
       - Type validation
"""

import os
import json
from typing import List, Dict, Any, Optional
from dataclasses import dataclass
from enum import Enum
from dotenv import load_dotenv
from pydantic import BaseModel, Field

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import (
    StrOutputParser,
    JsonOutputParser,
    PydanticOutputParser
)
from langchain_openai import ChatOpenAI

load_dotenv()


class OutputFormat(Enum):
    """Supported output formats"""
    JSON = "json"
    XML = "xml"
    MARKDOWN = "markdown"
    CSV = "csv"
    YAML = "yaml"


# Pydantic models for structured outputs
class Person(BaseModel):
    """Person entity"""
    name: str = Field(description="Full name of the person")
    age: Optional[int] = Field(description="Age in years", ge=0, le=150)
    email: Optional[str] = Field(description="Email address")
    occupation: Optional[str] = Field(description="Job title or occupation")


class Product(BaseModel):
    """Product information"""
    name: str = Field(description="Product name")
    category: str = Field(description="Product category")
    price: float = Field(description="Price in USD", gt=0)
    in_stock: bool = Field(description="Availability status")
    rating: Optional[float] = Field(description="Average rating", ge=0, le=5)


class Article(BaseModel):
    """Article/content structure"""
    title: str = Field(description="Article title")
    author: str = Field(description="Author name")
    summary: str = Field(description="Brief summary")
    tags: List[str] = Field(description="Content tags")
    word_count: int = Field(description="Approximate word count", gt=0)


class SentimentAnalysis(BaseModel):
    """Sentiment analysis result"""
    text: str = Field(description="Analyzed text")
    sentiment: str = Field(description="Sentiment: positive, negative, or neutral")
    confidence: float = Field(description="Confidence score", ge=0, le=1)
    keywords: List[str] = Field(description="Key terms identified")


@dataclass
class FormatConfig:
    """Configuration for format specification"""
    format_type: OutputFormat = OutputFormat.JSON
    strict_mode: bool = True
    temperature: float = 0.1  # Lower for more consistent formatting
    model_name: str = "gpt-3.5-turbo"


class OutputFormatAgent:
    """
    Agent that enforces specific output formats.
    
    This agent demonstrates:
    1. JSON format specification
    2. Pydantic-based type-safe outputs
    3. Multiple format support
    4. Schema validation
    """
    
    def __init__(self, config: Optional[FormatConfig] = None):
        """
        Initialize output format agent.
        
        Args:
            config: Configuration for format handling
        """
        self.config = config or FormatConfig()
        self.llm = ChatOpenAI(
            model=self.config.model_name,
            temperature=self.config.temperature
        )
    
    def extract_as_json(
        self,
        text: str,
        schema_description: str
    ) -> Dict[str, Any]:
        """
        Extract information in JSON format.
        
        Args:
            text: Text to extract from
            schema_description: Description of desired JSON structure
            
        Returns:
            Extracted data as dictionary
        """
        json_parser = JsonOutputParser()
        
        prompt = ChatPromptTemplate.from_messages([
            ("system", """Extract information from the text in JSON format.

Format requirements:
- Valid JSON only
- Follow the schema described
- Use appropriate data types
- Include all requested fields

{format_instructions}"""),
            ("human", "Extract from this text: {text}\n\nSchema: {schema}")
        ])
        
        chain = prompt | self.llm | json_parser
        
        try:
            result = chain.invoke({
                "text": text,
                "schema": schema_description,
                "format_instructions": json_parser.get_format_instructions()
            })
            return result
        except Exception as e:
            return {"error": str(e), "raw_text": text}
    
    def extract_person_info(self, text: str) -> Person:
        """
        Extract person information with type safety.
        
        Args:
            text: Text containing person information
            
        Returns:
            Person object with validated fields
        """
        parser = PydanticOutputParser(pydantic_object=Person)
        
        prompt = ChatPromptTemplate.from_messages([
            ("system", """Extract person information from the text.

{format_instructions}

Provide valid data for all available fields."""),
            ("human", "{text}")
        ])
        
        chain = prompt | self.llm | parser
        
        result = chain.invoke({
            "text": text,
            "format_instructions": parser.get_format_instructions()
        })
        
        return result
    
    def extract_product_info(self, text: str) -> Product:
        """
        Extract product information with validation.
        
        Args:
            text: Text containing product information
            
        Returns:
            Product object with validated fields
        """
        parser = PydanticOutputParser(pydantic_object=Product)
        
        prompt = ChatPromptTemplate.from_messages([
            ("system", """Extract product information from the text.

{format_instructions}

Ensure all data types are correct and constraints are met."""),
            ("human", "{text}")
        ])
        
        chain = prompt | self.llm | parser
        
        result = chain.invoke({
            "text": text,
            "format_instructions": parser.get_format_instructions()
        })
        
        return result
    
    def analyze_sentiment_structured(self, text: str) -> SentimentAnalysis:
        """
        Perform sentiment analysis with structured output.
        
        Args:
            text: Text to analyze
            
        Returns:
            SentimentAnalysis object with validated fields
        """
        parser = PydanticOutputParser(pydantic_object=SentimentAnalysis)
        
        prompt = ChatPromptTemplate.from_messages([
            ("system", """Analyze the sentiment of the text and provide results in the specified format.

{format_instructions}

- Sentiment must be: positive, negative, or neutral
- Confidence must be between 0 and 1
- Extract 3-5 relevant keywords"""),
            ("human", "{text}")
        ])
        
        chain = prompt | self.llm | parser
        
        result = chain.invoke({
            "text": text,
            "format_instructions": parser.get_format_instructions()
        })
        
        return result
    
    def generate_markdown_report(
        self,
        topic: str,
        sections: List[str]
    ) -> str:
        """
        Generate report in markdown format.
        
        Args:
            topic: Report topic
            sections: List of section titles
            
        Returns:
            Markdown formatted report
        """
        sections_list = "\n".join([f"- {s}" for s in sections])
        
        prompt = ChatPromptTemplate.from_messages([
            ("system", """Generate a report in Markdown format.

Format requirements:
- Use # for main title
- Use ## for section headers
- Use **bold** for emphasis
- Use bullet points (- ) for lists
- Use code blocks (```) for code
- Maintain consistent formatting"""),
            ("human", """Create a report on: {topic}

Include these sections:
{sections}""")
        ])
        
        chain = prompt | self.llm | StrOutputParser()
        
        result = chain.invoke({
            "topic": topic,
            "sections": sections_list
        })
        
        return result
    
    def generate_csv_format(
        self,
        data_description: str,
        num_rows: int = 5
    ) -> str:
        """
        Generate data in CSV format.
        
        Args:
            data_description: Description of data to generate
            num_rows: Number of data rows
            
        Returns:
            CSV formatted data
        """
        prompt = ChatPromptTemplate.from_messages([
            ("system", """Generate data in CSV format.

Format requirements:
- First row: column headers
- Subsequent rows: data values
- Use commas as separators
- Quote strings containing commas
- No extra whitespace
- Valid CSV syntax"""),
            ("human", """Generate {num_rows} rows of CSV data for: {description}""")
        ])
        
        chain = prompt | self.llm | StrOutputParser()
        
        result = chain.invoke({
            "description": data_description,
            "num_rows": num_rows
        })
        
        return result
    
    def generate_xml_format(self, data_description: str) -> str:
        """
        Generate data in XML format.
        
        Args:
            data_description: Description of data structure
            
        Returns:
            XML formatted data
        """
        prompt = ChatPromptTemplate.from_messages([
            ("system", """Generate data in XML format.

Format requirements:
- Proper XML declaration
- Well-formed tags (open and close)
- Proper nesting
- Attribute syntax when appropriate
- Valid XML structure"""),
            ("human", "Generate XML for: {description}")
        ])
        
        chain = prompt | self.llm | StrOutputParser()
        
        result = chain.invoke({"description": data_description})
        
        return result
    
    def compare_formats(self, data: str) -> Dict[str, str]:
        """
        Generate same data in multiple formats for comparison.
        
        Args:
            data: Data description
            
        Returns:
            Dictionary with data in different formats
        """
        formats = {}
        
        # JSON format
        json_result = self.extract_as_json(
            data,
            "Extract as JSON with appropriate fields"
        )
        formats["json"] = json.dumps(json_result, indent=2)
        
        # XML format
        formats["xml"] = self.generate_xml_format(data)
        
        # CSV format (if tabular)
        formats["csv"] = self.generate_csv_format(data, num_rows=3)
        
        return formats


def demonstrate_output_format():
    """Demonstrate output format specification patterns"""
    print("=" * 80)
    print("OUTPUT FORMAT SPECIFICATION PATTERN DEMONSTRATION")
    print("=" * 80)
    
    agent = OutputFormatAgent()
    
    # Example 1: JSON Extraction
    print("\n" + "=" * 80)
    print("EXAMPLE 1: JSON Format Extraction")
    print("=" * 80)
    
    text = "Apple Inc. released the iPhone 15 Pro in September 2023 priced at $999."
    schema = "Extract product name, company, release date, and price"
    print(f"\nText: {text}")
    print(f"Schema: {schema}\n")
    
    json_result = agent.extract_as_json(text, schema)
    print("JSON Output:")
    print(json.dumps(json_result, indent=2))
    
    # Example 2: Type-Safe Person Extraction
    print("\n" + "=" * 80)
    print("EXAMPLE 2: Type-Safe Person Information (Pydantic)")
    print("=" * 80)
    
    person_text = "Dr. Sarah Johnson is a 45-year-old cardiologist at Mayo Clinic. Her email is sarah.j@mayo.edu"
    print(f"\nText: {person_text}\n")
    
    person = agent.extract_person_info(person_text)
    print("Extracted Person (Type-Safe):")
    print(f"  Name: {person.name}")
    print(f"  Age: {person.age}")
    print(f"  Occupation: {person.occupation}")
    print(f"  Email: {person.email}")
    print(f"\nObject type: {type(person)}")
    
    # Example 3: Product Information
    print("\n" + "=" * 80)
    print("EXAMPLE 3: Structured Product Information")
    print("=" * 80)
    
    product_text = "The UltraBook Pro laptop in the Electronics category costs $1,299. Currently in stock with 4.5 star rating."
    print(f"\nText: {product_text}\n")
    
    product = agent.extract_product_info(product_text)
    print("Extracted Product:")
    print(f"  Name: {product.name}")
    print(f"  Category: {product.category}")
    print(f"  Price: ${product.price}")
    print(f"  In Stock: {product.in_stock}")
    print(f"  Rating: {product.rating}")
    
    # Example 4: Sentiment Analysis with Structure
    print("\n" + "=" * 80)
    print("EXAMPLE 4: Structured Sentiment Analysis")
    print("=" * 80)
    
    review = "This product is absolutely amazing! Best purchase I've made this year. Highly recommended!"
    print(f"\nReview: {review}\n")
    
    sentiment = agent.analyze_sentiment_structured(review)
    print("Sentiment Analysis Result:")
    print(f"  Sentiment: {sentiment.sentiment}")
    print(f"  Confidence: {sentiment.confidence:.2f}")
    print(f"  Keywords: {', '.join(sentiment.keywords)}")
    
    # Example 5: Markdown Report
    print("\n" + "=" * 80)
    print("EXAMPLE 5: Markdown Formatted Report")
    print("=" * 80)
    
    topic = "Quarterly Sales Performance"
    sections = ["Executive Summary", "Key Metrics", "Regional Analysis", "Recommendations"]
    print(f"\nTopic: {topic}")
    print(f"Sections: {sections}\n")
    
    markdown_report = agent.generate_markdown_report(topic, sections)
    print("Markdown Output:")
    print(markdown_report)
    
    # Example 6: CSV Format
    print("\n" + "=" * 80)
    print("EXAMPLE 6: CSV Format Generation")
    print("=" * 80)
    
    data_desc = "Employee records with name, department, salary, and hire date"
    print(f"\nData: {data_desc}\n")
    
    csv_data = agent.generate_csv_format(data_desc, num_rows=5)
    print("CSV Output:")
    print(csv_data)
    
    # Example 7: XML Format
    print("\n" + "=" * 80)
    print("EXAMPLE 7: XML Format Generation")
    print("=" * 80)
    
    xml_desc = "Book catalog with title, author, ISBN, and publication year"
    print(f"\nData: {xml_desc}\n")
    
    xml_data = agent.generate_xml_format(xml_desc)
    print("XML Output:")
    print(xml_data)
    
    # Summary
    print("\n" + "=" * 80)
    print("OUTPUT FORMAT SPECIFICATION SUMMARY")
    print("=" * 80)
    print("""
Output Format Pattern Benefits:
1. Programmatic Processing: Easy to parse and use in code
2. Type Safety: Validation ensures correct data types
3. Integration Ready: Works seamlessly with APIs and systems
4. Error Detection: Schema validation catches issues early
5. Documentation: Format serves as documentation
6. Testing: Enables automated testing

Key Formats Demonstrated:
1. JSON: Most common, flexible, widely supported
2. Pydantic Models: Type-safe Python objects with validation
3. Markdown: Human-readable formatted documents
4. CSV: Tabular data for spreadsheets
5. XML: Structured data for legacy systems

Format Specification Techniques:
1. Explicit Examples: Show exact format in prompt
2. Schema Definition: Use JSON Schema or Pydantic
3. Format Instructions: Parser-generated instructions
4. Validation: Automatic type and constraint checking
5. Error Recovery: Retry and fix malformed outputs

Best Practices:
1. Choose appropriate format for use case
   - JSON: APIs, web apps, general data
   - CSV: Tabular data, spreadsheets
   - XML: Legacy systems, complex hierarchies
   - Markdown: Documentation, reports
   
2. Use type safety when possible (Pydantic)
   - Automatic validation
   - IDE support
   - Clear error messages
   
3. Provide clear format instructions
   - Show complete examples
   - Specify all fields
   - Define data types and constraints
   
4. Handle format errors gracefully
   - Retry with corrections
   - Fallback formats
   - Clear error messages
   
5. Validate outputs
   - Schema compliance
   - Type checking
   - Constraint verification

Production Considerations:
- Lower temperature (0.1-0.3) for consistent formatting
- Implement retry logic for format violations
- Cache format instructions to save tokens
- Version schemas for evolution
- Monitor parsing success rates
- Log format errors for debugging
- Test edge cases thoroughly

Format Selection Guide:
- JSON: Default choice for most applications
- Pydantic: When type safety is critical
- CSV: For tabular data and spreadsheet integration
- XML: For legacy system compatibility
- Markdown: For human-readable documents
- YAML: For configuration files

Common Pitfalls:
- Insufficient format examples in prompts
- No validation of generated outputs
- Overly complex schemas
- Missing error handling
- Temperature too high (inconsistent outputs)
- Not handling optional fields properly
""")
    
    print("\n" + "=" * 80)
    print("Pattern 086 (Output Format Specification) demonstration complete!")
    print("=" * 80)


if __name__ == "__main__":
    demonstrate_output_format()

    

Agentic Design Pattern: Output Format

Pattern 086: Output Format – Line-by-Line Explanation

Overview

This document provides a detailed line-by-line explanation of the Output Format pattern implementation.

---

Pattern Description

Pattern 086: Output Format Specification Pattern

Description: The Output Format Specification pattern explicitly defines the structure, format, and schema of the LLM’s response. This pattern is crucial for integrating LLM outputs with downstream systems, APIs, and applications that require consistent, parseable, and structured data. By specifying the exact format upfront, this pattern eliminates ambiguity and ensures reliable programmatic processing.

Format specification is essential for:
- API integration and data exchange
- Structured data extraction
- Multi-step workflows with defined interfaces
- Database storage and retrieval
- User interface rendering
- Automated testing and validation

Components: 1. Format Type – JSON (JavaScript Object Notation) – XML (Extensible Markup Language) – YAML (YAML Ain’t Markup Language) – CSV (Comma-Separated Values) – Markdown (formatted text) – Custom formats (domain-specific)

2. Schema Definition
   - Field names and types
   - Required vs optional fields
   - Value constraints and ranges
   - Nested structures
   - Array/list specifications

3. Format Enforcement
   - Clear examples in prompts
   - Schema validation
   - Error handling for format violations
   - Retry mechanisms
   - Fallback strategies

4. Parsing and Validation
   - Automated parsing
   - Schema compliance checking
   - Type validation
   - Constraint verification
   - Error reporting

Use Cases: 1. Data Extraction – Entity extraction from text → JSON – Information retrieval → structured records – Document parsing → database entries – Web scraping → CSV data – Form filling → key-value pairs

2. API Integration
   - API request generation → JSON payloads
   - Response formatting → standardized structures
   - Webhook payloads → event schemas
   - Configuration files → YAML/JSON
   - Data exchange → XML documents

3. Report Generation
   - Business reports → Markdown tables
   - Data summaries → formatted text
   - Analytics → JSON metrics
   - Documentation → structured sections
   - Dashboards → visualization specs

4. Code Generation
   - Function signatures → specific syntax
   - Configuration files → language-specific format
   - Test cases → structured test specs
   - API documentation → OpenAPI/Swagger
   - Database schemas → SQL/DDL

5. Content Creation
   - Blog posts → Markdown with frontmatter
   - Product descriptions → structured fields
   - Email templates → HTML with placeholders
   - Social media → platform-specific formats
   - Translations → parallel text structures

LangChain Implementation: LangChain provides excellent format control through: – StructuredOutputParser for schema-based parsing – JsonOutputParser for JSON responses – PydanticOutputParser for type-safe outputs – Custom output parsers for specific formats – Output fixing parser for auto-correction

Key Features: 1. Type Safety – Pydantic models for validation – Automatic type checking – Type coercion when possible – Clear error messages

2. Schema Validation
   - JSON Schema compliance
   - Custom validation rules
   - Nested object validation
   - Array element validation

3. Error Recovery
   - Automatic retry with corrections
   - Format fixing parsers
   - Fallback to alternative formats
   - Graceful degradation

4. Multiple Format Support
   - JSON (most common)
   - XML for legacy systems
   - CSV for tabular data
   - Markdown for documents
   - Custom formats as needed

Best Practices: 1. Clear Format Specification – Provide explicit format examples – Use schema definitions when possible – Show complete structure (not partial) – Include edge cases in examples

2. Validation Strategy
   - Validate immediately after generation
   - Provide clear error messages
   - Implement retry logic
   - Log format violations

3. Error Handling
   - Graceful fallbacks
   - User-friendly error messages
   - Automatic format correction when possible
   - Manual intervention for critical failures

4. Documentation
   - Document expected formats
   - Provide format examples
   - Explain field meanings
   - Note format versions

Trade-offs: Advantages: – Reliable programmatic processing – Easy integration with systems – Reduced parsing errors – Type safety and validation – Better error detection – Automated testing possible

Disadvantages:
- More rigid (less flexibility)
- Format violations require handling
- Increases prompt complexity
- May constrain natural language quality
- Requires careful schema design
- Parsing overhead

Production Considerations: 1. Schema Evolution – Version format schemas – Backward compatibility – Migration strategies – Deprecation policies

2. Performance
   - Parsing efficiency
   - Validation overhead
   - Caching strategies
   - Batch processing

3. Reliability
   - Retry logic for format errors
   - Format fixing mechanisms
   - Fallback formats
   - Monitoring and alerting

4. Maintenance
   - Schema documentation
   - Format validation tests
   - Breaking change detection
   - Client SDK generation

5. Security
   - Input sanitization
   - Injection prevention
   - Size limits
   - Type validation

---

File Statistics

• Total Lines: 751
• Classes: 7
• Functions: 1
• Constants: 0

---

Module Documentation

"""
Pattern 086: Output Format Specification Pattern

Description:
    The Output Format Specification pattern explicitly defines the structure, format,
    and schema of the LLM's response. This pattern is crucial for integrating LLM
    outputs with downstream systems, APIs, and applications that require consistent,
    parseable, and structured data. By specifying the exact format upfront, this
    pattern eliminates ambiguity and ensures reliable programmatic processing.

    Format specification is essential for:
    - API integration and data exchange
    - Structured data extraction
    - Multi-step workflows with defined interfaces
    - Database storage and retrieval
    - User interface rendering
    - Automated testing and validation

Components:
    1. Format Type
       - JSON (JavaScript Object Notation)
       - XML (Extensible Markup Language)
       - YAML (YAML Ain't Markup Language)
       - CSV (Comma-Separated Values)
       - Markdown (formatted text)
       - Custom formats (domain-specific)

    2. Schema Definition
       - Field names and types
       - Required vs optional fields
       - Value constraints and ranges
       - Nested structures
       - Array/list specifications

    3. Format Enforcement
       - Clear examples in prompts
       - Schema validation
       - Error handling for format violations
       - Retry mechanisms
       - Fallback strategies

    4. Parsing and Validation
       - Automated parsing
       - Schema compliance checking
       - Type validation
       - Constraint verification
       - Error reporting

Use Cases:
    1. Data Extraction
       - Entity extraction from text → JSON
       - Information retrieval → structured records
       - Document parsing → database entries
       - Web scraping → CSV data
       - Form filling → key-value pairs

    2. API Integration
       - API request generation → JSON payloads
       - Response formatting → standardized structures
       - Webhook payloads → event schemas
       - Configuration files → YAML/JSON
       - Data exchange → XML documents

    3. Report Generation
       - Business reports → Markdown tables
       - Data summaries → formatted text
       - Analytics → JSON metrics
       - Documentation → structured sections
       - Dashboards → visualization specs

    4. Code Generation
       - Function signatures → specific syntax
       - Configuration files → language-specific format
       - Test cases → structured test specs
       - API documentation → OpenAPI/Swagger
       - Database schemas → SQL/DDL

    5. Content Creation
       - Blog posts → Markdown with frontmatter
       - Product descriptions → structured fields
       - Email templates → HTML with placeholders
       - Social media → platform-specific formats
       - Translations → parallel text structures

LangChain Implementation:
    LangChain provides excellent format control through:
    - StructuredOutputParser for schema-based parsing
    - JsonOutputParser for JSON responses
    - PydanticOutputParser for type-safe outputs
    - Custom output parsers for specific formats
    - Output fixing parser for auto-correction

Key Features:
    1. Type Safety
       - Pydantic models for validation
       - Automatic type checking
       - Type coercion when possible
       - Clear error messages

    2. Schema Validation
       - JSON Schema compliance
       - Custom validation rules
       - Nested object validation
       - Array element validation

    3. Error Recovery
       - Automatic retry with corrections
       - Format fixing parsers
       - Fallback to alternative formats
       - Graceful degradation

    4. Multiple Format Support
       - JSON (most common)
       - XML for legacy systems
       - CSV for tabular data
       - Markdown for documents
       - Custom formats as needed

Best Practices:
    1. Clear Format Specification
       - Provide explicit format examples
       - Use schema definitions when possible
       - Show complete structure (not partial)
       - Include edge cases in examples

    2. Validation Strategy
       - Validate immediately after generation
       - Provide clear error messages
       - Implement retry logic
       - Log format violations

    3. Error Handling
       - Graceful fallbacks
       - User-friendly error messages
       - Automatic format correction when possible
       - Manual intervention for critical failures

    4. Documentation
       - Document expected formats
       - Provide format examples
       - Explain field meanings
       - Note format versions

Trade-offs:
    Advantages:
    - Reliable programmatic processing
    - Easy integration with systems
    - Reduced parsing errors
    - Type safety and validation
    - Better error detection
    - Automated testing possible

    Disadvantages:
    - More rigid (less flexibility)
    - Format violations require handling
    - Increases prompt complexity
    - May constrain natural language quality
    - Requires careful schema design
    - Parsing overhead

Production Considerations:
    1. Schema Evolution
       - Version format schemas
       - Backward compatibility
       - Migration strategies
       - Deprecation policies

    2. Performance
       - Parsing efficiency
       - Validation overhead
       - Caching strategies
       - Batch processing

    3. Reliability
       - Retry logic for format errors
       - Format fixing mechanisms
       - Fallback formats
       - Monitoring and alerting

    4. Maintenance
       - Schema documentation
       - Format validation tests
       - Breaking change detection
       - Client SDK generation

    5. Security
       - Input sanitization
       - Injection prevention
       - Size limits
       - Type validation
"""

Purpose: This module implements the Output Format pattern, providing a structured approach to agent design and implementation.

---

Import Statements

Standard Library Imports

import dataclasses.dataclass
import enum.Enum
import json
import langchain_core.output_parsers.JsonOutputParser
import langchain_core.output_parsers.PydanticOutputParser
import langchain_core.output_parsers.StrOutputParser
import langchain_core.prompts.ChatPromptTemplate
import os
import typing.Any
import typing.Dict
import typing.List
import typing.Optional

Purpose: Core Python functionality: – dataclasses.dataclass: Simplified class definitions with @dataclass – enum.Enum: Enumeration support – json: JSON serialization/deserialization – langchain_core.output_parsers.JsonOutputParser: Regular expression operations – langchain_core.output_parsers.PydanticOutputParser: Regular expression operations – langchain_core.output_parsers.StrOutputParser: Regular expression operations – langchain_core.prompts.ChatPromptTemplate: Regular expression operations – os: Environment variable access and system operations – typing.Any: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Dict: Type hints for better code documentation (List, Dict, Any, Optional) – typing.List: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Optional: Type hints for better code documentation (List, Dict, Any, Optional)

LangChain Imports

from langchain_openai.ChatOpenAI

Purpose: LangChain framework components: – LLM Interface: Wrapper for OpenAI models – Chains: Composable building blocks for complex workflows

Other Imports

from dotenv.load_dotenv
from pydantic.BaseModel
from pydantic.Field

Purpose: Additional dependencies for specialized functionality.

---

Classes

Class: OutputFormat

Lines 212-218

Inherits from: Enum

Purpose:

Supported output formats

---

Class: Person

Lines 222-227

Inherits from: BaseModel

Purpose:

Person entity

---

Class: Product

Lines 230-236

Inherits from: BaseModel

Purpose:

Product information

---

Class: Article

Lines 239-245

Inherits from: BaseModel

Purpose:

Article/content structure

---

Class: SentimentAnalysis

Lines 248-253

Inherits from: BaseModel

Purpose:

Sentiment analysis result

---

Class: FormatConfig

Lines 257-262

Purpose:

Configuration for format specification

---

Class: OutputFormatAgent

Lines 265-555

Purpose:

Agent that enforces specific output formats.

This agent demonstrates:
1. JSON format specification
2. Pydantic-based type-safe outputs
3. Multiple format support
4. Schema validation

Methods:

__init__(self, config)

Lines 276-287

Purpose:

Initialize output format agent.

Args:
    config: Configuration for format handling

Parameters: – config: Config

extract_as_json(self, text, schema_description)

Lines 289-329

Purpose:

Extract information in JSON format.

Args:
    text: Text to extract from
    schema_description: Description of desired JSON structure

Returns:
    Extracted data as dictionary

Parameters: – text: Text – schema_description: Schema description

Returns: <ast.Subscript object at 0x000001A06D63E910>

extract_person_info(self, text)

Lines 331-359

Purpose:

Extract person information with type safety.

Args:
    text: Text containing person information

Returns:
    Person object with validated fields

Parameters: – text: Text

Returns: Person

extract_product_info(self, text)

Lines 361-389

Purpose:

Extract product information with validation.

Args:
    text: Text containing product information

Returns:
    Product object with validated fields

Parameters: – text: Text

Returns: Product

analyze_sentiment_structured(self, text)

Lines 391-421

Purpose:

Perform sentiment analysis with structured output.

Args:
    text: Text to analyze

Returns:
    SentimentAnalysis object with validated fields

Parameters: – text: Text

Returns: SentimentAnalysis

generate_markdown_report(self, topic, sections)

Lines 423-463

Purpose:

Generate report in markdown format.

Args:
    topic: Report topic
    sections: List of section titles

Returns:
    Markdown formatted report

Parameters: – topic: Topic – sections: Sections

Returns: str

generate_csv_format(self, data_description, num_rows)

Lines 465-500

Purpose:

Generate data in CSV format.

Args:
    data_description: Description of data to generate
    num_rows: Number of data rows

Returns:
    CSV formatted data

Parameters: – data_description: Data description – num_rows: Num rows

Returns: str

generate_xml_format(self, data_description)

Lines 502-528

Purpose:

Generate data in XML format.

Args:
    data_description: Description of data structure

Returns:
    XML formatted data

Parameters: – data_description: Data description

Returns: str

compare_formats(self, data)

Lines 530-555

Purpose:

Generate same data in multiple formats for comparison.

Args:
    data: Data description

Returns:
    Dictionary with data in different formats

Parameters: – data: Data

Returns: <ast.Subscript object at 0x000001A06D83C910>

---

Functions

demonstrate_output_format()

Lines 558-746

Purpose:

Demonstrate output format specification patterns

---

Key Concepts Summary

Pattern Overview

The Output Format pattern provides a structured approach to building AI agents with specific capabilities and behaviors. This implementation leverages LangChain’s composable components to create a robust and maintainable solution.

Core Components

1. Agent Class: Main implementation of the pattern logic
2. Prompt Engineering: Carefully crafted prompts to guide LLM behavior
3. Tool Integration: Optional tools for extended functionality
4. Memory Management: State handling across interactions
5. Output Parsing: Structured response processing

When to Use

✅ Good for: – Scenarios requiring this specific pattern’s strengths – Applications needing structured agent behavior – Systems where explainability is important – Production environments requiring reliability

❌ Not ideal for: – Simple single-prompt tasks – Scenarios requiring different pattern characteristics – Resource-constrained environments (if computationally intensive)

---

Best Practices Demonstrated

1. Environment Configuration

• ✅ Use .env files for API keys
• ✅ Never hardcode credentials
• ✅ Validate environment variables on startup

2. Error Handling

• ✅ Graceful degradation on failures
• ✅ Informative error messages
• ✅ Logging for debugging

3. Prompt Engineering

• ✅ Clear, specific instructions
• ✅ Structured output formats
• ✅ Few-shot examples when beneficial

4. Code Organization

• ✅ Separation of concerns (class-based design)
• ✅ Type hints for clarity
• ✅ Docstrings for documentation

5. Performance Optimization

• ✅ Appropriate temperature settings
• ✅ Token budget management
• ✅ Caching where applicable

---

LangChain Components Used

• ChatOpenAI: OpenAI model interface
• Chains: Sequential component composition

Why LangChain? – Composable abstractions for LLM applications – Rich ecosystem of integrations – Production-ready patterns – Active community and development

---

Execution Flow

1. Initialize agent with configuration
   ↓
2. Load environment and dependencies
   ↓
3. Prepare prompt with input data
   ↓
4. Invoke LLM through LangChain
   ↓
5. Parse and validate response
   ↓
6. Return formatted result

Key Decision Points: – Model selection (temperature, model type) – Prompt structure (few-shot vs zero-shot) – Error handling strategy – Output format validation

---

Performance Considerations

Computational Cost

• LLM Calls: Primary cost factor (based on tokens)
• Latency: Depends on model and input size
• Memory: Minimal for most patterns

Optimization Strategies

1. Prompt Optimization: Minimize tokens while maintaining clarity
2. Caching: Cache repeated queries/responses
3. Model Selection: Use smaller models when appropriate
4. Batch Processing: Group similar requests
5. Streaming: For long responses, stream tokens

Cost Estimates (Approximate)

• GPT-3.5-turbo: ~$0.002 per request
• GPT-4: ~$0.03 per request
• GPT-4-turbo: ~$0.01 per request

Actual costs vary based on input/output token counts

---

Common Pitfalls to Avoid

1. Prompt Design Issues

❌ Vague or ambiguous instructions
✅ Clear, specific, testable prompts

2. Error Handling

❌ Ignoring API errors or rate limits
✅ Implement retry logic and graceful degradation

3. Token Management

❌ Exceeding model token limits
✅ Truncate inputs, chunk processing

4. Security

❌ Exposing API keys in code
✅ Use environment variables

5. Testing

❌ Only testing happy paths
✅ Test edge cases, failures, malformed inputs

6. Cost Management

❌ Unmonitored API usage
✅ Implement budgets, alerts, and monitoring

---

Further Improvements

Production Enhancements

1. Monitoring: Add logging, metrics, tracing
2. Validation: Input/output schema validation
3. Rate Limiting: Implement request throttling
4. Caching: Redis or in-memory cache for responses
5. Async Support: Non-blocking operations for scalability

Feature Extensions

1. Multi-Model Support: Switch between providers
2. Fallback Models: Graceful degradation strategy
3. Custom Tools: Domain-specific function calling
4. Advanced Memory: Vector stores for long-term memory
5. Evaluation: Automated testing and benchmarking

Code Quality

1. Testing: Unit tests, integration tests
2. Documentation: API docs, usage examples
3. Type Safety: Stricter type hints
4. Configuration: Externalized config files

---

Related Patterns

• Pattern 085
• Pattern 087

Pattern Families: – Reasoning Patterns (001-011) – Multi-Agent Patterns (012-019) – Tool Use Patterns (020-025) – Memory Patterns (026-031) – And many more…

---

Summary

The Output Format pattern provides a powerful approach to building AI agents with specific capabilities. This implementation demonstrates:

✅ Key Strengths: – Clean, maintainable code structure – LangChain integration for composability – Type hints and documentation – Production-ready patterns

🎯 Best Use Cases: – Applications requiring this pattern’s characteristics – Systems needing structured agent behavior – Production environments requiring reliability

📚 Learning Resources: – LangChain Documentation: https://python.langchain.com/ – Pattern Documentation: See related patterns – OpenAI API Reference: https://platform.openai.com/docs/

---

File: 086_output_format.py
Total Lines: 751
Pattern Category: AI Agent Design Pattern #086

---

Generated from actual implementation code
For educational and reference purposes

    """
Pattern 085: Step-by-Step Instructions Pattern

Description:
    The Step-by-Step Instructions pattern structures prompts to guide the LLM through
    a process in discrete, ordered steps. This pattern is particularly effective for
    complex tasks that require systematic execution, clear sequencing, and validation
    at each stage. By breaking down tasks into explicit steps, this pattern ensures
    thorough coverage, reduces errors, and provides clear reasoning traces.

    This pattern is essential for:
    - Complex multi-stage workflows
    - Tasks requiring specific ordering
    - Processes with validation checkpoints
    - Teaching and tutoring applications
    - Troubleshooting and debugging
    - Recipe and instruction generation

Components:
    1. Step Definition
       - Numbered sequence (1, 2, 3...)
       - Clear action verbs (calculate, analyze, verify)
       - Specific objectives per step
       - Expected outcomes

    2. Step Organization
       - Main steps (high-level stages)
       - Sub-steps (detailed actions)
       - Conditional branches (if-then logic)
       - Iteration loops (repeat until)

    3. Validation Checkpoints
       - Completion criteria per step
       - Quality checks
       - Error detection
       - Progress verification

    4. Progress Tracking
       - Current step indication
       - Completed steps summary
       - Remaining steps preview
       - Overall progress percentage

Use Cases:
    1. Problem-Solving Workflows
       - Mathematical problem solving
       - Debugging code issues
       - Troubleshooting technical problems
       - Root cause analysis
       - Decision-making frameworks

    2. Instructional Content
       - Tutorial generation
       - How-to guides
       - Recipe creation
       - Assembly instructions
       - Learning modules

    3. Analysis Tasks
       - Data analysis workflows
       - Research methodology
       - Document review processes
       - Competitive analysis
       - Risk assessment

    4. Creative Processes
       - Story development (plot, characters, scenes)
       - Content creation workflow
       - Design thinking process
       - Brainstorming structure
       - Iterative refinement

    5. Compliance and Safety
       - Safety protocols
       - Audit procedures
       - Quality assurance checks
       - Regulatory compliance steps
       - Standard operating procedures

LangChain Implementation:
    LangChain enables step-by-step patterns through:
    - Structured prompt templates
    - Chain sequencing (LCEL)
    - State management between steps
    - Conditional routing
    - Progress tracking utilities

Key Features:
    1. Explicit Step Numbering
       - Clear sequential order
       - Easy progress tracking
       - Reference-able steps
       - Skippable sections

    2. Hierarchical Structure
       - Main steps and sub-steps
       - Nested levels (steps within steps)
       - Grouped related actions
       - Flexible depth

    3. Validation Integration
       - Check results at each step
       - Verify assumptions
       - Catch errors early
       - Confirm understanding

    4. Adaptive Sequencing
       - Conditional next steps
       - Skip irrelevant steps
       - Loop for iteration
       - Branch based on results

Best Practices:
    1. Step Clarity
       - Use clear, action-oriented language
       - One main action per step
       - Avoid ambiguity
       - Provide examples when helpful

    2. Appropriate Granularity
       - Not too high-level (vague)
       - Not too detailed (overwhelming)
       - Balance based on complexity
       - Adjust for target audience

    3. Logical Ordering
       - Dependencies respected
       - Natural flow
       - Build on previous steps
       - Group related steps

    4. Completeness
       - Cover all necessary steps
       - Don't skip crucial details
       - Include error handling
       - Provide alternative paths

Trade-offs:
    Advantages:
    - Reduced cognitive load (one step at a time)
    - Fewer errors (systematic approach)
    - Better traceability (clear reasoning path)
    - Easier debugging (identify problem step)
    - Improved learning (explicit process)
    - Higher consistency (standardized approach)

    Disadvantages:
    - Can be verbose (more tokens)
    - May feel rigid (less flexibility)
    - Slower for simple tasks (overhead)
    - Requires careful design (step definition)
    - May over-constrain creativity
    - Context window consumption

Production Considerations:
    1. Step Optimization
       - Balance detail with brevity
       - Remove redundant steps
       - Combine simple steps
       - Expand complex steps

    2. Error Handling
       - Define failure conditions
       - Provide recovery steps
       - Include validation checks
       - Offer troubleshooting guidance

    3. User Experience
       - Show progress indicators
       - Allow step skipping (when safe)
       - Provide step summaries
       - Enable backtracking

    4. Performance
       - Cache intermediate results
       - Parallelize independent steps
       - Optimize step execution
       - Monitor step durations

    5. Maintenance
       - Version control for procedures
       - Regular step reviews
       - Update based on feedback
       - Test step completeness
"""

import os
from typing import List, Dict, Any, Optional
from dataclasses import dataclass, field
from enum import Enum
from dotenv import load_dotenv

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_openai import ChatOpenAI

load_dotenv()


class StepStatus(Enum):
    """Status of a step"""
    PENDING = "pending"
    IN_PROGRESS = "in_progress"
    COMPLETED = "completed"
    FAILED = "failed"
    SKIPPED = "skipped"


@dataclass
class Step:
    """Represents a single step in a process"""
    number: int
    title: str
    description: str
    action: str
    validation: Optional[str] = None
    status: StepStatus = StepStatus.PENDING
    result: Optional[str] = None
    sub_steps: List['Step'] = field(default_factory=list)


@dataclass
class StepByStepConfig:
    """Configuration for step-by-step instructions"""
    include_validation: bool = True
    include_examples: bool = False
    include_reasoning: bool = True
    temperature: float = 0.3
    model_name: str = "gpt-3.5-turbo"


class StepByStepAgent:
    """
    Agent that structures tasks into clear step-by-step instructions.
    
    This agent demonstrates:
    1. Generating step-by-step processes
    2. Following structured procedures
    3. Progress tracking and validation
    4. Hierarchical step organization
    """
    
    def __init__(self, config: Optional[StepByStepConfig] = None):
        """
        Initialize step-by-step agent.
        
        Args:
            config: Configuration for step-by-step processing
        """
        self.config = config or StepByStepConfig()
        self.llm = ChatOpenAI(
            model=self.config.model_name,
            temperature=self.config.temperature
        )
    
    def generate_steps(
        self,
        task: str,
        num_steps: Optional[int] = None
    ) -> str:
        """
        Generate step-by-step instructions for a task.
        
        Args:
            task: Task to create steps for
            num_steps: Optional target number of steps
            
        Returns:
            Step-by-step instructions
        """
        num_steps_instruction = f" in approximately {num_steps} steps" if num_steps else ""
        
        prompt = ChatPromptTemplate.from_messages([
            ("system", """You are an expert at creating clear, actionable step-by-step instructions.

Your instructions should:
1. Break down the task into discrete, manageable steps
2. Use numbered steps (1, 2, 3, etc.)
3. Start each step with an action verb
4. Be specific and actionable
5. Include validation checks where appropriate
6. Maintain logical ordering"""),
            ("human", "Create step-by-step instructions for: {task}{num_steps_instruction}")
        ])
        
        chain = prompt | self.llm | StrOutputParser()
        
        result = chain.invoke({
            "task": task,
            "num_steps_instruction": num_steps_instruction
        })
        
        return result
    
    def solve_problem_step_by_step(
        self,
        problem: str
    ) -> str:
        """
        Solve a problem using step-by-step reasoning.
        
        Args:
            problem: Problem to solve
            
        Returns:
            Step-by-step solution
        """
        prompt = ChatPromptTemplate.from_messages([
            ("system", """You are a problem solver who thinks step-by-step.

For each problem:
1. Break it down into logical steps
2. Show your work for each step
3. Validate results as you go
4. State the final answer clearly

Use this format:
Step 1: [First action]
[Work/reasoning]

Step 2: [Second action]
[Work/reasoning]

...

Final Answer: [Conclusion]"""),
            ("human", "Solve this problem step-by-step: {problem}")
        ])
        
        chain = prompt | self.llm | StrOutputParser()
        result = chain.invoke({"problem": problem})
        
        return result
    
    def debug_code_step_by_step(
        self,
        code: str,
        error: str
    ) -> str:
        """
        Debug code using systematic step-by-step approach.
        
        Args:
            code: Code with error
            error: Error message
            
        Returns:
            Step-by-step debugging process
        """
        prompt = ChatPromptTemplate.from_messages([
            ("system", """You are a debugging expert who follows a systematic step-by-step process.

Debugging steps:
1. Understand the error message
2. Locate the error in the code
3. Identify the root cause
4. Propose a fix
5. Explain why the fix works
6. Provide the corrected code

Follow these steps explicitly in your response."""),
            ("human", """Debug this code step-by-step:

Code:
{code}

Error:
{error}""")
        ])
        
        chain = prompt | self.llm | StrOutputParser()
        result = chain.invoke({"code": code, "error": error})
        
        return result
    
    def create_tutorial_step_by_step(
        self,
        topic: str,
        audience: str = "beginners"
    ) -> str:
        """
        Create a tutorial with step-by-step instructions.
        
        Args:
            topic: Topic to teach
            audience: Target audience level
            
        Returns:
            Step-by-step tutorial
        """
        prompt = ChatPromptTemplate.from_messages([
            ("system", """You are an expert teacher creating step-by-step tutorials.

Your tutorials should:
1. Start with an overview
2. List prerequisites
3. Break content into numbered steps
4. Include examples in each step
5. Provide checkpoints to verify understanding
6. End with a summary and next steps

Adapt complexity to the audience level."""),
            ("human", "Create a step-by-step tutorial on '{topic}' for {audience}")
        ])
        
        chain = prompt | self.llm | StrOutputParser()
        result = chain.invoke({"topic": topic, "audience": audience})
        
        return result
    
    def analyze_data_step_by_step(
        self,
        data_description: str,
        analysis_goal: str
    ) -> str:
        """
        Perform data analysis using step-by-step methodology.
        
        Args:
            data_description: Description of the data
            analysis_goal: Goal of the analysis
            
        Returns:
            Step-by-step analysis process
        """
        prompt = ChatPromptTemplate.from_messages([
            ("system", """You are a data analyst who follows a structured step-by-step methodology.

Standard data analysis steps:
1. Understand the data and goal
2. Data cleaning and preparation
3. Exploratory analysis
4. Apply appropriate techniques
5. Interpret results
6. Draw conclusions
7. Recommend actions

Follow these steps in your analysis."""),
            ("human", """Analyze this data step-by-step:

Data: {data_description}
Goal: {analysis_goal}""")
        ])
        
        chain = prompt | self.llm | StrOutputParser()
        result = chain.invoke({
            "data_description": data_description,
            "analysis_goal": analysis_goal
        })
        
        return result
    
    def decision_framework_step_by_step(
        self,
        decision: str,
        context: str
    ) -> str:
        """
        Apply decision-making framework step-by-step.
        
        Args:
            decision: Decision to make
            context: Context and constraints
            
        Returns:
            Step-by-step decision analysis
        """
        prompt = ChatPromptTemplate.from_messages([
            ("system", """You are a decision analyst using a structured step-by-step framework.

Decision-making steps:
1. Define the decision clearly
2. Identify objectives and criteria
3. Generate alternatives
4. Evaluate each alternative against criteria
5. Assess risks and uncertainties
6. Make recommendation
7. Plan implementation

Follow these steps systematically."""),
            ("human", """Help me make this decision step-by-step:

Decision: {decision}
Context: {context}""")
        ])
        
        chain = prompt | self.llm | StrOutputParser()
        result = chain.invoke({"decision": decision, "context": context})
        
        return result
    
    def troubleshoot_step_by_step(
        self,
        problem_description: str,
        system: str = "general"
    ) -> str:
        """
        Troubleshoot a problem using systematic steps.
        
        Args:
            problem_description: Description of the problem
            system: System or domain
            
        Returns:
            Step-by-step troubleshooting guide
        """
        prompt = ChatPromptTemplate.from_messages([
            ("system", """You are a troubleshooting expert who uses a systematic step-by-step approach.

Troubleshooting methodology:
1. Gather information about the problem
2. Identify symptoms
3. Form hypotheses about causes
4. Test hypotheses systematically
5. Isolate the root cause
6. Implement solution
7. Verify resolution
8. Document for future reference

Follow these steps in your troubleshooting process."""),
            ("human", """Troubleshoot this problem step-by-step:

Problem: {problem_description}
System: {system}""")
        ])
        
        chain = prompt | self.llm | StrOutputParser()
        result = chain.invoke({
            "problem_description": problem_description,
            "system": system
        })
        
        return result


def demonstrate_step_by_step():
    """Demonstrate step-by-step instruction patterns"""
    print("=" * 80)
    print("STEP-BY-STEP INSTRUCTIONS PATTERN DEMONSTRATION")
    print("=" * 80)
    
    agent = StepByStepAgent()
    
    # Example 1: Generate Instructions for Task
    print("\n" + "=" * 80)
    print("EXAMPLE 1: Generate Step-by-Step Instructions")
    print("=" * 80)
    
    task = "Plan and execute a successful product launch"
    print(f"\nTask: {task}\n")
    instructions = agent.generate_steps(task, num_steps=7)
    print(instructions)
    
    # Example 2: Math Problem Solving
    print("\n" + "=" * 80)
    print("EXAMPLE 2: Step-by-Step Problem Solving")
    print("=" * 80)
    
    problem = "If a train travels 120 miles in 2 hours, then slows down and travels 80 miles in 2 hours, what is its average speed for the entire journey?"
    print(f"\nProblem: {problem}\n")
    solution = agent.solve_problem_step_by_step(problem)
    print(solution)
    
    # Example 3: Code Debugging
    print("\n" + "=" * 80)
    print("EXAMPLE 3: Step-by-Step Code Debugging")
    print("=" * 80)
    
    buggy_code = """
def calculate_average(numbers):
    total = 0
    for num in numbers:
        total += num
    return total / len(numbers)

result = calculate_average([])
print(result)
"""
    error = "ZeroDivisionError: division by zero"
    
    print(f"Code:\n{buggy_code}")
    print(f"\nError: {error}\n")
    debug_process = agent.debug_code_step_by_step(buggy_code, error)
    print(debug_process)
    
    # Example 4: Tutorial Creation
    print("\n" + "=" * 80)
    print("EXAMPLE 4: Step-by-Step Tutorial")
    print("=" * 80)
    
    topic = "Setting up a Python virtual environment"
    audience = "beginners"
    print(f"\nTopic: {topic}")
    print(f"Audience: {audience}\n")
    tutorial = agent.create_tutorial_step_by_step(topic, audience)
    print(tutorial)
    
    # Example 5: Data Analysis
    print("\n" + "=" * 80)
    print("EXAMPLE 5: Step-by-Step Data Analysis")
    print("=" * 80)
    
    data_desc = "Sales data for last quarter showing revenue, units sold, and customer segments"
    goal = "Identify which customer segment has the highest growth potential"
    print(f"\nData: {data_desc}")
    print(f"Goal: {goal}\n")
    analysis = agent.analyze_data_step_by_step(data_desc, goal)
    print(analysis)
    
    # Example 6: Decision Making
    print("\n" + "=" * 80)
    print("EXAMPLE 6: Step-by-Step Decision Framework")
    print("=" * 80)
    
    decision = "Should we expand to a new market?"
    context = "B2B software company with stable domestic revenue, considering Asian market expansion"
    print(f"\nDecision: {decision}")
    print(f"Context: {context}\n")
    decision_analysis = agent.decision_framework_step_by_step(decision, context)
    print(decision_analysis)
    
    # Example 7: Troubleshooting
    print("\n" + "=" * 80)
    print("EXAMPLE 7: Step-by-Step Troubleshooting")
    print("=" * 80)
    
    problem = "Website is loading very slowly for users"
    system = "web application"
    print(f"\nProblem: {problem}")
    print(f"System: {system}\n")
    troubleshooting = agent.troubleshoot_step_by_step(problem, system)
    print(troubleshooting)
    
    # Summary
    print("\n" + "=" * 80)
    print("STEP-BY-STEP INSTRUCTIONS SUMMARY")
    print("=" * 80)
    print("""
Step-by-Step Pattern Benefits:
1. Reduced Complexity: Breaks down complex tasks into manageable parts
2. Clear Progress: Easy to track where you are in the process
3. Fewer Errors: Systematic approach catches mistakes early
4. Better Learning: Explicit steps aid understanding
5. Reproducibility: Others can follow the same process
6. Debugging: Easy to identify where things went wrong

Key Applications Demonstrated:
1. Task Planning: Creating structured workflows
2. Problem Solving: Mathematical and logical problems
3. Code Debugging: Systematic error identification
4. Tutorial Creation: Teaching complex topics
5. Data Analysis: Structured analytical methodology
6. Decision Making: Rational choice frameworks
7. Troubleshooting: Systematic problem diagnosis

Step-by-Step Structure Elements:
- Numbered sequence (clear ordering)
- Action verbs (calculate, analyze, verify)
- Validation checkpoints (confirm results)
- Progress indicators (current step)
- Sub-steps (hierarchical detail)
- Conditional branches (if-then logic)

Best Practices:
1. Use clear, action-oriented language
2. Maintain logical dependencies between steps
3. Include validation checks at key points
4. Keep steps at appropriate granularity
5. Number steps clearly (1, 2, 3...)
6. Start steps with action verbs
7. Provide examples within steps when helpful
8. Include error handling and alternatives

When to Use Step-by-Step:
- Complex multi-stage tasks
- Teaching and learning scenarios
- Troubleshooting and debugging
- Compliance and safety procedures
- Quality assurance processes
- Standardized workflows
- Problem-solving that requires systematic approach

Advantages Over Free-Form:
- More thorough coverage
- Easier to follow and verify
- Better for complex tasks
- Reduces cognitive load
- Improves consistency
- Facilitates collaboration

Trade-offs:
- More verbose (uses more tokens)
- Can feel rigid for simple tasks
- Requires careful step design
- May over-structure creative tasks
- Slower for trivial operations

Production Tips:
- Cache common step sequences
- Allow step skipping when safe
- Provide progress indicators
- Enable backtracking to previous steps
- Version control procedural steps
- Gather feedback on step clarity
- Optimize step granularity based on usage
""")
    
    print("\n" + "=" * 80)
    print("Pattern 085 (Step-by-Step Instructions) demonstration complete!")
    print("=" * 80)


if __name__ == "__main__":
    demonstrate_step_by_step()

    

Agentic Design Pattern: Step By Step

Pattern 085: Step By Step – Line-by-Line Explanation

Overview

This document provides a detailed line-by-line explanation of the Step By Step pattern implementation.

---

Pattern Description

Pattern 085: Step-by-Step Instructions Pattern

Description: The Step-by-Step Instructions pattern structures prompts to guide the LLM through a process in discrete, ordered steps. This pattern is particularly effective for complex tasks that require systematic execution, clear sequencing, and validation at each stage. By breaking down tasks into explicit steps, this pattern ensures thorough coverage, reduces errors, and provides clear reasoning traces.

This pattern is essential for:
- Complex multi-stage workflows
- Tasks requiring specific ordering
- Processes with validation checkpoints
- Teaching and tutoring applications
- Troubleshooting and debugging
- Recipe and instruction generation

Components: 1. Step Definition – Numbered sequence (1, 2, 3…) – Clear action verbs (calculate, analyze, verify) – Specific objectives per step – Expected outcomes

2. Step Organization
   - Main steps (high-level stages)
   - Sub-steps (detailed actions)
   - Conditional branches (if-then logic)
   - Iteration loops (repeat until)

3. Validation Checkpoints
   - Completion criteria per step
   - Quality checks
   - Error detection
   - Progress verification

4. Progress Tracking
   - Current step indication
   - Completed steps summary
   - Remaining steps preview
   - Overall progress percentage

Use Cases: 1. Problem-Solving Workflows – Mathematical problem solving – Debugging code issues – Troubleshooting technical problems – Root cause analysis – Decision-making frameworks

2. Instructional Content
   - Tutorial generation
   - How-to guides
   - Recipe creation
   - Assembly instructions
   - Learning modules

3. Analysis Tasks
   - Data analysis workflows
   - Research methodology
   - Document review processes
   - Competitive analysis
   - Risk assessment

4. Creative Processes
   - Story development (plot, characters, scenes)
   - Content creation workflow
   - Design thinking process
   - Brainstorming structure
   - Iterative refinement

5. Compliance and Safety
   - Safety protocols
   - Audit procedures
   - Quality assurance checks
   - Regulatory compliance steps
   - Standard operating procedures

LangChain Implementation: LangChain enables step-by-step patterns through: – Structured prompt templates – Chain sequencing (LCEL) – State management between steps – Conditional routing – Progress tracking utilities

Key Features: 1. Explicit Step Numbering – Clear sequential order – Easy progress tracking – Reference-able steps – Skippable sections

2. Hierarchical Structure
   - Main steps and sub-steps
   - Nested levels (steps within steps)
   - Grouped related actions
   - Flexible depth

3. Validation Integration
   - Check results at each step
   - Verify assumptions
   - Catch errors early
   - Confirm understanding

4. Adaptive Sequencing
   - Conditional next steps
   - Skip irrelevant steps
   - Loop for iteration
   - Branch based on results

Best Practices: 1. Step Clarity – Use clear, action-oriented language – One main action per step – Avoid ambiguity – Provide examples when helpful

2. Appropriate Granularity
   - Not too high-level (vague)
   - Not too detailed (overwhelming)
   - Balance based on complexity
   - Adjust for target audience

3. Logical Ordering
   - Dependencies respected
   - Natural flow
   - Build on previous steps
   - Group related steps

4. Completeness
   - Cover all necessary steps
   - Don't skip crucial details
   - Include error handling
   - Provide alternative paths

Trade-offs: Advantages: – Reduced cognitive load (one step at a time) – Fewer errors (systematic approach) – Better traceability (clear reasoning path) – Easier debugging (identify problem step) – Improved learning (explicit process) – Higher consistency (standardized approach)

Disadvantages:
- Can be verbose (more tokens)
- May feel rigid (less flexibility)
- Slower for simple tasks (overhead)
- Requires careful design (step definition)
- May over-constrain creativity
- Context window consumption

Production Considerations: 1. Step Optimization – Balance detail with brevity – Remove redundant steps – Combine simple steps – Expand complex steps

2. Error Handling
   - Define failure conditions
   - Provide recovery steps
   - Include validation checks
   - Offer troubleshooting guidance

3. User Experience
   - Show progress indicators
   - Allow step skipping (when safe)
   - Provide step summaries
   - Enable backtracking

4. Performance
   - Cache intermediate results
   - Parallelize independent steps
   - Optimize step execution
   - Monitor step durations

5. Maintenance
   - Version control for procedures
   - Regular step reviews
   - Update based on feedback
   - Test step completeness

---

File Statistics

• Total Lines: 715
• Classes: 4
• Functions: 1
• Constants: 0

---

Module Documentation

"""
Pattern 085: Step-by-Step Instructions Pattern

Description:
    The Step-by-Step Instructions pattern structures prompts to guide the LLM through
    a process in discrete, ordered steps. This pattern is particularly effective for
    complex tasks that require systematic execution, clear sequencing, and validation
    at each stage. By breaking down tasks into explicit steps, this pattern ensures
    thorough coverage, reduces errors, and provides clear reasoning traces.

    This pattern is essential for:
    - Complex multi-stage workflows
    - Tasks requiring specific ordering
    - Processes with validation checkpoints
    - Teaching and tutoring applications
    - Troubleshooting and debugging
    - Recipe and instruction generation

Components:
    1. Step Definition
       - Numbered sequence (1, 2, 3...)
       - Clear action verbs (calculate, analyze, verify)
       - Specific objectives per step
       - Expected outcomes

    2. Step Organization
       - Main steps (high-level stages)
       - Sub-steps (detailed actions)
       - Conditional branches (if-then logic)
       - Iteration loops (repeat until)

    3. Validation Checkpoints
       - Completion criteria per step
       - Quality checks
       - Error detection
       - Progress verification

    4. Progress Tracking
       - Current step indication
       - Completed steps summary
       - Remaining steps preview
       - Overall progress percentage

Use Cases:
    1. Problem-Solving Workflows
       - Mathematical problem solving
       - Debugging code issues
       - Troubleshooting technical problems
       - Root cause analysis
       - Decision-making frameworks

    2. Instructional Content
       - Tutorial generation
       - How-to guides
       - Recipe creation
       - Assembly instructions
       - Learning modules

    3. Analysis Tasks
       - Data analysis workflows
       - Research methodology
       - Document review processes
       - Competitive analysis
       - Risk assessment

    4. Creative Processes
       - Story development (plot, characters, scenes)
       - Content creation workflow
       - Design thinking process
       - Brainstorming structure
       - Iterative refinement

    5. Compliance and Safety
       - Safety protocols
       - Audit procedures
       - Quality assurance checks
       - Regulatory compliance steps
       - Standard operating procedures

LangChain Implementation:
    LangChain enables step-by-step patterns through:
    - Structured prompt templates
    - Chain sequencing (LCEL)
    - State management between steps
    - Conditional routing
    - Progress tracking utilities

Key Features:
    1. Explicit Step Numbering
       - Clear sequential order
       - Easy progress tracking
       - Reference-able steps
       - Skippable sections

    2. Hierarchical Structure
       - Main steps and sub-steps
       - Nested levels (steps within steps)
       - Grouped related actions
       - Flexible depth

    3. Validation Integration
       - Check results at each step
       - Verify assumptions
       - Catch errors early
       - Confirm understanding

    4. Adaptive Sequencing
       - Conditional next steps
       - Skip irrelevant steps
       - Loop for iteration
       - Branch based on results

Best Practices:
    1. Step Clarity
       - Use clear, action-oriented language
       - One main action per step
       - Avoid ambiguity
       - Provide examples when helpful

    2. Appropriate Granularity
       - Not too high-level (vague)
       - Not too detailed (overwhelming)
       - Balance based on complexity
       - Adjust for target audience

    3. Logical Ordering
       - Dependencies respected
       - Natural flow
       - Build on previous steps
       - Group related steps

    4. Completeness
       - Cover all necessary steps
       - Don't skip crucial details
       - Include error handling
       - Provide alternative paths

Trade-offs:
    Advantages:
    - Reduced cognitive load (one step at a time)
    - Fewer errors (systematic approach)
    - Better traceability (clear reasoning path)
    - Easier debugging (identify problem step)
    - Improved learning (explicit process)
    - Higher consistency (standardized approach)

    Disadvantages:
    - Can be verbose (more tokens)
    - May feel rigid (less flexibility)
    - Slower for simple tasks (overhead)
    - Requires careful design (step definition)
    - May over-constrain creativity
    - Context window consumption

Production Considerations:
    1. Step Optimization
       - Balance detail with brevity
       - Remove redundant steps
       - Combine simple steps
       - Expand complex steps

    2. Error Handling
       - Define failure conditions
       - Provide recovery steps
       - Include validation checks
       - Offer troubleshooting guidance

    3. User Experience
       - Show progress indicators
       - Allow step skipping (when safe)
       - Provide step summaries
       - Enable backtracking

    4. Performance
       - Cache intermediate results
       - Parallelize independent steps
       - Optimize step execution
       - Monitor step durations

    5. Maintenance
       - Version control for procedures
       - Regular step reviews
       - Update based on feedback
       - Test step completeness
"""

Purpose: This module implements the Step By Step pattern, providing a structured approach to agent design and implementation.

---

Import Statements

Standard Library Imports

import dataclasses.dataclass
import dataclasses.field
import enum.Enum
import langchain_core.output_parsers.StrOutputParser
import langchain_core.prompts.ChatPromptTemplate
import os
import typing.Any
import typing.Dict
import typing.List
import typing.Optional

Purpose: Core Python functionality: – dataclasses.dataclass: Simplified class definitions with @dataclass – dataclasses.field: Simplified class definitions with @dataclass – enum.Enum: Enumeration support – langchain_core.output_parsers.StrOutputParser: Regular expression operations – langchain_core.prompts.ChatPromptTemplate: Regular expression operations – os: Environment variable access and system operations – typing.Any: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Dict: Type hints for better code documentation (List, Dict, Any, Optional) – typing.List: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Optional: Type hints for better code documentation (List, Dict, Any, Optional)

LangChain Imports

from langchain_openai.ChatOpenAI

Purpose: LangChain framework components: – LLM Interface: Wrapper for OpenAI models – Chains: Composable building blocks for complex workflows

Other Imports

from dotenv.load_dotenv

Purpose: Additional dependencies for specialized functionality.

---

Classes

Class: StepStatus

Lines 200-206

Inherits from: Enum

Purpose:

Status of a step

---

Class: Step

Lines 210-219

Purpose:

Represents a single step in a process

---

Class: StepByStepConfig

Lines 223-229

Purpose:

Configuration for step-by-step instructions

---

Class: StepByStepAgent

Lines 232-533

Purpose:

Agent that structures tasks into clear step-by-step instructions.

This agent demonstrates:
1. Generating step-by-step processes
2. Following structured procedures
3. Progress tracking and validation
4. Hierarchical step organization

Methods:

__init__(self, config)

Lines 243-254

Purpose:

Initialize step-by-step agent.

Args:
    config: Configuration for step-by-step processing

Parameters: – config: Config

generate_steps(self, task, num_steps)

Lines 256-293

Purpose:

Generate step-by-step instructions for a task.

Args:
    task: Task to create steps for
    num_steps: Optional target number of steps

Returns:
    Step-by-step instructions

Parameters: – task: Task – num_steps: Num steps

Returns: str

solve_problem_step_by_step(self, problem)

Lines 295-333

Purpose:

Solve a problem using step-by-step reasoning.

Args:
    problem: Problem to solve

Returns:
    Step-by-step solution

Parameters: – problem: Problem

Returns: str

debug_code_step_by_step(self, code, error)

Lines 335-374

Purpose:

Debug code using systematic step-by-step approach.

Args:
    code: Code with error
    error: Error message

Returns:
    Step-by-step debugging process

Parameters: – code: Code – error: Error

Returns: str

create_tutorial_step_by_step(self, topic, audience)

Lines 376-409

Purpose:

Create a tutorial with step-by-step instructions.

Args:
    topic: Topic to teach
    audience: Target audience level

Returns:
    Step-by-step tutorial

Parameters: – topic: Topic – audience: Audience

Returns: str

analyze_data_step_by_step(self, data_description, analysis_goal)

Lines 411-451

Purpose:

Perform data analysis using step-by-step methodology.

Args:
    data_description: Description of the data
    analysis_goal: Goal of the analysis

Returns:
    Step-by-step analysis process

Parameters: – data_description: Data description – analysis_goal: Analysis goal

Returns: str

decision_framework_step_by_step(self, decision, context)

Lines 453-490

Purpose:

Apply decision-making framework step-by-step.

Args:
    decision: Decision to make
    context: Context and constraints

Returns:
    Step-by-step decision analysis

Parameters: – decision: Decision – context: Context

Returns: str

troubleshoot_step_by_step(self, problem_description, system)

Lines 492-533

Purpose:

Troubleshoot a problem using systematic steps.

Args:
    problem_description: Description of the problem
    system: System or domain

Returns:
    Step-by-step troubleshooting guide

Parameters: – problem_description: Problem description – system: System

Returns: str

---

Functions

demonstrate_step_by_step()

Lines 536-710

Purpose:

Demonstrate step-by-step instruction patterns

---

Key Concepts Summary

Pattern Overview

The Step By Step pattern provides a structured approach to building AI agents with specific capabilities and behaviors. This implementation leverages LangChain’s composable components to create a robust and maintainable solution.

Core Components

1. Agent Class: Main implementation of the pattern logic
2. Prompt Engineering: Carefully crafted prompts to guide LLM behavior
3. Tool Integration: Optional tools for extended functionality
4. Memory Management: State handling across interactions
5. Output Parsing: Structured response processing

When to Use

✅ Good for: – Scenarios requiring this specific pattern’s strengths – Applications needing structured agent behavior – Systems where explainability is important – Production environments requiring reliability

❌ Not ideal for: – Simple single-prompt tasks – Scenarios requiring different pattern characteristics – Resource-constrained environments (if computationally intensive)

---

Best Practices Demonstrated

1. Environment Configuration

• ✅ Use .env files for API keys
• ✅ Never hardcode credentials
• ✅ Validate environment variables on startup

2. Error Handling

• ✅ Graceful degradation on failures
• ✅ Informative error messages
• ✅ Logging for debugging

3. Prompt Engineering

• ✅ Clear, specific instructions
• ✅ Structured output formats
• ✅ Few-shot examples when beneficial

4. Code Organization

• ✅ Separation of concerns (class-based design)
• ✅ Type hints for clarity
• ✅ Docstrings for documentation

5. Performance Optimization

• ✅ Appropriate temperature settings
• ✅ Token budget management
• ✅ Caching where applicable

---

LangChain Components Used

• ChatOpenAI: OpenAI model interface
• Chains: Sequential component composition

Why LangChain? – Composable abstractions for LLM applications – Rich ecosystem of integrations – Production-ready patterns – Active community and development

---

Execution Flow

1. Initialize agent with configuration
   ↓
2. Load environment and dependencies
   ↓
3. Prepare prompt with input data
   ↓
4. Invoke LLM through LangChain
   ↓
5. Parse and validate response
   ↓
6. Return formatted result

Key Decision Points: – Model selection (temperature, model type) – Prompt structure (few-shot vs zero-shot) – Error handling strategy – Output format validation

---

Performance Considerations

Computational Cost

• LLM Calls: Primary cost factor (based on tokens)
• Latency: Depends on model and input size
• Memory: Minimal for most patterns

Optimization Strategies

1. Prompt Optimization: Minimize tokens while maintaining clarity
2. Caching: Cache repeated queries/responses
3. Model Selection: Use smaller models when appropriate
4. Batch Processing: Group similar requests
5. Streaming: For long responses, stream tokens

Cost Estimates (Approximate)

• GPT-3.5-turbo: ~$0.002 per request
• GPT-4: ~$0.03 per request
• GPT-4-turbo: ~$0.01 per request

Actual costs vary based on input/output token counts

---

Common Pitfalls to Avoid

1. Prompt Design Issues

❌ Vague or ambiguous instructions
✅ Clear, specific, testable prompts

2. Error Handling

❌ Ignoring API errors or rate limits
✅ Implement retry logic and graceful degradation

3. Token Management

❌ Exceeding model token limits
✅ Truncate inputs, chunk processing

4. Security

❌ Exposing API keys in code
✅ Use environment variables

5. Testing

❌ Only testing happy paths
✅ Test edge cases, failures, malformed inputs

6. Cost Management

❌ Unmonitored API usage
✅ Implement budgets, alerts, and monitoring

---

Further Improvements

Production Enhancements

1. Monitoring: Add logging, metrics, tracing
2. Validation: Input/output schema validation
3. Rate Limiting: Implement request throttling
4. Caching: Redis or in-memory cache for responses
5. Async Support: Non-blocking operations for scalability

Feature Extensions

1. Multi-Model Support: Switch between providers
2. Fallback Models: Graceful degradation strategy
3. Custom Tools: Domain-specific function calling
4. Advanced Memory: Vector stores for long-term memory
5. Evaluation: Automated testing and benchmarking

Code Quality

1. Testing: Unit tests, integration tests
2. Documentation: API docs, usage examples
3. Type Safety: Stricter type hints
4. Configuration: Externalized config files

---

Related Patterns

• Pattern 084
• Pattern 086

Pattern Families: – Reasoning Patterns (001-011) – Multi-Agent Patterns (012-019) – Tool Use Patterns (020-025) – Memory Patterns (026-031) – And many more…

---

Summary

The Step By Step pattern provides a powerful approach to building AI agents with specific capabilities. This implementation demonstrates:

✅ Key Strengths: – Clean, maintainable code structure – LangChain integration for composability – Type hints and documentation – Production-ready patterns

🎯 Best Use Cases: – Applications requiring this pattern’s characteristics – Systems needing structured agent behavior – Production environments requiring reliability

📚 Learning Resources: – LangChain Documentation: https://python.langchain.com/ – Pattern Documentation: See related patterns – OpenAI API Reference: https://platform.openai.com/docs/

---

File: 085_step_by_step.py
Total Lines: 715
Pattern Category: AI Agent Design Pattern #085

---

Generated from actual implementation code
For educational and reference purposes

    """
Pattern 084: Role-Playing/Persona Pattern

Description:
    The Role-Playing/Persona pattern involves instructing the LLM to adopt a specific
    role, character, or persona when generating responses. This pattern leverages the
    model's ability to simulate different perspectives, expertise levels, personalities,
    and communication styles. By assuming a role, the LLM can provide more contextually
    appropriate, specialized, and engaging responses.

    Role-playing is a powerful prompt engineering technique that:
    - Enhances response quality through perspective-taking
    - Enables domain expertise simulation (doctor, lawyer, scientist)
    - Creates consistent personality in conversational agents
    - Improves user engagement through character-based interaction
    - Allows multi-perspective analysis of problems

Components:
    1. Persona Definition
       - Identity (who the persona is)
       - Expertise (what they know)
       - Personality traits (how they communicate)
       - Background (relevant experience)
       - Goals and motivations

    2. Communication Style
       - Tone (formal, casual, friendly, professional)
       - Vocabulary level (simple, technical, academic)
       - Response structure (detailed, concise, step-by-step)
       - Interaction patterns (questioning, advising, teaching)

    3. Persona Consistency
       - Maintaining character across turns
       - Memory of previous interactions
       - Consistent viewpoint and opinions
       - Appropriate domain knowledge boundaries

    4. Context Adaptation
       - Adjusting expertise level to user
       - Situational awareness
       - Emotional intelligence
       - Cultural sensitivity

Use Cases:
    1. Expert Consultation
       - Medical advice (doctor persona)
       - Legal guidance (lawyer persona)
       - Technical support (engineer persona)
       - Financial planning (financial advisor persona)
       - Career counseling (career coach persona)

    2. Educational Applications
       - Patient teacher (explaining complex topics)
       - Socratic tutor (guiding through questions)
       - Encouraging mentor (motivating learners)
       - Strict examiner (testing knowledge)

    3. Creative Writing
       - Character dialogue generation
       - Story narration from specific viewpoint
       - Historical figure simulation
       - Fictional character interaction

    4. Business Applications
       - Sales representative (persuasive, customer-focused)
       - Customer service (empathetic, problem-solving)
       - Business analyst (analytical, data-driven)
       - Executive advisor (strategic, big-picture)

    5. Therapeutic and Counseling
       - Supportive friend (empathetic listener)
       - Life coach (motivational, goal-oriented)
       - Therapist (reflective, non-judgmental)
       - Meditation guide (calm, mindful)

LangChain Implementation:
    LangChain facilitates role-playing through:
    - System message configuration (setting persona)
    - Context management (maintaining persona)
    - Memory integration (persona consistency)
    - Custom prompt templates (structured personas)
    - Chat history management (conversation continuity)

Key Features:
    1. Persona Library
       - Pre-defined expert personas
       - Customizable persona templates
       - Domain-specific knowledge bases
       - Personality trait combinations

    2. Dynamic Persona Switching
       - Context-aware persona selection
       - Smooth transitions between personas
       - Multi-persona conversations
       - Persona blending

    3. Expertise Calibration
       - Adjusting knowledge depth
       - Acknowledging limitations
       - Appropriate confidence levels
       - Domain boundary awareness

    4. Personality Traits
       - Empathy and emotional intelligence
       - Humor and engagement style
       - Assertiveness levels
       - Teaching vs directing approaches

Best Practices:
    1. Persona Definition
       - Be specific about role and expertise
       - Include relevant background details
       - Define communication style clearly
       - Set appropriate boundaries

    2. Consistency Maintenance
       - Use memory for multi-turn conversations
       - Reference persona context regularly
       - Maintain viewpoint across interactions
       - Avoid out-of-character responses

    3. Expertise Management
       - Stay within persona's knowledge domain
       - Acknowledge when unsure
       - Provide persona-appropriate disclaimers
       - Don't overreach expertise claims

    4. Engagement Optimization
       - Match tone to user expectations
       - Balance expertise with accessibility
       - Use persona-appropriate examples
       - Maintain engaging interaction style

Trade-offs:
    Advantages:
    - Enhanced response quality and relevance
    - More engaging and natural interactions
    - Domain-specific expertise simulation
    - Consistent personality and tone
    - Better user trust and satisfaction
    - Multi-perspective problem analysis

    Disadvantages:
    - Requires detailed persona definitions
    - May limit flexibility in responses
    - Risk of inappropriate persona adoption
    - Can be perceived as deceptive if not transparent
    - Harder to maintain over long conversations
    - May introduce unwanted biases

Production Considerations:
    1. Transparency
       - Clearly indicate AI persona usage
       - Provide disclaimers for professional advice
       - Explain limitations of simulated expertise
       - Allow users to request different personas

    2. Safety and Ethics
       - Avoid harmful or inappropriate personas
       - Include safety guardrails
       - Monitor for persona misuse
       - Implement content filtering
       - Respect professional boundaries

    3. Quality Assurance
       - Test personas across diverse inputs
       - Validate expertise accuracy
       - Monitor consistency metrics
       - Gather user feedback
       - Regular persona updates

    4. Scalability
       - Modular persona definitions
       - Reusable persona components
       - Version control for personas
       - A/B test persona variations
       - Performance monitoring per persona

    5. Legal Compliance
       - Medical/legal advice disclaimers
       - Professional credential clarification
       - Terms of service alignment
       - Privacy considerations
       - Liability protection
"""

import os
from typing import List, Dict, Any, Optional
from dataclasses import dataclass
from enum import Enum
from dotenv import load_dotenv

from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_core.output_parsers import StrOutputParser
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage, AIMessage, SystemMessage

load_dotenv()


class PersonaType(Enum):
    """Types of personas"""
    EXPERT = "expert"
    TEACHER = "teacher"
    CRITIC = "critic"
    FRIEND = "friend"
    COACH = "coach"
    ANALYST = "analyst"
    CREATIVE = "creative"
    COUNSELOR = "counselor"


@dataclass
class Persona:
    """Defines a role-playing persona"""
    name: str
    role: str
    expertise: List[str]
    personality_traits: List[str]
    communication_style: str
    background: str
    system_message: str


@dataclass
class PersonaConfig:
    """Configuration for persona-based interaction"""
    temperature: float = 0.7
    model_name: str = "gpt-3.5-turbo"
    max_tokens: Optional[int] = None
    maintain_history: bool = True


class RolePlayingAgent:
    """
    Agent that adopts different personas for specialized interactions.
    
    This agent demonstrates:
    1. Multiple pre-defined personas
    2. Persona switching
    3. Consistent character maintenance
    4. Domain-specific expertise simulation
    """
    
    def __init__(self, config: Optional[PersonaConfig] = None):
        """
        Initialize role-playing agent.
        
        Args:
            config: Configuration for persona interactions
        """
        self.config = config or PersonaConfig()
        self.llm = ChatOpenAI(
            model=self.config.model_name,
            temperature=self.config.temperature
        )
        self.personas = self._initialize_personas()
        self.conversation_history: List[Any] = []
    
    def _initialize_personas(self) -> Dict[str, Persona]:
        """Initialize library of personas"""
        return {
            "doctor": Persona(
                name="Dr. Sarah Johnson",
                role="Medical Doctor",
                expertise=["General Medicine", "Preventive Care", "Health Education"],
                personality_traits=["Empathetic", "Patient", "Thorough", "Professional"],
                communication_style="Clear, reassuring, and professional. Uses simple language to explain medical concepts.",
                background="15 years of experience in family medicine. Focus on patient education and preventive care.",
                system_message="""You are Dr. Sarah Johnson, a compassionate and experienced medical doctor with 15 years in family medicine.

Your approach:
- Speak in a warm, professional, and reassuring tone
- Explain medical concepts in simple, accessible language
- Ask clarifying questions about symptoms
- Provide general health guidance (not specific diagnoses)
- Always recommend consulting a healthcare provider for serious concerns
- Show empathy and understanding for patient concerns

Important: You provide general health information only. Always remind users to consult their healthcare provider for personalized medical advice."""
            ),
            
            "teacher": Persona(
                name="Professor Alex Chen",
                role="Patient Teacher",
                expertise=["Education", "Learning Science", "Curriculum Design"],
                personality_traits=["Patient", "Encouraging", "Thorough", "Enthusiastic"],
                communication_style="Clear, step-by-step explanations with examples and encouragement.",
                background="20 years teaching experience. Specializes in breaking down complex topics.",
                system_message="""You are Professor Alex Chen, a patient and enthusiastic teacher with 20 years of experience.

Your teaching approach:
- Break complex topics into manageable steps
- Use clear examples and analogies
- Check for understanding regularly
- Provide encouragement and positive reinforcement
- Adapt explanations to student's level
- Use the Socratic method to guide learning
- Make learning engaging and fun

Remember: Every student learns differently. Be patient, encouraging, and adaptive to their needs."""
            ),
            
            "critic": Persona(
                name="Marcus Stone",
                role="Constructive Critic",
                expertise=["Analysis", "Critical Thinking", "Quality Assurance"],
                personality_traits=["Analytical", "Direct", "Thorough", "Fair"],
                communication_style="Direct, analytical, focuses on both strengths and weaknesses.",
                background="Expert reviewer with focus on constructive feedback and improvement.",
                system_message="""You are Marcus Stone, a fair but thorough critic who provides constructive feedback.

Your approach:
- Analyze thoroughly and objectively
- Point out both strengths and weaknesses
- Provide specific, actionable feedback
- Explain the reasoning behind critiques
- Suggest concrete improvements
- Maintain professionalism and respect
- Balance criticism with recognition of merits

Your goal is to help improve quality through honest, constructive analysis."""
            ),
            
            "coach": Persona(
                name="Coach Sam Rivera",
                role="Life Coach",
                expertise=["Goal Setting", "Motivation", "Personal Development"],
                personality_traits=["Motivational", "Supportive", "Direct", "Energetic"],
                communication_style="Energetic, motivational, action-oriented with practical strategies.",
                background="Certified life coach helping people achieve their goals for 10 years.",
                system_message="""You are Coach Sam Rivera, an energetic and motivational life coach.

Your coaching style:
- Help clients clarify their goals
- Break big goals into actionable steps
- Provide motivation and encouragement
- Hold clients accountable
- Celebrate progress and wins
- Address obstacles and limiting beliefs
- Use powerful questions to unlock insights
- Focus on action and results

You believe everyone has untapped potential. Your job is to help them discover and achieve it!"""
            ),
            
            "analyst": Persona(
                name="Dr. Emily Watson",
                role="Business Analyst",
                expertise=["Data Analysis", "Strategic Planning", "Market Research"],
                personality_traits=["Analytical", "Objective", "Detail-oriented", "Strategic"],
                communication_style="Data-driven, structured, focuses on insights and recommendations.",
                background="15 years in business intelligence and strategic consulting.",
                system_message="""You are Dr. Emily Watson, a senior business analyst with expertise in data-driven decision making.

Your analytical approach:
- Examine problems systematically
- Look for patterns and trends
- Use data to support conclusions
- Consider multiple perspectives
- Identify risks and opportunities
- Provide structured recommendations
- Think strategically about implications
- Present insights clearly and concisely

You help organizations make informed decisions through rigorous analysis."""
            ),
            
            "creative": Persona(
                name="Luna Martinez",
                role="Creative Director",
                expertise=["Creative Writing", "Brainstorming", "Innovation"],
                personality_traits=["Imaginative", "Enthusiastic", "Bold", "Inspiring"],
                communication_style="Vibrant, imaginative, encourages thinking outside the box.",
                background="Award-winning creative director with passion for innovation.",
                system_message="""You are Luna Martinez, an imaginative and award-winning creative director.

Your creative approach:
- Think unconventionally and boldly
- Generate multiple creative ideas
- Encourage wild brainstorming
- Look for unexpected connections
- Challenge conventional thinking
- Inspire with enthusiasm
- Balance creativity with practicality
- Help refine and develop ideas

"Creativity is intelligence having fun" - and you bring both! Help others unlock their creative potential."""
            ),
        }
    
    def interact_with_persona(
        self,
        persona_key: str,
        user_message: str,
        reset_history: bool = False
    ) -> str:
        """
        Interact with a specific persona.
        
        Args:
            persona_key: Key of persona to use
            user_message: User's message
            reset_history: Whether to reset conversation history
            
        Returns:
            Persona's response
        """
        if persona_key not in self.personas:
            return f"Persona '{persona_key}' not found. Available: {list(self.personas.keys())}"
        
        persona = self.personas[persona_key]
        
        # Reset history if requested
        if reset_history or not self.config.maintain_history:
            self.conversation_history = []
        
        # Create prompt with persona
        if self.conversation_history:
            messages = [
                SystemMessage(content=persona.system_message),
                *self.conversation_history,
                HumanMessage(content=user_message)
            ]
        else:
            messages = [
                SystemMessage(content=persona.system_message),
                HumanMessage(content=user_message)
            ]
        
        # Get response
        response = self.llm.invoke(messages)
        
        # Update history
        if self.config.maintain_history:
            self.conversation_history.append(HumanMessage(content=user_message))
            self.conversation_history.append(AIMessage(content=response.content))
        
        return response.content
    
    def multi_persona_analysis(
        self,
        question: str,
        persona_keys: List[str]
    ) -> Dict[str, str]:
        """
        Get perspectives from multiple personas.
        
        Args:
            question: Question to analyze
            persona_keys: List of persona keys to consult
            
        Returns:
            Dictionary of persona responses
        """
        results = {}
        
        for key in persona_keys:
            if key in self.personas:
                # Reset history for clean analysis
                response = self.interact_with_persona(key, question, reset_history=True)
                results[key] = response
        
        return results
    
    def persona_debate(
        self,
        topic: str,
        persona1_key: str,
        persona2_key: str,
        rounds: int = 2
    ) -> List[Dict[str, str]]:
        """
        Have two personas debate a topic.
        
        Args:
            topic: Topic to debate
            persona1_key: First persona key
            persona2_key: Second persona key
            rounds: Number of debate rounds
            
        Returns:
            List of debate exchanges
        """
        if persona1_key not in self.personas or persona2_key not in self.personas:
            return [{"error": "One or both personas not found"}]
        
        persona1 = self.personas[persona1_key]
        persona2 = self.personas[persona2_key]
        
        debate = []
        
        # Initial statements
        prompt1 = f"Present your perspective on: {topic}"
        response1 = self.interact_with_persona(persona1_key, prompt1, reset_history=True)
        debate.append({
            "round": 1,
            "persona": persona1.name,
            "statement": response1
        })
        
        # Alternating responses
        for round_num in range(2, rounds + 2):
            # Persona 2 responds to Persona 1
            prompt2 = f"Respond to this perspective on '{topic}': {response1}"
            response2 = self.interact_with_persona(persona2_key, prompt2, reset_history=True)
            debate.append({
                "round": round_num,
                "persona": persona2.name,
                "statement": response2
            })
            
            if round_num  Optional[Persona]:
        """Get information about a persona"""
        return self.personas.get(persona_key)
    
    def list_personas(self) -> List[str]:
        """List available personas"""
        return list(self.personas.keys())


def demonstrate_role_playing():
    """Demonstrate role-playing/persona patterns"""
    print("=" * 80)
    print("ROLE-PLAYING/PERSONA PATTERN DEMONSTRATION")
    print("=" * 80)
    
    agent = RolePlayingAgent()
    
    # Example 1: Doctor Persona
    print("\n" + "=" * 80)
    print("EXAMPLE 1: Doctor Persona - Health Consultation")
    print("=" * 80)
    
    health_question = "I've been feeling tired lately and having trouble sleeping. What could be causing this?"
    print(f"\nPatient: {health_question}")
    print(f"\nDr. Johnson:")
    doctor_response = agent.interact_with_persona("doctor", health_question, reset_history=True)
    print(doctor_response)
    
    # Follow-up
    followup = "I'm also drinking a lot of coffee throughout the day."
    print(f"\n\nPatient (follow-up): {followup}")
    print(f"\nDr. Johnson:")
    doctor_followup = agent.interact_with_persona("doctor", followup)
    print(doctor_followup)
    
    # Example 2: Teacher Persona
    print("\n" + "=" * 80)
    print("EXAMPLE 2: Teacher Persona - Explaining Complex Topic")
    print("=" * 80)
    
    learning_question = "Can you explain how photosynthesis works?"
    print(f"\nStudent: {learning_question}")
    print(f"\nProfessor Chen:")
    teacher_response = agent.interact_with_persona("teacher", learning_question, reset_history=True)
    print(teacher_response)
    
    # Example 3: Critic Persona
    print("\n" + "=" * 80)
    print("EXAMPLE 3: Critic Persona - Essay Review")
    print("=" * 80)
    
    essay = """My essay argues that social media has both positive and negative impacts on society. 
It connects people but also spreads misinformation. It helps businesses but can harm mental health."""
    
    critique_request = f"Please review this essay introduction: {essay}"
    print(f"\nWriter: {critique_request}")
    print(f"\nMarcus Stone (Critic):")
    critic_response = agent.interact_with_persona("critic", critique_request, reset_history=True)
    print(critic_response)
    
    # Example 4: Coach Persona
    print("\n" + "=" * 80)
    print("EXAMPLE 4: Coach Persona - Goal Setting")
    print("=" * 80)
    
    goal_question = "I want to learn programming but I keep procrastinating. How can I stay motivated?"
    print(f"\nClient: {goal_question}")
    print(f"\nCoach Rivera:")
    coach_response = agent.interact_with_persona("coach", goal_question, reset_history=True)
    print(coach_response)
    
    # Example 5: Multi-Persona Analysis
    print("\n" + "=" * 80)
    print("EXAMPLE 5: Multi-Persona Analysis - Different Perspectives")
    print("=" * 80)
    
    question = "Should I pursue a career change at age 40?"
    print(f"\nQuestion: {question}")
    print("\nGetting perspectives from multiple personas...\n")
    
    personas_to_consult = ["coach", "analyst", "counselor"]
    # Note: counselor not defined, will be skipped
    personas_to_consult = ["coach", "analyst"]
    
    perspectives = agent.multi_persona_analysis(question, personas_to_consult)
    
    for persona_key, response in perspectives.items():
        persona = agent.get_persona_info(persona_key)
        print(f"\n{persona.name} ({persona.role}):")
        print("-" * 80)
        print(response)
    
    # Example 6: Persona Debate
    print("\n" + "=" * 80)
    print("EXAMPLE 6: Persona Debate - Creative vs Analyst")
    print("=" * 80)
    
    debate_topic = "What's more important for business success: creativity or data-driven decision making?"
    print(f"\nDebate Topic: {debate_topic}\n")
    
    debate_exchanges = agent.persona_debate(
        debate_topic,
        "creative",
        "analyst",
        rounds=2
    )
    
    for exchange in debate_exchanges:
        print(f"\nRound {exchange['round']} - {exchange['persona']}:")
        print("-" * 80)
        print(exchange['statement'])
    
    # Summary
    print("\n" + "=" * 80)
    print("ROLE-PLAYING/PERSONA PATTERN SUMMARY")
    print("=" * 80)
    print(f"""
Role-Playing Pattern Benefits:
1. Enhanced Expertise: Simulates domain-specific knowledge
2. Consistent Personality: Maintains character across interactions
3. Improved Engagement: More natural and relatable interactions
4. Context-Appropriate: Responses match expected communication style
5. Multi-Perspective Analysis: Diverse viewpoints on problems

Key Personas Demonstrated:
1. Doctor (Dr. Sarah Johnson): Empathetic health guidance
2. Teacher (Prof. Alex Chen): Patient, step-by-step explanations
3. Critic (Marcus Stone): Constructive analytical feedback
4. Coach (Coach Sam Rivera): Motivational goal-setting support
5. Analyst (Dr. Emily Watson): Data-driven strategic thinking
6. Creative (Luna Martinez): Imaginative idea generation

Persona Components:
- Identity: Name, role, background
- Expertise: Domain knowledge areas
- Personality: Communication traits
- Style: Tone, vocabulary, structure
- Approach: Problem-solving methodology

Best Practices:
1. Define personas thoroughly with clear characteristics
2. Maintain consistency across conversation turns
3. Use appropriate disclaimers (especially for professional advice)
4. Stay within persona's expertise boundaries
5. Enable history for multi-turn conversations
6. Test personas across diverse inputs
7. Monitor for inappropriate or biased responses

Advanced Techniques:
1. Multi-Persona Analysis: Get diverse perspectives
2. Persona Debates: Explore different viewpoints
3. Dynamic Switching: Change personas based on context
4. Persona Blending: Combine traits from multiple personas
5. Adaptive Expertise: Adjust knowledge depth to user level

Production Considerations:
- Clear AI disclosure (users know it's AI)
- Professional advice disclaimers
- Content safety guardrails
- Persona consistency monitoring
- User feedback collection
- Regular persona updates and improvements
- A/B testing persona variations

When to Use Role-Playing:
- Domain-specific expertise needed
- Consistent personality desired
- Engagement and relatability important
- Multiple perspectives valuable
- Specialized communication style required

Available Personas: {agent.list_personas()}
""")
    
    print("\n" + "=" * 80)
    print("Pattern 084 (Role-Playing/Persona) demonstration complete!")
    print("=" * 80)


if __name__ == "__main__":
    demonstrate_role_playing()

    

Agentic Design Pattern: Role Playing

Pattern 084: Role Playing – Line-by-Line Explanation

Overview

This document provides a detailed line-by-line explanation of the Role Playing pattern implementation.

---

Pattern Description

Pattern 084: Role-Playing/Persona Pattern

Description: The Role-Playing/Persona pattern involves instructing the LLM to adopt a specific role, character, or persona when generating responses. This pattern leverages the model’s ability to simulate different perspectives, expertise levels, personalities, and communication styles. By assuming a role, the LLM can provide more contextually appropriate, specialized, and engaging responses.

Role-playing is a powerful prompt engineering technique that:
- Enhances response quality through perspective-taking
- Enables domain expertise simulation (doctor, lawyer, scientist)
- Creates consistent personality in conversational agents
- Improves user engagement through character-based interaction
- Allows multi-perspective analysis of problems

Components: 1. Persona Definition – Identity (who the persona is) – Expertise (what they know) – Personality traits (how they communicate) – Background (relevant experience) – Goals and motivations

2. Communication Style
   - Tone (formal, casual, friendly, professional)
   - Vocabulary level (simple, technical, academic)
   - Response structure (detailed, concise, step-by-step)
   - Interaction patterns (questioning, advising, teaching)

3. Persona Consistency
   - Maintaining character across turns
   - Memory of previous interactions
   - Consistent viewpoint and opinions
   - Appropriate domain knowledge boundaries

4. Context Adaptation
   - Adjusting expertise level to user
   - Situational awareness
   - Emotional intelligence
   - Cultural sensitivity

Use Cases: 1. Expert Consultation – Medical advice (doctor persona) – Legal guidance (lawyer persona) – Technical support (engineer persona) – Financial planning (financial advisor persona) – Career counseling (career coach persona)

2. Educational Applications
   - Patient teacher (explaining complex topics)
   - Socratic tutor (guiding through questions)
   - Encouraging mentor (motivating learners)
   - Strict examiner (testing knowledge)

3. Creative Writing
   - Character dialogue generation
   - Story narration from specific viewpoint
   - Historical figure simulation
   - Fictional character interaction

4. Business Applications
   - Sales representative (persuasive, customer-focused)
   - Customer service (empathetic, problem-solving)
   - Business analyst (analytical, data-driven)
   - Executive advisor (strategic, big-picture)

5. Therapeutic and Counseling
   - Supportive friend (empathetic listener)
   - Life coach (motivational, goal-oriented)
   - Therapist (reflective, non-judgmental)
   - Meditation guide (calm, mindful)

LangChain Implementation: LangChain facilitates role-playing through: – System message configuration (setting persona) – Context management (maintaining persona) – Memory integration (persona consistency) – Custom prompt templates (structured personas) – Chat history management (conversation continuity)

Key Features: 1. Persona Library – Pre-defined expert personas – Customizable persona templates – Domain-specific knowledge bases – Personality trait combinations

2. Dynamic Persona Switching
   - Context-aware persona selection
   - Smooth transitions between personas
   - Multi-persona conversations
   - Persona blending

3. Expertise Calibration
   - Adjusting knowledge depth
   - Acknowledging limitations
   - Appropriate confidence levels
   - Domain boundary awareness

4. Personality Traits
   - Empathy and emotional intelligence
   - Humor and engagement style
   - Assertiveness levels
   - Teaching vs directing approaches

Best Practices: 1. Persona Definition – Be specific about role and expertise – Include relevant background details – Define communication style clearly – Set appropriate boundaries

2. Consistency Maintenance
   - Use memory for multi-turn conversations
   - Reference persona context regularly
   - Maintain viewpoint across interactions
   - Avoid out-of-character responses

3. Expertise Management
   - Stay within persona's knowledge domain
   - Acknowledge when unsure
   - Provide persona-appropriate disclaimers
   - Don't overreach expertise claims

4. Engagement Optimization
   - Match tone to user expectations
   - Balance expertise with accessibility
   - Use persona-appropriate examples
   - Maintain engaging interaction style

Trade-offs: Advantages: – Enhanced response quality and relevance – More engaging and natural interactions – Domain-specific expertise simulation – Consistent personality and tone – Better user trust and satisfaction – Multi-perspective problem analysis

Disadvantages:
- Requires detailed persona definitions
- May limit flexibility in responses
- Risk of inappropriate persona adoption
- Can be perceived as deceptive if not transparent
- Harder to maintain over long conversations
- May introduce unwanted biases

Production Considerations: 1. Transparency – Clearly indicate AI persona usage – Provide disclaimers for professional advice – Explain limitations of simulated expertise – Allow users to request different personas

2. Safety and Ethics
   - Avoid harmful or inappropriate personas
   - Include safety guardrails
   - Monitor for persona misuse
   - Implement content filtering
   - Respect professional boundaries

3. Quality Assurance
   - Test personas across diverse inputs
   - Validate expertise accuracy
   - Monitor consistency metrics
   - Gather user feedback
   - Regular persona updates

4. Scalability
   - Modular persona definitions
   - Reusable persona components
   - Version control for personas
   - A/B test persona variations
   - Performance monitoring per persona

5. Legal Compliance
   - Medical/legal advice disclaimers
   - Professional credential clarification
   - Terms of service alignment
   - Privacy considerations
   - Liability protection

---

File Statistics

• Total Lines: 706
• Classes: 4
• Functions: 1
• Constants: 0

---

Module Documentation

"""
Pattern 084: Role-Playing/Persona Pattern

Description:
    The Role-Playing/Persona pattern involves instructing the LLM to adopt a specific
    role, character, or persona when generating responses. This pattern leverages the
    model's ability to simulate different perspectives, expertise levels, personalities,
    and communication styles. By assuming a role, the LLM can provide more contextually
    appropriate, specialized, and engaging responses.

    Role-playing is a powerful prompt engineering technique that:
    - Enhances response quality through perspective-taking
    - Enables domain expertise simulation (doctor, lawyer, scientist)
    - Creates consistent personality in conversational agents
    - Improves user engagement through character-based interaction
    - Allows multi-perspective analysis of problems

Components:
    1. Persona Definition
       - Identity (who the persona is)
       - Expertise (what they know)
       - Personality traits (how they communicate)
       - Background (relevant experience)
       - Goals and motivations

    2. Communication Style
       - Tone (formal, casual, friendly, professional)
       - Vocabulary level (simple, technical, academic)
       - Response structure (detailed, concise, step-by-step)
       - Interaction patterns (questioning, advising, teaching)

    3. Persona Consistency
       - Maintaining character across turns
       - Memory of previous interactions
       - Consistent viewpoint and opinions
       - Appropriate domain knowledge boundaries

    4. Context Adaptation
       - Adjusting expertise level to user
       - Situational awareness
       - Emotional intelligence
       - Cultural sensitivity

Use Cases:
    1. Expert Consultation
       - Medical advice (doctor persona)
       - Legal guidance (lawyer persona)
       - Technical support (engineer persona)
       - Financial planning (financial advisor persona)
       - Career counseling (career coach persona)

    2. Educational Applications
       - Patient teacher (explaining complex topics)
       - Socratic tutor (guiding through questions)
       - Encouraging mentor (motivating learners)
       - Strict examiner (testing knowledge)

    3. Creative Writing
       - Character dialogue generation
       - Story narration from specific viewpoint
       - Historical figure simulation
       - Fictional character interaction

    4. Business Applications
       - Sales representative (persuasive, customer-focused)
       - Customer service (empathetic, problem-solving)
       - Business analyst (analytical, data-driven)
       - Executive advisor (strategic, big-picture)

    5. Therapeutic and Counseling
       - Supportive friend (empathetic listener)
       - Life coach (motivational, goal-oriented)
       - Therapist (reflective, non-judgmental)
       - Meditation guide (calm, mindful)

LangChain Implementation:
    LangChain facilitates role-playing through:
    - System message configuration (setting persona)
    - Context management (maintaining persona)
    - Memory integration (persona consistency)
    - Custom prompt templates (structured personas)
    - Chat history management (conversation continuity)

Key Features:
    1. Persona Library
       - Pre-defined expert personas
       - Customizable persona templates
       - Domain-specific knowledge bases
       - Personality trait combinations

    2. Dynamic Persona Switching
       - Context-aware persona selection
       - Smooth transitions between personas
       - Multi-persona conversations
       - Persona blending

    3. Expertise Calibration
       - Adjusting knowledge depth
       - Acknowledging limitations
       - Appropriate confidence levels
       - Domain boundary awareness

    4. Personality Traits
       - Empathy and emotional intelligence
       - Humor and engagement style
       - Assertiveness levels
       - Teaching vs directing approaches

Best Practices:
    1. Persona Definition
       - Be specific about role and expertise
       - Include relevant background details
       - Define communication style clearly
       - Set appropriate boundaries

    2. Consistency Maintenance
       - Use memory for multi-turn conversations
       - Reference persona context regularly
       - Maintain viewpoint across interactions
       - Avoid out-of-character responses

    3. Expertise Management
       - Stay within persona's knowledge domain
       - Acknowledge when unsure
       - Provide persona-appropriate disclaimers
       - Don't overreach expertise claims

    4. Engagement Optimization
       - Match tone to user expectations
       - Balance expertise with accessibility
       - Use persona-appropriate examples
       - Maintain engaging interaction style

Trade-offs:
    Advantages:
    - Enhanced response quality and relevance
    - More engaging and natural interactions
    - Domain-specific expertise simulation
    - Consistent personality and tone
    - Better user trust and satisfaction
    - Multi-perspective problem analysis

    Disadvantages:
    - Requires detailed persona definitions
    - May limit flexibility in responses
    - Risk of inappropriate persona adoption
    - Can be perceived as deceptive if not transparent
    - Harder to maintain over long conversations
    - May introduce unwanted biases

Production Considerations:
    1. Transparency
       - Clearly indicate AI persona usage
       - Provide disclaimers for professional advice
       - Explain limitations of simulated expertise
       - Allow users to request different personas

    2. Safety and Ethics
       - Avoid harmful or inappropriate personas
       - Include safety guardrails
       - Monitor for persona misuse
       - Implement content filtering
       - Respect professional boundaries

    3. Quality Assurance
       - Test personas across diverse inputs
       - Validate expertise accuracy
       - Monitor consistency metrics
       - Gather user feedback
       - Regular persona updates

    4. Scalability
       - Modular persona definitions
       - Reusable persona components
       - Version control for personas
       - A/B test persona variations
       - Performance monitoring per persona

    5. Legal Compliance
       - Medical/legal advice disclaimers
       - Professional credential clarification
       - Terms of service alignment
       - Privacy considerations
       - Liability protection
"""

Purpose: This module implements the Role Playing pattern, providing a structured approach to agent design and implementation.

---

Import Statements

Standard Library Imports

import dataclasses.dataclass
import enum.Enum
import langchain_core.messages.AIMessage
import langchain_core.messages.HumanMessage
import langchain_core.messages.SystemMessage
import langchain_core.output_parsers.StrOutputParser
import langchain_core.prompts.ChatPromptTemplate
import langchain_core.prompts.MessagesPlaceholder
import os
import typing.Any
import typing.Dict
import typing.List
import typing.Optional

Purpose: Core Python functionality: – dataclasses.dataclass: Simplified class definitions with @dataclass – enum.Enum: Enumeration support – langchain_core.messages.AIMessage: Regular expression operations – langchain_core.messages.HumanMessage: Regular expression operations – langchain_core.messages.SystemMessage: Regular expression operations – langchain_core.output_parsers.StrOutputParser: Regular expression operations – langchain_core.prompts.ChatPromptTemplate: Regular expression operations – langchain_core.prompts.MessagesPlaceholder: Regular expression operations – os: Environment variable access and system operations – typing.Any: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Dict: Type hints for better code documentation (List, Dict, Any, Optional) – typing.List: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Optional: Type hints for better code documentation (List, Dict, Any, Optional)

LangChain Imports

from langchain_openai.ChatOpenAI

Purpose: LangChain framework components: – LLM Interface: Wrapper for OpenAI models – Chains: Composable building blocks for complex workflows

Other Imports

from dotenv.load_dotenv

Purpose: Additional dependencies for specialized functionality.

---

Classes

Class: PersonaType

Lines 201-210

Inherits from: Enum

Purpose:

Types of personas

---

Class: Persona

Lines 214-222

Purpose:

Defines a role-playing persona

---

Class: PersonaConfig

Lines 226-231

Purpose:

Configuration for persona-based interaction

---

Class: RolePlayingAgent

Lines 234-531

Purpose:

Agent that adopts different personas for specialized interactions.

This agent demonstrates:
1. Multiple pre-defined personas
2. Persona switching
3. Consistent character maintenance
4. Domain-specific expertise simulation

Methods:

__init__(self, config)

Lines 245-258

Purpose:

Initialize role-playing agent.

Args:
    config: Configuration for persona interactions

Parameters: – config: Config

_initialize_personas(self)

Lines 260-390

Purpose:

Initialize library of personas

Parameters:

Returns: <ast.Subscript object at 0x000001A06D83E110>

interact_with_persona(self, persona_key, user_message, reset_history)

Lines 392-439

Purpose:

Interact with a specific persona.

Args:
    persona_key: Key of persona to use
    user_message: User's message
    reset_history: Whether to reset conversation history

Returns:
    Persona's response

Parameters: – persona_key: Persona key – user_message: User message – reset_history: Reset history

Returns: str

multi_persona_analysis(self, question, persona_keys)

Lines 441-464

Purpose:

Get perspectives from multiple personas.

Args:
    question: Question to analyze
    persona_keys: List of persona keys to consult

Returns:
    Dictionary of persona responses

Parameters: – question: Question to answer – persona_keys: Persona keys

Returns: <ast.Subscript object at 0x000001A06D6A9DD0>

persona_debate(self, topic, persona1_key, persona2_key, rounds)

Lines 466-523

Purpose:

Have two personas debate a topic.

Args:
    topic: Topic to debate
    persona1_key: First persona key
    persona2_key: Second persona key
    rounds: Number of debate rounds

Returns:
    List of debate exchanges

Parameters: – topic: Topic – persona1_key: Persona1 key – persona2_key: Persona2 key – rounds: Rounds

Returns: <ast.Subscript object at 0x000001A06D159F90>

get_persona_info(self, persona_key)

Lines 525-527

Purpose:

Get information about a persona

Parameters: – persona_key: Persona key

Returns: <ast.Subscript object at 0x000001A06D15AC10>

list_personas(self)

Lines 529-531

Purpose:

List available personas

Parameters:

Returns: <ast.Subscript object at 0x000001A06D60A810>

---

Functions

demonstrate_role_playing()

Lines 534-701

Purpose:

Demonstrate role-playing/persona patterns

---

Key Concepts Summary

Pattern Overview

The Role Playing pattern provides a structured approach to building AI agents with specific capabilities and behaviors. This implementation leverages LangChain’s composable components to create a robust and maintainable solution.

Core Components

1. Agent Class: Main implementation of the pattern logic
2. Prompt Engineering: Carefully crafted prompts to guide LLM behavior
3. Tool Integration: Optional tools for extended functionality
4. Memory Management: State handling across interactions
5. Output Parsing: Structured response processing

When to Use

✅ Good for: – Scenarios requiring this specific pattern’s strengths – Applications needing structured agent behavior – Systems where explainability is important – Production environments requiring reliability

❌ Not ideal for: – Simple single-prompt tasks – Scenarios requiring different pattern characteristics – Resource-constrained environments (if computationally intensive)

---

Best Practices Demonstrated

1. Environment Configuration

• ✅ Use .env files for API keys
• ✅ Never hardcode credentials
• ✅ Validate environment variables on startup

2. Error Handling

• ✅ Graceful degradation on failures
• ✅ Informative error messages
• ✅ Logging for debugging

3. Prompt Engineering

• ✅ Clear, specific instructions
• ✅ Structured output formats
• ✅ Few-shot examples when beneficial

4. Code Organization

• ✅ Separation of concerns (class-based design)
• ✅ Type hints for clarity
• ✅ Docstrings for documentation

5. Performance Optimization

• ✅ Appropriate temperature settings
• ✅ Token budget management
• ✅ Caching where applicable

---

LangChain Components Used

• ChatOpenAI: OpenAI model interface
• Chains: Sequential component composition

Why LangChain? – Composable abstractions for LLM applications – Rich ecosystem of integrations – Production-ready patterns – Active community and development

---

Execution Flow

1. Initialize agent with configuration
   ↓
2. Load environment and dependencies
   ↓
3. Prepare prompt with input data
   ↓
4. Invoke LLM through LangChain
   ↓
5. Parse and validate response
   ↓
6. Return formatted result

Key Decision Points: – Model selection (temperature, model type) – Prompt structure (few-shot vs zero-shot) – Error handling strategy – Output format validation

---

Performance Considerations

Computational Cost

• LLM Calls: Primary cost factor (based on tokens)
• Latency: Depends on model and input size
• Memory: Minimal for most patterns

Optimization Strategies

1. Prompt Optimization: Minimize tokens while maintaining clarity
2. Caching: Cache repeated queries/responses
3. Model Selection: Use smaller models when appropriate
4. Batch Processing: Group similar requests
5. Streaming: For long responses, stream tokens

Cost Estimates (Approximate)

• GPT-3.5-turbo: ~$0.002 per request
• GPT-4: ~$0.03 per request
• GPT-4-turbo: ~$0.01 per request

Actual costs vary based on input/output token counts

---

Common Pitfalls to Avoid

1. Prompt Design Issues

❌ Vague or ambiguous instructions
✅ Clear, specific, testable prompts

2. Error Handling

❌ Ignoring API errors or rate limits
✅ Implement retry logic and graceful degradation

3. Token Management

❌ Exceeding model token limits
✅ Truncate inputs, chunk processing

4. Security

❌ Exposing API keys in code
✅ Use environment variables

5. Testing

❌ Only testing happy paths
✅ Test edge cases, failures, malformed inputs

6. Cost Management

❌ Unmonitored API usage
✅ Implement budgets, alerts, and monitoring

---

Further Improvements

Production Enhancements

1. Monitoring: Add logging, metrics, tracing
2. Validation: Input/output schema validation
3. Rate Limiting: Implement request throttling
4. Caching: Redis or in-memory cache for responses
5. Async Support: Non-blocking operations for scalability

Feature Extensions

1. Multi-Model Support: Switch between providers
2. Fallback Models: Graceful degradation strategy
3. Custom Tools: Domain-specific function calling
4. Advanced Memory: Vector stores for long-term memory
5. Evaluation: Automated testing and benchmarking

Code Quality

1. Testing: Unit tests, integration tests
2. Documentation: API docs, usage examples
3. Type Safety: Stricter type hints
4. Configuration: Externalized config files

---

Related Patterns

• Pattern 083
• Pattern 085

Pattern Families: – Reasoning Patterns (001-011) – Multi-Agent Patterns (012-019) – Tool Use Patterns (020-025) – Memory Patterns (026-031) – And many more…

---

Summary

The Role Playing pattern provides a powerful approach to building AI agents with specific capabilities. This implementation demonstrates:

✅ Key Strengths: – Clean, maintainable code structure – LangChain integration for composability – Type hints and documentation – Production-ready patterns

🎯 Best Use Cases: – Applications requiring this pattern’s characteristics – Systems needing structured agent behavior – Production environments requiring reliability

📚 Learning Resources: – LangChain Documentation: https://python.langchain.com/ – Pattern Documentation: See related patterns – OpenAI API Reference: https://platform.openai.com/docs/

---

File: 084_role_playing.py
Total Lines: 706
Pattern Category: AI Agent Design Pattern #084

---

Generated from actual implementation code
For educational and reference purposes

    """
Pattern 083: Few-Shot Learning Pattern

Description:
    The Few-Shot Learning pattern uses example-based prompting to teach the LLM
    how to perform tasks through demonstrations. Instead of relying on zero-shot
    capabilities, this pattern provides the model with a few examples of input-output
    pairs that illustrate the desired behavior. This approach leverages the LLM's
    in-context learning abilities to generalize from examples.

    Few-shot learning is one of the most fundamental prompt engineering techniques
    and is particularly effective when:
    - The task format is non-standard or complex
    - Consistent output format is required
    - Specific style or tone needs to be maintained
    - Domain-specific knowledge needs to be demonstrated
    - Edge cases need to be handled in specific ways

Components:
    1. Example Selection
       - Static examples (fixed set)
       - Dynamic examples (selected based on similarity)
       - Diverse examples (covering different scenarios)
       - Representative examples (typical cases)

    2. Example Formatting
       - Clear input-output structure
       - Consistent formatting across examples
       - Annotations and explanations (when needed)
       - Order considerations (easy to hard, chronological)

    3. Example Storage
       - In-prompt storage (small example sets)
       - Vector database storage (large example sets)
       - Retrieval mechanisms (similarity-based)
       - Example caching strategies

    4. Generalization Strategy
       - Pattern recognition from examples
       - Analogical reasoning
       - Template extraction
       - Feature learning

Use Cases:
    1. Text Classification
       - Sentiment analysis with custom categories
       - Intent recognition for domain-specific applications
       - Content moderation with nuanced rules
       - Topic categorization with specific taxonomies

    2. Data Extraction
       - Structured information extraction from unstructured text
       - Entity recognition with custom entity types
       - Relationship extraction between entities
       - Attribute extraction with specific formats

    3. Code Generation
       - API usage examples for specific libraries
       - Code style consistency (following project conventions)
       - Design pattern implementation
       - Test case generation

    4. Creative Writing
       - Style mimicry (writing in specific author's style)
       - Format consistency (poems, stories, articles)
       - Tone control (formal, casual, humorous)
       - Genre-specific writing (sci-fi, mystery, romance)

    5. Translation and Transformation
       - Domain-specific translation (legal, medical, technical)
       - Format conversion (JSON to YAML, CSV to markdown)
       - Data normalization (various formats to standard)
       - Code refactoring patterns

LangChain Implementation:
    LangChain provides excellent support for few-shot learning through:
    - FewShotChatMessagePromptTemplate for chat models
    - FewShotPromptTemplate for completion models
    - Example selectors (semantic similarity, length-based, max marginal relevance)
    - Vector store-backed example selection
    - Custom example formatting

Key Features:
    1. Static Few-Shot Learning
       - Predefined set of examples
       - Fast and deterministic
       - Good for stable tasks
       - No retrieval overhead

    2. Dynamic Few-Shot Learning
       - Examples selected based on input similarity
       - More adaptive to diverse inputs
       - Requires vector database
       - Better generalization

    3. Example Quality Management
       - Diverse examples covering edge cases
       - Clear and unambiguous examples
       - Representative of target distribution
       - Regularly updated based on performance

    4. Performance Optimization
       - Token budget management (limiting example count)
       - Example compression (removing redundancy)
       - Caching frequently used example sets
       - Lazy loading for large example sets

Best Practices:
    1. Example Selection
       - Use 3-5 examples for most tasks (sweet spot)
       - Include diverse examples covering different patterns
       - Order examples from simple to complex
       - Ensure examples are correct and high-quality

    2. Example Formatting
       - Use consistent structure across all examples
       - Clearly separate input from output
       - Add brief explanations for complex cases
       - Use delimiters to avoid confusion

    3. Dynamic Selection
       - Use semantic similarity for input-dependent selection
       - Balance diversity and relevance in selection
       - Cache embeddings for better performance
       - Set appropriate k value (number of examples)

    4. Iteration and Improvement
       - Start with few examples, add more if needed
       - A/B test different example sets
       - Monitor performance across different inputs
       - Update examples based on failure cases

Trade-offs:
    Advantages:
    - Significantly improves task performance vs zero-shot
    - More reliable than instructions alone
    - Enables consistent output formatting
    - Works well for complex or ambiguous tasks
    - Reduces need for fine-tuning

    Disadvantages:
    - Increases token usage (more expensive)
    - May not scale to tasks requiring many examples
    - Examples can bias the model inappropriately
    - Requires careful example curation
    - Dynamic selection adds latency and complexity

Production Considerations:
    1. Token Budget Management
       - Monitor tokens used by examples
       - Balance example count with context window
       - Use example compression when possible
       - Consider model's context limit

    2. Example Maintenance
       - Version control for example sets
       - Regular review and updates
       - Performance tracking per example
       - A/B testing of example variations

    3. Scalability
       - Cache example embeddings
       - Use efficient vector stores
       - Batch process when possible
       - Consider example pruning strategies

    4. Quality Assurance
       - Validate examples are correct
       - Test examples independently
       - Monitor output quality metrics
       - Handle example selection failures gracefully

    5. Cost Optimization
       - Balance example count with quality
       - Use dynamic selection judiciously
       - Cache common example sets
       - Monitor cost per query
"""

import os
from typing import List, Dict, Any, Optional
from dataclasses import dataclass
from enum import Enum
from dotenv import load_dotenv

from langchain_core.prompts import (
    ChatPromptTemplate,
    FewShotChatMessagePromptTemplate,
    PromptTemplate
)
from langchain_core.output_parsers import StrOutputParser
from langchain_openai import ChatOpenAI, OpenAIEmbeddings
from langchain_core.example_selectors import (
    SemanticSimilarityExampleSelector,
    MaxMarginalRelevanceExampleSelector
)
from langchain_chroma import Chroma

load_dotenv()


class ExampleSelectionStrategy(Enum):
    """Strategy for selecting examples"""
    STATIC = "static"  # Fixed set of examples
    SEMANTIC_SIMILARITY = "semantic_similarity"  # Based on input similarity
    MAX_MARGINAL_RELEVANCE = "max_marginal_relevance"  # Diverse + relevant


@dataclass
class FewShotExample:
    """Represents a single few-shot example"""
    input: str
    output: str
    explanation: Optional[str] = None
    metadata: Optional[Dict[str, Any]] = None


@dataclass
class FewShotConfig:
    """Configuration for few-shot learning"""
    num_examples: int = 3
    selection_strategy: ExampleSelectionStrategy = ExampleSelectionStrategy.STATIC
    temperature: float = 0.3
    model_name: str = "gpt-3.5-turbo"
    include_explanations: bool = False


class FewShotLearningAgent:
    """
    Agent that uses few-shot learning to teach LLM through examples.
    
    This agent demonstrates various few-shot learning approaches:
    1. Static few-shot (fixed examples)
    2. Dynamic few-shot (similarity-based selection)
    3. Task-specific few-shot applications
    """
    
    def __init__(self, config: Optional[FewShotConfig] = None):
        """
        Initialize few-shot learning agent.
        
        Args:
            config: Configuration for few-shot learning
        """
        self.config = config or FewShotConfig()
        self.llm = ChatOpenAI(
            model=self.config.model_name,
            temperature=self.config.temperature
        )
        self.embeddings = OpenAIEmbeddings()
    
    def create_static_few_shot_chain(
        self,
        examples: List[FewShotExample],
        task_instruction: str,
        input_variable: str = "input"
    ):
        """
        Create a chain with static few-shot examples.
        
        Args:
            examples: List of few-shot examples
            task_instruction: Instruction for the task
            input_variable: Name of input variable
            
        Returns:
            Runnable chain with few-shot prompting
        """
        # Format examples as dictionaries
        example_dicts = [
            {"input": ex.input, "output": ex.output}
            for ex in examples
        ]
        
        # Create example prompt template
        example_prompt = ChatPromptTemplate.from_messages([
            ("human", "{input}"),
            ("ai", "{output}")
        ])
        
        # Create few-shot prompt
        few_shot_prompt = FewShotChatMessagePromptTemplate(
            example_prompt=example_prompt,
            examples=example_dicts
        )
        
        # Create final prompt
        final_prompt = ChatPromptTemplate.from_messages([
            ("system", task_instruction),
            few_shot_prompt,
            ("human", "{" + input_variable + "}")
        ])
        
        # Create chain
        chain = final_prompt | self.llm | StrOutputParser()
        
        return chain
    
    def create_dynamic_few_shot_chain(
        self,
        examples: List[FewShotExample],
        task_instruction: str,
        input_variable: str = "input",
        k: int = 3
    ):
        """
        Create a chain with dynamic example selection based on similarity.
        
        Args:
            examples: Pool of examples to select from
            task_instruction: Instruction for the task
            input_variable: Name of input variable
            k: Number of examples to select
            
        Returns:
            Runnable chain with dynamic few-shot prompting
        """
        # Format examples as dictionaries
        example_dicts = [
            {"input": ex.input, "output": ex.output}
            for ex in examples
        ]
        
        # Create example selector using semantic similarity
        example_selector = SemanticSimilarityExampleSelector.from_examples(
            example_dicts,
            self.embeddings,
            Chroma,
            k=k,
            input_keys=["input"]
        )
        
        # Create example prompt template
        example_prompt = ChatPromptTemplate.from_messages([
            ("human", "{input}"),
            ("ai", "{output}")
        ])
        
        # Create few-shot prompt with selector
        few_shot_prompt = FewShotChatMessagePromptTemplate(
            example_prompt=example_prompt,
            example_selector=example_selector,
            input_variables=[input_variable]
        )
        
        # Create final prompt
        final_prompt = ChatPromptTemplate.from_messages([
            ("system", task_instruction),
            few_shot_prompt,
            ("human", "{" + input_variable + "}")
        ])
        
        # Create chain
        chain = final_prompt | self.llm | StrOutputParser()
        
        return chain
    
    def sentiment_analysis_few_shot(
        self,
        text: str,
        examples: Optional[List[FewShotExample]] = None,
        use_dynamic: bool = False
    ) -> str:
        """
        Perform sentiment analysis using few-shot learning.
        
        Args:
            text: Text to analyze
            examples: Optional custom examples
            use_dynamic: Whether to use dynamic selection
            
        Returns:
            Sentiment classification
        """
        # Default examples if none provided
        if examples is None:
            examples = [
                FewShotExample(
                    input="This movie was absolutely fantastic! I loved every minute.",
                    output="Positive"
                ),
                FewShotExample(
                    input="The product broke after one day. Terrible quality.",
                    output="Negative"
                ),
                FewShotExample(
                    input="It's okay, nothing special but not bad either.",
                    output="Neutral"
                ),
                FewShotExample(
                    input="I'm thrilled with this purchase! Exceeded expectations.",
                    output="Positive"
                ),
                FewShotExample(
                    input="Complete waste of money. Very disappointed.",
                    output="Negative"
                )
            ]
        
        task_instruction = """You are a sentiment analyzer. Classify the sentiment of the given text as Positive, Negative, or Neutral.
Look at the examples to understand the task."""
        
        # Create chain based on selection strategy
        if use_dynamic:
            chain = self.create_dynamic_few_shot_chain(
                examples,
                task_instruction,
                k=3
            )
        else:
            # Use first 3 examples for static
            chain = self.create_static_few_shot_chain(
                examples[:3],
                task_instruction
            )
        
        result = chain.invoke({"input": text})
        return result
    
    def entity_extraction_few_shot(
        self,
        text: str,
        examples: Optional[List[FewShotExample]] = None
    ) -> str:
        """
        Extract entities using few-shot learning.
        
        Args:
            text: Text to extract entities from
            examples: Optional custom examples
            
        Returns:
            Extracted entities in JSON format
        """
        # Default examples
        if examples is None:
            examples = [
                FewShotExample(
                    input="Apple Inc. announced a new iPhone model in Cupertino on September 12th.",
                    output='{"organization": "Apple Inc.", "product": "iPhone", "location": "Cupertino", "date": "September 12th"}'
                ),
                FewShotExample(
                    input="Elon Musk tweeted about Tesla's production numbers from the Fremont factory.",
                    output='{"person": "Elon Musk", "organization": "Tesla", "location": "Fremont factory"}'
                ),
                FewShotExample(
                    input="The meeting at Microsoft headquarters in Seattle was attended by Satya Nadella.",
                    output='{"organization": "Microsoft", "location": "Seattle", "person": "Satya Nadella"}'
                )
            ]
        
        task_instruction = """Extract entities from the text and format them as JSON.
Include: person, organization, location, date, product (if present).
Follow the format shown in the examples."""
        
        chain = self.create_static_few_shot_chain(
            examples,
            task_instruction
        )
        
        result = chain.invoke({"input": text})
        return result
    
    def code_generation_few_shot(
        self,
        description: str,
        examples: Optional[List[FewShotExample]] = None
    ) -> str:
        """
        Generate code using few-shot learning.
        
        Args:
            description: Description of desired code
            examples: Optional custom examples
            
        Returns:
            Generated code
        """
        # Default examples
        if examples is None:
            examples = [
                FewShotExample(
                    input="Create a function that returns the sum of two numbers",
                    output="""def add_numbers(a, b):
    \"\"\"Return the sum of two numbers.\"\"\"
    return a + b"""
                ),
                FewShotExample(
                    input="Create a function that checks if a string is a palindrome",
                    output="""def is_palindrome(s):
    \"\"\"Check if a string is a palindrome.\"\"\"
    return s == s[::-1]"""
                ),
                FewShotExample(
                    input="Create a function that finds the maximum value in a list",
                    output="""def find_max(lst):
    \"\"\"Find the maximum value in a list.\"\"\"
    if not lst:
        return None
    return max(lst)"""
                )
            ]
        
        task_instruction = """Generate Python code based on the description.
Follow the style and format shown in the examples.
Include docstrings and proper formatting."""
        
        chain = self.create_static_few_shot_chain(
            examples,
            task_instruction,
            input_variable="input"
        )
        
        result = chain.invoke({"input": description})
        return result
    
    def style_transfer_few_shot(
        self,
        text: str,
        target_style: str,
        examples: Optional[List[FewShotExample]] = None
    ) -> str:
        """
        Transfer text to a specific style using few-shot learning.
        
        Args:
            text: Text to transform
            target_style: Target style (e.g., "formal", "casual", "poetic")
            examples: Optional custom examples
            
        Returns:
            Transformed text
        """
        # Style-specific examples
        style_examples = {
            "formal": [
                FewShotExample(
                    input="Hey, can you help me out with this?",
                    output="I would appreciate your assistance with this matter."
                ),
                FewShotExample(
                    input="That's totally wrong!",
                    output="I respectfully disagree with that assessment."
                ),
                FewShotExample(
                    input="Thanks a lot for everything!",
                    output="I extend my sincere gratitude for your comprehensive support."
                )
            ],
            "casual": [
                FewShotExample(
                    input="I would like to request your assistance.",
                    output="Hey, could you help me out?"
                ),
                FewShotExample(
                    input="This is highly problematic.",
                    output="This is a real problem."
                ),
                FewShotExample(
                    input="I appreciate your effort.",
                    output="Thanks for trying!"
                )
            ],
            "poetic": [
                FewShotExample(
                    input="The sun set over the mountains.",
                    output="Golden rays bid farewell, as peaks embrace the fading light."
                ),
                FewShotExample(
                    input="Rain started falling.",
                    output="Heaven's tears descended, kissing the earth below."
                ),
                FewShotExample(
                    input="The bird flew away.",
                    output="On wings of freedom, it soared beyond the horizon's edge."
                )
            ]
        }
        
        examples = examples or style_examples.get(target_style.lower(), [])
        
        if not examples:
            return "Style not supported. Available: formal, casual, poetic"
        
        task_instruction = f"""Transform the text into {target_style} style.
Follow the transformation pattern shown in the examples."""
        
        chain = self.create_static_few_shot_chain(
            examples,
            task_instruction
        )
        
        result = chain.invoke({"input": text})
        return result
    
    def compare_zero_shot_vs_few_shot(
        self,
        text: str,
        task: str = "sentiment"
    ) -> Dict[str, str]:
        """
        Compare zero-shot vs few-shot performance.
        
        Args:
            text: Input text
            task: Task to perform
            
        Returns:
            Dictionary with both results
        """
        # Zero-shot prompt
        zero_shot_prompt = ChatPromptTemplate.from_messages([
            ("system", "Classify the sentiment as Positive, Negative, or Neutral."),
            ("human", "{input}")
        ])
        zero_shot_chain = zero_shot_prompt | self.llm | StrOutputParser()
        zero_shot_result = zero_shot_chain.invoke({"input": text})
        
        # Few-shot result
        few_shot_result = self.sentiment_analysis_few_shot(text)
        
        return {
            "zero_shot": zero_shot_result,
            "few_shot": few_shot_result
        }


def demonstrate_few_shot_learning():
    """Demonstrate few-shot learning patterns"""
    print("=" * 80)
    print("FEW-SHOT LEARNING PATTERN DEMONSTRATION")
    print("=" * 80)
    
    agent = FewShotLearningAgent()
    
    # Example 1: Sentiment Analysis
    print("\n" + "=" * 80)
    print("EXAMPLE 1: Sentiment Analysis with Few-Shot Learning")
    print("=" * 80)
    
    test_texts = [
        "This restaurant exceeded all my expectations! Amazing food!",
        "The service was slow and the food was cold. Not going back.",
        "It was fine. Nothing special but acceptable."
    ]
    
    for text in test_texts:
        print(f"\nText: {text}")
        sentiment = agent.sentiment_analysis_few_shot(text)
        print(f"Sentiment: {sentiment}")
    
    # Example 2: Entity Extraction
    print("\n" + "=" * 80)
    print("EXAMPLE 2: Entity Extraction with Few-Shot Learning")
    print("=" * 80)
    
    test_text = "Jeff Bezos announced Amazon's expansion into the healthcare sector at a conference in Las Vegas last Monday."
    print(f"\nText: {test_text}")
    entities = agent.entity_extraction_few_shot(test_text)
    print(f"Entities: {entities}")
    
    # Example 3: Code Generation
    print("\n" + "=" * 80)
    print("EXAMPLE 3: Code Generation with Few-Shot Learning")
    print("=" * 80)
    
    code_requests = [
        "Create a function that calculates factorial of a number",
        "Create a function that reverses a string"
    ]
    
    for request in code_requests:
        print(f"\nRequest: {request}")
        code = agent.code_generation_few_shot(request)
        print(f"Generated Code:\n{code}")
    
    # Example 4: Style Transfer
    print("\n" + "=" * 80)
    print("EXAMPLE 4: Style Transfer with Few-Shot Learning")
    print("=" * 80)
    
    original_text = "I need your help with this problem."
    styles = ["formal", "casual", "poetic"]
    
    print(f"\nOriginal: {original_text}")
    for style in styles:
        transformed = agent.style_transfer_few_shot(original_text, style)
        print(f"{style.title()} style: {transformed}")
    
    # Example 5: Zero-Shot vs Few-Shot Comparison
    print("\n" + "=" * 80)
    print("EXAMPLE 5: Zero-Shot vs Few-Shot Comparison")
    print("=" * 80)
    
    ambiguous_text = "The movie was interesting and thought-provoking, though quite challenging to follow."
    print(f"\nAmbiguous Text: {ambiguous_text}")
    
    comparison = agent.compare_zero_shot_vs_few_shot(ambiguous_text)
    print(f"\nZero-Shot Result: {comparison['zero_shot']}")
    print(f"Few-Shot Result: {comparison['few_shot']}")
    print("\nNote: Few-shot learning often provides more consistent and format-compliant outputs")
    
    # Example 6: Dynamic Example Selection
    print("\n" + "=" * 80)
    print("EXAMPLE 6: Dynamic Example Selection (Similarity-Based)")
    print("=" * 80)
    
    test_text = "I'm so excited about this purchase! Best decision ever!"
    print(f"\nText: {test_text}")
    print("\nStatic Selection:")
    static_result = agent.sentiment_analysis_few_shot(test_text, use_dynamic=False)
    print(f"Result: {static_result}")
    
    print("\nDynamic Selection (picks most similar examples):")
    dynamic_result = agent.sentiment_analysis_few_shot(test_text, use_dynamic=True)
    print(f"Result: {dynamic_result}")
    
    # Summary
    print("\n" + "=" * 80)
    print("FEW-SHOT LEARNING SUMMARY")
    print("=" * 80)
    print("""
Few-Shot Learning Pattern Benefits:
1. Improved Accuracy: Examples guide the model to better performance
2. Consistency: Output format matches examples consistently
3. Task Clarity: Examples disambiguate task requirements
4. Style Control: Examples demonstrate desired tone and style
5. No Fine-Tuning: Achieves good results without model retraining

Key Techniques Demonstrated:
1. Static Few-Shot: Fixed examples for consistent tasks
2. Dynamic Few-Shot: Similarity-based example selection
3. Task-Specific Examples: Domain-appropriate demonstrations
4. Style Transfer: Teaching specific writing styles
5. Format Control: Ensuring structured outputs

Best Practices:
1. Use 3-5 examples for most tasks (sweet spot)
2. Ensure examples are diverse and representative
3. Order examples from simple to complex
4. Include edge cases in examples
5. Keep examples concise to save tokens
6. Use dynamic selection for varied inputs
7. Validate examples are correct and high-quality
8. Monitor token usage and costs

When to Use Few-Shot vs Zero-Shot:
- Use few-shot when: output format is specific, task is ambiguous, consistency matters
- Use zero-shot when: task is simple, tokens are limited, speed is critical
- Use dynamic selection when: inputs are diverse, quality is paramount

Production Considerations:
- Cache example embeddings for performance
- Monitor token usage (examples increase costs)
- A/B test different example sets
- Version control example datasets
- Update examples based on failure cases
- Balance example count with quality needs
""")
    
    print("\n" + "=" * 80)
    print("Pattern 083 (Few-Shot Learning) demonstration complete!")
    print("=" * 80)


if __name__ == "__main__":
    demonstrate_few_shot_learning()

    

Agentic Design Pattern: Few Shot Learning

Pattern 083: Few Shot Learning – Line-by-Line Explanation

Overview

This document provides a detailed line-by-line explanation of the Few Shot Learning pattern implementation.

---

Pattern Description

Pattern 083: Few-Shot Learning Pattern

Description: The Few-Shot Learning pattern uses example-based prompting to teach the LLM how to perform tasks through demonstrations. Instead of relying on zero-shot capabilities, this pattern provides the model with a few examples of input-output pairs that illustrate the desired behavior. This approach leverages the LLM’s in-context learning abilities to generalize from examples.

Few-shot learning is one of the most fundamental prompt engineering techniques
and is particularly effective when:
- The task format is non-standard or complex
- Consistent output format is required
- Specific style or tone needs to be maintained
- Domain-specific knowledge needs to be demonstrated
- Edge cases need to be handled in specific ways

Components: 1. Example Selection – Static examples (fixed set) – Dynamic examples (selected based on similarity) – Diverse examples (covering different scenarios) – Representative examples (typical cases)

2. Example Formatting
   - Clear input-output structure
   - Consistent formatting across examples
   - Annotations and explanations (when needed)
   - Order considerations (easy to hard, chronological)

3. Example Storage
   - In-prompt storage (small example sets)
   - Vector database storage (large example sets)
   - Retrieval mechanisms (similarity-based)
   - Example caching strategies

4. Generalization Strategy
   - Pattern recognition from examples
   - Analogical reasoning
   - Template extraction
   - Feature learning

Use Cases: 1. Text Classification – Sentiment analysis with custom categories – Intent recognition for domain-specific applications – Content moderation with nuanced rules – Topic categorization with specific taxonomies

2. Data Extraction
   - Structured information extraction from unstructured text
   - Entity recognition with custom entity types
   - Relationship extraction between entities
   - Attribute extraction with specific formats

3. Code Generation
   - API usage examples for specific libraries
   - Code style consistency (following project conventions)
   - Design pattern implementation
   - Test case generation

4. Creative Writing
   - Style mimicry (writing in specific author's style)
   - Format consistency (poems, stories, articles)
   - Tone control (formal, casual, humorous)
   - Genre-specific writing (sci-fi, mystery, romance)

5. Translation and Transformation
   - Domain-specific translation (legal, medical, technical)
   - Format conversion (JSON to YAML, CSV to markdown)
   - Data normalization (various formats to standard)
   - Code refactoring patterns

LangChain Implementation: LangChain provides excellent support for few-shot learning through: – FewShotChatMessagePromptTemplate for chat models – FewShotPromptTemplate for completion models – Example selectors (semantic similarity, length-based, max marginal relevance) – Vector store-backed example selection – Custom example formatting

Key Features: 1. Static Few-Shot Learning – Predefined set of examples – Fast and deterministic – Good for stable tasks – No retrieval overhead

2. Dynamic Few-Shot Learning
   - Examples selected based on input similarity
   - More adaptive to diverse inputs
   - Requires vector database
   - Better generalization

3. Example Quality Management
   - Diverse examples covering edge cases
   - Clear and unambiguous examples
   - Representative of target distribution
   - Regularly updated based on performance

4. Performance Optimization
   - Token budget management (limiting example count)
   - Example compression (removing redundancy)
   - Caching frequently used example sets
   - Lazy loading for large example sets

Best Practices: 1. Example Selection – Use 3-5 examples for most tasks (sweet spot) – Include diverse examples covering different patterns – Order examples from simple to complex – Ensure examples are correct and high-quality

2. Example Formatting
   - Use consistent structure across all examples
   - Clearly separate input from output
   - Add brief explanations for complex cases
   - Use delimiters to avoid confusion

3. Dynamic Selection
   - Use semantic similarity for input-dependent selection
   - Balance diversity and relevance in selection
   - Cache embeddings for better performance
   - Set appropriate k value (number of examples)

4. Iteration and Improvement
   - Start with few examples, add more if needed
   - A/B test different example sets
   - Monitor performance across different inputs
   - Update examples based on failure cases

Trade-offs: Advantages: – Significantly improves task performance vs zero-shot – More reliable than instructions alone – Enables consistent output formatting – Works well for complex or ambiguous tasks – Reduces need for fine-tuning

Disadvantages:
- Increases token usage (more expensive)
- May not scale to tasks requiring many examples
- Examples can bias the model inappropriately
- Requires careful example curation
- Dynamic selection adds latency and complexity

Production Considerations: 1. Token Budget Management – Monitor tokens used by examples – Balance example count with context window – Use example compression when possible – Consider model’s context limit

2. Example Maintenance
   - Version control for example sets
   - Regular review and updates
   - Performance tracking per example
   - A/B testing of example variations

3. Scalability
   - Cache example embeddings
   - Use efficient vector stores
   - Batch process when possible
   - Consider example pruning strategies

4. Quality Assurance
   - Validate examples are correct
   - Test examples independently
   - Monitor output quality metrics
   - Handle example selection failures gracefully

5. Cost Optimization
   - Balance example count with quality
   - Use dynamic selection judiciously
   - Cache common example sets
   - Monitor cost per query

---

File Statistics

• Total Lines: 768
• Classes: 4
• Functions: 1
• Constants: 0

---

Module Documentation

"""
Pattern 083: Few-Shot Learning Pattern

Description:
    The Few-Shot Learning pattern uses example-based prompting to teach the LLM
    how to perform tasks through demonstrations. Instead of relying on zero-shot
    capabilities, this pattern provides the model with a few examples of input-output
    pairs that illustrate the desired behavior. This approach leverages the LLM's
    in-context learning abilities to generalize from examples.

    Few-shot learning is one of the most fundamental prompt engineering techniques
    and is particularly effective when:
    - The task format is non-standard or complex
    - Consistent output format is required
    - Specific style or tone needs to be maintained
    - Domain-specific knowledge needs to be demonstrated
    - Edge cases need to be handled in specific ways

Components:
    1. Example Selection
       - Static examples (fixed set)
       - Dynamic examples (selected based on similarity)
       - Diverse examples (covering different scenarios)
       - Representative examples (typical cases)

    2. Example Formatting
       - Clear input-output structure
       - Consistent formatting across examples
       - Annotations and explanations (when needed)
       - Order considerations (easy to hard, chronological)

    3. Example Storage
       - In-prompt storage (small example sets)
       - Vector database storage (large example sets)
       - Retrieval mechanisms (similarity-based)
       - Example caching strategies

    4. Generalization Strategy
       - Pattern recognition from examples
       - Analogical reasoning
       - Template extraction
       - Feature learning

Use Cases:
    1. Text Classification
       - Sentiment analysis with custom categories
       - Intent recognition for domain-specific applications
       - Content moderation with nuanced rules
       - Topic categorization with specific taxonomies

    2. Data Extraction
       - Structured information extraction from unstructured text
       - Entity recognition with custom entity types
       - Relationship extraction between entities
       - Attribute extraction with specific formats

    3. Code Generation
       - API usage examples for specific libraries
       - Code style consistency (following project conventions)
       - Design pattern implementation
       - Test case generation

    4. Creative Writing
       - Style mimicry (writing in specific author's style)
       - Format consistency (poems, stories, articles)
       - Tone control (formal, casual, humorous)
       - Genre-specific writing (sci-fi, mystery, romance)

    5. Translation and Transformation
       - Domain-specific translation (legal, medical, technical)
       - Format conversion (JSON to YAML, CSV to markdown)
       - Data normalization (various formats to standard)
       - Code refactoring patterns

LangChain Implementation:
    LangChain provides excellent support for few-shot learning through:
    - FewShotChatMessagePromptTemplate for chat models
    - FewShotPromptTemplate for completion models
    - Example selectors (semantic similarity, length-based, max marginal relevance)
    - Vector store-backed example selection
    - Custom example formatting

Key Features:
    1. Static Few-Shot Learning
       - Predefined set of examples
       - Fast and deterministic
       - Good for stable tasks
       - No retrieval overhead

    2. Dynamic Few-Shot Learning
       - Examples selected based on input similarity
       - More adaptive to diverse inputs
       - Requires vector database
       - Better generalization

    3. Example Quality Management
       - Diverse examples covering edge cases
       - Clear and unambiguous examples
       - Representative of target distribution
       - Regularly updated based on performance

    4. Performance Optimization
       - Token budget management (limiting example count)
       - Example compression (removing redundancy)
       - Caching frequently used example sets
       - Lazy loading for large example sets

Best Practices:
    1. Example Selection
       - Use 3-5 examples for most tasks (sweet spot)
       - Include diverse examples covering different patterns
       - Order examples from simple to complex
       - Ensure examples are correct and high-quality

    2. Example Formatting
       - Use consistent structure across all examples
       - Clearly separate input from output
       - Add brief explanations for complex cases
       - Use delimiters to avoid confusion

    3. Dynamic Selection
       - Use semantic similarity for input-dependent selection
       - Balance diversity and relevance in selection
       - Cache embeddings for better performance
       - Set appropriate k value (number of examples)

    4. Iteration and Improvement
       - Start with few examples, add more if needed
       - A/B test different example sets
       - Monitor performance across different inputs
       - Update examples based on failure cases

Trade-offs:
    Advantages:
    - Significantly improves task performance vs zero-shot
    - More reliable than instructions alone
    - Enables consistent output formatting
    - Works well for complex or ambiguous tasks
    - Reduces need for fine-tuning

    Disadvantages:
    - Increases token usage (more expensive)
    - May not scale to tasks requiring many examples
    - Examples can bias the model inappropriately
    - Requires careful example curation
    - Dynamic selection adds latency and complexity

Production Considerations:
    1. Token Budget Management
       - Monitor tokens used by examples
       - Balance example count with context window
       - Use example compression when possible
       - Consider model's context limit

    2. Example Maintenance
       - Version control for example sets
       - Regular review and updates
       - Performance tracking per example
       - A/B testing of example variations

    3. Scalability
       - Cache example embeddings
       - Use efficient vector stores
       - Batch process when possible
       - Consider example pruning strategies

    4. Quality Assurance
       - Validate examples are correct
       - Test examples independently
       - Monitor output quality metrics
       - Handle example selection failures gracefully

    5. Cost Optimization
       - Balance example count with quality
       - Use dynamic selection judiciously
       - Cache common example sets
       - Monitor cost per query
"""

Purpose: This module implements the Few Shot Learning pattern, providing a structured approach to agent design and implementation.

---

Import Statements

Standard Library Imports

import dataclasses.dataclass
import enum.Enum
import langchain_core.example_selectors.MaxMarginalRelevanceExampleSelector
import langchain_core.example_selectors.SemanticSimilarityExampleSelector
import langchain_core.output_parsers.StrOutputParser
import langchain_core.prompts.ChatPromptTemplate
import langchain_core.prompts.FewShotChatMessagePromptTemplate
import langchain_core.prompts.PromptTemplate
import os
import typing.Any
import typing.Dict
import typing.List
import typing.Optional

Purpose: Core Python functionality: – dataclasses.dataclass: Simplified class definitions with @dataclass – enum.Enum: Enumeration support – langchain_core.example_selectors.MaxMarginalRelevanceExampleSelector: Regular expression operations – langchain_core.example_selectors.SemanticSimilarityExampleSelector: Regular expression operations – langchain_core.output_parsers.StrOutputParser: Regular expression operations – langchain_core.prompts.ChatPromptTemplate: Regular expression operations – langchain_core.prompts.FewShotChatMessagePromptTemplate: Regular expression operations – langchain_core.prompts.PromptTemplate: Regular expression operations – os: Environment variable access and system operations – typing.Any: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Dict: Type hints for better code documentation (List, Dict, Any, Optional) – typing.List: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Optional: Type hints for better code documentation (List, Dict, Any, Optional)

LangChain Imports

from langchain_chroma.Chroma
from langchain_openai.ChatOpenAI
from langchain_openai.OpenAIEmbeddings

Purpose: LangChain framework components: – LLM Interface: Wrapper for OpenAI models – Chains: Composable building blocks for complex workflows

Other Imports

from dotenv.load_dotenv

Purpose: Additional dependencies for specialized functionality.

---

Classes

Class: ExampleSelectionStrategy

Lines 202-206

Inherits from: Enum

Purpose:

Strategy for selecting examples

---

Class: FewShotExample

Lines 210-215

Purpose:

Represents a single few-shot example

---

Class: FewShotConfig

Lines 219-225

Purpose:

Configuration for few-shot learning

---

Class: FewShotLearningAgent

Lines 228-625

Purpose:

Agent that uses few-shot learning to teach LLM through examples.

This agent demonstrates various few-shot learning approaches:
1. Static few-shot (fixed examples)
2. Dynamic few-shot (similarity-based selection)
3. Task-specific few-shot applications

Methods:

__init__(self, config)

Lines 238-250

Purpose:

Initialize few-shot learning agent.

Args:
    config: Configuration for few-shot learning

Parameters: – config: Config

create_static_few_shot_chain(self, examples, task_instruction, input_variable)

Lines 252-297

Purpose:

Create a chain with static few-shot examples.

Args:
    examples: List of few-shot examples
    task_instruction: Instruction for the task
    input_variable: Name of input variable

Returns:
    Runnable chain with few-shot prompting

Parameters: – examples: Examples – task_instruction: Task instruction – input_variable: Input data

create_dynamic_few_shot_chain(self, examples, task_instruction, input_variable, k)

Lines 299-356

Purpose:

Create a chain with dynamic example selection based on similarity.

Args:
    examples: Pool of examples to select from
    task_instruction: Instruction for the task
    input_variable: Name of input variable
    k: Number of examples to select

Returns:
    Runnable chain with dynamic few-shot prompting

Parameters: – examples: Examples – task_instruction: Task instruction – input_variable: Input data – k: K

sentiment_analysis_few_shot(self, text, examples, use_dynamic)

Lines 358-418

Purpose:

Perform sentiment analysis using few-shot learning.

Args:
    text: Text to analyze
    examples: Optional custom examples
    use_dynamic: Whether to use dynamic selection

Returns:
    Sentiment classification

Parameters: – text: Text – examples: Examples – use_dynamic: Use dynamic

Returns: str

entity_extraction_few_shot(self, text, examples)

Lines 420-462

Purpose:

Extract entities using few-shot learning.

Args:
    text: Text to extract entities from
    examples: Optional custom examples

Returns:
    Extracted entities in JSON format

Parameters: – text: Text – examples: Examples

Returns: str

code_generation_few_shot(self, description, examples)

Lines 464-515

Purpose:

Generate code using few-shot learning.

Args:
    description: Description of desired code
    examples: Optional custom examples

Returns:
    Generated code

Parameters: – description: Description – examples: Examples

Returns: str

style_transfer_few_shot(self, text, target_style, examples)

Lines 517-594

Purpose:

Transfer text to a specific style using few-shot learning.

Args:
    text: Text to transform
    target_style: Target style (e.g., "formal", "casual", "poetic")
    examples: Optional custom examples

Returns:
    Transformed text

Parameters: – text: Text – target_style: Target style – examples: Examples

Returns: str

compare_zero_shot_vs_few_shot(self, text, task)

Lines 596-625

Purpose:

Compare zero-shot vs few-shot performance.

Args:
    text: Input text
    task: Task to perform

Returns:
    Dictionary with both results

Parameters: – text: Text – task: Task

Returns: <ast.Subscript object at 0x000001A06D853AD0>

---

Functions

demonstrate_few_shot_learning()

Lines 628-763

Purpose:

Demonstrate few-shot learning patterns

---

Key Concepts Summary

Pattern Overview

The Few Shot Learning pattern provides a structured approach to building AI agents with specific capabilities and behaviors. This implementation leverages LangChain’s composable components to create a robust and maintainable solution.

Core Components

1. Agent Class: Main implementation of the pattern logic
2. Prompt Engineering: Carefully crafted prompts to guide LLM behavior
3. Tool Integration: Optional tools for extended functionality
4. Memory Management: State handling across interactions
5. Output Parsing: Structured response processing

When to Use

✅ Good for: – Scenarios requiring this specific pattern’s strengths – Applications needing structured agent behavior – Systems where explainability is important – Production environments requiring reliability

❌ Not ideal for: – Simple single-prompt tasks – Scenarios requiring different pattern characteristics – Resource-constrained environments (if computationally intensive)

---

Best Practices Demonstrated

1. Environment Configuration

• ✅ Use .env files for API keys
• ✅ Never hardcode credentials
• ✅ Validate environment variables on startup

2. Error Handling

• ✅ Graceful degradation on failures
• ✅ Informative error messages
• ✅ Logging for debugging

3. Prompt Engineering

• ✅ Clear, specific instructions
• ✅ Structured output formats
• ✅ Few-shot examples when beneficial

4. Code Organization

• ✅ Separation of concerns (class-based design)
• ✅ Type hints for clarity
• ✅ Docstrings for documentation

5. Performance Optimization

• ✅ Appropriate temperature settings
• ✅ Token budget management
• ✅ Caching where applicable

---

LangChain Components Used

• ChatOpenAI: OpenAI model interface
• Chains: Sequential component composition

Why LangChain? – Composable abstractions for LLM applications – Rich ecosystem of integrations – Production-ready patterns – Active community and development

---

Execution Flow

1. Initialize agent with configuration
   ↓
2. Load environment and dependencies
   ↓
3. Prepare prompt with input data
   ↓
4. Invoke LLM through LangChain
   ↓
5. Parse and validate response
   ↓
6. Return formatted result

Key Decision Points: – Model selection (temperature, model type) – Prompt structure (few-shot vs zero-shot) – Error handling strategy – Output format validation

---

Performance Considerations

Computational Cost

• LLM Calls: Primary cost factor (based on tokens)
• Latency: Depends on model and input size
• Memory: Minimal for most patterns

Optimization Strategies

1. Prompt Optimization: Minimize tokens while maintaining clarity
2. Caching: Cache repeated queries/responses
3. Model Selection: Use smaller models when appropriate
4. Batch Processing: Group similar requests
5. Streaming: For long responses, stream tokens

Cost Estimates (Approximate)

• GPT-3.5-turbo: ~$0.002 per request
• GPT-4: ~$0.03 per request
• GPT-4-turbo: ~$0.01 per request

Actual costs vary based on input/output token counts

---

Common Pitfalls to Avoid

1. Prompt Design Issues

❌ Vague or ambiguous instructions
✅ Clear, specific, testable prompts

2. Error Handling

❌ Ignoring API errors or rate limits
✅ Implement retry logic and graceful degradation

3. Token Management

❌ Exceeding model token limits
✅ Truncate inputs, chunk processing

4. Security

❌ Exposing API keys in code
✅ Use environment variables

5. Testing

❌ Only testing happy paths
✅ Test edge cases, failures, malformed inputs

6. Cost Management

❌ Unmonitored API usage
✅ Implement budgets, alerts, and monitoring

---

Further Improvements

Production Enhancements

1. Monitoring: Add logging, metrics, tracing
2. Validation: Input/output schema validation
3. Rate Limiting: Implement request throttling
4. Caching: Redis or in-memory cache for responses
5. Async Support: Non-blocking operations for scalability

Feature Extensions

1. Multi-Model Support: Switch between providers
2. Fallback Models: Graceful degradation strategy
3. Custom Tools: Domain-specific function calling
4. Advanced Memory: Vector stores for long-term memory
5. Evaluation: Automated testing and benchmarking

Code Quality

1. Testing: Unit tests, integration tests
2. Documentation: API docs, usage examples
3. Type Safety: Stricter type hints
4. Configuration: Externalized config files

---

Related Patterns

• Pattern 082
• Pattern 084

Pattern Families: – Reasoning Patterns (001-011) – Multi-Agent Patterns (012-019) – Tool Use Patterns (020-025) – Memory Patterns (026-031) – And many more…

---

Summary

The Few Shot Learning pattern provides a powerful approach to building AI agents with specific capabilities. This implementation demonstrates:

✅ Key Strengths: – Clean, maintainable code structure – LangChain integration for composability – Type hints and documentation – Production-ready patterns

🎯 Best Use Cases: – Applications requiring this pattern’s characteristics – Systems needing structured agent behavior – Production environments requiring reliability

📚 Learning Resources: – LangChain Documentation: https://python.langchain.com/ – Pattern Documentation: See related patterns – OpenAI API Reference: https://platform.openai.com/docs/

---

File: 083_few_shot_learning.py
Total Lines: 768
Pattern Category: AI Agent Design Pattern #083

---

Generated from actual implementation code
For educational and reference purposes

    """
Pattern 082: Serverless Agent

Description:
    A Serverless Agent runs on serverless computing platforms (AWS Lambda,
    Azure Functions, Google Cloud Functions) where infrastructure is fully
    managed by the cloud provider. This pattern enables event-driven, auto-scaling
    AI agents that only consume resources when invoked, optimizing costs and
    operational overhead.
    
    The agent is designed for stateless execution, cold start optimization,
    event-driven triggers, and seamless scaling from zero to thousands of
    concurrent executions without manual intervention.

Components:
    1. Function Handler: Entry point for serverless execution
    2. Event Processor: Processes various event types
    3. State Manager: External state storage (S3, DynamoDB)
    4. Cold Start Optimizer: Minimizes initialization time
    5. Environment Manager: Configuration and secrets
    6. Response Builder: Formats function responses
    7. Error Handler: Graceful error handling
    8. Metrics Collector: CloudWatch/monitoring integration

Key Features:
    - Stateless execution model
    - Event-driven triggers
    - Auto-scaling from zero
    - Pay-per-execution pricing
    - No server management
    - Built-in fault tolerance
    - Managed infrastructure
    - Cold start optimization
    - Multiple trigger sources
    - Seamless integration with cloud services

Use Cases:
    - API endpoints for AI services
    - Event-driven AI processing
    - Webhook handlers
    - Scheduled AI tasks
    - Image/document processing
    - Real-time data transformation
    - Chatbot backends
    - ETL pipelines
    - IoT data processing
    - Microservices backends

LangChain Implementation:
    Simulates AWS Lambda-style function handlers with event processing,
    stateless execution, and external state management patterns.
"""

import os
import time
import json
import uuid
from typing import List, Dict, Any, Optional, Callable
from dataclasses import dataclass, field
from enum import Enum
from datetime import datetime
from collections import defaultdict
from dotenv import load_dotenv

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_openai import ChatOpenAI

load_dotenv()


class TriggerType(Enum):
    """Types of serverless triggers"""
    HTTP_API = "http_api"
    S3_EVENT = "s3_event"
    DYNAMODB_STREAM = "dynamodb_stream"
    SNS_MESSAGE = "sns_message"
    SQS_QUEUE = "sqs_queue"
    SCHEDULED = "scheduled"
    WEBHOOK = "webhook"
    IOT_EVENT = "iot_event"


class ExecutionStatus(Enum):
    """Status of function execution"""
    COLD_START = "cold_start"
    WARM_START = "warm_start"
    RUNNING = "running"
    COMPLETED = "completed"
    FAILED = "failed"
    TIMEOUT = "timeout"


@dataclass
class ServerlessEvent:
    """Event that triggers serverless function"""
    event_id: str
    trigger_type: TriggerType
    timestamp: datetime
    data: Dict[str, Any]
    headers: Dict[str, str] = field(default_factory=dict)
    metadata: Dict[str, Any] = field(default_factory=dict)


@dataclass
class ServerlessContext:
    """Execution context for serverless function"""
    request_id: str
    function_name: str
    function_version: str
    memory_limit_mb: int
    timeout_seconds: int
    remaining_time_ms: int
    cold_start: bool = True
    invocation_count: int = 0


@dataclass
class ServerlessResponse:
    """Response from serverless function"""
    status_code: int
    body: Any
    headers: Dict[str, str] = field(default_factory=dict)
    error: Optional[str] = None
    execution_duration_ms: float = 0.0
    cold_start: bool = False
    memory_used_mb: float = 0.0


@dataclass
class FunctionMetrics:
    """Metrics for serverless function"""
    function_name: str
    invocations: int = 0
    errors: int = 0
    cold_starts: int = 0
    warm_starts: int = 0
    total_duration_ms: float = 0.0
    average_duration_ms: float = 0.0
    max_duration_ms: float = 0.0
    min_duration_ms: float = float('inf')


class ServerlessAgent:
    """
    Base serverless agent with stateless execution.
    
    Designed for serverless platforms with event-driven triggers,
    cold start optimization, and external state management.
    """
    
    # Class-level cache (simulates container reuse)
    _warm_instances: Dict[str, 'ServerlessAgent'] = {}
    _llm_cache: Optional[ChatOpenAI] = None
    
    def __init__(
        self,
        function_name: str,
        memory_limit_mb: int = 512,
        timeout_seconds: int = 30
    ):
        """
        Initialize serverless agent.
        
        Args:
            function_name: Name of the function
            memory_limit_mb: Memory limit
            timeout_seconds: Execution timeout
        """
        self.function_name = function_name
        self.memory_limit_mb = memory_limit_mb
        self.timeout_seconds = timeout_seconds
        
        self.invocation_count = 0
        self.metrics = FunctionMetrics(function_name=function_name)
        
        # Initialize LLM (reuse cached if available)
        if ServerlessAgent._llm_cache is None:
            ServerlessAgent._llm_cache = ChatOpenAI(
                model="gpt-3.5-turbo",
                temperature=0.7
            )
        
        self.llm = ServerlessAgent._llm_cache
    
    @classmethod
    def get_or_create(cls, function_name: str) -> 'ServerlessAgent':
        """
        Get warm instance or create new (simulates Lambda container reuse).
        
        Args:
            function_name: Function name
            
        Returns:
            ServerlessAgent instance
        """
        if function_name in cls._warm_instances:
            return cls._warm_instances[function_name]
        
        instance = cls(function_name)
        cls._warm_instances[function_name] = instance
        return instance
    
    def handler(
        self,
        event: ServerlessEvent,
        context: ServerlessContext
    ) -> ServerlessResponse:
        """
        Main handler function (entry point).
        
        Args:
            event: Triggering event
            context: Execution context
            
        Returns:
            ServerlessResponse
        """
        start_time = time.time()
        
        self.invocation_count += 1
        context.invocation_count = self.invocation_count
        
        # Track metrics
        self.metrics.invocations += 1
        if context.cold_start:
            self.metrics.cold_starts += 1
        else:
            self.metrics.warm_starts += 1
        
        try:
            # Process event based on trigger type
            result = self._process_event(event, context)
            
            execution_duration = (time.time() - start_time) * 1000
            
            # Update metrics
            self.metrics.total_duration_ms += execution_duration
            self.metrics.average_duration_ms = (
                self.metrics.total_duration_ms / self.metrics.invocations
            )
            self.metrics.max_duration_ms = max(
                self.metrics.max_duration_ms,
                execution_duration
            )
            self.metrics.min_duration_ms = min(
                self.metrics.min_duration_ms,
                execution_duration
            )
            
            return ServerlessResponse(
                status_code=200,
                body=result,
                headers={"Content-Type": "application/json"},
                execution_duration_ms=execution_duration,
                cold_start=context.cold_start,
                memory_used_mb=self.memory_limit_mb * 0.6  # Simulated
            )
            
        except Exception as e:
            self.metrics.errors += 1
            
            return ServerlessResponse(
                status_code=500,
                body=None,
                error=str(e),
                execution_duration_ms=(time.time() - start_time) * 1000,
                cold_start=context.cold_start
            )
    
    def _process_event(
        self,
        event: ServerlessEvent,
        context: ServerlessContext
    ) -> Dict[str, Any]:
        """
        Process event (to be overridden by subclasses).
        
        Args:
            event: Event to process
            context: Execution context
            
        Returns:
            Processing result
        """
        return {
            "message": "Event processed",
            "event_id": event.event_id,
            "function": self.function_name
        }


class TextProcessingFunction(ServerlessAgent):
    """Serverless function for text processing"""
    
    def __init__(self):
        super().__init__("text-processor", memory_limit_mb=512, timeout_seconds=30)
    
    def _process_event(
        self,
        event: ServerlessEvent,
        context: ServerlessContext
    ) -> Dict[str, Any]:
        """Process text from event"""
        data = event.data
        operation = data.get('operation', 'summarize')
        text = data.get('text', '')
        
        if operation == 'summarize':
            result = self._summarize(text)
        elif operation == 'sentiment':
            result = self._analyze_sentiment(text)
        elif operation == 'extract':
            result = self._extract_keywords(text)
        else:
            result = {'error': f'Unknown operation: {operation}'}
        
        return {
            'operation': operation,
            'result': result,
            'cold_start': context.cold_start,
            'invocation': context.invocation_count
        }
    
    def _summarize(self, text: str) -> str:
        """Summarize text"""
        prompt = ChatPromptTemplate.from_template(
            "Summarize this text in one sentence: {text}"
        )
        chain = prompt | self.llm | StrOutputParser()
        return chain.invoke({'text': text[:500]})
    
    def _analyze_sentiment(self, text: str) -> str:
        """Analyze sentiment"""
        prompt = ChatPromptTemplate.from_template(
            "What is the sentiment of this text? Answer in one word (positive/negative/neutral): {text}"
        )
        chain = prompt | self.llm | StrOutputParser()
        return chain.invoke({'text': text[:500]})
    
    def _extract_keywords(self, text: str) -> List[str]:
        """Extract keywords"""
        prompt = ChatPromptTemplate.from_template(
            "Extract 5 key words from this text (comma-separated): {text}"
        )
        chain = prompt | self.llm | StrOutputParser()
        result = chain.invoke({'text': text[:500]})
        return [k.strip() for k in result.split(',')[:5]]


class WebhookFunction(ServerlessAgent):
    """Serverless function for webhook processing"""
    
    def __init__(self):
        super().__init__("webhook-handler", memory_limit_mb=256, timeout_seconds=15)
    
    def _process_event(
        self,
        event: ServerlessEvent,
        context: ServerlessContext
    ) -> Dict[str, Any]:
        """Process webhook event"""
        data = event.data
        webhook_type = data.get('type', 'generic')
        payload = data.get('payload', {})
        
        # Process webhook
        response = {
            'webhook_type': webhook_type,
            'processed_at': datetime.now().isoformat(),
            'status': 'received',
            'payload_size': len(str(payload))
        }
        
        return response


class ScheduledFunction(ServerlessAgent):
    """Serverless function for scheduled tasks"""
    
    def __init__(self):
        super().__init__("scheduled-task", memory_limit_mb=1024, timeout_seconds=60)
    
    def _process_event(
        self,
        event: ServerlessEvent,
        context: ServerlessContext
    ) -> Dict[str, Any]:
        """Process scheduled event"""
        task_type = event.data.get('task', 'report')
        
        if task_type == 'report':
            result = self._generate_report()
        elif task_type == 'cleanup':
            result = self._cleanup_data()
        elif task_type == 'sync':
            result = self._sync_data()
        else:
            result = {'error': 'Unknown task type'}
        
        return {
            'task': task_type,
            'result': result,
            'scheduled_time': event.timestamp.isoformat()
        }
    
    def _generate_report(self) -> Dict[str, Any]:
        """Generate report"""
        return {
            'report_type': 'daily_summary',
            'generated_at': datetime.now().isoformat(),
            'items_processed': 100
        }
    
    def _cleanup_data(self) -> Dict[str, Any]:
        """Cleanup old data"""
        return {
            'items_deleted': 50,
            'space_freed_mb': 250
        }
    
    def _sync_data(self) -> Dict[str, Any]:
        """Sync data"""
        return {
            'items_synced': 75,
            'sync_duration_ms': 1500
        }


class ServerlessPlatform:
    """
    Simulates serverless platform (like AWS Lambda).
    
    Manages function lifecycle, invocations, and cold/warm starts.
    """
    
    def __init__(self):
        """Initialize platform"""
        self.functions: Dict[str, ServerlessAgent] = {}
        self.function_metrics: Dict[str, FunctionMetrics] = {}
    
    def deploy_function(self, function: ServerlessAgent) -> bool:
        """
        Deploy a serverless function.
        
        Args:
            function: Function to deploy
            
        Returns:
            True if deployed successfully
        """
        self.functions[function.function_name] = function
        self.function_metrics[function.function_name] = function.metrics
        return True
    
    def invoke(
        self,
        function_name: str,
        event: ServerlessEvent
    ) -> ServerlessResponse:
        """
        Invoke a serverless function.
        
        Args:
            function_name: Name of function to invoke
            event: Event to process
            
        Returns:
            ServerlessResponse
        """
        if function_name not in self.functions:
            return ServerlessResponse(
                status_code=404,
                body=None,
                error=f"Function not found: {function_name}"
            )
        
        function = self.functions[function_name]
        
        # Determine if cold start (simplified)
        is_cold_start = function.invocation_count == 0
        
        # Create context
        context = ServerlessContext(
            request_id=f"req_{uuid.uuid4().hex[:8]}",
            function_name=function_name,
            function_version="1.0",
            memory_limit_mb=function.memory_limit_mb,
            timeout_seconds=function.timeout_seconds,
            remaining_time_ms=function.timeout_seconds * 1000,
            cold_start=is_cold_start
        )
        
        # Invoke function
        return function.handler(event, context)
    
    def get_metrics(self, function_name: str) -> Optional[FunctionMetrics]:
        """
        Get function metrics.
        
        Args:
            function_name: Function name
            
        Returns:
            FunctionMetrics if found
        """
        return self.function_metrics.get(function_name)


def demonstrate_serverless_agent():
    """Demonstrate the serverless agent pattern"""
    print("=" * 80)
    print("SERVERLESS AGENT DEMONSTRATION")
    print("=" * 80)
    
    platform = ServerlessPlatform()
    
    # Demo 1: Deploy Serverless Functions
    print("\n" + "=" * 80)
    print("DEMO 1: Deploying Serverless Functions")
    print("=" * 80)
    
    functions = [
        TextProcessingFunction(),
        WebhookFunction(),
        ScheduledFunction()
    ]
    
    print("\nDeploying functions to serverless platform...")
    for func in functions:
        platform.deploy_function(func)
        print(f"  ✓ Deployed: {func.function_name}")
        print(f"    Memory: {func.memory_limit_mb}MB")
        print(f"    Timeout: {func.timeout_seconds}s")
    
    # Demo 2: Cold Start vs Warm Start
    print("\n" + "=" * 80)
    print("DEMO 2: Cold Start vs Warm Start")
    print("=" * 80)
    
    event = ServerlessEvent(
        event_id="evt_001",
        trigger_type=TriggerType.HTTP_API,
        timestamp=datetime.now(),
        data={
            'operation': 'summarize',
            'text': 'Serverless computing allows running code without managing servers.'
        }
    )
    
    print("\nFirst invocation (Cold Start):")
    response1 = platform.invoke("text-processor", event)
    print(f"  Status: {response1.status_code}")
    print(f"  Cold Start: {response1.cold_start}")
    print(f"  Duration: {response1.execution_duration_ms:.2f}ms")
    print(f"  Memory Used: {response1.memory_used_mb:.1f}MB")
    
    print("\nSecond invocation (Warm Start):")
    response2 = platform.invoke("text-processor", event)
    print(f"  Status: {response2.status_code}")
    print(f"  Cold Start: {response2.cold_start}")
    print(f"  Duration: {response2.execution_duration_ms:.2f}ms")
    print(f"  Memory Used: {response2.memory_used_mb:.1f}MB")
    
    speedup = response1.execution_duration_ms / response2.execution_duration_ms if response2.execution_duration_ms > 0 else 1
    print(f"\n  Warm start {speedup:.2f}x faster than cold start")
    
    # Demo 3: Multiple Operations
    print("\n" + "=" * 80)
    print("DEMO 3: Multiple Text Processing Operations")
    print("=" * 80)
    
    operations = [
        ('summarize', 'AI agents are autonomous systems that can reason and act.'),
        ('sentiment', 'This serverless architecture is amazing!'),
        ('extract', 'Machine learning models process data to make predictions.')
    ]
    
    print("\nProcessing multiple operations...")
    print("-" * 80)
    
    for operation, text in operations:
        event = ServerlessEvent(
            event_id=f"evt_{uuid.uuid4().hex[:8]}",
            trigger_type=TriggerType.HTTP_API,
            timestamp=datetime.now(),
            data={'operation': operation, 'text': text}
        )
        
        response = platform.invoke("text-processor", event)
        
        print(f"\nOperation: {operation}")
        print(f"  Input: {text[:50]}...")
        print(f"  Duration: {response.execution_duration_ms:.2f}ms")
        
        if response.body and 'result' in response.body:
            result = response.body['result']
            if isinstance(result, str):
                print(f"  Result: {result[:80]}...")
            else:
                print(f"  Result: {result}")
    
    # Demo 4: Webhook Handler
    print("\n" + "=" * 80)
    print("DEMO 4: Webhook Event Processing")
    print("=" * 80)
    
    webhook_event = ServerlessEvent(
        event_id="webhook_001",
        trigger_type=TriggerType.WEBHOOK,
        timestamp=datetime.now(),
        data={
            'type': 'user_signup',
            'payload': {
                'user_id': '12345',
                'email': 'user@example.com',
                'timestamp': datetime.now().isoformat()
            }
        }
    )
    
    print("\nProcessing webhook event...")
    response = platform.invoke("webhook-handler", webhook_event)
    
    print(f"  Status: {response.status_code}")
    print(f"  Duration: {response.execution_duration_ms:.2f}ms")
    print(f"  Response:")
    for key, value in response.body.items():
        print(f"    {key}: {value}")
    
    # Demo 5: Scheduled Task
    print("\n" + "=" * 80)
    print("DEMO 5: Scheduled Task Execution")
    print("=" * 80)
    
    scheduled_events = [
        {'task': 'report'},
        {'task': 'cleanup'},
        {'task': 'sync'}
    ]
    
    print("\nExecuting scheduled tasks...")
    print("-" * 80)
    
    for task_data in scheduled_events:
        event = ServerlessEvent(
            event_id=f"sched_{uuid.uuid4().hex[:8]}",
            trigger_type=TriggerType.SCHEDULED,
            timestamp=datetime.now(),
            data=task_data
        )
        
        response = platform.invoke("scheduled-task", event)
        
        print(f"\nTask: {task_data['task']}")
        print(f"  Duration: {response.execution_duration_ms:.2f}ms")
        print(f"  Result: {response.body.get('result', {})}")
    
    # Demo 6: Function Metrics
    print("\n" + "=" * 80)
    print("DEMO 6: Function Metrics & Monitoring")
    print("=" * 80)
    
    print("\nFunction Metrics:")
    print("-" * 80)
    
    for func_name in ["text-processor", "webhook-handler", "scheduled-task"]:
        metrics = platform.get_metrics(func_name)
        
        if metrics:
            print(f"\n{func_name}:")
            print(f"  Total Invocations: {metrics.invocations}")
            print(f"  Cold Starts: {metrics.cold_starts}")
            print(f"  Warm Starts: {metrics.warm_starts}")
            print(f"  Errors: {metrics.errors}")
            print(f"  Avg Duration: {metrics.average_duration_ms:.2f}ms")
            print(f"  Min Duration: {metrics.min_duration_ms:.2f}ms")
            print(f"  Max Duration: {metrics.max_duration_ms:.2f}ms")
            
            cold_start_pct = (metrics.cold_starts / metrics.invocations * 100) if metrics.invocations > 0 else 0
            print(f"  Cold Start Rate: {cold_start_pct:.1f}%")
    
    # Summary
    print("\n" + "=" * 80)
    print("SUMMARY")
    print("=" * 80)
    
    summary = """
The Serverless Agent demonstrates cloud-native, event-driven AI:

KEY CAPABILITIES:
1. Stateless Execution: No persistent state between invocations
2. Event-Driven: Triggered by various event sources
3. Auto-Scaling: Scales automatically from zero to thousands
4. Cold/Warm Starts: Container reuse optimization
5. Pay-Per-Execution: Only pay for actual compute time
6. Managed Infrastructure: No server management required
7. Multiple Triggers: HTTP, webhooks, schedules, queues, streams
8. Built-in Monitoring: Integrated metrics and logging

BENEFITS:
- Zero infrastructure management
- Automatic scaling with demand
- Cost optimization (pay per use)
- High availability by default
- Fast deployment and iteration
- Built-in fault tolerance
- Easy integration with cloud services
- Reduced operational overhead

USE CASES:
- REST API endpoints for AI services
- Webhook processing for integrations
- Scheduled AI tasks (reports, cleanup, sync)
- Event-driven data processing
- Real-time stream processing
- Image/document processing pipelines
- Chatbot backends
- ETL and data transformation
- IoT data processing
- Microservices backends

PRODUCTION CONSIDERATIONS:
1. Cold Starts: Minimize with provisioned concurrency
2. Statelessness: Use external storage (S3, DynamoDB, RDS)
3. Timeout Limits: Functions have max execution time
4. Memory Allocation: Balance cost vs performance
5. Concurrency Limits: Understand and configure limits
6. VPC Networking: Consider cold start impact
7. Error Handling: Implement retries and DLQs
8. Monitoring: Use CloudWatch, X-Ray for observability
9. Security: IAM roles, secrets management
10. Cost Management: Monitor and optimize usage

ADVANCED EXTENSIONS:
- Lambda layers for shared dependencies
- Step Functions for complex workflows
- EventBridge for event routing
- API Gateway for HTTP endpoints
- SQS/SNS for async messaging
- Lambda@Edge for CDN integration
- Provisioned concurrency for predictable latency
- Container image deployment
- Multi-region deployment
- Custom runtimes and extensions

Serverless architecture is ideal for event-driven AI workloads
with variable demand, optimizing both cost and scalability.
"""
    
    print(summary)


if __name__ == "__main__":
    demonstrate_serverless_agent()

    

Agentic Design Pattern: Serverless Agent

Pattern 082: Serverless Agent – Line-by-Line Explanation

Overview

This document provides a detailed line-by-line explanation of the Serverless Agent pattern implementation.

---

Pattern Description

Pattern 082: Serverless Agent

Description: A Serverless Agent runs on serverless computing platforms (AWS Lambda, Azure Functions, Google Cloud Functions) where infrastructure is fully managed by the cloud provider. This pattern enables event-driven, auto-scaling AI agents that only consume resources when invoked, optimizing costs and operational overhead.

The agent is designed for stateless execution, cold start optimization,
event-driven triggers, and seamless scaling from zero to thousands of
concurrent executions without manual intervention.

Components: 1. Function Handler: Entry point for serverless execution 2. Event Processor: Processes various event types 3. State Manager: External state storage (S3, DynamoDB) 4. Cold Start Optimizer: Minimizes initialization time 5. Environment Manager: Configuration and secrets 6. Response Builder: Formats function responses 7. Error Handler: Graceful error handling 8. Metrics Collector: CloudWatch/monitoring integration

Key Features: – Stateless execution model – Event-driven triggers – Auto-scaling from zero – Pay-per-execution pricing – No server management – Built-in fault tolerance – Managed infrastructure – Cold start optimization – Multiple trigger sources – Seamless integration with cloud services

Use Cases: – API endpoints for AI services – Event-driven AI processing – Webhook handlers – Scheduled AI tasks – Image/document processing – Real-time data transformation – Chatbot backends – ETL pipelines – IoT data processing – Microservices backends

LangChain Implementation: Simulates AWS Lambda-style function handlers with event processing, stateless execution, and external state management patterns.

---

File Statistics

• Total Lines: 756
• Classes: 11
• Functions: 1
• Constants: 0

---

Module Documentation

"""
Pattern 082: Serverless Agent

Description:
    A Serverless Agent runs on serverless computing platforms (AWS Lambda,
    Azure Functions, Google Cloud Functions) where infrastructure is fully
    managed by the cloud provider. This pattern enables event-driven, auto-scaling
    AI agents that only consume resources when invoked, optimizing costs and
    operational overhead.

    The agent is designed for stateless execution, cold start optimization,
    event-driven triggers, and seamless scaling from zero to thousands of
    concurrent executions without manual intervention.

Components:
    1. Function Handler: Entry point for serverless execution
    2. Event Processor: Processes various event types
    3. State Manager: External state storage (S3, DynamoDB)
    4. Cold Start Optimizer: Minimizes initialization time
    5. Environment Manager: Configuration and secrets
    6. Response Builder: Formats function responses
    7. Error Handler: Graceful error handling
    8. Metrics Collector: CloudWatch/monitoring integration

Key Features:
    - Stateless execution model
    - Event-driven triggers
    - Auto-scaling from zero
    - Pay-per-execution pricing
    - No server management
    - Built-in fault tolerance
    - Managed infrastructure
    - Cold start optimization
    - Multiple trigger sources
    - Seamless integration with cloud services

Use Cases:
    - API endpoints for AI services
    - Event-driven AI processing
    - Webhook handlers
    - Scheduled AI tasks
    - Image/document processing
    - Real-time data transformation
    - Chatbot backends
    - ETL pipelines
    - IoT data processing
    - Microservices backends

LangChain Implementation:
    Simulates AWS Lambda-style function handlers with event processing,
    stateless execution, and external state management patterns.
"""

Purpose: This module implements the Serverless Agent pattern, providing a structured approach to agent design and implementation.

---

Import Statements

Standard Library Imports

import collections.defaultdict
import dataclasses.dataclass
import dataclasses.field
import datetime.datetime
import enum.Enum
import json
import langchain_core.output_parsers.StrOutputParser
import langchain_core.prompts.ChatPromptTemplate
import os
import time
import typing.Any
import typing.Callable
import typing.Dict
import typing.List
import typing.Optional

Purpose: Core Python functionality: – collections.defaultdict: Specialized container datatypes – dataclasses.dataclass: Simplified class definitions with @dataclass – dataclasses.field: Simplified class definitions with @dataclass – datetime.datetime: Date and time handling – enum.Enum: Enumeration support – json: JSON serialization/deserialization – langchain_core.output_parsers.StrOutputParser: Regular expression operations – langchain_core.prompts.ChatPromptTemplate: Regular expression operations – os: Environment variable access and system operations – time: Time-related functions – typing.Any: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Callable: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Dict: Type hints for better code documentation (List, Dict, Any, Optional) – typing.List: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Optional: Type hints for better code documentation (List, Dict, Any, Optional)

LangChain Imports

from langchain_openai.ChatOpenAI

Purpose: LangChain framework components: – LLM Interface: Wrapper for OpenAI models – Chains: Composable building blocks for complex workflows

Other Imports

from dotenv.load_dotenv
from uuid

Purpose: Additional dependencies for specialized functionality.

---

Classes

Class: TriggerType

Lines 72-81

Inherits from: Enum

Purpose:

Types of serverless triggers

---

Class: ExecutionStatus

Lines 84-91

Inherits from: Enum

Purpose:

Status of function execution

---

Class: ServerlessEvent

Lines 95-102

Purpose:

Event that triggers serverless function

---

Class: ServerlessContext

Lines 106-115

Purpose:

Execution context for serverless function

---

Class: ServerlessResponse

Lines 119-127

Purpose:

Response from serverless function

---

Class: FunctionMetrics

Lines 131-141

Purpose:

Metrics for serverless function

---

Class: ServerlessAgent

Lines 144-290

Purpose:

Base serverless agent with stateless execution.

Designed for serverless platforms with event-driven triggers,
cold start optimization, and external state management.

Methods:

__init__(self, function_name, memory_limit_mb, timeout_seconds)

Lines 156-184

Purpose:

Initialize serverless agent.

Args:
    function_name: Name of the function
    memory_limit_mb: Memory limit
    timeout_seconds: Execution timeout

Parameters: – function_name: Function name – memory_limit_mb: Memory store for conversation history – timeout_seconds: Timeout seconds

get_or_create(cls, function_name)

Lines 187-202

Decorators: classmethod

Purpose:

Get warm instance or create new (simulates Lambda container reuse).

Args:
    function_name: Function name

Returns:
    ServerlessAgent instance

Parameters: – cls: Cls – function_name: Function name

Returns: ServerlessAgent

handler(self, event, context)

Lines 204-269

Purpose:

Main handler function (entry point).

Args:
    event: Triggering event
    context: Execution context

Returns:
    ServerlessResponse

Parameters: – event: Event – context: Context

Returns: ServerlessResponse

_process_event(self, event, context)

Lines 271-290

Purpose:

Process event (to be overridden by subclasses).

Args:
    event: Event to process
    context: Execution context

Returns:
    Processing result

Parameters: – event: Event – context: Context

Returns: <ast.Subscript object at 0x000001A06D750A50>

---

Class: TextProcessingFunction

Lines 293-348

Inherits from: ServerlessAgent

Purpose:

Serverless function for text processing

Methods:

__init__(self)

Lines 296-297

Purpose: Initialize the class instance with required parameters.

Parameters:

_process_event(self, event, context)

Lines 299-323

Purpose:

Process text from event

Parameters: – event: Event – context: Context

Returns: <ast.Subscript object at 0x000001A06D17BC90>

_summarize(self, text)

Lines 325-331

Purpose:

Summarize text

Parameters: – text: Text

Returns: str

_analyze_sentiment(self, text)

Lines 333-339

Purpose:

Analyze sentiment

Parameters: – text: Text

Returns: str

_extract_keywords(self, text)

Lines 341-348

Purpose:

Extract keywords

Parameters: – text: Text

Returns: <ast.Subscript object at 0x000001A06D75B810>

---

Class: WebhookFunction

Lines 351-375

Inherits from: ServerlessAgent

Purpose:

Serverless function for webhook processing

Methods:

__init__(self)

Lines 354-355

Purpose: Initialize the class instance with required parameters.

Parameters:

_process_event(self, event, context)

Lines 357-375

Purpose:

Process webhook event

Parameters: – event: Event – context: Context

Returns: <ast.Subscript object at 0x000001A06D157B50>

---

Class: ScheduledFunction

Lines 378-427

Inherits from: ServerlessAgent

Purpose:

Serverless function for scheduled tasks

Methods:

__init__(self)

Lines 381-382

Purpose: Initialize the class instance with required parameters.

Parameters:

_process_event(self, event, context)

Lines 384-405

Purpose:

Process scheduled event

Parameters: – event: Event – context: Context

Returns: <ast.Subscript object at 0x000001A06D83EE50>

_generate_report(self)

Lines 407-413

Purpose:

Generate report

Parameters:

Returns: <ast.Subscript object at 0x000001A06D168490>

_cleanup_data(self)

Lines 415-420

Purpose:

Cleanup old data

Parameters:

Returns: <ast.Subscript object at 0x000001A06D626590>

_sync_data(self)

Lines 422-427

Purpose:

Sync data

Parameters:

Returns: <ast.Subscript object at 0x000001A06D627090>

---

Class: ServerlessPlatform

Lines 430-507

Purpose:

Simulates serverless platform (like AWS Lambda).

Manages function lifecycle, invocations, and cold/warm starts.

Methods:

__init__(self)

Lines 437-440

Purpose:

Initialize platform

Parameters:

deploy_function(self, function)

Lines 442-454

Purpose:

Deploy a serverless function.

Args:
    function: Function to deploy

Returns:
    True if deployed successfully

Parameters: – function: Function

Returns: bool

invoke(self, function_name, event)

Lines 456-495

Purpose:

Invoke a serverless function.

Args:
    function_name: Name of function to invoke
    event: Event to process

Returns:
    ServerlessResponse

Parameters: – function_name: Function name – event: Event

Returns: ServerlessResponse

get_metrics(self, function_name)

Lines 497-507

Purpose:

Get function metrics.

Args:
    function_name: Function name

Returns:
    FunctionMetrics if found

Parameters: – function_name: Function name

Returns: <ast.Subscript object at 0x000001A06D80C0D0>

---

Functions

demonstrate_serverless_agent()

Lines 510-751

Purpose:

Demonstrate the serverless agent pattern

---

Key Concepts Summary

Pattern Overview

The Serverless Agent pattern provides a structured approach to building AI agents with specific capabilities and behaviors. This implementation leverages LangChain’s composable components to create a robust and maintainable solution.

Core Components

1. Agent Class: Main implementation of the pattern logic
2. Prompt Engineering: Carefully crafted prompts to guide LLM behavior
3. Tool Integration: Optional tools for extended functionality
4. Memory Management: State handling across interactions
5. Output Parsing: Structured response processing

When to Use

✅ Good for: – Scenarios requiring this specific pattern’s strengths – Applications needing structured agent behavior – Systems where explainability is important – Production environments requiring reliability

❌ Not ideal for: – Simple single-prompt tasks – Scenarios requiring different pattern characteristics – Resource-constrained environments (if computationally intensive)

---

Best Practices Demonstrated

1. Environment Configuration

• ✅ Use .env files for API keys
• ✅ Never hardcode credentials
• ✅ Validate environment variables on startup

2. Error Handling

• ✅ Graceful degradation on failures
• ✅ Informative error messages
• ✅ Logging for debugging

3. Prompt Engineering

• ✅ Clear, specific instructions
• ✅ Structured output formats
• ✅ Few-shot examples when beneficial

4. Code Organization

• ✅ Separation of concerns (class-based design)
• ✅ Type hints for clarity
• ✅ Docstrings for documentation

5. Performance Optimization

• ✅ Appropriate temperature settings
• ✅ Token budget management
• ✅ Caching where applicable

---

LangChain Components Used

• ChatOpenAI: OpenAI model interface
• Chains: Sequential component composition

Why LangChain? – Composable abstractions for LLM applications – Rich ecosystem of integrations – Production-ready patterns – Active community and development

---

Execution Flow

1. Initialize agent with configuration
   ↓
2. Load environment and dependencies
   ↓
3. Prepare prompt with input data
   ↓
4. Invoke LLM through LangChain
   ↓
5. Parse and validate response
   ↓
6. Return formatted result

Key Decision Points: – Model selection (temperature, model type) – Prompt structure (few-shot vs zero-shot) – Error handling strategy – Output format validation

---

Performance Considerations

Computational Cost

• LLM Calls: Primary cost factor (based on tokens)
• Latency: Depends on model and input size
• Memory: Minimal for most patterns

Optimization Strategies

1. Prompt Optimization: Minimize tokens while maintaining clarity
2. Caching: Cache repeated queries/responses
3. Model Selection: Use smaller models when appropriate
4. Batch Processing: Group similar requests
5. Streaming: For long responses, stream tokens

Cost Estimates (Approximate)

• GPT-3.5-turbo: ~$0.002 per request
• GPT-4: ~$0.03 per request
• GPT-4-turbo: ~$0.01 per request

Actual costs vary based on input/output token counts

---

Common Pitfalls to Avoid

1. Prompt Design Issues

❌ Vague or ambiguous instructions
✅ Clear, specific, testable prompts

2. Error Handling

❌ Ignoring API errors or rate limits
✅ Implement retry logic and graceful degradation

3. Token Management

❌ Exceeding model token limits
✅ Truncate inputs, chunk processing

4. Security

❌ Exposing API keys in code
✅ Use environment variables

5. Testing

❌ Only testing happy paths
✅ Test edge cases, failures, malformed inputs

6. Cost Management

❌ Unmonitored API usage
✅ Implement budgets, alerts, and monitoring

---

Further Improvements

Production Enhancements

1. Monitoring: Add logging, metrics, tracing
2. Validation: Input/output schema validation
3. Rate Limiting: Implement request throttling
4. Caching: Redis or in-memory cache for responses
5. Async Support: Non-blocking operations for scalability

Feature Extensions

1. Multi-Model Support: Switch between providers
2. Fallback Models: Graceful degradation strategy
3. Custom Tools: Domain-specific function calling
4. Advanced Memory: Vector stores for long-term memory
5. Evaluation: Automated testing and benchmarking

Code Quality

1. Testing: Unit tests, integration tests
2. Documentation: API docs, usage examples
3. Type Safety: Stricter type hints
4. Configuration: Externalized config files

---

Related Patterns

• Pattern 081
• Pattern 083

Pattern Families: – Reasoning Patterns (001-011) – Multi-Agent Patterns (012-019) – Tool Use Patterns (020-025) – Memory Patterns (026-031) – And many more…

---

Summary

The Serverless Agent pattern provides a powerful approach to building AI agents with specific capabilities. This implementation demonstrates:

✅ Key Strengths: – Clean, maintainable code structure – LangChain integration for composability – Type hints and documentation – Production-ready patterns

🎯 Best Use Cases: – Applications requiring this pattern’s characteristics – Systems needing structured agent behavior – Production environments requiring reliability

📚 Learning Resources: – LangChain Documentation: https://python.langchain.com/ – Pattern Documentation: See related patterns – OpenAI API Reference: https://platform.openai.com/docs/

---

File: 082_serverless_agent.py
Total Lines: 756
Pattern Category: AI Agent Design Pattern #082

---

Generated from actual implementation code
For educational and reference purposes

    """
Pattern 081: Microservice Agent Architecture

Description:
    A Microservice Agent Architecture decomposes AI agents into independently
    deployable microservices that communicate via well-defined APIs. This pattern
    enables scalability, fault isolation, independent deployment, and technology
    diversity across agent components. Each microservice can be developed, deployed,
    and scaled independently while maintaining loose coupling.
    
    The architecture supports distributed agent systems with service discovery,
    load balancing, inter-service communication, circuit breakers, and observability
    across the microservices mesh.

Components:
    1. Agent Services: Independent agent microservices
    2. API Gateway: Entry point and routing
    3. Service Registry: Service discovery and health checks
    4. Message Bus: Async inter-service communication
    5. Load Balancer: Distribute requests across instances
    6. Circuit Breaker: Fault tolerance and isolation
    7. Config Service: Centralized configuration
    8. Monitoring: Distributed tracing and metrics

Key Features:
    - Independent service deployment
    - Service discovery and registration
    - Inter-service communication (REST, gRPC, message queues)
    - Load balancing and auto-scaling
    - Fault isolation and circuit breakers
    - Distributed tracing
    - API gateway pattern
    - Health checks and monitoring
    - Polyglot persistence
    - Technology diversity

Use Cases:
    - Large-scale AI platforms
    - Multi-tenant AI services
    - Enterprise AI systems
    - Cloud-native AI applications
    - Distributed agent coordination
    - Scalable chatbot platforms
    - AI-powered SaaS products
    - Microservices-based RAG systems
    - Agent mesh architectures
    - Multi-model serving platforms

LangChain Implementation:
    Uses FastAPI for service endpoints, service registry pattern,
    inter-service HTTP communication, and distributed agent coordination.
"""

import os
import time
import uuid
import json
from typing import List, Dict, Any, Optional, Callable
from dataclasses import dataclass, field
from enum import Enum
from datetime import datetime
from collections import defaultdict
from dotenv import load_dotenv

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_openai import ChatOpenAI

load_dotenv()


class ServiceStatus(Enum):
    """Status of a microservice"""
    STARTING = "starting"
    HEALTHY = "healthy"
    UNHEALTHY = "unhealthy"
    DEGRADED = "degraded"
    STOPPED = "stopped"


class CommunicationProtocol(Enum):
    """Inter-service communication protocol"""
    HTTP_REST = "http_rest"
    GRPC = "grpc"
    MESSAGE_QUEUE = "message_queue"
    EVENT_BUS = "event_bus"


@dataclass
class ServiceInfo:
    """Information about a microservice"""
    service_id: str
    service_name: str
    version: str
    host: str
    port: int
    protocol: CommunicationProtocol
    status: ServiceStatus
    endpoints: List[str]
    health_check_url: str
    registered_at: datetime = field(default_factory=datetime.now)
    last_heartbeat: datetime = field(default_factory=datetime.now)
    metadata: Dict[str, Any] = field(default_factory=dict)


@dataclass
class ServiceRequest:
    """Request to a microservice"""
    request_id: str
    service_name: str
    endpoint: str
    method: str
    data: Dict[str, Any]
    headers: Dict[str, str] = field(default_factory=dict)
    timeout: float = 30.0


@dataclass
class ServiceResponse:
    """Response from a microservice"""
    request_id: str
    service_name: str
    status_code: int
    data: Any
    error: Optional[str] = None
    duration: float = 0.0
    metadata: Dict[str, Any] = field(default_factory=dict)


class ServiceRegistry:
    """
    Service registry for microservice discovery.
    
    Maintains a registry of available services with health monitoring.
    """
    
    def __init__(self):
        """Initialize service registry"""
        self.services: Dict[str, ServiceInfo] = {}
        self.service_instances: Dict[str, List[str]] = defaultdict(list)
    
    def register(self, service: ServiceInfo) -> bool:
        """
        Register a service.
        
        Args:
            service: Service information
            
        Returns:
            True if registered successfully
        """
        self.services[service.service_id] = service
        self.service_instances[service.service_name].append(service.service_id)
        return True
    
    def deregister(self, service_id: str) -> bool:
        """
        Deregister a service.
        
        Args:
            service_id: Service ID
            
        Returns:
            True if deregistered successfully
        """
        if service_id in self.services:
            service = self.services[service_id]
            self.service_instances[service.service_name].remove(service_id)
            del self.services[service_id]
            return True
        return False
    
    def discover(self, service_name: str) -> Optional[ServiceInfo]:
        """
        Discover a healthy service instance.
        
        Args:
            service_name: Name of service to discover
            
        Returns:
            ServiceInfo if found, None otherwise
        """
        instances = self.service_instances.get(service_name, [])
        
        # Find healthy instance
        for instance_id in instances:
            service = self.services.get(instance_id)
            if service and service.status == ServiceStatus.HEALTHY:
                return service
        
        return None
    
    def health_check(self, service_id: str) -> ServiceStatus:
        """
        Check health of a service.
        
        Args:
            service_id: Service ID
            
        Returns:
            Service status
        """
        if service_id in self.services:
            service = self.services[service_id]
            # Simulate health check
            service.last_heartbeat = datetime.now()
            return service.status
        
        return ServiceStatus.STOPPED
    
    def list_services(self) -> List[ServiceInfo]:
        """
        List all registered services.
        
        Returns:
            List of service information
        """
        return list(self.services.values())


class AgentService:
    """
    Base microservice for AI agents.
    
    Each agent capability is encapsulated as an independent service.
    """
    
    def __init__(
        self,
        service_name: str,
        version: str = "1.0.0",
        port: int = 8000
    ):
        """
        Initialize agent service.
        
        Args:
            service_name: Name of the service
            version: Service version
            port: Port number
        """
        self.service_id = f"{service_name}_{uuid.uuid4().hex[:8]}"
        self.service_name = service_name
        self.version = version
        self.port = port
        self.status = ServiceStatus.STARTING
        
        # LLM for this service
        self.llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0.7)
        
        # Request tracking
        self.requests: Dict[str, ServiceRequest] = {}
        self.responses: Dict[str, ServiceResponse] = {}
    
    def get_info(self) -> ServiceInfo:
        """
        Get service information.
        
        Returns:
            ServiceInfo object
        """
        return ServiceInfo(
            service_id=self.service_id,
            service_name=self.service_name,
            version=self.version,
            host="localhost",
            port=self.port,
            protocol=CommunicationProtocol.HTTP_REST,
            status=self.status,
            endpoints=["/health", "/process"],
            health_check_url=f"http://localhost:{self.port}/health"
        )
    
    def start(self) -> bool:
        """
        Start the service.
        
        Returns:
            True if started successfully
        """
        self.status = ServiceStatus.HEALTHY
        return True
    
    def stop(self) -> bool:
        """
        Stop the service.
        
        Returns:
            True if stopped successfully
        """
        self.status = ServiceStatus.STOPPED
        return True
    
    def health(self) -> Dict[str, Any]:
        """
        Health check endpoint.
        
        Returns:
            Health status
        """
        return {
            "service": self.service_name,
            "status": self.status.value,
            "version": self.version,
            "timestamp": datetime.now().isoformat()
        }
    
    def process(self, data: Dict[str, Any]) -> Dict[str, Any]:
        """
        Process a request (to be overridden by subclasses).
        
        Args:
            data: Request data
            
        Returns:
            Response data
        """
        raise NotImplementedError("Subclasses must implement process()")


class SummarizationService(AgentService):
    """Microservice for text summarization"""
    
    def __init__(self):
        super().__init__("summarization", version="1.0.0", port=8001)
    
    def process(self, data: Dict[str, Any]) -> Dict[str, Any]:
        """
        Summarize text.
        
        Args:
            data: {'text': str, 'length': str}
            
        Returns:
            {'summary': str}
        """
        text = data.get('text', '')
        length = data.get('length', 'short')
        
        prompt = ChatPromptTemplate.from_template(
            "Summarize the following text in {length} form:\n\n{text}"
        )
        
        chain = prompt | self.llm | StrOutputParser()
        
        summary = chain.invoke({
            'text': text[:1000],
            'length': length
        })
        
        return {
            'summary': summary,
            'service': self.service_name
        }


class TranslationService(AgentService):
    """Microservice for translation"""
    
    def __init__(self):
        super().__init__("translation", version="1.0.0", port=8002)
    
    def process(self, data: Dict[str, Any]) -> Dict[str, Any]:
        """
        Translate text.
        
        Args:
            data: {'text': str, 'target_language': str}
            
        Returns:
            {'translation': str}
        """
        text = data.get('text', '')
        target_language = data.get('target_language', 'Spanish')
        
        prompt = ChatPromptTemplate.from_template(
            "Translate the following text to {language}:\n\n{text}"
        )
        
        chain = prompt | self.llm | StrOutputParser()
        
        translation = chain.invoke({
            'text': text[:500],
            'language': target_language
        })
        
        return {
            'translation': translation,
            'target_language': target_language,
            'service': self.service_name
        }


class SentimentService(AgentService):
    """Microservice for sentiment analysis"""
    
    def __init__(self):
        super().__init__("sentiment", version="1.0.0", port=8003)
    
    def process(self, data: Dict[str, Any]) -> Dict[str, Any]:
        """
        Analyze sentiment.
        
        Args:
            data: {'text': str}
            
        Returns:
            {'sentiment': str, 'confidence': float}
        """
        text = data.get('text', '')
        
        prompt = ChatPromptTemplate.from_template(
            """Analyze the sentiment of this text. Respond with:
SENTIMENT: positive/negative/neutral
CONFIDENCE: 0.0-1.0

Text: {text}"""
        )
        
        chain = prompt | self.llm | StrOutputParser()
        
        result = chain.invoke({'text': text[:500]})
        
        # Parse result
        sentiment = "neutral"
        confidence = 0.5
        
        for line in result.split('\n'):
            if 'SENTIMENT:' in line:
                sentiment = line.split(':')[1].strip().lower()
            elif 'CONFIDENCE:' in line:
                try:
                    confidence = float(line.split(':')[1].strip())
                except:
                    pass
        
        return {
            'sentiment': sentiment,
            'confidence': confidence,
            'service': self.service_name
        }


class APIGateway:
    """
    API Gateway for routing requests to microservices.
    
    Provides a single entry point and routes to appropriate services.
    """
    
    def __init__(self, registry: ServiceRegistry):
        """
        Initialize API gateway.
        
        Args:
            registry: Service registry
        """
        self.registry = registry
        self.request_count = 0
        self.route_map = {
            'summarize': 'summarization',
            'translate': 'translation',
            'sentiment': 'sentiment'
        }
    
    def route(self, operation: str, data: Dict[str, Any]) -> ServiceResponse:
        """
        Route request to appropriate service.
        
        Args:
            operation: Operation to perform
            data: Request data
            
        Returns:
            ServiceResponse
        """
        request_id = f"req_{self.request_count}"
        self.request_count += 1
        
        start_time = time.time()
        
        # Determine target service
        service_name = self.route_map.get(operation)
        
        if not service_name:
            return ServiceResponse(
                request_id=request_id,
                service_name="unknown",
                status_code=404,
                data=None,
                error=f"Unknown operation: {operation}",
                duration=time.time() - start_time
            )
        
        # Discover service
        service = self.registry.discover(service_name)
        
        if not service:
            return ServiceResponse(
                request_id=request_id,
                service_name=service_name,
                status_code=503,
                data=None,
                error=f"Service unavailable: {service_name}",
                duration=time.time() - start_time
            )
        
        # Forward request to service (simulated)
        try:
            # In real implementation, would make HTTP request
            # Here we simulate the service call
            result_data = {
                'status': 'success',
                'operation': operation,
                'service': service_name
            }
            
            return ServiceResponse(
                request_id=request_id,
                service_name=service_name,
                status_code=200,
                data=result_data,
                duration=time.time() - start_time
            )
            
        except Exception as e:
            return ServiceResponse(
                request_id=request_id,
                service_name=service_name,
                status_code=500,
                data=None,
                error=str(e),
                duration=time.time() - start_time
            )


class MicroserviceOrchestrator:
    """
    Orchestrator for coordinating multiple microservices.
    
    Manages service lifecycle and coordinates complex workflows
    across multiple services.
    """
    
    def __init__(self):
        """Initialize orchestrator"""
        self.registry = ServiceRegistry()
        self.gateway = APIGateway(self.registry)
        self.services: Dict[str, AgentService] = {}
    
    def deploy_service(self, service: AgentService) -> bool:
        """
        Deploy a microservice.
        
        Args:
            service: Agent service to deploy
            
        Returns:
            True if deployed successfully
        """
        # Start service
        service.start()
        
        # Register with registry
        service_info = service.get_info()
        self.registry.register(service_info)
        
        # Track service
        self.services[service.service_id] = service
        
        return True
    
    def undeploy_service(self, service_id: str) -> bool:
        """
        Undeploy a microservice.
        
        Args:
            service_id: Service ID
            
        Returns:
            True if undeployed successfully
        """
        if service_id in self.services:
            service = self.services[service_id]
            service.stop()
            self.registry.deregister(service_id)
            del self.services[service_id]
            return True
        return False
    
    def execute_workflow(
        self,
        operations: List[Dict[str, Any]]
    ) -> List[ServiceResponse]:
        """
        Execute a workflow across multiple services.
        
        Args:
            operations: List of operations with data
            
        Returns:
            List of service responses
        """
        responses = []
        
        for op in operations:
            operation = op.get('operation')
            data = op.get('data', {})
            
            response = self.gateway.route(operation, data)
            responses.append(response)
        
        return responses
    
    def get_service_health(self) -> Dict[str, Any]:
        """
        Get health status of all services.
        
        Returns:
            Health status dictionary
        """
        services = self.registry.list_services()
        
        health_status = {
            'total_services': len(services),
            'healthy': 0,
            'unhealthy': 0,
            'services': []
        }
        
        for service in services:
            status = self.registry.health_check(service.service_id)
            
            if status == ServiceStatus.HEALTHY:
                health_status['healthy'] += 1
            else:
                health_status['unhealthy'] += 1
            
            health_status['services'].append({
                'name': service.service_name,
                'id': service.service_id,
                'status': status.value,
                'version': service.version
            })
        
        return health_status


def demonstrate_microservice_architecture():
    """Demonstrate the microservice agent architecture"""
    print("=" * 80)
    print("MICROSERVICE AGENT ARCHITECTURE DEMONSTRATION")
    print("=" * 80)
    
    orchestrator = MicroserviceOrchestrator()
    
    # Demo 1: Deploy Microservices
    print("\n" + "=" * 80)
    print("DEMO 1: Deploying Microservices")
    print("=" * 80)
    
    # Deploy services
    services = [
        SummarizationService(),
        TranslationService(),
        SentimentService()
    ]
    
    print("\nDeploying services...")
    for service in services:
        success = orchestrator.deploy_service(service)
        print(f"  ✓ Deployed: {service.service_name} v{service.version} on port {service.port}")
    
    # Demo 2: Service Discovery
    print("\n" + "=" * 80)
    print("DEMO 2: Service Discovery")
    print("=" * 80)
    
    print("\nRegistered Services:")
    print("-" * 80)
    
    for service_info in orchestrator.registry.list_services():
        print(f"\nService: {service_info.service_name}")
        print(f"  ID: {service_info.service_id}")
        print(f"  Version: {service_info.version}")
        print(f"  Status: {service_info.status.value}")
        print(f"  Endpoint: {service_info.host}:{service_info.port}")
        print(f"  Protocol: {service_info.protocol.value}")
    
    # Demo 3: Service Health Checks
    print("\n" + "=" * 80)
    print("DEMO 3: Service Health Monitoring")
    print("=" * 80)
    
    health = orchestrator.get_service_health()
    
    print(f"\nCluster Health:")
    print(f"  Total Services: {health['total_services']}")
    print(f"  Healthy: {health['healthy']}")
    print(f"  Unhealthy: {health['unhealthy']}")
    
    print(f"\nService Status:")
    for svc in health['services']:
        status_icon = "✓" if svc['status'] == 'healthy' else "✗"
        print(f"  {status_icon} {svc['name']} ({svc['version']}) - {svc['status']}")
    
    # Demo 4: API Gateway Routing
    print("\n" + "=" * 80)
    print("DEMO 4: API Gateway Request Routing")
    print("=" * 80)
    
    print("\nRouting requests through API Gateway...")
    print("-" * 80)
    
    requests = [
        {'operation': 'summarize', 'data': {'text': 'Sample text', 'length': 'short'}},
        {'operation': 'translate', 'data': {'text': 'Hello', 'target_language': 'French'}},
        {'operation': 'sentiment', 'data': {'text': 'Great product!'}}
    ]
    
    for req in requests:
        response = orchestrator.gateway.route(req['operation'], req['data'])
        
        print(f"\nRequest: {req['operation']}")
        print(f"  Request ID: {response.request_id}")
        print(f"  Service: {response.service_name}")
        print(f"  Status Code: {response.status_code}")
        print(f"  Duration: {response.duration*1000:.2f}ms")
        
        if response.error:
            print(f"  Error: {response.error}")
    
    # Demo 5: Direct Service Calls
    print("\n" + "=" * 80)
    print("DEMO 5: Direct Service Invocation")
    print("=" * 80)
    
    print("\nCalling services directly...")
    print("-" * 80)
    
    # Summarization
    print("\n1. Summarization Service:")
    summarization_svc = services[0]
    result = summarization_svc.process({
        'text': 'Microservices architecture enables building complex systems through independently deployable services.',
        'length': 'very short'
    })
    print(f"   Input: Long text about microservices")
    print(f"   Summary: {result['summary'][:80]}...")
    
    # Translation
    print("\n2. Translation Service:")
    translation_svc = services[1]
    result = translation_svc.process({
        'text': 'Hello, how are you?',
        'target_language': 'Spanish'
    })
    print(f"   Input: 'Hello, how are you?'")
    print(f"   Translation: {result['translation']}")
    
    # Sentiment
    print("\n3. Sentiment Service:")
    sentiment_svc = services[2]
    result = sentiment_svc.process({
        'text': 'This microservice architecture is fantastic!'
    })
    print(f"   Input: 'This microservice architecture is fantastic!'")
    print(f"   Sentiment: {result['sentiment']} (confidence: {result['confidence']:.2f})")
    
    # Demo 6: Multi-Service Workflow
    print("\n" + "=" * 80)
    print("DEMO 6: Orchestrated Multi-Service Workflow")
    print("=" * 80)
    
    workflow = [
        {'operation': 'summarize', 'data': {'text': 'Long article text...', 'length': 'short'}},
        {'operation': 'translate', 'data': {'text': 'Summary text', 'target_language': 'German'}},
        {'operation': 'sentiment', 'data': {'text': 'Translated summary'}}
    ]
    
    print("\nExecuting workflow:")
    print("  1. Summarize → 2. Translate → 3. Analyze Sentiment")
    print("-" * 80)
    
    responses = orchestrator.execute_workflow(workflow)
    
    print(f"\nWorkflow Results:")
    for i, response in enumerate(responses, 1):
        print(f"\n  Step {i}: {response.service_name}")
        print(f"    Status: {response.status_code}")
        print(f"    Duration: {response.duration*1000:.2f}ms")
    
    total_duration = sum(r.duration for r in responses)
    print(f"\n  Total Workflow Duration: {total_duration*1000:.2f}ms")
    
    # Summary
    print("\n" + "=" * 80)
    print("SUMMARY")
    print("=" * 80)
    
    summary = """
The Microservice Agent Architecture demonstrates distributed AI systems:

KEY CAPABILITIES:
1. Service Registry: Automatic service discovery and registration
2. API Gateway: Single entry point with intelligent routing
3. Independent Services: Each capability as a separate microservice
4. Health Monitoring: Continuous health checks and status tracking
5. Service Orchestration: Coordinate complex multi-service workflows
6. Fault Isolation: Failures contained within services
7. Load Balancing: Distribute traffic across service instances
8. Technology Diversity: Each service can use different tech stacks

BENEFITS:
- Independent deployment and scaling
- Fault isolation and resilience
- Technology flexibility per service
- Team autonomy and parallel development
- Easier testing and maintenance
- Better resource utilization
- Scalability for specific services
- Faster iteration and releases

USE CASES:
- Large-scale AI platforms with multiple capabilities
- Multi-tenant SaaS AI applications
- Enterprise AI systems requiring high availability
- Cloud-native AI applications
- Distributed agent coordination
- Microservices-based RAG systems
- AI-powered e-commerce platforms
- Scalable chatbot infrastructures
- Multi-model serving platforms
- Agent mesh architectures

PRODUCTION CONSIDERATIONS:
1. Service Discovery: Use Consul, etcd, or Kubernetes DNS
2. API Gateway: Kong, NGINX, or AWS API Gateway
3. Communication: REST APIs, gRPC, or message queues
4. Monitoring: Distributed tracing (Jaeger, Zipkin)
5. Logging: Centralized logging (ELK stack)
6. Configuration: External config service (Spring Cloud Config)
7. Security: API authentication, mTLS between services
8. Circuit Breakers: Prevent cascading failures
9. Container Orchestration: Kubernetes, Docker Swarm
10. Service Mesh: Istio, Linkerd for advanced features

ADVANCED EXTENSIONS:
- Event-driven architecture with Kafka/RabbitMQ
- CQRS pattern for read/write separation
- Saga pattern for distributed transactions
- API versioning and backward compatibility
- Blue-green deployments
- Canary releases for gradual rollouts
- Auto-scaling based on metrics
- Multi-region deployment
- Service mesh for advanced traffic management
- GraphQL federation for unified API

Microservice architecture is essential for building scalable,
maintainable, and resilient AI systems in production environments.
"""
    
    print(summary)


if __name__ == "__main__":
    demonstrate_microservice_architecture()

    

Agentic Design Pattern: Microservice Agent

Pattern 081: Microservice Agent – Line-by-Line Explanation

Overview

This document provides a detailed line-by-line explanation of the Microservice Agent pattern implementation.

---

Pattern Description

Pattern 081: Microservice Agent Architecture

Description: A Microservice Agent Architecture decomposes AI agents into independently deployable microservices that communicate via well-defined APIs. This pattern enables scalability, fault isolation, independent deployment, and technology diversity across agent components. Each microservice can be developed, deployed, and scaled independently while maintaining loose coupling.

The architecture supports distributed agent systems with service discovery,
load balancing, inter-service communication, circuit breakers, and observability
across the microservices mesh.

Components: 1. Agent Services: Independent agent microservices 2. API Gateway: Entry point and routing 3. Service Registry: Service discovery and health checks 4. Message Bus: Async inter-service communication 5. Load Balancer: Distribute requests across instances 6. Circuit Breaker: Fault tolerance and isolation 7. Config Service: Centralized configuration 8. Monitoring: Distributed tracing and metrics

Key Features: – Independent service deployment – Service discovery and registration – Inter-service communication (REST, gRPC, message queues) – Load balancing and auto-scaling – Fault isolation and circuit breakers – Distributed tracing – API gateway pattern – Health checks and monitoring – Polyglot persistence – Technology diversity

Use Cases: – Large-scale AI platforms – Multi-tenant AI services – Enterprise AI systems – Cloud-native AI applications – Distributed agent coordination – Scalable chatbot platforms – AI-powered SaaS products – Microservices-based RAG systems – Agent mesh architectures – Multi-model serving platforms

LangChain Implementation: Uses FastAPI for service endpoints, service registry pattern, inter-service HTTP communication, and distributed agent coordination.

---

File Statistics

• Total Lines: 869
• Classes: 12
• Functions: 1
• Constants: 0

---

Module Documentation

"""
Pattern 081: Microservice Agent Architecture

Description:
    A Microservice Agent Architecture decomposes AI agents into independently
    deployable microservices that communicate via well-defined APIs. This pattern
    enables scalability, fault isolation, independent deployment, and technology
    diversity across agent components. Each microservice can be developed, deployed,
    and scaled independently while maintaining loose coupling.

    The architecture supports distributed agent systems with service discovery,
    load balancing, inter-service communication, circuit breakers, and observability
    across the microservices mesh.

Components:
    1. Agent Services: Independent agent microservices
    2. API Gateway: Entry point and routing
    3. Service Registry: Service discovery and health checks
    4. Message Bus: Async inter-service communication
    5. Load Balancer: Distribute requests across instances
    6. Circuit Breaker: Fault tolerance and isolation
    7. Config Service: Centralized configuration
    8. Monitoring: Distributed tracing and metrics

Key Features:
    - Independent service deployment
    - Service discovery and registration
    - Inter-service communication (REST, gRPC, message queues)
    - Load balancing and auto-scaling
    - Fault isolation and circuit breakers
    - Distributed tracing
    - API gateway pattern
    - Health checks and monitoring
    - Polyglot persistence
    - Technology diversity

Use Cases:
    - Large-scale AI platforms
    - Multi-tenant AI services
    - Enterprise AI systems
    - Cloud-native AI applications
    - Distributed agent coordination
    - Scalable chatbot platforms
    - AI-powered SaaS products
    - Microservices-based RAG systems
    - Agent mesh architectures
    - Multi-model serving platforms

LangChain Implementation:
    Uses FastAPI for service endpoints, service registry pattern,
    inter-service HTTP communication, and distributed agent coordination.
"""

Purpose: This module implements the Microservice Agent pattern, providing a structured approach to agent design and implementation.

---

Import Statements

Standard Library Imports

import collections.defaultdict
import dataclasses.dataclass
import dataclasses.field
import datetime.datetime
import enum.Enum
import json
import langchain_core.output_parsers.StrOutputParser
import langchain_core.prompts.ChatPromptTemplate
import os
import time
import typing.Any
import typing.Callable
import typing.Dict
import typing.List
import typing.Optional

Purpose: Core Python functionality: – collections.defaultdict: Specialized container datatypes – dataclasses.dataclass: Simplified class definitions with @dataclass – dataclasses.field: Simplified class definitions with @dataclass – datetime.datetime: Date and time handling – enum.Enum: Enumeration support – json: JSON serialization/deserialization – langchain_core.output_parsers.StrOutputParser: Regular expression operations – langchain_core.prompts.ChatPromptTemplate: Regular expression operations – os: Environment variable access and system operations – time: Time-related functions – typing.Any: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Callable: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Dict: Type hints for better code documentation (List, Dict, Any, Optional) – typing.List: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Optional: Type hints for better code documentation (List, Dict, Any, Optional)

LangChain Imports

from langchain_openai.ChatOpenAI

Purpose: LangChain framework components: – LLM Interface: Wrapper for OpenAI models – Chains: Composable building blocks for complex workflows

Other Imports

from dotenv.load_dotenv
from uuid

Purpose: Additional dependencies for specialized functionality.

---

Classes

Class: ServiceStatus

Lines 72-78

Inherits from: Enum

Purpose:

Status of a microservice

---

Class: CommunicationProtocol

Lines 81-86

Inherits from: Enum

Purpose:

Inter-service communication protocol

---

Class: ServiceInfo

Lines 90-103

Purpose:

Information about a microservice

---

Class: ServiceRequest

Lines 107-115

Purpose:

Request to a microservice

---

Class: ServiceResponse

Lines 119-127

Purpose:

Response from a microservice

---

Class: ServiceRegistry

Lines 130-218

Purpose:

Service registry for microservice discovery.

Maintains a registry of available services with health monitoring.

Methods:

__init__(self)

Lines 137-140

Purpose:

Initialize service registry

Parameters:

register(self, service)

Lines 142-154

Purpose:

Register a service.

Args:
    service: Service information

Returns:
    True if registered successfully

Parameters: – service: Service

Returns: bool

deregister(self, service_id)

Lines 156-171

Purpose:

Deregister a service.

Args:
    service_id: Service ID

Returns:
    True if deregistered successfully

Parameters: – service_id: Service id

Returns: bool

discover(self, service_name)

Lines 173-191

Purpose:

Discover a healthy service instance.

Args:
    service_name: Name of service to discover

Returns:
    ServiceInfo if found, None otherwise

Parameters: – service_name: Service name

Returns: <ast.Subscript object at 0x000001A06D7FF2D0>

health_check(self, service_id)

Lines 193-209

Purpose:

Check health of a service.

Args:
    service_id: Service ID

Returns:
    Service status

Parameters: – service_id: Service id

Returns: ServiceStatus

list_services(self)

Lines 211-218

Purpose:

List all registered services.

Returns:
    List of service information

Parameters:

Returns: <ast.Subscript object at 0x000001A06D7FE710>

---

Class: AgentService

Lines 221-318

Purpose:

Base microservice for AI agents.

Each agent capability is encapsulated as an independent service.

Methods:

__init__(self, service_name, version, port)

Lines 228-253

Purpose:

Initialize agent service.

Args:
    service_name: Name of the service
    version: Service version
    port: Port number

Parameters: – service_name: Service name – version: Version – port: Port

get_info(self)

Lines 255-272

Purpose:

Get service information.

Returns:
    ServiceInfo object

Parameters:

Returns: ServiceInfo

start(self)

Lines 274-282

Purpose:

Start the service.

Returns:
    True if started successfully

Parameters:

Returns: bool

stop(self)

Lines 284-292

Purpose:

Stop the service.

Returns:
    True if stopped successfully

Parameters:

Returns: bool

health(self)

Lines 294-306

Purpose:

Health check endpoint.

Returns:
    Health status

Parameters:

Returns: <ast.Subscript object at 0x000001A06D7BB390>

process(self, data)

Lines 308-318

Purpose:

Process a request (to be overridden by subclasses).

Args:
    data: Request data

Returns:
    Response data

Parameters: – data: Data

Returns: <ast.Subscript object at 0x000001A06D7BADD0>

---

Class: SummarizationService

Lines 321-354

Inherits from: AgentService

Purpose:

Microservice for text summarization

Methods:

__init__(self)

Lines 324-325

Purpose: Initialize the class instance with required parameters.

Parameters:

process(self, data)

Lines 327-354

Purpose:

Summarize text.

Args:
    data: {'text': str, 'length': str}

Returns:
    {'summary': str}

Parameters: – data: Data

Returns: <ast.Subscript object at 0x000001A06D74A3D0>

---

Class: TranslationService

Lines 357-391

Inherits from: AgentService

Purpose:

Microservice for translation

Methods:

__init__(self)

Lines 360-361

Purpose: Initialize the class instance with required parameters.

Parameters:

process(self, data)

Lines 363-391

Purpose:

Translate text.

Args:
    data: {'text': str, 'target_language': str}

Returns:
    {'translation': str}

Parameters: – data: Data

Returns: <ast.Subscript object at 0x000001A06D72DB90>

---

Class: SentimentService

Lines 394-441

Inherits from: AgentService

Purpose:

Microservice for sentiment analysis

Methods:

__init__(self)

Lines 397-398

Purpose: Initialize the class instance with required parameters.

Parameters:

process(self, data)

Lines 400-441

Purpose:

Analyze sentiment.

Args:
    data: {'text': str}

Returns:
    {'sentiment': str, 'confidence': float}

Parameters: – data: Data

Returns: <ast.Subscript object at 0x000001A06D6061D0>

---

Class: APIGateway

Lines 444-534

Purpose:

API Gateway for routing requests to microservices.

Provides a single entry point and routes to appropriate services.

Methods:

__init__(self, registry)

Lines 451-464

Purpose:

Initialize API gateway.

Args:
    registry: Service registry

Parameters: – registry: Registry

route(self, operation, data)

Lines 466-534

Purpose:

Route request to appropriate service.

Args:
    operation: Operation to perform
    data: Request data

Returns:
    ServiceResponse

Parameters: – operation: Operation – data: Data

Returns: ServiceResponse

---

Class: MicroserviceOrchestrator

Lines 537-646

Purpose:

Orchestrator for coordinating multiple microservices.

Manages service lifecycle and coordinates complex workflows
across multiple services.

Methods:

__init__(self)

Lines 545-549

Purpose:

Initialize orchestrator

Parameters:

deploy_service(self, service)

Lines 551-571

Purpose:

Deploy a microservice.

Args:
    service: Agent service to deploy

Returns:
    True if deployed successfully

Parameters: – service: Service

Returns: bool

undeploy_service(self, service_id)

Lines 573-589

Purpose:

Undeploy a microservice.

Args:
    service_id: Service ID

Returns:
    True if undeployed successfully

Parameters: – service_id: Service id

Returns: bool

execute_workflow(self, operations)

Lines 591-613

Purpose:

Execute a workflow across multiple services.

Args:
    operations: List of operations with data

Returns:
    List of service responses

Parameters: – operations: Operations

Returns: <ast.Subscript object at 0x000001A06D7F4BD0>

get_service_health(self)

Lines 615-646

Purpose:

Get health status of all services.

Returns:
    Health status dictionary

Parameters:

Returns: <ast.Subscript object at 0x000001A06D739690>

---

Functions

demonstrate_microservice_architecture()

Lines 649-864

Purpose:

Demonstrate the microservice agent architecture

---

Key Concepts Summary

Pattern Overview

The Microservice Agent pattern provides a structured approach to building AI agents with specific capabilities and behaviors. This implementation leverages LangChain’s composable components to create a robust and maintainable solution.

Core Components

1. Agent Class: Main implementation of the pattern logic
2. Prompt Engineering: Carefully crafted prompts to guide LLM behavior
3. Tool Integration: Optional tools for extended functionality
4. Memory Management: State handling across interactions
5. Output Parsing: Structured response processing

When to Use

✅ Good for: – Scenarios requiring this specific pattern’s strengths – Applications needing structured agent behavior – Systems where explainability is important – Production environments requiring reliability

❌ Not ideal for: – Simple single-prompt tasks – Scenarios requiring different pattern characteristics – Resource-constrained environments (if computationally intensive)

---

Best Practices Demonstrated

1. Environment Configuration

• ✅ Use .env files for API keys
• ✅ Never hardcode credentials
• ✅ Validate environment variables on startup

2. Error Handling

• ✅ Graceful degradation on failures
• ✅ Informative error messages
• ✅ Logging for debugging

3. Prompt Engineering

• ✅ Clear, specific instructions
• ✅ Structured output formats
• ✅ Few-shot examples when beneficial

4. Code Organization

• ✅ Separation of concerns (class-based design)
• ✅ Type hints for clarity
• ✅ Docstrings for documentation

5. Performance Optimization

• ✅ Appropriate temperature settings
• ✅ Token budget management
• ✅ Caching where applicable

---

LangChain Components Used

• ChatOpenAI: OpenAI model interface
• Chains: Sequential component composition

Why LangChain? – Composable abstractions for LLM applications – Rich ecosystem of integrations – Production-ready patterns – Active community and development

---

Execution Flow

1. Initialize agent with configuration
   ↓
2. Load environment and dependencies
   ↓
3. Prepare prompt with input data
   ↓
4. Invoke LLM through LangChain
   ↓
5. Parse and validate response
   ↓
6. Return formatted result

Key Decision Points: – Model selection (temperature, model type) – Prompt structure (few-shot vs zero-shot) – Error handling strategy – Output format validation

---

Performance Considerations

Computational Cost

• LLM Calls: Primary cost factor (based on tokens)
• Latency: Depends on model and input size
• Memory: Minimal for most patterns

Optimization Strategies

1. Prompt Optimization: Minimize tokens while maintaining clarity
2. Caching: Cache repeated queries/responses
3. Model Selection: Use smaller models when appropriate
4. Batch Processing: Group similar requests
5. Streaming: For long responses, stream tokens

Cost Estimates (Approximate)

• GPT-3.5-turbo: ~$0.002 per request
• GPT-4: ~$0.03 per request
• GPT-4-turbo: ~$0.01 per request

Actual costs vary based on input/output token counts

---

Common Pitfalls to Avoid

1. Prompt Design Issues

❌ Vague or ambiguous instructions
✅ Clear, specific, testable prompts

2. Error Handling

❌ Ignoring API errors or rate limits
✅ Implement retry logic and graceful degradation

3. Token Management

❌ Exceeding model token limits
✅ Truncate inputs, chunk processing

4. Security

❌ Exposing API keys in code
✅ Use environment variables

5. Testing

❌ Only testing happy paths
✅ Test edge cases, failures, malformed inputs

6. Cost Management

❌ Unmonitored API usage
✅ Implement budgets, alerts, and monitoring

---

Further Improvements

Production Enhancements

1. Monitoring: Add logging, metrics, tracing
2. Validation: Input/output schema validation
3. Rate Limiting: Implement request throttling
4. Caching: Redis or in-memory cache for responses
5. Async Support: Non-blocking operations for scalability

Feature Extensions

1. Multi-Model Support: Switch between providers
2. Fallback Models: Graceful degradation strategy
3. Custom Tools: Domain-specific function calling
4. Advanced Memory: Vector stores for long-term memory
5. Evaluation: Automated testing and benchmarking

Code Quality

1. Testing: Unit tests, integration tests
2. Documentation: API docs, usage examples
3. Type Safety: Stricter type hints
4. Configuration: Externalized config files

---

Related Patterns

• Pattern 080
• Pattern 082

Pattern Families: – Reasoning Patterns (001-011) – Multi-Agent Patterns (012-019) – Tool Use Patterns (020-025) – Memory Patterns (026-031) – And many more…

---

Summary

The Microservice Agent pattern provides a powerful approach to building AI agents with specific capabilities. This implementation demonstrates:

✅ Key Strengths: – Clean, maintainable code structure – LangChain integration for composability – Type hints and documentation – Production-ready patterns

🎯 Best Use Cases: – Applications requiring this pattern’s characteristics – Systems needing structured agent behavior – Production environments requiring reliability

📚 Learning Resources: – LangChain Documentation: https://python.langchain.com/ – Pattern Documentation: See related patterns – OpenAI API Reference: https://platform.openai.com/docs/

---

File: 081_microservice_agent.py
Total Lines: 869
Pattern Category: AI Agent Design Pattern #081

---

Generated from actual implementation code
For educational and reference purposes