aishastr.live

Blog

  • Agentic Design Pattern: Load Balancing

    
    """
Pattern 090: Load Balancing Pattern

Description:
    The Load Balancing Pattern distributes requests across multiple LLM endpoints or
    instances to achieve high availability, improved performance, and fault tolerance.
    In production LLM applications, load balancing is critical for handling high traffic
    volumes, preventing single points of failure, and ensuring consistent service
    quality even when individual endpoints fail or become overloaded.

    Load balancing strategies enable:
    - High availability through redundancy
    - Improved performance via parallel processing
    - Cost optimization through provider selection
    - Rate limit management across multiple accounts
    - Geographic distribution for lower latency
    - Failover and disaster recovery

Components:
    1. Load Balancing Strategies
       - Round Robin: Distribute equally across endpoints
       - Weighted Round Robin: Priority-based distribution
       - Least Loaded: Send to endpoint with lowest load
       - Random: Random endpoint selection
       - Geographic: Route based on location
       - Cost-Based: Choose cheapest available endpoint

    2. Health Checking
       - Endpoint availability monitoring
       - Response time tracking
       - Error rate monitoring
       - Periodic health probes
       - Automatic endpoint removal/recovery

    3. Request Management
       - Request queuing and buffering
       - Retry logic with backoff
       - Circuit breaker pattern
       - Request timeout handling
       - Priority-based routing

    4. Failover Mechanisms
       - Automatic endpoint switching
       - Degraded mode operation
       - Fallback to alternative providers
       - Request replay on failure
       - State preservation

Use Cases:
    1. High-Traffic Applications
       - Chatbots serving thousands of users
       - API services with SLA requirements
       - Real-time processing pipelines
       - Multi-tenant platforms
       - Peak load handling

    2. Multi-Provider Deployments
       - OpenAI + Anthropic + Google
       - Primary and backup providers
       - Cost optimization strategies
       - Feature-specific routing
       - Geographic compliance

    3. Rate Limit Management
       - Distribute across multiple API keys
       - Prevent rate limit exhaustion
       - Smooth request distribution
       - Burst handling
       - Cost control

    4. Disaster Recovery
       - Automatic failover
       - Multi-region deployment
       - Zero-downtime updates
       - Service degradation handling
       - Business continuity

    5. Performance Optimization
       - Parallel request processing
       - Load-based routing
       - Latency optimization
       - Resource utilization
       - Capacity planning

LangChain Implementation:
    LangChain doesn't have built-in load balancing, but you can implement
    it using:
    - Custom LLM wrappers
    - Callback handlers for monitoring
    - Retry logic with different models
    - Custom routing logic
    - Health check mechanisms

Key Features:
    1. Endpoint Management
       - Dynamic endpoint registration
       - Health status tracking
       - Automatic recovery
       - Configuration updates
       - Version management

    2. Intelligent Routing
       - Strategy selection
       - Real-time decision making
       - Performance-based routing
       - Cost optimization
       - SLA enforcement

    3. Fault Tolerance
       - Automatic retry with exponential backoff
       - Circuit breaker to prevent cascading failures
       - Graceful degradation
       - Error handling and recovery
       - Request preservation

    4. Monitoring
       - Request distribution metrics
       - Endpoint health status
       - Response time tracking
       - Error rate monitoring
       - Cost tracking per endpoint

Best Practices:
    1. Health Monitoring
       - Regular health checks (every 30-60s)
       - Multiple health metrics (latency, errors, availability)
       - Automatic endpoint recovery
       - Alert on degraded performance
       - Log all health transitions

    2. Load Distribution
       - Consider endpoint capacity
       - Account for different model speeds
       - Balance cost vs performance
       - Avoid overloading any single endpoint
       - Reserve capacity for spikes

    3. Failover Strategy
       - Fast failure detection (< 5s)
       - Quick failover ( float:
        """Calculate success rate"""
        if self.total_requests == 0:
            return 0.0
        return (self.successful_requests / self.total_requests) * 100
    
    @property
    def average_response_time(self) -> float:
        """Calculate average response time"""
        if self.successful_requests == 0:
            return 0.0
        return self.total_response_time / self.successful_requests


@dataclass
class LoadBalancerStats:
    """Overall load balancer statistics"""
    total_requests: int = 0
    successful_requests: int = 0
    failed_requests: int = 0
    failovers: int = 0
    total_response_time: float = 0.0
    endpoint_metrics: Dict[str, EndpointMetrics] = field(default_factory=dict)


class LLMEndpoint:
    """
    Represents a single LLM endpoint with health tracking.
    """
    
    def __init__(self, config: EndpointConfig):
        """
        Initialize endpoint.
        
        Args:
            config: Endpoint configuration
        """
        self.config = config
        self.metrics = EndpointMetrics()
        self.llm: Optional[ChatOpenAI] = None
        self.last_health_check: Optional[datetime] = None
        self.request_times: deque = deque(maxlen=100)  # Track recent request times
        
        self._initialize_llm()
    
    def _initialize_llm(self):
        """Initialize the LLM client"""
        try:
            # Use provided API key or default from environment
            api_key = self.config.api_key or os.getenv("OPENAI_API_KEY")
            
            self.llm = ChatOpenAI(
                model=self.config.model,
                openai_api_key=api_key,
                timeout=self.config.timeout,
                temperature=0.7
            )
            self.metrics.status = EndpointStatus.HEALTHY
        except Exception as e:
            print(f"Error initializing endpoint {self.config.id}: {e}")
            self.metrics.status = EndpointStatus.UNHEALTHY
            self.metrics.last_error = str(e)
            self.metrics.last_error_time = datetime.now()
    
    def is_available(self) -> bool:
        """Check if endpoint is available for requests"""
        return (
            self.metrics.status in [EndpointStatus.HEALTHY, EndpointStatus.DEGRADED]
            and self.llm is not None
            and self.metrics.active_requests  EndpointStatus:
        """
        Perform health check on endpoint.
        
        Returns:
            Current health status
        """
        try:
            if self.llm is None:
                self._initialize_llm()
            
            # Simple health check - try a minimal request
            start_time = time.time()
            self.llm.invoke("test")
            response_time = time.time() - start_time
            
            # Update status based on response time and error rate
            if response_time  95:
                self.metrics.status = EndpointStatus.HEALTHY
            elif response_time  80:
                self.metrics.status = EndpointStatus.DEGRADED
            else:
                self.metrics.status = EndpointStatus.UNHEALTHY
            
            self.last_health_check = datetime.now()
            return self.metrics.status
            
        except Exception as e:
            self.metrics.status = EndpointStatus.UNHEALTHY
            self.metrics.last_error = str(e)
            self.metrics.last_error_time = datetime.now()
            return EndpointStatus.UNHEALTHY
    
    def generate(self, prompt: str) -> Dict[str, Any]:
        """
        Generate response using this endpoint.
        
        Args:
            prompt: Input prompt
            
        Returns:
            Response dictionary
        """
        self.metrics.total_requests += 1
        self.metrics.active_requests += 1
        
        try:
            start_time = time.time()
            response = self.llm.invoke(prompt)
            response_time = time.time() - start_time
            
            self.metrics.successful_requests += 1
            self.metrics.total_response_time += response_time
            self.metrics.active_requests -= 1
            self.request_times.append(response_time)
            
            return {
                "response": response.content,
                "endpoint_id": self.config.id,
                "response_time": response_time,
                "success": True
            }
            
        except Exception as e:
            self.metrics.failed_requests += 1
            self.metrics.active_requests -= 1
            self.metrics.last_error = str(e)
            self.metrics.last_error_time = datetime.now()
            
            # Mark as unhealthy after repeated failures
            if self.metrics.success_rate  Optional[LLMEndpoint]:
        """Select endpoint using round-robin strategy"""
        attempts = 0
        while attempts  Optional[LLMEndpoint]:
        """Select endpoint using weighted round-robin strategy"""
        # Create weighted list
        weighted_endpoints = []
        for endpoint in self.endpoints:
            if endpoint.is_available():
                weighted_endpoints.extend([endpoint] * endpoint.config.weight)
        
        if not weighted_endpoints:
            return None
        
        # Round-robin through weighted list
        endpoint = weighted_endpoints[self.current_endpoint_index % len(weighted_endpoints)]
        self.current_endpoint_index += 1
        return endpoint
    
    def _select_endpoint_least_loaded(self) -> Optional[LLMEndpoint]:
        """Select endpoint with least active requests"""
        available = [e for e in self.endpoints if e.is_available()]
        if not available:
            return None
        
        return min(available, key=lambda e: e.metrics.active_requests)
    
    def _select_endpoint_random(self) -> Optional[LLMEndpoint]:
        """Select random available endpoint"""
        available = [e for e in self.endpoints if e.is_available()]
        if not available:
            return None
        
        return random.choice(available)
    
    def _select_endpoint_fastest(self) -> Optional[LLMEndpoint]:
        """Select endpoint with fastest average response time"""
        available = [e for e in self.endpoints if e.is_available()]
        if not available:
            return None
        
        # Sort by average response time (endpoints with no history get high priority)
        return min(
            available,
            key=lambda e: e.metrics.average_response_time if e.metrics.successful_requests > 0 else 0.0
        )
    
    def _select_endpoint(self) -> Optional[LLMEndpoint]:
        """
        Select endpoint based on strategy.
        
        Returns:
            Selected endpoint or None if none available
        """
        if self.strategy == LoadBalancingStrategy.ROUND_ROBIN:
            return self._select_endpoint_round_robin()
        elif self.strategy == LoadBalancingStrategy.WEIGHTED_ROUND_ROBIN:
            return self._select_endpoint_weighted()
        elif self.strategy == LoadBalancingStrategy.LEAST_LOADED:
            return self._select_endpoint_least_loaded()
        elif self.strategy == LoadBalancingStrategy.RANDOM:
            return self._select_endpoint_random()
        elif self.strategy == LoadBalancingStrategy.FASTEST_RESPONSE:
            return self._select_endpoint_fastest()
        else:
            return self._select_endpoint_round_robin()
    
    def _perform_health_checks(self):
        """Perform health checks on all endpoints if needed"""
        now = datetime.now()
        if (now - self.last_health_check).seconds  Dict[str, Any]:
        """
        Generate response with load balancing and failover.
        
        Args:
            prompt: Input prompt
            max_retries: Maximum retry attempts
            
        Returns:
            Response dictionary
        """
        self.stats.total_requests += 1
        
        # Perform health checks if needed
        self._perform_health_checks()
        
        attempts = 0
        last_error = None
        
        while attempts  Dict[str, Any]:
        """Get load balancer statistics"""
        return {
            "total_requests": self.stats.total_requests,
            "successful_requests": self.stats.successful_requests,
            "failed_requests": self.stats.failed_requests,
            "failovers": self.stats.failovers,
            "success_rate": (
                (self.stats.successful_requests / self.stats.total_requests * 100)
                if self.stats.total_requests > 0 else 0.0
            ),
            "average_response_time": (
                self.stats.total_response_time / self.stats.successful_requests
                if self.stats.successful_requests > 0 else 0.0
            ),
            "endpoints": [
                {
                    "id": e.config.id,
                    "model": e.config.model,
                    "status": e.metrics.status.value,
                    "total_requests": e.metrics.total_requests,
                    "success_rate": f"{e.metrics.success_rate:.2f}%",
                    "avg_response_time": f"{e.metrics.average_response_time:.3f}s",
                    "active_requests": e.metrics.active_requests
                }
                for e in self.endpoints
            ]
        }


def demonstrate_load_balancing():
    """Demonstrate load balancing patterns"""
    print("=" * 80)
    print("LOAD BALANCING PATTERN DEMONSTRATION")
    print("=" * 80)
    
    # Example 1: Basic Round-Robin Load Balancing
    print("\n" + "=" * 80)
    print("EXAMPLE 1: Round-Robin Load Balancing")
    print("=" * 80)
    
    # Configure multiple endpoints (simulating multiple API keys or models)
    endpoints = [
        EndpointConfig(id="endpoint-1", model="gpt-3.5-turbo", weight=1),
        EndpointConfig(id="endpoint-2", model="gpt-3.5-turbo", weight=1),
        EndpointConfig(id="endpoint-3", model="gpt-3.5-turbo", weight=1),
    ]
    
    lb = LoadBalancer(endpoints, strategy=LoadBalancingStrategy.ROUND_ROBIN)
    
    print("\nConfigured 3 endpoints with round-robin strategy\n")
    print("Sending 6 requests to observe distribution:\n")
    
    queries = [
        "What is 2+2?",
        "What is Python?",
        "What is AI?",
        "What is ML?",
        "What is DL?",
        "What is NLP?"
    ]
    
    for i, query in enumerate(queries, 1):
        print(f"Request {i}: {query}")
        result = lb.generate(query)
        
        if result["success"]:
            print(f"  ✓ Endpoint: {result['endpoint_id']}")
            print(f"  Response time: {result['response_time']:.3f}s")
            print(f"  Response: {result['response'][:60]}...")
        else:
            print(f"  ✗ Error: {result['error']}")
        print()
    
    stats = lb.get_stats()
    print("Load Distribution:")
    for endpoint in stats["endpoints"]:
        print(f"  {endpoint['id']}: {endpoint['total_requests']} requests")
    
    # Example 2: Weighted Load Balancing
    print("\n" + "=" * 80)
    print("EXAMPLE 2: Weighted Load Balancing")
    print("=" * 80)
    
    weighted_endpoints = [
        EndpointConfig(id="premium-endpoint", model="gpt-4", weight=3),  # Gets more traffic
        EndpointConfig(id="standard-endpoint", model="gpt-3.5-turbo", weight=1),
    ]
    
    weighted_lb = LoadBalancer(
        weighted_endpoints,
        strategy=LoadBalancingStrategy.WEIGHTED_ROUND_ROBIN
    )
    
    print("\nConfigured 2 endpoints:")
    print("  - Premium (GPT-4): weight=3")
    print("  - Standard (GPT-3.5): weight=1")
    print("\nSending 8 requests:\n")
    
    for i in range(8):
        result = weighted_lb.generate(f"Request {i+1}")
        if result["success"]:
            print(f"Request {i+1} → {result['endpoint_id']}")
    
    print("\nExpected distribution: ~6 to premium, ~2 to standard")
    stats = weighted_lb.get_stats()
    for endpoint in stats["endpoints"]:
        print(f"  {endpoint['id']}: {endpoint['total_requests']} requests")
    
    # Example 3: Least-Loaded Strategy
    print("\n" + "=" * 80)
    print("EXAMPLE 3: Least-Loaded Strategy")
    print("=" * 80)
    
    ll_endpoints = [
        EndpointConfig(id="endpoint-A", model="gpt-3.5-turbo"),
        EndpointConfig(id="endpoint-B", model="gpt-3.5-turbo"),
    ]
    
    ll_lb = LoadBalancer(ll_endpoints, strategy=LoadBalancingStrategy.LEAST_LOADED)
    
    print("\nLeast-loaded strategy routes to endpoint with fewest active requests")
    print("This optimizes for load distribution in real-time\n")
    
    for i in range(4):
        result = ll_lb.generate(f"Query {i+1}")
        if result["success"]:
            print(f"Query {i+1} → {result['endpoint_id']}")
    
    # Example 4: Failover Handling
    print("\n" + "=" * 80)
    print("EXAMPLE 4: Automatic Failover")
    print("=" * 80)
    
    print("\nSimulating endpoint failure scenario:")
    print("  - 3 endpoints configured")
    print("  - Mark one as unhealthy")
    print("  - Load balancer automatically routes around it\n")
    
    failover_endpoints = [
        EndpointConfig(id="healthy-1", model="gpt-3.5-turbo"),
        EndpointConfig(id="unhealthy", model="gpt-3.5-turbo"),
        EndpointConfig(id="healthy-2", model="gpt-3.5-turbo"),
    ]
    
    fo_lb = LoadBalancer(failover_endpoints, strategy=LoadBalancingStrategy.ROUND_ROBIN)
    
    # Simulate endpoint failure
    fo_lb.endpoints[1].metrics.status = EndpointStatus.UNHEALTHY
    print("Marked 'unhealthy' endpoint as UNHEALTHY\n")
    
    print("Sending 4 requests:\n")
    for i in range(4):
        result = fo_lb.generate(f"Query {i+1}")
        if result["success"]:
            print(f"Query {i+1} → {result['endpoint_id']} ✓")
    
    print("\nOnly healthy endpoints received requests!")
    
    # Example 5: Performance Comparison
    print("\n" + "=" * 80)
    print("EXAMPLE 5: Strategy Performance Comparison")
    print("=" * 80)
    
    strategies = [
        LoadBalancingStrategy.ROUND_ROBIN,
        LoadBalancingStrategy.RANDOM,
        LoadBalancingStrategy.LEAST_LOADED,
    ]
    
    print("\nComparing different load balancing strategies:\n")
    
    for strategy in strategies:
        test_endpoints = [
            EndpointConfig(id=f"ep-{i}", model="gpt-3.5-turbo")
            for i in range(3)
        ]
        
        test_lb = LoadBalancer(test_endpoints, strategy=strategy)
        
        # Send some requests
        for i in range(6):
            test_lb.generate(f"Test query {i}")
        
        stats = test_lb.get_stats()
        print(f"{strategy.value.upper()}:")
        print(f"  Success Rate: {stats['success_rate']:.2f}%")
        print(f"  Total Requests: {stats['total_requests']}")
        for ep in stats["endpoints"]:
            print(f"    {ep['id']}: {ep['total_requests']} requests")
        print()
    
    # Example 6: Monitoring and Statistics
    print("\n" + "=" * 80)
    print("EXAMPLE 6: Comprehensive Statistics")
    print("=" * 80)
    
    monitor_endpoints = [
        EndpointConfig(id="prod-1", model="gpt-3.5-turbo"),
        EndpointConfig(id="prod-2", model="gpt-3.5-turbo"),
    ]
    
    monitor_lb = LoadBalancer(monitor_endpoints, strategy=LoadBalancingStrategy.ROUND_ROBIN)
    
    print("\nSimulating production workload...\n")
    
    # Simulate various scenarios
    for i in range(10):
        result = monitor_lb.generate(f"Production query {i+1}")
    
    stats = monitor_lb.get_stats()
    
    print("Load Balancer Statistics:")
    print(f"  Total Requests: {stats['total_requests']}")
    print(f"  Successful: {stats['successful_requests']}")
    print(f"  Failed: {stats['failed_requests']}")
    print(f"  Success Rate: {stats['success_rate']:.2f}%")
    print(f"  Failovers: {stats['failovers']}")
    print(f"  Avg Response Time: {stats['average_response_time']:.3f}s")
    
    print("\nPer-Endpoint Metrics:")
    for ep in stats["endpoints"]:
        print(f"\n  {ep['id']} ({ep['model']}):")
        print(f"    Status: {ep['status']}")
        print(f"    Requests: {ep['total_requests']}")
        print(f"    Success Rate: {ep['success_rate']}")
        print(f"    Avg Response Time: {ep['avg_response_time']}")
        print(f"    Active Requests: {ep['active_requests']}")
    
    # Summary
    print("\n" + "=" * 80)
    print("LOAD BALANCING PATTERN SUMMARY")
    print("=" * 80)
    print("""
Load Balancing Benefits:
1. High Availability: 99.9%+ uptime through redundancy
2. Performance: Better throughput via parallel processing
3. Fault Tolerance: Automatic failover on endpoint failure
4. Scalability: Easy to add more endpoints
5. Cost Optimization: Route based on cost/performance
6. Rate Limit Management: Distribute across API keys

Load Balancing Strategies:
1. Round Robin
   - Equal distribution across endpoints
   - Simple and predictable
   - Good for homogeneous endpoints
   - Use when: All endpoints have similar capacity

2. Weighted Round Robin
   - Priority-based distribution
   - Higher weight = more requests
   - Good for heterogeneous endpoints
   - Use when: Different endpoint capacities

3. Least Loaded
   - Route to least busy endpoint
   - Optimizes for current load
   - Dynamic adjustment
   - Use when: Variable request durations

4. Random
   - Random endpoint selection
   - Stateless and simple
   - Good load distribution over time
   - Use when: Simplicity preferred

5. Fastest Response
   - Route to historically fastest endpoint
   - Performance-optimized
   - Adaptive to endpoint characteristics
   - Use when: Latency is critical

Health Checking:
1. Active Health Checks
   - Regular test requests
   - Verify endpoint availability
   - Monitor response times
   - Interval: 30-60 seconds

2. Passive Health Monitoring
   - Track request success/failure
   - Monitor error rates
   - Calculate response times
   - Continuous monitoring

3. Status Levels
   - HEALTHY:  95% success
   - DEGRADED:  80% success
   - UNHEALTHY: Slow or 1000 req/hr)
✓ SLA requirements (>99% uptime)
✓ Rate limit concerns
✓ Multi-provider deployments
✓ Geographic distribution needed
✗ Low-traffic applications
✗ Single endpoint sufficient
✗ Cost constraints (added complexity)

ROI Analysis:
- Uptime improvement: 99% → 99.9% (10x fewer outages)
- Performance: 2-3x throughput capacity
- User satisfaction: Reduced error rates
- Cost: +20% complexity, but worth it for scale
- Break-even: ~500 requests/hour
""")
    
    print("\n" + "=" * 80)
    print("Pattern 090 (Load Balancing) demonstration complete!")
    print("=" * 80)


if __name__ == "__main__":
    demonstrate_load_balancing()

    Agentic Design Pattern: Load Balancing

    Pattern 090: Load Balancing – Line-by-Line Explanation

    Overview

    This document provides a detailed line-by-line explanation of the Load Balancing pattern implementation.

    Pattern Description

    Pattern 090: Load Balancing Pattern

    Description: The Load Balancing Pattern distributes requests across multiple LLM endpoints or instances to achieve high availability, improved performance, and fault tolerance. In production LLM applications, load balancing is critical for handling high traffic volumes, preventing single points of failure, and ensuring consistent service quality even when individual endpoints fail or become overloaded.

    Load balancing strategies enable:
- High availability through redundancy
- Improved performance via parallel processing
- Cost optimization through provider selection
- Rate limit management across multiple accounts
- Geographic distribution for lower latency
- Failover and disaster recovery

    Components: 1. Load Balancing Strategies – Round Robin: Distribute equally across endpoints – Weighted Round Robin: Priority-based distribution – Least Loaded: Send to endpoint with lowest load – Random: Random endpoint selection – Geographic: Route based on location – Cost-Based: Choose cheapest available endpoint

    2. Health Checking
   - Endpoint availability monitoring
   - Response time tracking
   - Error rate monitoring
   - Periodic health probes
   - Automatic endpoint removal/recovery

3. Request Management
   - Request queuing and buffering
   - Retry logic with backoff
   - Circuit breaker pattern
   - Request timeout handling
   - Priority-based routing

4. Failover Mechanisms
   - Automatic endpoint switching
   - Degraded mode operation
   - Fallback to alternative providers
   - Request replay on failure
   - State preservation

    Use Cases: 1. High-Traffic Applications – Chatbots serving thousands of users – API services with SLA requirements – Real-time processing pipelines – Multi-tenant platforms – Peak load handling

    2. Multi-Provider Deployments
   - OpenAI + Anthropic + Google
   - Primary and backup providers
   - Cost optimization strategies
   - Feature-specific routing
   - Geographic compliance

3. Rate Limit Management
   - Distribute across multiple API keys
   - Prevent rate limit exhaustion
   - Smooth request distribution
   - Burst handling
   - Cost control

4. Disaster Recovery
   - Automatic failover
   - Multi-region deployment
   - Zero-downtime updates
   - Service degradation handling
   - Business continuity

5. Performance Optimization
   - Parallel request processing
   - Load-based routing
   - Latency optimization
   - Resource utilization
   - Capacity planning

    LangChain Implementation: LangChain doesn’t have built-in load balancing, but you can implement it using: – Custom LLM wrappers – Callback handlers for monitoring – Retry logic with different models – Custom routing logic – Health check mechanisms

    Key Features: 1. Endpoint Management – Dynamic endpoint registration – Health status tracking – Automatic recovery – Configuration updates – Version management

    2. Intelligent Routing
   - Strategy selection
   - Real-time decision making
   - Performance-based routing
   - Cost optimization
   - SLA enforcement

3. Fault Tolerance
   - Automatic retry with exponential backoff
   - Circuit breaker to prevent cascading failures
   - Graceful degradation
   - Error handling and recovery
   - Request preservation

4. Monitoring
   - Request distribution metrics
   - Endpoint health status
   - Response time tracking
   - Error rate monitoring
   - Cost tracking per endpoint

    Best Practices: 1. Health Monitoring – Regular health checks (every 30-60s) – Multiple health metrics (latency, errors, availability) – Automatic endpoint recovery – Alert on degraded performance – Log all health transitions

    2. Load Distribution
   - Consider endpoint capacity
   - Account for different model speeds
   - Balance cost vs performance
   - Avoid overloading any single endpoint
   - Reserve capacity for spikes

3. Failover Strategy
   - Fast failure detection (< 5s)
   - Quick failover (< 1s)
   - Preserve request context
   - Transparent to users
   - Log all failovers

4. Request Management
   - Set appropriate timeouts
   - Implement retry with backoff
   - Queue management for spikes
   - Priority handling
   - Request deduplication

    Trade-offs: Advantages: – High availability (99.9%+ uptime) – Better performance through parallelization – Fault tolerance and resilience – Flexible scaling – Cost optimization opportunities – Geographic distribution

    Disadvantages:
- Increased complexity
- Additional monitoring requirements
- Potential consistency issues
- Higher operational costs
- Configuration management overhead
- Testing complexity

    Production Considerations: 1. Endpoint Configuration – Multiple providers (OpenAI, Anthropic, etc.) – Multiple API keys per provider – Geographic distribution – Capacity planning – Cost monitoring

    2. Monitoring and Alerting
   - Real-time health dashboards
   - Performance metrics
   - Error tracking
   - Cost tracking
   - SLA compliance monitoring

3. Security
   - Secure credential management
   - Rate limiting per client
   - Access control
   - Audit logging
   - Encryption in transit

4. Scaling
   - Horizontal scaling capability
   - Auto-scaling based on load
   - Capacity planning
   - Performance testing
   - Load testing

5. Disaster Recovery
   - Multi-region deployment
   - Backup endpoints
   - Failover procedures
   - Data consistency
   - Recovery time objectives (RTO)

    File Statistics

    • Total Lines: 988
    • Classes: 7
    • Functions: 1
    • Constants: 0

    Module Documentation

    """
Pattern 090: Load Balancing Pattern

Description:
    The Load Balancing Pattern distributes requests across multiple LLM endpoints or
    instances to achieve high availability, improved performance, and fault tolerance.
    In production LLM applications, load balancing is critical for handling high traffic
    volumes, preventing single points of failure, and ensuring consistent service
    quality even when individual endpoints fail or become overloaded.

    Load balancing strategies enable:
    - High availability through redundancy
    - Improved performance via parallel processing
    - Cost optimization through provider selection
    - Rate limit management across multiple accounts
    - Geographic distribution for lower latency
    - Failover and disaster recovery

Components:
    1. Load Balancing Strategies
       - Round Robin: Distribute equally across endpoints
       - Weighted Round Robin: Priority-based distribution
       - Least Loaded: Send to endpoint with lowest load
       - Random: Random endpoint selection
       - Geographic: Route based on location
       - Cost-Based: Choose cheapest available endpoint

    2. Health Checking
       - Endpoint availability monitoring
       - Response time tracking
       - Error rate monitoring
       - Periodic health probes
       - Automatic endpoint removal/recovery

    3. Request Management
       - Request queuing and buffering
       - Retry logic with backoff
       - Circuit breaker pattern
       - Request timeout handling
       - Priority-based routing

    4. Failover Mechanisms
       - Automatic endpoint switching
       - Degraded mode operation
       - Fallback to alternative providers
       - Request replay on failure
       - State preservation

Use Cases:
    1. High-Traffic Applications
       - Chatbots serving thousands of users
       - API services with SLA requirements
       - Real-time processing pipelines
       - Multi-tenant platforms
       - Peak load handling

    2. Multi-Provider Deployments
       - OpenAI + Anthropic + Google
       - Primary and backup providers
       - Cost optimization strategies
       - Feature-specific routing
       - Geographic compliance

    3. Rate Limit Management
       - Distribute across multiple API keys
       - Prevent rate limit exhaustion
       - Smooth request distribution
       - Burst handling
       - Cost control

    4. Disaster Recovery
       - Automatic failover
       - Multi-region deployment
       - Zero-downtime updates
       - Service degradation handling
       - Business continuity

    5. Performance Optimization
       - Parallel request processing
       - Load-based routing
       - Latency optimization
       - Resource utilization
       - Capacity planning

LangChain Implementation:
    LangChain doesn't have built-in load balancing, but you can implement
    it using:
    - Custom LLM wrappers
    - Callback handlers for monitoring
    - Retry logic with different models
    - Custom routing logic
    - Health check mechanisms

Key Features:
    1. Endpoint Management
       - Dynamic endpoint registration
       - Health status tracking
       - Automatic recovery
       - Configuration updates
       - Version management

    2. Intelligent Routing
       - Strategy selection
       - Real-time decision making
       - Performance-based routing
       - Cost optimization
       - SLA enforcement

    3. Fault Tolerance
       - Automatic retry with exponential backoff
       - Circuit breaker to prevent cascading failures
       - Graceful degradation
       - Error handling and recovery
       - Request preservation

    4. Monitoring
       - Request distribution metrics
       - Endpoint health status
       - Response time tracking
       - Error rate monitoring
       - Cost tracking per endpoint

Best Practices:
    1. Health Monitoring
       - Regular health checks (every 30-60s)
       - Multiple health metrics (latency, errors, availability)
       - Automatic endpoint recovery
       - Alert on degraded performance
       - Log all health transitions

    2. Load Distribution
       - Consider endpoint capacity
       - Account for different model speeds
       - Balance cost vs performance
       - Avoid overloading any single endpoint
       - Reserve capacity for spikes

    3. Failover Strategy
       - Fast failure detection (< 5s)
       - Quick failover (< 1s)
       - Preserve request context
       - Transparent to users
       - Log all failovers

    4. Request Management
       - Set appropriate timeouts
       - Implement retry with backoff
       - Queue management for spikes
       - Priority handling
       - Request deduplication

Trade-offs:
    Advantages:
    - High availability (99.9%+ uptime)
    - Better performance through parallelization
    - Fault tolerance and resilience
    - Flexible scaling
    - Cost optimization opportunities
    - Geographic distribution

    Disadvantages:
    - Increased complexity
    - Additional monitoring requirements
    - Potential consistency issues
    - Higher operational costs
    - Configuration management overhead
    - Testing complexity

Production Considerations:
    1. Endpoint Configuration
       - Multiple providers (OpenAI, Anthropic, etc.)
       - Multiple API keys per provider
       - Geographic distribution
       - Capacity planning
       - Cost monitoring

    2. Monitoring and Alerting
       - Real-time health dashboards
       - Performance metrics
       - Error tracking
       - Cost tracking
       - SLA compliance monitoring

    3. Security
       - Secure credential management
       - Rate limiting per client
       - Access control
       - Audit logging
       - Encryption in transit

    4. Scaling
       - Horizontal scaling capability
       - Auto-scaling based on load
       - Capacity planning
       - Performance testing
       - Load testing

    5. Disaster Recovery
       - Multi-region deployment
       - Backup endpoints
       - Failover procedures
       - Data consistency
       - Recovery time objectives (RTO)
"""

    Purpose: This module implements the Load Balancing pattern, providing a structured approach to agent design and implementation.

    Import Statements

    Standard Library Imports

    import collections.deque
import dataclasses.dataclass
import dataclasses.field
import datetime.datetime
import datetime.timedelta
import enum.Enum
import langchain_core.prompts.ChatPromptTemplate
import os
import random
import time
import typing.Any
import typing.Callable
import typing.Dict
import typing.List
import typing.Optional

    Purpose: Core Python functionality: – collections.deque: Specialized container datatypes – dataclasses.dataclass: Simplified class definitions with @dataclass – dataclasses.field: Simplified class definitions with @dataclass – datetime.datetime: Date and time handling – datetime.timedelta: Date and time handling – enum.Enum: Enumeration support – langchain_core.prompts.ChatPromptTemplate: Regular expression operations – os: Environment variable access and system operations – random: Random number generation – time: Time-related functions – typing.Any: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Callable: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Dict: Type hints for better code documentation (List, Dict, Any, Optional) – typing.List: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Optional: Type hints for better code documentation (List, Dict, Any, Optional)

    LangChain Imports

    from langchain_openai.ChatOpenAI

    Purpose: LangChain framework components: – LLM Interface: Wrapper for OpenAI models – Chains: Composable building blocks for complex workflows

    Other Imports

    from dotenv.load_dotenv

    Purpose: Additional dependencies for specialized functionality.

    Classes

    Class: LoadBalancingStrategy

    Lines 222-228

    Inherits from: Enum

    Purpose:

    Load balancing strategies

    Class: EndpointStatus

    Lines 231-236

    Inherits from: Enum

    Purpose:

    Endpoint health status

    Class: EndpointConfig

    Lines 240-248

    Purpose:

    Configuration for an LLM endpoint

    Class: EndpointMetrics

    Lines 252-275

    Purpose:

    Metrics for an endpoint

    Methods:

    success_rate(self)

    Lines 264-268

    Decorators: property

    Purpose:

    Calculate success rate

    Parameters:

    Returns: float

    average_response_time(self)

    Lines 271-275

    Decorators: property

    Purpose:

    Calculate average response time

    Parameters:

    Returns: float

    Class: LoadBalancerStats

    Lines 279-286

    Purpose:

    Overall load balancer statistics

    Class: LLMEndpoint

    Lines 289-414

    Purpose:

    Represents a single LLM endpoint with health tracking.

    Methods:

    __init__(self, config)

    Lines 294-307

    Purpose:

    Initialize endpoint.

Args:
    config: Endpoint configuration

    Parameters: – config: Config

    _initialize_llm(self)

    Lines 309-326

    Purpose:

    Initialize the LLM client

    Parameters:

    is_available(self)

    Lines 328-334

    Purpose:

    Check if endpoint is available for requests

    Parameters:

    Returns: bool

    check_health(self)

    Lines 336-367

    Purpose:

    Perform health check on endpoint.

Returns:
    Current health status

    Parameters:

    Returns: EndpointStatus

    generate(self, prompt)

    Lines 369-414

    Purpose:

    Generate response using this endpoint.

Args:
    prompt: Input prompt

Returns:
    Response dictionary

    Parameters: – prompt: Prompt template or text

    Returns: <ast.Subscript object at 0x000001A06D7523D0>

    Class: LoadBalancer

    Lines 417-622

    Purpose:

    Load balancer for distributing requests across multiple LLM endpoints.

Supports multiple load balancing strategies and automatic failover.

    Methods:

    __init__(self, endpoints, strategy, health_check_interval)

    Lines 424-449

    Purpose:

    Initialize load balancer.

Args:
    endpoints: List of endpoint configurations
    strategy: Load balancing strategy
    health_check_interval: Seconds between health checks

    Parameters: – endpoints: Endpoints – strategy: Strategy – health_check_interval: Health check interval

    _select_endpoint_round_robin(self)

    Lines 451-463

    Purpose:

    Select endpoint using round-robin strategy

    Parameters:

    Returns: <ast.Subscript object at 0x000001A06D8881D0>

    _select_endpoint_weighted(self)

    Lines 465-479

    Purpose:

    Select endpoint using weighted round-robin strategy

    Parameters:

    Returns: <ast.Subscript object at 0x000001A06D88B410>

    _select_endpoint_least_loaded(self)

    Lines 481-487

    Purpose:

    Select endpoint with least active requests

    Parameters:

    Returns: <ast.Subscript object at 0x000001A06D75B250>

    _select_endpoint_random(self)

    Lines 489-495

    Purpose:

    Select random available endpoint

    Parameters:

    Returns: <ast.Subscript object at 0x000001A06D75A790>

    _select_endpoint_fastest(self)

    Lines 497-507

    Purpose:

    Select endpoint with fastest average response time

    Parameters:

    Returns: <ast.Subscript object at 0x000001A06D80CF90>

    _select_endpoint(self)

    Lines 509-527

    Purpose:

    Select endpoint based on strategy.

Returns:
    Selected endpoint or None if none available

    Parameters:

    Returns: <ast.Subscript object at 0x000001A06D157C90>

    _perform_health_checks(self)

    Lines 529-538

    Purpose:

    Perform health checks on all endpoints if needed

    Parameters:

    generate(self, prompt, max_retries)

    Lines 540-593

    Purpose:

    Generate response with load balancing and failover.

Args:
    prompt: Input prompt
    max_retries: Maximum retry attempts

Returns:
    Response dictionary

    Parameters: – prompt: Prompt template or text – max_retries: Max retries

    Returns: <ast.Subscript object at 0x000001A06D83E910>

    get_stats(self)

    Lines 595-622

    Purpose:

    Get load balancer statistics

    Parameters:

    Returns: <ast.Subscript object at 0x000001A06D850690>

    Functions

    demonstrate_load_balancing()

    Lines 625-983

    Purpose:

    Demonstrate load balancing patterns

    Key Concepts Summary

    Pattern Overview

    The Load Balancing pattern provides a structured approach to building AI agents with specific capabilities and behaviors. This implementation leverages LangChain’s composable components to create a robust and maintainable solution.

    Core Components

    1. Agent Class: Main implementation of the pattern logic
    2. Prompt Engineering: Carefully crafted prompts to guide LLM behavior
    3. Tool Integration: Optional tools for extended functionality
    4. Memory Management: State handling across interactions
    5. Output Parsing: Structured response processing

    When to Use

    ✅ Good for: – Scenarios requiring this specific pattern’s strengths – Applications needing structured agent behavior – Systems where explainability is important – Production environments requiring reliability

    ❌ Not ideal for: – Simple single-prompt tasks – Scenarios requiring different pattern characteristics – Resource-constrained environments (if computationally intensive)

    Best Practices Demonstrated

    1. Environment Configuration

    • ✅ Use .env files for API keys
    • ✅ Never hardcode credentials
    • ✅ Validate environment variables on startup

    2. Error Handling

    • ✅ Graceful degradation on failures
    • ✅ Informative error messages
    • ✅ Logging for debugging

    3. Prompt Engineering

    • ✅ Clear, specific instructions
    • ✅ Structured output formats
    • ✅ Few-shot examples when beneficial

    4. Code Organization

    • ✅ Separation of concerns (class-based design)
    • ✅ Type hints for clarity
    • ✅ Docstrings for documentation

    5. Performance Optimization

    • ✅ Appropriate temperature settings
    • ✅ Token budget management
    • ✅ Caching where applicable

    LangChain Components Used

    • ChatOpenAI: OpenAI model interface
    • Chains: Sequential component composition

    Why LangChain? – Composable abstractions for LLM applications – Rich ecosystem of integrations – Production-ready patterns – Active community and development

    Execution Flow

    1. Initialize agent with configuration
   ↓
2. Load environment and dependencies
   ↓
3. Prepare prompt with input data
   ↓
4. Invoke LLM through LangChain
   ↓
5. Parse and validate response
   ↓
6. Return formatted result

    Key Decision Points: – Model selection (temperature, model type) – Prompt structure (few-shot vs zero-shot) – Error handling strategy – Output format validation

    Performance Considerations

    Computational Cost

    • LLM Calls: Primary cost factor (based on tokens)
    • Latency: Depends on model and input size
    • Memory: Minimal for most patterns

    Optimization Strategies

    1. Prompt Optimization: Minimize tokens while maintaining clarity
    2. Caching: Cache repeated queries/responses
    3. Model Selection: Use smaller models when appropriate
    4. Batch Processing: Group similar requests
    5. Streaming: For long responses, stream tokens

    Cost Estimates (Approximate)

    • GPT-3.5-turbo: ~$0.002 per request
    • GPT-4: ~$0.03 per request
    • GPT-4-turbo: ~$0.01 per request

    Actual costs vary based on input/output token counts

    Common Pitfalls to Avoid

    1. Prompt Design Issues

    ❌ Vague or ambiguous instructions
    ✅ Clear, specific, testable prompts

    2. Error Handling

    ❌ Ignoring API errors or rate limits
    ✅ Implement retry logic and graceful degradation

    3. Token Management

    ❌ Exceeding model token limits
    ✅ Truncate inputs, chunk processing

    4. Security

    ❌ Exposing API keys in code
    ✅ Use environment variables

    5. Testing

    ❌ Only testing happy paths
    ✅ Test edge cases, failures, malformed inputs

    6. Cost Management

    ❌ Unmonitored API usage
    ✅ Implement budgets, alerts, and monitoring

    Further Improvements

    Production Enhancements

    1. Monitoring: Add logging, metrics, tracing
    2. Validation: Input/output schema validation
    3. Rate Limiting: Implement request throttling
    4. Caching: Redis or in-memory cache for responses
    5. Async Support: Non-blocking operations for scalability

    Feature Extensions

    1. Multi-Model Support: Switch between providers
    2. Fallback Models: Graceful degradation strategy
    3. Custom Tools: Domain-specific function calling
    4. Advanced Memory: Vector stores for long-term memory
    5. Evaluation: Automated testing and benchmarking

    Code Quality

    1. Testing: Unit tests, integration tests
    2. Documentation: API docs, usage examples
    3. Type Safety: Stricter type hints
    4. Configuration: Externalized config files

    Related Patterns

    • Pattern 089
    • Pattern 091

    Pattern Families: – Reasoning Patterns (001-011) – Multi-Agent Patterns (012-019) – Tool Use Patterns (020-025) – Memory Patterns (026-031) – And many more…

    Summary

    The Load Balancing pattern provides a powerful approach to building AI agents with specific capabilities. This implementation demonstrates:

    ✅ Key Strengths: – Clean, maintainable code structure – LangChain integration for composability – Type hints and documentation – Production-ready patterns

    🎯 Best Use Cases: – Applications requiring this pattern’s characteristics – Systems needing structured agent behavior – Production environments requiring reliability

    📚 Learning Resources: – LangChain Documentation: https://python.langchain.com/ – Pattern Documentation: See related patterns – OpenAI API Reference: https://platform.openai.com/docs/

    File: 090_load_balancing.py
    Total Lines: 988
    Pattern Category: AI Agent Design Pattern #090

    Generated from actual implementation code
    For educational and reference purposes

  • Agentic Design Pattern: Caching Patterns

    
    """
Pattern 089: Caching Patterns

Description:
    The Caching Patterns involve storing and reusing previously computed results to
    improve performance, reduce latency, and minimize costs. In the context of LLM
    applications, caching is critical for optimizing response times and reducing API
    calls for repeated or similar queries. Effective caching strategies can
    dramatically improve user experience while significantly cutting operational costs.

    Caching in LLM applications targets:
    - Complete responses (exact match caching)
    - Embeddings (vector representations)
    - Prompt templates (reusable structures)
    - Intermediate results (multi-step workflows)
    - API responses (rate limit optimization)

Components:
    1. Cache Types
       - In-memory cache (fast, volatile)
       - Disk cache (persistent, slower)
       - Distributed cache (Redis, Memcached)
       - Database cache (structured, queryable)
       - CDN cache (geographic distribution)

    2. Cache Keys
       - Exact match (full prompt hash)
       - Semantic match (embedding similarity)
       - Parametric keys (template + parameters)
       - Hierarchical keys (nested cache structures)
       - Time-based keys (temporal validity)

    3. Cache Policies
       - TTL (Time To Live) - expire after time
       - LRU (Least Recently Used) - evict old entries
       - LFU (Least Frequently Used) - evict unpopular
       - Size-based - limit total cache size
       - Custom policies - application-specific

    4. Invalidation Strategies
       - Time-based expiration
       - Event-triggered invalidation
       - Manual invalidation
       - Cascade invalidation (dependencies)
       - Version-based invalidation

Use Cases:
    1. Repeated Queries
       - FAQ responses
       - Common questions
       - Popular content
       - Standard procedures
       - Frequently accessed data

    2. Expensive Operations
       - Complex reasoning chains
       - Multi-step workflows
       - Large context processing
       - Batch operations
       - Resource-intensive computations

    3. Rate Limit Optimization
       - Reduce API calls
       - Stay within quotas
       - Handle traffic spikes
       - Cost optimization
       - Performance consistency

    4. Embeddings and Vectors
       - Document embeddings
       - Query embeddings
       - Semantic search optimization
       - Similarity computations
       - Vector database queries

    5. Multi-User Applications
       - Shared responses
       - Common workflows
       - Popular queries
       - Template responses
       - Collaborative features

LangChain Implementation:
    LangChain supports caching through:
    - InMemoryCache for simple caching
    - SQLiteCache for persistent caching
    - Custom cache implementations
    - Semantic similarity caching
    - Embedding caching

Key Features:
    1. Multiple Cache Layers
       - L1: In-memory (fastest)
       - L2: Local disk (persistent)
       - L3: Distributed cache (shared)
       - L4: Database (long-term)

    2. Intelligent Matching
       - Exact match for identical queries
       - Semantic match for similar queries
       - Fuzzy match with thresholds
       - Parametric templates

    3. Performance Optimization
       - Fast lookup ( 70%)
       - Measure latency improvements
       - Calculate cost savings
       - Identify cache misses patterns

    4. Handle Cache Misses
       - Graceful fallback to computation
       - Transparent to users
       - Background cache warming
       - Predictive pre-caching

Trade-offs:
    Advantages:
    - Dramatically improved response times
    - Significant cost reduction
    - Better user experience
    - Reduced API rate limiting
    - Lower resource consumption
    - Improved scalability

    Disadvantages:
    - Stale data risk
    - Memory/storage overhead
    - Cache consistency challenges
    - Complexity in invalidation
    - Initial cache warming time
    - Debugging complexity

Production Considerations:
    1. Cache Size Management
       - Monitor memory usage
       - Implement eviction policies
       - Set size limits
       - Regular cleanup
       - Compression strategies

    2. Cache Consistency
       - Version tracking
       - Update propagation
       - Invalidation timing
       - Distributed consistency
       - Race condition handling

    3. Security
       - Access control for cached data
       - Encryption for sensitive content
       - Cache poisoning prevention
       - User isolation in multi-tenant
       - Audit trails

    4. Monitoring
       - Hit/miss rates
       - Cache size trends
       - Eviction frequency
       - Cost savings
       - Performance impact

    5. Disaster Recovery
       - Cache rebuild procedures
       - Backup strategies
       - Failover mechanisms
       - Data loss handling
       - Recovery time objectives
"""

import os
import time
import hashlib
import json
from typing import List, Dict, Any, Optional, Callable
from dataclasses import dataclass, field
from enum import Enum
from datetime import datetime, timedelta
from collections import OrderedDict
from dotenv import load_dotenv

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_openai import ChatOpenAI, OpenAIEmbeddings
from langchain_core.caches import InMemoryCache
from langchain.globals import set_llm_cache

load_dotenv()


class CachePolicy(Enum):
    """Cache eviction policies"""
    LRU = "lru"  # Least Recently Used
    LFU = "lfu"  # Least Frequently Used
    FIFO = "fifo"  # First In First Out
    TTL = "ttl"  # Time To Live


@dataclass
class CacheEntry:
    """Represents a cached entry"""
    key: str
    value: Any
    created_at: datetime
    last_accessed: datetime
    access_count: int = 0
    ttl: Optional[int] = None  # TTL in seconds
    
    def is_expired(self) -> bool:
        """Check if entry is expired"""
        if self.ttl is None:
            return False
        age = (datetime.now() - self.created_at).total_seconds()
        return age > self.ttl
    
    def touch(self):
        """Update access information"""
        self.last_accessed = datetime.now()
        self.access_count += 1


@dataclass
class CacheStats:
    """Cache performance statistics"""
    hits: int = 0
    misses: int = 0
    evictions: int = 0
    total_requests: int = 0
    
    @property
    def hit_rate(self) -> float:
        """Calculate cache hit rate"""
        if self.total_requests == 0:
            return 0.0
        return (self.hits / self.total_requests) * 100
    
    @property
    def miss_rate(self) -> float:
        """Calculate cache miss rate"""
        return 100.0 - self.hit_rate


class SimpleCache:
    """
    Simple in-memory cache with multiple eviction policies.
    
    This cache supports:
    1. Multiple eviction policies (LRU, LFU, FIFO, TTL)
    2. Size limits
    3. TTL per entry
    4. Performance statistics
    """
    
    def __init__(
        self,
        max_size: int = 1000,
        policy: CachePolicy = CachePolicy.LRU,
        default_ttl: Optional[int] = None
    ):
        """
        Initialize cache.
        
        Args:
            max_size: Maximum number of entries
            policy: Eviction policy
            default_ttl: Default TTL in seconds
        """
        self.max_size = max_size
        self.policy = policy
        self.default_ttl = default_ttl
        self.cache: OrderedDict[str, CacheEntry] = OrderedDict()
        self.stats = CacheStats()
    
    def _make_key(self, key: Any) -> str:
        """Create cache key from input"""
        if isinstance(key, str):
            return hashlib.md5(key.encode()).hexdigest()
        return hashlib.md5(json.dumps(key, sort_keys=True).encode()).hexdigest()
    
    def _evict(self):
        """Evict entries based on policy"""
        if len(self.cache)  Optional[Any]:
        """
        Get value from cache.
        
        Args:
            key: Cache key
            
        Returns:
            Cached value or None
        """
        self.stats.total_requests += 1
        cache_key = self._make_key(key)
        
        # Remove expired entries
        self._remove_expired()
        
        if cache_key in self.cache:
            entry = self.cache[cache_key]
            
            if entry.is_expired():
                del self.cache[cache_key]
                self.stats.misses += 1
                return None
            
            entry.touch()
            
            # Move to end for LRU
            if self.policy == CachePolicy.LRU:
                self.cache.move_to_end(cache_key)
            
            self.stats.hits += 1
            return entry.value
        
        self.stats.misses += 1
        return None
    
    def set(self, key: Any, value: Any, ttl: Optional[int] = None):
        """
        Set value in cache.
        
        Args:
            key: Cache key
            value: Value to cache
            ttl: Optional TTL in seconds
        """
        cache_key = self._make_key(key)
        
        # Evict if necessary
        if cache_key not in self.cache:
            self._evict()
        
        entry = CacheEntry(
            key=cache_key,
            value=value,
            created_at=datetime.now(),
            last_accessed=datetime.now(),
            ttl=ttl or self.default_ttl
        )
        
        self.cache[cache_key] = entry
        
        # Move to end for LRU
        if self.policy == CachePolicy.LRU:
            self.cache.move_to_end(cache_key)
    
    def clear(self):
        """Clear all cache entries"""
        self.cache.clear()
        self.stats = CacheStats()
    
    def size(self) -> int:
        """Get current cache size"""
        return len(self.cache)
    
    def get_stats(self) -> Dict[str, Any]:
        """Get cache statistics"""
        return {
            "hits": self.stats.hits,
            "misses": self.stats.misses,
            "evictions": self.stats.evictions,
            "total_requests": self.stats.total_requests,
            "hit_rate": f"{self.stats.hit_rate:.2f}%",
            "miss_rate": f"{self.stats.miss_rate:.2f}%",
            "current_size": len(self.cache),
            "max_size": self.max_size
        }


class SemanticCache:
    """
    Semantic cache using embeddings for similarity matching.
    
    This cache finds similar queries using vector similarity,
    allowing cache hits for semantically similar but not identical queries.
    """
    
    def __init__(
        self,
        similarity_threshold: float = 0.95,
        max_size: int = 500
    ):
        """
        Initialize semantic cache.
        
        Args:
            similarity_threshold: Minimum similarity for cache hit
            max_size: Maximum cache entries
        """
        self.similarity_threshold = similarity_threshold
        self.max_size = max_size
        self.embeddings = OpenAIEmbeddings()
        self.cache: List[Dict[str, Any]] = []
        self.stats = CacheStats()
    
    def _compute_similarity(self, query1: str, query2: str) -> float:
        """
        Compute semantic similarity between queries.
        
        Args:
            query1: First query
            query2: Second query
            
        Returns:
            Similarity score (0-1)
        """
        # Simple implementation - in production use precomputed embeddings
        # This is for demonstration only
        query1_lower = query1.lower()
        query2_lower = query2.lower()
        
        # Simple word overlap similarity
        words1 = set(query1_lower.split())
        words2 = set(query2_lower.split())
        
        if not words1 or not words2:
            return 0.0
        
        intersection = words1.intersection(words2)
        union = words1.union(words2)
        
        return len(intersection) / len(union)
    
    def get(self, query: str) -> Optional[Any]:
        """
        Get value for semantically similar query.
        
        Args:
            query: Query string
            
        Returns:
            Cached value or None
        """
        self.stats.total_requests += 1
        
        # Find most similar cached query
        best_match = None
        best_similarity = 0.0
        
        for entry in self.cache:
            similarity = self._compute_similarity(query, entry["query"])
            if similarity > best_similarity:
                best_similarity = similarity
                best_match = entry
        
        if best_match and best_similarity >= self.similarity_threshold:
            self.stats.hits += 1
            # Update access time
            best_match["last_accessed"] = datetime.now()
            return best_match["value"]
        
        self.stats.misses += 1
        return None
    
    def set(self, query: str, value: Any):
        """
        Cache value for query.
        
        Args:
            query: Query string
            value: Value to cache
        """
        # Evict oldest if at capacity
        if len(self.cache) >= self.max_size:
            self.cache.pop(0)
            self.stats.evictions += 1
        
        self.cache.append({
            "query": query,
            "value": value,
            "created_at": datetime.now(),
            "last_accessed": datetime.now()
        })
    
    def clear(self):
        """Clear cache"""
        self.cache.clear()
        self.stats = CacheStats()
    
    def get_stats(self) -> Dict[str, Any]:
        """Get cache statistics"""
        return {
            "hits": self.stats.hits,
            "misses": self.stats.misses,
            "evictions": self.stats.evictions,
            "total_requests": self.stats.total_requests,
            "hit_rate": f"{self.stats.hit_rate:.2f}%",
            "current_size": len(self.cache),
            "max_size": self.max_size,
            "similarity_threshold": self.similarity_threshold
        }


class CachedLLMAgent:
    """
    LLM agent with caching support.
    
    This agent automatically caches responses and retrieves them
    for repeated or similar queries.
    """
    
    def __init__(
        self,
        cache: SimpleCache,
        model: str = "gpt-3.5-turbo"
    ):
        """
        Initialize cached agent.
        
        Args:
            cache: Cache instance to use
            model: LLM model name
        """
        self.cache = cache
        self.llm = ChatOpenAI(model=model, temperature=0.7)
    
    def generate(self, prompt: str) -> Dict[str, Any]:
        """
        Generate response with caching.
        
        Args:
            prompt: Input prompt
            
        Returns:
            Dictionary with response and cache info
        """
        # Check cache first
        cached_response = self.cache.get(prompt)
        
        if cached_response is not None:
            return {
                "response": cached_response,
                "from_cache": True,
                "cache_stats": self.cache.get_stats()
            }
        
        # Generate new response
        start_time = time.time()
        response = self.llm.invoke(prompt)
        generation_time = time.time() - start_time
        
        # Cache the response
        self.cache.set(prompt, response.content)
        
        return {
            "response": response.content,
            "from_cache": False,
            "generation_time": generation_time,
            "cache_stats": self.cache.get_stats()
        }


def demonstrate_caching_patterns():
    """Demonstrate caching patterns"""
    print("=" * 80)
    print("CACHING PATTERNS DEMONSTRATION")
    print("=" * 80)
    
    # Example 1: Basic LRU Cache
    print("\n" + "=" * 80)
    print("EXAMPLE 1: Basic LRU Cache")
    print("=" * 80)
    
    cache = SimpleCache(max_size=5, policy=CachePolicy.LRU)
    agent = CachedLLMAgent(cache)
    
    print("\nMaking requests (first time - cache misses):\n")
    
    queries = [
        "What is Python?",
        "What is JavaScript?",
        "What is Python?",  # Repeat
        "What is Java?",
        "What is Python?",  # Repeat again
    ]
    
    for i, query in enumerate(queries, 1):
        print(f"Request {i}: {query}")
        result = agent.generate(query)
        
        cache_status = "HIT ✓" if result["from_cache"] else "MISS ✗"
        print(f"  Status: {cache_status}")
        
        if not result["from_cache"]:
            print(f"  Generation time: {result['generation_time']:.2f}s")
        print(f"  Response: {result['response'][:80]}...")
        print()
    
    stats = cache.get_stats()
    print("Cache Statistics:")
    print(f"  Hit Rate: {stats['hit_rate']}")
    print(f"  Total Requests: {stats['total_requests']}")
    print(f"  Hits: {stats['hits']}")
    print(f"  Misses: {stats['misses']}")
    
    # Example 2: TTL-Based Caching
    print("\n" + "=" * 80)
    print("EXAMPLE 2: TTL-Based Caching (Time Expiration)")
    print("=" * 80)
    
    ttl_cache = SimpleCache(max_size=100, default_ttl=5)  # 5 second TTL
    
    print("\nCaching value with 5-second TTL...")
    ttl_cache.set("test_key", "test_value", ttl=5)
    
    print("Immediate retrieval:")
    value = ttl_cache.get("test_key")
    print(f"  Result: {'HIT' if value else 'MISS'} - {value}")
    
    print("\nWaiting 6 seconds for expiration...")
    time.sleep(6)
    
    print("Retrieval after TTL:")
    value = ttl_cache.get("test_key")
    print(f"  Result: {'HIT' if value else 'MISS'} - {value}")
    
    # Example 3: Cache Eviction Policies
    print("\n" + "=" * 80)
    print("EXAMPLE 3: Cache Eviction Policies Comparison")
    print("=" * 80)
    
    policies = [CachePolicy.LRU, CachePolicy.LFU, CachePolicy.FIFO]
    
    for policy in policies:
        print(f"\n{policy.value.upper()} Policy:")
        test_cache = SimpleCache(max_size=3, policy=policy)
        
        # Add 5 items to cache of size 3
        for i in range(5):
            key = f"item_{i}"
            test_cache.set(key, f"value_{i}")
            print(f"  Added {key}, cache size: {test_cache.size()}")
        
        stats = test_cache.get_stats()
        print(f"  Evictions: {stats['evictions']}")
    
    # Example 4: Semantic Cache
    print("\n" + "=" * 80)
    print("EXAMPLE 4: Semantic Cache (Similar Query Matching)")
    print("=" * 80)
    
    semantic_cache = SemanticCache(similarity_threshold=0.7, max_size=100)
    
    # Cache a response
    original_query = "What is machine learning?"
    original_response = "Machine learning is a subset of AI that enables systems to learn from data."
    semantic_cache.set(original_query, original_response)
    
    print(f"\nCached query: '{original_query}'")
    print(f"Response: {original_response}\n")
    
    # Try similar queries
    similar_queries = [
        "What is machine learning?",  # Exact match
        "What's machine learning?",   # Slight variation
        "Tell me about machine learning",  # Similar
        "Explain artificial intelligence",  # Different topic
    ]
    
    for query in similar_queries:
        result = semantic_cache.get(query)
        status = "HIT ✓" if result else "MISS ✗"
        print(f"Query: '{query}'")
        print(f"  Status: {status}")
        if result:
            print(f"  Retrieved: {result}")
        print()
    
    # Example 5: Multi-Layer Cache
    print("\n" + "=" * 80)
    print("EXAMPLE 5: Multi-Layer Cache Strategy")
    print("=" * 80)
    
    # L1: Small, fast, in-memory cache
    l1_cache = SimpleCache(max_size=10, policy=CachePolicy.LRU)
    
    # L2: Larger, still in-memory
    l2_cache = SimpleCache(max_size=100, policy=CachePolicy.LRU)
    
    print("\nMulti-layer cache simulation:")
    print("  L1: Small (10 items), fast")
    print("  L2: Large (100 items), moderate\n")
    
    def multi_layer_get(key: str) -> Optional[str]:
        """Get from multi-layer cache"""
        # Try L1 first
        value = l1_cache.get(key)
        if value:
            print(f"  L1 HIT for '{key}'")
            return value
        
        # Try L2
        value = l2_cache.get(key)
        if value:
            print(f"  L2 HIT for '{key}'")
            # Promote to L1
            l1_cache.set(key, value)
            return value
        
        print(f"  MISS for '{key}'")
        return None
    
    # Simulate access pattern
    l1_cache.set("frequently_accessed", "hot_data")
    l2_cache.set("occasionally_accessed", "warm_data")
    
    print("Access patterns:")
    multi_layer_get("frequently_accessed")
    multi_layer_get("occasionally_accessed")
    multi_layer_get("never_cached")
    
    # Example 6: Cache Performance Metrics
    print("\n" + "=" * 80)
    print("EXAMPLE 6: Cache Performance Analysis")
    print("=" * 80)
    
    perf_cache = SimpleCache(max_size=50, policy=CachePolicy.LRU)
    perf_agent = CachedLLMAgent(perf_cache)
    
    print("\nSimulating workload with repeated queries...\n")
    
    workload = [
        "What is AI?",
        "What is ML?",
        "What is AI?",  # Repeat
        "What is DL?",
        "What is AI?",  # Repeat
        "What is ML?",  # Repeat
        "What is NLP?",
        "What is AI?",  # Repeat
    ]
    
    total_time_with_cache = 0
    total_time_without_cache = 0
    
    for query in workload:
        result = perf_agent.generate(query)
        
        if result["from_cache"]:
            total_time_with_cache += 0.001  # Assume 1ms for cache hit
        else:
            gen_time = result.get("generation_time", 1.0)
            total_time_with_cache += gen_time
            total_time_without_cache += gen_time
    
    # Estimate time without cache (all queries would need generation)
    total_time_without_cache = len(workload) * 1.0  # Assume 1s per generation
    
    print(f"Performance Analysis:")
    print(f"  Total Queries: {len(workload)}")
    print(f"  Time WITH cache: {total_time_with_cache:.2f}s")
    print(f"  Time WITHOUT cache (estimated): {total_time_without_cache:.2f}s")
    print(f"  Speedup: {total_time_without_cache/total_time_with_cache:.1f}x")
    print(f"  Time Saved: {total_time_without_cache - total_time_with_cache:.2f}s")
    
    final_stats = perf_cache.get_stats()
    print(f"\nFinal Cache Statistics:")
    print(f"  Hit Rate: {final_stats['hit_rate']}")
    print(f"  Hits: {final_stats['hits']}")
    print(f"  Misses: {final_stats['misses']}")
    
    # Summary
    print("\n" + "=" * 80)
    print("CACHING PATTERNS SUMMARY")
    print("=" * 80)
    print("""
Caching Pattern Benefits:
1. Performance: Dramatically faster response times (10-1000x)
2. Cost Reduction: Fewer API calls = lower costs
3. Scalability: Handle more requests with same resources
4. Reliability: Reduce dependency on external services
5. User Experience: Instant responses for cached queries

Cache Types Demonstrated:
1. Simple Cache: Basic in-memory caching with eviction policies
2. TTL Cache: Time-based expiration for freshness
3. Semantic Cache: Similarity-based matching for related queries
4. Multi-Layer Cache: Hierarchical caching strategy
5. Performance-Optimized: Focus on hit rates and speedup

Eviction Policies:
1. LRU (Least Recently Used): Remove oldest accessed items
2. LFU (Least Frequently Used): Remove least popular items
3. FIFO (First In First Out): Remove oldest items
4. TTL (Time To Live): Expire after time period

Best Practices:
1. Cache Strategy Selection:
   - Exact match: Simple, fast, for identical queries
   - Semantic: Better coverage, for similar queries
   - TTL: For time-sensitive data
   - Multi-layer: For diverse access patterns

2. TTL Configuration:
   - Short (minutes): Dynamic, frequently changing data
   - Medium (hours): Semi-static content
   - Long (days): Stable, rarely changing data
   - No TTL: Static reference data

3. Size Management:
   - Monitor memory usage
   - Set appropriate limits
   - Implement eviction policies
   - Regular cleanup

4. Performance Monitoring:
   - Target >70% hit rate for good performance
   - Track cache size trends
   - Monitor eviction frequency
   - Measure latency improvements

When to Use Caching:
✓ Repeated identical queries
✓ Similar semantic queries
✓ Expensive computations
✓ Rate-limited APIs
✓ Stable, unchanging data
✗ Real-time data requirements
✗ Highly personalized responses
✗ Security-sensitive information
✗ Rapidly changing data

Production Considerations:
- Distributed caching (Redis) for multi-instance deployments
- Cache warming strategies for cold starts
- Monitoring and alerting on hit rates
- Regular cache analysis and optimization
- Security: encrypt sensitive cached data
- Compliance: respect data retention policies
- Backup and recovery procedures
- Cache versioning for updates

Cost-Benefit Analysis:
- Typical API call: $0.001-0.01 per request
- Cache hit cost: ~$0.000001 per request
- Potential savings: 90-99% cost reduction
- Performance gain: 10-1000x faster responses
- ROI: Immediate for high-volume applications
""")
    
    print("\n" + "=" * 80)
    print("Pattern 089 (Caching Patterns) demonstration complete!")
    print("=" * 80)


if __name__ == "__main__":
    demonstrate_caching_patterns()

    Agentic Design Pattern: Caching Patterns

    Pattern 089: Caching Patterns – Line-by-Line Explanation

    Overview

    This document provides a detailed line-by-line explanation of the Caching Patterns pattern implementation.

    Pattern Description

    Pattern 089: Caching Patterns

    Description: The Caching Patterns involve storing and reusing previously computed results to improve performance, reduce latency, and minimize costs. In the context of LLM applications, caching is critical for optimizing response times and reducing API calls for repeated or similar queries. Effective caching strategies can dramatically improve user experience while significantly cutting operational costs.

    Caching in LLM applications targets:
- Complete responses (exact match caching)
- Embeddings (vector representations)
- Prompt templates (reusable structures)
- Intermediate results (multi-step workflows)
- API responses (rate limit optimization)

    Components: 1. Cache Types – In-memory cache (fast, volatile) – Disk cache (persistent, slower) – Distributed cache (Redis, Memcached) – Database cache (structured, queryable) – CDN cache (geographic distribution)

    2. Cache Keys
   - Exact match (full prompt hash)
   - Semantic match (embedding similarity)
   - Parametric keys (template + parameters)
   - Hierarchical keys (nested cache structures)
   - Time-based keys (temporal validity)

3. Cache Policies
   - TTL (Time To Live) - expire after time
   - LRU (Least Recently Used) - evict old entries
   - LFU (Least Frequently Used) - evict unpopular
   - Size-based - limit total cache size
   - Custom policies - application-specific

4. Invalidation Strategies
   - Time-based expiration
   - Event-triggered invalidation
   - Manual invalidation
   - Cascade invalidation (dependencies)
   - Version-based invalidation

    Use Cases: 1. Repeated Queries – FAQ responses – Common questions – Popular content – Standard procedures – Frequently accessed data

    2. Expensive Operations
   - Complex reasoning chains
   - Multi-step workflows
   - Large context processing
   - Batch operations
   - Resource-intensive computations

3. Rate Limit Optimization
   - Reduce API calls
   - Stay within quotas
   - Handle traffic spikes
   - Cost optimization
   - Performance consistency

4. Embeddings and Vectors
   - Document embeddings
   - Query embeddings
   - Semantic search optimization
   - Similarity computations
   - Vector database queries

5. Multi-User Applications
   - Shared responses
   - Common workflows
   - Popular queries
   - Template responses
   - Collaborative features

    LangChain Implementation: LangChain supports caching through: – InMemoryCache for simple caching – SQLiteCache for persistent caching – Custom cache implementations – Semantic similarity caching – Embedding caching

    Key Features: 1. Multiple Cache Layers – L1: In-memory (fastest) – L2: Local disk (persistent) – L3: Distributed cache (shared) – L4: Database (long-term)

    2. Intelligent Matching
   - Exact match for identical queries
   - Semantic match for similar queries
   - Fuzzy match with thresholds
   - Parametric templates

3. Performance Optimization
   - Fast lookup (< 1ms for in-memory)
   - Efficient serialization
   - Compression for large entries
   - Lazy loading

4. Observability
   - Hit rate monitoring
   - Miss analysis
   - Performance metrics
   - Cost savings tracking

    Best Practices: 1. Cache Appropriately – Cache stable, reusable results – Don’t cache rapidly changing data – Consider data sensitivity – Balance freshness vs performance

    2. Set Proper TTLs
   - Shorter for dynamic content
   - Longer for stable content
   - Consider update frequency
   - Account for user expectations

3. Monitor Performance
   - Track hit rates (target > 70%)
   - Measure latency improvements
   - Calculate cost savings
   - Identify cache misses patterns

4. Handle Cache Misses
   - Graceful fallback to computation
   - Transparent to users
   - Background cache warming
   - Predictive pre-caching

    Trade-offs: Advantages: – Dramatically improved response times – Significant cost reduction – Better user experience – Reduced API rate limiting – Lower resource consumption – Improved scalability

    Disadvantages:
- Stale data risk
- Memory/storage overhead
- Cache consistency challenges
- Complexity in invalidation
- Initial cache warming time
- Debugging complexity

    Production Considerations: 1. Cache Size Management – Monitor memory usage – Implement eviction policies – Set size limits – Regular cleanup – Compression strategies

    2. Cache Consistency
   - Version tracking
   - Update propagation
   - Invalidation timing
   - Distributed consistency
   - Race condition handling

3. Security
   - Access control for cached data
   - Encryption for sensitive content
   - Cache poisoning prevention
   - User isolation in multi-tenant
   - Audit trails

4. Monitoring
   - Hit/miss rates
   - Cache size trends
   - Eviction frequency
   - Cost savings
   - Performance impact

5. Disaster Recovery
   - Cache rebuild procedures
   - Backup strategies
   - Failover mechanisms
   - Data loss handling
   - Recovery time objectives

    File Statistics

    • Total Lines: 897
    • Classes: 6
    • Functions: 1
    • Constants: 0

    Module Documentation

    """
Pattern 089: Caching Patterns

Description:
    The Caching Patterns involve storing and reusing previously computed results to
    improve performance, reduce latency, and minimize costs. In the context of LLM
    applications, caching is critical for optimizing response times and reducing API
    calls for repeated or similar queries. Effective caching strategies can
    dramatically improve user experience while significantly cutting operational costs.

    Caching in LLM applications targets:
    - Complete responses (exact match caching)
    - Embeddings (vector representations)
    - Prompt templates (reusable structures)
    - Intermediate results (multi-step workflows)
    - API responses (rate limit optimization)

Components:
    1. Cache Types
       - In-memory cache (fast, volatile)
       - Disk cache (persistent, slower)
       - Distributed cache (Redis, Memcached)
       - Database cache (structured, queryable)
       - CDN cache (geographic distribution)

    2. Cache Keys
       - Exact match (full prompt hash)
       - Semantic match (embedding similarity)
       - Parametric keys (template + parameters)
       - Hierarchical keys (nested cache structures)
       - Time-based keys (temporal validity)

    3. Cache Policies
       - TTL (Time To Live) - expire after time
       - LRU (Least Recently Used) - evict old entries
       - LFU (Least Frequently Used) - evict unpopular
       - Size-based - limit total cache size
       - Custom policies - application-specific

    4. Invalidation Strategies
       - Time-based expiration
       - Event-triggered invalidation
       - Manual invalidation
       - Cascade invalidation (dependencies)
       - Version-based invalidation

Use Cases:
    1. Repeated Queries
       - FAQ responses
       - Common questions
       - Popular content
       - Standard procedures
       - Frequently accessed data

    2. Expensive Operations
       - Complex reasoning chains
       - Multi-step workflows
       - Large context processing
       - Batch operations
       - Resource-intensive computations

    3. Rate Limit Optimization
       - Reduce API calls
       - Stay within quotas
       - Handle traffic spikes
       - Cost optimization
       - Performance consistency

    4. Embeddings and Vectors
       - Document embeddings
       - Query embeddings
       - Semantic search optimization
       - Similarity computations
       - Vector database queries

    5. Multi-User Applications
       - Shared responses
       - Common workflows
       - Popular queries
       - Template responses
       - Collaborative features

LangChain Implementation:
    LangChain supports caching through:
    - InMemoryCache for simple caching
    - SQLiteCache for persistent caching
    - Custom cache implementations
    - Semantic similarity caching
    - Embedding caching

Key Features:
    1. Multiple Cache Layers
       - L1: In-memory (fastest)
       - L2: Local disk (persistent)
       - L3: Distributed cache (shared)
       - L4: Database (long-term)

    2. Intelligent Matching
       - Exact match for identical queries
       - Semantic match for similar queries
       - Fuzzy match with thresholds
       - Parametric templates

    3. Performance Optimization
       - Fast lookup (< 1ms for in-memory)
       - Efficient serialization
       - Compression for large entries
       - Lazy loading

    4. Observability
       - Hit rate monitoring
       - Miss analysis
       - Performance metrics
       - Cost savings tracking

Best Practices:
    1. Cache Appropriately
       - Cache stable, reusable results
       - Don't cache rapidly changing data
       - Consider data sensitivity
       - Balance freshness vs performance

    2. Set Proper TTLs
       - Shorter for dynamic content
       - Longer for stable content
       - Consider update frequency
       - Account for user expectations

    3. Monitor Performance
       - Track hit rates (target > 70%)
       - Measure latency improvements
       - Calculate cost savings
       - Identify cache misses patterns

    4. Handle Cache Misses
       - Graceful fallback to computation
       - Transparent to users
       - Background cache warming
       - Predictive pre-caching

Trade-offs:
    Advantages:
    - Dramatically improved response times
    - Significant cost reduction
    - Better user experience
    - Reduced API rate limiting
    - Lower resource consumption
    - Improved scalability

    Disadvantages:
    - Stale data risk
    - Memory/storage overhead
    - Cache consistency challenges
    - Complexity in invalidation
    - Initial cache warming time
    - Debugging complexity

Production Considerations:
    1. Cache Size Management
       - Monitor memory usage
       - Implement eviction policies
       - Set size limits
       - Regular cleanup
       - Compression strategies

    2. Cache Consistency
       - Version tracking
       - Update propagation
       - Invalidation timing
       - Distributed consistency
       - Race condition handling

    3. Security
       - Access control for cached data
       - Encryption for sensitive content
       - Cache poisoning prevention
       - User isolation in multi-tenant
       - Audit trails

    4. Monitoring
       - Hit/miss rates
       - Cache size trends
       - Eviction frequency
       - Cost savings
       - Performance impact

    5. Disaster Recovery
       - Cache rebuild procedures
       - Backup strategies
       - Failover mechanisms
       - Data loss handling
       - Recovery time objectives
"""

    Purpose: This module implements the Caching Patterns pattern, providing a structured approach to agent design and implementation.

    Import Statements

    Standard Library Imports

    import collections.OrderedDict
import dataclasses.dataclass
import dataclasses.field
import datetime.datetime
import datetime.timedelta
import enum.Enum
import json
import langchain_core.caches.InMemoryCache
import langchain_core.output_parsers.StrOutputParser
import langchain_core.prompts.ChatPromptTemplate
import os
import time
import typing.Any
import typing.Callable
import typing.Dict
import typing.List
import typing.Optional

    Purpose: Core Python functionality: – collections.OrderedDict: Regular expression operations – dataclasses.dataclass: Simplified class definitions with @dataclass – dataclasses.field: Simplified class definitions with @dataclass – datetime.datetime: Date and time handling – datetime.timedelta: Date and time handling – enum.Enum: Enumeration support – json: JSON serialization/deserialization – langchain_core.caches.InMemoryCache: Regular expression operations – langchain_core.output_parsers.StrOutputParser: Regular expression operations – langchain_core.prompts.ChatPromptTemplate: Regular expression operations – os: Environment variable access and system operations – time: Time-related functions – typing.Any: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Callable: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Dict: Type hints for better code documentation (List, Dict, Any, Optional) – typing.List: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Optional: Type hints for better code documentation (List, Dict, Any, Optional)

    LangChain Imports

    from langchain.globals.set_llm_cache
from langchain_openai.ChatOpenAI
from langchain_openai.OpenAIEmbeddings

    Purpose: LangChain framework components: – LLM Interface: Wrapper for OpenAI models – Chains: Composable building blocks for complex workflows

    Other Imports

    from dotenv.load_dotenv
from hashlib

    Purpose: Additional dependencies for specialized functionality.

    Classes

    Class: CachePolicy

    Lines 215-220

    Inherits from: Enum

    Purpose:

    Cache eviction policies

    Class: CacheEntry

    Lines 224-243

    Purpose:

    Represents a cached entry

    Methods:

    is_expired(self)

    Lines 233-238

    Purpose:

    Check if entry is expired

    Parameters:

    Returns: bool

    touch(self)

    Lines 240-243

    Purpose:

    Update access information

    Parameters:

    Class: CacheStats

    Lines 247-264

    Purpose:

    Cache performance statistics

    Methods:

    hit_rate(self)

    Lines 255-259

    Decorators: property

    Purpose:

    Calculate cache hit rate

    Parameters:

    Returns: float

    miss_rate(self)

    Lines 262-264

    Decorators: property

    Purpose:

    Calculate cache miss rate

    Parameters:

    Returns: float

    Class: SimpleCache

    Lines 267-416

    Purpose:

    Simple in-memory cache with multiple eviction policies.

This cache supports:
1. Multiple eviction policies (LRU, LFU, FIFO, TTL)
2. Size limits
3. TTL per entry
4. Performance statistics

    Methods:

    __init__(self, max_size, policy, default_ttl)

    Lines 278-296

    Purpose:

    Initialize cache.

Args:
    max_size: Maximum number of entries
    policy: Eviction policy
    default_ttl: Default TTL in seconds

    Parameters: – max_size: Max size – policy: Policy – default_ttl: Default ttl

    _make_key(self, key)

    Lines 298-302

    Purpose:

    Create cache key from input

    Parameters: – key: Key

    Returns: str

    _evict(self)

    Lines 304-320

    Purpose:

    Evict entries based on policy

    Parameters:

    _remove_expired(self)

    Lines 322-329

    Purpose:

    Remove expired entries

    Parameters:

    get(self, key)

    Lines 331-365

    Purpose:

    Get value from cache.

Args:
    key: Cache key

Returns:
    Cached value or None

    Parameters: – key: Key

    Returns: <ast.Subscript object at 0x000001A06D62AED0>

    set(self, key, value, ttl)

    Lines 367-394

    Purpose:

    Set value in cache.

Args:
    key: Cache key
    value: Value to cache
    ttl: Optional TTL in seconds

    Parameters: – key: Key – value: Value – ttl: Ttl

    clear(self)

    Lines 396-399

    Purpose:

    Clear all cache entries

    Parameters:

    size(self)

    Lines 401-403

    Purpose:

    Get current cache size

    Parameters:

    Returns: int

    get_stats(self)

    Lines 405-416

    Purpose:

    Get cache statistics

    Parameters:

    Returns: <ast.Subscript object at 0x000001A06D74A3D0>

    Class: SemanticCache

    Lines 419-540

    Purpose:

    Semantic cache using embeddings for similarity matching.

This cache finds similar queries using vector similarity,
allowing cache hits for semantically similar but not identical queries.

    Methods:

    __init__(self, similarity_threshold, max_size)

    Lines 427-443

    Purpose:

    Initialize semantic cache.

Args:
    similarity_threshold: Minimum similarity for cache hit
    max_size: Maximum cache entries

    Parameters: – similarity_threshold: Similarity threshold – max_size: Max size

    _compute_similarity(self, query1, query2)

    Lines 445-471

    Purpose:

    Compute semantic similarity between queries.

Args:
    query1: First query
    query2: Second query

Returns:
    Similarity score (0-1)

    Parameters: – query1: Input query or question – query2: Input query or question

    Returns: float

    get(self, query)

    Lines 473-502

    Purpose:

    Get value for semantically similar query.

Args:
    query: Query string

Returns:
    Cached value or None

    Parameters: – query: Input query or question

    Returns: <ast.Subscript object at 0x000001A06D6A1F50>

    set(self, query, value)

    Lines 504-522

    Purpose:

    Cache value for query.

Args:
    query: Query string
    value: Value to cache

    Parameters: – query: Input query or question – value: Value

    clear(self)

    Lines 524-527

    Purpose:

    Clear cache

    Parameters:

    get_stats(self)

    Lines 529-540

    Purpose:

    Get cache statistics

    Parameters:

    Returns: <ast.Subscript object at 0x000001A06D7FC110>

    Class: CachedLLMAgent

    Lines 543-599

    Purpose:

    LLM agent with caching support.

This agent automatically caches responses and retrieves them
for repeated or similar queries.

    Methods:

    __init__(self, cache, model)

    Lines 551-564

    Purpose:

    Initialize cached agent.

Args:
    cache: Cache instance to use
    model: LLM model name

    Parameters: – cache: Cache – model: Language model instance

    generate(self, prompt)

    Lines 566-599

    Purpose:

    Generate response with caching.

Args:
    prompt: Input prompt

Returns:
    Dictionary with response and cache info

    Parameters: – prompt: Prompt template or text

    Returns: <ast.Subscript object at 0x000001A06D7C1A90>

    Functions

    demonstrate_caching_patterns()

    Lines 602-892

    Purpose:

    Demonstrate caching patterns

    Key Concepts Summary

    Pattern Overview

    The Caching Patterns pattern provides a structured approach to building AI agents with specific capabilities and behaviors. This implementation leverages LangChain’s composable components to create a robust and maintainable solution.

    Core Components

    1. Agent Class: Main implementation of the pattern logic
    2. Prompt Engineering: Carefully crafted prompts to guide LLM behavior
    3. Tool Integration: Optional tools for extended functionality
    4. Memory Management: State handling across interactions
    5. Output Parsing: Structured response processing

    When to Use

    ✅ Good for: – Scenarios requiring this specific pattern’s strengths – Applications needing structured agent behavior – Systems where explainability is important – Production environments requiring reliability

    ❌ Not ideal for: – Simple single-prompt tasks – Scenarios requiring different pattern characteristics – Resource-constrained environments (if computationally intensive)

    Best Practices Demonstrated

    1. Environment Configuration

    • ✅ Use .env files for API keys
    • ✅ Never hardcode credentials
    • ✅ Validate environment variables on startup

    2. Error Handling

    • ✅ Graceful degradation on failures
    • ✅ Informative error messages
    • ✅ Logging for debugging

    3. Prompt Engineering

    • ✅ Clear, specific instructions
    • ✅ Structured output formats
    • ✅ Few-shot examples when beneficial

    4. Code Organization

    • ✅ Separation of concerns (class-based design)
    • ✅ Type hints for clarity
    • ✅ Docstrings for documentation

    5. Performance Optimization

    • ✅ Appropriate temperature settings
    • ✅ Token budget management
    • ✅ Caching where applicable

    LangChain Components Used

    • ChatOpenAI: OpenAI model interface
    • Chains: Sequential component composition

    Why LangChain? – Composable abstractions for LLM applications – Rich ecosystem of integrations – Production-ready patterns – Active community and development

    Execution Flow

    1. Initialize agent with configuration
   ↓
2. Load environment and dependencies
   ↓
3. Prepare prompt with input data
   ↓
4. Invoke LLM through LangChain
   ↓
5. Parse and validate response
   ↓
6. Return formatted result

    Key Decision Points: – Model selection (temperature, model type) – Prompt structure (few-shot vs zero-shot) – Error handling strategy – Output format validation

    Performance Considerations

    Computational Cost

    • LLM Calls: Primary cost factor (based on tokens)
    • Latency: Depends on model and input size
    • Memory: Minimal for most patterns

    Optimization Strategies

    1. Prompt Optimization: Minimize tokens while maintaining clarity
    2. Caching: Cache repeated queries/responses
    3. Model Selection: Use smaller models when appropriate
    4. Batch Processing: Group similar requests
    5. Streaming: For long responses, stream tokens

    Cost Estimates (Approximate)

    • GPT-3.5-turbo: ~$0.002 per request
    • GPT-4: ~$0.03 per request
    • GPT-4-turbo: ~$0.01 per request

    Actual costs vary based on input/output token counts

    Common Pitfalls to Avoid

    1. Prompt Design Issues

    ❌ Vague or ambiguous instructions
    ✅ Clear, specific, testable prompts

    2. Error Handling

    ❌ Ignoring API errors or rate limits
    ✅ Implement retry logic and graceful degradation

    3. Token Management

    ❌ Exceeding model token limits
    ✅ Truncate inputs, chunk processing

    4. Security

    ❌ Exposing API keys in code
    ✅ Use environment variables

    5. Testing

    ❌ Only testing happy paths
    ✅ Test edge cases, failures, malformed inputs

    6. Cost Management

    ❌ Unmonitored API usage
    ✅ Implement budgets, alerts, and monitoring

    Further Improvements

    Production Enhancements

    1. Monitoring: Add logging, metrics, tracing
    2. Validation: Input/output schema validation
    3. Rate Limiting: Implement request throttling
    4. Caching: Redis or in-memory cache for responses
    5. Async Support: Non-blocking operations for scalability

    Feature Extensions

    1. Multi-Model Support: Switch between providers
    2. Fallback Models: Graceful degradation strategy
    3. Custom Tools: Domain-specific function calling
    4. Advanced Memory: Vector stores for long-term memory
    5. Evaluation: Automated testing and benchmarking

    Code Quality

    1. Testing: Unit tests, integration tests
    2. Documentation: API docs, usage examples
    3. Type Safety: Stricter type hints
    4. Configuration: Externalized config files

    Related Patterns

    • Pattern 088
    • Pattern 090

    Pattern Families: – Reasoning Patterns (001-011) – Multi-Agent Patterns (012-019) – Tool Use Patterns (020-025) – Memory Patterns (026-031) – And many more…

    Summary

    The Caching Patterns pattern provides a powerful approach to building AI agents with specific capabilities. This implementation demonstrates:

    ✅ Key Strengths: – Clean, maintainable code structure – LangChain integration for composability – Type hints and documentation – Production-ready patterns

    🎯 Best Use Cases: – Applications requiring this pattern’s characteristics – Systems needing structured agent behavior – Production environments requiring reliability

    📚 Learning Resources: – LangChain Documentation: https://python.langchain.com/ – Pattern Documentation: See related patterns – OpenAI API Reference: https://platform.openai.com/docs/

    File: 089_caching_patterns.py
    Total Lines: 897
    Pattern Category: AI Agent Design Pattern #089

    Generated from actual implementation code
    For educational and reference purposes

  • Agentic Design Pattern: Token Budget

    
    """
Pattern 088: Token Budget Management

Description:
    The Token Budget Management pattern monitors and controls token usage across LLM
    interactions to optimize costs, prevent budget overruns, and ensure efficient
    resource utilization. This pattern is critical for production deployments where
    token costs can escalate quickly, especially with high-volume applications or
    complex multi-step workflows.

    Token budget management involves:
    - Real-time token usage tracking
    - Cost estimation and forecasting
    - Budget enforcement and alerts
    - Optimization strategies for token efficiency
    - Usage analytics and reporting

Components:
    1. Token Counting
       - Input token counting (prompt tokens)
       - Output token counting (completion tokens)
       - Total token tracking per request
       - Cumulative usage tracking
       - Model-specific token limits

    2. Cost Calculation
       - Model pricing tables (per-token costs)
       - Request cost estimation
       - Cumulative cost tracking
       - Budget vs actual comparison
       - Cost forecasting

    3. Budget Enforcement
       - Hard limits (stop when exceeded)
       - Soft limits (warnings when approaching)
       - Rate-based budgets (tokens per hour/day)
       - User-specific quotas
       - Priority-based allocation

    4. Optimization Strategies
       - Prompt compression
       - Response length limiting
       - Model selection (cost vs quality)
       - Caching frequently used responses
       - Batching requests

Use Cases:
    1. Cost Control
       - Prevent unexpected charges
       - Stay within allocated budgets
       - Cost attribution per user/project
       - Predictable monthly spending
       - Resource optimization

    2. Multi-Tenant Applications
       - Per-user token quotas
       - Fair resource allocation
       - Usage-based billing
       - Tier-based limits (free/premium)
       - Abuse prevention

    3. Development and Testing
       - Sandbox environments with limits
       - Cost-effective experimentation
       - Testing budget allocation
       - Development vs production budgets
       - CI/CD pipeline limits

    4. High-Volume Applications
       - Aggregate usage tracking
       - Peak usage management
       - Resource planning
       - Scaling decisions
       - Cost optimization at scale

    5. Enterprise Deployments
       - Department-level budgets
       - Project-based allocation
       - Chargeback mechanisms
       - Compliance and auditing
       - Executive reporting

LangChain Implementation:
    LangChain supports token budget management through:
    - Callback handlers for token tracking
    - get_openai_callback for cost monitoring
    - Custom token counters
    - Usage analytics integration
    - Budget enforcement wrappers

Key Features:
    1. Real-Time Tracking
       - Immediate token counting
       - Running total updates
       - Cost calculation on-the-fly
       - Budget threshold monitoring
       - Alert triggering

    2. Multi-Model Support
       - Different pricing for different models
       - Model-specific token limits
       - Automatic model selection
       - Cost-aware routing
       - Fallback to cheaper models

    3. Granular Control
       - Request-level limits
       - Session-level budgets
       - User-level quotas
       - Project-level allocations
       - Global organization limits

    4. Analytics and Reporting
       - Usage trends over time
       - Cost breakdown by model
       - Per-user consumption
       - Peak usage identification
       - Cost forecasting

Best Practices:
    1. Set Appropriate Budgets
       - Based on historical data
       - Include buffer for peaks
       - Align with business value
       - Regular review and adjustment
       - Clear escalation procedures

    2. Implement Progressive Enforcement
       - Soft warnings at 70-80%
       - Stricter controls at 90%
       - Hard stops at 100%
       - Grace period for critical operations
       - Override mechanisms for emergencies

    3. Optimize Token Usage
       - Compress verbose prompts
       - Limit response lengths
       - Use cheaper models when sufficient
       - Cache common queries
       - Batch similar requests

    4. Monitor and Alert
       - Real-time dashboards
       - Automated alerts
       - Anomaly detection
       - Trend analysis
       - Regular reporting

Trade-offs:
    Advantages:
    - Predictable costs
    - Prevents budget overruns
    - Enables cost optimization
    - Fair resource allocation
    - Usage visibility and control
    - Informed scaling decisions

    Disadvantages:
    - Implementation complexity
    - Tracking overhead (minimal)
    - May limit functionality at budget
    - Requires ongoing management
    - False positives in alerting
    - User friction from limits

Production Considerations:
    1. Accuracy
       - Precise token counting
       - Up-to-date pricing information
       - Correct model identification
       - Cache hit accounting
       - Network overhead handling

    2. Performance
       - Minimal tracking overhead
       - Efficient storage of usage data
       - Fast budget checks
       - Optimized queries for analytics
       - Scalable architecture

    3. User Experience
       - Clear budget status visibility
       - Helpful error messages
       - Grace period handling
       - Budget increase requests
       - Usage optimization tips

    4. Maintenance
       - Regular pricing updates
       - Budget policy reviews
       - Alert threshold tuning
       - Anomaly investigation
       - Capacity planning

    5. Security
       - Access control for budget data
       - Secure token counting
       - Audit trail for usage
       - Fraud detection
       - Quota manipulation prevention
"""

import os
import time
from typing import List, Dict, Any, Optional, Callable
from dataclasses import dataclass, field
from enum import Enum
from datetime import datetime, timedelta
from collections import defaultdict
from dotenv import load_dotenv

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_openai import ChatOpenAI
from langchain.callbacks import get_openai_callback

load_dotenv()


class BudgetStatus(Enum):
    """Budget status levels"""
    NORMAL = "normal"
    WARNING = "warning"
    CRITICAL = "critical"
    EXCEEDED = "exceeded"


class ModelTier(Enum):
    """Model cost tiers"""
    CHEAP = "cheap"
    STANDARD = "standard"
    PREMIUM = "premium"


@dataclass
class ModelPricing:
    """Pricing information for a model"""
    model_name: str
    input_cost_per_1k: float  # Cost per 1000 input tokens
    output_cost_per_1k: float  # Cost per 1000 output tokens
    tier: ModelTier


@dataclass
class TokenUsage:
    """Token usage for a single request"""
    prompt_tokens: int
    completion_tokens: int
    total_tokens: int
    cost: float
    model: str
    timestamp: datetime = field(default_factory=datetime.now)


@dataclass
class BudgetConfig:
    """Configuration for token budget management"""
    total_budget: float  # Total budget in USD
    warning_threshold: float = 0.8  # Warning at 80%
    critical_threshold: float = 0.9  # Critical at 90%
    enforce_hard_limit: bool = True
    default_model: str = "gpt-3.5-turbo"
    allow_model_downgrade: bool = True


class TokenBudgetManager:
    """
    Manages token budgets and costs across LLM interactions.
    
    This manager:
    1. Tracks token usage in real-time
    2. Calculates costs based on model pricing
    3. Enforces budget limits
    4. Provides usage analytics
    """
    
    # Model pricing (as of 2025)
    MODEL_PRICING = {
        "gpt-3.5-turbo": ModelPricing("gpt-3.5-turbo", 0.0005, 0.0015, ModelTier.CHEAP),
        "gpt-4": ModelPricing("gpt-4", 0.03, 0.06, ModelTier.PREMIUM),
        "gpt-4-turbo": ModelPricing("gpt-4-turbo", 0.01, 0.03, ModelTier.STANDARD),
    }
    
    def __init__(self, config: BudgetConfig):
        """
        Initialize token budget manager.
        
        Args:
            config: Budget configuration
        """
        self.config = config
        self.total_spent = 0.0
        self.usage_history: List[TokenUsage] = []
        self.requests_count = 0
        self.model_usage: Dict[str, int] = defaultdict(int)
        self.alert_callbacks: List[Callable] = []
    
    def add_alert_callback(self, callback: Callable[[BudgetStatus, Dict], None]):
        """Add callback for budget alerts"""
        self.alert_callbacks.append(callback)
    
    def _trigger_alerts(self, status: BudgetStatus, details: Dict[str, Any]):
        """Trigger all registered alert callbacks"""
        for callback in self.alert_callbacks:
            try:
                callback(status, details)
            except Exception as e:
                print(f"Alert callback error: {e}")
    
    def get_status(self) -> BudgetStatus:
        """Get current budget status"""
        usage_ratio = self.total_spent / self.config.total_budget
        
        if usage_ratio >= 1.0:
            return BudgetStatus.EXCEEDED
        elif usage_ratio >= self.config.critical_threshold:
            return BudgetStatus.CRITICAL
        elif usage_ratio >= self.config.warning_threshold:
            return BudgetStatus.WARNING
        else:
            return BudgetStatus.NORMAL
    
    def can_proceed(self, estimated_cost: float = 0.0) -> tuple[bool, str]:
        """
        Check if request can proceed within budget.
        
        Args:
            estimated_cost: Estimated cost of request
            
        Returns:
            Tuple of (can_proceed, reason)
        """
        status = self.get_status()
        
        if status == BudgetStatus.EXCEEDED:
            if self.config.enforce_hard_limit:
                return False, "Budget exceeded. Request denied."
            else:
                return True, "Budget exceeded but enforcement disabled."
        
        projected_cost = self.total_spent + estimated_cost
        if projected_cost > self.config.total_budget:
            if self.config.enforce_hard_limit:
                return False, f"Request would exceed budget. Projected: ${projected_cost:.4f}"
            
        if status == BudgetStatus.CRITICAL:
            return True, "Budget critical. Proceed with caution."
        
        return True, "OK"
    
    def calculate_cost(
        self,
        prompt_tokens: int,
        completion_tokens: int,
        model: str
    ) -> float:
        """
        Calculate cost for token usage.
        
        Args:
            prompt_tokens: Number of prompt tokens
            completion_tokens: Number of completion tokens
            model: Model name
            
        Returns:
            Cost in USD
        """
        pricing = self.MODEL_PRICING.get(model)
        if not pricing:
            # Default to gpt-3.5-turbo pricing
            pricing = self.MODEL_PRICING["gpt-3.5-turbo"]
        
        input_cost = (prompt_tokens / 1000) * pricing.input_cost_per_1k
        output_cost = (completion_tokens / 1000) * pricing.output_cost_per_1k
        
        return input_cost + output_cost
    
    def record_usage(
        self,
        prompt_tokens: int,
        completion_tokens: int,
        model: str
    ) -> TokenUsage:
        """
        Record token usage and update budget.
        
        Args:
            prompt_tokens: Number of prompt tokens
            completion_tokens: Number of completion tokens
            model: Model name
            
        Returns:
            TokenUsage object
        """
        total_tokens = prompt_tokens + completion_tokens
        cost = self.calculate_cost(prompt_tokens, completion_tokens, model)
        
        usage = TokenUsage(
            prompt_tokens=prompt_tokens,
            completion_tokens=completion_tokens,
            total_tokens=total_tokens,
            cost=cost,
            model=model
        )
        
        # Update tracking
        self.total_spent += cost
        self.usage_history.append(usage)
        self.requests_count += 1
        self.model_usage[model] += total_tokens
        
        # Check status and trigger alerts if needed
        status = self.get_status()
        if status in [BudgetStatus.WARNING, BudgetStatus.CRITICAL, BudgetStatus.EXCEEDED]:
            self._trigger_alerts(status, {
                "total_spent": self.total_spent,
                "budget": self.config.total_budget,
                "usage_ratio": self.total_spent / self.config.total_budget,
                "latest_usage": usage
            })
        
        return usage
    
    def get_remaining_budget(self) -> float:
        """Get remaining budget"""
        return max(0, self.config.total_budget - self.total_spent)
    
    def get_usage_summary(self) -> Dict[str, Any]:
        """Get comprehensive usage summary"""
        status = self.get_status()
        
        return {
            "total_budget": self.config.total_budget,
            "total_spent": self.total_spent,
            "remaining": self.get_remaining_budget(),
            "usage_percentage": (self.total_spent / self.config.total_budget) * 100,
            "status": status.value,
            "total_requests": self.requests_count,
            "total_tokens": sum(u.total_tokens for u in self.usage_history),
            "model_breakdown": dict(self.model_usage),
            "average_cost_per_request": self.total_spent / max(1, self.requests_count)
        }
    
    def suggest_cheaper_model(self, current_model: str) -> Optional[str]:
        """Suggest a cheaper model alternative"""
        if not self.config.allow_model_downgrade:
            return None
        
        current_pricing = self.MODEL_PRICING.get(current_model)
        if not current_pricing:
            return None
        
        # Find cheaper alternatives
        cheaper_models = [
            model for model, pricing in self.MODEL_PRICING.items()
            if pricing.tier.value  Dict[str, Any]:
        """
        Generate response with budget tracking.
        
        Args:
            prompt: Input prompt
            max_tokens: Maximum tokens for response
            
        Returns:
            Dictionary with response and usage info
        """
        # Check budget before proceeding
        can_proceed, reason = self.budget_manager.can_proceed()
        if not can_proceed:
            return {
                "success": False,
                "error": reason,
                "response": None,
                "usage": None
            }
        
        # Generate with token tracking
        with get_openai_callback() as cb:
            try:
                response = self.llm.invoke(prompt)
                
                # Record usage
                usage = self.budget_manager.record_usage(
                    prompt_tokens=cb.prompt_tokens,
                    completion_tokens=cb.completion_tokens,
                    model=self.model
                )
                
                return {
                    "success": True,
                    "response": response.content,
                    "usage": {
                        "prompt_tokens": usage.prompt_tokens,
                        "completion_tokens": usage.completion_tokens,
                        "total_tokens": usage.total_tokens,
                        "cost": usage.cost
                    },
                    "budget_status": self.budget_manager.get_status().value
                }
                
            except Exception as e:
                return {
                    "success": False,
                    "error": str(e),
                    "response": None,
                    "usage": None
                }
    
    def generate_with_limit(
        self,
        prompt: str,
        max_cost: float
    ) -> Dict[str, Any]:
        """
        Generate response with per-request cost limit.
        
        Args:
            prompt: Input prompt
            max_cost: Maximum cost for this request
            
        Returns:
            Dictionary with response and usage info
        """
        # Estimate and check
        can_proceed, reason = self.budget_manager.can_proceed(max_cost)
        if not can_proceed:
            return {
                "success": False,
                "error": reason,
                "response": None
            }
        
        return self.generate(prompt)
    
    def batch_generate(
        self,
        prompts: List[str],
        max_total_cost: Optional[float] = None
    ) -> List[Dict[str, Any]]:
        """
        Generate responses for multiple prompts with budget tracking.
        
        Args:
            prompts: List of prompts
            max_total_cost: Optional cost limit for batch
            
        Returns:
            List of results
        """
        results = []
        batch_cost = 0.0
        
        for prompt in prompts:
            # Check batch cost limit
            if max_total_cost and batch_cost >= max_total_cost:
                results.append({
                    "success": False,
                    "error": "Batch cost limit reached",
                    "response": None
                })
                continue
            
            result = self.generate(prompt)
            results.append(result)
            
            if result["success"] and result["usage"]:
                batch_cost += result["usage"]["cost"]
        
        return results


def demonstrate_token_budget_management():
    """Demonstrate token budget management patterns"""
    print("=" * 80)
    print("TOKEN BUDGET MANAGEMENT PATTERN DEMONSTRATION")
    print("=" * 80)
    
    # Example 1: Basic Budget Tracking
    print("\n" + "=" * 80)
    print("EXAMPLE 1: Basic Budget Tracking")
    print("=" * 80)
    
    config = BudgetConfig(
        total_budget=1.00,  # $1.00 budget
        warning_threshold=0.7,
        critical_threshold=0.9,
        enforce_hard_limit=False  # Allow going over for demo
    )
    
    budget_manager = TokenBudgetManager(config)
    agent = BudgetAwareLLMAgent(budget_manager)
    
    print(f"\nInitial Budget: ${config.total_budget}")
    print("\nGenerating responses...\n")
    
    prompts = [
        "What is machine learning?",
        "Explain neural networks briefly.",
        "What is deep learning?"
    ]
    
    for i, prompt in enumerate(prompts, 1):
        print(f"Request {i}: {prompt}")
        result = agent.generate(prompt)
        
        if result["success"]:
            print(f"Response: {result['response'][:100]}...")
            print(f"Tokens: {result['usage']['total_tokens']}")
            print(f"Cost: ${result['usage']['cost']:.6f}")
            print(f"Status: {result['budget_status']}")
        else:
            print(f"Error: {result['error']}")
        print()
    
    # Show summary
    summary = budget_manager.get_usage_summary()
    print("\nBudget Summary:")
    print(f"  Total Spent: ${summary['total_spent']:.6f}")
    print(f"  Remaining: ${summary['remaining']:.6f}")
    print(f"  Usage: {summary['usage_percentage']:.1f}%")
    print(f"  Status: {summary['status']}")
    print(f"  Total Requests: {summary['total_requests']}")
    print(f"  Total Tokens: {summary['total_tokens']}")
    print(f"  Avg Cost/Request: ${summary['average_cost_per_request']:.6f}")
    
    # Example 2: Budget Alerts
    print("\n" + "=" * 80)
    print("EXAMPLE 2: Budget Alerts and Thresholds")
    print("=" * 80)
    
    def budget_alert_handler(status: BudgetStatus, details: Dict):
        """Custom alert handler"""
        print(f"\n🚨 BUDGET ALERT: {status.value.upper()}")
        print(f"   Spent: ${details['total_spent']:.6f}")
        print(f"   Budget: ${details['budget']:.6f}")
        print(f"   Usage: {details['usage_ratio']*100:.1f}%")
    
    config2 = BudgetConfig(
        total_budget=0.10,  # Small budget to trigger alerts
        warning_threshold=0.5,
        critical_threshold=0.8,
        enforce_hard_limit=False
    )
    
    budget_manager2 = TokenBudgetManager(config2)
    budget_manager2.add_alert_callback(budget_alert_handler)
    agent2 = BudgetAwareLLMAgent(budget_manager2)
    
    print(f"\nBudget: ${config2.total_budget}")
    print("Generating responses to trigger alerts...\n")
    
    test_prompts = [
        "Explain quantum computing in detail.",
        "What are the applications of AI?",
        "Describe blockchain technology."
    ]
    
    for prompt in test_prompts:
        result = agent2.generate(prompt)
        if result["success"]:
            print(f"✓ Generated response (${result['usage']['cost']:.6f})")
    
    # Example 3: Cost Estimation and Limits
    print("\n" + "=" * 80)
    print("EXAMPLE 3: Per-Request Cost Limits")
    print("=" * 80)
    
    config3 = BudgetConfig(total_budget=0.50, enforce_hard_limit=True)
    budget_manager3 = TokenBudgetManager(config3)
    agent3 = BudgetAwareLLMAgent(budget_manager3)
    
    print("\nTrying request with cost limit...\n")
    
    result = agent3.generate_with_limit(
        "Write a short poem about AI",
        max_cost=0.01
    )
    
    if result["success"]:
        print(f"Response: {result['response']}")
        print(f"Cost: ${result['usage']['cost']:.6f}")
    else:
        print(f"Request blocked: {result['error']}")
    
    # Example 4: Model Cost Comparison
    print("\n" + "=" * 80)
    print("EXAMPLE 4: Model Cost Comparison")
    print("=" * 80)
    
    print("\nModel Pricing:")
    for model_name, pricing in TokenBudgetManager.MODEL_PRICING.items():
        print(f"\n{model_name} ({pricing.tier.value}):")
        print(f"  Input: ${pricing.input_cost_per_1k}/1K tokens")
        print(f"  Output: ${pricing.output_cost_per_1k}/1K tokens")
    
    # Calculate example costs
    print("\n\nExample cost for 1000 input + 500 output tokens:")
    for model_name in TokenBudgetManager.MODEL_PRICING.keys():
        cost = budget_manager3.calculate_cost(1000, 500, model_name)
        print(f"  {model_name}: ${cost:.6f}")
    
    # Example 5: Batch Processing with Budget
    print("\n" + "=" * 80)
    print("EXAMPLE 5: Batch Processing with Budget Control")
    print("=" * 80)
    
    config5 = BudgetConfig(total_budget=0.50)
    budget_manager5 = TokenBudgetManager(config5)
    agent5 = BudgetAwareLLMAgent(budget_manager5, model="gpt-3.5-turbo")
    
    batch_prompts = [
        "What is Python?",
        "What is JavaScript?",
        "What is Java?",
        "What is C++?",
        "What is Ruby?"
    ]
    
    print(f"\nProcessing {len(batch_prompts)} prompts with batch cost limit...")
    print(f"Batch cost limit: $0.05\n")
    
    results = agent5.batch_generate(batch_prompts, max_total_cost=0.05)
    
    successful = sum(1 for r in results if r["success"])
    total_cost = sum(r["usage"]["cost"] for r in results if r["success"] and r["usage"])
    
    print(f"\nBatch Results:")
    print(f"  Successful: {successful}/{len(batch_prompts)}")
    print(f"  Total Cost: ${total_cost:.6f}")
    print(f"  Average Cost: ${total_cost/max(1, successful):.6f}")
    
    # Summary
    print("\n" + "=" * 80)
    print("TOKEN BUDGET MANAGEMENT SUMMARY")
    print("=" * 80)
    print("""
Token Budget Management Benefits:
1. Cost Control: Prevent unexpected charges and budget overruns
2. Visibility: Real-time tracking of token usage and costs
3. Optimization: Identify opportunities to reduce token consumption
4. Accountability: Per-user or per-project cost attribution
5. Planning: Forecast future costs based on usage trends
6. Alerts: Proactive notifications when approaching limits

Key Components Demonstrated:
1. Real-Time Tracking: Automatic token and cost monitoring
2. Budget Thresholds: Warning, critical, and exceeded states
3. Cost Calculation: Model-specific pricing and estimates
4. Enforcement: Hard and soft budget limits
5. Alerts: Custom callback handlers for notifications
6. Analytics: Usage summaries and breakdowns

Budget Management Strategies:
1. Progressive Thresholds:
   - Normal:  100% of budget

2. Cost Optimization:
   - Use cheaper models when quality allows
   - Compress verbose prompts
   - Limit response lengths
   - Cache common queries
   - Batch similar requests

3. Monitoring:
   - Real-time dashboards
   - Usage trend analysis
   - Anomaly detection
   - Regular reporting
   - Forecasting

Best Practices:
1. Set realistic budgets based on historical data
2. Implement progressive enforcement (soft → hard limits)
3. Monitor usage patterns and trends
4. Optimize prompts to reduce token consumption
5. Use alerts to catch issues early
6. Regular budget reviews and adjustments
7. Cost attribution by user/project/feature
8. Document budget policies clearly

Model Selection for Cost:
- gpt-3.5-turbo: Cheapest, good for simple tasks
- gpt-4-turbo: Balanced cost/quality
- gpt-4: Premium, best quality but expensive

Production Considerations:
- Accurate token counting (model-specific)
- Up-to-date pricing information
- Efficient usage data storage
- Scalable tracking architecture
- User-friendly budget notifications
- Grace periods for critical operations
- Emergency override procedures
- Regular auditing and reconciliation

Common Use Cases:
- SaaS applications with freemium tiers
- Multi-tenant platforms
- Development/testing environments
- High-volume production systems
- Enterprise cost centers
- Research project budgets
- Educational platforms with quotas
""")
    
    print("\n" + "=" * 80)
    print("Pattern 088 (Token Budget Management) demonstration complete!")
    print("=" * 80)


if __name__ == "__main__":
    demonstrate_token_budget_management()

    Agentic Design Pattern: Token Budget

    Pattern 088: Token Budget – Line-by-Line Explanation

    Overview

    This document provides a detailed line-by-line explanation of the Token Budget pattern implementation.

    Pattern Description

    Pattern 088: Token Budget Management

    Description: The Token Budget Management pattern monitors and controls token usage across LLM interactions to optimize costs, prevent budget overruns, and ensure efficient resource utilization. This pattern is critical for production deployments where token costs can escalate quickly, especially with high-volume applications or complex multi-step workflows.

    Token budget management involves:
- Real-time token usage tracking
- Cost estimation and forecasting
- Budget enforcement and alerts
- Optimization strategies for token efficiency
- Usage analytics and reporting

    Components: 1. Token Counting – Input token counting (prompt tokens) – Output token counting (completion tokens) – Total token tracking per request – Cumulative usage tracking – Model-specific token limits

    2. Cost Calculation
   - Model pricing tables (per-token costs)
   - Request cost estimation
   - Cumulative cost tracking
   - Budget vs actual comparison
   - Cost forecasting

3. Budget Enforcement
   - Hard limits (stop when exceeded)
   - Soft limits (warnings when approaching)
   - Rate-based budgets (tokens per hour/day)
   - User-specific quotas
   - Priority-based allocation

4. Optimization Strategies
   - Prompt compression
   - Response length limiting
   - Model selection (cost vs quality)
   - Caching frequently used responses
   - Batching requests

    Use Cases: 1. Cost Control – Prevent unexpected charges – Stay within allocated budgets – Cost attribution per user/project – Predictable monthly spending – Resource optimization

    2. Multi-Tenant Applications
   - Per-user token quotas
   - Fair resource allocation
   - Usage-based billing
   - Tier-based limits (free/premium)
   - Abuse prevention

3. Development and Testing
   - Sandbox environments with limits
   - Cost-effective experimentation
   - Testing budget allocation
   - Development vs production budgets
   - CI/CD pipeline limits

4. High-Volume Applications
   - Aggregate usage tracking
   - Peak usage management
   - Resource planning
   - Scaling decisions
   - Cost optimization at scale

5. Enterprise Deployments
   - Department-level budgets
   - Project-based allocation
   - Chargeback mechanisms
   - Compliance and auditing
   - Executive reporting

    LangChain Implementation: LangChain supports token budget management through: – Callback handlers for token tracking – get_openai_callback for cost monitoring – Custom token counters – Usage analytics integration – Budget enforcement wrappers

    Key Features: 1. Real-Time Tracking – Immediate token counting – Running total updates – Cost calculation on-the-fly – Budget threshold monitoring – Alert triggering

    2. Multi-Model Support
   - Different pricing for different models
   - Model-specific token limits
   - Automatic model selection
   - Cost-aware routing
   - Fallback to cheaper models

3. Granular Control
   - Request-level limits
   - Session-level budgets
   - User-level quotas
   - Project-level allocations
   - Global organization limits

4. Analytics and Reporting
   - Usage trends over time
   - Cost breakdown by model
   - Per-user consumption
   - Peak usage identification
   - Cost forecasting

    Best Practices: 1. Set Appropriate Budgets – Based on historical data – Include buffer for peaks – Align with business value – Regular review and adjustment – Clear escalation procedures

    2. Implement Progressive Enforcement
   - Soft warnings at 70-80%
   - Stricter controls at 90%
   - Hard stops at 100%
   - Grace period for critical operations
   - Override mechanisms for emergencies

3. Optimize Token Usage
   - Compress verbose prompts
   - Limit response lengths
   - Use cheaper models when sufficient
   - Cache common queries
   - Batch similar requests

4. Monitor and Alert
   - Real-time dashboards
   - Automated alerts
   - Anomaly detection
   - Trend analysis
   - Regular reporting

    Trade-offs: Advantages: – Predictable costs – Prevents budget overruns – Enables cost optimization – Fair resource allocation – Usage visibility and control – Informed scaling decisions

    Disadvantages:
- Implementation complexity
- Tracking overhead (minimal)
- May limit functionality at budget
- Requires ongoing management
- False positives in alerting
- User friction from limits

    Production Considerations: 1. Accuracy – Precise token counting – Up-to-date pricing information – Correct model identification – Cache hit accounting – Network overhead handling

    2. Performance
   - Minimal tracking overhead
   - Efficient storage of usage data
   - Fast budget checks
   - Optimized queries for analytics
   - Scalable architecture

3. User Experience
   - Clear budget status visibility
   - Helpful error messages
   - Grace period handling
   - Budget increase requests
   - Usage optimization tips

4. Maintenance
   - Regular pricing updates
   - Budget policy reviews
   - Alert threshold tuning
   - Anomaly investigation
   - Capacity planning

5. Security
   - Access control for budget data
   - Secure token counting
   - Audit trail for usage
   - Fraud detection
   - Quota manipulation prevention

    File Statistics

    • Total Lines: 863
    • Classes: 7
    • Functions: 1
    • Constants: 0

    Module Documentation

    """
Pattern 088: Token Budget Management

Description:
    The Token Budget Management pattern monitors and controls token usage across LLM
    interactions to optimize costs, prevent budget overruns, and ensure efficient
    resource utilization. This pattern is critical for production deployments where
    token costs can escalate quickly, especially with high-volume applications or
    complex multi-step workflows.

    Token budget management involves:
    - Real-time token usage tracking
    - Cost estimation and forecasting
    - Budget enforcement and alerts
    - Optimization strategies for token efficiency
    - Usage analytics and reporting

Components:
    1. Token Counting
       - Input token counting (prompt tokens)
       - Output token counting (completion tokens)
       - Total token tracking per request
       - Cumulative usage tracking
       - Model-specific token limits

    2. Cost Calculation
       - Model pricing tables (per-token costs)
       - Request cost estimation
       - Cumulative cost tracking
       - Budget vs actual comparison
       - Cost forecasting

    3. Budget Enforcement
       - Hard limits (stop when exceeded)
       - Soft limits (warnings when approaching)
       - Rate-based budgets (tokens per hour/day)
       - User-specific quotas
       - Priority-based allocation

    4. Optimization Strategies
       - Prompt compression
       - Response length limiting
       - Model selection (cost vs quality)
       - Caching frequently used responses
       - Batching requests

Use Cases:
    1. Cost Control
       - Prevent unexpected charges
       - Stay within allocated budgets
       - Cost attribution per user/project
       - Predictable monthly spending
       - Resource optimization

    2. Multi-Tenant Applications
       - Per-user token quotas
       - Fair resource allocation
       - Usage-based billing
       - Tier-based limits (free/premium)
       - Abuse prevention

    3. Development and Testing
       - Sandbox environments with limits
       - Cost-effective experimentation
       - Testing budget allocation
       - Development vs production budgets
       - CI/CD pipeline limits

    4. High-Volume Applications
       - Aggregate usage tracking
       - Peak usage management
       - Resource planning
       - Scaling decisions
       - Cost optimization at scale

    5. Enterprise Deployments
       - Department-level budgets
       - Project-based allocation
       - Chargeback mechanisms
       - Compliance and auditing
       - Executive reporting

LangChain Implementation:
    LangChain supports token budget management through:
    - Callback handlers for token tracking
    - get_openai_callback for cost monitoring
    - Custom token counters
    - Usage analytics integration
    - Budget enforcement wrappers

Key Features:
    1. Real-Time Tracking
       - Immediate token counting
       - Running total updates
       - Cost calculation on-the-fly
       - Budget threshold monitoring
       - Alert triggering

    2. Multi-Model Support
       - Different pricing for different models
       - Model-specific token limits
       - Automatic model selection
       - Cost-aware routing
       - Fallback to cheaper models

    3. Granular Control
       - Request-level limits
       - Session-level budgets
       - User-level quotas
       - Project-level allocations
       - Global organization limits

    4. Analytics and Reporting
       - Usage trends over time
       - Cost breakdown by model
       - Per-user consumption
       - Peak usage identification
       - Cost forecasting

Best Practices:
    1. Set Appropriate Budgets
       - Based on historical data
       - Include buffer for peaks
       - Align with business value
       - Regular review and adjustment
       - Clear escalation procedures

    2. Implement Progressive Enforcement
       - Soft warnings at 70-80%
       - Stricter controls at 90%
       - Hard stops at 100%
       - Grace period for critical operations
       - Override mechanisms for emergencies

    3. Optimize Token Usage
       - Compress verbose prompts
       - Limit response lengths
       - Use cheaper models when sufficient
       - Cache common queries
       - Batch similar requests

    4. Monitor and Alert
       - Real-time dashboards
       - Automated alerts
       - Anomaly detection
       - Trend analysis
       - Regular reporting

Trade-offs:
    Advantages:
    - Predictable costs
    - Prevents budget overruns
    - Enables cost optimization
    - Fair resource allocation
    - Usage visibility and control
    - Informed scaling decisions

    Disadvantages:
    - Implementation complexity
    - Tracking overhead (minimal)
    - May limit functionality at budget
    - Requires ongoing management
    - False positives in alerting
    - User friction from limits

Production Considerations:
    1. Accuracy
       - Precise token counting
       - Up-to-date pricing information
       - Correct model identification
       - Cache hit accounting
       - Network overhead handling

    2. Performance
       - Minimal tracking overhead
       - Efficient storage of usage data
       - Fast budget checks
       - Optimized queries for analytics
       - Scalable architecture

    3. User Experience
       - Clear budget status visibility
       - Helpful error messages
       - Grace period handling
       - Budget increase requests
       - Usage optimization tips

    4. Maintenance
       - Regular pricing updates
       - Budget policy reviews
       - Alert threshold tuning
       - Anomaly investigation
       - Capacity planning

    5. Security
       - Access control for budget data
       - Secure token counting
       - Audit trail for usage
       - Fraud detection
       - Quota manipulation prevention
"""

    Purpose: This module implements the Token Budget pattern, providing a structured approach to agent design and implementation.

    Import Statements

    Standard Library Imports

    import collections.defaultdict
import dataclasses.dataclass
import dataclasses.field
import datetime.datetime
import datetime.timedelta
import enum.Enum
import langchain_core.output_parsers.StrOutputParser
import langchain_core.prompts.ChatPromptTemplate
import os
import time
import typing.Any
import typing.Callable
import typing.Dict
import typing.List
import typing.Optional

    Purpose: Core Python functionality: – collections.defaultdict: Specialized container datatypes – dataclasses.dataclass: Simplified class definitions with @dataclass – dataclasses.field: Simplified class definitions with @dataclass – datetime.datetime: Date and time handling – datetime.timedelta: Date and time handling – enum.Enum: Enumeration support – langchain_core.output_parsers.StrOutputParser: Regular expression operations – langchain_core.prompts.ChatPromptTemplate: Regular expression operations – os: Environment variable access and system operations – time: Time-related functions – typing.Any: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Callable: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Dict: Type hints for better code documentation (List, Dict, Any, Optional) – typing.List: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Optional: Type hints for better code documentation (List, Dict, Any, Optional)

    LangChain Imports

    from langchain.callbacks.get_openai_callback
from langchain_openai.ChatOpenAI

    Purpose: LangChain framework components: – LLM Interface: Wrapper for OpenAI models – Chains: Composable building blocks for complex workflows

    Other Imports

    from dotenv.load_dotenv

    Purpose: Additional dependencies for specialized functionality.

    Classes

    Class: BudgetStatus

    Lines 220-225

    Inherits from: Enum

    Purpose:

    Budget status levels

    Class: ModelTier

    Lines 228-232

    Inherits from: Enum

    Purpose:

    Model cost tiers

    Class: ModelPricing

    Lines 236-241

    Purpose:

    Pricing information for a model

    Class: TokenUsage

    Lines 245-252

    Purpose:

    Token usage for a single request

    Class: BudgetConfig

    Lines 256-263

    Purpose:

    Configuration for token budget management

    Class: TokenBudgetManager

    Lines 266-469

    Purpose:

    Manages token budgets and costs across LLM interactions.

This manager:
1. Tracks token usage in real-time
2. Calculates costs based on model pricing
3. Enforces budget limits
4. Provides usage analytics

    Methods:

    __init__(self, config)

    Lines 284-296

    Purpose:

    Initialize token budget manager.

Args:
    config: Budget configuration

    Parameters: – config: Config

    add_alert_callback(self, callback)

    Lines 298-300

    Purpose:

    Add callback for budget alerts

    Parameters: – callback: Callback

    _trigger_alerts(self, status, details)

    Lines 302-308

    Purpose:

    Trigger all registered alert callbacks

    Parameters: – status: Status – details: Details

    get_status(self)

    Lines 310-321

    Purpose:

    Get current budget status

    Parameters:

    Returns: BudgetStatus

    can_proceed(self, estimated_cost)

    Lines 323-349

    Purpose:

    Check if request can proceed within budget.

Args:
    estimated_cost: Estimated cost of request

Returns:
    Tuple of (can_proceed, reason)

    Parameters: – estimated_cost: Estimated cost

    Returns: <ast.Subscript object at 0x000001A06D71B2D0>

    calculate_cost(self, prompt_tokens, completion_tokens, model)

    Lines 351-376

    Purpose:

    Calculate cost for token usage.

Args:
    prompt_tokens: Number of prompt tokens
    completion_tokens: Number of completion tokens
    model: Model name

Returns:
    Cost in USD

    Parameters: – prompt_tokens: Prompt template or text – completion_tokens: Completion tokens – model: Language model instance

    Returns: float

    record_usage(self, prompt_tokens, completion_tokens, model)

    Lines 378-422

    Purpose:

    Record token usage and update budget.

Args:
    prompt_tokens: Number of prompt tokens
    completion_tokens: Number of completion tokens
    model: Model name

Returns:
    TokenUsage object

    Parameters: – prompt_tokens: Prompt template or text – completion_tokens: Completion tokens – model: Language model instance

    Returns: TokenUsage

    get_remaining_budget(self)

    Lines 424-426

    Purpose:

    Get remaining budget

    Parameters:

    Returns: float

    get_usage_summary(self)

    Lines 428-442

    Purpose:

    Get comprehensive usage summary

    Parameters:

    Returns: <ast.Subscript object at 0x000001A06D857210>

    suggest_cheaper_model(self, current_model)

    Lines 444-459

    Purpose:

    Suggest a cheaper model alternative

    Parameters: – current_model: Language model instance

    Returns: <ast.Subscript object at 0x000001A06D8505D0>

    reset_budget(self, new_budget)

    Lines 461-469

    Purpose:

    Reset budget and usage tracking

    Parameters: – new_budget: New budget

    Class: BudgetAwareLLMAgent

    Lines 472-616

    Purpose:

    LLM agent with token budget awareness.

This agent:
1. Tracks token usage automatically
2. Enforces budget limits
3. Optimizes token usage
4. Provides cost visibility

    Methods:

    __init__(self, budget_manager, model)

    Lines 483-497

    Purpose:

    Initialize budget-aware agent.

Args:
    budget_manager: Token budget manager
    model: Model to use (defaults to config default)

    Parameters: – budget_manager: Budget manager – model: Language model instance

    generate(self, prompt, max_tokens)

    Lines 499-554

    Purpose:

    Generate response with budget tracking.

Args:
    prompt: Input prompt
    max_tokens: Maximum tokens for response

Returns:
    Dictionary with response and usage info

    Parameters: – prompt: Prompt template or text – max_tokens: Maximum tokens to generate

    Returns: <ast.Subscript object at 0x000001A06D83EF90>

    generate_with_limit(self, prompt, max_cost)

    Lines 556-580

    Purpose:

    Generate response with per-request cost limit.

Args:
    prompt: Input prompt
    max_cost: Maximum cost for this request

Returns:
    Dictionary with response and usage info

    Parameters: – prompt: Prompt template or text – max_cost: Max cost

    Returns: <ast.Subscript object at 0x000001A06D88B2D0>

    batch_generate(self, prompts, max_total_cost)

    Lines 582-616

    Purpose:

    Generate responses for multiple prompts with budget tracking.

Args:
    prompts: List of prompts
    max_total_cost: Optional cost limit for batch

Returns:
    List of results

    Parameters: – prompts: Prompt template or text – max_total_cost: Max total cost

    Returns: <ast.Subscript object at 0x000001A06D625290>

    Functions

    demonstrate_token_budget_management()

    Lines 619-858

    Purpose:

    Demonstrate token budget management patterns

    Key Concepts Summary

    Pattern Overview

    The Token Budget pattern provides a structured approach to building AI agents with specific capabilities and behaviors. This implementation leverages LangChain’s composable components to create a robust and maintainable solution.

    Core Components

    1. Agent Class: Main implementation of the pattern logic
    2. Prompt Engineering: Carefully crafted prompts to guide LLM behavior
    3. Tool Integration: Optional tools for extended functionality
    4. Memory Management: State handling across interactions
    5. Output Parsing: Structured response processing

    When to Use

    ✅ Good for: – Scenarios requiring this specific pattern’s strengths – Applications needing structured agent behavior – Systems where explainability is important – Production environments requiring reliability

    ❌ Not ideal for: – Simple single-prompt tasks – Scenarios requiring different pattern characteristics – Resource-constrained environments (if computationally intensive)

    Best Practices Demonstrated

    1. Environment Configuration

    • ✅ Use .env files for API keys
    • ✅ Never hardcode credentials
    • ✅ Validate environment variables on startup

    2. Error Handling

    • ✅ Graceful degradation on failures
    • ✅ Informative error messages
    • ✅ Logging for debugging

    3. Prompt Engineering

    • ✅ Clear, specific instructions
    • ✅ Structured output formats
    • ✅ Few-shot examples when beneficial

    4. Code Organization

    • ✅ Separation of concerns (class-based design)
    • ✅ Type hints for clarity
    • ✅ Docstrings for documentation

    5. Performance Optimization

    • ✅ Appropriate temperature settings
    • ✅ Token budget management
    • ✅ Caching where applicable

    LangChain Components Used

    • ChatOpenAI: OpenAI model interface
    • Chains: Sequential component composition

    Why LangChain? – Composable abstractions for LLM applications – Rich ecosystem of integrations – Production-ready patterns – Active community and development

    Execution Flow

    1. Initialize agent with configuration
   ↓
2. Load environment and dependencies
   ↓
3. Prepare prompt with input data
   ↓
4. Invoke LLM through LangChain
   ↓
5. Parse and validate response
   ↓
6. Return formatted result

    Key Decision Points: – Model selection (temperature, model type) – Prompt structure (few-shot vs zero-shot) – Error handling strategy – Output format validation

    Performance Considerations

    Computational Cost

    • LLM Calls: Primary cost factor (based on tokens)
    • Latency: Depends on model and input size
    • Memory: Minimal for most patterns

    Optimization Strategies

    1. Prompt Optimization: Minimize tokens while maintaining clarity
    2. Caching: Cache repeated queries/responses
    3. Model Selection: Use smaller models when appropriate
    4. Batch Processing: Group similar requests
    5. Streaming: For long responses, stream tokens

    Cost Estimates (Approximate)

    • GPT-3.5-turbo: ~$0.002 per request
    • GPT-4: ~$0.03 per request
    • GPT-4-turbo: ~$0.01 per request

    Actual costs vary based on input/output token counts

    Common Pitfalls to Avoid

    1. Prompt Design Issues

    ❌ Vague or ambiguous instructions
    ✅ Clear, specific, testable prompts

    2. Error Handling

    ❌ Ignoring API errors or rate limits
    ✅ Implement retry logic and graceful degradation

    3. Token Management

    ❌ Exceeding model token limits
    ✅ Truncate inputs, chunk processing

    4. Security

    ❌ Exposing API keys in code
    ✅ Use environment variables

    5. Testing

    ❌ Only testing happy paths
    ✅ Test edge cases, failures, malformed inputs

    6. Cost Management

    ❌ Unmonitored API usage
    ✅ Implement budgets, alerts, and monitoring

    Further Improvements

    Production Enhancements

    1. Monitoring: Add logging, metrics, tracing
    2. Validation: Input/output schema validation
    3. Rate Limiting: Implement request throttling
    4. Caching: Redis or in-memory cache for responses
    5. Async Support: Non-blocking operations for scalability

    Feature Extensions

    1. Multi-Model Support: Switch between providers
    2. Fallback Models: Graceful degradation strategy
    3. Custom Tools: Domain-specific function calling
    4. Advanced Memory: Vector stores for long-term memory
    5. Evaluation: Automated testing and benchmarking

    Code Quality

    1. Testing: Unit tests, integration tests
    2. Documentation: API docs, usage examples
    3. Type Safety: Stricter type hints
    4. Configuration: Externalized config files

    Related Patterns

    • Pattern 087
    • Pattern 089

    Pattern Families: – Reasoning Patterns (001-011) – Multi-Agent Patterns (012-019) – Tool Use Patterns (020-025) – Memory Patterns (026-031) – And many more…

    Summary

    The Token Budget pattern provides a powerful approach to building AI agents with specific capabilities. This implementation demonstrates:

    ✅ Key Strengths: – Clean, maintainable code structure – LangChain integration for composability – Type hints and documentation – Production-ready patterns

    🎯 Best Use Cases: – Applications requiring this pattern’s characteristics – Systems needing structured agent behavior – Production environments requiring reliability

    📚 Learning Resources: – LangChain Documentation: https://python.langchain.com/ – Pattern Documentation: See related patterns – OpenAI API Reference: https://platform.openai.com/docs/

    File: 088_token_budget.py
    Total Lines: 863
    Pattern Category: AI Agent Design Pattern #088

    Generated from actual implementation code
    For educational and reference purposes

  • Agentic Design Pattern: Constraint Specification

    
    """
Pattern 087: Constraint Specification Pattern

Description:
    The Constraint Specification pattern explicitly defines boundaries, rules, and
    limitations that the LLM must adhere to when generating responses. This pattern
    is crucial for controlling LLM behavior, ensuring compliance with requirements,
    and preventing undesired outputs. Constraints can be content-based, format-based,
    style-based, or domain-specific, and they guide the model toward appropriate
    responses while filtering out inappropriate ones.

    Constraint specification is essential for:
    - Safety and compliance requirements
    - Brand voice and style consistency
    - Content appropriateness filtering
    - Domain-specific requirements
    - Length and complexity control
    - Factual accuracy boundaries

Components:
    1. Constraint Types
       - Content constraints (what to include/exclude)
       - Format constraints (structure and presentation)
       - Style constraints (tone, formality, vocabulary)
       - Length constraints (word/character limits)
       - Domain constraints (topic boundaries)
       - Safety constraints (harmful content prevention)

    2. Constraint Expression
       - Explicit rules ("must", "must not")
       - Positive constraints (requirements)
       - Negative constraints (prohibitions)
       - Conditional constraints (if-then rules)
       - Priority levels (strict vs flexible)

    3. Enforcement Strategies
       - Pre-generation (prompt engineering)
       - Post-generation (validation and filtering)
       - Iterative refinement (generate-validate-refine)
       - Guardrails (safety layers)
       - Fallback handling (when constraints violated)

    4. Validation Methods
       - Rule-based checking
       - Pattern matching
       - Content analysis
       - Length verification
       - Format validation
       - Safety scoring

Use Cases:
    1. Content Safety
       - Prevent harmful content generation
       - Filter inappropriate language
       - Block sensitive topics
       - Ensure age-appropriate content
       - Compliance with platform policies

    2. Brand Consistency
       - Maintain brand voice
       - Use approved terminology
       - Follow style guidelines
       - Consistent tone across content
       - Corporate communication standards

    3. Domain Expertise
       - Stay within knowledge boundaries
       - Acknowledge uncertainty
       - Avoid medical/legal claims
       - Respect professional standards
       - Domain-specific terminology

    4. Format and Structure
       - Length limitations (tweets, abstracts)
       - Structural requirements (sections, headings)
       - Citation requirements
       - Formatting rules (markdown, HTML)
       - Template compliance

    5. Legal and Compliance
       - GDPR compliance (no PII without consent)
       - HIPAA compliance (no PHI exposure)
       - Copyright respect
       - Regulatory requirements
       - Terms of service adherence

LangChain Implementation:
    LangChain supports constraint specification through:
    - System messages with explicit rules
    - Constitutional AI principles
    - Output validators and filters
    - Custom guardrails
    - Retry mechanisms with corrections

Key Features:
    1. Multi-Level Constraints
       - Hard constraints (must be satisfied)
       - Soft constraints (preferred but flexible)
       - Conditional constraints (context-dependent)
       - Hierarchical constraints (priority ordering)

    2. Constraint Composition
       - Multiple constraints simultaneously
       - Constraint conflicts resolution
       - Priority-based enforcement
       - Constraint inheritance

    3. Validation and Feedback
       - Immediate validation
       - Detailed violation reporting
       - Correction suggestions
       - Iterative improvement

    4. Dynamic Constraints
       - Context-aware constraints
       - User-specific rules
       - Time-based constraints
       - Adaptive boundaries

Best Practices:
    1. Clarity and Specificity
       - State constraints explicitly
       - Provide examples of violations
       - Clarify edge cases
       - Use concrete language

    2. Prioritization
       - Identify critical constraints
       - Allow flexibility where possible
       - Balance constraints with quality
       - Avoid over-constraining

    3. Testing
       - Test constraint enforcement
       - Try boundary cases
       - Verify violation detection
       - Check false positives

    4. Documentation
       - Document all constraints
       - Explain rationale
       - Provide violation examples
       - Maintain constraint catalog

Trade-offs:
    Advantages:
    - Better control over outputs
    - Improved safety and compliance
    - Consistent brand voice
    - Reduced inappropriate content
    - Clear expectations
    - Easier validation

    Disadvantages:
    - May limit creativity
    - Can reduce output quality
    - Increases complexity
    - Requires careful design
    - May need frequent updates
    - Potential over-constraint

Production Considerations:
    1. Constraint Management
       - Centralized constraint definitions
       - Version control for constraints
       - Easy updates and modifications
       - Constraint testing framework
       - Documentation and examples

    2. Performance Impact
       - Validation overhead
       - Retry costs for violations
       - Caching validated outputs
       - Efficient constraint checking
       - Batch validation

    3. User Experience
       - Clear violation messages
       - Suggestion for corrections
       - Graceful degradation
       - Transparency about constraints
       - User constraint customization

    4. Monitoring
       - Track constraint violations
       - Analyze violation patterns
       - Measure enforcement effectiveness
       - Identify constraint conflicts
       - User feedback collection

    5. Evolution
       - Regular constraint reviews
       - Adapt to new requirements
       - Learn from violations
       - Community feedback
       - A/B test constraint changes
"""

import os
import re
from typing import List, Dict, Any, Optional, Callable
from dataclasses import dataclass
from enum import Enum
from dotenv import load_dotenv

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_openai import ChatOpenAI

load_dotenv()


class ConstraintType(Enum):
    """Types of constraints"""
    CONTENT = "content"
    LENGTH = "length"
    STYLE = "style"
    FORMAT = "format"
    SAFETY = "safety"
    DOMAIN = "domain"


class ConstraintSeverity(Enum):
    """Severity levels for constraints"""
    STRICT = "strict"  # Must be satisfied
    MODERATE = "moderate"  # Should be satisfied
    FLEXIBLE = "flexible"  # Nice to have


@dataclass
class Constraint:
    """Represents a single constraint"""
    name: str
    description: str
    constraint_type: ConstraintType
    severity: ConstraintSeverity
    validation_func: Optional[Callable[[str], bool]] = None
    violation_message: str = "Constraint violated"


@dataclass
class ConstraintConfig:
    """Configuration for constraint-based generation"""
    max_retries: int = 3
    temperature: float = 0.5
    model_name: str = "gpt-3.5-turbo"
    fail_on_violation: bool = False


class ConstraintSpecificationAgent:
    """
    Agent that enforces constraints on LLM outputs.
    
    This agent demonstrates:
    1. Defining multiple types of constraints
    2. Enforcing constraints during generation
    3. Validating outputs against constraints
    4. Handling constraint violations
    """
    
    def __init__(self, config: Optional[ConstraintConfig] = None):
        """
        Initialize constraint specification agent.
        
        Args:
            config: Configuration for constraint handling
        """
        self.config = config or ConstraintConfig()
        self.llm = ChatOpenAI(
            model=self.config.model_name,
            temperature=self.config.temperature
        )
        self.constraints: List[Constraint] = []
    
    def add_constraint(self, constraint: Constraint):
        """Add a constraint to the agent"""
        self.constraints.append(constraint)
    
    def clear_constraints(self):
        """Remove all constraints"""
        self.constraints = []
    
    def _build_constraint_instructions(self) -> str:
        """Build constraint instructions for prompt"""
        if not self.constraints:
            return ""
        
        instructions = ["You must follow these constraints:"]
        
        for i, constraint in enumerate(self.constraints, 1):
            severity_label = {
                ConstraintSeverity.STRICT: "MUST",
                ConstraintSeverity.MODERATE: "SHOULD",
                ConstraintSeverity.FLEXIBLE: "IDEALLY"
            }[constraint.severity]
            
            instructions.append(f"{i}. {severity_label}: {constraint.description}")
        
        return "\n".join(instructions)
    
    def _validate_constraints(self, text: str) -> Dict[str, Any]:
        """
        Validate text against all constraints.
        
        Args:
            text: Text to validate
            
        Returns:
            Validation results
        """
        violations = []
        
        for constraint in self.constraints:
            if constraint.validation_func:
                if not constraint.validation_func(text):
                    violations.append({
                        "constraint": constraint.name,
                        "severity": constraint.severity.value,
                        "message": constraint.violation_message
                    })
        
        return {
            "valid": len(violations) == 0,
            "violations": violations,
            "num_violations": len(violations)
        }
    
    def generate_with_constraints(
        self,
        prompt: str,
        task_description: str = "Generate a response"
    ) -> Dict[str, Any]:
        """
        Generate text while enforcing constraints.
        
        Args:
            prompt: User prompt
            task_description: Description of task
            
        Returns:
            Generation result with validation info
        """
        constraint_instructions = self._build_constraint_instructions()
        
        template = ChatPromptTemplate.from_messages([
            ("system", """{task_description}

{constraints}

Ensure your response satisfies all constraints."""),
            ("human", "{prompt}")
        ])
        
        chain = template | self.llm | StrOutputParser()
        
        for attempt in range(self.config.max_retries):
            # Generate
            result = chain.invoke({
                "task_description": task_description,
                "constraints": constraint_instructions,
                "prompt": prompt
            })
            
            # Validate
            validation = self._validate_constraints(result)
            
            if validation["valid"]:
                return {
                    "success": True,
                    "text": result,
                    "attempts": attempt + 1,
                    "validation": validation
                }
            
            # If not last attempt, provide feedback for retry
            if attempt  str:
        """
        Generate text with length constraints.
        
        Args:
            prompt: User prompt
            max_words: Maximum word count
            min_words: Optional minimum word count
            
        Returns:
            Generated text
        """
        self.clear_constraints()
        
        # Add length constraint
        def validate_length(text: str) -> bool:
            word_count = len(text.split())
            if min_words and word_count  max_words:
                return False
            return True
        
        length_desc = f"between {min_words} and {max_words} words" if min_words else f"at most {max_words} words"
        
        self.add_constraint(Constraint(
            name="length",
            description=f"Response must be {length_desc}",
            constraint_type=ConstraintType.LENGTH,
            severity=ConstraintSeverity.STRICT,
            validation_func=validate_length,
            violation_message=f"Length constraint violated (required: {length_desc})"
        ))
        
        result = self.generate_with_constraints(prompt)
        return result["text"] if result["success"] else result.get("error", "Generation failed")
    
    def generate_with_style_constraint(
        self,
        prompt: str,
        style: str = "professional"
    ) -> str:
        """
        Generate text with style constraints.
        
        Args:
            prompt: User prompt
            style: Desired style (professional, casual, formal)
            
        Returns:
            Generated text
        """
        self.clear_constraints()
        
        style_guidelines = {
            "professional": "Use professional language, avoid slang, maintain objective tone",
            "casual": "Use conversational language, contractions are fine, friendly tone",
            "formal": "Use formal language, no contractions, academic or business tone",
            "humorous": "Use humor and wit, light-hearted tone, creative wordplay"
        }
        
        guideline = style_guidelines.get(style.lower(), style_guidelines["professional"])
        
        self.add_constraint(Constraint(
            name="style",
            description=guideline,
            constraint_type=ConstraintType.STYLE,
            severity=ConstraintSeverity.MODERATE
        ))
        
        result = self.generate_with_constraints(
            prompt,
            task_description=f"Generate a {style} response"
        )
        
        return result["text"] if result["success"] else result.get("text", "")
    
    def generate_with_content_constraints(
        self,
        prompt: str,
        must_include: Optional[List[str]] = None,
        must_exclude: Optional[List[str]] = None
    ) -> str:
        """
        Generate text with content inclusion/exclusion constraints.
        
        Args:
            prompt: User prompt
            must_include: Terms that must appear
            must_exclude: Terms that must not appear
            
        Returns:
            Generated text
        """
        self.clear_constraints()
        
        # Inclusion constraint
        if must_include:
            def validate_inclusion(text: str) -> bool:
                text_lower = text.lower()
                return all(term.lower() in text_lower for term in must_include)
            
            self.add_constraint(Constraint(
                name="content_inclusion",
                description=f"Must include these terms: {', '.join(must_include)}",
                constraint_type=ConstraintType.CONTENT,
                severity=ConstraintSeverity.STRICT,
                validation_func=validate_inclusion,
                violation_message=f"Required terms missing: {', '.join(must_include)}"
            ))
        
        # Exclusion constraint
        if must_exclude:
            def validate_exclusion(text: str) -> bool:
                text_lower = text.lower()
                return not any(term.lower() in text_lower for term in must_exclude)
            
            self.add_constraint(Constraint(
                name="content_exclusion",
                description=f"Must NOT include these terms: {', '.join(must_exclude)}",
                constraint_type=ConstraintType.CONTENT,
                severity=ConstraintSeverity.STRICT,
                validation_func=validate_exclusion,
                violation_message=f"Prohibited terms found: {', '.join(must_exclude)}"
            ))
        
        result = self.generate_with_constraints(prompt)
        return result["text"] if result["success"] else result.get("text", "")
    
    def generate_with_format_constraint(
        self,
        prompt: str,
        format_type: str = "bullet_points"
    ) -> str:
        """
        Generate text with format constraints.
        
        Args:
            prompt: User prompt
            format_type: Format type (bullet_points, numbered_list, paragraphs)
            
        Returns:
            Generated text
        """
        self.clear_constraints()
        
        format_descriptions = {
            "bullet_points": "Use bullet points (- or •) for each item. No numbered lists.",
            "numbered_list": "Use numbered list format (1. 2. 3. etc.). No bullet points.",
            "paragraphs": "Write in paragraph form with clear topic sentences. No lists.",
            "single_sentence": "Provide answer as a single, concise sentence."
        }
        
        description = format_descriptions.get(format_type, format_descriptions["bullet_points"])
        
        self.add_constraint(Constraint(
            name="format",
            description=description,
            constraint_type=ConstraintType.FORMAT,
            severity=ConstraintSeverity.STRICT
        ))
        
        result = self.generate_with_constraints(
            prompt,
            task_description=f"Generate response in {format_type} format"
        )
        
        return result["text"] if result["success"] else result.get("text", "")
    
    def generate_with_safety_constraints(
        self,
        prompt: str
    ) -> str:
        """
        Generate text with safety constraints.
        
        Args:
            prompt: User prompt
            
        Returns:
            Generated text
        """
        self.clear_constraints()
        
        safety_rules = [
            "No harmful, hateful, or violent content",
            "No personal attacks or discrimination",
            "No illegal activities or advice",
            "No explicit or inappropriate content",
            "Respectful and inclusive language"
        ]
        
        for rule in safety_rules:
            self.add_constraint(Constraint(
                name=f"safety_{safety_rules.index(rule)}",
                description=rule,
                constraint_type=ConstraintType.SAFETY,
                severity=ConstraintSeverity.STRICT
            ))
        
        result = self.generate_with_constraints(
            prompt,
            task_description="Generate a safe and appropriate response"
        )
        
        return result["text"] if result["success"] else "Unable to generate safe content for this prompt."


def demonstrate_constraint_specification():
    """Demonstrate constraint specification patterns"""
    print("=" * 80)
    print("CONSTRAINT SPECIFICATION PATTERN DEMONSTRATION")
    print("=" * 80)
    
    agent = ConstraintSpecificationAgent()
    
    # Example 1: Length Constraints
    print("\n" + "=" * 80)
    print("EXAMPLE 1: Length Constraints")
    print("=" * 80)
    
    prompt = "Explain artificial intelligence"
    print(f"\nPrompt: {prompt}")
    print("\nConstraint: Maximum 50 words\n")
    
    short_response = agent.generate_with_length_constraint(prompt, max_words=50)
    print(f"Response ({len(short_response.split())} words):")
    print(short_response)
    
    print("\nConstraint: 80-120 words\n")
    medium_response = agent.generate_with_length_constraint(prompt, max_words=120, min_words=80)
    print(f"Response ({len(medium_response.split())} words):")
    print(medium_response)
    
    # Example 2: Style Constraints
    print("\n" + "=" * 80)
    print("EXAMPLE 2: Style Constraints")
    print("=" * 80)
    
    prompt = "Describe the benefits of exercise"
    print(f"\nPrompt: {prompt}\n")
    
    styles = ["professional", "casual", "formal"]
    for style in styles:
        print(f"\n{style.upper()} STYLE:")
        print("-" * 40)
        response = agent.generate_with_style_constraint(prompt, style=style)
        print(response)
    
    # Example 3: Content Inclusion/Exclusion
    print("\n" + "=" * 80)
    print("EXAMPLE 3: Content Constraints (Must Include/Exclude)")
    print("=" * 80)
    
    prompt = "Write about healthy eating habits"
    must_include = ["vegetables", "protein", "hydration"]
    must_exclude = ["diet pills", "supplements"]
    
    print(f"\nPrompt: {prompt}")
    print(f"Must include: {must_include}")
    print(f"Must exclude: {must_exclude}\n")
    
    response = agent.generate_with_content_constraints(
        prompt,
        must_include=must_include,
        must_exclude=must_exclude
    )
    print("Response:")
    print(response)
    
    # Example 4: Format Constraints
    print("\n" + "=" * 80)
    print("EXAMPLE 4: Format Constraints")
    print("=" * 80)
    
    prompt = "List the steps to start a business"
    print(f"\nPrompt: {prompt}\n")
    
    formats = ["bullet_points", "numbered_list", "paragraphs"]
    for fmt in formats:
        print(f"\n{fmt.upper().replace('_', ' ')} FORMAT:")
        print("-" * 40)
        response = agent.generate_with_format_constraint(prompt, format_type=fmt)
        print(response)
    
    # Example 5: Multiple Constraints
    print("\n" + "=" * 80)
    print("EXAMPLE 5: Multiple Simultaneous Constraints")
    print("=" * 80)
    
    agent.clear_constraints()
    
    # Add multiple constraints
    agent.add_constraint(Constraint(
        name="length",
        description="Response must be 60-80 words",
        constraint_type=ConstraintType.LENGTH,
        severity=ConstraintSeverity.STRICT,
        validation_func=lambda t: 60 <= len(t.split()) <= 80,
        violation_message="Length must be 60-80 words"
    ))
    
    agent.add_constraint(Constraint(
        name="style",
        description="Use professional, objective tone",
        constraint_type=ConstraintType.STYLE,
        severity=ConstraintSeverity.MODERATE
    ))
    
    agent.add_constraint(Constraint(
        name="content",
        description="Must mention 'innovation' and 'technology'",
        constraint_type=ConstraintType.CONTENT,
        severity=ConstraintSeverity.STRICT,
        validation_func=lambda t: "innovation" in t.lower() and "technology" in t.lower(),
        violation_message="Must include 'innovation' and 'technology'"
    ))
    
    prompt = "Describe the future of work"
    print(f"\nPrompt: {prompt}")
    print("\nConstraints:")
    print("1. Length: 60-80 words")
    print("2. Style: Professional, objective")
    print("3. Content: Must mention 'innovation' and 'technology'\n")
    
    result = agent.generate_with_constraints(prompt)
    print(f"Success: {result['success']}")
    print(f"Attempts: {result['attempts']}")
    print(f"\nResponse ({len(result['text'].split())} words):")
    print(result['text'])
    
    # Example 6: Safety Constraints
    print("\n" + "=" * 80)
    print("EXAMPLE 6: Safety Constraints")
    print("=" * 80)
    
    safe_prompt = "Give advice on managing workplace conflicts"
    print(f"\nPrompt: {safe_prompt}")
    print("\nApplying safety constraints...\n")
    
    response = agent.generate_with_safety_constraints(safe_prompt)
    print("Response:")
    print(response)
    
    # Summary
    print("\n" + "=" * 80)
    print("CONSTRAINT SPECIFICATION SUMMARY")
    print("=" * 80)
    print("""
Constraint Specification Pattern Benefits:
1. Control: Precise control over LLM outputs
2. Compliance: Ensures regulatory and policy adherence
3. Consistency: Maintains brand voice and standards
4. Safety: Prevents inappropriate content
5. Quality: Enforces quality standards
6. Integration: Easier to integrate with systems

Constraint Types Demonstrated:
1. Length Constraints: Word/character limits
2. Style Constraints: Tone, formality, voice
3. Content Constraints: Required/prohibited terms
4. Format Constraints: Structure and presentation
5. Safety Constraints: Harmful content prevention
6. Multiple Constraints: Combined requirements

Constraint Severity Levels:
- STRICT: Must be satisfied (violations cause failure)
- MODERATE: Should be satisfied (violations logged)
- FLEXIBLE: Nice to have (soft preferences)

Enforcement Strategies:
1. Pre-generation: Include constraints in prompt
2. Validation: Check outputs against rules
3. Iteration: Retry with feedback on violations
4. Fallback: Alternative approach when constraints can't be met

Best Practices:
1. Be explicit and specific with constraints
2. Prioritize constraints (strict vs flexible)
3. Provide examples of acceptable outputs
4. Test constraint enforcement thoroughly
5. Handle violations gracefully
6. Monitor constraint effectiveness
7. Update constraints based on feedback

Common Use Cases:
- Content moderation (safety)
- Brand consistency (style)
- Legal compliance (domain rules)
- Format standardization (structure)
- Length requirements (summaries, tweets)
- Quality control (standards)

Production Considerations:
- Balance constraints with creativity
- Allow appropriate flexibility
- Provide clear violation messages
- Implement retry mechanisms
- Monitor constraint violation rates
- A/B test constraint effectiveness
- Document all constraints clearly
- Version control constraint rules

Constraint Design Tips:
1. Start with essential constraints only
2. Add constraints incrementally
3. Test against diverse inputs
4. Avoid over-constraining
5. Make constraints measurable
6. Provide violation feedback
7. Allow for constraint evolution

When to Use Constraints:
- Safety-critical applications
- Regulated industries
- Brand-sensitive content
- System integration requirements
- Quality assurance needs
- Compliance requirements

Trade-offs:
✅ Better control and safety
✅ Consistent outputs
✅ Easier validation
❌ May limit creativity
❌ Requires careful design
❌ Can be overly restrictive
""")
    
    print("\n" + "=" * 80)
    print("Pattern 087 (Constraint Specification) demonstration complete!")
    print("=" * 80)


if __name__ == "__main__":
    demonstrate_constraint_specification()

    Agentic Design Pattern: Constraint Specification

    Pattern 087: Constraint Specification – Line-by-Line Explanation

    Overview

    This document provides a detailed line-by-line explanation of the Constraint Specification pattern implementation.

    Pattern Description

    Pattern 087: Constraint Specification Pattern

    Description: The Constraint Specification pattern explicitly defines boundaries, rules, and limitations that the LLM must adhere to when generating responses. This pattern is crucial for controlling LLM behavior, ensuring compliance with requirements, and preventing undesired outputs. Constraints can be content-based, format-based, style-based, or domain-specific, and they guide the model toward appropriate responses while filtering out inappropriate ones.

    Constraint specification is essential for:
- Safety and compliance requirements
- Brand voice and style consistency
- Content appropriateness filtering
- Domain-specific requirements
- Length and complexity control
- Factual accuracy boundaries

    Components: 1. Constraint Types – Content constraints (what to include/exclude) – Format constraints (structure and presentation) – Style constraints (tone, formality, vocabulary) – Length constraints (word/character limits) – Domain constraints (topic boundaries) – Safety constraints (harmful content prevention)

    2. Constraint Expression
   - Explicit rules ("must", "must not")
   - Positive constraints (requirements)
   - Negative constraints (prohibitions)
   - Conditional constraints (if-then rules)
   - Priority levels (strict vs flexible)

3. Enforcement Strategies
   - Pre-generation (prompt engineering)
   - Post-generation (validation and filtering)
   - Iterative refinement (generate-validate-refine)
   - Guardrails (safety layers)
   - Fallback handling (when constraints violated)

4. Validation Methods
   - Rule-based checking
   - Pattern matching
   - Content analysis
   - Length verification
   - Format validation
   - Safety scoring

    Use Cases: 1. Content Safety – Prevent harmful content generation – Filter inappropriate language – Block sensitive topics – Ensure age-appropriate content – Compliance with platform policies

    2. Brand Consistency
   - Maintain brand voice
   - Use approved terminology
   - Follow style guidelines
   - Consistent tone across content
   - Corporate communication standards

3. Domain Expertise
   - Stay within knowledge boundaries
   - Acknowledge uncertainty
   - Avoid medical/legal claims
   - Respect professional standards
   - Domain-specific terminology

4. Format and Structure
   - Length limitations (tweets, abstracts)
   - Structural requirements (sections, headings)
   - Citation requirements
   - Formatting rules (markdown, HTML)
   - Template compliance

5. Legal and Compliance
   - GDPR compliance (no PII without consent)
   - HIPAA compliance (no PHI exposure)
   - Copyright respect
   - Regulatory requirements
   - Terms of service adherence

    LangChain Implementation: LangChain supports constraint specification through: – System messages with explicit rules – Constitutional AI principles – Output validators and filters – Custom guardrails – Retry mechanisms with corrections

    Key Features: 1. Multi-Level Constraints – Hard constraints (must be satisfied) – Soft constraints (preferred but flexible) – Conditional constraints (context-dependent) – Hierarchical constraints (priority ordering)

    2. Constraint Composition
   - Multiple constraints simultaneously
   - Constraint conflicts resolution
   - Priority-based enforcement
   - Constraint inheritance

3. Validation and Feedback
   - Immediate validation
   - Detailed violation reporting
   - Correction suggestions
   - Iterative improvement

4. Dynamic Constraints
   - Context-aware constraints
   - User-specific rules
   - Time-based constraints
   - Adaptive boundaries

    Best Practices: 1. Clarity and Specificity – State constraints explicitly – Provide examples of violations – Clarify edge cases – Use concrete language

    2. Prioritization
   - Identify critical constraints
   - Allow flexibility where possible
   - Balance constraints with quality
   - Avoid over-constraining

3. Testing
   - Test constraint enforcement
   - Try boundary cases
   - Verify violation detection
   - Check false positives

4. Documentation
   - Document all constraints
   - Explain rationale
   - Provide violation examples
   - Maintain constraint catalog

    Trade-offs: Advantages: – Better control over outputs – Improved safety and compliance – Consistent brand voice – Reduced inappropriate content – Clear expectations – Easier validation

    Disadvantages:
- May limit creativity
- Can reduce output quality
- Increases complexity
- Requires careful design
- May need frequent updates
- Potential over-constraint

    Production Considerations: 1. Constraint Management – Centralized constraint definitions – Version control for constraints – Easy updates and modifications – Constraint testing framework – Documentation and examples

    2. Performance Impact
   - Validation overhead
   - Retry costs for violations
   - Caching validated outputs
   - Efficient constraint checking
   - Batch validation

3. User Experience
   - Clear violation messages
   - Suggestion for corrections
   - Graceful degradation
   - Transparency about constraints
   - User constraint customization

4. Monitoring
   - Track constraint violations
   - Analyze violation patterns
   - Measure enforcement effectiveness
   - Identify constraint conflicts
   - User feedback collection

5. Evolution
   - Regular constraint reviews
   - Adapt to new requirements
   - Learn from violations
   - Community feedback
   - A/B test constraint changes

    File Statistics

    • Total Lines: 835
    • Classes: 5
    • Functions: 1
    • Constants: 0

    Module Documentation

    """
Pattern 087: Constraint Specification Pattern

Description:
    The Constraint Specification pattern explicitly defines boundaries, rules, and
    limitations that the LLM must adhere to when generating responses. This pattern
    is crucial for controlling LLM behavior, ensuring compliance with requirements,
    and preventing undesired outputs. Constraints can be content-based, format-based,
    style-based, or domain-specific, and they guide the model toward appropriate
    responses while filtering out inappropriate ones.

    Constraint specification is essential for:
    - Safety and compliance requirements
    - Brand voice and style consistency
    - Content appropriateness filtering
    - Domain-specific requirements
    - Length and complexity control
    - Factual accuracy boundaries

Components:
    1. Constraint Types
       - Content constraints (what to include/exclude)
       - Format constraints (structure and presentation)
       - Style constraints (tone, formality, vocabulary)
       - Length constraints (word/character limits)
       - Domain constraints (topic boundaries)
       - Safety constraints (harmful content prevention)

    2. Constraint Expression
       - Explicit rules ("must", "must not")
       - Positive constraints (requirements)
       - Negative constraints (prohibitions)
       - Conditional constraints (if-then rules)
       - Priority levels (strict vs flexible)

    3. Enforcement Strategies
       - Pre-generation (prompt engineering)
       - Post-generation (validation and filtering)
       - Iterative refinement (generate-validate-refine)
       - Guardrails (safety layers)
       - Fallback handling (when constraints violated)

    4. Validation Methods
       - Rule-based checking
       - Pattern matching
       - Content analysis
       - Length verification
       - Format validation
       - Safety scoring

Use Cases:
    1. Content Safety
       - Prevent harmful content generation
       - Filter inappropriate language
       - Block sensitive topics
       - Ensure age-appropriate content
       - Compliance with platform policies

    2. Brand Consistency
       - Maintain brand voice
       - Use approved terminology
       - Follow style guidelines
       - Consistent tone across content
       - Corporate communication standards

    3. Domain Expertise
       - Stay within knowledge boundaries
       - Acknowledge uncertainty
       - Avoid medical/legal claims
       - Respect professional standards
       - Domain-specific terminology

    4. Format and Structure
       - Length limitations (tweets, abstracts)
       - Structural requirements (sections, headings)
       - Citation requirements
       - Formatting rules (markdown, HTML)
       - Template compliance

    5. Legal and Compliance
       - GDPR compliance (no PII without consent)
       - HIPAA compliance (no PHI exposure)
       - Copyright respect
       - Regulatory requirements
       - Terms of service adherence

LangChain Implementation:
    LangChain supports constraint specification through:
    - System messages with explicit rules
    - Constitutional AI principles
    - Output validators and filters
    - Custom guardrails
    - Retry mechanisms with corrections

Key Features:
    1. Multi-Level Constraints
       - Hard constraints (must be satisfied)
       - Soft constraints (preferred but flexible)
       - Conditional constraints (context-dependent)
       - Hierarchical constraints (priority ordering)

    2. Constraint Composition
       - Multiple constraints simultaneously
       - Constraint conflicts resolution
       - Priority-based enforcement
       - Constraint inheritance

    3. Validation and Feedback
       - Immediate validation
       - Detailed violation reporting
       - Correction suggestions
       - Iterative improvement

    4. Dynamic Constraints
       - Context-aware constraints
       - User-specific rules
       - Time-based constraints
       - Adaptive boundaries

Best Practices:
    1. Clarity and Specificity
       - State constraints explicitly
       - Provide examples of violations
       - Clarify edge cases
       - Use concrete language

    2. Prioritization
       - Identify critical constraints
       - Allow flexibility where possible
       - Balance constraints with quality
       - Avoid over-constraining

    3. Testing
       - Test constraint enforcement
       - Try boundary cases
       - Verify violation detection
       - Check false positives

    4. Documentation
       - Document all constraints
       - Explain rationale
       - Provide violation examples
       - Maintain constraint catalog

Trade-offs:
    Advantages:
    - Better control over outputs
    - Improved safety and compliance
    - Consistent brand voice
    - Reduced inappropriate content
    - Clear expectations
    - Easier validation

    Disadvantages:
    - May limit creativity
    - Can reduce output quality
    - Increases complexity
    - Requires careful design
    - May need frequent updates
    - Potential over-constraint

Production Considerations:
    1. Constraint Management
       - Centralized constraint definitions
       - Version control for constraints
       - Easy updates and modifications
       - Constraint testing framework
       - Documentation and examples

    2. Performance Impact
       - Validation overhead
       - Retry costs for violations
       - Caching validated outputs
       - Efficient constraint checking
       - Batch validation

    3. User Experience
       - Clear violation messages
       - Suggestion for corrections
       - Graceful degradation
       - Transparency about constraints
       - User constraint customization

    4. Monitoring
       - Track constraint violations
       - Analyze violation patterns
       - Measure enforcement effectiveness
       - Identify constraint conflicts
       - User feedback collection

    5. Evolution
       - Regular constraint reviews
       - Adapt to new requirements
       - Learn from violations
       - Community feedback
       - A/B test constraint changes
"""

    Purpose: This module implements the Constraint Specification pattern, providing a structured approach to agent design and implementation.

    Import Statements

    Standard Library Imports

    import dataclasses.dataclass
import enum.Enum
import langchain_core.output_parsers.StrOutputParser
import langchain_core.prompts.ChatPromptTemplate
import os
import re
import typing.Any
import typing.Callable
import typing.Dict
import typing.List
import typing.Optional

    Purpose: Core Python functionality: – dataclasses.dataclass: Simplified class definitions with @dataclass – enum.Enum: Enumeration support – langchain_core.output_parsers.StrOutputParser: Regular expression operations – langchain_core.prompts.ChatPromptTemplate: Regular expression operations – os: Environment variable access and system operations – re: Regular expression operations – typing.Any: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Callable: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Dict: Type hints for better code documentation (List, Dict, Any, Optional) – typing.List: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Optional: Type hints for better code documentation (List, Dict, Any, Optional)

    LangChain Imports

    from langchain_openai.ChatOpenAI

    Purpose: LangChain framework components: – LLM Interface: Wrapper for OpenAI models – Chains: Composable building blocks for complex workflows

    Other Imports

    from dotenv.load_dotenv

    Purpose: Additional dependencies for specialized functionality.

    Classes

    Class: ConstraintType

    Lines 213-220

    Inherits from: Enum

    Purpose:

    Types of constraints

    Class: ConstraintSeverity

    Lines 223-227

    Inherits from: Enum

    Purpose:

    Severity levels for constraints

    Class: Constraint

    Lines 231-238

    Purpose:

    Represents a single constraint

    Class: ConstraintConfig

    Lines 242-247

    Purpose:

    Configuration for constraint-based generation

    Class: ConstraintSpecificationAgent

    Lines 250-604

    Purpose:

    Agent that enforces constraints on LLM outputs.

This agent demonstrates:
1. Defining multiple types of constraints
2. Enforcing constraints during generation
3. Validating outputs against constraints
4. Handling constraint violations

    Methods:

    __init__(self, config)

    Lines 261-273

    Purpose:

    Initialize constraint specification agent.

Args:
    config: Configuration for constraint handling

    Parameters: – config: Config

    add_constraint(self, constraint)

    Lines 275-277

    Purpose:

    Add a constraint to the agent

    Parameters: – constraint: Constraint

    clear_constraints(self)

    Lines 279-281

    Purpose:

    Remove all constraints

    Parameters:

    _build_constraint_instructions(self)

    Lines 283-299

    Purpose:

    Build constraint instructions for prompt

    Parameters:

    Returns: str

    _validate_constraints(self, text)

    Lines 301-326

    Purpose:

    Validate text against all constraints.

Args:
    text: Text to validate

Returns:
    Validation results

    Parameters: – text: Text

    Returns: <ast.Subscript object at 0x000001A06D7F4B90>

    generate_with_constraints(self, prompt, task_description)

    Lines 328-392

    Purpose:

    Generate text while enforcing constraints.

Args:
    prompt: User prompt
    task_description: Description of task

Returns:
    Generation result with validation info

    Parameters: – prompt: Prompt template or text – task_description: Task description

    Returns: <ast.Subscript object at 0x000001A06D849590>

    generate_with_length_constraint(self, prompt, max_words, min_words)

    Lines 394-434

    Purpose:

    Generate text with length constraints.

Args:
    prompt: User prompt
    max_words: Maximum word count
    min_words: Optional minimum word count

Returns:
    Generated text

    Parameters: – prompt: Prompt template or text – max_words: Max words – min_words: Min words

    Returns: str

    generate_with_style_constraint(self, prompt, style)

    Lines 436-474

    Purpose:

    Generate text with style constraints.

Args:
    prompt: User prompt
    style: Desired style (professional, casual, formal)

Returns:
    Generated text

    Parameters: – prompt: Prompt template or text – style: Style

    Returns: str

    generate_with_content_constraints(self, prompt, must_include, must_exclude)

    Lines 476-526

    Purpose:

    Generate text with content inclusion/exclusion constraints.

Args:
    prompt: User prompt
    must_include: Terms that must appear
    must_exclude: Terms that must not appear

Returns:
    Generated text

    Parameters: – prompt: Prompt template or text – must_include: Must include – must_exclude: Must exclude

    Returns: str

    generate_with_format_constraint(self, prompt, format_type)

    Lines 528-566

    Purpose:

    Generate text with format constraints.

Args:
    prompt: User prompt
    format_type: Format type (bullet_points, numbered_list, paragraphs)

Returns:
    Generated text

    Parameters: – prompt: Prompt template or text – format_type: Format type

    Returns: str

    generate_with_safety_constraints(self, prompt)

    Lines 568-604

    Purpose:

    Generate text with safety constraints.

Args:
    prompt: User prompt

Returns:
    Generated text

    Parameters: – prompt: Prompt template or text

    Returns: str

    Functions

    demonstrate_constraint_specification()

    Lines 607-830

    Purpose:

    Demonstrate constraint specification patterns

    Key Concepts Summary

    Pattern Overview

    The Constraint Specification pattern provides a structured approach to building AI agents with specific capabilities and behaviors. This implementation leverages LangChain’s composable components to create a robust and maintainable solution.

    Core Components

    1. Agent Class: Main implementation of the pattern logic
    2. Prompt Engineering: Carefully crafted prompts to guide LLM behavior
    3. Tool Integration: Optional tools for extended functionality
    4. Memory Management: State handling across interactions
    5. Output Parsing: Structured response processing

    When to Use

    ✅ Good for: – Scenarios requiring this specific pattern’s strengths – Applications needing structured agent behavior – Systems where explainability is important – Production environments requiring reliability

    ❌ Not ideal for: – Simple single-prompt tasks – Scenarios requiring different pattern characteristics – Resource-constrained environments (if computationally intensive)

    Best Practices Demonstrated

    1. Environment Configuration

    • ✅ Use .env files for API keys
    • ✅ Never hardcode credentials
    • ✅ Validate environment variables on startup

    2. Error Handling

    • ✅ Graceful degradation on failures
    • ✅ Informative error messages
    • ✅ Logging for debugging

    3. Prompt Engineering

    • ✅ Clear, specific instructions
    • ✅ Structured output formats
    • ✅ Few-shot examples when beneficial

    4. Code Organization

    • ✅ Separation of concerns (class-based design)
    • ✅ Type hints for clarity
    • ✅ Docstrings for documentation

    5. Performance Optimization

    • ✅ Appropriate temperature settings
    • ✅ Token budget management
    • ✅ Caching where applicable

    LangChain Components Used

    • ChatOpenAI: OpenAI model interface
    • Chains: Sequential component composition

    Why LangChain? – Composable abstractions for LLM applications – Rich ecosystem of integrations – Production-ready patterns – Active community and development

    Execution Flow

    1. Initialize agent with configuration
   ↓
2. Load environment and dependencies
   ↓
3. Prepare prompt with input data
   ↓
4. Invoke LLM through LangChain
   ↓
5. Parse and validate response
   ↓
6. Return formatted result

    Key Decision Points: – Model selection (temperature, model type) – Prompt structure (few-shot vs zero-shot) – Error handling strategy – Output format validation

    Performance Considerations

    Computational Cost

    • LLM Calls: Primary cost factor (based on tokens)
    • Latency: Depends on model and input size
    • Memory: Minimal for most patterns

    Optimization Strategies

    1. Prompt Optimization: Minimize tokens while maintaining clarity
    2. Caching: Cache repeated queries/responses
    3. Model Selection: Use smaller models when appropriate
    4. Batch Processing: Group similar requests
    5. Streaming: For long responses, stream tokens

    Cost Estimates (Approximate)

    • GPT-3.5-turbo: ~$0.002 per request
    • GPT-4: ~$0.03 per request
    • GPT-4-turbo: ~$0.01 per request

    Actual costs vary based on input/output token counts

    Common Pitfalls to Avoid

    1. Prompt Design Issues

    ❌ Vague or ambiguous instructions
    ✅ Clear, specific, testable prompts

    2. Error Handling

    ❌ Ignoring API errors or rate limits
    ✅ Implement retry logic and graceful degradation

    3. Token Management

    ❌ Exceeding model token limits
    ✅ Truncate inputs, chunk processing

    4. Security

    ❌ Exposing API keys in code
    ✅ Use environment variables

    5. Testing

    ❌ Only testing happy paths
    ✅ Test edge cases, failures, malformed inputs

    6. Cost Management

    ❌ Unmonitored API usage
    ✅ Implement budgets, alerts, and monitoring

    Further Improvements

    Production Enhancements

    1. Monitoring: Add logging, metrics, tracing
    2. Validation: Input/output schema validation
    3. Rate Limiting: Implement request throttling
    4. Caching: Redis or in-memory cache for responses
    5. Async Support: Non-blocking operations for scalability

    Feature Extensions

    1. Multi-Model Support: Switch between providers
    2. Fallback Models: Graceful degradation strategy
    3. Custom Tools: Domain-specific function calling
    4. Advanced Memory: Vector stores for long-term memory
    5. Evaluation: Automated testing and benchmarking

    Code Quality

    1. Testing: Unit tests, integration tests
    2. Documentation: API docs, usage examples
    3. Type Safety: Stricter type hints
    4. Configuration: Externalized config files

    Related Patterns

    • Pattern 086
    • Pattern 088

    Pattern Families: – Reasoning Patterns (001-011) – Multi-Agent Patterns (012-019) – Tool Use Patterns (020-025) – Memory Patterns (026-031) – And many more…

    Summary

    The Constraint Specification pattern provides a powerful approach to building AI agents with specific capabilities. This implementation demonstrates:

    ✅ Key Strengths: – Clean, maintainable code structure – LangChain integration for composability – Type hints and documentation – Production-ready patterns

    🎯 Best Use Cases: – Applications requiring this pattern’s characteristics – Systems needing structured agent behavior – Production environments requiring reliability

    📚 Learning Resources: – LangChain Documentation: https://python.langchain.com/ – Pattern Documentation: See related patterns – OpenAI API Reference: https://platform.openai.com/docs/

    File: 087_constraint_specification.py
    Total Lines: 835
    Pattern Category: AI Agent Design Pattern #087

    Generated from actual implementation code
    For educational and reference purposes

  • Agentic Design Pattern: Output Format

    
    """
Pattern 086: Output Format Specification Pattern

Description:
    The Output Format Specification pattern explicitly defines the structure, format,
    and schema of the LLM's response. This pattern is crucial for integrating LLM
    outputs with downstream systems, APIs, and applications that require consistent,
    parseable, and structured data. By specifying the exact format upfront, this
    pattern eliminates ambiguity and ensures reliable programmatic processing.

    Format specification is essential for:
    - API integration and data exchange
    - Structured data extraction
    - Multi-step workflows with defined interfaces
    - Database storage and retrieval
    - User interface rendering
    - Automated testing and validation

Components:
    1. Format Type
       - JSON (JavaScript Object Notation)
       - XML (Extensible Markup Language)
       - YAML (YAML Ain't Markup Language)
       - CSV (Comma-Separated Values)
       - Markdown (formatted text)
       - Custom formats (domain-specific)

    2. Schema Definition
       - Field names and types
       - Required vs optional fields
       - Value constraints and ranges
       - Nested structures
       - Array/list specifications

    3. Format Enforcement
       - Clear examples in prompts
       - Schema validation
       - Error handling for format violations
       - Retry mechanisms
       - Fallback strategies

    4. Parsing and Validation
       - Automated parsing
       - Schema compliance checking
       - Type validation
       - Constraint verification
       - Error reporting

Use Cases:
    1. Data Extraction
       - Entity extraction from text → JSON
       - Information retrieval → structured records
       - Document parsing → database entries
       - Web scraping → CSV data
       - Form filling → key-value pairs

    2. API Integration
       - API request generation → JSON payloads
       - Response formatting → standardized structures
       - Webhook payloads → event schemas
       - Configuration files → YAML/JSON
       - Data exchange → XML documents

    3. Report Generation
       - Business reports → Markdown tables
       - Data summaries → formatted text
       - Analytics → JSON metrics
       - Documentation → structured sections
       - Dashboards → visualization specs

    4. Code Generation
       - Function signatures → specific syntax
       - Configuration files → language-specific format
       - Test cases → structured test specs
       - API documentation → OpenAPI/Swagger
       - Database schemas → SQL/DDL

    5. Content Creation
       - Blog posts → Markdown with frontmatter
       - Product descriptions → structured fields
       - Email templates → HTML with placeholders
       - Social media → platform-specific formats
       - Translations → parallel text structures

LangChain Implementation:
    LangChain provides excellent format control through:
    - StructuredOutputParser for schema-based parsing
    - JsonOutputParser for JSON responses
    - PydanticOutputParser for type-safe outputs
    - Custom output parsers for specific formats
    - Output fixing parser for auto-correction

Key Features:
    1. Type Safety
       - Pydantic models for validation
       - Automatic type checking
       - Type coercion when possible
       - Clear error messages

    2. Schema Validation
       - JSON Schema compliance
       - Custom validation rules
       - Nested object validation
       - Array element validation

    3. Error Recovery
       - Automatic retry with corrections
       - Format fixing parsers
       - Fallback to alternative formats
       - Graceful degradation

    4. Multiple Format Support
       - JSON (most common)
       - XML for legacy systems
       - CSV for tabular data
       - Markdown for documents
       - Custom formats as needed

Best Practices:
    1. Clear Format Specification
       - Provide explicit format examples
       - Use schema definitions when possible
       - Show complete structure (not partial)
       - Include edge cases in examples

    2. Validation Strategy
       - Validate immediately after generation
       - Provide clear error messages
       - Implement retry logic
       - Log format violations

    3. Error Handling
       - Graceful fallbacks
       - User-friendly error messages
       - Automatic format correction when possible
       - Manual intervention for critical failures

    4. Documentation
       - Document expected formats
       - Provide format examples
       - Explain field meanings
       - Note format versions

Trade-offs:
    Advantages:
    - Reliable programmatic processing
    - Easy integration with systems
    - Reduced parsing errors
    - Type safety and validation
    - Better error detection
    - Automated testing possible

    Disadvantages:
    - More rigid (less flexibility)
    - Format violations require handling
    - Increases prompt complexity
    - May constrain natural language quality
    - Requires careful schema design
    - Parsing overhead

Production Considerations:
    1. Schema Evolution
       - Version format schemas
       - Backward compatibility
       - Migration strategies
       - Deprecation policies

    2. Performance
       - Parsing efficiency
       - Validation overhead
       - Caching strategies
       - Batch processing

    3. Reliability
       - Retry logic for format errors
       - Format fixing mechanisms
       - Fallback formats
       - Monitoring and alerting

    4. Maintenance
       - Schema documentation
       - Format validation tests
       - Breaking change detection
       - Client SDK generation

    5. Security
       - Input sanitization
       - Injection prevention
       - Size limits
       - Type validation
"""

import os
import json
from typing import List, Dict, Any, Optional
from dataclasses import dataclass
from enum import Enum
from dotenv import load_dotenv
from pydantic import BaseModel, Field

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import (
    StrOutputParser,
    JsonOutputParser,
    PydanticOutputParser
)
from langchain_openai import ChatOpenAI

load_dotenv()


class OutputFormat(Enum):
    """Supported output formats"""
    JSON = "json"
    XML = "xml"
    MARKDOWN = "markdown"
    CSV = "csv"
    YAML = "yaml"


# Pydantic models for structured outputs
class Person(BaseModel):
    """Person entity"""
    name: str = Field(description="Full name of the person")
    age: Optional[int] = Field(description="Age in years", ge=0, le=150)
    email: Optional[str] = Field(description="Email address")
    occupation: Optional[str] = Field(description="Job title or occupation")


class Product(BaseModel):
    """Product information"""
    name: str = Field(description="Product name")
    category: str = Field(description="Product category")
    price: float = Field(description="Price in USD", gt=0)
    in_stock: bool = Field(description="Availability status")
    rating: Optional[float] = Field(description="Average rating", ge=0, le=5)


class Article(BaseModel):
    """Article/content structure"""
    title: str = Field(description="Article title")
    author: str = Field(description="Author name")
    summary: str = Field(description="Brief summary")
    tags: List[str] = Field(description="Content tags")
    word_count: int = Field(description="Approximate word count", gt=0)


class SentimentAnalysis(BaseModel):
    """Sentiment analysis result"""
    text: str = Field(description="Analyzed text")
    sentiment: str = Field(description="Sentiment: positive, negative, or neutral")
    confidence: float = Field(description="Confidence score", ge=0, le=1)
    keywords: List[str] = Field(description="Key terms identified")


@dataclass
class FormatConfig:
    """Configuration for format specification"""
    format_type: OutputFormat = OutputFormat.JSON
    strict_mode: bool = True
    temperature: float = 0.1  # Lower for more consistent formatting
    model_name: str = "gpt-3.5-turbo"


class OutputFormatAgent:
    """
    Agent that enforces specific output formats.
    
    This agent demonstrates:
    1. JSON format specification
    2. Pydantic-based type-safe outputs
    3. Multiple format support
    4. Schema validation
    """
    
    def __init__(self, config: Optional[FormatConfig] = None):
        """
        Initialize output format agent.
        
        Args:
            config: Configuration for format handling
        """
        self.config = config or FormatConfig()
        self.llm = ChatOpenAI(
            model=self.config.model_name,
            temperature=self.config.temperature
        )
    
    def extract_as_json(
        self,
        text: str,
        schema_description: str
    ) -> Dict[str, Any]:
        """
        Extract information in JSON format.
        
        Args:
            text: Text to extract from
            schema_description: Description of desired JSON structure
            
        Returns:
            Extracted data as dictionary
        """
        json_parser = JsonOutputParser()
        
        prompt = ChatPromptTemplate.from_messages([
            ("system", """Extract information from the text in JSON format.

Format requirements:
- Valid JSON only
- Follow the schema described
- Use appropriate data types
- Include all requested fields

{format_instructions}"""),
            ("human", "Extract from this text: {text}\n\nSchema: {schema}")
        ])
        
        chain = prompt | self.llm | json_parser
        
        try:
            result = chain.invoke({
                "text": text,
                "schema": schema_description,
                "format_instructions": json_parser.get_format_instructions()
            })
            return result
        except Exception as e:
            return {"error": str(e), "raw_text": text}
    
    def extract_person_info(self, text: str) -> Person:
        """
        Extract person information with type safety.
        
        Args:
            text: Text containing person information
            
        Returns:
            Person object with validated fields
        """
        parser = PydanticOutputParser(pydantic_object=Person)
        
        prompt = ChatPromptTemplate.from_messages([
            ("system", """Extract person information from the text.

{format_instructions}

Provide valid data for all available fields."""),
            ("human", "{text}")
        ])
        
        chain = prompt | self.llm | parser
        
        result = chain.invoke({
            "text": text,
            "format_instructions": parser.get_format_instructions()
        })
        
        return result
    
    def extract_product_info(self, text: str) -> Product:
        """
        Extract product information with validation.
        
        Args:
            text: Text containing product information
            
        Returns:
            Product object with validated fields
        """
        parser = PydanticOutputParser(pydantic_object=Product)
        
        prompt = ChatPromptTemplate.from_messages([
            ("system", """Extract product information from the text.

{format_instructions}

Ensure all data types are correct and constraints are met."""),
            ("human", "{text}")
        ])
        
        chain = prompt | self.llm | parser
        
        result = chain.invoke({
            "text": text,
            "format_instructions": parser.get_format_instructions()
        })
        
        return result
    
    def analyze_sentiment_structured(self, text: str) -> SentimentAnalysis:
        """
        Perform sentiment analysis with structured output.
        
        Args:
            text: Text to analyze
            
        Returns:
            SentimentAnalysis object with validated fields
        """
        parser = PydanticOutputParser(pydantic_object=SentimentAnalysis)
        
        prompt = ChatPromptTemplate.from_messages([
            ("system", """Analyze the sentiment of the text and provide results in the specified format.

{format_instructions}

- Sentiment must be: positive, negative, or neutral
- Confidence must be between 0 and 1
- Extract 3-5 relevant keywords"""),
            ("human", "{text}")
        ])
        
        chain = prompt | self.llm | parser
        
        result = chain.invoke({
            "text": text,
            "format_instructions": parser.get_format_instructions()
        })
        
        return result
    
    def generate_markdown_report(
        self,
        topic: str,
        sections: List[str]
    ) -> str:
        """
        Generate report in markdown format.
        
        Args:
            topic: Report topic
            sections: List of section titles
            
        Returns:
            Markdown formatted report
        """
        sections_list = "\n".join([f"- {s}" for s in sections])
        
        prompt = ChatPromptTemplate.from_messages([
            ("system", """Generate a report in Markdown format.

Format requirements:
- Use # for main title
- Use ## for section headers
- Use **bold** for emphasis
- Use bullet points (- ) for lists
- Use code blocks (```) for code
- Maintain consistent formatting"""),
            ("human", """Create a report on: {topic}

Include these sections:
{sections}""")
        ])
        
        chain = prompt | self.llm | StrOutputParser()
        
        result = chain.invoke({
            "topic": topic,
            "sections": sections_list
        })
        
        return result
    
    def generate_csv_format(
        self,
        data_description: str,
        num_rows: int = 5
    ) -> str:
        """
        Generate data in CSV format.
        
        Args:
            data_description: Description of data to generate
            num_rows: Number of data rows
            
        Returns:
            CSV formatted data
        """
        prompt = ChatPromptTemplate.from_messages([
            ("system", """Generate data in CSV format.

Format requirements:
- First row: column headers
- Subsequent rows: data values
- Use commas as separators
- Quote strings containing commas
- No extra whitespace
- Valid CSV syntax"""),
            ("human", """Generate {num_rows} rows of CSV data for: {description}""")
        ])
        
        chain = prompt | self.llm | StrOutputParser()
        
        result = chain.invoke({
            "description": data_description,
            "num_rows": num_rows
        })
        
        return result
    
    def generate_xml_format(self, data_description: str) -> str:
        """
        Generate data in XML format.
        
        Args:
            data_description: Description of data structure
            
        Returns:
            XML formatted data
        """
        prompt = ChatPromptTemplate.from_messages([
            ("system", """Generate data in XML format.

Format requirements:
- Proper XML declaration
- Well-formed tags (open and close)
- Proper nesting
- Attribute syntax when appropriate
- Valid XML structure"""),
            ("human", "Generate XML for: {description}")
        ])
        
        chain = prompt | self.llm | StrOutputParser()
        
        result = chain.invoke({"description": data_description})
        
        return result
    
    def compare_formats(self, data: str) -> Dict[str, str]:
        """
        Generate same data in multiple formats for comparison.
        
        Args:
            data: Data description
            
        Returns:
            Dictionary with data in different formats
        """
        formats = {}
        
        # JSON format
        json_result = self.extract_as_json(
            data,
            "Extract as JSON with appropriate fields"
        )
        formats["json"] = json.dumps(json_result, indent=2)
        
        # XML format
        formats["xml"] = self.generate_xml_format(data)
        
        # CSV format (if tabular)
        formats["csv"] = self.generate_csv_format(data, num_rows=3)
        
        return formats


def demonstrate_output_format():
    """Demonstrate output format specification patterns"""
    print("=" * 80)
    print("OUTPUT FORMAT SPECIFICATION PATTERN DEMONSTRATION")
    print("=" * 80)
    
    agent = OutputFormatAgent()
    
    # Example 1: JSON Extraction
    print("\n" + "=" * 80)
    print("EXAMPLE 1: JSON Format Extraction")
    print("=" * 80)
    
    text = "Apple Inc. released the iPhone 15 Pro in September 2023 priced at $999."
    schema = "Extract product name, company, release date, and price"
    print(f"\nText: {text}")
    print(f"Schema: {schema}\n")
    
    json_result = agent.extract_as_json(text, schema)
    print("JSON Output:")
    print(json.dumps(json_result, indent=2))
    
    # Example 2: Type-Safe Person Extraction
    print("\n" + "=" * 80)
    print("EXAMPLE 2: Type-Safe Person Information (Pydantic)")
    print("=" * 80)
    
    person_text = "Dr. Sarah Johnson is a 45-year-old cardiologist at Mayo Clinic. Her email is sarah.j@mayo.edu"
    print(f"\nText: {person_text}\n")
    
    person = agent.extract_person_info(person_text)
    print("Extracted Person (Type-Safe):")
    print(f"  Name: {person.name}")
    print(f"  Age: {person.age}")
    print(f"  Occupation: {person.occupation}")
    print(f"  Email: {person.email}")
    print(f"\nObject type: {type(person)}")
    
    # Example 3: Product Information
    print("\n" + "=" * 80)
    print("EXAMPLE 3: Structured Product Information")
    print("=" * 80)
    
    product_text = "The UltraBook Pro laptop in the Electronics category costs $1,299. Currently in stock with 4.5 star rating."
    print(f"\nText: {product_text}\n")
    
    product = agent.extract_product_info(product_text)
    print("Extracted Product:")
    print(f"  Name: {product.name}")
    print(f"  Category: {product.category}")
    print(f"  Price: ${product.price}")
    print(f"  In Stock: {product.in_stock}")
    print(f"  Rating: {product.rating}")
    
    # Example 4: Sentiment Analysis with Structure
    print("\n" + "=" * 80)
    print("EXAMPLE 4: Structured Sentiment Analysis")
    print("=" * 80)
    
    review = "This product is absolutely amazing! Best purchase I've made this year. Highly recommended!"
    print(f"\nReview: {review}\n")
    
    sentiment = agent.analyze_sentiment_structured(review)
    print("Sentiment Analysis Result:")
    print(f"  Sentiment: {sentiment.sentiment}")
    print(f"  Confidence: {sentiment.confidence:.2f}")
    print(f"  Keywords: {', '.join(sentiment.keywords)}")
    
    # Example 5: Markdown Report
    print("\n" + "=" * 80)
    print("EXAMPLE 5: Markdown Formatted Report")
    print("=" * 80)
    
    topic = "Quarterly Sales Performance"
    sections = ["Executive Summary", "Key Metrics", "Regional Analysis", "Recommendations"]
    print(f"\nTopic: {topic}")
    print(f"Sections: {sections}\n")
    
    markdown_report = agent.generate_markdown_report(topic, sections)
    print("Markdown Output:")
    print(markdown_report)
    
    # Example 6: CSV Format
    print("\n" + "=" * 80)
    print("EXAMPLE 6: CSV Format Generation")
    print("=" * 80)
    
    data_desc = "Employee records with name, department, salary, and hire date"
    print(f"\nData: {data_desc}\n")
    
    csv_data = agent.generate_csv_format(data_desc, num_rows=5)
    print("CSV Output:")
    print(csv_data)
    
    # Example 7: XML Format
    print("\n" + "=" * 80)
    print("EXAMPLE 7: XML Format Generation")
    print("=" * 80)
    
    xml_desc = "Book catalog with title, author, ISBN, and publication year"
    print(f"\nData: {xml_desc}\n")
    
    xml_data = agent.generate_xml_format(xml_desc)
    print("XML Output:")
    print(xml_data)
    
    # Summary
    print("\n" + "=" * 80)
    print("OUTPUT FORMAT SPECIFICATION SUMMARY")
    print("=" * 80)
    print("""
Output Format Pattern Benefits:
1. Programmatic Processing: Easy to parse and use in code
2. Type Safety: Validation ensures correct data types
3. Integration Ready: Works seamlessly with APIs and systems
4. Error Detection: Schema validation catches issues early
5. Documentation: Format serves as documentation
6. Testing: Enables automated testing

Key Formats Demonstrated:
1. JSON: Most common, flexible, widely supported
2. Pydantic Models: Type-safe Python objects with validation
3. Markdown: Human-readable formatted documents
4. CSV: Tabular data for spreadsheets
5. XML: Structured data for legacy systems

Format Specification Techniques:
1. Explicit Examples: Show exact format in prompt
2. Schema Definition: Use JSON Schema or Pydantic
3. Format Instructions: Parser-generated instructions
4. Validation: Automatic type and constraint checking
5. Error Recovery: Retry and fix malformed outputs

Best Practices:
1. Choose appropriate format for use case
   - JSON: APIs, web apps, general data
   - CSV: Tabular data, spreadsheets
   - XML: Legacy systems, complex hierarchies
   - Markdown: Documentation, reports
   
2. Use type safety when possible (Pydantic)
   - Automatic validation
   - IDE support
   - Clear error messages
   
3. Provide clear format instructions
   - Show complete examples
   - Specify all fields
   - Define data types and constraints
   
4. Handle format errors gracefully
   - Retry with corrections
   - Fallback formats
   - Clear error messages
   
5. Validate outputs
   - Schema compliance
   - Type checking
   - Constraint verification

Production Considerations:
- Lower temperature (0.1-0.3) for consistent formatting
- Implement retry logic for format violations
- Cache format instructions to save tokens
- Version schemas for evolution
- Monitor parsing success rates
- Log format errors for debugging
- Test edge cases thoroughly

Format Selection Guide:
- JSON: Default choice for most applications
- Pydantic: When type safety is critical
- CSV: For tabular data and spreadsheet integration
- XML: For legacy system compatibility
- Markdown: For human-readable documents
- YAML: For configuration files

Common Pitfalls:
- Insufficient format examples in prompts
- No validation of generated outputs
- Overly complex schemas
- Missing error handling
- Temperature too high (inconsistent outputs)
- Not handling optional fields properly
""")
    
    print("\n" + "=" * 80)
    print("Pattern 086 (Output Format Specification) demonstration complete!")
    print("=" * 80)


if __name__ == "__main__":
    demonstrate_output_format()

    Agentic Design Pattern: Output Format

    Pattern 086: Output Format – Line-by-Line Explanation

    Overview

    This document provides a detailed line-by-line explanation of the Output Format pattern implementation.

    Pattern Description

    Pattern 086: Output Format Specification Pattern

    Description: The Output Format Specification pattern explicitly defines the structure, format, and schema of the LLM’s response. This pattern is crucial for integrating LLM outputs with downstream systems, APIs, and applications that require consistent, parseable, and structured data. By specifying the exact format upfront, this pattern eliminates ambiguity and ensures reliable programmatic processing.

    Format specification is essential for:
- API integration and data exchange
- Structured data extraction
- Multi-step workflows with defined interfaces
- Database storage and retrieval
- User interface rendering
- Automated testing and validation

    Components: 1. Format Type – JSON (JavaScript Object Notation) – XML (Extensible Markup Language) – YAML (YAML Ain’t Markup Language) – CSV (Comma-Separated Values) – Markdown (formatted text) – Custom formats (domain-specific)

    2. Schema Definition
   - Field names and types
   - Required vs optional fields
   - Value constraints and ranges
   - Nested structures
   - Array/list specifications

3. Format Enforcement
   - Clear examples in prompts
   - Schema validation
   - Error handling for format violations
   - Retry mechanisms
   - Fallback strategies

4. Parsing and Validation
   - Automated parsing
   - Schema compliance checking
   - Type validation
   - Constraint verification
   - Error reporting

    Use Cases: 1. Data Extraction – Entity extraction from text → JSON – Information retrieval → structured records – Document parsing → database entries – Web scraping → CSV data – Form filling → key-value pairs

    2. API Integration
   - API request generation → JSON payloads
   - Response formatting → standardized structures
   - Webhook payloads → event schemas
   - Configuration files → YAML/JSON
   - Data exchange → XML documents

3. Report Generation
   - Business reports → Markdown tables
   - Data summaries → formatted text
   - Analytics → JSON metrics
   - Documentation → structured sections
   - Dashboards → visualization specs

4. Code Generation
   - Function signatures → specific syntax
   - Configuration files → language-specific format
   - Test cases → structured test specs
   - API documentation → OpenAPI/Swagger
   - Database schemas → SQL/DDL

5. Content Creation
   - Blog posts → Markdown with frontmatter
   - Product descriptions → structured fields
   - Email templates → HTML with placeholders
   - Social media → platform-specific formats
   - Translations → parallel text structures

    LangChain Implementation: LangChain provides excellent format control through: – StructuredOutputParser for schema-based parsing – JsonOutputParser for JSON responses – PydanticOutputParser for type-safe outputs – Custom output parsers for specific formats – Output fixing parser for auto-correction

    Key Features: 1. Type Safety – Pydantic models for validation – Automatic type checking – Type coercion when possible – Clear error messages

    2. Schema Validation
   - JSON Schema compliance
   - Custom validation rules
   - Nested object validation
   - Array element validation

3. Error Recovery
   - Automatic retry with corrections
   - Format fixing parsers
   - Fallback to alternative formats
   - Graceful degradation

4. Multiple Format Support
   - JSON (most common)
   - XML for legacy systems
   - CSV for tabular data
   - Markdown for documents
   - Custom formats as needed

    Best Practices: 1. Clear Format Specification – Provide explicit format examples – Use schema definitions when possible – Show complete structure (not partial) – Include edge cases in examples

    2. Validation Strategy
   - Validate immediately after generation
   - Provide clear error messages
   - Implement retry logic
   - Log format violations

3. Error Handling
   - Graceful fallbacks
   - User-friendly error messages
   - Automatic format correction when possible
   - Manual intervention for critical failures

4. Documentation
   - Document expected formats
   - Provide format examples
   - Explain field meanings
   - Note format versions

    Trade-offs: Advantages: – Reliable programmatic processing – Easy integration with systems – Reduced parsing errors – Type safety and validation – Better error detection – Automated testing possible

    Disadvantages:
- More rigid (less flexibility)
- Format violations require handling
- Increases prompt complexity
- May constrain natural language quality
- Requires careful schema design
- Parsing overhead

    Production Considerations: 1. Schema Evolution – Version format schemas – Backward compatibility – Migration strategies – Deprecation policies

    2. Performance
   - Parsing efficiency
   - Validation overhead
   - Caching strategies
   - Batch processing

3. Reliability
   - Retry logic for format errors
   - Format fixing mechanisms
   - Fallback formats
   - Monitoring and alerting

4. Maintenance
   - Schema documentation
   - Format validation tests
   - Breaking change detection
   - Client SDK generation

5. Security
   - Input sanitization
   - Injection prevention
   - Size limits
   - Type validation

    File Statistics

    • Total Lines: 751
    • Classes: 7
    • Functions: 1
    • Constants: 0

    Module Documentation

    """
Pattern 086: Output Format Specification Pattern

Description:
    The Output Format Specification pattern explicitly defines the structure, format,
    and schema of the LLM's response. This pattern is crucial for integrating LLM
    outputs with downstream systems, APIs, and applications that require consistent,
    parseable, and structured data. By specifying the exact format upfront, this
    pattern eliminates ambiguity and ensures reliable programmatic processing.

    Format specification is essential for:
    - API integration and data exchange
    - Structured data extraction
    - Multi-step workflows with defined interfaces
    - Database storage and retrieval
    - User interface rendering
    - Automated testing and validation

Components:
    1. Format Type
       - JSON (JavaScript Object Notation)
       - XML (Extensible Markup Language)
       - YAML (YAML Ain't Markup Language)
       - CSV (Comma-Separated Values)
       - Markdown (formatted text)
       - Custom formats (domain-specific)

    2. Schema Definition
       - Field names and types
       - Required vs optional fields
       - Value constraints and ranges
       - Nested structures
       - Array/list specifications

    3. Format Enforcement
       - Clear examples in prompts
       - Schema validation
       - Error handling for format violations
       - Retry mechanisms
       - Fallback strategies

    4. Parsing and Validation
       - Automated parsing
       - Schema compliance checking
       - Type validation
       - Constraint verification
       - Error reporting

Use Cases:
    1. Data Extraction
       - Entity extraction from text → JSON
       - Information retrieval → structured records
       - Document parsing → database entries
       - Web scraping → CSV data
       - Form filling → key-value pairs

    2. API Integration
       - API request generation → JSON payloads
       - Response formatting → standardized structures
       - Webhook payloads → event schemas
       - Configuration files → YAML/JSON
       - Data exchange → XML documents

    3. Report Generation
       - Business reports → Markdown tables
       - Data summaries → formatted text
       - Analytics → JSON metrics
       - Documentation → structured sections
       - Dashboards → visualization specs

    4. Code Generation
       - Function signatures → specific syntax
       - Configuration files → language-specific format
       - Test cases → structured test specs
       - API documentation → OpenAPI/Swagger
       - Database schemas → SQL/DDL

    5. Content Creation
       - Blog posts → Markdown with frontmatter
       - Product descriptions → structured fields
       - Email templates → HTML with placeholders
       - Social media → platform-specific formats
       - Translations → parallel text structures

LangChain Implementation:
    LangChain provides excellent format control through:
    - StructuredOutputParser for schema-based parsing
    - JsonOutputParser for JSON responses
    - PydanticOutputParser for type-safe outputs
    - Custom output parsers for specific formats
    - Output fixing parser for auto-correction

Key Features:
    1. Type Safety
       - Pydantic models for validation
       - Automatic type checking
       - Type coercion when possible
       - Clear error messages

    2. Schema Validation
       - JSON Schema compliance
       - Custom validation rules
       - Nested object validation
       - Array element validation

    3. Error Recovery
       - Automatic retry with corrections
       - Format fixing parsers
       - Fallback to alternative formats
       - Graceful degradation

    4. Multiple Format Support
       - JSON (most common)
       - XML for legacy systems
       - CSV for tabular data
       - Markdown for documents
       - Custom formats as needed

Best Practices:
    1. Clear Format Specification
       - Provide explicit format examples
       - Use schema definitions when possible
       - Show complete structure (not partial)
       - Include edge cases in examples

    2. Validation Strategy
       - Validate immediately after generation
       - Provide clear error messages
       - Implement retry logic
       - Log format violations

    3. Error Handling
       - Graceful fallbacks
       - User-friendly error messages
       - Automatic format correction when possible
       - Manual intervention for critical failures

    4. Documentation
       - Document expected formats
       - Provide format examples
       - Explain field meanings
       - Note format versions

Trade-offs:
    Advantages:
    - Reliable programmatic processing
    - Easy integration with systems
    - Reduced parsing errors
    - Type safety and validation
    - Better error detection
    - Automated testing possible

    Disadvantages:
    - More rigid (less flexibility)
    - Format violations require handling
    - Increases prompt complexity
    - May constrain natural language quality
    - Requires careful schema design
    - Parsing overhead

Production Considerations:
    1. Schema Evolution
       - Version format schemas
       - Backward compatibility
       - Migration strategies
       - Deprecation policies

    2. Performance
       - Parsing efficiency
       - Validation overhead
       - Caching strategies
       - Batch processing

    3. Reliability
       - Retry logic for format errors
       - Format fixing mechanisms
       - Fallback formats
       - Monitoring and alerting

    4. Maintenance
       - Schema documentation
       - Format validation tests
       - Breaking change detection
       - Client SDK generation

    5. Security
       - Input sanitization
       - Injection prevention
       - Size limits
       - Type validation
"""

    Purpose: This module implements the Output Format pattern, providing a structured approach to agent design and implementation.

    Import Statements

    Standard Library Imports

    import dataclasses.dataclass
import enum.Enum
import json
import langchain_core.output_parsers.JsonOutputParser
import langchain_core.output_parsers.PydanticOutputParser
import langchain_core.output_parsers.StrOutputParser
import langchain_core.prompts.ChatPromptTemplate
import os
import typing.Any
import typing.Dict
import typing.List
import typing.Optional

    Purpose: Core Python functionality: – dataclasses.dataclass: Simplified class definitions with @dataclass – enum.Enum: Enumeration support – json: JSON serialization/deserialization – langchain_core.output_parsers.JsonOutputParser: Regular expression operations – langchain_core.output_parsers.PydanticOutputParser: Regular expression operations – langchain_core.output_parsers.StrOutputParser: Regular expression operations – langchain_core.prompts.ChatPromptTemplate: Regular expression operations – os: Environment variable access and system operations – typing.Any: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Dict: Type hints for better code documentation (List, Dict, Any, Optional) – typing.List: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Optional: Type hints for better code documentation (List, Dict, Any, Optional)

    LangChain Imports

    from langchain_openai.ChatOpenAI

    Purpose: LangChain framework components: – LLM Interface: Wrapper for OpenAI models – Chains: Composable building blocks for complex workflows

    Other Imports

    from dotenv.load_dotenv
from pydantic.BaseModel
from pydantic.Field

    Purpose: Additional dependencies for specialized functionality.

    Classes

    Class: OutputFormat

    Lines 212-218

    Inherits from: Enum

    Purpose:

    Supported output formats

    Class: Person

    Lines 222-227

    Inherits from: BaseModel

    Purpose:

    Person entity

    Class: Product

    Lines 230-236

    Inherits from: BaseModel

    Purpose:

    Product information

    Class: Article

    Lines 239-245

    Inherits from: BaseModel

    Purpose:

    Article/content structure

    Class: SentimentAnalysis

    Lines 248-253

    Inherits from: BaseModel

    Purpose:

    Sentiment analysis result

    Class: FormatConfig

    Lines 257-262

    Purpose:

    Configuration for format specification

    Class: OutputFormatAgent

    Lines 265-555

    Purpose:

    Agent that enforces specific output formats.

This agent demonstrates:
1. JSON format specification
2. Pydantic-based type-safe outputs
3. Multiple format support
4. Schema validation

    Methods:

    __init__(self, config)

    Lines 276-287

    Purpose:

    Initialize output format agent.

Args:
    config: Configuration for format handling

    Parameters: – config: Config

    extract_as_json(self, text, schema_description)

    Lines 289-329

    Purpose:

    Extract information in JSON format.

Args:
    text: Text to extract from
    schema_description: Description of desired JSON structure

Returns:
    Extracted data as dictionary

    Parameters: – text: Text – schema_description: Schema description

    Returns: <ast.Subscript object at 0x000001A06D63E910>

    extract_person_info(self, text)

    Lines 331-359

    Purpose:

    Extract person information with type safety.

Args:
    text: Text containing person information

Returns:
    Person object with validated fields

    Parameters: – text: Text

    Returns: Person

    extract_product_info(self, text)

    Lines 361-389

    Purpose:

    Extract product information with validation.

Args:
    text: Text containing product information

Returns:
    Product object with validated fields

    Parameters: – text: Text

    Returns: Product

    analyze_sentiment_structured(self, text)

    Lines 391-421

    Purpose:

    Perform sentiment analysis with structured output.

Args:
    text: Text to analyze

Returns:
    SentimentAnalysis object with validated fields

    Parameters: – text: Text

    Returns: SentimentAnalysis

    generate_markdown_report(self, topic, sections)

    Lines 423-463

    Purpose:

    Generate report in markdown format.

Args:
    topic: Report topic
    sections: List of section titles

Returns:
    Markdown formatted report

    Parameters: – topic: Topic – sections: Sections

    Returns: str

    generate_csv_format(self, data_description, num_rows)

    Lines 465-500

    Purpose:

    Generate data in CSV format.

Args:
    data_description: Description of data to generate
    num_rows: Number of data rows

Returns:
    CSV formatted data

    Parameters: – data_description: Data description – num_rows: Num rows

    Returns: str

    generate_xml_format(self, data_description)

    Lines 502-528

    Purpose:

    Generate data in XML format.

Args:
    data_description: Description of data structure

Returns:
    XML formatted data

    Parameters: – data_description: Data description

    Returns: str

    compare_formats(self, data)

    Lines 530-555

    Purpose:

    Generate same data in multiple formats for comparison.

Args:
    data: Data description

Returns:
    Dictionary with data in different formats

    Parameters: – data: Data

    Returns: <ast.Subscript object at 0x000001A06D83C910>

    Functions

    demonstrate_output_format()

    Lines 558-746

    Purpose:

    Demonstrate output format specification patterns

    Key Concepts Summary

    Pattern Overview

    The Output Format pattern provides a structured approach to building AI agents with specific capabilities and behaviors. This implementation leverages LangChain’s composable components to create a robust and maintainable solution.

    Core Components

    1. Agent Class: Main implementation of the pattern logic
    2. Prompt Engineering: Carefully crafted prompts to guide LLM behavior
    3. Tool Integration: Optional tools for extended functionality
    4. Memory Management: State handling across interactions
    5. Output Parsing: Structured response processing

    When to Use

    ✅ Good for: – Scenarios requiring this specific pattern’s strengths – Applications needing structured agent behavior – Systems where explainability is important – Production environments requiring reliability

    ❌ Not ideal for: – Simple single-prompt tasks – Scenarios requiring different pattern characteristics – Resource-constrained environments (if computationally intensive)

    Best Practices Demonstrated

    1. Environment Configuration

    • ✅ Use .env files for API keys
    • ✅ Never hardcode credentials
    • ✅ Validate environment variables on startup

    2. Error Handling

    • ✅ Graceful degradation on failures
    • ✅ Informative error messages
    • ✅ Logging for debugging

    3. Prompt Engineering

    • ✅ Clear, specific instructions
    • ✅ Structured output formats
    • ✅ Few-shot examples when beneficial

    4. Code Organization

    • ✅ Separation of concerns (class-based design)
    • ✅ Type hints for clarity
    • ✅ Docstrings for documentation

    5. Performance Optimization

    • ✅ Appropriate temperature settings
    • ✅ Token budget management
    • ✅ Caching where applicable

    LangChain Components Used

    • ChatOpenAI: OpenAI model interface
    • Chains: Sequential component composition

    Why LangChain? – Composable abstractions for LLM applications – Rich ecosystem of integrations – Production-ready patterns – Active community and development

    Execution Flow

    1. Initialize agent with configuration
   ↓
2. Load environment and dependencies
   ↓
3. Prepare prompt with input data
   ↓
4. Invoke LLM through LangChain
   ↓
5. Parse and validate response
   ↓
6. Return formatted result

    Key Decision Points: – Model selection (temperature, model type) – Prompt structure (few-shot vs zero-shot) – Error handling strategy – Output format validation

    Performance Considerations

    Computational Cost

    • LLM Calls: Primary cost factor (based on tokens)
    • Latency: Depends on model and input size
    • Memory: Minimal for most patterns

    Optimization Strategies

    1. Prompt Optimization: Minimize tokens while maintaining clarity
    2. Caching: Cache repeated queries/responses
    3. Model Selection: Use smaller models when appropriate
    4. Batch Processing: Group similar requests
    5. Streaming: For long responses, stream tokens

    Cost Estimates (Approximate)

    • GPT-3.5-turbo: ~$0.002 per request
    • GPT-4: ~$0.03 per request
    • GPT-4-turbo: ~$0.01 per request

    Actual costs vary based on input/output token counts

    Common Pitfalls to Avoid

    1. Prompt Design Issues

    ❌ Vague or ambiguous instructions
    ✅ Clear, specific, testable prompts

    2. Error Handling

    ❌ Ignoring API errors or rate limits
    ✅ Implement retry logic and graceful degradation

    3. Token Management

    ❌ Exceeding model token limits
    ✅ Truncate inputs, chunk processing

    4. Security

    ❌ Exposing API keys in code
    ✅ Use environment variables

    5. Testing

    ❌ Only testing happy paths
    ✅ Test edge cases, failures, malformed inputs

    6. Cost Management

    ❌ Unmonitored API usage
    ✅ Implement budgets, alerts, and monitoring

    Further Improvements

    Production Enhancements

    1. Monitoring: Add logging, metrics, tracing
    2. Validation: Input/output schema validation
    3. Rate Limiting: Implement request throttling
    4. Caching: Redis or in-memory cache for responses
    5. Async Support: Non-blocking operations for scalability

    Feature Extensions

    1. Multi-Model Support: Switch between providers
    2. Fallback Models: Graceful degradation strategy
    3. Custom Tools: Domain-specific function calling
    4. Advanced Memory: Vector stores for long-term memory
    5. Evaluation: Automated testing and benchmarking

    Code Quality

    1. Testing: Unit tests, integration tests
    2. Documentation: API docs, usage examples
    3. Type Safety: Stricter type hints
    4. Configuration: Externalized config files

    Related Patterns

    • Pattern 085
    • Pattern 087

    Pattern Families: – Reasoning Patterns (001-011) – Multi-Agent Patterns (012-019) – Tool Use Patterns (020-025) – Memory Patterns (026-031) – And many more…

    Summary

    The Output Format pattern provides a powerful approach to building AI agents with specific capabilities. This implementation demonstrates:

    ✅ Key Strengths: – Clean, maintainable code structure – LangChain integration for composability – Type hints and documentation – Production-ready patterns

    🎯 Best Use Cases: – Applications requiring this pattern’s characteristics – Systems needing structured agent behavior – Production environments requiring reliability

    📚 Learning Resources: – LangChain Documentation: https://python.langchain.com/ – Pattern Documentation: See related patterns – OpenAI API Reference: https://platform.openai.com/docs/

    File: 086_output_format.py
    Total Lines: 751
    Pattern Category: AI Agent Design Pattern #086

    Generated from actual implementation code
    For educational and reference purposes

  • Agentic Design Pattern: Step By Step

    
    """
Pattern 085: Step-by-Step Instructions Pattern

Description:
    The Step-by-Step Instructions pattern structures prompts to guide the LLM through
    a process in discrete, ordered steps. This pattern is particularly effective for
    complex tasks that require systematic execution, clear sequencing, and validation
    at each stage. By breaking down tasks into explicit steps, this pattern ensures
    thorough coverage, reduces errors, and provides clear reasoning traces.

    This pattern is essential for:
    - Complex multi-stage workflows
    - Tasks requiring specific ordering
    - Processes with validation checkpoints
    - Teaching and tutoring applications
    - Troubleshooting and debugging
    - Recipe and instruction generation

Components:
    1. Step Definition
       - Numbered sequence (1, 2, 3...)
       - Clear action verbs (calculate, analyze, verify)
       - Specific objectives per step
       - Expected outcomes

    2. Step Organization
       - Main steps (high-level stages)
       - Sub-steps (detailed actions)
       - Conditional branches (if-then logic)
       - Iteration loops (repeat until)

    3. Validation Checkpoints
       - Completion criteria per step
       - Quality checks
       - Error detection
       - Progress verification

    4. Progress Tracking
       - Current step indication
       - Completed steps summary
       - Remaining steps preview
       - Overall progress percentage

Use Cases:
    1. Problem-Solving Workflows
       - Mathematical problem solving
       - Debugging code issues
       - Troubleshooting technical problems
       - Root cause analysis
       - Decision-making frameworks

    2. Instructional Content
       - Tutorial generation
       - How-to guides
       - Recipe creation
       - Assembly instructions
       - Learning modules

    3. Analysis Tasks
       - Data analysis workflows
       - Research methodology
       - Document review processes
       - Competitive analysis
       - Risk assessment

    4. Creative Processes
       - Story development (plot, characters, scenes)
       - Content creation workflow
       - Design thinking process
       - Brainstorming structure
       - Iterative refinement

    5. Compliance and Safety
       - Safety protocols
       - Audit procedures
       - Quality assurance checks
       - Regulatory compliance steps
       - Standard operating procedures

LangChain Implementation:
    LangChain enables step-by-step patterns through:
    - Structured prompt templates
    - Chain sequencing (LCEL)
    - State management between steps
    - Conditional routing
    - Progress tracking utilities

Key Features:
    1. Explicit Step Numbering
       - Clear sequential order
       - Easy progress tracking
       - Reference-able steps
       - Skippable sections

    2. Hierarchical Structure
       - Main steps and sub-steps
       - Nested levels (steps within steps)
       - Grouped related actions
       - Flexible depth

    3. Validation Integration
       - Check results at each step
       - Verify assumptions
       - Catch errors early
       - Confirm understanding

    4. Adaptive Sequencing
       - Conditional next steps
       - Skip irrelevant steps
       - Loop for iteration
       - Branch based on results

Best Practices:
    1. Step Clarity
       - Use clear, action-oriented language
       - One main action per step
       - Avoid ambiguity
       - Provide examples when helpful

    2. Appropriate Granularity
       - Not too high-level (vague)
       - Not too detailed (overwhelming)
       - Balance based on complexity
       - Adjust for target audience

    3. Logical Ordering
       - Dependencies respected
       - Natural flow
       - Build on previous steps
       - Group related steps

    4. Completeness
       - Cover all necessary steps
       - Don't skip crucial details
       - Include error handling
       - Provide alternative paths

Trade-offs:
    Advantages:
    - Reduced cognitive load (one step at a time)
    - Fewer errors (systematic approach)
    - Better traceability (clear reasoning path)
    - Easier debugging (identify problem step)
    - Improved learning (explicit process)
    - Higher consistency (standardized approach)

    Disadvantages:
    - Can be verbose (more tokens)
    - May feel rigid (less flexibility)
    - Slower for simple tasks (overhead)
    - Requires careful design (step definition)
    - May over-constrain creativity
    - Context window consumption

Production Considerations:
    1. Step Optimization
       - Balance detail with brevity
       - Remove redundant steps
       - Combine simple steps
       - Expand complex steps

    2. Error Handling
       - Define failure conditions
       - Provide recovery steps
       - Include validation checks
       - Offer troubleshooting guidance

    3. User Experience
       - Show progress indicators
       - Allow step skipping (when safe)
       - Provide step summaries
       - Enable backtracking

    4. Performance
       - Cache intermediate results
       - Parallelize independent steps
       - Optimize step execution
       - Monitor step durations

    5. Maintenance
       - Version control for procedures
       - Regular step reviews
       - Update based on feedback
       - Test step completeness
"""

import os
from typing import List, Dict, Any, Optional
from dataclasses import dataclass, field
from enum import Enum
from dotenv import load_dotenv

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_openai import ChatOpenAI

load_dotenv()


class StepStatus(Enum):
    """Status of a step"""
    PENDING = "pending"
    IN_PROGRESS = "in_progress"
    COMPLETED = "completed"
    FAILED = "failed"
    SKIPPED = "skipped"


@dataclass
class Step:
    """Represents a single step in a process"""
    number: int
    title: str
    description: str
    action: str
    validation: Optional[str] = None
    status: StepStatus = StepStatus.PENDING
    result: Optional[str] = None
    sub_steps: List['Step'] = field(default_factory=list)


@dataclass
class StepByStepConfig:
    """Configuration for step-by-step instructions"""
    include_validation: bool = True
    include_examples: bool = False
    include_reasoning: bool = True
    temperature: float = 0.3
    model_name: str = "gpt-3.5-turbo"


class StepByStepAgent:
    """
    Agent that structures tasks into clear step-by-step instructions.
    
    This agent demonstrates:
    1. Generating step-by-step processes
    2. Following structured procedures
    3. Progress tracking and validation
    4. Hierarchical step organization
    """
    
    def __init__(self, config: Optional[StepByStepConfig] = None):
        """
        Initialize step-by-step agent.
        
        Args:
            config: Configuration for step-by-step processing
        """
        self.config = config or StepByStepConfig()
        self.llm = ChatOpenAI(
            model=self.config.model_name,
            temperature=self.config.temperature
        )
    
    def generate_steps(
        self,
        task: str,
        num_steps: Optional[int] = None
    ) -> str:
        """
        Generate step-by-step instructions for a task.
        
        Args:
            task: Task to create steps for
            num_steps: Optional target number of steps
            
        Returns:
            Step-by-step instructions
        """
        num_steps_instruction = f" in approximately {num_steps} steps" if num_steps else ""
        
        prompt = ChatPromptTemplate.from_messages([
            ("system", """You are an expert at creating clear, actionable step-by-step instructions.

Your instructions should:
1. Break down the task into discrete, manageable steps
2. Use numbered steps (1, 2, 3, etc.)
3. Start each step with an action verb
4. Be specific and actionable
5. Include validation checks where appropriate
6. Maintain logical ordering"""),
            ("human", "Create step-by-step instructions for: {task}{num_steps_instruction}")
        ])
        
        chain = prompt | self.llm | StrOutputParser()
        
        result = chain.invoke({
            "task": task,
            "num_steps_instruction": num_steps_instruction
        })
        
        return result
    
    def solve_problem_step_by_step(
        self,
        problem: str
    ) -> str:
        """
        Solve a problem using step-by-step reasoning.
        
        Args:
            problem: Problem to solve
            
        Returns:
            Step-by-step solution
        """
        prompt = ChatPromptTemplate.from_messages([
            ("system", """You are a problem solver who thinks step-by-step.

For each problem:
1. Break it down into logical steps
2. Show your work for each step
3. Validate results as you go
4. State the final answer clearly

Use this format:
Step 1: [First action]
[Work/reasoning]

Step 2: [Second action]
[Work/reasoning]

...

Final Answer: [Conclusion]"""),
            ("human", "Solve this problem step-by-step: {problem}")
        ])
        
        chain = prompt | self.llm | StrOutputParser()
        result = chain.invoke({"problem": problem})
        
        return result
    
    def debug_code_step_by_step(
        self,
        code: str,
        error: str
    ) -> str:
        """
        Debug code using systematic step-by-step approach.
        
        Args:
            code: Code with error
            error: Error message
            
        Returns:
            Step-by-step debugging process
        """
        prompt = ChatPromptTemplate.from_messages([
            ("system", """You are a debugging expert who follows a systematic step-by-step process.

Debugging steps:
1. Understand the error message
2. Locate the error in the code
3. Identify the root cause
4. Propose a fix
5. Explain why the fix works
6. Provide the corrected code

Follow these steps explicitly in your response."""),
            ("human", """Debug this code step-by-step:

Code:
{code}

Error:
{error}""")
        ])
        
        chain = prompt | self.llm | StrOutputParser()
        result = chain.invoke({"code": code, "error": error})
        
        return result
    
    def create_tutorial_step_by_step(
        self,
        topic: str,
        audience: str = "beginners"
    ) -> str:
        """
        Create a tutorial with step-by-step instructions.
        
        Args:
            topic: Topic to teach
            audience: Target audience level
            
        Returns:
            Step-by-step tutorial
        """
        prompt = ChatPromptTemplate.from_messages([
            ("system", """You are an expert teacher creating step-by-step tutorials.

Your tutorials should:
1. Start with an overview
2. List prerequisites
3. Break content into numbered steps
4. Include examples in each step
5. Provide checkpoints to verify understanding
6. End with a summary and next steps

Adapt complexity to the audience level."""),
            ("human", "Create a step-by-step tutorial on '{topic}' for {audience}")
        ])
        
        chain = prompt | self.llm | StrOutputParser()
        result = chain.invoke({"topic": topic, "audience": audience})
        
        return result
    
    def analyze_data_step_by_step(
        self,
        data_description: str,
        analysis_goal: str
    ) -> str:
        """
        Perform data analysis using step-by-step methodology.
        
        Args:
            data_description: Description of the data
            analysis_goal: Goal of the analysis
            
        Returns:
            Step-by-step analysis process
        """
        prompt = ChatPromptTemplate.from_messages([
            ("system", """You are a data analyst who follows a structured step-by-step methodology.

Standard data analysis steps:
1. Understand the data and goal
2. Data cleaning and preparation
3. Exploratory analysis
4. Apply appropriate techniques
5. Interpret results
6. Draw conclusions
7. Recommend actions

Follow these steps in your analysis."""),
            ("human", """Analyze this data step-by-step:

Data: {data_description}
Goal: {analysis_goal}""")
        ])
        
        chain = prompt | self.llm | StrOutputParser()
        result = chain.invoke({
            "data_description": data_description,
            "analysis_goal": analysis_goal
        })
        
        return result
    
    def decision_framework_step_by_step(
        self,
        decision: str,
        context: str
    ) -> str:
        """
        Apply decision-making framework step-by-step.
        
        Args:
            decision: Decision to make
            context: Context and constraints
            
        Returns:
            Step-by-step decision analysis
        """
        prompt = ChatPromptTemplate.from_messages([
            ("system", """You are a decision analyst using a structured step-by-step framework.

Decision-making steps:
1. Define the decision clearly
2. Identify objectives and criteria
3. Generate alternatives
4. Evaluate each alternative against criteria
5. Assess risks and uncertainties
6. Make recommendation
7. Plan implementation

Follow these steps systematically."""),
            ("human", """Help me make this decision step-by-step:

Decision: {decision}
Context: {context}""")
        ])
        
        chain = prompt | self.llm | StrOutputParser()
        result = chain.invoke({"decision": decision, "context": context})
        
        return result
    
    def troubleshoot_step_by_step(
        self,
        problem_description: str,
        system: str = "general"
    ) -> str:
        """
        Troubleshoot a problem using systematic steps.
        
        Args:
            problem_description: Description of the problem
            system: System or domain
            
        Returns:
            Step-by-step troubleshooting guide
        """
        prompt = ChatPromptTemplate.from_messages([
            ("system", """You are a troubleshooting expert who uses a systematic step-by-step approach.

Troubleshooting methodology:
1. Gather information about the problem
2. Identify symptoms
3. Form hypotheses about causes
4. Test hypotheses systematically
5. Isolate the root cause
6. Implement solution
7. Verify resolution
8. Document for future reference

Follow these steps in your troubleshooting process."""),
            ("human", """Troubleshoot this problem step-by-step:

Problem: {problem_description}
System: {system}""")
        ])
        
        chain = prompt | self.llm | StrOutputParser()
        result = chain.invoke({
            "problem_description": problem_description,
            "system": system
        })
        
        return result


def demonstrate_step_by_step():
    """Demonstrate step-by-step instruction patterns"""
    print("=" * 80)
    print("STEP-BY-STEP INSTRUCTIONS PATTERN DEMONSTRATION")
    print("=" * 80)
    
    agent = StepByStepAgent()
    
    # Example 1: Generate Instructions for Task
    print("\n" + "=" * 80)
    print("EXAMPLE 1: Generate Step-by-Step Instructions")
    print("=" * 80)
    
    task = "Plan and execute a successful product launch"
    print(f"\nTask: {task}\n")
    instructions = agent.generate_steps(task, num_steps=7)
    print(instructions)
    
    # Example 2: Math Problem Solving
    print("\n" + "=" * 80)
    print("EXAMPLE 2: Step-by-Step Problem Solving")
    print("=" * 80)
    
    problem = "If a train travels 120 miles in 2 hours, then slows down and travels 80 miles in 2 hours, what is its average speed for the entire journey?"
    print(f"\nProblem: {problem}\n")
    solution = agent.solve_problem_step_by_step(problem)
    print(solution)
    
    # Example 3: Code Debugging
    print("\n" + "=" * 80)
    print("EXAMPLE 3: Step-by-Step Code Debugging")
    print("=" * 80)
    
    buggy_code = """
def calculate_average(numbers):
    total = 0
    for num in numbers:
        total += num
    return total / len(numbers)

result = calculate_average([])
print(result)
"""
    error = "ZeroDivisionError: division by zero"
    
    print(f"Code:\n{buggy_code}")
    print(f"\nError: {error}\n")
    debug_process = agent.debug_code_step_by_step(buggy_code, error)
    print(debug_process)
    
    # Example 4: Tutorial Creation
    print("\n" + "=" * 80)
    print("EXAMPLE 4: Step-by-Step Tutorial")
    print("=" * 80)
    
    topic = "Setting up a Python virtual environment"
    audience = "beginners"
    print(f"\nTopic: {topic}")
    print(f"Audience: {audience}\n")
    tutorial = agent.create_tutorial_step_by_step(topic, audience)
    print(tutorial)
    
    # Example 5: Data Analysis
    print("\n" + "=" * 80)
    print("EXAMPLE 5: Step-by-Step Data Analysis")
    print("=" * 80)
    
    data_desc = "Sales data for last quarter showing revenue, units sold, and customer segments"
    goal = "Identify which customer segment has the highest growth potential"
    print(f"\nData: {data_desc}")
    print(f"Goal: {goal}\n")
    analysis = agent.analyze_data_step_by_step(data_desc, goal)
    print(analysis)
    
    # Example 6: Decision Making
    print("\n" + "=" * 80)
    print("EXAMPLE 6: Step-by-Step Decision Framework")
    print("=" * 80)
    
    decision = "Should we expand to a new market?"
    context = "B2B software company with stable domestic revenue, considering Asian market expansion"
    print(f"\nDecision: {decision}")
    print(f"Context: {context}\n")
    decision_analysis = agent.decision_framework_step_by_step(decision, context)
    print(decision_analysis)
    
    # Example 7: Troubleshooting
    print("\n" + "=" * 80)
    print("EXAMPLE 7: Step-by-Step Troubleshooting")
    print("=" * 80)
    
    problem = "Website is loading very slowly for users"
    system = "web application"
    print(f"\nProblem: {problem}")
    print(f"System: {system}\n")
    troubleshooting = agent.troubleshoot_step_by_step(problem, system)
    print(troubleshooting)
    
    # Summary
    print("\n" + "=" * 80)
    print("STEP-BY-STEP INSTRUCTIONS SUMMARY")
    print("=" * 80)
    print("""
Step-by-Step Pattern Benefits:
1. Reduced Complexity: Breaks down complex tasks into manageable parts
2. Clear Progress: Easy to track where you are in the process
3. Fewer Errors: Systematic approach catches mistakes early
4. Better Learning: Explicit steps aid understanding
5. Reproducibility: Others can follow the same process
6. Debugging: Easy to identify where things went wrong

Key Applications Demonstrated:
1. Task Planning: Creating structured workflows
2. Problem Solving: Mathematical and logical problems
3. Code Debugging: Systematic error identification
4. Tutorial Creation: Teaching complex topics
5. Data Analysis: Structured analytical methodology
6. Decision Making: Rational choice frameworks
7. Troubleshooting: Systematic problem diagnosis

Step-by-Step Structure Elements:
- Numbered sequence (clear ordering)
- Action verbs (calculate, analyze, verify)
- Validation checkpoints (confirm results)
- Progress indicators (current step)
- Sub-steps (hierarchical detail)
- Conditional branches (if-then logic)

Best Practices:
1. Use clear, action-oriented language
2. Maintain logical dependencies between steps
3. Include validation checks at key points
4. Keep steps at appropriate granularity
5. Number steps clearly (1, 2, 3...)
6. Start steps with action verbs
7. Provide examples within steps when helpful
8. Include error handling and alternatives

When to Use Step-by-Step:
- Complex multi-stage tasks
- Teaching and learning scenarios
- Troubleshooting and debugging
- Compliance and safety procedures
- Quality assurance processes
- Standardized workflows
- Problem-solving that requires systematic approach

Advantages Over Free-Form:
- More thorough coverage
- Easier to follow and verify
- Better for complex tasks
- Reduces cognitive load
- Improves consistency
- Facilitates collaboration

Trade-offs:
- More verbose (uses more tokens)
- Can feel rigid for simple tasks
- Requires careful step design
- May over-structure creative tasks
- Slower for trivial operations

Production Tips:
- Cache common step sequences
- Allow step skipping when safe
- Provide progress indicators
- Enable backtracking to previous steps
- Version control procedural steps
- Gather feedback on step clarity
- Optimize step granularity based on usage
""")
    
    print("\n" + "=" * 80)
    print("Pattern 085 (Step-by-Step Instructions) demonstration complete!")
    print("=" * 80)


if __name__ == "__main__":
    demonstrate_step_by_step()

    Agentic Design Pattern: Step By Step

    Pattern 085: Step By Step – Line-by-Line Explanation

    Overview

    This document provides a detailed line-by-line explanation of the Step By Step pattern implementation.

    Pattern Description

    Pattern 085: Step-by-Step Instructions Pattern

    Description: The Step-by-Step Instructions pattern structures prompts to guide the LLM through a process in discrete, ordered steps. This pattern is particularly effective for complex tasks that require systematic execution, clear sequencing, and validation at each stage. By breaking down tasks into explicit steps, this pattern ensures thorough coverage, reduces errors, and provides clear reasoning traces.

    This pattern is essential for:
- Complex multi-stage workflows
- Tasks requiring specific ordering
- Processes with validation checkpoints
- Teaching and tutoring applications
- Troubleshooting and debugging
- Recipe and instruction generation

    Components: 1. Step Definition – Numbered sequence (1, 2, 3…) – Clear action verbs (calculate, analyze, verify) – Specific objectives per step – Expected outcomes

    2. Step Organization
   - Main steps (high-level stages)
   - Sub-steps (detailed actions)
   - Conditional branches (if-then logic)
   - Iteration loops (repeat until)

3. Validation Checkpoints
   - Completion criteria per step
   - Quality checks
   - Error detection
   - Progress verification

4. Progress Tracking
   - Current step indication
   - Completed steps summary
   - Remaining steps preview
   - Overall progress percentage

    Use Cases: 1. Problem-Solving Workflows – Mathematical problem solving – Debugging code issues – Troubleshooting technical problems – Root cause analysis – Decision-making frameworks

    2. Instructional Content
   - Tutorial generation
   - How-to guides
   - Recipe creation
   - Assembly instructions
   - Learning modules

3. Analysis Tasks
   - Data analysis workflows
   - Research methodology
   - Document review processes
   - Competitive analysis
   - Risk assessment

4. Creative Processes
   - Story development (plot, characters, scenes)
   - Content creation workflow
   - Design thinking process
   - Brainstorming structure
   - Iterative refinement

5. Compliance and Safety
   - Safety protocols
   - Audit procedures
   - Quality assurance checks
   - Regulatory compliance steps
   - Standard operating procedures

    LangChain Implementation: LangChain enables step-by-step patterns through: – Structured prompt templates – Chain sequencing (LCEL) – State management between steps – Conditional routing – Progress tracking utilities

    Key Features: 1. Explicit Step Numbering – Clear sequential order – Easy progress tracking – Reference-able steps – Skippable sections

    2. Hierarchical Structure
   - Main steps and sub-steps
   - Nested levels (steps within steps)
   - Grouped related actions
   - Flexible depth

3. Validation Integration
   - Check results at each step
   - Verify assumptions
   - Catch errors early
   - Confirm understanding

4. Adaptive Sequencing
   - Conditional next steps
   - Skip irrelevant steps
   - Loop for iteration
   - Branch based on results

    Best Practices: 1. Step Clarity – Use clear, action-oriented language – One main action per step – Avoid ambiguity – Provide examples when helpful

    2. Appropriate Granularity
   - Not too high-level (vague)
   - Not too detailed (overwhelming)
   - Balance based on complexity
   - Adjust for target audience

3. Logical Ordering
   - Dependencies respected
   - Natural flow
   - Build on previous steps
   - Group related steps

4. Completeness
   - Cover all necessary steps
   - Don't skip crucial details
   - Include error handling
   - Provide alternative paths

    Trade-offs: Advantages: – Reduced cognitive load (one step at a time) – Fewer errors (systematic approach) – Better traceability (clear reasoning path) – Easier debugging (identify problem step) – Improved learning (explicit process) – Higher consistency (standardized approach)

    Disadvantages:
- Can be verbose (more tokens)
- May feel rigid (less flexibility)
- Slower for simple tasks (overhead)
- Requires careful design (step definition)
- May over-constrain creativity
- Context window consumption

    Production Considerations: 1. Step Optimization – Balance detail with brevity – Remove redundant steps – Combine simple steps – Expand complex steps

    2. Error Handling
   - Define failure conditions
   - Provide recovery steps
   - Include validation checks
   - Offer troubleshooting guidance

3. User Experience
   - Show progress indicators
   - Allow step skipping (when safe)
   - Provide step summaries
   - Enable backtracking

4. Performance
   - Cache intermediate results
   - Parallelize independent steps
   - Optimize step execution
   - Monitor step durations

5. Maintenance
   - Version control for procedures
   - Regular step reviews
   - Update based on feedback
   - Test step completeness

    File Statistics

    • Total Lines: 715
    • Classes: 4
    • Functions: 1
    • Constants: 0

    Module Documentation

    """
Pattern 085: Step-by-Step Instructions Pattern

Description:
    The Step-by-Step Instructions pattern structures prompts to guide the LLM through
    a process in discrete, ordered steps. This pattern is particularly effective for
    complex tasks that require systematic execution, clear sequencing, and validation
    at each stage. By breaking down tasks into explicit steps, this pattern ensures
    thorough coverage, reduces errors, and provides clear reasoning traces.

    This pattern is essential for:
    - Complex multi-stage workflows
    - Tasks requiring specific ordering
    - Processes with validation checkpoints
    - Teaching and tutoring applications
    - Troubleshooting and debugging
    - Recipe and instruction generation

Components:
    1. Step Definition
       - Numbered sequence (1, 2, 3...)
       - Clear action verbs (calculate, analyze, verify)
       - Specific objectives per step
       - Expected outcomes

    2. Step Organization
       - Main steps (high-level stages)
       - Sub-steps (detailed actions)
       - Conditional branches (if-then logic)
       - Iteration loops (repeat until)

    3. Validation Checkpoints
       - Completion criteria per step
       - Quality checks
       - Error detection
       - Progress verification

    4. Progress Tracking
       - Current step indication
       - Completed steps summary
       - Remaining steps preview
       - Overall progress percentage

Use Cases:
    1. Problem-Solving Workflows
       - Mathematical problem solving
       - Debugging code issues
       - Troubleshooting technical problems
       - Root cause analysis
       - Decision-making frameworks

    2. Instructional Content
       - Tutorial generation
       - How-to guides
       - Recipe creation
       - Assembly instructions
       - Learning modules

    3. Analysis Tasks
       - Data analysis workflows
       - Research methodology
       - Document review processes
       - Competitive analysis
       - Risk assessment

    4. Creative Processes
       - Story development (plot, characters, scenes)
       - Content creation workflow
       - Design thinking process
       - Brainstorming structure
       - Iterative refinement

    5. Compliance and Safety
       - Safety protocols
       - Audit procedures
       - Quality assurance checks
       - Regulatory compliance steps
       - Standard operating procedures

LangChain Implementation:
    LangChain enables step-by-step patterns through:
    - Structured prompt templates
    - Chain sequencing (LCEL)
    - State management between steps
    - Conditional routing
    - Progress tracking utilities

Key Features:
    1. Explicit Step Numbering
       - Clear sequential order
       - Easy progress tracking
       - Reference-able steps
       - Skippable sections

    2. Hierarchical Structure
       - Main steps and sub-steps
       - Nested levels (steps within steps)
       - Grouped related actions
       - Flexible depth

    3. Validation Integration
       - Check results at each step
       - Verify assumptions
       - Catch errors early
       - Confirm understanding

    4. Adaptive Sequencing
       - Conditional next steps
       - Skip irrelevant steps
       - Loop for iteration
       - Branch based on results

Best Practices:
    1. Step Clarity
       - Use clear, action-oriented language
       - One main action per step
       - Avoid ambiguity
       - Provide examples when helpful

    2. Appropriate Granularity
       - Not too high-level (vague)
       - Not too detailed (overwhelming)
       - Balance based on complexity
       - Adjust for target audience

    3. Logical Ordering
       - Dependencies respected
       - Natural flow
       - Build on previous steps
       - Group related steps

    4. Completeness
       - Cover all necessary steps
       - Don't skip crucial details
       - Include error handling
       - Provide alternative paths

Trade-offs:
    Advantages:
    - Reduced cognitive load (one step at a time)
    - Fewer errors (systematic approach)
    - Better traceability (clear reasoning path)
    - Easier debugging (identify problem step)
    - Improved learning (explicit process)
    - Higher consistency (standardized approach)

    Disadvantages:
    - Can be verbose (more tokens)
    - May feel rigid (less flexibility)
    - Slower for simple tasks (overhead)
    - Requires careful design (step definition)
    - May over-constrain creativity
    - Context window consumption

Production Considerations:
    1. Step Optimization
       - Balance detail with brevity
       - Remove redundant steps
       - Combine simple steps
       - Expand complex steps

    2. Error Handling
       - Define failure conditions
       - Provide recovery steps
       - Include validation checks
       - Offer troubleshooting guidance

    3. User Experience
       - Show progress indicators
       - Allow step skipping (when safe)
       - Provide step summaries
       - Enable backtracking

    4. Performance
       - Cache intermediate results
       - Parallelize independent steps
       - Optimize step execution
       - Monitor step durations

    5. Maintenance
       - Version control for procedures
       - Regular step reviews
       - Update based on feedback
       - Test step completeness
"""

    Purpose: This module implements the Step By Step pattern, providing a structured approach to agent design and implementation.

    Import Statements

    Standard Library Imports

    import dataclasses.dataclass
import dataclasses.field
import enum.Enum
import langchain_core.output_parsers.StrOutputParser
import langchain_core.prompts.ChatPromptTemplate
import os
import typing.Any
import typing.Dict
import typing.List
import typing.Optional

    Purpose: Core Python functionality: – dataclasses.dataclass: Simplified class definitions with @dataclass – dataclasses.field: Simplified class definitions with @dataclass – enum.Enum: Enumeration support – langchain_core.output_parsers.StrOutputParser: Regular expression operations – langchain_core.prompts.ChatPromptTemplate: Regular expression operations – os: Environment variable access and system operations – typing.Any: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Dict: Type hints for better code documentation (List, Dict, Any, Optional) – typing.List: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Optional: Type hints for better code documentation (List, Dict, Any, Optional)

    LangChain Imports

    from langchain_openai.ChatOpenAI

    Purpose: LangChain framework components: – LLM Interface: Wrapper for OpenAI models – Chains: Composable building blocks for complex workflows

    Other Imports

    from dotenv.load_dotenv

    Purpose: Additional dependencies for specialized functionality.

    Classes

    Class: StepStatus

    Lines 200-206

    Inherits from: Enum

    Purpose:

    Status of a step

    Class: Step

    Lines 210-219

    Purpose:

    Represents a single step in a process

    Class: StepByStepConfig

    Lines 223-229

    Purpose:

    Configuration for step-by-step instructions

    Class: StepByStepAgent

    Lines 232-533

    Purpose:

    Agent that structures tasks into clear step-by-step instructions.

This agent demonstrates:
1. Generating step-by-step processes
2. Following structured procedures
3. Progress tracking and validation
4. Hierarchical step organization

    Methods:

    __init__(self, config)

    Lines 243-254

    Purpose:

    Initialize step-by-step agent.

Args:
    config: Configuration for step-by-step processing

    Parameters: – config: Config

    generate_steps(self, task, num_steps)

    Lines 256-293

    Purpose:

    Generate step-by-step instructions for a task.

Args:
    task: Task to create steps for
    num_steps: Optional target number of steps

Returns:
    Step-by-step instructions

    Parameters: – task: Task – num_steps: Num steps

    Returns: str

    solve_problem_step_by_step(self, problem)

    Lines 295-333

    Purpose:

    Solve a problem using step-by-step reasoning.

Args:
    problem: Problem to solve

Returns:
    Step-by-step solution

    Parameters: – problem: Problem

    Returns: str

    debug_code_step_by_step(self, code, error)

    Lines 335-374

    Purpose:

    Debug code using systematic step-by-step approach.

Args:
    code: Code with error
    error: Error message

Returns:
    Step-by-step debugging process

    Parameters: – code: Code – error: Error

    Returns: str

    create_tutorial_step_by_step(self, topic, audience)

    Lines 376-409

    Purpose:

    Create a tutorial with step-by-step instructions.

Args:
    topic: Topic to teach
    audience: Target audience level

Returns:
    Step-by-step tutorial

    Parameters: – topic: Topic – audience: Audience

    Returns: str

    analyze_data_step_by_step(self, data_description, analysis_goal)

    Lines 411-451

    Purpose:

    Perform data analysis using step-by-step methodology.

Args:
    data_description: Description of the data
    analysis_goal: Goal of the analysis

Returns:
    Step-by-step analysis process

    Parameters: – data_description: Data description – analysis_goal: Analysis goal

    Returns: str

    decision_framework_step_by_step(self, decision, context)

    Lines 453-490

    Purpose:

    Apply decision-making framework step-by-step.

Args:
    decision: Decision to make
    context: Context and constraints

Returns:
    Step-by-step decision analysis

    Parameters: – decision: Decision – context: Context

    Returns: str

    troubleshoot_step_by_step(self, problem_description, system)

    Lines 492-533

    Purpose:

    Troubleshoot a problem using systematic steps.

Args:
    problem_description: Description of the problem
    system: System or domain

Returns:
    Step-by-step troubleshooting guide

    Parameters: – problem_description: Problem description – system: System

    Returns: str

    Functions

    demonstrate_step_by_step()

    Lines 536-710

    Purpose:

    Demonstrate step-by-step instruction patterns

    Key Concepts Summary

    Pattern Overview

    The Step By Step pattern provides a structured approach to building AI agents with specific capabilities and behaviors. This implementation leverages LangChain’s composable components to create a robust and maintainable solution.

    Core Components

    1. Agent Class: Main implementation of the pattern logic
    2. Prompt Engineering: Carefully crafted prompts to guide LLM behavior
    3. Tool Integration: Optional tools for extended functionality
    4. Memory Management: State handling across interactions
    5. Output Parsing: Structured response processing

    When to Use

    ✅ Good for: – Scenarios requiring this specific pattern’s strengths – Applications needing structured agent behavior – Systems where explainability is important – Production environments requiring reliability

    ❌ Not ideal for: – Simple single-prompt tasks – Scenarios requiring different pattern characteristics – Resource-constrained environments (if computationally intensive)

    Best Practices Demonstrated

    1. Environment Configuration

    • ✅ Use .env files for API keys
    • ✅ Never hardcode credentials
    • ✅ Validate environment variables on startup

    2. Error Handling

    • ✅ Graceful degradation on failures
    • ✅ Informative error messages
    • ✅ Logging for debugging

    3. Prompt Engineering

    • ✅ Clear, specific instructions
    • ✅ Structured output formats
    • ✅ Few-shot examples when beneficial

    4. Code Organization

    • ✅ Separation of concerns (class-based design)
    • ✅ Type hints for clarity
    • ✅ Docstrings for documentation

    5. Performance Optimization

    • ✅ Appropriate temperature settings
    • ✅ Token budget management
    • ✅ Caching where applicable

    LangChain Components Used

    • ChatOpenAI: OpenAI model interface
    • Chains: Sequential component composition

    Why LangChain? – Composable abstractions for LLM applications – Rich ecosystem of integrations – Production-ready patterns – Active community and development

    Execution Flow

    1. Initialize agent with configuration
   ↓
2. Load environment and dependencies
   ↓
3. Prepare prompt with input data
   ↓
4. Invoke LLM through LangChain
   ↓
5. Parse and validate response
   ↓
6. Return formatted result

    Key Decision Points: – Model selection (temperature, model type) – Prompt structure (few-shot vs zero-shot) – Error handling strategy – Output format validation

    Performance Considerations

    Computational Cost

    • LLM Calls: Primary cost factor (based on tokens)
    • Latency: Depends on model and input size
    • Memory: Minimal for most patterns

    Optimization Strategies

    1. Prompt Optimization: Minimize tokens while maintaining clarity
    2. Caching: Cache repeated queries/responses
    3. Model Selection: Use smaller models when appropriate
    4. Batch Processing: Group similar requests
    5. Streaming: For long responses, stream tokens

    Cost Estimates (Approximate)

    • GPT-3.5-turbo: ~$0.002 per request
    • GPT-4: ~$0.03 per request
    • GPT-4-turbo: ~$0.01 per request

    Actual costs vary based on input/output token counts

    Common Pitfalls to Avoid

    1. Prompt Design Issues

    ❌ Vague or ambiguous instructions
    ✅ Clear, specific, testable prompts

    2. Error Handling

    ❌ Ignoring API errors or rate limits
    ✅ Implement retry logic and graceful degradation

    3. Token Management

    ❌ Exceeding model token limits
    ✅ Truncate inputs, chunk processing

    4. Security

    ❌ Exposing API keys in code
    ✅ Use environment variables

    5. Testing

    ❌ Only testing happy paths
    ✅ Test edge cases, failures, malformed inputs

    6. Cost Management

    ❌ Unmonitored API usage
    ✅ Implement budgets, alerts, and monitoring

    Further Improvements

    Production Enhancements

    1. Monitoring: Add logging, metrics, tracing
    2. Validation: Input/output schema validation
    3. Rate Limiting: Implement request throttling
    4. Caching: Redis or in-memory cache for responses
    5. Async Support: Non-blocking operations for scalability

    Feature Extensions

    1. Multi-Model Support: Switch between providers
    2. Fallback Models: Graceful degradation strategy
    3. Custom Tools: Domain-specific function calling
    4. Advanced Memory: Vector stores for long-term memory
    5. Evaluation: Automated testing and benchmarking

    Code Quality

    1. Testing: Unit tests, integration tests
    2. Documentation: API docs, usage examples
    3. Type Safety: Stricter type hints
    4. Configuration: Externalized config files

    Related Patterns

    • Pattern 084
    • Pattern 086

    Pattern Families: – Reasoning Patterns (001-011) – Multi-Agent Patterns (012-019) – Tool Use Patterns (020-025) – Memory Patterns (026-031) – And many more…

    Summary

    The Step By Step pattern provides a powerful approach to building AI agents with specific capabilities. This implementation demonstrates:

    ✅ Key Strengths: – Clean, maintainable code structure – LangChain integration for composability – Type hints and documentation – Production-ready patterns

    🎯 Best Use Cases: – Applications requiring this pattern’s characteristics – Systems needing structured agent behavior – Production environments requiring reliability

    📚 Learning Resources: – LangChain Documentation: https://python.langchain.com/ – Pattern Documentation: See related patterns – OpenAI API Reference: https://platform.openai.com/docs/

    File: 085_step_by_step.py
    Total Lines: 715
    Pattern Category: AI Agent Design Pattern #085

    Generated from actual implementation code
    For educational and reference purposes

  • Agentic Design Pattern: Role Playing

    
    """
Pattern 084: Role-Playing/Persona Pattern

Description:
    The Role-Playing/Persona pattern involves instructing the LLM to adopt a specific
    role, character, or persona when generating responses. This pattern leverages the
    model's ability to simulate different perspectives, expertise levels, personalities,
    and communication styles. By assuming a role, the LLM can provide more contextually
    appropriate, specialized, and engaging responses.

    Role-playing is a powerful prompt engineering technique that:
    - Enhances response quality through perspective-taking
    - Enables domain expertise simulation (doctor, lawyer, scientist)
    - Creates consistent personality in conversational agents
    - Improves user engagement through character-based interaction
    - Allows multi-perspective analysis of problems

Components:
    1. Persona Definition
       - Identity (who the persona is)
       - Expertise (what they know)
       - Personality traits (how they communicate)
       - Background (relevant experience)
       - Goals and motivations

    2. Communication Style
       - Tone (formal, casual, friendly, professional)
       - Vocabulary level (simple, technical, academic)
       - Response structure (detailed, concise, step-by-step)
       - Interaction patterns (questioning, advising, teaching)

    3. Persona Consistency
       - Maintaining character across turns
       - Memory of previous interactions
       - Consistent viewpoint and opinions
       - Appropriate domain knowledge boundaries

    4. Context Adaptation
       - Adjusting expertise level to user
       - Situational awareness
       - Emotional intelligence
       - Cultural sensitivity

Use Cases:
    1. Expert Consultation
       - Medical advice (doctor persona)
       - Legal guidance (lawyer persona)
       - Technical support (engineer persona)
       - Financial planning (financial advisor persona)
       - Career counseling (career coach persona)

    2. Educational Applications
       - Patient teacher (explaining complex topics)
       - Socratic tutor (guiding through questions)
       - Encouraging mentor (motivating learners)
       - Strict examiner (testing knowledge)

    3. Creative Writing
       - Character dialogue generation
       - Story narration from specific viewpoint
       - Historical figure simulation
       - Fictional character interaction

    4. Business Applications
       - Sales representative (persuasive, customer-focused)
       - Customer service (empathetic, problem-solving)
       - Business analyst (analytical, data-driven)
       - Executive advisor (strategic, big-picture)

    5. Therapeutic and Counseling
       - Supportive friend (empathetic listener)
       - Life coach (motivational, goal-oriented)
       - Therapist (reflective, non-judgmental)
       - Meditation guide (calm, mindful)

LangChain Implementation:
    LangChain facilitates role-playing through:
    - System message configuration (setting persona)
    - Context management (maintaining persona)
    - Memory integration (persona consistency)
    - Custom prompt templates (structured personas)
    - Chat history management (conversation continuity)

Key Features:
    1. Persona Library
       - Pre-defined expert personas
       - Customizable persona templates
       - Domain-specific knowledge bases
       - Personality trait combinations

    2. Dynamic Persona Switching
       - Context-aware persona selection
       - Smooth transitions between personas
       - Multi-persona conversations
       - Persona blending

    3. Expertise Calibration
       - Adjusting knowledge depth
       - Acknowledging limitations
       - Appropriate confidence levels
       - Domain boundary awareness

    4. Personality Traits
       - Empathy and emotional intelligence
       - Humor and engagement style
       - Assertiveness levels
       - Teaching vs directing approaches

Best Practices:
    1. Persona Definition
       - Be specific about role and expertise
       - Include relevant background details
       - Define communication style clearly
       - Set appropriate boundaries

    2. Consistency Maintenance
       - Use memory for multi-turn conversations
       - Reference persona context regularly
       - Maintain viewpoint across interactions
       - Avoid out-of-character responses

    3. Expertise Management
       - Stay within persona's knowledge domain
       - Acknowledge when unsure
       - Provide persona-appropriate disclaimers
       - Don't overreach expertise claims

    4. Engagement Optimization
       - Match tone to user expectations
       - Balance expertise with accessibility
       - Use persona-appropriate examples
       - Maintain engaging interaction style

Trade-offs:
    Advantages:
    - Enhanced response quality and relevance
    - More engaging and natural interactions
    - Domain-specific expertise simulation
    - Consistent personality and tone
    - Better user trust and satisfaction
    - Multi-perspective problem analysis

    Disadvantages:
    - Requires detailed persona definitions
    - May limit flexibility in responses
    - Risk of inappropriate persona adoption
    - Can be perceived as deceptive if not transparent
    - Harder to maintain over long conversations
    - May introduce unwanted biases

Production Considerations:
    1. Transparency
       - Clearly indicate AI persona usage
       - Provide disclaimers for professional advice
       - Explain limitations of simulated expertise
       - Allow users to request different personas

    2. Safety and Ethics
       - Avoid harmful or inappropriate personas
       - Include safety guardrails
       - Monitor for persona misuse
       - Implement content filtering
       - Respect professional boundaries

    3. Quality Assurance
       - Test personas across diverse inputs
       - Validate expertise accuracy
       - Monitor consistency metrics
       - Gather user feedback
       - Regular persona updates

    4. Scalability
       - Modular persona definitions
       - Reusable persona components
       - Version control for personas
       - A/B test persona variations
       - Performance monitoring per persona

    5. Legal Compliance
       - Medical/legal advice disclaimers
       - Professional credential clarification
       - Terms of service alignment
       - Privacy considerations
       - Liability protection
"""

import os
from typing import List, Dict, Any, Optional
from dataclasses import dataclass
from enum import Enum
from dotenv import load_dotenv

from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_core.output_parsers import StrOutputParser
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage, AIMessage, SystemMessage

load_dotenv()


class PersonaType(Enum):
    """Types of personas"""
    EXPERT = "expert"
    TEACHER = "teacher"
    CRITIC = "critic"
    FRIEND = "friend"
    COACH = "coach"
    ANALYST = "analyst"
    CREATIVE = "creative"
    COUNSELOR = "counselor"


@dataclass
class Persona:
    """Defines a role-playing persona"""
    name: str
    role: str
    expertise: List[str]
    personality_traits: List[str]
    communication_style: str
    background: str
    system_message: str


@dataclass
class PersonaConfig:
    """Configuration for persona-based interaction"""
    temperature: float = 0.7
    model_name: str = "gpt-3.5-turbo"
    max_tokens: Optional[int] = None
    maintain_history: bool = True


class RolePlayingAgent:
    """
    Agent that adopts different personas for specialized interactions.
    
    This agent demonstrates:
    1. Multiple pre-defined personas
    2. Persona switching
    3. Consistent character maintenance
    4. Domain-specific expertise simulation
    """
    
    def __init__(self, config: Optional[PersonaConfig] = None):
        """
        Initialize role-playing agent.
        
        Args:
            config: Configuration for persona interactions
        """
        self.config = config or PersonaConfig()
        self.llm = ChatOpenAI(
            model=self.config.model_name,
            temperature=self.config.temperature
        )
        self.personas = self._initialize_personas()
        self.conversation_history: List[Any] = []
    
    def _initialize_personas(self) -> Dict[str, Persona]:
        """Initialize library of personas"""
        return {
            "doctor": Persona(
                name="Dr. Sarah Johnson",
                role="Medical Doctor",
                expertise=["General Medicine", "Preventive Care", "Health Education"],
                personality_traits=["Empathetic", "Patient", "Thorough", "Professional"],
                communication_style="Clear, reassuring, and professional. Uses simple language to explain medical concepts.",
                background="15 years of experience in family medicine. Focus on patient education and preventive care.",
                system_message="""You are Dr. Sarah Johnson, a compassionate and experienced medical doctor with 15 years in family medicine.

Your approach:
- Speak in a warm, professional, and reassuring tone
- Explain medical concepts in simple, accessible language
- Ask clarifying questions about symptoms
- Provide general health guidance (not specific diagnoses)
- Always recommend consulting a healthcare provider for serious concerns
- Show empathy and understanding for patient concerns

Important: You provide general health information only. Always remind users to consult their healthcare provider for personalized medical advice."""
            ),
            
            "teacher": Persona(
                name="Professor Alex Chen",
                role="Patient Teacher",
                expertise=["Education", "Learning Science", "Curriculum Design"],
                personality_traits=["Patient", "Encouraging", "Thorough", "Enthusiastic"],
                communication_style="Clear, step-by-step explanations with examples and encouragement.",
                background="20 years teaching experience. Specializes in breaking down complex topics.",
                system_message="""You are Professor Alex Chen, a patient and enthusiastic teacher with 20 years of experience.

Your teaching approach:
- Break complex topics into manageable steps
- Use clear examples and analogies
- Check for understanding regularly
- Provide encouragement and positive reinforcement
- Adapt explanations to student's level
- Use the Socratic method to guide learning
- Make learning engaging and fun

Remember: Every student learns differently. Be patient, encouraging, and adaptive to their needs."""
            ),
            
            "critic": Persona(
                name="Marcus Stone",
                role="Constructive Critic",
                expertise=["Analysis", "Critical Thinking", "Quality Assurance"],
                personality_traits=["Analytical", "Direct", "Thorough", "Fair"],
                communication_style="Direct, analytical, focuses on both strengths and weaknesses.",
                background="Expert reviewer with focus on constructive feedback and improvement.",
                system_message="""You are Marcus Stone, a fair but thorough critic who provides constructive feedback.

Your approach:
- Analyze thoroughly and objectively
- Point out both strengths and weaknesses
- Provide specific, actionable feedback
- Explain the reasoning behind critiques
- Suggest concrete improvements
- Maintain professionalism and respect
- Balance criticism with recognition of merits

Your goal is to help improve quality through honest, constructive analysis."""
            ),
            
            "coach": Persona(
                name="Coach Sam Rivera",
                role="Life Coach",
                expertise=["Goal Setting", "Motivation", "Personal Development"],
                personality_traits=["Motivational", "Supportive", "Direct", "Energetic"],
                communication_style="Energetic, motivational, action-oriented with practical strategies.",
                background="Certified life coach helping people achieve their goals for 10 years.",
                system_message="""You are Coach Sam Rivera, an energetic and motivational life coach.

Your coaching style:
- Help clients clarify their goals
- Break big goals into actionable steps
- Provide motivation and encouragement
- Hold clients accountable
- Celebrate progress and wins
- Address obstacles and limiting beliefs
- Use powerful questions to unlock insights
- Focus on action and results

You believe everyone has untapped potential. Your job is to help them discover and achieve it!"""
            ),
            
            "analyst": Persona(
                name="Dr. Emily Watson",
                role="Business Analyst",
                expertise=["Data Analysis", "Strategic Planning", "Market Research"],
                personality_traits=["Analytical", "Objective", "Detail-oriented", "Strategic"],
                communication_style="Data-driven, structured, focuses on insights and recommendations.",
                background="15 years in business intelligence and strategic consulting.",
                system_message="""You are Dr. Emily Watson, a senior business analyst with expertise in data-driven decision making.

Your analytical approach:
- Examine problems systematically
- Look for patterns and trends
- Use data to support conclusions
- Consider multiple perspectives
- Identify risks and opportunities
- Provide structured recommendations
- Think strategically about implications
- Present insights clearly and concisely

You help organizations make informed decisions through rigorous analysis."""
            ),
            
            "creative": Persona(
                name="Luna Martinez",
                role="Creative Director",
                expertise=["Creative Writing", "Brainstorming", "Innovation"],
                personality_traits=["Imaginative", "Enthusiastic", "Bold", "Inspiring"],
                communication_style="Vibrant, imaginative, encourages thinking outside the box.",
                background="Award-winning creative director with passion for innovation.",
                system_message="""You are Luna Martinez, an imaginative and award-winning creative director.

Your creative approach:
- Think unconventionally and boldly
- Generate multiple creative ideas
- Encourage wild brainstorming
- Look for unexpected connections
- Challenge conventional thinking
- Inspire with enthusiasm
- Balance creativity with practicality
- Help refine and develop ideas

"Creativity is intelligence having fun" - and you bring both! Help others unlock their creative potential."""
            ),
        }
    
    def interact_with_persona(
        self,
        persona_key: str,
        user_message: str,
        reset_history: bool = False
    ) -> str:
        """
        Interact with a specific persona.
        
        Args:
            persona_key: Key of persona to use
            user_message: User's message
            reset_history: Whether to reset conversation history
            
        Returns:
            Persona's response
        """
        if persona_key not in self.personas:
            return f"Persona '{persona_key}' not found. Available: {list(self.personas.keys())}"
        
        persona = self.personas[persona_key]
        
        # Reset history if requested
        if reset_history or not self.config.maintain_history:
            self.conversation_history = []
        
        # Create prompt with persona
        if self.conversation_history:
            messages = [
                SystemMessage(content=persona.system_message),
                *self.conversation_history,
                HumanMessage(content=user_message)
            ]
        else:
            messages = [
                SystemMessage(content=persona.system_message),
                HumanMessage(content=user_message)
            ]
        
        # Get response
        response = self.llm.invoke(messages)
        
        # Update history
        if self.config.maintain_history:
            self.conversation_history.append(HumanMessage(content=user_message))
            self.conversation_history.append(AIMessage(content=response.content))
        
        return response.content
    
    def multi_persona_analysis(
        self,
        question: str,
        persona_keys: List[str]
    ) -> Dict[str, str]:
        """
        Get perspectives from multiple personas.
        
        Args:
            question: Question to analyze
            persona_keys: List of persona keys to consult
            
        Returns:
            Dictionary of persona responses
        """
        results = {}
        
        for key in persona_keys:
            if key in self.personas:
                # Reset history for clean analysis
                response = self.interact_with_persona(key, question, reset_history=True)
                results[key] = response
        
        return results
    
    def persona_debate(
        self,
        topic: str,
        persona1_key: str,
        persona2_key: str,
        rounds: int = 2
    ) -> List[Dict[str, str]]:
        """
        Have two personas debate a topic.
        
        Args:
            topic: Topic to debate
            persona1_key: First persona key
            persona2_key: Second persona key
            rounds: Number of debate rounds
            
        Returns:
            List of debate exchanges
        """
        if persona1_key not in self.personas or persona2_key not in self.personas:
            return [{"error": "One or both personas not found"}]
        
        persona1 = self.personas[persona1_key]
        persona2 = self.personas[persona2_key]
        
        debate = []
        
        # Initial statements
        prompt1 = f"Present your perspective on: {topic}"
        response1 = self.interact_with_persona(persona1_key, prompt1, reset_history=True)
        debate.append({
            "round": 1,
            "persona": persona1.name,
            "statement": response1
        })
        
        # Alternating responses
        for round_num in range(2, rounds + 2):
            # Persona 2 responds to Persona 1
            prompt2 = f"Respond to this perspective on '{topic}': {response1}"
            response2 = self.interact_with_persona(persona2_key, prompt2, reset_history=True)
            debate.append({
                "round": round_num,
                "persona": persona2.name,
                "statement": response2
            })
            
            if round_num  Optional[Persona]:
        """Get information about a persona"""
        return self.personas.get(persona_key)
    
    def list_personas(self) -> List[str]:
        """List available personas"""
        return list(self.personas.keys())


def demonstrate_role_playing():
    """Demonstrate role-playing/persona patterns"""
    print("=" * 80)
    print("ROLE-PLAYING/PERSONA PATTERN DEMONSTRATION")
    print("=" * 80)
    
    agent = RolePlayingAgent()
    
    # Example 1: Doctor Persona
    print("\n" + "=" * 80)
    print("EXAMPLE 1: Doctor Persona - Health Consultation")
    print("=" * 80)
    
    health_question = "I've been feeling tired lately and having trouble sleeping. What could be causing this?"
    print(f"\nPatient: {health_question}")
    print(f"\nDr. Johnson:")
    doctor_response = agent.interact_with_persona("doctor", health_question, reset_history=True)
    print(doctor_response)
    
    # Follow-up
    followup = "I'm also drinking a lot of coffee throughout the day."
    print(f"\n\nPatient (follow-up): {followup}")
    print(f"\nDr. Johnson:")
    doctor_followup = agent.interact_with_persona("doctor", followup)
    print(doctor_followup)
    
    # Example 2: Teacher Persona
    print("\n" + "=" * 80)
    print("EXAMPLE 2: Teacher Persona - Explaining Complex Topic")
    print("=" * 80)
    
    learning_question = "Can you explain how photosynthesis works?"
    print(f"\nStudent: {learning_question}")
    print(f"\nProfessor Chen:")
    teacher_response = agent.interact_with_persona("teacher", learning_question, reset_history=True)
    print(teacher_response)
    
    # Example 3: Critic Persona
    print("\n" + "=" * 80)
    print("EXAMPLE 3: Critic Persona - Essay Review")
    print("=" * 80)
    
    essay = """My essay argues that social media has both positive and negative impacts on society. 
It connects people but also spreads misinformation. It helps businesses but can harm mental health."""
    
    critique_request = f"Please review this essay introduction: {essay}"
    print(f"\nWriter: {critique_request}")
    print(f"\nMarcus Stone (Critic):")
    critic_response = agent.interact_with_persona("critic", critique_request, reset_history=True)
    print(critic_response)
    
    # Example 4: Coach Persona
    print("\n" + "=" * 80)
    print("EXAMPLE 4: Coach Persona - Goal Setting")
    print("=" * 80)
    
    goal_question = "I want to learn programming but I keep procrastinating. How can I stay motivated?"
    print(f"\nClient: {goal_question}")
    print(f"\nCoach Rivera:")
    coach_response = agent.interact_with_persona("coach", goal_question, reset_history=True)
    print(coach_response)
    
    # Example 5: Multi-Persona Analysis
    print("\n" + "=" * 80)
    print("EXAMPLE 5: Multi-Persona Analysis - Different Perspectives")
    print("=" * 80)
    
    question = "Should I pursue a career change at age 40?"
    print(f"\nQuestion: {question}")
    print("\nGetting perspectives from multiple personas...\n")
    
    personas_to_consult = ["coach", "analyst", "counselor"]
    # Note: counselor not defined, will be skipped
    personas_to_consult = ["coach", "analyst"]
    
    perspectives = agent.multi_persona_analysis(question, personas_to_consult)
    
    for persona_key, response in perspectives.items():
        persona = agent.get_persona_info(persona_key)
        print(f"\n{persona.name} ({persona.role}):")
        print("-" * 80)
        print(response)
    
    # Example 6: Persona Debate
    print("\n" + "=" * 80)
    print("EXAMPLE 6: Persona Debate - Creative vs Analyst")
    print("=" * 80)
    
    debate_topic = "What's more important for business success: creativity or data-driven decision making?"
    print(f"\nDebate Topic: {debate_topic}\n")
    
    debate_exchanges = agent.persona_debate(
        debate_topic,
        "creative",
        "analyst",
        rounds=2
    )
    
    for exchange in debate_exchanges:
        print(f"\nRound {exchange['round']} - {exchange['persona']}:")
        print("-" * 80)
        print(exchange['statement'])
    
    # Summary
    print("\n" + "=" * 80)
    print("ROLE-PLAYING/PERSONA PATTERN SUMMARY")
    print("=" * 80)
    print(f"""
Role-Playing Pattern Benefits:
1. Enhanced Expertise: Simulates domain-specific knowledge
2. Consistent Personality: Maintains character across interactions
3. Improved Engagement: More natural and relatable interactions
4. Context-Appropriate: Responses match expected communication style
5. Multi-Perspective Analysis: Diverse viewpoints on problems

Key Personas Demonstrated:
1. Doctor (Dr. Sarah Johnson): Empathetic health guidance
2. Teacher (Prof. Alex Chen): Patient, step-by-step explanations
3. Critic (Marcus Stone): Constructive analytical feedback
4. Coach (Coach Sam Rivera): Motivational goal-setting support
5. Analyst (Dr. Emily Watson): Data-driven strategic thinking
6. Creative (Luna Martinez): Imaginative idea generation

Persona Components:
- Identity: Name, role, background
- Expertise: Domain knowledge areas
- Personality: Communication traits
- Style: Tone, vocabulary, structure
- Approach: Problem-solving methodology

Best Practices:
1. Define personas thoroughly with clear characteristics
2. Maintain consistency across conversation turns
3. Use appropriate disclaimers (especially for professional advice)
4. Stay within persona's expertise boundaries
5. Enable history for multi-turn conversations
6. Test personas across diverse inputs
7. Monitor for inappropriate or biased responses

Advanced Techniques:
1. Multi-Persona Analysis: Get diverse perspectives
2. Persona Debates: Explore different viewpoints
3. Dynamic Switching: Change personas based on context
4. Persona Blending: Combine traits from multiple personas
5. Adaptive Expertise: Adjust knowledge depth to user level

Production Considerations:
- Clear AI disclosure (users know it's AI)
- Professional advice disclaimers
- Content safety guardrails
- Persona consistency monitoring
- User feedback collection
- Regular persona updates and improvements
- A/B testing persona variations

When to Use Role-Playing:
- Domain-specific expertise needed
- Consistent personality desired
- Engagement and relatability important
- Multiple perspectives valuable
- Specialized communication style required

Available Personas: {agent.list_personas()}
""")
    
    print("\n" + "=" * 80)
    print("Pattern 084 (Role-Playing/Persona) demonstration complete!")
    print("=" * 80)


if __name__ == "__main__":
    demonstrate_role_playing()

    Agentic Design Pattern: Role Playing

    Pattern 084: Role Playing – Line-by-Line Explanation

    Overview

    This document provides a detailed line-by-line explanation of the Role Playing pattern implementation.

    Pattern Description

    Pattern 084: Role-Playing/Persona Pattern

    Description: The Role-Playing/Persona pattern involves instructing the LLM to adopt a specific role, character, or persona when generating responses. This pattern leverages the model’s ability to simulate different perspectives, expertise levels, personalities, and communication styles. By assuming a role, the LLM can provide more contextually appropriate, specialized, and engaging responses.

    Role-playing is a powerful prompt engineering technique that:
- Enhances response quality through perspective-taking
- Enables domain expertise simulation (doctor, lawyer, scientist)
- Creates consistent personality in conversational agents
- Improves user engagement through character-based interaction
- Allows multi-perspective analysis of problems

    Components: 1. Persona Definition – Identity (who the persona is) – Expertise (what they know) – Personality traits (how they communicate) – Background (relevant experience) – Goals and motivations

    2. Communication Style
   - Tone (formal, casual, friendly, professional)
   - Vocabulary level (simple, technical, academic)
   - Response structure (detailed, concise, step-by-step)
   - Interaction patterns (questioning, advising, teaching)

3. Persona Consistency
   - Maintaining character across turns
   - Memory of previous interactions
   - Consistent viewpoint and opinions
   - Appropriate domain knowledge boundaries

4. Context Adaptation
   - Adjusting expertise level to user
   - Situational awareness
   - Emotional intelligence
   - Cultural sensitivity

    Use Cases: 1. Expert Consultation – Medical advice (doctor persona) – Legal guidance (lawyer persona) – Technical support (engineer persona) – Financial planning (financial advisor persona) – Career counseling (career coach persona)

    2. Educational Applications
   - Patient teacher (explaining complex topics)
   - Socratic tutor (guiding through questions)
   - Encouraging mentor (motivating learners)
   - Strict examiner (testing knowledge)

3. Creative Writing
   - Character dialogue generation
   - Story narration from specific viewpoint
   - Historical figure simulation
   - Fictional character interaction

4. Business Applications
   - Sales representative (persuasive, customer-focused)
   - Customer service (empathetic, problem-solving)
   - Business analyst (analytical, data-driven)
   - Executive advisor (strategic, big-picture)

5. Therapeutic and Counseling
   - Supportive friend (empathetic listener)
   - Life coach (motivational, goal-oriented)
   - Therapist (reflective, non-judgmental)
   - Meditation guide (calm, mindful)

    LangChain Implementation: LangChain facilitates role-playing through: – System message configuration (setting persona) – Context management (maintaining persona) – Memory integration (persona consistency) – Custom prompt templates (structured personas) – Chat history management (conversation continuity)

    Key Features: 1. Persona Library – Pre-defined expert personas – Customizable persona templates – Domain-specific knowledge bases – Personality trait combinations

    2. Dynamic Persona Switching
   - Context-aware persona selection
   - Smooth transitions between personas
   - Multi-persona conversations
   - Persona blending

3. Expertise Calibration
   - Adjusting knowledge depth
   - Acknowledging limitations
   - Appropriate confidence levels
   - Domain boundary awareness

4. Personality Traits
   - Empathy and emotional intelligence
   - Humor and engagement style
   - Assertiveness levels
   - Teaching vs directing approaches

    Best Practices: 1. Persona Definition – Be specific about role and expertise – Include relevant background details – Define communication style clearly – Set appropriate boundaries

    2. Consistency Maintenance
   - Use memory for multi-turn conversations
   - Reference persona context regularly
   - Maintain viewpoint across interactions
   - Avoid out-of-character responses

3. Expertise Management
   - Stay within persona's knowledge domain
   - Acknowledge when unsure
   - Provide persona-appropriate disclaimers
   - Don't overreach expertise claims

4. Engagement Optimization
   - Match tone to user expectations
   - Balance expertise with accessibility
   - Use persona-appropriate examples
   - Maintain engaging interaction style

    Trade-offs: Advantages: – Enhanced response quality and relevance – More engaging and natural interactions – Domain-specific expertise simulation – Consistent personality and tone – Better user trust and satisfaction – Multi-perspective problem analysis

    Disadvantages:
- Requires detailed persona definitions
- May limit flexibility in responses
- Risk of inappropriate persona adoption
- Can be perceived as deceptive if not transparent
- Harder to maintain over long conversations
- May introduce unwanted biases

    Production Considerations: 1. Transparency – Clearly indicate AI persona usage – Provide disclaimers for professional advice – Explain limitations of simulated expertise – Allow users to request different personas

    2. Safety and Ethics
   - Avoid harmful or inappropriate personas
   - Include safety guardrails
   - Monitor for persona misuse
   - Implement content filtering
   - Respect professional boundaries

3. Quality Assurance
   - Test personas across diverse inputs
   - Validate expertise accuracy
   - Monitor consistency metrics
   - Gather user feedback
   - Regular persona updates

4. Scalability
   - Modular persona definitions
   - Reusable persona components
   - Version control for personas
   - A/B test persona variations
   - Performance monitoring per persona

5. Legal Compliance
   - Medical/legal advice disclaimers
   - Professional credential clarification
   - Terms of service alignment
   - Privacy considerations
   - Liability protection

    File Statistics

    • Total Lines: 706
    • Classes: 4
    • Functions: 1
    • Constants: 0

    Module Documentation

    """
Pattern 084: Role-Playing/Persona Pattern

Description:
    The Role-Playing/Persona pattern involves instructing the LLM to adopt a specific
    role, character, or persona when generating responses. This pattern leverages the
    model's ability to simulate different perspectives, expertise levels, personalities,
    and communication styles. By assuming a role, the LLM can provide more contextually
    appropriate, specialized, and engaging responses.

    Role-playing is a powerful prompt engineering technique that:
    - Enhances response quality through perspective-taking
    - Enables domain expertise simulation (doctor, lawyer, scientist)
    - Creates consistent personality in conversational agents
    - Improves user engagement through character-based interaction
    - Allows multi-perspective analysis of problems

Components:
    1. Persona Definition
       - Identity (who the persona is)
       - Expertise (what they know)
       - Personality traits (how they communicate)
       - Background (relevant experience)
       - Goals and motivations

    2. Communication Style
       - Tone (formal, casual, friendly, professional)
       - Vocabulary level (simple, technical, academic)
       - Response structure (detailed, concise, step-by-step)
       - Interaction patterns (questioning, advising, teaching)

    3. Persona Consistency
       - Maintaining character across turns
       - Memory of previous interactions
       - Consistent viewpoint and opinions
       - Appropriate domain knowledge boundaries

    4. Context Adaptation
       - Adjusting expertise level to user
       - Situational awareness
       - Emotional intelligence
       - Cultural sensitivity

Use Cases:
    1. Expert Consultation
       - Medical advice (doctor persona)
       - Legal guidance (lawyer persona)
       - Technical support (engineer persona)
       - Financial planning (financial advisor persona)
       - Career counseling (career coach persona)

    2. Educational Applications
       - Patient teacher (explaining complex topics)
       - Socratic tutor (guiding through questions)
       - Encouraging mentor (motivating learners)
       - Strict examiner (testing knowledge)

    3. Creative Writing
       - Character dialogue generation
       - Story narration from specific viewpoint
       - Historical figure simulation
       - Fictional character interaction

    4. Business Applications
       - Sales representative (persuasive, customer-focused)
       - Customer service (empathetic, problem-solving)
       - Business analyst (analytical, data-driven)
       - Executive advisor (strategic, big-picture)

    5. Therapeutic and Counseling
       - Supportive friend (empathetic listener)
       - Life coach (motivational, goal-oriented)
       - Therapist (reflective, non-judgmental)
       - Meditation guide (calm, mindful)

LangChain Implementation:
    LangChain facilitates role-playing through:
    - System message configuration (setting persona)
    - Context management (maintaining persona)
    - Memory integration (persona consistency)
    - Custom prompt templates (structured personas)
    - Chat history management (conversation continuity)

Key Features:
    1. Persona Library
       - Pre-defined expert personas
       - Customizable persona templates
       - Domain-specific knowledge bases
       - Personality trait combinations

    2. Dynamic Persona Switching
       - Context-aware persona selection
       - Smooth transitions between personas
       - Multi-persona conversations
       - Persona blending

    3. Expertise Calibration
       - Adjusting knowledge depth
       - Acknowledging limitations
       - Appropriate confidence levels
       - Domain boundary awareness

    4. Personality Traits
       - Empathy and emotional intelligence
       - Humor and engagement style
       - Assertiveness levels
       - Teaching vs directing approaches

Best Practices:
    1. Persona Definition
       - Be specific about role and expertise
       - Include relevant background details
       - Define communication style clearly
       - Set appropriate boundaries

    2. Consistency Maintenance
       - Use memory for multi-turn conversations
       - Reference persona context regularly
       - Maintain viewpoint across interactions
       - Avoid out-of-character responses

    3. Expertise Management
       - Stay within persona's knowledge domain
       - Acknowledge when unsure
       - Provide persona-appropriate disclaimers
       - Don't overreach expertise claims

    4. Engagement Optimization
       - Match tone to user expectations
       - Balance expertise with accessibility
       - Use persona-appropriate examples
       - Maintain engaging interaction style

Trade-offs:
    Advantages:
    - Enhanced response quality and relevance
    - More engaging and natural interactions
    - Domain-specific expertise simulation
    - Consistent personality and tone
    - Better user trust and satisfaction
    - Multi-perspective problem analysis

    Disadvantages:
    - Requires detailed persona definitions
    - May limit flexibility in responses
    - Risk of inappropriate persona adoption
    - Can be perceived as deceptive if not transparent
    - Harder to maintain over long conversations
    - May introduce unwanted biases

Production Considerations:
    1. Transparency
       - Clearly indicate AI persona usage
       - Provide disclaimers for professional advice
       - Explain limitations of simulated expertise
       - Allow users to request different personas

    2. Safety and Ethics
       - Avoid harmful or inappropriate personas
       - Include safety guardrails
       - Monitor for persona misuse
       - Implement content filtering
       - Respect professional boundaries

    3. Quality Assurance
       - Test personas across diverse inputs
       - Validate expertise accuracy
       - Monitor consistency metrics
       - Gather user feedback
       - Regular persona updates

    4. Scalability
       - Modular persona definitions
       - Reusable persona components
       - Version control for personas
       - A/B test persona variations
       - Performance monitoring per persona

    5. Legal Compliance
       - Medical/legal advice disclaimers
       - Professional credential clarification
       - Terms of service alignment
       - Privacy considerations
       - Liability protection
"""

    Purpose: This module implements the Role Playing pattern, providing a structured approach to agent design and implementation.

    Import Statements

    Standard Library Imports

    import dataclasses.dataclass
import enum.Enum
import langchain_core.messages.AIMessage
import langchain_core.messages.HumanMessage
import langchain_core.messages.SystemMessage
import langchain_core.output_parsers.StrOutputParser
import langchain_core.prompts.ChatPromptTemplate
import langchain_core.prompts.MessagesPlaceholder
import os
import typing.Any
import typing.Dict
import typing.List
import typing.Optional

    Purpose: Core Python functionality: – dataclasses.dataclass: Simplified class definitions with @dataclass – enum.Enum: Enumeration support – langchain_core.messages.AIMessage: Regular expression operations – langchain_core.messages.HumanMessage: Regular expression operations – langchain_core.messages.SystemMessage: Regular expression operations – langchain_core.output_parsers.StrOutputParser: Regular expression operations – langchain_core.prompts.ChatPromptTemplate: Regular expression operations – langchain_core.prompts.MessagesPlaceholder: Regular expression operations – os: Environment variable access and system operations – typing.Any: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Dict: Type hints for better code documentation (List, Dict, Any, Optional) – typing.List: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Optional: Type hints for better code documentation (List, Dict, Any, Optional)

    LangChain Imports

    from langchain_openai.ChatOpenAI

    Purpose: LangChain framework components: – LLM Interface: Wrapper for OpenAI models – Chains: Composable building blocks for complex workflows

    Other Imports

    from dotenv.load_dotenv

    Purpose: Additional dependencies for specialized functionality.

    Classes

    Class: PersonaType

    Lines 201-210

    Inherits from: Enum

    Purpose:

    Types of personas

    Class: Persona

    Lines 214-222

    Purpose:

    Defines a role-playing persona

    Class: PersonaConfig

    Lines 226-231

    Purpose:

    Configuration for persona-based interaction

    Class: RolePlayingAgent

    Lines 234-531

    Purpose:

    Agent that adopts different personas for specialized interactions.

This agent demonstrates:
1. Multiple pre-defined personas
2. Persona switching
3. Consistent character maintenance
4. Domain-specific expertise simulation

    Methods:

    __init__(self, config)

    Lines 245-258

    Purpose:

    Initialize role-playing agent.

Args:
    config: Configuration for persona interactions

    Parameters: – config: Config

    _initialize_personas(self)

    Lines 260-390

    Purpose:

    Initialize library of personas

    Parameters:

    Returns: <ast.Subscript object at 0x000001A06D83E110>

    interact_with_persona(self, persona_key, user_message, reset_history)

    Lines 392-439

    Purpose:

    Interact with a specific persona.

Args:
    persona_key: Key of persona to use
    user_message: User's message
    reset_history: Whether to reset conversation history

Returns:
    Persona's response

    Parameters: – persona_key: Persona key – user_message: User message – reset_history: Reset history

    Returns: str

    multi_persona_analysis(self, question, persona_keys)

    Lines 441-464

    Purpose:

    Get perspectives from multiple personas.

Args:
    question: Question to analyze
    persona_keys: List of persona keys to consult

Returns:
    Dictionary of persona responses

    Parameters: – question: Question to answer – persona_keys: Persona keys

    Returns: <ast.Subscript object at 0x000001A06D6A9DD0>

    persona_debate(self, topic, persona1_key, persona2_key, rounds)

    Lines 466-523

    Purpose:

    Have two personas debate a topic.

Args:
    topic: Topic to debate
    persona1_key: First persona key
    persona2_key: Second persona key
    rounds: Number of debate rounds

Returns:
    List of debate exchanges

    Parameters: – topic: Topic – persona1_key: Persona1 key – persona2_key: Persona2 key – rounds: Rounds

    Returns: <ast.Subscript object at 0x000001A06D159F90>

    get_persona_info(self, persona_key)

    Lines 525-527

    Purpose:

    Get information about a persona

    Parameters: – persona_key: Persona key

    Returns: <ast.Subscript object at 0x000001A06D15AC10>

    list_personas(self)

    Lines 529-531

    Purpose:

    List available personas

    Parameters:

    Returns: <ast.Subscript object at 0x000001A06D60A810>

    Functions

    demonstrate_role_playing()

    Lines 534-701

    Purpose:

    Demonstrate role-playing/persona patterns

    Key Concepts Summary

    Pattern Overview

    The Role Playing pattern provides a structured approach to building AI agents with specific capabilities and behaviors. This implementation leverages LangChain’s composable components to create a robust and maintainable solution.

    Core Components

    1. Agent Class: Main implementation of the pattern logic
    2. Prompt Engineering: Carefully crafted prompts to guide LLM behavior
    3. Tool Integration: Optional tools for extended functionality
    4. Memory Management: State handling across interactions
    5. Output Parsing: Structured response processing

    When to Use

    ✅ Good for: – Scenarios requiring this specific pattern’s strengths – Applications needing structured agent behavior – Systems where explainability is important – Production environments requiring reliability

    ❌ Not ideal for: – Simple single-prompt tasks – Scenarios requiring different pattern characteristics – Resource-constrained environments (if computationally intensive)

    Best Practices Demonstrated

    1. Environment Configuration

    • ✅ Use .env files for API keys
    • ✅ Never hardcode credentials
    • ✅ Validate environment variables on startup

    2. Error Handling

    • ✅ Graceful degradation on failures
    • ✅ Informative error messages
    • ✅ Logging for debugging

    3. Prompt Engineering

    • ✅ Clear, specific instructions
    • ✅ Structured output formats
    • ✅ Few-shot examples when beneficial

    4. Code Organization

    • ✅ Separation of concerns (class-based design)
    • ✅ Type hints for clarity
    • ✅ Docstrings for documentation

    5. Performance Optimization

    • ✅ Appropriate temperature settings
    • ✅ Token budget management
    • ✅ Caching where applicable

    LangChain Components Used

    • ChatOpenAI: OpenAI model interface
    • Chains: Sequential component composition

    Why LangChain? – Composable abstractions for LLM applications – Rich ecosystem of integrations – Production-ready patterns – Active community and development

    Execution Flow

    1. Initialize agent with configuration
   ↓
2. Load environment and dependencies
   ↓
3. Prepare prompt with input data
   ↓
4. Invoke LLM through LangChain
   ↓
5. Parse and validate response
   ↓
6. Return formatted result

    Key Decision Points: – Model selection (temperature, model type) – Prompt structure (few-shot vs zero-shot) – Error handling strategy – Output format validation

    Performance Considerations

    Computational Cost

    • LLM Calls: Primary cost factor (based on tokens)
    • Latency: Depends on model and input size
    • Memory: Minimal for most patterns

    Optimization Strategies

    1. Prompt Optimization: Minimize tokens while maintaining clarity
    2. Caching: Cache repeated queries/responses
    3. Model Selection: Use smaller models when appropriate
    4. Batch Processing: Group similar requests
    5. Streaming: For long responses, stream tokens

    Cost Estimates (Approximate)

    • GPT-3.5-turbo: ~$0.002 per request
    • GPT-4: ~$0.03 per request
    • GPT-4-turbo: ~$0.01 per request

    Actual costs vary based on input/output token counts

    Common Pitfalls to Avoid

    1. Prompt Design Issues

    ❌ Vague or ambiguous instructions
    ✅ Clear, specific, testable prompts

    2. Error Handling

    ❌ Ignoring API errors or rate limits
    ✅ Implement retry logic and graceful degradation

    3. Token Management

    ❌ Exceeding model token limits
    ✅ Truncate inputs, chunk processing

    4. Security

    ❌ Exposing API keys in code
    ✅ Use environment variables

    5. Testing

    ❌ Only testing happy paths
    ✅ Test edge cases, failures, malformed inputs

    6. Cost Management

    ❌ Unmonitored API usage
    ✅ Implement budgets, alerts, and monitoring

    Further Improvements

    Production Enhancements

    1. Monitoring: Add logging, metrics, tracing
    2. Validation: Input/output schema validation
    3. Rate Limiting: Implement request throttling
    4. Caching: Redis or in-memory cache for responses
    5. Async Support: Non-blocking operations for scalability

    Feature Extensions

    1. Multi-Model Support: Switch between providers
    2. Fallback Models: Graceful degradation strategy
    3. Custom Tools: Domain-specific function calling
    4. Advanced Memory: Vector stores for long-term memory
    5. Evaluation: Automated testing and benchmarking

    Code Quality

    1. Testing: Unit tests, integration tests
    2. Documentation: API docs, usage examples
    3. Type Safety: Stricter type hints
    4. Configuration: Externalized config files

    Related Patterns

    • Pattern 083
    • Pattern 085

    Pattern Families: – Reasoning Patterns (001-011) – Multi-Agent Patterns (012-019) – Tool Use Patterns (020-025) – Memory Patterns (026-031) – And many more…

    Summary

    The Role Playing pattern provides a powerful approach to building AI agents with specific capabilities. This implementation demonstrates:

    ✅ Key Strengths: – Clean, maintainable code structure – LangChain integration for composability – Type hints and documentation – Production-ready patterns

    🎯 Best Use Cases: – Applications requiring this pattern’s characteristics – Systems needing structured agent behavior – Production environments requiring reliability

    📚 Learning Resources: – LangChain Documentation: https://python.langchain.com/ – Pattern Documentation: See related patterns – OpenAI API Reference: https://platform.openai.com/docs/

    File: 084_role_playing.py
    Total Lines: 706
    Pattern Category: AI Agent Design Pattern #084

    Generated from actual implementation code
    For educational and reference purposes

  • Agentic Design Pattern: Few Shot Learning

    
    """
Pattern 083: Few-Shot Learning Pattern

Description:
    The Few-Shot Learning pattern uses example-based prompting to teach the LLM
    how to perform tasks through demonstrations. Instead of relying on zero-shot
    capabilities, this pattern provides the model with a few examples of input-output
    pairs that illustrate the desired behavior. This approach leverages the LLM's
    in-context learning abilities to generalize from examples.

    Few-shot learning is one of the most fundamental prompt engineering techniques
    and is particularly effective when:
    - The task format is non-standard or complex
    - Consistent output format is required
    - Specific style or tone needs to be maintained
    - Domain-specific knowledge needs to be demonstrated
    - Edge cases need to be handled in specific ways

Components:
    1. Example Selection
       - Static examples (fixed set)
       - Dynamic examples (selected based on similarity)
       - Diverse examples (covering different scenarios)
       - Representative examples (typical cases)

    2. Example Formatting
       - Clear input-output structure
       - Consistent formatting across examples
       - Annotations and explanations (when needed)
       - Order considerations (easy to hard, chronological)

    3. Example Storage
       - In-prompt storage (small example sets)
       - Vector database storage (large example sets)
       - Retrieval mechanisms (similarity-based)
       - Example caching strategies

    4. Generalization Strategy
       - Pattern recognition from examples
       - Analogical reasoning
       - Template extraction
       - Feature learning

Use Cases:
    1. Text Classification
       - Sentiment analysis with custom categories
       - Intent recognition for domain-specific applications
       - Content moderation with nuanced rules
       - Topic categorization with specific taxonomies

    2. Data Extraction
       - Structured information extraction from unstructured text
       - Entity recognition with custom entity types
       - Relationship extraction between entities
       - Attribute extraction with specific formats

    3. Code Generation
       - API usage examples for specific libraries
       - Code style consistency (following project conventions)
       - Design pattern implementation
       - Test case generation

    4. Creative Writing
       - Style mimicry (writing in specific author's style)
       - Format consistency (poems, stories, articles)
       - Tone control (formal, casual, humorous)
       - Genre-specific writing (sci-fi, mystery, romance)

    5. Translation and Transformation
       - Domain-specific translation (legal, medical, technical)
       - Format conversion (JSON to YAML, CSV to markdown)
       - Data normalization (various formats to standard)
       - Code refactoring patterns

LangChain Implementation:
    LangChain provides excellent support for few-shot learning through:
    - FewShotChatMessagePromptTemplate for chat models
    - FewShotPromptTemplate for completion models
    - Example selectors (semantic similarity, length-based, max marginal relevance)
    - Vector store-backed example selection
    - Custom example formatting

Key Features:
    1. Static Few-Shot Learning
       - Predefined set of examples
       - Fast and deterministic
       - Good for stable tasks
       - No retrieval overhead

    2. Dynamic Few-Shot Learning
       - Examples selected based on input similarity
       - More adaptive to diverse inputs
       - Requires vector database
       - Better generalization

    3. Example Quality Management
       - Diverse examples covering edge cases
       - Clear and unambiguous examples
       - Representative of target distribution
       - Regularly updated based on performance

    4. Performance Optimization
       - Token budget management (limiting example count)
       - Example compression (removing redundancy)
       - Caching frequently used example sets
       - Lazy loading for large example sets

Best Practices:
    1. Example Selection
       - Use 3-5 examples for most tasks (sweet spot)
       - Include diverse examples covering different patterns
       - Order examples from simple to complex
       - Ensure examples are correct and high-quality

    2. Example Formatting
       - Use consistent structure across all examples
       - Clearly separate input from output
       - Add brief explanations for complex cases
       - Use delimiters to avoid confusion

    3. Dynamic Selection
       - Use semantic similarity for input-dependent selection
       - Balance diversity and relevance in selection
       - Cache embeddings for better performance
       - Set appropriate k value (number of examples)

    4. Iteration and Improvement
       - Start with few examples, add more if needed
       - A/B test different example sets
       - Monitor performance across different inputs
       - Update examples based on failure cases

Trade-offs:
    Advantages:
    - Significantly improves task performance vs zero-shot
    - More reliable than instructions alone
    - Enables consistent output formatting
    - Works well for complex or ambiguous tasks
    - Reduces need for fine-tuning

    Disadvantages:
    - Increases token usage (more expensive)
    - May not scale to tasks requiring many examples
    - Examples can bias the model inappropriately
    - Requires careful example curation
    - Dynamic selection adds latency and complexity

Production Considerations:
    1. Token Budget Management
       - Monitor tokens used by examples
       - Balance example count with context window
       - Use example compression when possible
       - Consider model's context limit

    2. Example Maintenance
       - Version control for example sets
       - Regular review and updates
       - Performance tracking per example
       - A/B testing of example variations

    3. Scalability
       - Cache example embeddings
       - Use efficient vector stores
       - Batch process when possible
       - Consider example pruning strategies

    4. Quality Assurance
       - Validate examples are correct
       - Test examples independently
       - Monitor output quality metrics
       - Handle example selection failures gracefully

    5. Cost Optimization
       - Balance example count with quality
       - Use dynamic selection judiciously
       - Cache common example sets
       - Monitor cost per query
"""

import os
from typing import List, Dict, Any, Optional
from dataclasses import dataclass
from enum import Enum
from dotenv import load_dotenv

from langchain_core.prompts import (
    ChatPromptTemplate,
    FewShotChatMessagePromptTemplate,
    PromptTemplate
)
from langchain_core.output_parsers import StrOutputParser
from langchain_openai import ChatOpenAI, OpenAIEmbeddings
from langchain_core.example_selectors import (
    SemanticSimilarityExampleSelector,
    MaxMarginalRelevanceExampleSelector
)
from langchain_chroma import Chroma

load_dotenv()


class ExampleSelectionStrategy(Enum):
    """Strategy for selecting examples"""
    STATIC = "static"  # Fixed set of examples
    SEMANTIC_SIMILARITY = "semantic_similarity"  # Based on input similarity
    MAX_MARGINAL_RELEVANCE = "max_marginal_relevance"  # Diverse + relevant


@dataclass
class FewShotExample:
    """Represents a single few-shot example"""
    input: str
    output: str
    explanation: Optional[str] = None
    metadata: Optional[Dict[str, Any]] = None


@dataclass
class FewShotConfig:
    """Configuration for few-shot learning"""
    num_examples: int = 3
    selection_strategy: ExampleSelectionStrategy = ExampleSelectionStrategy.STATIC
    temperature: float = 0.3
    model_name: str = "gpt-3.5-turbo"
    include_explanations: bool = False


class FewShotLearningAgent:
    """
    Agent that uses few-shot learning to teach LLM through examples.
    
    This agent demonstrates various few-shot learning approaches:
    1. Static few-shot (fixed examples)
    2. Dynamic few-shot (similarity-based selection)
    3. Task-specific few-shot applications
    """
    
    def __init__(self, config: Optional[FewShotConfig] = None):
        """
        Initialize few-shot learning agent.
        
        Args:
            config: Configuration for few-shot learning
        """
        self.config = config or FewShotConfig()
        self.llm = ChatOpenAI(
            model=self.config.model_name,
            temperature=self.config.temperature
        )
        self.embeddings = OpenAIEmbeddings()
    
    def create_static_few_shot_chain(
        self,
        examples: List[FewShotExample],
        task_instruction: str,
        input_variable: str = "input"
    ):
        """
        Create a chain with static few-shot examples.
        
        Args:
            examples: List of few-shot examples
            task_instruction: Instruction for the task
            input_variable: Name of input variable
            
        Returns:
            Runnable chain with few-shot prompting
        """
        # Format examples as dictionaries
        example_dicts = [
            {"input": ex.input, "output": ex.output}
            for ex in examples
        ]
        
        # Create example prompt template
        example_prompt = ChatPromptTemplate.from_messages([
            ("human", "{input}"),
            ("ai", "{output}")
        ])
        
        # Create few-shot prompt
        few_shot_prompt = FewShotChatMessagePromptTemplate(
            example_prompt=example_prompt,
            examples=example_dicts
        )
        
        # Create final prompt
        final_prompt = ChatPromptTemplate.from_messages([
            ("system", task_instruction),
            few_shot_prompt,
            ("human", "{" + input_variable + "}")
        ])
        
        # Create chain
        chain = final_prompt | self.llm | StrOutputParser()
        
        return chain
    
    def create_dynamic_few_shot_chain(
        self,
        examples: List[FewShotExample],
        task_instruction: str,
        input_variable: str = "input",
        k: int = 3
    ):
        """
        Create a chain with dynamic example selection based on similarity.
        
        Args:
            examples: Pool of examples to select from
            task_instruction: Instruction for the task
            input_variable: Name of input variable
            k: Number of examples to select
            
        Returns:
            Runnable chain with dynamic few-shot prompting
        """
        # Format examples as dictionaries
        example_dicts = [
            {"input": ex.input, "output": ex.output}
            for ex in examples
        ]
        
        # Create example selector using semantic similarity
        example_selector = SemanticSimilarityExampleSelector.from_examples(
            example_dicts,
            self.embeddings,
            Chroma,
            k=k,
            input_keys=["input"]
        )
        
        # Create example prompt template
        example_prompt = ChatPromptTemplate.from_messages([
            ("human", "{input}"),
            ("ai", "{output}")
        ])
        
        # Create few-shot prompt with selector
        few_shot_prompt = FewShotChatMessagePromptTemplate(
            example_prompt=example_prompt,
            example_selector=example_selector,
            input_variables=[input_variable]
        )
        
        # Create final prompt
        final_prompt = ChatPromptTemplate.from_messages([
            ("system", task_instruction),
            few_shot_prompt,
            ("human", "{" + input_variable + "}")
        ])
        
        # Create chain
        chain = final_prompt | self.llm | StrOutputParser()
        
        return chain
    
    def sentiment_analysis_few_shot(
        self,
        text: str,
        examples: Optional[List[FewShotExample]] = None,
        use_dynamic: bool = False
    ) -> str:
        """
        Perform sentiment analysis using few-shot learning.
        
        Args:
            text: Text to analyze
            examples: Optional custom examples
            use_dynamic: Whether to use dynamic selection
            
        Returns:
            Sentiment classification
        """
        # Default examples if none provided
        if examples is None:
            examples = [
                FewShotExample(
                    input="This movie was absolutely fantastic! I loved every minute.",
                    output="Positive"
                ),
                FewShotExample(
                    input="The product broke after one day. Terrible quality.",
                    output="Negative"
                ),
                FewShotExample(
                    input="It's okay, nothing special but not bad either.",
                    output="Neutral"
                ),
                FewShotExample(
                    input="I'm thrilled with this purchase! Exceeded expectations.",
                    output="Positive"
                ),
                FewShotExample(
                    input="Complete waste of money. Very disappointed.",
                    output="Negative"
                )
            ]
        
        task_instruction = """You are a sentiment analyzer. Classify the sentiment of the given text as Positive, Negative, or Neutral.
Look at the examples to understand the task."""
        
        # Create chain based on selection strategy
        if use_dynamic:
            chain = self.create_dynamic_few_shot_chain(
                examples,
                task_instruction,
                k=3
            )
        else:
            # Use first 3 examples for static
            chain = self.create_static_few_shot_chain(
                examples[:3],
                task_instruction
            )
        
        result = chain.invoke({"input": text})
        return result
    
    def entity_extraction_few_shot(
        self,
        text: str,
        examples: Optional[List[FewShotExample]] = None
    ) -> str:
        """
        Extract entities using few-shot learning.
        
        Args:
            text: Text to extract entities from
            examples: Optional custom examples
            
        Returns:
            Extracted entities in JSON format
        """
        # Default examples
        if examples is None:
            examples = [
                FewShotExample(
                    input="Apple Inc. announced a new iPhone model in Cupertino on September 12th.",
                    output='{"organization": "Apple Inc.", "product": "iPhone", "location": "Cupertino", "date": "September 12th"}'
                ),
                FewShotExample(
                    input="Elon Musk tweeted about Tesla's production numbers from the Fremont factory.",
                    output='{"person": "Elon Musk", "organization": "Tesla", "location": "Fremont factory"}'
                ),
                FewShotExample(
                    input="The meeting at Microsoft headquarters in Seattle was attended by Satya Nadella.",
                    output='{"organization": "Microsoft", "location": "Seattle", "person": "Satya Nadella"}'
                )
            ]
        
        task_instruction = """Extract entities from the text and format them as JSON.
Include: person, organization, location, date, product (if present).
Follow the format shown in the examples."""
        
        chain = self.create_static_few_shot_chain(
            examples,
            task_instruction
        )
        
        result = chain.invoke({"input": text})
        return result
    
    def code_generation_few_shot(
        self,
        description: str,
        examples: Optional[List[FewShotExample]] = None
    ) -> str:
        """
        Generate code using few-shot learning.
        
        Args:
            description: Description of desired code
            examples: Optional custom examples
            
        Returns:
            Generated code
        """
        # Default examples
        if examples is None:
            examples = [
                FewShotExample(
                    input="Create a function that returns the sum of two numbers",
                    output="""def add_numbers(a, b):
    \"\"\"Return the sum of two numbers.\"\"\"
    return a + b"""
                ),
                FewShotExample(
                    input="Create a function that checks if a string is a palindrome",
                    output="""def is_palindrome(s):
    \"\"\"Check if a string is a palindrome.\"\"\"
    return s == s[::-1]"""
                ),
                FewShotExample(
                    input="Create a function that finds the maximum value in a list",
                    output="""def find_max(lst):
    \"\"\"Find the maximum value in a list.\"\"\"
    if not lst:
        return None
    return max(lst)"""
                )
            ]
        
        task_instruction = """Generate Python code based on the description.
Follow the style and format shown in the examples.
Include docstrings and proper formatting."""
        
        chain = self.create_static_few_shot_chain(
            examples,
            task_instruction,
            input_variable="input"
        )
        
        result = chain.invoke({"input": description})
        return result
    
    def style_transfer_few_shot(
        self,
        text: str,
        target_style: str,
        examples: Optional[List[FewShotExample]] = None
    ) -> str:
        """
        Transfer text to a specific style using few-shot learning.
        
        Args:
            text: Text to transform
            target_style: Target style (e.g., "formal", "casual", "poetic")
            examples: Optional custom examples
            
        Returns:
            Transformed text
        """
        # Style-specific examples
        style_examples = {
            "formal": [
                FewShotExample(
                    input="Hey, can you help me out with this?",
                    output="I would appreciate your assistance with this matter."
                ),
                FewShotExample(
                    input="That's totally wrong!",
                    output="I respectfully disagree with that assessment."
                ),
                FewShotExample(
                    input="Thanks a lot for everything!",
                    output="I extend my sincere gratitude for your comprehensive support."
                )
            ],
            "casual": [
                FewShotExample(
                    input="I would like to request your assistance.",
                    output="Hey, could you help me out?"
                ),
                FewShotExample(
                    input="This is highly problematic.",
                    output="This is a real problem."
                ),
                FewShotExample(
                    input="I appreciate your effort.",
                    output="Thanks for trying!"
                )
            ],
            "poetic": [
                FewShotExample(
                    input="The sun set over the mountains.",
                    output="Golden rays bid farewell, as peaks embrace the fading light."
                ),
                FewShotExample(
                    input="Rain started falling.",
                    output="Heaven's tears descended, kissing the earth below."
                ),
                FewShotExample(
                    input="The bird flew away.",
                    output="On wings of freedom, it soared beyond the horizon's edge."
                )
            ]
        }
        
        examples = examples or style_examples.get(target_style.lower(), [])
        
        if not examples:
            return "Style not supported. Available: formal, casual, poetic"
        
        task_instruction = f"""Transform the text into {target_style} style.
Follow the transformation pattern shown in the examples."""
        
        chain = self.create_static_few_shot_chain(
            examples,
            task_instruction
        )
        
        result = chain.invoke({"input": text})
        return result
    
    def compare_zero_shot_vs_few_shot(
        self,
        text: str,
        task: str = "sentiment"
    ) -> Dict[str, str]:
        """
        Compare zero-shot vs few-shot performance.
        
        Args:
            text: Input text
            task: Task to perform
            
        Returns:
            Dictionary with both results
        """
        # Zero-shot prompt
        zero_shot_prompt = ChatPromptTemplate.from_messages([
            ("system", "Classify the sentiment as Positive, Negative, or Neutral."),
            ("human", "{input}")
        ])
        zero_shot_chain = zero_shot_prompt | self.llm | StrOutputParser()
        zero_shot_result = zero_shot_chain.invoke({"input": text})
        
        # Few-shot result
        few_shot_result = self.sentiment_analysis_few_shot(text)
        
        return {
            "zero_shot": zero_shot_result,
            "few_shot": few_shot_result
        }


def demonstrate_few_shot_learning():
    """Demonstrate few-shot learning patterns"""
    print("=" * 80)
    print("FEW-SHOT LEARNING PATTERN DEMONSTRATION")
    print("=" * 80)
    
    agent = FewShotLearningAgent()
    
    # Example 1: Sentiment Analysis
    print("\n" + "=" * 80)
    print("EXAMPLE 1: Sentiment Analysis with Few-Shot Learning")
    print("=" * 80)
    
    test_texts = [
        "This restaurant exceeded all my expectations! Amazing food!",
        "The service was slow and the food was cold. Not going back.",
        "It was fine. Nothing special but acceptable."
    ]
    
    for text in test_texts:
        print(f"\nText: {text}")
        sentiment = agent.sentiment_analysis_few_shot(text)
        print(f"Sentiment: {sentiment}")
    
    # Example 2: Entity Extraction
    print("\n" + "=" * 80)
    print("EXAMPLE 2: Entity Extraction with Few-Shot Learning")
    print("=" * 80)
    
    test_text = "Jeff Bezos announced Amazon's expansion into the healthcare sector at a conference in Las Vegas last Monday."
    print(f"\nText: {test_text}")
    entities = agent.entity_extraction_few_shot(test_text)
    print(f"Entities: {entities}")
    
    # Example 3: Code Generation
    print("\n" + "=" * 80)
    print("EXAMPLE 3: Code Generation with Few-Shot Learning")
    print("=" * 80)
    
    code_requests = [
        "Create a function that calculates factorial of a number",
        "Create a function that reverses a string"
    ]
    
    for request in code_requests:
        print(f"\nRequest: {request}")
        code = agent.code_generation_few_shot(request)
        print(f"Generated Code:\n{code}")
    
    # Example 4: Style Transfer
    print("\n" + "=" * 80)
    print("EXAMPLE 4: Style Transfer with Few-Shot Learning")
    print("=" * 80)
    
    original_text = "I need your help with this problem."
    styles = ["formal", "casual", "poetic"]
    
    print(f"\nOriginal: {original_text}")
    for style in styles:
        transformed = agent.style_transfer_few_shot(original_text, style)
        print(f"{style.title()} style: {transformed}")
    
    # Example 5: Zero-Shot vs Few-Shot Comparison
    print("\n" + "=" * 80)
    print("EXAMPLE 5: Zero-Shot vs Few-Shot Comparison")
    print("=" * 80)
    
    ambiguous_text = "The movie was interesting and thought-provoking, though quite challenging to follow."
    print(f"\nAmbiguous Text: {ambiguous_text}")
    
    comparison = agent.compare_zero_shot_vs_few_shot(ambiguous_text)
    print(f"\nZero-Shot Result: {comparison['zero_shot']}")
    print(f"Few-Shot Result: {comparison['few_shot']}")
    print("\nNote: Few-shot learning often provides more consistent and format-compliant outputs")
    
    # Example 6: Dynamic Example Selection
    print("\n" + "=" * 80)
    print("EXAMPLE 6: Dynamic Example Selection (Similarity-Based)")
    print("=" * 80)
    
    test_text = "I'm so excited about this purchase! Best decision ever!"
    print(f"\nText: {test_text}")
    print("\nStatic Selection:")
    static_result = agent.sentiment_analysis_few_shot(test_text, use_dynamic=False)
    print(f"Result: {static_result}")
    
    print("\nDynamic Selection (picks most similar examples):")
    dynamic_result = agent.sentiment_analysis_few_shot(test_text, use_dynamic=True)
    print(f"Result: {dynamic_result}")
    
    # Summary
    print("\n" + "=" * 80)
    print("FEW-SHOT LEARNING SUMMARY")
    print("=" * 80)
    print("""
Few-Shot Learning Pattern Benefits:
1. Improved Accuracy: Examples guide the model to better performance
2. Consistency: Output format matches examples consistently
3. Task Clarity: Examples disambiguate task requirements
4. Style Control: Examples demonstrate desired tone and style
5. No Fine-Tuning: Achieves good results without model retraining

Key Techniques Demonstrated:
1. Static Few-Shot: Fixed examples for consistent tasks
2. Dynamic Few-Shot: Similarity-based example selection
3. Task-Specific Examples: Domain-appropriate demonstrations
4. Style Transfer: Teaching specific writing styles
5. Format Control: Ensuring structured outputs

Best Practices:
1. Use 3-5 examples for most tasks (sweet spot)
2. Ensure examples are diverse and representative
3. Order examples from simple to complex
4. Include edge cases in examples
5. Keep examples concise to save tokens
6. Use dynamic selection for varied inputs
7. Validate examples are correct and high-quality
8. Monitor token usage and costs

When to Use Few-Shot vs Zero-Shot:
- Use few-shot when: output format is specific, task is ambiguous, consistency matters
- Use zero-shot when: task is simple, tokens are limited, speed is critical
- Use dynamic selection when: inputs are diverse, quality is paramount

Production Considerations:
- Cache example embeddings for performance
- Monitor token usage (examples increase costs)
- A/B test different example sets
- Version control example datasets
- Update examples based on failure cases
- Balance example count with quality needs
""")
    
    print("\n" + "=" * 80)
    print("Pattern 083 (Few-Shot Learning) demonstration complete!")
    print("=" * 80)


if __name__ == "__main__":
    demonstrate_few_shot_learning()

    Agentic Design Pattern: Few Shot Learning

    Pattern 083: Few Shot Learning – Line-by-Line Explanation

    Overview

    This document provides a detailed line-by-line explanation of the Few Shot Learning pattern implementation.

    Pattern Description

    Pattern 083: Few-Shot Learning Pattern

    Description: The Few-Shot Learning pattern uses example-based prompting to teach the LLM how to perform tasks through demonstrations. Instead of relying on zero-shot capabilities, this pattern provides the model with a few examples of input-output pairs that illustrate the desired behavior. This approach leverages the LLM’s in-context learning abilities to generalize from examples.

    Few-shot learning is one of the most fundamental prompt engineering techniques
and is particularly effective when:
- The task format is non-standard or complex
- Consistent output format is required
- Specific style or tone needs to be maintained
- Domain-specific knowledge needs to be demonstrated
- Edge cases need to be handled in specific ways

    Components: 1. Example Selection – Static examples (fixed set) – Dynamic examples (selected based on similarity) – Diverse examples (covering different scenarios) – Representative examples (typical cases)

    2. Example Formatting
   - Clear input-output structure
   - Consistent formatting across examples
   - Annotations and explanations (when needed)
   - Order considerations (easy to hard, chronological)

3. Example Storage
   - In-prompt storage (small example sets)
   - Vector database storage (large example sets)
   - Retrieval mechanisms (similarity-based)
   - Example caching strategies

4. Generalization Strategy
   - Pattern recognition from examples
   - Analogical reasoning
   - Template extraction
   - Feature learning

    Use Cases: 1. Text Classification – Sentiment analysis with custom categories – Intent recognition for domain-specific applications – Content moderation with nuanced rules – Topic categorization with specific taxonomies

    2. Data Extraction
   - Structured information extraction from unstructured text
   - Entity recognition with custom entity types
   - Relationship extraction between entities
   - Attribute extraction with specific formats

3. Code Generation
   - API usage examples for specific libraries
   - Code style consistency (following project conventions)
   - Design pattern implementation
   - Test case generation

4. Creative Writing
   - Style mimicry (writing in specific author's style)
   - Format consistency (poems, stories, articles)
   - Tone control (formal, casual, humorous)
   - Genre-specific writing (sci-fi, mystery, romance)

5. Translation and Transformation
   - Domain-specific translation (legal, medical, technical)
   - Format conversion (JSON to YAML, CSV to markdown)
   - Data normalization (various formats to standard)
   - Code refactoring patterns

    LangChain Implementation: LangChain provides excellent support for few-shot learning through: – FewShotChatMessagePromptTemplate for chat models – FewShotPromptTemplate for completion models – Example selectors (semantic similarity, length-based, max marginal relevance) – Vector store-backed example selection – Custom example formatting

    Key Features: 1. Static Few-Shot Learning – Predefined set of examples – Fast and deterministic – Good for stable tasks – No retrieval overhead

    2. Dynamic Few-Shot Learning
   - Examples selected based on input similarity
   - More adaptive to diverse inputs
   - Requires vector database
   - Better generalization

3. Example Quality Management
   - Diverse examples covering edge cases
   - Clear and unambiguous examples
   - Representative of target distribution
   - Regularly updated based on performance

4. Performance Optimization
   - Token budget management (limiting example count)
   - Example compression (removing redundancy)
   - Caching frequently used example sets
   - Lazy loading for large example sets

    Best Practices: 1. Example Selection – Use 3-5 examples for most tasks (sweet spot) – Include diverse examples covering different patterns – Order examples from simple to complex – Ensure examples are correct and high-quality

    2. Example Formatting
   - Use consistent structure across all examples
   - Clearly separate input from output
   - Add brief explanations for complex cases
   - Use delimiters to avoid confusion

3. Dynamic Selection
   - Use semantic similarity for input-dependent selection
   - Balance diversity and relevance in selection
   - Cache embeddings for better performance
   - Set appropriate k value (number of examples)

4. Iteration and Improvement
   - Start with few examples, add more if needed
   - A/B test different example sets
   - Monitor performance across different inputs
   - Update examples based on failure cases

    Trade-offs: Advantages: – Significantly improves task performance vs zero-shot – More reliable than instructions alone – Enables consistent output formatting – Works well for complex or ambiguous tasks – Reduces need for fine-tuning

    Disadvantages:
- Increases token usage (more expensive)
- May not scale to tasks requiring many examples
- Examples can bias the model inappropriately
- Requires careful example curation
- Dynamic selection adds latency and complexity

    Production Considerations: 1. Token Budget Management – Monitor tokens used by examples – Balance example count with context window – Use example compression when possible – Consider model’s context limit

    2. Example Maintenance
   - Version control for example sets
   - Regular review and updates
   - Performance tracking per example
   - A/B testing of example variations

3. Scalability
   - Cache example embeddings
   - Use efficient vector stores
   - Batch process when possible
   - Consider example pruning strategies

4. Quality Assurance
   - Validate examples are correct
   - Test examples independently
   - Monitor output quality metrics
   - Handle example selection failures gracefully

5. Cost Optimization
   - Balance example count with quality
   - Use dynamic selection judiciously
   - Cache common example sets
   - Monitor cost per query

    File Statistics

    • Total Lines: 768
    • Classes: 4
    • Functions: 1
    • Constants: 0

    Module Documentation

    """
Pattern 083: Few-Shot Learning Pattern

Description:
    The Few-Shot Learning pattern uses example-based prompting to teach the LLM
    how to perform tasks through demonstrations. Instead of relying on zero-shot
    capabilities, this pattern provides the model with a few examples of input-output
    pairs that illustrate the desired behavior. This approach leverages the LLM's
    in-context learning abilities to generalize from examples.

    Few-shot learning is one of the most fundamental prompt engineering techniques
    and is particularly effective when:
    - The task format is non-standard or complex
    - Consistent output format is required
    - Specific style or tone needs to be maintained
    - Domain-specific knowledge needs to be demonstrated
    - Edge cases need to be handled in specific ways

Components:
    1. Example Selection
       - Static examples (fixed set)
       - Dynamic examples (selected based on similarity)
       - Diverse examples (covering different scenarios)
       - Representative examples (typical cases)

    2. Example Formatting
       - Clear input-output structure
       - Consistent formatting across examples
       - Annotations and explanations (when needed)
       - Order considerations (easy to hard, chronological)

    3. Example Storage
       - In-prompt storage (small example sets)
       - Vector database storage (large example sets)
       - Retrieval mechanisms (similarity-based)
       - Example caching strategies

    4. Generalization Strategy
       - Pattern recognition from examples
       - Analogical reasoning
       - Template extraction
       - Feature learning

Use Cases:
    1. Text Classification
       - Sentiment analysis with custom categories
       - Intent recognition for domain-specific applications
       - Content moderation with nuanced rules
       - Topic categorization with specific taxonomies

    2. Data Extraction
       - Structured information extraction from unstructured text
       - Entity recognition with custom entity types
       - Relationship extraction between entities
       - Attribute extraction with specific formats

    3. Code Generation
       - API usage examples for specific libraries
       - Code style consistency (following project conventions)
       - Design pattern implementation
       - Test case generation

    4. Creative Writing
       - Style mimicry (writing in specific author's style)
       - Format consistency (poems, stories, articles)
       - Tone control (formal, casual, humorous)
       - Genre-specific writing (sci-fi, mystery, romance)

    5. Translation and Transformation
       - Domain-specific translation (legal, medical, technical)
       - Format conversion (JSON to YAML, CSV to markdown)
       - Data normalization (various formats to standard)
       - Code refactoring patterns

LangChain Implementation:
    LangChain provides excellent support for few-shot learning through:
    - FewShotChatMessagePromptTemplate for chat models
    - FewShotPromptTemplate for completion models
    - Example selectors (semantic similarity, length-based, max marginal relevance)
    - Vector store-backed example selection
    - Custom example formatting

Key Features:
    1. Static Few-Shot Learning
       - Predefined set of examples
       - Fast and deterministic
       - Good for stable tasks
       - No retrieval overhead

    2. Dynamic Few-Shot Learning
       - Examples selected based on input similarity
       - More adaptive to diverse inputs
       - Requires vector database
       - Better generalization

    3. Example Quality Management
       - Diverse examples covering edge cases
       - Clear and unambiguous examples
       - Representative of target distribution
       - Regularly updated based on performance

    4. Performance Optimization
       - Token budget management (limiting example count)
       - Example compression (removing redundancy)
       - Caching frequently used example sets
       - Lazy loading for large example sets

Best Practices:
    1. Example Selection
       - Use 3-5 examples for most tasks (sweet spot)
       - Include diverse examples covering different patterns
       - Order examples from simple to complex
       - Ensure examples are correct and high-quality

    2. Example Formatting
       - Use consistent structure across all examples
       - Clearly separate input from output
       - Add brief explanations for complex cases
       - Use delimiters to avoid confusion

    3. Dynamic Selection
       - Use semantic similarity for input-dependent selection
       - Balance diversity and relevance in selection
       - Cache embeddings for better performance
       - Set appropriate k value (number of examples)

    4. Iteration and Improvement
       - Start with few examples, add more if needed
       - A/B test different example sets
       - Monitor performance across different inputs
       - Update examples based on failure cases

Trade-offs:
    Advantages:
    - Significantly improves task performance vs zero-shot
    - More reliable than instructions alone
    - Enables consistent output formatting
    - Works well for complex or ambiguous tasks
    - Reduces need for fine-tuning

    Disadvantages:
    - Increases token usage (more expensive)
    - May not scale to tasks requiring many examples
    - Examples can bias the model inappropriately
    - Requires careful example curation
    - Dynamic selection adds latency and complexity

Production Considerations:
    1. Token Budget Management
       - Monitor tokens used by examples
       - Balance example count with context window
       - Use example compression when possible
       - Consider model's context limit

    2. Example Maintenance
       - Version control for example sets
       - Regular review and updates
       - Performance tracking per example
       - A/B testing of example variations

    3. Scalability
       - Cache example embeddings
       - Use efficient vector stores
       - Batch process when possible
       - Consider example pruning strategies

    4. Quality Assurance
       - Validate examples are correct
       - Test examples independently
       - Monitor output quality metrics
       - Handle example selection failures gracefully

    5. Cost Optimization
       - Balance example count with quality
       - Use dynamic selection judiciously
       - Cache common example sets
       - Monitor cost per query
"""

    Purpose: This module implements the Few Shot Learning pattern, providing a structured approach to agent design and implementation.

    Import Statements

    Standard Library Imports

    import dataclasses.dataclass
import enum.Enum
import langchain_core.example_selectors.MaxMarginalRelevanceExampleSelector
import langchain_core.example_selectors.SemanticSimilarityExampleSelector
import langchain_core.output_parsers.StrOutputParser
import langchain_core.prompts.ChatPromptTemplate
import langchain_core.prompts.FewShotChatMessagePromptTemplate
import langchain_core.prompts.PromptTemplate
import os
import typing.Any
import typing.Dict
import typing.List
import typing.Optional

    Purpose: Core Python functionality: – dataclasses.dataclass: Simplified class definitions with @dataclass – enum.Enum: Enumeration support – langchain_core.example_selectors.MaxMarginalRelevanceExampleSelector: Regular expression operations – langchain_core.example_selectors.SemanticSimilarityExampleSelector: Regular expression operations – langchain_core.output_parsers.StrOutputParser: Regular expression operations – langchain_core.prompts.ChatPromptTemplate: Regular expression operations – langchain_core.prompts.FewShotChatMessagePromptTemplate: Regular expression operations – langchain_core.prompts.PromptTemplate: Regular expression operations – os: Environment variable access and system operations – typing.Any: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Dict: Type hints for better code documentation (List, Dict, Any, Optional) – typing.List: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Optional: Type hints for better code documentation (List, Dict, Any, Optional)

    LangChain Imports

    from langchain_chroma.Chroma
from langchain_openai.ChatOpenAI
from langchain_openai.OpenAIEmbeddings

    Purpose: LangChain framework components: – LLM Interface: Wrapper for OpenAI models – Chains: Composable building blocks for complex workflows

    Other Imports

    from dotenv.load_dotenv

    Purpose: Additional dependencies for specialized functionality.

    Classes

    Class: ExampleSelectionStrategy

    Lines 202-206

    Inherits from: Enum

    Purpose:

    Strategy for selecting examples

    Class: FewShotExample

    Lines 210-215

    Purpose:

    Represents a single few-shot example

    Class: FewShotConfig

    Lines 219-225

    Purpose:

    Configuration for few-shot learning

    Class: FewShotLearningAgent

    Lines 228-625

    Purpose:

    Agent that uses few-shot learning to teach LLM through examples.

This agent demonstrates various few-shot learning approaches:
1. Static few-shot (fixed examples)
2. Dynamic few-shot (similarity-based selection)
3. Task-specific few-shot applications

    Methods:

    __init__(self, config)

    Lines 238-250

    Purpose:

    Initialize few-shot learning agent.

Args:
    config: Configuration for few-shot learning

    Parameters: – config: Config

    create_static_few_shot_chain(self, examples, task_instruction, input_variable)

    Lines 252-297

    Purpose:

    Create a chain with static few-shot examples.

Args:
    examples: List of few-shot examples
    task_instruction: Instruction for the task
    input_variable: Name of input variable

Returns:
    Runnable chain with few-shot prompting

    Parameters: – examples: Examples – task_instruction: Task instruction – input_variable: Input data

    create_dynamic_few_shot_chain(self, examples, task_instruction, input_variable, k)

    Lines 299-356

    Purpose:

    Create a chain with dynamic example selection based on similarity.

Args:
    examples: Pool of examples to select from
    task_instruction: Instruction for the task
    input_variable: Name of input variable
    k: Number of examples to select

Returns:
    Runnable chain with dynamic few-shot prompting

    Parameters: – examples: Examples – task_instruction: Task instruction – input_variable: Input data – k: K

    sentiment_analysis_few_shot(self, text, examples, use_dynamic)

    Lines 358-418

    Purpose:

    Perform sentiment analysis using few-shot learning.

Args:
    text: Text to analyze
    examples: Optional custom examples
    use_dynamic: Whether to use dynamic selection

Returns:
    Sentiment classification

    Parameters: – text: Text – examples: Examples – use_dynamic: Use dynamic

    Returns: str

    entity_extraction_few_shot(self, text, examples)

    Lines 420-462

    Purpose:

    Extract entities using few-shot learning.

Args:
    text: Text to extract entities from
    examples: Optional custom examples

Returns:
    Extracted entities in JSON format

    Parameters: – text: Text – examples: Examples

    Returns: str

    code_generation_few_shot(self, description, examples)

    Lines 464-515

    Purpose:

    Generate code using few-shot learning.

Args:
    description: Description of desired code
    examples: Optional custom examples

Returns:
    Generated code

    Parameters: – description: Description – examples: Examples

    Returns: str

    style_transfer_few_shot(self, text, target_style, examples)

    Lines 517-594

    Purpose:

    Transfer text to a specific style using few-shot learning.

Args:
    text: Text to transform
    target_style: Target style (e.g., "formal", "casual", "poetic")
    examples: Optional custom examples

Returns:
    Transformed text

    Parameters: – text: Text – target_style: Target style – examples: Examples

    Returns: str

    compare_zero_shot_vs_few_shot(self, text, task)

    Lines 596-625

    Purpose:

    Compare zero-shot vs few-shot performance.

Args:
    text: Input text
    task: Task to perform

Returns:
    Dictionary with both results

    Parameters: – text: Text – task: Task

    Returns: <ast.Subscript object at 0x000001A06D853AD0>

    Functions

    demonstrate_few_shot_learning()

    Lines 628-763

    Purpose:

    Demonstrate few-shot learning patterns

    Key Concepts Summary

    Pattern Overview

    The Few Shot Learning pattern provides a structured approach to building AI agents with specific capabilities and behaviors. This implementation leverages LangChain’s composable components to create a robust and maintainable solution.

    Core Components

    1. Agent Class: Main implementation of the pattern logic
    2. Prompt Engineering: Carefully crafted prompts to guide LLM behavior
    3. Tool Integration: Optional tools for extended functionality
    4. Memory Management: State handling across interactions
    5. Output Parsing: Structured response processing

    When to Use

    ✅ Good for: – Scenarios requiring this specific pattern’s strengths – Applications needing structured agent behavior – Systems where explainability is important – Production environments requiring reliability

    ❌ Not ideal for: – Simple single-prompt tasks – Scenarios requiring different pattern characteristics – Resource-constrained environments (if computationally intensive)

    Best Practices Demonstrated

    1. Environment Configuration

    • ✅ Use .env files for API keys
    • ✅ Never hardcode credentials
    • ✅ Validate environment variables on startup

    2. Error Handling

    • ✅ Graceful degradation on failures
    • ✅ Informative error messages
    • ✅ Logging for debugging

    3. Prompt Engineering

    • ✅ Clear, specific instructions
    • ✅ Structured output formats
    • ✅ Few-shot examples when beneficial

    4. Code Organization

    • ✅ Separation of concerns (class-based design)
    • ✅ Type hints for clarity
    • ✅ Docstrings for documentation

    5. Performance Optimization

    • ✅ Appropriate temperature settings
    • ✅ Token budget management
    • ✅ Caching where applicable

    LangChain Components Used

    • ChatOpenAI: OpenAI model interface
    • Chains: Sequential component composition

    Why LangChain? – Composable abstractions for LLM applications – Rich ecosystem of integrations – Production-ready patterns – Active community and development

    Execution Flow

    1. Initialize agent with configuration
   ↓
2. Load environment and dependencies
   ↓
3. Prepare prompt with input data
   ↓
4. Invoke LLM through LangChain
   ↓
5. Parse and validate response
   ↓
6. Return formatted result

    Key Decision Points: – Model selection (temperature, model type) – Prompt structure (few-shot vs zero-shot) – Error handling strategy – Output format validation

    Performance Considerations

    Computational Cost

    • LLM Calls: Primary cost factor (based on tokens)
    • Latency: Depends on model and input size
    • Memory: Minimal for most patterns

    Optimization Strategies

    1. Prompt Optimization: Minimize tokens while maintaining clarity
    2. Caching: Cache repeated queries/responses
    3. Model Selection: Use smaller models when appropriate
    4. Batch Processing: Group similar requests
    5. Streaming: For long responses, stream tokens

    Cost Estimates (Approximate)

    • GPT-3.5-turbo: ~$0.002 per request
    • GPT-4: ~$0.03 per request
    • GPT-4-turbo: ~$0.01 per request

    Actual costs vary based on input/output token counts

    Common Pitfalls to Avoid

    1. Prompt Design Issues

    ❌ Vague or ambiguous instructions
    ✅ Clear, specific, testable prompts

    2. Error Handling

    ❌ Ignoring API errors or rate limits
    ✅ Implement retry logic and graceful degradation

    3. Token Management

    ❌ Exceeding model token limits
    ✅ Truncate inputs, chunk processing

    4. Security

    ❌ Exposing API keys in code
    ✅ Use environment variables

    5. Testing

    ❌ Only testing happy paths
    ✅ Test edge cases, failures, malformed inputs

    6. Cost Management

    ❌ Unmonitored API usage
    ✅ Implement budgets, alerts, and monitoring

    Further Improvements

    Production Enhancements

    1. Monitoring: Add logging, metrics, tracing
    2. Validation: Input/output schema validation
    3. Rate Limiting: Implement request throttling
    4. Caching: Redis or in-memory cache for responses
    5. Async Support: Non-blocking operations for scalability

    Feature Extensions

    1. Multi-Model Support: Switch between providers
    2. Fallback Models: Graceful degradation strategy
    3. Custom Tools: Domain-specific function calling
    4. Advanced Memory: Vector stores for long-term memory
    5. Evaluation: Automated testing and benchmarking

    Code Quality

    1. Testing: Unit tests, integration tests
    2. Documentation: API docs, usage examples
    3. Type Safety: Stricter type hints
    4. Configuration: Externalized config files

    Related Patterns

    • Pattern 082
    • Pattern 084

    Pattern Families: – Reasoning Patterns (001-011) – Multi-Agent Patterns (012-019) – Tool Use Patterns (020-025) – Memory Patterns (026-031) – And many more…

    Summary

    The Few Shot Learning pattern provides a powerful approach to building AI agents with specific capabilities. This implementation demonstrates:

    ✅ Key Strengths: – Clean, maintainable code structure – LangChain integration for composability – Type hints and documentation – Production-ready patterns

    🎯 Best Use Cases: – Applications requiring this pattern’s characteristics – Systems needing structured agent behavior – Production environments requiring reliability

    📚 Learning Resources: – LangChain Documentation: https://python.langchain.com/ – Pattern Documentation: See related patterns – OpenAI API Reference: https://platform.openai.com/docs/

    File: 083_few_shot_learning.py
    Total Lines: 768
    Pattern Category: AI Agent Design Pattern #083

    Generated from actual implementation code
    For educational and reference purposes

  • Agentic Design Pattern: Serverless Agent

    
    """
Pattern 082: Serverless Agent

Description:
    A Serverless Agent runs on serverless computing platforms (AWS Lambda,
    Azure Functions, Google Cloud Functions) where infrastructure is fully
    managed by the cloud provider. This pattern enables event-driven, auto-scaling
    AI agents that only consume resources when invoked, optimizing costs and
    operational overhead.
    
    The agent is designed for stateless execution, cold start optimization,
    event-driven triggers, and seamless scaling from zero to thousands of
    concurrent executions without manual intervention.

Components:
    1. Function Handler: Entry point for serverless execution
    2. Event Processor: Processes various event types
    3. State Manager: External state storage (S3, DynamoDB)
    4. Cold Start Optimizer: Minimizes initialization time
    5. Environment Manager: Configuration and secrets
    6. Response Builder: Formats function responses
    7. Error Handler: Graceful error handling
    8. Metrics Collector: CloudWatch/monitoring integration

Key Features:
    - Stateless execution model
    - Event-driven triggers
    - Auto-scaling from zero
    - Pay-per-execution pricing
    - No server management
    - Built-in fault tolerance
    - Managed infrastructure
    - Cold start optimization
    - Multiple trigger sources
    - Seamless integration with cloud services

Use Cases:
    - API endpoints for AI services
    - Event-driven AI processing
    - Webhook handlers
    - Scheduled AI tasks
    - Image/document processing
    - Real-time data transformation
    - Chatbot backends
    - ETL pipelines
    - IoT data processing
    - Microservices backends

LangChain Implementation:
    Simulates AWS Lambda-style function handlers with event processing,
    stateless execution, and external state management patterns.
"""

import os
import time
import json
import uuid
from typing import List, Dict, Any, Optional, Callable
from dataclasses import dataclass, field
from enum import Enum
from datetime import datetime
from collections import defaultdict
from dotenv import load_dotenv

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_openai import ChatOpenAI

load_dotenv()


class TriggerType(Enum):
    """Types of serverless triggers"""
    HTTP_API = "http_api"
    S3_EVENT = "s3_event"
    DYNAMODB_STREAM = "dynamodb_stream"
    SNS_MESSAGE = "sns_message"
    SQS_QUEUE = "sqs_queue"
    SCHEDULED = "scheduled"
    WEBHOOK = "webhook"
    IOT_EVENT = "iot_event"


class ExecutionStatus(Enum):
    """Status of function execution"""
    COLD_START = "cold_start"
    WARM_START = "warm_start"
    RUNNING = "running"
    COMPLETED = "completed"
    FAILED = "failed"
    TIMEOUT = "timeout"


@dataclass
class ServerlessEvent:
    """Event that triggers serverless function"""
    event_id: str
    trigger_type: TriggerType
    timestamp: datetime
    data: Dict[str, Any]
    headers: Dict[str, str] = field(default_factory=dict)
    metadata: Dict[str, Any] = field(default_factory=dict)


@dataclass
class ServerlessContext:
    """Execution context for serverless function"""
    request_id: str
    function_name: str
    function_version: str
    memory_limit_mb: int
    timeout_seconds: int
    remaining_time_ms: int
    cold_start: bool = True
    invocation_count: int = 0


@dataclass
class ServerlessResponse:
    """Response from serverless function"""
    status_code: int
    body: Any
    headers: Dict[str, str] = field(default_factory=dict)
    error: Optional[str] = None
    execution_duration_ms: float = 0.0
    cold_start: bool = False
    memory_used_mb: float = 0.0


@dataclass
class FunctionMetrics:
    """Metrics for serverless function"""
    function_name: str
    invocations: int = 0
    errors: int = 0
    cold_starts: int = 0
    warm_starts: int = 0
    total_duration_ms: float = 0.0
    average_duration_ms: float = 0.0
    max_duration_ms: float = 0.0
    min_duration_ms: float = float('inf')


class ServerlessAgent:
    """
    Base serverless agent with stateless execution.
    
    Designed for serverless platforms with event-driven triggers,
    cold start optimization, and external state management.
    """
    
    # Class-level cache (simulates container reuse)
    _warm_instances: Dict[str, 'ServerlessAgent'] = {}
    _llm_cache: Optional[ChatOpenAI] = None
    
    def __init__(
        self,
        function_name: str,
        memory_limit_mb: int = 512,
        timeout_seconds: int = 30
    ):
        """
        Initialize serverless agent.
        
        Args:
            function_name: Name of the function
            memory_limit_mb: Memory limit
            timeout_seconds: Execution timeout
        """
        self.function_name = function_name
        self.memory_limit_mb = memory_limit_mb
        self.timeout_seconds = timeout_seconds
        
        self.invocation_count = 0
        self.metrics = FunctionMetrics(function_name=function_name)
        
        # Initialize LLM (reuse cached if available)
        if ServerlessAgent._llm_cache is None:
            ServerlessAgent._llm_cache = ChatOpenAI(
                model="gpt-3.5-turbo",
                temperature=0.7
            )
        
        self.llm = ServerlessAgent._llm_cache
    
    @classmethod
    def get_or_create(cls, function_name: str) -> 'ServerlessAgent':
        """
        Get warm instance or create new (simulates Lambda container reuse).
        
        Args:
            function_name: Function name
            
        Returns:
            ServerlessAgent instance
        """
        if function_name in cls._warm_instances:
            return cls._warm_instances[function_name]
        
        instance = cls(function_name)
        cls._warm_instances[function_name] = instance
        return instance
    
    def handler(
        self,
        event: ServerlessEvent,
        context: ServerlessContext
    ) -> ServerlessResponse:
        """
        Main handler function (entry point).
        
        Args:
            event: Triggering event
            context: Execution context
            
        Returns:
            ServerlessResponse
        """
        start_time = time.time()
        
        self.invocation_count += 1
        context.invocation_count = self.invocation_count
        
        # Track metrics
        self.metrics.invocations += 1
        if context.cold_start:
            self.metrics.cold_starts += 1
        else:
            self.metrics.warm_starts += 1
        
        try:
            # Process event based on trigger type
            result = self._process_event(event, context)
            
            execution_duration = (time.time() - start_time) * 1000
            
            # Update metrics
            self.metrics.total_duration_ms += execution_duration
            self.metrics.average_duration_ms = (
                self.metrics.total_duration_ms / self.metrics.invocations
            )
            self.metrics.max_duration_ms = max(
                self.metrics.max_duration_ms,
                execution_duration
            )
            self.metrics.min_duration_ms = min(
                self.metrics.min_duration_ms,
                execution_duration
            )
            
            return ServerlessResponse(
                status_code=200,
                body=result,
                headers={"Content-Type": "application/json"},
                execution_duration_ms=execution_duration,
                cold_start=context.cold_start,
                memory_used_mb=self.memory_limit_mb * 0.6  # Simulated
            )
            
        except Exception as e:
            self.metrics.errors += 1
            
            return ServerlessResponse(
                status_code=500,
                body=None,
                error=str(e),
                execution_duration_ms=(time.time() - start_time) * 1000,
                cold_start=context.cold_start
            )
    
    def _process_event(
        self,
        event: ServerlessEvent,
        context: ServerlessContext
    ) -> Dict[str, Any]:
        """
        Process event (to be overridden by subclasses).
        
        Args:
            event: Event to process
            context: Execution context
            
        Returns:
            Processing result
        """
        return {
            "message": "Event processed",
            "event_id": event.event_id,
            "function": self.function_name
        }


class TextProcessingFunction(ServerlessAgent):
    """Serverless function for text processing"""
    
    def __init__(self):
        super().__init__("text-processor", memory_limit_mb=512, timeout_seconds=30)
    
    def _process_event(
        self,
        event: ServerlessEvent,
        context: ServerlessContext
    ) -> Dict[str, Any]:
        """Process text from event"""
        data = event.data
        operation = data.get('operation', 'summarize')
        text = data.get('text', '')
        
        if operation == 'summarize':
            result = self._summarize(text)
        elif operation == 'sentiment':
            result = self._analyze_sentiment(text)
        elif operation == 'extract':
            result = self._extract_keywords(text)
        else:
            result = {'error': f'Unknown operation: {operation}'}
        
        return {
            'operation': operation,
            'result': result,
            'cold_start': context.cold_start,
            'invocation': context.invocation_count
        }
    
    def _summarize(self, text: str) -> str:
        """Summarize text"""
        prompt = ChatPromptTemplate.from_template(
            "Summarize this text in one sentence: {text}"
        )
        chain = prompt | self.llm | StrOutputParser()
        return chain.invoke({'text': text[:500]})
    
    def _analyze_sentiment(self, text: str) -> str:
        """Analyze sentiment"""
        prompt = ChatPromptTemplate.from_template(
            "What is the sentiment of this text? Answer in one word (positive/negative/neutral): {text}"
        )
        chain = prompt | self.llm | StrOutputParser()
        return chain.invoke({'text': text[:500]})
    
    def _extract_keywords(self, text: str) -> List[str]:
        """Extract keywords"""
        prompt = ChatPromptTemplate.from_template(
            "Extract 5 key words from this text (comma-separated): {text}"
        )
        chain = prompt | self.llm | StrOutputParser()
        result = chain.invoke({'text': text[:500]})
        return [k.strip() for k in result.split(',')[:5]]


class WebhookFunction(ServerlessAgent):
    """Serverless function for webhook processing"""
    
    def __init__(self):
        super().__init__("webhook-handler", memory_limit_mb=256, timeout_seconds=15)
    
    def _process_event(
        self,
        event: ServerlessEvent,
        context: ServerlessContext
    ) -> Dict[str, Any]:
        """Process webhook event"""
        data = event.data
        webhook_type = data.get('type', 'generic')
        payload = data.get('payload', {})
        
        # Process webhook
        response = {
            'webhook_type': webhook_type,
            'processed_at': datetime.now().isoformat(),
            'status': 'received',
            'payload_size': len(str(payload))
        }
        
        return response


class ScheduledFunction(ServerlessAgent):
    """Serverless function for scheduled tasks"""
    
    def __init__(self):
        super().__init__("scheduled-task", memory_limit_mb=1024, timeout_seconds=60)
    
    def _process_event(
        self,
        event: ServerlessEvent,
        context: ServerlessContext
    ) -> Dict[str, Any]:
        """Process scheduled event"""
        task_type = event.data.get('task', 'report')
        
        if task_type == 'report':
            result = self._generate_report()
        elif task_type == 'cleanup':
            result = self._cleanup_data()
        elif task_type == 'sync':
            result = self._sync_data()
        else:
            result = {'error': 'Unknown task type'}
        
        return {
            'task': task_type,
            'result': result,
            'scheduled_time': event.timestamp.isoformat()
        }
    
    def _generate_report(self) -> Dict[str, Any]:
        """Generate report"""
        return {
            'report_type': 'daily_summary',
            'generated_at': datetime.now().isoformat(),
            'items_processed': 100
        }
    
    def _cleanup_data(self) -> Dict[str, Any]:
        """Cleanup old data"""
        return {
            'items_deleted': 50,
            'space_freed_mb': 250
        }
    
    def _sync_data(self) -> Dict[str, Any]:
        """Sync data"""
        return {
            'items_synced': 75,
            'sync_duration_ms': 1500
        }


class ServerlessPlatform:
    """
    Simulates serverless platform (like AWS Lambda).
    
    Manages function lifecycle, invocations, and cold/warm starts.
    """
    
    def __init__(self):
        """Initialize platform"""
        self.functions: Dict[str, ServerlessAgent] = {}
        self.function_metrics: Dict[str, FunctionMetrics] = {}
    
    def deploy_function(self, function: ServerlessAgent) -> bool:
        """
        Deploy a serverless function.
        
        Args:
            function: Function to deploy
            
        Returns:
            True if deployed successfully
        """
        self.functions[function.function_name] = function
        self.function_metrics[function.function_name] = function.metrics
        return True
    
    def invoke(
        self,
        function_name: str,
        event: ServerlessEvent
    ) -> ServerlessResponse:
        """
        Invoke a serverless function.
        
        Args:
            function_name: Name of function to invoke
            event: Event to process
            
        Returns:
            ServerlessResponse
        """
        if function_name not in self.functions:
            return ServerlessResponse(
                status_code=404,
                body=None,
                error=f"Function not found: {function_name}"
            )
        
        function = self.functions[function_name]
        
        # Determine if cold start (simplified)
        is_cold_start = function.invocation_count == 0
        
        # Create context
        context = ServerlessContext(
            request_id=f"req_{uuid.uuid4().hex[:8]}",
            function_name=function_name,
            function_version="1.0",
            memory_limit_mb=function.memory_limit_mb,
            timeout_seconds=function.timeout_seconds,
            remaining_time_ms=function.timeout_seconds * 1000,
            cold_start=is_cold_start
        )
        
        # Invoke function
        return function.handler(event, context)
    
    def get_metrics(self, function_name: str) -> Optional[FunctionMetrics]:
        """
        Get function metrics.
        
        Args:
            function_name: Function name
            
        Returns:
            FunctionMetrics if found
        """
        return self.function_metrics.get(function_name)


def demonstrate_serverless_agent():
    """Demonstrate the serverless agent pattern"""
    print("=" * 80)
    print("SERVERLESS AGENT DEMONSTRATION")
    print("=" * 80)
    
    platform = ServerlessPlatform()
    
    # Demo 1: Deploy Serverless Functions
    print("\n" + "=" * 80)
    print("DEMO 1: Deploying Serverless Functions")
    print("=" * 80)
    
    functions = [
        TextProcessingFunction(),
        WebhookFunction(),
        ScheduledFunction()
    ]
    
    print("\nDeploying functions to serverless platform...")
    for func in functions:
        platform.deploy_function(func)
        print(f"  ✓ Deployed: {func.function_name}")
        print(f"    Memory: {func.memory_limit_mb}MB")
        print(f"    Timeout: {func.timeout_seconds}s")
    
    # Demo 2: Cold Start vs Warm Start
    print("\n" + "=" * 80)
    print("DEMO 2: Cold Start vs Warm Start")
    print("=" * 80)
    
    event = ServerlessEvent(
        event_id="evt_001",
        trigger_type=TriggerType.HTTP_API,
        timestamp=datetime.now(),
        data={
            'operation': 'summarize',
            'text': 'Serverless computing allows running code without managing servers.'
        }
    )
    
    print("\nFirst invocation (Cold Start):")
    response1 = platform.invoke("text-processor", event)
    print(f"  Status: {response1.status_code}")
    print(f"  Cold Start: {response1.cold_start}")
    print(f"  Duration: {response1.execution_duration_ms:.2f}ms")
    print(f"  Memory Used: {response1.memory_used_mb:.1f}MB")
    
    print("\nSecond invocation (Warm Start):")
    response2 = platform.invoke("text-processor", event)
    print(f"  Status: {response2.status_code}")
    print(f"  Cold Start: {response2.cold_start}")
    print(f"  Duration: {response2.execution_duration_ms:.2f}ms")
    print(f"  Memory Used: {response2.memory_used_mb:.1f}MB")
    
    speedup = response1.execution_duration_ms / response2.execution_duration_ms if response2.execution_duration_ms > 0 else 1
    print(f"\n  Warm start {speedup:.2f}x faster than cold start")
    
    # Demo 3: Multiple Operations
    print("\n" + "=" * 80)
    print("DEMO 3: Multiple Text Processing Operations")
    print("=" * 80)
    
    operations = [
        ('summarize', 'AI agents are autonomous systems that can reason and act.'),
        ('sentiment', 'This serverless architecture is amazing!'),
        ('extract', 'Machine learning models process data to make predictions.')
    ]
    
    print("\nProcessing multiple operations...")
    print("-" * 80)
    
    for operation, text in operations:
        event = ServerlessEvent(
            event_id=f"evt_{uuid.uuid4().hex[:8]}",
            trigger_type=TriggerType.HTTP_API,
            timestamp=datetime.now(),
            data={'operation': operation, 'text': text}
        )
        
        response = platform.invoke("text-processor", event)
        
        print(f"\nOperation: {operation}")
        print(f"  Input: {text[:50]}...")
        print(f"  Duration: {response.execution_duration_ms:.2f}ms")
        
        if response.body and 'result' in response.body:
            result = response.body['result']
            if isinstance(result, str):
                print(f"  Result: {result[:80]}...")
            else:
                print(f"  Result: {result}")
    
    # Demo 4: Webhook Handler
    print("\n" + "=" * 80)
    print("DEMO 4: Webhook Event Processing")
    print("=" * 80)
    
    webhook_event = ServerlessEvent(
        event_id="webhook_001",
        trigger_type=TriggerType.WEBHOOK,
        timestamp=datetime.now(),
        data={
            'type': 'user_signup',
            'payload': {
                'user_id': '12345',
                'email': 'user@example.com',
                'timestamp': datetime.now().isoformat()
            }
        }
    )
    
    print("\nProcessing webhook event...")
    response = platform.invoke("webhook-handler", webhook_event)
    
    print(f"  Status: {response.status_code}")
    print(f"  Duration: {response.execution_duration_ms:.2f}ms")
    print(f"  Response:")
    for key, value in response.body.items():
        print(f"    {key}: {value}")
    
    # Demo 5: Scheduled Task
    print("\n" + "=" * 80)
    print("DEMO 5: Scheduled Task Execution")
    print("=" * 80)
    
    scheduled_events = [
        {'task': 'report'},
        {'task': 'cleanup'},
        {'task': 'sync'}
    ]
    
    print("\nExecuting scheduled tasks...")
    print("-" * 80)
    
    for task_data in scheduled_events:
        event = ServerlessEvent(
            event_id=f"sched_{uuid.uuid4().hex[:8]}",
            trigger_type=TriggerType.SCHEDULED,
            timestamp=datetime.now(),
            data=task_data
        )
        
        response = platform.invoke("scheduled-task", event)
        
        print(f"\nTask: {task_data['task']}")
        print(f"  Duration: {response.execution_duration_ms:.2f}ms")
        print(f"  Result: {response.body.get('result', {})}")
    
    # Demo 6: Function Metrics
    print("\n" + "=" * 80)
    print("DEMO 6: Function Metrics & Monitoring")
    print("=" * 80)
    
    print("\nFunction Metrics:")
    print("-" * 80)
    
    for func_name in ["text-processor", "webhook-handler", "scheduled-task"]:
        metrics = platform.get_metrics(func_name)
        
        if metrics:
            print(f"\n{func_name}:")
            print(f"  Total Invocations: {metrics.invocations}")
            print(f"  Cold Starts: {metrics.cold_starts}")
            print(f"  Warm Starts: {metrics.warm_starts}")
            print(f"  Errors: {metrics.errors}")
            print(f"  Avg Duration: {metrics.average_duration_ms:.2f}ms")
            print(f"  Min Duration: {metrics.min_duration_ms:.2f}ms")
            print(f"  Max Duration: {metrics.max_duration_ms:.2f}ms")
            
            cold_start_pct = (metrics.cold_starts / metrics.invocations * 100) if metrics.invocations > 0 else 0
            print(f"  Cold Start Rate: {cold_start_pct:.1f}%")
    
    # Summary
    print("\n" + "=" * 80)
    print("SUMMARY")
    print("=" * 80)
    
    summary = """
The Serverless Agent demonstrates cloud-native, event-driven AI:

KEY CAPABILITIES:
1. Stateless Execution: No persistent state between invocations
2. Event-Driven: Triggered by various event sources
3. Auto-Scaling: Scales automatically from zero to thousands
4. Cold/Warm Starts: Container reuse optimization
5. Pay-Per-Execution: Only pay for actual compute time
6. Managed Infrastructure: No server management required
7. Multiple Triggers: HTTP, webhooks, schedules, queues, streams
8. Built-in Monitoring: Integrated metrics and logging

BENEFITS:
- Zero infrastructure management
- Automatic scaling with demand
- Cost optimization (pay per use)
- High availability by default
- Fast deployment and iteration
- Built-in fault tolerance
- Easy integration with cloud services
- Reduced operational overhead

USE CASES:
- REST API endpoints for AI services
- Webhook processing for integrations
- Scheduled AI tasks (reports, cleanup, sync)
- Event-driven data processing
- Real-time stream processing
- Image/document processing pipelines
- Chatbot backends
- ETL and data transformation
- IoT data processing
- Microservices backends

PRODUCTION CONSIDERATIONS:
1. Cold Starts: Minimize with provisioned concurrency
2. Statelessness: Use external storage (S3, DynamoDB, RDS)
3. Timeout Limits: Functions have max execution time
4. Memory Allocation: Balance cost vs performance
5. Concurrency Limits: Understand and configure limits
6. VPC Networking: Consider cold start impact
7. Error Handling: Implement retries and DLQs
8. Monitoring: Use CloudWatch, X-Ray for observability
9. Security: IAM roles, secrets management
10. Cost Management: Monitor and optimize usage

ADVANCED EXTENSIONS:
- Lambda layers for shared dependencies
- Step Functions for complex workflows
- EventBridge for event routing
- API Gateway for HTTP endpoints
- SQS/SNS for async messaging
- Lambda@Edge for CDN integration
- Provisioned concurrency for predictable latency
- Container image deployment
- Multi-region deployment
- Custom runtimes and extensions

Serverless architecture is ideal for event-driven AI workloads
with variable demand, optimizing both cost and scalability.
"""
    
    print(summary)


if __name__ == "__main__":
    demonstrate_serverless_agent()

    Agentic Design Pattern: Serverless Agent

    Pattern 082: Serverless Agent – Line-by-Line Explanation

    Overview

    This document provides a detailed line-by-line explanation of the Serverless Agent pattern implementation.

    Pattern Description

    Pattern 082: Serverless Agent

    Description: A Serverless Agent runs on serverless computing platforms (AWS Lambda, Azure Functions, Google Cloud Functions) where infrastructure is fully managed by the cloud provider. This pattern enables event-driven, auto-scaling AI agents that only consume resources when invoked, optimizing costs and operational overhead.

    The agent is designed for stateless execution, cold start optimization,
event-driven triggers, and seamless scaling from zero to thousands of
concurrent executions without manual intervention.

    Components: 1. Function Handler: Entry point for serverless execution 2. Event Processor: Processes various event types 3. State Manager: External state storage (S3, DynamoDB) 4. Cold Start Optimizer: Minimizes initialization time 5. Environment Manager: Configuration and secrets 6. Response Builder: Formats function responses 7. Error Handler: Graceful error handling 8. Metrics Collector: CloudWatch/monitoring integration

    Key Features: – Stateless execution model – Event-driven triggers – Auto-scaling from zero – Pay-per-execution pricing – No server management – Built-in fault tolerance – Managed infrastructure – Cold start optimization – Multiple trigger sources – Seamless integration with cloud services

    Use Cases: – API endpoints for AI services – Event-driven AI processing – Webhook handlers – Scheduled AI tasks – Image/document processing – Real-time data transformation – Chatbot backends – ETL pipelines – IoT data processing – Microservices backends

    LangChain Implementation: Simulates AWS Lambda-style function handlers with event processing, stateless execution, and external state management patterns.

    File Statistics

    • Total Lines: 756
    • Classes: 11
    • Functions: 1
    • Constants: 0

    Module Documentation

    """
Pattern 082: Serverless Agent

Description:
    A Serverless Agent runs on serverless computing platforms (AWS Lambda,
    Azure Functions, Google Cloud Functions) where infrastructure is fully
    managed by the cloud provider. This pattern enables event-driven, auto-scaling
    AI agents that only consume resources when invoked, optimizing costs and
    operational overhead.

    The agent is designed for stateless execution, cold start optimization,
    event-driven triggers, and seamless scaling from zero to thousands of
    concurrent executions without manual intervention.

Components:
    1. Function Handler: Entry point for serverless execution
    2. Event Processor: Processes various event types
    3. State Manager: External state storage (S3, DynamoDB)
    4. Cold Start Optimizer: Minimizes initialization time
    5. Environment Manager: Configuration and secrets
    6. Response Builder: Formats function responses
    7. Error Handler: Graceful error handling
    8. Metrics Collector: CloudWatch/monitoring integration

Key Features:
    - Stateless execution model
    - Event-driven triggers
    - Auto-scaling from zero
    - Pay-per-execution pricing
    - No server management
    - Built-in fault tolerance
    - Managed infrastructure
    - Cold start optimization
    - Multiple trigger sources
    - Seamless integration with cloud services

Use Cases:
    - API endpoints for AI services
    - Event-driven AI processing
    - Webhook handlers
    - Scheduled AI tasks
    - Image/document processing
    - Real-time data transformation
    - Chatbot backends
    - ETL pipelines
    - IoT data processing
    - Microservices backends

LangChain Implementation:
    Simulates AWS Lambda-style function handlers with event processing,
    stateless execution, and external state management patterns.
"""

    Purpose: This module implements the Serverless Agent pattern, providing a structured approach to agent design and implementation.

    Import Statements

    Standard Library Imports

    import collections.defaultdict
import dataclasses.dataclass
import dataclasses.field
import datetime.datetime
import enum.Enum
import json
import langchain_core.output_parsers.StrOutputParser
import langchain_core.prompts.ChatPromptTemplate
import os
import time
import typing.Any
import typing.Callable
import typing.Dict
import typing.List
import typing.Optional

    Purpose: Core Python functionality: – collections.defaultdict: Specialized container datatypes – dataclasses.dataclass: Simplified class definitions with @dataclass – dataclasses.field: Simplified class definitions with @dataclass – datetime.datetime: Date and time handling – enum.Enum: Enumeration support – json: JSON serialization/deserialization – langchain_core.output_parsers.StrOutputParser: Regular expression operations – langchain_core.prompts.ChatPromptTemplate: Regular expression operations – os: Environment variable access and system operations – time: Time-related functions – typing.Any: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Callable: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Dict: Type hints for better code documentation (List, Dict, Any, Optional) – typing.List: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Optional: Type hints for better code documentation (List, Dict, Any, Optional)

    LangChain Imports

    from langchain_openai.ChatOpenAI

    Purpose: LangChain framework components: – LLM Interface: Wrapper for OpenAI models – Chains: Composable building blocks for complex workflows

    Other Imports

    from dotenv.load_dotenv
from uuid

    Purpose: Additional dependencies for specialized functionality.

    Classes

    Class: TriggerType

    Lines 72-81

    Inherits from: Enum

    Purpose:

    Types of serverless triggers

    Class: ExecutionStatus

    Lines 84-91

    Inherits from: Enum

    Purpose:

    Status of function execution

    Class: ServerlessEvent

    Lines 95-102

    Purpose:

    Event that triggers serverless function

    Class: ServerlessContext

    Lines 106-115

    Purpose:

    Execution context for serverless function

    Class: ServerlessResponse

    Lines 119-127

    Purpose:

    Response from serverless function

    Class: FunctionMetrics

    Lines 131-141

    Purpose:

    Metrics for serverless function

    Class: ServerlessAgent

    Lines 144-290

    Purpose:

    Base serverless agent with stateless execution.

Designed for serverless platforms with event-driven triggers,
cold start optimization, and external state management.

    Methods:

    __init__(self, function_name, memory_limit_mb, timeout_seconds)

    Lines 156-184

    Purpose:

    Initialize serverless agent.

Args:
    function_name: Name of the function
    memory_limit_mb: Memory limit
    timeout_seconds: Execution timeout

    Parameters: – function_name: Function name – memory_limit_mb: Memory store for conversation history – timeout_seconds: Timeout seconds

    get_or_create(cls, function_name)

    Lines 187-202

    Decorators: classmethod

    Purpose:

    Get warm instance or create new (simulates Lambda container reuse).

Args:
    function_name: Function name

Returns:
    ServerlessAgent instance

    Parameters: – cls: Cls – function_name: Function name

    Returns: ServerlessAgent

    handler(self, event, context)

    Lines 204-269

    Purpose:

    Main handler function (entry point).

Args:
    event: Triggering event
    context: Execution context

Returns:
    ServerlessResponse

    Parameters: – event: Event – context: Context

    Returns: ServerlessResponse

    _process_event(self, event, context)

    Lines 271-290

    Purpose:

    Process event (to be overridden by subclasses).

Args:
    event: Event to process
    context: Execution context

Returns:
    Processing result

    Parameters: – event: Event – context: Context

    Returns: <ast.Subscript object at 0x000001A06D750A50>

    Class: TextProcessingFunction

    Lines 293-348

    Inherits from: ServerlessAgent

    Purpose:

    Serverless function for text processing

    Methods:

    __init__(self)

    Lines 296-297

    Purpose: Initialize the class instance with required parameters.

    Parameters:

    _process_event(self, event, context)

    Lines 299-323

    Purpose:

    Process text from event

    Parameters: – event: Event – context: Context

    Returns: <ast.Subscript object at 0x000001A06D17BC90>

    _summarize(self, text)

    Lines 325-331

    Purpose:

    Summarize text

    Parameters: – text: Text

    Returns: str

    _analyze_sentiment(self, text)

    Lines 333-339

    Purpose:

    Analyze sentiment

    Parameters: – text: Text

    Returns: str

    _extract_keywords(self, text)

    Lines 341-348

    Purpose:

    Extract keywords

    Parameters: – text: Text

    Returns: <ast.Subscript object at 0x000001A06D75B810>

    Class: WebhookFunction

    Lines 351-375

    Inherits from: ServerlessAgent

    Purpose:

    Serverless function for webhook processing

    Methods:

    __init__(self)

    Lines 354-355

    Purpose: Initialize the class instance with required parameters.

    Parameters:

    _process_event(self, event, context)

    Lines 357-375

    Purpose:

    Process webhook event

    Parameters: – event: Event – context: Context

    Returns: <ast.Subscript object at 0x000001A06D157B50>

    Class: ScheduledFunction

    Lines 378-427

    Inherits from: ServerlessAgent

    Purpose:

    Serverless function for scheduled tasks

    Methods:

    __init__(self)

    Lines 381-382

    Purpose: Initialize the class instance with required parameters.

    Parameters:

    _process_event(self, event, context)

    Lines 384-405

    Purpose:

    Process scheduled event

    Parameters: – event: Event – context: Context

    Returns: <ast.Subscript object at 0x000001A06D83EE50>

    _generate_report(self)

    Lines 407-413

    Purpose:

    Generate report

    Parameters:

    Returns: <ast.Subscript object at 0x000001A06D168490>

    _cleanup_data(self)

    Lines 415-420

    Purpose:

    Cleanup old data

    Parameters:

    Returns: <ast.Subscript object at 0x000001A06D626590>

    _sync_data(self)

    Lines 422-427

    Purpose:

    Sync data

    Parameters:

    Returns: <ast.Subscript object at 0x000001A06D627090>

    Class: ServerlessPlatform

    Lines 430-507

    Purpose:

    Simulates serverless platform (like AWS Lambda).

Manages function lifecycle, invocations, and cold/warm starts.

    Methods:

    __init__(self)

    Lines 437-440

    Purpose:

    Initialize platform

    Parameters:

    deploy_function(self, function)

    Lines 442-454

    Purpose:

    Deploy a serverless function.

Args:
    function: Function to deploy

Returns:
    True if deployed successfully

    Parameters: – function: Function

    Returns: bool

    invoke(self, function_name, event)

    Lines 456-495

    Purpose:

    Invoke a serverless function.

Args:
    function_name: Name of function to invoke
    event: Event to process

Returns:
    ServerlessResponse

    Parameters: – function_name: Function name – event: Event

    Returns: ServerlessResponse

    get_metrics(self, function_name)

    Lines 497-507

    Purpose:

    Get function metrics.

Args:
    function_name: Function name

Returns:
    FunctionMetrics if found

    Parameters: – function_name: Function name

    Returns: <ast.Subscript object at 0x000001A06D80C0D0>

    Functions

    demonstrate_serverless_agent()

    Lines 510-751

    Purpose:

    Demonstrate the serverless agent pattern

    Key Concepts Summary

    Pattern Overview

    The Serverless Agent pattern provides a structured approach to building AI agents with specific capabilities and behaviors. This implementation leverages LangChain’s composable components to create a robust and maintainable solution.

    Core Components

    1. Agent Class: Main implementation of the pattern logic
    2. Prompt Engineering: Carefully crafted prompts to guide LLM behavior
    3. Tool Integration: Optional tools for extended functionality
    4. Memory Management: State handling across interactions
    5. Output Parsing: Structured response processing

    When to Use

    ✅ Good for: – Scenarios requiring this specific pattern’s strengths – Applications needing structured agent behavior – Systems where explainability is important – Production environments requiring reliability

    ❌ Not ideal for: – Simple single-prompt tasks – Scenarios requiring different pattern characteristics – Resource-constrained environments (if computationally intensive)

    Best Practices Demonstrated

    1. Environment Configuration

    • ✅ Use .env files for API keys
    • ✅ Never hardcode credentials
    • ✅ Validate environment variables on startup

    2. Error Handling

    • ✅ Graceful degradation on failures
    • ✅ Informative error messages
    • ✅ Logging for debugging

    3. Prompt Engineering

    • ✅ Clear, specific instructions
    • ✅ Structured output formats
    • ✅ Few-shot examples when beneficial

    4. Code Organization

    • ✅ Separation of concerns (class-based design)
    • ✅ Type hints for clarity
    • ✅ Docstrings for documentation

    5. Performance Optimization

    • ✅ Appropriate temperature settings
    • ✅ Token budget management
    • ✅ Caching where applicable

    LangChain Components Used

    • ChatOpenAI: OpenAI model interface
    • Chains: Sequential component composition

    Why LangChain? – Composable abstractions for LLM applications – Rich ecosystem of integrations – Production-ready patterns – Active community and development

    Execution Flow

    1. Initialize agent with configuration
   ↓
2. Load environment and dependencies
   ↓
3. Prepare prompt with input data
   ↓
4. Invoke LLM through LangChain
   ↓
5. Parse and validate response
   ↓
6. Return formatted result

    Key Decision Points: – Model selection (temperature, model type) – Prompt structure (few-shot vs zero-shot) – Error handling strategy – Output format validation

    Performance Considerations

    Computational Cost

    • LLM Calls: Primary cost factor (based on tokens)
    • Latency: Depends on model and input size
    • Memory: Minimal for most patterns

    Optimization Strategies

    1. Prompt Optimization: Minimize tokens while maintaining clarity
    2. Caching: Cache repeated queries/responses
    3. Model Selection: Use smaller models when appropriate
    4. Batch Processing: Group similar requests
    5. Streaming: For long responses, stream tokens

    Cost Estimates (Approximate)

    • GPT-3.5-turbo: ~$0.002 per request
    • GPT-4: ~$0.03 per request
    • GPT-4-turbo: ~$0.01 per request

    Actual costs vary based on input/output token counts

    Common Pitfalls to Avoid

    1. Prompt Design Issues

    ❌ Vague or ambiguous instructions
    ✅ Clear, specific, testable prompts

    2. Error Handling

    ❌ Ignoring API errors or rate limits
    ✅ Implement retry logic and graceful degradation

    3. Token Management

    ❌ Exceeding model token limits
    ✅ Truncate inputs, chunk processing

    4. Security

    ❌ Exposing API keys in code
    ✅ Use environment variables

    5. Testing

    ❌ Only testing happy paths
    ✅ Test edge cases, failures, malformed inputs

    6. Cost Management

    ❌ Unmonitored API usage
    ✅ Implement budgets, alerts, and monitoring

    Further Improvements

    Production Enhancements

    1. Monitoring: Add logging, metrics, tracing
    2. Validation: Input/output schema validation
    3. Rate Limiting: Implement request throttling
    4. Caching: Redis or in-memory cache for responses
    5. Async Support: Non-blocking operations for scalability

    Feature Extensions

    1. Multi-Model Support: Switch between providers
    2. Fallback Models: Graceful degradation strategy
    3. Custom Tools: Domain-specific function calling
    4. Advanced Memory: Vector stores for long-term memory
    5. Evaluation: Automated testing and benchmarking

    Code Quality

    1. Testing: Unit tests, integration tests
    2. Documentation: API docs, usage examples
    3. Type Safety: Stricter type hints
    4. Configuration: Externalized config files

    Related Patterns

    • Pattern 081
    • Pattern 083

    Pattern Families: – Reasoning Patterns (001-011) – Multi-Agent Patterns (012-019) – Tool Use Patterns (020-025) – Memory Patterns (026-031) – And many more…

    Summary

    The Serverless Agent pattern provides a powerful approach to building AI agents with specific capabilities. This implementation demonstrates:

    ✅ Key Strengths: – Clean, maintainable code structure – LangChain integration for composability – Type hints and documentation – Production-ready patterns

    🎯 Best Use Cases: – Applications requiring this pattern’s characteristics – Systems needing structured agent behavior – Production environments requiring reliability

    📚 Learning Resources: – LangChain Documentation: https://python.langchain.com/ – Pattern Documentation: See related patterns – OpenAI API Reference: https://platform.openai.com/docs/

    File: 082_serverless_agent.py
    Total Lines: 756
    Pattern Category: AI Agent Design Pattern #082

    Generated from actual implementation code
    For educational and reference purposes

  • Agentic Design Pattern: Microservice Agent

    
    """
Pattern 081: Microservice Agent Architecture

Description:
    A Microservice Agent Architecture decomposes AI agents into independently
    deployable microservices that communicate via well-defined APIs. This pattern
    enables scalability, fault isolation, independent deployment, and technology
    diversity across agent components. Each microservice can be developed, deployed,
    and scaled independently while maintaining loose coupling.
    
    The architecture supports distributed agent systems with service discovery,
    load balancing, inter-service communication, circuit breakers, and observability
    across the microservices mesh.

Components:
    1. Agent Services: Independent agent microservices
    2. API Gateway: Entry point and routing
    3. Service Registry: Service discovery and health checks
    4. Message Bus: Async inter-service communication
    5. Load Balancer: Distribute requests across instances
    6. Circuit Breaker: Fault tolerance and isolation
    7. Config Service: Centralized configuration
    8. Monitoring: Distributed tracing and metrics

Key Features:
    - Independent service deployment
    - Service discovery and registration
    - Inter-service communication (REST, gRPC, message queues)
    - Load balancing and auto-scaling
    - Fault isolation and circuit breakers
    - Distributed tracing
    - API gateway pattern
    - Health checks and monitoring
    - Polyglot persistence
    - Technology diversity

Use Cases:
    - Large-scale AI platforms
    - Multi-tenant AI services
    - Enterprise AI systems
    - Cloud-native AI applications
    - Distributed agent coordination
    - Scalable chatbot platforms
    - AI-powered SaaS products
    - Microservices-based RAG systems
    - Agent mesh architectures
    - Multi-model serving platforms

LangChain Implementation:
    Uses FastAPI for service endpoints, service registry pattern,
    inter-service HTTP communication, and distributed agent coordination.
"""

import os
import time
import uuid
import json
from typing import List, Dict, Any, Optional, Callable
from dataclasses import dataclass, field
from enum import Enum
from datetime import datetime
from collections import defaultdict
from dotenv import load_dotenv

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_openai import ChatOpenAI

load_dotenv()


class ServiceStatus(Enum):
    """Status of a microservice"""
    STARTING = "starting"
    HEALTHY = "healthy"
    UNHEALTHY = "unhealthy"
    DEGRADED = "degraded"
    STOPPED = "stopped"


class CommunicationProtocol(Enum):
    """Inter-service communication protocol"""
    HTTP_REST = "http_rest"
    GRPC = "grpc"
    MESSAGE_QUEUE = "message_queue"
    EVENT_BUS = "event_bus"


@dataclass
class ServiceInfo:
    """Information about a microservice"""
    service_id: str
    service_name: str
    version: str
    host: str
    port: int
    protocol: CommunicationProtocol
    status: ServiceStatus
    endpoints: List[str]
    health_check_url: str
    registered_at: datetime = field(default_factory=datetime.now)
    last_heartbeat: datetime = field(default_factory=datetime.now)
    metadata: Dict[str, Any] = field(default_factory=dict)


@dataclass
class ServiceRequest:
    """Request to a microservice"""
    request_id: str
    service_name: str
    endpoint: str
    method: str
    data: Dict[str, Any]
    headers: Dict[str, str] = field(default_factory=dict)
    timeout: float = 30.0


@dataclass
class ServiceResponse:
    """Response from a microservice"""
    request_id: str
    service_name: str
    status_code: int
    data: Any
    error: Optional[str] = None
    duration: float = 0.0
    metadata: Dict[str, Any] = field(default_factory=dict)


class ServiceRegistry:
    """
    Service registry for microservice discovery.
    
    Maintains a registry of available services with health monitoring.
    """
    
    def __init__(self):
        """Initialize service registry"""
        self.services: Dict[str, ServiceInfo] = {}
        self.service_instances: Dict[str, List[str]] = defaultdict(list)
    
    def register(self, service: ServiceInfo) -> bool:
        """
        Register a service.
        
        Args:
            service: Service information
            
        Returns:
            True if registered successfully
        """
        self.services[service.service_id] = service
        self.service_instances[service.service_name].append(service.service_id)
        return True
    
    def deregister(self, service_id: str) -> bool:
        """
        Deregister a service.
        
        Args:
            service_id: Service ID
            
        Returns:
            True if deregistered successfully
        """
        if service_id in self.services:
            service = self.services[service_id]
            self.service_instances[service.service_name].remove(service_id)
            del self.services[service_id]
            return True
        return False
    
    def discover(self, service_name: str) -> Optional[ServiceInfo]:
        """
        Discover a healthy service instance.
        
        Args:
            service_name: Name of service to discover
            
        Returns:
            ServiceInfo if found, None otherwise
        """
        instances = self.service_instances.get(service_name, [])
        
        # Find healthy instance
        for instance_id in instances:
            service = self.services.get(instance_id)
            if service and service.status == ServiceStatus.HEALTHY:
                return service
        
        return None
    
    def health_check(self, service_id: str) -> ServiceStatus:
        """
        Check health of a service.
        
        Args:
            service_id: Service ID
            
        Returns:
            Service status
        """
        if service_id in self.services:
            service = self.services[service_id]
            # Simulate health check
            service.last_heartbeat = datetime.now()
            return service.status
        
        return ServiceStatus.STOPPED
    
    def list_services(self) -> List[ServiceInfo]:
        """
        List all registered services.
        
        Returns:
            List of service information
        """
        return list(self.services.values())


class AgentService:
    """
    Base microservice for AI agents.
    
    Each agent capability is encapsulated as an independent service.
    """
    
    def __init__(
        self,
        service_name: str,
        version: str = "1.0.0",
        port: int = 8000
    ):
        """
        Initialize agent service.
        
        Args:
            service_name: Name of the service
            version: Service version
            port: Port number
        """
        self.service_id = f"{service_name}_{uuid.uuid4().hex[:8]}"
        self.service_name = service_name
        self.version = version
        self.port = port
        self.status = ServiceStatus.STARTING
        
        # LLM for this service
        self.llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0.7)
        
        # Request tracking
        self.requests: Dict[str, ServiceRequest] = {}
        self.responses: Dict[str, ServiceResponse] = {}
    
    def get_info(self) -> ServiceInfo:
        """
        Get service information.
        
        Returns:
            ServiceInfo object
        """
        return ServiceInfo(
            service_id=self.service_id,
            service_name=self.service_name,
            version=self.version,
            host="localhost",
            port=self.port,
            protocol=CommunicationProtocol.HTTP_REST,
            status=self.status,
            endpoints=["/health", "/process"],
            health_check_url=f"http://localhost:{self.port}/health"
        )
    
    def start(self) -> bool:
        """
        Start the service.
        
        Returns:
            True if started successfully
        """
        self.status = ServiceStatus.HEALTHY
        return True
    
    def stop(self) -> bool:
        """
        Stop the service.
        
        Returns:
            True if stopped successfully
        """
        self.status = ServiceStatus.STOPPED
        return True
    
    def health(self) -> Dict[str, Any]:
        """
        Health check endpoint.
        
        Returns:
            Health status
        """
        return {
            "service": self.service_name,
            "status": self.status.value,
            "version": self.version,
            "timestamp": datetime.now().isoformat()
        }
    
    def process(self, data: Dict[str, Any]) -> Dict[str, Any]:
        """
        Process a request (to be overridden by subclasses).
        
        Args:
            data: Request data
            
        Returns:
            Response data
        """
        raise NotImplementedError("Subclasses must implement process()")


class SummarizationService(AgentService):
    """Microservice for text summarization"""
    
    def __init__(self):
        super().__init__("summarization", version="1.0.0", port=8001)
    
    def process(self, data: Dict[str, Any]) -> Dict[str, Any]:
        """
        Summarize text.
        
        Args:
            data: {'text': str, 'length': str}
            
        Returns:
            {'summary': str}
        """
        text = data.get('text', '')
        length = data.get('length', 'short')
        
        prompt = ChatPromptTemplate.from_template(
            "Summarize the following text in {length} form:\n\n{text}"
        )
        
        chain = prompt | self.llm | StrOutputParser()
        
        summary = chain.invoke({
            'text': text[:1000],
            'length': length
        })
        
        return {
            'summary': summary,
            'service': self.service_name
        }


class TranslationService(AgentService):
    """Microservice for translation"""
    
    def __init__(self):
        super().__init__("translation", version="1.0.0", port=8002)
    
    def process(self, data: Dict[str, Any]) -> Dict[str, Any]:
        """
        Translate text.
        
        Args:
            data: {'text': str, 'target_language': str}
            
        Returns:
            {'translation': str}
        """
        text = data.get('text', '')
        target_language = data.get('target_language', 'Spanish')
        
        prompt = ChatPromptTemplate.from_template(
            "Translate the following text to {language}:\n\n{text}"
        )
        
        chain = prompt | self.llm | StrOutputParser()
        
        translation = chain.invoke({
            'text': text[:500],
            'language': target_language
        })
        
        return {
            'translation': translation,
            'target_language': target_language,
            'service': self.service_name
        }


class SentimentService(AgentService):
    """Microservice for sentiment analysis"""
    
    def __init__(self):
        super().__init__("sentiment", version="1.0.0", port=8003)
    
    def process(self, data: Dict[str, Any]) -> Dict[str, Any]:
        """
        Analyze sentiment.
        
        Args:
            data: {'text': str}
            
        Returns:
            {'sentiment': str, 'confidence': float}
        """
        text = data.get('text', '')
        
        prompt = ChatPromptTemplate.from_template(
            """Analyze the sentiment of this text. Respond with:
SENTIMENT: positive/negative/neutral
CONFIDENCE: 0.0-1.0

Text: {text}"""
        )
        
        chain = prompt | self.llm | StrOutputParser()
        
        result = chain.invoke({'text': text[:500]})
        
        # Parse result
        sentiment = "neutral"
        confidence = 0.5
        
        for line in result.split('\n'):
            if 'SENTIMENT:' in line:
                sentiment = line.split(':')[1].strip().lower()
            elif 'CONFIDENCE:' in line:
                try:
                    confidence = float(line.split(':')[1].strip())
                except:
                    pass
        
        return {
            'sentiment': sentiment,
            'confidence': confidence,
            'service': self.service_name
        }


class APIGateway:
    """
    API Gateway for routing requests to microservices.
    
    Provides a single entry point and routes to appropriate services.
    """
    
    def __init__(self, registry: ServiceRegistry):
        """
        Initialize API gateway.
        
        Args:
            registry: Service registry
        """
        self.registry = registry
        self.request_count = 0
        self.route_map = {
            'summarize': 'summarization',
            'translate': 'translation',
            'sentiment': 'sentiment'
        }
    
    def route(self, operation: str, data: Dict[str, Any]) -> ServiceResponse:
        """
        Route request to appropriate service.
        
        Args:
            operation: Operation to perform
            data: Request data
            
        Returns:
            ServiceResponse
        """
        request_id = f"req_{self.request_count}"
        self.request_count += 1
        
        start_time = time.time()
        
        # Determine target service
        service_name = self.route_map.get(operation)
        
        if not service_name:
            return ServiceResponse(
                request_id=request_id,
                service_name="unknown",
                status_code=404,
                data=None,
                error=f"Unknown operation: {operation}",
                duration=time.time() - start_time
            )
        
        # Discover service
        service = self.registry.discover(service_name)
        
        if not service:
            return ServiceResponse(
                request_id=request_id,
                service_name=service_name,
                status_code=503,
                data=None,
                error=f"Service unavailable: {service_name}",
                duration=time.time() - start_time
            )
        
        # Forward request to service (simulated)
        try:
            # In real implementation, would make HTTP request
            # Here we simulate the service call
            result_data = {
                'status': 'success',
                'operation': operation,
                'service': service_name
            }
            
            return ServiceResponse(
                request_id=request_id,
                service_name=service_name,
                status_code=200,
                data=result_data,
                duration=time.time() - start_time
            )
            
        except Exception as e:
            return ServiceResponse(
                request_id=request_id,
                service_name=service_name,
                status_code=500,
                data=None,
                error=str(e),
                duration=time.time() - start_time
            )


class MicroserviceOrchestrator:
    """
    Orchestrator for coordinating multiple microservices.
    
    Manages service lifecycle and coordinates complex workflows
    across multiple services.
    """
    
    def __init__(self):
        """Initialize orchestrator"""
        self.registry = ServiceRegistry()
        self.gateway = APIGateway(self.registry)
        self.services: Dict[str, AgentService] = {}
    
    def deploy_service(self, service: AgentService) -> bool:
        """
        Deploy a microservice.
        
        Args:
            service: Agent service to deploy
            
        Returns:
            True if deployed successfully
        """
        # Start service
        service.start()
        
        # Register with registry
        service_info = service.get_info()
        self.registry.register(service_info)
        
        # Track service
        self.services[service.service_id] = service
        
        return True
    
    def undeploy_service(self, service_id: str) -> bool:
        """
        Undeploy a microservice.
        
        Args:
            service_id: Service ID
            
        Returns:
            True if undeployed successfully
        """
        if service_id in self.services:
            service = self.services[service_id]
            service.stop()
            self.registry.deregister(service_id)
            del self.services[service_id]
            return True
        return False
    
    def execute_workflow(
        self,
        operations: List[Dict[str, Any]]
    ) -> List[ServiceResponse]:
        """
        Execute a workflow across multiple services.
        
        Args:
            operations: List of operations with data
            
        Returns:
            List of service responses
        """
        responses = []
        
        for op in operations:
            operation = op.get('operation')
            data = op.get('data', {})
            
            response = self.gateway.route(operation, data)
            responses.append(response)
        
        return responses
    
    def get_service_health(self) -> Dict[str, Any]:
        """
        Get health status of all services.
        
        Returns:
            Health status dictionary
        """
        services = self.registry.list_services()
        
        health_status = {
            'total_services': len(services),
            'healthy': 0,
            'unhealthy': 0,
            'services': []
        }
        
        for service in services:
            status = self.registry.health_check(service.service_id)
            
            if status == ServiceStatus.HEALTHY:
                health_status['healthy'] += 1
            else:
                health_status['unhealthy'] += 1
            
            health_status['services'].append({
                'name': service.service_name,
                'id': service.service_id,
                'status': status.value,
                'version': service.version
            })
        
        return health_status


def demonstrate_microservice_architecture():
    """Demonstrate the microservice agent architecture"""
    print("=" * 80)
    print("MICROSERVICE AGENT ARCHITECTURE DEMONSTRATION")
    print("=" * 80)
    
    orchestrator = MicroserviceOrchestrator()
    
    # Demo 1: Deploy Microservices
    print("\n" + "=" * 80)
    print("DEMO 1: Deploying Microservices")
    print("=" * 80)
    
    # Deploy services
    services = [
        SummarizationService(),
        TranslationService(),
        SentimentService()
    ]
    
    print("\nDeploying services...")
    for service in services:
        success = orchestrator.deploy_service(service)
        print(f"  ✓ Deployed: {service.service_name} v{service.version} on port {service.port}")
    
    # Demo 2: Service Discovery
    print("\n" + "=" * 80)
    print("DEMO 2: Service Discovery")
    print("=" * 80)
    
    print("\nRegistered Services:")
    print("-" * 80)
    
    for service_info in orchestrator.registry.list_services():
        print(f"\nService: {service_info.service_name}")
        print(f"  ID: {service_info.service_id}")
        print(f"  Version: {service_info.version}")
        print(f"  Status: {service_info.status.value}")
        print(f"  Endpoint: {service_info.host}:{service_info.port}")
        print(f"  Protocol: {service_info.protocol.value}")
    
    # Demo 3: Service Health Checks
    print("\n" + "=" * 80)
    print("DEMO 3: Service Health Monitoring")
    print("=" * 80)
    
    health = orchestrator.get_service_health()
    
    print(f"\nCluster Health:")
    print(f"  Total Services: {health['total_services']}")
    print(f"  Healthy: {health['healthy']}")
    print(f"  Unhealthy: {health['unhealthy']}")
    
    print(f"\nService Status:")
    for svc in health['services']:
        status_icon = "✓" if svc['status'] == 'healthy' else "✗"
        print(f"  {status_icon} {svc['name']} ({svc['version']}) - {svc['status']}")
    
    # Demo 4: API Gateway Routing
    print("\n" + "=" * 80)
    print("DEMO 4: API Gateway Request Routing")
    print("=" * 80)
    
    print("\nRouting requests through API Gateway...")
    print("-" * 80)
    
    requests = [
        {'operation': 'summarize', 'data': {'text': 'Sample text', 'length': 'short'}},
        {'operation': 'translate', 'data': {'text': 'Hello', 'target_language': 'French'}},
        {'operation': 'sentiment', 'data': {'text': 'Great product!'}}
    ]
    
    for req in requests:
        response = orchestrator.gateway.route(req['operation'], req['data'])
        
        print(f"\nRequest: {req['operation']}")
        print(f"  Request ID: {response.request_id}")
        print(f"  Service: {response.service_name}")
        print(f"  Status Code: {response.status_code}")
        print(f"  Duration: {response.duration*1000:.2f}ms")
        
        if response.error:
            print(f"  Error: {response.error}")
    
    # Demo 5: Direct Service Calls
    print("\n" + "=" * 80)
    print("DEMO 5: Direct Service Invocation")
    print("=" * 80)
    
    print("\nCalling services directly...")
    print("-" * 80)
    
    # Summarization
    print("\n1. Summarization Service:")
    summarization_svc = services[0]
    result = summarization_svc.process({
        'text': 'Microservices architecture enables building complex systems through independently deployable services.',
        'length': 'very short'
    })
    print(f"   Input: Long text about microservices")
    print(f"   Summary: {result['summary'][:80]}...")
    
    # Translation
    print("\n2. Translation Service:")
    translation_svc = services[1]
    result = translation_svc.process({
        'text': 'Hello, how are you?',
        'target_language': 'Spanish'
    })
    print(f"   Input: 'Hello, how are you?'")
    print(f"   Translation: {result['translation']}")
    
    # Sentiment
    print("\n3. Sentiment Service:")
    sentiment_svc = services[2]
    result = sentiment_svc.process({
        'text': 'This microservice architecture is fantastic!'
    })
    print(f"   Input: 'This microservice architecture is fantastic!'")
    print(f"   Sentiment: {result['sentiment']} (confidence: {result['confidence']:.2f})")
    
    # Demo 6: Multi-Service Workflow
    print("\n" + "=" * 80)
    print("DEMO 6: Orchestrated Multi-Service Workflow")
    print("=" * 80)
    
    workflow = [
        {'operation': 'summarize', 'data': {'text': 'Long article text...', 'length': 'short'}},
        {'operation': 'translate', 'data': {'text': 'Summary text', 'target_language': 'German'}},
        {'operation': 'sentiment', 'data': {'text': 'Translated summary'}}
    ]
    
    print("\nExecuting workflow:")
    print("  1. Summarize → 2. Translate → 3. Analyze Sentiment")
    print("-" * 80)
    
    responses = orchestrator.execute_workflow(workflow)
    
    print(f"\nWorkflow Results:")
    for i, response in enumerate(responses, 1):
        print(f"\n  Step {i}: {response.service_name}")
        print(f"    Status: {response.status_code}")
        print(f"    Duration: {response.duration*1000:.2f}ms")
    
    total_duration = sum(r.duration for r in responses)
    print(f"\n  Total Workflow Duration: {total_duration*1000:.2f}ms")
    
    # Summary
    print("\n" + "=" * 80)
    print("SUMMARY")
    print("=" * 80)
    
    summary = """
The Microservice Agent Architecture demonstrates distributed AI systems:

KEY CAPABILITIES:
1. Service Registry: Automatic service discovery and registration
2. API Gateway: Single entry point with intelligent routing
3. Independent Services: Each capability as a separate microservice
4. Health Monitoring: Continuous health checks and status tracking
5. Service Orchestration: Coordinate complex multi-service workflows
6. Fault Isolation: Failures contained within services
7. Load Balancing: Distribute traffic across service instances
8. Technology Diversity: Each service can use different tech stacks

BENEFITS:
- Independent deployment and scaling
- Fault isolation and resilience
- Technology flexibility per service
- Team autonomy and parallel development
- Easier testing and maintenance
- Better resource utilization
- Scalability for specific services
- Faster iteration and releases

USE CASES:
- Large-scale AI platforms with multiple capabilities
- Multi-tenant SaaS AI applications
- Enterprise AI systems requiring high availability
- Cloud-native AI applications
- Distributed agent coordination
- Microservices-based RAG systems
- AI-powered e-commerce platforms
- Scalable chatbot infrastructures
- Multi-model serving platforms
- Agent mesh architectures

PRODUCTION CONSIDERATIONS:
1. Service Discovery: Use Consul, etcd, or Kubernetes DNS
2. API Gateway: Kong, NGINX, or AWS API Gateway
3. Communication: REST APIs, gRPC, or message queues
4. Monitoring: Distributed tracing (Jaeger, Zipkin)
5. Logging: Centralized logging (ELK stack)
6. Configuration: External config service (Spring Cloud Config)
7. Security: API authentication, mTLS between services
8. Circuit Breakers: Prevent cascading failures
9. Container Orchestration: Kubernetes, Docker Swarm
10. Service Mesh: Istio, Linkerd for advanced features

ADVANCED EXTENSIONS:
- Event-driven architecture with Kafka/RabbitMQ
- CQRS pattern for read/write separation
- Saga pattern for distributed transactions
- API versioning and backward compatibility
- Blue-green deployments
- Canary releases for gradual rollouts
- Auto-scaling based on metrics
- Multi-region deployment
- Service mesh for advanced traffic management
- GraphQL federation for unified API

Microservice architecture is essential for building scalable,
maintainable, and resilient AI systems in production environments.
"""
    
    print(summary)


if __name__ == "__main__":
    demonstrate_microservice_architecture()

    Agentic Design Pattern: Microservice Agent

    Pattern 081: Microservice Agent – Line-by-Line Explanation

    Overview

    This document provides a detailed line-by-line explanation of the Microservice Agent pattern implementation.

    Pattern Description

    Pattern 081: Microservice Agent Architecture

    Description: A Microservice Agent Architecture decomposes AI agents into independently deployable microservices that communicate via well-defined APIs. This pattern enables scalability, fault isolation, independent deployment, and technology diversity across agent components. Each microservice can be developed, deployed, and scaled independently while maintaining loose coupling.

    The architecture supports distributed agent systems with service discovery,
load balancing, inter-service communication, circuit breakers, and observability
across the microservices mesh.

    Components: 1. Agent Services: Independent agent microservices 2. API Gateway: Entry point and routing 3. Service Registry: Service discovery and health checks 4. Message Bus: Async inter-service communication 5. Load Balancer: Distribute requests across instances 6. Circuit Breaker: Fault tolerance and isolation 7. Config Service: Centralized configuration 8. Monitoring: Distributed tracing and metrics

    Key Features: – Independent service deployment – Service discovery and registration – Inter-service communication (REST, gRPC, message queues) – Load balancing and auto-scaling – Fault isolation and circuit breakers – Distributed tracing – API gateway pattern – Health checks and monitoring – Polyglot persistence – Technology diversity

    Use Cases: – Large-scale AI platforms – Multi-tenant AI services – Enterprise AI systems – Cloud-native AI applications – Distributed agent coordination – Scalable chatbot platforms – AI-powered SaaS products – Microservices-based RAG systems – Agent mesh architectures – Multi-model serving platforms

    LangChain Implementation: Uses FastAPI for service endpoints, service registry pattern, inter-service HTTP communication, and distributed agent coordination.

    File Statistics

    • Total Lines: 869
    • Classes: 12
    • Functions: 1
    • Constants: 0

    Module Documentation

    """
Pattern 081: Microservice Agent Architecture

Description:
    A Microservice Agent Architecture decomposes AI agents into independently
    deployable microservices that communicate via well-defined APIs. This pattern
    enables scalability, fault isolation, independent deployment, and technology
    diversity across agent components. Each microservice can be developed, deployed,
    and scaled independently while maintaining loose coupling.

    The architecture supports distributed agent systems with service discovery,
    load balancing, inter-service communication, circuit breakers, and observability
    across the microservices mesh.

Components:
    1. Agent Services: Independent agent microservices
    2. API Gateway: Entry point and routing
    3. Service Registry: Service discovery and health checks
    4. Message Bus: Async inter-service communication
    5. Load Balancer: Distribute requests across instances
    6. Circuit Breaker: Fault tolerance and isolation
    7. Config Service: Centralized configuration
    8. Monitoring: Distributed tracing and metrics

Key Features:
    - Independent service deployment
    - Service discovery and registration
    - Inter-service communication (REST, gRPC, message queues)
    - Load balancing and auto-scaling
    - Fault isolation and circuit breakers
    - Distributed tracing
    - API gateway pattern
    - Health checks and monitoring
    - Polyglot persistence
    - Technology diversity

Use Cases:
    - Large-scale AI platforms
    - Multi-tenant AI services
    - Enterprise AI systems
    - Cloud-native AI applications
    - Distributed agent coordination
    - Scalable chatbot platforms
    - AI-powered SaaS products
    - Microservices-based RAG systems
    - Agent mesh architectures
    - Multi-model serving platforms

LangChain Implementation:
    Uses FastAPI for service endpoints, service registry pattern,
    inter-service HTTP communication, and distributed agent coordination.
"""

    Purpose: This module implements the Microservice Agent pattern, providing a structured approach to agent design and implementation.

    Import Statements

    Standard Library Imports

    import collections.defaultdict
import dataclasses.dataclass
import dataclasses.field
import datetime.datetime
import enum.Enum
import json
import langchain_core.output_parsers.StrOutputParser
import langchain_core.prompts.ChatPromptTemplate
import os
import time
import typing.Any
import typing.Callable
import typing.Dict
import typing.List
import typing.Optional

    Purpose: Core Python functionality: – collections.defaultdict: Specialized container datatypes – dataclasses.dataclass: Simplified class definitions with @dataclass – dataclasses.field: Simplified class definitions with @dataclass – datetime.datetime: Date and time handling – enum.Enum: Enumeration support – json: JSON serialization/deserialization – langchain_core.output_parsers.StrOutputParser: Regular expression operations – langchain_core.prompts.ChatPromptTemplate: Regular expression operations – os: Environment variable access and system operations – time: Time-related functions – typing.Any: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Callable: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Dict: Type hints for better code documentation (List, Dict, Any, Optional) – typing.List: Type hints for better code documentation (List, Dict, Any, Optional) – typing.Optional: Type hints for better code documentation (List, Dict, Any, Optional)

    LangChain Imports

    from langchain_openai.ChatOpenAI

    Purpose: LangChain framework components: – LLM Interface: Wrapper for OpenAI models – Chains: Composable building blocks for complex workflows

    Other Imports

    from dotenv.load_dotenv
from uuid

    Purpose: Additional dependencies for specialized functionality.

    Classes

    Class: ServiceStatus

    Lines 72-78

    Inherits from: Enum

    Purpose:

    Status of a microservice

    Class: CommunicationProtocol

    Lines 81-86

    Inherits from: Enum

    Purpose:

    Inter-service communication protocol

    Class: ServiceInfo

    Lines 90-103

    Purpose:

    Information about a microservice

    Class: ServiceRequest

    Lines 107-115

    Purpose:

    Request to a microservice

    Class: ServiceResponse

    Lines 119-127

    Purpose:

    Response from a microservice

    Class: ServiceRegistry

    Lines 130-218

    Purpose:

    Service registry for microservice discovery.

Maintains a registry of available services with health monitoring.

    Methods:

    __init__(self)

    Lines 137-140

    Purpose:

    Initialize service registry

    Parameters:

    register(self, service)

    Lines 142-154

    Purpose:

    Register a service.

Args:
    service: Service information

Returns:
    True if registered successfully

    Parameters: – service: Service

    Returns: bool

    deregister(self, service_id)

    Lines 156-171

    Purpose:

    Deregister a service.

Args:
    service_id: Service ID

Returns:
    True if deregistered successfully

    Parameters: – service_id: Service id

    Returns: bool

    discover(self, service_name)

    Lines 173-191

    Purpose:

    Discover a healthy service instance.

Args:
    service_name: Name of service to discover

Returns:
    ServiceInfo if found, None otherwise

    Parameters: – service_name: Service name

    Returns: <ast.Subscript object at 0x000001A06D7FF2D0>

    health_check(self, service_id)

    Lines 193-209

    Purpose:

    Check health of a service.

Args:
    service_id: Service ID

Returns:
    Service status

    Parameters: – service_id: Service id

    Returns: ServiceStatus

    list_services(self)

    Lines 211-218

    Purpose:

    List all registered services.

Returns:
    List of service information

    Parameters:

    Returns: <ast.Subscript object at 0x000001A06D7FE710>

    Class: AgentService

    Lines 221-318

    Purpose:

    Base microservice for AI agents.

Each agent capability is encapsulated as an independent service.

    Methods:

    __init__(self, service_name, version, port)

    Lines 228-253

    Purpose:

    Initialize agent service.

Args:
    service_name: Name of the service
    version: Service version
    port: Port number

    Parameters: – service_name: Service name – version: Version – port: Port

    get_info(self)

    Lines 255-272

    Purpose:

    Get service information.

Returns:
    ServiceInfo object

    Parameters:

    Returns: ServiceInfo

    start(self)

    Lines 274-282

    Purpose:

    Start the service.

Returns:
    True if started successfully

    Parameters:

    Returns: bool

    stop(self)

    Lines 284-292

    Purpose:

    Stop the service.

Returns:
    True if stopped successfully

    Parameters:

    Returns: bool

    health(self)

    Lines 294-306

    Purpose:

    Health check endpoint.

Returns:
    Health status

    Parameters:

    Returns: <ast.Subscript object at 0x000001A06D7BB390>

    process(self, data)

    Lines 308-318

    Purpose:

    Process a request (to be overridden by subclasses).

Args:
    data: Request data

Returns:
    Response data

    Parameters: – data: Data

    Returns: <ast.Subscript object at 0x000001A06D7BADD0>

    Class: SummarizationService

    Lines 321-354

    Inherits from: AgentService

    Purpose:

    Microservice for text summarization

    Methods:

    __init__(self)

    Lines 324-325

    Purpose: Initialize the class instance with required parameters.

    Parameters:

    process(self, data)

    Lines 327-354

    Purpose:

    Summarize text.

Args:
    data: {'text': str, 'length': str}

Returns:
    {'summary': str}

    Parameters: – data: Data

    Returns: <ast.Subscript object at 0x000001A06D74A3D0>

    Class: TranslationService

    Lines 357-391

    Inherits from: AgentService

    Purpose:

    Microservice for translation

    Methods:

    __init__(self)

    Lines 360-361

    Purpose: Initialize the class instance with required parameters.

    Parameters:

    process(self, data)

    Lines 363-391

    Purpose:

    Translate text.

Args:
    data: {'text': str, 'target_language': str}

Returns:
    {'translation': str}

    Parameters: – data: Data

    Returns: <ast.Subscript object at 0x000001A06D72DB90>

    Class: SentimentService

    Lines 394-441

    Inherits from: AgentService

    Purpose:

    Microservice for sentiment analysis

    Methods:

    __init__(self)

    Lines 397-398

    Purpose: Initialize the class instance with required parameters.

    Parameters:

    process(self, data)

    Lines 400-441

    Purpose:

    Analyze sentiment.

Args:
    data: {'text': str}

Returns:
    {'sentiment': str, 'confidence': float}

    Parameters: – data: Data

    Returns: <ast.Subscript object at 0x000001A06D6061D0>

    Class: APIGateway

    Lines 444-534

    Purpose:

    API Gateway for routing requests to microservices.

Provides a single entry point and routes to appropriate services.

    Methods:

    __init__(self, registry)

    Lines 451-464

    Purpose:

    Initialize API gateway.

Args:
    registry: Service registry

    Parameters: – registry: Registry

    route(self, operation, data)

    Lines 466-534

    Purpose:

    Route request to appropriate service.

Args:
    operation: Operation to perform
    data: Request data

Returns:
    ServiceResponse

    Parameters: – operation: Operation – data: Data

    Returns: ServiceResponse

    Class: MicroserviceOrchestrator

    Lines 537-646

    Purpose:

    Orchestrator for coordinating multiple microservices.

Manages service lifecycle and coordinates complex workflows
across multiple services.

    Methods:

    __init__(self)

    Lines 545-549

    Purpose:

    Initialize orchestrator

    Parameters:

    deploy_service(self, service)

    Lines 551-571

    Purpose:

    Deploy a microservice.

Args:
    service: Agent service to deploy

Returns:
    True if deployed successfully

    Parameters: – service: Service

    Returns: bool

    undeploy_service(self, service_id)

    Lines 573-589

    Purpose:

    Undeploy a microservice.

Args:
    service_id: Service ID

Returns:
    True if undeployed successfully

    Parameters: – service_id: Service id

    Returns: bool

    execute_workflow(self, operations)

    Lines 591-613

    Purpose:

    Execute a workflow across multiple services.

Args:
    operations: List of operations with data

Returns:
    List of service responses

    Parameters: – operations: Operations

    Returns: <ast.Subscript object at 0x000001A06D7F4BD0>

    get_service_health(self)

    Lines 615-646

    Purpose:

    Get health status of all services.

Returns:
    Health status dictionary

    Parameters:

    Returns: <ast.Subscript object at 0x000001A06D739690>

    Functions

    demonstrate_microservice_architecture()

    Lines 649-864

    Purpose:

    Demonstrate the microservice agent architecture

    Key Concepts Summary

    Pattern Overview

    The Microservice Agent pattern provides a structured approach to building AI agents with specific capabilities and behaviors. This implementation leverages LangChain’s composable components to create a robust and maintainable solution.

    Core Components

    1. Agent Class: Main implementation of the pattern logic
    2. Prompt Engineering: Carefully crafted prompts to guide LLM behavior
    3. Tool Integration: Optional tools for extended functionality
    4. Memory Management: State handling across interactions
    5. Output Parsing: Structured response processing

    When to Use

    ✅ Good for: – Scenarios requiring this specific pattern’s strengths – Applications needing structured agent behavior – Systems where explainability is important – Production environments requiring reliability

    ❌ Not ideal for: – Simple single-prompt tasks – Scenarios requiring different pattern characteristics – Resource-constrained environments (if computationally intensive)

    Best Practices Demonstrated

    1. Environment Configuration

    • ✅ Use .env files for API keys
    • ✅ Never hardcode credentials
    • ✅ Validate environment variables on startup

    2. Error Handling

    • ✅ Graceful degradation on failures
    • ✅ Informative error messages
    • ✅ Logging for debugging

    3. Prompt Engineering

    • ✅ Clear, specific instructions
    • ✅ Structured output formats
    • ✅ Few-shot examples when beneficial

    4. Code Organization

    • ✅ Separation of concerns (class-based design)
    • ✅ Type hints for clarity
    • ✅ Docstrings for documentation

    5. Performance Optimization

    • ✅ Appropriate temperature settings
    • ✅ Token budget management
    • ✅ Caching where applicable

    LangChain Components Used

    • ChatOpenAI: OpenAI model interface
    • Chains: Sequential component composition

    Why LangChain? – Composable abstractions for LLM applications – Rich ecosystem of integrations – Production-ready patterns – Active community and development

    Execution Flow

    1. Initialize agent with configuration
   ↓
2. Load environment and dependencies
   ↓
3. Prepare prompt with input data
   ↓
4. Invoke LLM through LangChain
   ↓
5. Parse and validate response
   ↓
6. Return formatted result

    Key Decision Points: – Model selection (temperature, model type) – Prompt structure (few-shot vs zero-shot) – Error handling strategy – Output format validation

    Performance Considerations

    Computational Cost

    • LLM Calls: Primary cost factor (based on tokens)
    • Latency: Depends on model and input size
    • Memory: Minimal for most patterns

    Optimization Strategies

    1. Prompt Optimization: Minimize tokens while maintaining clarity
    2. Caching: Cache repeated queries/responses
    3. Model Selection: Use smaller models when appropriate
    4. Batch Processing: Group similar requests
    5. Streaming: For long responses, stream tokens

    Cost Estimates (Approximate)

    • GPT-3.5-turbo: ~$0.002 per request
    • GPT-4: ~$0.03 per request
    • GPT-4-turbo: ~$0.01 per request

    Actual costs vary based on input/output token counts

    Common Pitfalls to Avoid

    1. Prompt Design Issues

    ❌ Vague or ambiguous instructions
    ✅ Clear, specific, testable prompts

    2. Error Handling

    ❌ Ignoring API errors or rate limits
    ✅ Implement retry logic and graceful degradation

    3. Token Management

    ❌ Exceeding model token limits
    ✅ Truncate inputs, chunk processing

    4. Security

    ❌ Exposing API keys in code
    ✅ Use environment variables

    5. Testing

    ❌ Only testing happy paths
    ✅ Test edge cases, failures, malformed inputs

    6. Cost Management

    ❌ Unmonitored API usage
    ✅ Implement budgets, alerts, and monitoring

    Further Improvements

    Production Enhancements

    1. Monitoring: Add logging, metrics, tracing
    2. Validation: Input/output schema validation
    3. Rate Limiting: Implement request throttling
    4. Caching: Redis or in-memory cache for responses
    5. Async Support: Non-blocking operations for scalability

    Feature Extensions

    1. Multi-Model Support: Switch between providers
    2. Fallback Models: Graceful degradation strategy
    3. Custom Tools: Domain-specific function calling
    4. Advanced Memory: Vector stores for long-term memory
    5. Evaluation: Automated testing and benchmarking

    Code Quality

    1. Testing: Unit tests, integration tests
    2. Documentation: API docs, usage examples
    3. Type Safety: Stricter type hints
    4. Configuration: Externalized config files

    Related Patterns

    • Pattern 080
    • Pattern 082

    Pattern Families: – Reasoning Patterns (001-011) – Multi-Agent Patterns (012-019) – Tool Use Patterns (020-025) – Memory Patterns (026-031) – And many more…

    Summary

    The Microservice Agent pattern provides a powerful approach to building AI agents with specific capabilities. This implementation demonstrates:

    ✅ Key Strengths: – Clean, maintainable code structure – LangChain integration for composability – Type hints and documentation – Production-ready patterns

    🎯 Best Use Cases: – Applications requiring this pattern’s characteristics – Systems needing structured agent behavior – Production environments requiring reliability

    📚 Learning Resources: – LangChain Documentation: https://python.langchain.com/ – Pattern Documentation: See related patterns – OpenAI API Reference: https://platform.openai.com/docs/

    File: 081_microservice_agent.py
    Total Lines: 869
    Pattern Category: AI Agent Design Pattern #081

    Generated from actual implementation code
    For educational and reference purposes