Ai Systems Thinking

Patterns

AI Decision Framework

Can this be solved with rules/heuristics?
  → YES: Use deterministic logic. No AI needed.
  → NO: Is the task genuinely ambiguous?
    → YES: Use AI with validation + fallback.
    → NO: Reconsider the problem definition.

AI Component Isolation

Request → Preprocessor (deterministic)
        → AI Worker (LLM call, isolated)
        → Validator (schema check)
        → Postprocessor (deterministic)
        → Output

Model Abstraction Layer

type TextGenerator interface {
    Generate(ctx context.Context, prompt string, opts Options) (string, error)
}

// Implementations: OpenAIGenerator, AnthropicGenerator, LocalModelGenerator
// Swap at config level, not code level

Cost Budget Pattern

class AIBudget:
    def __init__(self, max_tokens_per_request: int, max_cost_per_day: float):
        self.max_tokens = max_tokens_per_request
        self.max_daily_cost = max_cost_per_day

    def check(self, estimated_tokens: int) -> bool:
        return (estimated_tokens <= self.max_tokens
                and self.daily_spend() < self.max_daily_cost)

Effectiveness Measurement

Track per AI component:
- Accuracy (validated output vs expected)
- Latency (p50, p95, p99)
- Cost per call (tokens × rate)
- Fallback rate (how often AI is bypassed)
- Error rate (validation failures)

Anti-Patterns

• AI for everything: Using LLMs for tasks solvable with a regex or lookup table
• Architecture around the model: Building the entire system assuming a specific AI model
• No fallback: System breaks completely when the AI provider is down
• Trust by default: Using AI outputs without validation
• Unmeasured AI: Shipping AI features with no accuracy or cost tracking
• AI as magic: Treating AI as an infallible oracle instead of a probabilistic tool
• Vendor lock-in: Hardcoding a single AI provider into the architecture
• Ignoring latency: Designing synchronous flows that depend on slow AI calls

Checklist

• AI is used only where deterministic logic cannot solve the problem
• Every AI component has a fallback/degradation path
• AI outputs are validated before use
• Model is swappable without architectural changes
• Cost and latency budgets are defined and monitored
• Effectiveness metrics are tracked per AI component
• AI failures are isolated (circuit breaker pattern)
• Human-in-the-loop option exists where appropriate