Llm Guardrails Design

Output Schema Validation

def validate_output(response: str, schema: Schema) -> Result:
    parsed = parse_response(response)
    if not schema.validate(parsed):
        return fallback_result()
    return parsed

Retry with Degradation

Attempt 1: Full prompt → validate response
Attempt 2: Simplified prompt → validate response
Attempt 3: Fallback to cached/default response

Circuit Breaker

if error_rate > THRESHOLD:
    return cached_fallback()
# Else proceed with LLM call

Anti-Patterns

• Unvalidated outputs: Using LLM responses directly without schema check
• No fallback: System crashes when LLM returns unexpected output
• Prompt injection vulnerability: User input embedded in prompts without sanitization
• Unbounded token usage: No limits on prompt or response size
• Direct code execution: Running LLM-generated code without sandboxing
• No rate limiting: Allowing unbounded API calls

Checklist

• Input validation before every LLM call
• Output schema validation after every LLM response
• Fallback behavior defined for all LLM integration points
• Token budgets enforced
• Sensitive data stripped from prompts
• Rate limiting applied
• No direct execution of LLM-generated code
• Circuit breaker for high error rates