Model Monitoring

4. Set Retraining Triggers

Define trigger strategy:

• Schedule: retrain weekly/monthly regardless of drift
• Performance-based: retrain when accuracy drops below threshold
• Drift-based: retrain when PSI > 0.25 on key features
• Combine: schedule + drift detection for critical models

5. Dashboard Essentials

• Prediction volume over time
• Feature distributions (current vs baseline)
• Model performance metrics (rolling 7-day)
• Latency P50/P95/P99
• Error rates (failed inferences)

Key Outputs

• Monitoring pipeline (drift detection + alerting)
• Model health dashboard
• Retraining trigger policy
• Runbook for drift alerts

Anti-Patterns

• No monitoring after deployment ("set and forget")
• Alerting on every metric — pick 3–5 critical signals
• No ground truth pipeline — can't measure real accuracy
• Retraining without validating the new model first