Logs + metrics + traces — structlog, OpenTelemetry, Prometheus, Sentry
All three correlated by trace_id + correlation_id.
structlog.configure(
processors=[
structlog.contextvars.merge_contextvars,
structlog.processors.add_log_level,
structlog.processors.TimeStamper(fmt="iso"),
structlog.processors.CallsiteParameterAdder([...]),
inject_otel_span_ids, # adds trace_id, span_id
structlog.processors.JSONRenderer(),
],
wrapper_class=structlog.make_filtering_bound_logger(logging.INFO),
)
Middleware reads/creates , binds to + OTel baggage. Every log line in the request gets it automatically.
X-Request-IDcontextvarsmodel_predictions_total{model, stage} — countermodel_prediction_latency_seconds{model} — histogramdrift_psi{feature} — gaugetraining_runs_total{status} — counter/health/live — process alive. Always 200 unless crashing./health/ready — dependencies reachable (DB, Redis, MLflow). Used by Koyeb.Distinct. Kubernetes/Koyeb needs both.
sentry-sdk[fastapi] with traces_sample_rate=0.1, release = git SHA.使用 Arthas 的 watch/trace 获取 EagleEye traceId / 获取请求的 traceId