Distributed tracing with OpenTelemetry — spans, context propagation, sampling. Use when instrumenting a distributed system, debugging requests that span services, setting up OpenTelemetry, choosing a sampling strategy, or understanding latency across service boundaries.
Distributed tracing follows a single request through every service it touches, exposing latency and error contributions per hop. OpenTelemetry is the vendor-neutral standard. Start with auto-instrumentation, propagate W3C Trace Context, and sample to control cost.
A trace is the entire journey of a request, identified by a 128-bit
trace_id. A span is one unit of work inside a trace with a name,
start time, duration, status, and attributes; spans form a tree where
every non-root span has a parent. Context carries trace_id,
span_id, and trace_flags across process boundaries via HTTP
headers, message metadata, or gRPC metadata. Baggage is small
key-value data propagated alongside context for cross-cutting concerns
like tenant ID — keep it tiny because every outgoing request carries it.
OpenTelemetry (OTel) is the standard. The is the vendor-neutral interface your code instruments against. The is the in-process library that creates and exports spans. ship spans to backends (Jaeger, Zipkin, Tempo, Datadog). The is an optional proxy that decouples apps from backends and is recommended for production. libraries create spans for common HTTP, gRPC, database, and messaging frameworks with zero code changes.
<component>.<operation> span names: OrderService.createOrderorder.idContext propagation is what stitches spans across service boundaries.
Use W3C Trace Context (the OTel default) with traceparent and
tracestate headers — example:
traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01.
B3 propagation (Zipkin's X-B3-* headers) is still supported. For
message queues, propagate context in message headers or metadata, not
in the body. For async work, pass context explicitly when spawning
background tasks — it does not flow automatically across thread or
goroutine boundaries in most languages.
In high-traffic systems, tracing every request is expensive. Pick a