Day 2: OpenTelemetry Rust SDK

Phase 6 · Sep 3, 2026

Agenda (2–3 hours)

• Read (45 min): opentelemetry and opentelemetry-otlp crate documentation; tracing-opentelemetry integration guide
• Study (45 min): What is the role of the OpenTelemetry Collector? Why not export directly from the application to Jaeger?
• Practice (45 min): Wire up opentelemetry-otlp exporter in a Tokio app; export spans to a local OTel Collector; verify spans appear in Jaeger UI
• Challenge (30 min): An OTLP exporter fails silently when the collector is down. Design a pattern to detect and alert on exporter failures without affecting the application's critical path

OpenTelemetry Architecture

Application (Rust)
  └── tracing-opentelemetry layer
        └── OpenTelemetry SDK
              └── OTLP Exporter (gRPC or HTTP)
                    └── OTel Collector (Docker sidecar / DaemonSet)
                          ├── Jaeger / Grafana Tempo
                          └── CloudWatch X-Ray (AWS backend)

The Collector buffers, batches, and retries — the app never blocks on exporter latency.

SDK Setup

[dependencies]
opentelemetry = { version = "0.22", features = ["trace"] }
opentelemetry-otlp = { version = "0.15", features = ["grpc-tonic"] }
opentelemetry_sdk = { version = "0.22", features = ["rt-tokio"] }
tracing-opentelemetry = "0.23"
tracing-subscriber = { version = "0.3", features = ["env-filter"] }

use opentelemetry_otlp::WithExportConfig;

let tracer = opentelemetry_otlp::new_pipeline()
    .tracing()
    .with_exporter(
        opentelemetry_otlp::new_exporter()
            .tonic()
            .with_endpoint("http://localhost:4317")
    )
    .with_trace_config(
        opentelemetry_sdk::trace::Config::default()
            .with_resource(Resource::new(vec![
                KeyValue::new("service.name", "task-service"),
            ]))
    )
    .install_batch(opentelemetry_sdk::runtime::Tokio)?;

Connecting to tracing

use tracing_subscriber::{layer::SubscriberExt, util::SubscriberInitExt};

tracing_subscriber::registry()
    .with(tracing_opentelemetry::layer().with_tracer(tracer))
    .with(tracing_subscriber::fmt::layer())
    .with(tracing_subscriber::EnvFilter::from_default_env())
    .init();

// Now all #[instrument] spans are exported via OTLP

Shutdown: call opentelemetry::global::shutdown_tracer_provider() on exit to flush the batch.

Key Takeaways

• tracing-opentelemetry bridges the tracing subscriber API to the OTel SDK
• The OTel Collector decouples the app from the backend — swap Jaeger for Tempo without code changes
• Batch processors are non-blocking; the app never waits for span export
• Always call shutdown_tracer_provider() at exit to flush buffered spans

Tomorrow: Jaeger and Grafana Tempo — querying traces and reading flame graphs.