Day 2: Tokio Runtime — Executor and Work-Stealing Scheduler

Phase 2 · Jun 11, 2026

Agenda (2–3 hours)

• Read (45 min): Tokio tutorial; Tokio's "Under the Hood" blog post; tokio::runtime documentation
• Study (45 min): Trace the lifecycle of tokio::spawn from task creation to completion; understand how work-stealing reduces idle threads
• Practice (45 min): Benchmark: 10,000 tasks that each sleep 1ms — compare multi-thread vs current-thread runtime; measure throughput
• Challenge (30 min): What happens when a Tokio task panics? How does JoinHandle::await propagate it? How does tokio::spawn_blocking fit in?

Tokio Runtime Components

┌─────────────────────────────────────────┐
│               Tokio Runtime              │
│  ┌──────────┐  ┌────────────────────┐   │
│  │ I/O       │  │ Scheduler           │   │
│  │ Reactor   │  │ (Work-stealing)     │   │
│  │ (epoll/   │  │                     │   │
│  │  kqueue)  │  │ Thread 1 | Thread 2 │   │
│  └──────────┘  └────────────────────┘   │
│  ┌──────────┐  ┌────────────────────┐   │
│  │ Timer     │  │ Blocking Thread    │   │
│  │ Wheel     │  │ Pool               │   │
│  └──────────┘  └────────────────────┘   │
└─────────────────────────────────────────┘

Work-Stealing Scheduler

Each worker thread has a local task queue. When empty, it steals from another thread's queue.

Benefits:

• No global lock on the task queue
• Cache locality: tasks run on the same thread they were spawned (usually)
• Load balancing: busy threads donate work to idle ones

Tokio's multi-thread runtime uses num_cpus worker threads by default.

let rt = tokio::runtime::Builder::new_multi_thread()
    .worker_threads(4)
    .enable_all()
    .build()?;

Spawning Tasks

// Spawn a non-blocking async task
let handle = tokio::spawn(async {
    do_async_work().await;
    42
});
let result = handle.await?; // JoinHandle<i32>

// Run blocking code without blocking a worker thread
let result = tokio::task::spawn_blocking(|| {
    // CPU-heavy or blocking I/O
    std::fs::read_to_string("/etc/hosts")
}).await?;

// Spawn on the current thread (no Send required)
tokio::task::spawn_local(async { /* ! Send */ });

I/O Reactor

Tokio wraps epoll (Linux) / kqueue (macOS) / IOCP (Windows) via the mio crate.

When a future awaits on a TcpStream read:

1. The stream registers interest with the reactor (edge-triggered)
2. The task stores its Waker and returns Poll::Pending
3. When the OS reports readiness, the reactor wakes the task's Waker
4. The runtime schedules the task on a worker thread

This is the entire I/O async machinery — all built on OS event notification.

Key Takeaways

• Tokio provides: executor (work-stealing scheduler), I/O reactor (mio/epoll), timer wheel, blocking thread pool
• Work-stealing avoids global lock contention and automatically load-balances
• spawn_blocking is essential for CPU-heavy or truly blocking operations — don't block worker threads
• One TcpListener can accept 100,000+ connections on a few worker threads

Tomorrow: Tokio I/O — TcpListener, TcpStream, AsyncRead, AsyncWrite.