← Week 2: Custom Binary Protocols

Day 14: Challenge — Multiplexed Binary Protocol

Phase 3 · Jul 14, 2026

← Week 2: Custom Binary Protocols

Challenge Overview

Build a complete multiplexed binary KV protocol client and server.

Requirements:

  1. Protocol: 16-byte header (magic, version, opcode, request_id, flags, key_len, value_len)
  2. Codec: Encoder<KvMessage> + Decoder using tokio-util
  3. Server: handles GET/SET/DELETE concurrently; dispatches by request_id
  4. Client: concurrent requests; MultiplexedClient::get/set/delete return futures
  5. Authentication: HMAC-SHA256 over header + body; constant-time verify
  6. Test: 1,000 concurrent GET requests; verify all responses match expected values
← Week 2: Custom Binary Protocols

Header Layout

Offset  Size  Field
0       2     magic (0xCAFE)
2       1     version (1)
3       1     opcode (1=GET, 2=SET, 3=DEL, 128=OK, 129=ERR, 130=NOTFOUND)
4       4     request_id (u32)
8       1     flags (bit 0: compressed, bit 1: encrypted)
9       1     reserved
10      2     key_len
12      4     value_len
16      32    hmac_sha256
Total: 48 bytes header
Followed by: key (key_len bytes) + value (value_len bytes)
← Week 2: Custom Binary Protocols

Client API

let client = MultiplexedClient::connect("127.0.0.1:7777").await?;

// Concurrent operations
let (a, b, c) = tokio::join!(
    client.get("key-one"),
    client.set("key-two", b"value"),
    client.delete("key-three"),
);

// Throughput test
let mut handles = JoinSet::new();
for i in 0..1000 {
    let c = client.clone();
    handles.spawn(async move { c.get(&format!("k{}", i)).await });
}
while let Some(r) = handles.join_next().await {
    r.unwrap().unwrap();
}
← Week 2: Custom Binary Protocols

Implementation Checklist

  • [ ] KvMessage struct with all header fields + body
  • [ ] KvCodec implementing Encoder<KvMessage> + Decoder
  • [ ] Server with FramedRead/FramedWrite + request dispatch loop
  • [ ] MultiplexedClient with shared connection + DashMap<u32, Sender>
  • [ ] HMAC signing on encode; verification on decode
  • [ ] Sequence number counter for replay prevention
  • [ ] Tests: happy path, NOT_FOUND, concurrent 1000 requests, tampered message detection
← Week 2: Custom Binary Protocols

Week 2 Recap

Topic Key insight
Protocol design Length-prefix framing; header includes magic, version, opcode
tokio-util codec Decoder returns None until full frame available
Multiplexing Correlation ID + DashMap<id, Sender>
Versioning Additive changes only; reserve removed field numbers
Zero-copy bytes::Bytes, rkyv, FlatBuffers for hot paths
Authentication HMAC-SHA256 + sequence numbers; or use TLS

Next week: service mesh — how to manage all of this at scale without embedding it in every service.