Day 8: Binary Protocol Design

Phase 3 · Jul 8, 2026

Agenda (2–3 hours)

• Read (45 min): Redis protocol specification (RESP3); PostgreSQL wire protocol frontend/backend overview; AMQP 0-9-1 frame structure
• Study (45 min): Compare length-prefix framing vs delimiter framing; analyze failure modes of each
• Practice (45 min): Design a binary protocol for a simple key-value store: header (magic, version, opcode, key_len, value_len), payload
• Challenge (30 min): How does the PostgreSQL wire protocol handle multiplexing? How does it differ from HTTP/2?

Why Custom Binary Protocols?

gRPC/protobuf adds ~2KB of overhead per connection (HTTP/2 SETTINGS, headers). For:

• Very low-latency loops (trading systems, game servers)
• Simple request-response with known message shapes
• Embedded/constrained environments
• Connection-oriented session state (databases, caches)

A custom binary protocol may be 10–100× simpler than gRPC.

Framing Strategies

Length-prefix framing (most common):

[4-byte length][payload bytes]

• Pro: O(1) to read: read 4 bytes → allocate buffer → read length bytes
• Con: Can't stream; must buffer entire payload before parsing

Delimiter framing (e.g., RESP, HTTP/1.1 headers):

..payload bytes..\r\n

• Pro: streaming-friendly; simple text protocols
• Con: delimiter must be escaped in payload; scanning required

Fixed-size frames (e.g., AMQP, ATM):

• Pro: no parsing; trivially parallel
• Con: wastes space for variable-length data

Designing a KV Protocol

Header (12 bytes):
  magic:     2 bytes  (0xCAFE — sanity check)
  version:   1 byte   (protocol version)
  opcode:    1 byte   (GET=1, SET=2, DELETE=3, RESP_OK=128, RESP_ERR=129)
  flags:     2 bytes  (compression, encryption, etc.)
  key_len:   2 bytes
  value_len: 4 bytes

Body:
  key:       key_len bytes
  value:     value_len bytes

Rules:

1. Always send the full header atomically (avoids partial-read confusion)
2. value_len = 0 for GET and DELETE requests
3. Error responses set opcode = RESP_ERR and encode error string in value field

Protocol State Machine

Each connection is in one of:

• Idle: waiting for next request header
• Reading header: accumulating 12 header bytes
• Reading body: accumulating key + value
• Processing: handler has the full request
• Writing response: sending response bytes

State must be explicitly tracked — partial reads are normal in async I/O.

Key Takeaways

• Choose framing based on your latency, streaming, and complexity tradeoffs
• Length-prefix framing is the default for most new binary protocols
• The header must include a version field and magic number from day one
• State machine is explicit: track how many bytes have been read in each phase

Tomorrow: tokio-util Codec — implementing the state machine with Encoder/Decoder traits.