Day 13: Idempotency Keys

Phase 4 · Aug 3, 2026

Agenda (2–3 hours)

• Read (45 min): Stripe "Idempotent Requests" API documentation; AWS SQS message deduplication documentation
• Study (45 min): Trace through a payment retry scenario: without idempotency keys vs with; what invariants does each approach maintain?
• Practice (45 min): Add idempotency key support to an Axum endpoint: store idempotency_key → response in a DashMap; return cached response on duplicate key
• Challenge (30 min): Design the idempotency key TTL policy. How long should you keep idempotency records? What are the tradeoffs?

The Retry Safety Problem

In distributed systems, delivery is often "at-least-once" — a request may be retried:

• Client times out, doesn't know if server received the request
• Network failure after server processed but before response was received
• Load balancer retry after server crash

For non-idempotent operations (create order, charge card):

• Without deduplication: duplicate orders, double charges
• With idempotency keys: second attempt returns the same result as the first

Idempotency Key Protocol

1. Client generates a UUID for each unique operation (idempotency-key: uuid-v4)
2. Server checks if this key was seen before
   • If not seen: execute operation, store (key → response), return response
   • If seen, in progress: return 409 Conflict (or wait)
   • If seen, completed: return the stored response directly
3. Client retries with the same key on failure → gets identical result

Server-Side Implementation

async fn create_order(
    State(state): State<AppState>,
    TypedHeader(idempotency_key): TypedHeader<IdempotencyKey>,
    Json(body): Json<CreateOrderRequest>,
) -> Result<Response, AppError> {
    let key = idempotency_key.0;

    // Check cache
    if let Some(cached) = state.idempotency_cache.get(&key) {
        return Ok(cached.into_response());
    }

    // Mark as in-progress (prevents concurrent duplicate)
    state.idempotency_cache.insert(key.clone(), CachedResponse::InProgress);

    // Execute
    let response = create_order_internal(&state, body).await?;
    let cached = CachedResponse::Complete(response.clone());
    state.idempotency_cache.insert(key, cached);
    Ok(response.into_response())
}

SQS Message Deduplication

SQS FIFO queues have built-in deduplication:

let result = sqs.send_message()
    .queue_url(queue_url)
    .message_body(serde_json::to_string(&order)?)
    .message_deduplication_id(order_id.to_string()) // unique per business event
    .message_group_id("orders")                      // FIFO ordering group
    .send()
    .await?;

SQS deduplicates within a 5-minute window. If the same MessageDeduplicationId is sent twice within 5 minutes, only one message is delivered.

Key Takeaways

• Idempotency keys transform at-least-once delivery into effectively-exactly-once
• Server stores (key → response) atomically with the operation (or as a separate step with a lock)
• TTL policy: keep idempotency records for the maximum expected retry window (usually 24–48 hours)
• SQS FIFO provides built-in message deduplication for event-driven architectures

Tomorrow: Phase 4 Week 2 Challenge — saga with compensation and idempotent handlers.