Reserve Cart Inventory Without Overselling

Scenario

In an on-demand grocery delivery platform, physical grocery stores have limited stock for thousands of SKUs. When a customer adds a scarce item (e.g., promotional produce or seasonal goods) to their shopping cart, the platform must guarantee the item's availability until checkout completes or a timeout expires. If multiple shoppers attempt to reserve the same last units concurrently, the system must strictly prevent overselling without degrading checkout latency or leaving items indefinitely locked when carts are abandoned.

Requirements

Functional Requirements

• Reserve Inventory: Attempt to reserve k units of a SKU for a user cart for a configurable duration (e.g., 15 minutes).
• Release / Reclaim: Automatically return reserved items to available inventory if the cart expires without purchase, or explicitly release items if a customer removes them from the cart.
• Commit / Fulfill: Transition reserved inventory into a permanently deducted state once payment succeeds at checkout.
• Inventory Visibility: Provide real-time available stock counts for catalog browsing and cart views.

Non-Functional Requirements

• Strict Consistency: No overselling under any concurrency level (zero phantom reservations).
• High Availability & Low Latency: Reservation and read operations must execute with p99 latency < 50ms.
• Scalability: Support current baseline traffic and gracefully scale to handle a 10x surge during peak holiday promotions.
• Fault Tolerance: Worker or node crashes must not result in permanently leaked (orphaned) reservations.

Scale & Traffic Assumptions

• Catalog: 50 million total SKUs across 20,000 store locations.
• Daily Active Shoppers: 10 million users.
• Write Operations (Add to Cart / Reserve): 20,000 requests per second (RPS) baseline, surging to 200,000 RPS during 10x promotional peaks.
• Read Operations (Catalog stock checks): 100,000 RPS baseline, surging to 1,000,000 RPS.

Key Discussion Areas

• Data Model & Storage Strategy: Contrast relational databases (e.g., row-level locks, conditional updates) vs. in-memory key-value stores (e.g., Redis Lua scripts) for the reservation ledger.
• Concurrency Control: Mechanisms to eliminate race conditions on hot SKUs (e.g., single-item inventory contention across hundreds of concurrent carts).
• Expiration & Cleanup Architecture: How to reliably expire reservations at scale (e.g., delayed message queues, TTL key events, sweeping jobs) without stampeding the primary database.
• Failure Modes & Reconciliation: Strategies to recover from payment processor timeouts, cart service crashes, and split-brain scenarios.