Design a Store Recommendation System and Explain ML Trade-offs

Scenario

Design the machine learning system powering the main store recommendation feed (e.g., "Stores Near You" or "Recommended For You") for a nationwide on-demand food delivery platform. The system should present an ordered list of merchants to users opening the app, maximizing user engagement and orders while balancing marketplace operational efficiency.

Requirements

Functional Requirements

• Provide a personalized ranking of merchant stores for each user session based on location, historical preferences, time of day, and contextual features.
• Support dynamic marketplace conditions such as store kitchen prep delays, delivery radius adjustments, and courier availability.
• Account for new stores (cold start) and merchant variety (avoiding feeds dominated by a single category or chain).

Non-Functional Requirements

• Scale: 30 million Daily Active Users (DAU), 500,000 active merchant stores nationwide, with peak query loads reaching 50,000 requests per second (RPS).
• Latency: End-to-end feed recommendation latency must be under 80 ms (P99).
• Freshness: User historical actions (e.g., recent clicks, search terms, cart additions) should reflect in recommendations within seconds to minutes.

What Your Design Should Cover

1. Objective Framing: Define the business goals and mathematical optimization objective (e.g., Click-Through Rate (CTR), Conversion Rate (CVR), Gross Merchandise Value (GMV), or multi-task learning formulation).
2. Architecture: Detail the multi-stage pipeline:
   • Candidate Generation (Retrieval): Geospatial filtering and candidate generation algorithms.
   • Scoring (Ranking): Model architecture, feature engineering (user, store, context, cross-features), and real-time inference strategy.
   • Re-ranking & Business Logic: Diversity, fulfillment constraints, fair exposure, and ad-sponsored blending.
3. Trade-off Analysis: Explicitly discuss technical trade-offs:
   • Real-time feature computation vs. precomputed batch embeddings.
   • Exploitation of historical top performers vs. exploration of new merchants.
   • Multi-task ranking (order probability vs. estimated delivery time) vs. separate cascading models.
4. Evaluation & Monitoring: Define offline evaluation metrics, online A/B testing strategy, and production risk mitigation (e.g., feedback loops, model drift, kitchen overload).