Katkat
Back

Design an online coding judge

MetaSoftware Engineer · Onsite
HardSoftware EngineerOnsiteSystem Design

Last updated

This problem tests the ability to architect a distributed, multi-tenant system for sandboxed execution of untrusted code, emphasizing security, scalability, and performance isolation.

Problem: Designing an Online Coding Judge

Design a scalable system that allows users to submit code (Python, Java, C++) to be executed against hidden test cases. The system must handle spikes in traffic during competitive programming events, ensure fair resource allocation, and maintain complete isolation between submissions.

Core Requirements

  • Submission Lifecycle: Accept code submissions, queue them, execute them in a secure environment, and return results (Passed, Failed, TLE, MLE, Runtime Error).
  • Sandbox Security: Prevent submissions from accessing the host file system, network, or other user processes.
  • Resource Management: Enforce strict CPU and memory limits per execution. Prevent one user's heavy workload from impacting the system's global availability.
  • Asynchronous Processing: Support high-throughput processing where results are pushed to the client via WebSockets or long-polling.
  • Test Data Versioning: Manage multiple versions of test cases and allow for efficient re-judging when a test case is updated.

Non-Functional Requirements

  • Scale: Handle 10,000 requests per minute with an average execution time of 2 seconds.
  • Availability: The system should remain operational even if a single worker node crashes while executing a malicious submission.
  • Latency: Provide feedback to the user within 5 seconds for 95% of submissions.

What to Cover

  1. API Design: Define the endpoints for submitting code and polling/receiving results.
  2. Data Model: Design the database schema for Users, Submissions, Problems, and Test Cases.
  3. System Architecture: Use a diagrammatic approach to explain the flow from Load Balancer to the Sandbox Worker.
  4. Sandboxing: Discuss how to isolate execution (e.g., cgroups, namespaces, gVisor, or Firecracker microVMs).
  5. Anti-Cheat & Fairness: How to prevent code from probing the environment or exhausting resources.

Hints

Hint 1

Consider using a message queue like Kafka or RabbitMQ to decouple the submission API from the worker fleet to handle traffic bursts.

Hint 2

For sandboxing, explore containerization technologies that provide strong kernel-level isolation rather than just standard Docker containers.

Hint 3

Think about the caching strategy for the 'compilation' step, as many submissions may be identical or re-runs of the same code.

Follow-up questions

  • How would you handle a 'Distributed Denial of Service' attack where a user submits millions of submissions that intentionally trigger a timeout?
  • If you needed to support language-specific libraries (e.g., importing heavy ML packages), how would you optimize the startup time for your worker nodes?
  • Describe how you would design a 're-judge' mechanism that updates submission statuses if a bug is found in the hidden test case logic.