Katkat
Back

Design a Code Review Agent

MetaSoftware Engineer · Onsite
MediumSoftware EngineerOnsiteML System Design

Last updated

This question evaluates a candidate's ability to design ML-powered developer tooling, covering competencies in LLM orchestration, deterministic tool integration, repository context management, code-analysis heuristics, safety and validation strategies, and inference cost estimation.

Design an AI-powered code review agent that assists developers by reviewing pull requests and producing actionable feedback.

The agent should be able to:

  • Inspect code changes in a pull request.
  • Understand repository context when necessary.
  • Identify bugs, security risks, style issues, maintainability problems, and missing tests.
  • Produce comments that are specific, actionable, and grounded in the code.
  • Decide which parts of the workflow require an LLM and which parts can be handled by deterministic tools.

Deep-dive areas:

  1. Which components need an LLM, and which should not use an LLM?
  2. How would you validate the quality and safety of the agent's generated review comments?
  3. How would the orchestrator work?
  4. What memory or repository context should the system maintain?
  5. What tools should the agent be able to use?
  6. How would you estimate and control the cost of each LLM inference?