Katkat
Back

When to automate operations work, and good-enough versus excellent automation

MicrosoftSite Reliability Engineer · Onsite
MediumSite Reliability EngineerOnsiteSoftware Engineering Fundamentals

Last updated

Reliability engineering discussion question: explain your philosophy on automating operational work to free engineers for development, your bar for good-enough versus excellent automation, and where automation stops being worth it. It tests toil measurement, cost-benefit reasoning, safety guardrails and concrete examples.

In a senior reliability engineering loop, the interviewer asks how you think about automating the operation of a system so that engineers win back time for development work. They then push on two points: what separates automation that is merely good enough from automation that is excellent, and how you decide that something is not worth automating at all. There is no system to draw; the interviewer wants a principled position backed by concrete examples.

Clarifying Questions

  • Should I focus on operational work for a live service (deployments, remediation, capacity changes), or include developer workflow tooling as well?
  • How large is the team that would build and then maintain the automation?
  • Is there existing data on where on-call and operational time goes?

Part 1 — Your automation philosophy

Explain how you choose what to automate first, how you build automation that is safe to run against production, and how you confirm that it actually gave engineers time back.

What This Part Should Cover

  • How operational toil is identified and measured
  • Prioritization by frequency, effort and risk
  • Safety properties for automation that changes production
  • Evidence that time was freed, not just moved elsewhere

Part 2 — Good enough, excellent, and where to stop

Describe your bar for good-enough versus excellent automation, and where you draw the line on what is worth automating.

What This Part Should Cover

  • Explicit criteria separating good-enough from excellent
  • A cost-benefit test for whether to automate at all
  • Cases where removing the work beats automating it
  • Concrete examples on both sides of the line

Hints

Hint 1

Measure before you build: Think about how you would find out which manual work costs the most, and how automation itself can become a new source of incidents.

Hint 2

Let the blast radius decide: Consider how often a task runs and how much damage a mistake can do, and how those two things should change the level of polish you require.

Follow-up questions

  • An automation you built caused an incident. What do you do next?
  • How do you keep on-call engineers able to act manually when most remediation is automated?
  • How would you convince a product-focused team to spend a sprint removing toil?