Katkat
Back

Explain and Diagnose a Segmentation Fault

Hudson River TradingSoftware Engineer · Technical Screen
MediumSoftware EngineerTechnical ScreenSoftware Engineering Fundamentals

Last updated

This question tests your understanding of virtual memory, the lifecycle of a segmentation fault, and your ability to diagnose memory corruption using standard tooling.

The Segmentation Fault Scenario

You are working on a high-frequency trading engine written in C++. During a performance regression test, the process crashes with a segmentation fault (SIGSEGV). The system logs indicate the crash occurred inside a standard library std::vector::push_back call, but your internal telemetry shows the memory address being accessed by the vector’s internal pointer is valid and properly aligned.

Your Tasks

  1. Theoretical Understanding: Explain the mechanism by which the CPU generates a segmentation fault via the Memory Management Unit (MMU) and the OS kernel. Why does the faulting instruction pointer often point to a different location than the line of code that caused the memory corruption?
  2. Diagnostic Workflow: You have access to the core dump, the binary (with debug symbols), and a Linux environment. Outline the specific steps you would take to investigate this crash. Mention the utility of tools like gdb, addr2line, and AddressSanitizer (ASan).
  3. Practical Debugging: If the crash is non-deterministic (a "heisenbug"), what strategies would you employ to force the error to trigger consistently? Provide an example of how a buffer overflow might lead to a crash far away from the site of the actual overwrite.

Expectations

Your response should demonstrate:

  • Knowledge of virtual address space, page tables, and memory protection.
  • Proficiency in leveraging diagnostic tooling to bridge the gap between binary-level faults and source-code bugs.
  • An understanding of heap corruption (e.g., writing past a buffer and overwriting adjacent metadata or pointers).

Hints

Hint 1

Consider the relationship between the MMU and the page table entries when the CPU attempts to access a virtual address that has not been mapped or has insufficient permissions.

Hint 2

Distinguish between "wild pointers" (dereferencing a garbage address) and "memory corruption" (overwriting heap headers or unrelated variables that only lead to a crash later).

Hint 3

Think about why compiler-level sanitizers are more effective than manual debugger inspection for tracking down memory corruption bugs.

Follow-up questions

  • How does the OS translate a virtual memory access violation into a signal delivered to the user-space process?
  • If you were tasked with running this in a production environment where you cannot afford the performance overhead of AddressSanitizer, what alternative monitoring tools would you implement?
  • Explain the role of the stack frame and how buffer overflows can be used to execute arbitrary code (the principle behind stack smashing).