Design web crawler for 1000 devices
LyftSoftware Engineer · Onsite
Last updated
This interview question evaluates requirements, scale assumptions, API/data design, architecture, trade-offs, failure modes, and rollout in a realistic interview setting. A strong answer for Design web crawler for 1000 devices states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.
Question
Design a web crawler that starts from a single link and distributes crawling across 1,000 different devices; address coordination, load balancing, fault tolerance, and scalability follow-ups
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.