Find the Shortest Click Path with a Fallible Link API
SnowflakeSoftware Engineer · Technical ScreenLast updated
Implement a breadth-first search to find the shortest URL click path between two pages using a fallible link-fetching API with retry and tie-breaking logic.
Problem Statement
You are building a link crawler to find the shortest navigation path from a start_url to a target_url. You are given an instance of a LinkFetcher interface that retrieves outgoing links for a given URL.
The link fetcher communicates with external web servers and is fallible: it can raise TransientFetchError due to network timeouts or rate limits.
Implement the function find_shortest_click_path:
class TransientFetchError(Exception):
pass
class LinkFetcher:
def get_links(self, url: str) -> list[str]:
"""Fetches outgoing links from url. May raise TransientFetchError."""
pass
def find_shortest_click_path(
start_url: str,
target_url: str,
fetcher: LinkFetcher,
max_retries: int = 3
) -> list[str]:
pass
Requirements
- Shortest Path: Return the path with the minimum number of clicks (edges). The result must be a list of URLs starting with
start_urland ending withtarget_url. - Tie-breaking: If multiple paths have the same minimum length, return the lexicographically smallest path (comparing URLs as strings from start to target).
- Transient Errors: If
fetcher.get_links(url)raisesTransientFetchError, retry the call up tomax_retriesadditional times (so at most1 + max_retriestotal attempts). If all attempts fail, treat the page as having 0 outgoing links. - Cycles and Redundancy: Avoid infinite loops caused by cyclic links. Do not call
get_linkson the same URL more than once across the entire search. - Unreachable / Edge Cases: If
start_url == target_url, return[start_url]without callingfetcher.get_links. If no valid path reachestarget_url, return[].
Constraints
- Number of reachable unique URLs <=
10,000 - URL length <=
256characters - `0 [/docs, /about]
/docs -> [/careers]
/about -> [/careers]
**Output:**
```python
["snowflake.com/home", "snowflake.com/about", "snowflake.com/careers"]
Explanation: Both /home -> /docs -> /careers and /home -> /about -> /careers have length 2 edges. Lexicographically, "snowflake.com/about" < "snowflake.com/docs", so the /about path is chosen.
Example 2
Input:
start_url = "a.com"
target_url = "b.com"
# fetcher.get_links("a.com") raises TransientFetchError twice, then returns ["b.com"]
max_retries = 2
Output:
["a.com", "b.com"]
Explanation: The first two attempts fail, but the third attempt (second retry) succeeds.
Hints
Hint 1
Use Breadth-First Search (BFS) to guarantee finding the path with the minimum number of clicks.
Hint 2
Sort outgoing links lexicographically before enqueuing them, and track predecessors with a dictionary to easily reconstruct the chosen path.
Hint 3
Encapsulate the retry loop in a helper function so network-handling logic is decoupled from BFS queue operations.
Follow-up questions
- How would you parallelize link fetching across multiple worker threads or coroutines using bounded concurrency while preserving BFS level-order semantics?
- If the graph were bidirectional and we knew incoming links as well, how would bidirectional BFS improve time complexity, and how would tie-breaking change?
- How would you design this crawler to be resumable across process restarts if searching large or slow graphs?
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.