Katkat

Find the Shortest Click Path with a Fallible Link API

SnowflakeSoftware Engineer · Technical Screen

Last updated

Implement a breadth-first search to find the shortest URL click path between two pages using a fallible link-fetching API with retry and tie-breaking logic.

Problem Statement

You are building a link crawler to find the shortest navigation path from a start_url to a target_url. You are given an instance of a LinkFetcher interface that retrieves outgoing links for a given URL.

The link fetcher communicates with external web servers and is fallible: it can raise TransientFetchError due to network timeouts or rate limits.

Implement the function find_shortest_click_path:

class TransientFetchError(Exception):
    pass

class LinkFetcher:
    def get_links(self, url: str) -> list[str]:
        """Fetches outgoing links from url. May raise TransientFetchError."""
        pass

def find_shortest_click_path(
    start_url: str,
    target_url: str,
    fetcher: LinkFetcher,
    max_retries: int = 3
) -> list[str]:
    pass

Requirements

  1. Shortest Path: Return the path with the minimum number of clicks (edges). The result must be a list of URLs starting with start_url and ending with target_url.
  2. Tie-breaking: If multiple paths have the same minimum length, return the lexicographically smallest path (comparing URLs as strings from start to target).
  3. Transient Errors: If fetcher.get_links(url) raises TransientFetchError, retry the call up to max_retries additional times (so at most 1 + max_retries total attempts). If all attempts fail, treat the page as having 0 outgoing links.
  4. Cycles and Redundancy: Avoid infinite loops caused by cyclic links. Do not call get_links on the same URL more than once across the entire search.
  5. Unreachable / Edge Cases: If start_url == target_url, return [start_url] without calling fetcher.get_links. If no valid path reaches target_url, return [].

Constraints

  • Number of reachable unique URLs <= 10,000
  • URL length <= 256 characters
  • `0 [/docs, /about]

/docs -> [/careers]

/about -> [/careers]

**Output:**
```python
["snowflake.com/home", "snowflake.com/about", "snowflake.com/careers"]

Explanation: Both /home -> /docs -> /careers and /home -> /about -> /careers have length 2 edges. Lexicographically, "snowflake.com/about" < "snowflake.com/docs", so the /about path is chosen.

Example 2

Input:

start_url = "a.com"
target_url = "b.com"
# fetcher.get_links("a.com") raises TransientFetchError twice, then returns ["b.com"]
max_retries = 2

Output:

["a.com", "b.com"]

Explanation: The first two attempts fail, but the third attempt (second retry) succeeds.

Hints

Hint 1

Use Breadth-First Search (BFS) to guarantee finding the path with the minimum number of clicks.

Hint 2

Sort outgoing links lexicographically before enqueuing them, and track predecessors with a dictionary to easily reconstruct the chosen path.

Hint 3

Encapsulate the retry loop in a helper function so network-handling logic is decoupled from BFS queue operations.

Follow-up questions

  • How would you parallelize link fetching across multiple worker threads or coroutines using bounded concurrency while preserving BFS level-order semantics?
  • If the graph were bidirectional and we knew incoming links as well, how would bidirectional BFS improve time complexity, and how would tie-breaking change?
  • How would you design this crawler to be resumable across process restarts if searching large or slow graphs?