Katkat

Traverse an HTTP Maze with Retry Handling

RampSoftware Engineer · Onsite

Last updated

This problem tests BFS traversal skills combined with state management for handling intermittent network failures and API constraints.

Problem Statement

You are given a web crawler interface that allows you to explore an HTTP-based maze. The goal is to find the shortest number of requests required to reach an "EXIT" URL from a "START" URL.

Your task is to implement the function findShortestPath(startUrl, httpClient).

The API

  • httpClient.get(url) returns a response object:
    • status: Integer status code.
    • links: A list of strings (neighbor URLs).
    • isExit: Boolean indicating if the URL is the exit.
  • Error Handling:
    • Status 200: Success. You receive the links.
    • Status 4xx: Client error. These are permanent; do not retry.
    • Status 5xx: Server error. You must retry up to 3 times for a specific URL before giving up on that path.

Constraints

  • Use Breadth-First Search (BFS) to find the shortest path.
  • Maintain a set of visited URLs to avoid cycles.
  • The function should return the minimum number of HTTP requests made to find the exit, or -1 if unreachable.

Example

# Interface
class HttpClient:
    def get(self, url: str) -> Response: pass

def findShortestPath(startUrl: str, httpClient: HttpClient) -> int:
    # Implementation here

Edge Cases

  • The maze contains cycles.
  • A URL returns a 500 error repeatedly, exceeding the 3-retry limit.
  • The startUrl is the exit.

Hints

Hint 1

Use a queue to store (url, distance) pairs, similar to standard BFS.

Hint 2

For the 5xx errors, store a separate counter or dictionary to track how many times you have attempted to request a specific URL before marking it as failed.

Hint 3

Ensure you only process a URL if it hasn't been successfully visited before, but allow retries for URLs that previously failed with 5xx.

Follow-up questions

  • How would you modify this algorithm if you needed to respect a rate limit (e.g., maximum 10 requests per second)?
  • What changes are needed if the goal is to find the path with the lowest cumulative latency, rather than the fewest hops?
  • If the maze is massive (billions of nodes), how would you optimize the memory usage of the visited set?