Katkat
Back

Fix an HTTP Client That Loses a Stream Body on Redirect Replay

StripeSoftware Engineer · Onsite
MediumSoftware EngineerOnsiteSoftware Engineering Fundamentals

Last updated

Debug a small HTTP client library where POST requests with a plain string body replay correctly after a redirect, but requests with a file-like stream body hang, and fix the library so a redirect resends the exact original bytes, including bytes from a stream the caller had already partly consumed.

You're working on FeatherHTTP, a small internal HTTP client library. When a server answers a POST with a 307 Temporary Redirect, the client is supposed to resend the same request, same method, same body bytes, to the new location. Three tests exercise this. The one that posts a plain string passes. The two that post a seekable, file-like stream (io.BytesIO) as the body hang and eventually time out.

The library accepts three shapes of data for a POST: a plain string, a fresh io.BytesIO stream, and a stream the caller has already partly read from before handing it to the client — in that last case, only the remaining, unread bytes count as the body.

What the three cases must produce

data argumentCaller's state before the callBytes the server must receive on both the first attempt and the redirected one
"ping"n/aping
io.BytesIO(b"ping")stream position 0ping
io.BytesIO(b"pingpong") after .read(4)stream position 4pong

The failing tests

def test_redirect_replays_string_body(transport):
    result = post_through(transport, "/start", data="ping")
    assert result == b"ping"

def test_redirect_replays_fresh_stream_body(transport):
    result = post_through(transport, "/start", data=io.BytesIO(b"ping"))
    assert result == b"ping"

def test_redirect_replays_partially_read_stream_body(transport):
    stream = io.BytesIO(b"pingpong")
    stream.read(4)
    result = post_through(transport, "/start", data=stream)
    assert result == b"pong"

transport is a fake in-process server: posting to /start always answers with a 307 pointing at /target; posting to /target echoes back whatever body bytes it received, or raises a TimeoutError if the body came through empty (standing in for a real server hanging while it waits for bytes that a Content-Length header promised but that never arrive).

Your task

Figure out why the plain-string case works while both stream cases fail, and fix the library so all three tests pass, including the constraint that a stream the caller already partly read must replay only its remaining bytes, both on the first send and on the redirected one.

Hints

Hint 1

Trace exactly what object gets handed to the code that sends the redirected request. Is it the original `data`, or something already derived from it?

Hint 2

A string behaves differently than a stream here because reading a string twice gives you the same string, but reading a stream twice does not give you the same bytes.

Hint 3

Decide the one moment you need to remember a stream's starting read position, and make sure you capture it before anything reads from the stream, not after.

Hint 4

A stream without `seek`/`tell` can't be replayed this way at all — decide what the library should do when it's handed one.

Follow-up questions

  • How would this design need to change for a stream that can't be seeked at all, like data arriving live from a pipe?
  • If the redirect target uses a different HTTP method than the original request (a 303 rather than a 307), how does that interact with whether the body should be resent at all?
  • How would you add a size limit so an extremely large stream body doesn't get buffered entirely in memory just to make it replayable?
  • What test would you add to catch a future regression where someone reads from `self.raw` directly somewhere else in the library, bypassing `PreparedBody`?