Katkat

Assemble async stream reads into objects

MicrosoftSoftware Engineer · Onsite

Last updated

This problem tests your ability to handle partial reads from an asynchronous stream and correctly parse length-prefixed binary data structures.

Problem Statement

You are building a streaming protocol handler where data arrives as a sequence of length-prefixed binary packets. Each packet consists of a 4-byte header representing the length of the payload as an unsigned 32-bit integer (big-endian), followed by exactly that many bytes of payload data.

Because the stream is asynchronous, you may receive partial packets, or multiple packets may be coalesced into a single read operation. You need to implement a class that consumes an IAsyncEnumerable> and yields fully formed byte[] objects representing the payloads.

Interface

public class PacketAssembler
{
    public async IAsyncEnumerable AssemblePacketsAsync(IAsyncEnumerable> stream)
    {
        // Implement this
    }
}

Constraints

  • The header length is always 4 bytes.
  • Maximum payload size is 10 MB.
  • The stream may close (EOF) at any time. If the stream ends while a packet is incomplete, you should throw an EndOfStreamException.

Examples

Input: Stream: [0, 0, 0, 2, 0xAA, 0xBB, 0, 0, 0, 1, 0xCC] Output: [0xAA, 0xBB], [0xCC] Explanation: The first 4 bytes indicate a 2-byte payload. The next 2 bytes are consumed as the payload. The following 4 bytes indicate a 1-byte payload, which is then consumed.

Edge Cases

  • A single ReadOnlyMemory contains only a partial header (e.g., 2 bytes of the 4-byte header).
  • A single ReadOnlyMemory contains multiple full packets.
  • A payload is spread across three different ReadOnlyMemory chunks.

Hints

Hint 1

Maintain an internal buffer (such as a MemoryStream or a List) to store bytes that haven't formed a complete packet yet.

Hint 2

First, attempt to read enough bytes to fill the 4-byte header, then use the length from that header to determine exactly how many more bytes are required for the payload.

Hint 3

Use a loop to continuously process your internal buffer, only yielding a result when the buffer size is greater than or equal to the expected packet size.

Follow-up questions

  • How would you optimize the memory usage to avoid excessive array allocations if you are processing high-throughput streams?
  • What changes would be necessary if the header size was variable or if the length was little-endian instead of big-endian?
  • How would you handle a scenario where a malicious sender sends an extremely large length value (e.g., 4GB) to trigger an OutOfMemoryException?