Assemble async stream reads into objects
MicrosoftSoftware Engineer · OnsiteLast updated
This problem tests your ability to handle partial reads from an asynchronous stream and correctly parse length-prefixed binary data structures.
Problem Statement
You are building a streaming protocol handler where data arrives as a sequence of length-prefixed binary packets. Each packet consists of a 4-byte header representing the length of the payload as an unsigned 32-bit integer (big-endian), followed by exactly that many bytes of payload data.
Because the stream is asynchronous, you may receive partial packets, or multiple packets may be coalesced into a single read operation. You need to implement a class that consumes an IAsyncEnumerable> and yields fully formed byte[] objects representing the payloads.
Interface
public class PacketAssembler
{
public async IAsyncEnumerable AssemblePacketsAsync(IAsyncEnumerable> stream)
{
// Implement this
}
}
Constraints
- The header length is always 4 bytes.
- Maximum payload size is 10 MB.
- The stream may close (EOF) at any time. If the stream ends while a packet is incomplete, you should throw an
EndOfStreamException.
Examples
Input:
Stream: [0, 0, 0, 2, 0xAA, 0xBB, 0, 0, 0, 1, 0xCC]
Output:
[0xAA, 0xBB], [0xCC]
Explanation: The first 4 bytes indicate a 2-byte payload. The next 2 bytes are consumed as the payload. The following 4 bytes indicate a 1-byte payload, which is then consumed.
Edge Cases
- A single
ReadOnlyMemorycontains only a partial header (e.g., 2 bytes of the 4-byte header). - A single
ReadOnlyMemorycontains multiple full packets. - A payload is spread across three different
ReadOnlyMemorychunks.
Hints
Hint 1
Maintain an internal buffer (such as a MemoryStream or a List) to store bytes that haven't formed a complete packet yet.
Hint 2
First, attempt to read enough bytes to fill the 4-byte header, then use the length from that header to determine exactly how many more bytes are required for the payload.
Hint 3
Use a loop to continuously process your internal buffer, only yielding a result when the buffer size is greater than or equal to the expected packet size.
Follow-up questions
- How would you optimize the memory usage to avoid excessive array allocations if you are processing high-throughput streams?
- What changes would be necessary if the header size was variable or if the length was little-endian instead of big-endian?
- How would you handle a scenario where a malicious sender sends an extremely large length value (e.g., 4GB) to trigger an OutOfMemoryException?
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.