Design a Streaming TLV Message Reassembler in C
Applied IntuitionSoftware Engineer · Technical ScreenLast updated
Implement an in-memory reassembler in C that reconstructs out-of-order chunked TLV messages, validates chunk bounds and duplicates, and detects message completion.
Problem Statement
In autonomous vehicle telemetry and sensor logging pipelines, large variable-length Type-Length-Value (TLV) payloads are fragmented into fixed- or bounded-size packets transmitted over UDP. Due to network jitter and multi-path routing, packets may arrive out of order or be duplicated.
Design and implement an in-memory packet reassembler in C for a single message stream. Each packet contains a stream header followed by a raw byte payload:
#include
#include
#include
#define MAX_CHUNKS 1024
#define MAX_PAYLOAD_SIZE (4 * 1024 * 1024) // 4 MB
typedef struct {
uint32_t message_id; // Unique ID for the message
uint16_t chunk_index; // 0-indexed sequence number
uint16_t total_chunks; // Total number of chunks in this message
uint32_t payload_len; // Byte length of this chunk's payload
const uint8_t *data; // Pointer to chunk bytes
} PacketChunk;
typedef struct MessageReassembler MessageReassembler;
Implement the following API:
// Initializes a new reassembler instance for a specific expected message.
MessageReassembler* reassembler_create(uint32_t message_id, uint16_t total_chunks);
// Processes an incoming chunk.
// Returns 0 on success (chunk accepted or harmless duplicate).
// Returns -1 on invalid input (out-of-bounds index, mismatched message_id/total_chunks, exceeding MAX_PAYLOAD_SIZE).
// Sets *is_complete = true if this chunk completes the entire message; false otherwise.
int reassembler_add_chunk(MessageReassembler *r, const PacketChunk *chunk, bool *is_complete);
// Returns a pointer to the contiguous reassembled buffer and populates *out_len.
// Can only be called once is_complete is true. Returns NULL if incomplete.
const uint8_t* reassembler_get_payload(const MessageReassembler *r, size_t *out_len);
// Frees all internal buffers and state.
void reassembler_destroy(MessageReassembler *r);
Requirements & Constraints
total_chunkswill be in the range[1, MAX_CHUNKS].- Chunks can arrive in arbitrary order. Repeated chunks (same
chunk_indexand identical content/length) must be accepted idempotently without corrupting state or double-counting received chunk counts. - A chunk with the same
chunk_indexbut differing length or conflicting bytes must be rejected as an error (-1). - Total message payload size cannot exceed
MAX_PAYLOAD_SIZE. Any chunk that pushes accumulated size beyond this must return-1. - Ensure zero undefined behavior: handle integer overflow in index/length arithmetic and protect against NULL pointers.
Example
MessageReassembler *r = reassembler_create(42, 3);
bool complete = false;
// Packet 1 (index 2, length 4): "DONE"
PacketChunk c1 = { .message_id = 42, .chunk_index = 2, .total_chunks = 3, .payload_len = 4, .data = (uint8_t*)"DONE" };
reassembler_add_chunk(r, &c1, &complete); // complete == false
// Packet 2 (index 0, length 5): "HELLO"
PacketChunk c2 = { .message_id = 42, .chunk_index = 0, .total_chunks = 3, .payload_len = 5, .data = (uint8_t*)"HELLO" };
reassembler_add_chunk(r, &c2, &complete); // complete == false
// Packet 3 (index 1, length 1): "_"
PacketChunk c3 = { .message_id = 42, .chunk_index = 1, .total_chunks = 3, .payload_len = 1, .data = (uint8_t*)"_" };
reassembler_add_chunk(r, &c3, &complete); // complete == true
size_t total_len = 0;
const uint8_t *buf = reassembler_get_payload(r, &total_len);
// total_len == 10, buf == "HELLO_DONE"
reassembler_destroy(r);
Hints
Hint 1
Track per-chunk metadata (e.g., received flag, offset, and length) in an array indexed by chunk_index, or store chunk buffers until all offsets are known.
Hint 2
Because chunk sizes can vary, you can either store pointers/copies of individual chunks and stitch them into a contiguous buffer upon completion, or compute offsets if chunks are fixed-size.
Hint 3
Use a bitmask or a boolean array alongside an integer counter `received_count` to detect completion in O(1) time per chunk.
Follow-up questions
- How would you handle streaming the reassembled payload directly to disk using atomic rename (`open`, `write`, `rename`) to prevent partial file exposure?
- If `total_chunks` is unknown upfront and the protocol only provides an `is_last_chunk` flag on the final packet, how does your tracking structure change?
- How would you make this reassembler thread-safe if multiple worker threads pull packets off the NIC simultaneously?
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.