Katkat

Design a Streaming TLV Message Reassembler in C

Applied IntuitionSoftware Engineer · Technical Screen

Last updated

Implement an in-memory reassembler in C that reconstructs out-of-order chunked TLV messages, validates chunk bounds and duplicates, and detects message completion.

Problem Statement

In autonomous vehicle telemetry and sensor logging pipelines, large variable-length Type-Length-Value (TLV) payloads are fragmented into fixed- or bounded-size packets transmitted over UDP. Due to network jitter and multi-path routing, packets may arrive out of order or be duplicated.

Design and implement an in-memory packet reassembler in C for a single message stream. Each packet contains a stream header followed by a raw byte payload:

#include 
#include 
#include 

#define MAX_CHUNKS 1024
#define MAX_PAYLOAD_SIZE (4 * 1024 * 1024) // 4 MB

typedef struct {
    uint32_t message_id;    // Unique ID for the message
    uint16_t chunk_index;   // 0-indexed sequence number
    uint16_t total_chunks;  // Total number of chunks in this message
    uint32_t payload_len;   // Byte length of this chunk's payload
    const uint8_t *data;    // Pointer to chunk bytes
} PacketChunk;

typedef struct MessageReassembler MessageReassembler;

Implement the following API:

// Initializes a new reassembler instance for a specific expected message.
MessageReassembler* reassembler_create(uint32_t message_id, uint16_t total_chunks);

// Processes an incoming chunk. 
// Returns 0 on success (chunk accepted or harmless duplicate).
// Returns -1 on invalid input (out-of-bounds index, mismatched message_id/total_chunks, exceeding MAX_PAYLOAD_SIZE).
// Sets *is_complete = true if this chunk completes the entire message; false otherwise.
int reassembler_add_chunk(MessageReassembler *r, const PacketChunk *chunk, bool *is_complete);

// Returns a pointer to the contiguous reassembled buffer and populates *out_len.
// Can only be called once is_complete is true. Returns NULL if incomplete.
const uint8_t* reassembler_get_payload(const MessageReassembler *r, size_t *out_len);

// Frees all internal buffers and state.
void reassembler_destroy(MessageReassembler *r);

Requirements & Constraints

  • total_chunks will be in the range [1, MAX_CHUNKS].
  • Chunks can arrive in arbitrary order. Repeated chunks (same chunk_index and identical content/length) must be accepted idempotently without corrupting state or double-counting received chunk counts.
  • A chunk with the same chunk_index but differing length or conflicting bytes must be rejected as an error (-1).
  • Total message payload size cannot exceed MAX_PAYLOAD_SIZE. Any chunk that pushes accumulated size beyond this must return -1.
  • Ensure zero undefined behavior: handle integer overflow in index/length arithmetic and protect against NULL pointers.

Example

MessageReassembler *r = reassembler_create(42, 3);
bool complete = false;

// Packet 1 (index 2, length 4): "DONE"
PacketChunk c1 = { .message_id = 42, .chunk_index = 2, .total_chunks = 3, .payload_len = 4, .data = (uint8_t*)"DONE" };
reassembler_add_chunk(r, &c1, &complete); // complete == false

// Packet 2 (index 0, length 5): "HELLO"
PacketChunk c2 = { .message_id = 42, .chunk_index = 0, .total_chunks = 3, .payload_len = 5, .data = (uint8_t*)"HELLO" };
reassembler_add_chunk(r, &c2, &complete); // complete == false

// Packet 3 (index 1, length 1): "_"
PacketChunk c3 = { .message_id = 42, .chunk_index = 1, .total_chunks = 3, .payload_len = 1, .data = (uint8_t*)"_" };
reassembler_add_chunk(r, &c3, &complete); // complete == true

size_t total_len = 0;
const uint8_t *buf = reassembler_get_payload(r, &total_len);
// total_len == 10, buf == "HELLO_DONE"
reassembler_destroy(r);

Hints

Hint 1

Track per-chunk metadata (e.g., received flag, offset, and length) in an array indexed by chunk_index, or store chunk buffers until all offsets are known.

Hint 2

Because chunk sizes can vary, you can either store pointers/copies of individual chunks and stitch them into a contiguous buffer upon completion, or compute offsets if chunks are fixed-size.

Hint 3

Use a bitmask or a boolean array alongside an integer counter `received_count` to detect completion in O(1) time per chunk.

Follow-up questions

  • How would you handle streaming the reassembled payload directly to disk using atomic rename (`open`, `write`, `rename`) to prevent partial file exposure?
  • If `total_chunks` is unknown upfront and the protocol only provides an `is_last_chunk` flag on the final packet, how does your tracking structure change?
  • How would you make this reassembler thread-safe if multiple worker threads pull packets off the NIC simultaneously?