Streaming Python Parser for Key-Value Log Lines with Quoted and Escaped Values

Host machines and IoT edge devices, such as network cameras, door-access controllers and gateways, continuously write operational logs. Every line starts with the timestamp of the event, followed by any number of key-value pairs joined by =:

<timestamp> <key1>=<val1> <key2>="<val 2 with spaces>" <key3>=<val3> ...

Write a Python program that reads such a log file and parses every line into a dict. The dict holds the line's timestamp under the key 'timestamp', plus one entry per field. Log files can be several gigabytes, so the program must stream the file through a generator rather than load it.

Line format

• Timestamp. Starts the line. It is either an ISO-8601 value with no spaces, such as 2026-09-29T22:15:30Z, or a standard date and time separated by a single space, such as 2026-09-29 22:15:31.450. Store it, as written, under 'timestamp'.
• Key. One or more letters, digits, underscores, hyphens or dots (matching [\w.-]+), immediately followed by =.
• Unquoted value. Contains no whitespace and ends at the next space or at the end of the line, for example host=cam-01, status=failed or retries=3.
• Quoted value. Wrapped in a matching pair of double quotes. Inside the quotes it may contain spaces, runs of several spaces, = signs (as in details="reboot=true") and escaped double quotes, written as a backslash followed by a quote (\"). The output drops the outer quotes and turns each escape back into the character it stands for.
• Empty value. note="" produces {'note': ''}.
• Separators. Consecutive pairs are separated by one or more spaces or tabs.
• Blank lines. Skip lines that are empty or contain only whitespace.

Examples

Mixed value types:

Input: 2026-09-29T22:15:30Z host=cam-01 event="camera offline" status=failed retries=3
Output: {'timestamp': '2026-09-29T22:15:30Z', 'host': 'cam-01', 'event': 'camera offline',
 'status': 'failed', 'retries': '3'}

Runs of spaces between pairs, and a timestamp that contains a space:

Input: 2026-09-29 22:15:31.450 host=gw-99 ip=192.168.1.1 msg="handshake failed: connection reset" code=500
Output: {'timestamp': '2026-09-29 22:15:31.450', 'host': 'gw-99', 'ip': '192.168.1.1',
 'msg': 'handshake failed: connection reset', 'code': '500'}

Escaped quotes, an inner = and an empty value:

Input: 2026-09-29T22:15:33Z event="firmware update" details="version=\"2.4.1\" reboot=true" note=""
Output: {'timestamp': '2026-09-29T22:15:33Z', 'event': 'firmware update',
 'details': 'version="2.4.1" reboot=true', 'note': ''}

Constraints and Clarifications

• Do not read the whole file at once: no read() and no readlines(). Yield one parsed record at a time from a generator, so that auxiliary memory stays constant with respect to file size.
• Each line is one record.
• All values are returned as strings, exactly as the examples show (retries=3 becomes '3'). The timestamp is also returned as a string, not converted to a datetime.

Clarifying Questions

• What should happen to a malformed line, such as one with no recognizable timestamp, an unterminated quote, or a token without =: skip it, raise an error, or report it with its line number?
• If a key appears twice on one line, which value wins?
• Besides \", which backslash escapes can appear inside quoted values (for example \\), and how should an unknown escape be treated?
• Can a field be named timestamp, and if so, how should it coexist with the leading timestamp?
• Is key= with nothing after the = legal, and does it mean an empty string?
• What encoding are the files in, and what should happen to bytes that do not decode?