Katkat

Parse JSON Accounts into Typed Python Objects and Prepare Features

DaveMachine Learning Engineer · Onsite

Last updated

This problem tests your ability to sanitize and transform raw JSON account data into a structured format while enforcing strict numerical integrity and data consistency.

Problem Statement

You are provided with a JSON string representing a snapshot of bank accounts. Each object in the list contains an account_id (string), a balance (numeric), and an updated_at (ISO 8601 string).

Implement a function process_accounts(json_data: str) -> dict that performs the following steps:

  1. Validation: Filter out any records with non-finite balances (e.g., NaN, Inf) or missing fields.
  2. Deduplication: If multiple records exist for the same account_id, retain only the record with the most recent updated_at timestamp.
  3. Aggregation: Return a dictionary containing two keys:
    • total_balance: The sum of all unique, valid account balances.
    • account_map: A dictionary mapping account_id to its validated balance.

Constraints

  • The input string may contain thousands of records.
  • balance values must be treated as Decimal types to avoid floating-point errors.
  • The output total_balance should be returned as a float, rounded to two decimal places.

Example

input_json = """
[
  {"account_id": "A1", "balance": 100.50, "updated_at": "2023-01-01T10:00:00Z"},
  {"account_id": "A1", "balance": 150.25, "updated_at": "2023-01-02T10:00:00Z"},
  {"account_id": "A2", "balance": "invalid", "updated_at": "2023-01-02T10:00:00Z"}
]
"""
# Result: {"total_balance": 150.25, "account_map": {"A1": 150.25}}

Hints

Hint 1

Use the `json` module to parse the input and the `decimal` module to handle financial values accurately.

Hint 2

Store the records in a dictionary keyed by `account_id` to naturally handle deduplication as you iterate through the list.

Hint 3

Parse the `updated_at` strings into datetime objects using `datetime.fromisoformat()` to ensure accurate chronological comparison.

Follow-up questions

  • How would you modify your implementation to process a multi-gigabyte JSON file that does not fit in memory?
  • If this pipeline were to fail halfway through a large batch, what strategy would you use to ensure atomicity and idempotency?
  • How would you validate that the schema of the incoming JSON matches the expected structure before processing begins?