Katkat

Extend and optimize an existing CLI tool

OracleSoftware Engineer · Onsite

Last updated

This problem tests your ability to refactor legacy CLI code, implement modular features, and optimize performance for large-scale data processing.

The Task

You are maintaining a CLI tool named log-analyzer written in Python. Currently, the tool reads a local file line-by-line, filters by a single status code, and prints the count of matching entries.

Requirements

  1. Refactor the existing code to support multiple status code filtering.
  2. Add a new feature to optionally output results to a JSON file instead of stdout.
  3. Optimize the tool to handle log files exceeding 10GB without loading them entirely into memory.
  4. Ensure the solution remains maintainable and unit-testable.

Current Codebase Excerpt

def process_logs(file_path, target_code):
    count = 0
    with open(file_path, 'r') as f:
        for line in f:
            # Assuming format: "timestamp status_code message"
            if str(target_code) in line.split()[1]:
                count += 1
    print(f"Total: {count}")

Your Deliverable

Provide a revised version of the process_logs function (or a class-based structure) that implements these requirements. Assume the file format is consistent. Use the following signature:

def process_logs(file_path, target_codes: list[int], output_path: str = None):
    # Implementation here
    pass

Constraints

  • Time complexity: O(N) where N is the number of lines.
  • Space complexity: O(K) where K is the number of target codes.
  • Edge cases: Handle missing files, empty lines, and malformed lines gracefully.

Hints

Hint 1

Consider using a 'set' for O(1) lookup of target status codes.

Hint 2

Use Python's generators or file object iteration to process the file line-by-line, ensuring memory usage stays constant regardless of file size.

Hint 3

Implement a strategy pattern or a simple conditional check to handle different output streams (stdout vs. file) without duplicating the processing logic.

Follow-up questions

  • How would you add unit tests to verify the logic without reading actual 10GB files?
  • If the log file contains duplicate entries, how would you modify the current logic to count only unique entries?
  • How would you transition this script to support multi-threading or parallel processing for faster execution on multi-core systems?