Katkat

Identify Files That Need Synchronization

Jump TradingSoftware Engineer · Online Assessment

Last updated

Determine which files from a source system need to be synchronized to a destination system based on permissions, presence, size, and modification timestamps.

Problem Statement

In distributed trading infrastructure, configuration and data files must be continuously replicated from a primary host to secondary nodes. You are tasked with writing a synchronization filter that identifies which files from a source node need to be synced to a destination node.

You are given two lists of file metadata objects: sources and destinations.

Each source metadata object contains:

  • name (str): The unique path/name of the file.
  • size (int): File size in bytes.
  • mtime (int): Last modified timestamp in seconds (Unix epoch).
  • is_readable (bool): Whether the source file can be read.

Each destination metadata object contains:

  • name (str): The path/name of the file.
  • size (int): File size in bytes.
  • mtime (int): Last modified timestamp in seconds (Unix epoch).
  • is_writable (bool): Whether the destination file can be overwritten.

A source file needs to be synchronized if and only if all of the following conditions are met:

  1. The source file is readable (is_readable == True).
  2. One of the following holds regarding the destination:
    • The file does not exist in destinations.
    • The file does exist in destinations, is writable (is_writable == True), and either:
      • source.size != dest.size, or
      • source.mtime > dest.mtime.

If a file exists at the destination but is not writable, it cannot be updated and must not be included in the synchronization list.

Return the list of file names that require synchronization, preserving their original order from sources.

Function Signature

def get_files_to_sync(sources: list[dict], destinations: list[dict]) -> list[str]:
    pass

Constraints

  • 1 = 0
  • mtime >= 0

Examples

Example 1

sources = [
    {"name": "config.xml", "size": 500, "mtime": 1000, "is_readable": True},
    {"name": "secrets.key", "size": 128, "mtime": 1000, "is_readable": False},
    {"name": "data.csv", "size": 2048, "mtime": 1050, "is_readable": True},
    {"name": "log.txt", "size": 100, "mtime": 1000, "is_readable": True},
]

destinations = [
    {"name": "config.xml", "size": 500, "mtime": 900, "is_writable": True},
    {"name": "data.csv", "size": 2048, "mtime": 1050, "is_writable": True},
    {"name": "log.txt", "size": 100, "mtime": 1000, "is_writable": False},
]

# Output: ["config.xml"]

Explanation:

  • config.xml: Readable, exists at dest, writable, and source.mtime (1000) > dest.mtime (900). Needs sync.
  • secrets.key: Not readable. Cannot sync.
  • data.csv: Sizes match and modification times match. Already up to date.
  • log.txt: Exists at dest but is_writable is False. Cannot sync.

Example 2

sources = [
    {"name": "a.bin", "size": 100, "mtime": 500, "is_readable": True},
    {"name": "b.bin", "size": 200, "mtime": 500, "is_readable": True},
]

destinations = [
    {"name": "a.bin", "size": 99, "mtime": 500, "is_writable": True},
]

# Output: ["a.bin", "b.bin"]

Explanation:

  • a.bin: Size differs (100 != 99) and destination is writable. Needs sync.
  • b.bin: Missing at destination and source is readable. Needs sync.

Hints

Hint 1

To avoid an O(n * m) lookup penalty, pre-index destination entries by their file name in a hash map.

Hint 2

Verify the non-negotiable preconditions first: if the source is not readable, it can be immediately skipped.

Hint 3

Be careful to handle the two distinct destination branches: completely missing versus present but not writable.

Follow-up questions

  • How would you handle checksum or hash verification when file size and mtime match but contents might differ?
  • If the metadata is streaming or too large to fit in memory, how would you design an external-memory or chunk-based solution?
  • How would you extend this logic to also detect destination files that no longer exist on the source (deletion synchronization)?