Design file deduplication at scale

Design an algorithm to identify duplicate files in a large directory tree. You are given an iterator over files providing (path, size) and a function read_chunks(path) -> Iterator[bytes]. Requirements: minimize I/O by comparing sizes and using rolling or cryptographic hashes; handle hash collisions safely; support datasets that do not fit in memory; and output groups of paths that are byte-for-byte identical. Explain time/space trade-offs and how you would parallelize the solution.