Katkat
Back

Count Files Safely from Bash

SquarepointQuantitative Researcher · Technical Screen
MediumQuantitative ResearcherTechnical ScreenSoftware Engineering Fundamentals

Last updated

Tests command-line proficiency and Bash scripting fundamentals by requiring a robust mechanism to count regular files across a directory tree while handling tricky filenames, symlinks, and recursion correctly.

Task

Write a POSIX-compliant Bash function or script named count_files that counts the total number of regular files in a given directory path (including subdirectories).

Your solution must be robust against filenames containing:

  • Whitespace (spaces, tabs)
  • Newline characters (\n)
  • Leading hyphens/dashes (e.g., -rf, --help)
  • Hidden files (dotfiles like .bashrc or .env)

Requirements

  1. Signature / Invocation: 
    count_files [-L] [--max-depth N]
    • ``: Target directory to inspect.
    • -L (optional): Follow symbolic links to directories during traversal.
    • --max-depth N (optional): Limit traversal to at most N directory levels deep (0 evaluates only the directory itself, 1 evaluates direct children, etc.). If omitted, recurse unbounded.
  2. File Filtering: Count only regular files. Do not count directories, symlinks (unless resolved via -L to a regular file), sockets, named pipes (FIFOs), or device nodes.
  3. Output: Print an integer representing the exact count to standard output, followed by a newline. Exit with code 0 on success, or a non-zero exit code if the directory does not exist or is unreadable.

Why ls | wc -l Fails

A common anti-pattern is ls -1 "$dir" | wc -l:

  • ls does not include hidden files unless -a is specified.
  • If a file name contains a newline, ls will output it across multiple lines, causing wc -l to overcount.
  • ls includes directories, devices, and special files unless filtered.

Examples

Assume directory test_dir/ contains:

  • normal.csv (regular file)
  • .config (hidden regular file)
  • "my file\nwith\nnewlines.txt" (regular file with two embedded newline characters)
  • sub/ (directory containing nested.parquet)
  • broken_link (symlink pointing to a nonexistent file)
$ count_files test_dir
4

Explanation: The regular files counted are normal.csv, .config, the newline-containing text file, and sub/nested.parquet.

$ count_files --max-depth 1 test_dir
3

Explanation: Depth is capped at 1, so sub/nested.parquet at depth 2 is excluded.

Constraints & Edge Cases

  • The directory may contain tens of thousands of files; avoid loading all file paths into a single shell array in memory.
  • Empty directories should return 0.
  • Non-existent directories must return a non-zero exit code and produce no valid count.

Hints

Hint 1

Using `find` with null-delimited outputs (`-print0`) is the standard POSIX pattern for handling filenames with arbitrary characters.

Hint 2

Pair `find -type f` with a stream consumer like `wc -l` after converting null bytes to newlines, or use `grep -c` or `awk` reading null-delimited records.

Hint 3

Be careful to pass `--` before user-supplied directory paths to `find` if the directory name might start with a hyphen.

Follow-up questions

  • How would you modify the solution to count files grouped by extension in a single pass without spawning separate processes per file?
  • What security risks arise if `find` output is passed into `xargs` without `-0`, and how can an adversary exploit filenames starting with `-`?
  • How does `-type f` behave when encountered with broken symlinks versus circular symlink loops when `-L` is active?