Katkat

Analyze the Fairness of an Incremental Shuffle

EliseAISoftware Engineer · Technical Screen

Last updated

This question tests the ability to analyze algorithm correctness through probability theory and identify common pitfalls in pseudo-random number generation that lead to selection bias.

Problem Statement

You are tasked with auditing a custom "incremental shuffle" function designed for a playlist system. Instead of the standard Fisher-Yates shuffle (which performs an in-place swap to reach any permutation with 1/n! probability), the engineer implemented the following logic to shuffle an array arr of size n:

def incremental_shuffle(arr):
    n = len(arr)
    for i in range(n):
        # Pick a random index from 0 to n-1 inclusive
        j = random.randint(0, n - 1)
        arr[i], arr[j] = arr[j], arr[i]
    return arr

Your Tasks

  1. Mathematical Analysis: Prove whether this algorithm produces a uniform distribution over all possible permutations of arr. If it is biased, explain why.
  2. Complexity: State the time and space complexity of the function.
  3. Practical Implementation: Discuss how the quality of the random.randint generator (e.g., modulo bias or PRNG state exhaustion) could further impact the shuffling quality.
  4. Testing: Propose a method to verify if a shuffle implementation is statistically fair.

Constraints

  • Assume random.randint(a, b) produces an integer in [a, b] with uniform probability.
  • Consider the behavior for small n (e.g., n=3) to build your probability argument.

Example

For an array [1, 2, 3]:

  • The algorithm performs 3 iterations.
  • Each step has 3^3 = 27 possible sequences of random choices.
  • Determine how many ways each permutation (e.g., [1, 2, 3], [3, 2, 1]) can be generated.

Hints

Hint 1

Consider the number of possible outcomes (sequences of indices chosen) versus the total number of permutations (n!).

Hint 2

Calculate if 27 outcomes can be mapped evenly onto the 6 permutations (3!) for an array of size 3.

Hint 3

Evaluate if the 'swap' operation at index `i` is dependent on previous swaps, and whether every position has an equal probability of landing in every slot.

Follow-up questions

  • How does the Fisher-Yates shuffle achieve a uniform distribution, and why is it preferred over this incremental approach?
  • If you had to shuffle an array that is too large to fit in memory, what architectural changes would you suggest?
  • How would you detect 'non-random' patterns in a shuffled output given a large dataset of results?