Katkat
Back

Reason About Coffee-Quality Comparison Probabilities

MercorSoftware Engineer · Technical Screen
MediumSoftware EngineerTechnical ScreenStatistics & Math

Last updated

Calculate conditional comparison probabilities for coffee quality scores under continuous and discrete distributions, accounting for ties and sample permutations.

Scenario

An automated coffee-cupping lab evaluates batches of coffee beans. In blind quality evaluations, judges or sensors assign scores to distinct coffee samples. You are asked to analyze comparison probabilities between samples under different scoring assumptions.

Let A, B, and C be three independent coffee samples, with quality scores denoted by random variables X_A, X_B, and X_C.

Questions to Solve

Part 1: Continuous Quality Scores (No Ties)

Assume X_A, X_B, and X_C are independent and identically distributed (i.i.d.) random variables drawn from a continuous distribution F (meaning the probability of a tie between any two scores is 0).

  1. By listing or counting the possible relative orderings (permutations) of the three scores, calculate the conditional probability P(X_A > X_C | X_A > X_B).
  2. State whether the result depends on the specific choice of the continuous distribution F.

Part 2: Discrete Quality Scores (Ties Allowed)

Now assume sensor ratings are discrete: each coffee sample independently receives an integer score chosen uniformly at random from {1, 2, ..., k}, where k >= 2.

  1. Express P(X_A = X_B) and P(X_A > X_B) in terms of k.
  2. Calculate the conditional probability P(X_A > X_C | X_A > X_B) as an explicit function of k.
  3. Verify the limiting value of P(X_A > X_C | X_A > X_B) as k -> infinity. Does it match the continuous result from Part 1?

Part 3: Multiple Comparisons

Returning to the continuous setting (no ties), introduce a fourth independent sample D with score X_D drawn from the same continuous distribution.

Calculate the probability P(X_C > X_D | X_A > X_B and X_A > X_C).

Hints

Hint 1

In the continuous i.i.d. case, all permutations of scores are equally likely by symmetry, so conditional probabilities can be found by counting permutations that satisfy the conditions.

Hint 2

For the discrete case with k values, condition on X_A = a; both X_B and X_C must be strictly less than a, which allows you to sum (a - 1)^2 over all possible values of a.

Hint 3

For Part 3, identify the relative rank of X_A among {A, B, C} first, and then determine how X_D can be inserted among the ordered samples.

Follow-up questions

  • How does P(X_A >= X_C | X_A >= X_B) behave in the discrete case when weak inequalities (ties included as favorable) are used instead?
  • If sample scores are not independent—for example, if score X_i = Mu + E_i with a shared day-effect Mu—does P(X_A > X_C | X_A > X_B) change?
  • How would you model and estimate the latent quality of each coffee bean if you only observe noisy pairwise win/loss/tie outcomes rather than raw numeric scores?