Katkat
Back

Optimize no-penalty test strategy

CoinbaseData Scientist · HR Screen
MediumData ScientistHR ScreenStatistics & Math

Last updated

This problem tests your ability to model expected value under time constraints by balancing the probability of success per question against a limited resource pool.

The Strategy Optimization Problem

You are taking a test consisting of 50 multiple-choice questions. Each question has 5 options, with only one correct answer. You have a total time limit of 900 seconds.

For each question, you have two choices:

  1. Blind Guess: Takes 0 seconds. The probability of picking the correct answer is 1/5 (20%).
  2. Analyze: Takes 20 seconds. With a 60% probability, you successfully identify two incorrect options (leaving 3 remaining options to choose from, one of which is correct). With a 40% probability, the analysis fails to eliminate any options, leaving you with the original 5 options to guess from.

Your Task

Calculate the optimal number of questions (k) you should choose to analyze to maximize your expected total score. Assume that after spending the time to analyze, you will always guess randomly among the remaining valid options.

Constraints

  • You must respect the 900-second time budget.
  • You cannot analyze more than 45 questions given the time constraint (45 * 20 = 900 seconds).
  • Provide the mathematical expression for the expected score E(k) as a function of k questions analyzed.

Example

If you choose to analyze 1 question:

  • Expected score = (Prob of success * Expected value given success) + (Prob of failure * Expected value given failure) + (Remaining questions * 0.2)
  • Calculate if this strategy is better than simply guessing on all 50 questions.

Hints

Hint 1

Determine the expected value of a single question if you choose to analyze it: calculate the weighted average of success between the 60% chance of 1/3 odds and the 40% chance of 1/5 odds.

Hint 2

Compare the expected value of an 'analyzed' question against the expected value of a 'blind guess' (0.2).

Hint 3

Since the expected value of an 'analyzed' question is strictly higher than a blind guess, the strategy is simply to maximize the number of analyzed questions within the 900-second limit.

Follow-up questions

  • How does the optimal strategy change if the analysis time per question increases to 30 seconds?
  • If you could choose which questions to analyze (e.g., they are not all the same difficulty), how would you prioritize them?
  • What is the variance of your total score under the optimal strategy, and why might high variance be undesirable depending on the passing threshold?