Katkat

Clean scores and return top 5 students

WayfairData Scientist · Online Assessment

Last updated

This problem tests your ability to clean tabular data by performing conditional imputation and applying window functions to rank performance metrics.

Problem Statement

You are provided with a DataFrame df_scores containing columns ['student_id', 'subject', 'score']. Some scores are missing (NaN). Your task is to process this data to identify the top 5 performing students based on their average score across all subjects.

Requirements:

  1. Data Cleaning: Fill missing scores with the median score of that specific subject. If a subject has no available scores, assume the median is 0.
  2. Aggregation: Calculate the average score for each student across all subjects.
  3. Ranking: Identify the top 5 students by their calculated average score. In the event of a tie in average scores, use student_id (ascending) as a tie-breaker.
  4. Output: Return a list of student_ids.

Function Signature

def get_top_students(df_scores: pd.DataFrame) -> list:
    # implementation here

Example

Input:

data = {
    'student_id': [1, 1, 2, 2, 3, 3],
    'subject': ['Math', 'Science', 'Math', 'Science', 'Math', 'Science'],
    'score': [80, None, 90, 85, None, 70]
}

Processing:

  • Math median: (80+90)/2 = 85. Science median: (85+70)/2 = 77.5.
  • Student 1 scores: Math 80, Science 77.5. Avg: 78.75.
  • Student 2 scores: Math 90, Science 85. Avg: 87.5.
  • Student 3 scores: Math 85, Science 70. Avg: 77.5.

Output: [2, 1, 3]

Constraints

  • student_id is an integer.
  • score is a float or NaN.
  • Handle cases where a student might not have any scores.

Hints

Hint 1

Use `groupby` on the 'subject' column to calculate the median score for each group before filling missing values.

Hint 2

After imputation, use `groupby` on 'student_id' to compute the mean of their scores.

Hint 3

Sort the resulting aggregated DataFrame using `sort_values` with a list of columns to handle the primary and secondary sort criteria.

Follow-up questions

  • How would your approach change if you needed to exclude students who have more than 50% of their scores missing?
  • If the dataset scale grew to billions of rows, how would you rewrite this logic using SQL instead of Pandas?
  • How would you handle a scenario where the median imputation needs to be calculated only on a subset of the data based on a 'semester' column?