Katkat

Compare Survey Satisfaction for New and Established Users

MetaData Scientist · Technical Screen

Last updated

Tests your ability to write PostgreSQL queries that segment users into cohorts based on date logic, filter invalid survey events, and calculate aggregate satisfaction metrics.

Problem Statement

Meta's User Experience Research team wants to compare survey satisfaction scores between newly registered users and established users across different countries.

You are provided with two tables: users and surveys.

Tables

users

  • user_id (integer, primary key): Unique identifier for the user.
  • signup_date (date): The date the user created their account.
  • country (varchar): The country code of the user (e.g., 'US', 'GB', 'IN').

surveys

  • survey_id (integer, primary key): Unique identifier for the survey submission.
  • user_id (integer, foreign key): References users.user_id.
  • survey_date (date): The date the survey was taken.
  • satisfaction_score (integer): User rating from 1 to 5. Can be NULL if skipped.
  • is_test (boolean): TRUE if generated by an internal test account, FALSE otherwise.

Requirements

Write a PostgreSQL query to calculate survey statistics grouped by country and user_cohort under the following rules:

  1. Data Cleaning & Filtering:
    • Exclude any test entries (is_test = TRUE).
    • Exclude records where satisfaction_score IS NULL.
    • Exclude anomalous events where survey_date < signup_date.
  2. Cohort Classification:
    • New: Surveys completed within 30 days of registration (survey_date - signup_date 30).
  3. Metrics Computed:
    • country: User country.
    • user_cohort: Either 'New' or 'Established'.
    • unique_respondents: Count of unique users who submitted at least one valid survey in that cohort.
    • total_responses: Total number of valid survey submissions.
    • avg_satisfaction: Average satisfaction score rounded to 2 decimal places.
  4. Ordering: Order the results by country ascending, then user_cohort ascending.

Example Input

users table:

| user_id | signup_date | country |
|---------|-------------|---------|
| 101     | 2024-01-01  | US      |
| 102     | 2024-01-01  | US      |
| 103     | 2024-02-15  | US      |
| 104     | 2024-03-01  | GB      |

surveys table:

| survey_id | user_id | survey_date | satisfaction_score | is_test |
|-----------|---------|-------------|--------------------|---------|
| 1         | 101     | 2024-01-15  | 4                  | false   |
| 2         | 101     | 2024-02-20  | 5                  | false   |
| 3         | 102     | 2024-02-10  | 3                  | false   |
| 4         | 103     | 2024-02-20  | null               | false   |
| 5         | 103     | 2024-02-22  | 2                  | true    |
| 6         | 104     | 2024-03-05  | 5                  | false   |

Example Output

| country | user_cohort  | unique_respondents | total_responses | avg_satisfaction |
|---------|--------------|--------------------|-----------------|------------------|
| GB      | New          | 1                  | 1               | 5.00             |
| US      | Established  | 2                  | 2               | 4.00             |
| US      | New          | 1                  | 1               | 4.00             |

Explanation:

  • For GB, user 104 completed survey 6 four days after signup (New, score 5.00).
  • For US:
    • User 101 survey 1 is within 14 days (New, score 4).
    • User 101 survey 2 is 50 days after signup (Established, score 5).
    • User 102 survey 3 is 40 days after signup (Established, score 3).
    • User 103 responses are either null score or is_test = true, so both are omitted.
    • Established cohort for US has scores 5 and 3, averaging 4.00 across 2 unique respondents.

Hints

Hint 1

Join `users` and `surveys` on `user_id`, applying data cleaning filters (`is_test = FALSE`, score not null, and date consistency) in the `WHERE` clause.

Hint 2

Use a `CASE` statement to assign `'New'` or `'Established'` depending on whether `(s.survey_date - u.signup_date) <= 30`.

Hint 3

Remember that `unique_respondents` requires `COUNT(DISTINCT u.user_id)`, and `avg_satisfaction` requires `ROUND(AVG(s.satisfaction_score)::numeric, 2)` in PostgreSQL.

Follow-up questions

  • How would you pivot this query so that each country occupies a single row with columns `new_avg_satisfaction` and `established_avg_satisfaction`?
  • If a single user takes multiple surveys within a cohort, their responses might bias the average. How would you calculate an unweighted average of user-level averages?
  • How would you perform a two-sample t-test or Welch's t-test in SQL/Python to determine if the satisfaction difference between cohorts in a given country is statistically significant?