Katkat
Back

Define Ultra success metrics and detect suspicious transactions

RevolutData Scientist · Technical Screen
EasyData ScientistTechnical ScreenAnalytics & Experimentation

Last updated

This question evaluates a data scientist's ability to define primary, diagnostic, and guardrail metrics for a product launch and to design rule-based suspicious-transaction detection using available schema, emphasizing metric definition, defensible proxy selection, SQL-based data modeling, and financial-crime domain knowledge.

You work on a fintech product with these existing tables (UTC timestamps). You may only use these tables/columns; if a metric cannot be measured directly, you must propose a defensible proxy using available data.

Tables

users

  • user_id BIGINT PRIMARY KEY
  • create_date TIMESTAMP
  • (optional if present) country VARCHAR

transactions

  • transaction_id BIGINT PRIMARY KEY
  • user_id BIGINT REFERENCES users(user_id)
  • transaction_time TIMESTAMP
  • product VARCHAR -- includes values like 'crypto', and may include 'ultra' if the plan is represented as a product
  • amount_gbp NUMERIC(18,2)
  • status VARCHAR -- 'completed' / 'declined'
  • (optional if present) ip_country VARCHAR

activity

  • user_id BIGINT REFERENCES users(user_id)
  • event_time TIMESTAMP
  • product VARCHAR -- may include 'ultra'
  • event_type VARCHAR -- 'view' / 'click'

Scenario A: Ultra subscription launch (open-ended)

A new Ultra subscription plan launches. In a 1-month evaluation window after launch:

  1. Define what “success” means with a primary metric, diagnostic metrics, and guardrail metrics.
  2. Describe key risks to validity (e.g., seasonality, marketing campaigns, selection bias) and how you’d mitigate them.
  3. Write example SQL (or pseudo-SQL) to compute your chosen metrics using only the tables above.

Scenario B: FinCrime—identify suspicious transactions (open-ended)

You partner with the Financial Crime team to flag suspicious behavior.

  1. Propose a rule-based detection approach that turns “suspicious” into actionable, testable rules using available data.
  2. Write example SQL (or pseudo-SQL) that outputs suspicious entities.

Required output for Scenario B (choose one and state which):

  • A list of suspicious user_ids with a risk_score and the reasons/rules triggered, or
  • A list of suspicious transaction_ids with flags/reasons.

Call out false-positive/false-negative tradeoffs and how you would evaluate and iterate.