Build a Payment Fraud Detection Model

Scenario

You are tasked with designing the initial machine learning pipeline for a real-time payment gateway processing millions of credit card transactions per day. The dataset contains 12 months of historical payment events, where each record includes:

• transaction_id: Unique identifier (string)
• timestamp: UTC timestamp of the transaction
• user_id: Customer identifier
• amount: Transaction amount in EUR (float)
• card_country and ip_country: ISO country codes (strings, with ~3% missing values in ip_country)
• merchant_category: Categorical identifier (approx. 250 distinct categories)
• is_fraud: Ground-truth label where 1 indicates a confirmed fraudulent chargeback and 0 indicates legitimate (fraud rate: ~0.15%)

Note that fraud labels suffer from a 30- to 60-day reporting lag because chargebacks take time to be filed by cardholders.

Tasks

Outline a structured approach for the following components of the modeling pipeline:

1. Data Splitting Strategy:

   • Explain how you would split the dataset into training, validation, and test sets. Discuss why traditional random k-fold cross-validation is inappropriate for this domain.
   • How do you account for the 60-day chargeback label delay?

2. Feature Preprocessing:

   • How should you handle missing values in ip_country?
   • Propose an encoding strategy for high-cardinality categorical features such as merchant_category.
   • Describe two domain-specific velocity or behavioral features you would engineer.

3. Class Imbalance & Modeling:

   • Choose an appropriate baseline algorithm for tabular fraud detection and justify your choice.
   • Compare two strategies for handling the severe 0.15% class imbalance during training (e.g., re-sampling vs. cost-sensitive learning).

4. Evaluation Metrics:

   • Why are Accuracy and standard ROC-AUC insufficient or misleading for this business use case?
   • Propose two primary evaluation metrics aligned with payment processing goals (minimizing financial loss while avoiding excessive false declines).