Katkat
Back

Design a game genre classifier

RobloxMachine Learning Engineer · Onsite
MediumMachine Learning EngineerOnsiteML System Design

Last updated

This question assesses the ability to design an end-to-end multi-modal machine learning system for classifying user-generated game content based on diverse input features and real-time serving requirements.

Scenario

Roblox hosts millions of user-created games. As an ML Engineer, you are tasked with designing a system that automatically classifies each game into a genre (e.g., 'Obby', 'Roleplay', 'Simulator', 'Tycoon'). The system must categorize new games upon upload and re-evaluate existing ones as their features evolve.

Available Data

  • Textual: Game title, description, and update logs.
  • Visual: Game thumbnail, promotional images, and captured gameplay screenshots.
  • Tabular/Metadata: Creator ID, play count, average session length, concurrent player count, and purchase history data.

Requirements

  1. Functional: The system must assign one or more genres to a game. It needs to handle 'cold starts' (new games with no play data) and active games.
  2. Non-functional: The inference pipeline must be highly scalable. The system should handle thousands of updates per hour. Latency requirements for initial upload are loose (minutes), but classification updates for existing games must be efficient.

Deliverables

  • Feature Engineering: How do you process and fuse these multi-modal inputs?
  • Model Architecture: Propose a model design that balances performance and extensibility.
  • Data/Labeling Strategy: How do you build a high-quality ground-truth dataset given the subjectivity of genre definitions?
  • Serving/Deployment: Describe the pipeline for both batch re-classification and real-time updates.
  • Monitoring: What metrics would you track to ensure the model remains accurate and unbiased?

Hints

Hint 1

Consider how you might use pre-trained embeddings (e.g., CLIP for images/text or Transformers for text) to reduce the complexity of raw data processing.

Hint 2

Address the cold-start problem by prioritizing content-based features (text/images) for new games and gradually incorporating collaborative-filtering style metrics (session length/play count) as the game matures.

Hint 3

Think about the lifecycle of the labels; consider a semi-supervised approach or human-in-the-loop (HITL) strategy to handle the large volume of games and potential 'genre drifting'.

Follow-up questions

  • How would you handle games that attempt to 'game' the classification system by using misleading titles or tags to appear in popular genres?
  • If you move from a single-genre classification to a multi-label classification system, how would your loss function or evaluation metrics change?
  • How would you implement a canary deployment strategy to validate the performance of a new model version against the current production model?