Design a Post-Training Pipeline to Improve a 7B Model's Coding for an Enterprise

Design the post-training process that improves the coding capabilities of a 7-billion-parameter language model for one enterprise customer. Cover the full pipeline: data collection, supervised fine-tuning (SFT), reinforcement learning from human feedback (RLHF), evaluation, the inference engine that serves the tuned model, and the feedback loop that brings production usage back into training.

Expect every stage to be probed in depth. The interviewer pays particular attention to how the datasets are built and how the reward model is chosen, and asks directly why an RLHF stage is needed at all.

Constraints and Clarifications

• Assume you start from a pretrained 7B decoder checkpoint that you are allowed to fine-tune, and that you own the training and serving stack.
• Assume the customer can share its code and developer interactions under an agreement you must respect. What exactly may be used for training is a clarifying question, not a given.

Clarifying Questions

• Which coding tasks matter most to the customer: inline completion, chat-based code generation, code review, test writing, or multi-file changes?
• Which languages, frameworks and internal libraries dominate the customer's codebase?
• May the customer's proprietary code and usage logs be used for training, and must the data and the tuned weights stay inside the customer's environment?
• Is the starting checkpoint a raw base model or one that is already instruction-tuned?
• What latency and hardware budget does serving have, and how many developers will use it?
• How will the customer judge success: suggestion acceptance, tasks completed, fewer defects, or something else?

Part 1 — Data collection and dataset construction

How do you collect and construct the training data for each later stage?

What This Part Should Cover

• Data sources, and the licensing, privacy and secret-handling constraints on each
• Quality filtering, deduplication and separation from evaluation data
• The format each stage needs: demonstrations, comparisons, or tasks with a checker

Part 2 — Supervised fine-tuning, and why RLHF is still needed

Describe the SFT stage. Then answer the interviewer's probe: "If SFT can also learn alignment when we add user preference data to it, why do we still need an RLHF step?"

What This Part Should Cover

• The SFT recipe: data mixture, full fine-tuning versus adapters, and protecting general capability
• What the SFT objective can and cannot extract from preference data
• On-policy versus off-policy learning, and when SFT alone is a reasonable choice

Part 3 — RLHF and the choice of reward model

Design the reinforcement-learning stage. How do you choose or build the reward model, and how do you know it is good enough to optimize against?

What This Part Should Cover

• Candidate reward sources and how they are combined
• How a reward model is validated before and during optimization
• The RL algorithm, regularization toward the SFT model, and reward hacking

Part 4 — Evaluation, inference engine and feedback loop

How do you evaluate the model before release, how do you serve it, and how do you close the loop from production usage back into training?

What This Part Should Cover

• Offline evaluation: customer-specific tasks, regression on general ability, and contamination control
• Serving a 7B model within the latency budget, including memory and throughput
• Which production signals become training data, and how often the model is retrained and released