Katkat
Back

Design ML deployment with GitHub and Jenkins

Capital OneData Scientist · Onsite
MediumData ScientistOnsiteMachine Learning

Last updated

This question tests the ability to design an end-to-end MLOps pipeline, focusing on CI/CD orchestration, model versioning, reproducibility, and security compliance in a regulated financial environment.

Problem Statement

Design an automated deployment pipeline for a credit risk prediction model that transitions from a GitHub code repository to a production service. Your solution must satisfy the following constraints:

  1. Reproducibility & Audit: Every prediction made in production must be linked to the specific model version, training dataset snapshot, and code commit used to generate it.
  2. Security: The pipeline must ensure PII (Personally Identifiable Information) is encrypted at rest and in transit. Access to model artifacts must be audited.
  3. Reliability: Implement a deployment strategy that allows for a full rollback to the previous stable model version within 15 minutes in the event of performance degradation.

Requirements

  • Define the CI/CD pipeline steps orchestrated by Jenkins.
  • Propose a strategy for model versioning and artifact storage.
  • Outline how you would handle data lineage for auditing.
  • Describe how you would monitor the model for drift after deployment.

Expectations

Provide an architecture diagram (described in text) and explain the choice of tools for each stage. Focus on the integration between GitHub, Jenkins, and the deployment target (e.g., Kubernetes/Cloud Foundry).

Hints

Hint 1

Consider using an artifact registry to store containerized model images and a metadata store to keep track of training runs and dataset references.

Hint 2

Think about the difference between code CI (unit testing/linting) and ML CI (model validation/data quality checks before deployment).

Hint 3

For the 15-minute rollback requirement, consider a blue-green deployment or canary release strategy using a service mesh or load balancer.

Follow-up questions

  • How would you ensure that the training data used in development is not biased or corrupted before it reaches the production pipeline?
  • If the model shows performance drift after one week in production, what specific metrics would you look at to decide whether to trigger a retraining cycle vs. a rollback?
  • How does your pipeline design change if the model needs to be deployed to an edge device or on-premises server rather than a cloud-based environment?