Katkat
Back

Measure platform success and drive adoption

AmazonSoftware Engineer · Onsite
MediumSoftware EngineerOnsiteSystem Design

Last updated

This question assesses your ability to design a telemetry and feedback system for an internal developer platform to measure adoption, identify friction points, and drive cross-team migration.

System Design: Internal Developer Platform Analytics

You are the lead engineer for an internal "Serverless Compute Platform" used by over 500 engineering teams at a large tech company. Your leadership team wants to understand why adoption of the new platform is stalling and requires a robust system to track usage, performance, and user satisfaction.

Requirements

Functional Requirements:

  1. Usage Tracking: Track which teams are using the platform, which features (e.g., auto-scaling, logging, secret management) are utilized, and the frequency of deployments.
  2. Performance Metrics: Monitor the latency of the platform’s control plane and the duration of deployment pipelines.
  3. Feedback Loop: Provide a mechanism for developers to report issues or "pain points" directly through the command-line interface (CLI) or dashboard.
  4. Adoption Dashboard: Create a high-level view showing the percentage of legacy services migrated to the platform versus new services created on it.

Non-Functional Requirements:

  1. Data Freshness: The dashboard should reflect usage data within a 5-minute latency.
  2. Scalability: The system must handle events from 10,000+ deployments per day.
  3. Privacy: Ensure sensitive application code or secrets are never logged in the telemetry system.

Scope of Response

Design the data architecture, including how events are collected, processed, and stored. Define the key metrics you would prioritize to convince team leads to migrate. Finally, describe your strategy for handling teams that are resistant to adopting the platform.

Hints

Hint 1

Consider using an asynchronous event-driven architecture to ensure that telemetry collection does not increase the latency of the platform's actual deployment pipelines.

Hint 2

Think about the difference between 'vanity metrics' (total deployments) and 'health metrics' (successful migration rate or mean time to recover from a platform-induced error).

Hint 3

Focus on the 'Golden Signals'—latency, traffic, errors, and saturation—tailored specifically to the developer experience rather than just server hardware.

Follow-up questions

  • How would you ensure data consistency if the platform is deployed across multiple global regions?
  • If adoption remains flat despite positive performance metrics, what qualitative data would you look for to diagnose the issue?
  • How would you design a 'deprecation signal' system to safely force-migrate teams off legacy infrastructure without breaking their production services?