Katkat
Back

Architecture and Data-Scaling Trade-offs

J.P. MorganSoftware Engineer · Onsite
MediumSoftware EngineerOnsiteSoftware Engineering Fundamentals

Last updated

This question assesses the ability to articulate trade-offs between architectural styles (monolith vs. microservices) and data storage strategies (sharding vs. replication) in the context of a high-throughput financial system.

The Scenario

You are a lead engineer at a financial institution tasked with architecting a new high-frequency transaction ledger system. The system must handle 50,000 transactions per second (TPS) while maintaining strict ACID compliance. The business is debating between deploying a monolithic architecture for simplicity or moving toward a microservices-based approach for agility. Additionally, as the ledger data is expected to grow by several terabytes per month, you must decide on an appropriate database scaling strategy.

Task

  1. Compare and contrast the monolithic and microservices architectures in the context of this specific financial ledger system. What are the major risks associated with transitioning to microservices for a system requiring strong consistency?
  2. Propose a data-scaling strategy. Explain the trade-offs between implementing horizontal sharding versus read-replication to handle the 50,000 TPS requirement.
  3. How would you handle cross-shard transactions to ensure that a balance update is atomic if the sender's account and receiver's account reside on different shards?

Hints

Hint 1

Consider how microservices impact data consistency (e.g., distributed transactions like 2PC or Saga patterns) compared to a single database instance.

Hint 2

Think about the difference between scaling for read-heavy workloads versus write-heavy workloads when choosing between sharding and replication.

Hint 3

For cross-shard transactions, consider the implications of latency and the 'CAP theorem' trade-offs between availability and consistency.

Follow-up questions

  • If we experience a 'hot shard' problem where a few accounts perform 40% of all transactions, how would you rebalance or modify your partitioning logic?
  • What monitoring or observability mechanisms would you implement to detect transaction failure rates across a distributed service boundary?
  • How does the choice of database (SQL vs. NoSQL) influence your decision-making regarding sharding and ACID guarantees?