Track Expiring GPU Credits
OpenAIMachine Learning Engineer · OnsiteLast updated
Design an in-memory credit ledger to track expiring GPU compute credits, support priority consumption, and manage resource debt.
Problem Statement
AI compute platforms often issue promotional or contractual compute credits that expire after a specified duration. When training workloads consume GPU hours, the platform must prioritize consuming credits that expire the earliest to maximize the user's usable credit value.
Implement the GPUCreditTracker class to simulate a real-time credit ledger that supports granting credits, consuming credits, and querying the current balance with debt handling.
Interface
class GPUCreditTracker:
def __init__(self):
pass
def grant(self, amount: int, timestamp: int, expiry: int) -> None:
"""Grants compute credits valid from timestamp until expiry (exclusive)."""
pass
def consume(self, amount: int, timestamp: int) -> int:
"""Consumes credits at timestamp. Returns the outstanding debt after consumption."""
pass
def get_balance(self, timestamp: int) -> int:
"""Returns net active credits at timestamp, or negative value representing debt."""
pass
Rules
- Priority Consumption: Credits that expire earliest must be consumed first.
- Expiration: A grant with expiration
expirycannot be used at anytimestamp >= expiry. - Debt Tracking: If a consumption request exceeds total available active credits, all active credits are consumed, and the remaining unsatisfied amount becomes outstanding debt (represented as a negative balance).
- Debt Settlement: When a new grant is added while outstanding debt exists, incoming credits must immediately offset the debt first. Any remaining credits after clearing the debt are retained with the given
expiry. - Monotonicity: All calls to
grant,consume, andget_balancehave non-decreasing timestamps (timestamp >= previous_timestamp).
Examples
Example 1
tracker = GPUCreditTracker()
# Grant 100 credits at t=0 expiring at t=10
tracker.grant(amount=100, timestamp=0, expiry=10)
# Grant 50 credits at t=2 expiring at t=5
tracker.grant(amount=50, timestamp=2, expiry=5)
# Consume 70 credits at t=3
# Earliest-expiring credits (expiry=5) are used first: 50 used.
# Remaining 20 credits are taken from the expiry=10 batch (80 left).
# Outstanding debt is 0.
assert tracker.consume(amount=70, timestamp=3) == 0
# Balance at t=4 is 80
assert tracker.get_balance(timestamp=4) == 80
# Consume 90 credits at t=8
# Only 80 active credits remain. 80 are consumed, 10 become debt.
assert tracker.consume(amount=90, timestamp=8) == 10
assert tracker.get_balance(timestamp=8) == -10
# Grant 30 credits at t=12 expiring at t=20
# 10 credits immediately settle the debt; 20 credits remain active.
tracker.grant(amount=30, timestamp=12, expiry=20)
assert tracker.get_balance(timestamp=12) == 20
Example 2: Expiration Handling
tracker = GPUCreditTracker()
tracker.grant(amount=50, timestamp=0, expiry=5)
# At t=5, the 50 credits have expired
assert tracker.get_balance(timestamp=5) == 0
# Consuming at t=6 leads directly to debt
assert tracker.consume(amount=20, timestamp=6) == 20
assert tracker.get_balance(timestamp=6) == -20
Constraints
1 = t_prev.- At most
2 * 10^5total method calls.
Edge Cases
- Consuming when the active balance is 0 increases existing debt.
- Grants that are immediately swallowed by existing debt should not be added to the expiration queue.
- Multiple grants with the exact same expiry timestamp.
Hints
Hint 1
Because credits must be consumed in order of earliest expiration date, a min-heap storing active credit batches ordered by expiry time is well-suited.
Hint 2
Before performing any operation at a given timestamp, lazily evict all credit batches from the min-heap whose expiration time is less than or equal to the current timestamp.
Hint 3
Maintain a running sum of all active credits and a separate scalar for outstanding debt so that `get_balance` operates in O(1) amortized time.
Follow-up questions
- How would you handle out-of-order events (e.g., delayed network messages where a consume event arrives with an earlier timestamp than a previously recorded grant)?
- What changes would be required if different credit grants had non-fungible restrictions (e.g., valid only on H100 clusters vs. A100 clusters)?
- How would you persist this state in a distributed system to ensure linearizable credit deduction across concurrent worker jobs?
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.