Design a News Subscription Processing Engine
OptiverSoftware Engineer · Online AssessmentLast updated
Design and implement an in-memory news subscription processing engine that supports attribute-based filtering, deduplication, unsubscription, and unacknowledged delivery tracking.
Problem Description
Design an in-memory news subscription processing engine that receives incoming news items, matches them against registered subscriber filter predicates, and manages delivery state.
In high-frequency trading and financial market environments, news feeds must be disseminated to internal consumers with strict delivery semantics, deduplication guarantees, and low processing overhead.
Implement the NewsSubscriptionEngine class with the following capabilities:
-
register_subscriber(subscriber_id: str, topic: str, min_urgency: int) -> None- Registers a subscriber. If
subscriber_idalready exists, update its subscription rule to the newtopicandmin_urgency. topiccan be a specific string (e.g.,"EQUITIES") or'*'to match all topics.- A subscriber matches an item if the topic matches and the item's urgency is greater than or equal to
min_urgency.
- Registers a subscriber. If
-
unregister_subscriber(subscriber_id: str) -> bool- Removes the subscriber. Purges all pending unacknowledged deliveries for this subscriber.
- Returns
Trueif the subscriber was registered, orFalseif not found.
-
publish_item(news_id: str, topic: str, urgency: int) -> List[str]- Publishes an item to all matching registered subscribers.
- Deduplication: If
news_idhas already been processed, the engine must ignore the publication and return an empty list[]. - When matched, the item is appended to each matched subscriber's pending queue in the order published.
- Returns a sorted list of
subscriber_ids that matched and were queued.
-
ack_delivery(subscriber_id: str, news_id: str) -> bool- Marks a pending item as acknowledged by the given subscriber and removes it from their pending list.
- Returns
Trueif the item was in the subscriber's pending list and successfully acknowledged; otherwiseFalse.
-
get_pending(subscriber_id: str) -> List[str]- Returns a list of
news_ids currently pending forsubscriber_idin FIFO order of publication. - Returns an empty list if the subscriber does not exist or has no pending items.
- Returns a list of
Class Interface
class NewsSubscriptionEngine:
def __init__(self):
pass
def register_subscriber(self, subscriber_id: str, topic: str, min_urgency: int) -> None:
pass
def unregister_subscriber(self, subscriber_id: str) -> bool:
pass
def publish_item(self, news_id: str, topic: str, urgency: int) -> List[str]:
pass
def ack_delivery(self, subscriber_id: str, news_id: str) -> bool:
pass
def get_pending(self, subscriber_id: str) -> List[str]:
pass
Example
engine = NewsSubscriptionEngine()
engine.register_subscriber("sub1", "EQUITIES", 2)
engine.register_subscriber("sub2", "*", 4)
engine.register_subscriber("sub3", "CRYPTO", 1)
# Item matching sub1 (topic EQUITIES, urgency 3 >= 2)
engine.publish_item("N1", "EQUITIES", 3) # Returns: ["sub1"]
# Item matching sub1 and sub2 (urgency 4 >= 2 and >= 4)
engine.publish_item("N2", "EQUITIES", 5) # Returns: ["sub1", "sub2"]
# Duplicate publication of N1 should be ignored
engine.publish_item("N1", "EQUITIES", 5) # Returns: []
# Check pending
engine.get_pending("sub1") # Returns: ["N1", "N2"]
engine.get_pending("sub2") # Returns: ["N2"]
# Acknowledge delivery
engine.ack_delivery("sub1", "N1") # Returns: True
engine.get_pending("sub1") # Returns: ["N2"]
# Unregistering subscriber clears their pending queue
engine.unregister_subscriber("sub1") # Returns: True
engine.get_pending("sub1") # Returns: []
Constraints
- Number of total calls:
N <= 100,000 subscriber_id,news_id,topicare non-empty strings of alphanumeric characters and underscores, length1 <= len(s) <= 64.urgency,min_urgencyare integers where1 <= urgency <= 10.- Time complexity per operation should be
O(M)orO(M log M)whereMis the number of matching subscribers forpublish_item, andO(1)amortized for registration, unregistration, and acknowledgment.
Hints
Hint 1
Use a set to track already seen `news_id` values to ensure O(1) deduplication on publication.
Hint 2
Index subscribers by topic (e.g., a dictionary mapping topic to subscriber IDs, plus a separate collection for wildcard subscribers) to avoid iterating through all subscribers on every publish.
Hint 3
For subscriber pending queues, combine an ordered data structure (like an `OrderedDict` or a doubly linked list coupled with a hash map) to maintain FIFO order while allowing O(1) removals during `ack_delivery`.
Follow-up questions
- How would you adapt the engine to handle concurrency safely if multiple worker threads call `publish_item` and `ack_delivery` simultaneously without coarse-grained locking?
- If subscribers are distributed across different machines, how would you implement flow control and backpressure when a subscriber is too slow to acknowledge messages?
- How would you bound the memory footprint if the stream of incoming news IDs is infinite, preventing the deduplication set from growing without bound?
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.