Debug and Extend a Priority-Aware Python Thread Pool
ByteDanceSoftware Engineer · OnsiteLast updated
This question tests your understanding of thread-safe synchronization primitives, priority queues, and robust resource management in a multi-threaded Python environment.
Problem Statement
You are tasked with implementing a PriorityThreadPool class. Unlike a standard pool, this implementation must prioritize tasks based on an integer priority (where a lower integer denotes a higher priority).
Requirements
submit(func, args, priority): Add a task to the internal queue. Threads should pick up the highest priority task available. Useheapqfor the underlying data structure.shutdown(wait=True): Prevent new tasks from being submitted and ensure worker threads exit gracefully. Ifwaitis True, block until all active tasks complete.Exception Handling: If a worker thread encounters an exception during task execution, the pool should log the error and ensure the thread remains alive to process subsequent tasks.
Interface
import threading
import heapq
import queue
class PriorityThreadPool:
def __init__(self, num_threads):
# Initialize threads and queue
pass
def submit(self, func, args=(), priority=0):
# Add task to queue with priority
pass
def shutdown(self, wait=True):
# Stop accepting tasks and shut down threads
pass
Constraints
- Use
threadingandqueueprimitives. - Ensure
submitis thread-safe. - Avoid busy-waiting loops.
Example
pool = PriorityThreadPool(num_threads=2)
pool.submit(lambda x: print(f"Task {x}"), args=(1,), priority=10)
pool.submit(lambda x: print(f"Task {x}"), args=(2,), priority=1)
# Output should ensure Task 2 executes before Task 1 due to priority.
Hints
Hint 1
Use a `queue.PriorityQueue` to store your tasks, as it is inherently thread-safe for both producers and consumers.
Hint 2
Implement a sentinel object (a 'poison pill') to notify worker threads that it is time to shut down during the shutdown process.
Hint 3
Wrap the task execution in a `try...except` block inside the worker loop to prevent individual task failures from crashing the background thread.
Follow-up questions
- How would you implement task cancellation if a user has a reference to the task ID before it starts execution?
- What specific risks are introduced by the Python Global Interpreter Lock (GIL) when using this thread pool for CPU-bound versus I/O-bound tasks?
- How would you modify this implementation to support 'task starvation prevention' if low-priority tasks are never getting executed?
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.