Clarify a BERT Keyphrase Span-Extraction Task

Scenario

In collaborative tools like FigJam, users write unstructured thoughts across sticky notes and comments. You are building an ML pipeline that extracts keyphrase spans from user notes using a BERT pointer model (similar to SQuAD-style span extraction). The model produces unnormalized start and end logits for each token.

Your task is to implement the post-processing span decoder that converts raw token logits and Hugging Face-style offset_mapping into clean, non-overlapping keyphrase text spans in the original string.

Interface

def extract_keyphrases(
    raw_text: str,
    token_offsets: list[tuple[int, int]],
    start_logits: list[float],
    end_logits: list[float],
    max_span_length: int = 5,
    top_k: int = 3,
    min_score: float = 0.0
) -> list[dict]:
    """
    Returns a list of up to `top_k` extracted keyphrases sorted by score descending.
    """
    pass

Requirements and Rules

1. Special Tokens: Any token whose offset is (0, 0) represents a special token (e.g., [CLS], [SEP], or padding) and cannot begin or end a valid span.
2. Span Validity: A candidate span is defined by token index pair (i, j) where i = min_score are considered.
3. Character Span: The raw text span begins at char_start = token_offsets[i][0] and ends at char_end = token_offsets[j][1]. The extracted text is raw_text[char_start:char_end].
4. Deduplication / Non-overlap: Candidate spans must not overlap in character offsets. Use a greedy approach: process candidate spans sorted by score descending, keeping a span only if its character range [char_start, char_end) does not overlap with any previously accepted span.
5. Return Format: List of dictionaries: [{"text": str, "char_start": int, "char_end": int, "score": float}, ...] truncated to at most top_k items.

Example

raw_text = "Design systems scale UI components."
token_offsets = [(0, 0), (0, 6), (7, 14), (15, 20), (21, 23), (24, 34), (34, 35), (0, 0)]
# Tokens: [CLS], Design, systems, scale, UI, components, ., [SEP]
start_logits = [0.0, 3.5, 0.2, 0.1, 2.8, 0.4, -1.0, 0.0]
end_logits   = [0.0, 0.1, 3.2, 0.0, 0.5, 3.0, -1.0, 0.0]

result = extract_keyphrases(
    raw_text, token_offsets, start_logits, end_logits, max_span_length=3, top_k=2
)
# Output:
# [
#   {"text": "Design systems", "char_start": 0, "char_end": 14, "score": 6.7},
#   {"text": "UI components", "char_start": 21, "char_end": 34, "score": 5.8}
# ]

Constraints

• len(token_offsets) == len(start_logits) == len(end_logits) = 1