Design Permission-Aware Retrieval for Design Files

Scenario

In a collaborative design platform like Figma, users create, share, and organize millions of design files, canvases, components, and design tokens across organizations, teams, projects, and individual share links.

You are tasked with designing a Permission-Aware Search Retrieval Engine that allows users to find files, frames, and components via short text queries, structured filters (e.g., file type, date modified, creator), and hybrid semantic search, while guaranteeing that no unauthorized metadata or content is ever leaked.

Requirements

Functional Requirements

1. Query & Filtering: Support keyword search (BM25/tokenized) and semantic retrieval (vector-based embeddings of frame titles/canvas contents) paired with metadata filters.
2. Hierarchical Permissions: Access control is inherited hierarchically: Org -> Team -> Project -> File -> Node/Frame. Direct grants, link-based access (e.g., "Anyone with link in org"), and explicit excludes can override inherited permissions.
3. Near Real-Time Updates: File edits, creation, renames, and permission mutations (grants/revocations) must reflect in search results within 2 seconds.
4. Revocation Safety: A revoked user must never see the file or snippet in subsequent queries, and search result counts must not reveal the existence of forbidden files.
5. Ranking Integration: Feed retrieved, permission-filtered candidates into a downstream scoring/ranking model while providing a deterministic fallback if the ranker degrades or times out.

Non-Functional Requirements & Scale

• Scale: 50 million total design files, 2 billion indexed components/frames, 10 million daily active users.
• Search Throughput: 15,000 queries per second (QPS) at peak.
• Latency: End-to-end p95 search latency < 150 ms (including candidate retrieval, authorization verification, and ranking).
• Observability: Metrics, traces, and logs must be strictly leakage-free (e.g., forbidden file IDs or title snippets must not appear in user query telemetry).

Key Architectural Questions to Address

1. Index Architecture & ACL Storage: How do you index documents and structured permissions? Do you perform pre-filtering (filtering permissions inside the inverted/vector index) or post-filtering (filtering candidate hits after retrieval), and how do you handle deep pagination?
2. Hierarchical ACL Resolution: How do you avoid traversing deep graph hierarchies on every candidate during high-throughput search queries?
3. Index Invalidation & Revocation: How do you handle high-cardinality permission updates (e.g., moving a project containing 10,000 files to a different team)?
4. Downstream Ranking & Fallback: How does candidate retrieval feed into the re-ranker, and how does the fallback operate under latency pressure?
5. Auditability & Observability: How do you monitor precision, recall, and system latency without leaking private document metadata in metrics or distributed traces?