System Design
System Design Interview Blueprint: High-Concurrency Architecture & The Technical STAR Method
In senior (L5/E5), staff (L6/E6), and principal engineering interviews, the System Design round is the single highest-weighted factor determining your level and compensation band. Yet more than 65% of experienced engineers struggle in these 45-minute sessions—not from a lack of technical knowledge, but from unstructured communication, jumping prematurely into database schemas, and failing to rigorously justify architectural trade-offs.
This guide outlines an end-to-end system design interview framework combined with the Technical STAR Method, complete with concrete architectures for distributed rate limiters, vector indexing (HNSW vs IVFFlat), and high-concurrency event streams.
+───────────────────────────────────────────────────────────────────────────+
| THE 45-MINUTE SYSTEM DESIGN TIMELINE |
+───────────────────────────────────────────────────────────────────────────+
| |
| [00:00 - 05:00] 1. Scope & Functional/Non-Functional Requirements |
| [05:00 - 15:00] 2. High-Level Blueprint & Data Flow Diagrams |
| [15:00 - 35:00] 3. Component Deep Dive & Concurrency Bottlenecks |
| [35:00 - 45:00] 4. Fault Tolerance, Edge Cases & Capacity Math |
| |
+───────────────────────────────────────────────────────────────────────────+
1. The Technical STAR Method for Architectural Decisions
While the classic STAR framework is widely used for behavioral rounds, applying the Technical STAR Framework allows you to defend system design decisions objectively:
+───────────────────────────────────────────────────────────────────────────+
| THE TECHNICAL STAR FRAMEWORK |
+───────────────────────────────────────────────────────────────────────────+
| |
| S - SITUATION & SCALE | 1,000,000 active WebSocket connections |
| | 50,000 write RPS, <50ms p99 latency SLA |
| ───────────────────────────┼───────────────────────────────────────── |
| T - TECHNICAL BOTTLENECK | Synchronous DB transactions cause lock |
| | contention and thread starvation at scale |
| ───────────────────────────┼───────────────────────────────────────── |
| A - ACTIONABLE ARCHITECTURE| Implement Redis Sliding Window Log + Kafka |
| | partitioned log for async write absorption |
| ───────────────────────────┼───────────────────────────────────────── |
| R - RESULT & TRADE-OFF | Achieved 18ms p99 latency; accepted 200ms |
| | eventual consistency on analytics views |
| |
+───────────────────────────────────────────────────────────────────────────+
2. Deep Dive: Distributed Rate Limiter Architecture
A classic system design interview question evaluates your ability to throttle abuse across globally distributed edge nodes.
+───────────────────────────────────────────────────────────────────────────+
| DISTRIBUTED SLIDING WINDOW RATE LIMITER PIPELINE |
+───────────────────────────────────────────────────────────────────────────+
| |
| Incoming HTTP Request (User ID: 8941) |
| │ |
| ▼ |
| [Envoy / API Gateway] |
| │ |
| ▼ |
| [Rate Limiting Service] |
| │ |
| ▼ |
| +───────────────────────────────────────────────────────────────────+ |
| | Redis Cluster (Atomic Lua Script Execution) | |
| | - ZREMRANGEBYSCORE key 0 (now - window) [Purge Expired Events] | |
| | - ZCARD key [Count Events in Window]| |
| | - IF count < limit THEN | |
| | ZADD key now requestId; EXPIRE key window; RETURN ALLOW | |
| | ELSE | |
| | RETURN DENY (429 Too Many Requests) | |
| +───────────────────────────────────────────────────────────────────+ |
| |
+───────────────────────────────────────────────────────────────────────────+
Rate Limiting Algorithm Comparison Matrix
| Algorithm | Memory Consumption | Burst Handling | Accuracy | Concurrency Race Condition Risk |
|---|---|---|---|---|
| Token Bucket | $O(1)$ per user (Tokens + Last Refill) | Excellent (burst up to bucket size) | Moderate | Low (atomic integer update) |
| Leaky Bucket | $O(1)$ per user (FIFO Queue) | Smooth constant outflow | High | Low (queue lock or ring buffer) |
| Fixed Window Counter | $O(1)$ per user (Integer counter) | Poor (2x burst at window boundary) | Low | Low (Redis INCR) |
| Sliding Window Log | $O(N)$ where $N$ = requests in window | Perfect | 100% Exact | High without atomic Lua script |
3. Deep Dive: Vector Database Indexing (HNSW vs IVFFlat)
Modern AI system designs (such as real-time RAG, semantic resume matching, and audio transcript search) require understanding approximate nearest neighbor (ANN) vector indexing.
GRAPH-BASED (HNSW) INVERTED CLUSTER (IVFFlat)
Layer 2: (•) ───────── (•) Centroid 1 ───> [ v1, v2, v3, v4 ]
│ │
Layer 1: (•) ── (•) ── (•) Centroid 2 ───> [ v5, v6, v7, v8 ]
│ │ │
Layer 0: (•)-(•)-(•)-(•)-(•) Centroid 3 ───> [ v9, v10, v11, v12 ]
(Hierarchical Skip-List Graph) (K-Means Voronoi Partition Cells)
Trade-Off Blueprint:
- HNSW (Hierarchical Navigable Small World):
* Query Latency: ~2-5ms (O(log N) graph traversal)
* Recall Rate: 98%+
* RAM Overhead: High (stores graph edges in memory)
* Best for: Real-time interactive applications (e.g., Kairo live interview cue retrieval)
- IVFFlat (Inverted File Flat):
* Query Latency: ~10-30ms (Requires scanning top-k nearest centroids)
* Recall Rate: 85-92%
* RAM Overhead: Low (flat quantized vector storage)
* Best for: Large-scale cold data indexing with billions of documents4. High-Concurrency Event Streams: Backpressure & Buffering
When designing ingestion systems handling $>500,000$ RPS, database write starvation is the primary failure mode. Use an event-driven backpressure pattern:
+───────────────────────────────────────────────────────────────────────────+
| HIGH-CONCURRENCY ASYNCHRONOUS INGESTION ENGINE |
+───────────────────────────────────────────────────────────────────────────+
| |
| [Clients] ──(HTTP/gRPC)──> [Load Balancer] |
| │ |
| ▼ |
| [Stateless Ingestion Pods] |
| │ |
| ▼ (Batch produce: 100ms / 500 msgs) |
| [Apache Kafka / Redpanda] |
| (Partitioned by Entity UUID) |
| │ |
| ▼ (Consumer Pull with Monotonic Queue) |
| [Worker Consumer Fleet] |
| │ |
| ▼ (Bulk Upsert: 2000 rows) |
| [ClickHouse / ScyllaDB] |
| |
+───────────────────────────────────────────────────────────────────────────+
5. Architectural Checklist for Senior & Staff System Design
Before concluding any system design round:
- Clarify Scale Early: Write down daily active users (DAU), read/write ratios, and payload sizes.
- Stateful vs Stateless Boundaries: Keep application services strictly stateless; delegate state to distributed caches and partitioned storage.
- Single Point of Failure (SPOF): Identify every SPOF and state how Multi-AZ failover and leader election (Raft/Paxos) prevent outages.
- Data Consistency vs Latency (CAP Theorem): Explicitly choose between Strong Consistency (CP) and Eventual Consistency (AP) based on business requirements.
Related System Design & Technical Guides
- Conquering Distributed Systems Design Interviews — Comprehensive patterns for caching, sharding, and consensus.
- Designing a Distributed Rate Limiter — Implementation walk-through of Redis Token Bucket.
- Kairo Live Architecture Practice — Practice real-time architectural defense with an AI interview copilot.