Question 1

What is a circuit breaker in system design?

Accepted Answer

A circuit breaker is a resilience pattern that monitors downstream service health and automatically stops routing requests when failure rates exceed a threshold. It transitions through three states: Closed (normal), Open (blocking requests to protect the downstream), and Half-Open (probing with limited traffic to test recovery). ScalingSutra's API Gateway sandbox lets you simulate circuit breaker state transitions interactively.

Question 2

How does Raft consensus algorithm work?

Accepted Answer

Raft is a distributed consensus algorithm that ensures all nodes in a cluster agree on a sequence of log entries. It works through leader election (nodes campaign after a randomised timeout), log replication (the leader replicates entries to followers before committing), and quorum commits (an entry is committed once a majority of nodes acknowledge it). ScalingSutra's Raft sandbox lets you trigger elections, pause nodes, and simulate network partitions.

Question 3

What is the difference between 2PC and Saga pattern?

Accepted Answer

Two-Phase Commit (2PC) uses a coordinator to achieve strong consistency by locking resources across all participants before committing — it is synchronous but can cause blocking under failures. The Saga pattern breaks a distributed transaction into local transactions each publishing an event; if a step fails, compensating transactions undo prior steps. Sagas trade strong consistency for higher availability and no distributed locking.

Question 4

How does database sharding work?

Accepted Answer

Database sharding horizontally partitions rows across multiple database instances to scale write throughput and storage. Range sharding assigns rows by key ranges which is prone to hotspots. Hash sharding applies a hash function for even distribution but complicates range queries. ScalingSutra's DB Sharding sandbox lets you switch strategies and observe replication lag and stale reads.

Question 5

What is the thundering herd problem in caching?

Accepted Answer

The thundering herd (cache stampede) occurs when a popular cached key expires simultaneously for many concurrent requests. All requests find a cache miss and simultaneously query the database, causing a spike. Mitigations include the singleflight pattern (coalescing duplicate in-flight requests), probabilistic early expiry, and mutex-based locks on cache population.

Question 6

What is Retrieval-Augmented Generation (RAG) and how does it work?

Accepted Answer

RAG enhances LLM responses by retrieving relevant documents at inference time and injecting them into the prompt context. The pipeline involves: ingesting documents and splitting them into chunks, embedding chunks as vectors and storing them in a vector database, then at query time embedding the question, retrieving the top-k nearest chunks via ANN search (HNSW or IVF), and injecting them into the LLM prompt as grounding context. ScalingSutra's AI course covers the full RAG pipeline across Chapters 6, 7, and 8.

Question 7

What is the ReAct pattern for AI agents?

Accepted Answer

ReAct (Reason + Act) is an agent framework where the LLM alternates between Thought steps (reasoning about what to do) and Action steps (calling external tools like calculators, search engines, or databases). The tool returns an Observation which feeds back into the next Thought. This loop continues until the agent produces a Final Answer. ScalingSutra's Chapter 9 visualises the full ReAct loop with tool error injection and retry simulation.

Question 8

What is LangGraph and how does it differ from LangChain?

Accepted Answer

LangGraph is a library built on top of LangChain that models LLM workflows as stateful directed graphs (StateGraph). Unlike LangChain chains which are linear, LangGraph supports cycles (loops), conditional branching via add_conditional_edges, and persistent checkpointing via SqliteSaver. This makes it ideal for self-correcting pipelines like Corrective RAG (CRAG) and human-in-the-loop workflows. ScalingSutra's Chapter 10 visualises the full CRAG graph with live TypedDict state mutations.

Question 9

How does KV cache work in LLM inference?

Accepted Answer

During LLM inference there are two phases. Prefill: all prompt tokens are processed in parallel to compute Q, K, and V matrices. Decode: the model generates one token at a time. Instead of recomputing K and V for all previous tokens on each decode step, the KV cache stores them in GPU VRAM — reducing computation from O(n²) to O(n) per step. Memory cost is 2 × layers × heads × head_dim × bytes_per_element × sequence_length. INT8 and INT4 quantisation halve or quarter memory at a small accuracy cost. Prefix caching reuses K/V for repeated system prompts.

Question 10

What are AI safety guardrails and why are they important?

Accepted Answer

AI safety guardrails are layered filters applied around an LLM to prevent misuse and data leaks. Key layers include: Prompt Injection filters (detecting attempts to override system instructions), PII Detectors (scrubbing personally identifiable information from input and output using tools like presidio), Jailbreak Classifiers (catching role-play bypass attempts with fine-tuned binary classifiers), and Toxicity Filters (scoring LLM output against thresholds using models like Detoxify or the Perspective API). Defence in depth means no single filter is perfect — stacking independent layers forces adversaries to bypass all simultaneously.

Question 11

How does token bucket rate limiting work?

Accepted Answer

Token Bucket rate limiting maintains a bucket that refills at a fixed rate (tokens per second). Each incoming request consumes one token. If the bucket is empty, the request is rejected or queued. The bucket has a maximum capacity, allowing short bursts up to the bucket size while enforcing the average rate. This differs from Leaky Bucket (which smooths traffic to a constant output rate) and Fixed Window (which resets a counter every time window).

Question 12

How does Kafka consumer group rebalancing work?

Accepted Answer

When a Kafka consumer joins or leaves a consumer group, the group coordinator triggers a rebalance — redistributing topic partition assignments across all active consumers. During a rebalance, consumption pauses. The new assignment ensures each partition is owned by exactly one consumer in the group. Strategies include Range (contiguous partition blocks per consumer) and RoundRobin (interleaved assignment for better balance).