System design concepts
199 concepts explained in plain words, each linked to the exercises that train it.
Fundamentals
- Client-server model A client sends a request over the network and a server returns a response. Every web system, however large, is built from this exchange repeated millions of times.
- Latency numbers to know Reading memory takes about 100 ns, an SSD read about 100 µs, a round trip in the same datacenter about 0.5 ms and across a continent 50 to 150 ms. These orders of magnitude decide where data must live.
- Back-of-the-envelope estimation Turn users into requests per second, storage and bandwidth before drawing anything: 10 million daily users making 20 requests is about 2,300 requests per second on average and several times that at peak.
- Latency versus throughput Latency is how long one request takes; throughput is how many requests are handled per second. You can raise throughput by adding servers, but latency only drops by doing less work or moving closer to the user.
- Availability and the nines 99.9 % availability allows 8.8 hours of downtime a year, 99.99 % only 53 minutes. Components in series multiply their availability, so a chain of three 99.9 % services is only about 99.7 % available.
- Little's law The number of requests in a system equals arrival rate times time spent: 1,000 requests per second at 200 ms means 200 requests in flight. It sizes thread pools, connection pools and queues.
- Queueing and utilisation As a server approaches full utilisation, waiting time explodes: at 50 % load a request waits about as long as it is served, at 90 % about ten times longer. Keep headroom instead of running at 95 %.
- Tail latency The p99 is the time under which 99 % of requests finish. When one page calls many services, it almost always meets one slow call, so the tail, not the average, decides user experience.
Scalability
- Vertical scaling Make one machine bigger: more vCPUs, more memory. It is simple and needs no code change, but it hits a ceiling, costs more per unit at the top and keeps a single point of failure.
- Horizontal scaling Add more machines behind a load balancer. Capacity grows almost linearly and one failure only removes a fraction of it, but the application must be stateless or share its state elsewhere.
- Stateless services A stateless server keeps no session data between requests, so any instance can answer any user. Sessions go to a shared cache or a signed token, which is what makes horizontal scaling and auto-scaling possible.
- Load balancing A load balancer spreads requests across servers and removes unhealthy ones. Round robin is simple, least-connections handles uneven requests better, and hashing keeps a client on the same server at the cost of hot spots.
- Sticky sessions Sending a user always to the same server lets it keep session state in memory. It breaks even load distribution and loses sessions when that server dies, so prefer stateless services.
- Auto-scaling Adding and removing instances automatically based on a metric such as CPU. New instances take minutes to become useful, so scaling follows gradual trends well but does not absorb an instant spike alone.
- Elasticity The ability to match capacity to demand in both directions and pay only for what is used. It is the main economic argument of the cloud, and only works with stateless, quick-to-start components.
Networking and protocols
- Layer 4 versus layer 7 Layer 4 devices route TCP or UDP packets by address and port, very fast and protocol-agnostic. Layer 7 devices read HTTP, so they can route by path, terminate TLS and inspect headers, at slightly higher cost.
- DNS The Domain Name System maps names to IP addresses through a hierarchy of servers, with answers cached for their TTL. Because of caching, DNS changes and DNS-based failover take effect only as caches expire.
- Geographic routing Send each user to the region closest to them, by DNS geolocation, latency measurements or anycast. It cuts round-trip time and can keep data in the user's jurisdiction.
- Anycast The same IP address is announced from many locations and the internet routes each user to the nearest one. CDNs and global load balancers rely on it to fail over without waiting for DNS caches.
- Polling, long polling, SSE and WebSockets Polling asks repeatedly, long polling holds the request until there is news, Server-Sent Events stream updates one way, and WebSockets keep a two-way connection open. Real-time chat needs persistent connections, which are stateful to scale.
- REST, gRPC and GraphQL REST is simple and cache-friendly over HTTP; gRPC uses binary Protocol Buffers over HTTP/2 for fast service-to-service calls; GraphQL lets clients ask for exactly the fields they need in one round trip.
- API pagination and versioning Cursor-based pagination stays fast and stable on large tables, unlike offsets. Version APIs explicitly and keep old versions alive long enough for clients to migrate.
- HTTP versions and TLS HTTP/2 multiplexes requests on one connection and HTTP/3 runs over QUIC to avoid head-of-line blocking. TLS 1.3 needs a single round trip to set up, and terminating it at the load balancer offloads servers.
- Forward and reverse proxies A forward proxy acts for clients, for example to filter outbound traffic; a reverse proxy sits in front of servers to balance load, cache and terminate TLS.
- TCP and UDP TCP guarantees ordered, reliable delivery with handshakes and retransmissions; UDP sends packets without guarantees and with less overhead, which suits video, games and DNS.
- Serialisation formats JSON is readable and universal; Protocol Buffers and Avro are compact, faster to parse and have schemas that support evolution. Compression such as gzip or zstd cuts bandwidth at a small CPU cost.
- Edge computing Running logic in points of presence close to users removes the round trip to a central region for redirects, personalisation or authentication. Anything that needs central data still pays that trip.
- Private connectivity Private endpoints, peering and dedicated links keep traffic between your network and cloud services or partners off the public internet, which reduces both attack surface and data transfer costs.
Caching
- Content delivery network A CDN caches static content in hundreds of locations near users. It cuts latency from hundreds of milliseconds to a few dozen, absorbs traffic spikes and is often cheaper per gigabyte than serving from your own servers.
- Caching Keep the result of an expensive operation close to where it is needed so the next request is cheap. Caches exist at every layer: browser, CDN, application, database. Every cache trades freshness for speed.
- Cache-aside The application reads the cache first and, on a miss, reads the database and fills the cache. It is the most common pattern: the cache only holds what is requested, and a cache failure degrades to slower reads rather than errors.
- Write-through Every write goes to the cache and the database together, so the cache is always fresh. Writes become slower, and data that is written but never read still occupies memory.
- Write-behind Writes go to the cache and are flushed to the database later in batches. Writes are very fast, but anything not yet flushed is lost if the cache node fails, so it suits counters more than payments.
- Eviction policies When memory is full, the cache must drop something. LRU removes the least recently used key, LFU the least frequently used, which keeps popular items better under scans. No eviction means writes fail instead.
- Time to live A TTL makes cached data expire after a set time. Short TTLs keep data fresh but lower the hit ratio; long TTLs do the opposite. Choose per type of data, not one value for everything.
- Cache invalidation Removing or updating cached copies when the source changes is famously hard: copies live in many places and updates race with reads. Common answers are short TTLs, deleting on write and versioned keys.
- Cache stampede When a popular key expires or the cache restarts, thousands of requests miss at the same moment and hit the database together. Request coalescing, early refresh and staggered TTLs prevent it.
Data and storage
- Hot keys and hot partitions When one key, such as a celebrity's profile, receives a large share of traffic, the partition or shard holding it saturates while others idle. Replicate hot keys, add a random suffix or cache them close to the app.
- Relational databases Data lives in tables linked by keys and is queried with SQL, joins included. Strong guarantees and a rigid schema make them the default for business data such as orders, accounts and payments.
- ACID transactions Atomicity, Consistency, Isolation, Durability: a transaction happens entirely or not at all, leaves data valid, does not see half-finished work of others and survives a crash once committed.
- BASE Basically Available, Soft state, Eventually consistent: the philosophy of many NoSQL systems that favour availability and scale over immediate consistency.
- Isolation levels Read committed avoids dirty reads, repeatable read also avoids values changing under you, and serializable makes transactions behave as if run one by one. Stronger levels prevent anomalies but cost throughput through locks and aborts.
- Indexes An index is a sorted structure, usually a B-tree, that finds rows without scanning the table. It turns a seconds-long query into a millisecond one, but every index slows writes and uses storage.
- B-trees Balanced trees with wide nodes that keep data sorted on disk, giving logarithmic lookups and range scans. They are the storage engine of most relational databases and are read-optimised.
- LSM trees Log-structured merge trees buffer writes in memory and flush them as sorted files that are merged in the background. Writes are very fast; reads may check several files, helped by Bloom filters.
- Write-ahead log Every change is appended to a log before being applied, so a crash can be recovered by replaying it. The same log feeds replication and change data capture.
- Multi-version concurrency control The database keeps several versions of a row so readers never block writers: each transaction sees a consistent snapshot. Old versions must be cleaned up, which is what vacuum does in PostgreSQL.
- Normalisation Store each fact once and link it by keys, which avoids inconsistent copies. Reads then need joins, which is why read-heavy systems sometimes denormalise on purpose.
- Denormalisation Duplicate data so a read needs one lookup instead of several joins. It speeds up reads at the cost of more storage and the job of keeping copies in sync.
- Connection pooling Opening a database connection is expensive and each one uses server memory. A pool such as PgBouncer shares a few connections among many application threads, so adding app servers does not exhaust the database.
- Replication Keep copies of the data on several machines for availability and read capacity. Leader-follower sends all writes to one node, multi-leader accepts writes in several places, and leaderless lets any replica accept them.
- Read replicas Followers that receive the leader's changes and serve reads. They scale read-heavy workloads cheaply, but replication lag means a user may not see their own recent write unless those reads go to the leader.
- Synchronous and asynchronous replication Synchronous replication waits for replicas before confirming a write: no data loss on failover, but higher latency. Asynchronous confirms immediately: faster, but the last writes can be lost if the leader dies.
- Leaderless replication Clients write to and read from several replicas directly, as in Dynamo and Cassandra. With read repair and hinted handoff the replicas converge; quorums decide how consistent each operation is.
- Partitioning strategies Range partitioning keeps related keys together for scans but creates hot spots on recent keys; hash partitioning spreads load evenly but loses ordering; directory-based partitioning is flexible but adds a lookup.
- Consistent hashing Keys and nodes are placed on a ring, and each key belongs to the next node clockwise. Adding or removing a node moves only about 1/N of the keys instead of nearly all of them; virtual nodes even out the load.
- Rebalancing Moving data when nodes are added or removed. It consumes network and disk while the system serves traffic, so it must be throttled and is far cheaper with consistent hashing or fixed partition counts.
- NoSQL families Key-value, document, wide-column and graph stores each drop some relational features to gain scale or flexibility. Pick by access pattern: how will the data be read and written, at what volume?
- Polyglot persistence Store each kind of data in the database whose guarantees it needs: strongly consistent SQL for money, a wide-column store for high-volume events, a search engine for text. One database for everything imposes the most expensive guarantees, and their cost, on all of it.
- Key-value stores The simplest model: get and put a value by its key. Extremely fast and easy to partition, but you can only query by key, so the access patterns must be known upfront.
- Document databases Each record is a JSON-like document with its own structure, queried by fields and secondary indexes. Great when the data is naturally hierarchical; awkward when you need many-to-many relationships.
- Graph databases Nodes and edges are first-class, so traversing relationships several hops deep stays fast. Used for social networks, recommendations and fraud detection.
- Time-series data Measurements indexed by time, written in huge volumes and read as aggregates over ranges. Specialised engines compress them heavily and expire old data automatically.
- Downsampling Replace old high-resolution points with coarser aggregates, such as per-minute averages after a week, to keep storage and query time under control.
- Object storage Files are stored as objects with a key and metadata in a flat namespace, served over HTTP. It is the right home for images, videos, backups and data lake files, not for data you update in place.
- Storing files outside the database Put the file in object storage and keep only its key and metadata in the database. Databases are expensive per gigabyte and slow to back up when they hold large binaries.
- Durability The guarantee that confirmed data is not lost. It comes from writing to disk, replicating across machines and zones, and keeping backups; eleven nines of durability means losing one object in a hundred billion per year.
- Erasure coding Data is split into fragments plus parity fragments, so it survives the loss of several disks while using far less space than full copies. Object stores use it to reach extreme durability cheaply.
- Inverted index A map from each word to the documents that contain it. It is what lets a search engine find all documents containing several words in milliseconds.
- Full-text search Searching by words with stemming, typo tolerance and relevance ranking. SQL LIKE queries cannot do this at scale; a search engine kept in sync from the primary database can.
- OLTP versus OLAP OLTP serves many small transactions for users in milliseconds; OLAP runs a few huge analytical queries over history. Running analytics on the production database slows down users, so the two are separated.
- Columnar storage Storing each column together makes scanning one column across billions of rows fast and compresses very well, which is why warehouses use it. Fetching one full row is slower.
- ETL and ELT Extract data from sources, transform it and load it into a warehouse, or load first and transform inside the warehouse. These pipelines run in batches or continuously, outside the user request path.
- Data pipelines Chains of jobs that move and reshape data between systems. They need idempotent steps, monitoring of freshness and a way to backfill history.
- Data lake A store of raw data in open formats on object storage, cheap enough to keep everything. Table formats add transactions and schema evolution on top.
- Schema on read Data is stored as it arrives and a schema is applied when it is queried. Flexible for exploration, but errors surface late and quality depends on discipline.
- Batch processing Process large bounded datasets in one go, for example every night, with frameworks like Spark or MapReduce. High throughput and simple recovery, but results are hours old.
- Change data capture Stream every insert, update and delete from the database log to other systems. It keeps caches, search indexes and warehouses in sync without dual writes from the application.
- Home region and write forwarding Each record has a home region that owns its writes, often for legal or consistency reasons. Other regions serve reads from local copies and forward writes to the home region asynchronously, trading a few seconds of freshness for local latency.
- Write concern and acknowledgement How many replicas must confirm a write before the client is told it succeeded. Waiting for a majority survives a failover; waiting for the primary only is faster but can lose acknowledged writes.
- Vector search Text, images and products are turned into embeddings, and similar items are those whose vectors are close. Approximate nearest-neighbour indexes such as HNSW make this fast at scale with a small loss of recall.
- Distributed and network file systems HDFS spreads large files across many machines for batch processing; NFS-style services let several servers mount the same files. Both are slower than local disks and object storage is usually a better default.
Consistency and consensus
- CAP theorem When a network partition separates nodes, a distributed system must choose between consistency (refuse requests it cannot serve correctly) and availability (answer, possibly with stale data). Partitions are not optional, so the real choice is C or A during one.
- PACELC Extends CAP: if there is a Partition, choose Availability or Consistency; Else, in normal operation, choose Latency or Consistency. Waiting for more replicas to agree is always slower, even when nothing is broken.
- Strong consistency Every read returns the latest confirmed write, as if there were a single copy. It requires coordination between replicas, which costs latency and availability during failures.
- Linearisability The strictest single-object guarantee: once a write completes, every later read anywhere sees it, in real-time order. Needed for locks, leader election and unique constraints.
- Eventual consistency If writes stop, all replicas eventually converge to the same value, but in the meantime reads can be stale. Fine for likes or view counts, dangerous for bank balances.
- Read-your-writes A user always sees their own updates, even if others may see them later. Achieved by reading from the leader for a short time after a write, or by sticking the user to an up-to-date replica.
- Monotonic reads A user never sees data go back in time, which can happen when successive reads hit replicas with different lag. Pinning a session to one replica prevents it.
- Causal consistency Operations that depend on each other are seen in the same order by everyone: nobody sees a reply before the message it answers. Stronger than eventual, cheaper than linearisable.
- Quorums With N replicas, writing to W and reading from R replicas guarantees that a read sees the latest write whenever R + W > N. With N = 3, QUORUM reads and writes (2 + 2 > 3) are consistent; ONE and ONE are not.
- Tunable consistency Some databases let you choose the consistency level per query. Use strong levels for money and inventory, weak ones for feeds and analytics, and accept the latency difference consciously.
- Consensus Getting several machines to agree on one value despite failures. It needs a majority, so a cluster of 2f + 1 nodes tolerates f failures, and it is what makes leader election and distributed SQL safe.
- Raft and Paxos The two classic consensus algorithms. Raft elects a leader that replicates a log to followers and commits an entry once a majority has stored it; it is used by etcd, Consul and CockroachDB.
- Leader election Choosing one node to coordinate work, such as accepting writes or running a scheduled job. It relies on consensus and leases so that a new leader is chosen when the old one stops responding.
- Split brain Two nodes both believe they are the leader, usually after a network partition, and accept conflicting writes. Majority quorums and fencing tokens prevent it.
- Fencing tokens Each new lock holder receives a higher number, and storage rejects writes carrying an older one. A paused old leader that wakes up cannot corrupt data.
- Distributed locks Mutual exclusion across machines, built on a consensus store with leases. Locks expire if the holder crashes, so they must be combined with fencing tokens to be safe.
- Vector clocks Each replica keeps a counter per node, which lets the system tell whether two versions are ordered or concurrent. Concurrent versions are conflicts that must be merged.
- Clock skew Machine clocks drift apart by milliseconds or more, so timestamps from different servers cannot be trusted to order events. Last-write-wins based on wall clocks silently loses data.
- CRDTs Conflict-free replicated data types, such as counters and sets, that merge concurrent updates automatically and deterministically. They power collaborative editing and multi-leader replication.
- Gossip protocols Nodes periodically exchange state with a few random peers, so information spreads to the whole cluster in a logarithmic number of rounds. Used for membership and failure detection.
- Merkle trees A tree of hashes where each parent hashes its children. Two replicas compare root hashes and walk down only the branches that differ, which makes anti-entropy repair cheap.
Algorithms and data structures
- Bloom filters A compact bit array that answers 'definitely not present' or 'maybe present'. Databases use them to skip files that cannot contain a key, saving disk reads.
- HyperLogLog and count-min sketch Probabilistic structures that count distinct visitors or estimate top items in a few kilobytes, with about 1 % error. Exact counting at that scale would need gigabytes.
- Geospatial indexing Geohashes, quadtrees and S2 cells turn a position into a key so nearby points share a prefix. That is how ride-hailing and delivery apps find drivers near you quickly.
- Unique ID generation Auto-increment does not work across shards. Snowflake IDs, UUIDv7 and ULIDs are generated independently on each server, stay unique and sort roughly by creation time.
Messaging and streaming
- Message queues A queue decouples the producer from the consumer: the request is accepted quickly, and the slow work happens later at the pace workers can sustain. It smooths spikes and isolates failures.
- Asynchronous processing Answer the user as soon as the work is safely recorded, then process it in the background. Latency drops and capacity can be sized for the average instead of the peak, at the cost of showing pending states.
- Delivery guarantees At-most-once may lose messages, at-least-once may duplicate them, and exactly-once processing is achieved by combining at-least-once delivery with idempotent handling or transactions.
- Exactly-once processing The effect of each message happens once even if it is delivered twice. Systems get there with idempotent writes, deduplication keys or transactional offsets, never by trusting the network.
- Dead-letter queues Messages that fail processing a set number of times are moved aside for inspection, so one bad message does not block the queue or burn worker capacity forever.
- Poison messages A message that crashes its consumer every time, for example because of malformed data. Without a retry limit it is redelivered endlessly.
- Publish-subscribe Producers publish events to a topic without knowing who consumes them, and every subscriber receives its own copy. New features subscribe without touching the producer.
- Fan-out on write and on read A social feed can be built by pushing each post into every follower's feed at write time, which makes reads cheap, or by gathering posts at read time, which suits accounts with millions of followers. Large systems mix both.
- Event streaming An append-only log of events kept for days or forever, that many consumers read at their own pace and can replay. It becomes the backbone that feeds caches, search, analytics and other services.
- Ordering guarantees Global ordering does not scale; most systems guarantee order per key by sending all events of one key to the same partition. Choose the key so that events that must stay ordered share it.
- Consumer groups Consumers in a group share the partitions of a topic, each partition read by one consumer. Adding consumers beyond the number of partitions leaves them idle.
- Stream processing Computing results continuously as events arrive: running counts, fraud detection, live dashboards. State and checkpoints let jobs recover without losing their place.
- Windowing Grouping an endless stream into tumbling, sliding or session windows so aggregates have a beginning and an end. Late events need a policy: wait with watermarks or correct afterwards.
Reliability
- Single point of failure Any component whose failure stops the whole system. Find them by asking, for every box, what happens if it disappears; remove them with redundancy across instances, zones or regions.
- N+1 capacity Size every tier so that it still carries the peak after losing its largest unit, whether an instance, a zone or a whole region. Two regions each sized for half of the traffic are not redundant: when one fails, the other collapses under twice its load.
- Health checks Periodic probes tell the load balancer which instances can take traffic. A shorter interval detects failures faster but adds load; a deep check that tests the database can take every server out at once if the database blips.
- Rate limiting Cap the requests a client may send per second. Token bucket allows short bursts, leaky bucket smooths output, fixed windows are simple but let double bursts through at window edges, and sliding windows fix that at a small memory cost.
- Backpressure When a component cannot keep up, it signals upstream to slow down instead of silently queueing forever. Bounded queues, 429 responses and pull-based consumers are all forms of backpressure.
- Thundering herd Many clients wake up and retry at exactly the same moment, for example after an outage, and knock the recovering service down again. Jitter, backoff and rate limits spread them out.
- Failover Automatically moving traffic or the leader role to a standby when the primary fails. It must detect the failure reliably, avoid two leaders at once and accept that a few seconds of requests fail.
- Idempotency An operation is idempotent if doing it twice has the same effect as doing it once. An idempotency key on a payment request lets clients retry safely without charging the customer twice.
- Circuit breaker After too many failures, stop calling a dependency for a while and fail fast or return a fallback, then probe it again. It prevents threads from piling up on timeouts and lets the dependency recover.
- Timeouts Every network call needs a deadline, otherwise one slow dependency holds threads forever. Set it from the dependency's p99, and make sure the total of nested timeouts fits within the user's patience.
- Retries, backoff and jitter Retrying hides transient failures, but immediate retries multiply load on a struggling service. Exponential backoff spaces attempts out and random jitter stops clients from retrying in lockstep.
- Cascading failures One slow component makes its callers wait, they run out of threads and slow down their own callers, until the whole system is down. Timeouts, circuit breakers, bulkheads and load shedding contain it.
- Bulkheads Isolate resources, such as thread pools or instances, per dependency or per customer, so one overloaded part cannot sink the others, like compartments in a ship.
- Graceful degradation When a non-essential dependency fails, keep serving the core feature with less: hide recommendations, show cached prices. Users notice a degraded page far less than an error page.
- Load shedding Deliberately reject part of the traffic when overloaded, preferably the least important, so the rest keeps fast responses. Better to serve 80 % well than 100 % badly.
- Self-healing The platform notices failed instances and replaces them without human action. Combined with redundancy, a crash becomes a non-event instead of an incident.
- Multi-region architecture Running in several regions protects against a regional outage and brings data closer to users. The hard part is data: synchronous replication adds cross-region latency to writes, asynchronous replication risks losing recent writes.
- Active-active and active-passive Active-passive keeps a standby region that takes over on failure: simpler, but the standby is idle and failover takes minutes. Active-active serves traffic from every region at once: better latency and instant failover, but write conflicts must be handled.
- Static stability A system is statically stable when it keeps working with what it already has while something it depends on is down: another region, a control plane, a configuration service. Each region runs its own copy of every dependency it calls at request time and is provisioned in advance, so nothing has to start or be reached during an outage.
- Disaster recovery strategies From cheapest to fastest: backup and restore, pilot light, warm standby, multi-site active-active. Each step lowers recovery time and data loss and raises cost.
- RPO and RTO The Recovery Point Objective is how much data you can afford to lose, the Recovery Time Objective how long you can be down. A five-minute RPO rules out daily backups as the only protection.
- Blast radius The share of users that a single failure, bad deployment or migration can hurt. Cells, shards, per-team databases and staged rollouts cap it, so that incidents become partial instead of total.
- Heartbeats and failure detection Nodes send periodic signals; missing several means a node is presumed dead. It is impossible to tell a dead node from a slow one, so detectors trade detection speed against false alarms.
- Backups and restore drills Replication copies mistakes as faithfully as data, so only backups protect against deletions, corruption and ransomware. Keep copies in another account or region, and rehearse restores to know the real recovery time.
- Third-party dependencies External APIs bring their latency, limits and outages into your system. Isolate them with timeouts, circuit breakers and queues, cache their answers where possible, and decide what your product does when they are down.
Architecture patterns
- API gateway A single entry point that handles authentication, quotas, routing and request transformation for many backend services. It simplifies clients and services, but becomes critical infrastructure that must itself scale.
- Backend for frontend A dedicated API layer per client type, such as mobile and web, shaped for that client's screens. It avoids chatty clients and lets each team evolve its API independently.
- Event-driven architecture Services communicate by emitting facts that happened, such as OrderPlaced, instead of calling each other. Coupling drops and new consumers are easy to add, but end-to-end flows become harder to follow.
- Loose coupling Components that know little about each other can change, scale and fail independently. Queues, events and stable APIs are the main tools.
- Event sourcing Store the sequence of events that changed an entity instead of its current state, and rebuild state by replaying them. You get a full audit trail and time travel, but schema evolution and queries need care.
- CQRS Command Query Responsibility Segregation separates the write model from one or more read models optimised for queries, kept in sync by events. Reads scale independently, at the cost of eventual consistency.
- Lambda and Kappa architectures Lambda runs a batch layer for accuracy next to a speed layer for freshness; Kappa uses a single streaming pipeline and replays the log to recompute. Kappa is simpler when the stream processor is reliable.
- Saga pattern A long business transaction split into local steps, each with a compensating action if a later step fails, such as refunding a payment when shipping is impossible. It replaces distributed locks across services.
- Two-phase commit A coordinator asks every participant to prepare, then tells all to commit. It is atomic across systems but blocks if the coordinator fails, so it is avoided across services.
- Distributed transactions Keeping data consistent across several databases or services. Options range from two-phase commit and consensus-based SQL to sagas and the outbox pattern, trading simplicity, latency and availability.
- Transactional outbox Write the business change and the event to publish in the same database transaction, then relay the event to the broker. It avoids losing events or publishing events for changes that rolled back.
- Durable execution Workflow engines persist each step of a long process so that it resumes after crashes, timeouts or deployments, which removes a lot of hand-written retry and state code.
- Serverless You deploy functions and the provider runs, scales and bills them per execution. No servers to manage and near-zero cost at idle, but cold starts, execution limits and per-request pricing at high volume.
- Cold starts The first request to a new function or container instance pays for loading the runtime and code, from about 100 ms in Go to around a second in Java. Provisioned capacity keeps instances warm.
- Containers An application packaged with its dependencies runs the same everywhere and starts in seconds. Orchestrators such as Kubernetes schedule containers, restart them and spread them across zones.
- Monolith and microservices A monolith is one deployable unit: simple to build, test and operate. Microservices split it into independently deployable services owned by teams, which helps large organisations but adds network calls, distributed data and operational cost.
- Service mesh Sidecar proxies next to each service handle retries, timeouts, mutual TLS and telemetry for service-to-service traffic, so this logic is not rewritten in every codebase.
- Service discovery Instances register themselves and clients look up current addresses, through DNS, a registry or the orchestrator. Static IP lists break as soon as auto-scaling replaces instances.
- Cell-based architecture The system is replicated into independent cells, each serving a slice of customers. A bad deployment or a failure affects one cell only, which caps the blast radius.
- Multi-tenancy One system serves many customers. Shared databases are cheap but risk noisy neighbours and data leaks; a database per tenant isolates well but multiplies operations.
- Strangler fig migration Replace a legacy system gradually by routing one feature at a time to the new implementation behind a facade, until the old system can be switched off, with no big-bang cutover.
- Conway's law Systems mirror the communication structure of the organisation that builds them. Design team boundaries and service boundaries together.
- Architecture decision records A short document per significant decision: context, options considered, choice and consequences. Future engineers learn why, not just what, and decisions can be revisited deliberately.
- Twelve-factor apps Principles for cloud-friendly services: config in the environment, stateless processes, backing services as attached resources, logs as streams, fast start-up and graceful shutdown.
- Serving machine learning models Inference is GPU-bound and slow per request, so capacity is planned in requests per GPU. Dynamic batching, response caching, smaller models and autoscaling on queue depth keep latency and cost under control.
- Notification systems Emails, SMS and push notifications are sent asynchronously from queues, with per-channel providers, retries, user preferences and deduplication so nobody receives the same alert twice.
Security
- Defense in depth Stack independent security layers, such as WAF, network isolation, authentication, encryption and least privilege, so that one failure does not expose everything.
- OWASP Top 10 The reference list of the most common web vulnerabilities: broken access control, injection, insecure design and more. A WAF helps, but most fixes belong in the code.
- Data residency and privacy law Laws such as Quebec's Law 25 and the GDPR govern where personal data may be stored and who may access it. Architecture must know which region holds each user's data and support deletion requests.
- Authentication Proving who a user or service is, with passwords, MFA, single sign-on or certificates. Centralise it in an identity provider rather than in each service.
- OAuth 2.0 and OpenID Connect OAuth 2.0 lets an application act on a user's behalf with limited scopes; OpenID Connect adds login on top of it. They power 'Sign in with Google' and API access tokens.
- JSON Web Tokens Signed tokens that carry claims, so any service can verify a user without calling the identity provider. Keep them short-lived, because a signed token cannot easily be revoked.
- Encryption in transit and at rest TLS protects data moving over the network; disk and database encryption protect stored data. Both are expected by default today and nearly free in managed services.
- Envelope encryption Data is encrypted with a data key, and the data key is encrypted with a master key kept in a KMS. Rotating the master key does not require re-encrypting all the data.
- Secrets management Credentials belong in a dedicated vault with access control, audit and rotation, not in code, images or plain environment files.
- Least privilege Every user, service and machine gets only the permissions it needs. When something is compromised, the damage stays small.
- Network segmentation Separate public, application and data tiers into subnets with strict rules between them. A compromised web server should not be able to reach the database directly.
- Presigned URLs The server signs a temporary URL so the browser uploads or downloads a file directly from object storage. Large files no longer pass through your application servers.
- DDoS protection Distributed denial-of-service attacks flood a system with traffic from many machines. Volumetric floods are absorbed by a provider's global network; application-layer floods are filtered by rate limits, WAF rules and bot detection.
- Zero-trust access No network location is trusted by default: every access to a server, database or tool is authenticated, authorised and logged, with short-lived credentials instead of long-lived keys.
Observability
- Observability Metrics, logs and traces together let you answer new questions about a running system without shipping new code. You cannot fix what you cannot see.
- The four golden signals Latency, traffic, errors and saturation describe the health of almost any service. Alert on them before alerting on CPU or memory.
- SLI, SLO and SLA An SLI is a measurement, such as the share of requests under 300 ms; an SLO is the target for it, such as 99.9 % per month; an SLA is the contractual promise, with penalties. The gap between 100 % and the SLO is the error budget.
- Structured logging Logs written as key-value fields instead of free text can be filtered and aggregated. Include a request ID in every line so one request can be followed across services.
Operations
- Feature flags Configuration switches that turn code paths on or off at runtime, per user or percentage. They separate deployment from release and provide an instant kill switch.
- Canary and blue-green releases Canary sends a small share of traffic to the new version and watches errors before going further; blue-green keeps two full environments and switches traffic at once, with instant rollback.
- Incident response Detect, mitigate first, communicate, then fix and learn. A blameless postmortem and an updated runbook are what turn an outage into lasting improvement.
- MTTR and MTBF Mean time to recovery and mean time between failures. Availability depends on both, and in practice it is usually far cheaper to shorten recovery than to prevent every failure.
- Chaos engineering Injecting failures on purpose, such as killing instances or adding latency, to verify that the system copes before a real incident does it for you. This simulator is a chaos lab.
- Capacity planning Forecast load from growth and seasonality, measure what one instance can handle, then keep enough headroom for peaks and for losing a zone.
- Load testing Replay realistic traffic at increasing volume to find the breaking point before users do. Measure p99 latency and errors, not just average response time.
- Cloud cost optimisation Right-size instances, use reservations or savings plans for steady load, autoscale the rest, and watch data transfer, which often surprises. Cost per request is the metric that matters.
No concept matches these filters.