Partitioning strategies

Where it shows up in real systems

• Key-value store

  Data is spread across partitions by the hash of its key, so throughput scales almost without limit as long as keys are well distributed. A single very popular key hits one partition's limit and gets throttled. On-demand mode follows traffic but absorbs sudden jumps above twice the previous peak with throttling.

  In production: Amazon DynamoDB, Azure Cosmos DB, Google Bigtable, Riak, FoundationDB

• Event log

  Events are kept for the retention period, so new consumers can replay history. Order is guaranteed only within a partition, and a consumer group can use at most one consumer per partition: partitions cap your parallelism.

  In production: Apache Kafka (Confluent, Amazon MSK), Amazon Kinesis, Redpanda, Azure Event Hubs, Apache Pulsar

• Geospatial index

  Positions are encoded as geohashes or S2 cells so nearby points share a key prefix. Keeping moving objects in memory, such as drivers updating every few seconds, avoids hammering the main database with writes.

  In production: Redis GEO, PostGIS, Elasticsearch geo queries, Uber H3, Google S2