System design

Expert

Telemetry that survives a region

A smart-meter platform ingests 50,000 readings per second from North American homes through two identical regional pipelines, in Virginia and Montréal, behind a global load balancer. Each pipeline was sized for half of the meters. At 14:00 Virginia disappears for two hours and every meter reconnects to Montréal. Do not drop or double-count a single reading.

Each partition of the event log accepts about 10,000 events per second, and a consumer group reads each partition with exactly one processor. Partitions cannot be added during an incident without reshuffling keys. Billing aggregates must stay less than a minute behind.

What your design must achieve

  • Availability ≥ 99.95 %
  • Worst queue lag < 60 s
  • Duplicated side effects = 0
  • Writes p99 < 200 ms

What you will master