Advanced
Cold starts in the tail
A bank moved its account API to serverless functions written in Java. The average latency looks great, yet the mobile team misses its 300 ms p99 target by a factor of three. Bring the tail down without blowing the 3,000-dollar monthly budget.
Traffic follows office hours, up to 1,000 requests per second. Each call keeps a function instance busy for about 15 ms, and every new instance first loads the Java runtime.
What your design must achieve
- Latency p99 < 300 ms
- Availability ≥ 99.9 %
- Infrastructure cost per month < 3,000 $