Cloud cost optimisation

Right-size instances, use reservations or savings plans for steady load, autoscale the rest, and watch data transfer, which often surprises. Cost per request is the metric that matters.

Where it shows up in real systems

  • NAT gateway

    A classic surprise on cloud bills: every gigabyte that leaves through NAT is charged on top of the hourly price. Private endpoints to cloud services avoid much of that traffic.

    In production: AWS NAT Gateway, Azure NAT Gateway, Cloud NAT

  • Model inference service

    Each request takes tens to hundreds of milliseconds of GPU time, so a few hundred requests per second already need several expensive GPUs. Batching requests, caching answers and choosing the right accelerator change the bill by an order of magnitude.

    In production: NVIDIA Triton, vLLM, Amazon SageMaker endpoints, Google Vertex AI, KServe

  • Private endpoint

    Traffic to storage, queues or partner APIs stays on the provider backbone instead of crossing the internet through a NAT gateway: safer, and often much cheaper per gigabyte.

    In production: AWS PrivateLink, Azure Private Link, Google Private Service Connect, VPC gateway endpoints

See it break, then fix it

Build an architecture, send it a day of traffic and outages, and learn why it holds or fails. Free to start.

Open the learning path Open the sandbox