Capacity Planning Starts at the First Saturated Resource

Capacity is the maximum useful demand a service can sustain while meeting its reliability target, and it is set by the first resource that saturates. Count requests per second only after connecting them to concurrency, work per request, memory residency, connection pools, queues, and downstream limits. Then preserve headroom for bursts, failover, and forecast error.

Measure a service envelope

Choose a representative demand unit: search query, checkout, job, or bytes ingested. Split it into workload classes because a cache hit and a complex export consume different resources. For each class, record arrival rate, service time, CPU time, allocated and retained memory, storage I/O, network bytes, open connections, and calls to constrained dependencies.

Concurrency follows arrival rate and time in the system. If 200 requests arrive each second and each occupies a worker for 250 milliseconds, average in-flight work is about 50 before burstiness and queueing. A pool of 60 may look adequate until latency rises, increasing concurrency further and consuming the remaining slots. This feedback is why saturation, not average utilization, defines the edge.

Google’s SRE discussion of intent-based capacity planning connects demand, dependencies, performance metrics, and placement constraints. Preserve those relationships in the model: one unit of frontend demand may create several database queries and a downstream API call. A forecast for the frontend must propagate through that dependency graph.

Find the bending point experimentally

Run a step load that increases each workload class while measuring useful throughput, success rate, latency percentiles, queue age, retries, and every candidate saturated resource. The bending point is where more offered load stops producing proportional successful throughput or violates the SLO. Record the first limiting signal and the configuration in effect.

Google’s handling overload chapter explains that accepting work beyond capacity can make a service less efficient and that load shedding should protect useful work. In a test, include overload controls. Without them, retry storms and long queues can turn a clear capacity edge into a prolonged collapse. Reject early with a bounded response and ensure callers use backoff and jitter.

Do not infer capacity from CPU alone. A service at 35% CPU can be out of database connections, memory, file descriptors, thread-pool slots, I/O bandwidth, or vendor quota. Likewise, 90% CPU can be acceptable if latency and scheduling remain stable and failover headroom exists. Capacity is a vector constrained by the minimum margin.

Convert the measurement into a forecast

For each resource, calculate demand per unit and divide usable supply by that coefficient. Use the smallest result as the current service limit. Then subtract reserves: the capacity needed to lose a zone or node, a burst allowance derived from traffic history, deployment overlap, and a forecast-error margin. Avoid adding percentages blindly; explain which failure or uncertainty each reserve covers.

Calibrate the model against production. Compare predicted and observed utilization at several demand levels, then update coefficients when code, instance type, cache hit rate, payload mix, or dependency behavior changes. Autoscaling helps allocate supply, but it does not eliminate startup time, quota ceilings, or a shared dependency that cannot scale at the same rate.

Make the model operational

Publish a small table per workload class with present demand, peak forecast, limiting resource, tested limit, usable limit after reserves, and expansion lead time. Alert on declining margin and on coefficient drift. Rehearse failover under realistic load; a plan that needs spare capacity only available in the failed zone is not a plan.

The next step is to load-test one representative workload until the first measured bend, identify that resource, and compare the observed limit with last month’s production model. Review by 2026-12-05 or after a major runtime, instance, or workload change.