Measure CI Queue Time Separately From Test Time

Measure queue delay before optimizing tests

A twelve-minute job that waits forty minutes is a capacity problem, not a test-speed problem. Record workflow created, job queued, assigned, started, completed, retried, and approved timestamps for each runner label and branch class. GitHub Actions concurrency supplies the primary specification or operating model for this decision.

Diagnose where GitHub Actions jobs wait

Plot queue percentiles against concurrent demand and available runners. Concurrency groups may intentionally serialize work, while missing labels or dependency chains create other queues. Autoscaling should respond to queued-job events and measure cold-start delay. GitHub self-hosted runners provides the complementary protocol or operational detail.

Separate waiting from execution

Emit four timestamps for every job: workflow created, runner assigned, step started, and workflow completed. Report queue time by workflow, branch class, runner label, and hour. A ten-minute test suite with forty minutes of waiting is a capacity problem; making tests one minute faster barely changes delivery.

Protect main, release, and urgent-fix lanes from pull-request bursts. Concurrency cancellation should target superseded work only. Cancelling a deployment or stateful integration job halfway through may leave an environment occupied and make the next run slower.

Fix the constraint you measured

If a rare runner label owns the delay, either add warm capacity, remove unnecessary label requirements, or schedule expensive suites after fast gates. Self-hosted autoscaling must include image boot, registration, dependency download, and teardown time in the queue objective. Cache restoration that saturates a shared network can move the bottleneck rather than remove it.

Set an objective such as “95 percent of main-branch jobs receive a runner within two minutes.” Alert on oldest queued critical job and unavailable label capacity. For the next week, graph queue and execution separately; change runner capacity only after the label-level distribution identifies the actual wait. Keep pull-request, main, and release percentiles separate so low-priority volume cannot hide delayed production fixes.