You are on call for a web service: a load balancer in front of a fleet of instances, a cache in front of a database, and a 99.9 % availability objective. Every minute the model computes queueing delay, timeouts, cache hits and database load from textbook formulas — and what your decisions cost.
Floor for the fleet. Raising it launches instances at once — they still need the boot delay before serving.
Target tracking on utilization; no new scale-out while instances are still booting. Off = exactly the minimum.
Lower = more headroom and more cost. Measured utilization cannot exceed 100 %, so a saturated fleet grows only step by step.
Longer TTL = more hits, but answers can be older (mean age ≈ TTL/2).
Loads hot keys for 6 minutes (+12 % of the cache per minute) at the price of 600 extra database queries/s.
Share of the low-priority 30 % (prefetch, batch, crawlers) rejected at the load balancer with “retry later”.
The heavy feature adds 20 ms of CPU and one database query per request. Off = graceful degradation.
Redeploys the last good build on a fresh fleet (5 min, billed twice), then switches traffic. Pressing again restarts the preparation; with no bad build live it only costs money.
| Burn rate (1 h) | 0.0 × |
|---|---|
| Requests | 1125 req/s |
| Instances serving | 10 |
| Instances booting | 0 |
| Cache hit rate | 77 % |
| Database utilization | 19 % |
| Traffic shed | 0 % |
| Fleet cost | 4.00 $/h |
| Cost so far | 0.00 $ |
| Mean age of cached answers | 30 s |
| Traffic on the bad build | 0 % |
| Recommendations available | 100 % |
A new build goes out at 09:10. The month has already been rough: only 20 % of the error budget is left. Minutes after the deploy the burn-rate page fires. Protect the budget.
A link to the service is spreading fast and a traffic surge is expected some time this morning — nobody knows when, or how big. New instances need 8 minutes to boot today. When it comes, keep errors and latency down without burning money on idle capacity.
At the midday peak a maintenance script flushes the whole cache. Every request now goes to the database, which was sized for the usual 85 % hit rate. Get the service back without overloading the database.
Every relation the simulator uses, with its source. Constants marked as assumptions are illustrative calibrations.
λ(t) = base × (1 + 0.25·sin(2π(t + clock − 6 h)/24 h)) × surge(t), clock = time of day at the start (peak at 12:00); count/min ≈ N(60λ, √(60λ)) × (1 + N(0, 0.02))[6]Assumption: worker counts, service times, database capacity, cache size, refill speed, prices and the bad build’s error rate are illustrative values for a mid-size web service.N = instances × 16 workers, a = λ·S; C(a, N) = B / (1 − (a/N)(1 − B)), B = Erlang B[3][6]P(W > t) = C·e^(−(N/S − λ)·t); timeouts = P(W > 2 s); a ≥ N ⇒ failed share = 1 − N/a[3]p99 ≈ S·ln 100 + ln(C/0.01)/(N/S − λ) (service + waiting quantile, an approximation)[3][6]Approximation: adding the service and waiting quantiles is not the exact p99 of their sum (it can be a little high or low); the queue is treated as steady within each minute because requests take milliseconds.busy workers L = λ·S ⇒ utilization = λ·S / N[4]hit = warm × rT/(1 + rT), r = λ / 20,000 objects; mean age of a cached answer ≈ T/2[5]DB load = λ·q·(1 − hit); query time = 5 ms/(1 − ρ_db) (≤ 250 ms); S = S_app + (1 − hit)·q·query time[6]Assumption: worker counts, service times, database capacity, cache size, refill speed, prices and the bad build’s error rate are illustrative values for a mid-size web service.budget = 1 − SLO = 0.1 %; burn = error rate / 0.1 %; Δbudget per min = burn / 43,200; burn (1 h) = mean error rate over the last 60 min / 0.1 % (window pre-filled with the opening minute)[1][2]desired = ⌈serving × utilization / target⌉ (utilization saturates at 100 %); new instances serve after the boot delay[6]Assumption: worker counts, service times, database capacity, cache size, refill speed, prices and the bad build’s error rate are illustrative values for a mid-size web service.16 workers/instance · app time 50 ms (+20 ms and +1 query with the feature on) · 2 queries/request · DB 4,000 queries/s · 20,000 hot objects · cache refill τ = 30 min (slower while the DB is saturated) · warm-up job +12 %/min for 6 min, +600 queries/s · timeout 2 s · 30 % low-priority traffic · bad build +5 % errors, ×1.25 CPU · rollback 5 min · $0.40 per instance-hour · scale-in by ≤ 20 % of the fleet after 10 quiet minutes · up to 40 instancesAssumption: worker counts, service times, database capacity, cache size, refill speed, prices and the bad build’s error rate are illustrative values for a mid-size web service.Randomness: a seeded mulberry32 generator; distributions used — uniform, exponential (inverse CDF), normal (Box–Muller), Poisson (Knuth). The seed is shown and shareable.
Educational model — not for operational decisions. Real sites calibrate every constant to their own equipment and data.