Teaching a load tester to find DoS bugs

Most load tests replay realistic traffic, and realistic traffic almost never breaks anything interesting. Real systems fall over on input that was crafted to hit a super-linear code path: a document nested deep enough to send a parser quadratic, an expansion bomb that turns a kilobyte into a gigabyte, a string that drives a validator's regex into catastrophic backtracking. Those are denial-of-service bugs, and a benchmark that hammers /health with a million happy requests will never find one. Commit 904b7ff (PR #74, landed 6 July 2026) teaches loadr to go looking for them on purpose — a new loadr-payload crate, a loadr payload command, a ${payload:…} template, and a loadr sweep --complexity mode that fits the response-time curve and fails CI when it bends the wrong way.

The manual method, and why it deserved a command

The technique itself is old. You take an adversarial input, scale one dimension of it — depth, count, byte length — fire it at a target, and watch what the response time does. If doubling the input doubles the latency, the endpoint is linear and boring. If doubling the input quadruples the latency, you have found an O(n²) code path, and an attacker with a single crafted request can burn a core for ten seconds. The problem was never the idea; it was that doing it by hand meant a spreadsheet, some curl loops, and an eyeball judgement about whether the curve "looked" super-linear. PR #74 turns that whole loop into two commands and an exit code.

The loadr-payload crate: 18 generators, one knob each

The core is a new crate, crates/loadr-payload (409 lines in the first cut). It is deliberately dull: pure, deterministic, no I/O, no randomness. generate takes a spec and returns bytes. Every kind is parameterised by exactly one magnitude — a depth, a count, a byte length or a level count — and carries a hard safety cap so a generator can never be asked to allocate the machine to death. The doctest captures the whole shape of it:

let bytes = generate_str("nested-json:8").unwrap();
assert_eq!(bytes, br#"{"a":{"a":{"a":{"a":{"a":{"a":{"a":{"a":1}}}}}}}}"#);

There are 18 kinds across seven categories. A representative slice:

The generators stay small and mechanical. nested-markdown-blockquote is ">".repeat(n) + " x". redos is "a".repeat(n) + "!". billion-laughs emits levels XML entities where each expands the previous ten times. The caps are per-kind and meaningful: billion-laughs tops out at 12 levels (already a trillion-fold expansion), nested-json at 5,000,000 depth, hash-collision at 1,000,000 keys. Ask for more and you get a typed PayloadError::TooLarge rather than an OOM.

Getting a payload out: loadr payload

The CLI surface is thin on purpose. loadr payload <kind>[:n] writes the raw bytes to stdout so it pipes straight into curl:

loadr payload nested-json:10000 | curl -X POST http://localhost:3001/api/parse \
  -H 'Content-Type: application/json' --data-binary @-

-o file writes to disk and prints a one-line summary (kind, magnitude, content-type, byte count); --list dumps the whole catalog with each kind's param, default and cap. Omitting :n uses the kind's default magnitude.

Scaling inside a plan: the ${payload:…} template

Piping bytes is fine for a one-off, but to sweep a magnitude you want the payload inside a plan and regenerated per request. That is what the ${payload:<kind>:<n>} template does — the body is generated at request time and never materialised into VU state, so a gigabyte payload costs nothing to hold between iterations. Critically, the magnitude can be a $ENVVAR reference:

body: '${payload:nested-markdown-blockquote:$LOADR_SWEEP_DEPTH}'

That $ENVVAR form is the hook for scaling. loadr sweep --var depth=… exports each value on the axis as LOADR_SWEEP_DEPTH (the LOADR_SWEEP_<NAME> convention), so one unedited plan runs at every depth. Only the magnitude after the last : is expanded; the kind name is left alone. An unset variable resolves to empty and the spec then fails to parse — an honest signal that you forgot to export the axis, rather than a silent zero-depth run.

The verdict: loadr sweep --complexity

loadr sweep gained two flags (about 220 lines in commands/sweep.rs). --complexity <AXIS> treats a swept numeric axis as an input size and fits the exponent k in latency ≈ c·size^k. --max-exponent <K> fails the run — exit 99 — when the fitted k crosses the bound, and implies --complexity.

The fit is the honest, boring choice: an ordinary least-squares regression of log(size) against log(p95 latency). On a log-log plot a power law y = c·x^k is a straight line of slope k, so the slope is the exponent. The implementation is a dozen lines — sum the log points, solve for the slope — and it needs at least two points with two distinct positive sizes, otherwise it returns None rather than a garbage number. Its unit tests pin it down: a synthetic quadratic series recovers k = 2.0 and a linear one recovers k = 1.0 to within 1e-9. The fitted slope is then bucketed into a human verdict:

The real result: catching goldmark red-handed

This is not a synthetic demo. The commit was verified end-to-end against a live Gitea instance's markup renderer. examples/49-payload-complexity.yaml posts a nested-markdown-blockquote bomb to /api/v1/markup, with the depth read from the swept axis:

body: '{"Text": "${payload:nested-markdown-blockquote:$LOADR_SWEEP_DEPTH}", "Mode": "markdown", "Context": "x/y"}'

Then one command sweeps the depth and gates the exponent:

loadr sweep examples/49-payload-complexity.yaml \
  --var depth=4000,8000,16000,32000,64000 \
  --complexity depth --max-exponent 1.2

Gitea renders markdown with Go's goldmark, which has a known super-linear blowup on deeply nested blockquotes. loadr's fit landed at O(n^1.6) — comfortably super-linear, comfortably past the 1.2 bound — and the run exited 99. No spreadsheet, no eyeballing: one command took "our markdown renderer probably scales badly on nesting" and turned it into a mechanical pass/fail. (The guide's worked example pushes an even worse parser, where each 2× in depth roughly 4×s the latency, and the fit lands near O(n^1.99) — textbook quadratic, the same exit 99.)

Why this belongs on every PR

A complexity probe is cheap. It is a single VU walking a handful of magnitudes — five requests, not a five-minute load test — so unlike a full sweep matrix it fits happily in a per-PR CI step:

- name: Complexity gate
  run: |
    loadr sweep examples/49-payload-complexity.yaml \
      --var depth=4000,8000,16000,32000,64000 \
      --complexity depth --max-exponent 1.2

The step fails on exit 99, blocking a merge that makes a parser scale worse than quasilinear. Swap the body for a nested-json, redos or hash-collision spec and you guard whichever code path you care about. That is the whole pitch: an algorithmic-DoS regression stops being a 2 a.m. production incident and becomes a failed check on the PR that introduced it.

One caveat, stated plainly. These payloads exist to exhaust CPU and memory on the receiving end — that is the point. Aim them only at systems you own or are explicitly authorised to test; pointed at a shared or production target they are indistinguishable from an attack. Start small, against a disposable environment.