How can I reduce a production failure into a small permanent test fixture?

Start by turning the production failure into a strict yes-or-no reproducer, then minimize the failing data only against that check. The goal is not realistic test data or a clever builder. The goal is the smallest sanitized fixture that still triggers the same observable failure. That matches the core idea behind delta debugging, which aims at a “failure-inducing minimum”, and reducer tools such as llvm-reduce, which remove content while “preserving their ability to detect bugs”.

What should the failure predicate look like?

Make it specific enough that a reducer cannot “succeed” by finding a different bug. A good predicate is one command or one test assertion that checks the same exception type, error code, invariant violation, or deterministic wrong output.

For example, prefer “import returns DUPLICATE_PRIMARY_CONTACT” over “import fails somehow.” llvm-reduce is explicit here: reduction is driven by an “interesting-ness test”. If your reproducer is flaky, stabilize it first; reduction makes ambiguity worse, not better.

How do I remove records and fields safely?

Reduce in layers, from coarse to fine:

1. Delete whole records, rows, or events.
2. Delete nested collections inside the survivors.
3. Delete fields.
4. Simplify remaining values.
5. Sanitize sensitive values without changing the relationships that matter.

The practical rule is simple: every edit must be followed by rerunning the predicate. The Debugging Book describes a reducer as taking a failing input and reducing it to the minimum that still reproduces the failure. That same discipline works for JSON fixtures, CSV imports, event streams, or a seed set for integration tests.

Preserve the relationships most likely to be causal: foreign keys, repeated IDs, ordering, timestamps, null-versus-empty differences, and duplicate values. Those are often the real bug trigger.

What does a small worked example look like?

Suppose a nightly customer import fails on a 4,000-line NDJSON file.

A useful minimization pass looks like this:

• Write reproduces(file): exit 0 only when the job returns the exact failing error code.
• Delete half the records and rerun.
• Keep the smaller half if it still fails; otherwise restore it and try the other half.
• Continue until you discover the bug needs only two customer records and one shared contact reference.
• Remove unrelated fields like notes, marketing flags, and audit metadata.
• Replace names, emails, and addresses with synthetic placeholders.
• Keep the exact shape that matters: both customers point at the same contact ID, and one conflicting flag marks that contact as primary.

Now your permanent fixture is three short records, not 4,000 lines of production baggage. That is much easier to review, safer to store, and clearer as a regression test.

When is this different from building synthetic test data?

Builders start from your theory of what matters. Minimization starts from a known failure and proves what can be deleted. Use builders for broad coverage. Use reduction when you need a trustworthy regression fixture for one production bug.

What if minimization changes timing?

Keep data minimization and execution conditions separate. If the bug only appears with concurrency, batching, or a clock boundary, encode those in the harness and reduce the data independently. Do not keep unrelated records just to recreate accidental timing.

How should I record fixture provenance?

Store a short note next to the fixture: incident or ticket ID, exact predicate, what was preserved, what was sanitized, and the command used to verify the reproducer. That prevents future cleanup from deleting the actual trigger.

Your next step: write the smallest possible reproduces() check for one real failure, then start deleting halves until the essential relationship is obvious.

Reviewed: 2026-09-05