Run mutation testing weekly in CI, and make it able to fail

The sweep found real gaps but nothing re-runs it, which puts it in exactly
the category this branch already has a bad history with: a check nobody runs
is not a check. Two changes make it one.

The script now exits non-zero. Exit 1 means a mutant survived — a security
check whose removal no test notices. That is not automatically a defect
(equivalent mutants exist) but it always needs a human to either fix it or
annotate it `mutation-equivalent:` with a reason, which is what a red check
should mean. Exit 2 means a target generated no mutants at all: not a pass,
and it must not share a signal with one.

The workflow is scheduled weekly rather than run per-push, deliberately. A
full sweep is ~132 mutants at roughly 25s each, and what it measures does not
change commit to commit — it drifts as checks and tests are added. Per-push
it would be an hour of CI for a signal that moves monthly.

It also verifies the tree is clean at the end. The script restores after
every mutant, verifies the restore, and traps SIGINT/SIGTERM/SIGHUP — but the
entire technique depends on deliberately broken security code never escaping
the run, so that gets checked rather than trusted.

Also corrects a stale TODO: the fuzzer and cross_impl_check were wired into
ts-parity.yml some commits ago and the list still claimed neither ran.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01Vtx94nXDmNgXEeThUZP4QN
This commit is contained in:
thePR0M3TH3ANandClaude Opus 5 committed 2026-08-19 19:26:24 -04:00
1 parent f043c95672
commit d6ffbac312
3 files changed
+92 -4

No files matched your search

+11
View File
@@ -369,6 +369,17 @@ def main() -> int:
print(f" was: {mutant.original.strip()[:110]}")
print(f" now: {mutant.mutated.strip()[:110]}")
print()
# Exit non-zero so this can gate CI. A survivor is not automatically a
# defect -- it may be an equivalent mutant -- but it is always something a
# human has to look at and then either fix or annotate with
# `mutation-equivalent:`, which is exactly what a red check should mean.
# A target that generated nothing gets its own code, because "not
# measured" and "measured and clean" must not be the same signal.
if overall_survivors:
return 1
if unmeasured:
return 2
return 0