Skip to content

Add a bitwise mutation operator - #199

Merged
martinus merged 3 commits into
mainfrom
mutate-bitwise-and-transpositions
Aug 16, 2026
Merged

Add a bitwise mutation operator#199
martinus merged 3 commits into
mainfrom
mutate-bitwise-and-transpositions

Conversation

@martinus

@martinus martinus commented Aug 15, 2026

Copy link
Copy Markdown
Owner

Adds one new operator to scripts/mutate/mutate.py: bitwise, which mutates ^ and |. Every operator can be asked for on its own.

scripts/mutate/mutate.py --operators bitwise      # ^ and | alone, four minutes
scripts/mutate/mutate.py --operators deletions    # unchanged

Why

^ appears 36 times in the header and | 20, and the token table mutates neither. & stays out for the same reason -> and :: do — it is three operators sharing a spelling (bitwise and, address-of, and the reference declarator in auto& x) and only a parser can tell them apart. ^ and | have one meaning each, so they cost nothing to be sure about.

In a header made of masks and fingerprints, with a hash that is essentially XOR and multiply, this lands hard. Swept over the whole header:

operator mutants time killed by a test survivors
tokens 841 ~47 min not re-measured
bitwise 76 4 min 99% 87% 1
deletions 665 14 min 94% 30% 37

87% killed by a test is the number worth comparing — deletions kills 94% but only 30% by a test, and the compiler telling you a mutant does not build says much less about the suite.

The single survivor is the pleasing part:

return Bucket::dist_inc | (static_cast<dist_and_fingerprint_type>(hash) & Bucket::fingerprint_mask);

|^ is the same function, because the two operands share no bits — precisely what static_assert(Bucket::fingerprint_mask < Bucket::dist_inc) two lines above guarantees. An equivalent mutant certified by an assertion already in the file. That is the mechanism worth remembering rather than the percentage: a surviving bitwise mutant usually means the operands are provably disjoint.

bitwise is in the default alongside tokens, so --diff — the everyday mode — actually runs it. Being independently selectable and being excluded from the default were two separate decisions and only the first had an argument behind it; without this, a change to a masking line comes back clean with ^ and | never tried. It costs 76 mutants on 841.

A reordering operator was tried and removed

I originally proposed statement transposition too, on the strength of bugs/invariants.txt saying outright that "a sweep cannot express pop_back before repointing instead of after". Built and measured, it killed 45% of what it generated and left ~100 survivors. Triaging every one of them produced no test worth writing: essentially all are two statements that never touched the same state — member-copy chains, the run of HASH_STATICCAST macros, blocks of declarations. The handful that do touch shared state either check out as equivalent on reading, or are already filed in #198 from the deletions sweep.

The reason is structural: reaching the orderings in invariants.txt means moving a statement out of its enclosing block, which adjacent-swapping cannot do. So it is worth building only alongside something that can. CLAUDE.md keeps the measurement and says plainly that there is nothing left in the tree to go and look at.

Review pass

Four review agents went over the diff; their agreed findings are applied in the second commit.

  • --help contradicted itself. The module docstring is argparse's description, and it described two operators directly above an option listing three.
  • A test that could not catch what it was named for. test_every_named_operator_is_one_the_runner_dispatches asserted the OPERATORS tuple against a copy of itself. The failure its own comment described — a name parse_operators accepts that nothing dispatches on, which sweeps nothing and reports a clean run — was exactly the one it could not catch, since adding the name and updating the literal is a single edit. It now reads main() and checks each name is actually dispatched on; the default is checked the same way instead of being restated.
  • A docstring claim that was false. It said the two tables "share no spelling". They share || — as a consumer, which is the whole point of it being there, since matching the | inside it would turn a || b into a &| b.
  • bitwise_sites() gives the operator a name so "this table and no words or numbers" is written once rather than at every call site.
  • The measurements now say which commit they describe, the way the optimization dead-ends section already does.

Efficiency was measured rather than guessed: the words_and_numbers=False path costs 4 ms once per run against a 4-minute sweep, and the bitwise scan is still cheaper in absolute terms than the tokens scan (17.6 ms vs 30.0 ms). Nothing to fix.

One finding deliberately not taken: folding both tables into one with per-row category tags would remove the duplicated || consumer row and the words_and_numbers flag. It touches ~25 lines of pre-existing table for a fourth subset that, on the evidence above, is not coming.

Tests

scripts/test_mutate.py goes 155 → 164, all hermetic, suite still ~0.5s.

🤖 Generated with Claude Code

martinus and others added 2 commits August 15, 2026 23:07
Both are asked for by name and can be run alone, which is the point: a header
of masks does not need all 841 token sites re-answered to find out whether ^
and | are checked.

bitwise mutates ^ and |, which the token table leaves alone. & is missing for
the same reason -> and :: are: it is three operators sharing a spelling --
bitwise and, address-of, and the reference declarator -- and only a parser can
tell them apart. ^ and | have one meaning each. Measured over the header this
is the best value of the four operators by a distance: 76 mutants, four
minutes, 99% killed and 87% of those by a test rather than the compiler.

Its single survivor is worth recording. `dist_inc | (hash & fingerprint_mask)`
turned into ^ is the same function, because the two operands share no bits --
which is exactly what the static_assert directly above it promises. An
equivalent mutant certified by an assertion already in the file.

transpositions puts two adjacent statements in the other order. That is the
half of bugs/invariants.txt the token sweep cannot reach, and the file says so
in as many words: "a sweep cannot express pop_back before repointing instead of
after". It is honestly the weaker of the two -- 45% killed, a hundred survivors
over the whole file, most of them two statements that never touched the same
state -- so CLAUDE.md says to prefer it with --diff, where it is a handful of
mutants and the reading is free. It also only reaches adjacent statements at
one indent, so the pop_back bug that motivated it is still out of reach; what
it does reach is the ordering within a block.

Two adjacent `auto` declarations are not transposed. That rule is a measurement
and not a proof -- 24 such pairs over the header, 11 that did not compile, 13
that survived and none ever caught -- so it reports how many it skipped, the
way every other "we will not run this" rule in this file does, and it is asked
after the line filter so --diff does not count pairs nowhere near the change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Drops the transposition operator and tidies what the review found around the
one that is left.

transpositions went because triaging its hundred survivors produced no test
worth writing: essentially all of them are two statements that never touched
the same state -- member-copy chains, the run of HASH_STATICCAST macros, blocks
of declarations. Reaching the orderings in bugs/invariants.txt means moving a
statement out of its enclosing block, which adjacent-swapping cannot do, so the
operator is worth building only alongside something that can. CLAUDE.md keeps
the measurement and says plainly that there is nothing in the tree to go and
look at.

bitwise joins the default. It was documented as the cheapest and sharpest of
the three and then left out of what --diff actually runs, which are two separate
decisions and only the first had an argument behind it -- so a change to a
masking line came back clean without ^ or | ever being tried. It costs 76
mutants on 841.

bitwise_sites() gives the operator a name, so "this table *and* no words or
numbers" is written once instead of at every call site; passing only the table
would quietly re-answer every comparison in the file, which is the one thing
asking for bitwise alone is meant to avoid.

test_every_named_operator_is_one_the_runner_dispatches asserted the OPERATORS
tuple against a copy of itself, so the failure its comment described -- a name
parse_operators accepts that nothing acts on, which sweeps nothing and reports a
clean run -- was exactly what it could not catch, because adding the name and
updating the literal is one edit. It now reads main() and checks each name is
dispatched on. The default is checked the same way rather than restated.

Also: the module docstring is argparse's description, and --help was describing
two operators above an option listing three; the docstring claiming the two
tables "share no spelling" was false (they share `||`, as a consumer, which is
the point of it being there); and the per-operator numbers now say which commit
they describe, the way the optimization dead-ends section already does.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@martinus martinus changed the title Add bitwise and transposition mutation operators Add a bitwise mutation operator Aug 16, 2026
A plain run now asks every question the tool knows how to ask, rather than the
two that change one token. Naming an operator is the way to ask less, which is
the cheaper thing to do and the reason they are named at all.

The default is `set(OPERATORS)` rather than a list spelled out beside it, so an
operator added later is in the default by construction and the two cannot
drift. The test checks that it is derived rather than checking its contents --
a restated list can only ever agree with whatever was typed next to it, which
is the same trap as the dispatch test above it.

Worth knowing before running one: over the whole header the default is now
~1600 mutants and something like an hour and a half. That is the case for
naming one operator for a full sweep, and no reason to for --diff, where the
cost is proportional to the lines you touched.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@martinus
martinus merged commit 2164505 into main Aug 16, 2026
31 checks passed
@martinus
martinus deleted the mutate-bitwise-and-transpositions branch August 16, 2026 04:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant