Add a bitwise mutation operator - #199
Merged
Merged
Conversation
Both are asked for by name and can be run alone, which is the point: a header of masks does not need all 841 token sites re-answered to find out whether ^ and | are checked. bitwise mutates ^ and |, which the token table leaves alone. & is missing for the same reason -> and :: are: it is three operators sharing a spelling -- bitwise and, address-of, and the reference declarator -- and only a parser can tell them apart. ^ and | have one meaning each. Measured over the header this is the best value of the four operators by a distance: 76 mutants, four minutes, 99% killed and 87% of those by a test rather than the compiler. Its single survivor is worth recording. `dist_inc | (hash & fingerprint_mask)` turned into ^ is the same function, because the two operands share no bits -- which is exactly what the static_assert directly above it promises. An equivalent mutant certified by an assertion already in the file. transpositions puts two adjacent statements in the other order. That is the half of bugs/invariants.txt the token sweep cannot reach, and the file says so in as many words: "a sweep cannot express pop_back before repointing instead of after". It is honestly the weaker of the two -- 45% killed, a hundred survivors over the whole file, most of them two statements that never touched the same state -- so CLAUDE.md says to prefer it with --diff, where it is a handful of mutants and the reading is free. It also only reaches adjacent statements at one indent, so the pop_back bug that motivated it is still out of reach; what it does reach is the ordering within a block. Two adjacent `auto` declarations are not transposed. That rule is a measurement and not a proof -- 24 such pairs over the header, 11 that did not compile, 13 that survived and none ever caught -- so it reports how many it skipped, the way every other "we will not run this" rule in this file does, and it is asked after the line filter so --diff does not count pairs nowhere near the change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Drops the transposition operator and tidies what the review found around the one that is left. transpositions went because triaging its hundred survivors produced no test worth writing: essentially all of them are two statements that never touched the same state -- member-copy chains, the run of HASH_STATICCAST macros, blocks of declarations. Reaching the orderings in bugs/invariants.txt means moving a statement out of its enclosing block, which adjacent-swapping cannot do, so the operator is worth building only alongside something that can. CLAUDE.md keeps the measurement and says plainly that there is nothing in the tree to go and look at. bitwise joins the default. It was documented as the cheapest and sharpest of the three and then left out of what --diff actually runs, which are two separate decisions and only the first had an argument behind it -- so a change to a masking line came back clean without ^ or | ever being tried. It costs 76 mutants on 841. bitwise_sites() gives the operator a name, so "this table *and* no words or numbers" is written once instead of at every call site; passing only the table would quietly re-answer every comparison in the file, which is the one thing asking for bitwise alone is meant to avoid. test_every_named_operator_is_one_the_runner_dispatches asserted the OPERATORS tuple against a copy of itself, so the failure its comment described -- a name parse_operators accepts that nothing acts on, which sweeps nothing and reports a clean run -- was exactly what it could not catch, because adding the name and updating the literal is one edit. It now reads main() and checks each name is dispatched on. The default is checked the same way rather than restated. Also: the module docstring is argparse's description, and --help was describing two operators above an option listing three; the docstring claiming the two tables "share no spelling" was false (they share `||`, as a consumer, which is the point of it being there); and the per-operator numbers now say which commit they describe, the way the optimization dead-ends section already does. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A plain run now asks every question the tool knows how to ask, rather than the two that change one token. Naming an operator is the way to ask less, which is the cheaper thing to do and the reason they are named at all. The default is `set(OPERATORS)` rather than a list spelled out beside it, so an operator added later is in the default by construction and the two cannot drift. The test checks that it is derived rather than checking its contents -- a restated list can only ever agree with whatever was typed next to it, which is the same trap as the dispatch test above it. Worth knowing before running one: over the whole header the default is now ~1600 mutants and something like an hour and a half. That is the case for naming one operator for a full sweep, and no reason to for --diff, where the cost is proportional to the lines you touched. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds one new operator to
scripts/mutate/mutate.py:bitwise, which mutates^and|. Every operator can be asked for on its own.Why
^appears 36 times in the header and|20, and the token table mutates neither.&stays out for the same reason->and::do — it is three operators sharing a spelling (bitwise and, address-of, and the reference declarator inauto& x) and only a parser can tell them apart.^and|have one meaning each, so they cost nothing to be sure about.In a header made of masks and fingerprints, with a hash that is essentially XOR and multiply, this lands hard. Swept over the whole header:
tokensbitwisedeletions87% killed by a test is the number worth comparing —
deletionskills 94% but only 30% by a test, and the compiler telling you a mutant does not build says much less about the suite.The single survivor is the pleasing part:
|→^is the same function, because the two operands share no bits — precisely whatstatic_assert(Bucket::fingerprint_mask < Bucket::dist_inc)two lines above guarantees. An equivalent mutant certified by an assertion already in the file. That is the mechanism worth remembering rather than the percentage: a surviving bitwise mutant usually means the operands are provably disjoint.bitwiseis in the default alongsidetokens, so--diff— the everyday mode — actually runs it. Being independently selectable and being excluded from the default were two separate decisions and only the first had an argument behind it; without this, a change to a masking line comes back clean with^and|never tried. It costs 76 mutants on 841.A reordering operator was tried and removed
I originally proposed statement transposition too, on the strength of
bugs/invariants.txtsaying outright that "a sweep cannot expresspop_backbefore repointing instead of after". Built and measured, it killed 45% of what it generated and left ~100 survivors. Triaging every one of them produced no test worth writing: essentially all are two statements that never touched the same state — member-copy chains, the run ofHASH_STATICCASTmacros, blocks of declarations. The handful that do touch shared state either check out as equivalent on reading, or are already filed in #198 from the deletions sweep.The reason is structural: reaching the orderings in
invariants.txtmeans moving a statement out of its enclosing block, which adjacent-swapping cannot do. So it is worth building only alongside something that can. CLAUDE.md keeps the measurement and says plainly that there is nothing left in the tree to go and look at.Review pass
Four review agents went over the diff; their agreed findings are applied in the second commit.
--helpcontradicted itself. The module docstring is argparse'sdescription, and it described two operators directly above an option listing three.test_every_named_operator_is_one_the_runner_dispatchesasserted theOPERATORStuple against a copy of itself. The failure its own comment described — a nameparse_operatorsaccepts that nothing dispatches on, which sweeps nothing and reports a clean run — was exactly the one it could not catch, since adding the name and updating the literal is a single edit. It now readsmain()and checks each name is actually dispatched on; the default is checked the same way instead of being restated.||— as a consumer, which is the whole point of it being there, since matching the|inside it would turna || bintoa &| b.bitwise_sites()gives the operator a name so "this table and no words or numbers" is written once rather than at every call site.Efficiency was measured rather than guessed: the
words_and_numbers=Falsepath costs 4 ms once per run against a 4-minute sweep, and the bitwise scan is still cheaper in absolute terms than the tokens scan (17.6 ms vs 30.0 ms). Nothing to fix.One finding deliberately not taken: folding both tables into one with per-row category tags would remove the duplicated
||consumer row and thewords_and_numbersflag. It touches ~25 lines of pre-existing table for a fourth subset that, on the evidence above, is not coming.Tests
scripts/test_mutate.pygoes 155 → 164, all hermetic, suite still ~0.5s.🤖 Generated with Claude Code