Skip to content

LT-22714: Stop the IPA populate path duplicating phoneme features - #1087

Draft
johnml1135 wants to merge 1 commit into
mainfrom
phon-features-duplicated
Draft

LT-22714: Stop the IPA populate path duplicating phoneme features#1087
johnml1135 wants to merge 1 commit into
mainfrom
phon-features-duplicated

Conversation

@johnml1135

@johnml1135 johnml1135 commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Setting or correcting a phoneme's Basic IPA Symbol no longer leaves it holding
the same phonological feature twice. Changing a symbol from p to t
previously appended t's entire standard feature set on top of p's: 16
duplicated features from one ordinary edit, many carrying contradictory values.

The unknown you are starting with is whether this is the whole cause or a
symptom. Every writer to PhPhoneme.FeaturesOA was audited. Both feature
choosers and the bulk-edit path clear or upsert before writing; the IPA
populate loop was the only one that appended blind. Two defects compound there

  • the shipped inventory names two features twice for IPA j, and a slice-local
    latch short-circuits the "features are not already filled" gate and can never
    reset while the symbol is non-empty. Review is better spent on whether an
    upsert is the right containment than on hunting for other writers.

Where to look

  • BasicIPASymbolSlice.cs upserts through IFsFeatStruc.GetOrCreateValue, the
    helper the choosers and bulk edit already use.
  • The feature lookup now requires an IFsClosedFeature, which subsumes the old
    null check and skips anything with no symbolic value to assign.
  • In BasicIPAInfo.xml the retained j rows are the ones in canonical
    position; its first eight features now match the order k uses.
  • All five new tests were verified to fail without the fix. One initially
    passed for the wrong reason and was corrected before being trusted.
  • A data test covers all 245 segment definitions, so a duplicated pair cannot
    be reintroduced silently.

Deliberately not here

  • The m_justChangedFeatures latch stays.
  • Features set in the chooser first still cause a later IPA symbol to populate
    nothing at all (recorded on LT-22714).
  • Phonemes already corrupted are not repaired (LT-22716).

Verification. Build clean, 0 warnings. MorphologyEditorDllTests 12/12.
Red and green confirmed in both directions. The full suite was not run.
build.ps1 -CommentHygiene could not complete: it trips on an untracked
scratch file that predates this branch, and that run confirmed zero violations
in these three files.


Reading this a year from now - start here

This branch is the containment half of a two-part story. The duplication had
two independent causes, and only the cheap one is fixed here. The expensive one

  • a UI-local flag standing in for knowledge the model does not record - is
    still present by choice.

If you are here because the duplication came back, check first whether the
latch is still in BasicIPASymbolSlice.SetFeaturesBasedOnIPA. If it is, the
bug you are looking at is probably not a duplicate but a stale feature: one
the previous IPA symbol specified and the current one does not name, which the
upsert has no reason to remove.

Decisions, and why

Upsert rather than skip. The obvious guard - "if this feature already
exists, skip it" - is wrong in a way that is easy to miss. On a p to t
edit it prevents the duplicate but silently discards t's values for every
shared feature, leaving the phoneme still described as p. Any guard here has
to update the existing specification, not step over it.

The latch was kept deliberately. m_justChangedFeatures is a crude record
of "these features are mine, not the user's". Deleting it would make the
mechanism idempotent, but it would also mean an IPA symbol overwrites
hand-picked feature values, and it would force a decision about whether a
symbol change owns the whole feature set or only the features the new symbol
names. The durable answer is a view showing where a phoneme's declared features
diverge from the standard features of its IPA symbol. There is nowhere to
present that today, so the latch remains a placeholder for the missing view
rather than an oversight to clean up.

Which duplicate rows to delete from the inventory. The two duplicated pairs
for j were byte-identical, so the choice looked arbitrary. It was not: j's
feature order was compared against neighbouring segments, and positions 1-8
match k exactly, with Anterior, Coronal and High at 4, 5 and 6. The
oddly-indented block was therefore in canonical position and the later,
normally-indented pair was the intruder - the opposite of what the stray
indentation suggests.

Paths not taken

Three fixes were costed on LT-22714.

Categorical, in the slice. Remove the latch from the gate and upsert. This
closes the bug class rather than the instance and also fixes the chooser-first
defect. Rejected for now only because it forces the ownership-semantics
decision described above, which wants the divergence view first.

Model invariant in liblcm. Enforce one specification per feature inside
FsFeatStruc so no caller anywhere can duplicate. Attractive, and the only
universal option, but it hits a concrete obstacle: every call site reviewed
adds the specification before setting its feature (FeatureSpecsOC.Add(value); value.FeatureRA = featDefn;). At Add time FeatureRA is still null, so an
add-time check cannot see the feature at all; the invariant would have to hook
the FeatureRA setter. Throwing there also risks breaking incremental object
construction during project load, XML import and undo replay. It is worth doing
later as a debug-only assertion, not a runtime invariant.

Surprising findings

Several plausible culprits were checked and cleared, which is worth recording
so they are not re-investigated:

  • Event handler accumulation. BasicIPASymbolChanged is an event on the
    long-lived model object subscribed from a UI slice, which is the classic leak
    shape. It does not leak: DataTree.Reset and DataTree.RemoveSlice both
    dispose, and the unsubscribe runs before base.Dispose.
  • Re-committing the same text. Retyping an identical symbol does not
    re-fire the side effect; the generated setter does a value-equality check.
  • LIFT import. It adds without a pre-check, but writes MSA inflection
    features into a freshly created structure, and LIFT carries no phonemes.
  • PriorityUnion matching. It reads as matching on FeatureRA.Name, but
    Name is an accessor cached per object with no operator== overload, so the
    comparison is really feature-object identity. It works, but it will not
    collapse two distinct feature definitions that share a name.
  • PriorityUnion cannot repair. It resolves matches with
    myFeatureValues.First(), so on an already-duplicated phoneme it updates one
    copy and leaves the twin. This is why bulk edit will not clean up existing
    corruption, and why LT-22716 exists.
Evidence

The regression tests were run against a reverted working tree to confirm they
are not vacuous. Pre-fix failures, which also quantify the defect:

Test Pre-fix result
SettingSymbol_AddsEachFeatureOnce j duplicated 2 features on a fresh phoneme, from the data defect alone
RepopulatingSameSymbol_DoesNotDuplicateFeatures repopulating p duplicated 18 features
ChangingSymbol_DoesNotDuplicateSharedFeatures p to t duplicated 16 features
SymbolEditAfterFeaturesAlreadySet_DoesNotDuplicateFeatures same 16
BasicIPAInfo_NoSegmentNamesTheSameFeatureTwice j: fPAAnterior, fPACoronal

RepopulatingSameSymbol initially failed with Not in the right state to register a change rather than on the assertion, because it called
SetFeaturesBasedOnIPA outside a unit of work. That also meant it passed green
for the wrong reason: post-fix the upsert writes the same value back, registers
no change, and so raised nothing. It was wrapped in a unit of work and both
directions re-run before being trusted.

A scripted audit of the shipped inventory found exactly one affected segment
out of 245.

Preflight review details

See .review/summary.md on the branch author's working copy. The findings it
records are reproduced in the pitch and accordions above: no Critical and no
Important findings; five Minor, of which three are the deliberate deferrals
listed under "Deliberately not here", one is a latent liblcm edge case that
this path cannot reach, and one is the comment-hygiene gate being blocked by an
untracked file predating the branch.


This change is Reviewable

The Basic IPA Symbol slice created a new closed value for every
FeatureValuePair the IPA inventory lists and appended it, so a phoneme
could end up holding several specifications for one phonological
feature. It now upserts through IFsFeatStruc.GetOrCreateValue, updating
the specification a feature already has rather than adding another. The
lookup requires a closed feature, which also covers the null case, and
skips a feature that has no symbolic value to assign.

The shipped inventory also named fPAAnterior and fPACoronal twice for
IPA j, which put two duplicates on a phoneme the first time that symbol
populated its features. The surviving rows are the ones in canonical
position, matching the feature order comparable segments use.

The m_justChangedFeatures latch stays. It stands in for a view of how a
phoneme's declared features diverge from the standard features of its
IPA symbol, which there is currently nowhere to show.

Five tests cover setting a symbol, repopulating the same symbol,
changing a symbol, and editing a symbol after features were already
set, plus a check that no segment in the shipped inventory names a
feature twice. All five fail without this change.

Phonemes corrupted before this change are not repaired by it and are
tracked in LT-22716.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@codecov-commenter

codecov-commenter commented Aug 19, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 75.00000% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 38.28%. Comparing base (6c6f9eb) to head (ebecedd).
⚠️ Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
Src/LexText/Morphology/BasicIPASymbolSlice.cs 75.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1087      +/-   ##
==========================================
+ Coverage   38.07%   38.28%   +0.21%     
==========================================
  Files        1499     1499              
  Lines      350141   350229      +88     
  Branches    40238    40256      +18     
==========================================
+ Hits       133304   134079     +775     
+ Misses     187558   186936     -622     
+ Partials    29279    29214      -65     
Files with missing lines Coverage Δ
Src/LexText/Morphology/BasicIPASymbolSlice.cs 77.31% <75.00%> (+77.31%) ⬆️

... and 24 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

Copy link
Copy Markdown

NUnit Tests

    1 files  ±0      1 suites  ±0   11m 8s ⏱️ +51s
5 801 tests +5  5 720 ✅ +5  81 💤 ±0  0 ❌ ±0 
5 810 runs  +5  5 729 ✅ +5  81 💤 ±0  0 ❌ ±0 

Results for commit ebecedd. ± Comparison against base commit 1cfe930.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants