fix: strip dangling conjunctions from extract_duration remainder#293
Draft
JarbasAl wants to merge 1 commit into
Draft
fix: strip dangling conjunctions from extract_duration remainder#293JarbasAl wants to merge 1 commit into
JarbasAl wants to merge 1 commit into
Conversation
The generic duration engine consumed <number> <unit> chunks by
replacing them with empty string, leaving connective tokens ("and",
",") that only joined two consumed chunks stranded in the remainder,
e.g. extract_duration("It will take about 2 hours and 30 minutes",
"en") returned "It will take about and" instead of "It will take
about".
Matches are now replaced with an internal sentinel so, once every
unit is consumed, a connector (with optional surrounding commas)
sandwiched between two sentinels can be told apart from one still
joining unconsumed text and collapsed away. A connector touching real
content on either side is left untouched. Opt-in per language via a
new DurationLexicon.connectors field, wired up for English ("and")
and Spanish ("y") where the bug was confirmed; other languages keep
their existing (tested) remainder behavior unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What was broken
extract_duration_generic(the shared engine backingextract_durationformost languages) consumed each matched
<number> <unit>chunk by replacing itwith an empty string. When two chunks in the same sentence were joined by a
conjunction or comma, that connector was left stranded in the remainder text.
How it fails
The remainder text is meant to be the sentence with the duration wording
removed, for use by callers (e.g. skills) that want to do something with
"the rest" of the utterance. A dangling "and"/"y" (optionally with commas)
is ungrammatical and can trip up anything downstream that expects clean
leftover text.
The fix
extract_duration_genericnow replaces consumed matches with an internalprivate-use sentinel character instead of
"". Once every unit has beenconsumed, a connector (with optional surrounding commas) that is sandwiched
between two sentinels — i.e. it only ever joined two consumed chunks — is
collapsed away along with them. A connector that still touches real,
unconsumed text on either side is left alone, so "two hours and rest and
relax" still returns remainder
"and rest and relax".This is opt-in per language via a new
DurationLexicon.connectorsfield(a tuple of regex fragments for the language's conjunction word(s)), so it
only changes behavior where explicitly wired up. It's wired up for English
(
"and") and Spanish ("y"), where the bug was confirmed. Other languages(e.g. German
"und", French"et") keep their existing, already-testedremainder behavior — those tests intentionally leave the connector in the
remainder and are unaffected.
Verification
Following this repo's NL-TDD convention, failing tests were written first
to reproduce the bug (
test/parse_tests/test_parse_en.pyandtest/parse_tests/test_parse_es.py), then the fix was applied and thetests turned green.
Ran the full suite before and after the change (
PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 pytest test/): the same 14 pre-existing failures are present ondevbefore this change (unrelated
test_dates_an*andtest_packagingfailures) — zero new failures, zero regressions.
Authored with AI assistance (Claude).