Skip to content

fix(zh): don't consume a leading Chinese numeral as a word boundary - #653

Open
Jaybhade wants to merge 1 commit into
wanasit:masterfrom
Jaybhade:fix/zh-leading-numeral-word-boundary
Open

fix(zh): don't consume a leading Chinese numeral as a word boundary#653
Jaybhade wants to merge 1 commit into
wanasit:masterfrom
Jaybhade:fix/zh-leading-numeral-word-boundary

Conversation

@Jaybhade

Copy link
Copy Markdown

The problem

Chinese date expressions that start at the very beginning of the text silently resolve to the wrong date. With refDate = 2012-08-10:

Input Means Parsed as result.text
十一月十日 Nov 10 Jan 10 一月十日
十二月十日 Dec 10 Feb 10 二月十日
二十天前 20 days ago 10 days ago 十天前
十五天前 15 days ago 5 days ago 五天前
二十天後 in 20 days in 10 days 十天後
十五天後 in 15 days in 5 days 五天後

This affects both zh.hans and zh.hant. These are silent wrong answers, not parse failures, so callers get a plausible-looking date that is off by up to ten months.

const chrono = require("chrono-node");
chrono.zh.hans.parseDate("十一月十日", new Date(2012, 7, 10));
// => Tue Jan 10 2012   (expected Sat Nov 10 2012)

Why

AbstractParserWithWordBoundaryChecking prepends patternLeftBoundary(), which defaults to:

return `(\\W|^)`;

JavaScript's \W is [^A-Za-z0-9_], so every CJK character satisfies \W. At index 0 the alternation tries \W before ^, consumes the leading character, and the parser then matches whatever still parses from index 1 — a shorter expression with a different value.

十月十日 (Oct 10) is unaffected only by luck: dropping leaves 月十日, which can't match, so the engine backtracks to ^. But dropping the of 十一月 leaves the perfectly parseable 一月, so no backtracking happens.

The same inputs parse correctly once they're preceded by a space or embedded in a sentence — the preceding character then serves as the boundary and nothing is eaten. That's why this hasn't shown up in normal use:

chrono.zh.hans.parseDate(" 十一月十日", ref); // Nov 10  ✅
chrono.zh.hans.parseDate("我在十一月十日", ref); // Nov 10  ✅
chrono.zh.hans.parseDate("十一月十日", ref); //  Jan 10  ❌

The fix

ZHHansTimeExpressionParser and ZHHantTimeExpressionParser already solve exactly this, with a comment naming the cause:

patternLeftBoundary(): string {
    // Return a capturing group to ensure that the match index is correct in the base class
    // while avoiding matching CJK characters as word boundaries.
    return "()";
}

This PR applies that same established override to the three parser families that still inherit the default — date, ago and deadline — in both script variants, keeping zh.hans and zh.hant symmetric:

  • ZHHansDateParser / ZHHantDateParser
  • ZHHansAgoFormatParser / ZHHantAgoFormatParser
  • ZHHansDeadlineFormatParser / ZHHantDeadlineFormatParser

The remaining zh parsers (casual, weekday, relation-weekday) don't need it: their patterns begin with non-numeric characters (, , 星期, …), so dropping the first character never leaves a parseable remainder.

Considered and rejected: fixing the base class

Reordering the default boundary to (^|\\W) also fixes all six cases with a one-line change, and the whole suite stays green — but a differential run over a 1,416-entry corpus showed it changes result.text/result.index for other locales, because the boundary character gets absorbed into the match:

Input text before text after
(Monday) Monday) (Monday)
,Monday Monday ,Monday

Not worth the cross-locale blast radius, so I kept the change scoped to the zh parsers, consistent with the existing precedent.

Verification

  • npm test146 suites / 680 tests green (674 before, +6 new).
  • The 6 added test blocks were confirmed to fail without the source change and pass with it.
  • A differential run over 1,416 locale×input combinations (en, de, fr, it, es, ru, uk, ja, pt, nl, zh.hans, zh.hant — including punctuation-leading inputs like (Monday), -5 days ago, @3pm) shows only the 12 intended zh corrections changed; no other locale is affected.
  • prettier --check clean; tsc -p tsconfig.build.json clean.

Tests follow the conventions in AGENTS.md: an explicit refDate on every case, assertions on both result.index and result.text, and matching Simplified/Traditional coverage. Each new block also keeps a control case (十月十日, 十天前, 十天後) so a future regression in the opposite direction is caught too.

One behaviour worth flagging

For ZHHans/HantDateParser, an input with leading whitespace now reports that whitespace inside result.text:

chrono.zh.hans.parse(" 十一月十日", ref)[0];
// before: index 1, text "十一月十日"
// after:  index 0, text " 十一月十日"   (date is correct either way: Nov 10)

That's inherent to return "()" when a pattern can begin with \s*, and it already happens on master for the time-expression parser that uses the same override (" 十点"text: " 十点"). I kept the fix consistent with that precedent rather than special-casing it, but happy to trim the leading whitespace in the date parser's pattern if you'd prefer result.text stay tight.

npm run eslint fails on this branch and on master alike (ESLint 9 expects eslint.config.js; reported in #606) — unrelated to this change, and CI runs npm run test.

The inherited patternLeftBoundary() is `(\W|^)`, and CJK characters
satisfy `\W`. When a Chinese expression starts at the very beginning of
the text, the alternation prefers `\W`, consumes the first character, and
the parser then matches a shorter expression with a different value:

  十一月十日  (Nov 10)      -> 一月十日  (Jan 10)
  十二月十日  (Dec 10)      -> 二月十日  (Feb 10)
  二十天前    (20 days ago) -> 十天前    (10 days ago)
  十五天前    (15 days ago) -> 五天前    (5 days ago)
  二十天後    (in 20 days)  -> 十天後    (in 10 days)

These are silent wrong answers rather than parse failures. The same
expressions resolve correctly once they are preceded by a space or
embedded in a sentence, which is why this went unnoticed.

ZHHansTimeExpressionParser and ZHHantTimeExpressionParser already avoid
this by returning an empty capturing group from patternLeftBoundary().
Apply that same override to the date, ago and deadline parsers, keeping
zh.hans and zh.hant symmetric.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant