Skip to content

preferences: watermark advances past un-enriched feedback, permanently skipping it #1054

Description

@electather

Module: preferences · Dimension: correctness · Severity: low · Location: apps/server/src/preferences/internal/incremental.ts:67

Problem

applyIncrementalUpdate reads records since the oldest partition watermark, and for any record whose provider enrichment returns null it does continue (line 53) without applying it. The final loop (lines 64-70) then unconditionally sets profile.lastUpdatedAt = now and writes every existing partition, regardless of whether that record was applied. Because the next run computes oldest = Math.min(...lastUpdatedAt) (line 40) and readSince uses a strict gt, any record with createdAt <= now is excluded from all future incremental runs.

Why it matters

Scenario: a user rates a brand-new title whose metadata is not yet cached and the fallback provider returns null transiently (CatalogPreferenceProvider miss + fallback miss). That record is skipped this run, but the watermark still advances to now, so no later incremental run ever retries it. The feedback is only recovered at the next full rebuild (nightly 3am job or a bursty-threshold rebuild), defeating the responsiveness the incremental path exists to provide.

Suggested fix

Only advance a partition's lastUpdatedAt to the max createdAt actually applied (or to now only when every in-window record for that partition was successfully enriched); leave the watermark behind un-enriched records so the next run retries them.

Fable-panel validated: 2/3 votes.


Filed from automated per-module audit workflow (Opus/Sonnet research → Fable-panel decision). Unverified by a human — triage before acting.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions