Skip to content

preferences: per-candidate metadata fetch is N+1 despite batch API #1050

Description

@electather

Module: preferences · Dimension: perf · Severity: medium · Location: apps/server/src/preferences/internal/catalog-provider.ts:49

Problem

CatalogPreferenceProvider.getItemFeatures issues one this.catalog.getMetadata(tmdbId, mediaType) DB call per candidate. It is invoked from rebuild.ts's fetchFeaturesForCandidates (apps/server/src/preferences/internal/rebuild.ts:248-297), which iterates candidates in batches of CONCURRENCY=10 (rebuild.ts:253) and calls fetchOneCandidate -> provider.getItemFeatures per item. For a user with a large candidate set (e.g. thousands of watched/rated/history items merged in aggregate()), this produces thousands of individual single-row SQL lookups instead of one batched lookup, even though CatalogService already exposes getMetadataBatch(items: MetadataKey[]) (apps/server/src/catalog/service/index.ts:79) for exactly this shape.

Why it matters

A user with a large history/ratings/watchlist union (easily 1000+ distinct tmdbIds after aggregation) triggers 1000+ sequential-ish DB round trips (10 at a time) during every full rebuild instead of one or a few batched queries, multiplying rebuild latency and DB load roughly linearly with catalog size and directly hurting the daily/manual rebuild jobs that fan this out across many users.

Suggested fix

Change CatalogPreferenceProvider to batch: collect all (tmdbId, mediaType) keys for the candidate set up front and call catalog.getMetadataBatch once (or in a few chunks) instead of one getMetadata call per candidate inside fetchOneCandidate; keep the existing cold-fill fallback for the batch misses.

Fable-panel validated: 3/3 votes.


Filed from automated per-module audit workflow (Opus/Sonnet research → Fable-panel decision). Unverified by a human — triage before acting.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions