Motivation
Deterministic channel-memory digesting is the safety baseline, but the Phase 2 design also needs an off-hot-path LLM worker for higher-quality topic and sequence rollups. This must not add latency to channel-awareness serving or blur exact source provenance.
Depends on #265. It can land after #267 if deterministic raw_window+digest ships first.
Scope
- Add an async worker that processes queued source-message windows into structured digest blocks.
- Keep hard events verbatim or near-verbatim; use lossy prose only for sparse
topic_rollup and sequence_rollup style blocks.
- Require structured JSON output and reject malformed or provenance-free results.
- Cache work by source message ids plus content hashes.
- Store model/provider/version, generated time, sparse flag, source window, and source-message provenance for every generated block.
- Enforce conservative per-channel/per-pod daily call and cost caps with clear telemetry.
- Fall back to deterministic-only digest output when the worker is disabled, over budget, or failing.
Out of scope
- The base SQLite adapter schema beyond extensions required for queued work and processor metadata.
- claw-wall push ingest and channel-awareness serving changes.
- Public YAML tuning surfaces beyond internal environment overrides.
- Release pin, changelog, or nav changes.
Acceptance
- Unit tests cover queue ordering, malformed LLM output rejection, cache reuse by content hash, cost-cap enforcement, and deterministic fallback.
- Processor tests prove edit/delete/forget marks affected derived blocks stale or dirty.
- Fixture tests show older verbose channel material compresses into sparse blocks while hard events remain faithful.
- No LLM call occurs during
/digest hot-path lookup.
- Public docs and examples stay deployment-agnostic.
Refs #262
Refs #263
Motivation
Deterministic channel-memory digesting is the safety baseline, but the Phase 2 design also needs an off-hot-path LLM worker for higher-quality topic and sequence rollups. This must not add latency to channel-awareness serving or blur exact source provenance.
Depends on #265. It can land after #267 if deterministic
raw_window+digestships first.Scope
topic_rollupandsequence_rollupstyle blocks.Out of scope
Acceptance
/digesthot-path lookup.Refs #262
Refs #263