Skip to content

channel-memory: add async LLM digest worker and cost caps #268

Description

@mostlydev

Motivation

Deterministic channel-memory digesting is the safety baseline, but the Phase 2 design also needs an off-hot-path LLM worker for higher-quality topic and sequence rollups. This must not add latency to channel-awareness serving or blur exact source provenance.

Depends on #265. It can land after #267 if deterministic raw_window+digest ships first.

Scope

  • Add an async worker that processes queued source-message windows into structured digest blocks.
  • Keep hard events verbatim or near-verbatim; use lossy prose only for sparse topic_rollup and sequence_rollup style blocks.
  • Require structured JSON output and reject malformed or provenance-free results.
  • Cache work by source message ids plus content hashes.
  • Store model/provider/version, generated time, sparse flag, source window, and source-message provenance for every generated block.
  • Enforce conservative per-channel/per-pod daily call and cost caps with clear telemetry.
  • Fall back to deterministic-only digest output when the worker is disabled, over budget, or failing.

Out of scope

  • The base SQLite adapter schema beyond extensions required for queued work and processor metadata.
  • claw-wall push ingest and channel-awareness serving changes.
  • Public YAML tuning surfaces beyond internal environment overrides.
  • Release pin, changelog, or nav changes.

Acceptance

  • Unit tests cover queue ordering, malformed LLM output rejection, cache reuse by content hash, cost-cap enforcement, and deterministic fallback.
  • Processor tests prove edit/delete/forget marks affected derived blocks stale or dirty.
  • Fixture tests show older verbose channel material compresses into sparse blocks while hard events remain faithful.
  • No LLM call occurs during /digest hot-path lookup.
  • Public docs and examples stay deployment-agnostic.

Refs #262
Refs #263

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions