Why
The first slice optimizes over tiny synthetic in-repo fixtures (src/mutalisk/eval_set.py). Component 2 of docs/ARCHITECTURE.md is real trace + eval ingestion: pull public-safe executed traces and eval results (provenance refs only; no raw prompts/customer data) as the optimizer training/validation sets.
Scope
- Define an ingestion interface that loads eval/train datasets from a public-safe source (D1/R2 provenance refs) into the
Example/dataset shape the optimizers consume.
- Enforce redaction/public-safety at the boundary: no customer prompts, no hidden reasoning traces, no wallet material, no secrets in traces or candidates.
- Carry real
eval_evidence_refs / trace_provenance_refs through to emitted candidates (today these point at the in-repo fixtures).
- Keep ingestion offline/batch; no online/hot-path reads.
Invariants (AGENTS.md)
- Fail closed when provenance or metric evidence is missing.
- Do not put production data, customer prompts, or wallet material into traces or candidates.
Done when
- Optimizers can run over an ingested public-safe dataset (with the synthetic fixtures retained as a smoke/default), provenance refs flow into candidates, and tests cover the redaction/fail-closed boundary.
Why
The first slice optimizes over tiny synthetic in-repo fixtures (
src/mutalisk/eval_set.py). Component 2 ofdocs/ARCHITECTURE.mdis real trace + eval ingestion: pull public-safe executed traces and eval results (provenance refs only; no raw prompts/customer data) as the optimizer training/validation sets.Scope
Example/dataset shape the optimizers consume.eval_evidence_refs/trace_provenance_refsthrough to emitted candidates (today these point at the in-repo fixtures).Invariants (AGENTS.md)
Done when