✨ feat(fixtures): pin the content-encoding decode policy to a shared corpus

fixtures/README.md

@@ -10,15 +10,20 @@ harness matrix the cut was taken against. Read the manifest first.
 Everything is generated downhill from one recording:
 
 ```
-L0  Envelope fixtures      fixtures/envelope/     header set -> expected parsed envelope (+ errors)
-L1  Wire recordings        fixtures/recordings/   turn-*/ bundles — verbatim transport bytes
-L2  Corpus (.jsonl.gz)     pkg/seed/corpus/       raw_turns rows — derive gates, seed
-L3  Rendered API fixtures  (generated downstream)  via `tapes dev trace-fixtures`
+L0  Envelope fixtures      fixtures/envelope/         header set -> expected parsed envelope (+ errors)
+L0  Thread fixtures        fixtures/thread/           header set -> resolved sub-thread id
+L0  Content-encoding       fixtures/content-encoding/ (body, encoding) -> decoded bytes | salvage | error
+L1  Wire recordings        fixtures/recordings/       turn-*/ bundles — verbatim transport bytes
+L2  Corpus (.jsonl.gz)     pkg/seed/corpus/           raw_turns rows — derive gates, seed
+L3  Rendered API fixtures  (generated downstream)     via `tapes dev trace-fixtures`
 ```
 
 One clean-room capture session -> the wire-trace recorder emits **L1** -> ingest +
 `tapes dev dump-corpus` emits **L2** -> `tapes dev trace-fixtures` emits **L3**. The
-**L0** envelope cases are synthesized directly from the header contract, not captured.
+**L0** families are synthesized directly from the contract they pin, not captured.
+Envelope and thread pin *header* contracts; content-encoding pins a *capture policy*,
+which is the same problem one layer in — a decision implemented independently in two
+languages, whose agreement nothing checked until it was made executable.
 
 ## Why some families live outside this directory
 
@@ -50,6 +55,14 @@ fixtures/
   envelope/          ← L0 synthetic header<->envelope cases
     README.md
     cases/*.json
+  thread/            ← L0 synthetic header->thread_id cases
+    README.md
+    DIGEST
+    cases/*.json
+  content-encoding/  ← L0 synthetic (body, encoding)->decode-outcome cases
+    README.md
+    DIGEST
+    cases/*.json
   recordings/        ← L1 wire recordings, one directory per capture set
     README.md
     claude-20260727-019fa57e/

fixtures/content-encoding/DIGEST

@@ -0,0 +1 @@
+sha256:28199a4a4775fbe355f92a43e5d7e74468b64d2370dc4824616e5e50526ca88f

fixtures/content-encoding/README.md

@@ -0,0 +1,253 @@
+# Content-encoding fixtures — the captured-body decode contract
+
+L0-layer fixtures, sibling to `fixtures/envelope/` and `fixtures/thread/`: small,
+synthetic, language-neutral JSON cases that pin how a captured body's
+`Content-Encoding` is undone before the bytes reach a reducer.
+
+Where the envelope corpus pins a *header* contract, this one pins a **capture
+policy**: which codings are readable, how stacked layers compose, how much
+output is allowed, and what happens when a stream is corrupt or arrives half
+finished. Those are decisions, not encodings — and they are currently made
+independently in three places across two languages:
+
+| Implementation | Where |
+| --- | --- |
+| Go, reference | `pkg/capture/contentencoding.go` (`DecodeContentEncoding`) |
+| Go, gateway lane | `extproc/` — calls the same function, deliberately |
+| Rust, client | `tapesctl` `crates/tapesctl/src/start/content_encoding.rs` |
+
+Their agreement is load-bearing: capture fidelity is supposed to be identical
+whether a session went through Paper's cloud or through `tapesctl start`. Until
+this corpus existed that agreement was a point-in-time claim by whoever last read
+both files, and the decay had already cost a bug — **PCC-1126**, where the
+client dropped every `content-encoding: zstd` request body (all of Codex/pi's
+traffic) while the gateway route decoded the same bytes fine. Nothing was red.
+
+`pkg/capture/contentencoding.go` is the **reference implementation**: where a
+case and the prose disagree, the case records what the reference actually does,
+and says so.
+
+## Layout
+
+```
+fixtures/content-encoding/
+  README.md          ← this file
+  DIGEST             ← seal over cases/, recomputed by consumers
+  cases/*.json       ← one case per file; consumers glob this directory
+```
+
+## How cases carry bytes
+
+Encoded bodies are binary, where the envelope corpus is pure JSON. Two forms are
+allowed, and which one a case uses is itself a statement about what the case
+asserts:
+
+* **`build`** — a *recipe*: a plaintext, the layers to apply to it, and an
+  optional truncation. The consumer compresses locally.
+* **`bytes_b64`** — the literal body, base64 (standard alphabet, padded).
+
+Recipes are the default, and deliberately so. Compressed output is not stable
+across implementations — Go's `compress/gzip` and Rust's `flate2` do not emit the
+same bytes for the same input, and neither do `klauspost/compress/zstd` and
+`libzstd`. A corpus that pinned compressed bytes would be asserting *compressor
+identity*, which is not the policy and which no consumer can satisfy. It would
+also be unreviewable: a diff of base64 blobs says nothing about what changed.
+With a recipe, the reviewable artifact is the plaintext and the layer list, and
+the assertion is the one that matters — that a gzip stream of X decodes to X.
+
+`bytes_b64` is kept for the cases where the exact bytes **are** the assertion:
+hand-built frames (`limit-zstd-window-*`), deliberately corrupt input, and the
+empty body. Those are short enough to annotate byte by byte in `notes`, and a
+recipe could not express them.
+
+Bodies larger than a few hundred bytes are never committed in either form. The
+cap cases decode to 32 MiB; a blob that size is not reviewable and its compressed
+form is compressor-specific, so it is a recipe with a `count`.
+
+## Case schema
+
+Each `cases/*.json` file is one object.
+
+| field | required | meaning |
+| --- | --- | --- |
+| `name` | yes | stable case id (matches the filename) |
+| `category` | yes | `identity` \| `supported` \| `stacked` \| `salvage` \| `limit` \| `error` |
+| `description` | yes | one line on what the case pins |
+| `encoding` | yes | the `Content-Encoding` header value, verbatim and un-normalised. `null` means the header is **absent**; `""` means present and empty |
+| `body` | yes | exactly one of `bytes_b64` or `build` (below) |
+| `expect` | yes | the expected outcome (below) |
+| `grounding` | yes | the policy rule the case pins, in behavioral terms |
+| `contested` | no | a decision this corpus was written to force; see below |
+| `notes` | no | anything a consumer needs to know |
+
+### `body.build`
+
+| field | required | meaning |
+| --- | --- | --- |
+| `plaintext` | yes | the logical content; exactly one of the plaintext forms below |
+| `layers` | yes | codings to apply, **left-to-right**, in the same order the header lists them. `[]` means the body is the plaintext |
+| `members` | no | encode the plaintext as this many independently-encoded, concatenated streams instead of one. Default `1` |
+| `truncate` | no | exactly one of `{"drop_tail_bytes": n}`, `{"keep_head_bytes": n}` or `{"keep_head_ratio": [num, den]}`, applied to the **encoded** bytes after all layers |
+
+`members` splits the **plaintext** into that many near-equal chunks (remainder
+on the last), runs each through `layers` on its own, and concatenates the
+results. Splitting the plaintext rather than the encoded stream is what makes
+the recipe compressor-independent: the member boundary is at the same logical
+offset whoever compressed it, so the case can still assert equality with the
+*whole* plaintext. A split of the encoded bytes could not — it would be cutting
+at an offset only one encoder's output has.
+
+Both codings this corpus decodes allow it. A gzip stream is a series of members
+(RFC 1952 §2.2) and zstd frames may be concatenated, and a streaming compressor
+that flushes mid-body produces exactly this. Every chunk must be non-empty, so
+`members` may not exceed the plaintext length.
+
+`keep_head_ratio` is integer arithmetic: keep `len * num / den` bytes, truncating
+the division. The encoded length differs per compressor, so a ratio-truncated
+case can only assert a property of its output, never a length.
+
+The three forms are not interchangeable, because each one holds a different
+thing fixed across compressors:
+
+* `drop_tail_bytes` fixes what is **missing** — a gzip stream short its 8-byte
+  trailer is short its trailer whoever compressed it.
+* `keep_head_ratio` fixes a **fraction**, so the surviving prefix is
+  compressor-specific and the case can only assert a property of the output.
+* `keep_head_bytes` fixes an **absolute prefix**, and is the only form that can
+  express a cut point derived from the container format rather than from one
+  encoder's output. Use it when the case asserts what is *not* yet possible at
+  that offset — `salvage-refused-when-nothing-produced` keeps 9 bytes because
+  that is the most a zstd frame can carry before any encoder's first byte of
+  block content, so "no output is producible" holds for all of them rather than
+  for the compressor that happened to build the fixture.
+
+Plaintext forms:
+
+| form | meaning |
+| --- | --- |
+| `{"utf8": "…"}` | the string's UTF-8 bytes |
+| `{"repeat_utf8": {"text": "…", "count": n}}` | that string's bytes, `n` times |
+| `{"repeat_byte": {"byte": b, "count": n}}` | the single byte value `b` (0–255), `n` times |
+
+### `expect`
+
+`outcome` is one of:
+
+| outcome | meaning |
+| --- | --- |
+| `decoded` | clean decode; the implementation reports no truncation |
+| `salvaged` | decode succeeded only by accepting partial output; the implementation **must** report the truncation |
+| `error` | the body was refused |
+
+For `decoded` and `salvaged`, `expect.decoded` holds exactly one of:
+
+| form | meaning |
+| --- | --- |
+| `{"equals_plaintext": true}` | byte-equal to `body.build.plaintext` |
+| `{"bytes_b64": "…"}` | byte-equal to these literal bytes |
+| `{"nonempty_prefix_of_plaintext": true}` | non-empty, and a prefix of the plaintext — the only assertion a ratio-truncated salvage can make |
+
+For `error`, `expect.error` holds:
+
+| field | required | meaning |
+| --- | --- | --- |
+| `class` | yes | `unsupported` \| `oversize` \| `undecodable` |
+| `message_contains` | no | substrings the error message must contain. **Asserted** — but used only where the message content is itself the contract |
+| `detail` | no | free text naming the sub-reason. **Never asserted**; it is for the reader |
+
+## The failure taxonomy
+
+Three classes, and only three:
+
+* **`unsupported`** — no decoder for the named coding (`br`, `deflate`). Nothing
+  was read; the bytes are irrelevant.
+* **`oversize`** — the decoded output crossed the per-layer cap. The bomb guard.
+* **`undecodable`** — the stream could not be read to a keepable result: corrupt
+  frame, bad header, a resource bound refused up front, or an early end that
+  produced nothing.
+
+The three exist because both implementations already distinguish them
+structurally — Rust as `DecodeError::{Unsupported, TooLarge, Read}`, Go by
+message — so pinning them costs nothing and catches a real class of drift: a
+decoder that starts passing an unknown coding through instead of refusing it
+still fails, because `unsupported` is not `decoded`.
+
+`undecodable` is deliberately coarse. Corrupt-body and window-bound-refused are
+separate things to a human and the same thing to both implementations, so the
+corpus records the difference in `detail` (unasserted) rather than inventing a
+distinction neither side carries. Splitting it is a change to both
+implementations first and to this corpus second, never the reverse.
+
+**This is not the drop-reason enum.** `extproc`'s `DropResponseDecode` collapses
+all three into one metrics label. That is a separate decision one layer up, and
+it stays as it is; this corpus does not touch it. It does make splitting it cheap
+if anyone wants to — the classes are already named and already asserted.
+
+## `contested`
+
+A few cases carry a `contested` object. These are the ones the corpus was written
+to force: places where the two implementations already diverged, or could, and
+where the divergence was invisible because nothing compared them. The object is
+prose — no consumer asserts it — and records `question`, `decision`, and whatever
+else the decision needs (`rationale`, `open`, `caller_precondition`,
+`suspected_bug`).
+
+It is deliberately part of the case rather than of this README, so that the
+reasoning travels with the bytes into every vendored copy, and so that the next
+person to change the case finds the argument before they change it.
+
+Three cases carry one today: `limit-zstd-window-over-cap`,
+`contested-empty-body-under-gzip`, and `divergence-empty-body-under-zstd`.
+
+## Consumers
+
+Each implementation table-tests over `cases/*.json`: build the body from
+`body`, call its decoder with `encoding`, and assert the outcome.
+
+In this repository:
+
+* `pkg/capture/contentencoding_corpus_test.go` — the reference oracle over
+  `DecodeContentEncoding`, and the authored-home gate (DIGEST + policy
+  coverage).
+
+Elsewhere, via a vendored copy:
+
+* `tapesctl` — `crates/tapesctl/src/start/content_encoding.rs`. Not yet wired up;
+  the corpus is the specification it should be wired to.
+
+## `DIGEST`
+
+Same sealing rule as `fixtures/envelope/DIGEST` and `fixtures/thread/DIGEST`: for
+each `cases/*.json`, sorted by base name, feed
+`"<basename>  <sha256-hex-of-file-bytes>\n"` into a SHA-256; the digest is
+`"sha256:" + hex` of that hash. Consumers vendor `DIGEST` alongside `cases/` and
+recompute it in their own suite, so a stale or hand-edited copy fails in the
+consumer's own CI.
+
+`DIGEST` makes a corpus change **visible**, not impossible. It is recomputed
+from whatever cases exist, so deleting a case and re-sealing is a legal, green
+two-line diff — which is why the coverage gate below names the case that pins
+each rule rather than trusting the seal to notice.
+
+## Coverage
+
+The authored home also asserts, rule by rule, that the case pinning each policy
+rule is still there and still pins it. That table is closed in both directions:
+a rule whose case is gone fails, and a case no rule names fails. So a case
+cannot be deleted, gutted in place, or renamed without the diff saying which
+policy rule moved — and a new case cannot be added without recording what it is
+for.
+
+## Adding a case
+
+1. Write `cases/<name>.json`; `name` must match the filename.
+2. Prefer a `build` recipe. Use `bytes_b64` only when the exact bytes are the
+   assertion, and annotate them in `notes`.
+3. Fill in `grounding` — the policy rule the case pins, in behavioral terms.
+4. If the case encodes a decision rather than a settled rule, add `contested`.
+5. Add the rule the case pins to the coverage table in the authored-home gate,
+   naming this case. The gate fails on any case no rule names.
+6. Run `go test ./pkg/capture/`, copy the new digest it prints into `DIGEST`,
+   and commit both.
+7. Re-sync any vendored copy from the same commit, together with whatever
+   implementation change the new case forces.

fixtures/content-encoding/cases/contested-empty-body-under-gzip.json

@@ -0,0 +1,15 @@
+{
+  "name": "contested-empty-body-under-gzip",
+  "category": "error",
+  "description": "A zero-byte body whose headers still claim gzip: the decoder errors, because an empty byte string is not a gzip stream.",
+  "encoding": "gzip",
+  "body": { "bytes_b64": "" },
+  "expect": { "outcome": "error", "error": { "class": "undecodable", "detail": "EOF reading the gzip header" } },
+  "grounding": "The decoder's contract is over (bytes, encoding) alone. Given a claimed coding and no stream to undo it on, erroring is the only honest answer — the decoder cannot know whether the body was empty by design or lost in flight.",
+  "contested": {
+    "question": "A bodiless request (e.g. a GET) whose headers still advertise an encoding. Go's decoder errors on empty input; the tapesctl proxy returns before decoding, so the same request is a debug-level non-event there. Which is correct?",
+    "decision": "Both, at different layers — and the corpus says so rather than picking one. This case pins the DECODER: empty input under a claimed coding is an error. The rule that a bodiless request must never reach the decoder is a CALLER PRECONDITION, stated in `caller_precondition` below and asserted by neither side's decoder test. tapesctl's early return satisfies it and is conformant, not divergent.",
+    "rationale": "The alternative — teaching the decoder to return empty-for-empty — would silently swallow a body that was lost in flight, which is the failure this whole corpus exists to make loud. And it cannot be done consistently anyway: see divergence-empty-body-under-zstd, where the Go decoder already does exactly that, for zstd only.",
+    "caller_precondition": "A caller MUST NOT invoke the decoder for a request with no body. A bodiless request is a non-event, not a dropped turn, and must not be counted or logged as a decode failure."
+  }
+}

fixtures/content-encoding/cases/divergence-empty-body-under-zstd.json

@@ -0,0 +1,15 @@
+{
+  "name": "divergence-empty-body-under-zstd",
+  "category": "error",
+  "description": "The same zero-byte body under zstd instead of gzip: the Go reference implementation returns success with zero bytes decoded. Recorded as observed, not as endorsed.",
+  "encoding": "zstd",
+  "body": { "bytes_b64": "" },
+  "expect": { "outcome": "decoded", "decoded": { "bytes_b64": "" } },
+  "grounding": "Documents actual current behaviour of the reference implementation so that a second implementation is compared against what exists rather than against what the prose claims.",
+  "contested": {
+    "question": "Is an empty body under zstd a success or an error?",
+    "decision": "The corpus records the reference implementation's answer (success, zero bytes) and flags it as a suspected bug rather than promoting it to a rule. Encoding the gzip answer here instead would make this case fail on the very implementation it is supposed to describe, and the corpus would be asserting an intent nobody has implemented.",
+    "suspected_bug": "Go's gzip reader consumes its header eagerly and so reports EOF on empty input, while the zstd reader yields zero bytes and no error. Identical inputs, opposite outcomes, decided by which decoder the header happened to name. Whichever answer is chosen, both codings should give it. Fixing this is out of scope here — see contested-empty-body-under-gzip: with the caller precondition honoured, neither answer is reachable in production.",
+    "open": "A Rust consumer wiring itself up should report what its zstd binding does on empty input rather than assuming. If it errors, this case is the first genuine cross-language divergence in the corpus and the pair should be resolved in the reference implementation, not papered over here."
+  }
+}

fixtures/content-encoding/cases/error-corrupt-gzip-header.json

@@ -0,0 +1,10 @@
+{
+  "name": "error-corrupt-gzip-header",
+  "category": "error",
+  "description": "Bytes that are not a gzip frame at all are refused.",
+  "encoding": "gzip",
+  "body": { "bytes_b64": "bm90IGEgZ3ppcCBmcmFtZSBhdCBhbGw=" },
+  "expect": { "outcome": "error", "error": { "class": "undecodable", "detail": "invalid gzip header" } },
+  "grounding": "A corrupt header is a hard failure, not a salvage: there is no partial turn inside it to keep. Salvage is scoped to streams that ended early after producing output.",
+  "notes": "Byte-exact: the ASCII string 'not a gzip frame at all'."
+}

fixtures/content-encoding/cases/error-corrupt-zstd-frame.json

@@ -0,0 +1,10 @@
+{
+  "name": "error-corrupt-zstd-frame",
+  "category": "error",
+  "description": "A zstd magic number followed by a malformed frame header is refused.",
+  "encoding": "zstd",
+  "body": { "bytes_b64": "KLUv/f8=" },
+  "expect": { "outcome": "error", "error": { "class": "undecodable", "detail": "reserved bit set on frame header" } },
+  "grounding": "Corruption that is detectable in the header must fail before any output exists, so it can never be mistaken for the salvage path.",
+  "notes": "Byte-exact: the zstd magic 28 b5 2f fd followed by 0xff, a frame-header descriptor with the reserved bit set."
+}