Skip to content

SPEC.md

Latest commit

 

History

History
310 lines (263 loc) · 18.1 KB

SPEC.md

File metadata and controls

310 lines (263 loc) · 18.1 KB
Raw

dig-urn-resolver — normative specification

This document is the authoritative contract for dig-urn-resolver. An independent reimplementation MUST behave as specified here. Where this spec references DIG read semantics it defers to the canonical sources: the Digstore store format + read-crypto (digstore-core), the client→node ladder (superproject CLAUDE.md §5.3), and the dig-node read wire (SYSTEM.md).

1. Purpose

Resolve a DIG URN to the bytes + content type of the resource it names, verifying integrity fail-closed, following the §5.3 node-first ladder.

This crate is INTENDED as the canonical, project-wide client-side URN→data resolver (#668). Its consumers today are the Rust crate (dig-node-service) and the wasm/npm package @dignetwork/dig-urn-resolver (consumed by dig-web-resolver); hub.dig.net, the Chrome extension and dig-sdk parse URNs with their own implementations and consume neither package. That convergence is tracked work, NOT a current invariant (dig_ecosystem#2725 / #2753); a new consumer MUST use this crate rather than add a parser. It sits strictly UPSTREAM of dig-node — a client that talks to a dig-node over the wire (node /s/ + /health, else the rpc gateway). dig-node performs all heavy lifting (sync, serve, decrypt, chain anchoring) and MUST NOT depend on this crate. Consumers use the Rust crate or @dignetwork/dig-urn-resolver (JS/wasm).

2. URN grammar

urn:dig:chia:<store_id>[:<root>]/<resource_key>[?salt=<hex>]
  • store_id — 64 lowercase hex chars (singleton launcher id). REQUIRED.
  • root — OPTIONAL 64 hex chars pinning one on-chain generation. The root is the trust anchor for inclusion verification ONLY; it is NOT a key input.
  • resource_key — the path within the store. An ABSENT or empty resource path resolves to the §8.5 default view index.html (a bare-store or trailing-slash URN is NOT rejected — it names the store's landing page).
  • ?salt=<hex> — OPTIONAL out-of-band private-store secret salt.

Parsing MUST reuse the canonical dig-urn-protocol URN parser (DigUrn::parse_with_salt) for the urn:dig:<chain>:<store>[:<root>]/<key>[?salt=<hex>] scheme — the single source of truth for the grammar, pinned by the frozen conformance corpus (dig-urn-protocol/tests/fixtures/urn_conformance.json). A syntactically invalid URN MUST produce a hard parse error.

3. Keys (reused, root-independent)

  • retrieval_key = SHA-256(canonical_rootless_urn), where the canonical rootless URN is urn:dig:chia:<store_id>/<resource_key> (root dropped). This is the ecosystem's on-wire lookup key (the value the node indexes as retrieval_key), and it is dig_urn_protocol::DigUrn::content_key() — NOT DigUrn::retrieval_key(), which is a DIFFERENT, root-PINNED hash. Implementations MUST map the resolver's retrieval_key to content_key.
  • decryption_key = digstore_core::crypto::derive_decryption_key(canonical_rootless_urn, salt?) (HKDF-SHA256, paper §11).

Both are root-independent so they are stable across generations. Implementations MUST NOT reimplement these: the URN scheme + content_key/retrieval_key derivation come from dig-urn-protocol, and the merkle codec/fold + symmetric read-crypto primitives come from digstore-core (injected via dig_urn_protocol::verify::ContentCrypto, reused unchanged).

4. The §5.3 ladder — node trust is LOOPBACK-ONLY

Resolution order, first that responds wins:

  1. explicit endpoint override (from options) — WINS, skips the ladder.
  2. http://dig.local:9778 (node, iff loopback — see below)
  3. http://localhost:9778 (node)
  4. https://rpc.dig.net (rpc) — the FINAL fallback.

Node trust (EndpointKind::Node) is granted ONLY to an ASSERTED-LOOPBACK host — a 127.0.0.0/8 or ::1 literal, the reserved name localhost, or dig.local iff it resolves (OS resolver / hosts) to loopback addresses only. The node /s/ path returns server-decrypted bytes with NO client-side crypto, so it is sound ONLY on the user's own machine. EVERY other host — including an explicit override at a remote host — MUST use the client-verified Rpc path. Implementations that cannot resolve names (e.g. a browser) MUST treat a non-literal, non-localhost name as NON-loopback (no node trust).

  • A node tier is selected only when it is an asserted-loopback host AND a cheap GET {base}/health returns 2xx within a short timeout; otherwise the ladder falls through.
  • An override is classified by HOST: a loopback host → node surface; ANY other host → the client-verified rpc surface. An override MUST NOT silently fall back to the public gateway.
  • The auto-ladder plan is [node(first-healthy-loopback), rpc(rpc.dig.net)] when a loopback node tier is healthy, else [rpc(rpc.dig.net)].
  • The resolved plan SHOULD be cached per resolver instance.

4.1 Cross-tier error classification — fall through on ABSENCE, fail closed on INTEGRITY

Walking the plan, a per-tier failure is classified into exactly one of three actions. Getting this split right is a SECURITY property, not a convenience:

  • TRY NEXT TIER (fall through) — a tier's NotFound (content genuinely absent here — a node 404, or a gateway total_length == 0) OR its transport failure (tier unreachable). This is the stranger's common case: the local node does not hold the content, so the ladder MUST continue to the public gateway that does. A local NotFound MUST NOT abort the ladder.
    • When EVERY tier returns NotFound, the resolve yields ONE branded final ResolveError::NotFound (the content exists nowhere reachable) — not Unreachable, not a raw per-tier error.
    • When the LAST tier is transport-unreachable, the resolve yields ResolveOutcome::Unreachable (§6).
  • ABORT THE WHOLE LADDER IMMEDIATELY (fail closed) — a tier's VerifyFailed / DecryptFailed (integrity: tampered bytes, a non-chaining proof, a wrong root, a decrypt-tag failure). This is surfaced as ResolveOutcome::IntegrityFailure and MUST NEVER fall through to another tier. Falling through after an integrity failure would let an attacker turn a tampered response at one tier into a silent retry that serves attacker-chosen bytes from another — the exact fail-closed hole this rule forbids (superproject CLAUDE.md §5.4).
  • SURFACE A HARD ERROR — a reachable protocol error (Rpc) or a RootRequired (rootless URN over the untrusted gateway) — returned as a hard Err (§6). Not absence, not unreachability, not integrity.

5. Read paths

5.1 Node path (EndpointKind::Node) — loopback only

GET {base}/s/<store_id>[:<root>]/<resource_key>. A loopback node may answer with verified PLAINTEXT or with CIPHERTEXT; the shape is detected DETERMINISTICALLY by headers (never assumed):

  • Verified plaintext — 2xx AND X-Dig-Verified: true → the body is the node-decrypted, node-verified plaintext (loopback trust). Content type: the response Content-Type, else derived (§7). No client-side crypto.
  • Ciphertext — 2xx AND (X-Dig-Encrypted: true OR an X-Dig-Inclusion-Proof header) → the body is opaque ciphertext that MUST be client-side verified+decrypted exactly like the rpc path (§5.2 step 4), reusing digstore-core and threading the URN salt. The trust root is the URN's pinned root, else the (loopback) node's X-Dig-Root; the proof is X-Dig-Inclusion-Proof; X-Dig-Chunk-Lens (comma- separated) gives the chunk layout. A node returning ciphertext is NOT trusted blindly.
  • 2xx that is neither attested plaintext nor decryptable ciphertext → hard VerifyFailed (fail-closed; §6 IntegrityFailure). Bytes are never returned.
  • Rootless resolution over this tier — a rootless URN is served as GET {base}/s/<store_id>/<resource_key> (no :<root>); the node resolves the store's current chain-anchored tip itself and reports it back via X-Dig-Root (both response shapes above). The client trusts that header as the resolve's root ONLY because this is the asserted-loopback node (never over rpc, §5.2 step 1); the ciphertext shape is still fully client-verified against it (gate-then-decrypt, same as a pinned root) — the node is trusted to NAME the tip, never to attest unverified bytes. A rootless URN with no healthy loopback node falls through the ladder to the rpc tier, which rejects it with RootRequired (§4.1, §5.2 step 1) — the wall never opens just because the node tier was unavailable.
  • 404 → NotFound (content absent at this tier; the ladder falls through to the next tier — §4.1).
  • other non-2xx / transport failure → a transport failure (ladder falls through).

5.2 RPC path (EndpointKind::Rpc) — the trust root comes from the URN

  1. The trust root MUST be the URN's pinned root. A ROOTLESS URN over this untrusted tier is REJECTED with a hard RootRequired error (its root would otherwise come from the same untrusted gateway, allowing a compromised gateway to prove attacker bytes for a public store against a fake root). No dig.getAnchoredRoot call is made on this tier.
  2. retrieval_key per §3.
  3. Stream windowed dig.getContent {store_id, root, retrieval_key, offset, length} -> {total_length, offset, next_offset?, complete?, ciphertext (b64), inclusion_proof (b64), chunk_lens}, accumulating ciphertext until complete or next_offset == null. total_length == 0 → NotFound (absent at this tier; falls through — §4.1).
  4. Verify then decrypt (gate-then-decrypt), via digstore-core:
    • inclusion: resource_leaf(ciphertext) == proof.leaf, proof.verify(), and proof.root == trusted_root. ANY failure → integrity failure (§6).
    • decrypt: split by chunk_lens (empty ⇒ single chunk), AES-256-GCM-SIV-open each under the URN key. A tag failure → integrity failure (§6).
  5. Content type derived per §7.

A JSON-RPC error object or a malformed/unexpected body is a hard Rpc error (the endpoint IS reachable). A transport failure / non-2xx HTTP is a transport failure.

6. Outcomes (fail-closed)

A resolve yields Result<ResolveOutcome, ResolveError>:

  • ResolveOutcome::Success(data) — verified content.
  • ResolveOutcome::IntegrityFailure — bytes were fetched but failed inclusion or decrypt verification. The unverified bytes MUST NOT be returned or carried. This is returned IMMEDIATELY at the producing tier (never cascaded, never masked as unreachable).
  • ResolveOutcome::Unreachable — every tier was transport-unreachable; nothing was fetched.
  • Err(ResolveError) — Parse, NotFound (the branded final not-found after EVERY tier reported absence — §4.1), RootRequired (a rootless URN over the untrusted rpc tier), or Rpc (a reachable protocol error).

A per-tier NotFound MUST NOT abort the ladder — it falls through (§4.1); only an integrity failure aborts. IntegrityFailure and Unreachable MUST be distinct and never conflated: integrity-fail = reached the network, bytes don't verify (security); unreachable = couldn't reach the network (retryable).

For a render/image path, IntegrityFailure MAY render a branded "Integrity Verification Failed" text/html document and Unreachable a branded "DIG Network unreachable" + Connect-to-Node document. An image/object-URL helper MUST NOT return the unverified bytes as content for an integrity failure — it returns the security document instead.

7. Content type

Derived from the resource path extension first, then a magic-byte sniff, falling back to application/octet-stream. The node path prefers the response Content-Type.

8. Transport & environments

The core logic depends only on an injected async transport (HttpTransport: get, post_json). Bundled implementations: reqwest (native, native feature) and the browser fetch (wasm, wasm feature, resolved off globalThis.fetch so it works in a browser window, a worker, AND Node.js ≥18). Node-class transports SHOULD use short connect timeouts so dead ladder tiers fall through quickly.

The wasm package (@dignetwork/dig-urn-resolver) MUST work in BOTH a browser and Node.js, degrading gracefully (never a hard env failure). The branded API + the three outcomes + MIME are IDENTICAL across environments; only env plumbing branches:

  • Ladder — a local-node /health probe that cannot run (e.g. CORS-blocked in a browser) MUST NOT throw; the ladder catches it and falls through to the verified rpc tier. A browser that can't reach a local node reaches the SAME fail-closed rpc path, never unverified bytes.
  • Cache (§11) — the disk tier is native-only; in the wasm package a cachePath MUST degrade to the in-memory cache without throwing.
  • Image URL — resolveImageUrl MUST return a usable URL in both: a blob: URL where URL.createObjectURL exists, else a data: URL. resolve() (bytes + contentType) MUST work regardless, so a consumer without blob-URL support can use the bytes directly.

The package ships a dual-target build: the wasm-bindgen web (ESM) build for browser bundlers and the nodejs (CommonJS) build for Node, routed by the package exports map (browser/import → web, node/require → node).

9. wasm surface (@dignetwork/dig-urn-resolver)

The front-door API is the branded DigNetwork class:

  • new DigNetwork(options?) — a single, named-field options object (NOT positional args). options is a DigNetworkOptions with all-optional fields { endpoint?, connectUrl?, cachePath? }; an omitted (or blank) field keeps its §5.3 default, so new DigNetwork() is all-defaults and new DigNetwork({ cachePath }) sets only the disk cache. Unknown properties are ignored. cachePath is an optional disk-cache directory (native/Node.js; ignored in the browser, see §11). The configured fields are readable back via the endpoint / connectUrl / cachePath getters. The package's generated .d.ts MUST export the DigNetworkOptions interface with those named fields (never any).
  • dig.resolve(urn) : Promise<{ outcome, bytes, contentType }>, outcome ∈ "success" | "integrity_failure" | "unreachable". contentType is present on EVERY result.
  • dig.resolveImageUrl(urn) : Promise<string> — an <img src> URL that ALWAYS resolves (never throws for a normal failure): a blob: URL of the real verified image on success, else a branded DIG error IMAGE as a data:image/png;base64 URI matching the failure (integrity / unreachable / not-found / invalid-URN / generic). An <img> cannot render the HTML error docs, so these prerendered PNGs are the image-path variant. FAIL-CLOSED: the integrity image is a STATIC branded placeholder — unverified bytes are NEVER returned as the image.

Low-level free functions resolve(urn, endpoint?, connectUrl?) and resolveObjectUrl(urn, endpoint?, connectUrl?) MAY also be exported (delegating to DigNetwork); the branded class is the documented surface.

10. Conformance

  • URN parse MUST match dig-urn-protocol byte-for-byte (the frozen conformance corpus); the retrieval key MUST equal dig_urn_protocol::DigUrn::content_key and the decryption key MUST match digstore-core byte-for-byte. A cross-parser equivalence test (tests/cross_parser_equivalence.rs) pins the wire key against both parsers over the frozen corpus, including a root-pinned URN (guarding the content_key vs retrieval_key trap).
  • A tampered ciphertext, a non-chaining proof, a wrong root, or a wrong/absent salt MUST yield IntegrityFailure, never data.
  • Only an asserted-loopback host is granted node trust; a remote host (incl. an override) MUST use the verified rpc path. A node response without X-Dig-Verified: true MUST fail closed.
  • A rootless URN over the rpc tier MUST be rejected (RootRequired); the rpc tier MUST NOT call dig.getAnchoredRoot.
  • Node-absent + a serving rpc gateway MUST resolve a valid ROOT-PINNED resource to Success.
  • A local-tier NotFound (node 404) MUST fall through to the next tier: a healthy node that lacks the resource + a serving gateway that has it MUST resolve to Success (§4.1). When EVERY tier reports absence the result MUST be one branded NotFound.
  • An integrity failure (VerifyFailed / DecryptFailed) at ANY tier MUST abort the whole ladder as IntegrityFailure and MUST NOT be retried on a later tier — a tampered response MUST NOT become a silent retry that serves other bytes (§4.1, §5.4).
  • A node CIPHERTEXT response MUST be client-side verified+decrypted (not trusted); a salted URN MUST decrypt salted content on BOTH tiers, and a wrong/absent salt MUST yield IntegrityFailure.
  • A ROOTLESS URN resolved over a healthy loopback node returning CIPHERTEXT MUST derive its trust root from X-Dig-Root and verify+decrypt against it to Success; the same rootless URN with no healthy loopback node MUST fall through to the rpc tier and be rejected RootRequired — the node is the ONLY tier ever trusted to resolve a rootless URN's tip.
  • Caching (§11) MUST NOT weaken fail-closed: only Success is cached; a disk hit is re-verified (a tampered file → IntegrityFailure).

11. Caching

Results are cached in front of resolve; this MUST NOT weaken fail-closed (§6).

  • Cacheable: ONLY a verified Success. IntegrityFailure / Unreachable / NotFound / any Err MUST NOT be cached.
  • Key: the content-addressed identity storeId:root:resourceKey:salt with the CONCRETE resolved root (a root-pinned URN's root, or the node's X-Dig-Root) — never the raw request URN. A rootless URN with no concrete root is not cached.
  • Memory tier (bounded LRU, both native + wasm): process-trusted — holds only what THIS process verified this run; a hit MAY skip re-verification. Bounded by an entry count AND a byte budget (no unbounded growth in a wallet).
  • Disk tier (optional, native, UNTRUSTED): stores the VERIFIABLE artifacts (ciphertext + inclusion proof + chunk lengths), NOT plaintext. A disk hit MUST be RE-VERIFIED against the URN's root before use (same merkle/decrypt gate); a tampered entry MUST fail verification → IntegrityFailure and MUST NOT be served. Filenames MUST be SHA-256(identity) (content-addressed, no path-traversal). Ignored where there is no filesystem (the browser); the memory tier still applies.