Pin each release's rust base image by digest

[fnando]

What

Replaces the shared rust_image_digests map in builds.json with a per-release base-image pin. Each rust_versions[] entry is fully-qualified as <label>@sha256:<digest> (e.g. 1.94.0-slim-trixie@sha256:…). The published image tag is the plain, human-readable <cli>-rust<label>[-<arch>]; the base digest pins the build's FROM but is not encoded in the tag.

Why

Closes #18. The old model keyed digests by label in one shared map, so a new CLI version couldn't adopt a freshly-rebuilt base without dragging older versions onto the same digest. Making the (release, label, digest) triple the unit that owns a digest decouples releases — a different cli version is a different tag, and builds.json records the exact base each release was built against.

Tags

• Published tags are <cli>-rust<label>[-<arch>] (e.g. 26.1.0-rust1.95.0-slim-trixie-amd64), plus the :<cli> and :latest moving aliases.
• Tags are mutable — re-publishing a pair overwrites in place; there is no skip-if-published guard. The only stable reference for SEP-58 bldimg is the per-arch content digest (@sha256:…).
• builds.json is append-only (a full ledger of every base a release was built against); resolve_matrix publishes only the newest pin per label (last in the list wins), so the mutable tag deterministically resolves to the latest base.

Notable details

• refresh.py (merging the two old refresh scripts) resolves the cli ref + each base's current digest and appends label@digest pins; release_prepare.py is a thin orchestrator over it.
• The build still pins FROM by digest and cross-checks it against the image's org.opencontainers.image.base.digest label in the smoke test; the matrix keeps a digest-bearing rust_base_id for collision-free per-image artifact names.
• Done test-first: 192 tests pass; ruff check and ruff format --check clean.

Validation

Validated end-to-end on a fork (publishing to fnando/stellar-cli-experimental); the publish run was green across every job:

• Release flow auto-generated and merged the release PR: https://github.com/fnando/stellar-cli-docker/pull/26
• Published GitHub Release (Tags + per-arch sections use the label-only tags): https://github.com/fnando/stellar-cli-docker/releases/tag/v26.1.0
• Publish run, all jobs ✓: https://github.com/fnando/stellar-cli-docker/actions/runs/26925993472
• Published images: https://hub.docker.com/repository/docker/fnando/stellar-cli-experimental/tags

refresh appended the new 1.96.0-slim-trixie pin to 26.1.0 alongside 1.95.0, and the published Docker Hub tags confirm the scheme:

26.1.0-rust1.95.0-slim-trixie[-amd64|-arm64]
26.1.0-rust1.96.0-slim-trixie[-amd64|-arm64]
26.1.0   latest

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0d14120002

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[Copilot]

Pull request overview

This PR updates the release/build pipeline to pin Rust base images per stellar-cli release by inlining each base as <label>@sha256:<digest> in builds.json, and to encode a short digest fragment (and shortened stellar-cli ref) into published image tags to preserve tag immutability across base rebuilds.

Changes:

• Replace the shared rust_image_digests map with per-release rust_versions[] pins (<label>@sha256:<digest>) and update the JSON schema + validation accordingly.
• Introduce scripts/refresh.py as the unified “append pins + resolve refs” mechanism and refactor release preparation to delegate to it.
• Thread the base digest into tag composition, matrix generation, build/publish workflows, and update docs/tests to match the new tag scheme.

Reviewed changes

Copilot reviewed 35 out of 35 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tests/unit/test_validate_json.py	Updates schema validation tests; adds cases for pinned entries/uniqueness.
tests/unit/test_tag_names.py	Updates tag composition expectations to include ref15/digest15 and required digest arg.
tests/unit/test_publish_manifests.py	Updates manifest ref composition to include digest fragment in tags.
tests/unit/test_publish_aliases.py	Adjusts fixture data to use pinned rust_versions[] entries.
tests/unit/test_newest_pair.py	Updates expectation: newest rust output is now a full pin string.
tests/unit/test_builds.py	Adds pin parsing helpers and updates default-rust selection semantics.
tests/unit/test_build_image.py	Updates local build command expectations to require/passthrough base digest.
tests/integration/test_release_prepare.py	Refactors integration tests for release_prepare delegating to refresh.py.
tests/integration/test_refresh.py	New integration coverage for refresh append-only pin behavior + ref resolution.
tests/integration/test_refresh_stellar_cli_digests.py	Removed (functionality folded into refresh.py).
tests/integration/test_refresh_rust_digests.py	Removed (functionality folded into refresh.py).
tests/fixtures/builds_unsorted_keys.json	Updates fixture to reflect new top-level schema (no digest map).
tests/fixtures/builds_unpinned_refs.json	Updates fixture to use pinned rust entries and note refresh’s new role.
tests/fixtures/builds_orphan_rust.json	Removed (orphan-map constraint no longer exists).
tests/fixtures/builds_multi_cli.json	Updates fixture to inline per-entry pins.
tests/fixtures/builds_minimal.json	Updates minimal fixture to inline per-entry pins.
tests/fixtures/builds_invalid_entry.json	Updates fixture to represent a bare-label schema violation.
scripts/validate_json.py	Drops cross-field constraints; validates sorted keys + JSON schema only.
scripts/tag_names.py	Requires base digest input; truncates ref/digest and appends digest fragment into tags.
scripts/resolve_matrix.py	Switches from label→digest map lookup to splitting pins into (label,digest).
scripts/release_prepare.py	Becomes orchestrator over refresh.py + validation + “no-op” guard.
scripts/release_pr_body.py	Updates generated PR body text to describe pinning behavior.
scripts/refresh.py	New unified refresh script: resolves refs/digests and appends pins without rewriting old ones.
scripts/publish_manifests.py	Updates manifest creation to derive label+digest from pins and pass digest to tag composer.
scripts/publish_aliases.py	Uses default pin; splits into (label,digest) for tag composition.
scripts/lib/builds.py	Replaces rust_image_digest() with pin helpers (split_entry, label_of, digest_of).
scripts/build_image.py	Requires base digest as input; asserts the full pin is declared and uses canonical tag naming.
RELEASE.md	Updates documentation for new tag scheme and “rebuilt base” adoption workflow.
README.md	Updates user-facing docs for pinned base + digest-bearing tags and new refresh script.
builds.schema.json	Removes shared digest map; enforces pinned entry format and per-entry uniqueness.
builds.json	Migrates to inline rust base pins per release entry.
.github/workflows/publish.yml	Threads base digest into tag computation for publish pipeline.
.github/workflows/build.yml	Updates CI smoke-build flow to extract label+digest from the newest pin and pass digest onward.

Comments suppressed due to low confidence (2)

scripts/resolve_matrix.py:42

• build_matrix() emits rust_base_key as the label (not the full pin). Since rust_versions[] can contain the same label multiple times with different digests, this can produce matrix rows that are identical in (stellar_cli_version, rust_base_key, arch) but differ only in rust_image_digest. Downstream workflows/artifact naming that key off rust_base_key will collide/overwrite; consider including a unique per-pin field in the matrix (e.g. rust_pin and/or rust_digest15) and using it everywhere a unique identifier is required.

        for pin in entry["rust_versions"]:
            label, digest = builds.split_entry(pin)
            parsed = rust_keys.parse(label)
            for arch in ARCHES:
                rows.append(
                    {
                        "arch": arch,
                        "platform": f"linux/{arch}",
                        "rust_base_key": label,
                        "rust_base_suffix": parsed.suffix,
                        "rust_image_digest": digest,
                        "rust_version": parsed.version,
                        "stellar_cli_ref": ref,
                        "stellar_cli_version": cli,
                    }

.github/workflows/publish.yml:87

• Because the tag now includes a digest fragment and builds.json can declare the same rust_base_key label multiple times with different rust_image_digest values, this workflow needs to treat (label,digest) as the unique unit. Right now the build job name, SBOM/provenance/metadata filenames, and uploaded artifact names are keyed only by ${{ matrix.rust_base_key }} + arch, so two pins with the same label will collide and overwrite each other’s outputs. Consider incorporating a per-pin identifier into filenames/artifact names (e.g. ${{ steps.tag.outputs.tag }} or a digest15/rust_pin field from the matrix).

      - name: resolve tag
        id: tag
        run: |
          tag="$(./scripts/tag_names.py \
            --stellar-cli-version ${{ matrix.stellar_cli_version }} \
            --rust-version ${{ matrix.rust_base_key }} \
            --rust-image-digest ${{ matrix.rust_image_digest }} \
            --platform ${{ matrix.platform }} \
            --stellar-cli-ref ${{ matrix.stellar_cli_ref }})"
          echo "tag=$tag" >> "$GITHUB_OUTPUT"
          echo "image=$REGISTRY:$tag" >> "$GITHUB_OUTPUT"

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3f59d79a23

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b790370c39

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".