Skip to content

Pin each release's rust base image by digest #20

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
fnando merged 8 commits into from
Jun 8, 2026
Merged

Pin each release's rust base image by digest #20

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
0d14120
Pin each release's rust base image by digest.
fnando Jun 3, 2026
3f59d79
Disambiguate release artifacts and tags by base digest.
fnando Jun 3, 2026
8d018b8
Fix smoke test to take the pinned base image digest.
fnando Jun 3, 2026
b790370
Simplify published image tags and allow overwriting.
fnando Jun 4, 2026
17b98c2
Publish only the newest base pin per rust label.
fnando Jun 4, 2026
2078d7b
Fix release PR body and docs for mutable tags.
fnando Jun 4, 2026
44d6495
Point release notes at the per-arch digest only.
fnando Jun 4, 2026
33bb3ed
Fix release body type hint and stale schema note.
fnando Jun 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 9 additions & 3 deletions .github/workflows/build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,25 +29,31 @@ jobs:
id: pair
run: |
cli="$(./scripts/newest_pair.py --stellar-cli-version)"
rust="$(./scripts/newest_pair.py --rust-version)"
# The newest pair is a fully-qualified pin: <label>@sha256:<digest>.
pin="$(./scripts/newest_pair.py --rust-version)"
rust="${pin%@*}"
digest="${pin#*@}"
tag="$(./scripts/tag_names.py \
--stellar-cli-version "$cli" --rust-version "$rust")"
{
echo "cli=$cli"
echo "rust=$rust"
echo "digest=$digest"
echo "image=stellar-cli:$tag"
} >> "$GITHUB_OUTPUT"
- name: build image
run: |
./scripts/build_image.py \
--stellar-cli-version "${{ steps.pair.outputs.cli }}" \
--rust-version "${{ steps.pair.outputs.rust }}"
--rust-version "${{ steps.pair.outputs.rust }}" \
--rust-image-digest "${{ steps.pair.outputs.digest }}"
Comment thread
fnando marked this conversation as resolved.
- name: smoke test
run: |
./scripts/smoke_test_image.py \
--image "${{ steps.pair.outputs.image }}" \
--stellar-cli-version "${{ steps.pair.outputs.cli }}" \
--rust-version "${{ steps.pair.outputs.rust }}"
--rust-version "${{ steps.pair.outputs.rust }}" \
--rust-image-digest "${{ steps.pair.outputs.digest }}"
- name: wasm reproducibility
run: |
./scripts/repro_test.py \
Expand Down
59 changes: 7 additions & 52 deletions .github/workflows/publish.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,8 +80,7 @@ jobs:
tag="$(./scripts/tag_names.py \
--stellar-cli-version ${{ matrix.stellar_cli_version }} \
--rust-version ${{ matrix.rust_base_key }} \
--platform ${{ matrix.platform }} \
--stellar-cli-ref ${{ matrix.stellar_cli_ref }})"
--platform ${{ matrix.platform }})"
echo "tag=$tag" >> "$GITHUB_OUTPUT"
echo "image=$REGISTRY:$tag" >> "$GITHUB_OUTPUT"

Expand All @@ -91,32 +90,13 @@ jobs:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: check whether already published
id: skip
run: |
if docker buildx imagetools inspect "${{ steps.tag.outputs.image }}" >/dev/null 2>&1; then
echo "::warning::${{ steps.tag.outputs.image }} already exists in the registry; skipping build (per-arch tags are immutable)"
{
echo "## ⚠️ Skipped — already published"
echo ""
echo "\`${{ steps.tag.outputs.image }}\` was already in the registry."
echo ""
echo "Per-arch tags are immutable, so no build or push happened for this row. If this is a corrupt-push recovery, delete the tag in Docker Hub by hand and re-run the workflow."
} >> "$GITHUB_STEP_SUMMARY"
echo "skipped=true" >> "$GITHUB_OUTPUT"
else
echo "skipped=false" >> "$GITHUB_OUTPUT"
fi

- name: build metadata
id: meta
if: steps.skip.outputs.skipped != 'true'
run: |
echo "build_date=$(date -u +%Y-%m-%dT%H:%M:%SZ)" >> "$GITHUB_OUTPUT"

- name: build and push
id: build
if: steps.skip.outputs.skipped != 'true'
uses: docker/build-push-action@bcafcacb16a39f128d818304e6c9c0c18556b85f # v7.1.0
with:
context: .
Expand All @@ -135,18 +115,16 @@ jobs:
SOURCE_REPO=${{ github.repository }}

- name: generate SBOM file
if: steps.skip.outputs.skipped != 'true'
uses: anchore/sbom-action@e22c389904149dbc22b58101806040fa8d37a610 # v0.24.0
with:
image: ${{ steps.tag.outputs.image }}@${{ steps.build.outputs.digest }}
format: spdx-json
output-file: sbom-${{ matrix.stellar_cli_version }}-rust${{ matrix.rust_base_key }}-${{ matrix.arch }}.spdx.json
output-file: sbom-${{ matrix.stellar_cli_version }}-rust${{ matrix.rust_base_id }}-${{ matrix.arch }}.spdx.json
upload-release-assets: false
upload-artifact: false

- name: attest build provenance
id: attest-prov
if: steps.skip.outputs.skipped != 'true'
uses: actions/attest-build-provenance@a2bbfa25375fe432b6a289bc6b6cd05ecd0c4c32 # v4.1.0
with:
subject-name: ${{ env.REGISTRY }}
Expand All @@ -158,19 +136,17 @@ jobs:

- name: attest SBOM
id: attest-sbom
if: steps.skip.outputs.skipped != 'true'
uses: actions/attest@59d89421af93a897026c735860bf21b6eb4f7b26 # v4.1.0
with:
subject-name: ${{ env.REGISTRY }}
subject-digest: ${{ steps.build.outputs.digest }}
sbom-path: sbom-${{ matrix.stellar_cli_version }}-rust${{ matrix.rust_base_key }}-${{ matrix.arch }}.spdx.json
sbom-path: sbom-${{ matrix.stellar_cli_version }}-rust${{ matrix.rust_base_id }}-${{ matrix.arch }}.spdx.json
push-to-registry: true

- name: write per-arch metadata
if: steps.skip.outputs.skipped != 'true'
run: |
./scripts/write_metadata.py \
--output "meta-${{ matrix.stellar_cli_version }}-rust${{ matrix.rust_base_key }}-${{ matrix.arch }}.json" \
--output "meta-${{ matrix.stellar_cli_version }}-rust${{ matrix.rust_base_id }}-${{ matrix.arch }}.json" \
--arch "${{ matrix.arch }}" \
--stellar-cli-version "${{ matrix.stellar_cli_version }}" \
--digest "${{ steps.build.outputs.digest }}" \
Expand All @@ -180,41 +156,20 @@ jobs:
--tag "${{ steps.tag.outputs.tag }}"

- name: rename provenance bundle
if: steps.skip.outputs.skipped != 'true'
run: |
cp "${{ steps.attest-prov.outputs.bundle-path }}" \
"prov-${{ matrix.stellar_cli_version }}-rust${{ matrix.rust_base_key }}-${{ matrix.arch }}.intoto.jsonl"

# Skipped pairs still need metadata so the release-body composer can
# show the full state of every declared pair, not just the freshly
# built ones. Omitting --digest tells write_metadata to resolve it
# from the existing tag in the registry; the SBOM/provenance files
# are not regenerated (they stay attached to the previously-
# published image's attestation store).
- name: write per-arch metadata (skipped pair)
if: steps.skip.outputs.skipped == 'true'
run: |
./scripts/write_metadata.py \
--output "meta-${{ matrix.stellar_cli_version }}-rust${{ matrix.rust_base_key }}-${{ matrix.arch }}.json" \
--arch "${{ matrix.arch }}" \
--stellar-cli-version "${{ matrix.stellar_cli_version }}" \
--image "${{ steps.tag.outputs.image }}" \
--rust-base-key "${{ matrix.rust_base_key }}" \
--rust-version "${{ matrix.rust_version }}" \
--tag "${{ steps.tag.outputs.tag }}"
"prov-${{ matrix.stellar_cli_version }}-rust${{ matrix.rust_base_id }}-${{ matrix.arch }}.intoto.jsonl"

- name: upload release artifacts
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: release-artifacts-${{ matrix.stellar_cli_version }}-rust${{ matrix.rust_base_key }}-${{ matrix.arch }}
name: release-artifacts-${{ matrix.stellar_cli_version }}-rust${{ matrix.rust_base_id }}-${{ matrix.arch }}
path: |
sbom-*.spdx.json
prov-*.intoto.jsonl
meta-*.json
retention-days: 7
# warn (not error) because the skipped path only produces meta-*.json;
# the sbom-*/prov-* globs find nothing in that case.
if-no-files-found: warn
if-no-files-found: error

manifest:
name: assemble manifest lists
Expand Down
47 changes: 23 additions & 24 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,13 +67,13 @@ docker run --rm \
## Verifiable builds ([SEP-58](https://github.com/stellar/stellar-protocol/blob/master/ecosystem/sep-0058.md))

For verifiable references, **always pin to a per-arch single-architecture
digest**, never to a moving tag like `:latest` or `:<cli>`, and never to a
multi-arch manifest list digest:
digest (`@sha256:…`)** — it is the only stable reference. Never use a tag or a
multi-arch manifest list digest in `bldimg`:

```sh
# Find the per-arch digest for the architecture you used to build.
# Pick any of the immutable manifest-list tags from the release notes,
# e.g. :26.0.0-<ref>-rust1.94.0-slim-trixie, or the :26.0.0 alias:
# Pick any of the manifest-list tags from the release notes,
# e.g. :26.0.0-rust1.94.0-slim-trixie, or the :26.0.0 alias:
docker buildx imagetools inspect docker.io/stellar/stellar-cli:26.0.0
```

Expand All @@ -83,36 +83,35 @@ compare the resulting WASM sha256.

## Repo layout

| Path | What |
| ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `Dockerfile` | Two-stage builder + runtime, args-driven. |
| `builds.json` | Source of truth for which (stellar-cli, rust base key) pairs we publish. |
| `builds.schema.json` | JSON Schema for `builds.json`. |
| `scripts/build_image.py` | Local single-image build. |
| `scripts/validate_json.py` | Validates every `*.json` for sorted keys and `builds.json` for schema + cross-field constraints. |
| `scripts/refresh_rust_digests.py` | Fills blank `rust_image_digests` entries by inspecting `rust:<key>` upstream (where `<key>` is the composite `<rust>-<suffix>` form). Does not touch already-pinned digests unless asked per-key. |
| `scripts/refresh_stellar_cli_digests.py` | Fills blank `stellar_cli_versions[].ref` entries by resolving the matching `v<version>` git tag in `stellar/stellar-cli`. Same per-target opt-in shape as the rust refresher. |
| `scripts/verify_image.py` | Consumer-facing verifier. Wraps `gh attestation verify` for both the SLSA build provenance and the SPDX SBOM attestations against a per-arch image digest. |
| `scripts/lib/` | Shared Python helpers imported by the other scripts (builds.json IO, semver/key parsing, subprocess + adapter wrappers). |
| Path | What |
| -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `Dockerfile` | Two-stage builder + runtime, args-driven. |
| `builds.json` | Source of truth for which (stellar-cli, rust base key) pairs we publish. |
| `builds.schema.json` | JSON Schema for `builds.json`. |
| `scripts/build_image.py` | Local single-image build. |
| `scripts/validate_json.py` | Validates every `*.json` for sorted keys and `builds.json` against the schema. |
| `scripts/refresh.py` | For one `--stellar-cli-version`: picks the rust base labels, resolves the upstream cli ref and each base's index digest, and appends the fully-qualified pins `<label>@<digest>` (append-only; already-published pins are retained). |
| `scripts/verify_image.py` | Consumer-facing verifier. Wraps `gh attestation verify` for both the SLSA build provenance and the SPDX SBOM attestations against a per-arch image digest. |
| `scripts/lib/` | Shared Python helpers imported by the other scripts (builds.json IO, semver/key parsing, subprocess + adapter wrappers). |

## Local development

```sh
# Validate builds.json.
./scripts/validate_json.py

# Build a local image for a declared (cli, rust base) pair.
./scripts/build_image.py --stellar-cli-version 26.0.0 --rust-version 1.94.0-slim-trixie
# Build a local image for a declared (cli, rust base) pair. The rust base is
# given as the label plus its pinned digest (copy the pin from builds.json).
./scripts/build_image.py --stellar-cli-version 26.0.0 \
--rust-version 1.94.0-slim-trixie \
--rust-image-digest sha256:f7bf1c266d9e48c8d724733fd97ba60464c44b743eb4f46f935577d3242d81d0

# Smoke-test the built image.
docker run --rm stellar-cli:26.0.0-rust1.94.0-slim-trixie --version
docker run --rm stellar-cli:26.0.0-rust1.94.0-slim-trixie contract build --help

# Resolve blank rust base image digests (maintainer task).
./scripts/refresh_rust_digests.py --dry-run

# Resolve blank stellar-cli refs from upstream git tags (maintainer task).
./scripts/refresh_stellar_cli_digests.py --dry-run
# Resolve + append rust base pins and the cli ref for a version (maintainer task).
./scripts/refresh.py --stellar-cli-version 26.1.0 --dry-run
```

Requirements: `docker` (with `buildx`) and [`uv`](https://docs.astral.sh/uv/).
Expand All @@ -121,8 +120,8 @@ Requirements: `docker` (with `buildx`) and [`uv`](https://docs.astral.sh/uv/).

Maintainers: see [`RELEASE.md`](./RELEASE.md) for the end-to-end release
process — how `builds.json` works, the PR-driven release flow that fires
the publish workflow when a GitHub Release is published, the tag-
immutability guard, and how to verify a freshly published image.
the publish workflow when a GitHub Release is published, the published tag
scheme, and how to verify a freshly published image.

## License

Expand Down
Loading