Skip to content

Store RHEL8 image layer cache in GHCR instead of the Actions cache - #14358

Merged
magnesj merged 1 commit into
OPM:devfrom
magnesj:rhel8-image-registry-cache
Jul 27, 2026
Merged

Store RHEL8 image layer cache in GHCR instead of the Actions cache#14358
magnesj merged 1 commit into
OPM:devfrom
magnesj:rhel8-image-registry-cache

Conversation

@magnesj

@magnesj magnesj commented Jul 27, 2026

Copy link
Copy Markdown
Member

The nightly "Build RHEL8 CI Image" workflow has failed every night since 2026-07-23 (on both OPM/ResInsight and forks) with:

#23 exporting to GitHub Actions Cache
#23 ERROR: not_found
ERROR: failed to build: failed to solve: not_found

The image itself builds and pushes to GHCR successfully — only the final cache-to: type=gha,mode=max export fails, marking the run red.

Root cause: the repository's Actions cache is at the 10 GB per-repo limit (10.4 GB on OPM/ResInsight), shared with all other workflows. The multi-GB BuildKit export is evicted as fast as it is written, which both breaks the export commit (not_found) and makes restores useless — the build log shows a 0% cache hit rate, so every nightly run did the full cold compile.

Changes:

  • Registry layer cache instead of the Actions cache. The BuildKit cache now lives in GHCR (ci-rhel8:buildcache), which has no size cap. The existing ${GITHUB_REPOSITORY,,} image-name pattern makes this work unchanged on upstream and forks. The buildcache tag is excluded from the daily image-version cleanup, alongside latest.
  • The vcpkg stage is keyed on its actual inputs. It previously copied the entire build context before compiling the dependencies, so any source commit invalidated the layer and every nightly build recompiled all vcpkg ports (~1 h). It now copies only vcpkg.json, the vcpkg configuration and the vcpkg tool, and runs a direct manifest install.
  • .git is excluded from the Docker build context. Its internals differ between checkouts of the same commit, which silently invalidated COPY . /src on every run. The in-image build does not need it (the CMake submodule init is guarded by EXISTS .git).

Verified on magnesj/ResInsight:

  • Cold run (2h24m): builds 101 vcpkg packages, exports the cache to GHCR without errors; the warmup configure logs Restored 101 package(s) from /opt/vcpkg-cache, confirming the ABI hashes from the direct manifest install match the CMake-driven configure.
  • Run after a source-only change (1h20m): the entire vcpkg-builder stage is CACHED from the registry cache; only the build-warmup stage recompiles. This is the nightly scenario — the ~1 h vcpkg portion is eliminated and the cache export no longer fails.

The nightly image build has failed since 2026-07-23 with "failed to
solve: not_found" while exporting the BuildKit layer cache, because the
repository's GitHub Actions cache is at the 10 GB limit and entries are
evicted as fast as they are written (0% cache hit rate on restore).

Use a registry cache (ci-rhel8:buildcache in GHCR) instead, which has no
size cap, and exclude the buildcache tag from the daily image cleanup.

Make the cache effective across nightly rebuilds: key the vcpkg stage on
its actual inputs (vcpkg.json, the vcpkg configuration and the vcpkg
tool) with a direct manifest install instead of copying the entire build
context, and exclude .git from the Docker build context since its
internals differ between checkouts of the same commit and invalidated
'COPY . /src' on every run. Nightly builds now reuse the compiled vcpkg
ports from the layer cache and only recompile the ResInsight warmup
stage.
@magnesj
magnesj force-pushed the rhel8-image-registry-cache branch from 82a96db to 044029a Compare July 27, 2026 13:38
@magnesj
magnesj merged commit 4b4bf29 into OPM:dev Jul 27, 2026
8 checks passed
@magnesj
magnesj deleted the rhel8-image-registry-cache branch July 27, 2026 13:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant