Fix: 🐛 Use upstream ghcr.io/spiffe/spiffe-helper image for operator sidecar - #501
Merged
Conversation
…ecar The operator's own spiffe-helper sidecar (injected into rossoctl-controller-manager when spiffe.operatorAuth.enabled=true) pointed at ghcr.io/rossoctl/cortex/spiffe-helper:latest, which was never published (403 Forbidden on pull) -- the operator pod gets stuck at 1/2 Ready and never obtains a JWT-SVID, so operator SPIFFE auth never actually authenticates at runtime even though the bootstrap Job successfully registers the Keycloak client. The container's CLI args (-config /etc/spiffe-helper/config.hcl) match the real upstream SPIFFE project tool exactly, and an earlier branch already used ghcr.io/spiffe/spiffe-helper:0.11.0 before this regressed to the broken rossoctl/cortex path. Point it back at the real upstream image. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Alan Cha <Alan.cha1@ibm.com>
evaline-ju
reviewed
Aug 3, 2026
evaline-ju
left a comment
evaline-ju
left a comment
Contributor
There was a problem hiding this comment.
q - should we avoid "latest" tags for images, especially if they're upstream/out of our control?
Member
Author
|
@evaline-ju Yes, you are correct, but I noticed that most of the images in the operator are using the latest tag like operator/charts/operator/values.yaml Line 249 in 47e7e30 However, as you mentioned, this is an external dependency so I think you're right to point out that we should tag this one. |
Unlike authbridge/envoyProxy/authbridgeLite/proxyInit (rossoctl-built images that default to :latest at the chart level and get pinned by the downstream rossoctl/rossoctl consumer chart), spiffe-helper is a genuine third-party upstream dependency with no downstream pin step for it. Tag it directly to the current latest stable release (ghcr.io/spiffe/spiffe-helper:0.11.0, confirmed to exist via docker manifest inspect) rather than tracking :latest. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Alan Cha <Alan.cha1@ibm.com>
Member
Author
|
The plan is to remove this dependency soon, but as of now, it's pointing at the wrong image |
Alan-Cha
changed the title
cwiklik
pushed a commit
that referenced
this pull request
Aug 4, 2026
…idecar (#501) * fix: use upstream ghcr.io/spiffe/spiffe-helper image for operator sidecar The operator's own spiffe-helper sidecar (injected into rossoctl-controller-manager when spiffe.operatorAuth.enabled=true) pointed at ghcr.io/rossoctl/cortex/spiffe-helper:latest, which was never published (403 Forbidden on pull) -- the operator pod gets stuck at 1/2 Ready and never obtains a JWT-SVID, so operator SPIFFE auth never actually authenticates at runtime even though the bootstrap Job successfully registers the Keycloak client. The container's CLI args (-config /etc/spiffe-helper/config.hcl) match the real upstream SPIFFE project tool exactly, and an earlier branch already used ghcr.io/spiffe/spiffe-helper:0.11.0 before this regressed to the broken rossoctl/cortex path. Point it back at the real upstream image. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Alan Cha <Alan.cha1@ibm.com> * fix: pin spiffe-helper to v0.11.0 instead of :latest Unlike authbridge/envoyProxy/authbridgeLite/proxyInit (rossoctl-built images that default to :latest at the chart level and get pinned by the downstream rossoctl/rossoctl consumer chart), spiffe-helper is a genuine third-party upstream dependency with no downstream pin step for it. Tag it directly to the current latest stable release (ghcr.io/spiffe/spiffe-helper:0.11.0, confirmed to exist via docker manifest inspect) rather than tracking :latest. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Alan Cha <Alan.cha1@ibm.com> --------- Signed-off-by: Alan Cha <Alan.cha1@ibm.com> (cherry picked from commit 5f4a8e3)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The operator's own
spiffe-helpersidecar (injected intorossoctl-controller-managerwhenspiffe.operatorAuth.enabled=true) points atghcr.io/rossoctl/cortex/spiffe-helper:latest, which was never published (403 Forbiddenon pull). The operator pod gets stuck at1/2 Readyand never obtains a JWT-SVID, so operator SPIFFE auth never actually authenticates at runtime — even though the bootstrap Job (separately, in rossoctl/rossoctl) successfully registers the Keycloak client.The container's CLI args (
-config /etc/spiffe-helper/config.hcl) match the real upstream SPIFFE project tool exactly, and an earlier branch already usedghcr.io/spiffe/spiffe-helper:0.11.0before this regressed to the brokenrossoctl/cortexpath.Key changes:
charts/operator/templates/manager/manager.yaml:196— swapghcr.io/rossoctl/cortex/spiffe-helper:latest→ghcr.io/spiffe/spiffe-helper:0.11.0.Pinned to a specific version rather than
:latest: unlikeauthbridge/envoyProxy/authbridgeLite/proxyInit(rossoctl-built images that default to:latestat this chart level and get pinned downstream by therossoctl/rossoctlconsumer chart),spiffe-helperis a genuine third-party upstream dependency with no downstream pin step — so it's tagged directly here to the current latest stable release (0.11.0, confirmed viadocker manifest inspect).Verified via
helm templatethat the renderedspiffe-helpercontainer now uses the correct pinned image, and confirmed on a live Kind cluster (loading a substitute image manually) that the operator comes up2/2 Readyand logs"SPIFFE ID authentication enabled: using JWT-SVID for client registration"once this image is resolvable.Related issue(s)
Fixes rossoctl/rossoctl#2334