CNTRLPLANE-3740: Add hypershift details for additional identity information sources - #2050
Conversation
openshift-ci-robot
added
the
jira/valid-reference
|
@liouk: This pull request references CNTRLPLANE-3740 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the sub-task to target the "5.0.0" version, but no target version was set. DetailsIn response to this: Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
liouk
changed the title
openshift-ci
Bot
added
the
do-not-merge/work-in-progress
liouk
changed the title
openshift-ci
Bot
removed
the
do-not-merge/work-in-progress
|
This PR currently contains a set of open questions; my goal is to resolve these before merging and adjust the PR accordingly. Putting a hold until these are resolved. /hold |
openshift-ci
Bot
added
the
do-not-merge/hold
everettraven
left a comment
everettraven
left a comment
There was a problem hiding this comment.
Overall, I think this looks pretty good. A handful of comments on the open questions.
| ##### Open Question: Compatibility across multiple Kubernetes API Servers | ||
|
|
||
| At the current state of HyperShift and the CPO v2, it seems like the CPO is baked into the payload, which means that the kube-apiserver version should be on par with CPO | ||
| within a payload. However, there has been discussion of evidence that this is not always true, and that we might still need to maintain compatibility with multiple | ||
| kube-apiserver versions. We need to clarify this before proceeding with decisions on how to manage the API types going forward. |
There was a problem hiding this comment.
I think some of the concern here may stem from the validation we added to prevent invalid KAS configurations where possible since HCP kind of eats KAS rollout failures IIRC.
More specifically: https://github.com/openshift/hypershift/blob/2d2b2d0805d36dcf401fdb5f3d913b9f7984ce42/support/validations/authentication.go#L51-L75
Maybe we ought to try to resolve that TODO in there to help have a more consistent validation pattern with the desired OCP installation version rather than always assuming the lowest possible version.
There was a problem hiding this comment.
Right, I see what you mean -- however I'm now thinking that maybe this is out of scope for this EP, as it's pre-existing in HyperShift and not specific to external claims sourcing. The existing approach would still work respectively with any validations we'd do for the new type anyway.
If you also agree, I'd drop this open question; we should definitely revisit that TODO at some point but that's probably orthogonal to the work we'll do in the context of this EP.
There was a problem hiding this comment.
I think the main thing I'd be concerned about here is whether or not we need to have version awareness in how we generate the configuration based on OCP version compatibility so that we never accidentally attempt to deploy an older oauth-apiserver with the newer configuration.
If we are confident that we do not need to be concerned with any potential version skew here, I think it would at least be worth explicitly documenting why we do not need to be concerned with it.
There was a problem hiding this comment.
@everettraven I've rewritten this question to describe better what my current understanding of versioning between CPO and control plane components is, but also our concerns both with version skew and with validations -- my aim is to get feedback from hypershift folks as well on this topic before we finalize it.
fabad31 to
e7dad96
Compare
everettraven
left a comment
everettraven
left a comment
There was a problem hiding this comment.
Aside from one comment, this LGTM
| ##### Open Question: Compatibility across multiple Kubernetes API Servers | ||
|
|
||
| At the current state of HyperShift and the CPO v2, it seems like the CPO is baked into the payload, which means that the kube-apiserver version should be on par with CPO | ||
| within a payload. However, there has been discussion of evidence that this is not always true, and that we might still need to maintain compatibility with multiple | ||
| kube-apiserver versions. We need to clarify this before proceeding with decisions on how to manage the API types going forward. |
There was a problem hiding this comment.
I think the main thing I'd be concerned about here is whether or not we need to have version awareness in how we generate the configuration based on OCP version compatibility so that we never accidentally attempt to deploy an older oauth-apiserver with the newer configuration.
If we are confident that we do not need to be concerned with any potential version skew here, I think it would at least be worth explicitly documenting why we do not need to be concerned with it.
e7dad96 to
fdb3169
Compare
|
@liouk: all tests passed! Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
| ##### Open Question: Version skew between the CPO and control plane components | ||
|
|
||
| Our current understanding is that the CPO is baked into the payload alongside the control plane components it configures (kube-apiserver, oauth-apiserver), meaning there is no version skew between them within a given payload. If this holds, the feature gate alone is sufficient to prevent the CPO from generating a configuration that the deployed component cannot understand, and no version-aware generation logic is needed. | ||
|
|
||
| However, it is not clear whether this is always the case. If a newer CPO could end up deploying an older oauth-apiserver that does not support the new configuration format, we would need version-aware generation logic to avoid deploying incompatible configurations. | ||
|
|
||
| Additionally, version skew affects admission-time validation: HyperShift currently compiles CEL expressions at admission time using a CEL environment pinned to the minimum supported OCP version, to avoid generating configurations that the target KAS cannot parse. If there is no version skew, validation could use the actual target version instead, reducing false rejections of valid CEL expressions. | ||
|
|
||
| We would like HyperShift maintainers to confirm whether version skew between the CPO and the components it deploys is a scenario we need to account for. |
There was a problem hiding this comment.
Maybe we can get @sjenning @enxebre or @csrwng to take a look to confirm our understanding here?
I suspect the HyperShift operator validation behavior is going to prove problematic here since there will be different validation based on the fact that the behavior will only be available starting with a specific openshift payload.
i.e We need to make sure we don't perform the new configuration generation and validation for a HostedCluster instance that wouldn't support it anyways.
4 participants
No description provided.