Remove kolla entrypoint pattern, harden SecurityContext - #747
Conversation
|
/test keystone-operator-build-deploy-kuttl |
|
This change depends on a change that failed to merge. Change openstack-k8s-operators/lib-common#728 is needed. |
bf39cea to
06b5d4c
Compare
|
/test functional |
486b47c to
7e41b98
Compare
|
Build failed (check pipeline). Post ✔️ keystone-openstack-meta-content-provider-master SUCCESS in 1h 36m 07s (non-voting) |
|
This change depends on a change that failed to merge. Changes openstack-k8s-operators/lib-common#732, openstack-k8s-operators/s2i-openstack-containers#46 are needed. |
|
recheck |
33e5731 to
53ca904
Compare
|
/test functional |
|
Merge Failed. This change or one of its cross-repo dependencies was unable to be automatically merged with the current state of its repository. Please rebase the change and upload a new patchset. |
|
recheck |
Eliminate kolla_start/kolla_set_configs and the config.json staging
pattern. Configs are now mounted directly to final paths via SubPath
volume mounts, removing the need for root privilege escalation at
container startup.
Key changes:
- Replace kolla_start with /usr/sbin/httpd -DFOREGROUND
- Remove kolla_set_configs from bootstrap command
- Remove all KOLLA_CONFIG_STRATEGY and KOLLA_BOOTSTRAP env vars
- Delete keystone-api-config.json (no longer needed)
- Mount configs via SubPath to /etc/keystone/, /etc/httpd/, /etc/my.cnf
- Add emptyDir volumes for writable paths (/run/httpd, /tmp, /var/log)
using lib-common volume.WritableDirVolume/Mount helpers with
standardized constants (volume.RunHttpdVolumeName, TmpVolumeName, etc.)
- Use pod.RestrictiveSecurityContext and RestrictivePodSecurityContext
from lib-common for all workloads (deployment, bootstrap, dbsync,
cronjob), setting ReadOnlyRootFilesystem, dropping ALL capabilities,
and enabling seccomp RuntimeDefault
- Set AutomountServiceAccountToken=false on all workloads — none need
k8s API access (ServiceAccountName is kept for SCC binding)
- Import KeystoneUID from lib-common modules/serviceuser
- Override TLS CertMount/KeyMount to /etc/pki/tls/ for Apache
- Migrate SCC from anyuid to nonroot-v2
- Update httpd.conf: User/Group keystone, PidFile /run/httpd/httpd.pid
- Support custom httpd config keys via dynamic SubPath mounts
- Config Secret DefaultMode set to 0440 (owner-read + group-read,
most restrictive — all SubPath mounts are ReadOnly: true anyway)
- Update functional and kuttl tests
Testing on a cluster surfaced a permission denial starting httpd:
httpd: Syntax error on line 16 of /etc/httpd/conf/httpd.conf:
Could not open configuration file /etc/httpd/conf.d/auth_openidc.conf:
Permission denied
mod_auth_openidc's auth_openidc.conf ships root:apache 0640 and is
baked into the container image rather than volume-mounted, so FSGroup
does not apply to it. Under the previous kolla/anyuid setup this was
never an issue: the httpd master process ran as root and could read
any file regardless of group. TCIB never adds the keystone service
user to the apache group (uid_gid_manage.sh only ever adds it to
"kolla"), so running the whole pod as non-root keystone requires the
apache group to be granted explicitly at the pod level via
RestrictivePodSecurityContext's new supplementalGroups parameter and
lib-common's serviceuser.ApacheGID constant.
Testing further surfaced httpd failing to start with:
(13)Permission denied: AH00091: httpd: could not open error log
file /etc/httpd/logs/error_log.
AH00015: Unable to open logs
/etc/httpd/logs is a symlink to /var/log/httpd (root:root 0700 in the
image). httpd.conf only redirected ErrorLog/CustomLog to /dev/stdout
inside <VirtualHost> blocks, so the global/startup error log fell back
to Apache's compiled-in default of "logs/error_log" relative to
ServerRoot. Fixed by adding a top-level ErrorLog directive, plus a
var-log-httpd emptyDir mounted at /var/log/httpd as defense-in-depth
for any other RPM-shipped conf.d file that references relative
"logs/*" paths.
ErrorLog (both the top-level and per-vhost directives) is sent to
/dev/stderr rather than /dev/stdout, following the convention used by
the official Apache httpd container image and OpenShift Logging's
per-stream labeling, keeping error diagnostics distinguishable from
routine access logs (CustomLog stays on /dev/stdout).
Testing also surfaced a keystone WARNING:
keystone.common.fernet_utils [...] key_repository is world readable:
/etc/keystone/fernet-keys
The fernet-keys and credential-keys Secret volumes had no DefaultMode
set, so Kubernetes defaulted to 0644 (world-readable), unlike
config-data which was already hardened to 0440. Fixed by setting
DefaultMode 0440 on both, matching config-data and silencing the
warning.
Jira: OSPRH-33504
Jira: OSPRH-33503
Depends-On: openstack-k8s-operators/lib-common#728
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Martin Schuppert <mschuppert@redhat.com>
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: abays, Deydra71, stuggi The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
3fb4181
into
openstack-k8s-operators:main
3 participants
Eliminate kolla_start/kolla_set_configs and the config.json staging pattern. Configs are now mounted directly to final paths via SubPath volume mounts, removing the need for root privilege escalation at container startup.
Key changes:
Testing on a cluster surfaced a permission denial starting httpd:
httpd: Syntax error on line 16 of /etc/httpd/conf/httpd.conf:
Could not open configuration file /etc/httpd/conf.d/auth_openidc.conf:
Permission denied
mod_auth_openidc's auth_openidc.conf ships root:apache 0640 and is baked into the container image rather than volume-mounted, so FSGroup does not apply to it. Under the previous kolla/anyuid setup this was never an issue: the httpd master process ran as root and could read any file regardless of group. TCIB never adds the keystone service user to the apache group (uid_gid_manage.sh only ever adds it to "kolla"), so running the whole pod as non-root keystone requires the apache group to be granted explicitly at the pod level via RestrictivePodSecurityContext's new supplementalGroups parameter and lib-common's serviceuser.ApacheGID constant.
Testing further surfaced httpd failing to start with:
(13)Permission denied: AH00091: httpd: could not open error log
file /etc/httpd/logs/error_log.
AH00015: Unable to open logs
/etc/httpd/logs is a symlink to /var/log/httpd (root:root 0700 in the image). httpd.conf only redirected ErrorLog/CustomLog to /dev/stdout inside blocks, so the global/startup error log fell back to Apache's compiled-in default of "logs/error_log" relative to ServerRoot. Fixed by adding a top-level ErrorLog directive, plus a var-log-httpd emptyDir mounted at /var/log/httpd as defense-in-depth for any other RPM-shipped conf.d file that references relative "logs/*" paths.
ErrorLog (both the top-level and per-vhost directives) is sent to /dev/stderr rather than /dev/stdout, following the convention used by the official Apache httpd container image and OpenShift Logging's per-stream labeling, keeping error diagnostics distinguishable from routine access logs (CustomLog stays on /dev/stdout).
Testing also surfaced a keystone WARNING:
keystone.common.fernet_utils [...] key_repository is world readable:
/etc/keystone/fernet-keys
The fernet-keys and credential-keys Secret volumes had no DefaultMode set, so Kubernetes defaulted to 0644 (world-readable), unlike config-data which was already hardened to 0440. Fixed by setting DefaultMode 0440 on both, matching config-data and silencing the warning.
Jira: OSPRH-33504
Jira: OSPRH-33503
Depends-On: openstack-k8s-operators/lib-common#728