Skip to content

Remove kolla entrypoint pattern from Cyborg, harden SecurityContext - #1182

Open
stuggi wants to merge 1 commit into
openstack-k8s-operators:mainfrom
stuggi:remove-kolla-cyborg
Open

Remove kolla entrypoint pattern from Cyborg, harden SecurityContext#1182
stuggi wants to merge 1 commit into
openstack-k8s-operators:mainfrom
stuggi:remove-kolla-cyborg

Conversation

@stuggi

@stuggi stuggi commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Eliminate kolla_start and the config.json staging pattern for the CyborgAPI, CyborgConductor and DB sync workloads. Configs are now mounted directly to their final paths via SubPath volume mounts, removing the need for root privilege escalation at container startup.

Standalone branch mirroring the Cyborg portion of the combined kolla-removal-test branch (which also migrated Nova and Placement), so Cyborg can be pushed and tested independently -- see sibling remove-kolla-nova and remove-kolla-placement branches, based on the same commit. Nova and Placement are still kolla-based here, so internal/common.ServiceCommand is intentionally kept: it is still their only consumer on this branch.

Key changes:

  • CyborgAPI: replace "/bin/bash -c kolla_start" with "/usr/sbin/httpd -DFOREGROUND"
  • CyborgConductor: replace kolla_start with "cyborg-conductor --config-dir /etc/cyborg/cyborg.conf.d"
  • DB sync: replace kolla_start with "cyborg-dbsync --config-dir /etc/cyborg/cyborg.conf.d/ upgrade"
  • Remove KOLLA_CONFIG_STRATEGY/KOLLA_BOOTSTRAP env vars
  • Delete cyborg-api-config.json, cyborg-conductor-config.json, cyborg-dbsync-config.json (no longer needed)
  • Mount 00-default.conf/01-service-custom.conf (only added when CustomServiceConfig is set) directly under /etc/cyborg/cyborg.conf.d/, plus /etc/my.cnf, via new cyborg.GetConfVolumeMounts() shared by API/Conductor/DB sync
  • CyborgAPI: add run-httpd and var-log-httpd emptyDir volumes (kolla used to chown /etc/httpd/run and /var/log/httpd at startup)
  • Use pod.RestrictiveSecurityContext/RestrictivePodSecurityContext from lib-common for all three workloads, passing users.CyborgUID and users.CyborgGID from lib-common modules/serviceuser (replaces the local CyborgUserID constant) -- the lib-common revision this pulls in takes an explicit gid parameter instead of deriving it from uid
  • httpd.conf: User/Group apache -> cyborg, matching the pre-existing WSGIDaemonProcess user=cyborg group=cyborg evidence
  • Override TLS CertMount/KeyMount to /etc/pki/tls/{certs,private}/, matching what 10-cyborg-wsgi-main.conf's SSLCertificateFile/ SSLCertificateKeyFile actually render -- svc.CreateVolumeMounts() otherwise defaults to lib-common's staging path, which nothing copies from once kolla's config.json is gone
  • RBAC: SCC anyuid -> nonroot-v2 (marker + rbacRules)

Security hardening (standardized across remove-kolla effort):

  • DefaultMode 0440 for all config Secret volumes (configMode in volumes.go, dbsync config-data volume), replacing 0640/0644 -- group-write is unnecessary since configs are immutable at runtime
  • AutomountServiceAccountToken: ptr.To(false) on all three workloads (CyborgAPI, CyborgConductor, DB sync) -- no container needs the projected service-account token
  • Adopt lib-common volume.WritableDirVolume/WritableDirVolumeMount helpers for logs, run-httpd, var-log-httpd emptyDir volumes, replacing local GetLogVolume/GetLogVolumeMount/GetRunHttpdVolume/ GetVarLogHttpdVolume wrappers; remove now-redundant RunHttpdVolume and VarLogHttpdVolume constants (use volume.RunHttpdVolumeName and volume.VarLogHttpdVolumeName from lib-common instead)
  • Update functional and kuttl tests for the new mount paths and SCC

Jira: OSPRH-33504
Jira: OSPRH-33503

Depends-On: openstack-k8s-operators/lib-common#728

@openshift-ci

openshift-ci Bot commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: stuggi

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@stuggi

stuggi commented Aug 12, 2026

Copy link
Copy Markdown
Contributor Author

/retest

@stuggi
stuggi force-pushed the remove-kolla-cyborg branch from 2710a4d to 277ae2c Compare August 12, 2026 15:35
@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/69d4def800a947b688c3df81fd29542e

✔️ openstack-meta-content-provider SUCCESS in 3h 28m 52s
nova-operator-kuttl NODE_FAILURE Node(set) request 099-0000168338 failed in 0s
nova-operator-kuttl-placement NODE_FAILURE Node(set) request 099-0000168339 failed in 0s
nova-operator-tempest-multinode NODE_FAILURE Node(set) request 099-0000168340 failed in 0s
✔️ nova-operator-tempest-multinode-ceph SUCCESS in 3h 04m 15s

@stuggi

stuggi commented Aug 12, 2026

Copy link
Copy Markdown
Contributor Author

recheck

@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/f0ff61a291b54c7a89283339bc0b058e

✔️ openstack-meta-content-provider SUCCESS in 3h 11m 17s
nova-operator-kuttl RETRY_LIMIT in 25m 02s
✔️ nova-operator-kuttl-placement SUCCESS in 1h 00m 22s
✔️ nova-operator-tempest-multinode SUCCESS in 2h 16m 22s
✔️ nova-operator-tempest-multinode-ceph SUCCESS in 2h 54m 17s

@stuggi

stuggi commented Aug 13, 2026

Copy link
Copy Markdown
Contributor Author

recheck

@stuggi
stuggi force-pushed the remove-kolla-cyborg branch from 277ae2c to e53f219 Compare August 14, 2026 07:43
@amoralej

Copy link
Copy Markdown
Contributor

It'd be great if we can test this with #1143

@stuggi

stuggi commented Aug 14, 2026

Copy link
Copy Markdown
Contributor Author

It'd be great if we can test this with #1143

I can rebase as soon it landed, or I should rebase on top of yours?

@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/6ae8a19258c1400e9e7c4999a472670b

✔️ openstack-meta-content-provider SUCCESS in 4h 23m 45s
nova-operator-kuttl FAILURE in 1h 09m 44s
✔️ nova-operator-kuttl-placement SUCCESS in 1h 03m 59s
✔️ nova-operator-tempest-multinode SUCCESS in 2h 48m 13s
✔️ nova-operator-tempest-multinode-ceph SUCCESS in 2h 56m 02s

@stuggi

stuggi commented Aug 17, 2026

Copy link
Copy Markdown
Contributor Author

recheck

@stuggi

stuggi commented Aug 19, 2026

Copy link
Copy Markdown
Contributor Author

rebased after #1181 merged

@coderabbitai

coderabbitai Bot commented Aug 19, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Central YAML (base), Organization UI (inherited)

Review profile: CHILL

Plan: Pro Plus

Run ID: 9b5e1fa2-723d-4ee8-b5e4-7a77986970ad

📥 Commits

Reviewing files that changed from the base of the PR and between 9123624 and b8ac3a9.

📒 Files selected for processing (2)
  • config/rbac/role.yaml
  • internal/cyborg/api/statefulset.go
💤 Files with no reviewable changes (1)
  • config/rbac/role.yaml

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.


📝 Walkthrough

Summary by CodeRabbit

  • New Features
    • Cyborg services now use direct configuration mounts and streamlined startup commands.
    • Added support for optional custom service configuration.
  • Security
    • Updated Cyborg workloads and permissions to use restrictive non-root security settings.
    • Improved protection for database synchronization configuration.
  • Bug Fixes
    • Updated HTTPD runtime, logging, and TLS paths for more reliable service operation.
  • Tests
    • Updated functional and integration coverage for the new configuration and security behavior.

Walkthrough

Cyborg workloads now use shared read-only configuration mounts, restrictive security contexts, and direct process commands. Legacy Kolla configuration files, environment variables, and shell wrappers were removed. RBAC and tests now expect nonroot-v2.

Changes

Cyborg runtime hardening

Layer / File(s) Summary
Shared configuration and identity contracts
internal/cyborg/constants.go, internal/cyborg/volumes.go, templates/cyborg/api/config/httpd.conf
Adds shared Secret volume and SubPath mount helpers. Adds the custom service configuration filename. Runs HTTPD as cyborg.
API StatefulSet runtime
internal/cyborg/api/statefulset.go
Mounts configuration and TLS files at final paths. Adds writable runtime and log volumes. Uses restrictive security contexts and starts HTTPD directly.
Conductor and database-sync runtimes
internal/cyborg/conductor/statefulset.go, internal/cyborg/dbsync.go
Uses shared configuration mounts, restrictive Cyborg identities, and direct conductor and database-sync commands. Removes legacy Kolla configuration wiring.
Security policy and functional validation
internal/controller/cyborg/cyborg_controller.go, config/rbac/role.yaml, test/functional/cyborg/*, test/kuttl/test-suites/nova/cyborg-tests/01-assert.yaml
Changes SCC access from anyuid to nonroot-v2. Updates assertions for configuration files, mounts, environment variables, and SCC annotations.

Estimated code review effort: 4 (Complex) | ~45 minutes

Merge Risk: ⚪ Minimal · up to b8ac3

The PR moves Cyborg configuration to direct mounts and tightens workload security settings; no actionable merge-blocking risk remains after normal checks and review.

Sequence Diagram(s)

sequenceDiagram
  participant ConfigSecret
  participant CyborgWorkloads
  participant CyborgProcesses
  ConfigSecret->>CyborgWorkloads: Provide read-only final-path configuration mounts
  CyborgWorkloads->>CyborgProcesses: Start direct API, conductor, or dbsync commands
  CyborgProcesses->>CyborgWorkloads: Write permitted runtime and log files
Loading

Suggested reviewers: amoralej

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the removal of the Kolla entrypoint pattern and the related security hardening.
Description check ✅ Passed The description directly explains the configuration, command, security, volume, and test changes in the pull request.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@stuggi
stuggi force-pushed the remove-kolla-cyborg branch from 71e8bd2 to 9123624 Compare August 20, 2026 08:03
@stuggi

stuggi commented Aug 20, 2026

Copy link
Copy Markdown
Contributor Author

rebased after #1143 merged

@amartyasinha
amartyasinha requested a review from amoralej August 20, 2026 09:54
Comment thread internal/cyborg/api/statefulset.go Outdated
@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/fe51c7498999451386a0d5b3280c0adb

✔️ openstack-meta-content-provider SUCCESS in 3h 51m 37s
nova-operator-kuttl RETRY_LIMIT in 1h 06m 59s
✔️ nova-operator-kuttl-placement SUCCESS in 1h 08m 09s
✔️ nova-operator-tempest-multinode SUCCESS in 2h 56m 03s
nova-operator-tempest-multinode-ceph TIMED_OUT in 3h 30m 03s
✔️ nova-operator-cyborg-tempest-multinode SUCCESS in 1h 52m 58s (non-voting)

Eliminate kolla_start and the config.json staging pattern for the
CyborgAPI, CyborgConductor and DB sync workloads. Configs are now
mounted directly to their final paths via SubPath volume mounts,
removing the need for root privilege escalation at container startup.

Standalone branch mirroring the Cyborg portion of the combined
kolla-removal-test branch (which also migrated Nova and Placement), so
Cyborg can be pushed and tested independently -- see sibling
remove-kolla-nova and remove-kolla-placement branches, based on the
same commit. Nova and Placement are still kolla-based here, so
internal/common.ServiceCommand is intentionally kept: it is still
their only consumer on this branch.

Key changes:
- CyborgAPI: replace "/bin/bash -c kolla_start" with
  "/usr/sbin/httpd -DFOREGROUND"
- CyborgConductor: replace kolla_start with
  "cyborg-conductor --config-dir /etc/cyborg/cyborg.conf.d"
- DB sync: replace kolla_start with
  "cyborg-dbsync --config-dir /etc/cyborg/cyborg.conf.d/ upgrade"
- Remove KOLLA_CONFIG_STRATEGY/KOLLA_BOOTSTRAP env vars
- Delete cyborg-api-config.json, cyborg-conductor-config.json,
  cyborg-dbsync-config.json (no longer needed)
- Mount 00-default.conf/01-service-custom.conf (only added when
  CustomServiceConfig is set) directly under
  /etc/cyborg/cyborg.conf.d/, plus /etc/my.cnf, via new
  cyborg.GetConfVolumeMounts() shared by API/Conductor/DB sync
- CyborgAPI: add run-httpd and var-log-httpd emptyDir volumes
  (kolla used to chown /etc/httpd/run and /var/log/httpd at startup)
- Use pod.RestrictiveSecurityContext/RestrictivePodSecurityContext
  from lib-common for all three workloads, passing users.CyborgUID
  and users.CyborgGID from lib-common modules/users
  (replaces the local CyborgUserID constant) -- the lib-common revision
  this pulls in takes an explicit gid parameter instead of deriving it
  from uid
- httpd.conf: User/Group apache -> cyborg, matching the pre-existing
  WSGIDaemonProcess user=cyborg group=cyborg evidence
- Override TLS CertMount/KeyMount to /etc/pki/tls/{certs,private}/,
  matching what 10-cyborg-wsgi-main.conf's SSLCertificateFile/
  SSLCertificateKeyFile actually render -- svc.CreateVolumeMounts()
  otherwise defaults to lib-common's staging path, which nothing
  copies from once kolla's config.json is gone
- RBAC: SCC anyuid -> nonroot-v2 (marker + rbacRules)

Security hardening (standardized across remove-kolla effort):
- DefaultMode 0440 for all config Secret volumes (configMode in
  volumes.go, dbsync config-data volume), replacing 0640/0644 --
  group-write is unnecessary since configs are immutable at runtime
- AutomountServiceAccountToken: ptr.To(false) on all three workloads
  (CyborgAPI, CyborgConductor, DB sync) -- no container needs the
  projected service-account token
- Adopt lib-common volume.WritableDirVolume/WritableDirVolumeMount
  helpers for logs, run-httpd, var-log-httpd emptyDir volumes,
  replacing local GetLogVolume/GetLogVolumeMount/GetRunHttpdVolume/
  GetVarLogHttpdVolume wrappers; remove now-redundant RunHttpdVolume
  and VarLogHttpdVolume constants (use volume.RunHttpdVolumeName and
  volume.VarLogHttpdVolumeName from lib-common instead)
- Update functional and kuttl tests for the new mount paths and SCC

Jira: OSPRH-33504
Jira: OSPRH-33503

Depends-On: openstack-k8s-operators/lib-common#728

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Martin Schuppert <mschuppert@redhat.com>
@stuggi
stuggi force-pushed the remove-kolla-cyborg branch from 9123624 to b8ac3a9 Compare August 20, 2026 13:55
@stuggi

stuggi commented Aug 20, 2026

Copy link
Copy Markdown
Contributor Author

/test functional

@amartyasinha amartyasinha left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 from my side. But will wait for @amoralej or @SeanMooney from Cyborg team to review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants