From 8c464e39bec35e278849814154916cbc567f3df0 Mon Sep 17 00:00:00 2001 From: Quinn James Date: Thu, 2 Jul 2026 17:13:44 -0400 Subject: [PATCH 01/10] Fixes #615 - Add documentation for upgrade procedure --- AGENTS.md | 20 +++++++++ docs/developer/development-environment.md | 23 +++++++++++ docs/user/upgrade.md | 50 +++++++++++++++++++++++ 3 files changed, 93 insertions(+) create mode 100644 docs/user/upgrade.md diff --git a/AGENTS.md b/AGENTS.md index bbd3de02c..6ec561348 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -46,3 +46,23 @@ AI agent specifications (rules, skills, agent personas) live under `.agents/`. - [Check Roles](docs/developer/checks.md) — check role catalog and integration patterns - [Testing](docs/developer/testing.md) — test infrastructure, fixtures, patterns - [Parameters](docs/user/parameters.md) — installation parameter map; update when adding parameters +Developer docs: +- [Check roles](docs/developer/checks.md) - How to integrate check roles; update as checks are created/modified +- [Container Image Builds](docs/developer/container-image-builds.md) - Info on image naming, registries +- [Deployment Architecture](docs/developer/deployment.md) +- [Development Environment](docs/developer/development-environment.md) - Dev environment setup with Foreman from source +- [How to Add a Feature](docs/developer/how-to-add-a-feature.md) - End-to-end feature development +- [Playbooks and Roles](docs/developer/playbooks-and-roles.md) - Playbook structure, naming, metadata +- [Testing](docs/developer/testing.md) - Additional info on test infrastructure, fixtures, patterns + +User docs: +- [Backup](docs/user/backup.md) - How to back up your data +- [Certificates](docs/user/certificates.md) - Overview of certificate sources +- [Parameters](docs/user/parameters.md) - Map of Foreman installation parameters; update as parameters are created/modified +- [Upgrade](docs/user/upgrade.md) - How to upgrade your Foreman server through foremanctl + +- [CONTRIBUTING](CONTRIBUTING.md) - How to contribute +- [Development](DEVELOPMENT.md) - Foremanctl development overview +- [IOP](docs/iop.md) - Overview of insights on premise +- [Migration Guide](docs/migration-guide.md) - Migrating from foreman-installer to foremanctl +- [Release](RELEASE.md) - Info on Foremanctl releases diff --git a/docs/developer/development-environment.md b/docs/developer/development-environment.md index f6db26fff..8fae509f5 100644 --- a/docs/developer/development-environment.md +++ b/docs/developer/development-environment.md @@ -141,6 +141,29 @@ After deployment, the environment includes: - `smart-proxy` and its plugins - the development smart proxy registered into Foreman +### Upgrading from source + +In a source install, git branches control versioning instead of RPM packages. Stable branches (e.g. `2.y-stable`) track a specific Foreman Y-stream, while the `master` branch tracks nightly changes (most recent). + +Specific versions of foremanctl always map to the same version of Foreman, regardless of parameter configuration: + +- 1.2.z+ -> Foreman 3.17 +- 2.0.z+ -> Foreman 3.18 +- 2.2.z+ -> Foreman 3.19 + +All steps must be run as the root user. These steps assume that foremanctl is being ran from the same machine that the source lives on. + +1. Please take a moment to determine your preferred Foreman version / git branch, we only support sequential updates. +2. We recommend a full foremanctl backup before all upgrade operations. Run `foremanctl backup `. Please see [Backup](backup.md) for more information on this process. +3. Begin the upgrade by stopping all Foreman processes with `systemctl stop foreman.target`. +4. Switch to the preferred target branch: + - Switch to a stable Y-stream: `git fetch origin && git checkout origin/X.y-stable` + - Switch to nightly: `git fetch origin && git checkout origin/master` +5. Run upgrade tasks by re-deploying foremanctl with your customized deploy command: `foremanctl deploy [...]`. Please see [Parameters](parameters.md) for available deploy options. + +This final deploy command will pull new images and run all upgrade jobs required by Foreman, its dependencies, and your configured plugins. Expect this deploy to take longer than typical deploys. + + ## Architecture ### Service Integration diff --git a/docs/user/upgrade.md b/docs/user/upgrade.md new file mode 100644 index 000000000..e3123d0f6 --- /dev/null +++ b/docs/user/upgrade.md @@ -0,0 +1,50 @@ +# Upgrading foremanctl + +Foremanctl releases bundle specific versions of Foreman with version-compatible dependencies and plugins. + +For nearly all install situations, upgrading your Foreman server should be approached through upgrading foremanctl. Scroll to the correct procedure below which matches your installation environment type (RPM, disconnected RPM, source). + +## Upgrading foremanctl from RPM install + +All steps must be run as root user. We also recommend that a `foremanctl health` check is run before these steps. + +1. We recommend setting up a dnf versionlock for foremanctl if control over Foreman versioning is critical to your use case. If applicable, please take a moment to review your current versionlock configuration: + - `dnf versionlock list` - View the current versionlock configuration + - `dnf versionlock add 'foremanctl-X.*'` - Lock upgrades to a certain X-stream (replace with exact X version). + - `dnf versionlock add 'foremanctl-X.Y.*'` - Lock upgrades to a certain Y-stream (replace with exact X.Y version). + - `dnf versionlock delete foremanctl` - Remove any foremanctl versionlock. +2. We recommend a full foremanctl backup before all upgrade operations. Run `foremanctl backup `. Please see [Backup](backup.md) for more information on this process. +3. Run upgrade tasks by re-deploying foremanctl with the deploy command: `foremanctl deploy [...]`. Please see [Parameters](parameters.md) for additional deploy options. + +This final deploy command will pull new images and run all upgrade jobs required by Foreman, its dependencies, and your configured plugins. Expect this deploy to take longer than typical deploys. + +## Upgrading foremanctl from disconnected RPM install + +All below steps must be run as root user. + +1. Stage the foremanctl RPM package + - The foremanctl RPM must be available in a repository accessible to your disconnected Foreman server. Please transfer the RPM to your disconnected system via an available transport mechanism (USB drive, rsync over a bastion, etc.). + - The foremanctl RPM can be downloaded from TODO: TBD + - Once staged, `dnf info foremanctl` will resolve as in a connected environment. +2. Stage the required container images + - On a connected machine, pull all required images with `foremanctl pull-images`. + - Confirm the correct images were downloaded by running `podman images` on both the connected and disconnected machines. All images from your previous-version disconnected environment should be present on the connected environment. If images are missing, ensure parameters are identical between machines. + - On the connected environment, run `podman save $(podman images --format "{{.Repository}}:{{.Tag}}" | tr '\n' ' ') -o .tar` to export all downloaded images as a tarball. + - Transfer the tar file to your disconnected environment via an available transport mechanism. + - Run `podman load -i .tar` to stage the required images. +3. Complete the [Upgrading foremanctl from RPM install](#upgrading-foremanctl-from-rpm-install) section above to install from locally staged packages and images. + +## Recovering from a failed upgrade + +In the event of a failed upgrade, don't panic! A failed deploy will typically reveal the details of what went wrong and can give hints regarding the nature of your issue. Here are some troubleshooting steps: + +#### (RPM install) `dnf upgrade foremanctl` had "Nothing to do" +Run `dnf versionlock list` to see if your system is configured to allow X or Y version upgrades. Update the versionlock using the steps above. + +#### `foremanctl deploy` could not pull images from remote +Ensure that https://quay.io is unblocked on your network. You can manually open https://quay.io/foreman/foreman in a browser to view available images. + +#### I'd like to roll back to a known working version (RPM install) +`dnf downgrade foremanctl-X.Y.Z` will roll back your foremanctl install to the requested version. Please follow instructions in our [Restore Guide](restore.md) to restore your system to a working state. + +TODO: this section could present the information better imho. Can you think of a way to put the roll back instructions into the main document above. From cae3f9b2fa716f08995c443ebacce48fa9b42d9f Mon Sep 17 00:00:00 2001 From: Aiden Fine Date: Wed, 22 Jul 2026 22:16:32 -0400 Subject: [PATCH 02/10] add foremanctl health recommendation --- docs/user/upgrade.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user/upgrade.md b/docs/user/upgrade.md index e3123d0f6..5fbbd1096 100644 --- a/docs/user/upgrade.md +++ b/docs/user/upgrade.md @@ -20,7 +20,7 @@ This final deploy command will pull new images and run all upgrade jobs required ## Upgrading foremanctl from disconnected RPM install -All below steps must be run as root user. +All below steps must be run as root user. We also recommend that a `foremanctl health` check is run before these steps. 1. Stage the foremanctl RPM package - The foremanctl RPM must be available in a repository accessible to your disconnected Foreman server. Please transfer the RPM to your disconnected system via an available transport mechanism (USB drive, rsync over a bastion, etc.). From 93ffb110115d2654b837849d1505f63f84681cdd Mon Sep 17 00:00:00 2001 From: Aiden Fine Date: Wed, 22 Jul 2026 22:30:30 -0400 Subject: [PATCH 03/10] Note that the Foreman repository is needed for foremanctl RPM deps --- docs/user/upgrade.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/user/upgrade.md b/docs/user/upgrade.md index 5fbbd1096..a0068f97f 100644 --- a/docs/user/upgrade.md +++ b/docs/user/upgrade.md @@ -23,6 +23,7 @@ This final deploy command will pull new images and run all upgrade jobs required All below steps must be run as root user. We also recommend that a `foremanctl health` check is run before these steps. 1. Stage the foremanctl RPM package + - The Foreman repository is needed for dependencies related to the foremanctl RPM. - The foremanctl RPM must be available in a repository accessible to your disconnected Foreman server. Please transfer the RPM to your disconnected system via an available transport mechanism (USB drive, rsync over a bastion, etc.). - The foremanctl RPM can be downloaded from TODO: TBD - Once staged, `dnf info foremanctl` will resolve as in a connected environment. From bdd2ec7b92a8a68d2329c9347be870e85d892b44 Mon Sep 17 00:00:00 2001 From: Aiden Fine Date: Thu, 23 Jul 2026 12:05:23 -0400 Subject: [PATCH 04/10] Update source to "Foreman" --- docs/developer/development-environment.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/developer/development-environment.md b/docs/developer/development-environment.md index 8fae509f5..c8b083abe 100644 --- a/docs/developer/development-environment.md +++ b/docs/developer/development-environment.md @@ -151,7 +151,7 @@ Specific versions of foremanctl always map to the same version of Foreman, regar - 2.0.z+ -> Foreman 3.18 - 2.2.z+ -> Foreman 3.19 -All steps must be run as the root user. These steps assume that foremanctl is being ran from the same machine that the source lives on. +All steps must be run as the root user. These steps assume that foremanctl is being ran from the same machine that the Foreman was deployed on. 1. Please take a moment to determine your preferred Foreman version / git branch, we only support sequential updates. 2. We recommend a full foremanctl backup before all upgrade operations. Run `foremanctl backup `. Please see [Backup](backup.md) for more information on this process. From 0c2ad4f54ea8a6a4b91464c9d204860f9c4b9b8c Mon Sep 17 00:00:00 2001 From: Aiden Fine Date: Thu, 23 Jul 2026 12:06:08 -0400 Subject: [PATCH 05/10] typo fix --- docs/developer/development-environment.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/developer/development-environment.md b/docs/developer/development-environment.md index c8b083abe..5ec3b2106 100644 --- a/docs/developer/development-environment.md +++ b/docs/developer/development-environment.md @@ -151,7 +151,7 @@ Specific versions of foremanctl always map to the same version of Foreman, regar - 2.0.z+ -> Foreman 3.18 - 2.2.z+ -> Foreman 3.19 -All steps must be run as the root user. These steps assume that foremanctl is being ran from the same machine that the Foreman was deployed on. +All steps must be run as the root user. These steps assume that foremanctl is being ran from the same machine that Foreman was deployed on. 1. Please take a moment to determine your preferred Foreman version / git branch, we only support sequential updates. 2. We recommend a full foremanctl backup before all upgrade operations. Run `foremanctl backup `. Please see [Backup](backup.md) for more information on this process. From c96ba8cb22338bea49cb5481da87392d819399e4 Mon Sep 17 00:00:00 2001 From: Aiden Fine Date: Mon, 27 Jul 2026 17:45:44 -0400 Subject: [PATCH 06/10] Add repo update steps and clean up docs --- docs/user/upgrade.md | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/docs/user/upgrade.md b/docs/user/upgrade.md index a0068f97f..0fa7e4438 100644 --- a/docs/user/upgrade.md +++ b/docs/user/upgrade.md @@ -13,8 +13,12 @@ All steps must be run as root user. We also recommend that a `foremanctl health` - `dnf versionlock add 'foremanctl-X.*'` - Lock upgrades to a certain X-stream (replace with exact X version). - `dnf versionlock add 'foremanctl-X.Y.*'` - Lock upgrades to a certain Y-stream (replace with exact X.Y version). - `dnf versionlock delete foremanctl` - Remove any foremanctl versionlock. -2. We recommend a full foremanctl backup before all upgrade operations. Run `foremanctl backup `. Please see [Backup](backup.md) for more information on this process. -3. Run upgrade tasks by re-deploying foremanctl with the deploy command: `foremanctl deploy [...]`. Please see [Parameters](parameters.md) for additional deploy options. +2. Update the Foreman repository to the target version: + - `dnf install https://yum.theforeman.org/releases//el9/x86_64/foreman-release.rpm` +3. Upgrade the foremanctl package: + - `dnf upgrade foremanctl` +4. We recommend a full foremanctl backup before running an upgrade. Run `foremanctl backup `. Please see [Backup](backup.md) for more information on this process. +5. Run upgrade tasks by re-deploying: `foremanctl deploy`. Please see [Parameters](parameters.md) for additional deploy options. This final deploy command will pull new images and run all upgrade jobs required by Foreman, its dependencies, and your configured plugins. Expect this deploy to take longer than typical deploys. @@ -25,7 +29,7 @@ All below steps must be run as root user. We also recommend that a `foremanctl h 1. Stage the foremanctl RPM package - The Foreman repository is needed for dependencies related to the foremanctl RPM. - The foremanctl RPM must be available in a repository accessible to your disconnected Foreman server. Please transfer the RPM to your disconnected system via an available transport mechanism (USB drive, rsync over a bastion, etc.). - - The foremanctl RPM can be downloaded from TODO: TBD + - The foremanctl RPM can be downloaded from `https://yum.theforeman.org` - Once staged, `dnf info foremanctl` will resolve as in a connected environment. 2. Stage the required container images - On a connected machine, pull all required images with `foremanctl pull-images`. @@ -33,7 +37,7 @@ All below steps must be run as root user. We also recommend that a `foremanctl h - On the connected environment, run `podman save $(podman images --format "{{.Repository}}:{{.Tag}}" | tr '\n' ' ') -o .tar` to export all downloaded images as a tarball. - Transfer the tar file to your disconnected environment via an available transport mechanism. - Run `podman load -i .tar` to stage the required images. -3. Complete the [Upgrading foremanctl from RPM install](#upgrading-foremanctl-from-rpm-install) section above to install from locally staged packages and images. +3. Complete the [Upgrading foremanctl from RPM install](#upgrading-foremanctl-from-rpm-install) section above starting from step 3 to install from locally staged packages and images. ## Recovering from a failed upgrade From f1ee6dc518c4a069b099c9472b83703fdd352e27 Mon Sep 17 00:00:00 2001 From: Aiden Fine Date: Mon, 27 Jul 2026 17:50:10 -0400 Subject: [PATCH 07/10] remove lock steps --- docs/user/upgrade.md | 13 ++++--------- 1 file changed, 4 insertions(+), 9 deletions(-) diff --git a/docs/user/upgrade.md b/docs/user/upgrade.md index 0fa7e4438..d75b84ba7 100644 --- a/docs/user/upgrade.md +++ b/docs/user/upgrade.md @@ -8,17 +8,12 @@ For nearly all install situations, upgrading your Foreman server should be appro All steps must be run as root user. We also recommend that a `foremanctl health` check is run before these steps. -1. We recommend setting up a dnf versionlock for foremanctl if control over Foreman versioning is critical to your use case. If applicable, please take a moment to review your current versionlock configuration: - - `dnf versionlock list` - View the current versionlock configuration - - `dnf versionlock add 'foremanctl-X.*'` - Lock upgrades to a certain X-stream (replace with exact X version). - - `dnf versionlock add 'foremanctl-X.Y.*'` - Lock upgrades to a certain Y-stream (replace with exact X.Y version). - - `dnf versionlock delete foremanctl` - Remove any foremanctl versionlock. -2. Update the Foreman repository to the target version: +1. Update the Foreman repository to the target version: - `dnf install https://yum.theforeman.org/releases//el9/x86_64/foreman-release.rpm` -3. Upgrade the foremanctl package: +2. Upgrade the foremanctl package: - `dnf upgrade foremanctl` -4. We recommend a full foremanctl backup before running an upgrade. Run `foremanctl backup `. Please see [Backup](backup.md) for more information on this process. -5. Run upgrade tasks by re-deploying: `foremanctl deploy`. Please see [Parameters](parameters.md) for additional deploy options. +3. We recommend a full foremanctl backup before running an upgrade. Run `foremanctl backup `. Please see [Backup](backup.md) for more information on this process. +4. Run upgrade tasks by re-deploying: `foremanctl deploy`. Please see [Parameters](parameters.md) for additional deploy options. This final deploy command will pull new images and run all upgrade jobs required by Foreman, its dependencies, and your configured plugins. Expect this deploy to take longer than typical deploys. From f1157f0f681d609b645beff83bb18c31e37e40ec Mon Sep 17 00:00:00 2001 From: Aiden Fine Date: Tue, 28 Jul 2026 12:13:03 -0400 Subject: [PATCH 08/10] Address PR comments --- docs/developer/development-environment.md | 6 ------ docs/user/upgrade.md | 12 +++--------- 2 files changed, 3 insertions(+), 15 deletions(-) diff --git a/docs/developer/development-environment.md b/docs/developer/development-environment.md index 5ec3b2106..eed522f26 100644 --- a/docs/developer/development-environment.md +++ b/docs/developer/development-environment.md @@ -145,12 +145,6 @@ After deployment, the environment includes: In a source install, git branches control versioning instead of RPM packages. Stable branches (e.g. `2.y-stable`) track a specific Foreman Y-stream, while the `master` branch tracks nightly changes (most recent). -Specific versions of foremanctl always map to the same version of Foreman, regardless of parameter configuration: - -- 1.2.z+ -> Foreman 3.17 -- 2.0.z+ -> Foreman 3.18 -- 2.2.z+ -> Foreman 3.19 - All steps must be run as the root user. These steps assume that foremanctl is being ran from the same machine that Foreman was deployed on. 1. Please take a moment to determine your preferred Foreman version / git branch, we only support sequential updates. diff --git a/docs/user/upgrade.md b/docs/user/upgrade.md index d75b84ba7..3c833ef89 100644 --- a/docs/user/upgrade.md +++ b/docs/user/upgrade.md @@ -8,12 +8,11 @@ For nearly all install situations, upgrading your Foreman server should be appro All steps must be run as root user. We also recommend that a `foremanctl health` check is run before these steps. -1. Update the Foreman repository to the target version: - - `dnf install https://yum.theforeman.org/releases//el9/x86_64/foreman-release.rpm` +1. Update the Foreman repository to the target version for the X or Y stream upgrades: + - `dnf upgrade https://yum.theforeman.org/releases//el9/x86_64/foreman-release.rpm` 2. Upgrade the foremanctl package: - `dnf upgrade foremanctl` -3. We recommend a full foremanctl backup before running an upgrade. Run `foremanctl backup `. Please see [Backup](backup.md) for more information on this process. -4. Run upgrade tasks by re-deploying: `foremanctl deploy`. Please see [Parameters](parameters.md) for additional deploy options. +3. Run upgrade tasks by re-deploying: `foremanctl deploy`. Please see [Parameters](parameters.md) for additional deploy options. This final deploy command will pull new images and run all upgrade jobs required by Foreman, its dependencies, and your configured plugins. Expect this deploy to take longer than typical deploys. @@ -43,8 +42,3 @@ Run `dnf versionlock list` to see if your system is configured to allow X or Y v #### `foremanctl deploy` could not pull images from remote Ensure that https://quay.io is unblocked on your network. You can manually open https://quay.io/foreman/foreman in a browser to view available images. - -#### I'd like to roll back to a known working version (RPM install) -`dnf downgrade foremanctl-X.Y.Z` will roll back your foremanctl install to the requested version. Please follow instructions in our [Restore Guide](restore.md) to restore your system to a working state. - -TODO: this section could present the information better imho. Can you think of a way to put the roll back instructions into the main document above. From 510e93e840de473d61b430c5eee5e185c47495bb Mon Sep 17 00:00:00 2001 From: Aiden Fine Date: Tue, 28 Jul 2026 20:22:37 -0400 Subject: [PATCH 09/10] Add local repo mirror steps --- docs/user/upgrade.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/user/upgrade.md b/docs/user/upgrade.md index 3c833ef89..79ff53a4e 100644 --- a/docs/user/upgrade.md +++ b/docs/user/upgrade.md @@ -20,6 +20,8 @@ This final deploy command will pull new images and run all upgrade jobs required All below steps must be run as root user. We also recommend that a `foremanctl health` check is run before these steps. +Disconnected users should also create a local repository mirror of foreman, an example of this from Red Hat is linked here https://access.redhat.com/solutions/7019225. + 1. Stage the foremanctl RPM package - The Foreman repository is needed for dependencies related to the foremanctl RPM. - The foremanctl RPM must be available in a repository accessible to your disconnected Foreman server. Please transfer the RPM to your disconnected system via an available transport mechanism (USB drive, rsync over a bastion, etc.). From fa970e24dcc8da62d7bd120a5c9ab8a37cfe9afb Mon Sep 17 00:00:00 2001 From: Aiden Fine Date: Fri, 31 Jul 2026 10:45:33 -0400 Subject: [PATCH 10/10] replace target naming with next --- docs/user/upgrade.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/user/upgrade.md b/docs/user/upgrade.md index 79ff53a4e..eaf4f038c 100644 --- a/docs/user/upgrade.md +++ b/docs/user/upgrade.md @@ -8,8 +8,8 @@ For nearly all install situations, upgrading your Foreman server should be appro All steps must be run as root user. We also recommend that a `foremanctl health` check is run before these steps. -1. Update the Foreman repository to the target version for the X or Y stream upgrades: - - `dnf upgrade https://yum.theforeman.org/releases//el9/x86_64/foreman-release.rpm` +1. Update the Foreman repository to the next X or Y release: + - `dnf upgrade https://yum.theforeman.org/releases//el9/x86_64/foreman-release.rpm` 2. Upgrade the foremanctl package: - `dnf upgrade foremanctl` 3. Run upgrade tasks by re-deploying: `foremanctl deploy`. Please see [Parameters](parameters.md) for additional deploy options.