AIL-418: feat(iso): switch osbuilder → AuroraBoot v0.26.2 for +build-iso and +build-uki-iso - #738
Merged
Conversation
kairos-io/osbuilder and kairos-io/enki are both archived, so the
osbuilder-tools image we've built ISOs against for years has no
maintained upstream. Switch to quay.io/kairos/auroraboot:v0.26.1
(kairos-io/AuroraBoot), the maintained successor.
Only +build-iso and +build-uki-iso move. +uki-genkey and
+build-provider-trustedboot-image still call the osbuilder entrypoint;
they touch container/UKI artifacts, not ISOs, and can migrate
separately once the ISO switch is boot-verified.
Mapping (osbuilder -> AuroraBoot):
/entrypoint.sh build-iso -> auroraboot build-iso
--name -> --override-name
--arch x86_64 -> --arch amd64 (Go arch names)
enki build-uki -t iso -> auroraboot build-uki -t iso
-k /keys (bundled) -> --public-keys / --sb-key /
--sb-cert / --tpm-pcr-private-key
(all still under /keys because the
three secure-boot/* source dirs are
flattened into the same COPY target)
Bonuses from the AuroraBoot base image (Fedora 44):
- xorriso 1.5.8.pl02 (osbuilder-tools ships 1.4.6, which the
Supermicro ATEN BMC virtual CD refuses to boot). This is the
xorriso fix vipsharm-isofix built from source; no longer needed.
- Actively maintained; kairos-sdk fixes land here.
Caveat: AuroraBoot still selects grubx64.efi.signed (installed-system
variant, prefix /EFI/ubuntu) via kairos-sdk utils.GetEfiGrubFiles.
That binary hangs immediately on Supermicro UEFI. This switch does
not fix defect 1 from the UEFI investigation on its own. Verifying
that on real hardware is the next step; if it reproduces, the
CD-variant swap workaround from vipsharm-isofix 22e00cb / 0a80d30
gets ported.
The grub.cfg cleanup (previous commit, ex-3faff86) is independent
of the toolchain switch and rides along.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
AuroraBoot v0.26.1's --override-name is not honored for "dir:" sources
(only image sources). A build-iso against dir:/build/image landed as
kairos-ubuntu-24.04-core-amd64-generic-v4.0.4.iso in /iso/, and the
subsequent
RUN sha256sum \$ISO_NAME.iso > \$ISO_NAME.iso.sha256
failed with "No such file or directory".
Rename /iso/*.iso to \$ISO_NAME.iso before checksumming, matching what
+build-uki-iso has done all along. This decouples us from whatever
naming scheme AuroraBoot uses internally.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
AuroraBoot v0.26.1's --output flag on build-iso is ignored for "dir:" sources: the log line "Generating iso '...' to '/tmp/auroraboot'" confirmed the ISO drops under /tmp/auroraboot, not /iso/, so `mv /iso/*.iso ...` failed with "cannot stat '/iso/*.iso'". Search /iso, /tmp/auroraboot, and (fall back) the whole filesystem for a .iso and move it into /iso/\$ISO_NAME.iso. Applied to both +build-iso and +build-uki-iso so we're covered if build-uki -d has the same bug. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…rename Both --output on build-iso and -d on build-uki are silently ignored by AuroraBoot v0.26.1 for "dir:" sources -- the ISO always drops at /tmp/auroraboot/*.iso. Two builds in a row confirmed the location, so drop the fallback find-tree and just mv from /tmp/auroraboot directly. Also drops --override-name (also inert for dir: sources on build-iso) so the invocation reads honestly. No behavior change -- the previous fallback path already worked. This is just less noisy shell. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Ubuntu's grub-efi-*-signed package contains two signed GRUB builds: grubx64.efi.signed prefix /EFI/ubuntu - for systems installed to disk gcdx64.efi.signed prefix /boot/grub - for CD / removable media osbuilder's copyGrub walks a hardcoded candidate list (kairos-sdk utils.GetEfiGrubFiles), takes the first path that exists, and names the destination after the source. On Ubuntu that always resolves to grubx64.efi.signed - there is no gcd candidate in the list at all - so our ISOs ship the installed-system binary. That binary hangs immediately when launched from removable media on UEFI firmware. Reproduced on Supermicro with an ATEN BMC: the firmware mounts the ESP correctly and reads both binaries, then executes nothing. Launching either EFI\BOOT\bootx64.efi or EFI\BOOT\grubx64.efi by hand from the UEFI Shell hangs with no output, with Secure Boot disabled. Only Dual/CSM boot ever worked, because that path uses the BIOS El Torito image and never touches this binary. It is not a version problem: ubuntu-24.04.3-live-server ships the identical GRUB package version (2.12-1ubuntu7.3) and boots on the same hardware. Substituting its gcd build into our ESP makes GRUB run on every boot. Changes: - Earthfile (+iso-image): stage the CD variant over the path osbuilder reads. Applied to the ISO rootfs only - cloud/raw disk images are installed systems and must keep the original binary. Handles x86_64 and arm64, and is a no-op on bases that do not ship the gcd build. - overlay/files-iso/boot/grub/grub.cfg: new stub. The CD-variant GRUB resolves its /boot/grub prefix against the ISO9660 volume, so it needs a config there; without it GRUB starts and drops to a rescue prompt. Hands off to the existing /boot/grub2/grub.cfg. This is a workaround, not a fix: kairos-io/osbuilder and kairos-io/enki are both archived, so the candidate list cannot be corrected upstream. The durable option is migrating ISO builds to AuroraBoot, which is maintained but inherits the same defect via kairos-sdk. Verified by hand-patching an ISO (CD-variant GRUB + this stub + the grub.cfg fixes in the previous commit): boots unattended in pure UEFI, menu renders on the EFI console, kernel boots with visible output. NOT yet verified through a CanvOS pipeline build.
+kairos-raw-image invokes +iso-image with no arguments (Earthfile:1259), so IS_CLOUD_IMAGE stays at its default of false. The 22e00cb guard IS_CLOUD_IMAGE=false therefore fires on MAAS builds too, replacing grubx64.efi.signed with the CD variant. AuroraBoot then writes that into the raw disk image's ESP, and a MAAS-deployed machine boots into the CD-variant GRUB, which looks for /boot/grub/grub.cfg on the installed disk and drops to a rescue prompt when it isn't there. Widen the guard to also skip when IS_MAAS=true. IS_MAAS is a top-level ARG (Earthfile:124) that already propagates into +iso-image on MAAS builds, so no call-site change is needed. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…t overlay files AuroraBoot v0.26.1's build-iso has two ISO-side defects that block boot on strict UEFI firmwares: 1. Produces a plain-MBR ISO with El Torito + a 0xEF appended-partition ESP entry. Works on lenient firmwares, but VMware ESXi/Workstation, OVMF/QEMU, and some SuperMicro BMC virtual CD implementations reject it as "No compatible bootloader found" because they only look for the ESP via a GPT partition entry. Ubuntu 24.04.3's live ISO uses hybrid GPT + protective MBR for exactly this reason and boots everywhere. 2. --overlay-iso is silently a no-op for "dir:" sources -- files placed under /overlay do NOT make it onto the ISO tree. Diagnosed by comparing overlay/files-iso/boot/grub2/grub.cfg (Palette-branded) with the same path on the built ISO (Kairos-branded default), and by finding /boot/grub/grub.cfg entirely missing. That in turn drops the CD-variant GRUB stub (from 22e00cb), the customized menu, user-data, content bundles, cluster config, and edge_custom_config. Fix: xorriso repack after the auroraboot run. -boot_image any appended_part_as=gpt Converts the appended ESP into a GPT partition entry and emits a protective MBR. UEFI firmware can now discover it. -add $OVERLAY_SPECS -- Enumerates every file under /overlay and inject-adds each to the ISO tree at its expected location. Also functions as the "pending change" that lets xorriso commit (without a tree change it would refuse with "No image modifications pending"). -overwrite on Some overlay files (like /boot/grub2/grub.cfg) already exist on the ISO tree with AuroraBoot's default content; -overwrite on lets -add replace them cleanly. Verified on the jump host: repacked ISO boots to GRUB menu in QEMU + OVMF and in VMware VM (Ubuntu 24.04.3 also broken on that VM until GPT was added). Only touches +build-iso. +build-uki-iso uses auroraboot build-uki -t iso which is a different upstream code path and may or may not have the same defects -- to be evaluated when UKI ISOs are next boot-tested. Also filed for upstream: kairos-io/AuroraBoot issue on --overlay-iso being ignored, and kairos-io/kairos-sdk issue on GetEfiGrubFiles never picking the gcd/CD-variant signed binary. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…hem upstream kairos-io/AuroraBoot v0.26.2 (PR #713) fixes the two defects that were blocking UEFI-only ISO boot, exactly as our RCA identified: * appended_part_as=gpt is now passed to xorriso, so ISOs get a GPT header alongside the protective MBR (fixes VMware ESXi/Workstation EFI, OVMF/QEMU, some SuperMicro BMC virtual-CD firmwares that previously reported "No compatible bootloader found"). * Live ISOs now prefer gcdx64.efi.signed / gcdaa64.efi.signed (CD-variant grub with prefix /boot/grub and iso9660 baked in) over grubx64.efi.signed (installed-system grub, prefix /EFI/ubuntu with no iso9660). cleanupGrubName renames gcd*.efi to grub*.efi on copy so shim chainloading still works under Secure Boot. * A /boot/grub/grub.cfg stub is written by AuroraBoot itself, both in the EFI image (USB path) and on the ISO9660 root (CD path). * Bumps kairos-sdk to v0.25.2 which carries the sdk-side changes. With upstream fixed we can drop every local workaround: Earthfile (+iso-image): remove the CD-variant GRUB swap block (was: cp gcdx64.efi.signed over grubx64.efi.signed, guarded by IS_CLOUD_IMAGE=false && IS_MAAS=false). AuroraBoot now picks the right binary itself for live media, and doesn't touch the installed-system variant that raw-disk/MAAS builds depend on -- the MAAS guard becomes moot along with the swap. Earthfile (+build-iso): remove the xorriso post-process step that added GPT via appended_part_as=gpt and injected every overlay/ file with -add. GPT is now emitted natively; --overlay-iso keeps the same behaviour it had (documented separately; not blocking us because AuroraBoot's own /boot/grub/grub.cfg + /boot/grub2/ grub.cfg cover the boot chain). overlay/files-iso/boot/grub/grub.cfg: delete. Was our stub telling the CD-variant GRUB where to find its real config; AuroraBoot now writes an equivalent file itself. AURORABOOT_VERSION v0.26.1 -> v0.26.2. Verified path is now: FROM auroraboot:v0.26.2 -> build-iso dir:... -> mv /tmp/auroraboot/*.iso /iso/\$ISO_NAME.iso -> sha256sum. No post-processing. Not yet verified end-to-end (rebuild + boot in QEMU/OVMF and VMware UEFI expected) -- filing this as the trust-upstream state and will test right after. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…iso subcommand skips StepInjectCC
Empirically: AuroraBoot v0.26.2 subcommand \`build-iso dir:...\`
with --overlay-iso produces an ISO whose /boot/grub2/grub.cfg is
AuroraBoot's default "Kairos"-branded template rather than our
overlay's "Palette eXtended Kubernetes Edge Installer" version.
Confirmed by extracting the file from the built ISO.
Root cause (pkg/ops/iso.go, internal/cmd/build-iso.go): the build-iso
subcommand runs only
PrepDirs -> StepCopyCloudConfig -> StepDumpSource -> StepGenISO
and never calls StepInjectCC. StepInjectCC is where AuroraBoot's
InjectISO() function xorriso-maps the overlay directory onto the
finalised ISO tree. Without it, --overlay-iso silently discards
every file we put under /overlay/ -- not just the grub menu, but
also user-data, cluster config (spc.tgz), content bundles, and
edge_custom_config.
Two ways to fix:
1. Switch +build-iso to AuroraBoot pipeline mode (docker run
auroraboot --set container_image=... --set iso.overlay_iso=...
--cloud-config ...). This invokes StepInjectCC. But it's a
~100-line rewrite involving DinD, container_image loading, and
matches the pattern +cloud-image / +kairos-raw-image already
use.
2. Do StepInjectCC's actual work ourselves inline. Its whole body
is one xorriso command:
xorriso -indev X -outdev X -map <overlay> / \
-boot_image any replay
No DinD, no rewrite. Same result.
Going with (2). Small, isolated, uses the same xorriso invocation
the upstream code path does, and can be trivially deleted the day
StepInjectCC is wired into the subcommand path.
Doesn't touch +build-uki-iso; that uses build-uki -t iso which is a
separate code path.
Filed separately as an upstream issue on kairos-io/AuroraBoot
(subcommand build-iso should call StepInjectCC too, or --overlay-iso
should be honored end-to-end within StepGenISO).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
vipsharm
requested review from
abhinavnagaraj,
nianyush and
rishi-anand
and removed request for
nianyush
August 6, 2026 22:40
rishi-anand
approved these changes
Aug 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Migrates CanvOS ISO builds off the archived
kairos-io/osbuilder(quay.io/kairos/osbuilder-tools) onto the maintainedkairos-io/AuroraBootv0.26.2. Fixes UEFI-only boot on strict firmware (SuperMicro ATEN BMC, VMware ESXi/Workstation, OVMF/QEMU) that previously refused CanvOS ISOs with "No compatible bootloader found" or a silent hang.Ticket: AIL-418
Why now
kairos-io/osbuilderandkairos-io/enkiare both archived. Our ISO toolchain had no upstream to accept fixes.+cloud-imageand+kairos-raw-image; the ISO path was the last piece still on osbuilder.What ends up in the tree
4e59375overlay/files-iso/boot/grub2/grub.cfgcleanup — three grub.cfg defects fixed (vga=795 gated to BIOS, gfxtermloadfontgated to BIOS,searchcorrected to/boot/kernel)978a633+build-isoand+build-uki-isofrom$OSBUILDER_IMAGEto$AURORABOOT_IMAGE(v0.26.1 initially)a179757/c8ed02b/b25262e--override-name/--outputare inert fordir:sources, so wemv /tmp/auroraboot/*.iso /iso/$ISO_NAME.isoourselves629a565AURORABOOT_VERSIONto v0.26.2 and drop every UEFI-only-boot workaround (CD-variant GRUB swap in+iso-image, MAAS guard, xorriso GPT +-addpost-process,overlay/files-iso/boot/grub/grub.cfgstub) — upstream v0.26.2 fixes both root causes and writes/boot/grub/grub.cfgitself9de6067+build-iso— required because AuroraBoot'sbuild-isosubcommand never callsStepInjectCC, so--overlay-isosilently drops overlay files. Filed as kairos-io/kairos#4283 and PR kairos-io/AuroraBoot#716Net Earthfile diff vs
vipsharm-GPU:+build-isoand+build-uki-isoswitched to AuroraBoot; a small xorriso post-process step in+build-isomimics AuroraBoot's ownInjectISOcode until the upstream PR merges.Local workarounds still shipping
Two remain, both cheap and both filed upstream:
xorriso overlay-injection post-process in
+build-iso(9de6067). Runsafter the AuroraBoot build. Same command AuroraBoot itself uses in
InjectISO. Upstream fix in fix(build-iso): apply --overlay-iso via StepInjectCC + run xorriso when only overlay is present kairos-io/AuroraBoot#716 (one-line addition ofd.StepInjectCCto the subcommand step list) — merges will let us drop this.mv /tmp/auroraboot/*.iso /iso/$ISO_NAME.iso(a179757+b25262e). AuroraBoot's--outputand--override-nameare inert fordir:sources. Cosmetic. Tracked in AuroraBoot:build-iso --outputand--override-namesilently ignored fordir:sources kairos-io/kairos#4284.Upstream issues + PR filed
build-isosubcommand silently drops--overlay-isocontent (StepInjectCC not called) kairos-io/kairos#4283 —build-isosubcommand silently drops--overlay-isocontent (StepInjectCC not called)build-iso --outputand--override-namesilently ignored fordir:sources kairos-io/kairos#4284 —build-iso --outputand--override-nameinert fordir:sourcesKairos AuroraBoot v0.26.2 fixed the UEFI boot part after our RCA landed with the maintainers.
Verification
fdisk -l+xorriso -report_el_torito): matches (GPT hybrid + El Torito UEFI, CD-variant grub binary) ✓/EFI/BOOT/grubx64.efiis 2,328,456 bytes (CD variantgcdx64.efi.signed), was 2,660,232 bytes (installed-system variant) pre-fix ✓/boot/grub2/grub.cfgis the Palette-branded version from our overlay, verified via extract ✓Merge conflict
There's a conflict against current
vipsharm-GPUtip (6244171— AIL-376 AMD GPU driver default bump) inoverlay/files-iso/boot/grub2/grub.cfg:pci=realloc=offfrom three menuentry kernel cmdlines.$consoleparams/$videoparamson${grub_platform}and gatedloadfontto BIOS.Both changes are legitimate and non-overlapping in intent. Resolution: keep the EFI/BIOS branching structure and drop
pci=realloc=offfrom all three entries. Happy to push the resolution commit when this PR opens.Related
AIL-AuroraBootISOtest on appliance-builder: branchvipsharm-test-ail-auroraboot-isowithCANVOS_BRANCH="AIL-AuroraBootISO"— do NOT merge; revert toCANVOS_BRANCH="vipsharm-GPU"once this PR lands.🤖 Co-authored-by: Claude Opus 4.7 noreply@anthropic.com