-
Notifications
You must be signed in to change notification settings - Fork 27
Embed a machine-readable firmware version record in the DXE FV #94
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
John Starks (jstarks)
wants to merge
3
commits into
microsoft:main
Choose a base branch
from
jstarks:version_blob
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
135 changes: 135 additions & 0 deletions
135
MsvmPkg/BuildPlugins/FirmwareVersionBlob/FirmwareVersionBlob.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,135 @@ | ||
| ## @file FirmwareVersionBlob.py | ||
| # Pre-build plugin that owns the firmware version data. | ||
| # | ||
| # It reads the single source of truth (MsvmPkg/FirmwareVersion.toml) plus the | ||
| # git state, then does two things from those same values so they can never | ||
| # drift: | ||
| # | ||
| # 1. Generates the machine-readable firmware version record | ||
| # (MSVM_FIRMWARE_VERSION_INFO) embedded as a RAW file in the DXE FV, which | ||
| # the host/VMM scans out of the loaded image ('MVFW' signature / file GUID). | ||
| # See MsvmPkg/Include/MsvmFirmwareVersion.h and the SECTION RAW reference in | ||
| # MsvmPkg/MsvmPkgX64.fdf / MsvmPkgAARCH64.fdf. | ||
| # | ||
| # 2. Injects the same values as build macros ("BLD_*_MSVM_FW_*") which the DSC | ||
| # binds to FixedAtBuild PCDs, so the firmware itself can read the version at | ||
| # runtime (e.g. to log it early). See the [PcdsFixedAtBuild] overrides in | ||
| # the platform DSC files. | ||
| # | ||
| # Runs before the platform build (do_pre_build), after stuart has populated the | ||
| # build environment, so BUILD_OUTPUT_BASE is guaranteed to be set and the macros | ||
| # are set in time for the build command. | ||
| # | ||
| ## | ||
| # Copyright (c) Microsoft Corporation. | ||
| # SPDX-License-Identifier: BSD-2-Clause-Patent | ||
| ## | ||
| import logging | ||
| import os | ||
| import struct | ||
| import tomllib | ||
|
|
||
| from edk2toolext.environment import repo_resolver | ||
| from edk2toolext.environment.plugintypes.uefi_build_plugin import IUefiBuildPlugin | ||
|
|
||
|
|
||
| class FirmwareVersionBlob(IUefiBuildPlugin): | ||
| # | ||
| # Layout of MSVM_FIRMWARE_VERSION_INFO (see | ||
| # MsvmPkg/Include/MsvmFirmwareVersion.h). All integers are little-endian; | ||
| # string fields are NUL-terminated ASCII. | ||
| # UINT32 Signature ('MVFW'), UINT16 StructVersion, UINT16 HeaderSize, | ||
| # UINT32 Flags, UINT16 InterfaceVersionMajor, UINT16 InterfaceVersionMinor, | ||
| # CHAR8 BaseVersion[16], CHAR8 GitCommit[48] | ||
| # | ||
| FW_VERSION_SIGNATURE = 0x5746564D # 'MVFW' little-endian | ||
| FW_VERSION_STRUCT_VERSION = 1 | ||
| FW_VERSION_FLAG_DIRTY = 0x1 # MSVM_FIRMWARE_VERSION_FLAG_DIRTY | ||
| FW_VERSION_FLAG_OFFICIAL = 0x2 # MSVM_FIRMWARE_VERSION_FLAG_OFFICIAL | ||
| FW_VERSION_BASE_VERSION_SIZE = 16 | ||
| FW_VERSION_GIT_COMMIT_SIZE = 48 | ||
| FW_VERSION_STRUCT_FORMAT = "<IHHIHH16s48s" | ||
| # Path of the generated blob, relative to the per-build output directory | ||
| # (BUILD_OUTPUT_BASE). The FDF references the same location via | ||
| # $(OUTPUT_DIRECTORY)/$(TARGET)_$(TOOL_CHAIN_TAG). | ||
| FW_VERSION_BLOB_SUBPATH = os.path.join("FwVersion", "FwVersionBlob.bin") | ||
| # Single source of truth for the version numbers, relative to the workspace. | ||
| FW_VERSION_TOML_SUBPATH = os.path.join("MsvmPkg", "FirmwareVersion.toml") | ||
|
|
||
| def _ReadVersionToml(self, workspace, env): | ||
| toml_path = os.path.join(workspace, self.FW_VERSION_TOML_SUBPATH) | ||
| with open(toml_path, "rb") as f: | ||
| data = tomllib.load(f) | ||
| major = int(data["interface"]["major"]) | ||
| minor = int(data["interface"]["minor"]) | ||
| # The CI BASE_VERSION env override wins so the release workflow can own | ||
| # the released value; the TOML provides the in-tree default. | ||
| release = env.GetValue("BASE_VERSION", "") or str(data["release"]["version"]) | ||
| return major, minor, release | ||
|
|
||
| def _GetGitCommit(self, workspace): | ||
| # Use edk2toolext's standard repo state rather than a bespoke check, so | ||
| # "dirty" means the same thing here as everywhere else in the build | ||
| # tooling (repo_resolver reports dirty for modified tracked files or | ||
| # untracked files; gitignored build outputs do not count). | ||
| try: | ||
| details = repo_resolver.repo_details(workspace) | ||
| return details["Head"]["HexSha"], bool(details["Dirty"]) | ||
| except Exception as e: | ||
| logging.warning(f"Could not determine git state for firmware version blob: {e}") | ||
| return "unknown", False | ||
|
|
||
| def do_pre_build(self, thebuilder): | ||
| workspace = thebuilder.GetWorkspaceRoot() | ||
|
|
||
| major, minor, base_version = self._ReadVersionToml(workspace, thebuilder.env) | ||
| commit, dirty = self._GetGitCommit(workspace) | ||
| flags = 0 | ||
| if dirty: | ||
| flags |= self.FW_VERSION_FLAG_DIRTY | ||
| # Official builds are marked by the CI pipeline (e.g. building main, not | ||
| # a PR) via the OFFICIAL_BUILD environment variable. Any non-empty value | ||
| # other than "0"/"false" counts as official. | ||
| if thebuilder.env.GetValue("OFFICIAL_BUILD", "").strip().lower() not in ("", "0", "false"): | ||
| flags |= self.FW_VERSION_FLAG_OFFICIAL | ||
|
|
||
| # (1) Expose the values to the firmware as FixedAtBuild PCDs. Setting | ||
| # BLD_*_<NAME> makes stuart pass "-D <NAME>=<value>" to the build, which | ||
| # the DSC binds to the PcdMsvmFirmware* PCDs. | ||
| thebuilder.env.SetValue("BLD_*_MSVM_FW_INTERFACE_MAJOR", str(major), "FirmwareVersion.toml", True) | ||
| thebuilder.env.SetValue("BLD_*_MSVM_FW_INTERFACE_MINOR", str(minor), "FirmwareVersion.toml", True) | ||
| thebuilder.env.SetValue("BLD_*_MSVM_FW_BASE_VERSION", base_version, "FirmwareVersion.toml", True) | ||
| thebuilder.env.SetValue("BLD_*_MSVM_FW_GIT_COMMIT", commit, "git", True) | ||
| thebuilder.env.SetValue("BLD_*_MSVM_FW_FLAGS", str(flags), "git", True) | ||
|
|
||
| # (2) Pack the same values into the host-facing FV record. | ||
| blob = struct.pack( | ||
| self.FW_VERSION_STRUCT_FORMAT, | ||
| self.FW_VERSION_SIGNATURE, | ||
| self.FW_VERSION_STRUCT_VERSION, | ||
| struct.calcsize(self.FW_VERSION_STRUCT_FORMAT), | ||
| flags, | ||
| major, | ||
| minor, | ||
| base_version.encode("ascii", "replace"), | ||
| commit.encode("ascii", "replace"), | ||
| ) | ||
|
|
||
| # Emit into the per-build output directory so the artifact stays out of | ||
| # the source tree. stuart computes BUILD_OUTPUT_BASE as | ||
| # <OUTPUT_DIRECTORY>/<TARGET>_<TOOL_CHAIN_TAG>, which matches the FDF | ||
| # SECTION RAW path. | ||
| out_dir = thebuilder.env.GetValue("BUILD_OUTPUT_BASE") | ||
| out_path = os.path.join(out_dir, self.FW_VERSION_BLOB_SUBPATH) | ||
| os.makedirs(os.path.dirname(out_path), exist_ok=True) | ||
| with open(out_path, "wb") as f: | ||
| f.write(blob) | ||
|
|
||
| logging.info( | ||
| f"Firmware version blob written: base={base_version} commit={commit}" | ||
| f"{' (dirty)' if dirty else ''}" | ||
| f"{' (official)' if flags & self.FW_VERSION_FLAG_OFFICIAL else ''}" | ||
| f" interface={major}.{minor}" | ||
| f" -> {out_path}" | ||
| ) | ||
| return 0 |
5 changes: 5 additions & 0 deletions
5
MsvmPkg/BuildPlugins/FirmwareVersionBlob/FirmwareVersionBlob_plug_in.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| { | ||
| "scope": "global", | ||
| "name": "Firmware Version Blob Uefi Build Plugin", | ||
| "module": "FirmwareVersionBlob" | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| # Single source of truth for the firmware version numbers. | ||
| # | ||
| # Bump these by hand in code review. The build (see | ||
| # MsvmPkg/BuildPlugins/FirmwareVersionBlob) embeds them into the DXE FV version | ||
| # record (MSVM_FIRMWARE_VERSION_INFO) *and* into the FixedAtBuild PCDs that the | ||
| # firmware reads at runtime, so there is exactly one place to edit. | ||
|
|
||
| # Firmware<->VMM interface (compatibility) contract version. Independent of the | ||
| # release below and of git state. It is a machine-comparable gate the VMM uses | ||
| # to decide whether it can talk to this firmware. | ||
| # - Bump major for a breaking change (an older VMM would malfunction). Reset | ||
| # minor to 0 on a major bump. | ||
| # - Bump minor for a backward-compatible addition the VMM can degrade without. | ||
| [interface] | ||
| major = 1 | ||
| minor = 0 | ||
|
|
||
| # Release version prefix ("major.minor"). The GitHub release workflow may | ||
| # override this via the BASE_VERSION environment variable and assigns the patch | ||
| # number later, so only the prefix lives here. | ||
| [release] | ||
| version = "26.0" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| ## @file | ||
| # Firmware version record PCD value bindings, shared by MsvmPkgX64.dsc and | ||
| # MsvmPkgAARCH64.dsc so the list lives in one place. | ||
| # | ||
| # Binds the firmware version PCDs (declared in MsvmPkg/MsvmPkg.dec) to the | ||
| # build-time MSVM_FW_* macros injected by the FirmwareVersionBlob pre-build | ||
| # plugin, which sources them from MsvmPkg/FirmwareVersion.toml and git state. | ||
| # The macros are supplied by the plugin as "-D MSVM_FW_*=..."; nothing here | ||
| # needs a default because the plugin always runs before the build. | ||
| # | ||
| # Copyright (c) Microsoft Corporation. | ||
| # SPDX-License-Identifier: BSD-2-Clause-Patent | ||
| ## | ||
|
|
||
| [PcdsFixedAtBuild.common] | ||
| gMsvmPkgTokenSpaceGuid.PcdMsvmFirmwareInterfaceVersionMajor|$(MSVM_FW_INTERFACE_MAJOR) | ||
| gMsvmPkgTokenSpaceGuid.PcdMsvmFirmwareInterfaceVersionMinor|$(MSVM_FW_INTERFACE_MINOR) | ||
| gMsvmPkgTokenSpaceGuid.PcdMsvmFirmwareBaseVersion|"$(MSVM_FW_BASE_VERSION)" | ||
| gMsvmPkgTokenSpaceGuid.PcdMsvmFirmwareGitCommit|"$(MSVM_FW_GIT_COMMIT)" | ||
| gMsvmPkgTokenSpaceGuid.PcdMsvmFirmwareFlags|$(MSVM_FW_FLAGS) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,135 @@ | ||
| /** @file | ||
| Defines the firmware version record that is embedded as a RAW file in the | ||
| firmware volume so the host/VMM can identify the firmware image mechanically | ||
| (i.e. by parsing the image bytes) at load time, without executing it. | ||
|
|
||
| The record is generated at build time by MsvmPkg/PlatformBuild.py and placed | ||
| into the DXE firmware volume (see MsvmPkg/MsvmPkgX64.fdf / | ||
| MsvmPkg/MsvmPkgAARCH64.fdf) as a leaf file named by | ||
| gMsvmFirmwareVersionFileGuid. | ||
|
|
||
| Host-side location strategy: | ||
| - Scan the loaded image for the 4-byte Signature ('MVFW'), OR | ||
| - Scan for the 16-byte file GUID (gMsvmFirmwareVersionFileGuid) that appears | ||
| in the FFS file header, then read the record that follows. | ||
|
|
||
| All multi-byte integer fields are little-endian. String fields are ASCII and | ||
| NUL-terminated. The layout is append-only: new fields may be added at the end | ||
| guarded by StructVersion / HeaderSize, but existing fields and the Signature | ||
| must never change meaning so the host parser stays stable. | ||
|
|
||
| Copyright (c) Microsoft Corporation. | ||
| SPDX-License-Identifier: BSD-2-Clause-Patent | ||
|
|
||
| **/ | ||
|
|
||
| #pragma once | ||
|
|
||
| // | ||
| // Signature 'MVFW' stored little-endian ('M','V','F','W'). | ||
| // | ||
| #define MSVM_FIRMWARE_VERSION_SIGNATURE SIGNATURE_32 ('M', 'V', 'F', 'W') | ||
|
|
||
| // | ||
| // Current StructVersion value. Bump when the layout is extended. | ||
| // | ||
| #define MSVM_FIRMWARE_VERSION_STRUCT_VERSION 1 | ||
|
|
||
| // | ||
| // Firmware/VMM interface (compatibility) version. This is a human-curated | ||
| // semantic contract version that is deliberately independent of the release | ||
| // (BaseVersion) and of git state. | ||
| // | ||
| // The value is NOT defined here. It lives in MsvmPkg/FirmwareVersion.toml (the | ||
| // single source of truth) and is embedded into this record and into the | ||
| // PcdMsvmFirmwareInterfaceVersionMajor/Minor FixedAtBuild PCDs at build time. | ||
| // Bump it there, by hand, in code review, whenever the contract between this | ||
| // firmware and the host/VMM changes: | ||
| // | ||
| // - Bump MAJOR for any breaking change: one where an existing VMM built | ||
| // against an older major would malfunction. This includes the firmware | ||
| // hard-requiring new VMM-provided behavior it cannot boot without. Reset | ||
| // MINOR to 0 on a major bump. | ||
| // - Bump MINOR for a backward-compatible addition: an optional capability the | ||
| // firmware can degrade gracefully without. | ||
| // | ||
| // The VMM supports a range of majors and refuses anything outside it (firmware | ||
| // too new or too old to talk to). A higher-than-known minor is always safe to | ||
| // proceed on; the VMM may additionally require a minimum minor for features it | ||
| // depends on. | ||
| // | ||
|
|
||
| // | ||
| // Bit definitions for the Flags field. | ||
| // | ||
| // MSVM_FIRMWARE_VERSION_FLAG_DIRTY: the build was produced from a tree with | ||
| // uncommitted changes (modified tracked files or untracked files present), so | ||
| // GitCommit identifies the base commit but not the exact source that was built. | ||
| // When clear, GitCommit identifies the exact committed source state. | ||
| // | ||
| // MSVM_FIRMWARE_VERSION_FLAG_OFFICIAL: the build was produced by the official | ||
| // CI pipeline (e.g. building main), as opposed to a developer or pull-request | ||
| // build. Set from the OFFICIAL_BUILD build environment; clear otherwise. | ||
| // | ||
| #define MSVM_FIRMWARE_VERSION_FLAG_DIRTY BIT0 | ||
| #define MSVM_FIRMWARE_VERSION_FLAG_OFFICIAL BIT1 | ||
|
|
||
| // | ||
| // Maximum sizes (including the terminating NUL) for the string fields. | ||
| // | ||
| #define MSVM_FIRMWARE_BASE_VERSION_SIZE 16 | ||
| #define MSVM_FIRMWARE_GIT_COMMIT_SIZE 48 | ||
|
|
||
| #pragma pack(1) | ||
|
|
||
| typedef struct { | ||
| // | ||
| // MSVM_FIRMWARE_VERSION_SIGNATURE. Lets the host find/validate the record by | ||
| // a flat byte scan regardless of FFS/section wrapping. | ||
| // | ||
| UINT32 Signature; | ||
|
|
||
| // | ||
| // MSVM_FIRMWARE_VERSION_STRUCT_VERSION. Incremented when fields are added. | ||
| // | ||
| UINT16 StructVersion; | ||
|
|
||
| // | ||
| // Size of this structure in bytes. Lets the host skip unknown trailing | ||
| // fields from a newer firmware build. | ||
| // | ||
| UINT16 HeaderSize; | ||
|
|
||
| // | ||
| // Bitmask of MSVM_FIRMWARE_VERSION_FLAG_* values describing the build. | ||
| // | ||
| UINT32 Flags; | ||
|
|
||
| // | ||
| // MSVM_FIRMWARE_INTERFACE_VERSION_MAJOR / _MINOR: the human-curated | ||
| // firmware/VMM compatibility contract version. Unlike BaseVersion this is a | ||
| // machine-comparable gate: the VMM checks these to decide whether it can talk | ||
| // to this firmware and should warn/bail early if not. The values live in | ||
| // MsvmPkg/FirmwareVersion.toml; see the interface version comment above for | ||
| // the bump rules. | ||
| // | ||
| UINT16 InterfaceVersionMajor; | ||
| UINT16 InterfaceVersionMinor; | ||
|
|
||
| // | ||
| // Static "major.minor" release prefix (e.g. "26.0"). The release patch | ||
| // number is assigned later by the GitHub release workflow and is therefore | ||
| // not present here; resolve the full version by finding the release whose | ||
| // tag targets GitCommit. | ||
| // | ||
| CHAR8 BaseVersion[MSVM_FIRMWARE_BASE_VERSION_SIZE]; | ||
|
|
||
| // | ||
| // Full 40-character git commit hash. Set to "unknown" when git information is | ||
| // unavailable at build time. Whether the tree was dirty at build time is | ||
| // reported separately via MSVM_FIRMWARE_VERSION_FLAG_DIRTY in Flags. | ||
| // | ||
| CHAR8 GitCommit[MSVM_FIRMWARE_GIT_COMMIT_SIZE]; | ||
| } MSVM_FIRMWARE_VERSION_INFO; | ||
|
|
||
| #pragma pack() | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should probably all come to an agreement on how this number gets populated between closed source and open source. Is the intent that both pipelines will need to maintain the same major.minor version? We do have a static .json file in our CI folders that define this as the source of truth.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added a TOML file that defines the versions for open source. We definitely should figure out how to have a coherent versioning story between open and closed source.