Skip to content

Warn when the CBMC on PATH does not match the pinned version - #4723

Open
ivmat wants to merge 1 commit into
model-checking:mainfrom
ivmat:cbmc-pin-rebase
Open

Warn when the CBMC on PATH does not match the pinned version#4723
ivmat wants to merge 1 commit into
model-checking:mainfrom
ivmat:cbmc-pin-rebase

Conversation

@ivmat

@ivmat ivmat commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Description

kani-dependencies pins a CBMC version, but the pin is only enforced at setup/CI time (install_deps.sh, kani-regression.sh). At runtime kani-driver uses whatever cbmc resolves from PATH with no check, so a locally installed CBMC can silently diverge from the pin while the user believes the pinned toolchain is in use. On a machine with several CBMC versions installed this is easy to hit and invisible when it happens — results get attributed to a toolchain that never ran.

This PR makes kani-driver resolve cbmc --version from PATH, compare it against the pin, and warn naming both versions when they differ. Design points:

  • The pin is embedded at compile time (include_str! of kani-dependencies) rather than read at runtime: release bundles don't ship that file, so a runtime read would silently disable the check for exactly the users least able to audit their toolchain. rustc tracks the included file, so editing the pin still rebuilds.
  • --version is handled explicitly (disable_version_flag) so kani --version / cargo kani --version actually run the check — clap's built-in flag exits during parsing, before any driver code runs. The first output line stays kani <version> / cargo-kani <version>, so scripts that parse it keep working; the CBMC lines are appended after it.
  • For cargo kani, the version flag is detected on the raw arguments before project configuration is merged, so --version cannot be broken by a malformed Cargo.toml (matching the robustness of clap's built-in, whose version action previously fired inside cargo_locate_project's early parse). The scan matches whole arguments only and stops at -- or --cbmc-args; a post-parse fallback catches spellings the scan cannot see, such as -V clustered with other short flags. Known residual: a clustered -V combined with a malformed Cargo.toml reports the TOML error instead of the version — loudly, not silently.
  • The warning is suppressed under --quiet during verification (preserving the zero-output contract in check-quiet.sh), but --version as an explicit query always reports.
  • tools/build-kani bundling whichever cbmc is on the builder's PATH is the build-time analogue of the same problem; left as a follow-up to keep this focused. With this check, a mismatched bundle at least reports itself at runtime.

Manual testing

Unit tests cover the version parse/compare and the raw --version scan. Verified live both ways — a matching CBMC 6.10.0 is silent, a mismatched 6.8.0 fires naming both versions. Both script-based version regression tests (kani-version-flag-version, cargo-kani-version-flag-version) pass. cargo kani --version verified to work from a directory whose Cargo.toml is malformed, and a --version after --cbmc-args is forwarded to CBMC, not intercepted.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 and MIT licenses.

…sion

`kani-dependencies` pins the CBMC version, but that pin was only enforced at setup/CI time by
`install_deps.sh` / `kani-regression.sh`. At runtime `kani-driver` used whatever `cbmc` it resolved
from `PATH` with no check, so a locally installed, unpinned CBMC could silently diverge from the pin
while every log claimed the pinned toolchain was in use.

`kani-driver` now resolves the CBMC version actually on `PATH` via `cbmc --version`, compares it
against the pin, and warns naming both versions when they differ. The pin is embedded at compile time
with `include_str!("../../kani-dependencies")` rather than read from the install root at runtime:
release bundles do not ship `kani-dependencies`, so a runtime read would silently disable the check
for exactly the users least able to audit their own toolchain. Embedding also removes the
dev-repo/release asymmetry entirely, and rustc tracks the included file as a build dependency, so
editing the pin still triggers a rebuild.

`--version` is handled explicitly rather than by clap's generated flag (`disable_version_flag`), so
that `kani --version` and `cargo kani --version` actually perform this check. Previously clap
intercepted the flag and exited before the driver ran, which would have made "check `--version`
before a campaign" useless advice. The explicit path preserves clap's conventional first line
(`kani <version>` / `cargo-kani <version>`) so scripts that parse it -- including the two
script-based version regression tests -- keep working, and appends the CBMC lines after it.

For `cargo kani`, the flag is detected by scanning the raw arguments before any project
configuration is merged, so `--version` cannot be broken by a malformed `Cargo.toml` -- matching the
robustness of clap's built-in, whose version action previously fired inside
`cargo_locate_project`'s early parse, before any TOML was read. The scan matches whole arguments
only and stops at `--` or `--cbmc-args` (everything beyond those belongs to CBMC, not Kani); a
post-parse fallback catches spellings the scan cannot see, such as `-V` clustered with other short
flags. Known residual: a clustered `-V` combined with a malformed `Cargo.toml` reports the TOML
error instead of the version -- loudly, not silently.

The check is deliberately not emitted under `--quiet` during verification, preserving the
zero-output contract asserted by `tests/script-based-pre/check-quiet/check-quiet.sh`; `--version`
is an explicit one-shot query, not verification output, so it reports regardless.

Note for reviewers: `tools/build-kani` bundles whichever `cbmc` is on the builder's `PATH`
(`which::which("cbmc")`), which is the build-time analogue of the same problem. That is left as a
follow-up to keep this change focused; with this check in place a mismatched bundle now at least
reports itself at runtime.
@ivmat
ivmat requested a review from a team as a code owner August 7, 2026 06:39
@feliperodri feliperodri added [C] Internal Tracks some internal work. I.e.: Users should not be affected. [I] CI / Infrastructure Work done to CI, tests and infrastructure. labels Aug 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

[C] Internal Tracks some internal work. I.e.: Users should not be affected. [I] CI / Infrastructure Work done to CI, tests and infrastructure.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants