fix(qdp-core): link cudart locally and stub FFI when toolkit is absent by andrewmusselman · Pull Request #1321 · apache/mahout

andrewmusselman · 2026-05-17T15:35:45Z

Closes #1318 .

qdp-core/build.rs (extended; it already existed for protoc) now:

probes for nvcc with the same logic as qdp-kernels/build.rs,
emits cargo:rustc-link-search=native=$CUDA_PATH/lib64 and
cargo:rustc-link-lib=cudart when found,
emits cargo:rustc-cfg=qdp_no_cuda when not found, with a clear
cargo:warning pointing at the toolkit install,
respects QDP_NO_CUDA=1 for explicit forcing (matching qdp-kernels).
qdp-core/src/gpu/cuda_ffi.rs wraps the extern "C" block in
#[cfg(not(qdp_no_cuda))] and adds matching #[cfg(qdp_no_cuda)] mod no_cuda_stubs { ... } with pub(crate) unsafe fn stubs for all 14
declared functions. Each stub returns 999 — the same sentinel
qdp-kernels uses for its kernel-launcher stubs — so existing caller
error paths (if ret != 0 { return Err(...) }) surface a clean runtime
error if anyone calls a CUDA function on a no-toolkit build, instead of
failing at link time. The whole stub module is wrapped in a single
#[allow(non_snake_case)] since the originals are camelCase to match
the real CUDA Runtime API.

Cross-crate behaviour after this PR:

environment	qdp-kernels	qdp-core	result
toolkit installed	links cudart	links cudart	normal GPU build
driver only / macOS / CI	stub launchers	stub Runtime API	links; runtime err 999
`QDP_NO_CUDA=1`	stub launchers	stub Runtime API	links; runtime err 999

Verified on Linux + CUDA 12.4: cargo build --workspace --tests --exclude qdp-python succeeds both with and without QDP_NO_CUDA=1;
make test_rust runs full integration tests on the GPU; 12 lint
warnings introduced by the stubs are silenced.

ryankert01 · 2026-05-19T11:16:04Z

Like the high-level idea:

With CUDA toolkit installed: normal GPU-enabled build.
Without CUDA toolkit: build still links successfully, but CUDA calls return a controlled runtime error.
With QDP_NO_CUDA=1: same forced no-CUDA behavior.

ryankert01

LGTM. Two small things worth doing. Also, needs to fix the pre-commit.

ryankert01 · 2026-05-19T11:31:25Z

+/// This function:
+///   * emits `cargo:rustc-link-lib=cudart` and the appropriate
+///     `cargo:rustc-link-search` path when nvcc is found, and
+///   * emits `cargo:rustc-cfg=qdp_no_cuda` when it is not, gating the


output().is_ok() is true even if nvcc exits non-zero — a half-installed nvcc would set has_cuda = true and fall through to a link error. Suggest .map(|o| o.status.success()).unwrap_or(false). Same idiom in qdp-kernels/build.rs:177; fix both so they can't disagree.

ryankert01 · 2026-05-19T11:31:25Z

+///
+/// `qdp-core` declares CUDA Runtime API extern symbols in `src/gpu/cuda_ffi.rs`
+/// (cudaHostAlloc, cudaMemGetInfo, cudaEventCreateWithFlags, ...). Those symbols
+/// must be resolved at link time, which requires `libcudart` from the CUDA


Missing cargo:rerun-if-env-changed=PATH. The whole decision hinges on nvcc-on-PATH, so installing CUDA after a stub build won't re-trigger the script until cargo clean. Same gap in qdp-kernels.

Address review feedback on the CUDA-detection probe shared by qdp-core and qdp-kernels build scripts: - output().is_ok() was true even when nvcc exited non-zero, so a half-installed toolkit would set has_cuda = true and fall through to a link error. Use .map(|o| o.status.success()).unwrap_or(false) in both scripts so the two probes can't disagree. - Add cargo:rerun-if-env-changed=PATH so installing the toolkit after a stub build re-triggers detection without `cargo clean`. Also run cargo fmt on the no_cuda_stubs module in cuda_ffi.rs to clear the pre-commit fmt hook (was failing the test job).

This PR lets _qdp build and import without the CUDA toolkit (stub CUDA Runtime symbols), so "extension importable" no longer implies a usable GPU. The conftest auto-skip only checked extension availability, so on a GPU-less runner the @pytest.mark.gpu tests ran against the stub engine and aborted the pytest worker (SIGABRT) instead of being skipped. - conftest: add a torch.cuda.is_available() probe and skip @pytest.mark.gpu tests when no device is present, matching the inline torch.cuda.is_available() guards already used across the QDP modules. The existing "extension missing" skip behaviour is unchanged. - mark test_synthetic_loader_batch_count @pytest.mark.gpu: it iterates a synthetic loader, which encodes on the GPU. Verified locally: with CUDA hidden the previously-crashing tests skip cleanly (0 failures across testing/qdp + testing/qdp_python); with a GPU present they run and pass.

Follow-up to #1321 (addresses #1414). After #1321, `_qdp` can build and import without the CUDA toolkit (stub runtime), so "extension importable" no longer implies a usable GPU. The test suite gated GPU tests on `torch.cuda.is_available()`, a proxy that is wrong on this PR's headline scenario -- a GPU host with PyTorch but no toolkit, where `_qdp` is a stub build yet torch still reports a device. - qdp-core: add `cuda_runtime_available()`, which queries `cudaGetDeviceCount` (false in a stub build via the existing 999 sentinel stub, and on hosts with no device). Re-exported from the crate root. - _qdp: expose it as `_qdp.cuda_available()`. - qumat_qdp: add `is_cuda_available()`, mirroring `is_triton_amd_available()`, as the single Python source of truth. - testing/conftest: gate the `@pytest.mark.gpu` auto-skip on the native signal (falling back to torch only if the helper is absent). - test_fallback: coverage that runs on a stub build too, guarding that querying availability returns a bool without aborting. Verified on GPU (tests run and pass) and with CUDA hidden (tests skip); full Rust suite, clippy --all-features, ruff, and ty all clean.

…on it (#1416) Follow-up to #1321 (addresses #1414). After #1321, `_qdp` can build and import without the CUDA toolkit (stub runtime), so "extension importable" no longer implies a usable GPU. The test suite gated GPU tests on `torch.cuda.is_available()`, a proxy that is wrong on this PR's headline scenario -- a GPU host with PyTorch but no toolkit, where `_qdp` is a stub build yet torch still reports a device. - qdp-core: add `cuda_runtime_available()`, which queries `cudaGetDeviceCount` (false in a stub build via the existing 999 sentinel stub, and on hosts with no device). Re-exported from the crate root. - _qdp: expose it as `_qdp.cuda_available()`. - qumat_qdp: add `is_cuda_available()`, mirroring `is_triton_amd_available()`, as the single Python source of truth. - testing/conftest: gate the `@pytest.mark.gpu` auto-skip on the native signal (falling back to torch only if the helper is absent). - test_fallback: coverage that runs on a stub build too, guarding that querying availability returns a bool without aborting. Verified on GPU (tests run and pass) and with CUDA hidden (tests skip); full Rust suite, clippy --all-features, ruff, and ty all clean.

andrewmusselman requested review from 400Ping, guan404ming and ryankert01 as code owners May 17, 2026 15:35

ryankert01 reviewed May 19, 2026

View reviewed changes

ryankert01 changed the title ~~qdp-core: own the cudart link directive; gate extern block on qdp_no_cuda~~ feat(qdp-core): own the cudart link directive; gate extern block on qdp_no_cuda May 19, 2026

ryankert01 changed the title ~~feat(qdp-core): own the cudart link directive; gate extern block on qdp_no_cuda~~ fix(qdp-core): link cudart locally and stub FFI when toolkit is absent May 19, 2026

ryankert01 force-pushed the qdp-core-cudart-linkage branch from 059cb3d to 1ff2f68 Compare June 22, 2026 17:14

ryankert01 assigned ryankert01 and unassigned ryankert01 Jun 22, 2026

fix(qdp): own -lcudart linkage in qdp-core, gate stubs on qdp_no_cuda

8b1f340

ryankert01 force-pushed the qdp-core-cudart-linkage branch from 1ff2f68 to 8b1f340 Compare June 26, 2026 18:05

ryankert01 approved these changes Jun 26, 2026

View reviewed changes

ryankert01 added 2 commits June 26, 2026 18:18

ryankert01 merged commit 205ce75 into main Jun 26, 2026
8 checks passed

ryankert01 deleted the qdp-core-cudart-linkage branch June 26, 2026 18:56

ryankert01 mentioned this pull request Jun 26, 2026

[QDP] Unify GPU-availability detection in tests; stub CUDA path should raise, not abort #1414

Closed

ryankert01 mentioned this pull request Jun 26, 2026

[MAHOUT-1414][QDP] Expose native cuda_available() and gate GPU tests on it #1416

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix(qdp-core): link cudart locally and stub FFI when toolkit is absent - #1321

fix(qdp-core): link cudart locally and stub FFI when toolkit is absent #1321
ryankert01 merged 3 commits into
mainfrom
qdp-core-cudart-linkage

andrewmusselman commented May 17, 2026 •

edited by ryankert01

Loading

Uh oh!

ryankert01 commented May 19, 2026 •

edited

Loading

Uh oh!

ryankert01 left a comment •

edited

Loading

Uh oh!

ryankert01 May 19, 2026

Uh oh!

ryankert01 May 19, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

andrewmusselman commented May 17, 2026 • edited by ryankert01 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ryankert01 commented May 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ryankert01 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ryankert01 May 19, 2026

Choose a reason for hiding this comment

Uh oh!

ryankert01 May 19, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

andrewmusselman commented May 17, 2026 •

edited by ryankert01

Loading

ryankert01 commented May 19, 2026 •

edited

Loading

ryankert01 left a comment •

edited

Loading