Skip to content

Fix DeviceType annotations for device and stream arguments - #4059

Open
XXXXRT666 wants to merge 4 commits into
ml-explore:mainfrom
XXXXRT666:fix-device-type-stream-typing
Open

Fix DeviceType annotations for device and stream arguments#4059
XXXXRT666 wants to merge 4 commits into
ml-explore:mainfrom
XXXXRT666:fix-device-type-stream-typing

Conversation

@XXXXRT666

@XXXXRT666 XXXXRT666 commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Proposed changes

mx.cpu and mx.gpu are DeviceType values. Nanobind accepts them at runtime through the existing implicit conversion from DeviceType to Device, but the generated Python stubs only exposed Device. This caused type checkers to reject valid calls such as:

mx.exp(x, stream=mx.gpu)

This PR:

  • Adds DeviceType as an explicit StreamOrDevice alternative.
  • Updates stream argument annotations across core, random, linalg, fast, and distributed APIs.
  • Includes ThreadLocalStream, which is already accepted by StreamOrDevice, in those annotations.
  • Adds DeviceType-aware annotations to device and stream APIs that rely on nanobind's implicit conversion.

Checklist

  • I have read the CONTRIBUTING document
  • I have run pre-commit run --all-files to format my code / installed pre-commit prior to committing changes
  • I have added tests that prove my fix is effective or that my feature works
  • I have updated the necessary documentation (if needed)

Comment thread python/src/distributed.cpp Outdated
Comment thread mlx/utils.cpp
@XXXXRT666

XXXXRT666 commented Aug 8, 2026

Copy link
Copy Markdown
Contributor Author

Another question: Do you prefer using Union/Optional, or the Python 3.10-style A | B type annotation?

Because nanobind automatically generates the A | B syntax, while the existing codebase still uses the legacy Union/Optional style.

@zcbenz

zcbenz commented Aug 8, 2026

Copy link
Copy Markdown
Member

I don't really have a preference, but it feels more reasonable following what nanobind does.

@XXXXRT666
XXXXRT666 force-pushed the fix-device-type-stream-typing branch from e9aa5d8 to f9ec808 Compare August 8, 2026 11:13
@XXXXRT666

Copy link
Copy Markdown
Contributor Author

I think the CI is failing because the non-Metal export cannot recognize DeviceType. Trying to fix it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants