Tracking Issue for LLVM AllocToken and Heap Partitioning Support for Rust

[rcvalle]

This is a tracking issue for the LLVM AllocToken and Heap Partitioning Support for Rust project (see the design document draft).

Steps

• Add option to the Rust compiler driver and frontend (i.e., -Zsanitizer=alloc-token and related options).
• Add support for emitting allocation token function attributes, module flags, and allocation token hint metadata to the Rust compiler code generation.
• Implement the default pointer-split heap partitioning scheme in the rustc_sanitizers crate.
• Define allocation token classifications and type name encodings for cross-language LLVM AllocToken and heap partitioning support.
• Change the initial implementation to use the defined classifications and type name encodings for cross-language LLVM AllocToken and heap partitioning support.
• Add typed allocation paths and allocation token hints to the Rust standard library.
• Add token-enabled allocator interface support to the Rust standard library and to the Rust compiler allocator shim generation.
• Add the alloc_token_infer intrinsic.
• Create documentation and tests.

Unresolved Questions

None.

Implementation history

• Add LLVM AllocToken and Heap Partitioning Support to the Rust compiler. #160298

[bjorn3]

Left a couple of review comments on rcvalle@41bf600

[bjorn3]

I see you reacted to my comments, but github just doesn't show any commit comments at all anymore. The file sidebar shows comment counts, but nothing is shown inside the actual diff: rcvalle@41bf600

[rcvalle]

Yes, it's happening to me as well. I had to expand all the files to be able to see your comments, but even doing so the third comment didn't show up--I wanted to reply I fixed it as well and I couldn't. Let me open a draft PR sometime later this week and we can continue the discussions and review there.

[nikic]

usize/isize should not be classified as pointer-containing types.

[rcvalle]

usize/isize should not be classified as pointer-containing types.

It's a bit complicated (see the Classification compatibility for the pointer-split heap partitioning scheme section in the design doc). The tl;dr is that the classification is conservative¹ and consistent with how Clang classifies uintptr_t and intptr_t, which end up as usize and isize across the FFI boundary in "mixed binaries"--for cross-language LLVM AllocToken and heap partitioning support--and because pointers may be stored as usize (e.g., via ptr::expose_provenance and ptr::with_exposed_provenance, and AtomicUsize).

Footnotes

1. Misclassifying a type not containing pointers into the partition containing pointers only loses part of the separation benefit for that type, while misclassifying a type containing pointers into the partition not containing pointers breaks the guarantee of the partition not containing pointers. ↩

[RalfJung]

and because pointers may be stored as usize (e.g., via ptr::expose_provenance and ptr::with_exposed_provenance, and AtomicUsize).

No, pointers can't be stored in usize. usize cannot hold provenance. The address part of a pointer can be stored in usize, but that's not a whole pointer. And almost all uses of usize are not for pointers.

usize should be treated like size_t, not like uintptr_t.

[RalfJung]

Tracking issues are typically for already accepted proposals. Where was the prior discussion and which team has accepted this proposal using which process?