Skip to content

Template Metal C2C FFT scalar lanes - #3969

Merged
zcbenz merged 1 commit into
ml-explore:mainfrom
PhysicistJohn:codex/generic-metal-fft
Aug 6, 2026
Merged

Template Metal C2C FFT scalar lanes#3969
zcbenz merged 1 commit into
ml-explore:mainfrom
PhysicistJohn:codex/generic-metal-fft

Conversation

@PhysicistJohn

@PhysicistJohn PhysicistJohn commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Motivation

MLX's Metal C2C FFT core hard-codes float2. This PR is the narrow Metal-kernel foundation for #2112: it removes that source-level coupling so a later, end-to-end reduced-precision complex proposal can be evaluated without rewriting the FFT implementation first.

The target use case is complex I/Q and signal-processing workloads, where storing real and imaginary lanes in FP16 or BF16 halves storage and bytes transferred relative to two FP32 lanes. This PR does not claim that lower precision is appropriate for every workload; public dtype semantics, host dispatch, operation coverage, accuracy policy, and new static instantiations remain separate work.

Existing accelerator-library precedent

This is not a novel representation:

  • NVIDIA cuFFT supports half-complex CUDA_C_16F and bfloat16-complex CUDA_C_16BF transforms through its type-generic planning API.
  • AMD rocFFT exposes rocfft_precision_half together with complex interleaved and planar array layouts.

Those are direct GPU-vendor FFT libraries. This patch only prepares MLX's C2C Metal implementation for equivalent storage lanes; it does not add a public reduced-precision complex type.

Scope

  • Template C2C radix arithmetic, shared memory, and I/O conversion over the scalar lane type.
  • Preserve the current public API, host dispatch, static kernel-instantiation table, and packed float32 RFFT/IRFFT paths.
  • Generalize the C2C four-step second pass so every eligible lane type retains the current float32 indexing, inversion, and normalization behavior.
  • Derive twiddle phase evaluation precision from the scalar lane and Metal's pi constant through the usual arithmetic promotion, rather than naming float inside generic FFT code.

Binary-size impact

No new static kernels or public exports are instantiated.

  • Local release mlx.metallib: 130,977,720 B → 130,994,264 B
  • Delta: +16,544 B (+0.01263%)
  • metal-nm -g exported-symbol output: exact match to the base build

Validation

  • Native xcrun metal compile and metallib link of the changed fft.metal using MLX's build flags.
  • Native CMake build with deployment target macOS 15.0 (NAX kernels excluded by the existing build gate): ctest 261/261 passed, including all FFT cases.
  • Branch-local Metal Python extension: python -m unittest test_fft -v16 passed, 2 expected skips.
  • Compile-only C2C instantiations for half2 and bfloat2 across Stockham, Rader, Bluestein, and four-step planners.
  • The phase-typing refinement produces byte-identical production and reduced-precision AIR relative to the previously reviewed generic FFT tree.
  • pre-commit (clang-format) and git diff --check passed.

Focused float32 regression checks

Path Correctness Generic / base median
C2C, N=256, batch=4096 Bit-identical 0.2952 / 0.2954 ms (0.9993x)
Four-step second pass Bit-identical 0.2384 / 0.2350 ms (1.0145x)

The second measurement is a focused dispatch-scale check, not a claim of a material performance change. There is no end-to-end reduced-precision performance claim until a future public dtype and dispatch path exists.

Downstream half-complex FFT result

PhysicistJohn/mlx#7 adds one
dependent commit on top of #3969, #3970, and the storage adapter in
PhysicistJohn/mlx#6. It
instantiates packed complex_t<half> C2C Stockham, Rader, Bluestein, and
four-step kernels.

A matched M5 Max benchmark compared the half kernels directly with untouched
main FP32 (fb5133e1) using identical plans, geometry, inputs, batches,
warmups, samples, and order-balanced runs:

Plan Forward half/main Inverse half/main
Stockham 2.151x 2.151x
Rader 1.273x 1.326x
Bluestein 1.165x 1.242x
Four-step 2.242x 2.220x

The candidate FP32 control was bit-identical to original FP32 on all four paths
and measured 0.992x-1.065x its throughput across forward and inverse. Packed
half-complex storage and external I/O are 50% smaller. A 39-length correctness
sweep through 1,048,576 points remained finite, with worst normalized RMSE of
0.2585% forward and 0.4101% round trip.

Comment thread mlx/backend/metal/kernels/fft/radix.h Outdated
Comment thread mlx/backend/metal/kernels/fft/radix.h Outdated
Comment thread mlx/backend/metal/kernels/fft/radix.h Outdated
Comment thread mlx/backend/metal/kernels/fft/readwrite.h Outdated
@zcbenz zcbenz added the await response This pull request is waiting for response from the author. label Aug 5, 2026
Template C2C FFT arithmetic and I/O over their scalar lane while retaining the existing static entry points and packed real paths.

Make the C2C four-step second pass use the same generic storage path so future scalar lanes preserve the existing indexing and scaling behavior.

Derive twiddle phase evaluation precision from the scalar lane and Metal's pi constant instead of naming float in generic FFT code, while preserving current code generation.
@PhysicistJohn
PhysicistJohn force-pushed the codex/generic-metal-fft branch from 611e080 to acca981 Compare August 6, 2026 06:17
@PhysicistJohn

Copy link
Copy Markdown
Contributor Author

Thanks for the review! Applied all four: dropped fft_make_complex and fft_splat, with get_twiddle returning the brace-initialized pair per your suggestion and the radix constants back to plain T scalars, so the codelets read like the originals again (x[0] - a_1 / 2, a_5 / 4); dropped the fft_complex_t<T> alias in favor of spelling vec<T, 2> directly; and dropped FFTValueTraits, with post_in/pre_out back to the original shape and the inverse inlined per your suggestion. The only trait left is the small scalar-type derivation the kernels need to get the lane type from the storage type.

Also rebased onto main to pick up #3970, which resolves the conflict, and folded in the explicit thread qualifiers from #3963 along the way. One consequence of restoring the original post_in: real-input loads under inv now yield (x, +0) rather than (x, −0), matching the pre-#3969 code.

@zcbenz zcbenz removed the await response This pull request is waiting for response from the author. label Aug 6, 2026

@zcbenz zcbenz left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

@zcbenz
zcbenz merged commit 39d9a8a into ml-explore:main Aug 6, 2026
28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants