DFlash outputs garbage (////) on RTX 4090 / Ada (sm_89) — reproduces on both v0.3.1 stable and v0.3.2-preview

[noonghunna]

Summary

DFlash speculative decoding produces degenerate ///// output on an RTX 4090 (Ada, sm_89) — the model returns only slash characters instead of text. This reproduces on both published CUDA server images:

• ghcr.io/anbeeld/beellama.cpp:server-cuda-v0.3.1 (stable)
• ghcr.io/anbeeld/beellama.cpp:server-cuda-preview-v0.3.2

The same GGUF weights serve correctly under mainline llama.cpp and ik_llama.cpp on the same rig, so it's specific to beellama's DFlash path — not the model, quant, chat template, driver, or CUDA stack.

Environment

• GPU: NVIDIA GeForce RTX 4090 (single card, Ada, sm_89)
• OS: Ubuntu 24.04 on WSL2 (Windows)
• Driver / CUDA: 610.43.02 / CUDA 13.3
• Images: server-cuda-v0.3.1 and server-cuda-preview-v0.3.2 — both reproduce
• Model: Qwen3.6-27B-Q5_K_S.gguf (main) + Qwen3.6-27B-DFlash-IQ4_XS.gguf (DFlash draft)

The image is compiled for Ada — the boot log's system_info shows CUDA : ARCHS = 500,610,700,750,800,860,890,900 (890 = sm_89 is present), so this is not a missing-cubin issue.

Symptom

Every request returns only /:

[basic completion — "capital of France"]
  ✗ reply didn't mention Paris: //////////////////////////////
[streaming]
  streamed 120 chunks, 120 chars:  /////////////////////////////////...

Boot-log evidence — DFlash drafter fails, then poisons the cross-ring

The DFlash drafter emits invalid logits on the very first generation and fail-closes, but output stays ///// even after DFlash disables itself:

E dflash: invalid reduced-logits token -1 in draft at row=1/5 (top_k=1 committed=28 ... score=-3.40282e+38 spec=-1 offset=0)
E dflash: invalid reduced-logits token -1 in draft at row=1/5 (... committed=29 ... invalid_reduced_logits_streak=2 ...)
E dflash: invalid reduced-logits token -1 in draft at row=1/5 (... committed=30 ... invalid_reduced_logits_streak=3 ...)
E dflash: invalid reduced-logits token -1 in draft at row=1/5 (... committed=31 ... invalid_reduced_logits_streak=4 ...)
E dflash: fail-closed after 4 consecutive invalid reduced-logits drafts; skipping DFlash drafting until the cross-ring state is reset

(score=-3.40282e+38 is -FLT_MAX.) Once the cross-ring enters this state, generation stays degenerate for the rest of the process.

Ruled out

• Not a cold-start / cudagraph warmup artifact — a real warm-up generation runs first, and every scored request after it (plus two independent interactive clients) is still /////.
• Not a stale/wrong image — reproduced on two independently-published builds (v0.3.1 stable + v0.3.2 preview).
• Not a missing-kernel / arch gap — sm_89 cubins are compiled (890 in ARCHS).
• Not the model / quant / chat template — the identical Q5_K_S GGUF serves clean under mainline llama.cpp and ik_llama.cpp on the same 4090.

What works on the same rig

• Mainline llama.cpp + ik_llama.cpp with the same weights → coherent output.
• Speculative decoding via MTP (a different drafter) on the same card → works.

So the fault is the DFlash cross-ring / reduced-logits path specifically, on Ada (sm_89).

Repro (approx.)

The beellama server, DFlash draft enabled:

llama-server \
  -m Qwen3.6-27B-Q5_K_S.gguf \
  --spec-draft-model Qwen3.6-27B-DFlash-IQ4_XS.gguf \
  --spec-type dflash --spec-dflash-cross-ctx 1024 \
  -ngl all --spec-draft-ngl all

Context

Surfaced and fully investigated downstream in the club-3090 project — complete logs, the cross-build confirmation, and the arch-detection details are at noonghunna/club-3090#693. Happy to run any diagnostic build, flag, or verbose-DFlash trace you'd like on the affected 4090.