Zyphra's whole family, four older architectures, and experts from the drive

Since 21 October 2026: Vulkan is no longer a route of the engine (engine #306): Zamba, Zamba2 and BlackMamba wait for an HRX port, and --moe-slots is not in the current build. See Every node on the GPU.

Zyphra, all of it#

ZAYA1-8B already ran on Vulkan, HRX and ROCm. The rest of the family followed:

• ZAYA1-74B-preview, with its sliding-window layers: Q4_K_M decodes at 35.4 tok/s on Vulkan.
• ZAYA1-base and ZAYA1-reasoning-base, in Zyphra's older checkpoint layout. Our converter reads that layout directly, and the tensors it writes are byte-identical to ZAYA1-8B's.
• ZAYA1-VL-8B, the vision model. It applies a vision-only LoRA to image tokens and attends to each image bidirectionally. Against Zyphra's own FP32 code it agrees at 100 of 101 teacher-forced positions. F16 decodes at 38-51 tok/s. It reads "HELLO 42" off a synthetic image, and it recognises the New York Times front page of the moon landing.
• Zamba, Zamba2 and BlackMamba, plus Zamba2-VL. Zamba v1 and BlackMamba match the reference at 96/96 positions. A new Vulkan scan for Mamba-1 layers took Mamba-370M from 25.8 to 173.5 tok/s; those layers used to run on the CPU.

Adding the vision model left ZAYA1-8B's text path unchanged. Its wikitext perplexity is identical on Vulkan (21.5731, and 21.6213 with four sequences per batch) and HRX (21.5518), and it still agrees with transformers at 95/96. Serve the vision models with 1bit serve --mmproj (docs/vulkan.md).

Four older architectures#

OPT, GPT-Neo, CodeGen and GPT-J now run from our llama.cpp. We checked each port against transformers FP32 before it merged. CodeGen started at 0/96: its fused projection stores query, value and key in that order, not query, key, value. After the fix all four agree at every checked position (docs/vulkan.md). With them, the model registry maps 323 Hugging Face architectures to a backend. "Mapped" means a backend's code accepts the architecture; the census reports separately how many have been checked (Model registry).

MoE experts from the drive#

1bit serve --moe-slots N keeps a MoE model's routed experts in the file on disk. It holds N of them in GPU memory and streams in the rest as the router asks for them. Perplexity is exactly that of the all-GPU model. On Qwen3-Coder-30B-A3B, with 75% of the experts resident and the prefetch moved off the decode path, it decodes at 43-45 tok/s (39 through serve). With 25% resident it still decodes at 8-10. The resident model is faster (79-91 tok/s), so streaming is for models that do not fit (MoE streaming).

HRX decode, now deterministic#

Repeating the same request exposed a problem in HRX's split flash-attention decode kernel: it gave different answers to identical requests, and on Qwen3-Coder-30B it faulted the GPU in two of three runs. 1bit serve --device hrx now decodes without it by default. That path is deterministic, and on most models faster: Qwen3-0.6B goes from 169 to 322 tok/s, and ZAYA1-8B from 23.5 to 25.5. Coder-30B runs 5 of 5 clean at 66-71 tok/s. ONEBIT_HRX_DECODE_SPLIT=1 turns the kernel back on for testing a fix (HRX).

Also#

• ONNX on the GPU on Linux: ONNX Runtime's WebGPU provider, over Vulkan. Qwen3-4B int4 decodes at 52.9 tok/s on the Radeon, against 26.8 on the CPU (ONNX Runtime).
• GPU matrix units: a short reference on the Radeon's matrix instructions. int8 runs at the f16 rate; only int4 × int4 doubles it (HRX).
• Issues and security:
  • issue forms, and RFCs for larger changes in Discussions;
  • a CI check that stops a pin from moving backwards;
  • CodeQL's extended suite, with no open alerts.

Packages for all of this ship on Sunday with the weekly release.