How the engine is built#

The 1bit engine runs inside AMD's Lemonade Server as its onebit backend, pinned to Lemonade's weekly release (third_party/lemonade, moved by the bump-lemonade workflow; Lemonade). Lemonade stays the server you talk to: its catalog, downloads, router and UI. For the models it hands to 1bit, it runs the engine as one of its backends, behind an OpenAI-compatible API. This page is the technical map of what runs where; the README is the short version.

Routes#

• HRX on the Radeon iGPU (HRX): AMD's ggml-hrx with its Loom kernels and our HIP kernels, the engine's one GPU route. Every ggml weight type runs on HRX, including NVFP4, Q2_0, MXFP4 and the ternary TQ, PTQ and PQ formats. For a GGUF, Lemonade starts the engine's llama-server directly, one HTTP hop (#475, 1bit serve). Placement is strict by default: the whole graph runs on HRX, or the load stops with the node that would not (#482, 1bit serve). --device cpu runs the same build on the CPU.
• The NPU engine on the XDNA 2 NPU, on our own NPU runtime with full-ELF kernel sets: correctness first, speed next (NPU engine).
• Hybrid: the prompt on the NPU engine, the tokens on HRX. 1bit serve --device hybrid hands the prompt's KV cache from a prefill provider to the HRX llama-server (Hybrid). The NPU and the GPU run at the same time; the measured cost of a busy GPU to the NPU's weight stream is recorded there.
• ONNX Runtime, linked in-process behind -DONEBIT_ORT, with our own execution provider that claims MatMul and runs it on the NPU runtime (ONNX Runtime, ONNX pack).
• ~~DwarfStar~~ — removed (owner decision 2026-11-01, out of the tree 2026-11-02): its DeepSeek V4 Flash / GLM 5.x / Qwen3.8-Flash-Next route was antirez's CUDA kernels through a HIP shim plus hipBLAS, not our HIP kernels. Those models run on HRX, and the engine keeps no 1BP reader. The reference fork 1bit-MONSTER/ds4 stays in the open for the history.
• Laya, opt-in, which picks where each request runs (Laya router), and the Project Manager, ZAYA1-8B delegating to expert models (Project Manager).
• Every Hugging Face model architecture, mapped to a backend by a registry and kept current by a daily census (Model registry).

Vulkan, ROCm, CUDA, ZINC, ComfyUI, Metal/MLX and Windows were cut from the engine's core (#306) and are not routes of the engine. Lemonade serves them with its own backends and builds (Lemonade).

Checked on the hardware#

Every HRX pin change and every PR labelled hw-gate passes the hardware gate on Strix Halo before merge: it builds our HIP kernels into the server as the release does, runs kernel tests, known answers, KLD against the release pin and long-prompt needles, and checks that every graph of every gate model is on the GPU (#478, #484, Hardware gate). The runtime keeps one memory budget for every buffer, and after a GPU fault the server refuses new work and exits cleanly, so Lemonade can restart the model (#486). Serve numbers in these docs come from tools/bench.py, which A/B-measures configurations in the same run (How we measure).

The kernel#

The box's Linux kernel is ours too: v7.4-rc1, plain upstream since the IOMMU PerfOpt series is native there, built from our config, MOK-signed for Secure Boot, and gated like a pin bump. Moving it to the next upstream tag is a written step list (Linux kernel).

Pinned sources#

HRX is pinned to AMD's hrx-system and ggml-hrx with our commits on top, kept current by bump-hrx (HRX); Lemonade, Laya and the tokenizers library each have their own bump workflow (Releases). A hand-raised bump carries the branch name and the derived data the checks expect (Releases).

Direction#

HRX with our HIP kernels, plus the NPU engine, embedded into Lemonade rather than Lemonade embedded into the engine.¹ HRX met the decode gates in RFC #213, which removed the engine's Vulkan and ROCm builds. ZAYA1-8B (Zyphra) runs from our llama.cpp on HRX at about 90 tok/s decode in Q4_K_M, matching transformers. Zyphra's Zamba, Zamba2 and BlackMamba ran only on the removed Vulkan build and are to be ported to HRX. The working engine is being ported from our private development repository; see Porting map. This repository holds the verified code without the development history.

View this page's source on GitHub ↗