Blog · field notes
Build notes: local LLMs on AMD Strix Halo
Notes on the work: what we port, what we measure, what we ship.
Serial · 2026
Notes, in order.
This week in the engine
The engine is pinned to this week's Lemonade release, the release candidate's checks are green, and two measurements this week say plainly where two slow things stand.
Ternary Bonsai-2-27B on the Ryzen AI NPU
PrismML's Ternary-Bonsai-2-27B now answers correctly on the XDNA 2 NPU, and its decode went from 11.5 to 8.87 seconds per token. This is work in progress: the model runs and is right, and it is slow, and the measurement says why.
The GPU and the NPU at the same time, measured
For four weeks the fleet that works on this engine ran every NPU job behind the GPU queue. The reason was a report from another project's documentation: GPU and NPU work at the same time could hang the machine, or corrupt NPU results, while the GPU's fabric clock (FCLK) changed level. The suggested fix was to hold…
Boot a stick
The feedback we hear most is "keep it simple". So here is the whole setup for local LLMs on a Ryzen AI MAX+ 395 (Strix Halo) machine:
BitNet b1.58 runs on HRX
Microsoft's BitNet b1.58 2B-4T, in the TQ20 ternary GGUF (1.11 GiB, 2.06 bits per weight), now runs end to end on the Radeon 8060S through HRX. Every node of the graph is on the GPU; nothing falls back to the CPU. Perplexity on HRX is 102.42 against 102.56 on the CPU for the same file, and decode is 123.4 tok/s.
Every node on the GPU: local LLMs on Strix Halo in Lemonade
On a Ryzen AI Max+ 395 (Strix Halo), the 1bit engine now runs every graph node of every local LLM and mode we test on the integrated Radeon 8060S GPU, through AMD's HRX runtime and our own HIP kernels. Nothing falls back to the CPU. Here is what landed ahead of the first release, measured on that machine.
Where the engine is going: HRX and the NPU
The engine is heading to AMD's HRX backend, whose kernels we write in Loom, plus the NPU.
Qwen3.8-27B at 45.7 tok/s: DFlash2 in 1bit serve, and 4-bit activations with a Hadamard rotation
Qwen3.8-27B is a dense 27B model, and on Strix Halo its decode speed is set by memory bandwidth: every token reads the whole 15 GB file, so plain decode sits near 11-12 tok/s. Prompt processing is set by the GPU's matrix units instead. This week we pushed on both. Every number below was measured on Strix Halo (Ryzen…
Laya routing: ask the model what it knows
The engine can run the same model on several devices of one machine: Vulkan and HRX on the Radeon, ROCm, ZINC, and the NPU for its own model format. Someone has to pick one for each request. Step 4 of the port gave that job to Laya, a small decision model that answers typed questions about a piece of text in one…
509 tokens a second of prompt: the Hadamard route ships
Earlier today we measured a way to make Qwen3.8-27B read prompts faster on Strix Halo: 4-bit activations on the GPU's matrix units, with a Hadamard rotation to keep them accurate. It is no longer an experiment. The engine now has a tool that makes the rotated file, 1bit serve recognises the file and runs it on the…
Milestone: the HRX decode race is solved
Yesterday's post ended with a workaround. HRX's split flash-attention decode kernel gave different answers to identical requests, and on Qwen3-Coder-30B it sometimes produced NaNs that faulted the GPU (#123, #140). So 1bit serve --device hrx stopped using it. Today we found the cause and fixed it, and the kernel is…
Zyphra's whole family, four older architectures, and experts from the drive
A lot landed this week. Zyphra's model family now runs end to end, including its vision models. Four older architectures that upstream llama.cpp never shipped are in. 1bit serve can run a MoE model whose experts do not fit in memory. And the HRX decode path became deterministic. Every number below was measured on…
The NPU drafts, the GPU checks
Speculative decoding makes a large model faster by letting a small one guess ahead: the small model proposes a few tokens, the large model checks them all in one pass, and every guess it agrees with is a token it did not have to produce one at a time. Usually both models run on the GPU. On Strix Halo there is a second…
Speculative decoding: where it pays, and where it does not
Speculative decoding lets a cheap drafter guess the next few tokens and the model check them all in one pass. When the guesses are right, several tokens come out of one pass instead of one. We measured it on the two most popular Unsloth models on Strix Halo, and the answer depends entirely on the model.
Smaller files: Unsloth, ROCmFPX and a ternary 27B
A smaller model file decodes faster, because decode is limited by how many bytes the GPU reads per token. The question is what each format costs in accuracy. We measured three families of small files on Strix Halo against their full-precision originals: KL divergence (how far the model's predictions move) and how…
One GPU, many users
A chat with one person and a server for many want different things from the same GPU. We measured Qwen3.8-27B (UD-Q4KXL) on Strix Halo from 1 to 24 simultaneous requests, and the engine now grows with the load.
1bit.MONSTER is now 1bit engine
1bit.MONSTER grew fast: an inference engine, a model zoo, benchmarks, a store, a voice assistant and a lot of experiments, all in one repository with all of its history. The engine that came out of it is now its own project, 1bit engine, and this site is its home.
Prefill on HRX, decode on Vulkan, one KV cache
On Strix Halo's Radeon iGPU, AMD's HRX runtime prefills a prompt faster than Vulkan, and Vulkan decodes faster than HRX. 1bit serve --device vulkan --prefill-device hrx now uses both on the same request, and the key-value cache never gets copied between them.
HRX answers long prompts correctly, and where Unsloth quants pay
Two results from the end of the day: a bug fixed in HRX decoding, and one table that answers "which Unsloth file should I download?" for five popular models.
The first week of 1bit engine
The clean repository started on 2026-09-22. In its first three days, the engine came up on every device it targets, inside Lemonade, with each step verified on real hardware before it merged. These are the milestones, with the numbers and where they come from.
Archive · 1bit.MONSTER
From the 1bit.MONSTER years.
The posts from before the clean repository, as they were written. They describe the old monorepo; the engine's current numbers are on the wiki.