Documentation
How to run the engine, what each device does, and how the pieces fit. Measured numbers are on the wiki.
Start
| Page | What it covers |
|---|
| 1bit serve | The engine runs inside Lemonade: Lemonade runs 1bit serve as a backend, the way it runs llama-server. 1bit serve exposes nothing but an OpenAI-compatible API, so it also works on… |
| Lemonade | The 1bit engine runs inside Lemonade. Lemonade is the server users talk to, with its own catalog, downloads, router and UI. For the models it serves with 1bit, it runs the engine… |
Devices
| Page | What it covers |
|---|
| NPU | The fast lane runs one whole-layer kernel per decoder layer, plus an lm-head kernel. For each token, the host submits the 28 layer runs and the lm head as one XRT runlist. This… |
| HRX + Vulkan | One llama.cpp build with two GPU backends, ggml-hrx (AMD's HRX runtime) and ggml-vulkan. Its single llama-server exposes both devices on Strix Halo: |
| Vulkan (upstream) | 1bit serve --device vulkan runs a llama-server built from upstream llama.cpp (MIT), pinned to its latest release in thirdparty/llama.cpp-vulkan. It is a separate tree from… |
| Lean (ROCmFP4, ROCmI4) | 1bit serve --lean trades accuracy for speed. It runs models in the AMD-focused formats of ROCmFPX (MIT), a llama.cpp fork that upstream llama.cpp cannot read, so the lean route… |
| ZINC | ZINC (MIT) is a single-binary GGUF engine written in Zig. It has its own kernels for four GPU backends, and serves /health, /v1/models and /v1/chat/completions: |
| Apple Silicon | On macOS, 1bit serve --device mlx serves MLX models. The executor is the server binary of lemon-mlx-engine (fork bong-water-water-bong/lemon-mlx-engine). It builds MLX's Metal… |
Components
| Page | What it covers |
|---|
| Laya router | Step 4 of the port (docs/PORTING.md): Laya picks where each request runs. Laya (Apache-2.0) is a non-autoregressive decision model: a ModernBERT-style encoder plus an RLCD… |
| Tokenizers | The engine reads any model's tokenizer.json through Hugging Face tokenizers, the Rust library behind the tokenizers npm and PyPI packages, at the same release as those packages. |
| Linux kernel | The engine's NPU driver (amdxdna) and GPU driver (amdgpu) come from the kernel, so the kernel is pinned like every other dependency: thirdparty/linux is upstream torvalds/linux at… |
Project
| Page | What it covers |
|---|
| Porting map | This repository is the working 1bit-MONSTER engine, ported without its history. Each step below is one PR (or a short series) that builds and runs on Strix Halo before the next… |