# Neutrino-1 8B

Neutrino-1 8B is Fermion Research's flagship small model, released 2026-07-27 under Apache 2.0. It is a dense 8.19B-parameter decoder-only transformer, a derivative of Alibaba Cloud's Qwen3-8B, and its whole pitch is the container format: all 252 transformer linears are stored in a proprietary ternary-family coded format at one eighth the bits of fp16, staying bit-packed at rest and decoded inside the matrix kernels. Nothing in the decode path is ever materialized as an fp16 or fp32 weight.

The claim built on that is a single artifact serving three very different machines — a datacenter GPU, an Apple laptop, a desktop CPU — with no per-platform conversion step. Everything below comes from Fermion's own model page.

## Geometry

Standard Qwen-shaped, read from the shipped container's header: 36 decoder layers, hidden width 4,096, gated SwiGLU feed-forward at 12,288 with three linears per layer, grouped-query attention 4:1 (32 query heads, 8 key-value heads, head width 128), rotary embeddings at base 1,000,000 applied across the full 128-wide head, RMSNorm at eps 1e-6 plus per-head query/key RMSNorm inside attention, 40,960-token context, 151,936 vocabulary, untied input and output embeddings.

The KV cache runs fp32 by default at 288 KiB per token, so 4k of context costs 1.21 GB and 32k costs 9.66 GB — against 3.88 GB of weights. At long context the cache is the larger number, which is the usual [[kv-cache-sizing]] arithmetic biting harder than normal precisely because the weights got so small.

## Where the bytes went

The 3,875,404,812-byte container splits into four classes:

| Class | Share | Size |
| --- | --- | --- |
| 252 transformer linears (coded lane) | 67.2% | 2,605 MB |
| Two untied int8 embedding tensors | 32.1% | 1,245 MB |
| Per-row metadata (dims, scales, row sums) | 0.6% | 25 MB |
| 145 fp32 normalization tensors | — | 1.2 MB |

Only the linears carry the coded format. The embeddings stay int8 because their rows are read one token at a time rather than multiplied against the whole activation stream, and the norms are too small to be worth coding. A third of the file is vocabulary — a reminder that at this scale, aggressive weight compression runs into a floor set by things that were never the problem. Compare [[llm-quantization]], where the format-versus-bit-width argument plays out on GGUF quants instead.

Fermion publishes an occupancy statistic that is more interesting than the size figures. Across the 6.95B coded weights, 62.63% sit at zero and the remainder splits 18.68% positive against 18.69% negative — sign-balanced to a hundredth of a point with nothing in the training objective asking for it. The balance is not flat with depth: the gate and down feed-forward projections spike to 70–72% zeros in layers 1 through 3, while all four attention projections stay within about a point of 62% at every depth from layer 0 to 35. The early feed-forward blocks shed weights the network does not need; attention keeps a constant code density throughout.

## Distribution: one container, three doors

The download is a 2,559,822,594-byte "coded transport" that expands bit-exactly to the 3.88 GB container — a transport encoding, not a lossy second compression of an fp16 model.

The **pip engine** (`pip install fermion-research`) fetches the container and a platform-matching native binary, with CPU runtimes for macOS arm64 and Linux x86-64 and a bit-exact torch reference underneath. The **GGUF pack** runs through Fermion's own llama.cpp fork on the `fermion-fv5` branch, using an FV5 tensor type. The **MLX pack** is a Python-native Apple runtime with custom Metal kernels; the container is memory-mapped and packed planes are decoded inside the GEMV kernels.

The GGUF door has a real catch that Fermion states outright: stock llama.cpp, Ollama, and LM Studio cannot load the FV5 type. You need their fork, built with `-DGGML_METAL=OFF` for the CPU/CUDA branch or the separate `fermion-fv5-metal` branch for Apple GPU kernels. "One artifact, three doors" is true about the *weights*; the runtimes are three separate builds, and two of the three are Fermion's own. A vendor-specific tensor type is the kind of thing an [[ik-llama-cpp|independent llama.cpp fork]] normally carries too, with the same consequence — the quants only work where the fork works.

The CLI covers `chat`, `serve` (a local OpenAI-compatible endpoint with tool calling and streaming), `generate`, `info` (header + SHA-256 against the manifest), `bench`, `inspect` (weight occupancy, per-layer bytes), and `verify` (certify a draft/target pair produces identical tokens).

## Benchmarks

Run on the shipped container with thinking disabled, on standard public harnesses, July 2026:

| Benchmark | Protocol | Score |
| --- | --- | --- |
| MMLU | 5-shot, all 57 subjects, 14,042 items | 72.1 |
| MMLU-Redux | generative, re-annotated subset | 67.8 |
| IFEval, prompt-strict | generative | 77.2 |
| IFEval, instruction-strict | same run, per-instruction grading | 80.2 |
| IFEval, prompt-loose | same run, loose extraction | 76.3 |
| BFCL v3 | macro over 13 subsets | 68.9 |
| GSM8K, flexible extraction | 0-shot generative, greedy, 256-token cap | 53.4 |
| GSM8K, stated format | same run, answer accepted only in requested form | 51.73 |

Publishing shot count, grading mode, and item count next to every row is unusual and worth the credit — the GSM8K pair in particular shows what format compliance costs (53.4 → 51.73 on the same run), which is the same axis [[structured-output-benchmark]] found separating JSON-Pass from Value-Accuracy by 15–30 points on frontier models.

## Speed, and the draft/verify pairing

Single-stream decode rates, same artifact on every row:

| Platform | Rate |
| --- | --- |
| H100 80 GB, drafted | 763 tok/s |
| H100 80 GB, plain greedy | 396 tok/s |
| Apple M5 MacBook (MLX, optimized) | 33.7 tok/s |
| NVIDIA L4, GGUF + CUDA fork, full offload | 30.7 tok/s (4.68 GiB at 4k) |
| Apple M5, drafted MLX | 25.7 tok/s |
| Apple M5, CPU only, 9 threads | 24.9 tok/s |

Single-stream decode is bound by bytes moved per token, which is the whole argument for the format: a 3.88 GB working set decodes at rates a 16 GB fp16 artifact cannot reach on the same memory system, and the model plus its cache fits on an 8 GB GPU or a 16 GB laptop.

The [[speculative-decoding]] setup pairs the 8B verifier with Neutrino-1 0.6B as drafter. Because both are the same format and run on the same binaries, the draft loads into the verifier's own process — no second deployment, no conversion. Its 328 MB beside the 8B's 3.88 GB is an 8.46% weight surcharge, and 4k of shared context costs 512 KiB per token across the pair (288 KiB on the 8B, 224 KiB on the draft).

A draft token is accepted only when it matches the 8B's own argmax, so the output stream is the plain greedy stream; Fermion reports 27,648 consecutive tokens with zero divergences. The speedup is therefore pure draft-acceptance physics and is quoted per prompt class over the 396 tok/s plain rate: counting and lists 763 tok/s (×1.93), factual short answers 613 (×1.55), prose continuation 532 (×1.34), conversational explanation 447 (×1.13), code 426 (×1.07). On counting prompts the full six-token draft is accepted every pass, about seven tokens emitted per 8B forward; on factual prompts acceptance holds at 96.5%. A dynamic controller sizes the draft per class, which is why even code clears the plain rate.

The same pairing works on a laptop: on a 16 GB M5 both models load into one MLX process under a 6 GiB cap and peak at 4.3 GiB together, 0.53 GiB of that the draft, with 6 of 6 prompts token-identical with drafting on and off and 25.71 tok/s drafted against 22.00 plain at 0.744 acceptance. Getting a draft/verify pair inside a laptop memory budget is the part that is hard to do with two separately-quantized models.

## Reading it skeptically

This is a vendor product page, and it reads like one — every number is Fermion's own, on Fermion's runtimes, and the coded format is proprietary and undocumented beyond its effects. What is unusually good is that the numbers are stated against the shipped artifact rather than a research checkpoint, with protocol attached, and that weights, native binaries, GGUF pack, and MLX pack ship in one public repository so no piece can drift out of sync with another. Apache 2.0 with no access request and no acceptance form, inherited from Qwen3-8B; the pip package is Apache-2.0 and the llama.cpp fork is MIT.

An 8B at 72.1 MMLU is not competing with [[kimi-k3]] or any frontier model, and Fermion doesn't claim it is. It is the far end of the same axis: where [[deltafin]] spends 14.6 seconds a token to run 2.8T parameters on one Mac, Neutrino spends 3.88 GB to put a competent small model everywhere at once. The realistic use is the one [[local-ai-is-not-opus]] argues for — bounded, well-scoped local work where fixed cost and data locality matter more than peak capability.