GLM5.2 on AMD MI355X

A July 2026 post by Wafer, an inference provider that sells AMD-based capacity, reporting GLM5.2 throughput on AMD MI355X hardware rented from TensorWave. Every number in it is self-reported by a party with a commercial interest in the conclusion, and none of it has been independently reproduced. The method section is unusually detailed for a vendor post, which is what makes it worth keeping — the engineering account is more useful than the headline.

What they claim

The pricing premise is that an MI355X costs roughly 2.75x less per GPU than a B300 on average, at comparable silicon specs, and that NVIDIA's advantage is software and day-0 model support rather than hardware.

On a 20k input / 1k output workload at a 60% cache hit rate, they report saturation at 2.4 requests per second with 2626 tok/s/node aggregate, TTFT p50 0.81s and p95 2.22s, 100% success, against a defined knee of 5s TTFT or better. They call this 80% of what they measured on a B200 while being over 2x cheaper. Their Blackwell reference figure, quoted later in the same post, is 3192 tok/s/node at 3.0 rps. Note the comparison switches parts: the price ratio is against a B300, the throughput ratio against a B200.

Single stream, following the Artificial Analysis benchmarking methodology at 10k in / 1.5k out, they report 213 tok/s. They concede this doesn't top the AA leaderboard and claim the win on performance per dollar only.

How they got there

Quantization first. They took the bf16 GLM-5.2 weights to MXFP4 using AMD Quark and call the result lossless against z-ai's official FP8 build (glm-5-2-step-change-for-open-agents covers why the model was worth porting). The evals they publish are GSM8K 0.955 ± 0.014 versus 0.965 ± 0.013, GPQA-Diamond 0.9026 ± 0.029 versus 0.9217 ± 0.027, and tau2 macro 0.834 versus 0.819. Two of the three moved down and one up, all inside the stated error bars, so "lossless" is their reading of within-noise rather than a measured equivalence. MXFP4 is the Blackwell-native 4-bit float format described in llm-quantization; the point of interest here is running it on AMD instead.

Framework choice went to sglang. vLLM had no working MXFP4 path for GlmMoeDsa, so the quantized weights bought nothing there, and ATOM's output degraded at long context.

Getting speculative decoding to work took two small fixes, neither of them kernel work. The MTP head keeps its shared expert in bf16 like every other layer, but it's registered under a different module prefix than Quark's un-quantized exclusion list names it — Quark writes model.layers.78.mlp.shared_experts.* while the MTP layer's real prefix is model.decoder.*. sglang's quantization lookup misses, builds that expert as MXFP4, and then crashes at load trying to read a full-width bf16 tensor into a 4-bit slot. They fixed it by duplicating the layer-78 entries in the exclusion list under the prefix sglang actually uses, which they credit with close to a 3x single-stream gain. The second fix: the fused multi-step metadata kernel needed for draft depth 4 or greater includes cuda_runtime.h with no ROCm guard, so deep configurations like the 5/1/6 z-ai suggests were blocked until they added one #ifdef USE_ROCM. With those plus --kv-cache-dtype fp8_e4m3 and --enable-aiter-allreduce-fusion they reached the 213 tok/s figure. See speculative-decoding for what MTP is and why draft depth matters.

Aggregate throughput needed different work, because at 20k input with 60% cache hits the workload is prefill-bound rather than decode-bound. TP8, the configuration tuned for single-stream decode, gave 1461 tok/s/node. Switching to TP4xDP2 got them to 1944 at 2.0 rps. The remaining gap came from GLM-5.2's fp4 MoE silently running on a slow FlyDSL heuristic fallback, because aiter only shipped tuned configs for the a8w8/fp8 path. Tuning MoE kernel selection for GLM's fp4 shapes (model_dim 6144, moe_inter 2048, E=256, topk=8) produced the headline 2626. The routing structure behind those shapes is in mixture-of-experts.

Reading it skeptically

Their conclusion is that they wrote no custom kernels this time, unlike their earlier Qwen3.5 397B work, so state of the art on AMD is now a matter of support rather than software, and "the CUDA moat is eroding in real time." The evidence they present for that is narrower than the claim: two upstream bugs and one kernel-selection tuning pass, on one model, on one node, by a team that does this for a living.

Things the post itself concedes or leaves open. It's single-node only, which they flag. Frontier models ship every few weeks and MI355X support is not day 0, so the porting work described here is a treadmill by their own framing — by the time it's done, the next model is out. And the headline throughput is still 80% of Blackwell on their own numbers, so the entire argument rests on the price ratio holding, which is a market condition rather than a property of the hardware.