More FP4-local testing with GLM-5.3
Trying Nvidia's next local optimized model
October 17, 2026
In addition to the new QAD quant of Qwen I looked at yesterday, Nvidia has released a GLM-5.3 NVFP4 quant designed for 2x Sparks as part of their new local optimized collection.
As a quick recap, Nvidia is using quantization-aware distillation on these. This one is a little beefer than Qwen, requiring two Sparks or two RTX 6000 Pros, with 1.8 and 2M token K/V respectively. They're using a new lossless compression on the NVFP4 scale factors, which requires the Blackwell specific b12x kernel. We give up expert parallelization but they tend to run faster than regular NVFP4 quants otherwise.
The bad first, as expected GLM-5.3 continues to be slower than Qwen 3.8 Flash Next. On a pair of DGX Sparks this is possibly too slow depending on your use case, but likely to be plenty fast on a pair of RTX 6000 Pros. Nvidia's recipe uses K=5 for 2x Sparks and K=3 for 2x RTX 6000s. In my testing K=3 was always faster on Sparks, with the 4th and 5th token rarely being accepted. For editing, K=5 and K=3 is basically equal, but for reasoning at max effort K=5 is about 20% slower.
Single-user decode, tok/s (500 tokens, median of 3, 1K and 128K context):
| ctx | task | GLM FP4-Local, effort low / max | GLM E224, low / max | Qwen FP4-Local (live), thinking off / on | Qwen config C, off / on |
|---|---|---|---|---|---|
| 1K | code | 38.9 / 38.5 | 29.1 / 28.1 | 64.0 / 50.0 | 66.3 / 56.4 |
| 1K | code_ts | 41.9 / 33.3 | 31.0 / 26.8 | 64.0 / 46.5 | 68.0 / 49.2 |
| 1K | edit | 45.1 / 37.9 | 31.6 / 29.7 | 66.8 / 57.4 | 70.1 / 54.8 |
| 128K | code | 40.6 / 35.1 | 29.5 / 27.1 | 61.7 / 48.5 | 65.2 / 51.3 |
| 128K | code_ts | 42.6 / 33.3 | 31.4 / 26.6 | 62.9 / 43.6 | 65.1 / 48.2 |
| 128K | edit | 37.0 / 42.4 | 27.1 / 29.0 | 60.4 / 51.4 | 67.6 / 52.5 |
| mean | 41.0 / 36.8 | 30.0 / 27.9 | 63.3 / 49.5 | 67.0 / 52.1 |
Aggregate under concurrency, tok/s (recipe bench, thinking off / effort low):
| Streams | GLM FP4-Local | GLM E224 | Qwen FP4-Local (live) | Qwen config C |
|---|---|---|---|---|
| 1 | 35.3 | 26.6 | 56.5 | 58.3 |
| 2 | 48.4 | 43.2 | 86.9 | 96.1 |
| 3 | 51.5 | 59.5 | 115.5 | 137.6 |
| 4 | 75.3 | 70.6 | 137.9 | 174.5 |
| 5 | 73.0 | 70.7 | 155.7 | 197.7 |
| 6 | 93.7 | 86.7 | 186.9 | 228.2 |
Cold prefill, tok/s (unique prompt; targets 2K / 8K / 32K / 64K):
| GLM FP4-Local | GLM E224 | Qwen FP4-Local | Qwen config C |
|---|---|---|---|
| 1,657 / 2,049 / 2,170 / 2,164 | 1,479 / 1,860 / 1,950 / 1,987 | 2,517 / 2,723 / 2,728 / 2,672 | 1,109 / 2,344 / 3,031 / 2,989 |
The good, it's the fastest GLM 5.3 yet and GLM tokens tend to be a little longer than Qwen tokens so this isn't truly apples to oranges.
The great, it's extremely smart. Smarter than the pruned GLM-5.3 quants I tested earlier this week. Nvidia's QAD is really delivering with these models.