MXFP and NVFP: Block Scales, Two-Level Scaling, and Quantization Error

The views expressed here are my own and do not represent those of my employer. The experiments are presented to discuss quantization methods and implementations; they do not describe product specifications or roadmaps.

Low-precision floating-point formats are often described only by their element type, such as E2M1 or E2M3. An element alone, however, is not enough to reconstruct the original value. The result also depends on the scale shared by a group of elements:

Here, is the low-precision element and is the block scale. NVFP4 adds a tensor-level scale:

Two formats may use the same 4-bit element type yet produce different errors because their scale formats and scaling schemes differ. This article starts from the public definitions of OCP MX and NVIDIA NVFP4 and focuses on block scales.

Why MXFP4 and NVFP4 Matter

Large-model inference is increasingly limited by memory capacity and bandwidth. Reducing weights from BF16 to 4 bits cuts both storage and weight traffic. Modern GPUs also provide high-throughput matrix instructions for microscaling formats, so low precision is no longer just a compression format—it is a compute format.

The following table shows dense matrix/Tensor Core peak throughput:

GPU	BF16	MXFP6 / FP6	MXFP4 / FP4	NVFP4
AMD Instinct MI355X	2.5 PFLOPS	10.1 PFLOPS	10.1 PFLOPS	Not natively supported
NVIDIA B200	2.25 PFLOPS	4.5 PFLOPS	9 PFLOPS	9 PFLOPS

Peak throughput is not an end-to-end model speedup. Quantization, scale handling, layout transformations, memory access, non-GEMM operators, and model shapes all reduce the realized gain. Still, the direction is clear: peak 4-bit matrix throughput is about four times BF16, while the weight payload drops from 16 bits to 4 bits.

MXFP4 offers a direct hardware path. NVFP4 improves 4-bit fidelity with a finer E4M3 scale, 16-element micro-blocks, and two-level scaling. B200 supports NVFP4 natively, while MI355X natively supports MXFP4 and MXFP6. This article discusses representation and weight-only error rather than converting peak FLOPS into model throughput.

1. Format Definitions

Format	Element	Block scale	Values per scale	Tensor scale	Dequantization
MXFP4	E2M1, 4 bit	E8M0, 8 bit	32	None
MXFP6	E2M3, 6 bit	E8M0, 8 bit	32	None
NVFP4	E2M1, 4 bit	E4M3, 8 bit	16	One FP32 value per tensor

The finite E2M1 values are:

E2M3 still has two exponent bits but uses three mantissa bits. Its maximum finite magnitude is , and its value grid is much denser. MXFP6 therefore has much lower error than MXFP4 in these experiments. That gain comes from the element precision, not from a finer block scale.

2. E8M0 and E4M3 Block Scales

E8M0 has exponent bits but no mantissa:

It covers a wide range, but it has no intermediate values between adjacent powers of two. If the ideal scale is , an amax-covering quantizer must round up to . The extra range carries no additional information and reduces the effective use of the element codebook.

E4M3 has three mantissa bits and can choose a scale closer to the ideal value between adjacent powers of two. The trade-off is a smaller exponent range.

E8M0: wide range, direct implementation, coarse scale grid;

E4M3: finer scale grid, narrower range.

3. Why NVFP4 Uses Two-Level Scaling

NVFP4 is not simply E4M3 × E2M1. Its full representation is:

Quantization first rescales the tensor:

then selects an E4M3 scale for every 16-element block:

Dequantization multiplies back in. The two scales have separate roles: controls the tensor's overall range, while fits each local block.

E4M3 block scales plus E2M1 elements are still not sufficient by themselves. The maximum E2M1 magnitude is 6, so a single block can represent at most:

Regardless of block size, must remain within the finite E4M3 range. A target above the upper bound causes overflow. A target below the lower bound forces the quantizer to use an overly large scale, pushing small weights toward only a few E2M1 codes such as 0 and 0.5. Block scales adapt to local variation, but they cannot move the absolute range of the tensor.

For normal E4M3FN values, the block-scale range is approximately to 448; including subnormals lowers the minimum to . BF16 and FP32 both use eight exponent bits and have roughly the same normal range:

Format	Exponent bits	Mantissa bits	Smallest normal	Maximum finite value
E4M3FN	4	3		448
BF16	8	7
FP32	8	23

BF16 and FP32 mainly differ in mantissa precision, not exponent range. NVFP4 stores the tensor scale in FP32 so that the full tensor is first normalized into a range that E4M3 can cover. E4M3 then fits each 16-element block, and E2M1 represents values within the block. Without the first level, block-scale targets at either boundary waste the precision provided by the E4M3 mantissa.

MX formats usually do not need this tensor scale because E8M0 has a much wider range. Their main block-scale limitation is the power-of-two grid rather than range.

4. Allocating Exponent and Mantissa Bits in the Scale

E8M0 and E4M3 represent two different allocations of an 8-bit scale: E8M0 gives all bits to the exponent, while E4M3 reserves three mantissa bits. Three additional experimental formats isolate this trade-off:

Experimental format	Element	Block scale	Purpose
FP4SE4	E2M1	E4M4	Add one scale mantissa bit
FP4SE5	E2M1	E5M3	Keep a 3-bit mantissa and extend the range
FP4SE6	E2M1	E6M2	Extend the range further with fewer mantissa bits

These formats are used only to study bit allocation; they are not OCP or NVIDIA standards. All use an FP32 tensor scale and a block size of 32.

5. Weight-Only Quantization Error

Weights are quantized and dequantized to . The metrics are:

All results use direct quantization: choose the smallest block scale that covers the block amax, then round each element to the nearest low-precision value. No rotation, SmoothQuant, AWQ, GPTQ, error compensation, or MSE scale search is applied. The goal is to compare data formats and scale designs rather than the best accuracy obtainable with advanced quantization algorithms.

The experiments cover four synthetic tensor distributions, Qwen3.5-4B weights, and Gemma 4 12B weights. The synthetic set contains normal, uniform, exponential, and outlier-mixture distributions, with 131,072 elements and a fixed random seed of 20260707.

NVFP4 denotes the standard 16-element micro-block. NVFP4 B32 is an experimental variant with a 32-element block. Local error is measured over 32-element windows, so each standard NVFP4 measurement window contains two independently scaled blocks. The p10/p50/p90/p99/max values come from the window distribution. The mean is the full-tensor or full-model rel-L2 and is shown separately as an open diamond.

Data	Method	Block scale	Block	p10	p50	mean	p90	p99	max
Synthetic	MXFP6	E8M0	32	2.22	2.66	2.77	3.31	4.21	5.07
Synthetic	FP4SE4	E4M4 + FP32	32	8.72	10.39	10.48	12.05	13.33	15.32
Synthetic	NVFP4	E4M3 + FP32	16	8.59	10.38	10.41	12.02	13.24	15.85
Synthetic	NVFP4 B32	E4M3 + FP32	32	9.02	10.80	10.96	12.58	13.99	16.41
Synthetic	FP4SE5	E5M3 + FP32	32	9.02	10.80	10.96	12.58	13.99	16.41
Synthetic	FP4SE6	E6M2 + FP32	32	9.39	11.38	11.96	14.30	17.04	19.97
Synthetic	MXFP4	E8M0	32	9.30	11.07	12.08	15.57	18.98	22.82
Qwen3.5-4B	MXFP6	E8M0	32	2.30	2.80	2.84	3.37	3.91	7.52
Qwen3.5-4B	FP4SE4	E4M4 + FP32	32	8.46	10.17	10.27	11.95	13.26	17.00
Qwen3.5-4B	NVFP4	E4M3 + FP32	16	8.27	9.94	10.03	11.55	12.96	16.73
Qwen3.5-4B	NVFP4 B32	E4M3 + FP32	32	8.76	10.41	10.55	12.23	13.57	19.58
Qwen3.5-4B	FP4SE5	E5M3 + FP32	32	8.76	10.41	10.55	12.23	13.57	19.58
Qwen3.5-4B	FP4SE6	E6M2 + FP32	32	9.17	11.16	11.32	13.26	15.05	25.48
Qwen3.5-4B	MXFP4	E8M0	32	9.61	11.55	11.80	13.72	15.94	30.07
Gemma 4 12B	MXFP6	E8M0	32	2.30	2.80	2.87	3.41	4.00	8.00
Gemma 4 12B	FP4SE4	E4M4 + FP32	32	8.46	10.17	10.28	11.95	13.26	17.66
Gemma 4 12B	NVFP4	E4M3 + FP32	16	8.27	9.94	10.02	11.55	12.96	17.64
Gemma 4 12B	NVFP4 B32	E4M3 + FP32	32	8.76	10.41	10.57	12.23	13.57	20.92
Gemma 4 12B	FP4SE5	E5M3 + FP32	32	8.76	10.41	10.57	12.23	13.57	21.12
Gemma 4 12B	FP4SE6	E6M2 + FP32	32	9.28	11.29	11.39	13.26	15.22	26.77
Gemma 4 12B	MXFP4	E8M0	32	9.61	11.55	11.88	13.88	16.50	30.70

All values are percentages.

Local error distribution for synthetic tensors and model weights

The box spans p10–p90; the vertical line is p50; the open diamond is the overall mean; the short tick is p99; and the cross is the maximum. Standard NVFP4 selects a scale every 16 elements, while error is still measured over 32-element windows.

The pattern is consistent:

MXFP6 is well below every 4-bit method because E2M3 has a denser element codebook;

with the same E2M1 elements, more scale mantissa bits generally reduce error: FP4SE4 is lowest, E4M3/E5M3 follow, E6M2 comes next, and E8M0 is highest;

FP4SE5 and NVFP4 B32 nearly overlap because both have a 3-bit scale mantissa and the tested data does not reach their codebook boundaries;

standard NVFP4 lowers p50 by about 0.47 percentage points relative to NVFP4 B32, with larger improvements at p90, p99, and max;

smaller blocks reduce the reach of outliers but require more scales. An 8-bit scale costs 0.5 bit per element at block 16 and 0.25 bit per element at block 32.

Synthetic tensors and both model-weight sets show the same direction. Synthetic data illustrates scale resolution and local dynamic range; model weights show that the same effects appear in real parameters.

These values describe weight reconstruction only. They do not directly predict GEMM output error or model accuracy.

6. Conclusion

Element width alone does not define a low-precision floating-point format. MXFP4 and NVFP4 both use E2M1 elements, but their scale designs differ:

MXFP4 uses E8M0 for wide range and power-of-two block scales;

NVFP4 uses E4M3 for finer local scales and an FP32 tensor scale for range;

NVFP4 also reduces the sharing group from 32 elements to 16.

In the weight experiments, standard NVFP4 has lower reconstruction error than MXFP4. Compared with NVFP4 B32, the 16-element micro-block mainly improves the error tail at the cost of additional scale metadata.

A useful format description must answer four questions: What is the element format? What is the scale format? How many elements share a scale? Is there another scaling level? Calling a format merely “4-bit” leaves out most of the design.