While comparing rowwise expert parallelism (EP) with DeepEP v2, we initially observed different expert-GEMM times after matching tensor shapes and routing. The two paths did not, however, contain comparable weights: one used normally initialized BF16 values, while the other consumed storage created with torch.empty(), whose documented contents are uninitialized (PyTorch Contributors 2026j). Once both paths used the same normal initialization and routing, the apparent rowwise advantage disappeared.
The effect reproduced without MoE, communication, routing, or Olmo-core. A minimal Blackwell benchmark measured one PyTorch grouped GEMM and one dense matrix multiplication while changing only the BF16 weight values. Table 34 shows the resulting median per-call GPU times, measured with CUDA events around each call.
| Weight values | Grouped GEMM | Dense GEMM |
|---|---|---|
| Normal random | 31.059 ms | 24.724 ms |
| Uniform random | 31.682 ms | 25.829 ms |
| Random sign, fixed magnitude | 25.979 ms | 24.160 ms |
| All zeros | 18.896 ms | 19.424 ms |
| Constant fill | 20.606 ms | 24.350 ms |
| Uninitialized storage | 30.213 ms | 24.321 ms |
Relative to normally distributed weights, zeros reduced the grouped-GEMM time by 39%, a constant fill reduced it by 34%, and random signs at a fixed magnitude reduced it by 16%. The dense GEMM also took 21% less time with zero weights, but its constant-fill time remained close to the normal case. The effect is therefore neither unique to grouped GEMM nor a simple monotonic law of value entropy. This is a single run with a fixed case order that preserved medians but not individual samples, clocks, power, temperature, or the complete software environment. The percentages characterize this run rather than a general Blackwell performance law.
19.1 Operand-Dependent Hardware Behavior Has Prior Evidence
On an NVIDIA A100, Bhalachandra et al. (2022) measured the same \(16{,}384^3\) DGEMM with fixed and random matrix values. The fixed-valued case reached 19.4 TFLOP/s, whereas the random case reached 18.6 TFLOP/s, about 4% lower, while drawing 67% more average power, close to the GPU’s thermal design power. Their measurements show that operand values affect the physical workload even when matrix dimensions match.
Other studies also find value-dependent power, without always observing a runtime difference. ALUPower measured substantial value-dependent energy variation in GPU arithmetic (Lucas and Juurlink 2016). More recently, Gregersen et al. (2024) varied GEMM value distributions, bit similarity, placement, and sparsity on an A100 GPU and repeated selected experiments on H100, V100, and RTX 6000 GPUs. They found power changes approaching 40%. In their main A100 experiments, run at a matrix size chosen to avoid consistent throttling, however, the average CUTLASS GEMM runtime for a fixed datatype remained consistent to the microsecond level. Together, these studies establish that operand bits can change physical GPU activity. They do not establish that every GPU, shape, or power regime will turn that change into a runtime difference.
19.2 Possible Mechanisms and Their Evidence
Our benchmark did not isolate a hardware mechanism. A plausible explanation is data-dependent switching activity. Similar bit patterns and low Hamming weight can reduce bit transitions through arithmetic and memory datapaths, which changes dynamic power. If a power-limited GPU responds by changing its operating frequency, the same instruction stream can then take a different amount of time. This explanation is consistent with the prior power studies, but our run did not record power or SM clocks and therefore does not test the causal chain.
Separate B300 stress measurements show that power limiting can reduce clocks and throughput. The grouped-GEMM diagnostic of Section 9 (Appendix A.12) drove B300 GPUs against their board power limit with very large expert shapes: the software power-cap limiter was active in effectively every telemetry sample, the median SM clock fell from 1174 to 960 MHz, and median grouped-GEMM throughput fell from 1366 to 1163 TFLOP/s, while per-trial throughput per MHz (each trial’s throughput divided by its mean SM clock) had per-shape medians within 1.16–1.19 TFLOP/s/MHz on the same kernel template. That run varied shape rather than operand values, so it speaks to how this hardware responds rather than to what triggered the response in Table 34. In the stress diagnostic, throughput fell with the lower clock. These measurements do not establish why operand values changed timing in the original run. Table 35 collects the observations relevant to the power-and-clock hypothesis. Matching operands, as described in Section 19.3, controls the observed timing effect without requiring a choice among mechanisms.
| Observation | What was measured | Source |
|---|---|---|
| Operand values change dynamic power | Random values drew 67% more average power than fixed values in a \(16{,}384^3\) DGEMM; value distributions moved GEMM power by up to about 40% | A100, prior work |
| A power-limited GPU lowers its clock | Power-cap limiter active in 99.3% of telemetry samples; SM clock 1174 to 960 MHz | B300, Section 9, Appendix A.12 |
| Lower throughput tracks the lower clock | Throughput 1366 to 1163 TFLOP/s at a nearly constant 1.16–1.19 TFLOP/s per MHz (per-shape medians), same kernel template | B300, Appendix A.12 |
| Operand values change time | Grouped-GEMM time fell by 16–39% and dense-GEMM time by up to 21% at fixed shape, layout, and dtype | B300, Table 34 |
Inline compression is another possible mechanism, but we have no evidence that it was active in this experiment. NVIDIA documents that Hopper can transfer compressible global memory using fewer physical bytes and thereby exceed nominal uncompressed bandwidth (NVIDIA 2026j). This is an opt-in property of an individual allocation, requested through the CUDA Driver API; it is not a general promise that ordinary tensors are compressed (NVIDIA 2026c). Our benchmark used ordinary PyTorch allocations on a Blackwell B300 and never requested the compressible-memory attribute. A later check on the same GPU class with the current PyTorch allocator found that an ordinary tensor, when it could be queried at all, reported no compression attribute; that check describes the current allocator rather than the environment of the original run. The high arithmetic intensity of these GEMMs and the different constant-fill behavior of grouped and dense kernels also argue against assuming a single bandwidth-only explanation.
The public GEMM interfaces do not declare that zero or constant operands may skip mathematical work, so we do not interpret the result as semantic zero skipping. We also did not preserve a kernel-identity or instruction trace that would rule out every library- or kernel-specific effect. The claim this report makes is limited to the observation: operand values changed the time of an otherwise identical GEMM. The measurements do not identify which combination of switching power, clock response, and memory-system behavior produced it.
19.3 Use the Same Initialization in Every Comparison
Operands Are Part of the Benchmark Configuration
For a backend comparison, use identical loaded weights whenever possible. Otherwise, use the same initializer, seed, scale, dtype, and layout on every path. Matching only the GEMM dimensions is insufficient.
This control should be applied after sharding, conversion, or expert placement so that no path retains uninitialized local storage. End-to-end comparisons should load the same checkpoint. Synthetic comparisons should use the same initialization method and random seed. Zeros, constant fills, and torch.empty() are useful diagnostic cases, but they are not substitutes for model-like values. If a communication change appears to alter an otherwise unchanged GEMM, operand statistics, clocks, and power should be checked before attributing the difference to the communication backend.