6954

NPUDure

Evidence

What NPUDure claims, the measurement each claim rests on, and the report in the repository you can check it against.

Total measurement runs: 421 (418 bench + 3 profile conditions), error rate 0

Every measurement and report link below is imported from docs/experiments/README.md in the NPUDure repository and rendered here. The repository is the source of truth; this page follows it.

Claims and their evidence

ClaimMeasured resultScaleReport
Three nodes scale near-linearly 112.9 / 229.0 / 338.4 inf/s, speedup 3.00×, eff 100%, 0 errors 30 runs S2
Tuning the transport raises throughput and costs scaling efficiency 135.5 / 263.3 / 387.2, 3N 2.86× (95.3%). Absolute +13.3% but eff 98.9→95.3% 36 runs S3.8
The lost efficiency shows up in the tail, not the median Server resources all ruled out. The loss shows up in the tail (p50 flat, p99 +36%). TCP retransmits 3.5× — though the micro-mechanism was not isolated 9 runs S3.9a
Sustained throughput holds under active cooling degradation 1.9%, zero clock downgrades. short-run = sustained 30 runs / 31 min S0
Fanless operation costs an order of magnitude more degradation 11.3%. CPU 2208→816 MHz (king), NPU pinned at 950. The king is 2.4× slower and round-robin still sends it 1/3 30 runs / 32 min S0
Our own scheduler defaults were herding on stale state The policies collapse throughput by 55–58%. Cause: herding on stale heartbeat state — a scheduler bug, found 15 runs S0-C
Fixing the herding bug cut tail latency RR 373.9 / LQ 380.9 / ECT 384.2. Collapse gone, p99 −37%, node latency spread 1.33×→1.00× 12 runs S0-C §8–11
Thermal heterogeneity is not explained by temperature Gate missed at 1.10×. Thermal conditions were identical (86.8 °C) — what sets heterogeneity is not temperature but the spread in CPU downgrade 1 run (aborted) S0-C §17–19
Heterogeneity can be reproduced on demand Yes. Cap 2208→600 moves the spread 1.12×→3.93×. Cap 816 reproduces S0-A (2.4×) to within 6 ms 12 runs S0-D
io_uring was rejected on measurement, not preference Not syscalls — ~1% of transport cost (8% being generous). User time exceeds kernel time (9.37 vs 6.99 ms/req). CPU is 48.9% idle = not a constraint. → S4 io_uring cancelled/shelved 4 conditions S3.9b

The claim wording is ours. Every number and every report link in this table is read from the experiment ledger at build time, so this page cannot drift from the repository.

What this does not prove

  • Not one virtual 18-TOPS NPU
  • Not tensor parallelism
  • Independent requests only — it raises throughput, not per-request latency
  • Measured at 1, 2 and 3 nodes. 4+ nodes not measured
  • Not production-ready — no authentication, no TLS; trusted private networks only
  • Many configurations were repeated 3–4 times; differences under 1% were never used to rank anything
  • Percentiles are aggregated at run level, so absolute p99 values should not be quoted out of context

Stating the bounds is what makes the rest of the table citable.

Go to the raw material

Raw data Experiment ledger Provenance Results