Entropy-governed dynamic precision routing and speculative verification for energy-constrained large language model inference on modern datacenter and edge accelerators.
Per-token energy reduced from 6.40 J down to 3.41 J via speculative scouting.
Memory bandwidth cut per projection layer via custom Triton INT4 GEMM kernel.
Sub-byte High-Gear duty cycle achieved on structured code and reasoning tokens.
Zero-stall on-chip SRAM dequantization on Blackwell Tensor Cores.
Decoupling wall-clock execution time from deterministic silicon precision without paying the software emulation tax.
Calculates instantaneous logit uncertainty in real time (Ht = −∑ pi log2 pi). Routine syntax and predictable tokens are routed to sub-byte execution, reserving full FP16 precision for complex reasoning transitions.
Prevents pipeline thrashing and "gear hunting" across sentence boundaries via a dual-threshold deadband (θlow, θhigh, h) and smoothed moving entropy window (W = 3).
Pairs ultra-lightweight draft scouts (1B / INT4) with target verifiers (8B–70B) in a unified VRAM pool. Achieves 50.4%+ draft acceptance, validating multi-token blocks in a single memory fetch.
Estimate enterprise power grid relief, carbon reductions, and annual cost savings using verified SDSIE metrics.
Permanent DOI minted via CERN / Zenodo repository.
@software{jacklin2026sdsie,
author = {Zanno Jacklin},
title = {{SDSIE: Entropy-Governed Dynamic Precision and Speculative
Verification for Energy-Constrained Inference on Modern Accelerators}},
month = aug,
year = 2026,
publisher = {Zenodo},
version = {v1.1.0},
doi = {10.5281/zenodo.21499379},
url = {https://doi.org/10.5281/zenodo.21499379}
}