The Work a Verifier Needs: Certified Int8 Output Heads That Match a Production Kernel's Rounding

Abstract

A greedy verifier in speculative decoding needs only the LM head’s winning token, yet the engine scores the whole vocabulary. We ask how little of that work an exact shortcut needs. Exact must mean the engine’s own token, BF16 rounding and tie rule included: for Qwen3.5-4B in SGLang on a GH200, the exact-arithmetic winner differs from it at 0.5% of greedy positions. A certified head bounds every score from an int8 copy of the head and recomputes exactly the few that could win; undecided positions fall back to the engine’s head. We prove when the bounds fix the token, under a tested but unproved error model of the closed-source kernel. The head recomputes about 1.8 entries per decision, and every token matches the engine head’s on the same batch, both in replay and when served beside the stock head. Against tuned stock configurations it raises throughput 1.3–4.6% at one request, but slows the fastest configuration at a few concurrent requests, where most verify calls fall back. Bounds carried from the drafters’ heads skip almost no vocabulary in any variant tested, because draft and target head inputs differ by a median 92–95% of the target’s norm. Attention, recurrent-state and admission changes gain more.