返回首页
arXiv AI··论文与技术

BlockLock: Reproducible LLM Inference with Fixed Batch Geometry

中文摘要

BlockLock 通过固定解码器数值路径并保留共享批处理 prefill,实现了高效且可重现的 LLM 推理,解决了现有方法在效率与一致性之间的权衡。

English Summary

BlockLock ensures reproducible LLM inference by pinning the decoder's numerical path while retaining efficient shared batched prefill, outperforming current batch-invariant or padding-heavy methods.

原文节选

arXiv:2611.06977v1 Announce Type: new Abstract: Reproducible LLM inference requires that a request produces the same output regardless of the other requests in the batch. Existing solutions pay for this in different ways: batch-invariant kernels constrain every operator, and isolating prefill with fixed-shape decode ({our CoRun-style baseline}) gives up shared prefill compute and pads decode. We present BlockLock, which keeps shared batched prefill and pins only the numerical path of the protected request: it fixes the geometry that the request sees, with {two modes: BlockLock-Rows uses fixed-width rows and retains batched decode of all requests; BlockLock-Pack uses terminal-anchored packing with an isolated dummy segment, then extracts the protected request's KV state for target-only decode. Under a stated per-deployment contract, both modes preserve the target's output bitwise when its companions change.} Across five {dense} models (Qwen3-8B/14B, Llama-3.1-8B-Instruct, and Gemma-3-4B/12B-IT) and three single-GPU deployments (NVIDIA A6000, A100, and H100), we evaluate our approach on both tinyBenchmarks and BFCL{; we also include initial tensor-parallel experiments on two NVIDIA A…