Governing the KV Cache: Preventing Timing Side-Channel Leakage in Multi-Tenant LLM Inference arxiv.org
KV cache timing leaks in multi-tenant LLMs, with KVGov adding per-principal salted keys and auditing to blunt the usual cache side channels.
0 comments