Cross-request KV cache exposure in the GLM-5.3 Workers AI model, and our remediation with SGLang
Curated from Cloudflare Blog
If you run large language models in production, this post is a necessary read. It details a subtle but severe isolation failure where a specific model configuration triggered a backend bug, leaking key-value cache data between unrelated user sessions. For SREs, the takeaway is not just about the fix, but the failure mode itself: infrastructure bugs can bypass logical access controls if state management is flawed. The article highlights the importance of treating model-specific parameters as security-critical inputs, not just performance tuning knobs. You need to verify that your inference engine properly isolates state per request, especially when using newer frameworks or custom model configurations. Takeaway: Audit your inference stack’s state isolation guarantees whenever you deploy new model versions, assuming that a simple configuration change might expose hidden backend vulnerabilities.
A configuration mismatch in GLM 5.3 on Workers AI triggered an SGLang bug that exposed KV cache pages across requests. Here is how we addressed the issue.
— Cloudflare Blog