LMCache – Building the foundation of AI memory tensor with KV Cache Infrastructure

LMCache – Building the foundation of AI memory tensor with KV Cache Infrastructure is a KV cache manager that can distribute the cache across GPU memory and SSDs and also helps when a model runs distributed across multiple machines. I primarily use it myself as an offloader for KV cache to disk for Qwen3.6 27B, because it has a slightly higher memory requirement for KV cache than the 35B model.