{"id":725,"date":"2025-09-18T09:30:00","date_gmt":"2025-09-18T16:30:00","guid":{"rendered":"https:\/\/identia.digital\/lmcache\/en\/?p=725"},"modified":"2025-10-30T14:18:10","modified_gmt":"2025-10-30T21:18:10","slug":"nvidia-dynamo-integrates-lmcache-accelerating-llm-inference","status":"publish","type":"post","link":"https:\/\/identia.digital\/lmcache\/en\/2025\/09\/18\/nvidia-dynamo-integrates-lmcache-accelerating-llm-inference\/","title":{"rendered":"NVIDIA Dynamo integrates LMCache, Accelerating LLM Inference"},"content":{"rendered":"\n

We’re thrilled to announce that Nvidia Dynamo<\/strong><\/a> has integrated LMCache<\/a> as a KV caching layer solution<\/a><\/strong>. This is a big milestone: Dynamo gets a battle-tested caching solution, and LMCache becomes part of a data center-scale inference platform used by many developers worldwide to deploy AI at scale.<\/p>\n\n\n\n

\"image\"\/<\/figure>\n\n\n\n

For comprehensive details about Dynamo’s KV cache optimization capabilities, see the NVIDIA Developer Blog post on reducing KV cache bottlenecks<\/a><\/strong>.<\/p>\n\n\n\n

Why KV Caching Matters<\/strong><\/h2>\n\n\n\n

KV caching is a foundational optimization for modern LLM inference. Instead of recomputing the expensive prefill phase for every new query, KV cache allows reuse of previously computed key\/value pairs. This reuse skips large portions of prefill computation<\/strong>, dramatically reducing end-to-end latency while increasing throughput and efficiency.<\/p>\n\n\n\n

We\u2019ve explored this in detail in earlier posts, such as our Turbocharging LLM Inference blog<\/a>, where we showed how KV cache reuse not only accelerates single-query latency but also enables more efficient multi-turn interactions and higher cluster utilization.<\/p>\n\n\n\n

With Dynamo now supporting LMCache as a caching layer, these benefits become first-class citizens<\/strong> in the Dynamo platform.<\/p>\n\n\n\n

What This Collaboration Delivers<\/strong><\/h2>\n\n\n\n

This collaboration focuses on two technical fronts:<\/p>\n\n\n\n

1. KV Cache Offloading and Reuse<\/strong><\/h3>\n\n\n\n

By default, KV cache is stored in GPU memory, which limits scale and context persistence. With this integration, Dynamo can now offload KV cache to external storage layers<\/strong> using LMCache while maintaining efficient reuse across queries. This integration is available on Dynamo repository: ai-dynamo\/dynamo#2079<\/a><\/p>\n\n\n\n

This combination enables scenarios like:<\/p>\n\n\n\n