https://terms-en.ai-term-hub.com/en/tags/memory-management/Recent content in Memory Management on English AI Terms DictionaryHugoen-usSat, 18 Jul 2026 11:44:44 +0000https://terms-en.ai-term-hub.com/en/terms/pagedattention/Sat, 18 Jul 2026 10:10:06 +0000https://terms-en.ai-term-hub.com/en/terms/pagedattention/<h2 id="definition">Definition</h2> <p>PagedAttention is a technique introduced by the vLLM project to improve the efficiency of Large Language Model inference. It addresses the fragmentation and overhead issues in managing the KV cache, which stores attention states for generating tokens. By treating the KV cache like virtual memory pages, PagedAttention allows for dynamic allocation and sharing of memory blocks between sequences. This results in significant reductions in memory waste and enables higher batch sizes and throughput without requiring hardware changes.</p>