archive

COMPUTER ORGANIZATION AND DESIGN MIPS EDITION : the hardware/software interface

Ask the publishers to restore access to 500,000+ books. An icon used to represent a menu that can be toggled by interacting with this icon. A line drawing of the Internet Archive headquarters building ...

Nvidia shrinks LLM memory 20x without changing model weights

Nvidia's KV Cache Transform Coding (KVTC) compresses LLM key-value cache by 20x without model changes, cutting GPU memory costs and time-to-first-token by up to 8x for multi-turn AI applications.
IEEE

Efficient KV Cache Spillover Management on Memory-Constrained GPU for LLM Inference

Abstract: The rapid growth of model parameters presents a significant challenge when deploying large generative models on GPU. Existing LLM runtime memory management solutions tend to maximize batch ...
IEEE

Effective Memory Management and Sparse Aware Cache for Row-Wise Sparse CNNs

Abstract: Convolutional Neural Networks (CNNs) are used in several image processing tasks like image recognition and object localization. For edge applications such as drones and autonomous vehicles, ...
NPR

There's a shortage of RAM (computer memory). How is this affecting the industry?

There's a RAM shortage at the moment. RAM, as in random access memory. The memory computer keeps immediately at hand, so it can perform tasks quickly. How can that be? Well, as with so much these days ...