MemDecay: Region-Aware KV Cache Eviction for Efficient LLM Agent Inference

MemDecay is a new memory management technique that optimizes LLM agent inference by using region-aware KV cache eviction based on semantic structure.

Firefly links to the original publisher. The summary above is AI-generated for orientation and may differ from the source. The “current rank score” decays over time so newer significant stories surface first.