Heykuki News
Top
New
Best
Ask
Show
Jobs
Toggle theme
Login
Top
New
Best
Ask
Show
Jobs
Kvcached: Virtualized, elastic KV cache for LLM serving on shared GPUs
notion.so
69 points
Jrxing
8 months ago
https://github.com/ovg-project/kvcached
13 comments
Loading...
Kvcached: Virtualized, elastic KV cache for LLM serving on shared GPUs | Heykuki News