The KV cache thing is a noise. Firstly this is a paper and it was publish about a year ago. But what caused the sell off is a Google blog post. See something similar? Isn't this like the deep seek thingy? The market is pricing a promising tech paper as it would be deployed tomorrow morning. Also, the saved memory may be used to offer more cap like say, more concurrent session or bigger context window? Then it wouldn't shrink traditional memory demand but may in fact boost it further? It's a sell on new move again.