Hey devs, A thought just stuck me while I was writing down a more detailed block caching documentation: with HFileV2, the indexes now live in the block cache which means that those who upgrade may all of a sudden get terrible cache hit ratios because of all that memory taken by the indexes. This is somewhat mitigated by the fact that people don't usually need to keep _all_ the index blocks in memory so in the end we're more efficient.
Which brings me to a question: should we set hfile.block.cache.size higher since indexes are now kept in the block cache? Currently it's set to 20%. Looking over my own production machines I see that the storefileIndexSize is around 600-700MB so that's potentially how much more data I'd have to block cache (more likely it's half of that that's really being used actively). What would be a good new default? 25%? 30%? How do we handle those that will be pushed over the BC+memstore size limit because of that change? Thx, J-D
