Re: Prefetching Indexes of HFiles

2015-09-25 Thread Anthony Nguyen
Thanks! If I understand it correctly, we need to at least open every HFile once to load root indexes into the cache, right? Another question - I went through Nick's BlockCache 101 and was intrigued by BucketCache / CombinedBlockCache. If I used BucketCache in file mode with SSDs, I see that I can

Prefetching Indexes of HFiles

2015-09-24 Thread Anthony Nguyen
Hi all, Is there a better way to warm HFile indexes other than scanning through my datasets? I did see PREFETCH_BLOCKS_ON_OPEN, but the warning that it "is not a good idea if the data to be preloaded will not fit into the blockcache" makes me wary. Why would this be a bad idea? Thanks!

Re: Prefetching Indexes of HFiles

2015-09-24 Thread Ted Yu
I think the option you mentioned is for data blocks. As for index blocks, please refer to HFile V2 design doc. Also see this: https://issues.apache.org/jira/browse/HBASE -3857?focusedCommentId=13031489=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13031489 Cheers On