There are couple of things here, one is direct byte buffers to put the blocks outside of heap, the other is MMap'ing the blocks directly from the underlying HDFS file.
I think they both make sense. And I'm not sure MapR's solution will be that much better if the latter is implemented in HBase. On Fri, Jul 8, 2011 at 6:26 PM, Ryan Rawson <ryano...@gmail.com> wrote: > The overhead in a byte buffer is the extra integers to keep track of the > mark, position, limit. > > I am not sure that putting the block cache in to heap is the way to go. > Getting faster local dfs reads is important, and if you run hbase on top of > Mapr, these things are taken care of for you. > On Jul 8, 2011 6:20 PM, "Jason Rutherglen" <jason.rutherg...@gmail.com> > wrote: >> Also, it's for a good cause, moving the blocks out of main heap using >> direct byte buffers or some other more native-like facility (if DBB's >> don't work). >> >> On Fri, Jul 8, 2011 at 5:34 PM, Ryan Rawson <ryano...@gmail.com> wrote: >>> Where? Everywhere? An array is 24 bytes, bb is 56 bytes. Also the API >>> is...annoying. >>> On Jul 8, 2011 4:51 PM, "Jason Rutherglen" <jason.rutherg...@gmail.com> >>> wrote: >>>> Is there an open issue for this? How hard will this be? :) >>> >
