On Mon, Oct 19, 2009 at 7:58 PM, Anty <[email protected]> wrote: > I do not mean to overwrite what is currently there of loadtable.rb. > Considering the following scenario，if i partition a dataset according to > the regions of a existing table in hbase,and generate many HFiles,then we > can move these HFiles to their appropriate directory .do you think this is > practicable? > > Thats something other than what hbase-48 does.
You'd need a partitioner that divided the key ranged based off current region set start and end keys (Not impossible, your partitioner on startup would do a lookup on current set of regions). While this new job ran, the table should be flipped to be read-only so it didn't split and make new regions. Before setting it read-only, it should flush so there is nothing in memstore. Somehow you'd have to get from hbase the latest sequenceid. I'm don't think this knowledge is currently easy to get at. You'd then run your job. You'd write with timestamps that were more modern than anything currently in your table. You'd need a version of loadtable.rb that then took the mapreduce produce and that moved the hfiles to the right region. If you want to work on something that does this Anty, make a new issue and I'll help you out. St.Ack
