On Thu, Jan 21, 2010 at 9:48 AM, Menno Luiten <[email protected]> wrote:
> ...Then I use the KeyValueSortReducer as Reducer, and > limit myself to 1 reducer only, to prevent writing a custom Partitioner. > > This all sounds like you are setting things up properly. > java.io.IOException: Added a key not lexically larger than previous > key=im2.jpgaspectcolor/blue1---some-color/p.995�������, > lastkey=im2.jpgaspectcolor/cyan---some-color/p.995������� > > This is a complaint that comes up out of hfile if keys are not in strictly increasing lexicographical order. > I thought the KeyValueSortReducer would sort the output to prevent these > errors from occuring. Yes. Me too. There is a bug in hbase-0.20.2 KVSR -- hbase-2101 -- but it would not explain what you are seeing, as far as I can tell. > Am I doing something wrong or should I write my > own comparator which compares actual row,column pairs? > > You shouldn't have to. See the comparator passed to the TreeMap used internally in KVSR. It should do the compare properly. I'm not sure whats going wrong. You seem to have it set up right yet keys are going into hfile out of order. Any chance of your digging in to figure what is going awry? Thanks, St.Ack
