Thanks for writing back to the list... I was wondering. St.Ack
On Thu, Jan 21, 2010 at 1:29 PM, Menno Luiten <[email protected]> wrote: > Turns out to be a major screw-up on my behalf :) > > Passed line number as key instead of actual row key, ofcourse resulting > in reducing and ordering 1 line at a time. Woops.. > > It's working fine now :) thanks for your time.. > Menno > > On do, 2010-01-21 at 20:35 +0100, Menno Luiten wrote: >> Manually added the "kv.clone()"-patch and it did not solve the issue, >> unfortunately. >> >> Will most certainly dig deeper as to why this is happening and will post >> the results here or in JIRA. >> >> Thanks >> >> On do, 2010-01-21 at 10:47 -0800, stack wrote: >> > On Thu, Jan 21, 2010 at 9:48 AM, Menno Luiten <[email protected]> wrote: >> > >> > > ...Then I use the KeyValueSortReducer as Reducer, and >> > > limit myself to 1 reducer only, to prevent writing a custom Partitioner. >> > > >> > > >> > This all sounds like you are setting things up properly. >> > >> > >> > >> > > java.io.IOException: Added a key not lexically larger than previous >> > > key=im2.jpgaspectcolor/blue1---some-color/p.995�������, >> > > lastkey=im2.jpgaspectcolor/cyan---some-color/p.995������� >> > > >> > > This is a complaint that comes up out of hfile if keys are not in >> > > strictly >> > increasing lexicographical order. >> > >> > >> > >> > > I thought the KeyValueSortReducer would sort the output to prevent these >> > > errors from occuring. >> > >> > >> > Yes. Me too. There is a bug in hbase-0.20.2 KVSR -- hbase-2101 -- but it >> > would not explain what you are seeing, as far as I can tell. >> > >> > >> > > Am I doing something wrong or should I write my >> > > own comparator which compares actual row,column pairs? >> > > >> > > You shouldn't have to. See the comparator passed to the TreeMap used >> > internally in KVSR. It should do the compare properly. >> > >> > I'm not sure whats going wrong. You seem to have it set up right yet keys >> > are going into hfile out of order. Any chance of your digging in to figure >> > what is going awry? >> > >> > Thanks, >> > St.Ack >> >> > > >
