Re: Shuffle speed?

2009-03-02 Thread hc busy
There are a few things that caused this to happen to me earlier on. Make sure to check that it actually makes progress. Sometimes, slowness is result of negative progress: it gets to say 10% complete on reduce, and then drop back down to 5%...In that case the output can output that line with the s

Re: hadoop mapper 100% but cannot complete?

2008-12-12 Thread hc busy
mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:534) 2008-12-12 12:06:31,107 INFO org.apache.hadoop.mapred.TaskTracker: attempt_200812121139_0001_r_07_0 0.29801327% reduce > copy (135 of 151 at 1.42 MB/s) > On Thu, Dec 11, 2008 at 12:13 PM, hc busy wrote: > And aside from r

Re: hadoop mapper 100% but cannot complete?

2008-12-11 Thread hc busy
with regards to these two problems? On Wed, Dec 10, 2008 at 4:37 PM, hc busy wrote: > Guys, I've just configured a hadoop cluster for the first time, and I'm > running a null map-reduction over the streaming interface. (/bin/cat for > both map and reducer). So I noticed tha

hadoop mapper 100% but cannot complete?

2008-12-10 Thread hc busy
Guys, I've just configured a hadoop cluster for the first time, and I'm running a null map-reduction over the streaming interface. (/bin/cat for both map and reducer). So I noticed that the mapper and reducer complete 100% in the web ui within a reasonable amount of time, but the job does not compl