Can somebody help me to unsubscribe. Even after unsubscribing I continue to get mails.
-----Original Message----- From: hadoop hive [mailto:hadooph...@gmail.com] Sent: 02 February 2012 12:29 To: common-user@hadoop.apache.org Subject: Re: Reduce > copy at 0.00 MB/s Hey , Can any1 help me with this, i have increases the reduce slowstart to .25 but its still hangs after copy . tell me what else i can change it to make it working fine. regards Vikas Srivastava On Wed, Jan 25, 2012 at 7:45 PM, praveenesh kumar <praveen...@gmail.com>wrote: > Yeah , I am doing it, currently its on 20 %, I guess I have to raise it > more. > Funny thing is, its still happening after map is 100% completed. > when map is completed, it should not wait, right. But I see it still give > same message, for some time. > > Thanks, > Praveenesh > > On Wed, Jan 25, 2012 at 7:29 PM, Harsh J <ha...@cloudera.com> wrote: > > > The copy phase fetches the map outputs. It may hang for a while if > > there are no newly completed map outputs to fetch yet. > > > > You can raise your reducers' slowstart value to have it not spend so > > many cycles waiting but rather start at 80-90% of map completions, > > instead of default 5%. This helps your MR performance overall, if you > > run multiple jobs at a time, as the reduce slots aren't wasted. > > > > On Wed, Jan 25, 2012 at 3:34 PM, praveenesh kumar <praveen...@gmail.com> > > wrote: > > > Hey, > > > > > > Can anyone explain me what is reduce > copy phase in the reducer > section > > ? > > > The (K,List(V)), is passed to the reducer. Is reduce > copy > representing > > > copying of K,List(V) on the reducer from all mappers ? > > > > > > I am monitoring my jobs on the cluster, using Jobtracker url. > > > I am seeing for most of my reducing jobs, something like this : > > > > > > task_201201250352_0001_r_000000 31.05% reduce > copy (395 of 424 at > > 0.00 > > > MB/s) > 25-Jan-2012 03:54:06 > > > > > > task_201201250352_0001_r_000001 30.73% reduce > copy (391 of 424 at > > 0.00 > > > MB/s) > 25-Jan-2012 03:54:06 > > > > > > task_201201250352_0001_r_000002 30.89% reduce > copy (393 of 424 at > > 0.00 > > > MB/s) > 25-Jan-2012 03:54:06 > > > > > > > > > ............................ > > > > > > Can anyone explain me why the speed is 0.00 MB/s. Job is running fine. > > > Is it because this reduce > copy is happening on the same machine. > > > > > > Thanks, > > > Praveenesh > > > > > > > > -- > > Harsh J > > Customer Ops. Engineer, Cloudera > > > ----- No virus found in this message. Checked by AVG - www.avg.com Version: 2012.0.1913 / Virus Database: 2112/4780 - Release Date: 02/01/12
