Re: When reduce tasks start in MapReduce Streaming?

2013-01-16 Thread Jeff Bean
like a > streaming application (The reduces don't receive data as long as it is > produced by the map tasks)? > > > On 16 January 2013 05:41, Jeff Bean wrote: > > me property. The reduce method is not called until the mappers are done, > and > > the r

Re: When reduce tasks start in MapReduce Streaming?

2013-01-15 Thread Jeff Bean
Hi Pedro, Yes, Hadoop Streaming has the same property. The reduce method is not called until the mappers are done, and the reducers are not scheduled before the threshold set by mapred.reduce.slowstart.completed.maps is reached. On Tue, Jan 15, 2013 at 3:06 PM, Pedro Sá da Costa wrote: > Hi, > >

Re: Profiler in Hadoop MapReduce

2012-12-30 Thread Jeff Bean
Hi Pedro, Have you read the documentation on profiling MapReduce? http://hadoop.apache.org/docs/r0.20.2/mapred_tutorial.html#Profiling Jeff On Sat, Dec 15, 2012 at 7:20 AM, Pedro Sá da Costa wrote: > Hi > > I want to attach jprofiler to Hadoop MapReduce (MR). DO I need to > configure MR to ope

Re: mapred.child.java.opts question

2011-06-14 Thread Jeff Bean
Does your class use GenericOptionsParser (does it implement Tool, and does it call ToolRunner.run(), for example? Jeff On Tue, Jun 14, 2011 at 8:47 AM, Alex Kozlov wrote: > There might be different reasons why this parameter is not passed to the > slave JVM: for example, it might have been decl