Re: MRv2 jobs fail when run with more than one slave

07-17 19:13:47,477 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container: Container container_1342570404456_0001_01_02 transitioned from RUNNING to EXITED_WITH_SUCCESS Suggestions of where to look next? Thanks, Trevor On Tue, Jul 17, 2012 at 6:33 PM, Trevor wrote: > Arun, I just verified that I get the same error

Re: MRv2 jobs fail when run with more than one slave

Arun, I just verified that I get the same error with 2.0.0-alpha (official tarball) and 2.0.1-alpha (built from svn). Karthik, thanks for forwarding. Thanks, Trevor On Tue, Jul 17, 2012 at 6:18 PM, Karthik Kambatla wrote: > Forwarding your email to the cdh-user group. > > Thanks

MRv2 jobs fail when run with more than one slave

rary isn't loaded on ARM. The master/client is the same x86 system in both scenarios. All nodes are running Ubuntu 12.04. Thanks for any guidance, Trevor

Re: Idle nodes with terasort and MRv2/YARN (0.23.1)

on the ARM work. Just keeping it building is a bit of work (e.g. HADOOP-8538), but now that I'm getting a handle on x86 performance tuning with MRv2, I should have some ARM results soon. -Trevor On Tue, Jun 5, 2012 at 7:35 AM, Arun C Murthy wrote: > Trevor, > > On May 30, 2012

Re: Idle nodes with terasort and MRv2/YARN (0.23.1)

, but the disk capacity/utilization situation will be more severe. Thanks, Trevor On Tue, May 29, 2012 at 6:21 PM, Arun C Murthy wrote: > What is the minimum container size? i.e. > yarn.scheduler.minimum-allocation-mb. > > I'd bump it up to at least 1G and use the CapacityScheduler

Idle nodes with terasort and MRv2/YARN (0.23.1)

ore running teragen. I've also tried adding delays, but they didn't seem to have any effect, so I don't *think* it's a start-up race issue. Thanks for any advice, Trevor

Re: TestDFSIO job hangs (0.23.1-cdh4b2)

Found the cause: https://issues.apache.org/jira/browse/MAPREDUCE-4250 https://issues.apache.org/jira/browse/HADOOP-8393 hadoop-config.sh and yarn-config.sh determine the correct home and config dirs, but don't export them. Applying the patches to those files fixes the MRAppMaster issue. -T

Re: TestDFSIO job hangs (0.23.1-cdh4b2)

ogram will exit. How can I fix the classpath to include the appropriate JAR (hadoop-mapreduce-client-app-0.23.1-cdh4.0.0b2.jar)? It does appear to be on the nodemanager classpath. Thanks, Trevor On Mon, May 14, 2012 at 3:46 PM, Trevor Robinson wrote: > Would someone please give me some troubl

TestDFSIO job hangs (0.23.1-cdh4b2)

The url to track the job: http://server1:8088/proxy/application_1337025701572_0001/ 12/05/14 15:02:13 INFO mapreduce.Job: Running job: job_1337025701572_0001 <30 minutes pass - no significant CPU or disk activity> Thanks, Trevor

Re: Is there a way to insure that different jobs have the same number of reducers

partitions to 1 bucket and is not split then you can get all the data going to one reducer. Doing it this way means there needs to be some transient property that carries over from the step 1 reducer and through the step 2 mapper. Most cases, I would assume, do not have that property. -Trevor On Wed

Re: Reduce method called same key twice

So after I created the RawComparator for the key it worked as expected. Thanks. -Trevor On Wed, Jun 29, 2011 at 2:47 PM, Aaron Baff wrote: > I dunno, I just know that when I use a separate comparator for my custom > key (does something similar to yours, although 2 or 3 additional sec

Re: Reduce method called same key twice

thing different. -Trevor On Wed, Jun 29, 2011 at 2:07 PM, Aaron Baff wrote: > You probably need to implement a custom comparator that you use as the > grouping comparator that compares the primary key, and then if they are the > same compares the int part of the ke

Reduce method called same key twice

isn't it bucketing? -Trevor