Re: Idle nodes with terasort and MRv2/YARN (0.23.1)

2012-05-30 Thread Trevor Robinson
se the latter). > > Also, FYI, currently the CS makes the tradeoff that node-locality is almost > same as rack-locality and hence you might see maps not spread out for > terasort. I'll fix that one soon. > > hth, > Arun > > On May 29, 2012, at 2:33 PM, Trevor Robinson

Idle nodes with terasort and MRv2/YARN (0.23.1)

2012-05-29 Thread Trevor Robinson
Hello, I'm trying to tune terasort on a small cluster (4 identical slave nodes w/ 4 disks and 16GB RAM each), but I'm having problems with very uneven load. For teragen, I specify 24 mappers, but for some reason, only 2 nodes out of 4 run them all, even though the web UI (for both YARN and HDFS)

Re: TestDFSIO job hangs (0.23.1-cdh4b2)

2012-05-14 Thread Trevor Robinson
revor On Mon, May 14, 2012 at 5:40 PM, Trevor Robinson wrote: > I figured out why this was hung in the single node case: The node > manager was failing to start because the resource manager was already > listening on the same port. I had followed Cloudera's example YARN > setup,

Re: TestDFSIO job hangs (0.23.1-cdh4b2)

2012-05-14 Thread Trevor Robinson
ogram will exit. How can I fix the classpath to include the appropriate JAR (hadoop-mapreduce-client-app-0.23.1-cdh4.0.0b2.jar)? It does appear to be on the nodemanager classpath. Thanks, Trevor On Mon, May 14, 2012 at 3:46 PM, Trevor Robinson wrote: > Would someone please give me some troubl

TestDFSIO job hangs (0.23.1-cdh4b2)

2012-05-14 Thread Trevor Robinson
Would someone please give me some troubleshooting tips for TestDFSIO hanging on a new 0.23.1-cdh4b2 cluster? I've tried both a 5-machine cluster and just running everything on a single node. It's my first time configuring YARN, so maybe I've misconfigured something. I don't see anything suspicious