whirr.instance-templates should be 1 hadoop-namenode+hadoop-jobtracker, 1 hadoop-datanode+hadoop-tasktracker.
This is important because roles are started in order per instance template group. -- Andrei Savu / andreisavu.ro On Wed, Jan 11, 2012 at 1:10 AM, Madhu Ramanna <[email protected]> wrote: > Here you go: > > whirr.cluster-name=test_bs_hadoop_cluster > whirr.instance-templates=1 hadoop-jobtracker+hadoop-namenode,1 > hadoop-datanode+hadoop-tasktracker > whirr.provider=aws-ec2 > whirr.identity=${env:AWS_ACCESS_KEY_ID} > whirr.credential=${env:AWS_SECRET_ACCESS_KEY} > whirr.private-key-file=/root/.ssh/id_rsa_whirr > whirr.public-key-file=/root/.ssh/id_rsa_whirr.pub > whirr.cluster-user=tuser > > Thanks, > Madhu > > From: Andrei Savu <[email protected]> > Reply-To: "[email protected]" <[email protected]> > Date: Tue, 10 Jan 2012 14:59:17 -0800 > To: "[email protected]" <[email protected]> > Subject: Re: [newbie] Unable to setup hadoop cluster on ec2 > > Can you share the .properties file you are using? I want to give it a try > on my machine. > > Thanks, > > -- Andrei Savu / andreisavu.ro > > On Wed, Jan 11, 2012 at 12:55 AM, Madhu Ramanna <[email protected]>wrote: > >> Hello everyone, >> >> I wanted to setup a hadoop cluster on ec2 and downloaded 0.7.0 version. >> >> I setup the relevant files as per the quick start guide. However the >> cluster setup failed with the following exceptions. What could be going >> wrong ? I'm sure (almost) my aws keys are correct >> >> Appreciate your response ! >> >> 2012-01-06 10:38:06,690 DEBUG [jclouds.compute] (user thread 2) << >> images(3758) >> 2012-01-06 10:38:07,210 DEBUG [jclouds.compute] (pool-4-thread-2) << >> images(3758) >> 2012-01-06 10:38:07,212 DEBUG [jclouds.compute] (pool-4-thread-1) >> >> blocking on socket [address=107.21.80.86, port=22] for 600000 MILLISECONDS >> 2012-01-06 10:38:07,212 DEBUG [jclouds.compute] (pool-4-thread-2) >> >> blocking on socket [address=107.22.64.12, port=22] for 600000 MILLISECONDS >> 2012-01-06 10:38:07,292 DEBUG [jclouds.compute] (pool-4-thread-2) << >> socket [address=107.22.64.12, port=22] opened >> 2012-01-06 10:38:10,293 DEBUG [jclouds.compute] (pool-4-thread-1) << >> socket [address=107.21.80.86, port=22] opened >> 2012-01-06 10:38:13,012 INFO >> [org.apache.whirr.actions.ScriptBasedClusterAction] (pool-4-thread-2) >> configure phase script run completed on: us-east-1/i-94bccef6 >> 2012-01-06 10:38:16,003 INFO >> [org.apache.whirr.actions.ScriptBasedClusterAction] (pool-4-thread-1) >> configure phase script run completed on: us-east-1/i-9abccef8 >> 2012-01-06 10:38:16,004 ERROR [org.apache.whirr.ClusterController] (main) >> Unable to start the cluster. Terminating all nodes. >> java.io.IOException: org.jclouds.rest.AuthorizationException: (root: >> <snip> ) error acquiring SSHClient(timeout=60000): Exhausted available >> authentication methods >> at >> org.apache.whirr.actions.ScriptBasedClusterAction.runScripts(ScriptBasedClusterAction.java:215) >> at >> org.apache.whirr.actions.ScriptBasedClusterAction.doAction(ScriptBasedClusterAction.java:128) >> at >> org.apache.whirr.actions.ScriptBasedClusterAction.execute(ScriptBasedClusterAction.java:107) >> at >> org.apache.whirr.ClusterController.launchCluster(ClusterController.java:109) >> at >> org.apache.whirr.cli.command.LaunchClusterCommand.run(LaunchClusterCommand.java:63) >> at org.apache.whirr.cli.Main.run(Main.java:64) >> at org.apache.whirr.cli.Main.main(Main.java:97) >> Caused by: org.jclouds.rest.AuthorizationException: (root:rsa[<snip>]@ >> 107.21.80.86:22) error acquiring SSHClient(timeout=60000): Exhausted >> available authentication methods >> at >> org.jclouds.sshj.SshjSshClient.propagate(SshjSshClient.java:413) >> at org.jclouds.sshj.SshjSshClient.acquire(SshjSshClient.java:244) >> at org.jclouds.sshj.SshjSshClient.connect(SshjSshClient.java:255) >> at >> org.jclouds.compute.callables.RunScriptOnNodeAsInitScriptUsingSsh.call(RunScriptOnNodeAsInitScriptUsingSsh.java:89) >> at >> org.jclouds.compute.internal.BaseComputeService.runScriptOnNode(BaseComputeService.java:612) >> at >> org.apache.whirr.actions.ScriptBasedClusterAction$2.call(ScriptBasedClusterAction.java:190) >> at >> org.apache.whirr.actions.ScriptBasedClusterAction$2.call(ScriptBasedClusterAction.java:178) >> at >> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) >> at java.util.concurrent.FutureTask.run(FutureTask.java:138) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) >> at java.lang.Thread.run(Thread.java:619) >> Caused by: net.schmizz.sshj.userauth.UserAuthException: Exhausted >> available authentication methods >> at >> net.schmizz.sshj.userauth.UserAuthImpl.authenticate(UserAuthImpl.java:114) >> at net.schmizz.sshj.SSHClient.auth(SSHClient.java:204) >> at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:304) >> at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:323) >> at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:199) >> at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:171) >> at org.jclouds.sshj.SshjSshClient.acquire(SshjSshClient.java:220) >> ... 10 more >> Caused by: net.schmizz.sshj.userauth.UserAuthException: publickey auth >> failed >> at >> net.schmizz.sshj.userauth.UserAuthImpl.handle(UserAuthImpl.java:157) >> at >> net.schmizz.sshj.transport.TransportImpl.handle(TransportImpl.java:474) >> at net.schmizz.sshj.transport.Decoder.decode(Decoder.java:127) >> at net.schmizz.sshj.transport.Decoder.received(Decoder.java:195) >> at net.schmizz.sshj.transport.Reader.run(Reader.java:72) >> 2012-01-06 10:38:16,007 INFO [org.apache.whirr.state.ClusterStateStore] >> (main) Unable to load cluster state, assuming it has no running nodes. >> java.io.FileNotFoundException: >> /root/.whirr/test_bs_hadoop_cluster/instances (No such file or directory) >> >> Thanks, >> Madhu >> >> >
