David I will give it a try later today. -- Andrei
On Wed, Jan 4, 2012 at 10:50 PM, David Dundas <[email protected]> wrote: > Actually it seems that this command returns the same error > > whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker, 2 > hadoop-datanode+hadoop-tasktracker > > but this one doesn't > > whirr.instance-templates=1 > hadoop-namenode+hadoop-jobtracker+hadoop-datanode+hadoop-tasktracker > > It looks like it's breaking when there are multiple groupings of > instances. > > > > On Wed, Jan 4, 2012 at 3:41 PM, Andrei Savu <[email protected]> wrote: > >> Let me know if a similar recipe for Hadoop works with Whirr 0.7.0 - it >> seems like the trunk is a bit broken now. >> >> -- Andrei Savu >> >> >> On Wed, Jan 4, 2012 at 10:30 PM, David Dundas <[email protected]>wrote: >> >>> Ok that recipe works. It also looks like something in my original >>> whir.instance-templates >>> >>> May have been causing an error as well. >>> >>> Thanks! >>> >>> On Wed, Jan 4, 2012 at 10:48 AM, Andrei Savu <[email protected]>wrote: >>> >>>> I have done some testing and I have been able to start a fully >>>> functional ZooKeeper cluster from a VM also running inside the Amazon Cloud >>>> using a recipe like this: >>>> >>>> whirr.cluster-name=zookeeper-asavu-from-ec2 >>>> whirr.instance-templates=3 zookeeper >>>> whirr.cluster-user=asavu >>>> whirr.provider=aws-ec2 >>>> whirr.identity=${env:AWS_ACCESS_KEY_ID} >>>> whirr.credential=${env:AWS_SECRET_ACCESS_KEY} >>>> >>>> Hadoop fails with the following exception but this is unrelated to this >>>> discussion: >>>> >>>> Authorizing firewall ingress to [us-east-1/i-e2187780] on ports [8020, >>>> 8021] for [184.72.197.142/32] >>>> Unable to start the cluster. Terminating all nodes. >>>> java.lang.NullPointerException >>>> at >>>> org.apache.whirr.service.hadoop.HadoopConfigurationBuilder.buildMapReduceConfiguration(HadoopConfigurationBuilder.java:113) >>>> at >>>> org.apache.whirr.service.hadoop.HadoopConfigurationBuilder.buildMapReduce(HadoopConfigurationBuilder.java:71) >>>> at >>>> org.apache.whirr.service.hadoop.HadoopClusterActionHandler.createHadoopConfigFiles(HadoopClusterActionHandler.java:102) >>>> at >>>> org.apache.whirr.service.hadoop.HadoopClusterActionHandler.beforeConfigure(HadoopClusterActionHandler.java:88) >>>> at >>>> org.apache.whirr.service.ClusterActionHandlerSupport.beforeAction(ClusterActionHandlerSupport.java:53) >>>> at >>>> org.apache.whirr.actions.ScriptBasedClusterAction.execute(ScriptBasedClusterAction.java:100) >>>> at >>>> org.apache.whirr.ClusterController.configureServices(ClusterController.java:148) >>>> at >>>> org.apache.whirr.ClusterController.launchCluster(ClusterController.java:110) >>>> at >>>> org.apache.whirr.cli.command.LaunchClusterCommand.run(LaunchClusterCommand.java:63) >>>> at org.apache.whirr.cli.Main.run(Main.java:64) >>>> at org.apache.whirr.cli.Main.main(Main.java:97) >>>> >>>> I am now running the integration tests on my machine to make sure that >>>> everything still works fine. >>>> >>>> -- Andrei Savu / andreisavu.ro >>>> >>>> On Wed, Jan 4, 2012 at 12:23 PM, Andrei Savu <[email protected]>wrote: >>>> >>>>> I will give it a try now and get back to you with more feedback. >>>>> >>>>> >>>>> On Wed, Jan 4, 2012 at 2:22 AM, David Dundas <[email protected]>wrote: >>>>> >>>>>> Also just added whir as the cluster-user, and it looks like I'm still >>>>>> getting the same errors. >>>>>> >>>>>> >>>>>> On Tue, Jan 3, 2012 at 7:21 PM, David Dundas <[email protected]>wrote: >>>>>> >>>>>>> Oh I understand now. I definitely misread your post. Ok then what is >>>>>>> the difference between >>>>>>> whirr.cluster-user and whirr.login-user? >>>>>>> >>>>>>> Do I need to create ssh key for the cluster user as well? >>>>>>> >>>>>>> Should I run the command as the cluster user or the command user? >>>>>>> >>>>>>> Can the login-user and cluster-user be the same? >>>>>>> >>>>>>> Thanks! >>>>>>> >>>>>>> On Tue, Jan 3, 2012 at 7:04 PM, Andrei Savu >>>>>>> <[email protected]>wrote: >>>>>>> >>>>>>>> >>>>>>>> On Wed, Jan 4, 2012 at 1:57 AM, David Dundas >>>>>>>> <[email protected]>wrote: >>>>>>>> >>>>>>>>> * have you added something like whirr.cluster-user=whirr to >>>>>>>>> hadoop.properties? >>>>>>>>> >>> No I havent - should I be? >>>>>>>>> >>>>>>>> >>>>>>>> Yes!! Whirr is trying to create the ec2-user on the remote machine >>>>>>>> and it fails because that user already exists. >>>>>>>> >>>>>>>> I have created the following two related JIRA issues: >>>>>>>> * https://issues.apache.org/jira/browse/WHIRR-463 >>>>>>>> * https://issues.apache.org/jira/browse/WHIRR-464 >>>>>>>> >>>>>>>> As I said already in a comment my advice is to add: >>>>>>>> >>>>>>>> whirr.cluster-user=whirr (to hadoop.properties) >>>>>>>> >>>>>>>> and you should be able to login using something like: >>>>>>>> >>>>>>>> ssh -i ~/.ssh/id_rsa_whirr whirr@remote-host-ip >>>>>>>> >>>>>>>> -- Andrei >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>> >>>> >>> >>> >> > >
