Let me know if a similar recipe for Hadoop works with Whirr 0.7.0 - it seems like the trunk is a bit broken now.
-- Andrei Savu On Wed, Jan 4, 2012 at 10:30 PM, David Dundas <[email protected]> wrote: > Ok that recipe works. It also looks like something in my original > whir.instance-templates > > May have been causing an error as well. > > Thanks! > > On Wed, Jan 4, 2012 at 10:48 AM, Andrei Savu <[email protected]>wrote: > >> I have done some testing and I have been able to start a fully functional >> ZooKeeper cluster from a VM also running inside the Amazon Cloud using a >> recipe like this: >> >> whirr.cluster-name=zookeeper-asavu-from-ec2 >> whirr.instance-templates=3 zookeeper >> whirr.cluster-user=asavu >> whirr.provider=aws-ec2 >> whirr.identity=${env:AWS_ACCESS_KEY_ID} >> whirr.credential=${env:AWS_SECRET_ACCESS_KEY} >> >> Hadoop fails with the following exception but this is unrelated to this >> discussion: >> >> Authorizing firewall ingress to [us-east-1/i-e2187780] on ports [8020, >> 8021] for [184.72.197.142/32] >> Unable to start the cluster. Terminating all nodes. >> java.lang.NullPointerException >> at >> org.apache.whirr.service.hadoop.HadoopConfigurationBuilder.buildMapReduceConfiguration(HadoopConfigurationBuilder.java:113) >> at >> org.apache.whirr.service.hadoop.HadoopConfigurationBuilder.buildMapReduce(HadoopConfigurationBuilder.java:71) >> at >> org.apache.whirr.service.hadoop.HadoopClusterActionHandler.createHadoopConfigFiles(HadoopClusterActionHandler.java:102) >> at >> org.apache.whirr.service.hadoop.HadoopClusterActionHandler.beforeConfigure(HadoopClusterActionHandler.java:88) >> at >> org.apache.whirr.service.ClusterActionHandlerSupport.beforeAction(ClusterActionHandlerSupport.java:53) >> at >> org.apache.whirr.actions.ScriptBasedClusterAction.execute(ScriptBasedClusterAction.java:100) >> at >> org.apache.whirr.ClusterController.configureServices(ClusterController.java:148) >> at >> org.apache.whirr.ClusterController.launchCluster(ClusterController.java:110) >> at >> org.apache.whirr.cli.command.LaunchClusterCommand.run(LaunchClusterCommand.java:63) >> at org.apache.whirr.cli.Main.run(Main.java:64) >> at org.apache.whirr.cli.Main.main(Main.java:97) >> >> I am now running the integration tests on my machine to make sure that >> everything still works fine. >> >> -- Andrei Savu / andreisavu.ro >> >> On Wed, Jan 4, 2012 at 12:23 PM, Andrei Savu <[email protected]>wrote: >> >>> I will give it a try now and get back to you with more feedback. >>> >>> >>> On Wed, Jan 4, 2012 at 2:22 AM, David Dundas <[email protected]>wrote: >>> >>>> Also just added whir as the cluster-user, and it looks like I'm still >>>> getting the same errors. >>>> >>>> >>>> On Tue, Jan 3, 2012 at 7:21 PM, David Dundas <[email protected]>wrote: >>>> >>>>> Oh I understand now. I definitely misread your post. Ok then what is >>>>> the difference between >>>>> whirr.cluster-user and whirr.login-user? >>>>> >>>>> Do I need to create ssh key for the cluster user as well? >>>>> >>>>> Should I run the command as the cluster user or the command user? >>>>> >>>>> Can the login-user and cluster-user be the same? >>>>> >>>>> Thanks! >>>>> >>>>> On Tue, Jan 3, 2012 at 7:04 PM, Andrei Savu <[email protected]>wrote: >>>>> >>>>>> >>>>>> On Wed, Jan 4, 2012 at 1:57 AM, David Dundas <[email protected]>wrote: >>>>>> >>>>>>> * have you added something like whirr.cluster-user=whirr to >>>>>>> hadoop.properties? >>>>>>> >>> No I havent - should I be? >>>>>>> >>>>>> >>>>>> Yes!! Whirr is trying to create the ec2-user on the remote machine >>>>>> and it fails because that user already exists. >>>>>> >>>>>> I have created the following two related JIRA issues: >>>>>> * https://issues.apache.org/jira/browse/WHIRR-463 >>>>>> * https://issues.apache.org/jira/browse/WHIRR-464 >>>>>> >>>>>> As I said already in a comment my advice is to add: >>>>>> >>>>>> whirr.cluster-user=whirr (to hadoop.properties) >>>>>> >>>>>> and you should be able to login using something like: >>>>>> >>>>>> ssh -i ~/.ssh/id_rsa_whirr whirr@remote-host-ip >>>>>> >>>>>> -- Andrei >>>>>> >>>>>> >>>>> >>>>> >>>>> >>> >> > >
