I have done some testing and I have been able to start a fully functional
ZooKeeper cluster from a VM also running inside the Amazon Cloud using a
recipe like this:
whirr.cluster-name=zookeeper-asavu-from-ec2
whirr.instance-templates=3 zookeeper
whirr.cluster-user=asavu
whirr.provider=aws-ec2
whirr.identity=${env:AWS_ACCESS_KEY_ID}
whirr.credential=${env:AWS_SECRET_ACCESS_KEY}
Hadoop fails with the following exception but this is unrelated to this
discussion:
Authorizing firewall ingress to [us-east-1/i-e2187780] on ports [8020,
8021] for [184.72.197.142/32]
Unable to start the cluster. Terminating all nodes.
java.lang.NullPointerException
at
org.apache.whirr.service.hadoop.HadoopConfigurationBuilder.buildMapReduceConfiguration(HadoopConfigurationBuilder.java:113)
at
org.apache.whirr.service.hadoop.HadoopConfigurationBuilder.buildMapReduce(HadoopConfigurationBuilder.java:71)
at
org.apache.whirr.service.hadoop.HadoopClusterActionHandler.createHadoopConfigFiles(HadoopClusterActionHandler.java:102)
at
org.apache.whirr.service.hadoop.HadoopClusterActionHandler.beforeConfigure(HadoopClusterActionHandler.java:88)
at
org.apache.whirr.service.ClusterActionHandlerSupport.beforeAction(ClusterActionHandlerSupport.java:53)
at
org.apache.whirr.actions.ScriptBasedClusterAction.execute(ScriptBasedClusterAction.java:100)
at
org.apache.whirr.ClusterController.configureServices(ClusterController.java:148)
at
org.apache.whirr.ClusterController.launchCluster(ClusterController.java:110)
at
org.apache.whirr.cli.command.LaunchClusterCommand.run(LaunchClusterCommand.java:63)
at org.apache.whirr.cli.Main.run(Main.java:64)
at org.apache.whirr.cli.Main.main(Main.java:97)
I am now running the integration tests on my machine to make sure that
everything still works fine.
-- Andrei Savu / andreisavu.ro
On Wed, Jan 4, 2012 at 12:23 PM, Andrei Savu <[email protected]> wrote:
> I will give it a try now and get back to you with more feedback.
>
>
> On Wed, Jan 4, 2012 at 2:22 AM, David Dundas <[email protected]> wrote:
>
>> Also just added whir as the cluster-user, and it looks like I'm still
>> getting the same errors.
>>
>>
>> On Tue, Jan 3, 2012 at 7:21 PM, David Dundas <[email protected]> wrote:
>>
>>> Oh I understand now. I definitely misread your post. Ok then what is the
>>> difference between
>>> whirr.cluster-user and whirr.login-user?
>>>
>>> Do I need to create ssh key for the cluster user as well?
>>>
>>> Should I run the command as the cluster user or the command user?
>>>
>>> Can the login-user and cluster-user be the same?
>>>
>>> Thanks!
>>>
>>> On Tue, Jan 3, 2012 at 7:04 PM, Andrei Savu <[email protected]>wrote:
>>>
>>>>
>>>> On Wed, Jan 4, 2012 at 1:57 AM, David Dundas <[email protected]>wrote:
>>>>
>>>>> * have you added something like whirr.cluster-user=whirr to
>>>>> hadoop.properties?
>>>>> >>> No I havent - should I be?
>>>>>
>>>>
>>>> Yes!! Whirr is trying to create the ec2-user on the remote machine and
>>>> it fails because that user already exists.
>>>>
>>>> I have created the following two related JIRA issues:
>>>> * https://issues.apache.org/jira/browse/WHIRR-463
>>>> * https://issues.apache.org/jira/browse/WHIRR-464
>>>>
>>>> As I said already in a comment my advice is to add:
>>>>
>>>> whirr.cluster-user=whirr (to hadoop.properties)
>>>>
>>>> and you should be able to login using something like:
>>>>
>>>> ssh -i ~/.ssh/id_rsa_whirr whirr@remote-host-ip
>>>>
>>>> -- Andrei
>>>>
>>>>
>>>
>>>
>>>
>
