Hi all,
When I start a hadoop cluster with overridden properties using
0.5-incubating I only get
a hadoop-site.xml and no mapred-site.xml with the additional
properties. I am running Ubuntu 11.4
I am using the following test setup:
whirr.service-name=hadoop
whirr.cluster-name=test-cluster
whirr.
You could also try to use computeService.runScriptOnNodesMatching and
upload the file using an AppendFile jclouds statement together with
the credentials from the cluster spec file.
This approach is similar to what RunScriptCommand is doing.
-- Andrei Savu / andreisavu.ro
On Mon, Jun 13, 2011 at
I've started a cluster using the same recipe and everything looks fine to me.
> When I start a hadoop cluster with overridden properties using
> 0.5-incubating I only get a hadoop-site.xml and no mapred-site.xml
> with the additional properties.
On the local machine or on the amazon cloud?
If yo
I get the whole "security is a good thing" thing, but could someone give me
a description as to why when whirr configures hadoop it sets up the ssh
proxy to disallow all coms to the data / task nodes except via the name node
over the proxy? If I'm running on EC2, wont correctly setting up security
So as an FYI, I just tested using the whirr API to start multiple clusters
at the same time using Futures, and it (seems to) works great. really cuts
down on the time to ramp up a set of clusters (like 4 or more). Yay
On Fri, Jun 10, 2011 at 11:38 AM, Andrei Savu wrote:
> John,
>
> I don't th
The proxy is not used for security (which would be better provided by
a firewall), but to make the datanode addresses resolve correctly for
the client. Without the proxy the datanodes return their internal
addresses which are not routable by the client (which runs in an
external network typically).
On Tue, Jun 14, 2011 at 3:46 PM, John Conwell wrote:
> So as an FYI, I just tested using the whirr API to start multiple clusters
> at the same time using Futures, and it (seems to) works great. really cuts
> down on the time to ramp up a set of clusters (like 4 or more). Yay
Great. This would