I get the whole "security is a good thing" thing, but could someone give me
a description as to why when whirr configures hadoop it sets up the ssh
proxy to disallow all coms to the data / task nodes except via the name node
over the proxy? If I'm running on EC2, wont correctly setting up security
The proxy is not used for security (which would be better provided by
a firewall), but to make the datanode addresses resolve correctly for
the client. Without the proxy the datanodes return their internal
addresses which are not routable by the client (which runs in an
external network typically).
Ok, that makes sense. Thanks for the clarification. It
is definitely unwieldy when trying to integrate whirr's API into another API
to wrap spinning up hadoop clusters, and getting it to work without any
manual steps.
On Tue, Jun 14, 2011 at 5:13 PM, Tom White wrote:
> The proxy is not used f
On Wed, Jun 15, 2011 at 10:18 AM, John Conwell wrote:
> Ok, that makes sense. Thanks for the clarification. It
> is definitely unwieldy when trying to integrate whirr's API into another API
> to wrap spinning up hadoop clusters, and getting it to work without any
> manual steps.
Agreed, but it
oh cool. Thanks for the pointer
On Wed, Jun 15, 2011 at 10:28 AM, Tom White wrote:
> On Wed, Jun 15, 2011 at 10:18 AM, John Conwell wrote:
> > Ok, that makes sense. Thanks for the clarification. It
> > is definitely unwieldy when trying to integrate whirr's API into another
> API
> > to wrap
Also the current trunk has an examples maven submodule. That code is mostly
extracted from tests.
On Jun 15, 2011 8:32 PM, "John Conwell" wrote:
> oh cool. Thanks for the pointer
>
> On Wed, Jun 15, 2011 at 10:28 AM, Tom White wrote:
>
>> On Wed, Jun 15, 2011 at 10:18 AM, John Conwell wrote:
>>
oh man. I didnt know there was a HadoopProxy class that actually had start
and stop methods. I was starting it via Runtime.getRuntime().exec(). Thats
so much nicer.
On Wed, Jun 15, 2011 at 10:41 AM, Andrei Savu wrote:
> Also the current trunk has an examples maven submodule. That code is most
