Tom White wrote:
Hi Joydeep,
The problem you are hitting may be because port 50001 isn't open,
whereas from within the cluster any node may talk to any other node
(because the security groups are set up to do this).
However I'm not sure this is a good approach. Configuring Hadoop to
use public
Philip Zeyliger wrote:
You could use ssh to set up a SOCKS proxy between your machine and
ec2, and setup org.apache.hadoop.net.SocksSocketFactory to be the
socket factory.
http://www.cloudera.com/blog/2008/12/03/securing-a-hadoop-cluster-through-a-gateway/
has more information.
very useful wri
to follow up on the
issues uncovered here.
-Original Message-
From: Tom White [mailto:t...@cloudera.com]
Sent: Thursday, May 14, 2009 7:07 AM
To: core-user@hadoop.apache.org
Subject: Re: public IP for datanode on EC2
Yes, you're absolutely right.
Tom
On Thu, May 14, 2009 at 2:19
-Original Message-
> From: Tom White [mailto:t...@cloudera.com]
> Sent: Thursday, May 14, 2009 5:58 AM
> To: core-user@hadoop.apache.org
> Subject: Re: public IP for datanode on EC2
>
> Hi Joydeep,
>
> The problem you are hitting may be because port 50001 is
ions from nodes within the ec2 cluster work
> just fine. I looked at the job.xml files of jobs submitted locally and
> remotely and don't see any relevant differences.
>
> Totally foxed now.
>
> Joydeep
>
> -Original Message-----
> From: Joydeep Sen Sarma [mailto:j
rk
> just fine. I looked at the job.xml files of jobs submitted locally and
> remotely and don't see any relevant differences.
>
> Totally foxed now.
>
> Joydeep
>
> -Original Message-
> From: Joydeep Sen Sarma [mailto:jssa...@facebook.com]
> Sent: Wednesd
d now.
Joydeep
-Original Message-
From: Joydeep Sen Sarma [mailto:jssa...@facebook.com]
Sent: Wednesday, May 13, 2009 9:38 PM
To: core-user@hadoop.apache.org
Cc: Tom White
Subject: RE: public IP for datanode on EC2
Thanks Philip. Very helpful (and great blog post)! This seems to make basi
M
To: core-user@hadoop.apache.org
Subject: Re: public IP for datanode on EC2
On Tue, May 12, 2009 at 9:11 PM, Joydeep Sen Sarma wrote:
> (raking up real old thread)
>
> After struggling with this issue for sometime now - it seems that accessing
> hdfs on ec2 from outside ec2 is not possible.
>
On Tue, May 12, 2009 at 9:11 PM, Joydeep Sen Sarma wrote:
> (raking up real old thread)
>
> After struggling with this issue for sometime now - it seems that accessing
> hdfs on ec2 from outside ec2 is not possible.
>
> This is primarily because of
> https://issues.apache.org/jira/browse/HADOOP-
e.org
Subject: Re: public IP for datanode on EC2
> I think most people try to avoid allowing remote access for security
> reasons. If you can add a file, I can mount your filesystem too, maybe
> even delete things. Whereas with EC2-only filesystems, your files are
> *only* exposed to e
> I think most people try to avoid allowing remote access for security
> reasons. If you can add a file, I can mount your filesystem too, maybe even
> delete things. Whereas with EC2-only filesystems, your files are *only*
> exposed to everyone else that knows or can scan for your IPAddr and ports.
Julien Nioche wrote:
I have tried using *slave.host.name* and give it the public address of my
data node. I can now see the node listed with its public address on the
dfshealth.jsp, however when I try to send a file to the HDFS from my
external server I still get :
*08/09/08 15:58:41 INFO dfs.DF
I have tried using *slave.host.name* and give it the public address of my
data node. I can now see the node listed with its public address on the
dfshealth.jsp, however when I try to send a file to the HDFS from my
external server I still get :
*08/09/08 15:58:41 INFO dfs.DFSClient: Waiting to fin
13 matches
Mail list logo
