Expanding on my point #3, if you run your own DNS that accepts updates, you can
use nsupdate to maintain a dynamic shadow of the internal zone with mappings to
public IPs. Update records when the cluster is up. Remove them when the cluster
is terminated.
You would also need to figure out how best to update should an instance fail
and be replaced, but this should be hopefully a rare event and elastic IPs can
help, though each account only gets 5 of them without justification to AWS.
- Andy
On Fri Mar 19th, 2010 9:45 AM PDT Andrew Purtell wrote:
>The IP addresses assigned on the cluster are all internal ones, so when the
>regionservers do a reverse lookup, they get something foo.internal. Then they
>report this to the master, which hands them out to the client library as
>region locations. So while you can telnet to 60020 on the slaves as you know
>the public DNS names, the client library is only able to learn of the internal
>ones.
>
>Some options:
>
>1) Run your clients up in the EC2 cloud also
>
>2) Use a connector like Stargate or the Thrift server which can in effect
>proxy your requests to the EC2 hosted cluster.
>
>3) Grab the latest scripts from 0.20 branch in SVN. In
>$HOME/.hbase-<cluster>-instances will be the list of instance identifiers of
>the slaves. Do:
>
> ec2-describe-instances `cat ~/.hbase-<cluster>-instances` | grep INSTANCE |
> grep running | awk '{print "$4 $5"}'
>
>This will give you a mapping between private and public names. Dump entries
>into your /etc/hosts which map public IP (use dig to look up) to private name.
>Yes, it's not a nice hack.
>
>4) You can use SSH as a SOCKS 5 proxy (ssh -f -N -D <local-port> <remote>),
>which will also forward DNS requests, but to do it that way you'd have to hack
>the client library some.
>
> - Andy
>
>> From: George Stathis
>> Subject: Remote Java client connection into EC2 instance
>> To: hbase-user@hadoop.apache.org
>> Date: Friday, March 19, 2010, 8:00 AM
>> This has come up
>> before<http://mail-archives.apache.org/mod_mbox/hadoop-hbase-user/200909.mbox/%3c587903.6843...@web65509.mail.ac4.yahoo.com%3e>but
>> I'm still unclear as to whether this is possible or not:
>> remotely connecting to an EC2 instance using the Java client
>> library.
>[...]
>> Now, I have gone though a lot of threads and posts and have
>> opened up all required ports (I think) on EC2: 60000, 60020
>> and 2181 (I can telnet into them). I have one test EC2
>> instance running in pseudo-distributed mode to
>> test the remote connection. I attempt to run a single unit
>> test.
>[...]
>
>
>
>
