Hbase on EC2 and issues with Amazon NAT Internal Addresses

Robert J Berger Fri, 23 Jan 2009 01:43:56 -0800

I've got HBase / Haddop 0.19 running on an Amazon EC2 instance using the instructions for the "Pseudo-Distributed Operation".


While ssh'd onto the machine running HBase/Haddop I can run


hbase shell

and do a list and see the rows from the HBase fine.

But when I try to run hbase shell from another host that is not on the Amazon network, it fails (see the example command where the real command uses a real host name not example.com)

I can access the web interface with a client not running on the Amazon network to HBase at example.com:60010 but it shows no User Tables and the Region Servers Address is the EC2 internal address:

domU-13-41-39-01-80-62.compute-1.internal:60020

When I click on that it of course it can not access it since its behind a NAT.

I've been struggling (and googling) with this for a while and can't figure out how to get it to use the public address. Any help or suggestions would be greatly appreciated!


Here is my hbase-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
configuration>
  <property>
    <name>hbase.rootdir</name>
    <value>hdfs://localhost:54310/hbase</value>
    <description>The directory shared by region servers.
    Should be fully-qualified to include the filesystem to use.
    E.g: hdfs://NAMENODE_SERVER:PORT/HBASE_ROOTDIR
    </description>
  </property>

  <property>
    <name>hbase.master</name>
    <value>0.0.0.0:60000</value>
    <description>The host and port that the HBase master runs at.
    </description>
  </property>

  <property>
    <name>hbase.regionserver</name>
    <value>fbase.cinchcorp.com:60020</value>
    <description>The host and port a HBase region server runs at.
    </description>
  </property>
</configuration>

Here is the command line and failure messages:

> /usr/local/hbase/bin/hbase shell --master=example.com:60000
HBase Shell; enter 'help<RETURN>' for list of supported commands.
Version: 0.18.1, r707159, Wed Oct 22 12:43:06 PDT 2008
hbase(main):001:0> list

09/01/23 08:56:58 INFO client.HConnectionManager$TableServers: Attempt 0 of 5 failed with <java.io.IOException: Call failed on local exception>. Retrying after sleep of 2000 09/01/23 08:57:00 INFO client.HConnectionManager$TableServers: Attempt 1 of 5 failed with <java.io.IOException: Call failed on local exception>. Retrying after sleep of 2000 09/01/23 08:57:03 INFO client.HConnectionManager$TableServers: Attempt 2 of 5 failed with <java.io.IOException: Call failed on local exception>. Retrying after sleep of 2000 09/01/23 08:57:05 INFO client.HConnectionManager$TableServers: Attempt 3 of 5 failed with <java.io.IOException: Call failed on local exception>. Retrying after sleep of 4000 NativeException: org.apache.hadoop.hbase.MasterNotRunningException: fbase.cinchcorp.com:60000 from org/apache/hadoop/hbase/client/HConnectionManager.java:221:in `getMaster'

        from org/apache/hadoop/hbase/client/HBaseAdmin.java:67:in `<init>'

from sun/reflect/NativeConstructorAccessorImpl.java:-2:in `newInstance0'

        from sun/reflect/NativeConstructorAccessorImpl.java:39:in `newInstance'

from sun/reflect/DelegatingConstructorAccessorImpl.java:27:in `newInstance'

        from java/lang/reflect/Constructor.java:513:in `newInstance'
        from org/jruby/javasupport/JavaConstructor.java:195:in `new_instance'

from org.jruby.javasupport.JavaConstructorInvoker $new_instance_method_0_0:-1:in `call'

        from org/jruby/runtime/CallSite.java:261:in `call'
        from org/jruby/evaluator/ASTInterpreter.java:670:in `callNode'
        from org/jruby/evaluator/ASTInterpreter.java:324:in `evalInternal'
        from org/jruby/evaluator/ASTInterpreter.java:2173:in `setupArgs'
        from org/jruby/evaluator/ASTInterpreter.java:571:in `attrAssignNode'
        from org/jruby/evaluator/ASTInterpreter.java:309:in `evalInternal'
        from org/jruby/evaluator/ASTInterpreter.java:620:in `blockNode'
        from org/jruby/evaluator/ASTInterpreter.java:318:in `evalInternal'
... 178 levels...

from ruby/usr/local/pkgs/hbase_minus_0_dot_18_dot_1/bin//usr/local/ hbase/bin/../bin/hirb.rb:-1:in `__file__' from ruby/usr/local/pkgs/hbase_minus_0_dot_18_dot_1/bin//usr/local/ hbase/bin/../bin/hirb.rb:-1:in `load'

        from org/jruby/Ruby.java:512:in `runScript'
        from org/jruby/Ruby.java:432:in `runNormally'
        from org/jruby/Ruby.java:312:in `runFromMain'
        from org/jruby/Main.java:144:in `run'
        from org/jruby/Main.java:89:in `run'
        from org/jruby/Main.java:80:in `main'

from file:/usr/local/pkgs/hbase-0.18.1/lib/jruby-complete-1.1.2.jar!/ builtin/javasupport/proxy/concrete.rb:23:in `initialize' from file:/usr/local/pkgs/hbase-0.18.1/lib/jruby-complete-1.1.2.jar!/ builtin/javasupport/proxy/concrete.rb:6:in `new' from file:/usr/local/pkgs/hbase-0.18.1/lib/jruby-complete-1.1.2.jar!/ builtin/javasupport/proxy/concrete.rb:6:in `new'

        from /usr/local/hbase/bin/../bin/HBase.rb:37:in `initialize'
        from /usr/local/hbase/bin/../bin/hirb.rb:218:in `new'
        from /usr/local/hbase/bin/../bin/hirb.rb:218:in `admin'
        from /usr/local/hbase/bin/../bin/hirb.rb:242:in `list'

from (hbase):2:in `binding'hbase(main):002:0>

smime.p7s
Description: S/MIME cryptographic signature

Hbase on EC2 and issues with Amazon NAT Internal Addresses

Reply via email to