Still having the same problem:

Here is the master log

2012-04-16 15:26:32,717 INFO
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation:
ZooKeeper available but no active master location found
2012-04-16 15:26:32,718 INFO
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation:
getMaster attempt 9 of 10 failed; no more retrying.
org.apache.hadoop.hbase.MasterNotRunningException
    at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:564)
    at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:95)
    at
org.apache.hadoop.hbase.master.MasterStatusServlet.doGet(MasterStatusServlet.java:55)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
    at
org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
    at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
    at
org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:829)
    at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
    at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
    at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
    at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
    at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
    at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
    at
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
    at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
    at org.mortbay.jetty.Server.handle(Server.java:326)
    at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
    at
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
    at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
    at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
    at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
    at
org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
    at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)

and here is what it says from zookeeper shell

[zk: localhost:2181(CONNECTED) 0] ls /hbase
[splitlog, unassigned, root-region-server, rs, table, master, shutdown]
[zk: localhost:2181(CONNECTED) 1] get /hbase/master
ip-10-251-27-130.ec2.internal:60000
cZxid = 0xd00000025
ctime = Mon Apr 16 15:25:18 EDT 2012
mZxid = 0xd00000025
mtime = Mon Apr 16 15:25:18 EDT 2012
pZxid = 0xd00000025
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x136bc97498d0003
dataLength = 35
numChildren = 0

and here is my /etc/hosts

[root@ip-10-251-27-130 bin]# cat /etc/hosts
127.0.0.1   localhost   localhost.localdomain
10.250.9.220    ip-10-250-9-220 zoo1
10.251.110.50    ip-10-251-110-50 zoo2
10.250.54.148    ip-10-250-54-148 datanode
10.251.27.130   ip-10-251-27-130 namenode ip-10-251-27-130.ec2.internal

I run zookeepers on namenode, zoo1 and zoo2

Thanks

Henri Pipe


On Fri, Apr 13, 2012 at 1:01 PM, Stack <st...@duboce.net> wrote:

> What do you see in the master log?
> St.Ack
>
> On Fri, Apr 13, 2012 at 11:00 AM, Henri Pipe <henri.p...@gmail.com> wrote:
> > I had tried zkCli (ls /hbase and get /hbase/master) , but it returns the
> > correct value.
> >
> > [zk: localhost:2181(CONNECTED) 2] get /hbase/master
> > ip-10-251-27-130:60000
> > cZxid = 0xa00000032
> > ctime = Thu Apr 12 20:03:23 EDT 2012
> > mZxid = 0xa00000032
> > mtime = Thu Apr 12 20:03:23 EDT 2012
> > pZxid = 0xa00000032
> >
> > Also, I do have the namenode listed in my config
> >
> > Here is my hbase-site.xml file:
> >
> > <configuration>
> >  <property>
> >    <name>hbase.rootdir</name>
> >    <value>hdfs://namenode:9000/hbase</value>
> >  </property>
> >
> > Henri Pipe
> >
> >
> > On Fri, Apr 13, 2012 at 1:58 AM, N Keywal <nkey...@gmail.com> wrote:
> >
> >> Hi,
> >>
> >> Literally, it means that ZooKeeper is there but the hbase client can't
> find
> >> the hbase master address in it.
> >> By default, the node used is /hbase/master, and it contains the hostname
> >> and port of the master.
> >>
> >> You can check its content in ZK by doing a "get /hbase/master" in
> >> bin/zkCli.sh (see
> >>
> >>
> http://zookeeper.apache.org/doc/r3.4.3/zookeeperStarted.html#sc_ConnectingToZooKeeper
> >> ).
> >>
> >> There should be a root cause for this, so it worths looking for other
> error
> >> messages in the logs (master especially).
> >>
> >> N.
> >>
> >> On Fri, Apr 13, 2012 at 1:23 AM, Henri Pipe <henri.p...@gmail.com>
> wrote:
> >>
> >> > "client.HConnectionManager$HConnectionImplementation: ZooKeeper
> available
> >> > but no active master location found"
> >> >
> >> > Having a problem with master startup that I have not seen before.
> >> >
> >> > running the following packages:
> >> >
> >> > hadoop-hbase-0.90.4+49.137-1
> >> > hadoop-0.20-secondarynamenode-0.20.2+923.197-1
> >> > hadoop-hbase-thrift-0.90.4+49.137-1
> >> > hadoop-zookeeper-3.3.4+19.3-1
> >> > hadoop-0.20-datanode-0.20.2+923.197-1
> >> > hadoop-0.20-namenode-0.20.2+923.197-1
> >> > hadoop-0.20-tasktracker-0.20.2+923.197-1
> >> > hadoop-hbase-regionserver-0.90.4+49.137-1
> >> > hadoop-zookeeper-server-3.3.4+19.3-1
> >> > hadoop-0.20-0.20.2+923.197-1
> >> > hadoop-0.20-jobtracker-0.20.2+923.197-1
> >> > hadoop-hbase-master-0.90.4+49.137-1
> >> > [root@ip-10-251-27-130 logs]# java -version
> >> > java version "1.6.0_31"
> >> > Java(TM) SE Runtime Environment (build 1.6.0_31-b04)
> >> > Java HotSpot(TM) 64-Bit Server VM (build 20.6-b01, mixed mode)
> >> >
> >> > I start master and region server on another node.
> >> >
> >> > Master is initialized, but as soon as I try to check the
> master_status or
> >> > do a zkdump via web interface, it blows up with:
> >> >
> >> > 2012-04-12 19:16:10,453 INFO
> >> >
> >> >
> >>
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation:
> >> > ZooKeeper available but no active master location found
> >> > 2012-04-12 19:16:10,453 INFO
> >> >
> >> >
> >>
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation:
> >> > getMaster attempt 10 of 10 failed; retrying after sleep of 16000
> >> >
> >> > I am running three zookeepers:
> >> >
> >> > # The number of milliseconds of each tick
> >> > tickTime=2000
> >> > # The number of ticks that the initial
> >> > # synchronization phase can take
> >> > initLimit=10
> >> > # The number of ticks that can pass between
> >> > # sending a request and getting an acknowledgement
> >> > syncLimit=5
> >> > # the directory where the snapshot is stored.
> >> > dataDir=/mnt/zookeeper
> >> > # The maximum number of zookeeper client connections
> >> > maxClientCnxns=2000
> >> > # the port at which the clients will connect
> >> > clientPort=2181
> >> > server.1=10.251.27.130:2888:3888
> >> > server.2=10.250.9.220:2888:3888
> >> > server.3=10.251.110.50:2888:3888
> >> >
> >> > I can telnet to the zookeepers just fine.
> >> >
> >> > Here is my hbase-site.xml file:
> >> >
> >> > <configuration>
> >> >  <property>
> >> >    <name>hbase.rootdir</name>
> >> >    <value>hdfs://namenode:9000/hbase</value>
> >> >  </property>
> >> >  <property>
> >> >    <name>hbase.cluster.distributed</name>
> >> >    <value>true</value>
> >> >  </property>
> >> > <property>
> >> >    <name>hbase.zookeeper.quorum</name>
> >> >    <value>10.251.27.130,10.250.9.220,10.251.110.50</value>
> >> > </property>
> >> > <property>
> >> >    <name>hbase.zookeeper.property.dataDir</name>
> >> >    <value>/hadoop/zookeeper/data</value>
> >> > </property>
> >> > <property>
> >> >    <name>hbase.zookeeper.property.maxClientCnxns</name>
> >> >    <value>2000</value>
> >> >    <final>true</final>
> >> > </property>
> >> > </configuration>
> >> >
> >> > Any thoughts? Any help is greatly appreciated.
> >> >
> >> > Thanks
> >> >
> >> > Henri Pipe
> >> >
> >>
>

Reply via email to