Re: ZooKeeper available but no active master location found
05:59:51 INFO zookeeper.ClientCnxn: Session establishment complete on server sandbox.hortonworks.com/10.0.2.15:2181, sessionid = 0x1453145e9500038, negotiated timeout = 4 14/04/05 05:59:51 INFO client.HConnectionManager$HConnectionImplementation: ZooKeeper available but no active master location found 14/04/05 05:59:51 INFO client.HConnectionManager$HConnectionImplementation: getMaster attempt 0 of 10 failed; retrying after sleep of 1000 org.apache.hadoop.hbase.MasterNotRunningException at org.apache.hadoop.hbase.client.HConnectionManager$ HConnectionImplementation.getMaster(HConnectionManager.java:634) at org.apache.hadoop.hbase.client.HBaseAdmin.init( HBaseAdmin.java:106) at HBaseIA.TwitBase.InitTables.main(InitTables.java:20) InitTables.java https://github.com/hbaseinaction/twitbase/blob/ master/src/main/java/HBaseIA/TwitBase/InitTables.java The line that drops error is: HBaseAdminadmin=newHBaseAdmin(conf); Log line: ZooKeeper available but no active master location found is from org/apache/hadoop/hbase/client/HConnectionManager.java [hbase@sandbox twitbase-master]$ jps 30473 Jps 28549 HMaster 28896 RESTServer 28916 HRegionServer 28902 ThriftServer [hbase@sandbox twitbase-master]$ As much I read ther might be somthing wrong with binding and DNS resolving. Any hints? -- Best regards, Margus (Margusja) Roo +372 51 48 780 http://margus.roo.ee http://ee.linkedin.com/in/margusroo skype: margusja ldapsearch -x -h ldap.sk.ee -b c=EE (serialNumber=37303140314)
Re: ZooKeeper available but no active master location found
14/04/05 05:59:50 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=10.0.2.15:2181 sessionTimeout=18 watcher=hconnection 14/04/05 05:59:50 INFO zookeeper.ClientCnxn: Opening socket connection to server /10.0.2.15:2181 14/04/05 05:59:50 INFO zookeeper.RecoverableZooKeeper: The identifier of this process is 30...@sandbox.hortonworks.com 14/04/05 05:59:50 INFO client.ZooKeeperSaslClient: Client will not SASL-authenticate because the default JAAS configuration section 'Client' could not be found. If you are not using SASL, you may ignore this. On the other hand, if you expected SASL to work, please fix your JAAS configuration. 14/04/05 05:59:51 INFO zookeeper.ClientCnxn: Socket connection established to sandbox.hortonworks.com/10.0.2.15:2181, initiating session 14/04/05 05:59:51 INFO zookeeper.ClientCnxn: Session establishment complete on server sandbox.hortonworks.com/10.0.2.15:2181, sessionid = 0x1453145e9500038, negotiated timeout = 4 14/04/05 05:59:51 INFO client.HConnectionManager$ HConnectionImplementation: ZooKeeper available but no active master location found 14/04/05 05:59:51 INFO client.HConnectionManager$ HConnectionImplementation: getMaster attempt 0 of 10 failed; retrying after sleep of 1000 org.apache.hadoop.hbase.MasterNotRunningException at org.apache.hadoop.hbase.client.HConnectionManager$ HConnectionImplementation.getMaster(HConnectionManager.java:634) at org.apache.hadoop.hbase.client.HBaseAdmin.init( HBaseAdmin.java:106) at HBaseIA.TwitBase.InitTables.main(InitTables.java:20) InitTables.java https://github.com/hbaseinaction/twitbase/blob/ master/src/main/java/HBaseIA/TwitBase/InitTables.java The line that drops error is: HBaseAdminadmin=newHBaseAdmin(conf); Log line: ZooKeeper available but no active master location found is from org/apache/hadoop/hbase/client/HConnectionManager.java [hbase@sandbox twitbase-master]$ jps 30473 Jps 28549 HMaster 28896 RESTServer 28916 HRegionServer 28902 ThriftServer [hbase@sandbox twitbase-master]$ As much I read ther might be somthing wrong with binding and DNS resolving. Any hints? -- Best regards, Margus (Margusja) Roo +372 51 48 780 http://margus.roo.ee http://ee.linkedin.com/in/margusroo skype: margusja ldapsearch -x -h ldap.sk.ee -b c=EE (serialNumber=37303140314)
ZooKeeper available but no active master location found
Hi I downloaded and installed hortonworks sandbox 2.0 for virtualbox. HBase version is: 0.96.0.2.0.6.0-76-hadoop2, re6d7a56f72914d01e55c0478d74e5cfd3778f231 [hbase@sandbox twitbase-master]$ cat /etc/hosts # Do not remove the following line, or various programs # that require network functionality will fail. 127.0.0.1 localhost.localdomain localhost 10.0.2.15 sandbox.hortonworks.com sandbox [hbase@sandbox twitbase-master]$ hostname sandbox.hortonworks.com [root@sandbox ~]# netstat -lnp | grep 2181 tcp0 0 0.0.0.0:2181 0.0.0.0:* LISTEN 19359/java [root@sandbox ~]# netstat -lnp | grep 6 tcp0 0 10.0.2.15:6 0.0.0.0:* LISTEN 28549/java [hbase@sandbox twitbase-master]$ hbase shell 14/04/05 05:56:44 INFO Configuration.deprecation: hadoop.native.lib is deprecated. Instead, use io.native.lib.available HBase Shell; enter 'helpRETURN' for list of supported commands. Type exitRETURN to leave the HBase Shell Version 0.96.0.2.0.6.0-76-hadoop2, re6d7a56f72914d01e55c0478d74e5cfd3778f231, Thu Oct 17 18:15:20 PDT 2013 hbase(main):001:0 list TABLE SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/usr/lib/zookeeper/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/lib/hadoop/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. ambarismoketest mytable simple_hcat_load_table users weblogs 5 row(s) in 4.6040 seconds = [ambarismoketest, mytable, simple_hcat_load_table, users, weblogs] hbase(main):002:0 So far is good. I'd like to play with a code: https://github.com/hbaseinaction/twitbase downloaded and made package: mvn package and got twitbase-1.0.0.jar. When I try to exec code I will get: 14/04/05 05:59:50 INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.3-1240972, built on 02/06/2012 10:48 GMT 14/04/05 05:59:50 INFO zookeeper.ZooKeeper: Client environment:host.name=sandbox.hortonworks.com 14/04/05 05:59:50 INFO zookeeper.ZooKeeper: Client environment:java.version=1.6.0_30 14/04/05 05:59:50 INFO zookeeper.ZooKeeper: Client environment:java.vendor=Sun Microsystems Inc. 14/04/05 05:59:50 INFO zookeeper.ZooKeeper: Client environment:java.home=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre 14/04/05 05:59:50 INFO zookeeper.ZooKeeper: Client environment:java.class.path=target/twitbase-1.0.0.jar 14/04/05 05:59:50 INFO zookeeper.ZooKeeper: Client environment:java.library.path=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/amd64/server:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/amd64:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 14/04/05 05:59:50 INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp 14/04/05 05:59:50 INFO zookeeper.ZooKeeper: Client environment:java.compiler=NA 14/04/05 05:59:50 INFO zookeeper.ZooKeeper: Client environment:os.name=Linux 14/04/05 05:59:50 INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64 14/04/05 05:59:50 INFO zookeeper.ZooKeeper: Client environment:os.version=2.6.32-431.11.2.el6.x86_64 14/04/05 05:59:50 INFO zookeeper.ZooKeeper: Client environment:user.name=hbase 14/04/05 05:59:50 INFO zookeeper.ZooKeeper: Client environment:user.home=/home/hbase 14/04/05 05:59:50 INFO zookeeper.ZooKeeper: Client environment:user.dir=/home/hbase/twitbase-master 14/04/05 05:59:50 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=10.0.2.15:2181 sessionTimeout=18 watcher=hconnection 14/04/05 05:59:50 INFO zookeeper.ClientCnxn: Opening socket connection to server /10.0.2.15:2181 14/04/05 05:59:50 INFO zookeeper.RecoverableZooKeeper: The identifier of this process is 30...@sandbox.hortonworks.com 14/04/05 05:59:50 INFO client.ZooKeeperSaslClient: Client will not SASL-authenticate because the default JAAS configuration section 'Client' could not be found. If you are not using SASL, you may ignore this. On the other hand, if you expected SASL to work, please fix your JAAS configuration. 14/04/05 05:59:51 INFO zookeeper.ClientCnxn: Socket connection established to sandbox.hortonworks.com/10.0.2.15:2181, initiating session 14/04/05 05:59:51 INFO zookeeper.ClientCnxn: Session establishment complete on server sandbox.hortonworks.com/10.0.2.15:2181, sessionid = 0x1453145e9500038, negotiated timeout = 4 14/04/05 05:59:51 INFO client.HConnectionManager$HConnectionImplementation: ZooKeeper available but no active master location found 14/04/05 05:59:51 INFO client.HConnectionManager$HConnectionImplementation: getMaster attempt 0 of 10 failed; retrying after sleep of 1000 org.apache.hadoop.hbase.MasterNotRunningException at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster
Re: ZooKeeper available but no active master location found
$HConnectionImplementation: ZooKeeper available but no active master location found 14/04/05 05:59:51 INFO client.HConnectionManager$HConnectionImplementation: getMaster attempt 0 of 10 failed; retrying after sleep of 1000 org.apache.hadoop.hbase.MasterNotRunningException at org.apache.hadoop.hbase.client.HConnectionManager$ HConnectionImplementation.getMaster(HConnectionManager.java:634) at org.apache.hadoop.hbase.client.HBaseAdmin.init( HBaseAdmin.java:106) at HBaseIA.TwitBase.InitTables.main(InitTables.java:20) InitTables.java https://github.com/hbaseinaction/twitbase/blob/ master/src/main/java/HBaseIA/TwitBase/InitTables.java The line that drops error is: HBaseAdminadmin=newHBaseAdmin(conf); Log line: ZooKeeper available but no active master location found is from org/apache/hadoop/hbase/client/HConnectionManager.java [hbase@sandbox twitbase-master]$ jps 30473 Jps 28549 HMaster 28896 RESTServer 28916 HRegionServer 28902 ThriftServer [hbase@sandbox twitbase-master]$ As much I read ther might be somthing wrong with binding and DNS resolving. Any hints? -- Best regards, Margus (Margusja) Roo +372 51 48 780 http://margus.roo.ee http://ee.linkedin.com/in/margusroo skype: margusja ldapsearch -x -h ldap.sk.ee -b c=EE (serialNumber=37303140314)
Re: Zookeeper available but no active master location found
Still having the same problem: Here is the master log 2012-04-16 15:26:32,717 INFO org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: ZooKeeper available but no active master location found 2012-04-16 15:26:32,718 INFO org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: getMaster attempt 9 of 10 failed; no more retrying. org.apache.hadoop.hbase.MasterNotRunningException at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:564) at org.apache.hadoop.hbase.client.HBaseAdmin.init(HBaseAdmin.java:95) at org.apache.hadoop.hbase.master.MasterStatusServlet.doGet(MasterStatusServlet.java:55) at javax.servlet.http.HttpServlet.service(HttpServlet.java:707) at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221) at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:829) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582) and here is what it says from zookeeper shell [zk: localhost:2181(CONNECTED) 0] ls /hbase [splitlog, unassigned, root-region-server, rs, table, master, shutdown] [zk: localhost:2181(CONNECTED) 1] get /hbase/master ip-10-251-27-130.ec2.internal:6 cZxid = 0xd0025 ctime = Mon Apr 16 15:25:18 EDT 2012 mZxid = 0xd0025 mtime = Mon Apr 16 15:25:18 EDT 2012 pZxid = 0xd0025 cversion = 0 dataVersion = 0 aclVersion = 0 ephemeralOwner = 0x136bc97498d0003 dataLength = 35 numChildren = 0 and here is my /etc/hosts [root@ip-10-251-27-130 bin]# cat /etc/hosts 127.0.0.1 localhost localhost.localdomain 10.250.9.220ip-10-250-9-220 zoo1 10.251.110.50ip-10-251-110-50 zoo2 10.250.54.148ip-10-250-54-148 datanode 10.251.27.130 ip-10-251-27-130 namenode ip-10-251-27-130.ec2.internal I run zookeepers on namenode, zoo1 and zoo2 Thanks Henri Pipe On Fri, Apr 13, 2012 at 1:01 PM, Stack st...@duboce.net wrote: What do you see in the master log? St.Ack On Fri, Apr 13, 2012 at 11:00 AM, Henri Pipe henri.p...@gmail.com wrote: I had tried zkCli (ls /hbase and get /hbase/master) , but it returns the correct value. [zk: localhost:2181(CONNECTED) 2] get /hbase/master ip-10-251-27-130:6 cZxid = 0xa0032 ctime = Thu Apr 12 20:03:23 EDT 2012 mZxid = 0xa0032 mtime = Thu Apr 12 20:03:23 EDT 2012 pZxid = 0xa0032 Also, I do have the namenode listed in my config Here is my hbase-site.xml file: configuration property namehbase.rootdir/name valuehdfs://namenode:9000/hbase/value /property Henri Pipe On Fri, Apr 13, 2012 at 1:58 AM, N Keywal nkey...@gmail.com wrote: Hi, Literally, it means that ZooKeeper is there but the hbase client can't find the hbase master address in it. By default, the node used is /hbase/master, and it contains the hostname and port of the master. You can check its content in ZK by doing a get /hbase/master in bin/zkCli.sh (see http://zookeeper.apache.org/doc/r3.4.3/zookeeperStarted.html#sc_ConnectingToZooKeeper ). There should be a root cause for this, so it worths looking for other error messages in the logs (master especially). N. On Fri, Apr 13, 2012 at 1:23 AM, Henri Pipe henri.p...@gmail.com wrote: client.HConnectionManager$HConnectionImplementation: ZooKeeper available but no active master location found Having a problem with master startup that I have not seen before. running the following packages: hadoop-hbase-0.90.4+49.137-1 hadoop-0.20-secondarynamenode-0.20.2+923.197-1
Re: Zookeeper available but no active master location found
Hi, Literally, it means that ZooKeeper is there but the hbase client can't find the hbase master address in it. By default, the node used is /hbase/master, and it contains the hostname and port of the master. You can check its content in ZK by doing a get /hbase/master in bin/zkCli.sh (see http://zookeeper.apache.org/doc/r3.4.3/zookeeperStarted.html#sc_ConnectingToZooKeeper ). There should be a root cause for this, so it worths looking for other error messages in the logs (master especially). N. On Fri, Apr 13, 2012 at 1:23 AM, Henri Pipe henri.p...@gmail.com wrote: client.HConnectionManager$HConnectionImplementation: ZooKeeper available but no active master location found Having a problem with master startup that I have not seen before. running the following packages: hadoop-hbase-0.90.4+49.137-1 hadoop-0.20-secondarynamenode-0.20.2+923.197-1 hadoop-hbase-thrift-0.90.4+49.137-1 hadoop-zookeeper-3.3.4+19.3-1 hadoop-0.20-datanode-0.20.2+923.197-1 hadoop-0.20-namenode-0.20.2+923.197-1 hadoop-0.20-tasktracker-0.20.2+923.197-1 hadoop-hbase-regionserver-0.90.4+49.137-1 hadoop-zookeeper-server-3.3.4+19.3-1 hadoop-0.20-0.20.2+923.197-1 hadoop-0.20-jobtracker-0.20.2+923.197-1 hadoop-hbase-master-0.90.4+49.137-1 [root@ip-10-251-27-130 logs]# java -version java version 1.6.0_31 Java(TM) SE Runtime Environment (build 1.6.0_31-b04) Java HotSpot(TM) 64-Bit Server VM (build 20.6-b01, mixed mode) I start master and region server on another node. Master is initialized, but as soon as I try to check the master_status or do a zkdump via web interface, it blows up with: 2012-04-12 19:16:10,453 INFO org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: ZooKeeper available but no active master location found 2012-04-12 19:16:10,453 INFO org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: getMaster attempt 10 of 10 failed; retrying after sleep of 16000 I am running three zookeepers: # The number of milliseconds of each tick tickTime=2000 # The number of ticks that the initial # synchronization phase can take initLimit=10 # The number of ticks that can pass between # sending a request and getting an acknowledgement syncLimit=5 # the directory where the snapshot is stored. dataDir=/mnt/zookeeper # The maximum number of zookeeper client connections maxClientCnxns=2000 # the port at which the clients will connect clientPort=2181 server.1=10.251.27.130:2888:3888 server.2=10.250.9.220:2888:3888 server.3=10.251.110.50:2888:3888 I can telnet to the zookeepers just fine. Here is my hbase-site.xml file: configuration property namehbase.rootdir/name valuehdfs://namenode:9000/hbase/value /property property namehbase.cluster.distributed/name valuetrue/value /property property namehbase.zookeeper.quorum/name value10.251.27.130,10.250.9.220,10.251.110.50/value /property property namehbase.zookeeper.property.dataDir/name value/hadoop/zookeeper/data/value /property property namehbase.zookeeper.property.maxClientCnxns/name value2000/value finaltrue/final /property /configuration Any thoughts? Any help is greatly appreciated. Thanks Henri Pipe
Re: Zookeeper available but no active master location found
What do you see in the master log? St.Ack On Fri, Apr 13, 2012 at 11:00 AM, Henri Pipe henri.p...@gmail.com wrote: I had tried zkCli (ls /hbase and get /hbase/master) , but it returns the correct value. [zk: localhost:2181(CONNECTED) 2] get /hbase/master ip-10-251-27-130:6 cZxid = 0xa0032 ctime = Thu Apr 12 20:03:23 EDT 2012 mZxid = 0xa0032 mtime = Thu Apr 12 20:03:23 EDT 2012 pZxid = 0xa0032 Also, I do have the namenode listed in my config Here is my hbase-site.xml file: configuration property namehbase.rootdir/name valuehdfs://namenode:9000/hbase/value /property Henri Pipe On Fri, Apr 13, 2012 at 1:58 AM, N Keywal nkey...@gmail.com wrote: Hi, Literally, it means that ZooKeeper is there but the hbase client can't find the hbase master address in it. By default, the node used is /hbase/master, and it contains the hostname and port of the master. You can check its content in ZK by doing a get /hbase/master in bin/zkCli.sh (see http://zookeeper.apache.org/doc/r3.4.3/zookeeperStarted.html#sc_ConnectingToZooKeeper ). There should be a root cause for this, so it worths looking for other error messages in the logs (master especially). N. On Fri, Apr 13, 2012 at 1:23 AM, Henri Pipe henri.p...@gmail.com wrote: client.HConnectionManager$HConnectionImplementation: ZooKeeper available but no active master location found Having a problem with master startup that I have not seen before. running the following packages: hadoop-hbase-0.90.4+49.137-1 hadoop-0.20-secondarynamenode-0.20.2+923.197-1 hadoop-hbase-thrift-0.90.4+49.137-1 hadoop-zookeeper-3.3.4+19.3-1 hadoop-0.20-datanode-0.20.2+923.197-1 hadoop-0.20-namenode-0.20.2+923.197-1 hadoop-0.20-tasktracker-0.20.2+923.197-1 hadoop-hbase-regionserver-0.90.4+49.137-1 hadoop-zookeeper-server-3.3.4+19.3-1 hadoop-0.20-0.20.2+923.197-1 hadoop-0.20-jobtracker-0.20.2+923.197-1 hadoop-hbase-master-0.90.4+49.137-1 [root@ip-10-251-27-130 logs]# java -version java version 1.6.0_31 Java(TM) SE Runtime Environment (build 1.6.0_31-b04) Java HotSpot(TM) 64-Bit Server VM (build 20.6-b01, mixed mode) I start master and region server on another node. Master is initialized, but as soon as I try to check the master_status or do a zkdump via web interface, it blows up with: 2012-04-12 19:16:10,453 INFO org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: ZooKeeper available but no active master location found 2012-04-12 19:16:10,453 INFO org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: getMaster attempt 10 of 10 failed; retrying after sleep of 16000 I am running three zookeepers: # The number of milliseconds of each tick tickTime=2000 # The number of ticks that the initial # synchronization phase can take initLimit=10 # The number of ticks that can pass between # sending a request and getting an acknowledgement syncLimit=5 # the directory where the snapshot is stored. dataDir=/mnt/zookeeper # The maximum number of zookeeper client connections maxClientCnxns=2000 # the port at which the clients will connect clientPort=2181 server.1=10.251.27.130:2888:3888 server.2=10.250.9.220:2888:3888 server.3=10.251.110.50:2888:3888 I can telnet to the zookeepers just fine. Here is my hbase-site.xml file: configuration property namehbase.rootdir/name valuehdfs://namenode:9000/hbase/value /property property namehbase.cluster.distributed/name valuetrue/value /property property namehbase.zookeeper.quorum/name value10.251.27.130,10.250.9.220,10.251.110.50/value /property property namehbase.zookeeper.property.dataDir/name value/hadoop/zookeeper/data/value /property property namehbase.zookeeper.property.maxClientCnxns/name value2000/value finaltrue/final /property /configuration Any thoughts? Any help is greatly appreciated. Thanks Henri Pipe
Zookeeper available but no active master location found
client.HConnectionManager$HConnectionImplementation: ZooKeeper available but no active master location found Having a problem with master startup that I have not seen before. running the following packages: hadoop-hbase-0.90.4+49.137-1 hadoop-0.20-secondarynamenode-0.20.2+923.197-1 hadoop-hbase-thrift-0.90.4+49.137-1 hadoop-zookeeper-3.3.4+19.3-1 hadoop-0.20-datanode-0.20.2+923.197-1 hadoop-0.20-namenode-0.20.2+923.197-1 hadoop-0.20-tasktracker-0.20.2+923.197-1 hadoop-hbase-regionserver-0.90.4+49.137-1 hadoop-zookeeper-server-3.3.4+19.3-1 hadoop-0.20-0.20.2+923.197-1 hadoop-0.20-jobtracker-0.20.2+923.197-1 hadoop-hbase-master-0.90.4+49.137-1 [root@ip-10-251-27-130 logs]# java -version java version 1.6.0_31 Java(TM) SE Runtime Environment (build 1.6.0_31-b04) Java HotSpot(TM) 64-Bit Server VM (build 20.6-b01, mixed mode) I start master and region server on another node. Master is initialized, but as soon as I try to check the master_status or do a zkdump via web interface, it blows up with: 2012-04-12 19:16:10,453 INFO org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: ZooKeeper available but no active master location found 2012-04-12 19:16:10,453 INFO org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation: getMaster attempt 10 of 10 failed; retrying after sleep of 16000 I am running three zookeepers: # The number of milliseconds of each tick tickTime=2000 # The number of ticks that the initial # synchronization phase can take initLimit=10 # The number of ticks that can pass between # sending a request and getting an acknowledgement syncLimit=5 # the directory where the snapshot is stored. dataDir=/mnt/zookeeper # The maximum number of zookeeper client connections maxClientCnxns=2000 # the port at which the clients will connect clientPort=2181 server.1=10.251.27.130:2888:3888 server.2=10.250.9.220:2888:3888 server.3=10.251.110.50:2888:3888 I can telnet to the zookeepers just fine. Here is my hbase-site.xml file: configuration property namehbase.rootdir/name valuehdfs://namenode:9000/hbase/value /property property namehbase.cluster.distributed/name valuetrue/value /property property namehbase.zookeeper.quorum/name value10.251.27.130,10.250.9.220,10.251.110.50/value /property property namehbase.zookeeper.property.dataDir/name value/hadoop/zookeeper/data/value /property property namehbase.zookeeper.property.maxClientCnxns/name value2000/value finaltrue/final /property /configuration Any thoughts? Any help is greatly appreciated. Thanks Henri Pipe
