Hi Ted, I used bin/hbase zkcli to check, and the result was: Connecting to pc277.emulab.net<http://pc277.emulab.net>:2181,pc273.emulab.net<http://pc273.emulab.net>:2181,pc312.emulab.net<http://pc312.emulab.net>:2181 2015-04-27 08:42:30,523 INFO [main] zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT 2015-04-27 08:42:30,525 INFO [main] zookeeper.ZooKeeper: Client environment:host.name=pc322.emulab.net<http://pc322.emulab.net> 2015-04-27 08:42:30,525 INFO [main] zookeeper.ZooKeeper: Client environment:java.version=1.7.0_79 2015-04-27 08:42:30,525 INFO [main] zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation 2015-04-27 08:42:30,526 INFO [main] zookeeper.ZooKeeper: Client environment:java.home=/usr/lib/jvm/java-7-openjdk-amd64/jre 2015-04-27 08:42:30,526 INFO [main] zookeeper.ZooKeeper: Client environment:java.class.path=/proj/ucare/bo/hbase-1.0.0/bin/../conf:/usr/lib/jvm/java-7-openjdk-amd64//lib/tools.jar:/proj/ucare/bo/hbase-1.0.0/bin/..:/proj/ucare/bo/hbase-1.0.0/bin/../lib/activation-1.1.jar:/pro………….. 2015-04-27 08:42:30,527 INFO [main] zookeeper.ZooKeeper: Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib 2015-04-27 08:42:30,527 INFO [main] zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp 2015-04-27 08:42:30,527 INFO [main] zookeeper.ZooKeeper: Client environment:java.compiler=<NA> 2015-04-27 08:42:30,528 INFO [main] zookeeper.ZooKeeper: Client environment:os.name=Linux 2015-04-27 08:42:30,528 INFO [main] zookeeper.ZooKeeper: Client environment:os.arch=amd64 2015-04-27 08:42:30,528 INFO [main] zookeeper.ZooKeeper: Client environment:os.version=3.2.0-56-generic 2015-04-27 08:42:30,528 INFO [main] zookeeper.ZooKeeper: Client environment:user.name=BF 2015-04-27 08:42:30,528 INFO [main] zookeeper.ZooKeeper: Client environment:user.home=/users/BF 2015-04-27 08:42:30,528 INFO [main] zookeeper.ZooKeeper: Client environment:user.dir=/proj/ucare/bo/hbase-1.0.0 2015-04-27 08:42:30,532 INFO [main] zookeeper.ZooKeeper: Initiating client connection, connectString=pc277.emulab.net<http://pc277.emulab.net>:2181,pc273.emulab.net<http://pc273.emulab.net>:2181,pc312.emulab.net<http://pc312.emulab.net>:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@920d014 Welcome to ZooKeeper! 2015-04-27 08:42:30,694 INFO [main-SendThread(pc277.emulab.net<http://pc277.emulab.net>:2181)] zookeeper.ClientCnxn: Opening socket connection to server pc277.emulab.net/155.98.39.77:2181<http://pc277.emulab.net/155.98.39.77:2181>. Will not attempt to authenticate using SASL (unknown error) 2015-04-27 08:42:30,718 INFO [main-SendThread(pc277.emulab.net<http://pc277.emulab.net>:2181)] zookeeper.ClientCnxn: Socket connection established to pc277.emulab.net/155.98.39.77:2181<http://pc277.emulab.net/155.98.39.77:2181>, initiating session JLine support is enabled 2015-04-27 08:42:30,760 INFO [main-SendThread(pc277.emulab.net<http://pc277.emulab.net>:2181)] zookeeper.ClientCnxn: Session establishment complete on server pc277.emulab.net/155.98.39.77:2181<http://pc277.emulab.net/155.98.39.77:2181>, sessionid = 0x14cfb53f8de0000, negotiated timeout = 30000
WATCHER:: WatchedEvent state:SyncConnected type:None path:null [zk: pc277.emulab.net<http://pc277.emulab.net>:2181,pc273.emulab.net<http://pc273.emulab.net>:2181,pc312.emulab.net<http://pc312.emulab.net>:2181(CONNECTED) 0] It seems that zookeeper is running correctly? Bo On Apr 27, 2015, at 9:19 AM, Ted Yu <yuzhih...@gmail.com<mailto:yuzhih...@gmail.com>> wrote: zkCli.sh is provided by zookeeper. See http://hbase.apache.org/book.html#quickstart for standalone HBase deployment. Cheers On Mon, Apr 27, 2015 at 6:53 AM, Bo Fu <b...@uchicago.edu<mailto:b...@uchicago.edu>> wrote: Hi Ted, I can’t find zkCli.sh. I’m using the zookeeper provided by HBase. So bad. Should I use my own zookeeper? Bo On Apr 26, 2015, at 3:23 PM, Ted Yu <yuzhih...@gmail.com<mailto:yuzhih...@gmail.com>> wrote: Can you use zkCli.sh to check the contents under /hbase ? Please check zookeeper logs to see if quorum was running fine. bq. [main-SendThread(pc273.emulab.net<http://pc273.emulab.net><http://pc273.emulab.net>:2181)] Please remove the '<http://...>' from hbase.zookeeper.quorum config. Cheers On Sun, Apr 26, 2015 at 11:08 AM, Bo Fu <b...@uchicago.edu<mailto:b...@uchicago.edu>> wrote: Hi all, I have problem starting a cluster of 1 master and 3 region server. When I started the cluster, the HMaster and HRegionserver will automatically exit. My abase-site.xml: <configuration> <property> <name>hbase.master</name> <value>hadoopmaster:60000</value> </property> <property> <name>hbase.rootdir</name> <value>hdfs://hadoopmaster:9000/hbase</value> </property> <property> <name>hbase.zookeeper.property.dataDir</name> <value>/proj/ucare/bo/hadoop_data/zookeeper</value> </property> <property> <name>hbase.zookeeper.quorum</name> <value>pc225.emulab.net<http://pc225.emulab.net><http://pc225.emulab.net>,pc273.emulab.net<http://pc273.emulab.net>< http://pc273.emulab.net>,pc210.emulab.net<http://pc210.emulab.net><http://pc210.emulab.net </value> <description>The directory shared by RegionServers. </description> </property> <property> <name>hbase.zookeeper.property.clientPort</name> <value>2181</value> </property> <property> <name>hbase.cluster.distributed</name> <value>true</value> </property> <property> <name>dfs.replication</name> <value>1</value> </property> </configuration> Log file: 2015-04-26 11:38:45,938 INFO [main-SendThread(pc273.emulab.net<http://pc273.emulab.net>< http://pc273.emulab.net>:2181)] zookeeper.ClientCnxn: Opening socket connection to server pc273.emulab.net/155.98.39.73:2181<http://pc273.emulab.net/155.98.39.73:2181>< http://pc273.emulab.net/155.98.39.73:2181>. Will not attempt to authenticate using SASL (unknown error) 2015-04-26 11:38:45,939 INFO [main-SendThread(pc273.emulab.net<http://pc273.emulab.net>< http://pc273.emulab.net>:2181)] zookeeper.ClientCnxn: Socket connection established to pc273.emulab.net/155.98.39.73:2181<http://pc273.emulab.net/155.98.39.73:2181>< http://pc273.emulab.net/155.98.39.73:2181>, initiating session 2015-04-26 11:38:45,940 INFO [main-SendThread(pc273.emulab.net<http://pc273.emulab.net>< http://pc273.emulab.net>:2181)] zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect 2015-04-26 11:38:47,022 INFO [main-SendThread(pc225.emulab.net<http://pc225.emulab.net>< http://pc225.emulab.net>:2181)] zookeeper.ClientCnxn: Opening socket connection to server pc225.emulab.net/155.98.39.25:2181<http://pc225.emulab.net/155.98.39.25:2181>< http://pc225.emulab.net/155.98.39.25:2181>. Will not attempt to authenticate using SASL (unknown error) 2015-04-26 11:38:47,023 INFO [main-SendThread(pc225.emulab.net<http://pc225.emulab.net>< http://pc225.emulab.net>:2181)] zookeeper.ClientCnxn: Socket connection established to pc225.emulab.net/155.98.39.25:2181<http://pc225.emulab.net/155.98.39.25:2181>< http://pc225.emulab.net/155.98.39.25:2181>, initiating session 2015-04-26 11:38:47,025 INFO [main-SendThread(pc225.emulab.net<http://pc225.emulab.net>< http://pc225.emulab.net>:2181)] zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect 2015-04-26 11:38:47,994 INFO [main-SendThread(pc332.emulab.net<http://pc332.emulab.net>< http://pc332.emulab.net>:2181)] zookeeper.ClientCnxn: Opening socket connection to server pc332.emulab.net/155.98.39.132:2181<http://pc332.emulab.net/155.98.39.132:2181>< http://pc332.emulab.net/155.98.39.132:2181>. Will not attempt to authenticate using SASL (unknown error) 2015-04-26 11:39:17,150 INFO [main-SendThread(pc332.emulab.net<http://pc332.emulab.net>< http://pc332.emulab.net>:2181)] zookeeper.ClientCnxn: Client session timed out, have not heard from server in 30025ms for sessionid 0x0, closing socket connection and attempting reconnect 2015-04-26 11:39:17,251 WARN [main] zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper, quorum=pc273.emulab.net<http://pc273.emulab.net>< http://pc273.emulab.net>:2181,pc225.emulab.net<http://pc225.emulab.net><http://pc225.emulab.net :2181,pc332.emulab.net<http://pc332.emulab.net><http://pc332.emulab.net>:2181, exception=org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase 2015-04-26 11:39:17,251 ERROR [main] zookeeper.RecoverableZooKeeper: ZooKeeper create failed after 4 attempts 2015-04-26 11:39:17,255 ERROR [main] master.HMasterCommandLine: Master exiting java.lang.RuntimeException: Failed construction of Master: class org.apache.hadoop.hbase.master.HMaster at org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:1982) at org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:198) at org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:139) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126) at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1996) Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase at org.apache.zookeeper.KeeperException.create(KeeperException.java:99) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783) at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.createNonSequential(RecoverableZooKeeper.java:512) at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.create(RecoverableZooKeeper.java:491) at org.apache.hadoop.hbase.zookeeper.ZKUtil.createWithParents(ZKUtil.java:1252) at org.apache.hadoop.hbase.zookeeper.ZKUtil.createWithParents(ZKUtil.java:1230) at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.createBaseZNodes(ZooKeeperWatcher.java:174) at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:167) at org.apache.hadoop.hbase.regionserver.HRegionServer.<init>(HRegionServer.java:531) at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:333) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) Can someone helps me figure out the problem? Is that because the number of region server and zookeeper nodes are incorrect? Thanks a lot! Bo
