Zhuqi Jin created HBASE-24556:
---------------------------------
Summary: HBase upgrade fails from 2.2 to 2.3 fails due to failure
in ZooKeeper
Key: HBASE-24556
URL: https://issues.apache.org/jira/browse/HBASE-24556
Project: HBase
Issue Type: Bug
Components: Zookeeper
Affects Versions: 2.3.0
Reporter: Zhuqi Jin
When we tested upgrading HBase from 2.2 to 2.3, the upgraded node failed to
start due to a known zookeeper failure - ZOOKEEPER-3056.
The error message is shown below:
{code:java}
2020-06-12 23:15:11,432 ERROR [main] master.HMasterCommandLine: Master
exiting2020-06-12 23:15:11,432 ERROR [main] master.HMasterCommandLine: Master
exitingjava.io.IOException: No snapshot found, but there are log entries.
Something is broken! at
org.apache.zookeeper.server.persistence.FileTxnSnapLog.restore(FileTxnSnapLog.java:240)
at org.apache.zookeeper.server.ZKDatabase.loadDataBase(ZKDatabase.java:240) at
org.apache.zookeeper.server.ZooKeeperServer.loadData(ZooKeeperServer.java:290)
at
org.apache.zookeeper.server.ZooKeeperServer.startdata(ZooKeeperServer.java:450)
at
org.apache.zookeeper.server.NIOServerCnxnFactory.startup(NIOServerCnxnFactory.java:764)
at
org.apache.zookeeper.server.ServerCnxnFactory.startup(ServerCnxnFactory.java:98)
at
org.apache.hadoop.hbase.zookeeper.MiniZooKeeperCluster.startup(MiniZooKeeperCluster.java:259)
at
org.apache.hadoop.hbase.zookeeper.MiniZooKeeperCluster.startup(MiniZooKeeperCluster.java:189)
at
org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:210)
at
org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:140)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at
org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:149)
at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:3035)
{code}
It can be reproduced through the following steps:
# Start a single-node HBase 2.2 with the following configuration.
{code:java}
<property>
<name>hbase.rootdir</name>
<value>/var/lib/hbase</value>
</property> <property>
<name>hbase.zookeeper.property.dataDir</name>
<value>/var/lib/zookeeper</value>
</property>{code}
# Use HBase performance evaluation tool to generate data.
{code:java}
/hbase/bin/hbase pe --nomapred --oneCon=true --valueSize=10 --rows=100
sequentialWrite 1{code}
# Upgrade the node to 2.3 with the same configuration. with the same
configuration. The new version node failed to start because of the zookeeper.
HBase 2.2 is using zookeeper-3.4.10.jar, and Kafka 2.3 are using
zookeeper-3.5.7.jar.
The bug has been fixed by ZOOKEEPER-3056 in zookeeper-3.6.0. Should we upgrade
the dependency of HBase 2.3 to use zookeeper-3.6.0.jar? or upgrade the
configuration file of zookeeper-3.5.7.jar?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
