Hi,
I'm currently trying to enable High Availability for my HBase cluster.
I'm using HBase version 1.2.0 provided by Cloudera's cdh5.8.0.
Everything works for a couple hours and then replication stops due to
the exception pasted below. We see sizeOfLogQueue continue to grow
every few minutes. Has anyone else run into this or know how we may
have gotten into this state?
Non Default Configs set:
hbase.region.replica.replication.enabled
hbase.replication
Exception seen:
Wed Nov 09 00:43:27 UTC 2016,
RpcRetryingCaller{globalStartTime=1478652206658, pause=100,
retries=35}, org.apache.hadoop.hbase.regionserver.UnexpectedStateException:
org.apache.hadoop.hbase.regionserver.UnexpectedStateException: Current
snapshot id is -1,passed 1478639480535
at
org.apache.hadoop.hbase.regionserver.DefaultMemStore.clearSnapshot(DefaultMemStore.java:191)
at
org.apache.hadoop.hbase.regionserver.HStore.updateStorefiles(HStore.java:1082)
at
org.apache.hadoop.hbase.regionserver.HStore.access$600(HStore.java:119)
at
org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.replayFlush(HStore.java:2377)
at
org.apache.hadoop.hbase.regionserver.HRegion.replayFlushInStores(HRegion.java:4565)
at
org.apache.hadoop.hbase.regionserver.HRegion.replayWALFlushCommitMarker(HRegion.java:4471)
at
org.apache.hadoop.hbase.regionserver.HRegion.replayWALFlushMarker(HRegion.java:4272)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.doReplayBatchOp(RSRpcServices.java:835)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.replay(RSRpcServices.java:1765)
at
org.apache.hadoop.hbase.protobuf.generated.AdminProtos$AdminService$2.callBlockingMethod(AdminProtos.java:22255)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2170)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:109)
at
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
at java.lang.Thread.run(Thread.java:745)
Thanks,
Tim
