Tim Burrage created AMQ-5671:
--------------------------------
Summary: two server ReplicatedDB cluster fails when Slave restarts
and becomes Master
Key: AMQ-5671
URL: https://issues.apache.org/jira/browse/AMQ-5671
Project: ActiveMQ
Issue Type: Bug
Components: activemq-leveldb-store
Affects Versions: 5.11.1
Environment: Red Hat Enterprise Linux Server release 6.5
Java 1_7_0_21_64bit
Reporter: Tim Burrage
Paired Master/Slave - with a client app publishing and subscribing to them.
I can fail the Master and restart it and the cluster will restart happily.
If I fail the Slave, and it restarts and remains the slave, again everything is
OK.
However, if I fail the Slave and it becomes the Master, it always fails and
shuts down.
This is repeatable 100% of the time assuming the failed Slave becomes the
master upon restart.
2015-03-17 16:11:23,092 | INFO | Stopping BrokerService[localhost] due to
exception, java.io.IOException: 283 | org.apache.activemq.util.DefaultIOExcep
tionHandler | LevelDB IOException handler.
java.io.IOException: 283
at
org.apache.activemq.util.IOExceptionSupport.create(IOExceptionSupport.java:39)[activemq-client-5.11.1.jar:5.11.1]
at
org.apache.activemq.leveldb.LevelDBClient.might_fail(LevelDBClient.scala:552)[activemq-leveldb-store-5.11.1.jar:5.11.1]
at
org.apache.activemq.leveldb.LevelDBClient.might_fail_using_index(LevelDBClient.scala:1044)[activemq-leveldb-store-5.11.1.jar:5.11.1]
at
org.apache.activemq.leveldb.LevelDBClient.collectionCursor(LevelDBClient.scala:1357)[activemq-leveldb-store-5.11.1.jar:5.11.1]
at
org.apache.activemq.leveldb.LevelDBClient.queueCursor(LevelDBClient.scala:1271)[activemq-leveldb-store-5.11.1.jar:5.11.1]
at
org.apache.activemq.leveldb.DBManager.cursorMessages(DBManager.scala:735)[activemq-leveldb-store-5.11.1.jar:5.11.1]
at
org.apache.activemq.leveldb.LevelDBStore$LevelDBMessageStore.recoverNextMessages(LevelDBStore.scala:860)[activemq-leveldb-store-5.11.1.jar:5.11.1]
at
org.apache.activemq.broker.region.cursors.QueueStorePrefetch.doFillBatch(QueueStorePrefetch.java:109)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.broker.region.cursors.AbstractStoreCursor.fillBatch(AbstractStoreCursor.java:381)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.broker.region.cursors.AbstractStoreCursor.reset(AbstractStoreCursor.java:142)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.broker.region.cursors.StoreQueueCursor.reset(StoreQueueCursor.java:159)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.broker.region.Queue.doPageInForDispatch(Queue.java:1897)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.broker.region.Queue.pageInMessages(Queue.java:2119)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.broker.region.Queue.iterate(Queue.java:1596)[activemq-broker-5.11.1.jar:5.11.1]
at
org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:133)[activemq-client-5.11.1.jar:5.11.1]
at
org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:48)[activemq-client-5.11.1.jar:5.11.1]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)[:1.7.0_21]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)[:1.7.0_21]
at java.lang.Thread.run(Thread.java:722)[:1.7.0_21]
2015-03-17 16:11:23,094 | INFO | Apache ActiveMQ 5.11.1 (localhost,
ID:efxln1i03.zit.commerzbank.com-52568-1426608682673-0:1) is shutting down |
org.apache.activemq.broker.BrokerService | IOExceptionHandler: stopping
BrokerService[localhost]
<replicatedLevelDB
hostname="efxln2i03.zit.commerzbank.com"
directory="${activemq.data}/broker/ldn-data-2"
securityToken="LDN.Prod.Broker"
weight="2"
replicas="2"
bind="tcp://efxln2i03.zit.commerzbank.com:61619"
zkAddress="efxln1i03.zit.commerzbank.com:2181,efxln2i03.zit.commerzbank.com:2181"
zkPassword=""
zkPath="/activemq/ldn-leveldb-stores"
/>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
