Hi,
Broker verison : 5.10.0
using Master-slave topology with shared kahadb.
Today we facing very critical production issue due to Kahadb. We got below
mentioned error in broker logs, and after that broker stopped it's transport
connectors and stopped it's services but still it didn't release the lock on
kahadb because of which even failover broker was not able to acquire the
lock and not able to serve the clients.
Broker was in this state for long time unless we manually restarted the
broker. The major concern here is that master broker didn't release the lock
on kahadb because of which failover was not able to get the lock and become
master.
Can you please let us know what was the reason caused this and why master
didn't release the lock ?
/[20150124 10:36:58.665 EST (ActiveMQ Data File Writer)
org.apache.activemq.store.kahadb.disk.journal.DataFileAppender#processQueue
382 INFO] - Journal fai
led while writing at: 1677639
[20150124 10:36:58.706 EST (ActiveMQ Journal Checkpoint Worker)
org.apache.activemq.store.kahadb.MessageDatabase$3#run 364 ERROR] -
Checkpoint failed
java.io.IOException: Input/output error
at java.io.RandomAccessFile.write0(Native Method)
at java.io.RandomAccessFile.write(RandomAccessFile.java:472)
at java.io.RandomAccessFile.writeLong(RandomAccessFile.java:1028)
at
org.apache.activemq.util.RecoverableRandomAccessFile.writeLong(RecoverableRandomAccessFile.java:305)
at
org.apache.activemq.store.kahadb.disk.page.PageFile.writeBatch(PageFile.java:1062)
at
org.apache.activemq.store.kahadb.disk.page.PageFile.flush(PageFile.java:516)
at
org.apache.activemq.store.kahadb.MessageDatabase.checkpointUpdate(MessageDatabase.java:1512)
at
org.apache.activemq.store.kahadb.MessageDatabase$17.execute(MessageDatabase.java:1484)
at
org.apache.activemq.store.kahadb.disk.page.Transaction.execute(Transaction.java:779)
at
org.apache.activemq.store.kahadb.MessageDatabase.checkpointUpdate(MessageDatabase.java:1481)
at
org.apache.activemq.store.kahadb.MessageDatabase.checkpointCleanup(MessageDatabase.java:929)
at
org.apache.activemq.store.kahadb.MessageDatabase$3.run(MessageDatabase.java:353)
/
Thanks,
Anuj
--
View this message in context:
http://activemq.2283324.n4.nabble.com/kahadb-corruption-Checkpoint-failed-java-io-IOException-Input-output-error-tp4690378.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
