Bhanu created AMQ-4264: -------------------------- Summary: ActiveMQ dying with exception detected missing/corrupt journal files Key: AMQ-4264 URL: https://issues.apache.org/jira/browse/AMQ-4264 Project: ActiveMQ Issue Type: Bug Components: Broker Affects Versions: 5.6.0 Environment: linux Reporter: Bhanu
Hi, Yesterday, our ActiveMQ broker suddenly died with the exception pasted below. It failed to come up on repeated attempts and we could only bring it back after clearing the kahadb data and redo files. We havent seen this issue in the past. Can you suggest some pointers here on why this would have happened suddenly? Is this a known issue which is going to be fixed in future versions ? ERROR: java.lang.RuntimeException: Failed to execute start task. Reason: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'org.apache.activemq.xbean.XBeanBrokerService#0' defined in URL [file:/prod/tools/base/etc/config/activemq/amq_prod_broker_config.xml]: Invocation of init method failed; nested exception is java.io.IOException: Detected missing/corrupt journal files. 124512 messages affected. java.lang.RuntimeException: Failed to execute start task. Reason: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'org.apache.activemq.xbean.XBeanBrokerService#0' defined in URL [file:/prod/tools/base/etc/config/activemq/amq_prod_broker_config.xml]: Invocation of init method failed; nested exception is java.io.IOException: Detected missing/corrupt journal files. 124512 messages affected. at org.apache.activemq.console.command.StartCommand.runTask(StartCommand.java:98) at org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:57) at org.apache.activemq.console.command.ShellCommand.runTask(ShellCommand.java:148) at org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:57) at org.apache.activemq.console.command.ShellCommand.main(ShellCommand.java:90) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.apache.activemq.console.Main.runTaskClass(Main.java:257) at org.apache.activemq.console.Main.main(Main.java:111) Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'org.apache.activemq.xbean.XBeanBrokerService#0' defined in URL [file:/prod/tools/base/etc/config/activemq/amq_prod_broker_config.xml]: Invocation of init method failed; nested exception is java.io.IOException: Detected missing/corrupt journal files. 124512 messages affected. at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1420) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:519) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456) at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:293) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222) at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:290) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:192) at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:585) at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:895) at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:425) at org.apache.xbean.spring.context.ResourceXmlApplicationContext.<init>(ResourceXmlApplicationContext.java:64) at org.apache.xbean.spring.context.ResourceXmlApplicationContext.<init>(ResourceXmlApplicationContext.java:52) at org.apache.activemq.xbean.XBeanBrokerFactory$1.<init>(XBeanBrokerFactory.java:108) at org.apache.activemq.xbean.XBeanBrokerFactory.createApplicationContext(XBeanBrokerFactory.java:108) at org.apache.activemq.xbean.XBeanBrokerFactory.createBroker(XBeanBrokerFactory.java:72) at org.apache.activemq.broker.BrokerFactory.createBroker(BrokerFactory.java:71) at org.apache.activemq.broker.BrokerFactory.createBroker(BrokerFactory.java:54) at org.apache.activemq.console.command.StartCommand.startBroker(StartCommand.java:115) at org.apache.activemq.console.command.StartCommand.runTask(StartCommand.java:91) ... 10 more Caused by: java.io.IOException: Detected missing/corrupt journal files. 124512 messages affected. at org.apache.activemq.store.kahadb.MessageDatabase.recoverIndex(MessageDatabase.java:740) at org.apache.activemq.store.kahadb.MessageDatabase$6.execute(MessageDatabase.java:550) at org.apache.kahadb.page.Transaction.execute(Transaction.java:769) at org.apache.activemq.store.kahadb.MessageDatabase.recover(MessageDatabase.java:548) at org.apache.activemq.store.kahadb.MessageDatabase.open(MessageDatabase.java:367) at org.apache.activemq.store.kahadb.MessageDatabase.load(MessageDatabase.java:424) at org.apache.activemq.store.kahadb.MessageDatabase.doStart(MessageDatabase.java:240) at org.apache.activemq.store.kahadb.KahaDBStore.doStart(KahaDBStore.java:180) at org.apache.activemq.util.ServiceSupport.start(ServiceSupport.java:54) at org.apache.activemq.store.kahadb.KahaDBPersistenceAdapter.start(KahaDBPersistenceAdapter.java:192) at org.apache.activemq.broker.BrokerService.start(BrokerService.java:525) at org.apache.activemq.xbean.XBeanBrokerService.afterPropertiesSet(XBeanBrokerService.java:60) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeCustomInitMethod(AbstractAutowireCapableBeanFactory.java:1544) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1485) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1417) ... 28 more -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira