[
https://issues.apache.org/jira/browse/AMQ-8432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Mensinger closed AMQ-8432.
---------------------------------
Resolution: Resolved
It stopped happening after some time, so it had probably something to do with
our setup...
> Broker shuts down randomly after some time
> ------------------------------------------
>
> Key: AMQ-8432
> URL: https://issues.apache.org/jira/browse/AMQ-8432
> Project: ActiveMQ
> Issue Type: Bug
> Affects Versions: 5.16.3
> Reporter: Daniel Mensinger
> Assignee: Jean-Baptiste Onofré
> Priority: Major
> Attachments: activemq.log
>
>
> Our ActiveMQ broker shuts down after a certain amount of time (days) of
> operation for no apparent reason:
>
> {code:java}
> 2021-12-09 15:36:56,221 | DEBUG | WriteChecker: 10017ms elapsed since last
> write check. | org.apache.activemq.transport.AbstractInactivityMonitor |
> ActiveMQ InactivityMonitor WriteCheckTimer
> 2021-12-09 15:36:56,227 | DEBUG | Running WriteCheck[tcp://127.0.0.1:33638] |
> org.apache.activemq.transport.AbstractInactivityMonitor | ActiveMQ
> InactivityMonitor Worker
> 2021-12-09 15:36:56,216 | DEBUG | Running WriteCheck[tcp://127.0.0.1:56489] |
> org.apache.activemq.transport.AbstractInactivityMonitor | ActiveMQ
> InactivityMonitor Worker
> 2021-12-09 15:36:56,254 | DEBUG | WriteChecker: 10000ms elapsed since last
> write check. | org.apache.activemq.transport.AbstractInactivityMonitor |
> ActiveMQ InactivityMonitor WriteCheckTimer
> 2021-12-09 15:36:56,221 | DEBUG | Running WriteCheck[tcp://127.0.0.1:53948] |
> org.apache.activemq.transport.AbstractInactivityMonitor | ActiveMQ
> InactivityMonitor Worker
> 2021-12-09 15:36:56,264 | DEBUG | WriteChecker: 10004ms elapsed since last
> write check. | org.apache.activemq.transport.AbstractInactivityMonitor |
> ActiveMQ InactivityMonitor WriteCheckTimer
> 2021-12-09 15:36:56,273 | DEBUG | Running WriteCheck[tcp://127.0.0.1:37383] |
> org.apache.activemq.transport.AbstractInactivityMonitor | ActiveMQ
> InactivityMonitor Worker
> 2021-12-09 15:36:56,284 | DEBUG | WriteChecker: 10000ms elapsed since last
> write check. | org.apache.activemq.transport.AbstractInactivityMonitor |
> ActiveMQ InactivityMonitor WriteCheckTimer
> 2021-12-09 15:36:56,299 | DEBUG | Running WriteCheck[tcp://127.0.0.1:45952] |
> org.apache.activemq.transport.AbstractInactivityMonitor | ActiveMQ
> InactivityMonitor Worker
> 2021-12-09 15:36:56,302 | DEBUG | Running WriteCheck[tcp://127.0.0.1:58722] |
> org.apache.activemq.transport.AbstractInactivityMonitor | ActiveMQ
> InactivityMonitor Worker
> 2021-12-09 15:36:56,302 | DEBUG | WriteChecker: 10001ms elapsed since last
> write check. | org.apache.activemq.transport.AbstractInactivityMonitor |
> ActiveMQ InactivityMonitor WriteCheckTimer
> 2021-12-09 15:36:56,316 | DEBUG | Running WriteCheck[tcp://127.0.0.1:35706] |
> org.apache.activemq.transport.AbstractInactivityMonitor | ActiveMQ
> InactivityMonitor Worker
> 2021-12-09 15:36:56,328 | DEBUG | WriteChecker: 10006ms elapsed since last
> write check. | org.apache.activemq.transport.AbstractInactivityMonitor |
> ActiveMQ InactivityMonitor WriteCheckTimer
> 2021-12-09 15:36:56,344 | DEBUG | WriteChecker: 10003ms elapsed since last
> write check. | org.apache.activemq.transport.AbstractInactivityMonitor |
> ActiveMQ InactivityMonitor WriteCheckTimer
> 2021-12-09 15:36:56,351 | DEBUG | Running WriteCheck[tcp://127.0.0.1:44245] |
> org.apache.activemq.transport.AbstractInactivityMonitor | ActiveMQ
> InactivityMonitor Worker
> 2021-12-09 15:36:56,344 | DEBUG | Running WriteCheck[tcp://127.0.0.1:48056] |
> org.apache.activemq.transport.AbstractInactivityMonitor | ActiveMQ
> InactivityMonitor Worker
> 2021-12-09 15:36:57,085 | DEBUG | Checkpoint started. |
> org.apache.activemq.store.kahadb.MessageDatabase | ActiveMQ Journal
> Checkpoint Worker
> 2021-12-09 15:36:57,088 | DEBUG | Checkpoint done. |
> org.apache.activemq.store.kahadb.MessageDatabase | ActiveMQ Journal
> Checkpoint Worker
> 2021-12-09 15:36:57,718 | INFO | Apache ActiveMQ 5.16.3 (localhost,
> ID:****************:1) is shutting down |
> org.apache.activemq.broker.BrokerService | ActiveMQ ShutdownHook
> 2021-12-09 15:36:57,722 | DEBUG | Caught exception, must be shutting down.
> This exception is ignored. | org.apache.activemq.broker.BrokerService |
> ActiveMQ ShutdownHook
> java.lang.IllegalStateException: Shutdown in progress
> at
> java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)[:1.8.0_201]
> at java.lang.Runtime.removeShutdownHook(Runtime.java:239)[:1.8.0_201]
> at
> org.apache.activemq.broker.BrokerService.removeShutdownHook(BrokerService.java:2576)[activemq-broker-5.16.3.jar:5.16.3]
> at
> org.apache.activemq.broker.BrokerService.stop(BrokerService.java:847)[activemq-broker-5.16.3.jar:5.16.3]
> at
> org.apache.activemq.xbean.XBeanBrokerService.stop(XBeanBrokerService.java:122)[activemq-spring-5.16.3.jar:5.16.3]
> at
> org.apache.activemq.broker.BrokerService.containerShutdown(BrokerService.java:2599)[activemq-broker-5.16.3.jar:5.16.3]
> at
> org.apache.activemq.broker.BrokerService$7.run(BrokerService.java:2566)[activemq-broker-5.16.3.jar:5.16.3]
> 2021-12-09 15:36:57,728 | DEBUG | Unregistering MBean
> org.apache.activemq:type=Broker,brokerName=localhost,connector=clientConnectors,connectorName=openwire
> | org.apache.activemq.broker.jmx.ManagementContext | ActiveMQ ShutdownHook
> 2021-12-09 15:36:57,746 | DEBUG | Stopping connection: tcp://127.0.0.1:46306
> | org.apache.activemq.broker.TransportConnection | ActiveMQ
> BrokerService[localhost] Task-17115
> 2021-12-09 15:36:57,746 | DEBUG | Unregistering MBean
> org.apache.activemq:type=Broker,brokerName=localhost,connector=clientConnectors,connectorName=openwire,connectionViewType=clientId,connectionName=ID_****************-0_1
> | org.apache.activemq.broker.jmx.ManagementContext | ActiveMQ ShutdownHook
> 2021-12-09 15:36:57,863 | DEBUG | Unregistering MBean
> org.apache.activemq:type=Broker,brokerName=localhost,connector=clientConnectors,connectorName=openwire,connectionViewType=remoteAddress,connectionName=tcp_//127.0.0.1_46306
> | org.apache.activemq.broker.jmx.ManagementContext | ActiveMQ ShutdownHook
> 2021-12-09 15:36:57,868 | DEBUG | Stopping transport
> tcp:///127.0.0.1:46306@61616 | org.apache.activemq.transport.tcp.TcpTransport
> | ActiveMQ BrokerService[localhost] Task-17115
> 2021-12-09 15:36:57,918 | DEBUG | Initialized TaskRunnerFactory[ActiveMQ
> Task] using ExecutorService:
> java.util.concurrent.ThreadPoolExecutor@38deec91[Running, pool size = 0,
> active threads = 0, queued tasks = 0, completed tasks = 0] |
> org.apache.activemq.thread.TaskRunnerFactory | ActiveMQ
> BrokerService[localhost] Task-17115
> 2021-12-09 15:36:57,955 | DEBUG | Closed socket
> Socket[addr=/127.0.0.1,port=46306,localport=61616] |
> org.apache.activemq.transport.tcp.TcpTransport | ActiveMQ Task-1
> 2021-12-09 15:36:57,970 | DEBUG | Forcing shutdown of ExecutorService:
> java.util.concurrent.ThreadPoolExecutor@38deec91[Running, pool size = 1,
> active threads = 0, queued tasks = 0, completed tasks = 1] |
> org.apache.activemq.util.ThreadPoolUtils | ActiveMQ BrokerService[localhost]
> Task-17115
> 2021-12-09 15:36:57,975 | DEBUG | Stopped transport: tcp://127.0.0.1:46306 |
> org.apache.activemq.broker.TransportConnection | ActiveMQ
> BrokerService[localhost] Task-17115
> 2021-12-09 15:36:58,086 | DEBUG | Cleaning up connection resources:
> tcp://127.0.0.1:46306 | org.apache.activemq.broker.TransportConnection |
> ActiveMQ BrokerService[localhost] Task-17115
> 2021-12-09 15:36:58,140 | DEBUG | remove connection id: ID:****************:1
> | org.apache.activemq.broker.TransportConnection | ActiveMQ
> BrokerService[localhost] Task-17115
> ......
> ......
> ......
> 2021-12-09 15:37:16,694 | DEBUG | Unregistering MBean
> org.apache.activemq:type=Broker,brokerName=localhost,destinationType=Queue,destinationName=tup.lisam.distribution
> | org.apache.activemq.broker.jmx.ManagementContext | ActiveMQ ShutdownHook
> 2021-12-09 15:37:16,698 | DEBUG | Unregistering MBean
> org.apache.activemq:type=Broker,brokerName=localhost,destinationType=Topic,destinationName=ActiveMQ.Advisory.MessageDLQd.Queue.tup.lisam.standort2lisam.normal
> | org.apache.activemq.broker.jmx.ManagementContext | ActiveMQ ShutdownHook
> 2021-12-09 15:37:16,703 | DEBUG | Unregistering MBean
> org.apache.activemq:type=Broker,brokerName=localhost,destinationType=Topic,destinationName=ActiveMQ.Advisory.MasterBroker
> | org.apache.activemq.broker.jmx.ManagementContext | ActiveMQ ShutdownHook
> 2021-12-09 15:37:16,708 | INFO |
> PListStore:[/proj/lisam/tmp/activemq-data/localhost/tmp_storage] stopped |
> org.apache.activemq.store.kahadb.plist.PListStoreImpl | ActiveMQ ShutdownHook
> 2021-12-09 15:37:16,711 | INFO | Stopping async queue tasks |
> org.apache.activemq.store.kahadb.KahaDBStore | ActiveMQ ShutdownHook
> 2021-12-09 15:37:16,715 | INFO | Stopping async topic tasks |
> org.apache.activemq.store.kahadb.KahaDBStore | ActiveMQ ShutdownHook
> 2021-12-09 15:37:16,718 | DEBUG | Forcing shutdown of ExecutorService:
> org.apache.activemq.store.kahadb.KahaDBStore$StoreTaskExecutor@66d10a8b[Running,
> pool size = 1, active threads = 0, queued tasks = 0, completed tasks = 3331]
> | org.apache.activemq.util.ThreadPoolUtils | ActiveMQ ShutdownHook
> 2021-12-09 15:37:16,724 | DEBUG | Forcing shutdown of ExecutorService:
> org.apache.activemq.store.kahadb.KahaDBStore$StoreTaskExecutor@11ab93c9[Running,
> pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0] |
> org.apache.activemq.util.ThreadPoolUtils | ActiveMQ ShutdownHook
> 2021-12-09 15:37:17,142 | INFO | Stopped KahaDB |
> org.apache.activemq.store.kahadb.KahaDBStore | ActiveMQ ShutdownHook
> 2021-12-09 15:37:17,215 | DEBUG | Checkpoint started. |
> org.apache.activemq.store.kahadb.MessageDatabase | ActiveMQ ShutdownHook
> 2021-12-09 15:37:17,218 | DEBUG | Checkpoint done. |
> org.apache.activemq.store.kahadb.MessageDatabase | ActiveMQ ShutdownHook
> 2021-12-09 15:37:17,300 | DEBUG | Shutdown of ExecutorService:
> java.util.concurrent.ScheduledThreadPoolExecutor@4b9a60ef[Terminated, pool
> size = 0, active threads = 0, queued tasks = 0, completed tasks = 29621] is
> shutdown: true and terminated: true took: 0.057 seconds. |
> org.apache.activemq.util.ThreadPoolUtils | ActiveMQ ShutdownHook
> 2021-12-09 15:37:17,308 | DEBUG | Shutdown of ExecutorService:
> java.util.concurrent.Executors$DelegatedScheduledExecutorService@219b3c9f is
> shutdown: true and terminated: true took: 0.001 seconds. |
> org.apache.activemq.util.ThreadPoolUtils | ActiveMQ ShutdownHook
> 2021-12-09 15:37:17,315 | DEBUG | Unregistering MBean
> org.apache.activemq:type=Broker,brokerName=localhost,service=PersistenceAdapter,instanceName=KahaDBPersistenceAdapter[/proj/lisam/tmp/activemq-data/kahadb_Index_/proj/lisam/tmp/activemq-data/kahadb]
> | org.apache.activemq.broker.jmx.ManagementContext | ActiveMQ ShutdownHook
> 2021-12-09 15:37:17,321 | DEBUG | Unregistering MBean
> org.apache.activemq:type=Broker,brokerName=localhost,service=Health |
> org.apache.activemq.broker.jmx.ManagementContext | ActiveMQ ShutdownHook
> 2021-12-09 15:37:17,326 | DEBUG | Unregistering MBean
> org.apache.activemq:type=Broker,brokerName=localhost |
> org.apache.activemq.broker.jmx.ManagementContext | ActiveMQ ShutdownHook
> 2021-12-09 15:37:17,533 | DEBUG | Unregistering MBean
> org.apache.activemq:type=Broker,brokerName=localhost,service=Log4JConfiguration
> | org.apache.activemq.broker.jmx.ManagementContext | ActiveMQ ShutdownHook
> 2021-12-09 15:37:18,026 | DEBUG | Shutdown of ExecutorService:
> java.util.concurrent.ThreadPoolExecutor@7331196b[Terminated, pool size = 0,
> active threads = 0, queued tasks = 0, completed tasks = 2156103] is shutdown:
> true and terminated: false took: 0.001 seconds. |
> org.apache.activemq.util.ThreadPoolUtils | ActiveMQ ShutdownHook
> 2021-12-09 15:37:18,035 | DEBUG | Forcing shutdown of ExecutorService:
> java.util.concurrent.ThreadPoolExecutor@6f7a041c[Running, pool size = 1,
> active threads = 0, queued tasks = 0, completed tasks = 1222] |
> org.apache.activemq.util.ThreadPoolUtils | ActiveMQ ShutdownHook
> 2021-12-09 15:37:18,042 | INFO | Apache ActiveMQ 5.16.3 (localhost,
> ID:****************:1) uptime 10 days 6 hours |
> org.apache.activemq.broker.BrokerService | ActiveMQ ShutdownHook
> 2021-12-09 15:37:18,048 | INFO | Apache ActiveMQ 5.16.3 (localhost,
> ID:****************:1) is shutdown | org.apache.activemq.broker.BrokerService
> | ActiveMQ ShutdownHook
> 2021-12-09 15:37:18,056 | INFO | Closing
> org.apache.activemq.xbean.XBeanBrokerFactory$1@22eeefeb: startup date [Mon
> Nov 29 08:45:17 CET 2021]; root of context hierarchy |
> org.apache.activemq.xbean.XBeanBrokerFactory$1 | ActiveMQ ShutdownHook
> 2021-12-09 15:37:18,696 | DEBUG | JspServlet.destroy() |
> org.apache.jasper.servlet.JspServlet | ActiveMQ ShutdownHook
> 2021-12-09 15:37:19,256 | DEBUG | JspServlet.destroy() |
> org.apache.jasper.servlet.JspServlet | ActiveMQ ShutdownHook{code}
>
> Is there something external from our processes that could trigger a shutdown?
> If yes, how can we figure out what this is?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
