Christopher G. Stach II wrote:
> Christopher G. Stach II wrote:
>> I am forcing a deadlock to generate rollbacks, and it doesn't look like
>> they are being retried [successfully]. Instead, this happens:
>>
>
> After getting about four deadlocks, rollbacks, and no retries (I think),
> I let the server sit for a while and eventually got this:
>
> com.caucho.server.admin.PingThread.checkJVMDeadlock JDK detected
> deadlock. Restarting Resin.
>
> I'm not sure what that's checking, but we have no problems like this
> with 3.2.2.
>
Here's a thread dump of the deadlock.
--
Christopher G. Stach II
Full thread dump Java HotSpot(TM) Client VM (1.5.0_03-b07 mixed mode):
"JDWP Command Reader" daemon prio=5 tid=0x101d4bc0 nid=0x1b78 runnable
[0x00000000..0x00000000]
"resin-46" daemon prio=5 tid=0x102b0638 nid=0x1a70 in Object.wait()
[0x0f51f000..0x0f51fa68]
at java.lang.Object.wait(Native Method)
^CTerminate batch job (Y/N)? - waiting on <0x032c0300> (a
com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:474)
- locked <0x032c0300> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"ActiveMQ Task" daemon prio=5 tid=0x102a8340 nid=0x1c20 waiting for monitor
entry [0x1125f000..0x1125fae8]
at
org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:44)
- waiting to lock <0x04abda20> (a java.lang.Object)
at
org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:56)
at
org.apache.activemq.ActiveMQConnection.asyncSendPacket(ActiveMQConnection.java:1088)
at
org.apache.activemq.ActiveMQConnection.close(ActiveMQConnection.java:543)
- locked <0x04aa67a8> (a org.apache.activemq.ActiveMQConnection)
at
org.apache.activemq.ra.ActiveMQEndpointWorker.safeClose(ActiveMQEndpointWorker.java:99)
at
org.apache.activemq.ra.ActiveMQEndpointWorker.disconnect(ActiveMQEndpointWorker.java:233)
- locked <0x04a9a8b8> (a org.apache.activemq.ra.ActiveMQEndpointWorker)
at
org.apache.activemq.ra.ActiveMQEndpointWorker.reconnect(ActiveMQEndpointWorker.java:249)
at
org.apache.activemq.ra.ActiveMQEndpointWorker.access$300(ActiveMQEndpointWorker.java:45)
at
org.apache.activemq.ra.ActiveMQEndpointWorker$1$1.onException(ActiveMQEndpointWorker.java:149)
at
org.apache.activemq.ActiveMQConnection.onAsyncException(ActiveMQConnection.java:1398)
at
org.apache.activemq.ActiveMQConnection.onCommand(ActiveMQConnection.java:1379)
at
org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:93)
at
org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:70)
at
org.apache.activemq.transport.vm.VMTransport.oneway(VMTransport.java:76)
at
org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:44)
- locked <0x04ab0f90> (a java.lang.Object)
at
org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:56)
at
org.apache.activemq.broker.TransportConnection.dispatch(TransportConnection.java:211)
at
org.apache.activemq.broker.AbstractConnection.processDispatch(AbstractConnection.java:575)
at
org.apache.activemq.broker.AbstractConnection.iterate(AbstractConnection.java:584)
at
org.apache.activemq.thread.SimpleTaskRunner.runTask(SimpleTaskRunner.java:110)
at
org.apache.activemq.thread.SimpleTaskRunner.access$100(SimpleTaskRunner.java:25)
at
org.apache.activemq.thread.SimpleTaskRunner$1.run(SimpleTaskRunner.java:43)
at
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
at
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
at java.lang.Thread.run(Thread.java:595)
"ActiveMQ Scheduler" daemon prio=5 tid=0x101b2090 nid=0x135c waiting for
monitor entry [0x1121f000..0x1121fd68]
at
org.apache.activemq.ra.ServerSessionPoolImpl.getServerSession(ServerSessionPoolImpl.java:95)
- waiting to lock <0x04aa6750> (a
org.apache.activemq.ra.ServerSessionPoolImpl)
at
org.apache.activemq.ActiveMQConnectionConsumer.dispatch(ActiveMQConnectionConsumer.java:135)
at org.apache.activemq.ActiveMQSession$2$1.run(ActiveMQSession.java:709)
at
edu.emory.mathcs.backport.java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:431)
at
edu.emory.mathcs.backport.java.util.concurrent.FutureTask.run(FutureTask.java:176)
at
edu.emory.mathcs.backport.java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExe
cutor.java:215)
at
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
at
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
at java.lang.Thread.run(Thread.java:595)
"ActiveMQ Scheduler" daemon prio=5 tid=0x101b1f10 nid=0x1cb8 waiting for
monitor entry [0x111df000..0x111dfc68]
at
org.apache.activemq.ra.ServerSessionPoolImpl.getServerSession(ServerSessionPoolImpl.java:95)
- waiting to lock <0x04aa6750> (a
org.apache.activemq.ra.ServerSessionPoolImpl)
at
org.apache.activemq.ActiveMQConnectionConsumer.dispatch(ActiveMQConnectionConsumer.java:135)
at org.apache.activemq.ActiveMQSession$2$1.run(ActiveMQSession.java:709)
at
edu.emory.mathcs.backport.java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:431)
at
edu.emory.mathcs.backport.java.util.concurrent.FutureTask.run(FutureTask.java:176)
at
edu.emory.mathcs.backport.java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExe
cutor.java:215)
at
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
at
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
at java.lang.Thread.run(Thread.java:595)
"ActiveMQ Scheduler" daemon prio=5 tid=0x102e86c0 nid=0x105c waiting for
monitor entry [0x1119f000..0x1119fbe8]
at
org.apache.activemq.ra.ServerSessionPoolImpl.getServerSession(ServerSessionPoolImpl.java:95)
- waiting to lock <0x04aa6750> (a
org.apache.activemq.ra.ServerSessionPoolImpl)
at
org.apache.activemq.ActiveMQConnectionConsumer.dispatch(ActiveMQConnectionConsumer.java:135)
at org.apache.activemq.ActiveMQSession$2$1.run(ActiveMQSession.java:709)
at
edu.emory.mathcs.backport.java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:431)
at
edu.emory.mathcs.backport.java.util.concurrent.FutureTask.run(FutureTask.java:176)
at
edu.emory.mathcs.backport.java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExe
cutor.java:215)
at
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
at
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
at java.lang.Thread.run(Thread.java:595)
"ActiveMQ Scheduler" daemon prio=5 tid=0x10330870 nid=0x11c4 waiting for
monitor entry [0x1115f000..0x1115fb68]
at
org.apache.activemq.ra.ServerSessionPoolImpl.getServerSession(ServerSessionPoolImpl.java:95)
- waiting to lock <0x04aa6750> (a
org.apache.activemq.ra.ServerSessionPoolImpl)
at
org.apache.activemq.ActiveMQConnectionConsumer.dispatch(ActiveMQConnectionConsumer.java:135)
at org.apache.activemq.ActiveMQSession$2$1.run(ActiveMQSession.java:709)
at
edu.emory.mathcs.backport.java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:431)
at
edu.emory.mathcs.backport.java.util.concurrent.FutureTask.run(FutureTask.java:176)
at
edu.emory.mathcs.backport.java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExe
cutor.java:215)
at
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
at
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
at java.lang.Thread.run(Thread.java:595)
"ActiveMQ Scheduler" daemon prio=5 tid=0x102e3e20 nid=0x1a94 waiting for
monitor entry [0x10edf000..0x10edfce8]
at
org.apache.activemq.ActiveMQConnection.checkClosedOrFailed(ActiveMQConnection.java:1165)
- waiting to lock <0x04aa67a8> (a
org.apache.activemq.ActiveMQConnection)
at
org.apache.activemq.ActiveMQConnection.createSession(ActiveMQConnection.java:251)
at
org.apache.activemq.ra.ServerSessionPoolImpl.createServerSessionImpl(ServerSessionPoolImpl.java:60)
at
org.apache.activemq.ra.ServerSessionPoolImpl.getServerSession(ServerSessionPoolImpl.java:113)
- locked <0x04aa6750> (a org.apache.activemq.ra.ServerSessionPoolImpl)
at
org.apache.activemq.ActiveMQConnectionConsumer.dispatch(ActiveMQConnectionConsumer.java:135)
at org.apache.activemq.ActiveMQSession$2$1.run(ActiveMQSession.java:709)
at
edu.emory.mathcs.backport.java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:431)
at
edu.emory.mathcs.backport.java.util.concurrent.FutureTask.run(FutureTask.java:176)
at
edu.emory.mathcs.backport.java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExe
cutor.java:215)
at
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
at
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
at java.lang.Thread.run(Thread.java:595)
"resin-45" daemon prio=5 tid=0x101ae848 nid=0x1a84 in Object.wait()
[0x1111f000..0x1111fae8]
at java.lang.Object.wait(Native Method)
- waiting on <0x05208550> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:474)
- locked <0x05208550> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"resin-44" daemon prio=5 tid=0x10370df0 nid=0x8e8 in Object.wait()
[0x110df000..0x110dfb68]
at java.lang.Object.wait(Native Method)
- waiting on <0x05208580> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:474)
- locked <0x05208580> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"resin-43" daemon prio=5 tid=0x1038a5b0 nid=0x1240 waiting for monitor entry
[0x1109f000..0x1109fce8]
at
org.apache.activemq.ra.ServerSessionPoolImpl.removeFromPool(ServerSessionPoolImpl.java:170)
- waiting to lock <0x04aa6750> (a
org.apache.activemq.ra.ServerSessionPoolImpl)
at
org.apache.activemq.ra.ServerSessionImpl.run(ServerSessionImpl.java:177)
- locked <0x05224f20> (a java.lang.Object)
- locked <0x05208ab0> (a org.apache.activemq.ra.ServerSessionImpl)
at com.caucho.jca.WorkThread.run(WorkThread.java:99)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:490)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"resin-tcp-connection-*:8088-27" daemon prio=5 tid=0x1038a1c8 nid=0x1f3c
runnable [0x1105f000..0x1105fd68]
at com.caucho.vfs.JniServerSocketImpl.nativeAccept(Native Method)
at
com.caucho.vfs.JniServerSocketImpl.accept(JniServerSocketImpl.java:184)
at com.caucho.server.port.Port.accept(Port.java:766)
at com.caucho.server.port.TcpConnection.run(TcpConnection.java:347)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:490)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"resin-tcp-connection-*:8088-1" daemon prio=5 tid=0x1033f460 nid=0x1a4c
runnable [0x1101f000..0x1101f9e8]
at com.caucho.vfs.JniServerSocketImpl.nativeAccept(Native Method)
at
com.caucho.vfs.JniServerSocketImpl.accept(JniServerSocketImpl.java:184)
at com.caucho.server.port.Port.accept(Port.java:766)
at com.caucho.server.port.TcpConnection.run(TcpConnection.java:347)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:490)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"resin-40" daemon prio=5 tid=0x102a3268 nid=0x14b4 in Object.wait()
[0x10fdf000..0x10fdfa68]
at java.lang.Object.wait(Native Method)
- waiting on <0x05192398> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:474)
- locked <0x05192398> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"resin-tcp-connection-*:8088-15" daemon prio=5 tid=0x1022a540 nid=0x1f48
runnable [0x10b5f000..0x10b5fae8]
at com.caucho.vfs.JniServerSocketImpl.nativeAccept(Native Method)
at
com.caucho.vfs.JniServerSocketImpl.accept(JniServerSocketImpl.java:184)
at com.caucho.server.port.Port.accept(Port.java:766)
at com.caucho.server.port.TcpConnection.run(TcpConnection.java:347)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:490)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"QuartzScheduler_BOEClusteredScheduler-dreamtime1144877191827_MisfireHandler"
prio=5 tid=0x1028d1d8 nid=0xec8 waiting on condition [
0x10f5f000..0x10f5fb68]
at java.lang.Thread.sleep(Native Method)
at
org.quartz.impl.jdbcjobstore.JobStoreSupport$MisfireHandler.run(JobStoreSupport.java:2481)
"QuartzScheduler_BOEClusteredScheduler-dreamtime1144877191827_ClusterManager"
prio=7 tid=0x103b5d18 nid=0xf3c waiting on condition [
0x10f1f000..0x10f1fbe8]
at java.lang.Thread.sleep(Native Method)
at
org.quartz.impl.jdbcjobstore.JobStoreSupport$ClusterManager.run(JobStoreSupport.java:2404)
"resin-38" daemon prio=5 tid=0x103db518 nid=0x18f4 waiting for monitor entry
[0x10f9f000..0x10f9fc68]
at
org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:44)
- waiting to lock <0x04ab0f90> (a java.lang.Object)
at
org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:56)
at
org.apache.activemq.broker.TransportConnection.dispatch(TransportConnection.java:211)
at
org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:64)
at
org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:93)
at
org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:70)
at
org.apache.activemq.transport.vm.VMTransport.oneway(VMTransport.java:76)
at
org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:44)
- locked <0x04abda20> (a java.lang.Object)
at
org.apache.activemq.transport.ResponseCorrelator.asyncRequest(ResponseCorrelator.java:64)
at
org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:69)
at
org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1108)
at
org.apache.activemq.TransactionContext.setXid(TransactionContext.java:550)
at
org.apache.activemq.TransactionContext.end(TransactionContext.java:327)
at
org.apache.activemq.ra.LocalAndXATransaction.end(LocalAndXATransaction.java:89)
at
org.apache.geronimo.transaction.manager.WrapperNamedXAResource.end(WrapperNamedXAResource.java:51)
at
com.caucho.transaction.TransactionImpl.delistResource(TransactionImpl.java:386)
at org.jencks.XAEndpoint.afterDelivery(XAEndpoint.java:97)
at
org.apache.activemq.ra.MessageEndpointProxy$MessageEndpointAlive.afterDelivery(MessageEndpointProxy.java:125)
at
org.apache.activemq.ra.MessageEndpointProxy.afterDelivery(MessageEndpointProxy.java:64)
at
org.apache.activemq.ra.ServerSessionImpl.afterDelivery(ServerSessionImpl.java:214)
at org.apache.activemq.ActiveMQSession.run(ActiveMQSession.java:723)
at
org.apache.activemq.ra.ServerSessionImpl.run(ServerSessionImpl.java:163)
- locked <0x0514ef50> (a org.apache.activemq.ra.ServerSessionImpl)
at com.caucho.jca.WorkThread.run(WorkThread.java:99)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:490)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"resin-37" daemon prio=5 tid=0x0ed099a0 nid=0x304 waiting for monitor entry
[0x10e9f000..0x10e9fa68]
at
org.apache.activemq.ra.ServerSessionPoolImpl.removeFromPool(ServerSessionPoolImpl.java:170)
- waiting to lock <0x04aa6750> (a
org.apache.activemq.ra.ServerSessionPoolImpl)
at
org.apache.activemq.ra.ServerSessionImpl.run(ServerSessionImpl.java:177)
- locked <0x050d0ab0> (a java.lang.Object)
- locked <0x050c2db8> (a org.apache.activemq.ra.ServerSessionImpl)
at com.caucho.jca.WorkThread.run(WorkThread.java:99)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:490)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"resin-36" daemon prio=5 tid=0x1038bc08 nid=0x1984 in Object.wait()
[0x10e5f000..0x10e5fae8]
at java.lang.Object.wait(Native Method)
- waiting on <0x04f965e8> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:474)
- locked <0x04f965e8> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"resin-35" daemon prio=5 tid=0x1027ea00 nid=0x1554 in Object.wait()
[0x10e1f000..0x10e1fc68]
at java.lang.Object.wait(Native Method)
- waiting on <0x04f96798> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:474)
- locked <0x04f96798> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"resin-tcp-connection-*:8088-7" daemon prio=5 tid=0x1039e7d8 nid=0x1dec
runnable [0x10ddf000..0x10ddfce8]
at com.caucho.vfs.JniServerSocketImpl.nativeAccept(Native Method)
at
com.caucho.vfs.JniServerSocketImpl.accept(JniServerSocketImpl.java:184)
at com.caucho.server.port.Port.accept(Port.java:766)
at com.caucho.server.port.TcpConnection.run(TcpConnection.java:347)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:490)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"resin-tcp-connection-*:8088-23" daemon prio=5 tid=0x1039cc00 nid=0x1d04
runnable [0x10d9f000..0x10d9fd68]
at com.caucho.vfs.JniServerSocketImpl.nativeAccept(Native Method)
at
com.caucho.vfs.JniServerSocketImpl.accept(JniServerSocketImpl.java:184)
at com.caucho.server.port.Port.accept(Port.java:766)
at com.caucho.server.port.TcpConnection.run(TcpConnection.java:347)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:490)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"resin-32" daemon prio=5 tid=0x10252458 nid=0x1f80 in Object.wait()
[0x10d5f000..0x10d5f9e8]
at java.lang.Object.wait(Native Method)
- waiting on <0x04f969a8> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:474)
- locked <0x04f969a8> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"resin-31" daemon prio=5 tid=0x1034f9e0 nid=0x1d8c in Object.wait()
[0x10d1f000..0x10d1fa68]
at java.lang.Object.wait(Native Method)
- waiting on <0x04f969d8> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:474)
- locked <0x04f969d8> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"resin-port-6802" daemon prio=5 tid=0x0f0101a0 nid=0x1eb8 in Object.wait()
[0x10cdf000..0x10cdfae8]
at java.lang.Object.wait(Native Method)
- waiting on <0x038e31f0> (a com.caucho.server.cluster.ClusterPort)
at com.caucho.server.port.Port.run(Port.java:905)
- locked <0x038e31f0> (a com.caucho.server.cluster.ClusterPort)
at java.lang.Thread.run(Thread.java:595)
"resin-port-8088" daemon prio=5 tid=0x1026fe70 nid=0x18cc in Object.wait()
[0x0ff2f000..0x0ff2fb68]
at java.lang.Object.wait(Native Method)
- waiting on <0x038f8098> (a com.caucho.server.port.Port)
at com.caucho.server.port.Port.run(Port.java:905)
- locked <0x038f8098> (a com.caucho.server.port.Port)
at java.lang.Thread.run(Thread.java:595)
"resin-30" daemon prio=5 tid=0x102204f0 nid=0x1b04 waiting for monitor entry
[0x10c9f000..0x10c9f9e8]
at
org.apache.activemq.ra.ServerSessionPoolImpl.removeFromPool(ServerSessionPoolImpl.java:170)
- waiting to lock <0x04aa6750> (a
org.apache.activemq.ra.ServerSessionPoolImpl)
at
org.apache.activemq.ra.ServerSessionImpl.run(ServerSessionImpl.java:177)
- locked <0x051a3c68> (a java.lang.Object)
- locked <0x05192708> (a org.apache.activemq.ra.ServerSessionImpl)
at com.caucho.jca.WorkThread.run(WorkThread.java:99)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:490)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"resin-tcp-connection-127.0.0.1:6802-13" daemon prio=5 tid=0x10245e28
nid=0x1b14 runnable [0x10c5f000..0x10c5fa68]
at com.caucho.vfs.JniServerSocketImpl.nativeAccept(Native Method)
at
com.caucho.vfs.JniServerSocketImpl.accept(JniServerSocketImpl.java:184)
at com.caucho.server.port.Port.accept(Port.java:766)
at com.caucho.server.port.TcpConnection.run(TcpConnection.java:347)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:490)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"resin-28" daemon prio=5 tid=0x1025e350 nid=0x16ec in Object.wait()
[0x10c1f000..0x10c1fae8]
at java.lang.Object.wait(Native Method)
- waiting on <0x04a92048> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:474)
- locked <0x04a92048> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"resin-tcp-connection-127.0.0.1:6802-17" daemon prio=5 tid=0x102225b0
nid=0x14a4 runnable [0x10bdf000..0x10bdfb68]
at com.caucho.vfs.JniServerSocketImpl.nativeAccept(Native Method)
at
com.caucho.vfs.JniServerSocketImpl.accept(JniServerSocketImpl.java:184)
at com.caucho.server.port.Port.accept(Port.java:766)
at com.caucho.server.port.TcpConnection.run(TcpConnection.java:347)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:490)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"resin-tcp-connection-127.0.0.1:6802-3" daemon prio=5 tid=0x10252b20 nid=0x12b0
runnable [0x10b9f000..0x10b9fbe8]
at com.caucho.vfs.JniServerSocketImpl.nativeAccept(Native Method)
at
com.caucho.vfs.JniServerSocketImpl.accept(JniServerSocketImpl.java:184)
at com.caucho.server.port.Port.accept(Port.java:766)
at com.caucho.server.port.TcpConnection.run(TcpConnection.java:347)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:490)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"BOEClusteredScheduler_QuartzSchedulerThread" prio=5 tid=0x0ed0a7f8 nid=0x19a0
waiting on condition [0x10b1f000..0x10b1fce8]
at java.lang.Thread.sleep(Native Method)
at
org.quartz.core.QuartzSchedulerThread.run(QuartzSchedulerThread.java:394)
"BOEClusteredScheduler_Worker-9" prio=5 tid=0x10224d30 nid=0x1ab8 in
Object.wait() [0x10adf000..0x10adfd68]
at java.lang.Object.wait(Native Method)
at
org.quartz.simpl.SimpleThreadPool.getNextRunnable(SimpleThreadPool.java:428)
- locked <0x049ab288> (a java.lang.Object)
at
org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool.java:47)
at
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:518)
"BOEClusteredScheduler_Worker-8" prio=5 tid=0x101a3430 nid=0x12f4 in
Object.wait() [0x10a9f000..0x10a9f9e8]
at java.lang.Object.wait(Native Method)
at
org.quartz.simpl.SimpleThreadPool.getNextRunnable(SimpleThreadPool.java:428)
- locked <0x049ab288> (a java.lang.Object)
at
org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool.java:47)
at
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:518)
"BOEClusteredScheduler_Worker-7" prio=5 tid=0x10224bb0 nid=0x1dc8 in
Object.wait() [0x10a5f000..0x10a5fa68]
at java.lang.Object.wait(Native Method)
at
org.quartz.simpl.SimpleThreadPool.getNextRunnable(SimpleThreadPool.java:428)
- locked <0x049ab288> (a java.lang.Object)
at
org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool.java:47)
at
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:518)
"BOEClusteredScheduler_Worker-6" prio=5 tid=0x102ca008 nid=0x868 in
Object.wait() [0x10a1f000..0x10a1fae8]
at java.lang.Object.wait(Native Method)
at
org.quartz.simpl.SimpleThreadPool.getNextRunnable(SimpleThreadPool.java:428)
- locked <0x049ab288> (a java.lang.Object)
at
org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool.java:47)
at
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:518)
"BOEClusteredScheduler_Worker-5" prio=5 tid=0x1034fcc8 nid=0xbfc in
Object.wait() [0x109df000..0x109dfb68]
at java.lang.Object.wait(Native Method)
at
org.quartz.simpl.SimpleThreadPool.getNextRunnable(SimpleThreadPool.java:428)
- locked <0x049ab288> (a java.lang.Object)
at
org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool.java:47)
at
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:518)
"BOEClusteredScheduler_Worker-4" prio=5 tid=0x101b9b28 nid=0x1b90 in
Object.wait() [0x1099f000..0x1099fbe8]
at java.lang.Object.wait(Native Method)
at
org.quartz.simpl.SimpleThreadPool.getNextRunnable(SimpleThreadPool.java:428)
- locked <0x049ab288> (a java.lang.Object)
at
org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool.java:47)
at
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:518)
"BOEClusteredScheduler_Worker-3" prio=5 tid=0x101a3608 nid=0xbcc in
Object.wait() [0x1095f000..0x1095fc68]
at java.lang.Object.wait(Native Method)
at
org.quartz.simpl.SimpleThreadPool.getNextRunnable(SimpleThreadPool.java:428)
- locked <0x049ab288> (a java.lang.Object)
at
org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool.java:47)
at
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:518)
"BOEClusteredScheduler_Worker-2" prio=5 tid=0x102236a8 nid=0x1c6c in
Object.wait() [0x1091f000..0x1091fce8]
at java.lang.Object.wait(Native Method)
at
org.quartz.simpl.SimpleThreadPool.getNextRunnable(SimpleThreadPool.java:428)
- locked <0x049ab288> (a java.lang.Object)
at
org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool.java:47)
at
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:518)
"BOEClusteredScheduler_Worker-1" prio=5 tid=0x10253658 nid=0x18d4 in
Object.wait() [0x108df000..0x108dfd68]
at java.lang.Object.wait(Native Method)
at
org.quartz.simpl.SimpleThreadPool.getNextRunnable(SimpleThreadPool.java:428)
- locked <0x049ab288> (a java.lang.Object)
at
org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool.java:47)
at
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:518)
"BOEClusteredScheduler_Worker-0" prio=5 tid=0x10369d70 nid=0xfb8 in
Object.wait() [0x1089f000..0x1089f9e8]
at java.lang.Object.wait(Native Method)
at
org.quartz.simpl.SimpleThreadPool.getNextRunnable(SimpleThreadPool.java:428)
- locked <0x049ab288> (a java.lang.Object)
at
org.quartz.simpl.SimpleThreadPool.access$000(SimpleThreadPool.java:47)
at
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:518)
"Transport Status Dector [EMAIL PROTECTED]" daemon prio=9 tid=0x1019bac8
nid=0x724 waiting on con
dition [0x1085f000..0x1085fa68]
at java.lang.Thread.sleep(Native Method)
at
org.apache.activemq.broker.TransportStatusDetector.run(TransportStatusDetector.java:98)
at java.lang.Thread.run(Thread.java:595)
"MERGE2.FindSubgroups thread" daemon prio=5 tid=0x100c3fc0 nid=0x10b4 waiting
on condition [0x0ffaf000..0x0ffafbe8]
at java.lang.Thread.sleep(Native Method)
at org.jgroups.util.Util.sleep(Util.java:374)
at org.jgroups.protocols.MERGE2$FindSubgroups.run(MERGE2.java:245)
at java.lang.Thread.run(Thread.java:595)
"UDP.UcastReceiverThread" daemon prio=5 tid=0x100c3e10 nid=0x1e14 runnable
[0x0ff6f000..0x0ff6fd68]
at java.net.PlainDatagramSocketImpl.receive0(Native Method)
- locked <0x03ffcbb8> (a java.net.PlainDatagramSocketImpl)
at
java.net.PlainDatagramSocketImpl.receive(PlainDatagramSocketImpl.java:136)
- locked <0x03ffcbb8> (a java.net.PlainDatagramSocketImpl)
at java.net.DatagramSocket.receive(DatagramSocket.java:712)
- locked <0x03ffcbf8> (a java.net.DatagramPacket)
- locked <0x03ffcc18> (a java.net.DatagramSocket)
at org.jgroups.protocols.UDP$UcastReceiver.run(UDP.java:1536)
at java.lang.Thread.run(Thread.java:595)
"Thread-10" daemon prio=5 tid=0x10161b30 nid=0x1bc8 in Object.wait()
[0x0feef000..0x0feefa68]
at java.lang.Object.wait(Native Method)
- waiting on <0x03ffcc90> (a java.lang.Object)
at java.lang.Object.wait(Object.java:474)
at org.jgroups.util.Queue.remove(Queue.java:226)
- locked <0x03ffcc90> (a java.lang.Object)
at
org.jgroups.blocks.MessageDispatcher$1.run(MessageDispatcher.java:701)
at java.lang.Thread.run(Thread.java:595)
"UpHandler (GMS)" daemon prio=5 tid=0x10108408 nid=0x1894 in Object.wait()
[0x0feaf000..0x0feafae8]
at java.lang.Object.wait(Native Method)
- waiting on <0x03ffccd8> (a java.lang.Object)
at java.lang.Object.wait(Object.java:474)
at org.jgroups.util.Queue.remove(Queue.java:226)
- locked <0x03ffccd8> (a java.lang.Object)
at org.jgroups.stack.UpHandler.run(Protocol.java:48)
"DownHandler (GMS)" daemon prio=5 tid=0x101c1e08 nid=0xc4 in Object.wait()
[0x0fe6f000..0x0fe6fb68]
at java.lang.Object.wait(Native Method)
- waiting on <0x03ffcd00> (a java.lang.Object)
at java.lang.Object.wait(Object.java:474)
at org.jgroups.util.Queue.remove(Queue.java:226)
- locked <0x03ffcd00> (a java.lang.Object)
at org.jgroups.stack.DownHandler.run(Protocol.java:104)
"UpHandler (UNICAST)" daemon prio=5 tid=0x101c1bc8 nid=0x12ec in Object.wait()
[0x0fe2f000..0x0fe2fbe8]
at java.lang.Object.wait(Native Method)
- waiting on <0x03ffcd28> (a java.lang.Object)
at java.lang.Object.wait(Object.java:474)
at org.jgroups.util.Queue.remove(Queue.java:226)
- locked <0x03ffcd28> (a java.lang.Object)
at org.jgroups.stack.UpHandler.run(Protocol.java:48)
"UpHandler (FD)" daemon prio=5 tid=0x00ab7a00 nid=0x119c in Object.wait()
[0x0fdef000..0x0fdefc68]
at java.lang.Object.wait(Native Method)
- waiting on <0x03ffcd50> (a java.lang.Object)
at java.lang.Object.wait(Object.java:474)
at org.jgroups.util.Queue.remove(Queue.java:226)
- locked <0x03ffcd50> (a java.lang.Object)
at org.jgroups.stack.UpHandler.run(Protocol.java:48)
"DownHandler (FD)" daemon prio=5 tid=0x101716d8 nid=0x12fc in Object.wait()
[0x0fdaf000..0x0fdafce8]
at java.lang.Object.wait(Native Method)
- waiting on <0x03ffcd78> (a java.lang.Object)
at java.lang.Object.wait(Object.java:474)
at org.jgroups.util.Queue.remove(Queue.java:226)
- locked <0x03ffcd78> (a java.lang.Object)
at org.jgroups.stack.DownHandler.run(Protocol.java:104)
"DownHandler (UDP)" daemon prio=5 tid=0x101729e0 nid=0x17dc in Object.wait()
[0x0fd6f000..0x0fd6fd68]
at java.lang.Object.wait(Native Method)
- waiting on <0x03ffcda0> (a java.lang.Object)
at java.lang.Object.wait(Object.java:474)
at org.jgroups.util.Queue.remove(Queue.java:226)
- locked <0x03ffcda0> (a java.lang.Object)
at org.jgroups.stack.DownHandler.run(Protocol.java:104)
"resin-select-manager" daemon prio=5 tid=0x0f1d21f0 nid=0x1aa4 runnable
[0x0fd0f000..0x0fd0f9e8]
at com.caucho.server.port.JniSelectManager.selectNative(Native Method)
at
com.caucho.server.port.JniSelectManager.run(JniSelectManager.java:285)
at java.lang.Thread.run(Thread.java:595)
"resin-tcp-connection-127.0.0.1:6802-9" daemon prio=5 tid=0x0f1efca8 nid=0x194c
runnable [0x0fbcf000..0x0fbcfa68]
at com.caucho.vfs.JniServerSocketImpl.nativeAccept(Native Method)
at
com.caucho.vfs.JniServerSocketImpl.accept(JniServerSocketImpl.java:184)
at com.caucho.server.port.Port.accept(Port.java:766)
at com.caucho.server.port.TcpConnection.run(TcpConnection.java:347)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:490)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"resin-tcp-connection-*:8088-11" daemon prio=5 tid=0x0f1d2070 nid=0x1934
runnable [0x0fb7f000..0x0fb7fae8]
at com.caucho.vfs.JniServerSocketImpl.nativeAccept(Native Method)
at
com.caucho.vfs.JniServerSocketImpl.accept(JniServerSocketImpl.java:184)
at com.caucho.server.port.Port.accept(Port.java:766)
at com.caucho.server.port.TcpConnection.run(TcpConnection.java:347)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:490)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"resin-tcp-connection-*:8088-25" daemon prio=5 tid=0x0f1cecb0 nid=0x1a00
runnable [0x0fb3f000..0x0fb3fb68]
at com.caucho.vfs.JniServerSocketImpl.nativeAccept(Native Method)
at
com.caucho.vfs.JniServerSocketImpl.accept(JniServerSocketImpl.java:184)
at com.caucho.server.port.Port.accept(Port.java:766)
at com.caucho.server.port.TcpConnection.run(TcpConnection.java:347)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:490)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"resin-22" daemon prio=5 tid=0x0f161a48 nid=0x1f34 in Object.wait()
[0x0faff000..0x0faffbe8]
at java.lang.Object.wait(Native Method)
- waiting on <0x03a34b78> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:474)
- locked <0x03a34b78> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"resin-tcp-connection-127.0.0.1:6802-5" daemon prio=5 tid=0x0ef2a738 nid=0x1330
runnable [0x0fabf000..0x0fabfc68]
at com.caucho.vfs.JniServerSocketImpl.nativeAccept(Native Method)
at
com.caucho.vfs.JniServerSocketImpl.accept(JniServerSocketImpl.java:184)
at com.caucho.server.port.Port.accept(Port.java:766)
at com.caucho.server.port.TcpConnection.run(TcpConnection.java:347)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:490)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"resin-20" daemon prio=5 tid=0x0f03b788 nid=0x18d8 in Object.wait()
[0x0fa7f000..0x0fa7fce8]
at java.lang.Object.wait(Native Method)
- waiting on <0x038816b0> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:474)
- locked <0x038816b0> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"resin-19" daemon prio=5 tid=0x0f136948 nid=0x1dbc in Object.wait()
[0x0fa3f000..0x0fa3fd68]
at java.lang.Object.wait(Native Method)
- waiting on <0x038816e0> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:474)
- locked <0x038816e0> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"resin-18" daemon prio=5 tid=0x0f136240 nid=0x15a8 in Object.wait()
[0x0f9ff000..0x0f9ff9e8]
at java.lang.Object.wait(Native Method)
- waiting on <0x03881710> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:474)
- locked <0x03881710> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"resin-17" daemon prio=5 tid=0x0f0474a0 nid=0x1fd4 in Object.wait()
[0x0f9bf000..0x0f9bfa68]
at java.lang.Object.wait(Native Method)
- waiting on <0x03881740> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:474)
- locked <0x03881740> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"resin-16" daemon prio=5 tid=0x0f112790 nid=0xcd4 in Object.wait()
[0x0f97f000..0x0f97fae8]
at java.lang.Object.wait(Native Method)
- waiting on <0x03881770> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:474)
- locked <0x03881770> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"resin-15" daemon prio=5 tid=0x0f165b80 nid=0x1f50 in Object.wait()
[0x0f93f000..0x0f93fb68]
at java.lang.Object.wait(Native Method)
- waiting on <0x038817a0> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:474)
- locked <0x038817a0> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"resin-ping" daemon prio=5 tid=0x0f165fe0 nid=0x1048 waiting on condition
[0x0f8ff000..0x0f8ffbe8]
at java.lang.Thread.sleep(Native Method)
at com.caucho.server.admin.PingThread.run(PingThread.java:365)
at java.lang.Thread.run(Thread.java:595)
"resin-14" daemon prio=5 tid=0x0f110b80 nid=0x1244 in Object.wait()
[0x0f8bf000..0x0f8bfc68]
at java.lang.Object.wait(Native Method)
- waiting on <0x03881838> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:474)
- locked <0x03881838> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"resin-13" daemon prio=5 tid=0x0f1c7d60 nid=0x1a9c in Object.wait()
[0x0f87f000..0x0f87fce8]
at java.lang.Object.wait(Native Method)
- waiting on <0x03881868> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:474)
- locked <0x03881868> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"resin-12" daemon prio=5 tid=0x0f17a398 nid=0x1980 in Object.wait()
[0x0f83f000..0x0f83fd68]
at java.lang.Object.wait(Native Method)
- waiting on <0x03881898> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:474)
- locked <0x03881898> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"resin-11" daemon prio=5 tid=0x0f138410 nid=0x1828 in Object.wait()
[0x0f7ff000..0x0f7ff9e8]
at java.lang.Object.wait(Native Method)
- waiting on <0x038818c8> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:474)
- locked <0x038818c8> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"resin-10" daemon prio=5 tid=0x0ef2ae80 nid=0x1d68 in Object.wait()
[0x0f7bf000..0x0f7bfa68]
at java.lang.Object.wait(Native Method)
- waiting on <0x038818f8> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:474)
- locked <0x038818f8> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"resin-9" daemon prio=5 tid=0x0f03c8c0 nid=0x1d50 in Object.wait()
[0x0f77f000..0x0f77fae8]
at java.lang.Object.wait(Native Method)
- waiting on <0x03881928> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:474)
- locked <0x03881928> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"resin-8" daemon prio=5 tid=0x0f16a008 nid=0x1960 in Object.wait()
[0x0f73f000..0x0f73fb68]
at java.lang.Object.wait(Native Method)
- waiting on <0x03881958> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:474)
- locked <0x03881958> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"resin-7" daemon prio=5 tid=0x0f162368 nid=0x1ebc in Object.wait()
[0x0f6ff000..0x0f6ffbe8]
at java.lang.Object.wait(Native Method)
- waiting on <0x03881988> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:474)
- locked <0x03881988> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"resin-6" daemon prio=5 tid=0x0f0eec60 nid=0x894 in Object.wait()
[0x0f6bf000..0x0f6bfc68]
at java.lang.Object.wait(Native Method)
- waiting on <0x038819b8> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:474)
- locked <0x038819b8> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"resin-5" daemon prio=5 tid=0x0f0dd438 nid=0x1690 in Object.wait()
[0x0f65f000..0x0f65fce8]
at java.lang.Object.wait(Native Method)
- waiting on <0x036c6978> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:474)
- locked <0x036c6978> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"resin-4" daemon prio=5 tid=0x0f0dc940 nid=0x1198 in Object.wait()
[0x0f61f000..0x0f61fd68]
at java.lang.Object.wait(Native Method)
- waiting on <0x036ca9b8> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:474)
- locked <0x036ca9b8> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"resin-3" daemon prio=5 tid=0x0f0dbf30 nid=0xd34 in Object.wait()
[0x0f5df000..0x0f5df9e8]
at java.lang.Object.wait(Native Method)
- waiting on <0x036ca9e8> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:474)
- locked <0x036ca9e8> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"resin-2" daemon prio=5 tid=0x0f0d78f8 nid=0x1854 in Object.wait()
[0x0f59f000..0x0f59fa68]
at java.lang.Object.wait(Native Method)
- waiting on <0x036caa18> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:474)
- locked <0x036caa18> (a com.caucho.util.ThreadPool)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
"resin-thread-scheduler" daemon prio=5 tid=0x0f000740 nid=0x19ac in
Object.wait() [0x0f4df000..0x0f4dfbe8]
at java.lang.Object.wait(Native Method)
- waiting on <0x036caaa8> (a java.util.ArrayList)
at com.caucho.util.ThreadPool$ScheduleThread.run(ThreadPool.java:661)
- locked <0x036caaa8> (a java.util.ArrayList)
at java.lang.Thread.run(Thread.java:595)
"resin-thread-launcher" daemon prio=5 tid=0x0f0005c0 nid=0x560 in Object.wait()
[0x0f49f000..0x0f49fc68]
at java.lang.Object.wait(Native Method)
- waiting on <0x036caac8> (a com.caucho.util.ThreadPool$ThreadLauncher)
at
com.caucho.util.ThreadPool$ThreadLauncher.startConnection(ThreadPool.java:584)
- locked <0x036caac8> (a com.caucho.util.ThreadPool$ThreadLauncher)
at com.caucho.util.ThreadPool$ThreadLauncher.run(ThreadPool.java:607)
at java.lang.Thread.run(Thread.java:595)
"resin-alarm" daemon prio=5 tid=0x0f00fa10 nid=0x1ef8 waiting on condition
[0x0f45f000..0x0f45fce8]
at java.lang.Thread.sleep(Native Method)
at com.caucho.util.Alarm$AlarmThread.run(Alarm.java:521)
"RMI TCP Accept-0" daemon prio=5 tid=0x0f0ac870 nid=0x18b4 runnable
[0x0f3df000..0x0f3dfd68]
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
- locked <0x036cab18> (a java.net.SocksSocketImpl)
at java.net.ServerSocket.implAccept(ServerSocket.java:450)
at java.net.ServerSocket.accept(ServerSocket.java:421)
at sun.rmi.transport.tcp.TCPTransport.run(TCPTransport.java:334)
at java.lang.Thread.run(Thread.java:595)
"RMI TCP Accept-6969" daemon prio=5 tid=0x0f0d1d48 nid=0x18b8 runnable
[0x0f39f000..0x0f39fa68]
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
- locked <0x036caca8> (a java.net.SocksSocketImpl)
at java.net.ServerSocket.implAccept(ServerSocket.java:450)
at java.net.ServerSocket.accept(ServerSocket.java:421)
at sun.rmi.transport.tcp.TCPTransport.run(TCPTransport.java:334)
at java.lang.Thread.run(Thread.java:595)
"RMI TCP Accept-0" daemon prio=5 tid=0x0f09bc70 nid=0x18dc runnable
[0x0f35f000..0x0f35fae8]
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
- locked <0x036cae38> (a java.net.SocksSocketImpl)
at java.net.ServerSocket.implAccept(ServerSocket.java:450)
at java.net.ServerSocket.accept(ServerSocket.java:421)
at
javax.rmi.ssl.SslRMIServerSocketFactory$1.accept(SslRMIServerSocketFactory.java:191)
at sun.rmi.transport.tcp.TCPTransport.run(TCPTransport.java:334)
at java.lang.Thread.run(Thread.java:595)
"Timer-0" daemon prio=5 tid=0x0f056e00 nid=0xf4 in Object.wait()
[0x0f31f000..0x0f31fb68]
at java.lang.Object.wait(Native Method)
- waiting on <0x036caf88> (a java.util.TaskQueue)
at java.lang.Object.wait(Object.java:474)
at java.util.TimerThread.mainLoop(Timer.java:483)
- locked <0x036caf88> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:462)
"Low Memory Detector" daemon prio=5 tid=0x00aa01e8 nid=0x10d0 runnable
[0x00000000..0x00000000]
"CompilerThread0" daemon prio=10 tid=0x00038f88 nid=0x1318 waiting on condition
[0x00000000..0x0ec8f9cc]
"Signal Dispatcher" daemon prio=10 tid=0x00a9dce0 nid=0xc30 waiting on
condition [0x00000000..0x00000000]
"JDWP Event Helper Thread" daemon prio=5 tid=0x00a9caa8 nid=0x4e0 runnable
[0x00000000..0x00000000]
"JDWP Transport Listener: dt_socket" daemon prio=5 tid=0x00a95f78 nid=0x1a68
runnable [0x00000000..0x0ebcf7e8]
"Finalizer" daemon prio=9 tid=0x00a8bf00 nid=0xb10 in Object.wait()
[0x0eb8f000..0x0eb8fa68]
at java.lang.Object.wait(Native Method)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
- locked <0x034b46b0> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
"Reference Handler" daemon prio=10 tid=0x00a8aa70 nid=0x17d4 in Object.wait()
[0x0eb4f000..0x0eb4fae8]
at java.lang.Object.wait(Native Method)
- waiting on <0x034b46d0> (a java.lang.ref.Reference$Lock)
at java.lang.Object.wait(Object.java:474)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
- locked <0x034b46d0> (a java.lang.ref.Reference$Lock)
"main" prio=5 tid=0x0003bd78 nid=0x1a34 runnable [0x0007f000..0x0007fc3c]
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at java.net.SocketInputStream.read(SocketInputStream.java:182)
at com.caucho.server.resin.Resin.waitForExit(Resin.java:524)
at com.caucho.server.resin.Resin.main(Resin.java:614)
"VM Thread" prio=10 tid=0x00a88d98 nid=0x100 runnable
"VM Periodic Task Thread" prio=10 tid=0x0f05be08 nid=0x9fc waiting on condition
Found one Java-level deadlock:
=============================
"ActiveMQ Task":
waiting to lock monitor 0x00a8ba3c (object 0x04abda20, a java.lang.Object),
which is held by "resin-38"
"resin-38":
waiting to lock monitor 0x00a8bbbc (object 0x04ab0f90, a java.lang.Object),
which is held by "ActiveMQ Task"
Java stack information for the threads listed above:
===================================================
"ActiveMQ Task":
at
org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:44)
- waiting to lock <0x04abda20> (a java.lang.Object)
at
org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:56)
at
org.apache.activemq.ActiveMQConnection.asyncSendPacket(ActiveMQConnection.java:1088)
at
org.apache.activemq.ActiveMQConnection.close(ActiveMQConnection.java:543)
- locked <0x04aa67a8> (a org.apache.activemq.ActiveMQConnection)
at
org.apache.activemq.ra.ActiveMQEndpointWorker.safeClose(ActiveMQEndpointWorker.java:99)
at
org.apache.activemq.ra.ActiveMQEndpointWorker.disconnect(ActiveMQEndpointWorker.java:233)
- locked <0x04a9a8b8> (a org.apache.activemq.ra.ActiveMQEndpointWorker)
at
org.apache.activemq.ra.ActiveMQEndpointWorker.reconnect(ActiveMQEndpointWorker.java:249)
at
org.apache.activemq.ra.ActiveMQEndpointWorker.access$300(ActiveMQEndpointWorker.java:45)
at
org.apache.activemq.ra.ActiveMQEndpointWorker$1$1.onException(ActiveMQEndpointWorker.java:149)
at
org.apache.activemq.ActiveMQConnection.onAsyncException(ActiveMQConnection.java:1398)
at
org.apache.activemq.ActiveMQConnection.onCommand(ActiveMQConnection.java:1379)
at
org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:93)
at
org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:70)
at
org.apache.activemq.transport.vm.VMTransport.oneway(VMTransport.java:76)
at
org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:44)
- locked <0x04ab0f90> (a java.lang.Object)
at
org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:56)
at
org.apache.activemq.broker.TransportConnection.dispatch(TransportConnection.java:211)
at
org.apache.activemq.broker.AbstractConnection.processDispatch(AbstractConnection.java:575)
at
org.apache.activemq.broker.AbstractConnection.iterate(AbstractConnection.java:584)
at
org.apache.activemq.thread.SimpleTaskRunner.runTask(SimpleTaskRunner.java:110)
at
org.apache.activemq.thread.SimpleTaskRunner.access$100(SimpleTaskRunner.java:25)
at
org.apache.activemq.thread.SimpleTaskRunner$1.run(SimpleTaskRunner.java:43)
at
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
at
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
at java.lang.Thread.run(Thread.java:595)
"resin-38":
at
org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:44)
- waiting to lock <0x04ab0f90> (a java.lang.Object)
at
org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:56)
at
org.apache.activemq.broker.TransportConnection.dispatch(TransportConnection.java:211)
at
org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:64)
at
org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:93)
at
org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:70)
at
org.apache.activemq.transport.vm.VMTransport.oneway(VMTransport.java:76)
at
org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:44)
- locked <0x04abda20> (a java.lang.Object)
at
org.apache.activemq.transport.ResponseCorrelator.asyncRequest(ResponseCorrelator.java:64)
at
org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:69)
at
org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1108)
at
org.apache.activemq.TransactionContext.setXid(TransactionContext.java:550)
at
org.apache.activemq.TransactionContext.end(TransactionContext.java:327)
at
org.apache.activemq.ra.LocalAndXATransaction.end(LocalAndXATransaction.java:89)
at
org.apache.geronimo.transaction.manager.WrapperNamedXAResource.end(WrapperNamedXAResource.java:51)
at
com.caucho.transaction.TransactionImpl.delistResource(TransactionImpl.java:386)
at org.jencks.XAEndpoint.afterDelivery(XAEndpoint.java:97)
at
org.apache.activemq.ra.MessageEndpointProxy$MessageEndpointAlive.afterDelivery(MessageEndpointProxy.java:125)
at
org.apache.activemq.ra.MessageEndpointProxy.afterDelivery(MessageEndpointProxy.java:64)
at
org.apache.activemq.ra.ServerSessionImpl.afterDelivery(ServerSessionImpl.java:214)
at org.apache.activemq.ActiveMQSession.run(ActiveMQSession.java:723)
at
org.apache.activemq.ra.ServerSessionImpl.run(ServerSessionImpl.java:163)
- locked <0x0514ef50> (a org.apache.activemq.ra.ServerSessionImpl)
at com.caucho.jca.WorkThread.run(WorkThread.java:99)
at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:490)
at com.caucho.util.ThreadPool.run(ThreadPool.java:423)
at java.lang.Thread.run(Thread.java:595)
Found 1 deadlock.
closing server
com.caucho.lifecycle.Lifecycle.toStopping Server[dev1-a] stopping
