On Thu, Oct 1, 2009 at 5:13 AM, Jamie McCrindle
<jamiemccrin...@gmail.com> wrote:
> Hi All,
>
> I've just got hit by an OutOfMemoryError: unable to create new native
> thread issue. I've switched UseDedicatedTaskRunner to false which
> appears to have resolved the issue for now but I have a feeling I'm
> just masking a deeper problem.
>
> I have the following set up:
>
> 2 ActiveMQ brokers configured as a network of brokers
> Several applications that use
> - a Spring JmsTemplate with a PooledConnectionFactory
> - a Spring DefaultMessageListenerContainer that uses a regular
> ConnectionFactory. The DefaultMessageListenerContainers are set to
> CACHE_CONSUMER
> - a static failover broker URL for both the JmsTemplate and listener
> container to the two brokers
>
> In the applications themselves, I'm getting this:
>
> java.lang.OutOfMemoryError: unable to create new native thread
> at java.lang.Thread.start0(Native Method)
> at java.lang.Thread.start(Thread.java:574)
> at
> java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:455)
> at
> java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:863)
> at
> org.apache.activemq.thread.PooledTaskRunner.wakeup(PooledTaskRunner.java:76)
> at
> org.apache.activemq.transport.failover.FailoverTransport.reconnect(FailoverTransport.java:536)
> at
> org.apache.activemq.transport.failover.FailoverTransport.start(FailoverTransport.java:239)
> at
> org.apache.activemq.transport.TransportFilter.start(TransportFilter.java:57)
> at
> org.apache.activemq.transport.TransportFilter.start(TransportFilter.java:57)
> at
> org.apache.activemq.ActiveMQConnectionFactory.createActiveMQConnection(ActiveMQConnectionFactory.java:260)
> at
> org.apache.activemq.ActiveMQConnectionFactory.createActiveMQConnection(ActiveMQConnectionFactory.java:224)
> at
> org.apache.activemq.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:172)
> at
> org.apache.activemq.pool.PooledConnectionFactory.createConnection(PooledConnectionFactory.java:125)
> at
> org.apache.activemq.pool.PooledConnectionFactory.createConnection(PooledConnectionFactory.java:110)
> at
> org.apache.activemq.pool.PooledConnectionFactory.createConnection(PooledConnectionFactory.java:81)
> at
> org.springframework.jms.support.JmsAccessor.createConnection(JmsAccessor.java:184)
> at
> org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:461)
> at org.springframework.jms.core.JmsTemplate.send(JmsTemplate.java:539)
>
> In the brokers I'm getting this:
>
> INFO | jvm 1 | 2009/09/30 12:28:41 | Exception in thread
> "ActiveMQ Transport Server Thread Hand
> ler: tcp://localhost:41414" java.lang.OutOfMemoryError: unable to
> create new native thread
> INFO | jvm 1 | 2009/09/30 12:28:41 | at
> java.lang.Thread.start0(Native Method)
> INFO | jvm 1 | 2009/09/30 12:28:41 | at
> java.lang.Thread.start(Thread.java:574)
> INFO | jvm 1 | 2009/09/30 12:28:41 | at
> org.apache.activemq.broker.TransportConnector$1.onAccept(TransportConnector.java:221)
> INFO | jvm 1 | 2009/09/30 12:28:41 | at
> org.apache.activemq.transport.tcp.TcpTransportServer.handleSocket(TcpTransportServer.java:409)
> INFO | jvm 1 | 2009/09/30 12:28:41 | at
> org.apache.activemq.transport.tcp.TcpTransportServer$1.run(TcpTransportServer.java:341)
> INFO | jvm 1 | 2009/09/30 12:28:41 | at
> java.lang.Thread.run(Thread.java:595)
> INFO | jvm 1 | 2009/09/30 12:59:27 | Exception in thread
> "DefaultMessageListenerContainer-80896" java.lang.OutOfMemoryError:
> unable to create new native thread
> INFO | jvm 1 | 2009/09/30 12:59:27 | at
> java.lang.Thread.start0(Native Method)
> INFO | jvm 1 | 2009/09/30 12:59:27 | at
> java.lang.Thread.start(Thread.java:574)
> INFO | jvm 1 | 2009/09/30 12:59:27 | at
> org.springframework.core.task.SimpleAsyncTaskExecutor.doExecute(SimpleAsyncTaskExecutor.java:157)
> INFO | jvm 1 | 2009/09/30 12:59:27 | at
> org.springframework.core.task.SimpleAsyncTaskExecutor.execute(SimpleAsyncTaskExecutor.java:145)
> INFO | jvm 1 | 2009/09/30 12:59:27 | at
> org.springframework.core.task.SimpleAsyncTaskExecutor.execute(SimpleAsyncTaskExecutor.java:126)
> INFO | jvm 1 | 2009/09/30 12:59:27 | at
> org.springframework.jms.listener.DefaultMessageListenerContainer.doRescheduleTask(DefaultMessageListenerContainer.java:598)
> INFO | jvm 1 | 2009/09/30 12:59:27 | at
> org.springframework.jms.listener.AbstractJmsListeningContainer.rescheduleTaskIfNecessary(AbstractJmsListeningContainer.java:509)
> INFO | jvm 1 | 2009/09/30 12:59:27 | at
> org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:918)
>
> I'm also getting a lot of these in the logs:
>
> FailoverTransport.doReconnect | Successfully connected to tcp://xxxxxxxx:xxxx
>
> Other information:
>
> We've got a similar set up in our load testing environment and our
> other production environment and neither of them appear to have the
> same problem (yet?).
> I've read the JmsTemplate gotchas and outofmemory pages on the
> ActiveMQ site. I think I'm within the guidelines...
> We're using Spring 2.5.6
> We aren't using a transaction manager
>
> any ideas?
Not using the DedicatedTaskRunner means that less threads are created
which will indeed help to alleviate the problem. But you should look
at the JVM settings being used, especially the ThreadStackSize for the
error you were receiving.
Bruce
--
perl -e 'print
unpack("u30","D0G)u8...@4vyy9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*"
);'
ActiveMQ in Action: http://bit.ly/2je6cQ
Blog: http://bruceblog.org/
Twitter: http://twitter.com/brucesnyder
