Hello,
I have a question about the effects of the the scheduled pool max size
in artemis (2.10.1).
I had a recurring issue with an embedded artemis instance which I had
significant trouble to track down.
The application stack uses a small number (3) of internal consumers do
process messages, connected using the vm transport using pooledJms (and
spring DefaultMessageListenerContainer) and a a few JmsTemplate
instances for sending responses (also using pooledJMS ; removing
pooledJMS did not help)
The failure occured while sending a message inside the server
environment; from "client" side it looks like that :
Caused by: javax.jms.JMSException
at
org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.sendBlocking(ChannelImpl.java:464)
at
org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.sendBlocking(ChannelImpl.java:358)
at
org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQClientProtocolManager.createSessionContext(ActiveMQClientProtocolManager.java:300)
at
org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQClientProtocolManager.createSessionContext(ActiveMQClientProtocolManager.java:249)
at
org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.createSessionChannel(ClientSessionFactoryImpl.java:1348)
at
org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.createSessionInternal(ClientSessionFactoryImpl.java:673)
at
org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.createSession(ClientSessionFactoryImpl.java:298)
at
org.apache.activemq.artemis.jms.client.ActiveMQConnection.authorize(ActiveMQConnection.java:684)
at
org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory.createConnectionInternal(ActiveMQConnectionFactory.java:920)
at
org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:299)
at
org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:294)
at
org.messaginghub.pooled.jms.JmsPoolConnectionFactory.createProviderConnection(JmsPoolConnectionFactory.java:650)
at
org.messaginghub.pooled.jms.JmsPoolConnectionFactory$1.makeObject(JmsPoolConnectionFactory.java:107)
at
org.messaginghub.pooled.jms.JmsPoolConnectionFactory$1.makeObject(JmsPoolConnectionFactory.java:104)
at
org.apache.commons.pool2.impl.GenericKeyedObjectPool.create(GenericKeyedObjectPool.java:1041)
at
org.apache.commons.pool2.impl.GenericKeyedObjectPool.addObject(GenericKeyedObjectPool.java:1233)
at
org.messaginghub.pooled.jms.JmsPoolConnectionFactory.createJmsPoolConnection(JmsPoolConnectionFactory.java:704)
at
org.messaginghub.pooled.jms.JmsPoolConnectionFactory.createConnection(JmsPoolConnectionFactory.java:236)
at
org.messaginghub.pooled.jms.JmsPoolConnectionFactory.createConnection(JmsPoolConnectionFactory.java:231)
at
org.springframework.jms.support.JmsAccessor.createConnection(JmsAccessor.java:196)
at
org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:494)
Caused by: ActiveMQInternalErrorException[errorType=INTERNAL_ERROR
message=null] ... 28 more
On "server" (artemis-side), it produces a debug trace like that
2019-10-17 10:44:07,853 DEBUG {Thread-8
(ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6@66c08833)}
[o.a.a.artemis.core.server.impl.QueueImpl] :
QueueImpl[name=FullEvent.2.In, postOffice=PostOfficeImpl
[server=ActiveMQServerImpl::serverUUID=c7fa0fe7-6688-11e9-9127-005056aaab0e],
temp=false]@6c22dc9a::All the consumers were busy, giving up now
Note : the failing send occurs in the context of a thread of a scheduled
timer task created by the application (end of batch notification emitted
every 30s).
At some point, if the real client (i.e. client side, netty connector
distinct from server-side clients using the vm transport) emit new
messages, the consumer processes, another send is triggered and the
previous "stuck" send would appear in its destination queue.
After tinkering with quite a lot the artemis parameters, the issue
finally went away when I increased <scheduled-thread-pool-max-size> from
10 to 20.
According to the documentation here :
https://activemq.apache.org/components/artemis/documentation/latest/thread-pooling.html
the default value is 5 and "usually sufficient". Is there any ressources
available with deeper explanation of this parameter and guidance on the
sizing policy by use cases or is this caused by an issue with the
implementation ?
Regards,
Sébastien
