Hello,
there are various reasons why this may happen, but with your low amount of
threads - are you by any chance running this at 32-bit OS or at least JVM?
If yes, it isn't that difficult to achieve that with large heap sizes when
you're operating near the address space limitations.
In such case you can:
- Decrease the heap size (-Xmx) a little in order to leave more room for the
native memory purposes. I think that even decreasing the MaxPermSize or
MaxMetaspaceSize (Java 8+) might work, but I'm not really sure about the
metaspace.
- Decrease the thread stack size (-Xss) a little to reduce the per-thread
native memory consumption.
Both settings would need to be adjusted sensibly as lowering them too much
would have other negative effects (performance, OutOfMemoryError on heap
space, StackOverflowError).
As for the thread stack size you can determine the default value via the -
XX:+PrintFlagsFinal JVM option and then reduce it slightly, example:
# /opt/jre1.8.0/bin/java -XX:+PrintFlagsFinal -version 2>&1|grep StackSize
...
intx ThreadStackSize = 320
{pd product}
...
In this case I'd start with setting it to 256k, e.g. -Xss256k
If it's at 64-bit system/JVM then that would be more interesting and in such
case I'd start with reviewing the OS limits (at Linux it would be the
maximum user processes limit that you can review by executing "ulimit -u"
(soft limit) and "ulimit -Hu" (hard limit) under the user running your JVM
process). But with those claimed ~200 threads you shouldn't be hitting even
the default limit I'd say. Maybe if you'd be running many concurrent
instances...
Best regards,
Petr
"Hi
I am using ActiveMQ v5.14.4 and have observed below multiple times now. I am
using NIO connector so thread count is not high (~203 threads when this
issue was happening).
Exception in thread "ActiveMQ InactivityMonitor WriteCheckTimer"
java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:714)
at
java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:
950)
at
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:
1368)
at
org.apache.activemq.transport.AbstractInactivityMonitor.writeCheck
(AbstractInactivityMonitor.java:189)
at
org.apache.activemq.transport.AbstractInactivityMonitor$3.run
(AbstractInactivityMonitor.java:153)
at
org.apache.activemq.thread.SchedulerTimerTask.run(SchedulerTimerTask.java:
33)
at java.util.TimerThread.mainLoop(Timer.java:555)
at java.util.TimerThread.run(Timer.java:505)
which then triggers following warnings and no new clients are able to
connect after this.
[20180702 06:49:50.829 EDT (ActiveMQ NIO Worker 8)
org.apache.activemq.broker.TransportConnection#serviceTransportException 243
WARN] - Transport Connection to: tcp://10.219.31.188:34470 failed:
java.io.IOException: Timer already cancelled.
[20180702 06:49:50.945 EDT (ActiveMQ NIO Worker 7)
org.apache.activemq.broker.TransportConnection#serviceTransportException 243
WARN] - Transport Connection to: tcp://10.219.31.184:41305 failed:
java.io.IOException: Timer already cancelled.
[20180702 06:49:50.945 EDT (ActiveMQ NIO Worker 5)
org.apache.activemq.broker.TransportConnection#serviceTransportException 243
WARN] - Transport Connection to: tcp://10.219.31.189:41305 failed:
java.io.IOException: Timer already cancelled.
[20180702 06:49:50.945 EDT (ActiveMQ NIO Worker 3)
org.apache.activemq.broker.TransportConnection#serviceTransportException 243
WARN] - Transport Connection to: tcp://10.218.24.250:46430 failed:
java.io.IOException: Timer already cancelled.
What can cause this issue to happen? 203 threads doesn't seem too high. I
also found this related JIRA- https://issues.apache.org/jira/browse/AMQ-5543
<https://issues.apache.org/jira/browse/AMQ-5543> which seems to ahve fixed
this but i am still seeing this issue?
How should i go about fixing this? Does this look like a bug or a
configuration problem?
Thanks,
Abhi
--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
"
