Re: ActiveMQ 5.5.1 - Connections stuck in CLOSE_WAIT state, reaching file descriptor limit

2014-01-23 Thread David O'C
As mentioned before, the broker wasn't at the maximum heap limit. The heapdump was manually created using jmap but by using the -F option as ActiveMQ wouldn't accept even JMX connections. -- View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-5-5-1-Connections-stuck-

Re: ActiveMQ 5.5.1 - Connections stuck in CLOSE_WAIT state, reaching file descriptor limit

2014-01-22 Thread artnaseef
For the broker memory - is that looking at the Heap Size and Permgen Size, checking for broker producer-flow-contorl-messages, or something else? Not being able to connect JMX is a typical symptom of a JVM that's out of Heap or Permgen space. If you connect jconsole, visualvm, or the like before

Re: ActiveMQ 5.5.1 - Connections stuck in CLOSE_WAIT state, reaching file descriptor limit

2014-01-22 Thread David O'C
The broker wasn't at its memory limit at the time when this had occurred, but it wouldn't allow for JMX connections. I'd assume I was able to collect a heapdump before ActiveMQ was restarted and it does mention the Server Transport thread mentioned earlier. I'm not able to identify the thread to

Re: ActiveMQ 5.5.1 - Connections stuck in CLOSE_WAIT state, reaching file descriptor limit

2014-01-21 Thread Gary Tully
I have seen something like this when the hand off thread that deals with accepted sockets die. The error handling needs to be improved to make that case more obvious. It is a long shot, but you could disable the use of an accept queue and see if that resolves the problem. The thread name (fro

Re: ActiveMQ 5.5.1 - Connections stuck in CLOSE_WAIT state, reaching file descriptor limit

2014-01-21 Thread artnaseef
Without looking at the stack trace carefully, problems getting stack traces is often seen with a memory leak or overloaded broker (i.e. running out of heap or permgen space). Can you connect jconsole or visualvm to the broker and check the heap and permgen space when the problem occurs? If you wa

Re: ActiveMQ 5.5.1 - Connections stuck in CLOSE_WAIT state, reaching file descriptor limit

2014-01-21 Thread David O'C
Hi Christian, Apologies for the late reply. I've a thread dump of the broker when it had reached the limit for open sockets. actmqjstack.txt This had to be taken with force with jstack as ActiveMQ at that stage would not ac

Re: ActiveMQ 5.5.1 - Connections stuck in CLOSE_WAIT state, reaching file descriptor limit

2013-09-23 Thread Christian Posta
When you see this next, can you take stack traces and post back here? On Sun, Sep 22, 2013 at 3:37 AM, David O'Connor D < david.d.o.con...@ericsson.com> wrote: > Hi, > > There has been an issue seen where ActiveMQ 5.5.1 would keep outbound > connections in a CLOSE_WAIT state, to the point where

ActiveMQ 5.5.1 - Connections stuck in CLOSE_WAIT state, reaching file descriptor limit

2013-09-23 Thread David O'Connor D
Hi, There has been an issue seen where ActiveMQ 5.5.1 would keep outbound connections in a CLOSE_WAIT state, to the point where the application reaches the file descriptor limit. The broker uses the Openwire format on TCP with closeAsync set to false. It is also non-persistant, storing all me