[
https://issues.apache.org/jira/browse/QPID-5786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126305#comment-14126305
]
Rob Godfrey commented on QPID-5786:
-----------------------------------
Just recording the analysis I have done so far:
Looking at the heap dump, there are ~1.4 million messages in the broker, all in
one queue:
Running the following OOL query using jhat:
{noformat}
select q._name.toString() + " : " + q._atomicQueueCount.value from
org.apache.qpid.server.queue.StandardQueue q where q._atomicQueueCount.value > 0
{noformat}
we get
{noformat}
tmp_58c26acf-f4be-40a8-b6a7-7b11a7cbd2f4 : 1398340
{noformat}
So... all the other queues are empty, but this one has 1.4 million messages on
it...
What else can we tell about this queue... does it have any consumers?
{noformat}
select q._consumerList._size.value from
org.apache.qpid.server.queue.StandardQueue q where q._atomicQueueCount.value > 0
{noformat}
resulting in:
{noformat}
1.0
{noformat}
OK.. so one consumer... let's have a look at it - in particular, let's look at
the message credit used vs. the credit limit
{noformat}
select "Credit used: " + c._target._creditManager._messageCredit + " with
limit: " + c._target._creditManager._messageCreditLimit from
org.apache.qpid.server.queue.QueueConsumerImpl c where identical(c._queue,
heap.findObject("0x747212d0"))
{noformat}
which gives
{noformat}
Credit used: 1 with limit: 1
{noformat}
So, it looks like there is no credit to send messages... however when we look
at the unacknowledged message map:
{noformat}
select "Unacked Map Size: " +
c._target._channel._unacknowledgedMessageMap._map.size from
org.apache.qpid.server.queue.QueueConsumerImpl c where identical(c._queue,
heap.findObject("0x747212d0"))
{noformat}
we get
{noformat}
Unacked Map Size: 0
{noformat}
So - this is interesting... something seems out of whack here as the credit
manager is saying there is no credit but the unacked message map is saying
there are no unacknowledged messages.
Looking at the other fields in the unacknowledged message map we see
_lastDeliveryTag (J) : 523646
So it seems like it delivered over half a million messages before something
went wrong...
A quick review of the code in the broker doesn't lead me to see any obvious way
these two values can get out of sync... though the code here has changed a bit
between 0.28 and 0.30
> Memory leak in qpid-java-broker-0.26.tar.gz
> -------------------------------------------
>
> Key: QPID-5786
> URL: https://issues.apache.org/jira/browse/QPID-5786
> Project: Qpid
> Issue Type: Bug
> Components: Java Broker
> Affects Versions: 0.26
> Environment: Linux Mint 15 (64 Bit) - any Linux version should do
> Java // OpenJDK Runtime Environment (IcedTea 2.4.4)
> also tested with Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
> Ruby 1.8.7
> Ruby Bunny Gem 1.2.2
> Reporter: Bobby Flitter
> Attachments: amqp_bunny_qpid.pcap
>
>
> When using the hello bunny example program from Ruby together with Apache
> qpid server the memory consumption of qpid seems to grow continuously.
> Steps to reproduce the problem in Linux:
> wget
> http://ftp-stud.hs-esslingen.de/pub/Mirrors/ftp.apache.org/dist/qpid/0.26/qpid-java-broker-0.26.tar.gz
> tar xzvf qpid-java-broker-0.26.tar.gz
> cd qpid-broker-0.26/bin/
> ./qpid-server
> - in a new terminal
> - meanwhile bunny 1.2.2. is available
> - install ruby amqp bunny
> sudo gem install bunny
> - download hello bunny example
> wget http://pastebin.com/raw.php?i=f6MesMCZ -O hello_bunny.rb
> while true; do ruby hello_bunny.rb ; done
> - in another terminal:
> while true; do grep "^VmRSS" /proc/$(pgrep -f qpid)/status ; sleep 1;done
> - memory consumption of java qpid is growing continuously until
> OutOfMemoryError ends the server
> Unhandled Exception java.lang.OutOfMemoryError: Java heap space in Thread
> Thread-1
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
