On 09/12/2013 03:39 PM, Jimmy Jones wrote:
Hi,
I've finally managed to isolate the issue and can reproduce it with the attached
scripts. Running rx-test.pl followed by tx-test.pl results in a system where the
receiver can keep up with the producer (gets a message every <1s) (tx-test 118%
CPU, qpidd 97% CPU, rx-test 60% CPU). However, if you stop rx-test and restart it
(even after only a second or so), it starts to take 2s+ to receive messages, going
up to about 6s on my system, so the ring quickly fills and overflows. Even if the
producer is then stopped, messages are still only received every 3s - with qpidd
on 100% CPU and the receiver on 5%. Also the resident size of qpidd reaches 5GB,
yet the queue is only 2GB.
Hopefully I can now regain my sanity :)
Well done! Unfortunately your scripts seem to have been stripped off at
some stage. Could you attach them to a JIRA perhaps? This was with 0.22,
right?
Created QPID-5135.
Thanks, Jimmy. I have been looking into this issue a little more. I
couldn't exactly duplicate your numbers as my test machine did not have
sufficient memory but I believe I have identified the key symptom (JIRA
updated accordingly), though as yet not the root cause.
As noted in the JIRA, it may be possible to tune your receivers to
mitigate the issue. How feasible that is probably depends on how closely
your real system follows the test scenario in the JIRA. For large
messages, reducing the capacity seems to be the most effective
improvement. As message size decreases, acknowledging in larger batches
becomes more effective.
One other question was just to confirm that the case as reported does
match your real system. Initially there was a suspicion that the ingest
process was blocked on send which would I think would be a different issue.
I'll do some more digging on what the root cause from the drop in
throughput for large messages on a full ring queue might be and update
the JIRA with any progress.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
