On 09/03/12 14:22, Gordon Sim wrote:
Can you (or have you) tracked the queue depth, connection and session
stats for the broker exhibiting the problem? Anything you can think of
that might correlate with the rate of growth (e.g. does it look like
its per message)?
Unfortunately nothing looks especially anomalous. As I mentioned earlier
in this thread we did have what I suspect was a dodgy network switch
which caused the producer host NIC to negotiate half-duplex. That hosed
network performance and caused things to back up on the (circular) queue
and was also causing regular reconnections (not mega-fast ~every 10
minutes) so initially I wondered if that was where the problem may lie,
however since we sorted the network problem it has been running as fast
as I'd expect with nothing obvious in the logs (no reconnections now)
and a queue that now stays pretty close to empty.
As I say below we've got a 0.8 qpid::client producer delivering to
amq.match on a broker co-located on the same host which is federated to
another 0.8 broker (all brokers are c++) via a source queue route.
That is all that's happening on the problem broker?
Yeah the problem broker doesn't have much going on at all. It has a
single queue, the producer is writing to amq.match and there are a
couple of matches in the x-match-all binding. The queue is sized to 1G
the server has 24GB. The queue is a basic in-memory ring queue so no
persistence or anything going on.
We are using source routes set up with qpid-config -s
That has me thinking I guess that I should standard routes just to rule
out source routes as a potential problem. I don't suppose you are aware
of any potential issues with source routes (he says clutching at straws....)
The producer is a slightly unusual fairly real time thing that mmaps
it's main chunk of memory and uses an internal memory pool to optimise
multi-threaded memory access but I believe that it is only grabbing ~8GB
rpm -qv <name> will give you the versioned name of the rpms which may
shed some light...
Is there a list of libraries that qpid has dependencies on available
anywhere so I can compare the info I get from the above command with
what *should* be in place on the host?
Nothing that should cause leaks... are you running any RHEL6?
I *think* it might be RHEL5, unfortunately I can't check at the moment
as I'm mailing from home. Why did you ask about RHEL6, are there any
issues with that?
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
