Rob,
Our consumer client version is 1.37:
bash 0 3: rpm -qa | grep -i qpid
qpid-cpp-client-devel-1.37.0-1.el7.x86_64
python-qpid-1.37.0-1.el7.noarch
qpid-qmf-1.37.0-1.el7.x86_64
qpid-cpp-client-1.37.0-1.el7.x86_64
python-qpid-qmf-1.37.0-1.el7.x86_64
qpid-cpp-server-1.37.0-1.el7.x86_64
qpid-proton-c-0.18.1-1.el7.x86_64
qpid-tools-1.37.0-1.el7.noarch
Our producer client version is 6.1.5:
-rw-r--r-- 1 NSDA.NSDA NSDA 570873 Jul 26 15:42
qpid-client-6.1.5.jar
-rw-r--r-- 1 NSDA.NSDA NSDA 864493 Jul 26 15:42
qpid-common-6.1.5.jar
Both producer and consumer clients use AMQP_0_10 protocol.
Thanks for your assistance,
Mike
FYI - There may be a typo on the past releases web page. I believe "2017"
should be "2018" in "Qpid JMS AMQP 0-x 6.3.2, July 2017". URL:
https://qpid.apache.org/releases/index.html#past-releases
-----Original Message-----
From: Rob Godfrey [mailto:rob.j.godf...@gmail.com]
Sent: Thursday, August 2, 2018 5:57 PM
To: users@qpid.apache.org
Cc: Mears, David B <david.me...@hpe.com>; Herren, Elaine
<elaine.her...@hpe.com>; Rao, Shobha (NonStop) <shobha.jayatheer...@hpe.com>
Subject: Re: Java Broker (7.0.6) stops delivering queue/consumer messages after
4 GB data transfer
Hi Mike,
On Fri, 3 Aug 2018 at 01:25, Dyslin, Mike <mike.dys...@hpe.com> wrote:
> This is my first submit to this email group. Hopefully this is the
> correct place to post this problem.
>
>
This is exactly the right place to post this problem.
> We are running a continuous stream of message (about 5K each) from
> producer to consumer over a single java broker queue at a rate of
> about 600 messages/second. Outbound message flow stops after
> transferring 4 GB of message data (about 770,000 messages in 25
> minutes). The Web Management Console page for our consumer connection shows
> the total "Outbound Bytes"
> growing steadily until it reaches 4.0 GB and stops with "Last I/O time"
> unchanging thereafter.
>
> After outbound messages stop:
> Inbound messages continue on the producer connection (well past 4.0
> GB) and are kept in the queue until they expire with a time-to-live
> value of 3 minutes. The queue grows until is stabilizes with a steady
> 600 m/s inbound, and 600 m/s expiring and being deleted from the queue
> (as expected). The Web Management Console shows that the consumer
> connection remains open and is a consumer on the queue, and the queue
> shows the connection as a consumer on the queue.
>
> If I run the exact same test replacing the Java Broker with a C++
> broker (1.37.0), message flow continues well past the 4 GB barrier. I
> kept it running for about 17 hours reaching about 37 million messages,
> about 180 GB data transferred on the queue.
>
> Since the only difference seems to be the broker, this seems to point
> to a problem with the Java Broker, and not issues with our producer,
> consumer or network issues. Could there be some problem with our java
> broker configuration that would explain this behaviour?
>
Unfortunately this sounds like it may be a bug in the Java Broker :-(
>
> Has anyone out there experienced more than 4 GB of outbound data on a
> single java broker connection or queue?
>
>
Can you confirm which client you are using, and which version of AMQP is in use
(as you have identified I don't expect this to be a client problem, but knowing
the client will help us track down the issue in the broker)?
> Any help would be appreciated.
>
> Other comments/observations:
>
> I do not know if the 4 GB barrier is associated with the connection
> and/or the queue because all our message traffic is over one consumer
> connection and one queue. I could determine this by changing our
> consumer code to spread message traffic over one connection and multiple
> queues.
>
>
We are using the heartbeat feature with a 5 minute timeout. Since the
> connection stays open beyond the 5 minute timeout after the messages
> stop, I assume the heartbeat messages are still being sent between
> consumer and broker, indicating that the consumer and broker are able
> to communicate over the socket. It has been awhile since I have
> tested that the heartbeat feature is working correctly.
>
> If I close the consumer connection from the Web Management Console,
> the broker deletes the queue (I believe) and our consumer detects the
> closed connection, establishes a new connection and new queue, and
> messages start flowing again until . . . we reach the 4 GB barrier and
> messages stop being delivered once again.
>
> We have run with the Java Broker on both Linux (RHEL 7.4) and
> proprietary NonStop POSIX platform with the same results.
> Unfortunately, the C++ broker is not yet an option on the NonStop
> POSIX platform where we require the broker to be.
>
>
Hopefully we can quickly track down the issue in the Java Broker and push out a
fix,
> Thanks,
> Mike
>
>
Apologies you've run into this issue,
Rob
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For
> additional commands, e-mail: users-h...@qpid.apache.org
>
>
