Thank you Tim!
https://issues.apache.org/jira/browse/AMQ-7061
BR,
- Simon
On Fri, 2018-09-21 at 14:55:14 +0200, Tim Bain wrote:
Please submit this as a bug in JIRA, and provide the link here.
My first guess would be that the message is dispatched to the consumer
before it expires, but expires before the consumer pulls it from the
dispatch queue (so this is consumer-side expiration rather than broker-side
expiration), and that the broker code isn't detecting that this particular
Throwable counts as expiration. But I could be wrong about those guesses,
and whoever investigates will have to see if it's that or something else.
One thing you can do to help the investigation is to allow a backlog of
messages to build up on the queue (e.g. by pointing the consumer at a
different queue for a little while) and then see if all the built-up
messages are properly discarded once the consumer reconnects. That would
help test whether broker-side expiration is working.
For your setup, I suspect that fixing your 60ms problem will reduce the
frequency at which you see this to nearly zero, so I expect that this won't
affect you in any meaningful way, but it's still a bug that should be
fixed.
Thanks,
Tim
On Fri, Sep 21, 2018, 3:03 AM Simon Lundström <si...@su.se> wrote:
Hey all!
We're running 5.15.3 and we're using
<https://github.com/ARGOeu/nagios-plugins-activemq/> for monitoring
(SSL'ed) OpenWire. Healthchecks fail and to not get our aliveness-check
queue filled with old messages we added TTL via
<
https://github.com/ARGOeu/nagios-plugins-activemq/blob/3d6d43c/lib/OpenWireProbe/src/org/activemq/probes/OpenWireProbe.java#L124-L126
>.
We got one DLQ'd OpenWire check message and just discarded it but when
we got the second one I started to investigate.
The message which got put into the DLQ has the propery:
dlqDeliveryFailureCause java.lang.Throwable: Message Expired.
Expiration:1537468027378
Our configuration related to the queue aliveness-test in activemq.xml
is:
[...]
<authorizationEntry
queue="aliveness-test.>"
read="monitoring"
write="monitoring"
admin="admins"
/>
[...]
<queue physicalName="aliveness-test"/>
[...]
<policyEntry queue="aliveness-test">
<!--
Tell the dead letter strategy not to process expired messages
so that they will just be discarded instead of being sent to
the DLQ
-->
<deadLetterStrategy>
<sharedDeadLetterStrategy processExpired="false" />
</deadLetterStrategy>
</policyEntry>
[...]
I just noticed a bug that our TTL is set to 60ms (instead of seconds)
but that's just how fast ActiveMQ is ; ) So I don't think that's
related.
So what we expect is for those messages to just get deleted but instead
we they get DLQ'd into ActiveMQ.DLQ (since no DLQ is configured).
Thanks and have a great weekend!
BR,
- Simon
____________________________________
Simon Lundström
Section for Infrastructure
IT Services
Stockholm University
SE-106 91 Stockholm, Sweden
www.su.se/english/staff-info/it
https://www.su.se/english/about-this-website/privacy-policy
On Sep 21, 2018 3:03 AM, "Simon Lundström" <si...@su.se> wrote:
Hey all!
We're running 5.15.3 and we're using
<https://github.com/ARGOeu/nagios-plugins-activemq/> for monitoring
(SSL'ed) OpenWire. Healthchecks fail and to not get our aliveness-check
queue filled with old messages we added TTL via
<
https://github.com/ARGOeu/nagios-plugins-activemq/blob/3d6d43c/lib/OpenWireProbe/src/org/activemq/probes/OpenWireProbe.java#L124-L126
.
We got one DLQ'd OpenWire check message and just discarded it but when
we got the second one I started to investigate.
The message which got put into the DLQ has the propery:
dlqDeliveryFailureCause java.lang.Throwable: Message Expired.
Expiration:1537468027378
Our configuration related to the queue aliveness-test in activemq.xml
is:
[...]
<authorizationEntry
queue="aliveness-test.>"
read="monitoring"
write="monitoring"
admin="admins"
/>
[...]
<queue physicalName="aliveness-test"/>
[...]
<policyEntry queue="aliveness-test">
<!--
Tell the dead letter strategy not to process expired messages
so that they will just be discarded instead of being sent to
the DLQ
-->
<deadLetterStrategy>
<sharedDeadLetterStrategy processExpired="false" />
</deadLetterStrategy>
</policyEntry>
[...]
I just noticed a bug that our TTL is set to 60ms (instead of seconds)
but that's just how fast ActiveMQ is ; ) So I don't think that's
related.
So what we expect is for those messages to just get deleted but instead
we they get DLQ'd into ActiveMQ.DLQ (since no DLQ is configured).
Thanks and have a great weekend!
BR,
- Simon
____________________________________
Simon Lundström
Section for Infrastructure
IT Services
Stockholm University
SE-106 91 Stockholm, Sweden
www.su.se/english/staff-info/it
https://www.su.se/english/about-this-website/privacy-policy
