[
https://issues.apache.org/jira/browse/ARTEMIS-4371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17764670#comment-17764670
]
Nicolas Filotto edited comment on ARTEMIS-4371 at 9/13/23 1:21 PM:
-------------------------------------------------------------------
I confirm your behavior and after a deeper investigation, it appears that it is
due to ARTEMIS-3962 more specifically this commit
[https://github.com/apache/activemq-artemis/commit/8b56c04293] which changed
the logic of how to compare 2 operands when one of them is null and
consequently the filter of the notification queue.
The filter of the notification queue is \{{_AMQ_Binding_Type<>2 AND
_AMQ_NotifType IN
('SESSION_CREATED','BINDING_ADDED','BINDING_REMOVED','CONSUMER_CREATED','CONSUMER_CLOSED','PROPOSAL','PROPOSAL_RESPONSE','UNPROPOSAL')
AND _AMQ_Distance<1 AND (((_AMQ_Address NOT LIKE '$.artemis.internal.sf.%')
AND (_AMQ_Address NOT LIKE 'activemq.management%') AND (_AMQ_Address NOT LIKE
'ActiveMQ.Advisory.%'))) AND (_AMQ_NotifType = 'SESSION_CREATED' OR
(_AMQ_Address NOT LIKE 'activemq.notifications%'))}} in version 2.19.1 while it
is {{(_AMQ_Binding_Type <> 2 OR _AMQ_Binding_Type IS NULL) AND _AMQ_NotifType
IN ('SESSION_CREATED', 'BINDING_ADDED', 'BINDING_REMOVED', 'CONSUMER_CREATED',
'CONSUMER_CLOSED', 'PROPOSAL', 'PROPOSAL_RESPONSE', 'UNPROPOSAL') AND
_AMQ_Distance < 1 AND (((_AMQ_Address NOT LIKE '$.artemis.internal.sf.%') AND
(_AMQ_Address NOT LIKE 'activemq.management%') AND (_AMQ_Address NOT LIKE
'ActiveMQ.Advisory.%'))) AND (_AMQ_NotifType = 'SESSION_CREATED' OR
(_AMQ_Address NOT LIKE 'activemq.notifications%'))}} in version 2.26.0. Note
that {{_AMQ_Binding_Type<>2}} has been replaced by {{{}(_AMQ_Binding_Type <> 2
OR _AMQ_Binding_Type IS NULL){}}}.
AFAIU from the code, because of this behavior change, the notification allowing
your first broker to know that there is a consumer connected to the second
broker is simply filtered out such that the messages received by your first
broker are not broadcasted to the second broker hence the behavior that you get.
As long as the first broker is not upgraded and restarted, in theory, one way
to work around this problem is to modify the filter of the notification queue
of the first broker directly on the second broker thanks to JMX or the
WebConsole but in practice, I don't know if it is recommended as I'm not
ActiveMQ Artemis expert so let's see what others will say/propose.
was (Author: JIRAUSER285918):
I confirm your behavior and after a deeper investigation, it appears that it is
due to ARTEMIS-3962 more specifically this commit
[https://github.com/apache/activemq-artemis/commit/8b56c04293] which changed
the logic of how to compare 2 operands when one of them is null and
consequently the filter of the notification queue.
The filter of the notification queue is {{_AMQ_Binding_Type<>2 AND
_AMQ_NotifType IN
('SESSION_CREATED','BINDING_ADDED','BINDING_REMOVED','CONSUMER_CREATED','CONSUMER_CLOSED','PROPOSAL','PROPOSAL_RESPONSE','UNPROPOSAL')
AND _AMQ_Distance<1 AND (((_AMQ_Address NOT LIKE '$.artemis.internal.sf.%')
AND (_AMQ_Address NOT LIKE 'activemq.management%') AND (_AMQ_Address NOT LIKE
'ActiveMQ.Advisory.%'))) AND (_AMQ_NotifType = 'SESSION_CREATED' OR
(_AMQ_Address NOT LIKE 'activemq.notifications%'))
}} in version 2.19.1 while it is \{{(_AMQ_Binding_Type <> 2 OR
_AMQ_Binding_Type IS NULL) AND _AMQ_NotifType IN ('SESSION_CREATED',
'BINDING_ADDED', 'BINDING_REMOVED', 'CONSUMER_CREATED', 'CONSUMER_CLOSED',
'PROPOSAL', 'PROPOSAL_RESPONSE', 'UNPROPOSAL') AND _AMQ_Distance < 1 AND
(((_AMQ_Address NOT LIKE '$.artemis.internal.sf.%') AND (_AMQ_Address NOT LIKE
'activemq.management%') AND (_AMQ_Address NOT LIKE 'ActiveMQ.Advisory.%'))) AND
(_AMQ_NotifType = 'SESSION_CREATED' OR (_AMQ_Address NOT LIKE
'activemq.notifications%'))}} in version 2.26.0. Note that
\{{_AMQ_Binding_Type<>2}} has been replaced by \{{(_AMQ_Binding_Type <> 2 OR
_AMQ_Binding_Type IS NULL)}}.
AFAIU from the code, because of this behavior change, the notification allowing
your first broker to know that there is a consumer connected to the second
broker is simply filtered out such that the messages received by your first
broker are not broadcasted to the second broker hence the behavior that you get.
As long as the first broker is not upgraded and restarted, in theory, one way
to work around this problem is to modify the filter of the notification queue
of the first broker directly on the second broker thanks to JMX or the
WebConsole but in practice, I don't know if it is recommended as I'm not
ActiveMQ Artemis expert so let's see what others will say/propose.
> Rolling upgrade causes messages to not be forwarded between members of cluster
> ------------------------------------------------------------------------------
>
> Key: ARTEMIS-4371
> URL: https://issues.apache.org/jira/browse/ARTEMIS-4371
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Affects Versions: 2.26.0, 2.27.0, 2.28.0
> Reporter: Andy Gustafson
> Priority: Major
> Attachments: artemisbugconfig.tar.gz
>
>
> I've found what appears to me to be a bug in forwarding messages between
> brokers in a symmetric cluster while doing a rolling upgrade. From my brief
> testing this problem starts on version 2.26. I have not tested with a
> starting version other than 2.19.1.
>
> *How to reproduce*
> I've attached a basic configuration in the tar file. Due to size limits of
> Jira I wasn't able to include the Artemis releases.
> Download the attached tar.gz with the broker configs and extract in /opt.
> Download/extract artemis 2.19.1 and 2.26.0 to the /opt/artemisbug directory.
> *From the /opt/artemisbug directory*
> Start broker1
> {code:java}
> broker1/bin/artemis run{code}
> Start broker2
> {code:java}
> broker2/bin/artemis run{code}
> Create a producer against broker1
> {code:java}
> ./apache-artemis-2.19.1/bin/artemis producer --url=tcp://localhost:61618
> --clientID=123 --destination=queue://test --message-count=10000000
> --non-persistent --sleep=2{code}
> Create consumer against broker2
> {code:java}
> ./apache-artemis-2.19.1/bin/artemis consumer --url=tcp://localhost:61619
> --clientID=321 --destination=queue://test --message-count=10000000{code}
> Stop broker2
> upgrade broker2 (script included for convenience)
> {code:java}
> ./broker2/etc/cutover26.sh{code}
> Start broker2
> {code:java}
> broker2/bin/artemis run{code}
> Restart consumer against broker2
> {code:java}
> ./apache-artemis-2.19.1/bin/artemis consumer --url=tcp://localhost:61619
> --clientID=321 --destination=queue://test --message-count=10000000{code}
>
> At this point you should see no messages are being consumed even even though
> broker2 is a member of the cluster and there is a producer on broker1. No
> errors appear to be logged server side either.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
