[ https://issues.apache.org/jira/browse/ARTEMIS-4314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17733204#comment-17733204 ]
Gary Tully commented on ARTEMIS-4314: ------------------------------------- [~tabish] had a great observation to suggest that with local producers keeping local consumers busy, messages on the upstream will be stranded b/c the local queue will never be empty. That is fair, and I think a solution is to replace the check for empty with a check for pendingMessageCount == 0, such that if we have spare capacity, we pull a batch. If local producers are keeping us full, we will still leave the messages stranded, but in that case we are already at capacity. I think that is a better match. thoughts? > Federation, support consumerWindowSize zero and federate in batches only when > the local queue is empty > ------------------------------------------------------------------------------------------------------ > > Key: ARTEMIS-4314 > URL: https://issues.apache.org/jira/browse/ARTEMIS-4314 > Project: ActiveMQ Artemis > Issue Type: Improvement > Components: Federation > Affects Versions: 2.28.0 > Reporter: Gary Tully > Assignee: Gary Tully > Priority: Major > Fix For: 2.29.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Dual queue federation, where clusters federate in both direction can suffer > from message flip flopping once the priority adjustment kicks in. > If there is a large backlog, the lower priority federation consumer is in > play once all of the local consumer credit is exhausted and the backlog can > drain to the other cluster. > If demand is low there, the process can repeat. limiting the rate of the > federation consumer can help but it is not ideal b/c when there is no local > demand, we want to have a high rate of migration. > > A possible solution is to have the federation consumer manage its own credit > and only flow messages when the local queue is empty. Then flow a batch of > messages, and await again that the local queue is empty. In this way, there > is no thundering heard effect, but there is also fast migration of messages > once there is demand. > the consumerWindowSize=0 is already in play for consumer.receive calls and > there is already a defaultConsumerWindowSize for an address. These can be > combined to realise batchFederationOnEmpty semantics. -- This message was sent by Atlassian Jira (v8.20.10#820010)