Further to the below, there is perhaps also a middle-ground option that certainly I've not even been considering so far.
The prior multicast behaviour accepted all the time, but forwarded unsettled messages as pre-settled and so couldnt govern sender credit replenishment, and could suffer from congestion handling inside a router network. The new multicast behaviour releases if there are no consumers, or accepts/releases/modifies/rejects based on some decision list if there are, and uses unsettled forwarding to help govern sender credit and avoid congestion inside a router network. These are to an extent two extremes, and in the middle there is maybe something of a compromise like 'accept like a topic as before, but govern credit like now'. Maybe that might be another option? Robbie On Mon, 4 Nov 2019 at 11:48, Robbie Gemmell <robbie.gemm...@gmail.com> wrote: > > On Fri, 1 Nov 2019 at 16:16, Ted Ross <tr...@redhat.com> wrote: > > > > On Fri, Nov 1, 2019 at 10:46 AM Robbie Gemmell <robbie.gemm...@gmail.com> > > wrote: > > > > > On Thu, 31 Oct 2019 at 17:31, Ted Ross <tr...@redhat.com> wrote: > > > > > > > > On Thu, Oct 31, 2019 at 6:57 AM Robbie Gemmell <robbie.gemm...@gmail.com > > > > > > > > wrote: > > > > > > > > > With the below, are you saying that previously the router would always > > > > > immediately accept an unsettled message sent to a multicast address, > > > > > and then either send it on (pre-settled) or drop it if there was > > > > > nowhere to direct it > > > > > > > > > > > > Yes. > > > > > > > > > > > > > ....but in the latter case, now it would release > > > > > it instead? > > > > > > > > > > > > Yes. > > > > > > > > > > > > > If so, is it possible to configure the old behaviour, for > > > > > folks that actually wanted that? > > > > > > > > > > > > > Perhaps. Does anyone want it? It _would_ prevent the looping behavior. > > > > > > > > > > Who knows, did we ask? I dont think it is at all out of the realms of > > > possibility that some do, especially given it is essentially how a > > > brokered topic might often work, and how dispatch behaved until now. > > > > > > My main point is jsut that it was a fairly large change in behaviour > > > given it worked the way it had for some time deliberately (as opposed > > > to some bug), and now all of a sudden it isnt possible to get that > > > behaviour at all. I dont think its a very nice change to have made in > > > a minor release without providing any option to keep things working as > > > they had. As suggested, sending pre-settled is the closest equivalent > > > now (though not quite identical), but that unfortunately may require > > > updating all your senders too. > > > > > > > I agree with you in general about the change in behavior, however the old > > behavior in this case was never a desirable behavior. > > > > A little history on this feature: > > > > When we first supported multicast distribution, the default behavior was to > > reject all unsettled deliveries to multicast addresses. This was > > technically correct in that multicast was only appropriate for pre-settled > > QoS. In practice, however, this was a pretty serious problem. Not reading > > the fine print, developers would send unsettled deliveries to multicast > > addresses and get rejections and not understand why their communication > > didn't work. > > > > Because of this, we changed the behavior to accept unsettled deliveries > > even if they were not delivered to any consumer. I believe there was a > > detailed discussion of this on this email list. > > > > Yes, I remember discussing this a few times on the list and elsewhere. > I always advocated that unsettled deliveries should be allowed and so > didnt like the earlier behaviour to forbid them, I was glad of the > change to accepting unsettled deliveries by default. I didnt, and > still dont, believe you should have to change to send pre-settled to > influence certain server behaviours (particularly given the way the > router can handle congestion with pre-settled deliveries), but rather > only do so if its what you originally wanted at the client. > > > Now, we have a much more proper way to handle unsettled deliveries on > > multicast addresses. The question that arose was how to handle the > > situation when there are no consumers. We could behave like anycast > > addresses and throttle the senders, or we could behave more like a topic > > and allow the sending of messages, but release the deliveries if there are > > no consumers. > > > > In light of the current conversation, I would propose that the behavior be > > made the same as anycast distribution without any configurable options. I > > don't think that releasing deliveries is all that "topic-like" and > > accepting them is incorrect and misleading. > > > > I agree that releasing deliveries to multicast addresses without > consumers is not all that 'topic-like'. However, I dont think > accepting them is incorrect or misleading. It might not be what > someone who really wanted more queue-like behaviour expected, but its > not particularly out of character for 'topic-like' behaviour. > > I think the current 1.9.0 behaviour is the wrong default/sole > behaviour personally, and the old behaviour would be preferable as a > default (and also nice is that would mean not significantly changing > the default behaviour in a minor release). I appreciate the new > behaviour has some properties that are very useful in certain cases, > and so much preferable to some folks, so please dont think I'm saying > I'm against such behaviour existing as I'm not. > > We already know from the previous bug report mentioned in the thread > (DISPATCH-779) that there is at least one user out there who expected > multicast addresses to grant credit and permit sending even if there > are no consumers. I would expect the same personally, per the above. I > expect they would also have been surprised to get releases back if > they were sending unsettled. > > > What would a broker do if messages were sent to a non-persistent topic for > > which there were no subscribers? > > > > Brokers would often behave in the same way that Dispatch 1.9.0 did for > sends...issuing credit to allow sends, accepting messages sent, and > dropping them if there are currently no topic subscriptions to add the > message to. On send at least they (routers, brokers) used to behave > essentially the same here, but as of 1.9.0 they are now very > different. > > > > > > > > > > > > > > > > > > > > Or does it still accept in that case and you just meant that the end > > > > > receiver outcomes are now interpreted to decide the response to the > > > > > sender? What if the different multicast points return different > > > > > outcomes? > > > > > > > > > > > > If there are different dispositions, there is a priority list that > > > > determines which of the provided dispositions will be returned to the > > > > sender. > > > > > > > > > > > > > What if one recipient doesnt provide an outcome for ages? > > > > > > > > > > > > Then that delivery will not be settled for ages. > > > > > > > > > > > > > Or > > > > > goes away without providing one? > > > > > > > > > > > > > Then the outcome from that recipient will be MODIFIED at the time the > > > > receiver detaches. > > > > > > > > > > > > > > > > > > Or maybe it was both of the above? :) > > > > > > > > > > In some ways either of these seem like odd changes for a minor release > > > > > unless its possible to toggle the previous long standing behaviour, > > > > > and not say have to switch all your senders to pre-settled to mimmick > > > > > but still not quite match it (since at least with the unsettled case > > > > > before, youd at least know whether a [first, if multiple hops] router > > > > > processed the message at all). > > > > > > > > > > Robbie > > > > > > > > > > On Wed, 30 Oct 2019 at 13:54, Ken Giusti <kgiu...@redhat.com> wrote: > > > > > > > > > > > > On Tue, Oct 29, 2019 at 6:23 PM VERMEULEN Olivier < > > > > > > olivier.vermeu...@murex.com> wrote: > > > > > > > > > > > > > Hello, > > > > > > > > > > > > > > Yes the waypoint address (in from broker) is using a multicast > > > > > > > distribution. > > > > > > > Unfortunately skipping the broker is not an option for us right > > > now. > > > > > > > Our whole architecture relies on the broker to guarantee that no > > > > > messages > > > > > > > will ever be lost... > > > > > > > > > > > > > > > > > > > That won't be the case for multicast actually. Prior to release > > > 1.9.0 of > > > > > > the router multicast messages would be dropped without notification > > > when > > > > > > under load. > > > > > > > > > > > > This relates to the issue you're experiencing now I believe. In > > > 1.9.0 we > > > > > > fixed this via > > > > > > > > > > > > https://issues.apache.org/jira/browse/DISPATCH-1266 > > > > > > > > > > > > Previously multicast messages were marked a pre-settled on entry to > > > the > > > > > > router and an "accepted" status was returned to the sender _before_ > > > the > > > > > > multicast was forwarded at all. Since the message was marked > > > pre-settled > > > > > > the router mesh will be more likely to drop it should congestion > > > occur. > > > > > > (Note this not the case with unsettled anycast messages - the router > > > will > > > > > > send a "release" status should the message need be discarded). > > > > > > > > > > > > This auto-settle behavior was undesirable for a number of reasons as > > > you > > > > > > can imagine, so in 1.9.0 we changed the behavior of multicast > > > deliveries: > > > > > > > > > > > > *unsettled* messages sent to multicast addresses are no longer > > > > > pre-settled > > > > > > by the router. The router will send back the final acknowledgement > > > > > > (accepted, release, etc) once all *present* subscribers for the > > > multicast > > > > > > address return acknowledgements. > > > > > > > > > > > > The behavior of pre-settled multicast did not change. > > > > > > > > > > > > So you can probably restore the original behavior by reverted back > > > to a > > > > > > pre-1.9.0 release, however be aware that even then there's no > > > guarantee > > > > > the > > > > > > message will be dropped. In fact it's _more_ likely to be dropped > > > (and > > > > > > signalled as accepted) in pre-1.9.0 releases. > > > > > > > > > > > > > > > > > > > > > > > > > For information we're asking for a quick workaround because we're > > > > > facing > > > > > > > this problem on a client production environment... > > > > > > > > > > > > > > Thanks, > > > > > > > Olivier > > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: Ken Giusti <kgiu...@redhat.com> > > > > > > > Sent: mardi 29 octobre 2019 18:07 > > > > > > > To: users <users@qpid.apache.org> > > > > > > > Subject: Re: multicast without consumers > > > > > > > > > > > > > > On Tue, Oct 29, 2019 at 11:54 AM jeremy <jeremyao...@gmail.com> > > > wrote: > > > > > > > > > > > > > > > Hello Gordon, > > > > > > > > > > > > > > > > We debugged the dispatch router, and fell on the code which > > > releases > > > > > > > > undeliverable messages( > > > > > > > > > > > > > https://github.com/apache/qpid-dispatch/blob/1.5.0/src/router_core/tra > > > > > > > > nsfer.c#L869 > > > > > > > > ). > > > > > > > > > > > > > > > > Check the comment on line 879. It states that if the > > > distribution is > > > > > > > > multicast, the credit will be replenished after the release. The > > > > > issue > > > > > > > > that introduced this behavior is: > > > > > > > > https://issues.apache.org/jira/browse/DISPATCH-1012 > > > > > > > > > > > > > > > > > > > > > > > Is the waypoint address (in from broker) using multicast > > > distribution? > > > > > > > > > > > > > > The router treats multicast addresses like topics - you can > > > publish to > > > > > a > > > > > > > multicast address (topic) regardless of the presence of consumers. > > > > > That's > > > > > > > the reason credit is being replenished even when no consumers are > > > > > present. > > > > > > > > > > > > > > That's probably what's happening here - broker sends first queued > > > > > message > > > > > > > to the router, which attempts to send it to the topic. Since > > > there > > > > > are no > > > > > > > consumers (and the message is sent from the broker as unsettled) > > > the > > > > > > > router cannot deliver it so it returns the released status. The > > > > > released > > > > > > > status causes the broker to redeliver the message. Repeat. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > In fact, we need an urgent fix/workaround for this. Perhaps > > > there is > > > > > a > > > > > > > > quick workaround, awaiting the full analysis of this problem? > > > > > > > > > > > > > > > > > > > > > > > As a work around can you avoid sending these multicast messages to > > > the > > > > > > > broker queue? In other words send them directly to the router > > > instead > > > > > of > > > > > > > using a waypoint? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ----- > > > > > > > > Cheers, > > > > > > > > Jeremy > > > > > > > > -- > > > > > > > > Sent from: > > > > > > > > > > > http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > > > To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For > > > > > > > > additional commands, e-mail: users-h...@qpid.apache.org > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > -K > > > > > > > ******************************* > > > > > > > This e-mail contains information for the intended recipient only. > > > It > > > > > may > > > > > > > contain proprietary material or confidential information. If you > > > are > > > > > not > > > > > > > the intended recipient you are not authorized to distribute, copy > > > or > > > > > use > > > > > > > this e-mail or any attachment to it. Murex cannot guarantee that > > > it is > > > > > > > virus free and accepts no responsibility for any loss or damage > > > arising > > > > > > > from its use. If you have received this e-mail in error please > > > notify > > > > > > > immediately the sender and delete the original email received, any > > > > > > > attachments and all copies from your system. > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > -K > > > > > > > > > > --------------------------------------------------------------------- > > > > > To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org > > > > > For additional commands, e-mail: users-h...@qpid.apache.org > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org > > > For additional commands, e-mail: users-h...@qpid.apache.org > > > > > > --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
