Hi Hasitha, We have a similar boundary in MB4 as well. The transport will use a "Broker" object to interact with the broker core. Delivering messages with proper flags matching the protocol semantics is a responsibility of the transport since it can change depending on the transport type (in AMQP we have redelivery flag, in HTTP we might have to use an HTTP header, etc). Broker core will track and provide the required information to transport. Additionally, the transport is responsible for telling the broker core about external events like redelivery since transport is the module which knows what it did with the message. WDYT?
On Mon, Jan 15, 2018 at 9:39 AM, Hasitha Hiranya <hasit...@wso2.com> wrote: > Hi Asanka and all, > > What would be the bounty between the transport and broker core? > In pervious version we defined an interface (bridge) with operations like > message metadata received, content chunk received, recover, exchange > declare, exchange delete etc. > So, tracking messages and giving them to deliver with proper flags would > not a responsibility of the transport in that case. Transport would not > keep any status of a message. It would just translate between broker core > and the wire. If we keep status in transports, if we write a new transport, > we will have to repeat the logic there as well. > > Thoughts? > > Thanks > > On Mon, Jan 15, 2018 at 9:09 AM, Asanka Abeyweera <asank...@wso2.com> > wrote: > >> Hi all, >> >> According to the 0-9-1 spec all the exchanges and queues prefixed with >> "amq." are reserved for internal use by the broker. Therefore I used >> following as reserved entities to support dead letter functionality. >> >> - *amq.dlq* - dead letter queue name >> - *amq.dlx* - dead letter exchange name >> >> Additionally, we are setting the following headers in the message to >> identify the original headers of the message. >> >> - x-origin-queue >> - x-origin-exchange >> - x-origin-routing-key >> >> Additionally, the responsibility of setting the redelivery flag is given >> to the transport module. The main reason is that the transport can >> redeliver without sending it back to the broker-core. For example if >> basic.recover frame is received with the requeue flag set to false, the >> transport will send the messages back to the client without going to the >> broker core. >> >> >> >> On Fri, Jan 12, 2018 at 1:39 PM, Eranda Rajapakshe <eran...@wso2.com> >> wrote: >> >>> >>> >>> On Fri, Jan 12, 2018 at 11:27 AM, Hasitha Hiranya <hasit...@wso2.com> >>> wrote: >>> >>>> Hi Asanka, >>>> >>>> +1. Seems we need to consider all clients w:r:t a queue. >>>> In previous model also we had a property to count number of times a >>>> message was delivered and a delivery rule to evaluate but it was >>>> channel-wise (per subscriber). >>>> >>>> So if a queue or a shared durable topic subscriber has more than one >>>> subscriber, we should consider deliveries to all consumers of that >>>> queue/internal queue of durable topic subscriber. Let us go with that >>>> approach. >>>> >>>> Thanks >>>> >>>> >>>> On Fri, Jan 12, 2018 at 10:55 AM, Asanka Abeyweera <asank...@wso2.com> >>>> wrote: >>>> >>>>> In the previous MB versions, we were counting the redelivery count >>>>> independently for each consumer. But according to the spec, this is >>>>> counted >>>>> independently of the delivered consumer. >>>>> >>>>> The server SHOULD track the number of times a message has been >>>>>> delivered to *clients* and when a message is redelivered a certain >>>>>> number of times e.g. 5 times without being acknowledged, the server >>>>>> SHOULD consider the message to be unprocessable (possibly causing client >>>>>> applications to abort), and move the message to a dead letter queue. >>>>>> >>>>> >>>>> Therefore we are planning to keep the redelivery count independent of >>>>> the consumer and publish the message to DLX when the count exceeds the >>>>> configured limit. >>>>> >>>> +1, it makes more sense to keep the redelivery count independent from >>> the consumer. There can be situations where client application does not >>> cache the consumer for each retry, even this scenario will work with the >>> new design. >>> >>>> >>>>> >>>>> On Fri, Jan 12, 2018 at 10:15 AM, Asanka Abeyweera <asank...@wso2.com> >>>>> wrote: >>>>> >>>>>> Hi all, >>>>>> >>>>>> For message brokers, we need to provide a location to dump >>>>>> undeliverable messages. Messages can be undeliverable due to following >>>>>> reasons. >>>>>> >>>>>> 1. Message redelivery count exceeds a configured limit. >>>>>> 2. The queue length limit is exceeded. >>>>>> >>>>>> In previous versions of MB, we were using a dedicated queue (DLC) to >>>>>> persist those undeliverable messages. This was adding some complexity in >>>>>> handling permission and different policies with DLC. In MB4 we are >>>>>> planning >>>>>> to implement a Dead Letter Exchange (DLX) instead. >>>>>> In the first iteration of the implementation, we planning to have an >>>>>> inbuilt DLX which is used to publish undeliverable messages. Type of the >>>>>> DLX is of type direct. We will also create a default dead letter queue >>>>>> and >>>>>> have a binding added to the DLX for the default dead letter queue. This >>>>>> will be the default path of any undeliverable message. >>>>>> >>>>>> In a later iteration, we can add the capability to define custom >>>>>> queues and exchanges for routing undeliverable messages. This will make >>>>>> it >>>>>> easy ti define different policies for different dead letter scenarios. >>>>>> >>>>> +1, If we can restrict putting messages directly into queues and use >>> exchanges always to publish messages(into queues), it will make the broker >>> configurations more flexiable and architechture will be clear to >>> understand. >>> With that design, external parties can only publish messages into >>> exchanges, and consume only from queues. Exchanges are the only ones who is >>> capable of routing messages into queues. I think it will be better to >>> maintain such a policy all the time. >>> >>>> WDYT? >>>>>> -- >>>>>> Asanka Abeyweera >>>>>> Associate Technical Lead >>>>>> WSO2 Inc. >>>>>> >>>>>> Phone: +94 712228648 <+94%2071%20222%208648> >>>>>> Blog: a5anka.github.io >>>>>> >>>>>> <https://wso2.com/signature> >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Asanka Abeyweera >>>>> Associate Technical Lead >>>>> WSO2 Inc. >>>>> >>>>> Phone: +94 712228648 <+94%2071%20222%208648> >>>>> Blog: a5anka.github.io >>>>> >>>>> <https://wso2.com/signature> >>>>> >>>> >>>> >>>> >>>> -- >>>> *Hasitha Abeykoon* >>>> Associate Technical Lead; WSO2, Inc.; http://wso2.com >>>> *cell:* *+94 719363063* >>>> *blog: **abeykoon.blogspot.com* <http://abeykoon.blogspot.com> >>>> >>>> >>>> _______________________________________________ >>>> Architecture mailing list >>>> Architecture@wso2.org >>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>> >>>> >>> >>> >>> -- >>> *Eranda Rajapakshe* >>> Software Engineer >>> WSO2 Inc. >>> Mobile : +94784822608 >>> >> >> >> >> -- >> Asanka Abeyweera >> Associate Technical Lead >> WSO2 Inc. >> >> Phone: +94 712228648 <+94%2071%20222%208648> >> Blog: a5anka.github.io >> >> <https://wso2.com/signature> >> >> _______________________________________________ >> Architecture mailing list >> Architecture@wso2.org >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > *Hasitha Abeykoon* > Associate Technical Lead; WSO2, Inc.; http://wso2.com > *cell:* *+94 719363063* > *blog: **abeykoon.blogspot.com* <http://abeykoon.blogspot.com> > > > _______________________________________________ > Architecture mailing list > Architecture@wso2.org > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- Asanka Abeyweera Associate Technical Lead WSO2 Inc. Phone: +94 712228648 Blog: a5anka.github.io <https://wso2.com/signature>
_______________________________________________ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
