Delayed subscription is simpler, and probably worth doing in the broker IF done right.
How is this different from a subscription running behind? Why does supporting that require this complex a change in the dispatcher, when we already support backlogged subscribers? I am extremely wary of changes in the dispatcher. The recent changes made to support DLQ caused major problems with garbage collection, broker failure and service interruptions for us. Even though we ARE NOT using the DLQ feature. Not a pleasant experience. This is a very performance sensitive piece of code, and it should be treated as such. Joe On Wed, Feb 13, 2019 at 3:58 PM Sijie Guo <guosi...@gmail.com> wrote: > Hi all, > > I am going to wrap up the discussion regarding delayed delivery use cases. > > For arbitrary delayed delivery, there are a few +1s to doing PIP-26 in > functions. I am assuming that we will go down this path, unless there are > other proposals. > > However there is a use case Lovelle pointed out about "Fixed Delayed > Message". More specifically it is > https://github.com/apache/pulsar/pull/3155 > (The caption in #3155 is a bit misleading). IMO it is a "delayed > subscription", basically all messages in the subscription is delayed to > dispatch in a given time interval. The consensus of this feature is not yet > achieved. Basically, there will be two approaches for this: > > a) DONT treat "fixed delayed message" as a different case. Just use the > same approach as in PIP-26. > b) treat "fixed delayed message" as a different case, e.g. we can better > call it "delayed subscription" or whatever can distinguish it from general > arbitrary delayed delivery. Use the approach proposed/discussed in #3155. > > I would like the community to discuss this and also come to an agreement. > So Lovelle can move forward with the approach agreed by the community. > > Thanks, > Sijie > > On Tue, Jan 29, 2019 at 6:30 AM Ezequiel Lovelle < > ezequiellove...@gmail.com> > wrote: > > > "I agree, but that is *not what #3155 tries to achieve." > > > > This typo made this phrase nonsense, sorry! > > > > On Mon, 28 Jan 2019, 16:44 Ezequiel Lovelle <ezequiellove...@gmail.com > > wrote: > > > > > > What exactly is the delayed delivery use case? > > > > > > This is helpful on systems relaying on pulsar for persistent guarantees > > > and using it for synchronization or some sort of checks, but on such > > > systems is common to have some overhead committing data on persistent > > > storage maybe due to buffered mechanism or distributing the data across > > > the network before being available. > > > > > > Surely would be more use cases I don't came across right now. > > > > > > > Random insertion and deletion is not what FIFO queues like Pulsar are > > > designed for. > > > > > > I agree, but that is now what #3155 tries to achieve. #3155 is just a > > > fixed delay for all message in a consumer, that's the reason that the > > > implementation of #3155 is quite trivial. > > > > > > +1 from me for doing PIP-26 in functions. > > > > > > -- > > > *Ezequiel Lovelle* > > > > > > > > > On Sat, 26 Jan 2019 at 09:57, Yuva raj <uvar...@gmail.com> wrote: > > > > > >> Considering the way pulsar is built +1 for doing PIP-26 in functions. > I > > am > > >> more of thinking in a way like publish it pulsar we will make it > > available > > >> in a different queuing system if you need priority and delay messages > > >> support. Pulsar functions would go enough for this kind of use cases. > > >> > > >> On Fri, 25 Jan 2019 at 22:29, Ivan Kelly <iv...@apache.org> wrote: > > >> > > >> > > Correct. PIP-26 can be implemented in Functions. I believe the > last > > >> > > discussion in PIP-26 thread kind of agree on functions approach. > > >> > > If the community is okay with PIP-26 in functions, I think that is > > >> > probably > > >> > > a good approach to start. > > >> > > > >> > +1 for doing it in functions. > > >> > > > >> > -Ivan > > >> > > > >> > > >> > > >> -- > > >> *Thanks* > > >> > > >> *Yuvaraj L* > > >> > > > > > >
