Re: [DISCUSS] PIP 33: Replicated subscriptions

2019-06-12 Thread Ivan Kelly
Joe, Matteo What was the conclusion of this discussion? Was there a conclusion? Has the algorithm used in the implementation changed from the initial draft? -Ivan On Fri, May 3, 2019 at 4:00 AM Joe F wrote: > > Let me try to explain my suggestion better. > > First, about positions in an ordered

Re: [DISCUSS] PIP 33: Replicated subscriptions

2019-05-02 Thread Joe F
Let me try to explain my suggestion better. First, about positions in an ordered stream: Consider a simple stream of events when there is no identifier on each event about its relative position in the stream. But after every 'n' events there is a ticker event carrying a monotonically increasing

Re: [DISCUSS] PIP 33: Replicated subscriptions

2019-05-01 Thread Matteo Merli
On Mon, Apr 29, 2019 at 1:57 PM Joe F wrote: > > I have suggestions for an alternate solution. > > If source message-ids were known for replicated messages, a composite > cursor can be maintained for replicated subscriptions as an n-tuple. Since > messages are ordered from a source, it would be p

Re: [DISCUSS] PIP 33: Replicated subscriptions

2019-04-29 Thread Joe F
I have suggestions for an alternate solution. If source message-ids were known for replicated messages, a composite cursor can be maintained for replicated subscriptions as an n-tuple. Since messages are ordered from a source, it would be possible to restart from a known cursor n-tuple in any clu

Re: [DISCUSS] PIP 33: Replicated subscriptions

2019-03-25 Thread Sijie Guo
On Mon, Mar 25, 2019 at 4:14 PM Matteo Merli wrote: > On Sun, Mar 24, 2019 at 9:54 PM Sijie Guo wrote: > > > > Ivan, Matteo, thank you for the writeup! > > > > I have a few more questions > > > > - How does this handle ack individual vs ack cumulatively? It seems it is > > ignored at this moment

Re: [DISCUSS] PIP 33: Replicated subscriptions

2019-03-25 Thread Matteo Merli
On Sun, Mar 24, 2019 at 9:54 PM Sijie Guo wrote: > > Ivan, Matteo, thank you for the writeup! > > I have a few more questions > > - How does this handle ack individual vs ack cumulatively? It seems it is > ignored at this moment. But it is good to have some discussions around how > to extend the a

Re: [DISCUSS] PIP 33: Replicated subscriptions

2019-03-24 Thread Sijie Guo
Ivan, Matteo, thank you for the writeup! I have a few more questions - How does this handle ack individual vs ack cumulatively? It seems it is ignored at this moment. But it is good to have some discussions around how to extend the approach to support them and how easy to do that. - Do we need to

Re: [DISCUSS] PIP 33: Replicated subscriptions

2019-03-23 Thread Matteo Merli
Hi Chris, thanks for the comments, replies inline: On Sat, Mar 23, 2019 at 5:53 AM Chris Bartholomew wrote: > > Hi, > > I have a few thoughts that are hopefully useful. I have some experience > testing a similar feature for a proprietary messaging system and understand > that system's design

Re: [DISCUSS] PIP 33: Replicated subscriptions

2019-03-23 Thread Chris Bartholomew
Hi, I have a few thoughts that are hopefully useful. I have some experience testing a similar feature for a proprietary messaging system and understand that system's design well, which is similar to this proposal. Would it make sense to have the snapshot frequency based on messages not time, o

Re: [DISCUSS] PIP 33: Replicated subscriptions

2019-03-22 Thread Matteo Merli
...and link https://docs.google.com/document/d/1WLN9vbhcqv-ZQ-9udlUBkFURpGPO9GSB6GsI7guJvZQ/edit?usp=sharing -- Matteo Merli On Fri, Mar 22, 2019 at 2:32 PM Matteo Merli wrote: > > BTW: I also added to a Google doc if someone prefers to add > comments/suggestions there rather here inline. > --

Re: [DISCUSS] PIP 33: Replicated subscriptions

2019-03-22 Thread Matteo Merli
BTW: I also added to a Google doc if someone prefers to add comments/suggestions there rather here inline. -- Matteo Merli On Fri, Mar 22, 2019 at 12:14 PM Matteo Merli wrote: > > https://github.com/apache/pulsar/wiki/PIP-33%3A-Replicated-subscriptions > > ---

[DISCUSS] PIP 33: Replicated subscriptions

2019-03-22 Thread Matteo Merli
https://github.com/apache/pulsar/wiki/PIP-33%3A-Replicated-subscriptions * **Status**: Proposal * **Author**: Ivan Kelly, Matteo Merli * **Pull Request**: * **Mailing List discussion**: * **Release**: ## Goa