Re: [DISCUSS] PIP-157 Exclusive Producer: new mode ExclusiveWithFencing
+1 for "Preemptive". Le ven. 27 mai 2022 à 00:22, Neng Lu a écrit : > Hi All, > > Would "Preemptive" Mode make sense? > > On Wed, May 11, 2022 at 8:56 AM Matteo Merli > wrote: > > > +1 > > > > > > On Tue, May 10, 2022 at 5:56 AM Enrico Olivelli > > wrote: > > > > > Hello, > > > I created a new PIP about a new AccessMode for the Producer. > > > https://github.com/apache/pulsar/issues/15528 > > > > > > This is the PR: https://github.com/apache/pulsar/pull/15488 > > > > > > Honestly I don't like the name "ExclusiveWithFencing", any suggestion > > > is really appreciated. > > > > > > Enrico > > > > > > Motivation > > > > > > In PIP-68 we introduced two access modes for the Producer: > > > > > > Exclusive: The producer is the only one who can publish to the topic. > > > Fail if there is another Exclusive Producer connected to the topic > > > while creating the new Producer. > > > WaitForExclusive: Like Exclusive, but instead of Failing we are going > > > to wait for the current Exclusive Producer to disconnect. > > > > > > Those two modes are very powerful and allow you to perform some kind > > > of Locking on a topic. > > > > > > We are missing a third more, in which the Producer always succeeds to > > > acquire the Exclusive lock on the topic by fencing out any other > > > Producer that is connected, even the current Exclusive Producer and > > > the other Producers waiting in WaitForExclusive mode. > > > > > > Goal > > > > > > The modes that are available with PIP-68 require a writer to acquire > > > the lock and release it as soon as possible in order to allow other > > > clients to write to the topic. > > > > > > With the new mode it will be possible to implement locking in another > > > way: the Producer holds the lock until someone else steals it. This > > > way when you have very low contention you can achieve better latency > > > for writes because you don't have to acquire the lock every time you > > > want to write,. > > > > > > API and Wire protocol Changes > > > > > > Changes: > > > > > > a new constant on the Wire Protocol for AccessMode > > > a new constant in the Java Client API AccessMode#ExclusiveWithFencing > > > > > > Implementation > > > > > > The new mode will behave mostly like AccessMode#Exclusive but instead > > > of failing in case of the presence of other Producers it will force > > > all of the current connected Producers to be removed and invalidated > > > (they will see ProducerFencedException). > > > > > > Reject Alternatives > > > > > > None > > > > > -- > > -- > > Matteo Merli > > > > > > > -- > Best Regards, > Neng >
Re: [DISCUSS] PIP-157 Exclusive Producer: new mode ExclusiveWithFencing
Hi All, Would "Preemptive" Mode make sense? On Wed, May 11, 2022 at 8:56 AM Matteo Merli wrote: > +1 > > > On Tue, May 10, 2022 at 5:56 AM Enrico Olivelli > wrote: > > > Hello, > > I created a new PIP about a new AccessMode for the Producer. > > https://github.com/apache/pulsar/issues/15528 > > > > This is the PR: https://github.com/apache/pulsar/pull/15488 > > > > Honestly I don't like the name "ExclusiveWithFencing", any suggestion > > is really appreciated. > > > > Enrico > > > > Motivation > > > > In PIP-68 we introduced two access modes for the Producer: > > > > Exclusive: The producer is the only one who can publish to the topic. > > Fail if there is another Exclusive Producer connected to the topic > > while creating the new Producer. > > WaitForExclusive: Like Exclusive, but instead of Failing we are going > > to wait for the current Exclusive Producer to disconnect. > > > > Those two modes are very powerful and allow you to perform some kind > > of Locking on a topic. > > > > We are missing a third more, in which the Producer always succeeds to > > acquire the Exclusive lock on the topic by fencing out any other > > Producer that is connected, even the current Exclusive Producer and > > the other Producers waiting in WaitForExclusive mode. > > > > Goal > > > > The modes that are available with PIP-68 require a writer to acquire > > the lock and release it as soon as possible in order to allow other > > clients to write to the topic. > > > > With the new mode it will be possible to implement locking in another > > way: the Producer holds the lock until someone else steals it. This > > way when you have very low contention you can achieve better latency > > for writes because you don't have to acquire the lock every time you > > want to write,. > > > > API and Wire protocol Changes > > > > Changes: > > > > a new constant on the Wire Protocol for AccessMode > > a new constant in the Java Client API AccessMode#ExclusiveWithFencing > > > > Implementation > > > > The new mode will behave mostly like AccessMode#Exclusive but instead > > of failing in case of the presence of other Producers it will force > > all of the current connected Producers to be removed and invalidated > > (they will see ProducerFencedException). > > > > Reject Alternatives > > > > None > > > -- > -- > Matteo Merli > > -- Best Regards, Neng
Re: [DISCUSS] PIP-157 Exclusive Producer: new mode ExclusiveWithFencing
+1 On Tue, May 10, 2022 at 5:56 AM Enrico Olivelli wrote: > Hello, > I created a new PIP about a new AccessMode for the Producer. > https://github.com/apache/pulsar/issues/15528 > > This is the PR: https://github.com/apache/pulsar/pull/15488 > > Honestly I don't like the name "ExclusiveWithFencing", any suggestion > is really appreciated. > > Enrico > > Motivation > > In PIP-68 we introduced two access modes for the Producer: > > Exclusive: The producer is the only one who can publish to the topic. > Fail if there is another Exclusive Producer connected to the topic > while creating the new Producer. > WaitForExclusive: Like Exclusive, but instead of Failing we are going > to wait for the current Exclusive Producer to disconnect. > > Those two modes are very powerful and allow you to perform some kind > of Locking on a topic. > > We are missing a third more, in which the Producer always succeeds to > acquire the Exclusive lock on the topic by fencing out any other > Producer that is connected, even the current Exclusive Producer and > the other Producers waiting in WaitForExclusive mode. > > Goal > > The modes that are available with PIP-68 require a writer to acquire > the lock and release it as soon as possible in order to allow other > clients to write to the topic. > > With the new mode it will be possible to implement locking in another > way: the Producer holds the lock until someone else steals it. This > way when you have very low contention you can achieve better latency > for writes because you don't have to acquire the lock every time you > want to write,. > > API and Wire protocol Changes > > Changes: > > a new constant on the Wire Protocol for AccessMode > a new constant in the Java Client API AccessMode#ExclusiveWithFencing > > Implementation > > The new mode will behave mostly like AccessMode#Exclusive but instead > of failing in case of the presence of other Producers it will force > all of the current connected Producers to be removed and invalidated > (they will see ProducerFencedException). > > Reject Alternatives > > None > -- -- Matteo Merli
Re: [DISCUSS] PIP-157 Exclusive Producer: new mode ExclusiveWithFencing
Kezhu, Il giorno mer 11 mag 2022 alle ore 04:04 Kezhu Wang ha scritto: > > Nice improvement. > > I was searching for similar functionalities to ban/fence old producers and > found a similar improvement/suggestion from [PIP 68 Suggestions]( > https://github.com/apache/pulsar/issues/7584) in addition to PIP 68. This (7584) is interesting. My proposal is in fact to extend PIP-68. Enrico > > Best, > Kezhu Wang > > On May 10, 2022 at 21:59:43, Enrico Olivelli (eolive...@gmail.com) wrote: > > I have renamed this to PIP-161 (there is some mess in the wiki page) > > Enrico > > Il giorno mar 10 mag 2022 alle ore 14:56 Enrico Olivelli > ha scritto: > > > > Hello, > > I created a new PIP about a new AccessMode for the Producer. > > https://github.com/apache/pulsar/issues/15528 > > > > This is the PR: https://github.com/apache/pulsar/pull/15488 > > > > Honestly I don't like the name "ExclusiveWithFencing", any suggestion > > is really appreciated. > > > > Enrico > > > > Motivation > > > > In PIP-68 we introduced two access modes for the Producer: > > > > Exclusive: The producer is the only one who can publish to the topic. > > Fail if there is another Exclusive Producer connected to the topic > > while creating the new Producer. > > WaitForExclusive: Like Exclusive, but instead of Failing we are going > > to wait for the current Exclusive Producer to disconnect. > > > > Those two modes are very powerful and allow you to perform some kind > > of Locking on a topic. > > > > We are missing a third more, in which the Producer always succeeds to > > acquire the Exclusive lock on the topic by fencing out any other > > Producer that is connected, even the current Exclusive Producer and > > the other Producers waiting in WaitForExclusive mode. > > > > Goal > > > > The modes that are available with PIP-68 require a writer to acquire > > the lock and release it as soon as possible in order to allow other > > clients to write to the topic. > > > > With the new mode it will be possible to implement locking in another > > way: the Producer holds the lock until someone else steals it. This > > way when you have very low contention you can achieve better latency > > for writes because you don't have to acquire the lock every time you > > want to write,. > > > > API and Wire protocol Changes > > > > Changes: > > > > a new constant on the Wire Protocol for AccessMode > > a new constant in the Java Client API AccessMode#ExclusiveWithFencing > > > > Implementation > > > > The new mode will behave mostly like AccessMode#Exclusive but instead > > of failing in case of the presence of other Producers it will force > > all of the current connected Producers to be removed and invalidated > > (they will see ProducerFencedException). > > > > Reject Alternatives > > > > None
Re: [DISCUSS] PIP-157 Exclusive Producer: new mode ExclusiveWithFencing
Nice improvement. I was searching for similar functionalities to ban/fence old producers and found a similar improvement/suggestion from [PIP 68 Suggestions]( https://github.com/apache/pulsar/issues/7584) in addition to PIP 68. Best, Kezhu Wang On May 10, 2022 at 21:59:43, Enrico Olivelli (eolive...@gmail.com) wrote: I have renamed this to PIP-161 (there is some mess in the wiki page) Enrico Il giorno mar 10 mag 2022 alle ore 14:56 Enrico Olivelli ha scritto: > > Hello, > I created a new PIP about a new AccessMode for the Producer. > https://github.com/apache/pulsar/issues/15528 > > This is the PR: https://github.com/apache/pulsar/pull/15488 > > Honestly I don't like the name "ExclusiveWithFencing", any suggestion > is really appreciated. > > Enrico > > Motivation > > In PIP-68 we introduced two access modes for the Producer: > > Exclusive: The producer is the only one who can publish to the topic. > Fail if there is another Exclusive Producer connected to the topic > while creating the new Producer. > WaitForExclusive: Like Exclusive, but instead of Failing we are going > to wait for the current Exclusive Producer to disconnect. > > Those two modes are very powerful and allow you to perform some kind > of Locking on a topic. > > We are missing a third more, in which the Producer always succeeds to > acquire the Exclusive lock on the topic by fencing out any other > Producer that is connected, even the current Exclusive Producer and > the other Producers waiting in WaitForExclusive mode. > > Goal > > The modes that are available with PIP-68 require a writer to acquire > the lock and release it as soon as possible in order to allow other > clients to write to the topic. > > With the new mode it will be possible to implement locking in another > way: the Producer holds the lock until someone else steals it. This > way when you have very low contention you can achieve better latency > for writes because you don't have to acquire the lock every time you > want to write,. > > API and Wire protocol Changes > > Changes: > > a new constant on the Wire Protocol for AccessMode > a new constant in the Java Client API AccessMode#ExclusiveWithFencing > > Implementation > > The new mode will behave mostly like AccessMode#Exclusive but instead > of failing in case of the presence of other Producers it will force > all of the current connected Producers to be removed and invalidated > (they will see ProducerFencedException). > > Reject Alternatives > > None
Re: [DISCUSS] PIP-157 Exclusive Producer: new mode ExclusiveWithFencing
I have renamed this to PIP-161 (there is some mess in the wiki page) Enrico Il giorno mar 10 mag 2022 alle ore 14:56 Enrico Olivelli ha scritto: > > Hello, > I created a new PIP about a new AccessMode for the Producer. > https://github.com/apache/pulsar/issues/15528 > > This is the PR: https://github.com/apache/pulsar/pull/15488 > > Honestly I don't like the name "ExclusiveWithFencing", any suggestion > is really appreciated. > > Enrico > > Motivation > > In PIP-68 we introduced two access modes for the Producer: > > Exclusive: The producer is the only one who can publish to the topic. > Fail if there is another Exclusive Producer connected to the topic > while creating the new Producer. > WaitForExclusive: Like Exclusive, but instead of Failing we are going > to wait for the current Exclusive Producer to disconnect. > > Those two modes are very powerful and allow you to perform some kind > of Locking on a topic. > > We are missing a third more, in which the Producer always succeeds to > acquire the Exclusive lock on the topic by fencing out any other > Producer that is connected, even the current Exclusive Producer and > the other Producers waiting in WaitForExclusive mode. > > Goal > > The modes that are available with PIP-68 require a writer to acquire > the lock and release it as soon as possible in order to allow other > clients to write to the topic. > > With the new mode it will be possible to implement locking in another > way: the Producer holds the lock until someone else steals it. This > way when you have very low contention you can achieve better latency > for writes because you don't have to acquire the lock every time you > want to write,. > > API and Wire protocol Changes > > Changes: > > a new constant on the Wire Protocol for AccessMode > a new constant in the Java Client API AccessMode#ExclusiveWithFencing > > Implementation > > The new mode will behave mostly like AccessMode#Exclusive but instead > of failing in case of the presence of other Producers it will force > all of the current connected Producers to be removed and invalidated > (they will see ProducerFencedException). > > Reject Alternatives > > None
[DISCUSS] PIP-157 Exclusive Producer: new mode ExclusiveWithFencing
Hello, I created a new PIP about a new AccessMode for the Producer. https://github.com/apache/pulsar/issues/15528 This is the PR: https://github.com/apache/pulsar/pull/15488 Honestly I don't like the name "ExclusiveWithFencing", any suggestion is really appreciated. Enrico Motivation In PIP-68 we introduced two access modes for the Producer: Exclusive: The producer is the only one who can publish to the topic. Fail if there is another Exclusive Producer connected to the topic while creating the new Producer. WaitForExclusive: Like Exclusive, but instead of Failing we are going to wait for the current Exclusive Producer to disconnect. Those two modes are very powerful and allow you to perform some kind of Locking on a topic. We are missing a third more, in which the Producer always succeeds to acquire the Exclusive lock on the topic by fencing out any other Producer that is connected, even the current Exclusive Producer and the other Producers waiting in WaitForExclusive mode. Goal The modes that are available with PIP-68 require a writer to acquire the lock and release it as soon as possible in order to allow other clients to write to the topic. With the new mode it will be possible to implement locking in another way: the Producer holds the lock until someone else steals it. This way when you have very low contention you can achieve better latency for writes because you don't have to acquire the lock every time you want to write,. API and Wire protocol Changes Changes: a new constant on the Wire Protocol for AccessMode a new constant in the Java Client API AccessMode#ExclusiveWithFencing Implementation The new mode will behave mostly like AccessMode#Exclusive but instead of failing in case of the presence of other Producers it will force all of the current connected Producers to be removed and invalidated (they will see ProducerFencedException). Reject Alternatives None
