Hi Dhruvil, Thanks for the KIP. I think there should be some way to turn this off, in case that becomes necessary. I'm also curious how long we intend to wait between detecting the duplication and deleting the extra logs. The KIP says "scheduled for deletion" but doesn't give a time frame -- is it assumed to be immediate?
best, Colin On Tue, Jan 14, 2020, at 05:56, Dhruvil Shah wrote: > If there are no more questions or concerns, I will start a vote thread > tomorrow. > > Thanks, > Dhruvil > > On Mon, Jan 13, 2020 at 6:59 PM Dhruvil Shah <dhru...@confluent.io> wrote: > > > Hi Nikhil, > > > > Thanks for looking at the KIP. The kind of race condition you mention is > > not possible as stray partition detection is done synchronously while > > handling the LeaderAndIsrRequest. In other words, we atomically evaluate > > the partitions the broker must host and the extra partitions it is hosting > > and schedule deletions based on that. > > > > One possible shortcoming of the KIP is that we do not have the ability to > > detect a stray partition if the topic has been recreated since. We will > > have the ability to disambiguate between different generations of a > > partition with KIP-516. > > > > Thanks, > > Dhruvil > > > > On Sat, Jan 11, 2020 at 11:40 AM Nikhil Bhatia <nik...@confluent.io> > > wrote: > > > >> Thanks Dhruvil, the proposal looks reasonable to me. > >> > >> is there a potential of a race between a new topic being assigned to the > >> same node that is still performing a cleanup of the stray partition ? > >> Topic > >> ID will definitely solve this issue. > >> > >> Thanks > >> Nikhil > >> > >> On 2020/01/06 04:30:20, Dhruvil Shah <d...@confluent.io> wrote: > >> > Here is the link to the KIP:> > >> > > >> > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-550%3A+Mechanism+to+Delete+Stray+Partitions+on+Broker > >> > > >> > >> > > >> > On Mon, Jan 6, 2020 at 9:59 AM Dhruvil Shah <dh...@confluent.io> > >> wrote:> > >> > > >> > > Hi all, I would like to kick off discussion for KIP-550 which proposes > >> a> > >> > > mechanism to detect and delete stray partitions on a broker. > >> Suggestions> > >> > > and feedback are welcome.> > >> > >> > >> > > - Dhruvil> > >> > >> > >> > > >> > > >
