Hi Jun and everyone, I would like to change the KIP in the following way. Currently, if any replica if offline, the purge result for a partition will be NotEnoughReplicasException and its low_watermark will be 0. The motivation for this approach is that we want to guarantee that the data before purgedOffset has been deleted on all replicas of this partition if purge result indicates success.
But this approach seems too conservative. It should be sufficient in most cases to just tell user success and set low_watermark to minimum logStartOffset of all live replicas in the PurgeResponse if logStartOffset of all live replicas have reached purgedOffset. This is because for an offline replicas to become online and be elected leader, it should have received one FetchReponse from the current leader which should tell it to purge beyond purgedOffset. The benefit of doing this change is that we can allow purge operation to succeed when some replica is offline. Are you OK with this change? If so, I will go ahead to update the KIP and implement this behavior. Thanks, Dong On Tue, Jan 17, 2017 at 10:18 AM, Dong Lin <lindon...@gmail.com> wrote: > Hey Jun, > > Do you have time to review the KIP again or vote for it? > > Hey Ewen, > > Can you also review the KIP again or vote for it? I have discussed with > Radai and Becket regarding your concern. We still think putting it in Admin > Client seems more intuitive because there is use-case where application > which manages topic or produces data may also want to purge data. It seems > weird if they need to create a consumer to do this. > > Thanks, > Dong > > On Thu, Jan 12, 2017 at 9:34 AM, Mayuresh Gharat < > gharatmayures...@gmail.com> wrote: > >> +1 (non-binding) >> >> Thanks, >> >> Mayuresh >> >> On Wed, Jan 11, 2017 at 1:03 PM, Dong Lin <lindon...@gmail.com> wrote: >> >> > Sorry for the duplicated email. It seems that gmail will put the voting >> > email in this thread if I simply replace DISCUSS with VOTE in the >> subject. >> > >> > On Wed, Jan 11, 2017 at 12:57 PM, Dong Lin <lindon...@gmail.com> wrote: >> > >> > > Hi all, >> > > >> > > It seems that there is no further concern with the KIP-107. At this >> point >> > > we would like to start the voting process. The KIP can be found at >> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-107 >> > > %3A+Add+purgeDataBefore%28%29+API+in+AdminClient. >> > > >> > > Thanks, >> > > Dong >> > > >> > >> >> >> >> -- >> -Regards, >> Mayuresh R. Gharat >> (862) 250-7125 >> > >
