Hi all, Quick update: I've filed a separate ticket, https://issues.apache.org/jira/browse/KAFKA-15425, to track the behavior change in Admin::listOffsets. For the full history of the ticket, it's worth reading the comment thread on the old ticket at https://issues.apache.org/jira/browse/KAFKA-12879.
I've also published https://github.com/apache/kafka/pull/14314 as a fairly lightweight PR to revert the behavior of Admin::listOffsets without also reverting the refactoring to use the internal admin driver API. Would appreciate a review on that if anyone can spare the cycles. Cheers, Chris On Wed, Aug 30, 2023 at 1:01 PM Chris Egerton <chr...@aiven.io> wrote: > Hi Satish, > > Wanted to let you know that KAFKA-12879 ( > https://issues.apache.org/jira/browse/KAFKA-12879), a breaking change in > Admin::listOffsets, has been reintroduced into the code base. Since we > haven't yet published a release with this change (at least, not the more > recent instance of it), I was hoping we could treat it as a blocker for > 3.6.0. I'd also like to solicit the input of people familiar with the admin > client to weigh in on the Jira ticket about whether we should continue to > preserve the current behavior (if the consensus is that we should, I'm > happy to file a fix). > > Please let me know if you agree that this qualifies as a blocker. I plan > on publishing a potential fix sometime this week. > > Cheers, > > Chris > > On Wed, Aug 30, 2023 at 9:19 AM Satish Duggana <satish.dugg...@gmail.com> > wrote: > >> Hi, >> Please plan to continue merging pull requests associated with any >> outstanding minor features and stabilization changes to 3.6 branch >> before September 3rd. Kindly update the KIP's implementation status in >> the 3.6.0 release notes. >> >> Thanks, >> Satish. >> >> On Fri, 25 Aug 2023 at 21:37, Justine Olshan >> <jols...@confluent.io.invalid> wrote: >> > >> > Hey Satish, >> > Everything should be in 3.6, and I will update the release plan wiki. >> > Thanks! >> > >> > On Fri, Aug 25, 2023 at 4:08 AM Satish Duggana < >> satish.dugg...@gmail.com> >> > wrote: >> > >> > > Hi Justine, >> > > Adding KIP-890 part-1 to 3.6.0 seems reasonable to me. This part looks >> > > to be addressing a critical issue of consumers getting stuck. Please >> > > update the release plan wiki and merge all the required changes to 3.6 >> > > branch. >> > > >> > > Thanks, >> > > Satish. >> > > >> > > On Thu, 24 Aug 2023 at 22:19, Justine Olshan >> > > <jols...@confluent.io.invalid> wrote: >> > > > >> > > > Hey Satish, >> > > > Does it make sense to include KIP-890 part 1? It prevents hanging >> > > > transactions for older clients. (An optimization and stronger EOS >> > > > guarantees will be included in part 2) >> > > > >> > > > Thanks, >> > > > Justine >> > > > >> > > > On Mon, Aug 21, 2023 at 3:29 AM Satish Duggana < >> satish.dugg...@gmail.com >> > > > >> > > > wrote: >> > > > >> > > > > Hi, >> > > > > 3.6 branch is created. Please make sure any PRs targeted for 3.6.0 >> > > > > should be merged to 3.6 branch once those are merged to trunk. >> > > > > >> > > > > Thanks, >> > > > > Satish. >> > > > > >> > > > > On Wed, 16 Aug 2023 at 15:58, Satish Duggana < >> satish.dugg...@gmail.com >> > > > >> > > > > wrote: >> > > > > > >> > > > > > Hi, >> > > > > > Please plan to merge PRs(including the major features) targeted >> for >> > > > > > 3.6.0 by the end of Aug 20th UTC. Starting from August 21st, >> any pull >> > > > > > requests intended for the 3.6.0 release must include the changes >> > > > > > merged into the 3.6 branch as mentioned in the release plan. >> > > > > > >> > > > > > Thanks, >> > > > > > Satish. >> > > > > > >> > > > > > On Fri, 4 Aug 2023 at 18:39, Chris Egerton >> <chr...@aiven.io.invalid> >> > > > > wrote: >> > > > > > > >> > > > > > > Thanks for adding KIP-949, Satish! >> > > > > > > >> > > > > > > On Fri, Aug 4, 2023 at 7:06 AM Satish Duggana < >> > > > > satish.dugg...@gmail.com> >> > > > > > > wrote: >> > > > > > > >> > > > > > > > Hi, >> > > > > > > > Myself and Divij discussed and added the wiki for Kafka >> > > TieredStorage >> > > > > > > > Early Access Release[1]. If you have any comments or >> feedback, >> > > please >> > > > > > > > feel free to share them. >> > > > > > > > >> > > > > > > > 1. >> > > > > > > > >> > > > > >> > > >> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Tiered+Storage+Early+Access+Release+Notes >> > > > > > > > >> > > > > > > > Thanks, >> > > > > > > > Satish. >> > > > > > > > >> > > > > > > > On Fri, 4 Aug 2023 at 08:40, Satish Duggana < >> > > > > satish.dugg...@gmail.com> >> > > > > > > > wrote: >> > > > > > > > > >> > > > > > > > > Hi Chris, >> > > > > > > > > Thanks for the update. This looks to be a minor change >> and is >> > > also >> > > > > > > > > useful for backward compatibility. I added it to the >> release >> > > plan >> > > > > as >> > > > > > > > > an exceptional case. >> > > > > > > > > >> > > > > > > > > ~Satish. >> > > > > > > > > >> > > > > > > > > On Thu, 3 Aug 2023 at 21:34, Chris Egerton >> > > <chr...@aiven.io.invalid >> > > > > > >> > > > > > > > wrote: >> > > > > > > > > > >> > > > > > > > > > Hi Satish, >> > > > > > > > > > >> > > > > > > > > > Would it be possible to include KIP-949 ( >> > > > > > > > > > >> > > > > > > > >> > > > > >> > > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-949%3A+Add+flag+to+enable+the+usage+of+topic+separator+in+MM2+DefaultReplicationPolicy >> > > > > > > > ) >> > > > > > > > > > in the 3.6.0 release? It passed voting yesterday, and >> is a >> > > very >> > > > > small, >> > > > > > > > > > low-risk change that we'd like to put out as soon as >> > > possible in >> > > > > order >> > > > > > > > to >> > > > > > > > > > patch an accidental break in backwards compatibility >> caused >> > > a few >> > > > > > > > versions >> > > > > > > > > > ago. >> > > > > > > > > > >> > > > > > > > > > Best, >> > > > > > > > > > >> > > > > > > > > > Chris >> > > > > > > > > > >> > > > > > > > > > On Fri, Jul 28, 2023 at 2:35 AM Satish Duggana < >> > > > > > > > satish.dugg...@gmail.com> >> > > > > > > > > > wrote: >> > > > > > > > > > >> > > > > > > > > > > Hi All, >> > > > > > > > > > > Whoever has KIP entries in the 3.6.0 release plan. >> Please >> > > > > update it >> > > > > > > > > > > with the latest status by tomorrow(end of the day >> 29th Jul >> > > UTC >> > > > > ). >> > > > > > > > > > > >> > > > > > > > > > > Thanks >> > > > > > > > > > > Satish. >> > > > > > > > > > > >> > > > > > > > > > > On Fri, 28 Jul 2023 at 12:01, Satish Duggana < >> > > > > > > > satish.dugg...@gmail.com> >> > > > > > > > > > > wrote: >> > > > > > > > > > > > >> > > > > > > > > > > > Thanks Ismael and Divij for the suggestions. >> > > > > > > > > > > > >> > > > > > > > > > > > One way was to follow the earlier guidelines that >> we set >> > > for >> > > > > any >> > > > > > > > early >> > > > > > > > > > > > access release. It looks Ismael already mentioned >> the >> > > > > example of >> > > > > > > > > > > > KRaft. >> > > > > > > > > > > > >> > > > > > > > > > > > KIP-405 mentions upgrade/downgrade and limitations >> > > sections. >> > > > > We can >> > > > > > > > > > > > clarify that in the release notes for users on how >> this >> > > > > feature >> > > > > > > > can be >> > > > > > > > > > > > used for early access. >> > > > > > > > > > > > >> > > > > > > > > > > > Divij, We do not want users to enable this feature >> on >> > > > > production >> > > > > > > > > > > > environments in early access release. Let us work >> > > together >> > > > > on the >> > > > > > > > > > > > followups Ismael suggested. >> > > > > > > > > > > > >> > > > > > > > > > > > ~Satish. >> > > > > > > > > > > > >> > > > > > > > > > > > On Fri, 28 Jul 2023 at 02:24, Divij Vaidya < >> > > > > > > > divijvaidy...@gmail.com> >> > > > > > > > > > > wrote: >> > > > > > > > > > > > > >> > > > > > > > > > > > > Those are great suggestions, thank you. We will >> > > continue >> > > > > this >> > > > > > > > > > > discussion >> > > > > > > > > > > > > forward in a separate KIP for release plan for >> Tiered >> > > > > Storage. >> > > > > > > > > > > > > >> > > > > > > > > > > > > On Thu 27. Jul 2023 at 21:46, Ismael Juma < >> > > > > m...@ismaeljuma.com> >> > > > > > > > wrote: >> > > > > > > > > > > > > >> > > > > > > > > > > > > > Hi Divij, >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > I think the points you bring up for discussion >> are >> > > all >> > > > > good. >> > > > > > > > My main >> > > > > > > > > > > > > > feedback is that they should be discussed in the >> > > context >> > > > > of >> > > > > > > > KIPs vs >> > > > > > > > > > > the >> > > > > > > > > > > > > > release template. That's why we have a backwards >> > > > > compatibility >> > > > > > > > > > > section for >> > > > > > > > > > > > > > every KIP, it's precisely to ensure we think >> > > carefully >> > > > > about >> > > > > > > > some of >> > > > > > > > > > > the >> > > > > > > > > > > > > > points you're bringing up. When it comes to >> defining >> > > the >> > > > > > > > meaning of >> > > > > > > > > > > early >> > > > > > > > > > > > > > access, we have two options: >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > 1. Have a KIP specifically for tiered storage. >> > > > > > > > > > > > > > 2. Have a KIP to define general guidelines for >> what >> > > early >> > > > > > > > access >> > > > > > > > > > > means. >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > Does this make sense? >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > Ismael >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > On Thu, Jul 27, 2023 at 6:38 PM Divij Vaidya < >> > > > > > > > > > > divijvaidy...@gmail.com> >> > > > > > > > > > > > > > wrote: >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > > Thank you for the response, Ismael. >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > 1. Specifically in context of 3.6, I wanted >> this >> > > > > > > > compatibility >> > > > > > > > > > > > > > > guarantee point to encourage a discussion on >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > >> > > > > >> > > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-952%3A+Regenerate+segment-aligned+producer+snapshots+when+upgrading+to+a+Kafka+version+supporting+Tiered+Storage >> > > > > > > > > > > > > > > . >> > > > > > > > > > > > > > > Due to lack of producer snapshots in <2.8 >> > > versions, a >> > > > > > > > customer may >> > > > > > > > > > > not >> > > > > > > > > > > > > > > be able to upgrade to 3.6 and use TS on a >> topic >> > > which >> > > > > was >> > > > > > > > created >> > > > > > > > > > > when >> > > > > > > > > > > > > > > the cluster was on <2.8 version (see >> motivation for >> > > > > > > > details). We >> > > > > > > > > > > can >> > > > > > > > > > > > > > > discuss and agree that it does not break >> > > compatibility, >> > > > > > > > which is >> > > > > > > > > > > fine. >> > > > > > > > > > > > > > > But I want to ensure that we have a discussion >> > > soon on >> > > > > this >> > > > > > > > to >> > > > > > > > > > > reach a >> > > > > > > > > > > > > > > conclusion. >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > 2. I will start a KIP on this for further >> > > discussion. >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > 3. In the context of 3.6, this would mean that >> > > there >> > > > > should >> > > > > > > > be >> > > > > > > > > > > > > > > no-regression, if a user does "not" turn-on >> remote >> > > > > storage >> > > > > > > > (early >> > > > > > > > > > > > > > > access feature) at a cluster level. We have >> some >> > > known >> > > > > cases >> > > > > > > > (such >> > > > > > > > > > > as >> > > > > > > > > > > > > > > >> https://issues.apache.org/jira/browse/KAFKA-15189) >> > > > > which >> > > > > > > > violate >> > > > > > > > > > > this >> > > > > > > > > > > > > > > compatibility requirement. Having this >> guarantee >> > > > > mentioned >> > > > > > > > in the >> > > > > > > > > > > > > > > release plan will ensure that we are all in >> > > agreement >> > > > > with >> > > > > > > > which >> > > > > > > > > > > cases >> > > > > > > > > > > > > > > are truly blockers and which aren't. >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > 4. Fair, instead of a general goal, let me >> put it >> > > > > > > > specifically in >> > > > > > > > > > > the >> > > > > > > > > > > > > > > context of 3.6. Let me know if this is not the >> > > right >> > > > > forum >> > > > > > > > for this >> > > > > > > > > > > > > > > discussion. >> > > > > > > > > > > > > > > Once a user "turns on" tiered storage (TS) at >> a >> > > cluster >> > > > > > > > level, I am >> > > > > > > > > > > > > > > proposing that they should have the ability to >> > > turn it >> > > > > off >> > > > > > > > as well >> > > > > > > > > > > at >> > > > > > > > > > > > > > > a cluster level. Since this is a topic level >> > > feature, >> > > > > folks >> > > > > > > > may not >> > > > > > > > > > > > > > > spin up a separate cluster to try this >> feature, >> > > hence, >> > > > > we >> > > > > > > > need to >> > > > > > > > > > > > > > > ensure that we provide them with the ability >> to try >> > > > > tiered >> > > > > > > > storage >> > > > > > > > > > > for >> > > > > > > > > > > > > > > a topic which could be deleted and featured >> > > > > turned-off, so >> > > > > > > > that >> > > > > > > > > > > rest >> > > > > > > > > > > > > > > of the production cases are not impacted. >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > 5. Agree on not making public interface >> change as a >> > > > > > > > requirement >> > > > > > > > > > > but we >> > > > > > > > > > > > > > > should define what "early access" means in >> that >> > > case. >> > > > > Users >> > > > > > > > may >> > > > > > > > > > > not be >> > > > > > > > > > > > > > > aware that "early access" public APIs may >> change >> > > > > (unless I am >> > > > > > > > > > > missing >> > > > > > > > > > > > > > > some documentation somewhere completely, in >> which >> > > case >> > > > > I >> > > > > > > > apologize >> > > > > > > > > > > for >> > > > > > > > > > > > > > > bringing this naive point). >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > -- >> > > > > > > > > > > > > > > Divij Vaidya >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > On Thu, Jul 27, 2023 at 2:27 PM Ismael Juma < >> > > > > > > > m...@ismaeljuma.com> >> > > > > > > > > > > wrote: >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > Hi Divij, >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > Some of these are launch checklist items >> (not >> > > really >> > > > > > > > goals) and >> > > > > > > > > > > some >> > > > > > > > > > > > > > are >> > > > > > > > > > > > > > > > compatibility guarantees. More below. >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > On Thu, Jul 27, 2023, 12:10 PM Divij Vaidya >> < >> > > > > > > > > > > divijvaidy...@gmail.com> >> > > > > > > > > > > > > > > wrote: >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > Hey Satish >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > Could we consider adding "launch goals" >> in the >> > > > > release >> > > > > > > > plan. >> > > > > > > > > > > While >> > > > > > > > > > > > > > > > > some of these may be implicit, it would be >> > > nice to >> > > > > list >> > > > > > > > them >> > > > > > > > > > > down in >> > > > > > > > > > > > > > > > > the release plan. For this release, our >> launch >> > > > > > > > requirements >> > > > > > > > > > > would be: >> > > > > > > > > > > > > > > > > 1. Users should be able to upgrade from >> any >> > > prior >> > > > > Kafka >> > > > > > > > > > > version to >> > > > > > > > > > > > > > this >> > > > > > > > > > > > > > > > > version. >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > This is part of the compatibility >> guarantees. The >> > > > > upgrade >> > > > > > > > notes >> > > > > > > > > > > mention >> > > > > > > > > > > > > > > > this already. If there is a change in a >> given >> > > > > release, it >> > > > > > > > should >> > > > > > > > > > > > > > > definitely >> > > > > > > > > > > > > > > > be highlighted. >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > 2. On release, this version (or it's >> > > dependencies) >> > > > > would >> > > > > > > > not >> > > > > > > > > > > have any >> > > > > > > > > > > > > > > > > known MEDIUM/HIGH CVE. >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > This is a new policy and the details should >> be >> > > > > discussed. >> > > > > > > > In >> > > > > > > > > > > > > > particular, >> > > > > > > > > > > > > > > > the threshold (medium or high). >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > 3. Presence of any "early access"/"beta" >> feature >> > > > > should not >> > > > > > > > > > > impact >> > > > > > > > > > > > > > > > > other production features when it is not >> > > enabled. >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > This is a general guideline for early access >> > > > > features and >> > > > > > > > not >> > > > > > > > > > > specific >> > > > > > > > > > > > > > to >> > > > > > > > > > > > > > > > this release. It would be good to have a >> page >> > > that >> > > > > talks >> > > > > > > > about >> > > > > > > > > > > these >> > > > > > > > > > > > > > > things. >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > 4. Once enabled, users should have an >> option to >> > > > > disable any >> > > > > > > > > > > "early >> > > > > > > > > > > > > > > > > access"/"beta" feature and resume normal >> > > production >> > > > > > > > features, >> > > > > > > > > > > i.e. >> > > > > > > > > > > > > > > > > impact of beta features should be >> reversible. >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > This needs discussion and I don't think it's >> > > > > reasonable as >> > > > > > > > a >> > > > > > > > > > > general >> > > > > > > > > > > > > > > rule. >> > > > > > > > > > > > > > > > For example, Kraft early access wasn't >> reversible >> > > > > and it >> > > > > > > > was not >> > > > > > > > > > > > > > feasible >> > > > > > > > > > > > > > > > for it to be. >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > 5. KIP-405 will be available in "early >> > > access"/"beta" >> > > > > > > > mode. Early >> > > > > > > > > > > > > > > > > access/beta means that the public facing >> > > > > interfaces won't >> > > > > > > > > > > change in >> > > > > > > > > > > > > > > > > future but the implementation is not >> > > recommended >> > > > > to be >> > > > > > > > used in >> > > > > > > > > > > > > > > > > production. >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > I don't think it's ok to make this a >> requirement. >> > > > > Early >> > > > > > > > access >> > > > > > > > > > > is a way >> > > > > > > > > > > > > > > to >> > > > > > > > > > > > > > > > get early feedback and all types of changes >> > > should >> > > > > be on >> > > > > > > > the >> > > > > > > > > > > table. >> > > > > > > > > > > > > > They >> > > > > > > > > > > > > > > > would be discussed via KIPs as usual. I >> believe >> > > > > there were >> > > > > > > > some >> > > > > > > > > > > > > > > > incompatible changes for Kraft during the >> early >> > > > > access >> > > > > > > > period >> > > > > > > > > > > although >> > > > > > > > > > > > > > > the >> > > > > > > > > > > > > > > > team aimed to minimize work required during >> > > > > upgrades. I >> > > > > > > > have >> > > > > > > > > > > mentioned >> > > > > > > > > > > > > > > > Kraft a couple of times since it's a good >> > > example of >> > > > > a >> > > > > > > > large >> > > > > > > > > > > feature >> > > > > > > > > > > > > > that >> > > > > > > > > > > > > > > > went through this process. >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > Ismael >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > -- >> > > > > > > > > > > > > Divij Vaidya >> > > > > > > > > > > >> > > > > > > > >> > > > > >> > > >> >
