Hi, Please plan to continue merging pull requests associated with any outstanding minor features and stabilization changes to 3.6 branch before September 3rd. Kindly update the KIP's implementation status in the 3.6.0 release notes.
Thanks, Satish. On Fri, 25 Aug 2023 at 21:37, Justine Olshan <jols...@confluent.io.invalid> wrote: > > Hey Satish, > Everything should be in 3.6, and I will update the release plan wiki. > Thanks! > > On Fri, Aug 25, 2023 at 4:08 AM Satish Duggana <satish.dugg...@gmail.com> > wrote: > > > Hi Justine, > > Adding KIP-890 part-1 to 3.6.0 seems reasonable to me. This part looks > > to be addressing a critical issue of consumers getting stuck. Please > > update the release plan wiki and merge all the required changes to 3.6 > > branch. > > > > Thanks, > > Satish. > > > > On Thu, 24 Aug 2023 at 22:19, Justine Olshan > > <jols...@confluent.io.invalid> wrote: > > > > > > Hey Satish, > > > Does it make sense to include KIP-890 part 1? It prevents hanging > > > transactions for older clients. (An optimization and stronger EOS > > > guarantees will be included in part 2) > > > > > > Thanks, > > > Justine > > > > > > On Mon, Aug 21, 2023 at 3:29 AM Satish Duggana <satish.dugg...@gmail.com > > > > > > wrote: > > > > > > > Hi, > > > > 3.6 branch is created. Please make sure any PRs targeted for 3.6.0 > > > > should be merged to 3.6 branch once those are merged to trunk. > > > > > > > > Thanks, > > > > Satish. > > > > > > > > On Wed, 16 Aug 2023 at 15:58, Satish Duggana <satish.dugg...@gmail.com > > > > > > > wrote: > > > > > > > > > > Hi, > > > > > Please plan to merge PRs(including the major features) targeted for > > > > > 3.6.0 by the end of Aug 20th UTC. Starting from August 21st, any pull > > > > > requests intended for the 3.6.0 release must include the changes > > > > > merged into the 3.6 branch as mentioned in the release plan. > > > > > > > > > > Thanks, > > > > > Satish. > > > > > > > > > > On Fri, 4 Aug 2023 at 18:39, Chris Egerton <chr...@aiven.io.invalid> > > > > wrote: > > > > > > > > > > > > Thanks for adding KIP-949, Satish! > > > > > > > > > > > > On Fri, Aug 4, 2023 at 7:06 AM Satish Duggana < > > > > satish.dugg...@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > Hi, > > > > > > > Myself and Divij discussed and added the wiki for Kafka > > TieredStorage > > > > > > > Early Access Release[1]. If you have any comments or feedback, > > please > > > > > > > feel free to share them. > > > > > > > > > > > > > > 1. > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Tiered+Storage+Early+Access+Release+Notes > > > > > > > > > > > > > > Thanks, > > > > > > > Satish. > > > > > > > > > > > > > > On Fri, 4 Aug 2023 at 08:40, Satish Duggana < > > > > satish.dugg...@gmail.com> > > > > > > > wrote: > > > > > > > > > > > > > > > > Hi Chris, > > > > > > > > Thanks for the update. This looks to be a minor change and is > > also > > > > > > > > useful for backward compatibility. I added it to the release > > plan > > > > as > > > > > > > > an exceptional case. > > > > > > > > > > > > > > > > ~Satish. > > > > > > > > > > > > > > > > On Thu, 3 Aug 2023 at 21:34, Chris Egerton > > <chr...@aiven.io.invalid > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > Hi Satish, > > > > > > > > > > > > > > > > > > Would it be possible to include KIP-949 ( > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-949%3A+Add+flag+to+enable+the+usage+of+topic+separator+in+MM2+DefaultReplicationPolicy > > > > > > > ) > > > > > > > > > in the 3.6.0 release? It passed voting yesterday, and is a > > very > > > > small, > > > > > > > > > low-risk change that we'd like to put out as soon as > > possible in > > > > order > > > > > > > to > > > > > > > > > patch an accidental break in backwards compatibility caused > > a few > > > > > > > versions > > > > > > > > > ago. > > > > > > > > > > > > > > > > > > Best, > > > > > > > > > > > > > > > > > > Chris > > > > > > > > > > > > > > > > > > On Fri, Jul 28, 2023 at 2:35 AM Satish Duggana < > > > > > > > satish.dugg...@gmail.com> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Hi All, > > > > > > > > > > Whoever has KIP entries in the 3.6.0 release plan. Please > > > > update it > > > > > > > > > > with the latest status by tomorrow(end of the day 29th Jul > > UTC > > > > ). > > > > > > > > > > > > > > > > > > > > Thanks > > > > > > > > > > Satish. > > > > > > > > > > > > > > > > > > > > On Fri, 28 Jul 2023 at 12:01, Satish Duggana < > > > > > > > satish.dugg...@gmail.com> > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > Thanks Ismael and Divij for the suggestions. > > > > > > > > > > > > > > > > > > > > > > One way was to follow the earlier guidelines that we set > > for > > > > any > > > > > > > early > > > > > > > > > > > access release. It looks Ismael already mentioned the > > > > example of > > > > > > > > > > > KRaft. > > > > > > > > > > > > > > > > > > > > > > KIP-405 mentions upgrade/downgrade and limitations > > sections. > > > > We can > > > > > > > > > > > clarify that in the release notes for users on how this > > > > feature > > > > > > > can be > > > > > > > > > > > used for early access. > > > > > > > > > > > > > > > > > > > > > > Divij, We do not want users to enable this feature on > > > > production > > > > > > > > > > > environments in early access release. Let us work > > together > > > > on the > > > > > > > > > > > followups Ismael suggested. > > > > > > > > > > > > > > > > > > > > > > ~Satish. > > > > > > > > > > > > > > > > > > > > > > On Fri, 28 Jul 2023 at 02:24, Divij Vaidya < > > > > > > > divijvaidy...@gmail.com> > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > Those are great suggestions, thank you. We will > > continue > > > > this > > > > > > > > > > discussion > > > > > > > > > > > > forward in a separate KIP for release plan for Tiered > > > > Storage. > > > > > > > > > > > > > > > > > > > > > > > > On Thu 27. Jul 2023 at 21:46, Ismael Juma < > > > > m...@ismaeljuma.com> > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > Hi Divij, > > > > > > > > > > > > > > > > > > > > > > > > > > I think the points you bring up for discussion are > > all > > > > good. > > > > > > > My main > > > > > > > > > > > > > feedback is that they should be discussed in the > > context > > > > of > > > > > > > KIPs vs > > > > > > > > > > the > > > > > > > > > > > > > release template. That's why we have a backwards > > > > compatibility > > > > > > > > > > section for > > > > > > > > > > > > > every KIP, it's precisely to ensure we think > > carefully > > > > about > > > > > > > some of > > > > > > > > > > the > > > > > > > > > > > > > points you're bringing up. When it comes to defining > > the > > > > > > > meaning of > > > > > > > > > > early > > > > > > > > > > > > > access, we have two options: > > > > > > > > > > > > > > > > > > > > > > > > > > 1. Have a KIP specifically for tiered storage. > > > > > > > > > > > > > 2. Have a KIP to define general guidelines for what > > early > > > > > > > access > > > > > > > > > > means. > > > > > > > > > > > > > > > > > > > > > > > > > > Does this make sense? > > > > > > > > > > > > > > > > > > > > > > > > > > Ismael > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Jul 27, 2023 at 6:38 PM Divij Vaidya < > > > > > > > > > > divijvaidy...@gmail.com> > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > Thank you for the response, Ismael. > > > > > > > > > > > > > > > > > > > > > > > > > > > > 1. Specifically in context of 3.6, I wanted this > > > > > > > compatibility > > > > > > > > > > > > > > guarantee point to encourage a discussion on > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-952%3A+Regenerate+segment-aligned+producer+snapshots+when+upgrading+to+a+Kafka+version+supporting+Tiered+Storage > > > > > > > > > > > > > > . > > > > > > > > > > > > > > Due to lack of producer snapshots in <2.8 > > versions, a > > > > > > > customer may > > > > > > > > > > not > > > > > > > > > > > > > > be able to upgrade to 3.6 and use TS on a topic > > which > > > > was > > > > > > > created > > > > > > > > > > when > > > > > > > > > > > > > > the cluster was on <2.8 version (see motivation for > > > > > > > details). We > > > > > > > > > > can > > > > > > > > > > > > > > discuss and agree that it does not break > > compatibility, > > > > > > > which is > > > > > > > > > > fine. > > > > > > > > > > > > > > But I want to ensure that we have a discussion > > soon on > > > > this > > > > > > > to > > > > > > > > > > reach a > > > > > > > > > > > > > > conclusion. > > > > > > > > > > > > > > > > > > > > > > > > > > > > 2. I will start a KIP on this for further > > discussion. > > > > > > > > > > > > > > > > > > > > > > > > > > > > 3. In the context of 3.6, this would mean that > > there > > > > should > > > > > > > be > > > > > > > > > > > > > > no-regression, if a user does "not" turn-on remote > > > > storage > > > > > > > (early > > > > > > > > > > > > > > access feature) at a cluster level. We have some > > known > > > > cases > > > > > > > (such > > > > > > > > > > as > > > > > > > > > > > > > > https://issues.apache.org/jira/browse/KAFKA-15189) > > > > which > > > > > > > violate > > > > > > > > > > this > > > > > > > > > > > > > > compatibility requirement. Having this guarantee > > > > mentioned > > > > > > > in the > > > > > > > > > > > > > > release plan will ensure that we are all in > > agreement > > > > with > > > > > > > which > > > > > > > > > > cases > > > > > > > > > > > > > > are truly blockers and which aren't. > > > > > > > > > > > > > > > > > > > > > > > > > > > > 4. Fair, instead of a general goal, let me put it > > > > > > > specifically in > > > > > > > > > > the > > > > > > > > > > > > > > context of 3.6. Let me know if this is not the > > right > > > > forum > > > > > > > for this > > > > > > > > > > > > > > discussion. > > > > > > > > > > > > > > Once a user "turns on" tiered storage (TS) at a > > cluster > > > > > > > level, I am > > > > > > > > > > > > > > proposing that they should have the ability to > > turn it > > > > off > > > > > > > as well > > > > > > > > > > at > > > > > > > > > > > > > > a cluster level. Since this is a topic level > > feature, > > > > folks > > > > > > > may not > > > > > > > > > > > > > > spin up a separate cluster to try this feature, > > hence, > > > > we > > > > > > > need to > > > > > > > > > > > > > > ensure that we provide them with the ability to try > > > > tiered > > > > > > > storage > > > > > > > > > > for > > > > > > > > > > > > > > a topic which could be deleted and featured > > > > turned-off, so > > > > > > > that > > > > > > > > > > rest > > > > > > > > > > > > > > of the production cases are not impacted. > > > > > > > > > > > > > > > > > > > > > > > > > > > > 5. Agree on not making public interface change as a > > > > > > > requirement > > > > > > > > > > but we > > > > > > > > > > > > > > should define what "early access" means in that > > case. > > > > Users > > > > > > > may > > > > > > > > > > not be > > > > > > > > > > > > > > aware that "early access" public APIs may change > > > > (unless I am > > > > > > > > > > missing > > > > > > > > > > > > > > some documentation somewhere completely, in which > > case > > > > I > > > > > > > apologize > > > > > > > > > > for > > > > > > > > > > > > > > bringing this naive point). > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > Divij Vaidya > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Jul 27, 2023 at 2:27 PM Ismael Juma < > > > > > > > m...@ismaeljuma.com> > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Divij, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Some of these are launch checklist items (not > > really > > > > > > > goals) and > > > > > > > > > > some > > > > > > > > > > > > > are > > > > > > > > > > > > > > > compatibility guarantees. More below. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Jul 27, 2023, 12:10 PM Divij Vaidya < > > > > > > > > > > divijvaidy...@gmail.com> > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hey Satish > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Could we consider adding "launch goals" in the > > > > release > > > > > > > plan. > > > > > > > > > > While > > > > > > > > > > > > > > > > some of these may be implicit, it would be > > nice to > > > > list > > > > > > > them > > > > > > > > > > down in > > > > > > > > > > > > > > > > the release plan. For this release, our launch > > > > > > > requirements > > > > > > > > > > would be: > > > > > > > > > > > > > > > > 1. Users should be able to upgrade from any > > prior > > > > Kafka > > > > > > > > > > version to > > > > > > > > > > > > > this > > > > > > > > > > > > > > > > version. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This is part of the compatibility guarantees. The > > > > upgrade > > > > > > > notes > > > > > > > > > > mention > > > > > > > > > > > > > > > this already. If there is a change in a given > > > > release, it > > > > > > > should > > > > > > > > > > > > > > definitely > > > > > > > > > > > > > > > be highlighted. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 2. On release, this version (or it's > > dependencies) > > > > would > > > > > > > not > > > > > > > > > > have any > > > > > > > > > > > > > > > > known MEDIUM/HIGH CVE. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This is a new policy and the details should be > > > > discussed. > > > > > > > In > > > > > > > > > > > > > particular, > > > > > > > > > > > > > > > the threshold (medium or high). > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 3. Presence of any "early access"/"beta" feature > > > > should not > > > > > > > > > > impact > > > > > > > > > > > > > > > > other production features when it is not > > enabled. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This is a general guideline for early access > > > > features and > > > > > > > not > > > > > > > > > > specific > > > > > > > > > > > > > to > > > > > > > > > > > > > > > this release. It would be good to have a page > > that > > > > talks > > > > > > > about > > > > > > > > > > these > > > > > > > > > > > > > > things. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 4. Once enabled, users should have an option to > > > > disable any > > > > > > > > > > "early > > > > > > > > > > > > > > > > access"/"beta" feature and resume normal > > production > > > > > > > features, > > > > > > > > > > i.e. > > > > > > > > > > > > > > > > impact of beta features should be reversible. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This needs discussion and I don't think it's > > > > reasonable as > > > > > > > a > > > > > > > > > > general > > > > > > > > > > > > > > rule. > > > > > > > > > > > > > > > For example, Kraft early access wasn't reversible > > > > and it > > > > > > > was not > > > > > > > > > > > > > feasible > > > > > > > > > > > > > > > for it to be. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 5. KIP-405 will be available in "early > > access"/"beta" > > > > > > > mode. Early > > > > > > > > > > > > > > > > access/beta means that the public facing > > > > interfaces won't > > > > > > > > > > change in > > > > > > > > > > > > > > > > future but the implementation is not > > recommended > > > > to be > > > > > > > used in > > > > > > > > > > > > > > > > production. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I don't think it's ok to make this a requirement. > > > > Early > > > > > > > access > > > > > > > > > > is a way > > > > > > > > > > > > > > to > > > > > > > > > > > > > > > get early feedback and all types of changes > > should > > > > be on > > > > > > > the > > > > > > > > > > table. > > > > > > > > > > > > > They > > > > > > > > > > > > > > > would be discussed via KIPs as usual. I believe > > > > there were > > > > > > > some > > > > > > > > > > > > > > > incompatible changes for Kraft during the early > > > > access > > > > > > > period > > > > > > > > > > although > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > team aimed to minimize work required during > > > > upgrades. I > > > > > > > have > > > > > > > > > > mentioned > > > > > > > > > > > > > > > Kraft a couple of times since it's a good > > example of > > > > a > > > > > > > large > > > > > > > > > > feature > > > > > > > > > > > > > that > > > > > > > > > > > > > > > went through this process. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Ismael > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > Divij Vaidya > > > > > > > > > > > > > > > > > > > > > > >
