Thank you for the response, Ismael. 1. Specifically in context of 3.6, I wanted this compatibility guarantee point to encourage a discussion on https://cwiki.apache.org/confluence/display/KAFKA/KIP-952%3A+Regenerate+segment-aligned+producer+snapshots+when+upgrading+to+a+Kafka+version+supporting+Tiered+Storage. Due to lack of producer snapshots in <2.8 versions, a customer may not be able to upgrade to 3.6 and use TS on a topic which was created when the cluster was on <2.8 version (see motivation for details). We can discuss and agree that it does not break compatibility, which is fine. But I want to ensure that we have a discussion soon on this to reach a conclusion.
2. I will start a KIP on this for further discussion. 3. In the context of 3.6, this would mean that there should be no-regression, if a user does "not" turn-on remote storage (early access feature) at a cluster level. We have some known cases (such as https://issues.apache.org/jira/browse/KAFKA-15189) which violate this compatibility requirement. Having this guarantee mentioned in the release plan will ensure that we are all in agreement with which cases are truly blockers and which aren't. 4. Fair, instead of a general goal, let me put it specifically in the context of 3.6. Let me know if this is not the right forum for this discussion. Once a user "turns on" tiered storage (TS) at a cluster level, I am proposing that they should have the ability to turn it off as well at a cluster level. Since this is a topic level feature, folks may not spin up a separate cluster to try this feature, hence, we need to ensure that we provide them with the ability to try tiered storage for a topic which could be deleted and featured turned-off, so that rest of the production cases are not impacted. 5. Agree on not making public interface change as a requirement but we should define what "early access" means in that case. Users may not be aware that "early access" public APIs may change (unless I am missing some documentation somewhere completely, in which case I apologize for bringing this naive point). -- Divij Vaidya On Thu, Jul 27, 2023 at 2:27 PM Ismael Juma <m...@ismaeljuma.com> wrote: > > Hi Divij, > > Some of these are launch checklist items (not really goals) and some are > compatibility guarantees. More below. > > On Thu, Jul 27, 2023, 12:10 PM Divij Vaidya <divijvaidy...@gmail.com> wrote: > > > Hey Satish > > > > Could we consider adding "launch goals" in the release plan. While > > some of these may be implicit, it would be nice to list them down in > > the release plan. For this release, our launch requirements would be: > > 1. Users should be able to upgrade from any prior Kafka version to this > > version. > > > > This is part of the compatibility guarantees. The upgrade notes mention > this already. If there is a change in a given release, it should definitely > be highlighted. > > 2. On release, this version (or it's dependencies) would not have any > > known MEDIUM/HIGH CVE. > > > > This is a new policy and the details should be discussed. In particular, > the threshold (medium or high). > > 3. Presence of any "early access"/"beta" feature should not impact > > other production features when it is not enabled. > > > > This is a general guideline for early access features and not specific to > this release. It would be good to have a page that talks about these things. > > 4. Once enabled, users should have an option to disable any "early > > access"/"beta" feature and resume normal production features, i.e. > > impact of beta features should be reversible. > > > > This needs discussion and I don't think it's reasonable as a general rule. > For example, Kraft early access wasn't reversible and it was not feasible > for it to be. > > 5. KIP-405 will be available in "early access"/"beta" mode. Early > > access/beta means that the public facing interfaces won't change in > > future but the implementation is not recommended to be used in > > production. > > > I don't think it's ok to make this a requirement. Early access is a way to > get early feedback and all types of changes should be on the table. They > would be discussed via KIPs as usual. I believe there were some > incompatible changes for Kraft during the early access period although the > team aimed to minimize work required during upgrades. I have mentioned > Kraft a couple of times since it's a good example of a large feature that > went through this process. > > Ismael
