Hello Colin, Thank you for taking the time to review the proposal!
I have attached a compatibility matrix to aid the explanation below - if the mailing system rejects it I will find another way to share it. For the avoidance of doubt, I am not proposing to drop support for rolling upgrade from old Kafka versions to new ones. What I am saying is that additional care will need to be taken when upgrading to the latest Kafka versions depending on the version of the accompanying Zookeeper cluster. This additional care means one might have to upgrade to a Kafka version which falls in the intersection of the two sets in the accompanying diagram before upgrading the accompanying Zookeeper cluster. As a concrete example let's say you want to upgrade to Kafka 3.5 from Kafka 2.3 and Zookeeper 3.4. You will have to: 1. Carry out a rolling upgrade of your Kafka cluster to a version between 2.4 and 3.4. 2. Carry out a rolling upgrade of your Zookeeper cluster to 3.8.1 (with a possible stop at 3.4.6 due to https://zookeeper.apache.org/doc/r3.8.1/zookeeperReconfig.html#ch_reconfig_upgrade ). 3. Carry out a rolling upgrade of your Kafka cluster from 3.4 to 3.5. It is true that Zookeeper is to be deprecated in Kafka 4.0, but as far as I looked there is no concrete release date for that version yet. Until this is the case and unless we carry out a Zookeeper version upgrade we leave users to run on an end-of-life version with unpatched CVEs addressed in later versions. Some users have compliance requirements to only run on stable versions of a software and its dependencies and not keeping the dependencies up to date renders them unable to use Kafka. Please, let me know your thoughts on the matter! Best, Christo On Thu, 9 Mar 2023 at 21:56, Colin McCabe <cmcc...@apache.org> wrote: > Hi, > > I'm struggling a bit with this KIP, because dropping support for rolling > upgrades from old Kafka versions doesn't seem like something we should do > in a minor release. But on the other hand, the next Kafka release won't > have ZK at all. Maybe we should punt on this until and unless there is a > security issue that requires some action from us. > > I would also add, that a major ZK version bump is pretty risky. Last time > we did this we hit several bugs. I remember we hit one where there was an > incompatible change with regard to formatting (sorry, I can't seem to find > the JIRA right now). > > Sorry, but for now I have to vote -1 until I can understand this better > > best, > Colin > > > On Thu, Feb 23, 2023, at 06:48, Divij Vaidya wrote: > > Thanks for the KIP Christo. > > > > Having Zk 3.6 reach EOL in Dec 2022 is a good enough reason to upgrade, > > hence I completely agree with the motivation. Your experiments have > > demonstrated that the new version of Zk is stable at scale and the > backward > > compatibility risks are acceptable since Apache Kafka 2.4.x is an EOL > > version. > > > > Vote +1 (non binding) > > > > -- > > Divij Vaidya > > > > > > > > On Thu, Feb 23, 2023 at 3:32 PM Christo Lolov <christolo...@gmail.com> > > wrote: > > > >> Hello! > >> > >> I would like to start the vote for KIP-902, which upgrades Zookeeper to > >> version 3.8.1. > >> > >> The KIP can be found at > >> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=240882784 > >> > >> The discussion thread is > >> https://lists.apache.org/thread/5jbn2x0rtmqz5scyoygbdbj4vo0mpbw1 > >> > >> Thanks > >> Christo >
