> Stretch clusters are possible with the KRaft architecture. For example, if you had a cluster with nodes in us-west-1, us-west-2 and us-central-1, you could put a KRaft controller node in each region. This is similar to how with ZK you'd put a ZK node in each region.
Ah ha! TIL about process.roles <https://github.com/apache/kafka/blob/trunk/config/kraft/README.md#process-roles>. Thank you! On Wed, May 11, 2022 at 4:15 PM Colin McCabe <cmcc...@apache.org> wrote: > On Thu, May 5, 2022, at 15:57, Israel Ekpo wrote: > > Thanks Colin. > > > > I think we may need to update the KIP name to reflect the intent of the > KIP > > and convey everything it’s about if all the 3 action items will be > covered > > by the same KIP > > > > It contains three parts: > > > > Marking KRraft as Production Ready > > Deprecating ZK Mode and > > Removing Zookeeper Mode > > > > Should this be broken up in three separate KIPs since it will be done in > > multiple releases? > > Hi Israel, > > I think these three things are closely related, and it makes sense to have > them in a single KIP. Obviously we cannot remove ZK before deprecating it, > or deprecate it before marking it as production-ready. So if we had 3 KIPs > the discussion would get quite confusing since we'd constantly be > referencing the other discussion thread(s). > > > > > The current name of the KIP only conveys the first item and not the next > > two > > > > It is just a thought. I will like to get your perspective > > > > Names are always a hard part :) I didn't want the name to get too long. > Maybe if others want to see a longer name I can change it -- I'm open. > > best, > Colin > > > > > > > On Thu, May 5, 2022 at 1:19 PM Colin McCabe <cmcc...@apache.org> wrote: > > > >> Hi all, > >> > >> Thanks for the comments. I agree that we should split out declaring > KRaft > >> going production for new clusters from deprecating ZK. We can do the > former > >> in the next release, 3.3, and the latter in the release after that, 3.4. > >> > >> I also talked offline with some of the people working on upgrade from ZK > >> and it seems like 3.4 is a more realistic target for that work. Partly > this > >> is because 3.3 will be a bit earlier than I originally thought (for some > >> reason I thought it might be October, but Ismael pointed out it's > planned > >> for August) > >> > >> I also agree that it will probably be useful to have a 3.5 release > >> following the 3.4 one, which will also support ZK. Hopefully we will not > >> need a 3.6, but we don't have to decide that now. > >> > >> I added a timeline section to the KIP to make this all clearer. To be > >> clear, it is a preliminary timeline, which may change. It's difficult to > >> fully plan out the next 1.5 years of Apache Kafka releases right now -- > and > >> obviously, there are things which may come up to change our plans. > However, > >> I think it is still helpful to have the section to give us a feeling for > >> the general roadmap. > >> > >> When you read the KIP, please consider it all speculative except for the > >> three proposed changes at the top: > >> > >> 1. Mark KRaft as production-ready for new clusters in the upcoming Kafka > >> 3.3 release. > >> 2. Deprecate ZooKeeper mode in the upcoming Kafka 3.4 release > >> 3. Plan to remove ZooKeeper mode entirely in Kafka 4.0. > >> > >> best, > >> Colin > >> > >> > >> On Wed, May 4, 2022, at 19:31, Ismael Juma wrote: > >> > Yes, all features supported by zk mode will be available in kraft > mode in > >> > the 3.x series. > >> > > >> > Ismael > >> > > >> > On Wed, May 4, 2022, 5:28 PM Israel Ekpo <israele...@gmail.com> > wrote: > >> > > >> >> Ismael, > >> >> > >> >> I like the timeline. However, does this or will this also account for > >> >> features users rely on today in Zookeeper mode being available when > >> >> Zookeeper is dropped? > >> >> > >> >> That’s my main concern > >> >> > >> >> On Wed, May 4, 2022 at 8:12 PM Ismael Juma <ism...@juma.me.uk> > wrote: > >> >> > >> >> > Hi Colin, > >> >> > > >> >> > Thanks for the KIP, this is exciting. Trying to balance progress > and > >> >> > compatibility, how about the following? > >> >> > > >> >> > 1. 3.3 (~August 2022): kraft is production ready for new clusters > >> >> > 2. 3.4 (~December 2022/January 2023): migration from zk to kraft is > >> >> > production ready and zk mode is deprecated > >> >> > 3. 3.5 (~April 2023): buffer release > >> >> > 4. 4.0 (~August 2023): kraft mode is on by default and zk mode is > >> removed > >> >> > > >> >> > This would mean about 1 year from kraft being production ready to > zk > >> >> > removal and 8 months from zk deprecation to zk removal. > >> >> > > >> >> > If necessary (due to important bugs or security issues), we can do > a > >> >> couple > >> >> > of additional bug fix releases in the 3.5 series after 4.0 is > >> released. > >> >> > > >> >> > Thoughts? > >> >> > > >> >> > Ismael > >> >> > > >> >> > On Tue, May 3, 2022, 6:03 PM Colin McCabe <cmcc...@apache.org> > wrote: > >> >> > > >> >> > > Hi all, > >> >> > > > >> >> > > I've written a KIP for marking KRaft as production ready. Please > >> take a > >> >> > > look if you have a chance: > >> >> > > > >> >> > > https://cwiki.apache.org/confluence/x/8xKhD > >> >> > > > >> >> > > thanks, > >> >> > > Colin > >> >> > > > >> >> > > >> >> -- > >> >> Israel Ekpo > >> >> Lead Instructor, IzzyAcademy.com > >> >> https://www.youtube.com/c/izzyacademy > >> >> https://izzyacademy.com/ > >> >> > >> > > -- > > Israel Ekpo > > Lead Instructor, IzzyAcademy.com > > https://www.youtube.com/c/izzyacademy > > https://izzyacademy.com/ >
