Re: [DISCUSS] KIP-833: Mark KRaft as Production Ready

2022-06-20 Thread Christo
Hello Colin, Thank you for the swift and informative reply! I will have a look at the suggested tickets and pick up one of the free ones. Best, Christo On Sunday, 19 June 2022, 22:46:41 BST, Colin McCabe wrote: Hi Christo NUKK, We have been trying to label KRaft issues on JIRA with

Re: [DISCUSS] KIP-833: Mark KRaft as Production Ready

2022-06-19 Thread Colin McCabe
Hi Christo NUKK, We have been trying to label KRaft issues on JIRA with "kip-500". So you should be able to do a search like this for them: https://issues.apache.org/jira/browse/KAFKA-13912?jql=project%20%3D%20KAFKA%20AND%20labels%20%3D%20kip-500%20AND%20statusCategory%20!%3D%20Done In JQL

Re: [DISCUSS] KIP-833: Mark KRaft as Production Ready

2022-06-19 Thread Christo NUKK
Hello! I went through JIRA, GitHub and the mailing list, but I was not able to find a list of developer tasks to be picked up for what is left to achieve feature parity with Zookeeper mode. I will have spare bandwidth in the next month and would like to contribute. I have touched

Re: [DISCUSS] KIP-833: Mark KRaft as Production Ready

2022-05-23 Thread Colin McCabe
Hi José, Yes, that matches my understanding. I added a section on "bridge releases" to the KIP. This section basically reiterates what we discussed in KIP-500, but it's good to have anyway. best, Colin On Wed, May 18, 2022, at 14:49, José Armando García Sancio wrote: > Hi Colin, > > Thanks

Re: [DISCUSS] KIP-833: Mark KRaft as Production Ready

2022-05-18 Thread José Armando García Sancio
Hi Colin, Thanks for the KIP. >The rationale for deprecating ZK in the 3.4 release is so that we can remove >it in the 4.0 release. (In general, Kafka requires features to be deprecated >for at least one release before they can be removed in the following major >release.) During the

Re: [DISCUSS] KIP-833: Mark KRaft as Production Ready

2022-05-11 Thread Andrew Otto
> Stretch clusters are possible with the KRaft architecture. For example, if you had a cluster with nodes in us-west-1, us-west-2 and us-central-1, you could put a KRaft controller node in each region. This is similar to how with ZK you'd put a ZK node in each region. Ah ha! TIL about

Re: [DISCUSS] KIP-833: Mark KRaft as Production Ready

2022-05-11 Thread Colin McCabe
On Thu, May 5, 2022, at 15:57, Israel Ekpo wrote: > Thanks Colin. > > I think we may need to update the KIP name to reflect the intent of the KIP > and convey everything it’s about if all the 3 action items will be covered > by the same KIP > > It contains three parts: > > Marking KRraft as

Re: [DISCUSS] KIP-833: Mark KRaft as Production Ready

2022-05-11 Thread Colin McCabe
Hi Andrew, Stretch clusters are possible with the KRaft architecture. For example, if you had a cluster with nodes in us-west-1, us-west-2 and us-central-1, you could put a KRaft controller node in each region. This is similar to how with ZK you'd put a ZK node in each region. best, Colin

Re: [DISCUSS] KIP-833: Mark KRaft as Production Ready

2022-05-09 Thread Andrew Otto
> Deprecating ZK Mode and > Removing Zookeeper Mode I'm excited about KRaft, but quick Q. I'm researching Kafka 'stretch' cluster deployments, and as far as I can tell stretch clusters require Zookeeper to function properly, is this correct? If so, we might want to solve that before Deprecating

Re: [DISCUSS] KIP-833: Mark KRaft as Production Ready

2022-05-05 Thread Israel Ekpo
Thanks Colin. I think we may need to update the KIP name to reflect the intent of the KIP and convey everything it’s about if all the 3 action items will be covered by the same KIP It contains three parts: Marking KRraft as Production Ready Deprecating ZK Mode and Removing Zookeeper Mode

Re: [DISCUSS] KIP-833: Mark KRaft as Production Ready

2022-05-05 Thread Colin McCabe
Hi all, Thanks for the comments. I agree that we should split out declaring KRaft going production for new clusters from deprecating ZK. We can do the former in the next release, 3.3, and the latter in the release after that, 3.4. I also talked offline with some of the people working on

Re: [DISCUSS] KIP-833: Mark KRaft as Production Ready

2022-05-04 Thread Ismael Juma
Yes, all features supported by zk mode will be available in kraft mode in the 3.x series. Ismael On Wed, May 4, 2022, 5:28 PM Israel Ekpo wrote: > Ismael, > > I like the timeline. However, does this or will this also account for > features users rely on today in Zookeeper mode being available

Re: [DISCUSS] KIP-833: Mark KRaft as Production Ready

2022-05-04 Thread Israel Ekpo
Ismael, I like the timeline. However, does this or will this also account for features users rely on today in Zookeeper mode being available when Zookeeper is dropped? That’s my main concern On Wed, May 4, 2022 at 8:12 PM Ismael Juma wrote: > Hi Colin, > > Thanks for the KIP, this is

Re: [DISCUSS] KIP-833: Mark KRaft as Production Ready

2022-05-04 Thread Ismael Juma
Hi Colin, Thanks for the KIP, this is exciting. Trying to balance progress and compatibility, how about the following? 1. 3.3 (~August 2022): kraft is production ready for new clusters 2. 3.4 (~December 2022/January 2023): migration from zk to kraft is production ready and zk mode is deprecated

Re: [DISCUSS] KIP-833: Mark KRaft as Production Ready

2022-05-04 Thread Israel Ekpo
Thanks Colin for the update and clarification and the additional details you added. I second Igor's thoughts that we should not deprecate ZK before having feature parity. However, I think marking KRaft mode as production-ready is something that can be done independent of the status of feature

Re: [DISCUSS] KIP-833: Mark KRaft as Production Ready

2022-05-04 Thread Igor Soarez
Hi Colin, It's very exciting to see KRaft ready for production! I see the value of marking it ready for production even with the current missing features. However, I'm worried about marking ZK mode as deprecated before these missing features are ready. It might be hard to estimate right now

Re: [DISCUSS] KIP-833: Mark KRaft as Production Ready

2022-05-04 Thread Colin McCabe
On Tue, May 3, 2022, at 23:16, Colin McCabe wrote: > > To be clear, the proposal here is to have the bridge release be 3.4 and > then move on to a ZK-free 4.0. With a 3.5 release as an option (but not > requirement) if we can't finish everything in time after 3.4. So that > would be two more

Re: [DISCUSS] KIP-833: Mark KRaft as Production Ready

2022-05-04 Thread Colin McCabe
On Tue, May 3, 2022, at 19:32, Luke Chen wrote: > Hi Colin, > > So exciting to see the KIP to mark the Kraft production ready! > > Just one comment: We should make sure the period between ZK deprecation > (i.e. v3.4.0) to ZK removal (i.e. v4.0.0) is not too short. > Do we have any expectation for

Re: [DISCUSS] KIP-833: Mark KRaft as Production Ready

2022-05-04 Thread Colin McCabe
On Tue, May 3, 2022, at 20:48, Israel Ekpo wrote: > Hi Colin, > > Thanks for this KIP. I am very excited to see the proposal. > > A lot in the community have been asking when KRaft mode will be ready and I > think this KIP will provide a lot of the answers and guidance desperately > needed. > >

Re: [DISCUSS] KIP-833: Mark KRaft as Production Ready

2022-05-03 Thread Israel Ekpo
Hi Colin, Thanks for this KIP. I am very excited to see the proposal. A lot in the community have been asking when KRaft mode will be ready and I think this KIP will provide a lot of the answers and guidance desperately needed. Thanks for working on it. I also have some of the questions from

Re: [DISCUSS] KIP-833: Mark KRaft as Production Ready

2022-05-03 Thread Luke Chen
Hi Colin, So exciting to see the KIP to mark the Kraft production ready! Just one comment: We should make sure the period between ZK deprecation (i.e. v3.4.0) to ZK removal (i.e. v4.0.0) is not too short. Do we have any expectation for the deprecation period? After all, this is not a small

[DISCUSS] KIP-833: Mark KRaft as Production Ready

2022-05-03 Thread Colin McCabe
Hi all, I've written a KIP for marking KRaft as production ready. Please take a look if you have a chance: https://cwiki.apache.org/confluence/x/8xKhD thanks, Colin