Re: [VOTE] 3.5.0 RC0

[David Arthur]

Mickael, all of our migration fixes are in.

Thanks!
David

On Thu, Jun 1, 2023 at 6:10 PM Colin McCabe  wrote:

> Hi Mickael,
>
> Can you start the new RC tomorrow? There's one last PR we'd like to get in.
>
> If we can't get it in by tomorrow then let's go ahead anyway.
>
> Thanks very much,
> Colin
>
>
> On Thu, Jun 1, 2023, at 14:15, Mickael Maison wrote:
> > Hi David,
> >
> > The PR you mentioned is merged now. Can I start working on a new RC or is
> > there more work needed? It seems the associated ticket is still open.
> >
> > Thanks,
> > Mickael
> >
> > On Wed, 31 May 2023, 23:52 Justine Olshan,  >
> > wrote:
> >
> >> Hey Mickael --
> >> This is done. Thanks!
> >>
> >> On Wed, May 31, 2023 at 11:24 AM Mickael Maison <
> mickael.mai...@gmail.com>
> >> wrote:
> >>
> >> > Hi Justine,
> >> >
> >> > Yes you can merge that into 3.5.
> >> >
> >> > Thanks,
> >> > Mickael
> >> >
> >> > On Wed, May 31, 2023 at 7:56 PM Justine Olshan
> >> >  wrote:
> >> > >
> >> > > FYI -- I just saw this PR regarding a dependency for ARM. We may
> want
> >> to
> >> > > get this in for 3.5 as well. It should be quick.
> >> > >
> >> > > https://issues.apache.org/jira/browse/KAFKA-15044
> >> > > https://github.com/apache/kafka/pull/13786
> >> > >
> >> > > Justine
> >> > >
> >> > > On Wed, May 31, 2023 at 9:28 AM David Arthur
> >> > >  wrote:
> >> > >
> >> > > > Mickael,
> >> > > >
> >> > > > Colin has approved my patch for KAFKA-15010, I'm just waiting on a
> >> > build
> >> > > > before merging. I'll go ahead and backport the other fixes that
> need
> >> to
> >> > > > precede this one into 3.5.
> >> > > >
> >> > > > -David
> >> > > >
> >> > > > On Wed, May 31, 2023 at 11:52 AM Mickael Maison <
> >> > mickael.mai...@gmail.com>
> >> > > > wrote:
> >> > > >
> >> > > > > Hi,
> >> > > > >
> >> > > > > The issue mentioned by Greg has been fixed. As soon as the fix
> for
> >> > > > > KAFKA-15010 is merged I'll build another RC.
> >> > > > >
> >> > > > > Thanks,
> >> > > > > Mickael
> >> > > > >
> >> > > > > On Tue, May 30, 2023 at 10:33 AM Mickael Maison
> >> > > > >  wrote:
> >> > > > > >
> >> > > > > > Hi David,
> >> > > > > >
> >> > > > > > Feel free to backport the necessary fixes to 3.5.
> >> > > > > >
> >> > > > > > Thanks,
> >> > > > > > Mickael
> >> > > > > >
> >> > > > > > On Tue, May 30, 2023 at 10:32 AM Mickael Maison
> >> > > > > >  wrote:
> >> > > > > > >
> >> > > > > > > Hi Greg,
> >> > > > > > >
> >> > > > > > > Thanks for the heads up, this indeed looks like something we
> >> > want in
> >> > > > > > > 3.5. I've replied in the PR.
> >> > > > > > >
> >> > > > > > > Mickael
> >> > > > > > >
> >> > > > > > > On Sat, May 27, 2023 at 11:44 PM David Arthur
> >> > > > > > >  wrote:
> >> > > > > > > >
> >> > > > > > > > Mickael, after looking more closely, I definitely think
> >> > KAFKA-15010
> >> > > > > is a
> >> > > > > > > > blocker. It creates the case where the controller can
> totally
> >> > miss
> >> > > > a
> >> > > > > > > > metadata update and not write it back to ZK. Since things
> >> like
> >> > > > > dynamic
> >> > > > > > > > configs and ACLs are only read from ZK by the ZK brokers,
> we
> >> > could
> >> > > > > have
> >> > > > > > > > significant problems while the brokers are being migrated
> >> (when
> >> > > > some
> >> > > > > are
> >> > > > > > > > KRaft and some are ZK). E.g., ZK brokers could be totally
> >> > unaware
> >> > > > of
> >> > > > > an ACL
> >> > > > > > > > change while the KRaft brokers have it. I have a fix ready
> >> here
> >> > > > > > > > https://github.com/apache/kafka/pull/13758. I think we
> can
> >> > get it
> >> > > > > committed
> >> > > > > > > > soon.
> >> > > > > > > >
> >> > > > > > > > Another blocker is KAFKA-15004 which was just merged to
> >> trunk.
> >> > This
> >> > > > > is
> >> > > > > > > > another dual-write bug where new topic/broker configs will
> >> not
> >> > be
> >> > > > > written
> >> > > > > > > > back to ZK by the controller.
> >> > > > > > > >
> >> > > > > > > > The fix for KAFKA-15010 has a few dependencies on fixes we
> >> made
> >> > > > this
> >> > > > > past
> >> > > > > > > > week, so we'll need to cherry-pick a few commits. The
> changes
> >> > are
> >> > > > > totally
> >> > > > > > > > contained within the migration area of code, so I think
> the
> >> > risk in
> >> > > > > > > > including them is fairly low.
> >> > > > > > > >
> >> > > > > > > > -David
> >> > > > > > > >
> >> > > > > > > > On Thu, May 25, 2023 at 2:15 PM Greg Harris
> >> > > > > 
> >> > > > > > > > wrote:
> >> > > > > > > >
> >> > > > > > > > > Hey all,
> >> > > > > > > > >
> >> > > > > > > > > A contributor just pointed out a small but noticeable
> flaw
> >> > in the
> >> > > > > > > > > implementation of KIP-581
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > >
> >> > > >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-581%3A+Value+of+optional+null+field+which+has+default+value
> >> > > > > > > > > which is planned for this release.
> >> > > > > > > >

Re: [VOTE] 3.5.0 RC0

[Colin McCabe]

Hi Mickael,

Can you start the new RC tomorrow? There's one last PR we'd like to get in.

If we can't get it in by tomorrow then let's go ahead anyway.

Thanks very much,
Colin


On Thu, Jun 1, 2023, at 14:15, Mickael Maison wrote:
> Hi David,
>
> The PR you mentioned is merged now. Can I start working on a new RC or is
> there more work needed? It seems the associated ticket is still open.
>
> Thanks,
> Mickael
>
> On Wed, 31 May 2023, 23:52 Justine Olshan, 
> wrote:
>
>> Hey Mickael --
>> This is done. Thanks!
>>
>> On Wed, May 31, 2023 at 11:24 AM Mickael Maison 
>> wrote:
>>
>> > Hi Justine,
>> >
>> > Yes you can merge that into 3.5.
>> >
>> > Thanks,
>> > Mickael
>> >
>> > On Wed, May 31, 2023 at 7:56 PM Justine Olshan
>> >  wrote:
>> > >
>> > > FYI -- I just saw this PR regarding a dependency for ARM. We may want
>> to
>> > > get this in for 3.5 as well. It should be quick.
>> > >
>> > > https://issues.apache.org/jira/browse/KAFKA-15044
>> > > https://github.com/apache/kafka/pull/13786
>> > >
>> > > Justine
>> > >
>> > > On Wed, May 31, 2023 at 9:28 AM David Arthur
>> > >  wrote:
>> > >
>> > > > Mickael,
>> > > >
>> > > > Colin has approved my patch for KAFKA-15010, I'm just waiting on a
>> > build
>> > > > before merging. I'll go ahead and backport the other fixes that need
>> to
>> > > > precede this one into 3.5.
>> > > >
>> > > > -David
>> > > >
>> > > > On Wed, May 31, 2023 at 11:52 AM Mickael Maison <
>> > mickael.mai...@gmail.com>
>> > > > wrote:
>> > > >
>> > > > > Hi,
>> > > > >
>> > > > > The issue mentioned by Greg has been fixed. As soon as the fix for
>> > > > > KAFKA-15010 is merged I'll build another RC.
>> > > > >
>> > > > > Thanks,
>> > > > > Mickael
>> > > > >
>> > > > > On Tue, May 30, 2023 at 10:33 AM Mickael Maison
>> > > > >  wrote:
>> > > > > >
>> > > > > > Hi David,
>> > > > > >
>> > > > > > Feel free to backport the necessary fixes to 3.5.
>> > > > > >
>> > > > > > Thanks,
>> > > > > > Mickael
>> > > > > >
>> > > > > > On Tue, May 30, 2023 at 10:32 AM Mickael Maison
>> > > > > >  wrote:
>> > > > > > >
>> > > > > > > Hi Greg,
>> > > > > > >
>> > > > > > > Thanks for the heads up, this indeed looks like something we
>> > want in
>> > > > > > > 3.5. I've replied in the PR.
>> > > > > > >
>> > > > > > > Mickael
>> > > > > > >
>> > > > > > > On Sat, May 27, 2023 at 11:44 PM David Arthur
>> > > > > > >  wrote:
>> > > > > > > >
>> > > > > > > > Mickael, after looking more closely, I definitely think
>> > KAFKA-15010
>> > > > > is a
>> > > > > > > > blocker. It creates the case where the controller can totally
>> > miss
>> > > > a
>> > > > > > > > metadata update and not write it back to ZK. Since things
>> like
>> > > > > dynamic
>> > > > > > > > configs and ACLs are only read from ZK by the ZK brokers, we
>> > could
>> > > > > have
>> > > > > > > > significant problems while the brokers are being migrated
>> (when
>> > > > some
>> > > > > are
>> > > > > > > > KRaft and some are ZK). E.g., ZK brokers could be totally
>> > unaware
>> > > > of
>> > > > > an ACL
>> > > > > > > > change while the KRaft brokers have it. I have a fix ready
>> here
>> > > > > > > > https://github.com/apache/kafka/pull/13758. I think we can
>> > get it
>> > > > > committed
>> > > > > > > > soon.
>> > > > > > > >
>> > > > > > > > Another blocker is KAFKA-15004 which was just merged to
>> trunk.
>> > This
>> > > > > is
>> > > > > > > > another dual-write bug where new topic/broker configs will
>> not
>> > be
>> > > > > written
>> > > > > > > > back to ZK by the controller.
>> > > > > > > >
>> > > > > > > > The fix for KAFKA-15010 has a few dependencies on fixes we
>> made
>> > > > this
>> > > > > past
>> > > > > > > > week, so we'll need to cherry-pick a few commits. The changes
>> > are
>> > > > > totally
>> > > > > > > > contained within the migration area of code, so I think the
>> > risk in
>> > > > > > > > including them is fairly low.
>> > > > > > > >
>> > > > > > > > -David
>> > > > > > > >
>> > > > > > > > On Thu, May 25, 2023 at 2:15 PM Greg Harris
>> > > > > 
>> > > > > > > > wrote:
>> > > > > > > >
>> > > > > > > > > Hey all,
>> > > > > > > > >
>> > > > > > > > > A contributor just pointed out a small but noticeable flaw
>> > in the
>> > > > > > > > > implementation of KIP-581
>> > > > > > > > >
>> > > > > > > > >
>> > > > >
>> > > >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-581%3A+Value+of+optional+null+field+which+has+default+value
>> > > > > > > > > which is planned for this release.
>> > > > > > > > > Impact: the feature works for root values in a record, but
>> > does
>> > > > not
>> > > > > > > > > work for any fields within structs. Fields within structs
>> > will
>> > > > > > > > > continue to have their previous, backwards-compatible
>> > behavior.
>> > > > > > > > > The contributor has submitted a bug-fix PR which reports
>> the
>> > > > > problem
>> > > > > > > > > and does not yet have a merge-able solution, but they are
>> > > > actively
>> > > > > > > > > respondin

Re: [VOTE] 3.5.0 RC0

[Mickael Maison]

Hi David,

The PR you mentioned is merged now. Can I start working on a new RC or is
there more work needed? It seems the associated ticket is still open.

Thanks,
Mickael

On Wed, 31 May 2023, 23:52 Justine Olshan, 
wrote:

> Hey Mickael --
> This is done. Thanks!
>
> On Wed, May 31, 2023 at 11:24 AM Mickael Maison 
> wrote:
>
> > Hi Justine,
> >
> > Yes you can merge that into 3.5.
> >
> > Thanks,
> > Mickael
> >
> > On Wed, May 31, 2023 at 7:56 PM Justine Olshan
> >  wrote:
> > >
> > > FYI -- I just saw this PR regarding a dependency for ARM. We may want
> to
> > > get this in for 3.5 as well. It should be quick.
> > >
> > > https://issues.apache.org/jira/browse/KAFKA-15044
> > > https://github.com/apache/kafka/pull/13786
> > >
> > > Justine
> > >
> > > On Wed, May 31, 2023 at 9:28 AM David Arthur
> > >  wrote:
> > >
> > > > Mickael,
> > > >
> > > > Colin has approved my patch for KAFKA-15010, I'm just waiting on a
> > build
> > > > before merging. I'll go ahead and backport the other fixes that need
> to
> > > > precede this one into 3.5.
> > > >
> > > > -David
> > > >
> > > > On Wed, May 31, 2023 at 11:52 AM Mickael Maison <
> > mickael.mai...@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > The issue mentioned by Greg has been fixed. As soon as the fix for
> > > > > KAFKA-15010 is merged I'll build another RC.
> > > > >
> > > > > Thanks,
> > > > > Mickael
> > > > >
> > > > > On Tue, May 30, 2023 at 10:33 AM Mickael Maison
> > > > >  wrote:
> > > > > >
> > > > > > Hi David,
> > > > > >
> > > > > > Feel free to backport the necessary fixes to 3.5.
> > > > > >
> > > > > > Thanks,
> > > > > > Mickael
> > > > > >
> > > > > > On Tue, May 30, 2023 at 10:32 AM Mickael Maison
> > > > > >  wrote:
> > > > > > >
> > > > > > > Hi Greg,
> > > > > > >
> > > > > > > Thanks for the heads up, this indeed looks like something we
> > want in
> > > > > > > 3.5. I've replied in the PR.
> > > > > > >
> > > > > > > Mickael
> > > > > > >
> > > > > > > On Sat, May 27, 2023 at 11:44 PM David Arthur
> > > > > > >  wrote:
> > > > > > > >
> > > > > > > > Mickael, after looking more closely, I definitely think
> > KAFKA-15010
> > > > > is a
> > > > > > > > blocker. It creates the case where the controller can totally
> > miss
> > > > a
> > > > > > > > metadata update and not write it back to ZK. Since things
> like
> > > > > dynamic
> > > > > > > > configs and ACLs are only read from ZK by the ZK brokers, we
> > could
> > > > > have
> > > > > > > > significant problems while the brokers are being migrated
> (when
> > > > some
> > > > > are
> > > > > > > > KRaft and some are ZK). E.g., ZK brokers could be totally
> > unaware
> > > > of
> > > > > an ACL
> > > > > > > > change while the KRaft brokers have it. I have a fix ready
> here
> > > > > > > > https://github.com/apache/kafka/pull/13758. I think we can
> > get it
> > > > > committed
> > > > > > > > soon.
> > > > > > > >
> > > > > > > > Another blocker is KAFKA-15004 which was just merged to
> trunk.
> > This
> > > > > is
> > > > > > > > another dual-write bug where new topic/broker configs will
> not
> > be
> > > > > written
> > > > > > > > back to ZK by the controller.
> > > > > > > >
> > > > > > > > The fix for KAFKA-15010 has a few dependencies on fixes we
> made
> > > > this
> > > > > past
> > > > > > > > week, so we'll need to cherry-pick a few commits. The changes
> > are
> > > > > totally
> > > > > > > > contained within the migration area of code, so I think the
> > risk in
> > > > > > > > including them is fairly low.
> > > > > > > >
> > > > > > > > -David
> > > > > > > >
> > > > > > > > On Thu, May 25, 2023 at 2:15 PM Greg Harris
> > > > > 
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hey all,
> > > > > > > > >
> > > > > > > > > A contributor just pointed out a small but noticeable flaw
> > in the
> > > > > > > > > implementation of KIP-581
> > > > > > > > >
> > > > > > > > >
> > > > >
> > > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-581%3A+Value+of+optional+null+field+which+has+default+value
> > > > > > > > > which is planned for this release.
> > > > > > > > > Impact: the feature works for root values in a record, but
> > does
> > > > not
> > > > > > > > > work for any fields within structs. Fields within structs
> > will
> > > > > > > > > continue to have their previous, backwards-compatible
> > behavior.
> > > > > > > > > The contributor has submitted a bug-fix PR which reports
> the
> > > > > problem
> > > > > > > > > and does not yet have a merge-able solution, but they are
> > > > actively
> > > > > > > > > responding and interested in having this fixed:
> > > > > > > > > https://github.com/apache/kafka/pull/13748
> > > > > > > > > The overall fix should be a one-liner + some unit tests.
> > While
> > > > > this is
> > > > > > > > > not a regression, it does make the feature largely useless,
> > as
> > > > the
> > > > > > > > > majority of use-cases will be for struct fields.
> > > > > > > > >
> > > > > > > > > 

Re: [VOTE] 3.5.0 RC0

[Justine Olshan]

Hey Mickael --
This is done. Thanks!

On Wed, May 31, 2023 at 11:24 AM Mickael Maison 
wrote:

> Hi Justine,
>
> Yes you can merge that into 3.5.
>
> Thanks,
> Mickael
>
> On Wed, May 31, 2023 at 7:56 PM Justine Olshan
>  wrote:
> >
> > FYI -- I just saw this PR regarding a dependency for ARM. We may want to
> > get this in for 3.5 as well. It should be quick.
> >
> > https://issues.apache.org/jira/browse/KAFKA-15044
> > https://github.com/apache/kafka/pull/13786
> >
> > Justine
> >
> > On Wed, May 31, 2023 at 9:28 AM David Arthur
> >  wrote:
> >
> > > Mickael,
> > >
> > > Colin has approved my patch for KAFKA-15010, I'm just waiting on a
> build
> > > before merging. I'll go ahead and backport the other fixes that need to
> > > precede this one into 3.5.
> > >
> > > -David
> > >
> > > On Wed, May 31, 2023 at 11:52 AM Mickael Maison <
> mickael.mai...@gmail.com>
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > The issue mentioned by Greg has been fixed. As soon as the fix for
> > > > KAFKA-15010 is merged I'll build another RC.
> > > >
> > > > Thanks,
> > > > Mickael
> > > >
> > > > On Tue, May 30, 2023 at 10:33 AM Mickael Maison
> > > >  wrote:
> > > > >
> > > > > Hi David,
> > > > >
> > > > > Feel free to backport the necessary fixes to 3.5.
> > > > >
> > > > > Thanks,
> > > > > Mickael
> > > > >
> > > > > On Tue, May 30, 2023 at 10:32 AM Mickael Maison
> > > > >  wrote:
> > > > > >
> > > > > > Hi Greg,
> > > > > >
> > > > > > Thanks for the heads up, this indeed looks like something we
> want in
> > > > > > 3.5. I've replied in the PR.
> > > > > >
> > > > > > Mickael
> > > > > >
> > > > > > On Sat, May 27, 2023 at 11:44 PM David Arthur
> > > > > >  wrote:
> > > > > > >
> > > > > > > Mickael, after looking more closely, I definitely think
> KAFKA-15010
> > > > is a
> > > > > > > blocker. It creates the case where the controller can totally
> miss
> > > a
> > > > > > > metadata update and not write it back to ZK. Since things like
> > > > dynamic
> > > > > > > configs and ACLs are only read from ZK by the ZK brokers, we
> could
> > > > have
> > > > > > > significant problems while the brokers are being migrated (when
> > > some
> > > > are
> > > > > > > KRaft and some are ZK). E.g., ZK brokers could be totally
> unaware
> > > of
> > > > an ACL
> > > > > > > change while the KRaft brokers have it. I have a fix ready here
> > > > > > > https://github.com/apache/kafka/pull/13758. I think we can
> get it
> > > > committed
> > > > > > > soon.
> > > > > > >
> > > > > > > Another blocker is KAFKA-15004 which was just merged to trunk.
> This
> > > > is
> > > > > > > another dual-write bug where new topic/broker configs will not
> be
> > > > written
> > > > > > > back to ZK by the controller.
> > > > > > >
> > > > > > > The fix for KAFKA-15010 has a few dependencies on fixes we made
> > > this
> > > > past
> > > > > > > week, so we'll need to cherry-pick a few commits. The changes
> are
> > > > totally
> > > > > > > contained within the migration area of code, so I think the
> risk in
> > > > > > > including them is fairly low.
> > > > > > >
> > > > > > > -David
> > > > > > >
> > > > > > > On Thu, May 25, 2023 at 2:15 PM Greg Harris
> > > > 
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hey all,
> > > > > > > >
> > > > > > > > A contributor just pointed out a small but noticeable flaw
> in the
> > > > > > > > implementation of KIP-581
> > > > > > > >
> > > > > > > >
> > > >
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-581%3A+Value+of+optional+null+field+which+has+default+value
> > > > > > > > which is planned for this release.
> > > > > > > > Impact: the feature works for root values in a record, but
> does
> > > not
> > > > > > > > work for any fields within structs. Fields within structs
> will
> > > > > > > > continue to have their previous, backwards-compatible
> behavior.
> > > > > > > > The contributor has submitted a bug-fix PR which reports the
> > > > problem
> > > > > > > > and does not yet have a merge-able solution, but they are
> > > actively
> > > > > > > > responding and interested in having this fixed:
> > > > > > > > https://github.com/apache/kafka/pull/13748
> > > > > > > > The overall fix should be a one-liner + some unit tests.
> While
> > > > this is
> > > > > > > > not a regression, it does make the feature largely useless,
> as
> > > the
> > > > > > > > majority of use-cases will be for struct fields.
> > > > > > > >
> > > > > > > > Thanks!
> > > > > > > > Greg Harris
> > > > > > > >
> > > > > > > > On Wed, May 24, 2023 at 7:05 PM Ismael Juma <
> ism...@juma.me.uk>
> > > > wrote:
> > > > > > > > >
> > > > > > > > > I agree the migration should be functional - it wasn't
> obvious
> > > > if the
> > > > > > > > > migration issues are edge cases or not. If they are edge
> cases,
> > > > I think
> > > > > > > > > 3.5.1 would be fine given the preview status.
> > > > > > > > >
> > > > > > > > > I understand that a new RC is needed, but that doesn't
> mean we
> > > > should let

Re: [VOTE] 3.5.0 RC0

[Mickael Maison]

Hi Justine,

Yes you can merge that into 3.5.

Thanks,
Mickael

On Wed, May 31, 2023 at 7:56 PM Justine Olshan
 wrote:
>
> FYI -- I just saw this PR regarding a dependency for ARM. We may want to
> get this in for 3.5 as well. It should be quick.
>
> https://issues.apache.org/jira/browse/KAFKA-15044
> https://github.com/apache/kafka/pull/13786
>
> Justine
>
> On Wed, May 31, 2023 at 9:28 AM David Arthur
>  wrote:
>
> > Mickael,
> >
> > Colin has approved my patch for KAFKA-15010, I'm just waiting on a build
> > before merging. I'll go ahead and backport the other fixes that need to
> > precede this one into 3.5.
> >
> > -David
> >
> > On Wed, May 31, 2023 at 11:52 AM Mickael Maison 
> > wrote:
> >
> > > Hi,
> > >
> > > The issue mentioned by Greg has been fixed. As soon as the fix for
> > > KAFKA-15010 is merged I'll build another RC.
> > >
> > > Thanks,
> > > Mickael
> > >
> > > On Tue, May 30, 2023 at 10:33 AM Mickael Maison
> > >  wrote:
> > > >
> > > > Hi David,
> > > >
> > > > Feel free to backport the necessary fixes to 3.5.
> > > >
> > > > Thanks,
> > > > Mickael
> > > >
> > > > On Tue, May 30, 2023 at 10:32 AM Mickael Maison
> > > >  wrote:
> > > > >
> > > > > Hi Greg,
> > > > >
> > > > > Thanks for the heads up, this indeed looks like something we want in
> > > > > 3.5. I've replied in the PR.
> > > > >
> > > > > Mickael
> > > > >
> > > > > On Sat, May 27, 2023 at 11:44 PM David Arthur
> > > > >  wrote:
> > > > > >
> > > > > > Mickael, after looking more closely, I definitely think KAFKA-15010
> > > is a
> > > > > > blocker. It creates the case where the controller can totally miss
> > a
> > > > > > metadata update and not write it back to ZK. Since things like
> > > dynamic
> > > > > > configs and ACLs are only read from ZK by the ZK brokers, we could
> > > have
> > > > > > significant problems while the brokers are being migrated (when
> > some
> > > are
> > > > > > KRaft and some are ZK). E.g., ZK brokers could be totally unaware
> > of
> > > an ACL
> > > > > > change while the KRaft brokers have it. I have a fix ready here
> > > > > > https://github.com/apache/kafka/pull/13758. I think we can get it
> > > committed
> > > > > > soon.
> > > > > >
> > > > > > Another blocker is KAFKA-15004 which was just merged to trunk. This
> > > is
> > > > > > another dual-write bug where new topic/broker configs will not be
> > > written
> > > > > > back to ZK by the controller.
> > > > > >
> > > > > > The fix for KAFKA-15010 has a few dependencies on fixes we made
> > this
> > > past
> > > > > > week, so we'll need to cherry-pick a few commits. The changes are
> > > totally
> > > > > > contained within the migration area of code, so I think the risk in
> > > > > > including them is fairly low.
> > > > > >
> > > > > > -David
> > > > > >
> > > > > > On Thu, May 25, 2023 at 2:15 PM Greg Harris
> > > 
> > > > > > wrote:
> > > > > >
> > > > > > > Hey all,
> > > > > > >
> > > > > > > A contributor just pointed out a small but noticeable flaw in the
> > > > > > > implementation of KIP-581
> > > > > > >
> > > > > > >
> > >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-581%3A+Value+of+optional+null+field+which+has+default+value
> > > > > > > which is planned for this release.
> > > > > > > Impact: the feature works for root values in a record, but does
> > not
> > > > > > > work for any fields within structs. Fields within structs will
> > > > > > > continue to have their previous, backwards-compatible behavior.
> > > > > > > The contributor has submitted a bug-fix PR which reports the
> > > problem
> > > > > > > and does not yet have a merge-able solution, but they are
> > actively
> > > > > > > responding and interested in having this fixed:
> > > > > > > https://github.com/apache/kafka/pull/13748
> > > > > > > The overall fix should be a one-liner + some unit tests. While
> > > this is
> > > > > > > not a regression, it does make the feature largely useless, as
> > the
> > > > > > > majority of use-cases will be for struct fields.
> > > > > > >
> > > > > > > Thanks!
> > > > > > > Greg Harris
> > > > > > >
> > > > > > > On Wed, May 24, 2023 at 7:05 PM Ismael Juma 
> > > wrote:
> > > > > > > >
> > > > > > > > I agree the migration should be functional - it wasn't obvious
> > > if the
> > > > > > > > migration issues are edge cases or not. If they are edge cases,
> > > I think
> > > > > > > > 3.5.1 would be fine given the preview status.
> > > > > > > >
> > > > > > > > I understand that a new RC is needed, but that doesn't mean we
> > > should let
> > > > > > > > everything in. Each change carries some risk. And if we don't
> > > agree on
> > > > > > > the
> > > > > > > > bar for the migration work, we may be having the same
> > discussion
> > > next
> > > > > > > week.
> > > > > > > > :)
> > > > > > > >
> > > > > > > > Ismael
> > > > > > > >
> > > > > > > > On Wed, May 24, 2023, 12:00 PM Josep Prat
> > > 
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi there,
> > > > > > > > > Is the plan des

Re: [VOTE] 3.5.0 RC0

[Justine Olshan]

FYI -- I just saw this PR regarding a dependency for ARM. We may want to
get this in for 3.5 as well. It should be quick.

https://issues.apache.org/jira/browse/KAFKA-15044
https://github.com/apache/kafka/pull/13786

Justine

On Wed, May 31, 2023 at 9:28 AM David Arthur
 wrote:

> Mickael,
>
> Colin has approved my patch for KAFKA-15010, I'm just waiting on a build
> before merging. I'll go ahead and backport the other fixes that need to
> precede this one into 3.5.
>
> -David
>
> On Wed, May 31, 2023 at 11:52 AM Mickael Maison 
> wrote:
>
> > Hi,
> >
> > The issue mentioned by Greg has been fixed. As soon as the fix for
> > KAFKA-15010 is merged I'll build another RC.
> >
> > Thanks,
> > Mickael
> >
> > On Tue, May 30, 2023 at 10:33 AM Mickael Maison
> >  wrote:
> > >
> > > Hi David,
> > >
> > > Feel free to backport the necessary fixes to 3.5.
> > >
> > > Thanks,
> > > Mickael
> > >
> > > On Tue, May 30, 2023 at 10:32 AM Mickael Maison
> > >  wrote:
> > > >
> > > > Hi Greg,
> > > >
> > > > Thanks for the heads up, this indeed looks like something we want in
> > > > 3.5. I've replied in the PR.
> > > >
> > > > Mickael
> > > >
> > > > On Sat, May 27, 2023 at 11:44 PM David Arthur
> > > >  wrote:
> > > > >
> > > > > Mickael, after looking more closely, I definitely think KAFKA-15010
> > is a
> > > > > blocker. It creates the case where the controller can totally miss
> a
> > > > > metadata update and not write it back to ZK. Since things like
> > dynamic
> > > > > configs and ACLs are only read from ZK by the ZK brokers, we could
> > have
> > > > > significant problems while the brokers are being migrated (when
> some
> > are
> > > > > KRaft and some are ZK). E.g., ZK brokers could be totally unaware
> of
> > an ACL
> > > > > change while the KRaft brokers have it. I have a fix ready here
> > > > > https://github.com/apache/kafka/pull/13758. I think we can get it
> > committed
> > > > > soon.
> > > > >
> > > > > Another blocker is KAFKA-15004 which was just merged to trunk. This
> > is
> > > > > another dual-write bug where new topic/broker configs will not be
> > written
> > > > > back to ZK by the controller.
> > > > >
> > > > > The fix for KAFKA-15010 has a few dependencies on fixes we made
> this
> > past
> > > > > week, so we'll need to cherry-pick a few commits. The changes are
> > totally
> > > > > contained within the migration area of code, so I think the risk in
> > > > > including them is fairly low.
> > > > >
> > > > > -David
> > > > >
> > > > > On Thu, May 25, 2023 at 2:15 PM Greg Harris
> > 
> > > > > wrote:
> > > > >
> > > > > > Hey all,
> > > > > >
> > > > > > A contributor just pointed out a small but noticeable flaw in the
> > > > > > implementation of KIP-581
> > > > > >
> > > > > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-581%3A+Value+of+optional+null+field+which+has+default+value
> > > > > > which is planned for this release.
> > > > > > Impact: the feature works for root values in a record, but does
> not
> > > > > > work for any fields within structs. Fields within structs will
> > > > > > continue to have their previous, backwards-compatible behavior.
> > > > > > The contributor has submitted a bug-fix PR which reports the
> > problem
> > > > > > and does not yet have a merge-able solution, but they are
> actively
> > > > > > responding and interested in having this fixed:
> > > > > > https://github.com/apache/kafka/pull/13748
> > > > > > The overall fix should be a one-liner + some unit tests. While
> > this is
> > > > > > not a regression, it does make the feature largely useless, as
> the
> > > > > > majority of use-cases will be for struct fields.
> > > > > >
> > > > > > Thanks!
> > > > > > Greg Harris
> > > > > >
> > > > > > On Wed, May 24, 2023 at 7:05 PM Ismael Juma 
> > wrote:
> > > > > > >
> > > > > > > I agree the migration should be functional - it wasn't obvious
> > if the
> > > > > > > migration issues are edge cases or not. If they are edge cases,
> > I think
> > > > > > > 3.5.1 would be fine given the preview status.
> > > > > > >
> > > > > > > I understand that a new RC is needed, but that doesn't mean we
> > should let
> > > > > > > everything in. Each change carries some risk. And if we don't
> > agree on
> > > > > > the
> > > > > > > bar for the migration work, we may be having the same
> discussion
> > next
> > > > > > week.
> > > > > > > :)
> > > > > > >
> > > > > > > Ismael
> > > > > > >
> > > > > > > On Wed, May 24, 2023, 12:00 PM Josep Prat
> > 
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi there,
> > > > > > > > Is the plan described in KIP-833[1] still valid? In there it
> > states
> > > > > > that
> > > > > > > > 3.5.0 should aim at deprecation of Zookeeper, so
> conceptually,
> > the
> > > > > > path to
> > > > > > > > migrate to Kraft should be somewhat functional (in my
> > opinion). If we
> > > > > > don't
> > > > > > > > want to deprecate Zookeeper in 3.5.0, then I share Ismael's
> > opinion
> > > > > > that
> > > > > > > > t

Re: [VOTE] 3.5.0 RC0

[David Arthur]

Mickael,

Colin has approved my patch for KAFKA-15010, I'm just waiting on a build
before merging. I'll go ahead and backport the other fixes that need to
precede this one into 3.5.

-David

On Wed, May 31, 2023 at 11:52 AM Mickael Maison 
wrote:

> Hi,
>
> The issue mentioned by Greg has been fixed. As soon as the fix for
> KAFKA-15010 is merged I'll build another RC.
>
> Thanks,
> Mickael
>
> On Tue, May 30, 2023 at 10:33 AM Mickael Maison
>  wrote:
> >
> > Hi David,
> >
> > Feel free to backport the necessary fixes to 3.5.
> >
> > Thanks,
> > Mickael
> >
> > On Tue, May 30, 2023 at 10:32 AM Mickael Maison
> >  wrote:
> > >
> > > Hi Greg,
> > >
> > > Thanks for the heads up, this indeed looks like something we want in
> > > 3.5. I've replied in the PR.
> > >
> > > Mickael
> > >
> > > On Sat, May 27, 2023 at 11:44 PM David Arthur
> > >  wrote:
> > > >
> > > > Mickael, after looking more closely, I definitely think KAFKA-15010
> is a
> > > > blocker. It creates the case where the controller can totally miss a
> > > > metadata update and not write it back to ZK. Since things like
> dynamic
> > > > configs and ACLs are only read from ZK by the ZK brokers, we could
> have
> > > > significant problems while the brokers are being migrated (when some
> are
> > > > KRaft and some are ZK). E.g., ZK brokers could be totally unaware of
> an ACL
> > > > change while the KRaft brokers have it. I have a fix ready here
> > > > https://github.com/apache/kafka/pull/13758. I think we can get it
> committed
> > > > soon.
> > > >
> > > > Another blocker is KAFKA-15004 which was just merged to trunk. This
> is
> > > > another dual-write bug where new topic/broker configs will not be
> written
> > > > back to ZK by the controller.
> > > >
> > > > The fix for KAFKA-15010 has a few dependencies on fixes we made this
> past
> > > > week, so we'll need to cherry-pick a few commits. The changes are
> totally
> > > > contained within the migration area of code, so I think the risk in
> > > > including them is fairly low.
> > > >
> > > > -David
> > > >
> > > > On Thu, May 25, 2023 at 2:15 PM Greg Harris
> 
> > > > wrote:
> > > >
> > > > > Hey all,
> > > > >
> > > > > A contributor just pointed out a small but noticeable flaw in the
> > > > > implementation of KIP-581
> > > > >
> > > > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-581%3A+Value+of+optional+null+field+which+has+default+value
> > > > > which is planned for this release.
> > > > > Impact: the feature works for root values in a record, but does not
> > > > > work for any fields within structs. Fields within structs will
> > > > > continue to have their previous, backwards-compatible behavior.
> > > > > The contributor has submitted a bug-fix PR which reports the
> problem
> > > > > and does not yet have a merge-able solution, but they are actively
> > > > > responding and interested in having this fixed:
> > > > > https://github.com/apache/kafka/pull/13748
> > > > > The overall fix should be a one-liner + some unit tests. While
> this is
> > > > > not a regression, it does make the feature largely useless, as the
> > > > > majority of use-cases will be for struct fields.
> > > > >
> > > > > Thanks!
> > > > > Greg Harris
> > > > >
> > > > > On Wed, May 24, 2023 at 7:05 PM Ismael Juma 
> wrote:
> > > > > >
> > > > > > I agree the migration should be functional - it wasn't obvious
> if the
> > > > > > migration issues are edge cases or not. If they are edge cases,
> I think
> > > > > > 3.5.1 would be fine given the preview status.
> > > > > >
> > > > > > I understand that a new RC is needed, but that doesn't mean we
> should let
> > > > > > everything in. Each change carries some risk. And if we don't
> agree on
> > > > > the
> > > > > > bar for the migration work, we may be having the same discussion
> next
> > > > > week.
> > > > > > :)
> > > > > >
> > > > > > Ismael
> > > > > >
> > > > > > On Wed, May 24, 2023, 12:00 PM Josep Prat
> 
> > > > > > wrote:
> > > > > >
> > > > > > > Hi there,
> > > > > > > Is the plan described in KIP-833[1] still valid? In there it
> states
> > > > > that
> > > > > > > 3.5.0 should aim at deprecation of Zookeeper, so conceptually,
> the
> > > > > path to
> > > > > > > migrate to Kraft should be somewhat functional (in my
> opinion). If we
> > > > > don't
> > > > > > > want to deprecate Zookeeper in 3.5.0, then I share Ismael's
> opinion
> > > > > that
> > > > > > > these could be fixed in subsequent patches of 3.5.x. Just my
> 5cts.
> > > > > > >
> > > > > > > [1]:
> > > > > > >
> > > > > > >
> > > > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-833:+Mark+KRaft+as+Production+Ready#KIP833:MarkKRaftasProductionReady-Kafka3.5
> > > > > > > Best,
> > > > > > >
> > > > > > > On Wed, May 24, 2023 at 8:51 PM Ismael Juma 
> wrote:
> > > > > > >
> > > > > > > > Are all these blockers? For example, zk to kraft migration
> are is
> > > > > still
> > > > > > > in
> > > > > > > > preview - can we fix some of these in 3.5.1?
> > > > 

Re: [VOTE] 3.5.0 RC0

[Mickael Maison]

Hi,

The issue mentioned by Greg has been fixed. As soon as the fix for
KAFKA-15010 is merged I'll build another RC.

Thanks,
Mickael

On Tue, May 30, 2023 at 10:33 AM Mickael Maison
 wrote:
>
> Hi David,
>
> Feel free to backport the necessary fixes to 3.5.
>
> Thanks,
> Mickael
>
> On Tue, May 30, 2023 at 10:32 AM Mickael Maison
>  wrote:
> >
> > Hi Greg,
> >
> > Thanks for the heads up, this indeed looks like something we want in
> > 3.5. I've replied in the PR.
> >
> > Mickael
> >
> > On Sat, May 27, 2023 at 11:44 PM David Arthur
> >  wrote:
> > >
> > > Mickael, after looking more closely, I definitely think KAFKA-15010 is a
> > > blocker. It creates the case where the controller can totally miss a
> > > metadata update and not write it back to ZK. Since things like dynamic
> > > configs and ACLs are only read from ZK by the ZK brokers, we could have
> > > significant problems while the brokers are being migrated (when some are
> > > KRaft and some are ZK). E.g., ZK brokers could be totally unaware of an 
> > > ACL
> > > change while the KRaft brokers have it. I have a fix ready here
> > > https://github.com/apache/kafka/pull/13758. I think we can get it 
> > > committed
> > > soon.
> > >
> > > Another blocker is KAFKA-15004 which was just merged to trunk. This is
> > > another dual-write bug where new topic/broker configs will not be written
> > > back to ZK by the controller.
> > >
> > > The fix for KAFKA-15010 has a few dependencies on fixes we made this past
> > > week, so we'll need to cherry-pick a few commits. The changes are totally
> > > contained within the migration area of code, so I think the risk in
> > > including them is fairly low.
> > >
> > > -David
> > >
> > > On Thu, May 25, 2023 at 2:15 PM Greg Harris 
> > > wrote:
> > >
> > > > Hey all,
> > > >
> > > > A contributor just pointed out a small but noticeable flaw in the
> > > > implementation of KIP-581
> > > >
> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-581%3A+Value+of+optional+null+field+which+has+default+value
> > > > which is planned for this release.
> > > > Impact: the feature works for root values in a record, but does not
> > > > work for any fields within structs. Fields within structs will
> > > > continue to have their previous, backwards-compatible behavior.
> > > > The contributor has submitted a bug-fix PR which reports the problem
> > > > and does not yet have a merge-able solution, but they are actively
> > > > responding and interested in having this fixed:
> > > > https://github.com/apache/kafka/pull/13748
> > > > The overall fix should be a one-liner + some unit tests. While this is
> > > > not a regression, it does make the feature largely useless, as the
> > > > majority of use-cases will be for struct fields.
> > > >
> > > > Thanks!
> > > > Greg Harris
> > > >
> > > > On Wed, May 24, 2023 at 7:05 PM Ismael Juma  wrote:
> > > > >
> > > > > I agree the migration should be functional - it wasn't obvious if the
> > > > > migration issues are edge cases or not. If they are edge cases, I 
> > > > > think
> > > > > 3.5.1 would be fine given the preview status.
> > > > >
> > > > > I understand that a new RC is needed, but that doesn't mean we should 
> > > > > let
> > > > > everything in. Each change carries some risk. And if we don't agree on
> > > > the
> > > > > bar for the migration work, we may be having the same discussion next
> > > > week.
> > > > > :)
> > > > >
> > > > > Ismael
> > > > >
> > > > > On Wed, May 24, 2023, 12:00 PM Josep Prat 
> > > > > 
> > > > > wrote:
> > > > >
> > > > > > Hi there,
> > > > > > Is the plan described in KIP-833[1] still valid? In there it states
> > > > that
> > > > > > 3.5.0 should aim at deprecation of Zookeeper, so conceptually, the
> > > > path to
> > > > > > migrate to Kraft should be somewhat functional (in my opinion). If 
> > > > > > we
> > > > don't
> > > > > > want to deprecate Zookeeper in 3.5.0, then I share Ismael's opinion
> > > > that
> > > > > > these could be fixed in subsequent patches of 3.5.x. Just my 5cts.
> > > > > >
> > > > > > [1]:
> > > > > >
> > > > > >
> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-833:+Mark+KRaft+as+Production+Ready#KIP833:MarkKRaftasProductionReady-Kafka3.5
> > > > > > Best,
> > > > > >
> > > > > > On Wed, May 24, 2023 at 8:51 PM Ismael Juma  
> > > > > > wrote:
> > > > > >
> > > > > > > Are all these blockers? For example, zk to kraft migration are is
> > > > still
> > > > > > in
> > > > > > > preview - can we fix some of these in 3.5.1?
> > > > > > >
> > > > > > > Ismael
> > > > > > >
> > > > > > > On Wed, May 24, 2023, 10:22 AM Colin McCabe 
> > > > wrote:
> > > > > > >
> > > > > > > > Hi Mickael,
> > > > > > > >
> > > > > > > > Thanks for putting together this RC. Unfortunately, we've
> > > > identified
> > > > > > > > several blocker issues in this release candidate.
> > > > > > > >
> > > > > > > > KAFKA-15009: New ACLs are not written to ZK during migration
> > > > > > > > KAFKA-15007: MV is not 

Re: [VOTE] 3.5.0 RC0

[Mickael Maison]

Hi David,

Feel free to backport the necessary fixes to 3.5.

Thanks,
Mickael

On Tue, May 30, 2023 at 10:32 AM Mickael Maison
 wrote:
>
> Hi Greg,
>
> Thanks for the heads up, this indeed looks like something we want in
> 3.5. I've replied in the PR.
>
> Mickael
>
> On Sat, May 27, 2023 at 11:44 PM David Arthur
>  wrote:
> >
> > Mickael, after looking more closely, I definitely think KAFKA-15010 is a
> > blocker. It creates the case where the controller can totally miss a
> > metadata update and not write it back to ZK. Since things like dynamic
> > configs and ACLs are only read from ZK by the ZK brokers, we could have
> > significant problems while the brokers are being migrated (when some are
> > KRaft and some are ZK). E.g., ZK brokers could be totally unaware of an ACL
> > change while the KRaft brokers have it. I have a fix ready here
> > https://github.com/apache/kafka/pull/13758. I think we can get it committed
> > soon.
> >
> > Another blocker is KAFKA-15004 which was just merged to trunk. This is
> > another dual-write bug where new topic/broker configs will not be written
> > back to ZK by the controller.
> >
> > The fix for KAFKA-15010 has a few dependencies on fixes we made this past
> > week, so we'll need to cherry-pick a few commits. The changes are totally
> > contained within the migration area of code, so I think the risk in
> > including them is fairly low.
> >
> > -David
> >
> > On Thu, May 25, 2023 at 2:15 PM Greg Harris 
> > wrote:
> >
> > > Hey all,
> > >
> > > A contributor just pointed out a small but noticeable flaw in the
> > > implementation of KIP-581
> > >
> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-581%3A+Value+of+optional+null+field+which+has+default+value
> > > which is planned for this release.
> > > Impact: the feature works for root values in a record, but does not
> > > work for any fields within structs. Fields within structs will
> > > continue to have their previous, backwards-compatible behavior.
> > > The contributor has submitted a bug-fix PR which reports the problem
> > > and does not yet have a merge-able solution, but they are actively
> > > responding and interested in having this fixed:
> > > https://github.com/apache/kafka/pull/13748
> > > The overall fix should be a one-liner + some unit tests. While this is
> > > not a regression, it does make the feature largely useless, as the
> > > majority of use-cases will be for struct fields.
> > >
> > > Thanks!
> > > Greg Harris
> > >
> > > On Wed, May 24, 2023 at 7:05 PM Ismael Juma  wrote:
> > > >
> > > > I agree the migration should be functional - it wasn't obvious if the
> > > > migration issues are edge cases or not. If they are edge cases, I think
> > > > 3.5.1 would be fine given the preview status.
> > > >
> > > > I understand that a new RC is needed, but that doesn't mean we should 
> > > > let
> > > > everything in. Each change carries some risk. And if we don't agree on
> > > the
> > > > bar for the migration work, we may be having the same discussion next
> > > week.
> > > > :)
> > > >
> > > > Ismael
> > > >
> > > > On Wed, May 24, 2023, 12:00 PM Josep Prat 
> > > > wrote:
> > > >
> > > > > Hi there,
> > > > > Is the plan described in KIP-833[1] still valid? In there it states
> > > that
> > > > > 3.5.0 should aim at deprecation of Zookeeper, so conceptually, the
> > > path to
> > > > > migrate to Kraft should be somewhat functional (in my opinion). If we
> > > don't
> > > > > want to deprecate Zookeeper in 3.5.0, then I share Ismael's opinion
> > > that
> > > > > these could be fixed in subsequent patches of 3.5.x. Just my 5cts.
> > > > >
> > > > > [1]:
> > > > >
> > > > >
> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-833:+Mark+KRaft+as+Production+Ready#KIP833:MarkKRaftasProductionReady-Kafka3.5
> > > > > Best,
> > > > >
> > > > > On Wed, May 24, 2023 at 8:51 PM Ismael Juma  wrote:
> > > > >
> > > > > > Are all these blockers? For example, zk to kraft migration are is
> > > still
> > > > > in
> > > > > > preview - can we fix some of these in 3.5.1?
> > > > > >
> > > > > > Ismael
> > > > > >
> > > > > > On Wed, May 24, 2023, 10:22 AM Colin McCabe 
> > > wrote:
> > > > > >
> > > > > > > Hi Mickael,
> > > > > > >
> > > > > > > Thanks for putting together this RC. Unfortunately, we've
> > > identified
> > > > > > > several blocker issues in this release candidate.
> > > > > > >
> > > > > > > KAFKA-15009: New ACLs are not written to ZK during migration
> > > > > > > KAFKA-15007: MV is not set correctly in the MetadataPropagator in
> > > > > > > migration.
> > > > > > > KAFKA-15004: Topic config changes are not synced during zk to 
> > > > > > > kraft
> > > > > > > migration (dual-write)
> > > > > > > KAFKA-15003: TopicIdReplicaAssignment is not updated in migration
> > > > > > > (dual-write) when partitions are changed for topic
> > > > > > > KAFKA-14996: The KRaft controller should properly handle overly
> > > large
> > > > > > user
> > > > > > > operations
> > > > > > >
>

Re: [VOTE] 3.5.0 RC0

[Mickael Maison]

Hi Greg,

Thanks for the heads up, this indeed looks like something we want in
3.5. I've replied in the PR.

Mickael

On Sat, May 27, 2023 at 11:44 PM David Arthur
 wrote:
>
> Mickael, after looking more closely, I definitely think KAFKA-15010 is a
> blocker. It creates the case where the controller can totally miss a
> metadata update and not write it back to ZK. Since things like dynamic
> configs and ACLs are only read from ZK by the ZK brokers, we could have
> significant problems while the brokers are being migrated (when some are
> KRaft and some are ZK). E.g., ZK brokers could be totally unaware of an ACL
> change while the KRaft brokers have it. I have a fix ready here
> https://github.com/apache/kafka/pull/13758. I think we can get it committed
> soon.
>
> Another blocker is KAFKA-15004 which was just merged to trunk. This is
> another dual-write bug where new topic/broker configs will not be written
> back to ZK by the controller.
>
> The fix for KAFKA-15010 has a few dependencies on fixes we made this past
> week, so we'll need to cherry-pick a few commits. The changes are totally
> contained within the migration area of code, so I think the risk in
> including them is fairly low.
>
> -David
>
> On Thu, May 25, 2023 at 2:15 PM Greg Harris 
> wrote:
>
> > Hey all,
> >
> > A contributor just pointed out a small but noticeable flaw in the
> > implementation of KIP-581
> >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-581%3A+Value+of+optional+null+field+which+has+default+value
> > which is planned for this release.
> > Impact: the feature works for root values in a record, but does not
> > work for any fields within structs. Fields within structs will
> > continue to have their previous, backwards-compatible behavior.
> > The contributor has submitted a bug-fix PR which reports the problem
> > and does not yet have a merge-able solution, but they are actively
> > responding and interested in having this fixed:
> > https://github.com/apache/kafka/pull/13748
> > The overall fix should be a one-liner + some unit tests. While this is
> > not a regression, it does make the feature largely useless, as the
> > majority of use-cases will be for struct fields.
> >
> > Thanks!
> > Greg Harris
> >
> > On Wed, May 24, 2023 at 7:05 PM Ismael Juma  wrote:
> > >
> > > I agree the migration should be functional - it wasn't obvious if the
> > > migration issues are edge cases or not. If they are edge cases, I think
> > > 3.5.1 would be fine given the preview status.
> > >
> > > I understand that a new RC is needed, but that doesn't mean we should let
> > > everything in. Each change carries some risk. And if we don't agree on
> > the
> > > bar for the migration work, we may be having the same discussion next
> > week.
> > > :)
> > >
> > > Ismael
> > >
> > > On Wed, May 24, 2023, 12:00 PM Josep Prat 
> > > wrote:
> > >
> > > > Hi there,
> > > > Is the plan described in KIP-833[1] still valid? In there it states
> > that
> > > > 3.5.0 should aim at deprecation of Zookeeper, so conceptually, the
> > path to
> > > > migrate to Kraft should be somewhat functional (in my opinion). If we
> > don't
> > > > want to deprecate Zookeeper in 3.5.0, then I share Ismael's opinion
> > that
> > > > these could be fixed in subsequent patches of 3.5.x. Just my 5cts.
> > > >
> > > > [1]:
> > > >
> > > >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-833:+Mark+KRaft+as+Production+Ready#KIP833:MarkKRaftasProductionReady-Kafka3.5
> > > > Best,
> > > >
> > > > On Wed, May 24, 2023 at 8:51 PM Ismael Juma  wrote:
> > > >
> > > > > Are all these blockers? For example, zk to kraft migration are is
> > still
> > > > in
> > > > > preview - can we fix some of these in 3.5.1?
> > > > >
> > > > > Ismael
> > > > >
> > > > > On Wed, May 24, 2023, 10:22 AM Colin McCabe 
> > wrote:
> > > > >
> > > > > > Hi Mickael,
> > > > > >
> > > > > > Thanks for putting together this RC. Unfortunately, we've
> > identified
> > > > > > several blocker issues in this release candidate.
> > > > > >
> > > > > > KAFKA-15009: New ACLs are not written to ZK during migration
> > > > > > KAFKA-15007: MV is not set correctly in the MetadataPropagator in
> > > > > > migration.
> > > > > > KAFKA-15004: Topic config changes are not synced during zk to kraft
> > > > > > migration (dual-write)
> > > > > > KAFKA-15003: TopicIdReplicaAssignment is not updated in migration
> > > > > > (dual-write) when partitions are changed for topic
> > > > > > KAFKA-14996: The KRaft controller should properly handle overly
> > large
> > > > > user
> > > > > > operations
> > > > > >
> > > > > > We are working on PRs for these issues and will get them in soon,
> > we
> > > > > think!
> > > > > >
> > > > > > So unfortunately I have to leave a -1 here for RC0. Let's aim for
> > > > another
> > > > > > RC next week.
> > > > > >
> > > > > > best,
> > > > > > Colin
> > > > > >
> > > > > > On Wed, May 24, 2023, at 07:05, Mickael Maison wrote:
> > > > > > > Hi David,
> > > > > > >
>

Re: [VOTE] 3.5.0 RC0

[David Arthur]

Mickael, after looking more closely, I definitely think KAFKA-15010 is a
blocker. It creates the case where the controller can totally miss a
metadata update and not write it back to ZK. Since things like dynamic
configs and ACLs are only read from ZK by the ZK brokers, we could have
significant problems while the brokers are being migrated (when some are
KRaft and some are ZK). E.g., ZK brokers could be totally unaware of an ACL
change while the KRaft brokers have it. I have a fix ready here
https://github.com/apache/kafka/pull/13758. I think we can get it committed
soon.

Another blocker is KAFKA-15004 which was just merged to trunk. This is
another dual-write bug where new topic/broker configs will not be written
back to ZK by the controller.

The fix for KAFKA-15010 has a few dependencies on fixes we made this past
week, so we'll need to cherry-pick a few commits. The changes are totally
contained within the migration area of code, so I think the risk in
including them is fairly low.

-David

On Thu, May 25, 2023 at 2:15 PM Greg Harris 
wrote:

> Hey all,
>
> A contributor just pointed out a small but noticeable flaw in the
> implementation of KIP-581
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-581%3A+Value+of+optional+null+field+which+has+default+value
> which is planned for this release.
> Impact: the feature works for root values in a record, but does not
> work for any fields within structs. Fields within structs will
> continue to have their previous, backwards-compatible behavior.
> The contributor has submitted a bug-fix PR which reports the problem
> and does not yet have a merge-able solution, but they are actively
> responding and interested in having this fixed:
> https://github.com/apache/kafka/pull/13748
> The overall fix should be a one-liner + some unit tests. While this is
> not a regression, it does make the feature largely useless, as the
> majority of use-cases will be for struct fields.
>
> Thanks!
> Greg Harris
>
> On Wed, May 24, 2023 at 7:05 PM Ismael Juma  wrote:
> >
> > I agree the migration should be functional - it wasn't obvious if the
> > migration issues are edge cases or not. If they are edge cases, I think
> > 3.5.1 would be fine given the preview status.
> >
> > I understand that a new RC is needed, but that doesn't mean we should let
> > everything in. Each change carries some risk. And if we don't agree on
> the
> > bar for the migration work, we may be having the same discussion next
> week.
> > :)
> >
> > Ismael
> >
> > On Wed, May 24, 2023, 12:00 PM Josep Prat 
> > wrote:
> >
> > > Hi there,
> > > Is the plan described in KIP-833[1] still valid? In there it states
> that
> > > 3.5.0 should aim at deprecation of Zookeeper, so conceptually, the
> path to
> > > migrate to Kraft should be somewhat functional (in my opinion). If we
> don't
> > > want to deprecate Zookeeper in 3.5.0, then I share Ismael's opinion
> that
> > > these could be fixed in subsequent patches of 3.5.x. Just my 5cts.
> > >
> > > [1]:
> > >
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-833:+Mark+KRaft+as+Production+Ready#KIP833:MarkKRaftasProductionReady-Kafka3.5
> > > Best,
> > >
> > > On Wed, May 24, 2023 at 8:51 PM Ismael Juma  wrote:
> > >
> > > > Are all these blockers? For example, zk to kraft migration are is
> still
> > > in
> > > > preview - can we fix some of these in 3.5.1?
> > > >
> > > > Ismael
> > > >
> > > > On Wed, May 24, 2023, 10:22 AM Colin McCabe 
> wrote:
> > > >
> > > > > Hi Mickael,
> > > > >
> > > > > Thanks for putting together this RC. Unfortunately, we've
> identified
> > > > > several blocker issues in this release candidate.
> > > > >
> > > > > KAFKA-15009: New ACLs are not written to ZK during migration
> > > > > KAFKA-15007: MV is not set correctly in the MetadataPropagator in
> > > > > migration.
> > > > > KAFKA-15004: Topic config changes are not synced during zk to kraft
> > > > > migration (dual-write)
> > > > > KAFKA-15003: TopicIdReplicaAssignment is not updated in migration
> > > > > (dual-write) when partitions are changed for topic
> > > > > KAFKA-14996: The KRaft controller should properly handle overly
> large
> > > > user
> > > > > operations
> > > > >
> > > > > We are working on PRs for these issues and will get them in soon,
> we
> > > > think!
> > > > >
> > > > > So unfortunately I have to leave a -1 here for RC0. Let's aim for
> > > another
> > > > > RC next week.
> > > > >
> > > > > best,
> > > > > Colin
> > > > >
> > > > > On Wed, May 24, 2023, at 07:05, Mickael Maison wrote:
> > > > > > Hi David,
> > > > > >
> > > > > > We're already quite a bit behind schedule. If you think these
> fixes
> > > > > > are really important and can be ready in the next couple of
> days, I'm
> > > > > > open to backport them and build another release candidate. Let me
> > > know
> > > > > > once you've investigated the severity of KAFKA-15010.
> > > > > >
> > > > > > Thanks,
> > > > > > Mickael
> > > > > >
> > > > > >
> > > > > > On Tue, M

Re: [VOTE] 3.5.0 RC0

[Greg Harris]

Hey all,

A contributor just pointed out a small but noticeable flaw in the
implementation of KIP-581
https://cwiki.apache.org/confluence/display/KAFKA/KIP-581%3A+Value+of+optional+null+field+which+has+default+value
which is planned for this release.
Impact: the feature works for root values in a record, but does not
work for any fields within structs. Fields within structs will
continue to have their previous, backwards-compatible behavior.
The contributor has submitted a bug-fix PR which reports the problem
and does not yet have a merge-able solution, but they are actively
responding and interested in having this fixed:
https://github.com/apache/kafka/pull/13748
The overall fix should be a one-liner + some unit tests. While this is
not a regression, it does make the feature largely useless, as the
majority of use-cases will be for struct fields.

Thanks!
Greg Harris

On Wed, May 24, 2023 at 7:05 PM Ismael Juma  wrote:
>
> I agree the migration should be functional - it wasn't obvious if the
> migration issues are edge cases or not. If they are edge cases, I think
> 3.5.1 would be fine given the preview status.
>
> I understand that a new RC is needed, but that doesn't mean we should let
> everything in. Each change carries some risk. And if we don't agree on the
> bar for the migration work, we may be having the same discussion next week.
> :)
>
> Ismael
>
> On Wed, May 24, 2023, 12:00 PM Josep Prat 
> wrote:
>
> > Hi there,
> > Is the plan described in KIP-833[1] still valid? In there it states that
> > 3.5.0 should aim at deprecation of Zookeeper, so conceptually, the path to
> > migrate to Kraft should be somewhat functional (in my opinion). If we don't
> > want to deprecate Zookeeper in 3.5.0, then I share Ismael's opinion that
> > these could be fixed in subsequent patches of 3.5.x. Just my 5cts.
> >
> > [1]:
> >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-833:+Mark+KRaft+as+Production+Ready#KIP833:MarkKRaftasProductionReady-Kafka3.5
> > Best,
> >
> > On Wed, May 24, 2023 at 8:51 PM Ismael Juma  wrote:
> >
> > > Are all these blockers? For example, zk to kraft migration are is still
> > in
> > > preview - can we fix some of these in 3.5.1?
> > >
> > > Ismael
> > >
> > > On Wed, May 24, 2023, 10:22 AM Colin McCabe  wrote:
> > >
> > > > Hi Mickael,
> > > >
> > > > Thanks for putting together this RC. Unfortunately, we've identified
> > > > several blocker issues in this release candidate.
> > > >
> > > > KAFKA-15009: New ACLs are not written to ZK during migration
> > > > KAFKA-15007: MV is not set correctly in the MetadataPropagator in
> > > > migration.
> > > > KAFKA-15004: Topic config changes are not synced during zk to kraft
> > > > migration (dual-write)
> > > > KAFKA-15003: TopicIdReplicaAssignment is not updated in migration
> > > > (dual-write) when partitions are changed for topic
> > > > KAFKA-14996: The KRaft controller should properly handle overly large
> > > user
> > > > operations
> > > >
> > > > We are working on PRs for these issues and will get them in soon, we
> > > think!
> > > >
> > > > So unfortunately I have to leave a -1 here for RC0. Let's aim for
> > another
> > > > RC next week.
> > > >
> > > > best,
> > > > Colin
> > > >
> > > > On Wed, May 24, 2023, at 07:05, Mickael Maison wrote:
> > > > > Hi David,
> > > > >
> > > > > We're already quite a bit behind schedule. If you think these fixes
> > > > > are really important and can be ready in the next couple of days, I'm
> > > > > open to backport them and build another release candidate. Let me
> > know
> > > > > once you've investigated the severity of KAFKA-15010.
> > > > >
> > > > > Thanks,
> > > > > Mickael
> > > > >
> > > > >
> > > > > On Tue, May 23, 2023 at 6:34 PM David Arthur
> > > > >  wrote:
> > > > >>
> > > > >> Mickael, we have some migration fixes on trunk, is it okay to
> > > > cherry-pick
> > > > >> these to 3.5?
> > > > >>
> > > > >> KAFKA-15007 Use the correct MetadataVersion in MigrationPropagator
> > > > >> KAFKA-15009 Handle new ACLs in KRaft snapshot during migration
> > > > >>
> > > > >> There is another issue KAFKA-15010 that I'm also investigating to
> > > > determine
> > > > >> the impact and likelihood of seeing it in practice. This one may be
> > a
> > > > >> significant migration blocker
> > > > >>
> > > > >> Cheers,
> > > > >> David
> > > > >>
> > > > >> On Tue, May 23, 2023 at 9:57 AM Mickael Maison <
> > > > mickael.mai...@gmail.com>
> > > > >> wrote:
> > > > >>
> > > > >> > Hi Christo,
> > > > >> >
> > > > >> > Yes this is expected. This happens when nested fields also accept
> > > > >> > optional tagged fields. The tables list all fields, so they may
> > > > >> > include _tagged_fields multiple times.
> > > > >> > Clearly the layout of this page could be improved, if you have
> > ideas
> > > > >> > how to describe the protocol in a better way, feel free to share
> > > them.
> > > > >> >
> > > > >> > Thanks,
> > > > >> > Mickael
> > > > >> >
> > > > >> > On Tue, May 23, 2023 at 3:

Re: [VOTE] 3.5.0 RC0

[Ismael Juma]

I agree the migration should be functional - it wasn't obvious if the
migration issues are edge cases or not. If they are edge cases, I think
3.5.1 would be fine given the preview status.

I understand that a new RC is needed, but that doesn't mean we should let
everything in. Each change carries some risk. And if we don't agree on the
bar for the migration work, we may be having the same discussion next week.
:)

Ismael

On Wed, May 24, 2023, 12:00 PM Josep Prat 
wrote:

> Hi there,
> Is the plan described in KIP-833[1] still valid? In there it states that
> 3.5.0 should aim at deprecation of Zookeeper, so conceptually, the path to
> migrate to Kraft should be somewhat functional (in my opinion). If we don't
> want to deprecate Zookeeper in 3.5.0, then I share Ismael's opinion that
> these could be fixed in subsequent patches of 3.5.x. Just my 5cts.
>
> [1]:
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-833:+Mark+KRaft+as+Production+Ready#KIP833:MarkKRaftasProductionReady-Kafka3.5
> Best,
>
> On Wed, May 24, 2023 at 8:51 PM Ismael Juma  wrote:
>
> > Are all these blockers? For example, zk to kraft migration are is still
> in
> > preview - can we fix some of these in 3.5.1?
> >
> > Ismael
> >
> > On Wed, May 24, 2023, 10:22 AM Colin McCabe  wrote:
> >
> > > Hi Mickael,
> > >
> > > Thanks for putting together this RC. Unfortunately, we've identified
> > > several blocker issues in this release candidate.
> > >
> > > KAFKA-15009: New ACLs are not written to ZK during migration
> > > KAFKA-15007: MV is not set correctly in the MetadataPropagator in
> > > migration.
> > > KAFKA-15004: Topic config changes are not synced during zk to kraft
> > > migration (dual-write)
> > > KAFKA-15003: TopicIdReplicaAssignment is not updated in migration
> > > (dual-write) when partitions are changed for topic
> > > KAFKA-14996: The KRaft controller should properly handle overly large
> > user
> > > operations
> > >
> > > We are working on PRs for these issues and will get them in soon, we
> > think!
> > >
> > > So unfortunately I have to leave a -1 here for RC0. Let's aim for
> another
> > > RC next week.
> > >
> > > best,
> > > Colin
> > >
> > > On Wed, May 24, 2023, at 07:05, Mickael Maison wrote:
> > > > Hi David,
> > > >
> > > > We're already quite a bit behind schedule. If you think these fixes
> > > > are really important and can be ready in the next couple of days, I'm
> > > > open to backport them and build another release candidate. Let me
> know
> > > > once you've investigated the severity of KAFKA-15010.
> > > >
> > > > Thanks,
> > > > Mickael
> > > >
> > > >
> > > > On Tue, May 23, 2023 at 6:34 PM David Arthur
> > > >  wrote:
> > > >>
> > > >> Mickael, we have some migration fixes on trunk, is it okay to
> > > cherry-pick
> > > >> these to 3.5?
> > > >>
> > > >> KAFKA-15007 Use the correct MetadataVersion in MigrationPropagator
> > > >> KAFKA-15009 Handle new ACLs in KRaft snapshot during migration
> > > >>
> > > >> There is another issue KAFKA-15010 that I'm also investigating to
> > > determine
> > > >> the impact and likelihood of seeing it in practice. This one may be
> a
> > > >> significant migration blocker
> > > >>
> > > >> Cheers,
> > > >> David
> > > >>
> > > >> On Tue, May 23, 2023 at 9:57 AM Mickael Maison <
> > > mickael.mai...@gmail.com>
> > > >> wrote:
> > > >>
> > > >> > Hi Christo,
> > > >> >
> > > >> > Yes this is expected. This happens when nested fields also accept
> > > >> > optional tagged fields. The tables list all fields, so they may
> > > >> > include _tagged_fields multiple times.
> > > >> > Clearly the layout of this page could be improved, if you have
> ideas
> > > >> > how to describe the protocol in a better way, feel free to share
> > them.
> > > >> >
> > > >> > Thanks,
> > > >> > Mickael
> > > >> >
> > > >> > On Tue, May 23, 2023 at 3:50 PM Mickael Maison <
> > > mickael.mai...@gmail.com>
> > > >> > wrote:
> > > >> > >
> > > >> > > Hi Josep,
> > > >> > >
> > > >> > > Good catch! I opened a PR to fix this:
> > > >> > > https://github.com/apache/kafka-site/pull/514
> > > >> > >
> > > >> > > Thanks,
> > > >> > > Mickael
> > > >> > >
> > > >> > >
> > > >> > > On Tue, May 23, 2023 at 3:36 PM Christo Lolov <
> > > christolo...@gmail.com>
> > > >> > wrote:
> > > >> > > >
> > > >> > > > Hey Mickael!
> > > >> > > >
> > > >> > > > I am giving a +1 (non-binding) for this candidate release.
> > > >> > > >
> > > >> > > > * Built from the binary tar.gz source with Java 17 and Scala
> > 2.13
> > > on
> > > >> > Intel
> > > >> > > > (m5.4xlarge) and ARM (m6g.4xlarge) machines.
> > > >> > > > * Ran unit and integration tests on Intel and ARM machines.
> > > >> > > > * Ran the Quickstart in both Zookeeper and KRaft modes on
> Intel
> > > and ARM
> > > >> > > > machines.
> > > >> > > >
> > > >> > > > Question:
> > > >> > > > * I went through https://kafka.apache.org/35/protocol.html
> and
> > > there
> > > >> > are
> > > >> > > > quite a few repetitive __tagged_fileds fields within the s

Re: [VOTE] 3.5.0 RC0

[Colin McCabe]

Hi Ismael,

I do personally think they are blockers. But just as importantly, KAFKA-14996 
is not related to zk to kraft migration, and is a blocker. So we will need 
another RC in any case.

best,
Colin

On Wed, May 24, 2023, at 11:50, Ismael Juma wrote:
> Are all these blockers? For example, zk to kraft migration are is still in
> preview - can we fix some of these in 3.5.1?
>
> Ismael
>
> On Wed, May 24, 2023, 10:22 AM Colin McCabe  wrote:
>
>> Hi Mickael,
>>
>> Thanks for putting together this RC. Unfortunately, we've identified
>> several blocker issues in this release candidate.
>>
>> KAFKA-15009: New ACLs are not written to ZK during migration
>> KAFKA-15007: MV is not set correctly in the MetadataPropagator in
>> migration.
>> KAFKA-15004: Topic config changes are not synced during zk to kraft
>> migration (dual-write)
>> KAFKA-15003: TopicIdReplicaAssignment is not updated in migration
>> (dual-write) when partitions are changed for topic
>> KAFKA-14996: The KRaft controller should properly handle overly large user
>> operations
>>
>> We are working on PRs for these issues and will get them in soon, we think!
>>
>> So unfortunately I have to leave a -1 here for RC0. Let's aim for another
>> RC next week.
>>
>> best,
>> Colin
>>
>> On Wed, May 24, 2023, at 07:05, Mickael Maison wrote:
>> > Hi David,
>> >
>> > We're already quite a bit behind schedule. If you think these fixes
>> > are really important and can be ready in the next couple of days, I'm
>> > open to backport them and build another release candidate. Let me know
>> > once you've investigated the severity of KAFKA-15010.
>> >
>> > Thanks,
>> > Mickael
>> >
>> >
>> > On Tue, May 23, 2023 at 6:34 PM David Arthur
>> >  wrote:
>> >>
>> >> Mickael, we have some migration fixes on trunk, is it okay to
>> cherry-pick
>> >> these to 3.5?
>> >>
>> >> KAFKA-15007 Use the correct MetadataVersion in MigrationPropagator
>> >> KAFKA-15009 Handle new ACLs in KRaft snapshot during migration
>> >>
>> >> There is another issue KAFKA-15010 that I'm also investigating to
>> determine
>> >> the impact and likelihood of seeing it in practice. This one may be a
>> >> significant migration blocker
>> >>
>> >> Cheers,
>> >> David
>> >>
>> >> On Tue, May 23, 2023 at 9:57 AM Mickael Maison <
>> mickael.mai...@gmail.com>
>> >> wrote:
>> >>
>> >> > Hi Christo,
>> >> >
>> >> > Yes this is expected. This happens when nested fields also accept
>> >> > optional tagged fields. The tables list all fields, so they may
>> >> > include _tagged_fields multiple times.
>> >> > Clearly the layout of this page could be improved, if you have ideas
>> >> > how to describe the protocol in a better way, feel free to share them.
>> >> >
>> >> > Thanks,
>> >> > Mickael
>> >> >
>> >> > On Tue, May 23, 2023 at 3:50 PM Mickael Maison <
>> mickael.mai...@gmail.com>
>> >> > wrote:
>> >> > >
>> >> > > Hi Josep,
>> >> > >
>> >> > > Good catch! I opened a PR to fix this:
>> >> > > https://github.com/apache/kafka-site/pull/514
>> >> > >
>> >> > > Thanks,
>> >> > > Mickael
>> >> > >
>> >> > >
>> >> > > On Tue, May 23, 2023 at 3:36 PM Christo Lolov <
>> christolo...@gmail.com>
>> >> > wrote:
>> >> > > >
>> >> > > > Hey Mickael!
>> >> > > >
>> >> > > > I am giving a +1 (non-binding) for this candidate release.
>> >> > > >
>> >> > > > * Built from the binary tar.gz source with Java 17 and Scala 2.13
>> on
>> >> > Intel
>> >> > > > (m5.4xlarge) and ARM (m6g.4xlarge) machines.
>> >> > > > * Ran unit and integration tests on Intel and ARM machines.
>> >> > > > * Ran the Quickstart in both Zookeeper and KRaft modes on Intel
>> and ARM
>> >> > > > machines.
>> >> > > >
>> >> > > > Question:
>> >> > > > * I went through https://kafka.apache.org/35/protocol.html and
>> there
>> >> > are
>> >> > > > quite a few repetitive __tagged_fileds fields within the same
>> >> > structures -
>> >> > > > is this expected?
>> >> > > >
>> >> > > > On Tue, 23 May 2023 at 12:01, Josep Prat
>> 
>> >> > > > wrote:
>> >> > > >
>> >> > > > > Hi Mickael,
>> >> > > > > I just wanted to point out that I think the documentation you
>> >> > recently
>> >> > > > > merged on Kafka site regarding the 3.5.0 version has a problem
>> when
>> >> > it
>> >> > > > > states the version number and the sub-menu that links to
>> previous
>> >> > versions.
>> >> > > > > Left a comment here:
>> >> > > > >
>> >> >
>> https://github.com/apache/kafka-site/pull/513#pullrequestreview-1438927939
>> >> > > > >
>> >> > > > > Best,
>> >> > > > >
>> >> > > > > On Tue, May 23, 2023 at 9:29 AM Josep Prat > >
>> >> > wrote:
>> >> > > > >
>> >> > > > > > Hi Mickael,
>> >> > > > > >
>> >> > > > > > I can +1 this candidate. I verified the following:
>> >> > > > > > - Built from source with Java 17 and Scala 2.13
>> >> > > > > > - Signatures and hashes of the artifacts generated
>> >> > > > > > - Navigated through Javadoc including links to JDK classes
>> >> > > > > > - Run the unit tests
>> >> > > > > > - Run integration tests
>> >> > > > > > - Run the quicksta

Re: [VOTE] 3.5.0 RC0

[Josep Prat]

Hi there,
Is the plan described in KIP-833[1] still valid? In there it states that
3.5.0 should aim at deprecation of Zookeeper, so conceptually, the path to
migrate to Kraft should be somewhat functional (in my opinion). If we don't
want to deprecate Zookeeper in 3.5.0, then I share Ismael's opinion that
these could be fixed in subsequent patches of 3.5.x. Just my 5cts.

[1]:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-833:+Mark+KRaft+as+Production+Ready#KIP833:MarkKRaftasProductionReady-Kafka3.5
Best,

On Wed, May 24, 2023 at 8:51 PM Ismael Juma  wrote:

> Are all these blockers? For example, zk to kraft migration are is still in
> preview - can we fix some of these in 3.5.1?
>
> Ismael
>
> On Wed, May 24, 2023, 10:22 AM Colin McCabe  wrote:
>
> > Hi Mickael,
> >
> > Thanks for putting together this RC. Unfortunately, we've identified
> > several blocker issues in this release candidate.
> >
> > KAFKA-15009: New ACLs are not written to ZK during migration
> > KAFKA-15007: MV is not set correctly in the MetadataPropagator in
> > migration.
> > KAFKA-15004: Topic config changes are not synced during zk to kraft
> > migration (dual-write)
> > KAFKA-15003: TopicIdReplicaAssignment is not updated in migration
> > (dual-write) when partitions are changed for topic
> > KAFKA-14996: The KRaft controller should properly handle overly large
> user
> > operations
> >
> > We are working on PRs for these issues and will get them in soon, we
> think!
> >
> > So unfortunately I have to leave a -1 here for RC0. Let's aim for another
> > RC next week.
> >
> > best,
> > Colin
> >
> > On Wed, May 24, 2023, at 07:05, Mickael Maison wrote:
> > > Hi David,
> > >
> > > We're already quite a bit behind schedule. If you think these fixes
> > > are really important and can be ready in the next couple of days, I'm
> > > open to backport them and build another release candidate. Let me know
> > > once you've investigated the severity of KAFKA-15010.
> > >
> > > Thanks,
> > > Mickael
> > >
> > >
> > > On Tue, May 23, 2023 at 6:34 PM David Arthur
> > >  wrote:
> > >>
> > >> Mickael, we have some migration fixes on trunk, is it okay to
> > cherry-pick
> > >> these to 3.5?
> > >>
> > >> KAFKA-15007 Use the correct MetadataVersion in MigrationPropagator
> > >> KAFKA-15009 Handle new ACLs in KRaft snapshot during migration
> > >>
> > >> There is another issue KAFKA-15010 that I'm also investigating to
> > determine
> > >> the impact and likelihood of seeing it in practice. This one may be a
> > >> significant migration blocker
> > >>
> > >> Cheers,
> > >> David
> > >>
> > >> On Tue, May 23, 2023 at 9:57 AM Mickael Maison <
> > mickael.mai...@gmail.com>
> > >> wrote:
> > >>
> > >> > Hi Christo,
> > >> >
> > >> > Yes this is expected. This happens when nested fields also accept
> > >> > optional tagged fields. The tables list all fields, so they may
> > >> > include _tagged_fields multiple times.
> > >> > Clearly the layout of this page could be improved, if you have ideas
> > >> > how to describe the protocol in a better way, feel free to share
> them.
> > >> >
> > >> > Thanks,
> > >> > Mickael
> > >> >
> > >> > On Tue, May 23, 2023 at 3:50 PM Mickael Maison <
> > mickael.mai...@gmail.com>
> > >> > wrote:
> > >> > >
> > >> > > Hi Josep,
> > >> > >
> > >> > > Good catch! I opened a PR to fix this:
> > >> > > https://github.com/apache/kafka-site/pull/514
> > >> > >
> > >> > > Thanks,
> > >> > > Mickael
> > >> > >
> > >> > >
> > >> > > On Tue, May 23, 2023 at 3:36 PM Christo Lolov <
> > christolo...@gmail.com>
> > >> > wrote:
> > >> > > >
> > >> > > > Hey Mickael!
> > >> > > >
> > >> > > > I am giving a +1 (non-binding) for this candidate release.
> > >> > > >
> > >> > > > * Built from the binary tar.gz source with Java 17 and Scala
> 2.13
> > on
> > >> > Intel
> > >> > > > (m5.4xlarge) and ARM (m6g.4xlarge) machines.
> > >> > > > * Ran unit and integration tests on Intel and ARM machines.
> > >> > > > * Ran the Quickstart in both Zookeeper and KRaft modes on Intel
> > and ARM
> > >> > > > machines.
> > >> > > >
> > >> > > > Question:
> > >> > > > * I went through https://kafka.apache.org/35/protocol.html and
> > there
> > >> > are
> > >> > > > quite a few repetitive __tagged_fileds fields within the same
> > >> > structures -
> > >> > > > is this expected?
> > >> > > >
> > >> > > > On Tue, 23 May 2023 at 12:01, Josep Prat
> > 
> > >> > > > wrote:
> > >> > > >
> > >> > > > > Hi Mickael,
> > >> > > > > I just wanted to point out that I think the documentation you
> > >> > recently
> > >> > > > > merged on Kafka site regarding the 3.5.0 version has a problem
> > when
> > >> > it
> > >> > > > > states the version number and the sub-menu that links to
> > previous
> > >> > versions.
> > >> > > > > Left a comment here:
> > >> > > > >
> > >> >
> >
> https://github.com/apache/kafka-site/pull/513#pullrequestreview-1438927939
> > >> > > > >
> > >> > > > > Best,
> > >> > > > >
> > >> > > > > On Tue, May 23, 2023 at 9:29 AM Josep Prat <
> jos

Re: [VOTE] 3.5.0 RC0

[Ismael Juma]

Are all these blockers? For example, zk to kraft migration are is still in
preview - can we fix some of these in 3.5.1?

Ismael

On Wed, May 24, 2023, 10:22 AM Colin McCabe  wrote:

> Hi Mickael,
>
> Thanks for putting together this RC. Unfortunately, we've identified
> several blocker issues in this release candidate.
>
> KAFKA-15009: New ACLs are not written to ZK during migration
> KAFKA-15007: MV is not set correctly in the MetadataPropagator in
> migration.
> KAFKA-15004: Topic config changes are not synced during zk to kraft
> migration (dual-write)
> KAFKA-15003: TopicIdReplicaAssignment is not updated in migration
> (dual-write) when partitions are changed for topic
> KAFKA-14996: The KRaft controller should properly handle overly large user
> operations
>
> We are working on PRs for these issues and will get them in soon, we think!
>
> So unfortunately I have to leave a -1 here for RC0. Let's aim for another
> RC next week.
>
> best,
> Colin
>
> On Wed, May 24, 2023, at 07:05, Mickael Maison wrote:
> > Hi David,
> >
> > We're already quite a bit behind schedule. If you think these fixes
> > are really important and can be ready in the next couple of days, I'm
> > open to backport them and build another release candidate. Let me know
> > once you've investigated the severity of KAFKA-15010.
> >
> > Thanks,
> > Mickael
> >
> >
> > On Tue, May 23, 2023 at 6:34 PM David Arthur
> >  wrote:
> >>
> >> Mickael, we have some migration fixes on trunk, is it okay to
> cherry-pick
> >> these to 3.5?
> >>
> >> KAFKA-15007 Use the correct MetadataVersion in MigrationPropagator
> >> KAFKA-15009 Handle new ACLs in KRaft snapshot during migration
> >>
> >> There is another issue KAFKA-15010 that I'm also investigating to
> determine
> >> the impact and likelihood of seeing it in practice. This one may be a
> >> significant migration blocker
> >>
> >> Cheers,
> >> David
> >>
> >> On Tue, May 23, 2023 at 9:57 AM Mickael Maison <
> mickael.mai...@gmail.com>
> >> wrote:
> >>
> >> > Hi Christo,
> >> >
> >> > Yes this is expected. This happens when nested fields also accept
> >> > optional tagged fields. The tables list all fields, so they may
> >> > include _tagged_fields multiple times.
> >> > Clearly the layout of this page could be improved, if you have ideas
> >> > how to describe the protocol in a better way, feel free to share them.
> >> >
> >> > Thanks,
> >> > Mickael
> >> >
> >> > On Tue, May 23, 2023 at 3:50 PM Mickael Maison <
> mickael.mai...@gmail.com>
> >> > wrote:
> >> > >
> >> > > Hi Josep,
> >> > >
> >> > > Good catch! I opened a PR to fix this:
> >> > > https://github.com/apache/kafka-site/pull/514
> >> > >
> >> > > Thanks,
> >> > > Mickael
> >> > >
> >> > >
> >> > > On Tue, May 23, 2023 at 3:36 PM Christo Lolov <
> christolo...@gmail.com>
> >> > wrote:
> >> > > >
> >> > > > Hey Mickael!
> >> > > >
> >> > > > I am giving a +1 (non-binding) for this candidate release.
> >> > > >
> >> > > > * Built from the binary tar.gz source with Java 17 and Scala 2.13
> on
> >> > Intel
> >> > > > (m5.4xlarge) and ARM (m6g.4xlarge) machines.
> >> > > > * Ran unit and integration tests on Intel and ARM machines.
> >> > > > * Ran the Quickstart in both Zookeeper and KRaft modes on Intel
> and ARM
> >> > > > machines.
> >> > > >
> >> > > > Question:
> >> > > > * I went through https://kafka.apache.org/35/protocol.html and
> there
> >> > are
> >> > > > quite a few repetitive __tagged_fileds fields within the same
> >> > structures -
> >> > > > is this expected?
> >> > > >
> >> > > > On Tue, 23 May 2023 at 12:01, Josep Prat
> 
> >> > > > wrote:
> >> > > >
> >> > > > > Hi Mickael,
> >> > > > > I just wanted to point out that I think the documentation you
> >> > recently
> >> > > > > merged on Kafka site regarding the 3.5.0 version has a problem
> when
> >> > it
> >> > > > > states the version number and the sub-menu that links to
> previous
> >> > versions.
> >> > > > > Left a comment here:
> >> > > > >
> >> >
> https://github.com/apache/kafka-site/pull/513#pullrequestreview-1438927939
> >> > > > >
> >> > > > > Best,
> >> > > > >
> >> > > > > On Tue, May 23, 2023 at 9:29 AM Josep Prat  >
> >> > wrote:
> >> > > > >
> >> > > > > > Hi Mickael,
> >> > > > > >
> >> > > > > > I can +1 this candidate. I verified the following:
> >> > > > > > - Built from source with Java 17 and Scala 2.13
> >> > > > > > - Signatures and hashes of the artifacts generated
> >> > > > > > - Navigated through Javadoc including links to JDK classes
> >> > > > > > - Run the unit tests
> >> > > > > > - Run integration tests
> >> > > > > > - Run the quickstart in KRaft and Zookeeper mode
> >> > > > > >
> >> > > > > > Best,
> >> > > > > >
> >> > > > > > On Mon, May 22, 2023 at 5:30 PM Mickael Maison <
> >> > mimai...@apache.org>
> >> > > > > > wrote:
> >> > > > > >
> >> > > > > >> Hello Kafka users, developers and client-developers,
> >> > > > > >>
> >> > > > > >> This is the first candidate for release of Apache Kafka
> 3.5.0.
> >> > Some of
> >> > > > > t

Re: [VOTE] 3.5.0 RC0

[Colin McCabe]

Hi Mickael,

Thanks for putting together this RC. Unfortunately, we've identified several 
blocker issues in this release candidate.

KAFKA-15009: New ACLs are not written to ZK during migration
KAFKA-15007: MV is not set correctly in the MetadataPropagator in migration.
KAFKA-15004: Topic config changes are not synced during zk to kraft migration 
(dual-write)
KAFKA-15003: TopicIdReplicaAssignment is not updated in migration (dual-write) 
when partitions are changed for topic
KAFKA-14996: The KRaft controller should properly handle overly large user 
operations

We are working on PRs for these issues and will get them in soon, we think!

So unfortunately I have to leave a -1 here for RC0. Let's aim for another RC 
next week.

best,
Colin

On Wed, May 24, 2023, at 07:05, Mickael Maison wrote:
> Hi David,
>
> We're already quite a bit behind schedule. If you think these fixes
> are really important and can be ready in the next couple of days, I'm
> open to backport them and build another release candidate. Let me know
> once you've investigated the severity of KAFKA-15010.
>
> Thanks,
> Mickael
>
>
> On Tue, May 23, 2023 at 6:34 PM David Arthur
>  wrote:
>>
>> Mickael, we have some migration fixes on trunk, is it okay to cherry-pick
>> these to 3.5?
>>
>> KAFKA-15007 Use the correct MetadataVersion in MigrationPropagator
>> KAFKA-15009 Handle new ACLs in KRaft snapshot during migration
>>
>> There is another issue KAFKA-15010 that I'm also investigating to determine
>> the impact and likelihood of seeing it in practice. This one may be a
>> significant migration blocker
>>
>> Cheers,
>> David
>>
>> On Tue, May 23, 2023 at 9:57 AM Mickael Maison 
>> wrote:
>>
>> > Hi Christo,
>> >
>> > Yes this is expected. This happens when nested fields also accept
>> > optional tagged fields. The tables list all fields, so they may
>> > include _tagged_fields multiple times.
>> > Clearly the layout of this page could be improved, if you have ideas
>> > how to describe the protocol in a better way, feel free to share them.
>> >
>> > Thanks,
>> > Mickael
>> >
>> > On Tue, May 23, 2023 at 3:50 PM Mickael Maison 
>> > wrote:
>> > >
>> > > Hi Josep,
>> > >
>> > > Good catch! I opened a PR to fix this:
>> > > https://github.com/apache/kafka-site/pull/514
>> > >
>> > > Thanks,
>> > > Mickael
>> > >
>> > >
>> > > On Tue, May 23, 2023 at 3:36 PM Christo Lolov 
>> > wrote:
>> > > >
>> > > > Hey Mickael!
>> > > >
>> > > > I am giving a +1 (non-binding) for this candidate release.
>> > > >
>> > > > * Built from the binary tar.gz source with Java 17 and Scala 2.13 on
>> > Intel
>> > > > (m5.4xlarge) and ARM (m6g.4xlarge) machines.
>> > > > * Ran unit and integration tests on Intel and ARM machines.
>> > > > * Ran the Quickstart in both Zookeeper and KRaft modes on Intel and ARM
>> > > > machines.
>> > > >
>> > > > Question:
>> > > > * I went through https://kafka.apache.org/35/protocol.html and there
>> > are
>> > > > quite a few repetitive __tagged_fileds fields within the same
>> > structures -
>> > > > is this expected?
>> > > >
>> > > > On Tue, 23 May 2023 at 12:01, Josep Prat 
>> > > > wrote:
>> > > >
>> > > > > Hi Mickael,
>> > > > > I just wanted to point out that I think the documentation you
>> > recently
>> > > > > merged on Kafka site regarding the 3.5.0 version has a problem when
>> > it
>> > > > > states the version number and the sub-menu that links to previous
>> > versions.
>> > > > > Left a comment here:
>> > > > >
>> > https://github.com/apache/kafka-site/pull/513#pullrequestreview-1438927939
>> > > > >
>> > > > > Best,
>> > > > >
>> > > > > On Tue, May 23, 2023 at 9:29 AM Josep Prat 
>> > wrote:
>> > > > >
>> > > > > > Hi Mickael,
>> > > > > >
>> > > > > > I can +1 this candidate. I verified the following:
>> > > > > > - Built from source with Java 17 and Scala 2.13
>> > > > > > - Signatures and hashes of the artifacts generated
>> > > > > > - Navigated through Javadoc including links to JDK classes
>> > > > > > - Run the unit tests
>> > > > > > - Run integration tests
>> > > > > > - Run the quickstart in KRaft and Zookeeper mode
>> > > > > >
>> > > > > > Best,
>> > > > > >
>> > > > > > On Mon, May 22, 2023 at 5:30 PM Mickael Maison <
>> > mimai...@apache.org>
>> > > > > > wrote:
>> > > > > >
>> > > > > >> Hello Kafka users, developers and client-developers,
>> > > > > >>
>> > > > > >> This is the first candidate for release of Apache Kafka 3.5.0.
>> > Some of
>> > > > > the
>> > > > > >> major features include:
>> > > > > >> - KIP-710: Full support for distributed mode in dedicated
>> > MirrorMaker
>> > > > > >> 2.0 clusters
>> > > > > >> - KIP-881: Rack-aware Partition Assignment for Kafka Consumers
>> > > > > >> - KIP-887: Add ConfigProvider to make use of environment variables
>> > > > > >> - KIP-889: Versioned State Stores
>> > > > > >> - KIP-894: Use incrementalAlterConfig for syncing topic
>> > configurations
>> > > > > >> - KIP-900: KRaft kafka-storage.sh API additions to support SCRAM
>> > for
>> > > > > >>

Re: [VOTE] 3.5.0 RC0

[Mickael Maison]

Hi David,

We're already quite a bit behind schedule. If you think these fixes
are really important and can be ready in the next couple of days, I'm
open to backport them and build another release candidate. Let me know
once you've investigated the severity of KAFKA-15010.

Thanks,
Mickael


On Tue, May 23, 2023 at 6:34 PM David Arthur
 wrote:
>
> Mickael, we have some migration fixes on trunk, is it okay to cherry-pick
> these to 3.5?
>
> KAFKA-15007 Use the correct MetadataVersion in MigrationPropagator
> KAFKA-15009 Handle new ACLs in KRaft snapshot during migration
>
> There is another issue KAFKA-15010 that I'm also investigating to determine
> the impact and likelihood of seeing it in practice. This one may be a
> significant migration blocker
>
> Cheers,
> David
>
> On Tue, May 23, 2023 at 9:57 AM Mickael Maison 
> wrote:
>
> > Hi Christo,
> >
> > Yes this is expected. This happens when nested fields also accept
> > optional tagged fields. The tables list all fields, so they may
> > include _tagged_fields multiple times.
> > Clearly the layout of this page could be improved, if you have ideas
> > how to describe the protocol in a better way, feel free to share them.
> >
> > Thanks,
> > Mickael
> >
> > On Tue, May 23, 2023 at 3:50 PM Mickael Maison 
> > wrote:
> > >
> > > Hi Josep,
> > >
> > > Good catch! I opened a PR to fix this:
> > > https://github.com/apache/kafka-site/pull/514
> > >
> > > Thanks,
> > > Mickael
> > >
> > >
> > > On Tue, May 23, 2023 at 3:36 PM Christo Lolov 
> > wrote:
> > > >
> > > > Hey Mickael!
> > > >
> > > > I am giving a +1 (non-binding) for this candidate release.
> > > >
> > > > * Built from the binary tar.gz source with Java 17 and Scala 2.13 on
> > Intel
> > > > (m5.4xlarge) and ARM (m6g.4xlarge) machines.
> > > > * Ran unit and integration tests on Intel and ARM machines.
> > > > * Ran the Quickstart in both Zookeeper and KRaft modes on Intel and ARM
> > > > machines.
> > > >
> > > > Question:
> > > > * I went through https://kafka.apache.org/35/protocol.html and there
> > are
> > > > quite a few repetitive __tagged_fileds fields within the same
> > structures -
> > > > is this expected?
> > > >
> > > > On Tue, 23 May 2023 at 12:01, Josep Prat 
> > > > wrote:
> > > >
> > > > > Hi Mickael,
> > > > > I just wanted to point out that I think the documentation you
> > recently
> > > > > merged on Kafka site regarding the 3.5.0 version has a problem when
> > it
> > > > > states the version number and the sub-menu that links to previous
> > versions.
> > > > > Left a comment here:
> > > > >
> > https://github.com/apache/kafka-site/pull/513#pullrequestreview-1438927939
> > > > >
> > > > > Best,
> > > > >
> > > > > On Tue, May 23, 2023 at 9:29 AM Josep Prat 
> > wrote:
> > > > >
> > > > > > Hi Mickael,
> > > > > >
> > > > > > I can +1 this candidate. I verified the following:
> > > > > > - Built from source with Java 17 and Scala 2.13
> > > > > > - Signatures and hashes of the artifacts generated
> > > > > > - Navigated through Javadoc including links to JDK classes
> > > > > > - Run the unit tests
> > > > > > - Run integration tests
> > > > > > - Run the quickstart in KRaft and Zookeeper mode
> > > > > >
> > > > > > Best,
> > > > > >
> > > > > > On Mon, May 22, 2023 at 5:30 PM Mickael Maison <
> > mimai...@apache.org>
> > > > > > wrote:
> > > > > >
> > > > > >> Hello Kafka users, developers and client-developers,
> > > > > >>
> > > > > >> This is the first candidate for release of Apache Kafka 3.5.0.
> > Some of
> > > > > the
> > > > > >> major features include:
> > > > > >> - KIP-710: Full support for distributed mode in dedicated
> > MirrorMaker
> > > > > >> 2.0 clusters
> > > > > >> - KIP-881: Rack-aware Partition Assignment for Kafka Consumers
> > > > > >> - KIP-887: Add ConfigProvider to make use of environment variables
> > > > > >> - KIP-889: Versioned State Stores
> > > > > >> - KIP-894: Use incrementalAlterConfig for syncing topic
> > configurations
> > > > > >> - KIP-900: KRaft kafka-storage.sh API additions to support SCRAM
> > for
> > > > > >> Kafka Brokers
> > > > > >>
> > > > > >> Release notes for the 3.5.0 release:
> > > > > >>
> > https://home.apache.org/~mimaison/kafka-3.5.0-rc0/RELEASE_NOTES.html
> > > > > >>
> > > > > >> *** Please download, test and vote by Friday, May 26, 5pm PT
> > > > > >>
> > > > > >> Kafka's KEYS file containing PGP keys we use to sign the release:
> > > > > >> https://kafka.apache.org/KEYS
> > > > > >>
> > > > > >> * Release artifacts to be voted upon (source and binary):
> > > > > >> https://home.apache.org/~mimaison/kafka-3.5.0-rc0/
> > > > > >>
> > > > > >> * Maven artifacts to be voted upon:
> > > > > >>
> > https://repository.apache.org/content/groups/staging/org/apache/kafka/
> > > > > >>
> > > > > >> * Javadoc:
> > > > > >> https://home.apache.org/~mimaison/kafka-3.5.0-rc0/javadoc/
> > > > > >>
> > > > > >> * Tag to be voted upon (off 3.5 branch) is the 3.5.0 tag:
> > > > > >> https://github.com/apache/kafka/releases/tag/3.5.

Re: [VOTE] 3.5.0 RC0

[David Arthur]

Mickael, we have some migration fixes on trunk, is it okay to cherry-pick
these to 3.5?

KAFKA-15007 Use the correct MetadataVersion in MigrationPropagator
KAFKA-15009 Handle new ACLs in KRaft snapshot during migration

There is another issue KAFKA-15010 that I'm also investigating to determine
the impact and likelihood of seeing it in practice. This one may be a
significant migration blocker

Cheers,
David

On Tue, May 23, 2023 at 9:57 AM Mickael Maison 
wrote:

> Hi Christo,
>
> Yes this is expected. This happens when nested fields also accept
> optional tagged fields. The tables list all fields, so they may
> include _tagged_fields multiple times.
> Clearly the layout of this page could be improved, if you have ideas
> how to describe the protocol in a better way, feel free to share them.
>
> Thanks,
> Mickael
>
> On Tue, May 23, 2023 at 3:50 PM Mickael Maison 
> wrote:
> >
> > Hi Josep,
> >
> > Good catch! I opened a PR to fix this:
> > https://github.com/apache/kafka-site/pull/514
> >
> > Thanks,
> > Mickael
> >
> >
> > On Tue, May 23, 2023 at 3:36 PM Christo Lolov 
> wrote:
> > >
> > > Hey Mickael!
> > >
> > > I am giving a +1 (non-binding) for this candidate release.
> > >
> > > * Built from the binary tar.gz source with Java 17 and Scala 2.13 on
> Intel
> > > (m5.4xlarge) and ARM (m6g.4xlarge) machines.
> > > * Ran unit and integration tests on Intel and ARM machines.
> > > * Ran the Quickstart in both Zookeeper and KRaft modes on Intel and ARM
> > > machines.
> > >
> > > Question:
> > > * I went through https://kafka.apache.org/35/protocol.html and there
> are
> > > quite a few repetitive __tagged_fileds fields within the same
> structures -
> > > is this expected?
> > >
> > > On Tue, 23 May 2023 at 12:01, Josep Prat 
> > > wrote:
> > >
> > > > Hi Mickael,
> > > > I just wanted to point out that I think the documentation you
> recently
> > > > merged on Kafka site regarding the 3.5.0 version has a problem when
> it
> > > > states the version number and the sub-menu that links to previous
> versions.
> > > > Left a comment here:
> > > >
> https://github.com/apache/kafka-site/pull/513#pullrequestreview-1438927939
> > > >
> > > > Best,
> > > >
> > > > On Tue, May 23, 2023 at 9:29 AM Josep Prat 
> wrote:
> > > >
> > > > > Hi Mickael,
> > > > >
> > > > > I can +1 this candidate. I verified the following:
> > > > > - Built from source with Java 17 and Scala 2.13
> > > > > - Signatures and hashes of the artifacts generated
> > > > > - Navigated through Javadoc including links to JDK classes
> > > > > - Run the unit tests
> > > > > - Run integration tests
> > > > > - Run the quickstart in KRaft and Zookeeper mode
> > > > >
> > > > > Best,
> > > > >
> > > > > On Mon, May 22, 2023 at 5:30 PM Mickael Maison <
> mimai...@apache.org>
> > > > > wrote:
> > > > >
> > > > >> Hello Kafka users, developers and client-developers,
> > > > >>
> > > > >> This is the first candidate for release of Apache Kafka 3.5.0.
> Some of
> > > > the
> > > > >> major features include:
> > > > >> - KIP-710: Full support for distributed mode in dedicated
> MirrorMaker
> > > > >> 2.0 clusters
> > > > >> - KIP-881: Rack-aware Partition Assignment for Kafka Consumers
> > > > >> - KIP-887: Add ConfigProvider to make use of environment variables
> > > > >> - KIP-889: Versioned State Stores
> > > > >> - KIP-894: Use incrementalAlterConfig for syncing topic
> configurations
> > > > >> - KIP-900: KRaft kafka-storage.sh API additions to support SCRAM
> for
> > > > >> Kafka Brokers
> > > > >>
> > > > >> Release notes for the 3.5.0 release:
> > > > >>
> https://home.apache.org/~mimaison/kafka-3.5.0-rc0/RELEASE_NOTES.html
> > > > >>
> > > > >> *** Please download, test and vote by Friday, May 26, 5pm PT
> > > > >>
> > > > >> Kafka's KEYS file containing PGP keys we use to sign the release:
> > > > >> https://kafka.apache.org/KEYS
> > > > >>
> > > > >> * Release artifacts to be voted upon (source and binary):
> > > > >> https://home.apache.org/~mimaison/kafka-3.5.0-rc0/
> > > > >>
> > > > >> * Maven artifacts to be voted upon:
> > > > >>
> https://repository.apache.org/content/groups/staging/org/apache/kafka/
> > > > >>
> > > > >> * Javadoc:
> > > > >> https://home.apache.org/~mimaison/kafka-3.5.0-rc0/javadoc/
> > > > >>
> > > > >> * Tag to be voted upon (off 3.5 branch) is the 3.5.0 tag:
> > > > >> https://github.com/apache/kafka/releases/tag/3.5.0-rc0
> > > > >>
> > > > >> The PR adding the 35 documentation is not merged yet
> > > > >> (https://github.com/apache/kafka-site/pull/513)
> > > > >> * Documentation:
> > > > >> https://kafka.apache.org/35/documentation.html
> > > > >> * Protocol:
> > > > >> https://kafka.apache.org/35/protocol.html
> > > > >>
> > > > >> * Successful Jenkins builds for the 3.5 branch:
> > > > >> Unit/integration tests: Jenkins is not detecting the 3.5 branch,
> > > > >> working with INFRA to sort it out:
> > > > >> https://issues.apache.org/jira/browse/INFRA-24577
> > > > >> System tests: The build is still running, I'l

Re: [VOTE] 3.5.0 RC0

[Mickael Maison]

Hi Christo,

Yes this is expected. This happens when nested fields also accept
optional tagged fields. The tables list all fields, so they may
include _tagged_fields multiple times.
Clearly the layout of this page could be improved, if you have ideas
how to describe the protocol in a better way, feel free to share them.

Thanks,
Mickael

On Tue, May 23, 2023 at 3:50 PM Mickael Maison  wrote:
>
> Hi Josep,
>
> Good catch! I opened a PR to fix this:
> https://github.com/apache/kafka-site/pull/514
>
> Thanks,
> Mickael
>
>
> On Tue, May 23, 2023 at 3:36 PM Christo Lolov  wrote:
> >
> > Hey Mickael!
> >
> > I am giving a +1 (non-binding) for this candidate release.
> >
> > * Built from the binary tar.gz source with Java 17 and Scala 2.13 on Intel
> > (m5.4xlarge) and ARM (m6g.4xlarge) machines.
> > * Ran unit and integration tests on Intel and ARM machines.
> > * Ran the Quickstart in both Zookeeper and KRaft modes on Intel and ARM
> > machines.
> >
> > Question:
> > * I went through https://kafka.apache.org/35/protocol.html and there are
> > quite a few repetitive __tagged_fileds fields within the same structures -
> > is this expected?
> >
> > On Tue, 23 May 2023 at 12:01, Josep Prat 
> > wrote:
> >
> > > Hi Mickael,
> > > I just wanted to point out that I think the documentation you recently
> > > merged on Kafka site regarding the 3.5.0 version has a problem when it
> > > states the version number and the sub-menu that links to previous 
> > > versions.
> > > Left a comment here:
> > > https://github.com/apache/kafka-site/pull/513#pullrequestreview-1438927939
> > >
> > > Best,
> > >
> > > On Tue, May 23, 2023 at 9:29 AM Josep Prat  wrote:
> > >
> > > > Hi Mickael,
> > > >
> > > > I can +1 this candidate. I verified the following:
> > > > - Built from source with Java 17 and Scala 2.13
> > > > - Signatures and hashes of the artifacts generated
> > > > - Navigated through Javadoc including links to JDK classes
> > > > - Run the unit tests
> > > > - Run integration tests
> > > > - Run the quickstart in KRaft and Zookeeper mode
> > > >
> > > > Best,
> > > >
> > > > On Mon, May 22, 2023 at 5:30 PM Mickael Maison 
> > > > wrote:
> > > >
> > > >> Hello Kafka users, developers and client-developers,
> > > >>
> > > >> This is the first candidate for release of Apache Kafka 3.5.0. Some of
> > > the
> > > >> major features include:
> > > >> - KIP-710: Full support for distributed mode in dedicated MirrorMaker
> > > >> 2.0 clusters
> > > >> - KIP-881: Rack-aware Partition Assignment for Kafka Consumers
> > > >> - KIP-887: Add ConfigProvider to make use of environment variables
> > > >> - KIP-889: Versioned State Stores
> > > >> - KIP-894: Use incrementalAlterConfig for syncing topic configurations
> > > >> - KIP-900: KRaft kafka-storage.sh API additions to support SCRAM for
> > > >> Kafka Brokers
> > > >>
> > > >> Release notes for the 3.5.0 release:
> > > >> https://home.apache.org/~mimaison/kafka-3.5.0-rc0/RELEASE_NOTES.html
> > > >>
> > > >> *** Please download, test and vote by Friday, May 26, 5pm PT
> > > >>
> > > >> Kafka's KEYS file containing PGP keys we use to sign the release:
> > > >> https://kafka.apache.org/KEYS
> > > >>
> > > >> * Release artifacts to be voted upon (source and binary):
> > > >> https://home.apache.org/~mimaison/kafka-3.5.0-rc0/
> > > >>
> > > >> * Maven artifacts to be voted upon:
> > > >> https://repository.apache.org/content/groups/staging/org/apache/kafka/
> > > >>
> > > >> * Javadoc:
> > > >> https://home.apache.org/~mimaison/kafka-3.5.0-rc0/javadoc/
> > > >>
> > > >> * Tag to be voted upon (off 3.5 branch) is the 3.5.0 tag:
> > > >> https://github.com/apache/kafka/releases/tag/3.5.0-rc0
> > > >>
> > > >> The PR adding the 35 documentation is not merged yet
> > > >> (https://github.com/apache/kafka-site/pull/513)
> > > >> * Documentation:
> > > >> https://kafka.apache.org/35/documentation.html
> > > >> * Protocol:
> > > >> https://kafka.apache.org/35/protocol.html
> > > >>
> > > >> * Successful Jenkins builds for the 3.5 branch:
> > > >> Unit/integration tests: Jenkins is not detecting the 3.5 branch,
> > > >> working with INFRA to sort it out:
> > > >> https://issues.apache.org/jira/browse/INFRA-24577
> > > >> System tests: The build is still running, I'll send an update once I
> > > >> have the results
> > > >>
> > > >> Thanks,
> > > >> Mickael
> > > >>
> > > >
> > > >
> > > > --
> > > > [image: Aiven] 
> > > >
> > > > *Josep Prat*
> > > > Open Source Engineering Director, *Aiven*
> > > > josep.p...@aiven.io   |   +491715557497
> > > > aiven.io    |
> > > > 
> > > >    <
> > > https://twitter.com/aiven_io>
> > > > *Aiven Deutschland GmbH*
> > > > Alexanderufer 3-7, 10117 Berlin
> > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > > Amtsgericht Charlottenburg, HRB 209739 B
> > > >
> > >
> > >
> > > --
> > > [image: Aiven] 
> > >

Re: [VOTE] 3.5.0 RC0

[Mickael Maison]

Hi Josep,

Good catch! I opened a PR to fix this:
https://github.com/apache/kafka-site/pull/514

Thanks,
Mickael


On Tue, May 23, 2023 at 3:36 PM Christo Lolov  wrote:
>
> Hey Mickael!
>
> I am giving a +1 (non-binding) for this candidate release.
>
> * Built from the binary tar.gz source with Java 17 and Scala 2.13 on Intel
> (m5.4xlarge) and ARM (m6g.4xlarge) machines.
> * Ran unit and integration tests on Intel and ARM machines.
> * Ran the Quickstart in both Zookeeper and KRaft modes on Intel and ARM
> machines.
>
> Question:
> * I went through https://kafka.apache.org/35/protocol.html and there are
> quite a few repetitive __tagged_fileds fields within the same structures -
> is this expected?
>
> On Tue, 23 May 2023 at 12:01, Josep Prat 
> wrote:
>
> > Hi Mickael,
> > I just wanted to point out that I think the documentation you recently
> > merged on Kafka site regarding the 3.5.0 version has a problem when it
> > states the version number and the sub-menu that links to previous versions.
> > Left a comment here:
> > https://github.com/apache/kafka-site/pull/513#pullrequestreview-1438927939
> >
> > Best,
> >
> > On Tue, May 23, 2023 at 9:29 AM Josep Prat  wrote:
> >
> > > Hi Mickael,
> > >
> > > I can +1 this candidate. I verified the following:
> > > - Built from source with Java 17 and Scala 2.13
> > > - Signatures and hashes of the artifacts generated
> > > - Navigated through Javadoc including links to JDK classes
> > > - Run the unit tests
> > > - Run integration tests
> > > - Run the quickstart in KRaft and Zookeeper mode
> > >
> > > Best,
> > >
> > > On Mon, May 22, 2023 at 5:30 PM Mickael Maison 
> > > wrote:
> > >
> > >> Hello Kafka users, developers and client-developers,
> > >>
> > >> This is the first candidate for release of Apache Kafka 3.5.0. Some of
> > the
> > >> major features include:
> > >> - KIP-710: Full support for distributed mode in dedicated MirrorMaker
> > >> 2.0 clusters
> > >> - KIP-881: Rack-aware Partition Assignment for Kafka Consumers
> > >> - KIP-887: Add ConfigProvider to make use of environment variables
> > >> - KIP-889: Versioned State Stores
> > >> - KIP-894: Use incrementalAlterConfig for syncing topic configurations
> > >> - KIP-900: KRaft kafka-storage.sh API additions to support SCRAM for
> > >> Kafka Brokers
> > >>
> > >> Release notes for the 3.5.0 release:
> > >> https://home.apache.org/~mimaison/kafka-3.5.0-rc0/RELEASE_NOTES.html
> > >>
> > >> *** Please download, test and vote by Friday, May 26, 5pm PT
> > >>
> > >> Kafka's KEYS file containing PGP keys we use to sign the release:
> > >> https://kafka.apache.org/KEYS
> > >>
> > >> * Release artifacts to be voted upon (source and binary):
> > >> https://home.apache.org/~mimaison/kafka-3.5.0-rc0/
> > >>
> > >> * Maven artifacts to be voted upon:
> > >> https://repository.apache.org/content/groups/staging/org/apache/kafka/
> > >>
> > >> * Javadoc:
> > >> https://home.apache.org/~mimaison/kafka-3.5.0-rc0/javadoc/
> > >>
> > >> * Tag to be voted upon (off 3.5 branch) is the 3.5.0 tag:
> > >> https://github.com/apache/kafka/releases/tag/3.5.0-rc0
> > >>
> > >> The PR adding the 35 documentation is not merged yet
> > >> (https://github.com/apache/kafka-site/pull/513)
> > >> * Documentation:
> > >> https://kafka.apache.org/35/documentation.html
> > >> * Protocol:
> > >> https://kafka.apache.org/35/protocol.html
> > >>
> > >> * Successful Jenkins builds for the 3.5 branch:
> > >> Unit/integration tests: Jenkins is not detecting the 3.5 branch,
> > >> working with INFRA to sort it out:
> > >> https://issues.apache.org/jira/browse/INFRA-24577
> > >> System tests: The build is still running, I'll send an update once I
> > >> have the results
> > >>
> > >> Thanks,
> > >> Mickael
> > >>
> > >
> > >
> > > --
> > > [image: Aiven] 
> > >
> > > *Josep Prat*
> > > Open Source Engineering Director, *Aiven*
> > > josep.p...@aiven.io   |   +491715557497
> > > aiven.io    |
> > > 
> > >    <
> > https://twitter.com/aiven_io>
> > > *Aiven Deutschland GmbH*
> > > Alexanderufer 3-7, 10117 Berlin
> > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > Amtsgericht Charlottenburg, HRB 209739 B
> > >
> >
> >
> > --
> > [image: Aiven] 
> >
> > *Josep Prat*
> > Open Source Engineering Director, *Aiven*
> > josep.p...@aiven.io   |   +491715557497
> > aiven.io    |    > >
> >      <
> > https://twitter.com/aiven_io>
> > *Aiven Deutschland GmbH*
> > Alexanderufer 3-7, 10117 Berlin
> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > Amtsgericht Charlottenburg, HRB 209739 B
> >

Re: [VOTE] 3.5.0 RC0

[Christo Lolov]

Hey Mickael!

I am giving a +1 (non-binding) for this candidate release.

* Built from the binary tar.gz source with Java 17 and Scala 2.13 on Intel
(m5.4xlarge) and ARM (m6g.4xlarge) machines.
* Ran unit and integration tests on Intel and ARM machines.
* Ran the Quickstart in both Zookeeper and KRaft modes on Intel and ARM
machines.

Question:
* I went through https://kafka.apache.org/35/protocol.html and there are
quite a few repetitive __tagged_fileds fields within the same structures -
is this expected?

On Tue, 23 May 2023 at 12:01, Josep Prat 
wrote:

> Hi Mickael,
> I just wanted to point out that I think the documentation you recently
> merged on Kafka site regarding the 3.5.0 version has a problem when it
> states the version number and the sub-menu that links to previous versions.
> Left a comment here:
> https://github.com/apache/kafka-site/pull/513#pullrequestreview-1438927939
>
> Best,
>
> On Tue, May 23, 2023 at 9:29 AM Josep Prat  wrote:
>
> > Hi Mickael,
> >
> > I can +1 this candidate. I verified the following:
> > - Built from source with Java 17 and Scala 2.13
> > - Signatures and hashes of the artifacts generated
> > - Navigated through Javadoc including links to JDK classes
> > - Run the unit tests
> > - Run integration tests
> > - Run the quickstart in KRaft and Zookeeper mode
> >
> > Best,
> >
> > On Mon, May 22, 2023 at 5:30 PM Mickael Maison 
> > wrote:
> >
> >> Hello Kafka users, developers and client-developers,
> >>
> >> This is the first candidate for release of Apache Kafka 3.5.0. Some of
> the
> >> major features include:
> >> - KIP-710: Full support for distributed mode in dedicated MirrorMaker
> >> 2.0 clusters
> >> - KIP-881: Rack-aware Partition Assignment for Kafka Consumers
> >> - KIP-887: Add ConfigProvider to make use of environment variables
> >> - KIP-889: Versioned State Stores
> >> - KIP-894: Use incrementalAlterConfig for syncing topic configurations
> >> - KIP-900: KRaft kafka-storage.sh API additions to support SCRAM for
> >> Kafka Brokers
> >>
> >> Release notes for the 3.5.0 release:
> >> https://home.apache.org/~mimaison/kafka-3.5.0-rc0/RELEASE_NOTES.html
> >>
> >> *** Please download, test and vote by Friday, May 26, 5pm PT
> >>
> >> Kafka's KEYS file containing PGP keys we use to sign the release:
> >> https://kafka.apache.org/KEYS
> >>
> >> * Release artifacts to be voted upon (source and binary):
> >> https://home.apache.org/~mimaison/kafka-3.5.0-rc0/
> >>
> >> * Maven artifacts to be voted upon:
> >> https://repository.apache.org/content/groups/staging/org/apache/kafka/
> >>
> >> * Javadoc:
> >> https://home.apache.org/~mimaison/kafka-3.5.0-rc0/javadoc/
> >>
> >> * Tag to be voted upon (off 3.5 branch) is the 3.5.0 tag:
> >> https://github.com/apache/kafka/releases/tag/3.5.0-rc0
> >>
> >> The PR adding the 35 documentation is not merged yet
> >> (https://github.com/apache/kafka-site/pull/513)
> >> * Documentation:
> >> https://kafka.apache.org/35/documentation.html
> >> * Protocol:
> >> https://kafka.apache.org/35/protocol.html
> >>
> >> * Successful Jenkins builds for the 3.5 branch:
> >> Unit/integration tests: Jenkins is not detecting the 3.5 branch,
> >> working with INFRA to sort it out:
> >> https://issues.apache.org/jira/browse/INFRA-24577
> >> System tests: The build is still running, I'll send an update once I
> >> have the results
> >>
> >> Thanks,
> >> Mickael
> >>
> >
> >
> > --
> > [image: Aiven] 
> >
> > *Josep Prat*
> > Open Source Engineering Director, *Aiven*
> > josep.p...@aiven.io   |   +491715557497
> > aiven.io    |
> > 
> >    <
> https://twitter.com/aiven_io>
> > *Aiven Deutschland GmbH*
> > Alexanderufer 3-7, 10117 Berlin
> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > Amtsgericht Charlottenburg, HRB 209739 B
> >
>
>
> --
> [image: Aiven] 
>
> *Josep Prat*
> Open Source Engineering Director, *Aiven*
> josep.p...@aiven.io   |   +491715557497
> aiven.io    |    >
>      <
> https://twitter.com/aiven_io>
> *Aiven Deutschland GmbH*
> Alexanderufer 3-7, 10117 Berlin
> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> Amtsgericht Charlottenburg, HRB 209739 B
>

Re: [VOTE] 3.5.0 RC0

[Josep Prat]

Hi Mickael,
I just wanted to point out that I think the documentation you recently
merged on Kafka site regarding the 3.5.0 version has a problem when it
states the version number and the sub-menu that links to previous versions.
Left a comment here:
https://github.com/apache/kafka-site/pull/513#pullrequestreview-1438927939

Best,

On Tue, May 23, 2023 at 9:29 AM Josep Prat  wrote:

> Hi Mickael,
>
> I can +1 this candidate. I verified the following:
> - Built from source with Java 17 and Scala 2.13
> - Signatures and hashes of the artifacts generated
> - Navigated through Javadoc including links to JDK classes
> - Run the unit tests
> - Run integration tests
> - Run the quickstart in KRaft and Zookeeper mode
>
> Best,
>
> On Mon, May 22, 2023 at 5:30 PM Mickael Maison 
> wrote:
>
>> Hello Kafka users, developers and client-developers,
>>
>> This is the first candidate for release of Apache Kafka 3.5.0. Some of the
>> major features include:
>> - KIP-710: Full support for distributed mode in dedicated MirrorMaker
>> 2.0 clusters
>> - KIP-881: Rack-aware Partition Assignment for Kafka Consumers
>> - KIP-887: Add ConfigProvider to make use of environment variables
>> - KIP-889: Versioned State Stores
>> - KIP-894: Use incrementalAlterConfig for syncing topic configurations
>> - KIP-900: KRaft kafka-storage.sh API additions to support SCRAM for
>> Kafka Brokers
>>
>> Release notes for the 3.5.0 release:
>> https://home.apache.org/~mimaison/kafka-3.5.0-rc0/RELEASE_NOTES.html
>>
>> *** Please download, test and vote by Friday, May 26, 5pm PT
>>
>> Kafka's KEYS file containing PGP keys we use to sign the release:
>> https://kafka.apache.org/KEYS
>>
>> * Release artifacts to be voted upon (source and binary):
>> https://home.apache.org/~mimaison/kafka-3.5.0-rc0/
>>
>> * Maven artifacts to be voted upon:
>> https://repository.apache.org/content/groups/staging/org/apache/kafka/
>>
>> * Javadoc:
>> https://home.apache.org/~mimaison/kafka-3.5.0-rc0/javadoc/
>>
>> * Tag to be voted upon (off 3.5 branch) is the 3.5.0 tag:
>> https://github.com/apache/kafka/releases/tag/3.5.0-rc0
>>
>> The PR adding the 35 documentation is not merged yet
>> (https://github.com/apache/kafka-site/pull/513)
>> * Documentation:
>> https://kafka.apache.org/35/documentation.html
>> * Protocol:
>> https://kafka.apache.org/35/protocol.html
>>
>> * Successful Jenkins builds for the 3.5 branch:
>> Unit/integration tests: Jenkins is not detecting the 3.5 branch,
>> working with INFRA to sort it out:
>> https://issues.apache.org/jira/browse/INFRA-24577
>> System tests: The build is still running, I'll send an update once I
>> have the results
>>
>> Thanks,
>> Mickael
>>
>
>
> --
> [image: Aiven] 
>
> *Josep Prat*
> Open Source Engineering Director, *Aiven*
> josep.p...@aiven.io   |   +491715557497
> aiven.io    |
> 
>    
> *Aiven Deutschland GmbH*
> Alexanderufer 3-7, 10117 Berlin
> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> Amtsgericht Charlottenburg, HRB 209739 B
>


-- 
[image: Aiven] 

*Josep Prat*
Open Source Engineering Director, *Aiven*
josep.p...@aiven.io   |   +491715557497
aiven.io    |   
     
*Aiven Deutschland GmbH*
Alexanderufer 3-7, 10117 Berlin
Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
Amtsgericht Charlottenburg, HRB 209739 B

Re: [VOTE] 3.5.0 RC0

[Josep Prat]

Hi Mickael,

I can +1 this candidate. I verified the following:
- Built from source with Java 17 and Scala 2.13
- Signatures and hashes of the artifacts generated
- Navigated through Javadoc including links to JDK classes
- Run the unit tests
- Run integration tests
- Run the quickstart in KRaft and Zookeeper mode

Best,

On Mon, May 22, 2023 at 5:30 PM Mickael Maison  wrote:

> Hello Kafka users, developers and client-developers,
>
> This is the first candidate for release of Apache Kafka 3.5.0. Some of the
> major features include:
> - KIP-710: Full support for distributed mode in dedicated MirrorMaker
> 2.0 clusters
> - KIP-881: Rack-aware Partition Assignment for Kafka Consumers
> - KIP-887: Add ConfigProvider to make use of environment variables
> - KIP-889: Versioned State Stores
> - KIP-894: Use incrementalAlterConfig for syncing topic configurations
> - KIP-900: KRaft kafka-storage.sh API additions to support SCRAM for
> Kafka Brokers
>
> Release notes for the 3.5.0 release:
> https://home.apache.org/~mimaison/kafka-3.5.0-rc0/RELEASE_NOTES.html
>
> *** Please download, test and vote by Friday, May 26, 5pm PT
>
> Kafka's KEYS file containing PGP keys we use to sign the release:
> https://kafka.apache.org/KEYS
>
> * Release artifacts to be voted upon (source and binary):
> https://home.apache.org/~mimaison/kafka-3.5.0-rc0/
>
> * Maven artifacts to be voted upon:
> https://repository.apache.org/content/groups/staging/org/apache/kafka/
>
> * Javadoc:
> https://home.apache.org/~mimaison/kafka-3.5.0-rc0/javadoc/
>
> * Tag to be voted upon (off 3.5 branch) is the 3.5.0 tag:
> https://github.com/apache/kafka/releases/tag/3.5.0-rc0
>
> The PR adding the 35 documentation is not merged yet
> (https://github.com/apache/kafka-site/pull/513)
> * Documentation:
> https://kafka.apache.org/35/documentation.html
> * Protocol:
> https://kafka.apache.org/35/protocol.html
>
> * Successful Jenkins builds for the 3.5 branch:
> Unit/integration tests: Jenkins is not detecting the 3.5 branch,
> working with INFRA to sort it out:
> https://issues.apache.org/jira/browse/INFRA-24577
> System tests: The build is still running, I'll send an update once I
> have the results
>
> Thanks,
> Mickael
>


-- 
[image: Aiven] 

*Josep Prat*
Open Source Engineering Director, *Aiven*
josep.p...@aiven.io   |   +491715557497
aiven.io    |   
     
*Aiven Deutschland GmbH*
Alexanderufer 3-7, 10117 Berlin
Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
Amtsgericht Charlottenburg, HRB 209739 B

[VOTE] 3.5.0 RC0

[Mickael Maison]

Hello Kafka users, developers and client-developers,

This is the first candidate for release of Apache Kafka 3.5.0. Some of the
major features include:
- KIP-710: Full support for distributed mode in dedicated MirrorMaker
2.0 clusters
- KIP-881: Rack-aware Partition Assignment for Kafka Consumers
- KIP-887: Add ConfigProvider to make use of environment variables
- KIP-889: Versioned State Stores
- KIP-894: Use incrementalAlterConfig for syncing topic configurations
- KIP-900: KRaft kafka-storage.sh API additions to support SCRAM for
Kafka Brokers

Release notes for the 3.5.0 release:
https://home.apache.org/~mimaison/kafka-3.5.0-rc0/RELEASE_NOTES.html

*** Please download, test and vote by Friday, May 26, 5pm PT

Kafka's KEYS file containing PGP keys we use to sign the release:
https://kafka.apache.org/KEYS

* Release artifacts to be voted upon (source and binary):
https://home.apache.org/~mimaison/kafka-3.5.0-rc0/

* Maven artifacts to be voted upon:
https://repository.apache.org/content/groups/staging/org/apache/kafka/

* Javadoc:
https://home.apache.org/~mimaison/kafka-3.5.0-rc0/javadoc/

* Tag to be voted upon (off 3.5 branch) is the 3.5.0 tag:
https://github.com/apache/kafka/releases/tag/3.5.0-rc0

The PR adding the 35 documentation is not merged yet
(https://github.com/apache/kafka-site/pull/513)
* Documentation:
https://kafka.apache.org/35/documentation.html
* Protocol:
https://kafka.apache.org/35/protocol.html

* Successful Jenkins builds for the 3.5 branch:
Unit/integration tests: Jenkins is not detecting the 3.5 branch,
working with INFRA to sort it out:
https://issues.apache.org/jira/browse/INFRA-24577
System tests: The build is still running, I'll send an update once I
have the results

Thanks,
Mickael