Re: [DISCUSS] Apache Kafka 3.5.0 release

Hi, Traditionally we used to publish an announcement on the Apache blog for major and minor releases. Unfortunately the Apache blog is going away so we can't use it anymore. There has been a few discussions on updating our website to have a proper blog (and use a markup language instead of raw

Re: [DISCUSS] Apache Kafka 3.5.0 release

Makes sense Mickael. Thanks! Ismael On Mon, May 22, 2023 at 4:56 AM Mickael Maison wrote: > Hi Ismael, > > Yes we're getting a bit behind schedule. KAFKA-14980 was a significant > regression that would have broken a lot of MirrorMaker deployments. > Chris and I were at Kafka Summit last week,

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hi Ismael, Yes we're getting a bit behind schedule. KAFKA-14980 was a significant regression that would have broken a lot of MirrorMaker deployments. Chris and I were at Kafka Summit last week, so it took a bit longer to fix. The remaining issues are all about migration and at this point I'm not

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hi Mickael, What's the plan for the first RC? In my opinion, we should really tighten the criteria for what's considered a blocker and release RC0 soon (given where we are timeline wise). Ismael On Sat, May 20, 2023, 8:15 AM Mickael Maison wrote: > Hi Matthias, > > Yes feel free to backport

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hi Matthias, Yes feel free to backport to 3.5. Thanks, Mickael On Sat, May 20, 2023 at 12:53 AM Akhilesh Chaganti wrote: > > Hi Mickael, > > I raised the blockers for AK 3.5 and raised PRs for them. All of them are > in the Zk -> KRaft migration path and critical for the Zk -> KRaft >

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hi Mickael, I raised the blockers for AK 3.5 and raised PRs for them. All of them are in the Zk -> KRaft migration path and critical for the Zk -> KRaft migration to be successful. KAFKA-15003 -- Topic metadata is not correctly synced with

Re: [DISCUSS] Apache Kafka 3.5.0 release

Mickael, we included a bug-fix into 3.5, and just discovered a critical bug in the fix itself, that would introduce a regression into 3.5. We have already a PR to fix-forward: https://github.com/apache/kafka/pull/13734 As we don't have an RC yet, I would like to cherry-pick this back to

Re: [DISCUSS] Apache Kafka 3.5.0 release

Thanks Mickael -- the fix has been merged to 3.5 now On Wed, May 10, 2023 at 1:12 AM Mickael Maison wrote: > Hi Sophie, > > Yes that's fine, thanks for letting me know! > > Mickael > > On Tue, May 9, 2023 at 10:54 PM Sophie Blee-Goldman > wrote: > > > > Hey Mickael, I noticed a bug in the new

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hi Sophie, Yes that's fine, thanks for letting me know! Mickael On Tue, May 9, 2023 at 10:54 PM Sophie Blee-Goldman wrote: > > Hey Mickael, I noticed a bug in the new versioned key-value byte store > where it's delegating to the wrong API > (copy/paste error I assume). I extracted this into

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hey Mickael, I noticed a bug in the new versioned key-value byte store where it's delegating to the wrong API (copy/paste error I assume). I extracted this into its own PR which I think should be included in the 3.5 release. The tests are still running, but it's just a one-liner so I'll merge it

Re: [DISCUSS] Apache Kafka 3.5.0 release

Thanks, Mickael. I've cherry-picked that commit to the `3.5` branch ( https://issues.apache.org/jira/browse/KAFKA-14974). Best regards, Randall On Tue, May 9, 2023 at 2:13 PM Mickael Maison wrote: > Hi Randall/Luke, > > Yes you can go ahead and merge these into 3.5. I've not started making >

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hi Randall/Luke, Yes you can go ahead and merge these into 3.5. I've not started making a release yet because: - I found a regression today in MirrorMaker: https://issues.apache.org/jira/browse/KAFKA-14980 - The 3.5 branch builder job in Jenkins has been disabled:

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hi Mickael, Since we haven't had the CR created yet, I'm thinking we should backport this doc improvement to v3.5.0 to make the doc complete. https://github.com/apache/kafka/pull/13660 What do you think? Luke On Sat, May 6, 2023 at 11:31 PM David Arthur wrote: > I resolved these three: > *

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hi, Mickael. Would it be okay for me to merge the commit from https://github.com/apache/kafka/pull/13688 to the 3.5 branch? This brings back two methods to maintain backward compatibility to the KafkaBasedLog class (https://issues.apache.org/jira/browse/KAFKA-14974), which were refactored in Feb

Re: [DISCUSS] Apache Kafka 3.5.0 release

I resolved these three: * KAFKA-14840 is merged to trunk and 3.5. I removed the 3.4.1 fix version * KAFKA-14805 is merged to trunk and 3.5 * KAFKA-14918 is merged to trunk and 3.5 KAFKA-14692 (docs issue) is still a not done Looks like KAFKA-14084 is now resolved as well (it's in trunk and 3.5).

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hi David, Thanks for the update! You still own 4 other tickets targeting 3.5: KAFKA-14840, KAFKA-14805, KAFKA-14918, KAFKA-14692. Should I move all of them to the next release? Also KAFKA-14698 and KAFKA-14084 are somewhat related to the migration. Should I move them too? Thanks, Mickael On

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hey Mickael, my two ZK migration fixes are in 3.5 now. Cheers, David On Fri, May 5, 2023 at 9:37 AM Mickael Maison wrote: > Hi Divij, > > Some dependencies (ZooKeeper, Snappy, Swagger, zstd, etc) have been > updated since 3.4. > Regarding your PR, I would have been in favor of bringing this to

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hi Divij, Some dependencies (ZooKeeper, Snappy, Swagger, zstd, etc) have been updated since 3.4. Regarding your PR, I would have been in favor of bringing this to 3.5 a couple of weeks ago, but we're now a week past code freeze for 3.5. Apart if this fixes CVEs, or significant bugs, I think we

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hey Mickael Should we consider performing an update of the minor versions of the dependencies in 3.5 (per https://github.com/apache/kafka/pull/13673)? -- Divij Vaidya On Tue, May 2, 2023 at 5:48 PM Mickael Maison wrote: > Hi Luke, > > Yes I think it makes sense to backport both to 3.5. > >

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hi Luke, Yes I think it makes sense to backport both to 3.5. Thanks, Mickael On Tue, May 2, 2023 at 11:38 AM Luke Chen wrote: > > Hi Mickael, > > There are 1 bug and 1 improvement that I'd like to backport to 3.5. > 1. A small improvement for ZK migration based on KAFKA-14840 (mentioned >

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hi Mickael, There are 1 bug and 1 improvement that I'd like to backport to 3.5. 1. A small improvement for ZK migration based on KAFKA-14840 (mentioned above in David's mail). PR is already merged to trunk. https://issues.apache.org/jira/browse/KAFKA-14909 2. A bug will cause the KRaft

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hi David, Yes you can backport these to 3.5. Let me know when you are done. Thanks, Mickael On Thu, Apr 27, 2023 at 9:02 PM David Arthur wrote: > > Hey Mickael, > > I have one major ZK migration improvement (KAFKA-14805) that landed in > trunk this week that I'd like to merge to 3.5 (once we

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hey Mickael, I have one major ZK migration improvement (KAFKA-14805) that landed in trunk this week that I'd like to merge to 3.5 (once we fix some test failures it introduced). After that, I have another PR for KAFKA-14840 which is essentially a huge bug in the ZK migration logic that needs to

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hi Justine, That makes sense. Feel free to revert that commit in 3.5. Thanks, Mickael On Mon, Apr 24, 2023 at 7:16 PM Mickael Maison wrote: > > Hi Josep, > > Thanks for letting me know! > > On Mon, Apr 24, 2023 at 6:58 PM Justine Olshan > wrote: > > > > Hey Mickael, > > > > I've just opened

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hi Josep, Thanks for letting me know! On Mon, Apr 24, 2023 at 6:58 PM Justine Olshan wrote: > > Hey Mickael, > > I've just opened a blocker to revert KAFKA-14561 in 3.5. There are a few > blocker bugs that I don't think I can fix before the code freeze, so I > think for the quality of the

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hey Mickael, I've just opened a blocker to revert KAFKA-14561 in 3.5. There are a few blocker bugs that I don't think I can fix before the code freeze, so I think for the quality of the release, we should just revert the commit. Thanks, Justine On Fri, Apr 21, 2023 at 1:23 PM Josep Prat wrote:

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hi Mickael, Greg Harris managed to fix a flaky test in https://github.com/apache/kafka/pull/13575, I cherry-picked it to the 3.5 (and 3.4) branch. I updated the Jira to reflect that is now fixed on 3.5.0 as well as 3.6.0. Let me know if I forgot anything. Best, On Fri, Apr 21, 2023 at 3:44 PM

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hi, Just a quick reminder that code freeze is next week. We still have 27 JIRAs targeting 3.5 [0] including quite a few bugs and flaky test issues opened recently. If you have time, take one of these items or help with the reviews. I'll send another update next once we've entered code freeze.

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hi Ron, Yes feel free to merge that fix. Thanks for letting me know! Mickael On Thu, Apr 20, 2023 at 8:15 PM Ron Dagostino wrote: > > Hi Mickael. I would like to merge > https://github.com/apache/kafka/pull/13532 (KAFKA-14887: No shutdown > for ZK session expiration in feature processing) to

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hi Mickael. I would like to merge https://github.com/apache/kafka/pull/13532 (KAFKA-14887: No shutdown for ZK session expiration in feature processing) to the 3.5 branch. It is a very small and focused fix that can cause unexpected broker shutdowns when there is instability in the connectivity to

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hi David, Thanks for the update. I've marked KAFKA-14869 as fixed in 3.5.0, I guess you'll only resolve this ticket once you merge the backports to earlier branches. The ticket will have to be resolved to run the release but that should leave you enough time. Thanks, Mickael On Tue, Apr 18,

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hi Mickael, FYI - I just merged the two PRs for KIP-915 to trunk/3.5. We are all good. Cheers, David On Mon, Apr 17, 2023 at 5:10 PM Mickael Maison wrote: > Hi Chris, > > I was looking at that just now! As you said, the PRs merged provide > some functionality so I think it's fine to deliver

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hi Chris, I was looking at that just now! As you said, the PRs merged provide some functionality so I think it's fine to deliver the KIP across 2 releases. I left a comment in https://issues.apache.org/jira/browse/KAFKA-14876 to document what's in 3.5. Thanks, Mickael On Mon, Apr 17, 2023 at

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hi Mickael, It looks like we missed the feature freeze cutoff for part but not all of KIP-875 [1]. The features we've been able to merge so far are the new STOPPED state for connectors [2] and the API for reading offsets [3]. The features we have not been able to merge yet are the APIs for

Re: [DISCUSS] Apache Kafka 3.5.0 release

Thanks a lot! On 4/14/23 5:32 AM, Mickael Maison wrote: Hi Matthias, I merged the PR before cutting the 3.5 branch. Thanks, Mickael On Fri, Apr 14, 2023 at 2:31 PM Mickael Maison wrote: Hi David, I've created the 3.5 branch. Feel free to cherry pick these 2 commits when they are ready.

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hi Satish, Thanks for the update! If the changes are complex and not enough to have the feature in early access, let's defer them to the next release. I've updated the JIRA and moved the KIP to the postponed section for 3.5. Thanks, Mickael On Fri, Apr 14, 2023 at 2:32 PM Mickael Maison wrote:

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hi Matthias, I merged the PR before cutting the 3.5 branch. Thanks, Mickael On Fri, Apr 14, 2023 at 2:31 PM Mickael Maison wrote: > > Hi David, > > I've created the 3.5 branch. Feel free to cherry pick these 2 commits > when they are ready. > > Thanks, > Mickael > > On Fri, Apr 14, 2023 at

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hi David, I've created the 3.5 branch. Feel free to cherry pick these 2 commits when they are ready. Thanks, Mickael On Fri, Apr 14, 2023 at 11:23 AM Satish Duggana wrote: > > Thanks Luke for helping with the reviews and adding a few tests in a > couple of PRs. > > Hi Mickael, > I raised 3 PRs

Re: [DISCUSS] Apache Kafka 3.5.0 release

Thanks Luke for helping with the reviews and adding a few tests in a couple of PRs. Hi Mickael, I raised 3 PRs recently for tiered storage, one is merged. The other 2 PRs are in the critical path of non-tiered storage changes also. Especially, in consumer fetch and retention cleanup paths. These

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hey Mickael, we have one open PR for KIP-914 left. Would be great if you could merge it before cutting the 3.5 branch. If you don't want to merge it and prefer that I cherry-pick it to 3.5 branch later, also works for me. I did close the ticket already as resolved. It's just a minor change

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hi Mickael, Thanks for the heads up. As raised by Jeff earlier in this thread, we would like to get the two small patches [1][2] for KIP-915 in 3.5. The PRs are in review and I should be able to merge them in the next few days. I will cherry-pick them to the release branch in your create it in

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hi Luke, Thanks for the heads up. This would be great to get Tiered Storage in Early Access. Let me know if you can't get everything done this week. Mickael On Thu, Apr 13, 2023 at 12:54 PM Mickael Maison wrote: > > Hi, > > We've now reached feature freeze for 3.5.0. From now on, only bug >

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hi, We've now reached feature freeze for 3.5.0. From now on, only bug fixes and changes related to stabilizing the release should be merged. I plan to create the release branch tomorrow (Friday 14). After this point, you'll have to cherry pick changes to the release branch when merging a PR.

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hi Mickael, I'd like to ask for some more days for KIP-405 tiered storage PRs to include in v3.5. Currently, we have 1 PR under reviewing ( https://github.com/apache/kafka/pull/13535), and 1 PR soon will be opened for review. After these 2 PRs merged, we can have an "Early Access" for tiered

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hi Mickael, Thank you. Best, Jeff On Wed, Apr 5, 2023 at 1:28 PM Mickael Maison wrote: > Hi Jeff, > > Ok, I've added KIP-915 to the release plan. > > Thanks, > Mickael > > On Wed, Apr 5, 2023 at 6:48 PM Jeff Kim > wrote: > > > > Hi Mickael, > > > > I would like to bring up that KIP-915

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hi Jeff, Ok, I've added KIP-915 to the release plan. Thanks, Mickael On Wed, Apr 5, 2023 at 6:48 PM Jeff Kim wrote: > > Hi Mickael, > > I would like to bring up that KIP-915 proposes to patch 3.5 > although it missed the KIP freeze date. If the patch is done before the > feature freeze date,

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hi Mickael, I would like to bring up that KIP-915 proposes to patch 3.5 although it missed the KIP freeze date. If the patch is done before the feature freeze date, 4/13, would this be acceptable? If so, should this be added to the 3.5.0 Release Plan wiki? Best, Jeff On Mon, Mar 27, 2023 at

Re: [DISCUSS] Apache Kafka 3.5.0 release

Mickael, Just wanted to let you know that I will not be including KIP-898 in the 3.5.0 release. I think the change needed is not reviewable before the feature freeze deadline, and would take resources away from other more necessary changes. Thanks! Greg On Thu, Mar 23, 2023 at 9:01 AM Chia-Ping

Re: [DISCUSS] Apache Kafka 3.5.0 release

> If you have a KIP that is accepted, make sure it is listed in > https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+3.5.0 > and that it's status is accurate. Thanks for the reminder. Have added KIP-641 to the list. Thanks, Chia-Ping > Mickael Maison 於 2023年3月23日 下午11:51 寫道： > >

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hi Christo, Thanks for opening KIP-902. Since the timeline to Kafka 4.0 is still unclear, I think this KIP makes sense overall. Kafka should not depend on end of life dependencies. However the timing is unfortunate and considering the risks of upgrading ZooKeeper I think it's a bit too late to

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hi all, KIP Freeze was yesterday. The next milestone is feature freeze on April 12. If you have a KIP that is accepted, make sure it is listed in https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+3.5.0 and that it's status is accurate. Thanks, Mickael On Fri, Mar 17, 2023 at 6:22 

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hello! What would you suggest as the best way to get more eyes on KIP-902 as I would like it to be included it in 3.5.0? Best, Christo > On 16 Mar 2023, at 10:33, Mickael Maison wrote: > > Hi, > > This is a reminder that KIP freeze is less than a week away (22 Mar). > For a KIP to be

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hi, This is a reminder that KIP freeze is less than a week away (22 Mar). For a KIP to be considered for this release, it must be voted and accepted by that date. Feature freeze will be 3 weeks after this, so if you want KIPs or other significant changes in the release, please get them ready

Re: [DISCUSS] Apache Kafka 3.5.0 release

Thanks! Ismael On Tue, Feb 14, 2023 at 1:07 PM Mickael Maison wrote: > Hi Ismael, > > Good call. I shifted all dates by 2 weeks and moved them to Wednesdays. > > Thanks, > Mickael > > On Tue, Feb 14, 2023 at 6:01 PM Ismael Juma wrote: > > > > Thanks Mickael. A couple of notes: > > > > 1. We

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hi Ismael, Good call. I shifted all dates by 2 weeks and moved them to Wednesdays. Thanks, Mickael On Tue, Feb 14, 2023 at 6:01 PM Ismael Juma wrote: > > Thanks Mickael. A couple of notes: > > 1. We typically choose a Wednesday for the various freeze dates - there are > often 1-2 day slips and

Re: [DISCUSS] Apache Kafka 3.5.0 release

Thanks Mickael. A couple of notes: 1. We typically choose a Wednesday for the various freeze dates - there are often 1-2 day slips and it's better if that doesn't require people working through the weekend. 2. Looks like we're over a month later compared to the equivalent release last year (

Re: [DISCUSS] Apache Kafka 3.5.0 release

Thanks for the work Mickael! On Tue, Feb 14, 2023 at 5:01 PM Mickael Maison wrote: > Hi, > > I've created a release plan for 3.5.0 in the wiki: > https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+3.5.0 > > Current dates are: > 1) KIP Freeze: 07 Apr 2023 > 2) Feature Freeze: 27 Apr

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hi, I've created a release plan for 3.5.0 in the wiki: https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+3.5.0 Current dates are: 1) KIP Freeze: 07 Apr 2023 2) Feature Freeze: 27 Apr 2023 3) Code Freeze: 11 May 2023 Please take a look at the plan. Let me know if there are other

Re: [DISCUSS] Apache Kafka 3.5.0 release

Thanks, Mickael! Best, Bruno On 09.02.23 03:15, Luke Chen wrote: Hi Mickael, Thanks for volunteering! Luke On Thu, Feb 9, 2023 at 6:23 AM Chris Egerton wrote: Thanks for volunteering, Mickael! On Wed, Feb 8, 2023 at 1:12 PM José Armando García Sancio wrote: Thanks for volunteering

Re: [DISCUSS] Apache Kafka 3.5.0 release

Hi Mickael, Thanks for volunteering! Luke On Thu, Feb 9, 2023 at 6:23 AM Chris Egerton wrote: > Thanks for volunteering, Mickael! > > On Wed, Feb 8, 2023 at 1:12 PM José Armando García Sancio > wrote: > > > Thanks for volunteering Mickael. > > > > -- > > -José > > >

Re: [DISCUSS] Apache Kafka 3.5.0 release

Thanks for volunteering, Mickael! On Wed, Feb 8, 2023 at 1:12 PM José Armando García Sancio wrote: > Thanks for volunteering Mickael. > > -- > -José >

Re: [DISCUSS] Apache Kafka 3.5.0 release

Thanks for volunteering Mickael. -- -José

Re: [DISCUSS] Apache Kafka 3.5.0 release

Thanks for volunteering Mickael! On Wed, Feb 8, 2023 at 4:23 PM Ismael Juma wrote: > Thanks Mickael! > > Ismael > > On Wed, Feb 8, 2023 at 7:06 AM Mickael Maison > wrote: > > > Hi, > > > > I'd like to volunteer as release manager for the next release, Apache > > Kafka 3.5.0. > > If there are

Re: [DISCUSS] Apache Kafka 3.5.0 release

Thanks Mickael! Ismael On Wed, Feb 8, 2023 at 7:06 AM Mickael Maison wrote: > Hi, > > I'd like to volunteer as release manager for the next release, Apache > Kafka 3.5.0. > If there are no objections, I'll start building a release plan in the > wiki in the next couple of weeks. > > Thanks, >

[DISCUSS] Apache Kafka 3.5.0 release

Hi, I'd like to volunteer as release manager for the next release, Apache Kafka 3.5.0. If there are no objections, I'll start building a release plan in the wiki in the next couple of weeks. Thanks, Mickael