Re: [VOTE] 3.7.0 RC2

Thanks Mickael, sounds good. KAFKA-160195 and KAFKA-16157 were both merged! I was made aware of one final blocker, this time for streams - KAFKA-16221. Matthias was prompt with a short hotfix PR: https://github.com/apache/kafka/pull/15315 After that goes into 3.7, I think I will be free to

Re: [VOTE] 3.7.0 RC2

Hi Stanislav, I merged https://github.com/apache/kafka/pull/15308 in trunk. I let you cherry-pick it to 3.7. I think fixing the absolute show stoppers and calling JBOD support in KRaft early access in 3.7.0 is probably the right call. Even without the bugs we found, there's still quite a few

Re: [VOTE] 3.7.0 RC2

Thanks for the work everybody. Providing a status update at the end of the week: - docs change explaining migration was merged - the blocker KAFKA-16162 was merged - the blocker KAFKA-14616

Re: [VOTE] 3.7.0 RC2

Hi Stan and Gaurav, Just to clarify some points mentioned here before KAFKA-14616: I raised a year ago so it's not related to JBOD work. It is rather a blocker bug for KRAFT in general. The PR from Colin should fix this. Am not sure if it is a blocker for 3.7 per-say as it was a major bug

Re: [VOTE] 3.7.0 RC2

Apologies, I duplicated KAFKA-16157 twice in my previous message. I intended to mention KAFKA-16195 with the PR at https://github.com/apache/kafka/pull/15262 as the second JIRA. Thanks, Gaurav > On 26 Jan 2024, at 15:34, ka...@gnarula.com wrote: > > Hi Stan, > > I wanted to share some updates

Re: [VOTE] 3.7.0 RC2

Hi Stan, I wanted to share some updates about the bugs you shared earlier. - KAFKA-14616: I've reviewed and tested the PR from Colin and have observed the fix works as intended. - KAFKA-16162: I reviewed Proven's PR and found some gaps in the proposed fix. I've therefore raised

Re: [VOTE] 3.7.0 RC2

Hi Stanislav, Thanks for bringing these JIRAs/PRs up. I'll be testing the open PRs for KAFKA-14616 and KAFKA-16162 this week and I hope to have some feedback by Friday. I gather the latter JIRA is marked as a WIP by Proven and he's away. I'll try to build on his work in the meantime. As for

Re: [VOTE] 3.7.0 RC2

Oops sorry I got confused between https://issues.apache.org/jira/browse/KAFKA-16120 (migration issue) and https://issues.apache.org/jira/browse/KAFKA-14616 (not migration issue) However, both do not seem related to JBOD based on the jira and PRs Justine On Tue, Jan 23, 2024 at 1:51 PM Justine

Re: [VOTE] 3.7.0 RC2

Hey Stan, Just wanted to clarify -- KAFKA-14616 is not particularly related to JBOD but to ZK -> KRaft migration. There were some other related migration bugs like https://issues.apache.org/jira/browse/KAFKA-16180. This may or may not influence decisions, but wanted to paint the full picture of

Re: [VOTE] 3.7.0 RC2

Hey all, I figured I'd give an update about what known blockers we have right now: - KAFKA-16101: KRaft migration rollback documentation is incorrect - https://github.com/apache/kafka/pull/15193; This need not block RC creation, but we need the docs updated so that people can test properly -

Re: [VOTE] 3.7.0 RC2

Hi all, I think I've found another blocker issue: KAFKA-16162 . The impact is after upgrading to 3.7.0, any new created topics/partitions will be unavailable. I've put my findings in the JIRA. Thanks. Luke On Thu, Jan 18, 2024 at 9:50 AM

Re: [VOTE] 3.7.0 RC2

Stan, thanks for driving this all forward! Excellent job. About StreamsStandbyTask - https://issues.apache.org/jira/browse/KAFKA-16141 StreamsUpgradeTest - https://issues.apache.org/jira/browse/KAFKA-16139 For `StreamsUpgradeTest` it was a test setup issue and should be fixed now in trunk

Re: [VOTE] 3.7.0 RC2

We have another blocking issue for the RC : https://issues.apache.org/jira/browse/KAFKA-16157. This bug is similar to https://issues.apache.org/jira/browse/KAFKA-14616. The new issue however can lead to the new topic having partitions that a producer cannot write to. --Proven On Tue, Jan 16,

Re: [VOTE] 3.7.0 RC2

I have a PR https://github.com/apache/kafka/pull/15197 for https://issues.apache.org/jira/browse/KAFKA-16131 that is building now. --Proven On Mon, Jan 15, 2024 at 5:03 AM Jakub Scholz wrote: > *> Hi Jakub,> > Thanks for trying the RC. I think what you found is a > blocker bug because it * > *>

Re: [VOTE] 3.7.0 RC2

Hi Kirk, Given we are going to have to roll a new RC anyway, and the change is so simple - might as well get it in! On Mon, Jan 15, 2024 at 8:26 PM Kirk True wrote: > Hi Stanislav, > > On Sun, Jan 14, 2024, at 1:17 PM, Stanislav Kozlovski wrote: > > Hey Kirk and Chris, > > > > Unless I'm

Re: [VOTE] 3.7.0 RC2

Hi Stanislav, On Sun, Jan 14, 2024, at 1:17 PM, Stanislav Kozlovski wrote: > Hey Kirk and Chris, > > Unless I'm missing something - KAFKALESS-16029 is simply a bad log due to > improper closing. And the PR description implies this has been present > since 3.5. While annoying, I don't see a

Re: [VOTE] 3.7.0 RC2

I wanted to circle back and confirm the integration tests + system tests, plus give an overall update regarding status. The integration tests have a fair amount of flakes. I ran and inspected 3 consecutive builds (57 , 58

Re: [VOTE] 3.7.0 RC2

*> Hi Jakub,> > Thanks for trying the RC. I think what you found is a blocker bug because it * *> will generate huge amount of logspam. I guess we didn't find it in junit tests * *> since logspam doesn't fail the automated tests. But certainly it's not suitable * *> for production. Did you file a

Re: [VOTE] 3.7.0 RC2

Hi Paolo, Colin, Let's discuss detail about this issue in the PR: https://github.com/apache/kafka/pull/15193 . On Mon, Jan 15, 2024 at 4:21 PM Paolo Patierno wrote: > Hi Colin, > I was the one raising the issue about rollback and I also already tried > what you mentioned but with no success. >

Re: [VOTE] 3.7.0 RC2

Hi Colin, I was the one raising the issue about rollback and I also already tried what you mentioned but with no success. During the first rolling, I left the zookeeper.metadata.migration.enable=true but removed controller.quorum.voters and controller.listener.names. This is what I get from the

Re: [VOTE] 3.7.0 RC2

Docs fix discussed in the thread is here: https://github.com/apache/kafka/pull/15193 best, Colin On Sun, Jan 14, 2024, at 23:56, Colin McCabe wrote: > Hi Stanislav, > > Thanks for making the first RC. The fact that it's titled RC2 is > messing with my mind a bit. I hope this doesn't make

Re: [VOTE] 3.7.0 RC2

Hi Stanislav, Thanks for making the first RC. The fact that it's titled RC2 is messing with my mind a bit. I hope this doesn't make people think that we're farther along than we are, heh. On Sun, Jan 14, 2024, at 13:54, Jakub Scholz wrote: > *> Nice catch! It does seem like we should have

Re: [VOTE] 3.7.0 RC2

*> Nice catch! It does seem like we should have gated this behind the metadata> version as KIP-858 implies. Is the cluster configured with multiple log> dirs? What is the impact of the error messages?* I did not observe any obvious impact. I was able to send and receive messages as normally. But

Re: [VOTE] 3.7.0 RC2

Hey Luke, This is an interesting problem. Given the fact that the KIP for having a 3.8 release passed, I think it weights the scale towards not calling this a blocker and expecting it to be solved in 3.7.1. It is unfortunate that it would not seem safe to migrate to KRaft in 3.7.0 (given the

Re: [VOTE] 3.7.0 RC2

Hi, I was trying the RC2 and run into the following issue ... when I run 3.7.0-RC2 KRaft cluster with metadata version set to 3.6-IV2 metadata version, I seem to be getting repeated errors like this in the controller logs: 2024-01-13 16:58:01,197 INFO [QuorumController id=0]

Re: [VOTE] 3.7.0 RC2

Hi Stanislav, I commented in the "Apache Kafka 3.7.0 Release" thread, but maybe you missed it. cross-posting here: There is a bug KAFKA-16101 reporting that "Kafka cluster will be unavailable during KRaft migration rollback". The impact for

Re: [VOTE] 3.7.0 RC2

Thanks, Kirk! @Stanislav--do you believe that this warrants a new RC? On Fri, Jan 12, 2024, 19:08 Kirk True wrote: > Hi Chris/Stanislav, > > I'm working on the 'Unable to find FetchSessionHandler' log problem > (KAFKA-16029) and have put out a draft PR ( >

Re: [VOTE] 3.7.0 RC2

Hi Chris/Stanislav, I'm working on the 'Unable to find FetchSessionHandler' log problem (KAFKA-16029) and have put out a draft PR (https://github.com/apache/kafka/pull/15186). I will use the quickstart approach as a second means to reproduce/verify while I wait for the PR's Jenkins job to

Re: [VOTE] 3.7.0 RC2

Hi Stanislav, Thanks for running this release! To verify, I: - Built from source using Java 11 with both: - - the 3.7.0-rc2 tag on GitHub - - the kafka-3.7.0-src.tgz artifact from https://home.apache.org/~stanislavkozlovski/kafka-3.7.0-rc2/ - Checked signatures and checksums - Ran the

[VOTE] 3.7.0 RC2

Hello Kafka users, developers, and client-developers, This is the first candidate for release of Apache Kafka 3.7.0. Note it's named "RC2" because I had a few "failed" RCs that I had cut/uploaded but ultimately had to scrap prior to announcing due to new blockers arriving before I could even