Not sure if all folks can see this link but the results are here: https://confluent-kafka-branch-builder-system-test-results.s3-us-west-2.amazonaws.com/system-test-kafka-branch-builder--1701739886--apache--3.6--ec5faaa90d/2023-12-04--001./2023-12-04--001./report.html
Looks like there were a few failures but most of them were due to known versioning issues with tests (ie the dev version string causes issues when trying to assert on it or versions 2.6.3, 2.7.2, and 3.3.2 are not in the tests -- see PR here where I tried to fix this https://github.com/apache/kafka/pull/14641) I believe the quota tests have also been broken for a while and I've talked to folks about fixing them in the past. Taking in the dev (3.6.1-SNAPSHOT), 2.6.3, 2.7.2, and 3.3.2, and the quota tests I think the failures look reasonable. +1 binding from me given I did the previous validations. Justine P.S. I will get to that versioning fix again soon! On Mon, Dec 4, 2023 at 2:10 PM Justine Olshan <jols...@confluent.io> wrote: > Just wanted to give an update on the system tests since I said I started > mine last week. I accidentally misconfigured my build, so I restarted them > today. I should have results and if things look good, I should be able to > make a vote early tomorrow. > > Justine > > On Mon, Dec 4, 2023 at 1:55 PM David Arthur > <david.art...@confluent.io.invalid> wrote: > >> I have a fix for KAFKA-15968 >> <https://issues.apache.org/jira/browse/KAFKA-15968> here >> https://github.com/apache/kafka/pull/14919/. After a bit of digging, I >> found that this behavior has existed in the KRaft controller since the >> beginning, so it is not a regression. >> >> Another thing I observed while investigating this is that MetadataLoader >> *does* treat CorruptRecordExceptions as fatal, which leads to the crash we >> want. RaftClient calls handleCommit serially for all its listeners, so if >> QuorumController#handleCommit is called first and does not crash, the call >> to MetadataLoader#handleCommit will crash. >> >> Considering these two factors, I don't strongly feel like we need to block >> the release for this fix. >> >> -David >> >> >> On Mon, Dec 4, 2023 at 10:49 AM David Arthur <david.art...@confluent.io> >> wrote: >> >> > Mickael, >> > >> > I just filed https://issues.apache.org/jira/browse/KAFKA-15968 while >> > investigating a log corruption issue on the controller. I'm still >> > investigating the issue to see how far back this goes, but I think this >> > could be a blocker. >> > >> > Essentially, the bug is that the controller does not treat a >> > CorruptRecordException as fatal, so the process will continue running. >> If >> > this happens on an active controller, it could corrupt the cluster's >> > metadata in general (since missing a single metadata record can cause >> lots >> > of downstream problems). >> > >> > I'll update this thread by the end of day with a stronger >> > blocker/non-blocker opinion. >> > >> > Thanks, >> > David >> > >> > >> > On Mon, Dec 4, 2023 at 6:48 AM Luke Chen <show...@gmail.com> wrote: >> > >> >> Hi Mickael: >> >> >> >> I did: >> >> 1. Validated all checksums, signatures, and hashes >> >> 2. Ran quick start for KRaft using scala 2.12 artifacts >> >> 3. Spot checked the documentation and Javadoc >> >> 4. Validated the licence file >> >> >> >> When running the validation to scala 2.12 package, I found these >> libraries >> >> are missing: (We only include scala 2.13 libraries in licence file) >> >> scala-java8-compat_2.12-1.0.2 is missing in license file >> >> scala-library-2.12.18 is missing in license file >> >> scala-logging_2.12-3.9.4 is missing in license file >> >> scala-reflect-2.12.18 is missing in license file >> >> >> >> It looks like this issue has been there for a long time, so it won't >> be a >> >> block issue for v3.6.1. >> >> >> >> +1 (binding) from me. >> >> >> >> Thank you. >> >> Luke >> >> >> >> On Sat, Dec 2, 2023 at 5:46 AM Bill Bejeck <bbej...@gmail.com> wrote: >> >> >> >> > Hi Mickael, >> >> > >> >> > I did the following: >> >> > >> >> > 1. Validated all checksums, signatures, and hashes >> >> > 2. Built from source >> >> > 3. Ran all the unit tests >> >> > 4. Spot checked the documentation and Javadoc >> >> > 5. Ran the ZK, Kraft, and Kafka Streams quickstart guides >> >> > >> >> > I did notice that the `fillDotVersion` in `js/templateData.js` needs >> >> > updating to `3.6.1`, but this is minor and should not block the >> release. >> >> > >> >> > It's a +1(binding) for me, pending the successful system test run >> >> > >> >> > Thanks, >> >> > Bill >> >> > >> >> > On Fri, Dec 1, 2023 at 1:49 PM Justine Olshan >> >> <jols...@confluent.io.invalid >> >> > > >> >> > wrote: >> >> > >> >> > > I've started a system test run on my end. >> >> > > >> >> > > Justine >> >> > > >> >> > > On Wed, Nov 29, 2023 at 1:55 PM Justine Olshan < >> jols...@confluent.io> >> >> > > wrote: >> >> > > >> >> > > > I built from source and ran a simple transactional produce >> bench. I >> >> > ran a >> >> > > > handful of unit tests as well. >> >> > > > I scanned the docs and everything looked reasonable. >> >> > > > >> >> > > > I was wondering if we got the system test results mentioned > >> System >> >> > > > tests: Still running I'll post an update once they complete. >> >> > > > >> >> > > > Justine >> >> > > > >> >> > > > On Wed, Nov 29, 2023 at 6:33 AM Mickael Maison < >> >> > mickael.mai...@gmail.com >> >> > > > >> >> > > > wrote: >> >> > > > >> >> > > >> Hi Josep, >> >> > > >> >> >> > > >> Good catch! >> >> > > >> If it's the only issue we find, I don't think we should block >> the >> >> > > >> release just to fix that. >> >> > > >> >> >> > > >> If we find another issue, I'll backport it before running >> another >> >> RC, >> >> > > >> otherwise I'll backport it once 3.6.1 is released. >> >> > > >> >> >> > > >> Thanks, >> >> > > >> Mickael >> >> > > >> >> >> > > >> On Wed, Nov 29, 2023 at 11:55 AM Josep Prat >> >> > <josep.p...@aiven.io.invalid >> >> > > > >> >> > > >> wrote: >> >> > > >> > >> >> > > >> > Hi Mickael, >> >> > > >> > This PR[1] made me realize NOTICE-binary is missing the notice >> >> for >> >> > > >> > commons-io. I don't know if it's a blocker or not. I can >> cherry >> >> pick >> >> > > the >> >> > > >> > commit to the 3.6 branch if you want. >> >> > > >> > >> >> > > >> > Best, >> >> > > >> > >> >> > > >> > >> >> > > >> > [1]: https://github.com/apache/kafka/pull/14865 >> >> > > >> > >> >> > > >> > On Tue, Nov 28, 2023 at 10:25 AM Josep Prat < >> josep.p...@aiven.io >> >> > >> >> > > >> wrote: >> >> > > >> > >> >> > > >> > > Hi Mickael, >> >> > > >> > > Thanks for running the release. It's a +1 for me >> (non-binding). >> >> > > >> > > I did the following: >> >> > > >> > > - Verified artifact's signatures and hashes >> >> > > >> > > - Checked JavaDoc (with navigation to Oracle JavaDoc) >> >> > > >> > > - Compiled source code >> >> > > >> > > - Run unit tests and integration tests >> >> > > >> > > - Run getting started with ZK and KRaft >> >> > > >> > > >> >> > > >> > > Best, >> >> > > >> > > >> >> > > >> > > On Tue, Nov 28, 2023 at 8:51 AM Kamal Chandraprakash < >> >> > > >> > > kamal.chandraprak...@gmail.com> wrote: >> >> > > >> > > >> >> > > >> > >> +1 (non-binding) >> >> > > >> > >> >> >> > > >> > >> 1. Built the source from 3.6.1-rc0 tag in scala 2.12 and >> 2.13 >> >> > > >> > >> 2. Ran all the unit and integration tests. >> >> > > >> > >> 3. Ran quickstart and verified the produce-consume on a 3 >> node >> >> > > >> cluster. >> >> > > >> > >> 4. Verified the tiered storage functionality with >> local-tiered >> >> > > >> storage. >> >> > > >> > >> >> >> > > >> > >> On Tue, Nov 28, 2023 at 12:55 AM Federico Valeri < >> >> > > >> fedeval...@gmail.com> >> >> > > >> > >> wrote: >> >> > > >> > >> >> >> > > >> > >> > Hi Mickael, >> >> > > >> > >> > >> >> > > >> > >> > - Build from source (Java 17, Scala 2.13) >> >> > > >> > >> > - Run unit and integration tests >> >> > > >> > >> > - Run custom client apps using staging artifacts >> >> > > >> > >> > >> >> > > >> > >> > +1 (non binding) >> >> > > >> > >> > >> >> > > >> > >> > Thanks >> >> > > >> > >> > Fede >> >> > > >> > >> > >> >> > > >> > >> > >> >> > > >> > >> > >> >> > > >> > >> > On Sun, Nov 26, 2023 at 11:34 AM Jakub Scholz < >> >> ja...@scholz.cz >> >> > > >> >> > > >> wrote: >> >> > > >> > >> > > >> >> > > >> > >> > > +1 non-binding. I used the staged Scala 2.13 artifacts >> and >> >> > the >> >> > > >> staged >> >> > > >> > >> > Maven >> >> > > >> > >> > > repo for my tests. All seems to work fine. >> >> > > >> > >> > > >> >> > > >> > >> > > Thanks >> >> > > >> > >> > > Jakub >> >> > > >> > >> > > >> >> > > >> > >> > > On Fri, Nov 24, 2023 at 4:37 PM Mickael Maison < >> >> > > >> mimai...@apache.org> >> >> > > >> > >> > wrote: >> >> > > >> > >> > > >> >> > > >> > >> > > > Hello Kafka users, developers and client-developers, >> >> > > >> > >> > > > >> >> > > >> > >> > > > This is the first candidate for release of Apache >> Kafka >> >> > > 3.6.1. >> >> > > >> > >> > > > >> >> > > >> > >> > > > This is a bugfix release with several fixes, >> including >> >> > > >> dependency >> >> > > >> > >> > > > version bumps for CVEs. >> >> > > >> > >> > > > >> >> > > >> > >> > > > Release notes for the 3.6.1 release: >> >> > > >> > >> > > > >> >> > > >> > >> >> >> > > >> https://home.apache.org/~mimaison/kafka-3.6.1-rc0/RELEASE_NOTES.html >> >> > > >> > >> > > > >> >> > > >> > >> > > > *** Please download, test and vote by Friday, >> December 1 >> >> > > >> > >> > > > >> >> > > >> > >> > > > Kafka's KEYS file containing PGP keys we use to sign >> the >> >> > > >> release: >> >> > > >> > >> > > > https://kafka.apache.org/KEYS >> >> > > >> > >> > > > >> >> > > >> > >> > > > * Release artifacts to be voted upon (source and >> >> binary): >> >> > > >> > >> > > > https://home.apache.org/~mimaison/kafka-3.6.1-rc0/ >> >> > > >> > >> > > > >> >> > > >> > >> > > > * Maven artifacts to be voted upon: >> >> > > >> > >> > > > >> >> > > >> > >> >> >> > > >> >> >> > >> https://repository.apache.org/content/groups/staging/org/apache/kafka/ >> >> > > >> > >> > > > >> >> > > >> > >> > > > * Javadoc: >> >> > > >> > >> > > > >> >> https://home.apache.org/~mimaison/kafka-3.6.1-rc0/javadoc/ >> >> > > >> > >> > > > >> >> > > >> > >> > > > * Tag to be voted upon (off 3.6 branch) is the 3.6.1 >> >> tag: >> >> > > >> > >> > > > >> https://github.com/apache/kafka/releases/tag/3.6.1-rc0 >> >> > > >> > >> > > > >> >> > > >> > >> > > > PR for updating docs: >> >> > > >> > >> > > > https://github.com/apache/kafka-site/pull/568 >> >> > > >> > >> > > > >> >> > > >> > >> > > > * Documentation: >> >> > > >> > >> > > > https://kafka.apache.org/36/documentation.html >> >> > > >> > >> > > > >> >> > > >> > >> > > > * Protocol: >> >> > > >> > >> > > > https://kafka.apache.org/36/protocol.html >> >> > > >> > >> > > > >> >> > > >> > >> > > > * Successful Jenkins builds for the 3.6 branch: >> >> > > >> > >> > > > Unit/integration tests: We still have a lot of flaky >> >> tests >> >> > in >> >> > > >> the >> >> > > >> > >> 3.6 >> >> > > >> > >> > > > branch. Looking at the last few 3.6 builds in >> >> > > >> > >> > > > >> >> https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.6/ >> >> > it >> >> > > >> seems >> >> > > >> > >> all >> >> > > >> > >> > > > tests passed at least once apart from >> >> > > >> > >> > > > ClusterConnectionStatesTest.testSingleIP(). There's >> >> > > >> > >> > > > https://issues.apache.org/jira/browse/KAFKA-15762 to >> >> fix >> >> > > that >> >> > > >> test. >> >> > > >> > >> > > > System tests: Still running I'll post an update once >> >> they >> >> > > >> complete. >> >> > > >> > >> > > > >> >> > > >> > >> > > > Thanks, >> >> > > >> > >> > > > Mickael >> >> > > >> > >> > > > >> >> > > >> > >> > >> >> > > >> > >> >> >> > > >> > > >> >> > > >> > > >> >> > > >> > > -- >> >> > > >> > > [image: Aiven] <https://www.aiven.io> >> >> > > >> > > >> >> > > >> > > *Josep Prat* >> >> > > >> > > Open Source Engineering Director, *Aiven* >> >> > > >> > > josep.p...@aiven.io | +491715557497 >> >> > > >> > > aiven.io <https://www.aiven.io> | >> >> > > >> > > <https://www.facebook.com/aivencloud> >> >> > > >> > > <https://www.linkedin.com/company/aiven/> < >> >> > > >> https://twitter.com/aiven_io> >> >> > > >> > > *Aiven Deutschland GmbH* >> >> > > >> > > Alexanderufer 3-7, 10117 Berlin >> >> > > >> > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen >> >> > > >> > > Amtsgericht Charlottenburg, HRB 209739 B >> >> > > >> > > >> >> > > >> > >> >> > > >> > >> >> > > >> > -- >> >> > > >> > [image: Aiven] <https://www.aiven.io> >> >> > > >> > >> >> > > >> > *Josep Prat* >> >> > > >> > Open Source Engineering Director, *Aiven* >> >> > > >> > josep.p...@aiven.io | +491715557497 >> >> > > >> > aiven.io <https://www.aiven.io> | < >> >> > > >> https://www.facebook.com/aivencloud> >> >> > > >> > <https://www.linkedin.com/company/aiven/> < >> >> > > >> https://twitter.com/aiven_io> >> >> > > >> > *Aiven Deutschland GmbH* >> >> > > >> > Alexanderufer 3-7, 10117 Berlin >> >> > > >> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen >> >> > > >> > Amtsgericht Charlottenburg, HRB 209739 B >> >> > > >> >> >> > > > >> >> > > >> >> > >> >> >> > >> > >> > -- >> > -David >> > >> >> >> -- >> -David >> >
