Just wanted to give an update on the system tests since I said I started mine last week. I accidentally misconfigured my build, so I restarted them today. I should have results and if things look good, I should be able to make a vote early tomorrow.
Justine On Mon, Dec 4, 2023 at 1:55 PM David Arthur <david.art...@confluent.io.invalid> wrote: > I have a fix for KAFKA-15968 > <https://issues.apache.org/jira/browse/KAFKA-15968> here > https://github.com/apache/kafka/pull/14919/. After a bit of digging, I > found that this behavior has existed in the KRaft controller since the > beginning, so it is not a regression. > > Another thing I observed while investigating this is that MetadataLoader > *does* treat CorruptRecordExceptions as fatal, which leads to the crash we > want. RaftClient calls handleCommit serially for all its listeners, so if > QuorumController#handleCommit is called first and does not crash, the call > to MetadataLoader#handleCommit will crash. > > Considering these two factors, I don't strongly feel like we need to block > the release for this fix. > > -David > > > On Mon, Dec 4, 2023 at 10:49 AM David Arthur <david.art...@confluent.io> > wrote: > > > Mickael, > > > > I just filed https://issues.apache.org/jira/browse/KAFKA-15968 while > > investigating a log corruption issue on the controller. I'm still > > investigating the issue to see how far back this goes, but I think this > > could be a blocker. > > > > Essentially, the bug is that the controller does not treat a > > CorruptRecordException as fatal, so the process will continue running. If > > this happens on an active controller, it could corrupt the cluster's > > metadata in general (since missing a single metadata record can cause > lots > > of downstream problems). > > > > I'll update this thread by the end of day with a stronger > > blocker/non-blocker opinion. > > > > Thanks, > > David > > > > > > On Mon, Dec 4, 2023 at 6:48 AM Luke Chen <show...@gmail.com> wrote: > > > >> Hi Mickael: > >> > >> I did: > >> 1. Validated all checksums, signatures, and hashes > >> 2. Ran quick start for KRaft using scala 2.12 artifacts > >> 3. Spot checked the documentation and Javadoc > >> 4. Validated the licence file > >> > >> When running the validation to scala 2.12 package, I found these > libraries > >> are missing: (We only include scala 2.13 libraries in licence file) > >> scala-java8-compat_2.12-1.0.2 is missing in license file > >> scala-library-2.12.18 is missing in license file > >> scala-logging_2.12-3.9.4 is missing in license file > >> scala-reflect-2.12.18 is missing in license file > >> > >> It looks like this issue has been there for a long time, so it won't be > a > >> block issue for v3.6.1. > >> > >> +1 (binding) from me. > >> > >> Thank you. > >> Luke > >> > >> On Sat, Dec 2, 2023 at 5:46 AM Bill Bejeck <bbej...@gmail.com> wrote: > >> > >> > Hi Mickael, > >> > > >> > I did the following: > >> > > >> > 1. Validated all checksums, signatures, and hashes > >> > 2. Built from source > >> > 3. Ran all the unit tests > >> > 4. Spot checked the documentation and Javadoc > >> > 5. Ran the ZK, Kraft, and Kafka Streams quickstart guides > >> > > >> > I did notice that the `fillDotVersion` in `js/templateData.js` needs > >> > updating to `3.6.1`, but this is minor and should not block the > release. > >> > > >> > It's a +1(binding) for me, pending the successful system test run > >> > > >> > Thanks, > >> > Bill > >> > > >> > On Fri, Dec 1, 2023 at 1:49 PM Justine Olshan > >> <jols...@confluent.io.invalid > >> > > > >> > wrote: > >> > > >> > > I've started a system test run on my end. > >> > > > >> > > Justine > >> > > > >> > > On Wed, Nov 29, 2023 at 1:55 PM Justine Olshan < > jols...@confluent.io> > >> > > wrote: > >> > > > >> > > > I built from source and ran a simple transactional produce bench. > I > >> > ran a > >> > > > handful of unit tests as well. > >> > > > I scanned the docs and everything looked reasonable. > >> > > > > >> > > > I was wondering if we got the system test results mentioned > > System > >> > > > tests: Still running I'll post an update once they complete. > >> > > > > >> > > > Justine > >> > > > > >> > > > On Wed, Nov 29, 2023 at 6:33 AM Mickael Maison < > >> > mickael.mai...@gmail.com > >> > > > > >> > > > wrote: > >> > > > > >> > > >> Hi Josep, > >> > > >> > >> > > >> Good catch! > >> > > >> If it's the only issue we find, I don't think we should block the > >> > > >> release just to fix that. > >> > > >> > >> > > >> If we find another issue, I'll backport it before running another > >> RC, > >> > > >> otherwise I'll backport it once 3.6.1 is released. > >> > > >> > >> > > >> Thanks, > >> > > >> Mickael > >> > > >> > >> > > >> On Wed, Nov 29, 2023 at 11:55 AM Josep Prat > >> > <josep.p...@aiven.io.invalid > >> > > > > >> > > >> wrote: > >> > > >> > > >> > > >> > Hi Mickael, > >> > > >> > This PR[1] made me realize NOTICE-binary is missing the notice > >> for > >> > > >> > commons-io. I don't know if it's a blocker or not. I can cherry > >> pick > >> > > the > >> > > >> > commit to the 3.6 branch if you want. > >> > > >> > > >> > > >> > Best, > >> > > >> > > >> > > >> > > >> > > >> > [1]: https://github.com/apache/kafka/pull/14865 > >> > > >> > > >> > > >> > On Tue, Nov 28, 2023 at 10:25 AM Josep Prat < > josep.p...@aiven.io > >> > > >> > > >> wrote: > >> > > >> > > >> > > >> > > Hi Mickael, > >> > > >> > > Thanks for running the release. It's a +1 for me > (non-binding). > >> > > >> > > I did the following: > >> > > >> > > - Verified artifact's signatures and hashes > >> > > >> > > - Checked JavaDoc (with navigation to Oracle JavaDoc) > >> > > >> > > - Compiled source code > >> > > >> > > - Run unit tests and integration tests > >> > > >> > > - Run getting started with ZK and KRaft > >> > > >> > > > >> > > >> > > Best, > >> > > >> > > > >> > > >> > > On Tue, Nov 28, 2023 at 8:51 AM Kamal Chandraprakash < > >> > > >> > > kamal.chandraprak...@gmail.com> wrote: > >> > > >> > > > >> > > >> > >> +1 (non-binding) > >> > > >> > >> > >> > > >> > >> 1. Built the source from 3.6.1-rc0 tag in scala 2.12 and > 2.13 > >> > > >> > >> 2. Ran all the unit and integration tests. > >> > > >> > >> 3. Ran quickstart and verified the produce-consume on a 3 > node > >> > > >> cluster. > >> > > >> > >> 4. Verified the tiered storage functionality with > local-tiered > >> > > >> storage. > >> > > >> > >> > >> > > >> > >> On Tue, Nov 28, 2023 at 12:55 AM Federico Valeri < > >> > > >> fedeval...@gmail.com> > >> > > >> > >> wrote: > >> > > >> > >> > >> > > >> > >> > Hi Mickael, > >> > > >> > >> > > >> > > >> > >> > - Build from source (Java 17, Scala 2.13) > >> > > >> > >> > - Run unit and integration tests > >> > > >> > >> > - Run custom client apps using staging artifacts > >> > > >> > >> > > >> > > >> > >> > +1 (non binding) > >> > > >> > >> > > >> > > >> > >> > Thanks > >> > > >> > >> > Fede > >> > > >> > >> > > >> > > >> > >> > > >> > > >> > >> > > >> > > >> > >> > On Sun, Nov 26, 2023 at 11:34 AM Jakub Scholz < > >> ja...@scholz.cz > >> > > > >> > > >> wrote: > >> > > >> > >> > > > >> > > >> > >> > > +1 non-binding. I used the staged Scala 2.13 artifacts > and > >> > the > >> > > >> staged > >> > > >> > >> > Maven > >> > > >> > >> > > repo for my tests. All seems to work fine. > >> > > >> > >> > > > >> > > >> > >> > > Thanks > >> > > >> > >> > > Jakub > >> > > >> > >> > > > >> > > >> > >> > > On Fri, Nov 24, 2023 at 4:37 PM Mickael Maison < > >> > > >> mimai...@apache.org> > >> > > >> > >> > wrote: > >> > > >> > >> > > > >> > > >> > >> > > > Hello Kafka users, developers and client-developers, > >> > > >> > >> > > > > >> > > >> > >> > > > This is the first candidate for release of Apache > Kafka > >> > > 3.6.1. > >> > > >> > >> > > > > >> > > >> > >> > > > This is a bugfix release with several fixes, including > >> > > >> dependency > >> > > >> > >> > > > version bumps for CVEs. > >> > > >> > >> > > > > >> > > >> > >> > > > Release notes for the 3.6.1 release: > >> > > >> > >> > > > > >> > > >> > >> > >> > > > https://home.apache.org/~mimaison/kafka-3.6.1-rc0/RELEASE_NOTES.html > >> > > >> > >> > > > > >> > > >> > >> > > > *** Please download, test and vote by Friday, > December 1 > >> > > >> > >> > > > > >> > > >> > >> > > > Kafka's KEYS file containing PGP keys we use to sign > the > >> > > >> release: > >> > > >> > >> > > > https://kafka.apache.org/KEYS > >> > > >> > >> > > > > >> > > >> > >> > > > * Release artifacts to be voted upon (source and > >> binary): > >> > > >> > >> > > > https://home.apache.org/~mimaison/kafka-3.6.1-rc0/ > >> > > >> > >> > > > > >> > > >> > >> > > > * Maven artifacts to be voted upon: > >> > > >> > >> > > > > >> > > >> > >> > >> > > >> > >> > > https://repository.apache.org/content/groups/staging/org/apache/kafka/ > >> > > >> > >> > > > > >> > > >> > >> > > > * Javadoc: > >> > > >> > >> > > > > >> https://home.apache.org/~mimaison/kafka-3.6.1-rc0/javadoc/ > >> > > >> > >> > > > > >> > > >> > >> > > > * Tag to be voted upon (off 3.6 branch) is the 3.6.1 > >> tag: > >> > > >> > >> > > > > https://github.com/apache/kafka/releases/tag/3.6.1-rc0 > >> > > >> > >> > > > > >> > > >> > >> > > > PR for updating docs: > >> > > >> > >> > > > https://github.com/apache/kafka-site/pull/568 > >> > > >> > >> > > > > >> > > >> > >> > > > * Documentation: > >> > > >> > >> > > > https://kafka.apache.org/36/documentation.html > >> > > >> > >> > > > > >> > > >> > >> > > > * Protocol: > >> > > >> > >> > > > https://kafka.apache.org/36/protocol.html > >> > > >> > >> > > > > >> > > >> > >> > > > * Successful Jenkins builds for the 3.6 branch: > >> > > >> > >> > > > Unit/integration tests: We still have a lot of flaky > >> tests > >> > in > >> > > >> the > >> > > >> > >> 3.6 > >> > > >> > >> > > > branch. Looking at the last few 3.6 builds in > >> > > >> > >> > > > > >> https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.6/ > >> > it > >> > > >> seems > >> > > >> > >> all > >> > > >> > >> > > > tests passed at least once apart from > >> > > >> > >> > > > ClusterConnectionStatesTest.testSingleIP(). There's > >> > > >> > >> > > > https://issues.apache.org/jira/browse/KAFKA-15762 to > >> fix > >> > > that > >> > > >> test. > >> > > >> > >> > > > System tests: Still running I'll post an update once > >> they > >> > > >> complete. > >> > > >> > >> > > > > >> > > >> > >> > > > Thanks, > >> > > >> > >> > > > Mickael > >> > > >> > >> > > > > >> > > >> > >> > > >> > > >> > >> > >> > > >> > > > >> > > >> > > > >> > > >> > > -- > >> > > >> > > [image: Aiven] <https://www.aiven.io> > >> > > >> > > > >> > > >> > > *Josep Prat* > >> > > >> > > Open Source Engineering Director, *Aiven* > >> > > >> > > josep.p...@aiven.io | +491715557497 > >> > > >> > > aiven.io <https://www.aiven.io> | > >> > > >> > > <https://www.facebook.com/aivencloud> > >> > > >> > > <https://www.linkedin.com/company/aiven/> < > >> > > >> https://twitter.com/aiven_io> > >> > > >> > > *Aiven Deutschland GmbH* > >> > > >> > > Alexanderufer 3-7, 10117 Berlin > >> > > >> > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > >> > > >> > > Amtsgericht Charlottenburg, HRB 209739 B > >> > > >> > > > >> > > >> > > >> > > >> > > >> > > >> > -- > >> > > >> > [image: Aiven] <https://www.aiven.io> > >> > > >> > > >> > > >> > *Josep Prat* > >> > > >> > Open Source Engineering Director, *Aiven* > >> > > >> > josep.p...@aiven.io | +491715557497 > >> > > >> > aiven.io <https://www.aiven.io> | < > >> > > >> https://www.facebook.com/aivencloud> > >> > > >> > <https://www.linkedin.com/company/aiven/> < > >> > > >> https://twitter.com/aiven_io> > >> > > >> > *Aiven Deutschland GmbH* > >> > > >> > Alexanderufer 3-7, 10117 Berlin > >> > > >> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > >> > > >> > Amtsgericht Charlottenburg, HRB 209739 B > >> > > >> > >> > > > > >> > > > >> > > >> > > > > > > -- > > -David > > > > > -- > -David >
