Thanks for testing the release! And thanks for the review on the documentation. Good catch on the license too.
I have addressed the comments in the blog PR, and opened a few other PRs to the website in relation to the release. - 37: Add download section for the latest 3.7 release <https://github.com/apache/kafka-site/pull/583/files> - 37: Update default docs to point to the 3.7.0 release docs <https://github.com/apache/kafka-site/pull/582> - 3.7: Add blog post for Kafka 3.7 <https://github.com/apache/kafka-site/pull/578> - MINOR: Update stale upgrade_3_6_0 header links in documentation <https://github.com/apache/kafka-site/pull/580> - 37: Add upgrade notes for the 3.7.0 release <https://github.com/apache/kafka-site/pull/581> I am a bit unclear on the precise process regarding what parts of this get merged at what time, and whether the release first needs to be done or not. Best, Stanislav On Mon, Feb 19, 2024 at 8:34 PM Divij Vaidya <divijvaidy...@gmail.com> wrote: > Great. In that case we can fix the license issue retrospectively. I have > created a JIRA for it https://issues.apache.org/jira/browse/KAFKA-16278 > and > also updated the release process (which redirects to > https://issues.apache.org/jira/browse/KAFKA-12622) to check for the > correct > license in both the kafka binaries. > > I am +1 (binding) assuming Mickael's concerns about update notes to 3.7 are > addressed before release. > > -- > Divij Vaidya > > > > On Mon, Feb 19, 2024 at 6:08 PM Mickael Maison <mickael.mai...@gmail.com> > wrote: > > > Hi, > > > > I agree with Josep, I don't think it's worth making a new RC just for > this. > > > > Thanks Stanislav for sharing the test results. The last thing holding > > me from casting my vote is the missing upgrade notes for 3.7.0. > > > > Thanks, > > Mickael > > > > > > > > On Mon, Feb 19, 2024 at 4:28 PM Josep Prat <josep.p...@aiven.io.invalid> > > wrote: > > > > > > I think I remember finding a similar problem (NOTICE_binary) and it > > didn't > > > qualify for an extra RC > > > > > > Best, > > > > > > On Mon, Feb 19, 2024 at 3:44 PM Divij Vaidya <divijvaidy...@gmail.com> > > > wrote: > > > > > > > I have performed the following checks. The only thing I would like to > > call > > > > out is the missing licenses before providing a vote. How do we want > > > > to proceed on this? What have we done in the past? (Creating a new RC > > is > > > > overkill IMO for this license issue). > > > > > > > > ## License check > > > > > > > > Test: Validate license of dependencies for both 2.12 & 2.13 binary. > > > > Result: Missing license for some scala* libraries specifically for > > 2.12. > > > > Seems like we have been missing these licenses for quite some version > > now. > > > > > > > > ``` > > > > for f in $(ls libs | grep -v "^kafka\|connect\|trogdor"); do if ! > grep > > -q > > > > ${f%.*} LICENSE; then echo "${f%.*} is missing in license file"; fi; > > done > > > > scala-collection-compat_2.12-2.10.0 is missing in license file > > > > scala-java8-compat_2.12-1.0.2 is missing in license file > > > > scala-library-2.12.18 is missing in license file > > > > scala-logging_2.12-3.9.4 is missing in license file > > > > scala-reflect-2.12.18 is missing in license file > > > > ``` > > > > > > > > ## Long running tests for memory leak (on ARM machine with zstd) > > > > > > > > Test: Run produce/consume for a few hours and verify no gradual > > increase in > > > > heap. > > > > Result: No heap increase observed. The overall CPU utilization is > lower > > > > compared to 3.5.1. > > > > > > > > ## Verify system test results > > > > > > > > Test: Spot check the results of system tests. > > > > Result: I have verified that the system tests are passing across > > different > > > > runs. > > > > > > > > -- > > > > Divij Vaidya > > > > > > > > > > > > > > > > On Sun, Feb 18, 2024 at 2:50 PM Stanislav Kozlovski > > > > <stanis...@confluent.io.invalid> wrote: > > > > > > > > > The latest system test build completed successfully - > > > > > > > > > > > > > > > > > https://confluent-kafka-branch-builder-system-test-results.s3-us-west-2.amazonaws.com/system-test-kafka-branch-builder--1708250728--apache--3.7--02197edaaa/2024-02-18--001./2024-02-18--001./report.html > > > > > > > > > > *System tests are therefore all good*. We just have some flakes > > > > > > > > > > On Sun, Feb 18, 2024 at 10:45 AM Stanislav Kozlovski < > > > > > stanis...@confluent.io> > > > > > wrote: > > > > > > > > > > > The upgrade test passed -> > > > > > > > > > > > > > > > > > > https://confluent-kafka-branch-builder-system-test-results.s3-us-west-2.amazonaws.com/system-test-kafka-branch-builder--1708103771--apache--3.7--bb6990114b/2024-02-16--001./2024-02-16--001./report.html > > > > > > > > > > > > The replica verification test succeeded in ZK mode, but failed in > > > > > > ISOLATED_KRAFT. It just seems to be very flaky. > > > > > > > > > > > > > > > > > > https://confluent-kafka-branch-builder-system-test-results.s3-us-west-2.amazonaws.com/system-test-kafka-branch-builder--1708100119--apache--3.7--bb6990114b/2024-02-16--001./2024-02-16--001./report.html > > > > > > > > > > > > Scheduling another run in > > > > > > > > > > > > https://jenkins.confluent.io/job/system-test-kafka-branch-builder/6062/ > > > > > > > > > > > > On Fri, Feb 16, 2024 at 6:39 PM Stanislav Kozlovski < > > > > > > stanis...@confluent.io> wrote: > > > > > > > > > > > >> Thanks all for the help in verifying. > > > > > >> > > > > > >> I have updated > > > > > >> > > > > > > > > > > > > https://gist.github.com/stanislavkozlovski/820976fc7bfb5f4dcdf9742fd96a9982 > > > > > >> with the system tests. > > > > > >> There were two builds ran, and across those - the following > tests > > > > failed > > > > > >> two times in a row: > > > > > >> > > > > > >> > > > > > >> > > > > > > > > > > > > *kafkatest.tests.tools.replica_verification_test.ReplicaVerificationToolTest#test_replica_lagsArguments:{ > > > > > >> "metadata_quorum": "ZK"}*Fails with the same error of > > > > > >> *`TimeoutError('Timed out waiting to reach non-zero number of > > replica > > > > > >> lags.')`* > > > > > >> I have scheduled a re-run of this specific test here -> > > > > > >> > > > > > https://jenkins.confluent.io/job/system-test-kafka-branch-builder/6057 > > > > > >> > > > > > >> > > *kafkatest.tests.core.upgrade_test.TestUpgrade#test_upgradeArguments:{ > > > > > >> "compression_types": [ "zstd" ], "from_kafka_version": "2.4.1", > > > > > >> "to_message_format_version": null}* > > > > > >> Fails with the same error of > > > > > >> *`TimeoutError('Producer failed to produce messages for 20s.')`* > > > > > >> > > *kafkatest.tests.core.upgrade_test.TestUpgrade#test_upgradeArguments:{ > > > > > >> "compression_types": [ "lz4" ], "from_kafka_version": "3.0.2", > > > > > >> "to_message_format_version": null}* > > > > > >> Fails with the same error of *`TimeoutError('Producer failed to > > > > produce > > > > > >> messages for 20s.')`* > > > > > >> > > > > > >> I have scheduled a re-run of this test here -> > > > > > >> > > > > > > https://jenkins.confluent.io/job/system-test-kafka-branch-builder/6058/ > > > > > >> > > > > > >> On Fri, Feb 16, 2024 at 12:15 PM Vedarth Sharma < > > > > > vedarth.sha...@gmail.com> > > > > > >> wrote: > > > > > >> > > > > > >>> Hey Stanislav, > > > > > >>> > > > > > >>> Thanks for the release candidate. > > > > > >>> > > > > > >>> +1 (non-binding) > > > > > >>> > > > > > >>> I tested and verified the docker image artifact > > > > > apache/kafka:3.7.0-rc4:- > > > > > >>> - verified create topic, produce messages and consume messages > > flow > > > > > when > > > > > >>> running the docker image with > > > > > >>> - default configs > > > > > >>> - configs provided via env variables > > > > > >>> - configs provided via file input > > > > > >>> - verified the html documentation for docker image. > > > > > >>> - ran the example docker compose files successfully. > > > > > >>> > > > > > >>> All looks good for the docker image artifact! > > > > > >>> > > > > > >>> Thanks and regards, > > > > > >>> Vedarth > > > > > >>> > > > > > >>> > > > > > >>> On Thu, Feb 15, 2024 at 10:58 PM Mickael Maison < > > > > > >>> mickael.mai...@gmail.com> > > > > > >>> wrote: > > > > > >>> > > > > > >>> > Hi Stanislav, > > > > > >>> > > > > > > >>> > Thanks for running the release. > > > > > >>> > > > > > > >>> > I did the following testing: > > > > > >>> > - verified the check sums and signatures > > > > > >>> > - ran ZooKeeper and KRaft quickstarts with Scala 2.13 > binaries > > > > > >>> > - ran a successful migration from ZooKeeper to KRaft > > > > > >>> > > > > > > >>> > We seem to be missing the upgrade notes for 3.7.0 in the > docs. > > See > > > > > >>> > https://kafka.apache.org/37/documentation.html#upgrade that > > still > > > > > >>> > points to 3.6.0 > > > > > >>> > Before voting I'd like to see results from the system tests > > too. > > > > > >>> > > > > > > >>> > Thanks, > > > > > >>> > Mickael > > > > > >>> > > > > > > >>> > On Thu, Feb 15, 2024 at 6:06 PM Andrew Schofield > > > > > >>> > <andrew_schofield_j...@outlook.com> wrote: > > > > > >>> > > > > > > > >>> > > +1 (non-binding). I used the staged binaries with Scala > > 2.13. I > > > > > tried > > > > > >>> > the new group coordinator > > > > > >>> > > and consumer group protocol which is included with the > Early > > > > Access > > > > > >>> > release of KIP-848. > > > > > >>> > > Also verified the availability of the new APIs. All working > > as > > > > > >>> expected. > > > > > >>> > > > > > > > >>> > > Thanks, > > > > > >>> > > Andrew > > > > > >>> > > > > > > > >>> > > > On 15 Feb 2024, at 05:07, Paolo Patierno < > > > > > paolo.patie...@gmail.com > > > > > >>> > > > > > > >>> > wrote: > > > > > >>> > > > > > > > > >>> > > > +1 (non-binding). I used the staged binaries with Scala > > 2.13 > > > > and > > > > > >>> mostly > > > > > >>> > > > focused on the ZooKeeper to KRaft migration with multiple > > > > tests. > > > > > >>> > Everything > > > > > >>> > > > works fine. > > > > > >>> > > > > > > > > >>> > > > Thanks > > > > > >>> > > > Paolo > > > > > >>> > > > > > > > > >>> > > > On Mon, 12 Feb 2024, 22:06 Jakub Scholz, < > ja...@scholz.cz> > > > > > wrote: > > > > > >>> > > > > > > > > >>> > > >> +1 (non-binding). I used the staged binaries with Scala > > 2.13 > > > > and > > > > > >>> the > > > > > >>> > staged > > > > > >>> > > >> Maven artifacts to run my tests. All seems to work fine. > > > > Thanks. > > > > > >>> > > >> > > > > > >>> > > >> Jakub > > > > > >>> > > >> > > > > > >>> > > >> On Fri, Feb 9, 2024 at 4:20 PM Stanislav Kozlovski > > > > > >>> > > >> <stanis...@confluent.io.invalid> wrote: > > > > > >>> > > >> > > > > > >>> > > >>> Hello Kafka users, developers and client-developers, > > > > > >>> > > >>> > > > > > >>> > > >>> This is the second candidate we are considering for > > release > > > > of > > > > > >>> Apache > > > > > >>> > > >> Kafka > > > > > >>> > > >>> 3.7.0. > > > > > >>> > > >>> > > > > > >>> > > >>> Major changes include: > > > > > >>> > > >>> - Early Access to KIP-848 - the next generation of the > > > > consumer > > > > > >>> > rebalance > > > > > >>> > > >>> protocol > > > > > >>> > > >>> - Early Access to KIP-858: Adding JBOD support to KRaft > > > > > >>> > > >>> - KIP-714: Observability into Client metrics via a > > > > standardized > > > > > >>> > interface > > > > > >>> > > >>> > > > > > >>> > > >>> Release notes for the 3.7.0 release: > > > > > >>> > > >>> > > > > > >>> > > >>> > > > > > >>> > > >> > > > > > >>> > > > > > > >>> > > > > > > > > > > > > https://home.apache.org/~stanislavkozlovski/kafka-3.7.0-rc4/RELEASE_NOTES.html > > > > > >>> > > >>> > > > > > >>> > > >>> *** Please download, test and vote by Thursday, > February > > > > 15th, > > > > > >>> 9AM > > > > > >>> > PST > > > > > >>> > > >> *** > > > > > >>> > > >>> > > > > > >>> > > >>> Kafka's KEYS file containing PGP keys we use to sign > the > > > > > release: > > > > > >>> > > >>> https://kafka.apache.org/KEYS > > > > > >>> > > >>> > > > > > >>> > > >>> * Release artifacts to be voted upon (source and > binary): > > > > > >>> > > >>> > > https://home.apache.org/~stanislavkozlovski/kafka-3.7.0-rc4/ > > > > > >>> > > >>> > > > > > >>> > > >>> * Docker release artifact to be voted upon: > > > > > >>> > > >>> apache/kafka:3.7.0-rc4 > > > > > >>> > > >>> > > > > > >>> > > >>> * Maven artifacts to be voted upon: > > > > > >>> > > >>> > > > > > >>> > > > > > > > > https://repository.apache.org/content/groups/staging/org/apache/kafka/ > > > > > >>> > > >>> > > > > > >>> > > >>> * Javadoc: > > > > > >>> > > >>> > > > > > >>> > > https://home.apache.org/~stanislavkozlovski/kafka-3.7.0-rc4/javadoc/ > > > > > >>> > > >>> > > > > > >>> > > >>> * Tag to be voted upon (off 3.7 branch) is the 3.7.0 > tag: > > > > > >>> > > >>> https://github.com/apache/kafka/releases/tag/3.7.0-rc4 > > > > > >>> > > >>> > > > > > >>> > > >>> * Documentation: > > > > > >>> > > >>> https://kafka.apache.org/37/documentation.html > > > > > >>> > > >>> > > > > > >>> > > >>> * Protocol: > > > > > >>> > > >>> https://kafka.apache.org/37/protocol.html > > > > > >>> > > >>> > > > > > >>> > > >>> * Successful Jenkins builds for the 3.7 branch: > > > > > >>> > > >>> > > > > > >>> > > >>> Unit/integration tests: I am in the process of running > > and > > > > > >>> analyzing > > > > > >>> > > >> these. > > > > > >>> > > >>> System tests: I am in the process of running these. > > > > > >>> > > >>> > > > > > >>> > > >>> Expect a follow-up over the weekend > > > > > >>> > > >>> > > > > > >>> > > >>> * Successful Docker Image Github Actions Pipeline for > 3.7 > > > > > branch: > > > > > >>> > > >>> Docker Build Test Pipeline: > > > > > >>> > > >>> > https://github.com/apache/kafka/actions/runs/7845614846 > > > > > >>> > > >>> > > > > > >>> > > >>> /************************************** > > > > > >>> > > >>> > > > > > >>> > > >>> Best, > > > > > >>> > > >>> Stanislav > > > > > >>> > > >>> > > > > > >>> > > >> > > > > > >>> > > > > > > > >>> > > > > > > >>> > > > > > >> > > > > > >> > > > > > >> -- > > > > > >> Best, > > > > > >> Stanislav > > > > > >> > > > > > > > > > > > > > > > > > > -- > > > > > > Best, > > > > > > Stanislav > > > > > > > > > > > > > > > > > > > > > -- > > > > > Best, > > > > > Stanislav > > > > > > > > > > > > > > > > > > -- > > > [image: Aiven] <https://www.aiven.io> > > > > > > *Josep Prat* > > > Open Source Engineering Director, *Aiven* > > > josep.p...@aiven.io | +491715557497 > > > aiven.io <https://www.aiven.io> | < > > https://www.facebook.com/aivencloud> > > > <https://www.linkedin.com/company/aiven/> < > > https://twitter.com/aiven_io> > > > *Aiven Deutschland GmbH* > > > Alexanderufer 3-7, 10117 Berlin > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > > Amtsgericht Charlottenburg, HRB 209739 B > > > -- Best, Stanislav
