Re: [VOTE] 3.6.0 RC2
Hi, Thank you all for participating in the 3.6.0 release candidate vote threads. This vote passes with 11 +1 votes(4 binding) and no 0 or -1 votes. +1 votes PMC Members: * Luke Chen * Bill Bejeck * Chris Egerton * Justine Olshan Community: * Jakub Scholz * Federico Valeri * Divij Vaidya * Kamal Chandraprakash * Josep Prat * Proven Provenzano * Greg Harris 0 votes * No votes -1 votes * No votes I will continue with the release process and the release announcement will follow in the next few days. Thanks, Satish. On Tue, 3 Oct 2023 at 12:35, Satish Duggana wrote: > > Hi, > Thank you all for participating in the 3.6.0 release candidate vote threads. > > This vote passes with 11 +1 votes(4 binding) and no 0 or -1 votes. > > +1 votes > PMC Members: > * Luke Chen > * Bill Bejeck > * Chris Egerton > * Justine Olshan > > Community: > * Jakub Scholz > * Federico Valeri > * Divij Vaidya > * Kamal Chandraprakash > * Josep Prat > * Proven Provenzano > * Greg Harris > > 0 votes > * No votes > > -1 votes > * No votes > > I will continue with the release process and the release announcement > will follow in the next few days. > > Thanks, > Satish. > > On Tue, 3 Oct 2023 at 09:54, Justine Olshan > wrote: > > > > Thanks folks for following up. Given my previous testing and the results > > you've provided, I'm +1 (binding) > > > > I will also follow up with the non-blocking metrics documentation. > > > > Thanks! > > Justine > > > > On Tue, Oct 3, 2023 at 8:17 AM Chris Egerton > > wrote: > > > > > Hi Satish, > > > > > > Thanks for running this release! > > > > > > To verify, I: > > > - Built from source using Java 11 with both: > > > - - the 3.6.0-rc2 tag on GitHub > > > - - the kafka-3.6.0-src.tgz artifact from > > > https://home.apache.org/~satishd/kafka-3.6.0-rc2/ > > > - Checked signatures and checksums > > > - Ran the quickstart using the kafka_2.13-3.6.0.tgz artifact from > > > https://home.apache.org/~satishd/kafka-3.6.0-rc2/ with Java 11 and Scala > > > 13 > > > in KRaft mode > > > - Ran all unit tests > > > - Ran all integration tests for Connect and MM2 > > > - Verified that the connect-test-plugins module is present in the staging > > > Maven artifacts (https://issues.apache.org/jira/browse/KAFKA-15249) > > > > > > Everything looks good to me! > > > > > > +1 (binding) > > > > > > Cheers, > > > > > > Chris > > > > > > On Tue, Oct 3, 2023 at 6:43 AM Satish Duggana > > > wrote: > > > > > > > Thanks Luke for helping on running system tests on RCs and updating > > > > the status on this email thread. > > > > > > > > ~Satish. > > > > > > > > On Tue, 3 Oct 2023 at 05:04, Luke Chen wrote: > > > > > > > > > > Hi Justine and all, > > > > > > > > > > The system test result for 3.6.0 RC2 can be found below. > > > > > In short, no failed tests. The flaky tests will pass in the 2nd run. > > > > > > > > > > > > https://drive.google.com/drive/folders/1qwIKg-B4CBrswUeo5fBRv65KWpDsGUiS?usp=sharing > > > > > > > > > > Thank you. > > > > > Luke > > > > > > > > > > On Tue, Oct 3, 2023 at 7:08 AM Justine Olshan > > > > > > > > > wrote: > > > > > > > > > > > I realized Luke shared the results here for RC1 > > > > > > > > > > https://drive.google.com/drive/folders/1S2XYd79f6_AeWj9f9qEkliRg7JtL04AC > > > > > > Given we had some runs that looked reasonable, and we made a small > > > > change, > > > > > > I'm ok with this. But I wouldn't be upset if we had another set of > > > > runs :) > > > > > > > > > > > > As for the validation: > > > > > > > > > > > >- I've compiled from source with java 17, 2.13, run the > > > > transactional > > > > > >produce bench > > > > > >- Run unit tests > > > > > >- Validated the checksums > > > > > >- Downloaded and ran the 2.12 version of the release > > > > > >- Briefly took a look at the documentation > > > > > >- I was browsing through the site html files and I noticed the > > > html > > > > for > > > > > >documentation.html seemed to be for 3.4. Not sure if this is a > > > > blocker, > > > > > > but > > > > > >wanted to flag it. This seems to be the case for the previous > > > > release > > > > > >candidates as well. (As well as 3.5 release it seems) > > > > > > > > > > > > > > > > > > I will hold off on voting until we figure that part out. I will also > > > > follow > > > > > > up with the documentation Divij mentioned outside this thread. > > > > > > > > > > > > Thanks, > > > > > > Justine > > > > > > > > > > > > On Mon, Oct 2, 2023 at 3:05 PM Greg Harris > > > > > > > > > > wrote: > > > > > > > > > > > > > Hey Satish, > > > > > > > > > > > > > > I verified KIP-898 functionality and the KAFKA-15473 patch. > > > > > > > +1 (non-binding) > > > > > > > > > > > > > > Thanks! > > > > > > > > > > > > > > On Mon, Oct 2, 2023 at 1:28 PM Justine Olshan > > > > > > > wrote: > > > > > > > > > > > > > > > > Hey all -- I noticed we still have the system tests as something > > > > that > > > > > > > will > > > > > > > > be updated. Did we get a run for this RC
Re: [VOTE] 3.6.0 RC2
Thanks Divij for the observations. KAFKA-15483 is added to address the missing metrics related to KRaft and ZK to KRaft migration. Please leave a comment at https://github.com/apache/kafka-site/pull/548 in which related metrics documentation is added. https://issues.apache.org/jira/browse/KAFKA-15530 is raised to followup on the missing documentation of metrics introduced in KIP-890. ~Satish. On Mon, 2 Oct 2023 at 05:37, Divij Vaidya wrote: > > + 1 (non-binding) > > Verifications: > 1. I ran a produce-consume workload with plaintext auth, JDK17, zstd > compression using an open messaging benchmark and found 3.6 to be better > than or equal to 3.5.1 across all dimensions. Notably, 3.6 had consistently > 6-7% lower CPU utilization, lesser spikes on P99 produce latencies and > overall lower P99.8 latencies. > > 2. I have verified that detached signature is correct using > https://www.apache.org/info/verification.html and the release manager > public keys are available at > https://keys.openpgp.org/search?q=F65DC3423D4CD7B9 > > 3. I have verified that all metrics emitted in 3.5.1 (with Zk) are also > being emitted in 3.6.0 (with Zk). > > Problems (but not blockers): > 1. Metrics added in > https://github.com/apache/kafka/commit/2f71708955b293658cec3b27e9a5588d39c38d7e > aren't available in the documentation (cc: Justine). I don't consider this > as a release blocker but we should add it as a fast follow-up. > > 2. Metric added in > https://github.com/apache/kafka/commit/a900794ace4dcf1f9dadee27fbd8b63979532a18 > isn't available in documentation (cc: David). I don't consider this as a > release blocker but we should add it as a fast follow-up. > > -- > Divij Vaidya > > > > On Mon, Oct 2, 2023 at 9:50 AM Federico Valeri wrote: > > > Hi Satish, I did the following to verify the release: > > > > - Built from source with Java 17 and Scala 2.13 > > - Ran all unit and integration tests > > - Spot checked documentation > > - Ran custom client applications using staging artifacts on a 3-nodes > > cluster > > - Tested tiered storage with one of the available RSM implementations > > > > +1 (non binding) > > > > Thanks > > Fede > > > > On Mon, Oct 2, 2023 at 8:50 AM Luke Chen wrote: > > > > > > Hi Satish, > > > > > > I verified with: > > > 1. Ran quick start in KRaft for scala 2.12 artifact > > > 2. Making sure the checksum are correct > > > 3. Browsing release notes, documents, javadocs, protocols. > > > 4. Verified the tiered storage feature works well. > > > > > > +1 (binding). > > > > > > Thanks. > > > Luke > > > > > > > > > > > > On Mon, Oct 2, 2023 at 5:23 AM Jakub Scholz wrote: > > > > > > > +1 (non-binding). I used the Scala 2.13 binaries and the staged Maven > > > > artifacts and run my tests. Everything seems to work fine for me. > > > > > > > > Thanks > > > > Jakub > > > > > > > > On Fri, Sep 29, 2023 at 8:17 PM Satish Duggana < > > satish.dugg...@gmail.com> > > > > wrote: > > > > > > > > > Hello Kafka users, developers and client-developers, > > > > > > > > > > This is the third candidate for the release of Apache Kafka 3.6.0. > > > > > Some of the major features include: > > > > > > > > > > * KIP-405 : Kafka Tiered Storage > > > > > * KIP-868 : KRaft Metadata Transactions > > > > > * KIP-875: First-class offsets support in Kafka Connect > > > > > * KIP-898: Modernize Connect plugin discovery > > > > > * KIP-938: Add more metrics for measuring KRaft performance > > > > > * KIP-902: Upgrade Zookeeper to 3.8.1 > > > > > * KIP-917: Additional custom metadata for remote log segment > > > > > > > > > > Release notes for the 3.6.0 release: > > > > > https://home.apache.org/~satishd/kafka-3.6.0-rc2/RELEASE_NOTES.html > > > > > > > > > > *** Please download, test and vote by Tuesday, October 3, 12pm PT > > > > > > > > > > Kafka's KEYS file containing PGP keys we use to sign the release: > > > > > https://kafka.apache.org/KEYS > > > > > > > > > > * Release artifacts to be voted upon (source and binary): > > > > > https://home.apache.org/~satishd/kafka-3.6.0-rc2/ > > > > > > > > > > * Maven artifacts to be voted upon: > > > > > > > https://repository.apache.org/content/groups/staging/org/apache/kafka/ > > > > > > > > > > * Javadoc: > > > > > https://home.apache.org/~satishd/kafka-3.6.0-rc2/javadoc/ > > > > > > > > > > * Tag to be voted upon (off 3.6 branch) is the 3.6.0-rc2 tag: > > > > > https://github.com/apache/kafka/releases/tag/3.6.0-rc2 > > > > > > > > > > * Documentation: > > > > > https://kafka.apache.org/36/documentation.html > > > > > > > > > > * Protocol: > > > > > https://kafka.apache.org/36/protocol.html > > > > > > > > > > * Successful Jenkins builds for the 3.6 branch: > > > > > There are a few runs of unit/integration tests. You can see the > > latest > > > > > at https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.6/. We > > will > > > > > continue running a few more iterations. > > > > > System tests: > > > > > We will send an update once we have the results. > > > > > > > > > > Thanks, > >
Re: [VOTE] 3.6.0 RC2
Hi, To verify the release of release 3.6.0 RC2 I did the following: - Downloaded the source, built and ran the tests. - Validated SCRAM with KRaft including creating credentials with kafka-storage. - Validated Delegation Tokens with KRaft +1 (non-binding) --Proven On Mon, Oct 2, 2023 at 8:37 AM Divij Vaidya wrote: > + 1 (non-binding) > > Verifications: > 1. I ran a produce-consume workload with plaintext auth, JDK17, zstd > compression using an open messaging benchmark and found 3.6 to be better > than or equal to 3.5.1 across all dimensions. Notably, 3.6 had consistently > 6-7% lower CPU utilization, lesser spikes on P99 produce latencies and > overall lower P99.8 latencies. > > 2. I have verified that detached signature is correct using > https://www.apache.org/info/verification.html and the release manager > public keys are available at > https://keys.openpgp.org/search?q=F65DC3423D4CD7B9 > > 3. I have verified that all metrics emitted in 3.5.1 (with Zk) are also > being emitted in 3.6.0 (with Zk). > > Problems (but not blockers): > 1. Metrics added in > > https://github.com/apache/kafka/commit/2f71708955b293658cec3b27e9a5588d39c38d7e > aren't available in the documentation (cc: Justine). I don't consider this > as a release blocker but we should add it as a fast follow-up. > > 2. Metric added in > > https://github.com/apache/kafka/commit/a900794ace4dcf1f9dadee27fbd8b63979532a18 > isn't available in documentation (cc: David). I don't consider this as a > release blocker but we should add it as a fast follow-up. > > -- > Divij Vaidya > > > > On Mon, Oct 2, 2023 at 9:50 AM Federico Valeri > wrote: > > > Hi Satish, I did the following to verify the release: > > > > - Built from source with Java 17 and Scala 2.13 > > - Ran all unit and integration tests > > - Spot checked documentation > > - Ran custom client applications using staging artifacts on a 3-nodes > > cluster > > - Tested tiered storage with one of the available RSM implementations > > > > +1 (non binding) > > > > Thanks > > Fede > > > > On Mon, Oct 2, 2023 at 8:50 AM Luke Chen wrote: > > > > > > Hi Satish, > > > > > > I verified with: > > > 1. Ran quick start in KRaft for scala 2.12 artifact > > > 2. Making sure the checksum are correct > > > 3. Browsing release notes, documents, javadocs, protocols. > > > 4. Verified the tiered storage feature works well. > > > > > > +1 (binding). > > > > > > Thanks. > > > Luke > > > > > > > > > > > > On Mon, Oct 2, 2023 at 5:23 AM Jakub Scholz wrote: > > > > > > > +1 (non-binding). I used the Scala 2.13 binaries and the staged Maven > > > > artifacts and run my tests. Everything seems to work fine for me. > > > > > > > > Thanks > > > > Jakub > > > > > > > > On Fri, Sep 29, 2023 at 8:17 PM Satish Duggana < > > satish.dugg...@gmail.com> > > > > wrote: > > > > > > > > > Hello Kafka users, developers and client-developers, > > > > > > > > > > This is the third candidate for the release of Apache Kafka 3.6.0. > > > > > Some of the major features include: > > > > > > > > > > * KIP-405 : Kafka Tiered Storage > > > > > * KIP-868 : KRaft Metadata Transactions > > > > > * KIP-875: First-class offsets support in Kafka Connect > > > > > * KIP-898: Modernize Connect plugin discovery > > > > > * KIP-938: Add more metrics for measuring KRaft performance > > > > > * KIP-902: Upgrade Zookeeper to 3.8.1 > > > > > * KIP-917: Additional custom metadata for remote log segment > > > > > > > > > > Release notes for the 3.6.0 release: > > > > > > https://home.apache.org/~satishd/kafka-3.6.0-rc2/RELEASE_NOTES.html > > > > > > > > > > *** Please download, test and vote by Tuesday, October 3, 12pm PT > > > > > > > > > > Kafka's KEYS file containing PGP keys we use to sign the release: > > > > > https://kafka.apache.org/KEYS > > > > > > > > > > * Release artifacts to be voted upon (source and binary): > > > > > https://home.apache.org/~satishd/kafka-3.6.0-rc2/ > > > > > > > > > > * Maven artifacts to be voted upon: > > > > > > > https://repository.apache.org/content/groups/staging/org/apache/kafka/ > > > > > > > > > > * Javadoc: > > > > > https://home.apache.org/~satishd/kafka-3.6.0-rc2/javadoc/ > > > > > > > > > > * Tag to be voted upon (off 3.6 branch) is the 3.6.0-rc2 tag: > > > > > https://github.com/apache/kafka/releases/tag/3.6.0-rc2 > > > > > > > > > > * Documentation: > > > > > https://kafka.apache.org/36/documentation.html > > > > > > > > > > * Protocol: > > > > > https://kafka.apache.org/36/protocol.html > > > > > > > > > > * Successful Jenkins builds for the 3.6 branch: > > > > > There are a few runs of unit/integration tests. You can see the > > latest > > > > > at https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.6/. We > > will > > > > > continue running a few more iterations. > > > > > System tests: > > > > > We will send an update once we have the results. > > > > > > > > > > Thanks, > > > > > Satish. > > > > > > > > > > > >
Re: [VOTE] 3.6.0 RC2
Hi Satish, Thanks for running the release. I performed the following steps: - Validated all the checksums, signatures, and keys - Built the release from source - Ran all unit tests - Quick start validations - ZK and Kraft - Connect - Kafka Streams - Spot checked java docs and documentation +1 (binding) - Bill On Mon, Oct 2, 2023 at 10:23 AM Proven Provenzano wrote: > Hi, > > To verify the release of release 3.6.0 RC2 I did the following: > >- Downloaded the source, built and ran the tests. >- Validated SCRAM with KRaft including creating credentials with >kafka-storage. >- Validated Delegation Tokens with KRaft > > +1 (non-binding) > > --Proven > > > > On Mon, Oct 2, 2023 at 8:37 AM Divij Vaidya > wrote: > > > + 1 (non-binding) > > > > Verifications: > > 1. I ran a produce-consume workload with plaintext auth, JDK17, zstd > > compression using an open messaging benchmark and found 3.6 to be better > > than or equal to 3.5.1 across all dimensions. Notably, 3.6 had > consistently > > 6-7% lower CPU utilization, lesser spikes on P99 produce latencies and > > overall lower P99.8 latencies. > > > > 2. I have verified that detached signature is correct using > > https://www.apache.org/info/verification.html and the release manager > > public keys are available at > > https://keys.openpgp.org/search?q=F65DC3423D4CD7B9 > > > > 3. I have verified that all metrics emitted in 3.5.1 (with Zk) are also > > being emitted in 3.6.0 (with Zk). > > > > Problems (but not blockers): > > 1. Metrics added in > > > > > https://github.com/apache/kafka/commit/2f71708955b293658cec3b27e9a5588d39c38d7e > > aren't available in the documentation (cc: Justine). I don't consider > this > > as a release blocker but we should add it as a fast follow-up. > > > > 2. Metric added in > > > > > https://github.com/apache/kafka/commit/a900794ace4dcf1f9dadee27fbd8b63979532a18 > > isn't available in documentation (cc: David). I don't consider this as a > > release blocker but we should add it as a fast follow-up. > > > > -- > > Divij Vaidya > > > > > > > > On Mon, Oct 2, 2023 at 9:50 AM Federico Valeri > > wrote: > > > > > Hi Satish, I did the following to verify the release: > > > > > > - Built from source with Java 17 and Scala 2.13 > > > - Ran all unit and integration tests > > > - Spot checked documentation > > > - Ran custom client applications using staging artifacts on a 3-nodes > > > cluster > > > - Tested tiered storage with one of the available RSM implementations > > > > > > +1 (non binding) > > > > > > Thanks > > > Fede > > > > > > On Mon, Oct 2, 2023 at 8:50 AM Luke Chen wrote: > > > > > > > > Hi Satish, > > > > > > > > I verified with: > > > > 1. Ran quick start in KRaft for scala 2.12 artifact > > > > 2. Making sure the checksum are correct > > > > 3. Browsing release notes, documents, javadocs, protocols. > > > > 4. Verified the tiered storage feature works well. > > > > > > > > +1 (binding). > > > > > > > > Thanks. > > > > Luke > > > > > > > > > > > > > > > > On Mon, Oct 2, 2023 at 5:23 AM Jakub Scholz wrote: > > > > > > > > > +1 (non-binding). I used the Scala 2.13 binaries and the staged > Maven > > > > > artifacts and run my tests. Everything seems to work fine for me. > > > > > > > > > > Thanks > > > > > Jakub > > > > > > > > > > On Fri, Sep 29, 2023 at 8:17 PM Satish Duggana < > > > satish.dugg...@gmail.com> > > > > > wrote: > > > > > > > > > > > Hello Kafka users, developers and client-developers, > > > > > > > > > > > > This is the third candidate for the release of Apache Kafka > 3.6.0. > > > > > > Some of the major features include: > > > > > > > > > > > > * KIP-405 : Kafka Tiered Storage > > > > > > * KIP-868 : KRaft Metadata Transactions > > > > > > * KIP-875: First-class offsets support in Kafka Connect > > > > > > * KIP-898: Modernize Connect plugin discovery > > > > > > * KIP-938: Add more metrics for measuring KRaft performance > > > > > > * KIP-902: Upgrade Zookeeper to 3.8.1 > > > > > > * KIP-917: Additional custom metadata for remote log segment > > > > > > > > > > > > Release notes for the 3.6.0 release: > > > > > > > > https://home.apache.org/~satishd/kafka-3.6.0-rc2/RELEASE_NOTES.html > > > > > > > > > > > > *** Please download, test and vote by Tuesday, October 3, 12pm PT > > > > > > > > > > > > Kafka's KEYS file containing PGP keys we use to sign the release: > > > > > > https://kafka.apache.org/KEYS > > > > > > > > > > > > * Release artifacts to be voted upon (source and binary): > > > > > > https://home.apache.org/~satishd/kafka-3.6.0-rc2/ > > > > > > > > > > > > * Maven artifacts to be voted upon: > > > > > > > > > https://repository.apache.org/content/groups/staging/org/apache/kafka/ > > > > > > > > > > > > * Javadoc: > > > > > > https://home.apache.org/~satishd/kafka-3.6.0-rc2/javadoc/ > > > > > > > > > > > > * Tag to be voted upon (off 3.6 branch) is the 3.6.0-rc2 tag: > > > > > > https://github.com/apache/kafka/releases/tag/3.6.0-rc2 > > > > >
Re: [VOTE] 3.6.0 RC2
Hi Satish, +1 (non-binding) from me as well! Thanks for running the release I run the following validations steps: - Built from source with Java 11 and Scala 2.13 - Verified Signatures and hashes of the artifacts generated - Navigated through Javadoc including links to JDK classes - Run the unit tests - Run integration tests - Run the quickstart in KRaft and Zookeeper mode On Mon, Oct 2, 2023 at 3:13 PM Kamal Chandraprakash < kamal.chandraprak...@gmail.com> wrote: > +1 (non-binding) > > 1. Built the source from 3.6 branch in scala 2.12 and 2.13 > 2. Ran all the unit and integration tests. > 3. Ran quickstart and verified the produce-consume on a 3 node cluster. > 4. Verified the tiered storage functionality with local-tiered storage. > > Thanks, > Kamal > > On Mon, Oct 2, 2023 at 6:07 PM Divij Vaidya > wrote: > > > + 1 (non-binding) > > > > Verifications: > > 1. I ran a produce-consume workload with plaintext auth, JDK17, zstd > > compression using an open messaging benchmark and found 3.6 to be better > > than or equal to 3.5.1 across all dimensions. Notably, 3.6 had > consistently > > 6-7% lower CPU utilization, lesser spikes on P99 produce latencies and > > overall lower P99.8 latencies. > > > > 2. I have verified that detached signature is correct using > > https://www.apache.org/info/verification.html and the release manager > > public keys are available at > > https://keys.openpgp.org/search?q=F65DC3423D4CD7B9 > > > > 3. I have verified that all metrics emitted in 3.5.1 (with Zk) are also > > being emitted in 3.6.0 (with Zk). > > > > Problems (but not blockers): > > 1. Metrics added in > > > > > https://github.com/apache/kafka/commit/2f71708955b293658cec3b27e9a5588d39c38d7e > > aren't available in the documentation (cc: Justine). I don't consider > this > > as a release blocker but we should add it as a fast follow-up. > > > > 2. Metric added in > > > > > https://github.com/apache/kafka/commit/a900794ace4dcf1f9dadee27fbd8b63979532a18 > > isn't available in documentation (cc: David). I don't consider this as a > > release blocker but we should add it as a fast follow-up. > > > > -- > > Divij Vaidya > > > > > > > > On Mon, Oct 2, 2023 at 9:50 AM Federico Valeri > > wrote: > > > > > Hi Satish, I did the following to verify the release: > > > > > > - Built from source with Java 17 and Scala 2.13 > > > - Ran all unit and integration tests > > > - Spot checked documentation > > > - Ran custom client applications using staging artifacts on a 3-nodes > > > cluster > > > - Tested tiered storage with one of the available RSM implementations > > > > > > +1 (non binding) > > > > > > Thanks > > > Fede > > > > > > On Mon, Oct 2, 2023 at 8:50 AM Luke Chen wrote: > > > > > > > > Hi Satish, > > > > > > > > I verified with: > > > > 1. Ran quick start in KRaft for scala 2.12 artifact > > > > 2. Making sure the checksum are correct > > > > 3. Browsing release notes, documents, javadocs, protocols. > > > > 4. Verified the tiered storage feature works well. > > > > > > > > +1 (binding). > > > > > > > > Thanks. > > > > Luke > > > > > > > > > > > > > > > > On Mon, Oct 2, 2023 at 5:23 AM Jakub Scholz wrote: > > > > > > > > > +1 (non-binding). I used the Scala 2.13 binaries and the staged > Maven > > > > > artifacts and run my tests. Everything seems to work fine for me. > > > > > > > > > > Thanks > > > > > Jakub > > > > > > > > > > On Fri, Sep 29, 2023 at 8:17 PM Satish Duggana < > > > satish.dugg...@gmail.com> > > > > > wrote: > > > > > > > > > > > Hello Kafka users, developers and client-developers, > > > > > > > > > > > > This is the third candidate for the release of Apache Kafka > 3.6.0. > > > > > > Some of the major features include: > > > > > > > > > > > > * KIP-405 : Kafka Tiered Storage > > > > > > * KIP-868 : KRaft Metadata Transactions > > > > > > * KIP-875: First-class offsets support in Kafka Connect > > > > > > * KIP-898: Modernize Connect plugin discovery > > > > > > * KIP-938: Add more metrics for measuring KRaft performance > > > > > > * KIP-902: Upgrade Zookeeper to 3.8.1 > > > > > > * KIP-917: Additional custom metadata for remote log segment > > > > > > > > > > > > Release notes for the 3.6.0 release: > > > > > > > > https://home.apache.org/~satishd/kafka-3.6.0-rc2/RELEASE_NOTES.html > > > > > > > > > > > > *** Please download, test and vote by Tuesday, October 3, 12pm PT > > > > > > > > > > > > Kafka's KEYS file containing PGP keys we use to sign the release: > > > > > > https://kafka.apache.org/KEYS > > > > > > > > > > > > * Release artifacts to be voted upon (source and binary): > > > > > > https://home.apache.org/~satishd/kafka-3.6.0-rc2/ > > > > > > > > > > > > * Maven artifacts to be voted upon: > > > > > > > > > https://repository.apache.org/content/groups/staging/org/apache/kafka/ > > > > > > > > > > > > * Javadoc: > > > > > > https://home.apache.org/~satishd/kafka-3.6.0-rc2/javadoc/ > > > > > > > > > > > > * Tag to be voted upon (off 3.6 branch) is the 3.6.0-rc2 tag: > > > > > > https://g
Re: [VOTE] 3.6.0 RC2
+1 (non-binding) 1. Built the source from 3.6 branch in scala 2.12 and 2.13 2. Ran all the unit and integration tests. 3. Ran quickstart and verified the produce-consume on a 3 node cluster. 4. Verified the tiered storage functionality with local-tiered storage. Thanks, Kamal On Mon, Oct 2, 2023 at 6:07 PM Divij Vaidya wrote: > + 1 (non-binding) > > Verifications: > 1. I ran a produce-consume workload with plaintext auth, JDK17, zstd > compression using an open messaging benchmark and found 3.6 to be better > than or equal to 3.5.1 across all dimensions. Notably, 3.6 had consistently > 6-7% lower CPU utilization, lesser spikes on P99 produce latencies and > overall lower P99.8 latencies. > > 2. I have verified that detached signature is correct using > https://www.apache.org/info/verification.html and the release manager > public keys are available at > https://keys.openpgp.org/search?q=F65DC3423D4CD7B9 > > 3. I have verified that all metrics emitted in 3.5.1 (with Zk) are also > being emitted in 3.6.0 (with Zk). > > Problems (but not blockers): > 1. Metrics added in > > https://github.com/apache/kafka/commit/2f71708955b293658cec3b27e9a5588d39c38d7e > aren't available in the documentation (cc: Justine). I don't consider this > as a release blocker but we should add it as a fast follow-up. > > 2. Metric added in > > https://github.com/apache/kafka/commit/a900794ace4dcf1f9dadee27fbd8b63979532a18 > isn't available in documentation (cc: David). I don't consider this as a > release blocker but we should add it as a fast follow-up. > > -- > Divij Vaidya > > > > On Mon, Oct 2, 2023 at 9:50 AM Federico Valeri > wrote: > > > Hi Satish, I did the following to verify the release: > > > > - Built from source with Java 17 and Scala 2.13 > > - Ran all unit and integration tests > > - Spot checked documentation > > - Ran custom client applications using staging artifacts on a 3-nodes > > cluster > > - Tested tiered storage with one of the available RSM implementations > > > > +1 (non binding) > > > > Thanks > > Fede > > > > On Mon, Oct 2, 2023 at 8:50 AM Luke Chen wrote: > > > > > > Hi Satish, > > > > > > I verified with: > > > 1. Ran quick start in KRaft for scala 2.12 artifact > > > 2. Making sure the checksum are correct > > > 3. Browsing release notes, documents, javadocs, protocols. > > > 4. Verified the tiered storage feature works well. > > > > > > +1 (binding). > > > > > > Thanks. > > > Luke > > > > > > > > > > > > On Mon, Oct 2, 2023 at 5:23 AM Jakub Scholz wrote: > > > > > > > +1 (non-binding). I used the Scala 2.13 binaries and the staged Maven > > > > artifacts and run my tests. Everything seems to work fine for me. > > > > > > > > Thanks > > > > Jakub > > > > > > > > On Fri, Sep 29, 2023 at 8:17 PM Satish Duggana < > > satish.dugg...@gmail.com> > > > > wrote: > > > > > > > > > Hello Kafka users, developers and client-developers, > > > > > > > > > > This is the third candidate for the release of Apache Kafka 3.6.0. > > > > > Some of the major features include: > > > > > > > > > > * KIP-405 : Kafka Tiered Storage > > > > > * KIP-868 : KRaft Metadata Transactions > > > > > * KIP-875: First-class offsets support in Kafka Connect > > > > > * KIP-898: Modernize Connect plugin discovery > > > > > * KIP-938: Add more metrics for measuring KRaft performance > > > > > * KIP-902: Upgrade Zookeeper to 3.8.1 > > > > > * KIP-917: Additional custom metadata for remote log segment > > > > > > > > > > Release notes for the 3.6.0 release: > > > > > > https://home.apache.org/~satishd/kafka-3.6.0-rc2/RELEASE_NOTES.html > > > > > > > > > > *** Please download, test and vote by Tuesday, October 3, 12pm PT > > > > > > > > > > Kafka's KEYS file containing PGP keys we use to sign the release: > > > > > https://kafka.apache.org/KEYS > > > > > > > > > > * Release artifacts to be voted upon (source and binary): > > > > > https://home.apache.org/~satishd/kafka-3.6.0-rc2/ > > > > > > > > > > * Maven artifacts to be voted upon: > > > > > > > https://repository.apache.org/content/groups/staging/org/apache/kafka/ > > > > > > > > > > * Javadoc: > > > > > https://home.apache.org/~satishd/kafka-3.6.0-rc2/javadoc/ > > > > > > > > > > * Tag to be voted upon (off 3.6 branch) is the 3.6.0-rc2 tag: > > > > > https://github.com/apache/kafka/releases/tag/3.6.0-rc2 > > > > > > > > > > * Documentation: > > > > > https://kafka.apache.org/36/documentation.html > > > > > > > > > > * Protocol: > > > > > https://kafka.apache.org/36/protocol.html > > > > > > > > > > * Successful Jenkins builds for the 3.6 branch: > > > > > There are a few runs of unit/integration tests. You can see the > > latest > > > > > at https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.6/. We > > will > > > > > continue running a few more iterations. > > > > > System tests: > > > > > We will send an update once we have the results. > > > > > > > > > > Thanks, > > > > > Satish. > > > > > > > > > > > >
Re: [VOTE] 3.6.0 RC2
+ 1 (non-binding) Verifications: 1. I ran a produce-consume workload with plaintext auth, JDK17, zstd compression using an open messaging benchmark and found 3.6 to be better than or equal to 3.5.1 across all dimensions. Notably, 3.6 had consistently 6-7% lower CPU utilization, lesser spikes on P99 produce latencies and overall lower P99.8 latencies. 2. I have verified that detached signature is correct using https://www.apache.org/info/verification.html and the release manager public keys are available at https://keys.openpgp.org/search?q=F65DC3423D4CD7B9 3. I have verified that all metrics emitted in 3.5.1 (with Zk) are also being emitted in 3.6.0 (with Zk). Problems (but not blockers): 1. Metrics added in https://github.com/apache/kafka/commit/2f71708955b293658cec3b27e9a5588d39c38d7e aren't available in the documentation (cc: Justine). I don't consider this as a release blocker but we should add it as a fast follow-up. 2. Metric added in https://github.com/apache/kafka/commit/a900794ace4dcf1f9dadee27fbd8b63979532a18 isn't available in documentation (cc: David). I don't consider this as a release blocker but we should add it as a fast follow-up. -- Divij Vaidya On Mon, Oct 2, 2023 at 9:50 AM Federico Valeri wrote: > Hi Satish, I did the following to verify the release: > > - Built from source with Java 17 and Scala 2.13 > - Ran all unit and integration tests > - Spot checked documentation > - Ran custom client applications using staging artifacts on a 3-nodes > cluster > - Tested tiered storage with one of the available RSM implementations > > +1 (non binding) > > Thanks > Fede > > On Mon, Oct 2, 2023 at 8:50 AM Luke Chen wrote: > > > > Hi Satish, > > > > I verified with: > > 1. Ran quick start in KRaft for scala 2.12 artifact > > 2. Making sure the checksum are correct > > 3. Browsing release notes, documents, javadocs, protocols. > > 4. Verified the tiered storage feature works well. > > > > +1 (binding). > > > > Thanks. > > Luke > > > > > > > > On Mon, Oct 2, 2023 at 5:23 AM Jakub Scholz wrote: > > > > > +1 (non-binding). I used the Scala 2.13 binaries and the staged Maven > > > artifacts and run my tests. Everything seems to work fine for me. > > > > > > Thanks > > > Jakub > > > > > > On Fri, Sep 29, 2023 at 8:17 PM Satish Duggana < > satish.dugg...@gmail.com> > > > wrote: > > > > > > > Hello Kafka users, developers and client-developers, > > > > > > > > This is the third candidate for the release of Apache Kafka 3.6.0. > > > > Some of the major features include: > > > > > > > > * KIP-405 : Kafka Tiered Storage > > > > * KIP-868 : KRaft Metadata Transactions > > > > * KIP-875: First-class offsets support in Kafka Connect > > > > * KIP-898: Modernize Connect plugin discovery > > > > * KIP-938: Add more metrics for measuring KRaft performance > > > > * KIP-902: Upgrade Zookeeper to 3.8.1 > > > > * KIP-917: Additional custom metadata for remote log segment > > > > > > > > Release notes for the 3.6.0 release: > > > > https://home.apache.org/~satishd/kafka-3.6.0-rc2/RELEASE_NOTES.html > > > > > > > > *** Please download, test and vote by Tuesday, October 3, 12pm PT > > > > > > > > Kafka's KEYS file containing PGP keys we use to sign the release: > > > > https://kafka.apache.org/KEYS > > > > > > > > * Release artifacts to be voted upon (source and binary): > > > > https://home.apache.org/~satishd/kafka-3.6.0-rc2/ > > > > > > > > * Maven artifacts to be voted upon: > > > > > https://repository.apache.org/content/groups/staging/org/apache/kafka/ > > > > > > > > * Javadoc: > > > > https://home.apache.org/~satishd/kafka-3.6.0-rc2/javadoc/ > > > > > > > > * Tag to be voted upon (off 3.6 branch) is the 3.6.0-rc2 tag: > > > > https://github.com/apache/kafka/releases/tag/3.6.0-rc2 > > > > > > > > * Documentation: > > > > https://kafka.apache.org/36/documentation.html > > > > > > > > * Protocol: > > > > https://kafka.apache.org/36/protocol.html > > > > > > > > * Successful Jenkins builds for the 3.6 branch: > > > > There are a few runs of unit/integration tests. You can see the > latest > > > > at https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.6/. We > will > > > > continue running a few more iterations. > > > > System tests: > > > > We will send an update once we have the results. > > > > > > > > Thanks, > > > > Satish. > > > > > > > >
Re: [VOTE] 3.6.0 RC2
Hi Satish, I did the following to verify the release: - Built from source with Java 17 and Scala 2.13 - Ran all unit and integration tests - Spot checked documentation - Ran custom client applications using staging artifacts on a 3-nodes cluster - Tested tiered storage with one of the available RSM implementations +1 (non binding) Thanks Fede On Mon, Oct 2, 2023 at 8:50 AM Luke Chen wrote: > > Hi Satish, > > I verified with: > 1. Ran quick start in KRaft for scala 2.12 artifact > 2. Making sure the checksum are correct > 3. Browsing release notes, documents, javadocs, protocols. > 4. Verified the tiered storage feature works well. > > +1 (binding). > > Thanks. > Luke > > > > On Mon, Oct 2, 2023 at 5:23 AM Jakub Scholz wrote: > > > +1 (non-binding). I used the Scala 2.13 binaries and the staged Maven > > artifacts and run my tests. Everything seems to work fine for me. > > > > Thanks > > Jakub > > > > On Fri, Sep 29, 2023 at 8:17 PM Satish Duggana > > wrote: > > > > > Hello Kafka users, developers and client-developers, > > > > > > This is the third candidate for the release of Apache Kafka 3.6.0. > > > Some of the major features include: > > > > > > * KIP-405 : Kafka Tiered Storage > > > * KIP-868 : KRaft Metadata Transactions > > > * KIP-875: First-class offsets support in Kafka Connect > > > * KIP-898: Modernize Connect plugin discovery > > > * KIP-938: Add more metrics for measuring KRaft performance > > > * KIP-902: Upgrade Zookeeper to 3.8.1 > > > * KIP-917: Additional custom metadata for remote log segment > > > > > > Release notes for the 3.6.0 release: > > > https://home.apache.org/~satishd/kafka-3.6.0-rc2/RELEASE_NOTES.html > > > > > > *** Please download, test and vote by Tuesday, October 3, 12pm PT > > > > > > Kafka's KEYS file containing PGP keys we use to sign the release: > > > https://kafka.apache.org/KEYS > > > > > > * Release artifacts to be voted upon (source and binary): > > > https://home.apache.org/~satishd/kafka-3.6.0-rc2/ > > > > > > * Maven artifacts to be voted upon: > > > https://repository.apache.org/content/groups/staging/org/apache/kafka/ > > > > > > * Javadoc: > > > https://home.apache.org/~satishd/kafka-3.6.0-rc2/javadoc/ > > > > > > * Tag to be voted upon (off 3.6 branch) is the 3.6.0-rc2 tag: > > > https://github.com/apache/kafka/releases/tag/3.6.0-rc2 > > > > > > * Documentation: > > > https://kafka.apache.org/36/documentation.html > > > > > > * Protocol: > > > https://kafka.apache.org/36/protocol.html > > > > > > * Successful Jenkins builds for the 3.6 branch: > > > There are a few runs of unit/integration tests. You can see the latest > > > at https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.6/. We will > > > continue running a few more iterations. > > > System tests: > > > We will send an update once we have the results. > > > > > > Thanks, > > > Satish. > > > > >
Re: [VOTE] 3.6.0 RC2
Hi Satish, I verified with: 1. Ran quick start in KRaft for scala 2.12 artifact 2. Making sure the checksum are correct 3. Browsing release notes, documents, javadocs, protocols. 4. Verified the tiered storage feature works well. +1 (binding). Thanks. Luke On Mon, Oct 2, 2023 at 5:23 AM Jakub Scholz wrote: > +1 (non-binding). I used the Scala 2.13 binaries and the staged Maven > artifacts and run my tests. Everything seems to work fine for me. > > Thanks > Jakub > > On Fri, Sep 29, 2023 at 8:17 PM Satish Duggana > wrote: > > > Hello Kafka users, developers and client-developers, > > > > This is the third candidate for the release of Apache Kafka 3.6.0. > > Some of the major features include: > > > > * KIP-405 : Kafka Tiered Storage > > * KIP-868 : KRaft Metadata Transactions > > * KIP-875: First-class offsets support in Kafka Connect > > * KIP-898: Modernize Connect plugin discovery > > * KIP-938: Add more metrics for measuring KRaft performance > > * KIP-902: Upgrade Zookeeper to 3.8.1 > > * KIP-917: Additional custom metadata for remote log segment > > > > Release notes for the 3.6.0 release: > > https://home.apache.org/~satishd/kafka-3.6.0-rc2/RELEASE_NOTES.html > > > > *** Please download, test and vote by Tuesday, October 3, 12pm PT > > > > Kafka's KEYS file containing PGP keys we use to sign the release: > > https://kafka.apache.org/KEYS > > > > * Release artifacts to be voted upon (source and binary): > > https://home.apache.org/~satishd/kafka-3.6.0-rc2/ > > > > * Maven artifacts to be voted upon: > > https://repository.apache.org/content/groups/staging/org/apache/kafka/ > > > > * Javadoc: > > https://home.apache.org/~satishd/kafka-3.6.0-rc2/javadoc/ > > > > * Tag to be voted upon (off 3.6 branch) is the 3.6.0-rc2 tag: > > https://github.com/apache/kafka/releases/tag/3.6.0-rc2 > > > > * Documentation: > > https://kafka.apache.org/36/documentation.html > > > > * Protocol: > > https://kafka.apache.org/36/protocol.html > > > > * Successful Jenkins builds for the 3.6 branch: > > There are a few runs of unit/integration tests. You can see the latest > > at https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.6/. We will > > continue running a few more iterations. > > System tests: > > We will send an update once we have the results. > > > > Thanks, > > Satish. > > >
Re: [VOTE] 3.6.0 RC2
+1 (non-binding). I used the Scala 2.13 binaries and the staged Maven artifacts and run my tests. Everything seems to work fine for me. Thanks Jakub On Fri, Sep 29, 2023 at 8:17 PM Satish Duggana wrote: > Hello Kafka users, developers and client-developers, > > This is the third candidate for the release of Apache Kafka 3.6.0. > Some of the major features include: > > * KIP-405 : Kafka Tiered Storage > * KIP-868 : KRaft Metadata Transactions > * KIP-875: First-class offsets support in Kafka Connect > * KIP-898: Modernize Connect plugin discovery > * KIP-938: Add more metrics for measuring KRaft performance > * KIP-902: Upgrade Zookeeper to 3.8.1 > * KIP-917: Additional custom metadata for remote log segment > > Release notes for the 3.6.0 release: > https://home.apache.org/~satishd/kafka-3.6.0-rc2/RELEASE_NOTES.html > > *** Please download, test and vote by Tuesday, October 3, 12pm PT > > Kafka's KEYS file containing PGP keys we use to sign the release: > https://kafka.apache.org/KEYS > > * Release artifacts to be voted upon (source and binary): > https://home.apache.org/~satishd/kafka-3.6.0-rc2/ > > * Maven artifacts to be voted upon: > https://repository.apache.org/content/groups/staging/org/apache/kafka/ > > * Javadoc: > https://home.apache.org/~satishd/kafka-3.6.0-rc2/javadoc/ > > * Tag to be voted upon (off 3.6 branch) is the 3.6.0-rc2 tag: > https://github.com/apache/kafka/releases/tag/3.6.0-rc2 > > * Documentation: > https://kafka.apache.org/36/documentation.html > > * Protocol: > https://kafka.apache.org/36/protocol.html > > * Successful Jenkins builds for the 3.6 branch: > There are a few runs of unit/integration tests. You can see the latest > at https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.6/. We will > continue running a few more iterations. > System tests: > We will send an update once we have the results. > > Thanks, > Satish. >
[VOTE] 3.6.0 RC2
Hello Kafka users, developers and client-developers, This is the third candidate for the release of Apache Kafka 3.6.0. Some of the major features include: * KIP-405 : Kafka Tiered Storage * KIP-868 : KRaft Metadata Transactions * KIP-875: First-class offsets support in Kafka Connect * KIP-898: Modernize Connect plugin discovery * KIP-938: Add more metrics for measuring KRaft performance * KIP-902: Upgrade Zookeeper to 3.8.1 * KIP-917: Additional custom metadata for remote log segment Release notes for the 3.6.0 release: https://home.apache.org/~satishd/kafka-3.6.0-rc2/RELEASE_NOTES.html *** Please download, test and vote by Tuesday, October 3, 12pm PT Kafka's KEYS file containing PGP keys we use to sign the release: https://kafka.apache.org/KEYS * Release artifacts to be voted upon (source and binary): https://home.apache.org/~satishd/kafka-3.6.0-rc2/ * Maven artifacts to be voted upon: https://repository.apache.org/content/groups/staging/org/apache/kafka/ * Javadoc: https://home.apache.org/~satishd/kafka-3.6.0-rc2/javadoc/ * Tag to be voted upon (off 3.6 branch) is the 3.6.0-rc2 tag: https://github.com/apache/kafka/releases/tag/3.6.0-rc2 * Documentation: https://kafka.apache.org/36/documentation.html * Protocol: https://kafka.apache.org/36/protocol.html * Successful Jenkins builds for the 3.6 branch: There are a few runs of unit/integration tests. You can see the latest at https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.6/. We will continue running a few more iterations. System tests: We will send an update once we have the results. Thanks, Satish.