[ANNOUNCE] Apache Kafka 3.7.0
The Apache Kafka community is pleased to announce the release of Apache Kafka 3.7.0 This is a minor release that includes new features, fixes, and improvements from 296 JIRAs An overview of the release and its notable changes can be found in the release blog post: https://kafka.apache.org/blog#apache_kafka_370_release_announcement All of the changes in this release can be found in the release notes: https://www.apache.org/dist/kafka/3.7.0/RELEASE_NOTES.html You can download the source and binary release (Scala 2.12, 2.13) from: https://kafka.apache.org/downloads#3.7.0 --- Apache Kafka is a distributed streaming platform with four core APIs: ** The Producer API allows an application to publish a stream of records to one or more Kafka topics. ** The Consumer API allows an application to subscribe to one or more topics and process the stream of records produced to them. ** The Streams API allows an application to act as a stream processor, consuming an input stream from one or more topics and producing an output stream to one or more output topics, effectively transforming the input streams to output streams. ** The Connector API allows building and running reusable producers or consumers that connect Kafka topics to existing applications or data systems. For example, a connector to a relational database might capture every change to a table. With these APIs, Kafka can be used for two broad classes of application: ** Building real-time streaming data pipelines that reliably get data between systems or applications. ** Building real-time streaming applications that transform or react to the streams of data. Apache Kafka is in use at large and small companies worldwide, including Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, Rabobank, Target, The New York Times, Uber, Yelp, and Zalando, among others. A big thank you to the following 146 contributors to this release! (Please report an unintended omission) Abhijeet Kumar, Akhilesh Chaganti, Alieh, Alieh Saeedi, Almog Gavra, Alok Thatikunta, Alyssa Huang, Aman Singh, Andras Katona, Andrew Schofield, Anna Sophie Blee-Goldman, Anton Agestam, Apoorv Mittal, Arnout Engelen, Arpit Goyal, Artem Livshits, Ashwin Pankaj, ashwinpankaj, atu-sharm, bachmanity1, Bob Barrett, Bruno Cadonna, Calvin Liu, Cerchie, chern, Chris Egerton, Christo Lolov, Colin Patrick McCabe, Colt McNealy, Crispin Bernier, David Arthur, David Jacot, David Mao, Deqi Hu, Dimitar Dimitrov, Divij Vaidya, Dongnuo Lyu, Eaugene Thomas, Eduwer Camacaro, Eike Thaden, Federico Valeri, Florin Akermann, Gantigmaa Selenge, Gaurav Narula, gongzhongqiang, Greg Harris, Guozhang Wang, Gyeongwon, Do, Hailey Ni, Hanyu Zheng, Hao Li, Hector Geraldino, hudeqi, Ian McDonald, Iblis Lin, Igor Soarez, iit2009060, Ismael Juma, Jakub Scholz, James Cheng, Jason Gustafson, Jay Wang, Jeff Kim, Jim Galasyn, John Roesler, Jorge Esteban Quilcate Otoya, Josep Prat, José Armando García Sancio, Jotaniya Jeel, Jouni Tenhunen, Jun Rao, Justine Olshan, Kamal Chandraprakash, Kirk True, kpatelatwork, kumarpritam863, Laglangyue, Levani Kokhreidze, Lianet Magrans, Liu Zeyu, Lucas Brutschy, Lucia Cerchie, Luke Chen, maniekes, Manikumar Reddy, mannoopj, Maros Orsak, Matthew de Detrich, Matthias J. Sax, Max Riedel, Mayank Shekhar Narula, Mehari Beyene, Michael Westerby, Mickael Maison, Nick Telford, Nikhil Ramakrishnan, Nikolay, Okada Haruki, olalamichelle, Omnia G.H Ibrahim, Owen Leung, Paolo Patierno, Philip Nee, Phuc-Hong-Tran, Proven Provenzano, Purshotam Chauhan, Qichao Chu, Matthias J. Sax, Rajini Sivaram, Renaldo Baur Filho, Ritika Reddy, Robert Wagner, Rohan, Ron Dagostino, Roon, runom, Ruslan Krivoshein, rykovsi, Sagar Rao, Said Boudjelda, Satish Duggana, shuoer86, Stanislav Kozlovski, Taher Ghaleb, Tang Yunzi, TapDang, Taras Ledkov, tkuramoto33, Tyler Bertrand, vamossagar12, Vedarth Sharma, Viktor Somogyi-Vass, Vincent Jiang, Walker Carlson, Wuzhengyu97, Xavier Léauté, Xiaobing Fang, yangy, Ritika Reddy, Yanming Zhou, Yash Mayya, yuyli, zhaohaidao, Zihao Lin, Ziming Deng We welcome your help and feedback. For more information on how to report problems, and to get involved, visit the project website at https://kafka.apache.org/ Thank you! Regards, Stanislav Kozlovski Release Manager for Apache Kafka 3.7.0
Re: [VOTE] 3.7.0 RC4
*(double-sending this to kafka-clients & users, as I forgot initially)* This vote passes with *10 +1 votes* (3 bindings) and no 0 or -1 votes. +1 votes PMC Members (binding): * Mickael Maison * Justine Olshan * Divij Vaidya Community (non-binding): * Proven Provenzano * Federico Valeri * Vedarth Sharma * Andrew Schofield * Paolo Patierno * Jakub Scholz * Josep Prat 0 votes * No votes -1 votes * No votes Vote thread: https://lists.apache.org/thread/71djwz292y2lzgwzm7n6n8o7x56zbgh9 I'll continue with the release process and the release announcement will follow ASAP. On Mon, Feb 26, 2024 at 10:42 AM Stanislav Kozlovski wrote: > > This vote passes with *10 +1 votes* (3 bindings) and no 0 or -1 votes. > > +1 votes > > PMC Members (binding): > * Mickael Maison > * Justine Olshan > * Divij Vaidya > > Community (non-binding): > * Proven Provenzano > * Federico Valeri > * Vedarth Sharma > * Andrew Schofield > * Paolo Patierno > * Jakub Scholz > * Josep Prat > > > > 0 votes > > * No votes > > > > -1 votes > > * No votes > > > > Vote thread: > https://lists.apache.org/thread/71djwz292y2lzgwzm7n6n8o7x56zbgh9 > > I'll continue with the release process and the release announcement will > follow ASAP. > > Best, > Stanislav > > On Sun, Feb 25, 2024 at 7:08 PM Mickael Maison > wrote: > >> Hi, >> >> Thanks for sorting out the docs issues. >> +1 (binding) >> >> Mickael >> >> On Fri, Feb 23, 2024 at 11:50 AM Stanislav Kozlovski >> wrote: >> > >> > Some quick updates: >> > >> > There were some inconsistencies between the documentation in the >> > apache/kafka repo and the one in kafka-site. The process is such that >> the >> > apache/kafka docs are the source of truth, but we had a few divergences >> in >> > the other repo. I have worked on correcting those with: >> > - MINOR: Reconcile upgrade.html with kafka-site/36's version >> > <https://github.com/apache/kafka/pull/15406> and cherry-picked it into >> the >> > 3.6 and 3.7 branches too >> > >> > Additionally, the 3.7 upgrade notes have been merged in apache/kafka - >> MINOR: >> > Add 3.7 upgrade notes <https://github.com/apache/kafka/pull/15407/files >> >. >> > >> > With that, I have opened a PR to move them to the kafka-site repository >> - >> > https://github.com/apache/kafka-site/pull/587. That is awaiting review. >> > >> > Similarly, the 3.7 blog post is ready for review again >> > <https://github.com/apache/kafka-site/pull/578> and awaiting a review >> on 37: >> > Update default docs to point to the 3.7.0 release docs >> > <https://github.com/apache/kafka-site/pull/582> >> > >> > I also have a WIP for fixing the 3.6 docs in the kafka-site repo >> > <https://github.com/apache/kafka-site/pull/586>. This isn't really >> related >> > to the release, but it's good to do. >> > >> > On Wed, Feb 21, 2024 at 4:55 AM Luke Chen wrote: >> > >> > > Hi all, >> > > >> > > I found there is a bug (KAFKA-16283 >> > > <https://issues.apache.org/jira/browse/KAFKA-16283>) in the built-in >> > > `RoundRobinPartitioner`, and it will cause only half of the partitions >> > > receiving the records. (I'm surprised our tests didn't catch it!?) >> > > After further testing, I found this issue already existed in v3.0.0. >> (I >> > > didn't test 2.x versions) >> > > I think this should not be a blocker to v3.7.0 since it's been there >> for >> > > years. >> > > But I think we should, at least, document it to notify users about >> this >> > > known issue. >> > > I've created 2 PRs to document it: >> > > https://github.com/apache/kafka/pull/15400 >> > > https://github.com/apache/kafka-site/pull/585 >> > > >> > > Let me know what you think. >> > > >> > > Thanks. >> > > Luke >> > > >> > > On Wed, Feb 21, 2024 at 10:52 AM Proven Provenzano >> > > wrote: >> > > >> > > > HI, >> > > > >> > > > I've downloaded, built from source and then validated JBOD with >> KRaft >> > > works >> > > > along with migrating a cluster with JBOD from ZK to KRaft works. >> > > > >> > > > +1 (nonbinding) from me. >> > >
[VOTE] 3.7.0 RC4
Hello Kafka users, developers and client-developers, This is the second candidate we are considering for release of Apache Kafka 3.7.0. Major changes include: - Early Access to KIP-848 - the next generation of the consumer rebalance protocol - Early Access to KIP-858: Adding JBOD support to KRaft - KIP-714: Observability into Client metrics via a standardized interface Release notes for the 3.7.0 release: https://home.apache.org/~stanislavkozlovski/kafka-3.7.0-rc4/RELEASE_NOTES.html *** Please download, test and vote by Thursday, February 15th, 9AM PST *** Kafka's KEYS file containing PGP keys we use to sign the release: https://kafka.apache.org/KEYS * Release artifacts to be voted upon (source and binary): https://home.apache.org/~stanislavkozlovski/kafka-3.7.0-rc4/ * Docker release artifact to be voted upon: apache/kafka:3.7.0-rc4 * Maven artifacts to be voted upon: https://repository.apache.org/content/groups/staging/org/apache/kafka/ * Javadoc: https://home.apache.org/~stanislavkozlovski/kafka-3.7.0-rc4/javadoc/ * Tag to be voted upon (off 3.7 branch) is the 3.7.0 tag: https://github.com/apache/kafka/releases/tag/3.7.0-rc4 * Documentation: https://kafka.apache.org/37/documentation.html * Protocol: https://kafka.apache.org/37/protocol.html * Successful Jenkins builds for the 3.7 branch: Unit/integration tests: I am in the process of running and analyzing these. System tests: I am in the process of running these. Expect a follow-up over the weekend * Successful Docker Image Github Actions Pipeline for 3.7 branch: Docker Build Test Pipeline: https://github.com/apache/kafka/actions/runs/7845614846 /** Best, Stanislav
Re: [VOTE] 3.7.0 RC2
Thanks for the work everybody. Providing a status update at the end of the week: - docs change explaining migration <https://github.com/apache/kafka/pull/15193> was merged - the blocker KAFKA-16162 <https://github.com/apache/kafka/pull/15270> was merged - the blocker KAFKA-14616 <https://github.com/apache/kafka/pull/15230> was merged - a small blocker problem with the shadow jar plugin <https://github.com/apache/kafka/pull/15308> - the blockers KAFKALESS-16157 & KAFKALESS-16195 aren't merged - the good-to-have KAFKA-16082 isn't merged I think we should prioritize merging KAFKALESS-16195 and *call JBOD EA*. I question whether we may find more blocker bugs in the next RC. The release is late by approximately a month so far, so I do want to scope down aggressively to meet the time-based goal. Best, Stanislav On Mon, Jan 29, 2024 at 5:46 PM Omnia Ibrahim wrote: > Hi Stan and Gaurav, > Just to clarify some points mentioned here before > KAFKA-14616: I raised a year ago so it's not related to JBOD work. It is > rather a blocker bug for KRAFT in general. The PR from Colin should fix > this. Am not sure if it is a blocker for 3.7 per-say as it was a major bug > since 3.3 and got missed from all other releases. > > Regarding the JBOD's work: > KAFKA-16082: Is not a blocker for 3.7 instead it's nice fix. The pr > https://github.com/apache/kafka/pull/15136 is quite a small one and was > approved by Proven and I but it is waiting for a committer's approval. > KAFKA-16162: This is a blocker for 3.7. Same it’s a small pr > https://github.com/apache/kafka/pull/15270 and it is approved Proven and > I and the PR is waiting for committer's approval. > KAFKA-16157: This is a blocker for 3.7. There is one small suggestion for > the pr https://github.com/apache/kafka/pull/15263 but I don't think any > of the current feedback is blocking the pr from getting approved. Assuming > we get a committer's approval on it. > KAFKA-16195: Same it's a blocker but it has approval from Proven and I > and we are waiting for committer's approval on the pr > https://github.com/apache/kafka/pull/15262. > > If we can’t get a committer approval for KAFKA-16162, KAFKA-16157 and > KAFKA-16195 in time for 3.7 then we can mark JBOD as early release > assuming we merge at least KAFKA-16195. > > Regards, > Omnia > > > On 26 Jan 2024, at 15:39, ka...@gnarula.com wrote: > > > > Apologies, I duplicated KAFKA-16157 twice in my previous message. I > intended to mention KAFKA-16195 > > with the PR at https://github.com/apache/kafka/pull/15262 as the second > JIRA. > > > > Thanks, > > Gaurav > > > >> On 26 Jan 2024, at 15:34, ka...@gnarula.com wrote: > >> > >> Hi Stan, > >> > >> I wanted to share some updates about the bugs you shared earlier. > >> > >> - KAFKA-14616: I've reviewed and tested the PR from Colin and have > observed > >> the fix works as intended. > >> - KAFKA-16162: I reviewed Proven's PR and found some gaps in the > proposed fix. I've > >> therefore raised https://github.com/apache/kafka/pull/15270 following > a discussion with Luke in JIRA. > >> - KAFKA-16082: I don't think this is marked as a blocker anymore. I'm > awaiting > >> feedback/reviews at https://github.com/apache/kafka/pull/15136 > >> > >> In addition to the above, there are 2 JIRAs I'd like to bring > everyone's attention to: > >> > >> - KAFKA-16157: This is similar to KAFKA-14616 and is marked as a > blocker. I've raised > >> https://github.com/apache/kafka/pull/15263 and am awaiting reviews on > it. > >> - KAFKA-16157: I raised this yesterday and have addressed feedback from > Luke. This should > >> hopefully get merged soon. > >> > >> Regards, > >> Gaurav > >> > >> > >>> On 24 Jan 2024, at 11:51, ka...@gnarula.com wrote: > >>> > >>> Hi Stanislav, > >>> > >>> Thanks for bringing these JIRAs/PRs up. > >>> > >>> I'll be testing the open PRs for KAFKA-14616 and KAFKA-16162 this week > and I hope to have some feedback > >>> by Friday. I gather the latter JIRA is marked as a WIP by Proven and > he's away. I'll try to build on his work in the meantime. > >>> > >>> As for KAFKA-16082, we haven't been able to deduce a data loss > scenario. There's a PR open > >>> by me for promoting an abandoned future replica with approvals from > Omnia and Proven, > >>> so I'd appreciate a committer reviewing it. > >>> > >>> Regards, > >>> Gaurav > >>> > >>> On 23 Jan 2024
Re: [VOTE] 3.7.0 RC2
mpact of the error messages?* > > >>>>> > > >>>>> I did not observe any obvious impact. I was able to send and > receive > > >>>>> messages as normally. But to be honest, I have no idea what else > > >>>>> this might impact, so I did not try anything special. > > >>>>> > > >>>>> I think everyone upgrading an existing KRaft cluster will go > through > > >>> this > > >>>>> stage (running Kafka 3.7 with an older metadata version for at > least > > a > > >>>>> while). So even if it is just a logged exception without any other > > >>>> impact I > > >>>>> wonder if it might scare users from upgrading. But I leave it to > > >>> others > > >>>> to > > >>>>> decide if this is a blocker or not. > > >>>>> > > >>>> > > >>>> Hi Jakub, > > >>>> > > >>>> Thanks for trying the RC. I think what you found is a blocker bug > > >>> because > > >>>> it will generate huge amount of logspam. I guess we didn't find it > in > > >>> junit > > >>>> tests since logspam doesn't fail the automated tests. But certainly > > it's > > >>>> not suitable for production. Did you file a JIRA yet? > > >>>> > > >>>>> On Sun, Jan 14, 2024 at 10:17 PM Stanislav Kozlovski > > >>>>> wrote: > > >>>>> > > >>>>>> Hey Luke, > > >>>>>> > > >>>>>> This is an interesting problem. Given the fact that the KIP for > > >>> having a > > >>>>>> 3.8 release passed, I think it weights the scale towards not > calling > > >>>> this a > > >>>>>> blocker and expecting it to be solved in 3.7.1. > > >>>>>> > > >>>>>> It is unfortunate that it would not seem safe to migrate to KRaft > in > > >>>> 3.7.0 > > >>>>>> (given the inability to rollback safely), but if that's true - the > > >>> same > > >>>>>> case would apply for 3.6.0. So in any case users w\ould be > expected > > >>> to > > >>>> use a > > >>>>>> patch release for this. > > >>>> > > >>>> Hi Luke, > > >>>> > > >>>> Thanks for testing rollback. I think this is a case where the > > >>>> documentation is wrong. The intention was to for the steps to > > basically > > >>> be: > > >>>> > > >>>> 1. roll all the brokers into zk mode, but with migration enabled > > >>>> 2. take down the kraft quorum > > >>>> 3. rmr /controller, allowing a hybrid broker to take over. > > >>>> 4. roll all the brokers into zk mode without migration enabled (if > > >>> desired) > > >>>> > > >>>> With these steps, there isn't really unavailability since a ZK > > >>> controller > > >>>> can be elected quickly after the kraft quorum is gone. > > >>>> > > >>>>>> Further, since we will have a 3.8 release - it is > > >>>>>> likely we will ultimately recommend users upgrade from that > version > > >>>> given > > >>>>>> its aim is to have strategic KRaft feature parity with ZK. > > >>>>>> That being said, I am not 100% on this. Let me know whether you > > think > > >>>> this > > >>>>>> should block the release, Luke. I am also tagging Colin and David > to > > >>>> weigh > > >>>>>> in with their opinions, as they worked on the migration logic. > > >>>> > > >>>> The rollback docs are new in 3.7 so the fact that they're wrong is a > > >>> clear > > >>>> blocker, I think. But easy to fix, I believe. I will create a PR. > > >>>> > > >>>> best, > > >>>> Colin > > >>>> > > >>>>>> > > >>>>>> Hey Kirk and Chris, > > >>>>>> > > >>>>>> Unless I'm missing something - KAFKALESS-16029 is simply a bad log > > >>> due &g
Re: [VOTE] 3.7.0 RC2
I wanted to circle back and confirm the integration tests + system tests, plus give an overall update regarding status. The integration tests have a fair amount of flakes. I ran and inspected 3 consecutive builds (57 <https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.7/57/>, 58 <https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.7/58/>, 59 <https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.7/59/>), then cross-checked each run's failures via a script of mine to see any consistent failures. Three tests proved very flaky. Two are related to KIP-848 running under KRaft. The third one is a Trogdor test. All 3 tests pass locally, hence I deem them not blockers for the release. Especially since KIP-848 is in early access, I am not particularly concerned with a flaky test. I opened three JIRAs to track them: - https://issues.apache.org/jira/browse/KAFKA-16134 - https://issues.apache.org/jira/browse/KAFKA-16135 - https://issues.apache.org/jira/browse/KAFKA-16136 As for the system tests, I again ran 2 consecutive builds (1 <https://confluent-kafka-branch-builder-system-test-results.s3-us-west-2.amazonaws.com/system-test-kafka-branch-builder--1705045120--apache--3.7--d071cceffc/2024-01-11--001./2024-01-11--001./report.html>, 2 <https://confluent-kafka-branch-builder-system-test-results.s3-us-west-2.amazonaws.com/system-test-kafka-branch-builder--1705113592--apache--3.7--d071cceffc/2024-01-12--001./2024-01-12--001./report.html>) and I found 4 tests that exhibit consecutive failures. - The whole analysis: https://hackmd.io/@hOneAGCrSmKSpL8VF-1HWQ/HyRgRJmta The failing tests: StreamsStandbyTask - https://issues.apache.org/jira/browse/KAFKA-16141 StreamsUpgradeTest - https://issues.apache.org/jira/browse/KAFKA-16139 QuotaTest - https://issues.apache.org/jira/browse/KAFKA-16138 ZookeeperMigrationTest - https://issues.apache.org/jira/browse/KAFKA-16140 I am reaching out to subject matter experts regarding the failures. Thanks to everyone who contributed in testing the release. Here is a general update regarding known blockers that were recently found: We are treating https://issues.apache.org/jira/browse/KAFKA-16131 and https://issues.apache.org/jira/browse/KAFKA-16101 as blockers. https://issues.apache.org/jira/browse/KAFKA-16132 is a potential other issue that will likely be treated as a blocker Best, Stanislav On Mon, Jan 15, 2024 at 12:04 PM Jakub Scholz wrote: > *> Hi Jakub,> > Thanks for trying the RC. I think what you found is a > blocker bug because it * > *> will generate huge amount of logspam. I guess we didn't find it in junit > tests * > *> since logspam doesn't fail the automated tests. But certainly it's not > suitable * > *> for production. Did you file a JIRA yet?* > > Hi Colin, > > I opened https://issues.apache.org/jira/browse/KAFKA-16131. > > Thanks & Regards > Jakub > > On Mon, Jan 15, 2024 at 8:57 AM Colin McCabe wrote: > > > Hi Stanislav, > > > > Thanks for making the first RC. The fact that it's titled RC2 is messing > > with my mind a bit. I hope this doesn't make people think that we're > > farther along than we are, heh. > > > > On Sun, Jan 14, 2024, at 13:54, Jakub Scholz wrote: > > > *> Nice catch! It does seem like we should have gated this behind the > > > metadata> version as KIP-858 implies. Is the cluster configured with > > > multiple log> dirs? What is the impact of the error messages?* > > > > > > I did not observe any obvious impact. I was able to send and receive > > > messages as normally. But to be honest, I have no idea what else > > > this might impact, so I did not try anything special. > > > > > > I think everyone upgrading an existing KRaft cluster will go through > this > > > stage (running Kafka 3.7 with an older metadata version for at least a > > > while). So even if it is just a logged exception without any other > > impact I > > > wonder if it might scare users from upgrading. But I leave it to others > > to > > > decide if this is a blocker or not. > > > > > > > Hi Jakub, > > > > Thanks for trying the RC. I think what you found is a blocker bug because > > it will generate huge amount of logspam. I guess we didn't find it in > junit > > tests since logspam doesn't fail the automated tests. But certainly it's > > not suitable for production. Did you file a JIRA yet? > > > > > On Sun, Jan 14, 2024 at 10:17 PM Stanislav Kozlovski > > > wrote: > > > > > >> Hey Luke, > > >> > > >> This is an interesting problem. Given the fact that the KIP for > having a > > >> 3.8 release passed, I think it weights the scale towards not call
[VOTE] 3.7.0 RC2
Hello Kafka users, developers, and client-developers, This is the first candidate for release of Apache Kafka 3.7.0. Note it's named "RC2" because I had a few "failed" RCs that I had cut/uploaded but ultimately had to scrap prior to announcing due to new blockers arriving before I could even announce them. Further - I haven't yet been able to set up the system tests successfully. And the integration/unit tests do have a few failures that I have to spend time triaging. I would appreciate any help in case anyone notices any tests failing that they're subject matters experts in. Expect me to follow up in a day or two with more detailed analysis. Major changes include: - Early Access to KIP-848 - the next generation of the consumer rebalance protocol - KIP-858: Adding JBOD support to KRaft - KIP-714: Observability into Client metrics via a standardized interface Check more information in the WIP blog post: https://github.com/apache/kafka-site/pull/578 Release notes for the 3.7.0 release: https://home.apache.org/~stanislavkozlovski/kafka-3.7.0-rc2/RELEASE_NOTES.html *** Please download, test and vote by Thursday, January 18, 9am PT *** Usually these deadlines tend to be 2-3 days, but due to this being the first RC and the tests not having ran yet, I am giving it a bit more time. Kafka's KEYS file containing PGP keys we use to sign the release: https://kafka.apache.org/KEYS * Release artifacts to be voted upon (source and binary): https://home.apache.org/~stanislavkozlovski/kafka-3.7.0-rc2/ * Docker release artifact to be voted upon: apache/kafka:3.7.0-rc2 * Maven artifacts to be voted upon: https://repository.apache.org/content/groups/staging/org/apache/kafka/ * Javadoc: https://home.apache.org/~stanislavkozlovski/kafka-3.7.0-rc2/javadoc/ * Tag to be voted upon (off 3.7 branch) is the 3.7.0 tag: https://github.com/apache/kafka/releases/tag/3.7.0-rc2 * Documentation: https://kafka.apache.org/37/documentation.html * Protocol: https://kafka.apache.org/37/protocol.html * Successful Jenkins builds for the 3.7 branch: Unit/integration tests: https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.7/58/ There are failing tests here. I have to follow up with triaging some of the failures and figuring out if they're actual problems or simply flakes. System tests: https://jenkins.confluent.io/job/system-test-kafka/job/3.7/ No successful system test runs yet. I am working on getting the job to run. * Successful Docker Image Github Actions Pipeline for 3.7 branch: Attached are the scan_report and report_jvm output files from the Docker Build run: https://github.com/apache/kafka/actions/runs/7486094960/job/20375761673 And the final docker image build job - Docker Build Test Pipeline: https://github.com/apache/kafka/actions/runs/7486178277 The image is apache/kafka:3.7.0-rc2 - https://hub.docker.com/layers/apache/kafka/3.7.0-rc2/images/sha256-5b4707c08170d39549fbb6e2a3dbb83936a50f987c0c097f23cb26b4c210c226?context=explore /** Thanks, Stanislav Kozlovski kafka/test:test (alpine 3.18.5) === Total: 0 (HIGH: 0, CRITICAL: 0)
Re: [ANNOUNCE] Apache Kafka 2.6.0
Thanks for driving the release Randall! Congratulations to everybody involved - awesome work! On Thu, Aug 6, 2020 at 5:21 PM Randall Hauch wrote: > The Apache Kafka community is pleased to announce the release for Apache > Kafka 2.6.0 > > * TLSv1.3 has been enabled by default for Java 11 or newer. > * Significant performance improvements, especially when the broker has > large numbers of partitions > * Smooth scaling out of Kafka Streams applications > * Kafka Streams support for emit on change > * New metrics for better operational insight > * Kafka Connect can automatically create topics for source connectors > * Improved error reporting options for sink connectors in Kafka Connect > * New Filter and conditional SMTs in Kafka Connect > * The default value for the `client.dns.lookup` configuration is > now `use_all_dns_ips` > * Upgrade Zookeeper to 3.5.8 > > This release also includes other features, 74 improvements, 175 bug fixes, > plus other changes. > > All of the changes in this release can be found in the release notes: > https://www.apache.org/dist/kafka/2.6.0/RELEASE_NOTES.html > > > You can download the source and binary release (Scala 2.12 and 2.13) from: > https://kafka.apache.org/downloads#2.6.0 > > > --- > > > Apache Kafka is a distributed streaming platform with four core APIs: > > > ** The Producer API allows an application to publish a stream of records to > one or more Kafka topics. > > ** The Consumer API allows an application to subscribe to one or more > topics and process the stream of records produced to them. > > ** The Streams API allows an application to act as a stream processor, > consuming an input stream from one or more topics and producing an > output stream to one or more output topics, effectively transforming the > input streams to output streams. > > ** The Connector API allows building and running reusable producers or > consumers that connect Kafka topics to existing applications or data > systems. For example, a connector to a relational database might > capture every change to a table. > > > With these APIs, Kafka can be used for two broad classes of application: > > ** Building real-time streaming data pipelines that reliably get data > between systems or applications. > > ** Building real-time streaming applications that transform or react > to the streams of data. > > > Apache Kafka is in use at large and small companies worldwide, including > Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, Rabobank, > Target, The New York Times, Uber, Yelp, and Zalando, among others. > > A big thank you for the following 127 contributors to this release! > > 17hao, A. Sophie Blee-Goldman, Aakash Shah, Adam Bellemare, Agam Brahma, > Alaa Zbair, Alexandra Rodoni, Andras Katona, Andrew Olson, Andy Coates, > Aneel Nazareth, Anna Povzner, Antony Stubbs, Arjun Satish, Auston, avalsa, > Badai Aqrandista, belugabehr, Bill Bejeck, Bob Barrett, Boyang Chen, Brian > Bushree, Brian Byrne, Bruno Cadonna, Charles Feduke, Chia-Ping Tsai, Chris > Egerton, Colin Patrick McCabe, Daniel, Daniel Beskin, David Arthur, David > Jacot, David Mao, dengziming, Dezhi “Andy” Fang, Dima Reznik, Dominic > Evans, Ego, Eric Bolinger, Evelyn Bayes, Ewen Cheslack-Postava, fantayeneh, > feyman2016, Florian Hussonnois, Gardner Vickers, Greg Harris, Gunnar > Morling, Guozhang Wang, high.lee, Hossein Torabi, huxi, Ismael Juma, Jason > Gustafson, Jeff Huang, jeff kim, Jeff Widman, Jeremy Custenborder, Jiamei > Xie, jiameixie, jiao, Jim Galasyn, Joel Hamill, John Roesler, Jorge Esteban > Quilcate Otoya, José Armando García Sancio, Konstantine Karantasis, Kowshik > Prakasam, Kun Song, Lee Dongjin, Leonard Ge, Lev Zemlyanov, Levani > Kokhreidze, Liam Clarke-Hutchinson, Lucas Bradstreet, Lucent-Wong, Magnus > Edenhill, Manikumar Reddy, Mario Molina, Matthew Wong, Matthias J. Sax, > maulin-vasavada, Michael Viamari, Michal T, Mickael Maison, Mitch, Navina > Ramesh, Navinder Pal Singh Brar, nicolasguyomar, Nigel Liang, Nikolay, > Okada Haruki, Paul, Piotr Fras, Radai Rosenblatt, Rajini Sivaram, Randall > Hauch, Rens Groothuijsen, Richard Yu, Rigel Bezerra de Melo, Rob Meng, > Rohan, Ron Dagostino, Sanjana Kaundinya, Scott, Scott Hendricks, sebwills, > Shailesh Panwar, showuon, SoontaekLim, Stanislav Kozlovski, Steve > Rodrigues, Svend Vanderveken, Sönke Liebau, THREE LEVEL HELMET, Tom > Bentley, Tu V. Tran, Valeria, Vikas Singh, Viktor Somogyi, vinoth chandar, > Vito Jeng, Xavier Léauté, xiaodongdu, Zach Zhang, zhaohaidao, zshuo, 阿洋 > > We welcome your help and feedback. For more information on how to > report problems, and to get involved, visit the project website at > https://kafka.apache.org/ > > Thank you! > > > Regards, > > Randall Hauch > -- Best, Stanislav
