Re: [kafka-clients] Re: [ANNOUNCE] Apache Kafka 3.7.0

[Satish Duggana]

Thanks Stanislav for all your hard work on running the release. Thanks
to all the contributors to this release.


On Wed, 28 Feb 2024 at 13:43, Bruno Cadonna  wrote:
>
> Thanks Stan and all contributors for the release!
>
> Best,
> Bruno
>
> On 2/27/24 7:01 PM, Stanislav Kozlovski wrote:
> > The Apache Kafka community is pleased to announce the release of
> > Apache Kafka 3.7.0
> >
> > This is a minor release that includes new features, fixes, and
> > improvements from 296 JIRAs
> >
> > An overview of the release and its notable changes can be found in the
> > release blog post:
> > https://kafka.apache.org/blog#apache_kafka_370_release_announcement
> >
> > All of the changes in this release can be found in the release notes:
> > https://www.apache.org/dist/kafka/3.7.0/RELEASE_NOTES.html
> >
> > You can download the source and binary release (Scala 2.12, 2.13) from:
> > https://kafka.apache.org/downloads#3.7.0
> >
> > ---
> >
> >
> > Apache Kafka is a distributed streaming platform with four core APIs:
> >
> >
> > ** The Producer API allows an application to publish a stream of records to
> > one or more Kafka topics.
> >
> > ** The Consumer API allows an application to subscribe to one or more
> > topics and process the stream of records produced to them.
> >
> > ** The Streams API allows an application to act as a stream processor,
> > consuming an input stream from one or more topics and producing an
> > output stream to one or more output topics, effectively transforming the
> > input streams to output streams.
> >
> > ** The Connector API allows building and running reusable producers or
> > consumers that connect Kafka topics to existing applications or data
> > systems. For example, a connector to a relational database might
> > capture every change to a table.
> >
> >
> > With these APIs, Kafka can be used for two broad classes of application:
> >
> > ** Building real-time streaming data pipelines that reliably get data
> > between systems or applications.
> >
> > ** Building real-time streaming applications that transform or react
> > to the streams of data.
> >
> >
> > Apache Kafka is in use at large and small companies worldwide, including
> > Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, Rabobank,
> > Target, The New York Times, Uber, Yelp, and Zalando, among others.
> >
> > A big thank you to the following 146 contributors to this release!
> > (Please report an unintended omission)
> >
> > Abhijeet Kumar, Akhilesh Chaganti, Alieh, Alieh Saeedi, Almog Gavra,
> > Alok Thatikunta, Alyssa Huang, Aman Singh, Andras Katona, Andrew
> > Schofield, Anna Sophie Blee-Goldman, Anton Agestam, Apoorv Mittal,
> > Arnout Engelen, Arpit Goyal, Artem Livshits, Ashwin Pankaj,
> > ashwinpankaj, atu-sharm, bachmanity1, Bob Barrett, Bruno Cadonna,
> > Calvin Liu, Cerchie, chern, Chris Egerton, Christo Lolov, Colin
> > Patrick McCabe, Colt McNealy, Crispin Bernier, David Arthur, David
> > Jacot, David Mao, Deqi Hu, Dimitar Dimitrov, Divij Vaidya, Dongnuo
> > Lyu, Eaugene Thomas, Eduwer Camacaro, Eike Thaden, Federico Valeri,
> > Florin Akermann, Gantigmaa Selenge, Gaurav Narula, gongzhongqiang,
> > Greg Harris, Guozhang Wang, Gyeongwon, Do, Hailey Ni, Hanyu Zheng, Hao
> > Li, Hector Geraldino, hudeqi, Ian McDonald, Iblis Lin, Igor Soarez,
> > iit2009060, Ismael Juma, Jakub Scholz, James Cheng, Jason Gustafson,
> > Jay Wang, Jeff Kim, Jim Galasyn, John Roesler, Jorge Esteban Quilcate
> > Otoya, Josep Prat, José Armando García Sancio, Jotaniya Jeel, Jouni
> > Tenhunen, Jun Rao, Justine Olshan, Kamal Chandraprakash, Kirk True,
> > kpatelatwork, kumarpritam863, Laglangyue, Levani Kokhreidze, Lianet
> > Magrans, Liu Zeyu, Lucas Brutschy, Lucia Cerchie, Luke Chen, maniekes,
> > Manikumar Reddy, mannoopj, Maros Orsak, Matthew de Detrich, Matthias
> > J. Sax, Max Riedel, Mayank Shekhar Narula, Mehari Beyene, Michael
> > Westerby, Mickael Maison, Nick Telford, Nikhil Ramakrishnan, Nikolay,
> > Okada Haruki, olalamichelle, Omnia G.H Ibrahim, Owen Leung, Paolo
> > Patierno, Philip Nee, Phuc-Hong-Tran, Proven Provenzano, Purshotam
> > Chauhan, Qichao Chu, Matthias J. Sax, Rajini Sivaram, Renaldo Baur
> > Filho, Ritika Reddy, Robert Wagner, Rohan, Ron Dagostino, Roon, runom,
> > Ruslan Krivoshein, rykovsi, Sagar Rao, Said Boudjelda, Satish Duggana,
> > shuoer86, Stanislav Kozlovski, Taher Ghaleb, Tang Yunzi, TapDa

Re: [ANNOUNCE] Apache Kafka 3.6.0

[Satish Duggana]

I'd like to clarify calling out 3.6.0 as a minor release.

We use semantic versioning*(Major.Minor.Patch) in Apache Kafka
releases. Releases like 3.5.0 or 3.6.0 are called minor releases
according to the semantic versioning. The next major release will be
4.0.0.

3.6.0 release is packed with big features and improvements as
mentioned in the blog post[1], and release notes[2], even though it is
called a minor release :)

1. https://kafka.apache.org/blog
2. https://downloads.apache.org/kafka/3.6.0/RELEASE_NOTES.html
* https://semver.org/

Thanks,
Satish.


On Wed, 11 Oct 2023 at 12:00, Luke Chen  wrote:
>
> Thanks for running the release, Satish!
>
> BTW, 3.6.0 should be a major release, not a minor one. :)
>
> Luke
>
> On Wed, Oct 11, 2023 at 1:39 PM Satish Duggana  wrote:
>
> > The Apache Kafka community is pleased to announce the release for
> > Apache Kafka 3.6.0
> >
> > This is a minor release and it includes fixes and improvements from 238
> > JIRAs.
> >
> > All of the changes in this release can be found in the release notes:
> > https://www.apache.org/dist/kafka/3.6.0/RELEASE_NOTES.html
> >
> > An overview of the release can be found in our announcement blog post:
> > https://kafka.apache.org/blog
> >
> > You can download the source and binary release (Scala 2.12 and Scala 2.13)
> > from:
> > https://kafka.apache.org/downloads#3.6.0
> >
> >
> > ---
> >
> >
> > Apache Kafka is a distributed streaming platform with four core APIs:
> >
> >
> > ** The Producer API allows an application to publish a stream of records to
> > one or more Kafka topics.
> >
> > ** The Consumer API allows an application to subscribe to one or more
> > topics and process the stream of records produced to them.
> >
> > ** The Streams API allows an application to act as a stream processor,
> > consuming an input stream from one or more topics and producing an
> > output stream to one or more output topics, effectively transforming the
> > input streams to output streams.
> >
> > ** The Connector API allows building and running reusable producers or
> > consumers that connect Kafka topics to existing applications or data
> > systems. For example, a connector to a relational database might
> > capture every change to a table.
> >
> >
> > With these APIs, Kafka can be used for two broad classes of application:
> >
> > ** Building real-time streaming data pipelines that reliably get data
> > between systems or applications.
> >
> > ** Building real-time streaming applications that transform or react
> > to the streams of data.
> >
> >
> > Apache Kafka is in use at large and small companies worldwide, including
> > Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, Rabobank,
> > Target, The New York Times, Uber, Yelp, and Zalando, among others.
> >
> > A big thank you for the following 139 contributors to this release!
> > (Please report an unintended omission)
> >
> > This was a community effort, so thank you to everyone who contributed
> > to this release, including all our users and our 139 contributors:
> > A. Sophie Blee-Goldman, Aaron Ai, Abhijeet Kumar, aindriu-aiven,
> > Akhilesh Chaganti, Alexandre Dupriez, Alexandre Garnier, Alok
> > Thatikunta, Alyssa Huang, Aman Singh, Andras Katona, Andrew Schofield,
> > Andrew Grant, Aneel Kumar, Anton Agestam, Artem Livshits, atu-sharm,
> > bachmanity1, Bill Bejeck, Bo Gao, Bruno Cadonna, Calvin Liu, Chaitanya
> > Mukka, Chase Thomas, Cheryl Simmons, Chia-Ping Tsai, Chris Egerton,
> > Christo Lolov, Clay Johnson, Colin P. McCabe, Colt McNealy, d00791190,
> > Damon Xie, Danica Fine, Daniel Scanteianu, Daniel Urban, David Arthur,
> > David Jacot, David Mao, dengziming, Deqi Hu, Dimitar Dimitrov, Divij
> > Vaidya, DL1231, Dániel Urbán, Erik van Oosten, ezio, Farooq Qaiser,
> > Federico Valeri, flashmouse, Florin Akermann, Gabriel Oliveira,
> > Gantigmaa Selenge, Gaurav Narula, GeunJae Jeon, Greg Harris, Guozhang
> > Wang, Hailey Ni, Hao Li, Hector Geraldino, hudeqi, hzh0425, Iblis Lin,
> > iit2009060, Ismael Juma, Ivan Yurchenko, James Shaw, Jason Gustafson,
> > Jeff Kim, Jim Galasyn, John Roesler, Joobi S B, Jorge Esteban Quilcate
> > Otoya, Josep Prat, Joseph (Ting-Chou) Lin, José Armando García Sancio,
> > Jun Rao, Justine Olshan, Kamal Chandraprakash, Keith Wall, Kirk True,
> > Lianet Magrans, LinShunKang, Liu Zeyu, lixy, Lucas Bradstreet, Lucas
> > Brutschy, Lucent-Wong, Lucia Cerchie, Luke Chen, M

[ANNOUNCE] Apache Kafka 3.6.0

[Satish Duggana]

The Apache Kafka community is pleased to announce the release for
Apache Kafka 3.6.0

This is a minor release and it includes fixes and improvements from 238 JIRAs.

All of the changes in this release can be found in the release notes:
https://www.apache.org/dist/kafka/3.6.0/RELEASE_NOTES.html

An overview of the release can be found in our announcement blog post:
https://kafka.apache.org/blog

You can download the source and binary release (Scala 2.12 and Scala 2.13) from:
https://kafka.apache.org/downloads#3.6.0

---


Apache Kafka is a distributed streaming platform with four core APIs:


** The Producer API allows an application to publish a stream of records to
one or more Kafka topics.

** The Consumer API allows an application to subscribe to one or more
topics and process the stream of records produced to them.

** The Streams API allows an application to act as a stream processor,
consuming an input stream from one or more topics and producing an
output stream to one or more output topics, effectively transforming the
input streams to output streams.

** The Connector API allows building and running reusable producers or
consumers that connect Kafka topics to existing applications or data
systems. For example, a connector to a relational database might
capture every change to a table.


With these APIs, Kafka can be used for two broad classes of application:

** Building real-time streaming data pipelines that reliably get data
between systems or applications.

** Building real-time streaming applications that transform or react
to the streams of data.


Apache Kafka is in use at large and small companies worldwide, including
Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, Rabobank,
Target, The New York Times, Uber, Yelp, and Zalando, among others.

A big thank you for the following 139 contributors to this release!
(Please report an unintended omission)

This was a community effort, so thank you to everyone who contributed
to this release, including all our users and our 139 contributors:
A. Sophie Blee-Goldman, Aaron Ai, Abhijeet Kumar, aindriu-aiven,
Akhilesh Chaganti, Alexandre Dupriez, Alexandre Garnier, Alok
Thatikunta, Alyssa Huang, Aman Singh, Andras Katona, Andrew Schofield,
Andrew Grant, Aneel Kumar, Anton Agestam, Artem Livshits, atu-sharm,
bachmanity1, Bill Bejeck, Bo Gao, Bruno Cadonna, Calvin Liu, Chaitanya
Mukka, Chase Thomas, Cheryl Simmons, Chia-Ping Tsai, Chris Egerton,
Christo Lolov, Clay Johnson, Colin P. McCabe, Colt McNealy, d00791190,
Damon Xie, Danica Fine, Daniel Scanteianu, Daniel Urban, David Arthur,
David Jacot, David Mao, dengziming, Deqi Hu, Dimitar Dimitrov, Divij
Vaidya, DL1231, Dániel Urbán, Erik van Oosten, ezio, Farooq Qaiser,
Federico Valeri, flashmouse, Florin Akermann, Gabriel Oliveira,
Gantigmaa Selenge, Gaurav Narula, GeunJae Jeon, Greg Harris, Guozhang
Wang, Hailey Ni, Hao Li, Hector Geraldino, hudeqi, hzh0425, Iblis Lin,
iit2009060, Ismael Juma, Ivan Yurchenko, James Shaw, Jason Gustafson,
Jeff Kim, Jim Galasyn, John Roesler, Joobi S B, Jorge Esteban Quilcate
Otoya, Josep Prat, Joseph (Ting-Chou) Lin, José Armando García Sancio,
Jun Rao, Justine Olshan, Kamal Chandraprakash, Keith Wall, Kirk True,
Lianet Magrans, LinShunKang, Liu Zeyu, lixy, Lucas Bradstreet, Lucas
Brutschy, Lucent-Wong, Lucia Cerchie, Luke Chen, Manikumar Reddy,
Manyanda Chitimbo, Maros Orsak, Matthew de Detrich, Matthias J. Sax,
maulin-vasavada, Max Riedel, Mehari Beyene, Michal Cabak (@miccab),
Mickael Maison, Milind Mantri, minjian.cai, mojh7, Nikolay, Okada
Haruki, Omnia G H Ibrahim, Owen Leung, Philip Nee, prasanthV, Proven
Provenzano, Purshotam Chauhan, Qichao Chu, Rajini Sivaram, Randall
Hauch, Renaldo Baur Filho, Ritika Reddy, Rittika Adhikari, Rohan, Ron
Dagostino, Sagar Rao, Said Boudjelda, Sambhav Jain, Satish Duggana,
sciclon2, Shekhar Rajak, Sungyun Hur, Sushant Mahajan, Tanay
Karmarkar, tison, Tom Bentley, vamossagar12, Victoria Xia, Vincent
Jiang, vveicc, Walker Carlson, Yash Mayya, Yi-Sheng Lien, Ziming Deng,
蓝士钦

We welcome your help and feedback. For more information on how to
report problems, and to get involved, visit the project website at
https://kafka.apache.org/

Thank you!

Regards,
Satish Duggana

Re: [VOTE] 3.6.0 RC2

[Satish Duggana]

Hi,
Thank you all for participating in the 3.6.0 release candidate vote threads.

This vote passes with 11 +1 votes(4 binding) and no 0 or -1 votes.

+1 votes
PMC Members:
* Luke Chen
* Bill Bejeck
* Chris Egerton
* Justine Olshan

Community:
* Jakub Scholz
* Federico Valeri
* Divij Vaidya
* Kamal Chandraprakash
* Josep Prat
* Proven Provenzano
* Greg Harris

0 votes
* No votes

-1 votes
* No votes

I will continue with the release process and the release announcement
will follow in the next few days.

Thanks,
Satish.

On Tue, 3 Oct 2023 at 12:35, Satish Duggana  wrote:
>
> Hi,
> Thank you all for participating in the 3.6.0 release candidate vote threads.
>
> This vote passes with 11 +1 votes(4 binding) and no 0 or -1 votes.
>
> +1 votes
> PMC Members:
> * Luke Chen
> * Bill Bejeck
> * Chris Egerton
> * Justine Olshan
>
> Community:
> * Jakub Scholz
> * Federico Valeri
> * Divij Vaidya
> * Kamal Chandraprakash
> * Josep Prat
> * Proven Provenzano
> * Greg Harris
>
> 0 votes
> * No votes
>
> -1 votes
> * No votes
>
> I will continue with the release process and the release announcement
> will follow in the next few days.
>
> Thanks,
> Satish.
>
> On Tue, 3 Oct 2023 at 09:54, Justine Olshan
>  wrote:
> >
> > Thanks folks for following up. Given my previous testing and the results
> > you've provided, I'm +1 (binding)
> >
> > I will also follow up with the non-blocking metrics documentation.
> >
> > Thanks!
> > Justine
> >
> > On Tue, Oct 3, 2023 at 8:17 AM Chris Egerton 
> > wrote:
> >
> > > Hi Satish,
> > >
> > > Thanks for running this release!
> > >
> > > To verify, I:
> > > - Built from source using Java 11 with both:
> > > - - the 3.6.0-rc2 tag on GitHub
> > > - - the kafka-3.6.0-src.tgz artifact from
> > > https://home.apache.org/~satishd/kafka-3.6.0-rc2/
> > > - Checked signatures and checksums
> > > - Ran the quickstart using the kafka_2.13-3.6.0.tgz artifact from
> > > https://home.apache.org/~satishd/kafka-3.6.0-rc2/ with Java 11 and Scala
> > > 13
> > > in KRaft mode
> > > - Ran all unit tests
> > > - Ran all integration tests for Connect and MM2
> > > - Verified that the connect-test-plugins module is present in the staging
> > > Maven artifacts (https://issues.apache.org/jira/browse/KAFKA-15249)
> > >
> > > Everything looks good to me!
> > >
> > > +1 (binding)
> > >
> > > Cheers,
> > >
> > > Chris
> > >
> > > On Tue, Oct 3, 2023 at 6:43 AM Satish Duggana 
> > > wrote:
> > >
> > > > Thanks Luke for helping on running system tests on RCs and updating
> > > > the status on this email thread.
> > > >
> > > > ~Satish.
> > > >
> > > > On Tue, 3 Oct 2023 at 05:04, Luke Chen  wrote:
> > > > >
> > > > > Hi Justine and all,
> > > > >
> > > > > The system test result for 3.6.0 RC2 can be found below.
> > > > > In short, no failed tests. The flaky tests will pass in the 2nd run.
> > > > >
> > > >
> > > https://drive.google.com/drive/folders/1qwIKg-B4CBrswUeo5fBRv65KWpDsGUiS?usp=sharing
> > > > >
> > > > > Thank you.
> > > > > Luke
> > > > >
> > > > > On Tue, Oct 3, 2023 at 7:08 AM Justine Olshan
> > > > 
> > > > > wrote:
> > > > >
> > > > > > I realized Luke shared the results here for RC1
> > > > > >
> > > > https://drive.google.com/drive/folders/1S2XYd79f6_AeWj9f9qEkliRg7JtL04AC
> > > > > > Given we had some runs that looked reasonable, and we made a small
> > > > change,
> > > > > > I'm ok with this. But I wouldn't be upset if we had another set of
> > > > runs :)
> > > > > >
> > > > > > As for the validation:
> > > > > >
> > > > > >- I've compiled from source with java 17, 2.13, run the
> > > > transactional
> > > > > >produce bench
> > > > > >- Run unit tests
> > > > > >- Validated the checksums
> > > > > >- Downloaded and ran the 2.12 version of the release
> > > > > >- Briefly took a look at the documentation
> > > > > >- I was browsing through the site html files and I noticed the
> > > html
> > > > for
> &

Re: [VOTE] 3.6.0 RC2

[Satish Duggana]

Thanks Divij for the observations.

KAFKA-15483 is added to address the missing metrics related to KRaft
and ZK to KRaft migration. Please leave a comment at
https://github.com/apache/kafka-site/pull/548 in which related metrics
documentation is added.

https://issues.apache.org/jira/browse/KAFKA-15530 is raised to
followup on the missing documentation of metrics introduced in
KIP-890.

~Satish.

On Mon, 2 Oct 2023 at 05:37, Divij Vaidya  wrote:
>
> + 1 (non-binding)
>
> Verifications:
> 1. I ran a produce-consume workload with plaintext auth, JDK17, zstd
> compression using an open messaging benchmark and found 3.6 to be better
> than or equal to 3.5.1 across all dimensions. Notably, 3.6 had consistently
> 6-7% lower CPU utilization, lesser spikes on P99 produce latencies and
> overall lower P99.8 latencies.
>
> 2. I have verified that detached signature is correct using
> https://www.apache.org/info/verification.html and the release manager
> public keys are available at
> https://keys.openpgp.org/search?q=F65DC3423D4CD7B9
>
> 3. I have verified that all metrics emitted in 3.5.1 (with Zk) are also
> being emitted in 3.6.0 (with Zk).
>
> Problems (but not blockers):
> 1. Metrics added in
> https://github.com/apache/kafka/commit/2f71708955b293658cec3b27e9a5588d39c38d7e
> aren't available in the documentation (cc: Justine). I don't consider this
> as a release blocker but we should add it as a fast follow-up.
>
> 2. Metric added in
> https://github.com/apache/kafka/commit/a900794ace4dcf1f9dadee27fbd8b63979532a18
> isn't available in documentation (cc: David). I don't consider this as a
> release blocker but we should add it as a fast follow-up.
>
> --
> Divij Vaidya
>
>
>
> On Mon, Oct 2, 2023 at 9:50 AM Federico Valeri  wrote:
>
> > Hi Satish, I did the following to verify the release:
> >
> > - Built from source with Java 17 and Scala 2.13
> > - Ran all unit and integration tests
> > - Spot checked documentation
> > - Ran custom client applications using staging artifacts on a 3-nodes
> > cluster
> > - Tested tiered storage with one of the available RSM implementations
> >
> > +1 (non binding)
> >
> > Thanks
> > Fede
> >
> > On Mon, Oct 2, 2023 at 8:50 AM Luke Chen  wrote:
> > >
> > > Hi Satish,
> > >
> > > I verified with:
> > > 1. Ran quick start in KRaft for scala 2.12 artifact
> > > 2. Making sure the checksum are correct
> > > 3. Browsing release notes, documents, javadocs, protocols.
> > > 4. Verified the tiered storage feature works well.
> > >
> > > +1 (binding).
> > >
> > > Thanks.
> > > Luke
> > >
> > >
> > >
> > > On Mon, Oct 2, 2023 at 5:23 AM Jakub Scholz  wrote:
> > >
> > > > +1 (non-binding). I used the Scala 2.13 binaries and the staged Maven
> > > > artifacts and run my tests. Everything seems to work fine for me.
> > > >
> > > > Thanks
> > > > Jakub
> > > >
> > > > On Fri, Sep 29, 2023 at 8:17 PM Satish Duggana <
> > satish.dugg...@gmail.com>
> > > > wrote:
> > > >
> > > > > Hello Kafka users, developers and client-developers,
> > > > >
> > > > > This is the third candidate for the release of Apache Kafka 3.6.0.
> > > > > Some of the major features include:
> > > > >
> > > > > * KIP-405 : Kafka Tiered Storage
> > > > > * KIP-868 : KRaft Metadata Transactions
> > > > > * KIP-875: First-class offsets support in Kafka Connect
> > > > > * KIP-898: Modernize Connect plugin discovery
> > > > > * KIP-938: Add more metrics for measuring KRaft performance
> > > > > * KIP-902: Upgrade Zookeeper to 3.8.1
> > > > > * KIP-917: Additional custom metadata for remote log segment
> > > > >
> > > > > Release notes for the 3.6.0 release:
> > > > > https://home.apache.org/~satishd/kafka-3.6.0-rc2/RELEASE_NOTES.html
> > > > >
> > > > > *** Please download, test and vote by Tuesday, October 3, 12pm PT
> > > > >
> > > > > Kafka's KEYS file containing PGP keys we use to sign the release:
> > > > > https://kafka.apache.org/KEYS
> > > > >
> > > > > * Release artifacts to be voted upon (source and binary):
> > > > > https://home.apache.org/~satishd/kafka-3.6.0-rc2/
> > > > >
> > > > > * Maven artifacts to be voted upon:
> > > > >
> > https://reposit

[VOTE] 3.6.0 RC2

[Satish Duggana]

Hello Kafka users, developers and client-developers,

This is the third candidate for the release of Apache Kafka 3.6.0.
Some of the major features include:

* KIP-405 : Kafka Tiered Storage
* KIP-868 : KRaft Metadata Transactions
* KIP-875: First-class offsets support in Kafka Connect
* KIP-898: Modernize Connect plugin discovery
* KIP-938: Add more metrics for measuring KRaft performance
* KIP-902: Upgrade Zookeeper to 3.8.1
* KIP-917: Additional custom metadata for remote log segment

Release notes for the 3.6.0 release:
https://home.apache.org/~satishd/kafka-3.6.0-rc2/RELEASE_NOTES.html

*** Please download, test and vote by Tuesday, October 3, 12pm PT

Kafka's KEYS file containing PGP keys we use to sign the release:
https://kafka.apache.org/KEYS

* Release artifacts to be voted upon (source and binary):
https://home.apache.org/~satishd/kafka-3.6.0-rc2/

* Maven artifacts to be voted upon:
https://repository.apache.org/content/groups/staging/org/apache/kafka/

* Javadoc:
https://home.apache.org/~satishd/kafka-3.6.0-rc2/javadoc/

* Tag to be voted upon (off 3.6 branch) is the 3.6.0-rc2 tag:
https://github.com/apache/kafka/releases/tag/3.6.0-rc2

* Documentation:
https://kafka.apache.org/36/documentation.html

* Protocol:
https://kafka.apache.org/36/protocol.html

* Successful Jenkins builds for the 3.6 branch:
There are a few runs of unit/integration tests. You can see the latest
at https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.6/. We will
continue running a few more iterations.
System tests:
We will send an update once we have the results.

Thanks,
Satish.

Re: [kafka-clients] [VOTE] 3.6.0 RC1

[Satish Duggana]

Thanks Luke for reporting KRaft issue[1].

I am not sure whether it is a release blocker for 3.6.0. Need input
from other KRaft experts also to finalize the decision. Even if we
adopt a fix, do not we need to bake it for some time before it is
pushed to production to avoid any regressions as this change is in the
critical paths?

1. https://issues.apache.org/jira/browse/KAFKA-15489

Thanks,
Satish.

On Sat, 23 Sept 2023 at 03:08, Luke Chen  wrote:
>
> Hi Satish,
>
> I found the current KRaft implementation will have "split brain" issue when
> network partition happens, which will cause inconsistent metadata returned
> from the controller.
> Filed KAFKA-15489 <https://issues.apache.org/jira/browse/KAFKA-15489> for
> this issue, and PR <https://github.com/apache/kafka/pull/14428> is ready
> for review.
>
> Even though this is not a regression issue (this has already existed since
> the 1st release of KRaft feature), I think this is an important issue since
> KRaft is announced production ready.
> Not sure what other people's thoughts are.
>
> Thank you.
> Luke
>
> On Thu, Sep 21, 2023 at 6:33 PM Josep Prat 
> wrote:
>
> > Hi Satish,
> >
> > I ran the following validation steps:
> > - Built from source with Java 11 and Scala 2.13
> > - Verified Signatures and hashes of the artifacts generated
> > - Navigated through Javadoc including links to JDK classes
> > - Run the unit tests
> > - Run integration tests
> > - Run the quickstart in KRaft and Zookeeper mode
> >
> >
> > I +1 this release (non-binding)
> >
> > Thanks for your efforts!
> >
> > On Thu, Sep 21, 2023 at 2:59 AM Satish Duggana 
> > wrote:
> >
> > > Thanks Greg for verifying the release including the earlier
> > > blocker(KAFKA-15473) verification.
> > >
> > > ~Satish.
> > >
> > > On Wed, 20 Sept 2023 at 22:30, Greg Harris  > >
> > > wrote:
> > >
> > > > Hi all,
> > > >
> > > > I verified the functionality of KIP-898 and the recent fix for
> > > > KAFKA-15473 with the following steps:
> > > >
> > > > 1. I started a 3.5.1 broker, and a 3.5.1 worker with most (>400)
> > > > publicly available plugins installed
> > > > 2. I captured the output of /connector-plugins
> > > > 3. I upgraded the worker to 3.6.0-rc1
> > > > 4. I captured the output of /connector-plugins with various settings
> > > > of plugin.discovery
> > > > 5. I ran the migration script to add manifests to my plugins
> > > > 6. I captured the output of /connector-plugins with various settings
> > > > of plugin.discovery
> > > > 7. I downgraded the worker to 3.5.1
> > > > 8. I diffed the output of /connector-plugins across the different
> > > > cases and observed the expected changes.
> > > > a. When plugins are migrated for 3.6.0, all modes produce identical
> > > > results.
> > > > b. When plugins are not migrated for 3.6.0, only_scan and
> > > > hybrid_warn produce identical results, hybrid_fail crashes, and
> > > > service_load is missing plugins
> > > > c. When upgrading from 3.5.1 I see that plugins with invalid
> > > > constructors are hidden, AK plugins now have versions, multi-interface
> > > > plugins now show each interface type, and plugins using AppInfoParser
> > > > change versions.
> > > > d. The startup logs now include descriptive errors for invalid
> > > > plugins that otherwise would have been thrown at runtime
> > > > d. The fix for KAFKA-15473 prevents duplicates
> > > > e. The output for 3.5.1 after downgrading is identical to before.
> > > >
> > > > +1 (non-binding)
> > > >
> > > > Thanks Satish for running the release!
> > > >
> > > > On Wed, Sep 20, 2023 at 8:36 AM Divij Vaidya 
> > wrote:
> > > > >
> > > > > Hey Satish
> > > > >
> > > > > My comments about documentation misses from RC0 vote thread [1] are
> > > > > still not addressed (such as missing metric documentation, formatting
> > > > > problems etc). Could you please mention why we shouldn't consider
> > them
> > > > > as blockers to make RC1 as the final release?
> > > > >
> > > > > [1] https://lists.apache.org/thread/cokoxzd0jtgjtrlxoq7kkzmvpm75381t
> > > > >
> > > > > On Wed, Sep 20, 2023 at 4:53 PM Satish Dugga

Re: [kafka-clients] [VOTE] 3.6.0 RC1

[Satish Duggana]

Thanks Greg for verifying the release including the earlier
blocker(KAFKA-15473) verification.

~Satish.

On Wed, 20 Sept 2023 at 22:30, Greg Harris 
wrote:

> Hi all,
>
> I verified the functionality of KIP-898 and the recent fix for
> KAFKA-15473 with the following steps:
>
> 1. I started a 3.5.1 broker, and a 3.5.1 worker with most (>400)
> publicly available plugins installed
> 2. I captured the output of /connector-plugins
> 3. I upgraded the worker to 3.6.0-rc1
> 4. I captured the output of /connector-plugins with various settings
> of plugin.discovery
> 5. I ran the migration script to add manifests to my plugins
> 6. I captured the output of /connector-plugins with various settings
> of plugin.discovery
> 7. I downgraded the worker to 3.5.1
> 8. I diffed the output of /connector-plugins across the different
> cases and observed the expected changes.
> a. When plugins are migrated for 3.6.0, all modes produce identical
> results.
> b. When plugins are not migrated for 3.6.0, only_scan and
> hybrid_warn produce identical results, hybrid_fail crashes, and
> service_load is missing plugins
> c. When upgrading from 3.5.1 I see that plugins with invalid
> constructors are hidden, AK plugins now have versions, multi-interface
> plugins now show each interface type, and plugins using AppInfoParser
> change versions.
> d. The startup logs now include descriptive errors for invalid
> plugins that otherwise would have been thrown at runtime
> d. The fix for KAFKA-15473 prevents duplicates
> e. The output for 3.5.1 after downgrading is identical to before.
>
> +1 (non-binding)
>
> Thanks Satish for running the release!
>
> On Wed, Sep 20, 2023 at 8:36 AM Divij Vaidya  wrote:
> >
> > Hey Satish
> >
> > My comments about documentation misses from RC0 vote thread [1] are
> > still not addressed (such as missing metric documentation, formatting
> > problems etc). Could you please mention why we shouldn't consider them
> > as blockers to make RC1 as the final release?
> >
> > [1] https://lists.apache.org/thread/cokoxzd0jtgjtrlxoq7kkzmvpm75381t
> >
> > On Wed, Sep 20, 2023 at 4:53 PM Satish Duggana 
> wrote:
> > >
> > > Hello Kafka users, developers and client-developers,
> > >
> > > This is the second candidate for the release of Apache Kafka 3.6.0.
> Some of the major features include:
> > >
> > > * KIP-405 : Kafka Tiered Storage
> > > * KIP-868 : KRaft Metadata Transactions
> > > * KIP-875: First-class offsets support in Kafka Connect
> > > * KIP-898: Modernize Connect plugin discovery
> > > * KIP-938: Add more metrics for measuring KRaft performance
> > > * KIP-902: Upgrade Zookeeper to 3.8.1
> > > * KIP-917: Additional custom metadata for remote log segment
> > >
> > > Release notes for the 3.6.0 release:
> > > https://home.apache.org/~satishd/kafka-3.6.0-rc1/RELEASE_NOTES.html
> > >
> > > *** Please download, test and vote by Saturday, September 23, 8am PT
> > >
> > > Kafka's KEYS file containing PGP keys we use to sign the release:
> > > https://kafka.apache.org/KEYS
> > >
> > > * Release artifacts to be voted upon (source and binary):
> > > https://home.apache.org/~satishd/kafka-3.6.0-rc1/
> > >
> > > * Maven artifacts to be voted upon:
> > > https://repository.apache.org/content/groups/staging/org/apache/kafka/
> > >
> > > * Javadoc:
> > > https://home.apache.org/~satishd/kafka-3.6.0-rc1/javadoc/
> > >
> > > * Tag to be voted upon (off 3.6 branch) is the 3.6.0 tag:
> > > https://github.com/apache/kafka/releases/tag/3.6.0-rc1
> > >
> > > * Documentation:
> > > https://kafka.apache.org/36/documentation.html
> > >
> > > * Protocol:
> > > https://kafka.apache.org/36/protocol.html
> > >
> > > * Successful Jenkins builds for the 3.6 branch:
> > > There are a few runs of unit/integration tests. You can see the latest
> at https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.6/. We will
> continue running a few more iterations.
> > > System tests:
> > > We will send an update once we have the results.
> > >
> > > Thanks,
> > > Satish.
> > >
> > > --
> > > You received this message because you are subscribed to the Google
> Groups "kafka-clients" group.
> > > To unsubscribe from this group and stop receiving emails from it, send
> an email to kafka-clients+unsubscr...@googlegroups.com.
> > > To view this discussion on the web visit
> https://groups.google.com/d/msgid/kafka-clients/CAM-aUZ%3DuJ-SKeVFtBZwBjhLHKw4CbxF_ws%2BvQqaymGHFsC%2Bmdg%40mail.gmail.com
> .
>

Re: [kafka-clients] [VOTE] 3.6.0 RC1

[Satish Duggana]

Hi Divij,
Thanks for calling out documentation issues. Some of them are being
addressed in the existing documentation PRs. We will keep addressing the
remaining documentation issues by making respective changes in kafka-site
repo for 3.6.0 release.

I think documentation is important but I do not consider documentation
issues as release blockers. I want others to review and test the release
candidate sooner to finalize the RC for 3.6.0 release.

~Satish.

On Wed, 20 Sept 2023 at 21:04, Divij Vaidya  wrote:

> Hey Satish
>
> My comments about documentation misses from RC0 vote thread [1] are
> still not addressed (such as missing metric documentation, formatting
> problems etc). Could you please mention why we shouldn't consider them
> as blockers to make RC1 as the final release?
>
> [1] https://lists.apache.org/thread/cokoxzd0jtgjtrlxoq7kkzmvpm75381t
>
> On Wed, Sep 20, 2023 at 4:53 PM Satish Duggana 
> wrote:
> >
> > Hello Kafka users, developers and client-developers,
> >
> > This is the second candidate for the release of Apache Kafka 3.6.0. Some
> of the major features include:
> >
> > * KIP-405 : Kafka Tiered Storage
> > * KIP-868 : KRaft Metadata Transactions
> > * KIP-875: First-class offsets support in Kafka Connect
> > * KIP-898: Modernize Connect plugin discovery
> > * KIP-938: Add more metrics for measuring KRaft performance
> > * KIP-902: Upgrade Zookeeper to 3.8.1
> > * KIP-917: Additional custom metadata for remote log segment
> >
> > Release notes for the 3.6.0 release:
> > https://home.apache.org/~satishd/kafka-3.6.0-rc1/RELEASE_NOTES.html
> >
> > *** Please download, test and vote by Saturday, September 23, 8am PT
> >
> > Kafka's KEYS file containing PGP keys we use to sign the release:
> > https://kafka.apache.org/KEYS
> >
> > * Release artifacts to be voted upon (source and binary):
> > https://home.apache.org/~satishd/kafka-3.6.0-rc1/
> >
> > * Maven artifacts to be voted upon:
> > https://repository.apache.org/content/groups/staging/org/apache/kafka/
> >
> > * Javadoc:
> > https://home.apache.org/~satishd/kafka-3.6.0-rc1/javadoc/
> >
> > * Tag to be voted upon (off 3.6 branch) is the 3.6.0 tag:
> > https://github.com/apache/kafka/releases/tag/3.6.0-rc1
> >
> > * Documentation:
> > https://kafka.apache.org/36/documentation.html
> >
> > * Protocol:
> > https://kafka.apache.org/36/protocol.html
> >
> > * Successful Jenkins builds for the 3.6 branch:
> > There are a few runs of unit/integration tests. You can see the latest
> at https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.6/. We will
> continue running a few more iterations.
> > System tests:
> > We will send an update once we have the results.
> >
> > Thanks,
> > Satish.
> >
> > --
> > You received this message because you are subscribed to the Google
> Groups "kafka-clients" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> an email to kafka-clients+unsubscr...@googlegroups.com.
> > To view this discussion on the web visit
> https://groups.google.com/d/msgid/kafka-clients/CAM-aUZ%3DuJ-SKeVFtBZwBjhLHKw4CbxF_ws%2BvQqaymGHFsC%2Bmdg%40mail.gmail.com
> .
>

[VOTE] 3.6.0 RC1

[Satish Duggana]

Hello Kafka users, developers and client-developers,

This is the second candidate for the release of Apache Kafka 3.6.0. Some of
the major features include:

* KIP-405 : Kafka Tiered Storage
* KIP-868 : KRaft Metadata Transactions
* KIP-875: First-class offsets support in Kafka Connect
* KIP-898: Modernize Connect plugin discovery
* KIP-938: Add more metrics for measuring KRaft performance
* KIP-902: Upgrade Zookeeper to 3.8.1
* KIP-917: Additional custom metadata for remote log segment

Release notes for the 3.6.0 release:
https://home.apache.org/~satishd/kafka-3.6.0-rc1/RELEASE_NOTES.html

*** Please download, test and vote by Saturday, September 23, 8am PT

Kafka's KEYS file containing PGP keys we use to sign the release:
https://kafka.apache.org/KEYS

* Release artifacts to be voted upon (source and binary):
https://home.apache.org/~satishd/kafka-3.6.0-rc1/

* Maven artifacts to be voted upon:
https://repository.apache.org/content/groups/staging/org/apache/kafka/

* Javadoc:
https://home.apache.org/~satishd/kafka-3.6.0-rc1/javadoc/

* Tag to be voted upon (off 3.6 branch) is the 3.6.0 tag:
https://github.com/apache/kafka/releases/tag/3.6.0-rc1

* Documentation:
https://kafka.apache.org/36/documentation.html

* Protocol:
https://kafka.apache.org/36/protocol.html

* Successful Jenkins builds for the 3.6 branch:
There are a few runs of unit/integration tests. You can see the latest at
https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.6/. We will continue
running a few more iterations.
System tests:
We will send an update once we have the results.

Thanks,
Satish.

Re: [VOTE] 3.6.0 RC0

[Satish Duggana]

On Sun, 17 Sept 2023 at 23:00, Satish Duggana 
wrote:

> Hello Kafka users, developers and client-developers,
>
> This is the first candidate for the release of Apache Kafka 3.6.0. Some of
> the major features include:
>
> * KIP-405 : Kafka Tiered Storage
> * KIP-868 : KRaft Metadata Transactions
> * KIP-875: First-class offsets support in Kafka Connect
> * KIP-898: Modernize Connect plugin discovery
> * KIP-938: Add more metrics for measuring KRaft performance
> * KIP-902: Upgrade Zookeeper to 3.8.1
> * KIP-917: Additional custom metadata for remote log segment
>
> Release notes for the 3.6.0 release:
> https://home.apache.org/~satishd/kafka-3.6.0-rc0/RELEASE_NOTES.html
>
> *** Please download, test and vote by Wednesday, September 21, 12pm PT
>
> Kafka's KEYS file containing PGP keys we use to sign the release:
> https://kafka.apache.org/KEYS
>
> * Release artifacts to be voted upon (source and binary):
> https://home.apache.org/~satishd/kafka-3.6.0-rc0/
>
> * Maven artifacts to be voted upon:
> https://repository.apache.org/content/groups/staging/org/apache/kafka/
>
> * Javadoc:
> https://home.apache.org/~satishd/kafka-3.6.0-rc0/javadoc/
>
> * Tag to be voted upon (off 3.6 branch) is the 3.6.0 tag:
> https://github.com/apache/kafka/releases/tag/3.6.0-rc0
>
> * Documentation:
> https://kafka.apache.org/36/documentation.html
>
> * Protocol:
> https://kafka.apache.org/36/protocol.html
>
> * Successful Jenkins builds for the 3.6 branch:
> There are a few runs of unit/integration tests. You can see the latest at
> https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.6/. We will
> continue running a few more iterations.
> System tests:
> We will send an update once we have the results.
>
> Thanks,
> Satish.
>

[VOTE] 3.6.0 RC0

[Satish Duggana]

Hello Kafka users, developers and client-developers,

This is the first candidate for the release of Apache Kafka 3.6.0. Some of
the major features include:

* KIP-405 : Kafka Tiered Storage
* KIP-868 : KRaft Metadata Transactions
* KIP-875: First-class offsets support in Kafka Connect
* KIP-898: Modernize Connect plugin discovery
* KIP-938: Add more metrics for measuring KRaft performance
* KIP-902: Upgrade Zookeeper to 3.8.1
* KIP-917: Additional custom metadata for remote log segment

Release notes for the 3.6.0 release:
https://home.apache.org/~satishd/kafka-3.6.0-rc0/RELEASE_NOTES.html

*** Please download, test and vote by Wednesday, September 21, 12pm PT

Kafka's KEYS file containing PGP keys we use to sign the release:
https://kafka.apache.org/KEYS

* Release artifacts to be voted upon (source and binary):
https://home.apache.org/~satishd/kafka-3.6.0-rc0/

* Maven artifacts to be voted upon:
https://repository.apache.org/content/groups/staging/org/apache/kafka/

* Javadoc:
https://home.apache.org/~satishd/kafka-3.6.0-rc0/javadoc/

* Tag to be voted upon (off 3.6 branch) is the 3.6.0 tag:
https://github.com/apache/kafka/releases/tag/3.6.0-rc0

* Documentation:
https://kafka.apache.org/36/documentation.html

* Protocol:
https://kafka.apache.org/36/protocol.html

* Successful Jenkins builds for the 3.6 branch:
There are a few runs of unit/integration tests. You can see the latest at
https://ci-builds.apache.org/job/Kafka/job/kafka/job/3.6/. We will continue
running a few more iterations.
System tests:
We will send an update once we have the results.

Thanks,
Satish.

Re: [ANNOUNCE] New Kafka PMC Member: David Arthur

[Satish Duggana]

Congratulations David!

On Mon, 13 Mar 2023 at 16:08, Rajini Sivaram 
wrote:

> Congratulations, David!
>
> Regards,
>
> Rajini
>
> On Mon, Mar 13, 2023 at 9:06 AM Bruno Cadonna  wrote:
>
> > Congrats, David!
> >
> > Bruno
> >
> > On 10.03.23 01:36, Matthias J. Sax wrote:
> > > Congrats!
> > >
> > > On 3/9/23 2:59 PM, José Armando García Sancio wrote:
> > >> Congrats David!
> > >>
> > >> On Thu, Mar 9, 2023 at 2:00 PM Kowshik Prakasam 
> > >> wrote:
> > >>>
> > >>> Congrats David!
> > >>>
> > >>> On Thu, Mar 9, 2023 at 12:09 PM Lucas Brutschy
> > >>>  wrote:
> > >>>
> >  Congratulations!
> > 
> >  On Thu, Mar 9, 2023 at 8:37 PM Manikumar  >
> >  wrote:
> > >
> > > Congrats David!
> > >
> > >
> > > On Fri, Mar 10, 2023 at 12:24 AM Josep Prat
> > >  > >
> > > wrote:
> > >>
> > >> Congrats David!
> > >>
> > >> ———
> > >> Josep Prat
> > >>
> > >> Aiven Deutschland GmbH
> > >>
> > >> Alexanderufer 3-7, 10117 Berlin
> > >>
> > >> Amtsgericht Charlottenburg, HRB 209739 B
> > >>
> > >> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > >>
> > >> m: +491715557497
> > >>
> > >> w: aiven.io
> > >>
> > >> e: josep.p...@aiven.io
> > >>
> > >> On Thu, Mar 9, 2023, 19:22 Mickael Maison <
> mickael.mai...@gmail.com
> > >
> > > wrote:
> > >>
> > >>> Congratulations David!
> > >>>
> > >>> On Thu, Mar 9, 2023 at 7:20 PM Chris Egerton
> > >>>  > >
> > >>> wrote:
> > 
> >  Congrats David!
> > 
> >  On Thu, Mar 9, 2023 at 1:17 PM Bill Bejeck 
> >  wrote:
> > 
> > > Congratulations David!
> > >
> > > On Thu, Mar 9, 2023 at 1:12 PM Jun Rao
>  > >
> > >>> wrote:
> > >
> > >> Hi, Everyone,
> > >>
> > >> David Arthur has been a Kafka committer since 2013. He has
> been
> > > very
> > >> instrumental to the community since becoming a committer. It's
> >  my
> > > pleasure
> > >> to announce that David is now a member of Kafka PMC.
> > >>
> > >> Congratulations David!
> > >>
> > >> Jun
> > >> on behalf of Apache Kafka PMC
> > >>
> > >
> > >>>
> > 
> > >>
> > >>
> > >>
> >
>

Re: [ANNOUNCE] New committer: Satish Duggana

[Satish Duggana]

Thanks again, everyone!

On Sat, 31 Dec 2022 at 01:27, Dhiraj Dwarapudi
 wrote:
>
> Congratulations Sato :-)
>
> Regards,
> Dhiraj
>
> On Thu, Dec 29, 2022 at 10:17 AM Bill Bejeck  wrote:
>
> > Congratulations Satish!
> >
> > -Bill
> >
> > On Thu, Dec 29, 2022 at 11:05 AM Satish Duggana 
> > wrote:
> >
> > > Thank you, everyone!
> > >
> > > On Wed, 28 Dec 2022 at 00:02, Matthias J. Sax  wrote:
> > > >
> > > > Congrats!
> > > >
> > > > On 12/27/22 10:20 AM, Kirk True wrote:
> > > > > Congrats, Satish!
> > > > >
> > > > > On Fri, Dec 23, 2022, at 10:07 AM, Jun Rao wrote:
> > > > >> Hi, Everyone,
> > > > >>
> > > > >> The PMC of Apache Kafka is pleased to announce a new Kafka committer
> > > Satish
> > > > >> Duggana.
> > > > >>
> > > > >> Satish has been a long time Kafka contributor since 2017. He is the
> > > main
> > > > >> driver behind KIP-405 that integrates Kafka with remote storage, a
> > > > >> significant and much anticipated feature in Kafka.
> > > > >>
> > > > >> Congratulations, Satish!
> > > > >>
> > > > >> Thanks,
> > > > >>
> > > > >> Jun (on behalf of the Apache Kafka PMC)
> > > > >>
> > > > >
> > >
> >

Re: [ANNOUNCE] New committer: Satish Duggana

[Satish Duggana]

Thank you, everyone!

On Wed, 28 Dec 2022 at 00:02, Matthias J. Sax  wrote:
>
> Congrats!
>
> On 12/27/22 10:20 AM, Kirk True wrote:
> > Congrats, Satish!
> >
> > On Fri, Dec 23, 2022, at 10:07 AM, Jun Rao wrote:
> >> Hi, Everyone,
> >>
> >> The PMC of Apache Kafka is pleased to announce a new Kafka committer Satish
> >> Duggana.
> >>
> >> Satish has been a long time Kafka contributor since 2017. He is the main
> >> driver behind KIP-405 that integrates Kafka with remote storage, a
> >> significant and much anticipated feature in Kafka.
> >>
> >> Congratulations, Satish!
> >>
> >> Thanks,
> >>
> >> Jun (on behalf of the Apache Kafka PMC)
> >>
> >

Re: [ANNOUNCE] New Kafka PMC Member: Konstantine Karantasis

[Satish Duggana]

Congratulations Konstantine!!

On Tue, 22 Jun 2021 at 00:34, Bill Bejeck  wrote:

> Congrats Konstantine!
>
> -Bill
>
> On Mon, Jun 21, 2021 at 12:44 PM Tom Bentley  wrote:
>
> > Congratulations Konstantine!
> >
> > On Mon, Jun 21, 2021 at 5:33 PM David Jacot  >
> > wrote:
> >
> > > Congrats, Konstantine. Well deserved!
> > >
> > > Best,
> > > David
> > >
> > > On Mon, Jun 21, 2021 at 6:14 PM Ramesh Krishnan <
> ramesh.154...@gmail.com
> > >
> > > wrote:
> > >
> > > > Congrats Konstantine
> > > >
> > > > On Mon, 21 Jun 2021 at 8:58 PM, Mickael Maison 
> > > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > It's my pleasure to announce that Konstantine Karantasis is now a
> > > > > member of the Kafka PMC.
> > > > >
> > > > > Konstantine has been a Kafka committer since Feb 2020. He has
> > remained
> > > > > active in the community since becoming a committer.
> > > > >
> > > > > Congratulations Konstantine!
> > > > >
> > > > > Mickael, on behalf of the Apache Kafka PMC
> > > > >
> > > >
> > >
> >
>

Re: [ANNOUNCE] New Kafka PMC Member: Randall Hauch

[Satish Duggana]

Congratulations Randall!!

On Sat, 17 Apr, 2021, 5:29 AM Raymond Ng,  wrote:

> Congrats Randall!
>
> On Fri, Apr 16, 2021 at 4:45 PM Matthias J. Sax  wrote:
>
> > Hi,
> >
> > It's my pleasure to announce that Randall Hauch in now a member of the
> > Kafka PMC.
> >
> > Randall has been a Kafka committer since Feb 2019. He has remained
> > active in the community since becoming a committer.
> >
> >
> >
> > Congratulations Randall!
> >
> >  -Matthias, on behalf of Apache Kafka PMC
> >
>

Re: [ANNOUNCE] New Kafka PMC Member: Bill Bejeck

[Satish Duggana]

Congratulations Bill!!

On Thu, 8 Apr 2021 at 13:24, Tom Bentley  wrote:

> Congratulations Bill!
>
> On Thu, Apr 8, 2021 at 2:36 AM Luke Chen  wrote:
>
> > Congratulations Bill!
> >
> > Luke
> >
> > On Thu, Apr 8, 2021 at 9:17 AM Matthias J. Sax  wrote:
> >
> > > Hi,
> > >
> > > It's my pleasure to announce that Bill Bejeck in now a member of the
> > > Kafka PMC.
> > >
> > > Bill has been a Kafka committer since Feb 2019. He has remained
> > > active in the community since becoming a committer.
> > >
> > >
> > >
> > > Congratulations Bill!
> > >
> > >  -Matthias, on behalf of Apache Kafka PMC
> > >
> >
>

Re: Subject: [VOTE] 2.2.2 RC2

[Satish Duggana]

+1 (non-binding)

- Ran testAll/releaseTarGzAll successfully with no failures.
- Ran through quickstart of core/streams on builds generated from 2.3.0-rc2
tag.
- Ran a few internal apps with ~2500 topic-partitions on 5 a node cluster.

Thanks,
Satish.

On Sat, Nov 9, 2019 at 4:41 AM Bill Bejeck  wrote:

>- verified signatures
>- built it from source
>- ran unit tests
>
> +1 (non-binding)
>
> Thanks for running the release, Randall!
>
> -Bill
>
> On Fri, Nov 8, 2019 at 4:40 PM David Arthur  wrote:
>
> > * Glanced through docs, release notes
> > * Downloaded RC2 binaries, verified signatures
> > * Ran through quickstart
> >
> > +1 binding
> >
> > Thanks for managing this release, Randall!
> >
> > -David
> >
> > On Wed, Nov 6, 2019 at 7:39 PM Eric Lalonde  wrote:
> >
> > > Hello,
> > >
> > > In an effort to assist in the verification of release candidates, I
> have
> > > authored the following quick-and-dirty utility to help people verify
> > > release candidate artifacts:
> > > https://github.com/elalonde/kafka/blob/master/bin/verify-kafka-rc.sh <
> > > https://github.com/elalonde/kafka/blob/master/bin/verify-kafka-rc.sh>
> .
> > I
> > > have executed this script for 2.2.2 rc2 and everything looks good:
> > > - all checksums verify
> > > - all executed gradle commands succeed
> > > - all unit and integration tests pass.
> > >
> > > Hope this helps in the release of 2.2.2.
> > >
> > > - Eric
> > >
> > > > On Nov 5, 2019, at 7:55 AM, Randall Hauch  wrote:
> > > >
> > > > Thanks, Mickael!
> > > >
> > > > Anyone else get a chance to validate the 2.2.2 RC2 build? It'd be
> great
> > > to
> > > > get this out the door.
> > > >
> > > > Randall
> > > >
> > > > On Tue, Nov 5, 2019 at 6:34 AM Mickael Maison <
> > mickael.mai...@gmail.com>
> > > > wrote:
> > > >
> > > >> +1 (non binding)
> > > >> I verified signatures, built it from source, ran unit tests and
> > > quickstart
> > > >>
> > > >>
> > > >>
> > > >> On Fri, Oct 25, 2019 at 3:10 PM Randall Hauch 
> > wrote:
> > > >>>
> > > >>> Hello all, we identified around three dozen bug fixes, including an
> > > >> update
> > > >>> of a third party dependency, and wanted to release a patch release
> > for
> > > >> the
> > > >>> Apache Kafka 2.2.0 release.
> > > >>>
> > > >>> This is the *second* candidate for release of Apache Kafka 2.2.2.
> > (RC1
> > > >> did
> > > >>> not include a fix for
> > https://issues.apache.org/jira/browse/KAFKA-9053
> > > ,
> > > >> but
> > > >>> the fix appeared before RC1 was announced so it was easier to just
> > > create
> > > >>> RC2.)
> > > >>>
> > > >>> Check out the release notes for a complete list of the changes in
> > this
> > > >>> release candidate:
> > > >>> https://home.apache.org/~rhauch/kafka-2.2.2-rc2/RELEASE_NOTES.html
> > > >>>
> > > >>> *** Please download, test and vote by Wednesday, October 30, 9am
> PT>
> > > >>>
> > > >>> Kafka's KEYS file containing PGP keys we use to sign the release:
> > > >>> https://kafka.apache.org/KEYS
> > > >>>
> > > >>> * Release artifacts to be voted upon (source and binary):
> > > >>> https://home.apache.org/~rhauch/kafka-2.2.2-rc2/
> > > >>>
> > > >>> * Maven artifacts to be voted upon:
> > > >>>
> > https://repository.apache.org/content/groups/staging/org/apache/kafka/
> > > >>>
> > > >>> * Javadoc:
> > > >>> https://home.apache.org/~rhauch/kafka-2.2.2-rc2/javadoc/
> > > >>>
> > > >>> * Tag to be voted upon (off 2.2 branch) is the 2.2.2 tag:
> > > >>> https://github.com/apache/kafka/releases/tag/2.2.2-rc2
> > > >>>
> > > >>> * Documentation:
> > > >>> https://kafka.apache.org/22/documentation.html
> > > >>>
> > > >>> * Protocol:
> > > >>> https://kafka.apache.org/22/protocol.html
> > > >>>
> > > >>> * Successful Jenkins builds for the 2.2 branch:
> > > >>> Unit/integration tests:
> > > https://builds.apache.org/job/kafka-2.2-jdk8/1/
> > > >>> System tests:
> > > >>> https://jenkins.confluent.io/job/system-test-kafka/job/2.2/216/
> > > >>>
> > > >>> /**
> > > >>>
> > > >>> Thanks,
> > > >>>
> > > >>> Randall Hauch
> > > >>
> > >
> > >
> >
> > --
> > David Arthur
> >
>

Re: [ANNOUNCE] New committer: Mickael Maison

[Satish Duggana]

Congratulations Mickael!!

On Fri, Nov 8, 2019 at 2:50 PM Rajini Sivaram  wrote:
>
> Congratulations, Mickael, well deserved!!
>
> Regards,
>
> Rajini
>
> On Fri, Nov 8, 2019 at 9:08 AM David Jacot  wrote:
>
> > Congrats Mickeal, well deserved!
> >
> > On Fri, Nov 8, 2019 at 8:56 AM Tom Bentley  wrote:
> >
> > > Congratulations Mickael!
> > >
> > > On Fri, Nov 8, 2019 at 6:41 AM Vahid Hashemian <
> > vahid.hashem...@gmail.com>
> > > wrote:
> > >
> > > > Congrats Mickael,
> > > >
> > > > Well deserved!
> > > >
> > > > --Vahid
> > > >
> > > > On Thu, Nov 7, 2019 at 9:10 PM Maulin Vasavada <
> > > maulin.vasav...@gmail.com>
> > > > wrote:
> > > >
> > > > > Congratulations Mickael!
> > > > >
> > > > > On Thu, Nov 7, 2019 at 8:27 PM Manikumar 
> > > > > wrote:
> > > > >
> > > > > > Congrats Mickeal!
> > > > > >
> > > > > > On Fri, Nov 8, 2019 at 9:05 AM Dong Lin 
> > wrote:
> > > > > >
> > > > > > > Congratulations Mickael!
> > > > > > >
> > > > > > > On Thu, Nov 7, 2019 at 1:38 PM Jun Rao  wrote:
> > > > > > >
> > > > > > > > Hi, Everyone,
> > > > > > > >
> > > > > > > > The PMC of Apache Kafka is pleased to announce a new Kafka
> > > > committer
> > > > > > > > Mickael
> > > > > > > > Maison.
> > > > > > > >
> > > > > > > > Mickael has been contributing to Kafka since 2016. He proposed
> > > and
> > > > > > > > implemented multiple KIPs. He has also been propomating Kafka
> > > > through
> > > > > > > blogs
> > > > > > > > and public talks.
> > > > > > > >
> > > > > > > > Congratulations, Mickael!
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > >
> > > > > > > > Jun (on behalf of the Apache Kafka PMC)
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > >
> > > > Thanks!
> > > > --Vahid
> > > >
> > >
> >

Re: [ANNOUNCE] New Kafka PMC member: Sriharsh Chintalapan

[Satish Duggana]

Congrats Harsha!

On Fri, Apr 19, 2019 at 2:58 PM Mickael Maison 
wrote:

> Congratulations Harsha!
>
>
> On Fri, Apr 19, 2019 at 5:49 AM Manikumar 
> wrote:
> >
> > Congrats Harsha!.
> >
> > On Fri, Apr 19, 2019 at 7:43 AM Dong Lin  wrote:
> >
> > > Congratulations Sriharsh!
> > >
> > > On Thu, Apr 18, 2019 at 11:46 AM Jun Rao  wrote:
> > >
> > > > Hi, Everyone,
> > > >
> > > > Sriharsh Chintalapan has been active in the Kafka community since he
> > > became
> > > > a Kafka committer in 2015. I am glad to announce that Harsh is now a
> > > member
> > > > of Kafka PMC.
> > > >
> > > > Congratulations, Harsh!
> > > >
> > > > Jun
> > > >
> > >
>

Re: [VOTE] 2.2.0 RC2

[Satish Duggana]

+1 (non-binding)

- Ran testAll/releaseTarGzAll successfully with no failures.
- Ran through quickstart of core/streams on builds generated from 2.2.0-rc2
tag
- Ran few internal apps targeting to topics on 3 node cluster.

Thanks for running the release Matthias!

On Wed, Mar 20, 2019 at 12:43 AM Manikumar 
wrote:

> +1 (non-binding)
>
> - Verified the artifacts, build from src, ran tests
> - Verified the quickstart, ran producer/consumer performance tests.
>
> Thanks for running release!.
>
> Thanks,
> Manikumar
>
> On Wed, Mar 20, 2019 at 12:19 AM David Arthur 
> wrote:
>
> > +1
> >
> > Validated signatures, and ran through quick-start.
> >
> > Thanks!
> >
> > On Mon, Mar 18, 2019 at 4:00 AM Jakub Scholz  wrote:
> >
> > > +1 (non-binding). I used the staged binaries and run some of my tests
> > > against them. All seems to look good to me.
> > >
> > > On Sat, Mar 9, 2019 at 11:56 PM Matthias J. Sax  >
> > > wrote:
> > >
> > > > Hello Kafka users, developers and client-developers,
> > > >
> > > > This is the third candidate for release of Apache Kafka 2.2.0.
> > > >
> > > >  - Added SSL support for custom principal name
> > > >  - Allow SASL connections to periodically re-authenticate
> > > >  - Command line tool bin/kafka-topics.sh adds AdminClient support
> > > >  - Improved consumer group management
> > > >- default group.id is `null` instead of empty string
> > > >  - API improvement
> > > >- Producer: introduce close(Duration)
> > > >- AdminClient: introduce close(Duration)
> > > >- Kafka Streams: new flatTransform() operator in Streams DSL
> > > >- KafkaStreams (and other classed) now implement AutoClosable to
> > > > support try-with-resource
> > > >- New Serdes and default method implementations
> > > >  - Kafka Streams exposed internal client.id via ThreadMetadata
> > > >  - Metric improvements:  All `-min`, `-avg` and `-max` metrics will
> now
> > > > output `NaN` as default value
> > > > Release notes for the 2.2.0 release:
> > > > https://home.apache.org/~mjsax/kafka-2.2.0-rc2/RELEASE_NOTES.html
> > > >
> > > > *** Please download, test, and vote by Thursday, March 14, 9am PST.
> > > >
> > > > Kafka's KEYS file containing PGP keys we use to sign the release:
> > > > https://kafka.apache.org/KEYS
> > > >
> > > > * Release artifacts to be voted upon (source and binary):
> > > > https://home.apache.org/~mjsax/kafka-2.2.0-rc2/
> > > >
> > > > * Maven artifacts to be voted upon:
> > > >
> https://repository.apache.org/content/groups/staging/org/apache/kafka/
> > > >
> > > > * Javadoc:
> > > > https://home.apache.org/~mjsax/kafka-2.2.0-rc2/javadoc/
> > > >
> > > > * Tag to be voted upon (off 2.2 branch) is the 2.2.0 tag:
> > > > https://github.com/apache/kafka/releases/tag/2.2.0-rc2
> > > >
> > > > * Documentation:
> > > > https://kafka.apache.org/22/documentation.html
> > > >
> > > > * Protocol:
> > > > https://kafka.apache.org/22/protocol.html
> > > >
> > > > * Jenkins builds for the 2.2 branch:
> > > > Unit/integration tests:
> https://builds.apache.org/job/kafka-2.2-jdk8/
> > > > System tests:
> > > https://jenkins.confluent.io/job/system-test-kafka/job/2.2/
> > > >
> > > > /**
> > > >
> > > > Thanks,
> > > >
> > > > -Matthias
> > > >
> > > >
> > >
> >
>

Re: [ANNOUNCE] New Committer: Randall Hauch

[Satish Duggana]

Congratulations Randall!

On Fri, Feb 15, 2019 at 1:51 PM Mickael Maison  wrote:
>
> Congrats Randall!
>
> On Fri, Feb 15, 2019 at 6:37 AM James Cheng  wrote:
> >
> > Congrats, Randall! Well deserved!
> >
> > -James
> >
> > Sent from my iPhone
> >
> > > On Feb 14, 2019, at 6:16 PM, Guozhang Wang  wrote:
> > >
> > > Hello all,
> > >
> > > The PMC of Apache Kafka is happy to announce another new committer joining
> > > the project today: we have invited Randall Hauch as a project committer 
> > > and
> > > he has accepted.
> > >
> > > Randall has been participating in the Kafka community for the past 3 
> > > years,
> > > and is well known as the founder of the Debezium project, a popular 
> > > project
> > > for database change-capture streams using Kafka (https://debezium.io). 
> > > More
> > > recently he has become the main person keeping Kafka Connect moving
> > > forward, participated in nearly all KIP discussions and QAs on the mailing
> > > list. He's authored 6 KIPs and authored 50 pull requests and conducted 
> > > over
> > > a hundred reviews around Kafka Connect, and has also been evangelizing
> > > Kafka Connect at several Kafka Summit venues.
> > >
> > >
> > > Thank you very much for your contributions to the Connect community 
> > > Randall
> > > ! And looking forward to many more :)
> > >
> > >
> > > Guozhang, on behalf of the Apache Kafka PMC

Re: [ANNOUNCE] New Committer: Bill Bejeck

[Satish Duggana]

Congratulations Bill!

On Thu, Feb 14, 2019 at 6:41 AM Marcelo Barbosa
 wrote:
>
> Wow! Congrats Bill!
> Cheers,
> Barbosa
> Em quarta-feira, 13 de fevereiro de 2019 23:03:54 BRST, Guozhang Wang 
>  escreveu:
>
>  Hello all,
>
> The PMC of Apache Kafka is happy to announce that we've added Bill Bejeck
> as our newest project committer.
>
> Bill has been active in the Kafka community since 2015. He has made
> significant contributions to the Kafka Streams project with more than 100
> PRs and 4 authored KIPs, including the streams topology optimization
> framework. Bill's also very keen on tightening Kafka's unit test / system
> tests coverage, which is a great value to our project codebase.
>
> In addition, Bill has been very active in evangelizing Kafka for stream
> processing in the community. He has given several Kafka meetup talks in the
> past year, including a presentation at Kafka Summit SF. He's also authored
> a book about Kafka Streams (
> https://www.manning.com/books/kafka-streams-in-action), as well as various
> of posts in public venues like DZone as well as his personal blog (
> http://codingjunkie.net/).
>
> We really appreciate the contributions and are looking forward to see more
> from him. Congratulations, Bill !
>
>
> Guozhang, on behalf of the Apache Kafka PMC
>

Re: [VOTE] 2.1.1 RC2

[Satish Duggana]

+1 (non-binding)

- Ran testAll/releaseTarGzAll  successfully with NO failures.
- Ran through quickstart of core/streams on builds generated from tag
- Ran few internal apps targeting to topics on 3 node cluster.

Thanks for running the release Colin!

Satish.

On Mon, Feb 11, 2019 at 2:29 PM Mickael Maison  wrote:
>
> +1 (non binding)
> I built from source and ran Quickstart. All good. Thanks Colin
>
>
> On Mon, Feb 11, 2019 at 12:43 AM Vahid Hashemian
>  wrote:
> >
> > +1
> >
> > I built from the source, and ran the Quickstart on Ubuntu 18.04 with Java 8.
> > Some unit tests fail for me but none consistently across several runs.
> >
> > Thanks for running the release Colin.
> >
> > --Vahid
> >
> > On Sat, Feb 9, 2019 at 2:24 PM Jakub Scholz  wrote:
> >
> > > +1 (non-binding). I built it from source and run my tests. Everything 
> > > seems
> > > to be fine.
> > >
> > > On Sat, Feb 9, 2019 at 12:10 AM Magnus Edenhill 
> > > wrote:
> > >
> > > > +1
> > > >
> > > > Passes librdkafka test suite.
> > > >
> > > > Den fre 8 feb. 2019 kl 21:02 skrev Colin McCabe :
> > > >
> > > > > Hi all,
> > > > >
> > > > > This is the third candidate for release of Apache Kafka 2.1.1.  This
> > > > > release includes many bug fixes for Apache Kafka 2.1.
> > > > >
> > > > > Compared to rc1, this release includes the following changes:
> > > > > * MINOR: release.py: fix some compatibility problems.
> > > > > * KAFKA-7897; Disable leader epoch cache when older message formats 
> > > > > are
> > > > > used
> > > > > * KAFKA-7902: Replace original loginContext if SASL/OAUTHBEARER 
> > > > > refresh
> > > > > login fails
> > > > > * MINOR: Fix more places where the version should be bumped from 2.1.0
> > > ->
> > > > > 2.1.1
> > > > > * KAFKA-7890: Invalidate ClusterConnectionState cache for a broker if
> > > the
> > > > > hostname of the broker changes.
> > > > > * KAFKA-7873; Always seek to beginning in KafkaBasedLog
> > > > > * MINOR: Correctly set dev version in version.py
> > > > >
> > > > > Check out the release notes here:
> > > > > http://home.apache.org/~cmccabe/kafka-2.1.1-rc2/RELEASE_NOTES.html
> > > > >
> > > > > The vote will go until Wednesday, February 13st.
> > > > >
> > > > > * Release artifacts to be voted upon (source and binary):
> > > > > http://home.apache.org/~cmccabe/kafka-2.1.1-rc2/
> > > > >
> > > > > * Maven artifacts to be voted upon:
> > > > > https://repository.apache.org/content/groups/staging/
> > > > >
> > > > > * Javadoc:
> > > > > http://home.apache.org/~cmccabe/kafka-2.1.1-rc2/javadoc/
> > > > >
> > > > > * Tag to be voted upon (off 2.1 branch) is the 2.1.1 tag:
> > > > > https://github.com/apache/kafka/releases/tag/2.1.1-rc2
> > > > >
> > > > > * Jenkins builds for the 2.1 branch:
> > > > > Unit/integration tests: https://builds.apache.org/job/kafka-2.1-jdk8/
> > > > >
> > > > > Thanks to everyone who tested the earlier RCs.
> > > > >
> > > > > cheers,
> > > > > Colin
> > > > >
> > > >
> > >

Re: [ANNOUNCE] New Committer: Vahid Hashemian

[Satish Duggana]

Congratulations Vahid!

On Tue, Jan 15, 2019 at 5:47 PM Yishun Guan  wrote:
>
> Congratulations!
>
> On Tue, Jan 15, 2019, 16:46 James Cheng  wrote:
>
> > Congrats, Vahid!!
> >
> > -James
> >
> > > On Jan 15, 2019, at 2:44 PM, Jason Gustafson  wrote:
> > >
> > > Hi All,
> > >
> > > The PMC for Apache Kafka has invited Vahid Hashemian as a project
> > committer and
> > > we are
> > > pleased to announce that he has accepted!
> > >
> > > Vahid has made numerous contributions to the Kafka community over the
> > past
> > > few years. He has authored 13 KIPs with core improvements to the consumer
> > > and the tooling around it. He has also contributed nearly 100 patches
> > > affecting all parts of the codebase. Additionally, Vahid puts a lot of
> > > effort into community engagement, helping others on the mail lists and
> > > sharing his experience at conferences and meetups.
> > >
> > > We appreciate the contributions and we are looking forward to more.
> > > Congrats Vahid!
> > >
> > > Jason, on behalf of the Apache Kafka PMC
> >
> >

Re: [VOTE] 2.1.0 RC1

[Satish Duggana]

+1 (non-binding)

- Ran testAll/releaseTarGzAll on 2.1.0 successfully without any failures.
- Ran through quickstart of core/streams on builds generated from tag
- Ran few internal apps targeting to topics on 3 node cluster.

Thanks,
Satish.
On Sun, Nov 18, 2018 at 4:58 AM Jakub Scholz  wrote:
>
> +1 (non-binding) ... I used the staged artifacts and binaries and run my
> tests
>
> On Sat, Nov 10, 2018 at 12:33 AM Dong Lin  wrote:
>
> > Hello Kafka users, developers and client-developers,
> >
> > This is the second candidate for feature release of Apache Kafka 2.1.0.
> >
> > This is a major version release of Apache Kafka. It includes 28 new KIPs
> > and
> >
> > critical bug fixes. Please see the Kafka 2.1.0 release plan for more
> > details:
> >
> > *
> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=91554044*
> >  > >
> >
> > Here are a few notable highlights:
> >
> > - Java 11 support
> > - Support for Zstandard, which achieves compression comparable to gzip with
> > higher compression and especially decompression speeds(KIP-110)
> > - Avoid expiring committed offsets for active consumer group (KIP-211)
> > - Provide Intuitive User Timeouts in The Producer (KIP-91)
> > - Kafka's replication protocol now supports improved fencing of zombies.
> > Previously, under certain rare conditions, if a broker became partitioned
> > from Zookeeper but not the rest of the cluster, then the logs of replicated
> > partitions could diverge and cause data loss in the worst case (KIP-320)
> > - Streams API improvements (KIP-319, KIP-321, KIP-330, KIP-353, KIP-356)
> > - Admin script and admin client API improvements to simplify admin
> > operation (KIP-231, KIP-308, KIP-322, KIP-324, KIP-338, KIP-340)
> > - DNS handling improvements (KIP-235, KIP-302)
> >
> > Release notes for the 2.1.0 release:
> > http://home.apache.org/~lindong/kafka-2.1.0-rc0/RELEASE_NOTES.html
> >
> > *** Please download, test and vote by Thursday, Nov 15, 12 pm PT ***
> >
> > * Kafka's KEYS file containing PGP keys we use to sign the release:
> > http://kafka.apache.org/KEYS
> >
> > * Release artifacts to be voted upon (source and binary):
> > http://home.apache.org/~lindong/kafka-2.1.0-rc1/
> >
> > * Maven artifacts to be voted upon:
> > https://repository.apache.org/content/groups/staging/
> >
> > * Javadoc:
> > http://home.apache.org/~lindong/kafka-2.1.0-rc1/javadoc/
> >
> > * Tag to be voted upon (off 2.1 branch) is the 2.1.0-rc1 tag:
> > https://github.com/apache/kafka/tree/2.1.0-rc1
> >
> > * Documentation:
> > *http://kafka.apache.org/21/documentation.html*
> > 
> >
> > * Protocol:
> > http://kafka.apache.org/21/protocol.html
> >
> > * Successful Jenkins builds for the 2.1 branch:
> > Unit/integration tests: *https://builds.apache.org/job/kafka-2.1-jdk8/50/
> > *
> >
> > Please test and verify the release artifacts and submit a vote for this RC,
> > or report any issues so we can fix them and get a new RC out ASAP. Although
> > this release vote requires PMC votes to pass, testing, votes, and bug
> > reports are valuable and appreciated from everyone.
> >
> > Cheers,
> > Dong
> >

Re: [VOTE] 2.1.0 RC0

[Satish Duggana]

Hi Dong,
Is there a RC1 planned with configs documentation fixes and
https://github.com/apache/kafka/pull/5857 ?

Thanks,
Satish.
On Thu, Nov 1, 2018 at 4:05 PM Jakub Scholz  wrote:
>
> +1 (non-binding) ... I used the staged binaries and checked it with
> different clients.
>
> On Wed, Oct 24, 2018 at 10:17 AM Dong Lin  wrote:
>
> > Hello Kafka users, developers and client-developers,
> >
> > This is the first candidate for feature release of Apache Kafka 2.1.0.
> >
> > This is a major version release of Apache Kafka. It includes 28 new KIPs
> > and
> >
> > critical bug fixes. Please see the Kafka 2.1.0 release plan for more
> > details:
> >
> > *
> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=91554044*
> >  > >
> >
> > Here are a few notable highlights:
> >
> > - Java 11 support
> > - Support for Zstandard, which achieves compression comparable to gzip with
> > higher compression and especially decompression speeds(KIP-110)
> > - Avoid expiring committed offsets for active consumer group (KIP-211)
> > - Provide Intuitive User Timeouts in The Producer (KIP-91)
> > - Kafka's replication protocol now supports improved fencing of zombies.
> > Previously, under certain rare conditions, if a broker became partitioned
> > from Zookeeper but not the rest of the cluster, then the logs of replicated
> > partitions could diverge and cause data loss in the worst case (KIP-320)
> > - Streams API improvements (KIP-319, KIP-321, KIP-330, KIP-353, KIP-356)
> > - Admin script and admin client API improvements to simplify admin
> > operation (KIP-231, KIP-308, KIP-322, KIP-324, KIP-338, KIP-340)
> > - DNS handling improvements (KIP-235, KIP-302)
> >
> > Release notes for the 2.1.0 release:
> > http://home.apache.org/~lindong/kafka-2.1.0-rc0/RELEASE_NOTES.html
> >
> > *** Please download, test and vote ***
> >
> > * Kafka's KEYS file containing PGP keys we use to sign the release:
> > http://kafka.apache.org/KEYS
> >
> > * Release artifacts to be voted upon (source and binary):
> > http://home.apache.org/~lindong/kafka-2.1.0-rc0/
> >
> > * Maven artifacts to be voted upon:
> > https://repository.apache.org/content/groups/staging/
> >
> > * Javadoc:
> > http://home.apache.org/~lindong/kafka-2.1.0-rc0/javadoc/
> >
> > * Tag to be voted upon (off 2.1 branch) is the 2.1.0-rc0 tag:
> > https://github.com/apache/kafka/tree/2.1.0-rc0
> >
> > * Documentation:
> > *http://kafka.apache.org/21/documentation.html*
> > 
> >
> > * Protocol:
> > http://kafka.apache.org/21/protocol.html
> >
> > * Successful Jenkins builds for the 2.1 branch:
> > Unit/integration tests: *https://builds.apache.org/job/kafka-2.1-jdk8/38/
> > *
> >
> > Please test and verify the release artifacts and submit a vote for this RC,
> > or report any issues so we can fix them and get a new RC out ASAP. Although
> > this release vote requires PMC votes to pass, testing, votes, and bug
> > reports are valuable and appreciated from everyone.
> >
> > Cheers,
> > Dong
> >

Re: [VOTE] 2.0.1 RC0

[Satish Duggana]

+1 (non-binding)

- Ran testAll/releaseTarGzAll on 2.0.1
- Ran through quickstart of core/streams on builds generated from tag
- Ran few internal apps targeting to topics on 3 node cluster.

Thanks,
Satish.
On Sat, Nov 3, 2018 at 2:32 AM Ewen Cheslack-Postava  wrote:
>
> +1
>
> -Ewen
>
> On Thu, Nov 1, 2018 at 10:10 AM Manikumar  wrote:
>
> > We were waiting for the system test results. There were few failures:
> > KAFKA-7579,  KAFKA-7559, KAFKA-7561
> > they are not blockers for 2.0.1 release. We need more votes from
> > PMC/committers :)
> >
> > Thanks Stanislav! for the system test results.
> >
> > Thanks,
> > Manikumar
> >
> > On Thu, Nov 1, 2018 at 10:20 PM Eno Thereska 
> > wrote:
> >
> > > Anything else holding this up?
> > >
> > > Thanks
> > > Eno
> > >
> > > On Thu, Nov 1, 2018 at 10:27 AM Jakub Scholz  wrote:
> > >
> > > > +1 (non-binding) ... I used the staged binaries and run tests with
> > > > different clients.
> > > >
> > > > On Fri, Oct 26, 2018 at 4:29 AM Manikumar 
> > > > wrote:
> > > >
> > > > > Hello Kafka users, developers and client-developers,
> > > > >
> > > > > This is the first candidate for release of Apache Kafka 2.0.1.
> > > > >
> > > > > This is a bug fix release closing 49 tickets:
> > > > > https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+2.0.1
> > > > >
> > > > > Release notes for the 2.0.1 release:
> > > > > http://home.apache.org/~manikumar/kafka-2.0.1-rc0/RELEASE_NOTES.html
> > > > >
> > > > > *** Please download, test and vote by  Tuesday, October 30, end of
> > day
> > > > >
> > > > > Kafka's KEYS file containing PGP keys we use to sign the release:
> > > > > http://kafka.apache.org/KEYS
> > > > >
> > > > > * Release artifacts to be voted upon (source and binary):
> > > > > http://home.apache.org/~manikumar/kafka-2.0.1-rc0/
> > > > >
> > > > > * Maven artifacts to be voted upon:
> > > > > https://repository.apache.org/content/groups/staging/
> > > > >
> > > > > * Javadoc:
> > > > > http://home.apache.org/~manikumar/kafka-2.0.1-rc0/javadoc/
> > > > >
> > > > > * Tag to be voted upon (off 2.0 branch) is the 2.0.1 tag:
> > > > > https://github.com/apache/kafka/releases/tag/2.0.1-rc0
> > > > >
> > > > > * Documentation:
> > > > > http://kafka.apache.org/20/documentation.html
> > > > >
> > > > > * Protocol:
> > > > > http://kafka.apache.org/20/protocol.html
> > > > >
> > > > > * Successful Jenkins builds for the 2.0 branch:
> > > > > Unit/integration tests:
> > > > https://builds.apache.org/job/kafka-2.0-jdk8/177/
> > > > >
> > > > > /**
> > > > >
> > > > > Thanks,
> > > > > Manikumar
> > > > >
> > > >
> > >
> >

Re: Kafka consumer offset topic index not getting deleted

[Satish Duggana]

>>Offsets.retention.minutes (default is 7 days, not 24 hours).
In 0.11.x , default value was 24 hrs, it is changed to 7 days in
2.0[1]. Kaushik mentions that they are using 0.11.xx

1. http://kafka.apache.org/documentation/#upgrade_200_notable

On Mon, Sep 24, 2018 at 8:42 PM, Kaushik Nambiar
 wrote:
> Hello,
> Thankyou for your reply.
> I have attached a text file containing the data within the server.properties
> file.
>
> Also I could see that it was the .log files within the __consumer_offset
> topic that were sizing around 100 mb each.
> So due to many such log files,the disk is getting maxed out.
>
> Your comments on the same.
>
> Regards,
> Kaushik Nambiar
>
>
> On Mon, Sep 24, 2018, 5:06 PM M. Manna  wrote:
>>
>> What are your settings for:
>>
>> 1) Offsets.retention.check.interval.ms
>> 2) Offsets.retention.minutes (default is 7 days, not 24 hours).
>>
>> Also, did this occur even after you restarted any individual brokers?
>> Please share the server.properties "As is" for your case.
>>
>> Regards,
>>
>> On Mon, 24 Sep 2018 at 12:14, Kaushik Nambiar 
>> wrote:
>>
>> > Hello,
>> > I am using a Kafka with version 0.11.xx.
>> > When I check the logs I can see the index segments for user defined
>> > topics
>> > are getting deleted.
>> > But I cannot find the indices for the consumer_offset topic getting
>> > deleted.
>> > That's causing around GBs of data getting accumulated in our persistent
>> > disk.
>> > Based on the server.properties file,we haven't set any
>> > offsets.retention.minutes.
>> > So I am assuming the default values would be used which is 24 hours.
>> > But still I can find .index files in the __consumer_offsets topic which
>> > are
>> > months old.
>> > Cannot you guys tell me what could be the reason for this.
>> > Also a workaround to fix this issue would help too.
>> >
>> > Any kind of help is greatly appreciated.
>> >
>> > Regards,
>> > Kaushik Nambiar
>> >

Re: Determining broker max buffer size from producer.

[Satish Duggana]

Are you asking about the maximum message size that can be sent from a
producer?
max.message.bytes is used to control that at broker/topic level. You can
find more details
at the below links.

*"The largest record batch size allowed by Kafka. If this is increased and
there are consumers older than 0.10.2, the consumers' fetch size must also
be increased so that the they can fetch record batches this large. In the
latest message format version, records are always grouped into batches for
efficiency. In previous message format versions, uncompressed records are
not grouped into batches and this limit only applies to a single record in
that case."*


https://kafka.apache.org/documentation/#topicconfigs
https://kafka.apache.org/documentation/#brokerconfigs

Thanks,
Satish.

On Mon, Sep 3, 2018 at 2:12 PM, claude.war...@wipro.com <
claude.war...@wipro.com> wrote:

> Greetings,
>
>
> I have spent several hours looking through documentation and historical
> email chains and have been unable to find a solution to my problem.
>
>
> I have a case where I need to construct a producer and I need to know the
> broker max buffer size so that I don't get any nasty buffer too big errors
> from the server when serializing large objects.  I have been unable to find
> any way to do this short of reading the broker configuration and I can't do
> that from a remote call (as far as I can tell).  Does anybody have a
> solution?
>
>
> Claude
>
> The information contained in this electronic message and any attachments
> to this message are intended for the exclusive use of the addressee(s) and
> may contain proprietary, confidential or privileged information. If you are
> not the intended recipient, you should not disseminate, distribute or copy
> this e-mail. Please notify the sender immediately and destroy all copies of
> this message and any attachments. WARNING: Computer viruses can be
> transmitted via email. The recipient should check this email and any
> attachments for the presence of viruses. The company accepts no liability
> for any damage caused by any virus transmitted by this email.
> www.wipro.com
>

Re: Kafka Polling alternate

[Satish Duggana]

There is no push mechanism available for consumers in Kafka. What is the
current timeout passed to KafkaConsumer#poll(timeout)? You can increase
that timeout to avoid calling poll frequently.

Thanks,
Satish.

On Fri, Aug 31, 2018 at 12:27 AM, Pulkit Manchanda 
wrote:

> HI All,
> I have a consumer application continuously polling for the record in a
> while loop wasting the CPU cycles.
> Is there any alternative like I get a callback/event from Kafka sooner the
> producer publishes the record to the topic.
> Thanks
> Pulkit
>

Re: [DISCUSS] KIP-369 Alternative Partitioner to Support "Always Round-Robin" Selection

[Satish Duggana]

+including both dev and user mailing lists.

Hi,
Thanks for the KIP.

"* For us, the message keys represent some metadata which we use to either
ignore messages (if a loop-back to the sender), or log some information.*"

Above statement was mentioned in the KIP about how key value is used. I
guess the topic is not configured to be compacted and you do not want to
have partitioning based on that key. IMHO, it qualifies more as a header
than a key. What do you think about building records with a specific header
and consumers to execute the logic whether to process or ignore the
messages based on that header value.

Thanks,
Satish.


On Fri, Aug 31, 2018 at 1:32 AM, Satish Duggana 
wrote:

> Hi,
> Thanks for the KIP.
>
> "* For us, the message keys represent some metadata which we use to
> either ignore messages (if a loop-back to the sender), or log some
> information.*"
>
> Above statement was mentioned in the KIP about how key value is used. I
> guess the topic is not configured to be compacted and you do not want to
> have partitioning based on that key. IMHO, it qualifies more as a header
> than a key. What do you think about building records with a specific header
> and consumers to execute the logic whether to process or ignore the
> messages based on that header value.
>
> Thanks,
> Satish.
>
>
> On Fri, Aug 31, 2018 at 12:02 AM, M. Manna  wrote:
>
>> Hi Harsha,
>>
>> thanks for reading the KIP.
>>
>> The intent is to use the DefaultPartitioner logic for round-robin
>> selection
>> of partition regardless of key type.
>>
>> Implementing Partitioner interface isn’t the issue here, you would have to
>> do that anyway if  you are implementing your own. But we also want this to
>> be part of formal codebase.
>>
>> Regards,
>>
>> On Thu, 30 Aug 2018 at 16:58, Harsha  wrote:
>>
>> > Hi,
>> >   Thanks for the KIP. I am trying to understand the intent of the
>> > KIP.  Is the use case you specified can't be achieved by implementing
>> the
>> > Partitioner interface here?
>> > https://github.com/apache/kafka/blob/trunk/clients/src/main/
>> java/org/apache/kafka/clients/producer/Partitioner.java#L28
>> > .
>> > Use your custom partitioner to be configured in your producer clients.
>> >
>> > Thanks,
>> > Harsha
>> >
>> > On Thu, Aug 30, 2018, at 1:45 AM, M. Manna wrote:
>> > > Hello,
>> > >
>> > > I opened a very simple KIP and there exists a JIRA for it.
>> > >
>> > > I would be grateful if any comments are available for action.
>> > >
>> > > Regards,
>> >
>>
>
>

Re: [DISCUSS] KIP-369 Alternative Partitioner to Support "Always Round-Robin" Selection

[Satish Duggana]

Hi,
Thanks for the KIP.

"* For us, the message keys represent some metadata which we use to either
ignore messages (if a loop-back to the sender), or log some information.*"

Above statement was mentioned in the KIP about how key value is used. I
guess the topic is not configured to be compacted and you do not want to
have partitioning based on that key. IMHO, it qualifies more as a header
than a key. What do you think about building records with a specific header
and consumers to execute the logic whether to process or ignore the
messages based on that header value.

Thanks,
Satish.


On Fri, Aug 31, 2018 at 12:02 AM, M. Manna  wrote:

> Hi Harsha,
>
> thanks for reading the KIP.
>
> The intent is to use the DefaultPartitioner logic for round-robin selection
> of partition regardless of key type.
>
> Implementing Partitioner interface isn’t the issue here, you would have to
> do that anyway if  you are implementing your own. But we also want this to
> be part of formal codebase.
>
> Regards,
>
> On Thu, 30 Aug 2018 at 16:58, Harsha  wrote:
>
> > Hi,
> >   Thanks for the KIP. I am trying to understand the intent of the
> > KIP.  Is the use case you specified can't be achieved by implementing the
> > Partitioner interface here?
> > https://github.com/apache/kafka/blob/trunk/clients/src/
> main/java/org/apache/kafka/clients/producer/Partitioner.java#L28
> > .
> > Use your custom partitioner to be configured in your producer clients.
> >
> > Thanks,
> > Harsha
> >
> > On Thu, Aug 30, 2018, at 1:45 AM, M. Manna wrote:
> > > Hello,
> > >
> > > I opened a very simple KIP and there exists a JIRA for it.
> > >
> > > I would be grateful if any comments are available for action.
> > >
> > > Regards,
> >
>

Re: Offset for consumer group set to Null in __consumer_offsets, leads to data loss

[Satish Duggana]

In Kafka 2.0, default offsets retention period is changed to 7 days[a] and
better expiration semantics of consumer groups was added as part of
KIP-211[b].


a - https://kafka.apache.org/documentation/#upgrade_200_notable
b -
https://cwiki.apache.org/confluence/display/KAFKA/KIP-211%3A+Revise+Expiration+Semantics+of+Consumer+Group+Offsets


On Mon, Aug 27, 2018 at 7:26 PM, Biplob Biswas 
wrote:

> Hi Manikumar,
>
> Thanks for the reply, it seems you caught the correct nerve as we never set
> this property, and if kafka is taking the default value then it would mean
> that the consumer group would expire in a day, which completely explains
> our behaviour.
>
> Thanks a lot.  We will test this and verify and report back.
>
> Thanks & Regards
> Biplob Biswas
>
>
> On Mon, Aug 27, 2018 at 10:12 AM Manikumar 
> wrote:
>
> > What is the value set for offsets.retention.minutes?  Default value is
> > `1day` in Kafka 0.11 release.
> > Based on your requirement, you may need to increase this value.
> >
> > On Mon, Aug 27, 2018 at 1:33 PM Biplob Biswas 
> > wrote:
> >
> > > Hi,
> > >
> > > Can anyone provide any insights to the issue we faced?
> > >
> > > Thanks & Regards
> > > Biplob Biswas
> > >
> > >
> > > On Fri, Aug 24, 2018 at 4:50 PM Biplob Biswas <
> revolutioni...@gmail.com>
> > > wrote:
> > >
> > > > Hi everyone,
> > > >
> > > > *Short Version:*
> > > > We had an unusual situation where after restart of our spark job,
> > rather
> > > > than reading from the last known offset of the consumer group, it
> > started
> > > > consuming from the latest offset, thus leading to data loss.
> > > >
> > > > *Long Version:*
> > > > We have a spark job which crashed last Friday due to an NPE
> introduced
> > by
> > > > developer code. Over the weekend the job was not restarted, although
> on
> > > > monday the bug was fixed and the job was restarted. The
> consumer-group
> > > was
> > > > *NOT* changed, but when we restarted the job, the spark job started
> > > > reading form the latest offset.
> > > >
> > > > We started investigating the reasons for the same and we verified our
> > > > spark job configuration and it looks good, we tried to reproduce the
> > > issue
> > > > by killing the job and restarting while new events are coming in and
> it
> > > > reads the buffered data properly. We also verified that the consumer
> > > group
> > > > had a valid lag which was reducing over time.
> > > >
> > > > Next we checked __consumer_offsets topic and we found the something
> > which
> > > > could explain why spark job started reading fro latest. We found the
> > > > following lines which were not compressed by the topic itself
> > > >
> > > > [spark-job-consumer-group,spark-job-reading-topic,1]::NULL
> > > >> [spark-job-consumer-group,spark-job-reading-topic,3]::NULL
> > > >> [spark-job-consumer-group,spark-job-reading-topic,0]::NULL
> > > >> [spark-job-consumer-group,spark-job-reading-topic,2]::NULL
> > > >
> > > > --
> > > > These following offset commits came after we started ingesting again
> on
> > > > monday after bug fix:
> > > >
> > > >
> > >
> > [spark-job-consumer-group,spark-job-reading-topic,3]::[
> OffsetMetadata[606979,NO_METADATA],CommitTime
> > > >> 1534778505013,ExpirationTime 1534864905013]
> > > >>
> > >
> > [spark-job-consumer-group,spark-job-reading-topic,1]::[
> OffsetMetadata[607195,NO_METADATA],CommitTime
> > > >> 1534778505013,ExpirationTime 1534864905013]
> > > >>
> > >
> > [spark-job-consumer-group,spark-job-reading-topic,2]::[
> OffsetMetadata[607100,NO_METADATA],CommitTime
> > > >> 1534778505013,ExpirationTime 1534864905013]
> > > >>
> > >
> > [spark-job-consumer-group,spark-job-reading-topic,0]::[
> OffsetMetadata[607330,NO_METADATA],CommitTime
> > > >> 1534778505013,ExpirationTime 1534864905013]
> > > >>
> > >
> > [spark-job-consumer-group,spark-job-reading-topic,3]::[
> OffsetMetadata[606979,NO_METADATA],CommitTime
> > > >> 1534778508010,ExpirationTime 1534864908010]
> > > >>
> > >
> > [spark-job-consumer-group,spark-job-reading-topic,1]::[
> OffsetMetadata[607195,NO_METADATA],CommitTime
> > > >> 1534778508010,ExpirationTime 1534864908010]
> > > >>
> > >
> > [spark-job-consumer-group,spark-job-reading-topic,2]::[
> OffsetMetadata[607100,NO_METADATA],CommitTime
> > > >> 1534778508010,ExpirationTime 1534864908010]
> > > >>
> > >
> > [spark-job-consumer-group,spark-job-reading-topic,0]::[
> OffsetMetadata[607330,NO_METADATA],CommitTime
> > > >> 1534778508010,ExpirationTime 1534864908010]
> > > >>
> > >
> > [spark-job-consumer-group,spark-job-reading-topic,3]::[
> OffsetMetadata[606979,NO_METADATA],CommitTime
> > > >> 1534778511010,ExpirationTime 1534864911010]
> > > >>
> > >
> > [spark-job-consumer-group,spark-job-reading-topic,1]::[
> OffsetMetadata[607195,NO_METADATA],CommitTime
> > > >> 1534778511010,ExpirationTime 1534864911010]
> > > >>
> > >
> > [spark-job-consumer-group,spark-job-reading-topic,2]::[
> OffsetMetadata[607100,NO_METADATA],CommitTime
> > > >> 1534778511010,ExpirationTime 

Re: [VOTE] 2.0.0 RC2

[Satish Duggana]

+1 (non-binding)

- Ran testAll/releaseTarGzAll on 2.0.0-rc2 tag
- Ran through quickstart of core/streams on builds generated from tag
- Ran few internal apps targeting to topics on 3 node cluster.

Thanks,
Satish.

On Sun 15 Jul, 2018, 9:55 PM Rajini Sivaram, 
wrote:

> Hi Ismael,
>
> Thank you for pointing that out. I have re-uploaded the RC2 artifacts to
> maven including streams-scala_2.12. Also submitted a PR to update build &
> release scripts to include this.
>
> Thank you,
>
> Rajini
>
>
>
> On Fri, Jul 13, 2018 at 7:19 AM, Ismael Juma  wrote:
>
> > Hi Rajini,
> >
> > Thanks for generating the RC. It seems like the kafka-streams-scala 2.12
> > artifact is missing from the Maven repository:
> >
> > https://repository.apache.org/content/groups/staging/org/apache/kafka/
> >
> > Since this is the first time we are publishing this artifact, it is
> > possible that this never worked properly.
> >
> > Ismael
> >
> > On Tue, Jul 10, 2018 at 10:17 AM Rajini Sivaram  >
> > wrote:
> >
> > > Hello Kafka users, developers and client-developers,
> > >
> > >
> > > This is the third candidate for release of Apache Kafka 2.0.0.
> > >
> > >
> > > This is a major version release of Apache Kafka. It includes 40 new
> KIPs
> > > and
> > >
> > > several critical bug fixes. Please see the 2.0.0 release plan for more
> > > details:
> > >
> > > https://cwiki.apache.org/confluence/pages/viewpage.
> > action?pageId=80448820
> > >
> > >
> > > A few notable highlights:
> > >
> > >- Prefixed wildcard ACLs (KIP-290), Fine grained ACLs for
> CreateTopics
> > >(KIP-277)
> > >- SASL/OAUTHBEARER implementation (KIP-255)
> > >- Improved quota communication and customization of quotas (KIP-219,
> > >KIP-257)
> > >- Efficient memory usage for down conversion (KIP-283)
> > >- Fix log divergence between leader and follower during fast leader
> > >failover (KIP-279)
> > >- Drop support for Java 7 and remove deprecated code including old
> > scala
> > >clients
> > >- Connect REST extension plugin, support for externalizing secrets
> and
> > >improved error handling (KIP-285, KIP-297, KIP-298 etc.)
> > >- Scala API for Kafka Streams and other Streams API improvements
> > >(KIP-270, KIP-150, KIP-245, KIP-251 etc.)
> > >
> > >
> > > Release notes for the 2.0.0 release:
> > >
> > > http://home.apache.org/~rsivaram/kafka-2.0.0-rc2/RELEASE_NOTES.html
> > >
> > >
> > > *** Please download, test and vote by Friday, July 13, 4pm PT
> > >
> > >
> > > Kafka's KEYS file containing PGP keys we use to sign the release:
> > >
> > > http://kafka.apache.org/KEYS
> > >
> > >
> > > * Release artifacts to be voted upon (source and binary):
> > >
> > > http://home.apache.org/~rsivaram/kafka-2.0.0-rc2/
> > >
> > >
> > > * Maven artifacts to be voted upon:
> > >
> > > https://repository.apache.org/content/groups/staging/
> > >
> > >
> > > * Javadoc:
> > >
> > > http://home.apache.org/~rsivaram/kafka-2.0.0-rc2/javadoc/
> > >
> > >
> > > * Tag to be voted upon (off 2.0 branch) is the 2.0.0 tag:
> > >
> > > https://github.com/apache/kafka/tree/2.0.0-rc2
> > >
> > >
> > >
> > > * Documentation:
> > >
> > > http://kafka.apache.org/20/documentation.html
> > >
> > >
> > > * Protocol:
> > >
> > > http://kafka.apache.org/20/protocol.html
> > >
> > >
> > > * Successful Jenkins builds for the 2.0 branch:
> > >
> > > Unit/integration tests:
> https://builds.apache.org/job/kafka-2.0-jdk8/72/
> > >
> > > System tests:
> > > https://jenkins.confluent.io/job/system-test-kafka/job/2.0/27/
> > >
> > >
> > > /**
> > >
> > >
> > > Thanks,
> > >
> > >
> > > Rajini
> > >
> >
>

Re: [VOTE] 1.1.1 RC3

[Satish Duggana]

+1 (non-binding)

- Ran testAll/releaseTarGzAll on 1.1.0-rc0 tag with jdk-8.
- Ran through quickstart of core/streams on builds.
- Ran few internal apps targeting to topics on 3 node cluster.

Thanks,
Satish.


On Mon, Jul 9, 2018 at 11:39 PM, Harsha  wrote:

> +1.
>
> * Ran unit tests
> * Installed in a cluster and ran simple tests
>
> Thanks,
> Harsha
>
> On Mon, Jul 9th, 2018 at 6:38 AM, Ted Yu  wrote:
>
> >
> >
> >
> > +1
> >
> > Ran test suite.
> >
> > Checked signatures.
> >
> >
> >
> > On Sun, Jul 8, 2018 at 3:36 PM Dong Lin < lindon...@gmail.com > wrote:
> >
> > > Hello Kafka users, developers and client-developers,
> > >
> > > This is the fourth candidate for release of Apache Kafka 1.1.1.
> > >
> > > Apache Kafka 1.1.1 is a bug-fix release for the 1.1 branch that was
> > first
> > > released with 1.1.0 about 3 months ago. We have fixed about 25 issues
> > since
> > > that release. A few of the more significant fixes include:
> > >
> > > KAFKA-6925 < https://issues.apache.org/jira/browse/KAFKA-6925> - Fix
> > memory
> > > leak in StreamsMetricsThreadImpl
> > > KAFKA-6937 < https://issues.apache.org/jira/browse/KAFKA-6937> -
> In-sync
> >
> > > replica delayed during fetch if replica throttle is exceeded
> > > KAFKA-6917 < https://issues.apache.org/jira/browse/KAFKA-6917> -
> Process
> >
> > > txn
> > > completion asynchronously to avoid deadlock
> > > KAFKA-6893 < https://issues.apache.org/jira/browse/KAFKA-6893> -
> Create
> > > processors before starting acceptor to avoid ArithmeticException
> > > KAFKA-6870 < https://issues.apache.org/jira/browse/KAFKA-6870> -
> > > Fix ConcurrentModificationException in SampledStat
> > > KAFKA-6878 < https://issues.apache.org/jira/browse/KAFKA-6878> - Fix
> > > NullPointerException when querying global state store
> > > KAFKA-6879 < https://issues.apache.org/jira/browse/KAFKA-6879> -
> Invoke
> > > session init callbacks outside lock to avoid Controller deadlock
> > > KAFKA-6857 < https://issues.apache.org/jira/browse/KAFKA-6857> -
> Prevent
> >
> > > follower from truncating to the wrong offset if undefined leader epoch
> > is
> > > requested
> > > KAFKA-6854 < https://issues.apache.org/jira/browse/KAFKA-6854> - Log
> > > cleaner
> > > fails with transaction markers that are deleted during clean
> > > KAFKA-6747 < https://issues.apache.org/jira/browse/KAFKA-6747> - Check
> > > whether there is in-flight transaction before aborting transaction
> > > KAFKA-6748 < https://issues.apache.org/jira/browse/KAFKA-6748> -
> Double
> > > check before scheduling a new task after the punctuate call
> > > KAFKA-6739 < https://issues.apache.org/jira/browse/KAFKA-6739> -
> > > Fix IllegalArgumentException when down-converting from V2 to V0/V1
> > > KAFKA-6728 < https://issues.apache.org/jira/browse/KAFKA-6728> -
> > > Fix NullPointerException when instantiating the HeaderConverter
> > >
> > > Kafka 1.1.1 release plan:
> > > https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+1.1.1
> > >
> > > Release notes for the 1.1.1 release:
> > > http://home.apache.org/~lindong/kafka-1.1.1-rc3/RELEASE_NOTES.html
> > >
> > > *** Please download, test and vote by Thursday, July 12, 12pm PT ***
> > >
> > > Kafka's KEYS file containing PGP keys we use to sign the release:
> > > http://kafka.apache.org/KEYS
> > >
> > > * Release artifacts to be voted upon (source and binary):
> > > http://home.apache.org/~lindong/kafka-1.1.1-rc3/
> > >
> > > * Maven artifacts to be voted upon:
> > > https://repository.apache.org/content/groups/staging/
> > >
> > > * Javadoc:
> > > http://home.apache.org/~lindong/kafka-1.1.1-rc3/javadoc/
> > >
> > > * Tag to be voted upon (off 1.1 branch) is the 1.1.1-rc3 tag:
> > > https://github.com/apache/kafka/tree/1.1.1-rc3
> > >
> > > * Documentation:
> > > http://kafka.apache.org/11/documentation.html
> > >
> > > * Protocol:
> > > http://kafka.apache.org/11/protocol.html
> > >
> > > * Successful Jenkins builds for the 1.1 branch:
> > > Unit/integration tests: * https://builds.apache.org/job/
> kafka-1.1-jdk7/162
> >
> > > < https://builds.apache.org/job/kafka-1.1-jdk7/162>*
> > > System tests:
> > > https://jenkins.confluent.io/job/system-test-kafka/job/1.1/156/
> > >
> > > Please test and verify the release artifacts and submit a vote for this
> > RC,
> > > or report any issues so we can fix them and get a new RC out ASAP.
> > Although
> > > this release vote requires PMC votes to pass, testing, votes, and bug
> > > reports are valuable and appreciated from everyone.
> > >
> > >
> > > Regards,
> > > Dong
> > >
> >
> >
> >
> >
> >
> >
>

Re: 答复: [ANNOUNCE] New Committer: Dong Lin

[Satish Duggana]

Congratulations Dong!


On Thu, Mar 29, 2018 at 5:12 PM, Sandor Murakozi 
wrote:

> Congrats, Dong!
>
>
> On Thu, Mar 29, 2018 at 2:15 AM, Dong Lin  wrote:
>
> > Thanks everyone!!
> >
> > It is my great pleasure to be part of the Apache Kafka community and help
> > make Apache Kafka more useful to its users. I am super excited to be a
> > Kafka committer and I am hoping to contribute more to its design,
> > implementation and review etc in the future.
> >
> > Thanks!
> > Dong
> >
> > On Wed, Mar 28, 2018 at 4:04 PM, Hu Xi  wrote:
> >
> > > Congrats, Dong Lin!
> > >
> > >
> > > 
> > > 发件人: Matthias J. Sax 
> > > 发送时间: 2018年3月29日 6:37
> > > 收件人: users@kafka.apache.org; d...@kafka.apache.org
> > > 主题: Re: [ANNOUNCE] New Committer: Dong Lin
> > >
> > > Congrats!
> > >
> > > On 3/28/18 1:16 PM, James Cheng wrote:
> > > > Congrats, Dong!
> > > >
> > > > -James
> > > >
> > > >> On Mar 28, 2018, at 10:58 AM, Becket Qin 
> > wrote:
> > > >>
> > > >> Hello everyone,
> > > >>
> > > >> The PMC of Apache Kafka is pleased to announce that Dong Lin has
> > > accepted
> > > >> our invitation to be a new Kafka committer.
> > > >>
> > > >> Dong started working on Kafka about four years ago, since which he
> has
> > > >> contributed numerous features and patches. His work on Kafka core
> has
> > > been
> > > >> consistent and important. Among his contributions, most noticeably,
> > Dong
> > > >> developed JBOD (KIP-112, KIP-113) to handle disk failures and to
> > reduce
> > > >> overall cost, added deleteDataBefore() API (KIP-107) to allow users
> > > >> actively remove old messages. Dong has also been active in the
> > > community,
> > > >> participating in KIP discussions and doing code reviews.
> > > >>
> > > >> Congratulations and looking forward to your future contribution,
> Dong!
> > > >>
> > > >> Jiangjie (Becket) Qin, on behalf of Apache Kafka PMC
> > > >
> > >
> > >
> >
>

Re: [VOTE] 1.1.0 RC2

[Satish Duggana]

Hi Damian,
Thanks for starting vote thread for 1.1.0 release.

There may be a typo on the tag to be voted upon for this release candidate.
I guess it should be https://github.com/apache/kafka/tree/1.1.0-rc2 instead
of https://github.com/apache/kafka/tree/1.1.0-rc.

On Wed, Mar 14, 2018 at 8:27 AM, Satish Duggana <satish.dugg...@gmail.com>
wrote:

> Hi Damian,
> Given release plan link in earlier mail is about 1.0 release. You may want
> to replace that with 1.1.0 release plan link[1].
>
> 1 - https://cwiki.apache.org/confluence/pages/viewpage.
> action?pageId=75957546
>
> Thanks,
> Satish.
>
> On Wed, Mar 14, 2018 at 12:47 AM, Damian Guy <damian@gmail.com> wrote:
>
>> Hello Kafka users, developers and client-developers,
>>
>> This is the third candidate for release of Apache Kafka 1.1.0.
>>
>> This is minor version release of Apache Kakfa. It Includes 29 new KIPs.
>> Please see the release plan for more details:
>>
>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=71764913
>>
>> A few highlights:
>>
>> * Significant Controller improvements (much faster and session expiration
>> edge cases fixed)
>> * Data balancing across log directories (JBOD)
>> * More efficient replication when the number of partitions is large
>> * Dynamic Broker Configs
>> * Delegation tokens (KIP-48)
>> * Kafka Streams API improvements (KIP-205 / 210 / 220 / 224 / 239)
>>
>> Release notes for the 1.1.0 release:
>> http://home.apache.org/~damianguy/kafka-1.1.0-rc2/RELEASE_NOTES.html
>>
>> *** Please download, test and vote by Friday, March 16, 1pm PDT>
>>
>> Kafka's KEYS file containing PGP keys we use to sign the release:
>> http://kafka.apache.org/KEYS
>>
>> * Release artifacts to be voted upon (source and binary):
>> http://home.apache.org/~damianguy/kafka-1.1.0-rc2/
>>
>> * Maven artifacts to be voted upon:
>> https://repository.apache.org/content/groups/staging/
>>
>> * Javadoc:
>> http://home.apache.org/~damianguy/kafka-1.1.0-rc2/javadoc/
>>
>> * Tag to be voted upon (off 1.1 branch) is the 1.1.0 tag:
>> https://github.com/apache/kafka/tree/1.1.0-rc
>> <https://github.com/apache/kafka/tree/1.1.0-rc1>2
>>
>>
>> * Documentation:
>> http://kafka.apache.org/11/documentation.html
>> <http://kafka.apache.org/1/documentation.html>
>>
>> * Protocol:
>> http://kafka.apache.org/11/protocol.html
>> <http://kafka.apache.org/1/protocol.html>
>>
>> * Successful Jenkins builds for the 1.1 branch:
>> Unit/integration tests: https://builds.apache.org/job/kafka-1.1-jdk7/78
>> System tests: https://jenkins.confluent.io/j
>> ob/system-test-kafka/job/1.1/38/
>>
>> /**
>>
>> Thanks,
>> Damian
>>
>>
>> *
>>
>
>

Re: [VOTE] 1.1.0 RC2

[Satish Duggana]

Hi Damian,
Given release plan link in earlier mail is about 1.0 release. You may want
to replace that with 1.1.0 release plan link[1].

1 -
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=75957546

Thanks,
Satish.

On Wed, Mar 14, 2018 at 12:47 AM, Damian Guy  wrote:

> Hello Kafka users, developers and client-developers,
>
> This is the third candidate for release of Apache Kafka 1.1.0.
>
> This is minor version release of Apache Kakfa. It Includes 29 new KIPs.
> Please see the release plan for more details:
>
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=71764913
>
> A few highlights:
>
> * Significant Controller improvements (much faster and session expiration
> edge cases fixed)
> * Data balancing across log directories (JBOD)
> * More efficient replication when the number of partitions is large
> * Dynamic Broker Configs
> * Delegation tokens (KIP-48)
> * Kafka Streams API improvements (KIP-205 / 210 / 220 / 224 / 239)
>
> Release notes for the 1.1.0 release:
> http://home.apache.org/~damianguy/kafka-1.1.0-rc2/RELEASE_NOTES.html
>
> *** Please download, test and vote by Friday, March 16, 1pm PDT>
>
> Kafka's KEYS file containing PGP keys we use to sign the release:
> http://kafka.apache.org/KEYS
>
> * Release artifacts to be voted upon (source and binary):
> http://home.apache.org/~damianguy/kafka-1.1.0-rc2/
>
> * Maven artifacts to be voted upon:
> https://repository.apache.org/content/groups/staging/
>
> * Javadoc:
> http://home.apache.org/~damianguy/kafka-1.1.0-rc2/javadoc/
>
> * Tag to be voted upon (off 1.1 branch) is the 1.1.0 tag:
> https://github.com/apache/kafka/tree/1.1.0-rc
> 2
>
>
> * Documentation:
> http://kafka.apache.org/11/documentation.html
> 
>
> * Protocol:
> http://kafka.apache.org/11/protocol.html
> 
>
> * Successful Jenkins builds for the 1.1 branch:
> Unit/integration tests: https://builds.apache.org/job/kafka-1.1-jdk7/78
> System tests: https://jenkins.confluent.io/job/system-test-kafka/job/1.1/
> 38/
>
> /**
>
> Thanks,
> Damian
>
>
> *
>

Re: [VOTE] 1.0.1 RC1

[Satish Duggana]

+1 (non-binding)

- Ran testAll/releaseTarGzAll on 1.0.1-rc1
 tag
- Ran through quickstart of core/streams

Thanks,
Satish.


On Tue, Feb 13, 2018 at 11:30 PM, Damian Guy  wrote:

> +1
>
> Ran tests, verified streams quickstart works
>
> On Tue, 13 Feb 2018 at 17:52 Damian Guy  wrote:
>
> > Thanks Ewen - i had the staging repo set up as profile that i forgot to
> > add to my maven command. All good.
> >
> > On Tue, 13 Feb 2018 at 17:41 Ewen Cheslack-Postava 
> > wrote:
> >
> >> Damian,
> >>
> >> Which quickstart are you referring to? The streams quickstart only
> >> executes
> >> pre-built stuff afaict.
> >>
> >> In any case, if you're building a maven streams project, did you modify
> it
> >> to point to the staging repository at
> >> https://repository.apache.org/content/groups/staging/ in addition to
> the
> >> default repos? During rc it wouldn't fetch from maven central since it
> >> hasn't been published there yet.
> >>
> >> If that is configured, more compete maven output would be helpful to
> track
> >> down where it is failing to resolve the necessary archetype.
> >>
> >> -Ewen
> >>
> >> On Tue, Feb 13, 2018 at 3:03 AM, Damian Guy 
> wrote:
> >>
> >> > Hi Ewen,
> >> >
> >> > I'm trying to run the streams quickstart and I'm getting:
> >> > [ERROR] Failed to execute goal
> >> > org.apache.maven.plugins:maven-archetype-plugin:3.0.1:generate
> >> > (default-cli) on project standalone-pom: The desired archetype does
> not
> >> > exist (org.apache.kafka:streams-quickstart-java:1.0.1)
> >> >
> >> > Something i'm missing?
> >> >
> >> > Thanks,
> >> > Damian
> >> >
> >> > On Tue, 13 Feb 2018 at 10:16 Manikumar 
> >> wrote:
> >> >
> >> > > +1 (non-binding)
> >> > >
> >> > > ran quick-start, unit tests on the src.
> >> > >
> >> > >
> >> > >
> >> > > On Tue, Feb 13, 2018 at 5:31 AM, Ewen Cheslack-Postava <
> >> > e...@confluent.io>
> >> > > wrote:
> >> > >
> >> > > > Thanks for the heads up, I forgot to drop the old ones, I've done
> >> that
> >> > > and
> >> > > > rc1 artifacts should be showing up now.
> >> > > >
> >> > > > -Ewen
> >> > > >
> >> > > >
> >> > > > On Mon, Feb 12, 2018 at 12:57 PM, Ted Yu 
> >> wrote:
> >> > > >
> >> > > > > +1
> >> > > > >
> >> > > > > Ran test suite which passed.
> >> > > > >
> >> > > > > BTW it seems the staging repo hasn't been updated yet:
> >> > > > >
> >> > > > > https://repository.apache.org/content/groups/staging/org/
> >> > > > > apache/kafka/kafka-clients/
> >> > > > >
> >> > > > > On Mon, Feb 12, 2018 at 10:16 AM, Ewen Cheslack-Postava <
> >> > > > e...@confluent.io
> >> > > > > >
> >> > > > > wrote:
> >> > > > >
> >> > > > > > And of course I'm +1 since I've already done normal release
> >> > > validation
> >> > > > > > before posting this.
> >> > > > > >
> >> > > > > > -Ewen
> >> > > > > >
> >> > > > > > On Mon, Feb 12, 2018 at 10:15 AM, Ewen Cheslack-Postava <
> >> > > > > e...@confluent.io
> >> > > > > > >
> >> > > > > > wrote:
> >> > > > > >
> >> > > > > > > Hello Kafka users, developers and client-developers,
> >> > > > > > >
> >> > > > > > > This is the second candidate for release of Apache Kafka
> >> 1.0.1.
> >> > > > > > >
> >> > > > > > > This is a bugfix release for the 1.0 branch that was first
> >> > released
> >> > > > > with
> >> > > > > > > 1.0.0 about 3 months ago. We've fixed 49 significant issues
> >> since
> >> > > > that
> >> > > > > > > release. Most of these are non-critical, but in aggregate
> >> these
> >> > > fixes
> >> > > > > > will
> >> > > > > > > have significant impact. A few of the more significant fixes
> >> > > include:
> >> > > > > > >
> >> > > > > > > * KAFKA-6277: Make loadClass thread-safe for class loaders
> of
> >> > > Connect
> >> > > > > > > plugins
> >> > > > > > > * KAFKA-6185: Selector memory leak with high likelihood of
> >> OOM in
> >> > > > case
> >> > > > > of
> >> > > > > > > down conversion
> >> > > > > > > * KAFKA-6269: KTable state restore fails after rebalance
> >> > > > > > > * KAFKA-6190: GlobalKTable never finishes restoring when
> >> > consuming
> >> > > > > > > transactional messages
> >> > > > > > > * KAFKA-6529: Stop file descriptor leak when client
> >> disconnects
> >> > > with
> >> > > > > > > staged receives
> >> > > > > > >
> >> > > > > > > Release notes for the 1.0.1 release:
> >> > > > > > > http://home.apache.org/~ewencp/kafka-1.0.1-rc1/
> >> > RELEASE_NOTES.html
> >> > > > > > >
> >> > > > > > > *** Please download, test and vote by Thursday, Feb 15, 5pm
> PT
> >> > ***
> >> > > > > > >
> >> > > > > > > Kafka's KEYS file containing PGP keys we use to sign the
> >> release:
> >> > > > > > > http://kafka.apache.org/KEYS
> >> > > > > > >
> >> > > > > > > * Release artifacts to be voted upon (source and binary):
> >> > > > > > > http://home.apache.org/~ewencp/kafka-1.0.1-rc1/
> >> > > > > > >
> >> > > > > > > * Maven