[ANNOUNCE] Apache Kafka 3.6.2

, David Mao, Divij Vaidya, Edoardo Comar, Emma Humber, Gaurav Narula, Greg Harris, hudeqi, Ismael Juma, Jason Gustafson, Jim Galasyn, Joel Hamill, Johnny Hsu, José Armando García Sancio, Justine Olshan, Luke Chen, Manikumar Reddy, Matthias J. Sax, Mayank Shekhar Narula, Mickael Maison, Mike Lloyd, Paolo

Re: [VOTE] 3.6.2 RC2

Thanks all for voting. I'm now closing the vote. The vote passes with - 3 +1 bindings votes from Divij Vaidya, Justine, and Manikumar - 3 +1 non-binding votes from Andrew, Jakub, and Lianet - 0 -1 votes I'll go ahead and finish the release process. Thanks, On Thu, Apr 4, 2024 at 8:53 PM

Re: [VOTE] 3.6.2 RC2

+1 (binding) from my side also. - verified the signatures, artifacts - ran the tests on the source archive - verified the quickstarts Thanks On Thu, Apr 4, 2024 at 7:46 PM Justine Olshan wrote: > Thanks for clarifying Manikumar! > > +1 (binding) from me > > Justine > >

Re: [VOTE] 3.6.2 RC2

Gentle reminder. Please download, test and vote for the release. Thanks, On Fri, Mar 29, 2024 at 4:57 PM Manikumar wrote: > Hi All, > > System test runs are green. There were 13 test failures in the first run. > All the failed tests passed in the second run. > > System tes

Re: [VOTE] 3.6.2 RC2

Hi All, System test runs are green. There were 13 test failures in the first run. All the failed tests passed in the second run. System test results: https://gist.github.com/omkreddy/17d23d3eb36ef840011f2494d65bbd4f Thanks, On Thu, Mar 28, 2024 at 3:21 PM Manikumar wrote: > Hello Kafka us

[VOTE] 3.6.2 RC2

/~manikumar/kafka-3.6.2-rc2/RELEASE_NOTES.html *** Please download, test and vote by by Wednesday, April 3rd Kafka's KEYS file containing PGP keys we use to sign the release: https://kafka.apache.org/KEYS * Release artifacts to be voted upon (source and binary): https://home.apache.org/~manikumar

Re: [VOTE] 3.6.2 RC1

browse/KAFKA-16222 was fixed, and it > appears to still be an issue with 3.6.2 RC1. The quota on the default > resource is still getting translated improperly. > > I am looking into what the issue is here. > > best, > Colin > > > On Thu, Mar 21, 2024, at 19:32, Chia-Ping

[VOTE] 3.6.2 RC1

Hello Kafka users, developers and client-developers, This is the first candidate for release of Apache Kafka 3.6.2. This is a bugfix release with several fixes, including dependency version bumps for CVEs. Release notes for the 3.6.2 release: https://home.apache.org/~manikumar/kafka-3.6.2-rc1

CVE-2023-25194: Apache Kafka: Possible RCE/Denial of service attack via SASL JAAS JndiLoginModule configuration using Kafka Connect

Severity: important Description: A possible security vulnerability has been identified in Apache Kafka Connect. This requires access to a Kafka Connect worker, and the ability to create/modify connectors on it with an arbitrary Kafka client SASL JAAS config and a SASL-based security protocol,

CVE-2022-34917: Unauthenticated clients may cause OutOfMemoryError on Apache Kafka Brokers

Severity: High Description: A security vulnerability has been identified in Apache Kafka. It affects all releases since 2.8.0. The vulnerability allows malicious unauthenticated clients to allocate large amounts of memory on brokers. This can lead to brokers hitting OutOfMemoryException and

[ANNOUNCE] Apache Kafka 3.2.3

, Netflix, Pinterest, Rabobank, Target, The New York Times, Uber, Yelp, and Zalando, among others. A big thank you for the following 12 contributors to this release! Andrew Borley, Andrew Dean, Colin Patrick McCabe, David Arthur, Derek Troy-West, Divij Vaidya, Jason Gustafson, Manikumar Reddy

[ANNOUNCE] Apache Kafka 3.1.2

Gustafson, Kirk True, Lucas Bradstreet, Manikumar Reddy, Mickael Maison, nicolasguyomar, Niket, Tom Bentley Thanks to Mickael Maison and Tom Bentley for driving this release. We welcome your help and feedback. For more information on how to report problems, and to get involved, visit the project website

[ANNOUNCE] Apache Kafka 3.0.2

, Justine Olshan, Konstantine Karantasis, Lucas Bradstreet, Manikumar Reddy, Mickael Maison, nicolasguyomar, Niket, Philip Nee, Randall Hauch, Stanislav Vodetskyi, Tom Bentley, Vincent Jiang, Xiaoyue Xue, Yang Yu Thanks to Tom Bentley for driving this release. We welcome your help and feedback

[ANNOUNCE] Apache Kafka 2.8.2

, Luke Chen, Manikumar Reddy, Matthias J. Sax, Oliver Hutchison, Philip Nee, Prateek Agarwal, prince-mahajan, Randall Hauch, Stanislav Vodetskyi We welcome your help and feedback. For more information on how to report problems, and to get involved, visit the project website at https://kafka.apache.org

Re: [kafka-clients] [VOTE] 2.7.2 RC0

Hi, +1 (binding) - verified the signatures - verified the quickstart with binary Thanks for running the release! Thanks, Manikumar On Tue, Nov 2, 2021 at 11:16 PM Mickael Maison wrote: > Bumping the thread. > > Contributors, committers and PMC, please take some time to test this

Re: [VOTE] 2.8.1 RC1

Hi, +1 (binding) - verified the signatures - ran the tests on the source archive - verified the quickstart with binary - verified the artifacts, javadoc Thanks for running the release! Thanks, Manikumar On Tue, Sep 14, 2021 at 10:36 PM Chris Egerton wrote: > Hi David, > > I to

Re: [VOTE] 2.6.2 RC0

Hi, +1 (binding) - verified the signatures - ran the tests on the source archive with Scala 2.13 - verified the quickstart with Scala 2.12 binary - verified the artifacts, javadoc Thanks for running the release! Thanks, Manikumar On Tue, Mar 16, 2021 at 9:41 AM Sophie Blee-Goldman wrote

Re: Apache Kafka project CLA requirement

Hi, Apache Software Foundation does not require code contributors (non-committers) to sign CLA. Contributions are considered to be under the Apache License. Committers need to sign a CLA. Thanks, Manikumar On Wed, Jan 20, 2021 at 7:53 PM Radoslava Zheleva wrote: > Hello Team, > >

Re: [VOTE] 2.6.1 RC3

Hi, +1 (binding) - verified signatures - ran the tests on the source archive with Scala 2.13 - verified the core/connect/streams quickstart with Scala 2.13 binary archive. - verified the artifacts, javadoc Thanks for running the release! Thanks, Manikumar On Tue, Dec 15, 2020 at 9:01 PM

Re: [VOTE] 2.5.1 RC0

Hi John, +1 (binding) - Verified signatures, artifacts, Release notes - Built from sources, ran tests - Ran core/connect/streams quick start for Scala 2.13 release, ran few manual tests Thanks for driving the release. Thanks, Manikumar On Thu, Jul 30, 2020 at 10:32 PM Ismael Juma wrote

Re: Request for adding into contributors list

Hi, Thanks for your interest. I just added you to the the contributors list. Thanks, On Mon, Apr 13, 2020 at 6:45 PM Wong Lucent wrote: > Hi, > > > May I add my JIRA ID into the contributors list of Apache kafka. > > Here is my JIRA profile: > > Username: LucentWong > Full name: Yu

Re: [kafka-clients] Re: [VOTE] 2.5.0 RC3

the release. Thanks, Manikumar On Fri, Apr 10, 2020 at 7:53 PM Bill Bejeck wrote: > Hi David, > > +1 (non-binding) Verified signatures, built jars from source, ran unit and > integration tests, all passed. > > Thanks for running the release. > > -Bill > > On Wed, Apr 8,

Re: [ANNOUNCE] New committer: Konstantine Karantasis

Congrats Konstantine! On Thu, Feb 27, 2020 at 7:46 AM Matthias J. Sax wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA512 > > Congrats! > > On 2/27/20 2:21 AM, Jeremy Custenborder wrote: > > Congrats Konstantine! > > > > On Wed, Feb 26, 2020 at 2:39 PM Bill Bejeck > > wrote: > >> > >>

Re: Minor version upgrade instructions from 0.11.0.0 to 0.11.0.3

Hi, Your approach is correct. For minor version upgrade (0.11.0.0 to 0.11.0.3), we can just update the brokers to new version. Thanks, On Mon, Jan 20, 2020 at 1:27 AM Sarath Babu wrote: > Hi all, > Appreciate any help/ pointers on how to upgrade.One thought is to start > the cluster brokers

Re: [ANNOUNCE] Apache Kafka 2.4.0

Thanks to all who contributed to the Apache blog post about 2.4. It's live now at https://blogs.apache.org/kafka/date/20191216 On Mon, Dec 16, 2019 at 10:30 PM Manikumar wrote: > The Apache Kafka community is pleased to announce the release for Apache > Kafka 2.4.0 > > This rele

[ANNOUNCE] Apache Kafka 2.4.0

Bradstreet, LuyingLiu, Magesh Nandakumar, Manikumar Reddy, Matthias J. Sax, Michał Borowiecki, Michał Siatkowski, Mickael Maison, mjarvie, mmanna-sapfgl, Nacho Muñoz Gómez, Nathan Murthy, Nigel Liang, NIkhil Bhatia, Nikolay, Omar Al-Safi, Omkar Mestry, Paul, pkleindl, Rajan Chauhan, Rajini Sivaram, Randall

[RESULTS] [VOTE] Release Kafka version 2.4.0

/qlira627sqbmmzz4 I'll continue with the release process and the release announcement will follow in the next few days. Manikumar

Re: [VOTE] 2.4.0 RC4

--confluentinc--2.4--9a7f26d/report.html On Mon, Dec 9, 2019 at 11:02 PM Manikumar wrote: > Hello Kafka users, developers and client-developers, > > This is the fifth candidate for release of Apache Kafka 2.4.0. > > This release includes many new features, including: > - Allow consu

Re: [VOTE] 2.4.0 RC4

Hi Israel, Thanks for testing the release candidate. We can use https://issues.apache.org/jira/browse/KAFKA-9283 for fixing the flaky test. Thanks, On Thu, Dec 12, 2019 at 8:18 PM Israel Ekpo wrote: > > @Manikumar , I have rerun the latest release > candidate (RC4) and all the t

Re: [kafka-clients] [VOTE] 2.4.0 RC3

Hi all, We have merged the PR for KAFKA-9212. Thanks to Jason for the fixing the issue. Thanks to Yannick for reporting the issue and Michael Jaschob for providing extra details. I am canceling this vote and will create new RC shortly. Thanks, Manikumar On Sat, Dec 7, 2019 at 3:19 AM Israel

Re: [VOTE] 2.4.0 RC2

Hi all, We merged all the blockers and released a new release candidate RC3. Hopefully this will be a final RC for 2.4 release. Thanks, On Thu, Dec 5, 2019 at 6:56 AM Israel Ekpo wrote: > Thanks @Randall for the update. I will run the tests on RC3 when it becomes > available. > > On Wed,

[VOTE] 2.4.0 RC3

Internal connect REST endpoints - API to delete consumer offsets and expose it via the AdminClient. Release notes for the 2.4.0 release: https://home.apache.org/~manikumar/kafka-2.4.0-rc3/RELEASE_NOTES.html *** Please download, test and vote by Monday, December 9th, 9 pm PT Kafka's KEYS file

Re: [VOTE] 2.4.0 RC2

t; https://github.com/elalonde/kafka> > $ ./kafka/bin/verifykafka-rc.sh 2.4.0 > https://home.apache.org/~manikumar/kafka-2.4.0-rc2 < > https://home.apache.org/~manikumar/kafka-2.4.0-rc2> > > - All sha1, sha256 and md5 sums look good > - pgp signatures look good > - unit tests

[VOTE] 2.4.0 RC2

Internal connect REST endpoints - API to delete consumer offsets and expose it via the AdminClient. Release notes for the 2.4.0 release: https://home.apache.org/~manikumar/kafka-2.4.0-rc2/RELEASE_NOTES.html *** Please download, test and vote by Thursday, December 5th, 9am PT Kafka's KEYS file

Re: [VOTE] 2.4.0 RC1

eate task ':clients:spotbugsMain'. > >>> Could not create task of type 'SpotBugsTask'. > >> > Could not create an instance of type > >> com.github.spotbugs.internal.SpotBugsReportsImpl. > >> > > >> > >> > org.gradle.api.re

Re: [VOTE] 2.4.0 RC1

in/verify-kafka-rc.sh 2.4.0 > https://home.apache.org/~manikumar/kafka-2.4.0-rc1 > <https://home.apache.org/~manikumar/kafka-2.4.0-rc0> > > - All PGP signatures are good > - All md5, sha1sums and sha512sums pass > - Had a few intermittent failures in tests that passed upon

Re: Preliminary blog post about the Apache Kafka 2.4.0 release

Sure, I will mention the details about ZK upgrade to 3.5 Thanks. On Tue, Nov 26, 2019 at 1:57 PM Sönke Liebau wrote: > Hi Manikumar, > > looks great, thanks for the effort! > > Like Ismael, I think it would make sense to mention the ZK upgrade to 3.5 I > know a few peopl

[VOTE] 2.4.0 RC1

Internal connect REST endpoints - API to delete consumer offsets and expose it via the AdminClient. Release notes for the 2.4.0 release: https://home.apache.org/~manikumar/kafka-2.4.0-rc1/RELEASE_NOTES.html ** Please download, test and vote by Tuesday, November 26, 9am PT ** Kafka's KEYS file

Re: [VOTE] 2.4.0 RC0

Hi All, Thanks you for testing RC0. The following blocker issues were fixed after RC0. https://issues.apache.org/jira/browse/KAFKA-9196 https://github.com/apache/kafka/pull/7691 I am canceling RC0 VOTE and will create new RC soon. Thanks, Manikumar On Thu, Nov 14, 2019 at 11:51 PM Manikumar

Preliminary blog post about the Apache Kafka 2.4.0 release

, Manikumar

[VOTE] 2.4.0 RC0

for replica reassignment - Sticky partitioner - Return topic metadata and configs in CreateTopics response - Securing Internal connect REST endpoints - API to delete consumer offsets and expose it via the AdminClient. Release notes for the 2.4.0 release: https://home.apache.org/~manikumar/kafka

Re: [ANNOUNCE] New committer: Mickael Maison

Congrats Mickeal! On Fri, Nov 8, 2019 at 9:05 AM Dong Lin wrote: > Congratulations Mickael! > > On Thu, Nov 7, 2019 at 1:38 PM Jun Rao wrote: > > > Hi, Everyone, > > > > The PMC of Apache Kafka is pleased to announce a new Kafka committer > > Mickael > > Maison. > > > > Mickael has been

Re: [DISCUSS] Apache Kafka 2.4.0 release

Date: Oct 30, 2019 (tentative) Please plan accordingly for the features you want push into Apache Kafka 2.4.0 release. Regards, Manikumar On Mon, Aug 12, 2019 at 9:08 PM Ismael Juma wrote: > Thanks for volunteering Manikumar. +1 > > Ismael > > On Mon, Aug 12, 2019 at 7:54 AM

[DISCUSS] Apache Kafka 2.4.0 release

Hi all, I would like to volunteer to be the release manager for our next time-based feature release (v2.4.0). If that sounds good, I'll post the release plan over the next few days. Thanks, Manikumar

Re: Kafka - JMX Metrics for Fetch and Produce

Hi, RequestsPerSec metric was changed to include the API version in Kafka 2.0 release. You may need to update jmx query. http://kafka.apache.org/documentation/#upgrade_200_notable https://cwiki.apache.org/confluence/display/KAFKA/KIP-272%3A+Add+API+version+tag+to+broker%27s+RequestsPerSec+metric

Re: [ANNOUNCE] New Kafka PMC member: Sriharsh Chintalapan

Congrats Harsha!. On Fri, Apr 19, 2019 at 7:43 AM Dong Lin wrote: > Congratulations Sriharsh! > > On Thu, Apr 18, 2019 at 11:46 AM Jun Rao wrote: > > > Hi, Everyone, > > > > Sriharsh Chintalapan has been active in the Kafka community since he > became > > a Kafka committer in 2015. I am glad

Re: [ANNOUNCE] New Kafka PMC member: Matthias J. Sax

Congrats Matthias!. well deserved. On Fri, Apr 19, 2019 at 7:44 AM Dong Lin wrote: > Congratulations Matthias! > > Very well deserved! > > On Thu, Apr 18, 2019 at 2:35 PM Guozhang Wang wrote: > > > Hello Everyone, > > > > I'm glad to announce that Matthias J. Sax is now a member of Kafka PMC.

Re: Kafka console consumer group

Hi, auto offset commit is enabled by default in consumer. When auto offset commit is enabled, consumer commits the offset (based on auto.offset.reset config, default : latest), when it first joins the group. Thanks, Manikumar On Fri, Mar 29, 2019 at 8:33 PM Sharmadha Sainath wrote: > Hi

Re: [VOTE] 2.2.0 RC2

+1 (non-binding) - Verified the artifacts, build from src, ran tests - Verified the quickstart, ran producer/consumer performance tests. Thanks for running release!. Thanks, Manikumar On Wed, Mar 20, 2019 at 12:19 AM David Arthur wrote: > +1 > > Validated signatures, and ran thro

Re: Kafka 2.1.0 JMX mbean broken ?

Hi Raghav, As you know, KIP-372 added "version" tag to RequestsPerSec metric to monitor requests for each version. As mentioned in the KIP, to get total count per request (across all versions), we need to aggregate over all versions. You may need to automate this by fetching all metrics using

Re: [ANNOUNCE] New Committer: Vahid Hashemian

Congrats, Vahid! On Wed 16 Jan, 2019, 6:53 AM Ismael Juma Congratulations Vahid! > > On Tue, Jan 15, 2019, 2:45 PM Jason Gustafson > > Hi All, > > > > The PMC for Apache Kafka has invited Vahid Hashemian as a project > > committer and > > we are > > pleased to announce that he has accepted! > >

Re: Setup GC log rotate policy for kafka broker servers

Hi Parth, We need to pass JVM system properties to rotate gc logs. These props added in Kafka 0.11.release. For releases before 0.11, we need to add these JVM props. Broker restart is required for changes to take effect. https://github.com/apache/kafka/blob/trunk/bin/kafka-run-class.sh#L290

[ANNOUNCE] Apache Kafka 2.0.1

Roesler, Jon Lee, Kamal Chandraprakash, Kevin Lafferty, Konstantine Karantasis, lambdaliu, Lincong Li, Lucas Wang, Maciej Bryński, Manikumar Reddy, Matthias J. Sax, Max Zheng, Michal Dziemianko, Michał Borowiecki, radai-rosenblatt, Rajini Sivaram, Randall Hauch, Robert Yokota, Simon Clark, Stanislav

[RESULTS] [VOTE] Release Kafka version 2.0.1

This vote passes with 12 +1 votes (3 bindings) and no 0 or -1 votes. +1 votes PMC Members: * Jun Rao * Gwen Shapira * Rajini Sivaram Committers: * Harsha * Ewen Cheslack-Postava * Manikumar Community: * Magnus Edenhill * Eno Thereska * Vahid Hashemian * Jakub Scholz * Satish Duggana * Mickael

Re: [kafka-clients] Re: [VOTE] 2.0.1 RC0

Yes, I was about to send closing mail. On Thu, Nov 8, 2018 at 9:39 PM Ismael Juma wrote: > Manikumar, looks like you have the necessary votes. :) > > Ismael > > On Wed, Nov 7, 2018, 10:45 AM Rajini Sivaram wrote: > >> +1 (binding) >> >> Checked source

Re: [kafka-clients] Re: [VOTE] 2.0.1 RC0

+1 (binding) Built from src and ran tests, verified the artifacts, signatures, Ran core quick start. Thanks, Manikumar On Thu, Nov 8, 2018 at 12:15 AM Rajini Sivaram wrote: > +1 (binding) > > Checked source build and unit tests. Ran quickstart with source and binary. > > Thank y

Re: [VOTE] 2.0.1 RC0

We were waiting for the system test results. There were few failures: KAFKA-7579, KAFKA-7559, KAFKA-7561 they are not blockers for 2.0.1 release. We need more votes from PMC/committers :) Thanks Stanislav! for the system test results. Thanks, Manikumar On Thu, Nov 1, 2018 at 10:20 PM Eno

Re: How to turn-off Time-Based index files in Kafka?

We can't disable time-based indexing. What is the issue you are facing? On Thu, Oct 25, 2018 at 6:04 PM Ashwin Sinha wrote: > Hi Users, > > How to turn off time-based index files in Kafka (.timeindex files). Tried > searching for it in topic and broker configs documentation but could not > find

[VOTE] 2.0.1 RC0

/~manikumar/kafka-2.0.1-rc0/RELEASE_NOTES.html *** Please download, test and vote by Tuesday, October 30, end of day Kafka's KEYS file containing PGP keys we use to sign the release: http://kafka.apache.org/KEYS * Release artifacts to be voted upon (source and binary): http://home.apache.org

[DISCUSS] 2.0.1 bug fix release

( https://issues.apache.org/jira/browse/KAFKA-7464). Next week, Once the blocking issue gets addressed, I plan to create the first RC for 2.0.1 release. Thanks, Manikumar

Re: [ANNOUNCE] New Committer: Manikumar Reddy

Thanks everyone! It has been a great learning experience working Apache Kafka community. I am thankful to the committers/community members for all the support and help. Thanks Manikumar On Fri, Oct 12, 2018 at 1:55 PM Edoardo Comar wrote: > Well done Maniku

Re: kafka-consumers-group.sh version 2.0.0 does not recognize --zookeeper option

Deprecated scala consumers/producers are removed from 2.0 release. Old consumer related options are removed from /kafka-consumer-groups.sh tool. http://kafka.apache.org/documentation/#upgrade_200_notable We missed updating some of the docs. Will be fixed in next release docs. On Thu, Oct 4, 2018

Re: Kafka ACL issue - Operation denied despite having full access to the topic

PrincipalType string "User" is case sensitive. Try creating acls for "User:storm-mytestcluster" principal. On Sat, Sep 29, 2018 at 9:04 PM Bala wrote: > Yes, I followed all the docs and thing is it's connecting as expected and > sending the right user to the server. The problem is that when I

Re: unable to start kafka when zookeeper cluster is in working but unhealthy state

lve IP for zookeeper-2 thus an > NullPointerException is thrown. > @Manikumar, ZOOKEEPER-2184 is to re-resolve IP for new instance of > zookeeper-2, however, zookeeper-2 stays down hence now IP to be resolved. > Or does it also work for this case? > > Thanks for your responses. >

Re: unable to start kafka when zookeeper cluster is in working but unhealthy state

You can try using Kafka 2.0 release. Original issue is handled in ZOOKEEPER-2184 and corresponding zookeeper version is used in Kafka 2.0. On Wed, Sep 26, 2018 at 3:58 PM Liam Clarke wrote: > Hi James, > > That's not an unresponsive node that's killing Kafka, that's a failure to > resolve the

Re: Kafka producer huge memory usage (leak?)

server logs for any errors. try increasing stack size by adding -Xss2048k. thanks, On Fri, Sep 21, 2018 at 2:44 PM Shantanu Deshmukh wrote: > Hi Manikumar, > I am using correct protocol. SSL and truststore is also correctly > configured and there is a single port which is for SSL.

Re: Kafka producer huge memory usage (leak?)

Hi, Instead trying the PR, make sure you are setting valid security protocol and connecting to valid broker port. also looks for any errors in producer logs. Thanks, On Fri, Sep 21, 2018 at 12:35 PM Shantanu Deshmukh wrote: > Hi Manikumar, > > I checked this issue. There is t

Re: How to enable Kafka Authentication as on-demand feature and later make mandatory feature?

One option is to enable multiple listeners (PLAINTEXT, SASL_PLAINTEXT) on brokers and slowly migrate the clients/services. On Wed, Sep 19, 2018 at 10:38 PM Ashwin Sinha wrote: > Hi, > > We have tried setting up Kafka Authentication by SASL/PLAIN >

Re: Kafka producer huge memory usage (leak?)

Similar issue reported here:KAFKA-7304, but on broker side. maybe you can create a JIRA and upload the heap dump for analysis. On Wed, Sep 19, 2018 at 11:59 AM Shantanu Deshmukh wrote: > Any thoughts on this matter? Someone, please help. > > On Tue, Sep 18, 2018 at 6:05 PM Shantanu Deshmukh >

Re: Determining broker max buffer size from producer.

You can use AdminClient describeConfigs API to fetch topic/broker configs. https://kafka.apache.org/20/javadoc/org/apache/kafka/clients/admin/KafkaAdminClient.html On Tue, Sep 4, 2018 at 1:47 PM claude.war...@wipro.com < claude.war...@wipro.com> wrote: > Thank you for the reference but I don't

Re: Offset for consumer group set to Null in __consumer_offsets, leads to data loss

What is the value set for offsets.retention.minutes? Default value is `1day` in Kafka 0.11 release. Based on your requirement, you may need to increase this value. On Mon, Aug 27, 2018 at 1:33 PM Biplob Biswas wrote: > Hi, > > Can anyone provide any insights to the issue we faced? > > Thanks &

Re: Cannot create topics when zookeeper.set.acl=true

*zk does NOT support PlainLoginModule.* On Thu, Aug 23, 2018 at 5:54 PM Manikumar wrote: > No, zk does support PlainLoginModule. while using kafka-acls.sh script > with kerberized zk, > we need to pass required kerberos credentials. > > AdminClient API is an api to perform

Re: Cannot create topics when zookeeper.set.acl=true

y.plain.PlainLoginModule required >username="user" >password="user-secret"; > }; > > Does that make a difference? > > What do you mean with AdminClient API? > > Regards Hans > > Op do 23 aug. 2018 om 13:34 schreef Manikumar : > > > W

Re: Cannot create topics when zookeeper.set.acl=true

; I searched for an option with which I can provide credentials but I did not > find them. > Is there another way to reach the same goal? > Regards Hans > > Op do 23 aug. 2018 om 13:00 schreef Manikumar : > > > "kafka-acls.sh" script communicates directly with zoo

Re: Cannot create topics when zookeeper.set.acl=true

"kafka-acls.sh" script communicates directly with zookeeper. We should run kafka-acls.sh as kafka user (super user) to get write permission on zk. We should pass required jaas conf to the script. On Thu, Aug 23, 2018 at 3:02 PM HG wrote: > Hi, > > I have an environment with SSL, SASL and ACL's

Re: Kafka SASL (Kerberos) - UNKNOWN_TOPIC_OR_PARTITION

you using any custom authorizer? On Tue, Aug 21, 2018 at 12:21 AM Matt L wrote: > Thanks for the pointer Manikumar! > > It looks like it was my interbroker communication, this was set to SSL so > inter broker users were coming in as ANONYMOUS. Once i changed this to > SASL_SSL I w

Re: Kafka SASL (Kerberos) - UNKNOWN_TOPIC_OR_PARTITION

is auto topic creation enabled on server? Any deny logs in kafka-authorizer.log? What is the inter-broker protocol configured? If it is SSL, SSL user should have ClusterAction permission. On Mon, Aug 20, 2018 at 3:33 PM Matt L wrote: > Hello, > > Having trouble when publishing and consuming

Re: unable to reconfigure ssl truststore dynamically on broker

Yes, we have open JIRA for this: https://issues.apache.org/jira/browse/KAFKA-4493 On Wed, Aug 15, 2018 at 10:12 PM John Calcote wrote: > Thanks Manikumar - that's very helpful. I never thought to treat the > AdminClient like the broker or clients and look for a configuration options

Re: unable to reconfigure ssl truststore dynamically on broker

Hi, There is no specific doc. "--command-config" option takes admin client configs (. AdminClient configs are listed here: http://kafka.apache.org/documentation/#adminclientconfigs On Tue, Aug 14, 2018 at 10:35 PM John Calcote wrote: > Manikumar, > > Thank

Re: unable to reconfigure ssl truststore dynamically on broker

looks like port is wrong or security.protocol config is not passed to adminclient.properties file On Tue, Aug 14, 2018 at 7:23 PM John Calcote wrote: > Hi, > > I'm using the latest kafka broker - 2.0.0 with scala 2.12. I have a > complete SSL configuration working, but I add clients

Re: Recovering from Leader : -1 and No ISRs after upgrading to Kafka 1.0

Related issue: https://lists.apache.org/thread.html/55df8a7310669a568c3b1e575e9f034ecc7cb242132c17e6df533134@%3Cusers.kafka.apache.org%3E On Thu, Aug 2, 2018 at 4:29 PM Darpan Patel wrote: > Hi guys, > > Recently I upgraded from Kafka 0.10 version to kafka 1.0 on one of the > clusters. > After

Re: [ANNOUNCE] Apache Kafka 2.0.0 Released

y, Dong Lin, Edoardo Comar, emmanuel Harel, > > Eugene Sevastyanov, Ewen Cheslack-Postava, Fedor Bobin, fedosov-alexander, > > Filipe Agapito, Florian Hussonnois, fredfp, Gilles Degols, gitlw, Gitomain, > > Guangxian, Gunju Ko, Gunnar Morling, Guozhang Wang, hmcl, huxi, huxihx, >

Re: cleanup.policy - doesn't accept compact,delete

Impl.java:45) > > at > org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32) > > at > org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:89) > > at > org.apache.kafka.c

Re: [kafka-clients] Re: [VOTE] 1.0.2 RC1

+1 (non-binding) Verified tests, quick start, producer/consumer perf tests. On Sun, Jul 1, 2018 at 11:27 PM Ismael Juma wrote: > +1 (binding) > > Verified signature of source artifact, ran tests and verified quickstart > on source artifact with Java 7, verified quickstart on binary artifact

Re: [kafka-clients] Re: [VOTE] 0.11.0.3 RC0

+1 (non-binding) Verified quick start, ran producer/consumer perf tests, Thanks On Sun, Jul 1, 2018 at 8:29 PM Ismael Juma wrote: > +1 (binding) > > Verified signature of source artifact, ran tests and verified quickstart > on source artifact with Java 7, verified quickstart on binary

Re: Partitions reassignment is failing in Kafka 1.1.0

It will be taken as "any" directory for each replica, which means replica will placed on any one of the configured directory on that broker. Since it is "log_dirs" optional, you can remove from the json. On Sat, Jun 30, 2018 at 12:02 PM Debraj Manna wrote: > It is problem on my side. The code

Re: [kafka-clients] [VOTE] 2.0.0 RC1

looks like maven artifacts are not updated in the staging repo. They are still at old timestamp. https://repository.apache.org/content/groups/staging/org/apache/kafka/kafka_2.11/2.0.0/ On Sat, Jun 30, 2018 at 12:06 AM Rajini Sivaram wrote: > Hello Kafka users, developers and client-developers,

Re: Unclean Leader Election Expected Behavior

In your case, you need to restart B2 with unclean.leader.election=true. This will enable B2 to become leader with 90 messages. On Thu, Jun 28, 2018 at 11:51 PM Jordan Pilat wrote: > If I restart the broker, won't that cause all 100 messages to be lost? > > On 2018/06/28 02:59:15,

Re: [VOTE] 1.1.1 RC1

Yes, looks like maven artifacts are missing on staging repo https://repository.apache.org/content/groups/staging/org/apache/kafka/kafka_2.11/ On Thu, Jun 28, 2018 at 4:18 PM Odin wrote: > There are no 1.1.1-rc1 artifacts in the staging repo listed. Where can > they be found? > > Sincerely >

Re: Unclean Leader Election Expected Behavior

You can enable unclean.leader.election temporarily for specific topic by using kafka-topics.sh command. This requires broker restart to take effect. http://kafka.apache.org/documentation/#topicconfigs On Thu, Jun 28, 2018 at 2:27 AM Jordan Pilat wrote: > Heya, > > I had a question about what

Re: [kafka-clients] Re: [VOTE] 1.1.1 RC1

+1 (non-binding) Ran tests, Verified quick start, producer/consumer perf tests On Sat, Jun 23, 2018 at 8:11 AM Dong Lin wrote: > Thank you for testing and voting the release! > > I noticed that the date for 1.1.1-rc1 is wrong. Please kindly test and > vote by Tuesday, June 26, 12 pm PT. > >

Re: ISR shrink rate & IsrExpandsPerSec issue

These metrics are meter type metrics, which tracks count, mean rate, 1-, 5-, and 15-minute moving averages. You maybe observing count measure, which gives number of events which have been marked. You can try monitoring 1/5/15 miniture averages. On Wed, Jun 20, 2018 at 12:33 AM Arunkumar wrote:

Re: How set log compaction policies at cluster level

Those configs are topic-level config names. To configure in server.properties, we need to use server config property names (log.cleanup.policy, log.cleaner.delete.retention.ms etc..). check the "SERVER DEFAULT PROPERTY" column in the table given in the below link

Re: Problem consuming from broker 1.1.0

Can you post consumer debug logs? You can enable console consumer debug logs here: kafka/config/tools-log4j.properties On Wed, Jun 13, 2018 at 9:55 AM Craig Ching wrote: > Hi! > > We’re having a problem with a new kafka cluster at 1.1.0. The problem is, > in general, that consumers can’t

Re: cleanup.policy - doesn't accept compact,delete

As described in usage description, to group the values which contain commas, we need to use square brackets. ex: --add-config cleanup.policy=[compact,delete] On Thu, Jun 7, 2018 at 8:49 AM, Jayaraman, AshokKumar (CCI-Atlanta-CON) < ashokkumar.jayara...@cox.com> wrote: > Hi, > > We are on Kafka

Re: Custom AuthenticateCallbackHandler

This feature will be part upcoming Kafka 2.0.0 release. Doc PR is here : https://github.com/apache/kafka/pull/4890 configs here: https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/common/config/SaslConfigs.java#L57 On Fri, Jun 1, 2018 at 10:51 PM, Roy van der Valk

Re: Logging authentication/authorization failures

Currently authentication logs are not available. In recent Kafka versions, authorization failures will be logged in logs/kafka-authorizer.log On Thu, May 31, 2018 at 5:34 PM, Gérald Quintana wrote: > Hello, > > I am using SASL Plaintext authentication and ACLs. > I'd like to be able to detect

Re: Long start time for consumer

Pls check "group.initial.rebalance.delay.ms" broker config property. This will be the delay for the initial consumer rebalance. from docs "The rebalance will be further delayed by the value of group.initial.rebalance.delay.ms as new members join the group, up to a maximum of

Re: Frequent consumer rebalance, auto commit failures

heartbeat.interval.ms should be lower than session.timeout.ms. Check here: http://kafka.apache.org/0101/documentation.html#newconsumerconfigs On Thu, May 24, 2018 at 2:39 PM, Shantanu Deshmukh wrote: > Someone please help me. I am suffering due to this issue since a

Re: Transfer data between kerberized kafka clusters (different principals)

t;( > producerConfig) > > //Consumer with specific config: principal 'xyz' > Properties consumerConfig = new Properties(); > consumerConfig.put("sasl.jaas.config" , ) > KafkaConsumerr<String, String> producer = new KafkaConsumer<>( > consumerConfig) > &g

  1   2   3   4   >