Re: [ANNOUNCE] New committer: Christo Lolov

Congratulations! On Wed, 27 Mar 2024 at 21:10, Matthias J. Sax wrote: > Congrats! > > On 3/26/24 9:39 PM, Christo Lolov wrote: > > Thank you everyone! > > > > It wouldn't have been possible without quite a lot of reviews and > extremely > > helpful inputs from you and the rest of the community!

Re: [VOTE] 3.4.1 RC3

Hi Luke, Thanks for running the release. I've checked signatures, eyeballed the Javadocs, built from source and run the unit and integration tests. DynamicBrokerReconfigurationTest.testAdvertisedListenerUpdate fails for me repeatedly. I opened https://issues.apache.org/jira/browse/KAFKA-15049

[ANNOUNCE] Apache Kafka 3.1.1

, Mike Lothian, Philip Nee, prince-mahajan, Randall Hauch, Stanislav Vodetskyi, sunshujie1990, Tom Bentley, Vincent Jiang, Xiaobing Fang, Xiaoyue Xue, Yang Yu We welcome your help and feedback. For more information on how to report problems, and to get involved, visit the project website at https

[VOTE] 3.1.1 RC1

Hello Kafka users, developers and client-developers, This is the first candidate for release of Apache Kafka 3.1.1. Apache Kafka 3.1.1 is a bugfix release and 30 issues have been fixed since 3.1.0. Release notes for the 3.1.1 release:

[VOTE] 3.1.1 RC0

Hello Kafka users, developers and client-developers, This is the first candidate for release of Apache Kafka 3.1.1. Apache Kafka 3.1.1 is a bugfix release and 29 issues have been fixed since 3.1.0. Release notes for the 3.1.1 release:

Re: [ANNOUNCE] New committer: Luke Chen

Congratulations Luke! On Thu, 10 Feb 2022 at 06:41, Josep Prat wrote: > Congrats Luke! > > ——— > Josep Prat > > Aiven Deutschland GmbH > > Immanuelkirchstraße 26, 10405 Berlin > > Amtsgericht Charlottenburg, HRB 209739 B > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > m:

Re: [kafka-clients] [VOTE] 2.7.2 RC0

Hi Mickael, As with 2.6.3 RC0, I have: * Verified checksums and signatures * Built jars and docs from the source jar * Run the unit and integration tests +1 non-binding Kind regards, Tom On Sun, Oct 24, 2021 at 3:05 PM Israel Ekpo wrote: > Mickael, > > Do we need to do another RC? Were

Re: warning messages from kafka admin client

This was fixed in 2.8, see https://issues.apache.org/jira/browse/KAFKA-10090 On Wed, Jul 14, 2021 at 12:19 PM Dhirendra Singh wrote: > Hi All, > whenever i run any kafka CLI tool i am getting following warning messages > > kafka-topics.sh --bootstrap-server kafka-dev:9092 --command-config >

Re: How to avoid storing password in clear text in server.properties file

It was done in KIP-421 , though I guess it's not really documented very well (feel free to open a JIRA for that). By default Apache Kafka has a FileConfigProvider and DirectoryConfigProvider which both use the local file

Re: [ANNOUNCE] New Kafka PMC Member: Konstantine Karantasis

Congratulations Konstantine! On Mon, Jun 21, 2021 at 5:33 PM David Jacot wrote: > Congrats, Konstantine. Well deserved! > > Best, > David > > On Mon, Jun 21, 2021 at 6:14 PM Ramesh Krishnan > wrote: > > > Congrats Konstantine > > > > On Mon, 21 Jun 2021 at 8:58 PM, Mickael Maison > > wrote: >

Re: Mirror Maker2 Event filter capability from topic before replication

seems we will have to > build something of our own together to work with MM2. > So just wanted to confirm that it is not an inbuilt feature.. right? Like > confluent replicator provide this. > Also are we planning to add this in near future? > > > On Fri, 7 May 2021 13:33 Tom

Re: Mirror Maker2 Event filter capability from topic before replication

Hi Anup, This should be possible using the Filter SMT in Kafka Connect together with a custom Predicate which you'd have to write yourself and provide on the plugin path. See http://kafka.apache.org/documentation.html#connect_predicates and

Re: Request to be added the contributor list

Hi Wenhao, I added you as a contributor in Jira, so you should now be able to assign issues to yourself etc. We normally discuss development of Kafka on the d...@kafka.apache.org mailing list. Thanks for your interest. Tom On Wed, Apr 28, 2021 at 3:39 PM Wenhao Ji wrote: > Hi everyone, > > I

Re: [ANNOUNCE] New Kafka PMC Member: Randall Hauch

Congratulations Randall! On Sat, Apr 17, 2021 at 7:36 AM feyman2009 wrote: > Congratulations Randall! > > Haoran > -- > 发件人：Luke Chen > 发送时间：2021年4月17日(星期六) 12:05 > 收件人：Kafka Users > 抄 送：dev > 主 题：Re: [ANNOUNCE] New Kafka PMC

Re: [ANNOUNCE] New Kafka PMC Member: Bill Bejeck

Congratulations Bill! On Thu, Apr 8, 2021 at 2:36 AM Luke Chen wrote: > Congratulations Bill! > > Luke > > On Thu, Apr 8, 2021 at 9:17 AM Matthias J. Sax wrote: > > > Hi, > > > > It's my pleasure to announce that Bill Bejeck in now a member of the > > Kafka PMC. > > > > Bill has been a Kafka

Re: [ANNOUNCE] New Committer: Bruno Cadonna

Congratulations Bruno! On Thu, Apr 8, 2021 at 6:21 AM Konstantine Karantasis wrote: > Congratulations Bruno! > > Konstantine > > On Wed, Apr 7, 2021 at 8:08 PM Sophie Blee-Goldman > wrote: > > > Congrats! > > > > On Wed, Apr 7, 2021 at 6:32 PM Luke Chen wrote: > > > > > Congrats Bruno!! > > >

Re: Filter Kafka Messages

e | ro...@confluent.io | @rmoff > > > On Wed, 10 Mar 2021 at 09:55, raman gugnani > wrote: > > > HI Tom, > > > > For which connector class connector do I need to submit ? > > Can you share some samples for the same? > > > > On Wed, 10

Re: Filter Kafka Messages

Kafka Connect has the Filter SMT which can be used with a predicate to filter out certain messages, see https://kafka.apache.org/documentation/#connect_predicates Kinds regards, Tom On Wed, Mar 10, 2021 at 8:46 AM raman gugnani wrote: > HI Team, > > I want to filter out some of the kafka

Re: Re: hi,guys kafka 2.7.0 builld fail, any suggestion?

It should work with 6.6[1], or use the included gradlew script to bootstrap and use the right gradle version. Kind regards, Tom [1]: https://github.com/apache/kafka/blob/2.7.0/gradle/wrapper/gradle-wrapper.properties On Wed, Jan 27, 2021 at 9:52 AM felixzh wrote: > Hi, > Gradle 5.6.4 > > > >

Re: --override option for bin/connect-distributed.sh

Hi Aki, Since this is a change to a public API of the project it would need to be done through the KIP process [1]. Since writing the KIP in this case isn't much work, I suggest you write it up as a proposal and start a KIP discussion thread on the dev@ mailing list, then interested people can

Re: Kafka in-sync replicas

The leader should send the produce response (the acknowledgement) to the producer once the leader has persisted the batch to its log *and* the leader knows that one of the followers has persisted it to its log. On Mon, Dec 21, 2020 at 9:52 AM Miroslav Tsvetanov wrote: > Hello everyone, > > I`m

Re: [VOTE] 2.7.0 RC3

Hi Bill, One very minor problem I just spotted is https://kafka.apache.org/27/documentation.html#brokerconfigs_listeners, because the tag is not properly closed the HTML doesn't render property (in Chromium and Firefox at least) and all the rest of the configs are shown in a monospaced font. I

Re: LeaderElectionNotNeededException

Hi Liam, I think you have a misunderstanding of what preferred leader election does. All it does is ensure that the "preferred leader" (the first in the list of replicas for a partition) becomes the actual leader if it can (that is, if it's in the ISR) and if the current leader is already the

Re: [ANNOUNCE] New committer: David Jacot

Congratulations David! On Fri, Oct 16, 2020 at 5:10 PM Bill Bejeck wrote: > Congrats David! Well deserved. > > -Bill > > On Fri, Oct 16, 2020 at 12:01 PM Gwen Shapira wrote: > > > The PMC for Apache Kafka has invited David Jacot as a committer, and > > we are excited to say that he accepted! >

Re: Intended behavior when a broker loses its log volume

Hi Peter, When an unexpected IOException happens when accessing a file in a log directory, the broker will take that log directory offline. That means follower fetchers for partitions on that log dir will be stopped, the broker will stop serving requests from those logs and a notification is sent

Re: Getting the suggested partition assignment via the Admin client

Hi Stig, AdminUtils is not a public API, so you're right to not want to depend on it. Unfortunately the Admin client API doesn't offer functionality equivalent to `kafka-reassign-partitions.sh --generate`. That is, while you can get the broker to generate the assignments for you when you create a

Re: Documentation suggestion / issue and question about ticket openning

Hi Ahmed, I have an open PR (https://github.com/apache/kafka/pull/8878) which, although far from being a perfect solution, would be of _some_ help. Some configs are common between broker, producer and consumer and right now it's not possible to link directly to, say, the consumer one because the

Re: New Website Layout

ks. > > On Thu, 6 Aug 2020 at 18:59, Ben Weintraub wrote: > > > Plus one to Tom's request - the ability to easily generate links to > > specific config options is extremely valuable. > > > > On Thu, Aug 6, 2020 at 10:09 AM Tom Bentley wrote: > > > >

Re: New Website Layout

Hi Ben, The documentation for the configs (broker, producer etc) used to function as links as well as anchors, which made the url fragments more discoverable, because you could click on the link and then copy+paste the browser URL: batch.size What seems to have happened with the new

Re: Upgrade connectors logging from log4j to log4j2

out restarting the > > worker process. > > > > Can you please share any suggestion on this with log4j. > > > > Regards and Thanks > > Deepak Raghav > > > > > > > > On Mon, Jul 27, 2020 at 7:09 PM Tom Bentley wrote: > > > >> Hi

Re: Upgrade connectors logging from log4j to log4j2

Hi Deepak, https://issues.apache.org/jira/browse/KAFKA-9366 already proposes to replace log4j with log4j2 throughout Kafka. Kind regards, Tom On Mon, Jul 27, 2020 at 2:04 PM Deepak Raghav wrote: > Hi Team > > Request you to please have a look. > > Regards and Thanks > Deepak Raghav > > > >

Re: [ANNOUNCE] New committer: Xi Hu

Congratulations Xi! On Wed, Jun 24, 2020 at 5:34 PM Guozhang Wang wrote: > The PMC for Apache Kafka has invited Xi Hu as a committer and we are > pleased to announce that he has accepted! > > Xi Hu has been actively contributing to Kafka since 2016, and is well > recognized especially for his

Re: Using encrypted distributed worker file

Hi Ashish, KIP-297[1] added support for "config providers", which allow a config file to have an indirect reference to secrets stored elsewhere. While it doesn't sound like the provided FileConfigProvider would be suitable for your needs, you could provide your own ConfigProvider to implement a

Re: Custom Connector

data from one Rest Endpoint and post that data from > endpoint to topic. > > > Is it possible or any other connector available for that? > > On Tue, May 12, 2020, 13:24 Tom Bentley wrote: > > > Hi Vishnu, > > > > These are probably a good place to start: > >

Re: Custom Connector

Hi Vishnu, These are probably a good place to start: 1. https://docs.confluent.io/current/connect/devguide.html 2. https://www.confluent.io/blog/create-dynamic-kafka-connect-source-connectors/ Cheers, Tom On Tue, May 12, 2020 at 7:34 AM vishnu murali wrote: > Hi Guys, > > i am trying to

Re: [VOTE] 2.5.0 RC1

Hi David, I verified signatures, built the tagged branch and ran unit and integration tests. I found some flaky tests, as follows: https://issues.apache.org/jira/browse/KAFKA-9691 (new) https://issues.apache.org/jira/browse/KAFKA-9692 (new) https://issues.apache.org/jira/browse/KAFKA-9283

Re: Deleting single record from kafka topic

Hi Khoi, Short answer: No. Broker logs are append only, so your options are either to delete all records before a certain offset (for a non-compacted topic) or to have a compacted topic and send a message with a null value. For some applications (such as GDPR compliance, where there's a legal

Re: ingesting web server logs, or log4j log files from a boss server

Hi George, Since you mentioned CDC specifically you might want to check out Debezium ( https://debezium.io/) which operates as a connector of the sort Robin referred to and does CDC for MySQL and others. Cheers, Tom On Wed, Jan 15, 2020 at 10:18 AM Robin Moffatt wrote: > The integration part

Re: Public IPs for brokers

Hi Himanshu, Short answer: yes. The way a Kafka client works is to connect to one of the given bootstrap brokers and ask it about the rest of the cluster. The client then connects to those brokers as necessary. So in general a client will need to connect to all brokers in the cluster and

Re: callback function

Tavares, if you're asking about the consumer then I think you might have a misconception about how it works: The application calls poll() to fetch the latest records from the broker(s). The broker is not pushing records into some queue in the consumer. It might be worth reading

Re: Broker Interceptors

l > > On Fri, Dec 6, 2019 at 1:56 AM Tom Bentley wrote: > > > Hi, > > > > Couldn't this be done without exposing broker internals at the slightly > > higher level of AbstractRequest and AbstractResponse? Those classes are > > public. If the observer interface us

Re: Broker Interceptors

Hi, Couldn't this be done without exposing broker internals at the slightly higher level of AbstractRequest and AbstractResponse? Those classes are public. If the observer interface used Java default methods then adding a new request type would not break existing implementations. I'm thinking

Re: [ANNOUNCE] New committer: Mickael Maison

Congratulations Mickael! On Fri, Nov 8, 2019 at 6:41 AM Vahid Hashemian wrote: > Congrats Mickael, > > Well deserved! > > --Vahid > > On Thu, Nov 7, 2019 at 9:10 PM Maulin Vasavada > wrote: > > > Congratulations Mickael! > > > > On Thu, Nov 7, 2019 at 8:27 PM Manikumar > > wrote: > > > > >

Re: [ANNOUNCE] New Committer: Vahid Hashemian

Congratulations Vahid! On Wed, Jan 16, 2019 at 10:13 AM Edoardo Comar wrote: > Bravo Vahid!!! > > Edo > > > > From: Mickael Maison > To: Users > Cc: dev > Date: 16/01/2019 08:48 > Subject:Re: [ANNOUNCE] New Committer: Vahid Hashemian > > > > Congratulations Vahid! Well

Re: Kafka Mirror Maker Automation

Hi, The documentation on MirrorMaker is available on the website: https://kafka.apache.org/documentation/#basic_ops_mirror_maker Did you have a specific question or are you having a specific problem? If so, do post the details. Cheers, Tom On 5 October 2017 at 19:03, Kannappan, Saravanan

Re: Problems with SSL and consumer groups

Hi, What are the ACLs for the topics and groups concerned? btw my kafka-console-consumer.sh supports offset, were you using some other command? Cheers, Tom On 2 August 2017 at 15:31, Alisa Bohac wrote: > Hello, > > I recently integrated my Kafka broker with SSL

Re: Unit testing changes in core Scala sources

Hi, I'm no gradle expert, but it looks like gradle core:cleanTest core:test would do what you want. Have you looked at the example command lines in the README.md? Cheers, Tom On 27 July 2017 at 15:36, M. Manna wrote: > Hello, > > Please forgive me for asking too simply

Re: Kafka shutdown gracefully

FTR, the problem with the 4096 character limit is a known issue: https://issues.apache.org/jira/browse/KAFKA-4931 Cheers, Tom On 6 July 2017 at 13:55, Kamal C wrote: > Don't use `kill -9 PID`. Use `kill -s TERM PID` - sends a signal to the > process to end, and

Re: Consumers re-consuming messages again after re-balance?

Hi Dmitriy, FTR, https://issues.apache.org/jira/browse/KAFKA-3806 is the issue Damian is referring to, but it doesn't quite fit what you describe because you said your consumer was configured with enable.auto.commit = true, which should keep committing even if there are no messages being

Re: mirroring Kafka while preserving the order

I believe so. You need to be careful that the mirror maker producer doesn't reorder messages; in particular if retries > 0 then max.in.flight.requests.per.connection must be 1. If retries=0 then it doesn't matter what max.in.flight.requests.per.connection is. On 29 June 2017 at 05:52, Sunil

Re: Kafka Sometimes Fails to Start on Boot

Have you tried changing the configured JMX port? After all, it's possible the conflict is between kafka and some other software running on the same server. On 28 June 2017 at 21:06, Eric Coan wrote: > Hello, > > > Unfortunately Kafka does indeed startup and run for a

Re: Kafka Streams vs Spark Streaming : reduce by window

It sounds like you want a tumbling time window, rather than a sliding window https://kafka.apache.org/documentation/streams#streams_dsl_windowing On 15 June 2017 at 14:38, Paolo Patierno wrote: > Hi, > > > using the streams library I noticed a difference (or there is a lack

Re: Why authorization deny Read ACL doesn't work

Hi, Did you set authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer as described here at http://docs.confluent.io/current/kafka/authorization.html#further-configuration HTH, Tom On 11 June 2017 at 04:40, linbo liao wrote: > Hi, > > I try to set Kafka ACL for