Re: Need Help: Kafka - ElasticSearch Use case

Take a look at kafka-connect. Here is the list of connectors. Thanks, Suman On Sat, Apr 11, 2020 at 10:11 PM dave.d...@yahoo.com.INVALID wrote: > Hi Team, > First of all thank you for developing and maintaining such system.I

Need help in setting up security in Kafka systems

Team, Can anyone help me share the configs to be set to achieve the below security in Kafka systems? - Broker-Broker should be PLAINTEXT(No Authentication and Authorization between brokers) - Zookeeper-Broker should be PLAINTEXT(No Authentication and Authorization between brokers and

Permissions to create KIP

Hi Apache Kafka Team, Could you please grant me permission to create a KIP for Apache Kafka, I wanted to propose a feature to ConsoleProducer in Kafka. Following are my details: - Full Name: Suman BN - Email: sumannew...@gmail.com - Wiki ID: sumannewton -- *Suman* *OlaCabs*

Re: [ANNOUNCE] New Committer: Vahid Hashemian

Congrats Vahid! On Wed, Jan 16, 2019 at 12:02 PM Dong Lin wrote: > Congratulations Vahid! > > On Tue, Jan 15, 2019 at 2:45 PM Jason Gustafson > wrote: > > > Hi All, > > > > The PMC for Apache Kafka has invited Vahid Hashemian as a project > > committer and > > we are > > pleased to announce

Re: Doubts regarding kafka-reassign-partitions.sh

Can you paste the producer and consumer configs as well? Thanks, Suman On Tue, Dec 18, 2018 at 11:49 AM Abhimanyu Nagrath < abhimanyunagr...@gmail.com> wrote: > so we're currently trying to use Kafka 2.1.0 and are pretty much in a > proof-of-concept-phase. We've only just started to look into

Re: [KAFKA-7382] Guarantee atleast one replica of partition to be alive during topic creation

Team, I have addressed review comments from Mani Kumar Reddy. Please review and let me know if any changes are required. -Suman On Thu, Nov 22, 2018 at 12:39 PM Suman B N wrote: > Committers, > Please check this. > > On Thu, Nov 15, 2018 at 1:01 PM Suman B N wrote: >

Re: Kafka replication error

t; of the requests handler on node 6? > > thanks. > > ty > > On Sat, Dec 8, 2018 at 9:02 PM Suman B N wrote: > > > Still hoping for some help here. > > > > On Fri, Dec 7, 2018 at 12:24 AM Suman B N wrote: > > > > > Guys, > > > Another ob

Re: Kafka replication error

Still hoping for some help here. On Fri, Dec 7, 2018 at 12:24 AM Suman B N wrote: > Guys, > Another observation is 90% of under-replicated partitions have the same > node as the follower. > > *Any help in here is very much appreciated. We have very less time to > stabilize k

Re: Kafka Cluster Setup

-On adding a new Kafka Broker(B) to the cluster. Will all data present on broker A will be distributed automatically? If not what I need to do distribute the data. Automatic distribution doesn't happen. Use kafka-reassign-partitions.sh command to distribute load manually. - Not let's

Re: Kafka replication error

Guys, Another observation is 90% of under-replicated partitions have the same node as the follower. *Any help in here is very much appreciated. We have very less time to stabilize kafka. Thanks a lot in advance.* -Suman On Thu, Dec 6, 2018 at 9:08 PM Suman B N wrote: > +users > > On

Re: Kafka replication error

+users On Thu, Dec 6, 2018 at 9:01 PM Suman B N wrote: > Team, > > We are observing ISR shrink and expand very frequently. In the logs of the > follower, below errors are observed: > > [2018-12-06 20:00:42,709] WARN [ReplicaFetcherThread-2-15], Error in fetch > kafka.server

Kafka replication error

Team, We are observing ISR shrink and expand very frequently. In the logs of the follower, below errors are observed: [2018-12-06 20:00:42,709] WARN [ReplicaFetcherThread-2-15], Error in fetch kafka.server.ReplicaFetcherThread$FetchRequest@a0f9ba9 (kafka.server.ReplicaFetcherThread)

Re: [KAFKA-7382] Guarantee atleast one replica of partition to be alive during topic creation

Committers, Please check this. On Thu, Nov 15, 2018 at 1:01 PM Suman B N wrote: > Any update on this? > > On Tue, Nov 13, 2018 at 1:17 PM Suman B N wrote: > >> Team, >> >> Review and merge - https://github.com/apache/kafka/pull/5822. >> One round of revi

Re: [KAFKA-7382] Guarantee atleast one replica of partition to be alive during topic creation

Any update on this? On Tue, Nov 13, 2018 at 1:17 PM Suman B N wrote: > Team, > > Review and merge - https://github.com/apache/kafka/pull/5822. > One round of review has been done by Mani Kumar Reddy in PR - > https://github.com/apache/kafka/pull/5665. > > -- > *Suman* &g

[KAFKA-7382] Guarantee atleast one replica of partition to be alive during topic creation

Team, Review and merge - https://github.com/apache/kafka/pull/5822. One round of review has been done by Mani Kumar Reddy in PR - https://github.com/apache/kafka/pull/5665. -- *Suman* *OlaCabs*

Re: Frequent under-replicated partitions

[-.\w]+),partition=([0-9]+) > > On Thu, Oct 25, 2018 at 7:18 PM Suman B N wrote: > > > Still looking for some response here. Pls assist. > > > > On Sat, Oct 20, 2018 at 12:43 AM Suman B N > wrote: > > > > > Rate of ingestion is not 150-200rps. I

Re: Frequent under-replicated partitions

Still looking for some response here. Pls assist. On Sat, Oct 20, 2018 at 12:43 AM Suman B N wrote: > Rate of ingestion is not 150-200rps. Its 150k-200k rps. > > On Fri, Oct 19, 2018 at 11:12 PM Suman B N wrote: > >> Team, >> We have been observing some partition

Re: classes with Test in their name

Those are for unit and integration tests only. Looks like you are very new to Java/Scala and Kafka. On Wed, Oct 24, 2018 at 2:38 PM wrote: > Hi, > > there, in the code, is lot of classes with "Test" in their name. For > example KafkaProducerTest, KafkaAdminClientTest or >

Re: [DISCUSS] KIP-386: Make Min metrics' default value consistent with Max metrics

Looks good to me. Maintains uniformity. +1 from me. But its more of a bug rather than improvement proposal. Let's see what contributors got to say. On Mon, Oct 22, 2018 at 7:23 PM Stanislav Kozlovski wrote: > Hey everybody, > > I've opened up a very short KIP to make the Max and Min metrics'

Re: Frequent under-replicated partitions

Rate of ingestion is not 150-200rps. Its 150k-200k rps. On Fri, Oct 19, 2018 at 11:12 PM Suman B N wrote: > Team, > We have been observing some partitions being under-replicated. Broker > version 0.10.2.1. Below actions were carried out but in vain: > >- Tried restarting node

Frequent under-replicated partitions

Team, We have been observing some partitions being under-replicated. Broker version 0.10.2.1. Below actions were carried out but in vain: - Tried restarting nodes. - Tried increasing replica fetcher threads. Recommend ideal replica fetcher threads for a 20 node cluster with 150-200rps

Re: [DISCUSS] KIP-379: Multiple Consumer Group Management

This eases debugging metadata information of consumer groups and offsets in case of client hungs which we have been facing frequently. +1 from me. Well done Alex! -Suman On Fri, Oct 19, 2018 at 8:36 PM Vahid Hashemian wrote: > Thanks for proposing the KIP. Looks good to me overall. > > I agree

Re: Kafka 7382 - Guarantee atleast one replica of partition be alive during create topic.

Any update on this? This has been reviewed by one reviewer. Need more inputs on this one. Please take a look. Sorry to bother again and again. Just nagging as documented . On Monday, October 1, 2018, Suman B N wrote: > Any update on this? Should this be considered with KIP? > > On

Re: [ANNOUNCE] New Committer: Manikumar Reddy

Congratulations Manikumar! On Thu, Oct 11, 2018 at 11:09 PM Jason Gustafson wrote: > Hi all, > > The PMC for Apache Kafka has invited Manikumar Reddy as a committer and we > are > pleased to announce that he has accepted! > > Manikumar has contributed 134 commits including significant work to

Re run build of a pull request

Team, Every once in a while, PR builds fails due to below test failure with Null Pointer Exception. - org.apache.kafka.streams.scala.kstream.KStreamTest.join 2 KStreams should join correctly records But locally it is successfully run. Sometimes, when re-run, it just passes. Now I have two

[jira] [Created] (KAFKA-7465) kafka-topics.sh command not honouring --disable-rack-aware property when adding partitions.

Suman B N created KAFKA-7465: Summary: kafka-topics.sh command not honouring --disable-rack-aware property when adding partitions. Key: KAFKA-7465 URL: https://issues.apache.org/jira/browse/KAFKA-7465

Re: Kafka 7382 - Guarantee atleast one replica of partition be alive during create topic.

Any update on this? Should this be considered with KIP? On Wednesday, September 26, 2018, Suman B N wrote: > Team, > > Issue KAFKA-7382 <https://issues.apache.org/jira/browse/KAFKA-7382> has > been fixed. Please review and merge. We need this fix in one of our > producti

Kafka 7382 - Guarantee atleast one replica of partition be alive during create topic.

Team, Issue KAFKA-7382 has been fixed. Please review and merge. We need this fix in one of our production clusters. Pull request is here . -- *Suman* *OlaCabs*

Re: [ANNOUNCE] New committer: Colin McCabe

Congrats Colin! On Tue, Sep 25, 2018 at 7:46 PM Kamal Chandraprakash < kamal.chandraprak...@gmail.com> wrote: > Congratulations Colin! > > On Tue, Sep 25, 2018 at 7:42 PM Dhruvil Shah wrote: > > > Congratulations Colin! > > > > On Tue, Sep 25, 2018 at 6:47 AM Stephane Maarek < > >

Re: Review and merge - KAFKA-6764

This has been approved by one reviewer. Can anyone review and merge this - https://github.com/apache/kafka/pull/5637? On Fri, Sep 14, 2018 at 5:37 PM Suman B N wrote: > Can you pls check this? > > On Wed, Sep 12, 2018 at 3:12 PM Suman B N wrote: > >> Team, >> >&g

Re: Review and merge - KAFKA-6764

Can you pls check this? On Wed, Sep 12, 2018 at 3:12 PM Suman B N wrote: > Team, > > Review and merge below pull request. > Merge Request: https://github.com/apache/kafka/pull/5637 > Jira: https://issues.apache.org/jira/browse/KAFKA-6764 > > -- > *Suman* > *OlaCabs* > -- *Suman* *OlaCabs*

Review and merge - KAFKA-6764

Team, Review and merge below pull request. Merge Request: https://github.com/apache/kafka/pull/5637 Jira: https://issues.apache.org/jira/browse/KAFKA-6764 -- *Suman* *OlaCabs*