Re: High write operations rate on disk

2020-04-07 Thread Suman B N
ing on each disk, but I don't think > that > > matters as most of the time the brokers run flawlessly (even during peak > > traffic hours). > > > > Thanks! > > > > On Mon, Apr 6, 2020 at 11:39 PM Suman B N wrote: > > > > > We too have a simila

Re: High write operations rate on disk

2020-04-06 Thread Suman B N
We too have a similar setup but we never observed any such spikes. Are you sure your disk IOPS is good enough? Check if that is throttling. After a broker restarts, there might be more traffic as well because of followers trying to catch up with the leader. -Suman On Tue, Apr 7, 2020 at 11:59 A

Re: Doubts in Kafka

2019-07-15 Thread Suman B N
Explore and set max.poll.records. On Mon, Jul 15, 2019 at 5:47 PM aruna ramachandran wrote: > Is there a way to get the Consumer to only read one message at a time and > commit the offset after processing a single message. > -- *Suman* *OlaCabs*

Need help in setting up security in Kafka systems

2019-04-04 Thread Suman B N
Team, Can anyone help me share the configs to be set to achieve the below security in Kafka systems? - Broker-Broker should be PLAINTEXT(No Authentication and Authorization between brokers) - Zookeeper-Broker should be PLAINTEXT(No Authentication and Authorization between brokers and

Re: [ANNOUNCE] New Committer: Vahid Hashemian

2019-01-15 Thread Suman B N
Congrats Vahid! On Wed, Jan 16, 2019 at 12:02 PM Dong Lin wrote: > Congratulations Vahid! > > On Tue, Jan 15, 2019 at 2:45 PM Jason Gustafson > wrote: > > > Hi All, > > > > The PMC for Apache Kafka has invited Vahid Hashemian as a project > > committer and > > we are > > pleased to announce tha

Re: Kafka replication error

2018-12-09 Thread Suman B N
e percentage > of the requests handler on node 6? > > thanks. > > ty > > On Sat, Dec 8, 2018 at 9:02 PM Suman B N wrote: > > > Still hoping for some help here. > > > > On Fri, Dec 7, 2018 at 12:24 AM Suman B N wrote: > > > > > Guys, > > >

Re: Kafka replication error

2018-12-08 Thread Suman B N
Still hoping for some help here. On Fri, Dec 7, 2018 at 12:24 AM Suman B N wrote: > Guys, > Another observation is 90% of under-replicated partitions have the same > node as the follower. > > *Any help in here is very much appreciated. We have very less time to > stabilize k

Re: Kafka replication error

2018-12-06 Thread Suman B N
Guys, Another observation is 90% of under-replicated partitions have the same node as the follower. *Any help in here is very much appreciated. We have very less time to stabilize kafka. Thanks a lot in advance.* -Suman On Thu, Dec 6, 2018 at 9:08 PM Suman B N wrote: > +users > > On

Re: Kafka replication error

2018-12-06 Thread Suman B N
+users On Thu, Dec 6, 2018 at 9:01 PM Suman B N wrote: > Team, > > We are observing ISR shrink and expand very frequently. In the logs of the > follower, below errors are observed: > > [2018-12-06 20:00:42,709] WARN [ReplicaFetcherThread-2-15], Error in fetch > kafka.server

Re: High CPU on kafka nodes

2018-11-29 Thread Suman B N
+1 We too face similar issue sometime. I am interested to know more about it. On Thursday, November 29, 2018, Alexander Filipchik wrote: > Hello! > > I inherited Kafka cluster which runs on AWS (I3.4xl instances). > Each node ingests around 8k messages per second. > Instance stats: > Network in

Re: [ANNOUNCE] New Committer: Manikumar Reddy

2018-10-11 Thread Suman B N
Congratulations Manikumar! On Thu, Oct 11, 2018 at 11:09 PM Jason Gustafson wrote: > Hi all, > > The PMC for Apache Kafka has invited Manikumar Reddy as a committer and we > are > pleased to announce that he has accepted! > > Manikumar has contributed 134 commits including significant work to ad

Re: How many consumers can subscribe to the same topic

2018-09-14 Thread Suman B N
- With same consumer group, you can't have consumers more than the number of partitions. Even if you do, extra consumers are just idle. Each partition can be read only by one consumer. - With different consumer groups, you can have as many consumer groups as you want for a single top