Re: High CPU usage for idle kafka server

2015-06-08 Thread Jiangjie Qin
to:users@kafka.apache.org>> Date: Monday, June 8, 2015 at 1:53 PM To: "users@kafka.apache.org<mailto:users@kafka.apache.org>" mailto:users@kafka.apache.org>> Subject: Re: High CPU usage for idle kafka server Hi Jiangjie, Resending profiling screenshot for High CPU usage on

Re: High CPU usage for idle kafka server

2015-06-08 Thread pundlik.anuja
> pundlik.an...@gmail.com>> >> > Reply-To: "users@kafka.apache.org<mailto:users@kafka.apache.org>" < >> > users@kafka.apache.org<mailto:users@kafka.apache.org>> >> > Date: Monday, June 8, 2015 at 12:20 PM >> > To: "users@kafka.a

Re: High CPU usage for idle kafka server

2015-06-08 Thread pundlik.anuja
rs@kafka.apache.org<mailto:users@kafka.apache.org>> > > Date: Monday, June 8, 2015 at 12:20 PM > > To: "users@kafka.apache.org<mailto:users@kafka.apache.org>" < > > users@kafka.apache.org<mailto:users@kafka.apache.org>> > > Subject: Re:

Re: High CPU usage for idle kafka server

2015-06-08 Thread Jay Kreps
pache.org<mailto:users@kafka.apache.org>" < > users@kafka.apache.org<mailto:users@kafka.apache.org>> > Date: Monday, June 8, 2015 at 12:20 PM > To: "users@kafka.apache.org<mailto:users@kafka.apache.org>" < > users@kafka.apache.org<mailto:users@kafka.

Re: High CPU usage for idle kafka server

2015-06-08 Thread Jiangjie Qin
sers@kafka.apache.org<mailto:users@kafka.apache.org>" mailto:users@kafka.apache.org>> Date: Monday, June 8, 2015 at 12:20 PM To: "users@kafka.apache.org<mailto:users@kafka.apache.org>" mailto:users@kafka.apache.org>> Subject: Re: High CPU usage for idle kafka serve

Re: High CPU usage for idle kafka server

2015-06-08 Thread pundlik.anuja
Seems to be LogCleaner and FetchRequest threads. Attached is the profiling screenshot On Fri, Jun 5, 2015 at 3:06 PM, Jiangjie Qin wrote: > Has this to do with KAFKA-1461? > Can you see which thread is taking a lot of cpu? Some jconsole plugin can > get that information. > > Jiangjie (Becket) Qi

Re: High CPU usage for idle kafka server

2015-06-05 Thread Jiangjie Qin
Has this to do with KAFKA-1461? Can you see which thread is taking a lot of cpu? Some jconsole plugin can get that information. Jiangjie (Becket) Qin On 6/5/15, 2:57 PM, "pundlik.anuja" wrote: >Hi Jay, > >Good to hear from you. I met you at the kafka meetup at linkedin. > >- No, I am running ka

Re: High CPU usage for idle kafka server

2015-06-05 Thread pundlik.anuja
Hi Jay, Good to hear from you. I met you at the kafka meetup at linkedin. - No, I am running kafka_2.11-0.8.2.1 Are there any logs/ any info that I can provide that will help you understand what could be the issue? Thanks, Anuja On Fri, Jun 5, 2015 at 2:36 PM, Jay Kreps wrote: > This sounds

Re: High CPU usage for idle kafka server

2015-06-05 Thread Jay Kreps
This sounds a lot like a bug we fixed in 0.8.2.0, no chance you are running that pre-release version is there? -Jay On Wed, Jun 3, 2015 at 9:43 PM, Anuja Pundlik (apundlik) wrote: > Hi, > > I am using Kafka 0.8.2.1. > We have 1 zookeeper, 3 kafka brokers. > We have 9 topics, out of which 1 topi

Re: High CPU usage for idle kafka server

2015-06-05 Thread pundlik.anuja
There are no messages being sent or received. The system is idle. however, there seems to be some GC going on in the kafka broker and some socket reads and writes. It is using approx 500MB of memory. On Fri, Jun 5, 2015 at 1:44 PM, pundlik.anuja wrote: > Hi Otis, > How do I check garbage collect

Re: High CPU usage for idle kafka server

2015-06-05 Thread pundlik.anuja
Hi Otis, How do I check garbage collection on kafka broker? On Thu, Jun 4, 2015 at 1:24 PM, Otis Gospodnetic wrote: > How's their garbage collection doing? > > Otis > -- > Monitoring * Alerting * Anomaly Detection * Centralized Log Management > Solr & Elasticsearch Support * http://sematext.com

Re: High CPU usage for idle kafka server

2015-06-04 Thread Otis Gospodnetic
How's their garbage collection doing? Otis -- Monitoring * Alerting * Anomaly Detection * Centralized Log Management Solr & Elasticsearch Support * http://sematext.com/ On Thu, Jun 4, 2015 at 12:43 AM, Anuja Pundlik (apundlik) < apund...@cisco.com> wrote: > Hi, > > I am using Kafka 0.8.2.1. > W

High CPU usage for idle kafka server

2015-06-04 Thread Anuja Pundlik (apundlik)
Hi, I am using Kafka 0.8.2.1. We have 1 zookeeper, 3 kafka brokers. We have 9 topics, out of which 1 topic has 18 partitions, while another has 12 partitions. All other topics have 1 partition each. We see that idle kafka brokers (not carrying any message) are using more than 50% of CPU. See to