Query on kafka topic metadata

2015-06-05 Thread Pavan Chenduluru
Hi, I am new to kafka and I have a doubt. How to read specified topic statistics from kafka server? I want to read below parameters about existing topic from kafka. 1) How many activeMessages 2) How many activeSubscriptions 3) How many totalMessages 4) How many totalSubscriptions 5) How mnay

Re: Consumer lag lies - orphaned offsets?

2015-06-05 Thread Joel Koshy
On Fri, Jun 05, 2015 at 12:53:00AM -0400, Otis Gospodnetić wrote: Hi Joel, On Thu, Jun 4, 2015 at 8:52 PM, Joel Koshy jjkosh...@gmail.com wrote: Hi Otis, Yes this is a limitation in the old consumer. i.e., a number of per-topic/partition mbeans remain even on a rebalance. Those need

Re: High CPU usage for idle kafka server

2015-06-05 Thread Jay Kreps
This sounds a lot like a bug we fixed in 0.8.2.0, no chance you are running that pre-release version is there? -Jay On Wed, Jun 3, 2015 at 9:43 PM, Anuja Pundlik (apundlik) apund...@cisco.com wrote: Hi, I am using Kafka 0.8.2.1. We have 1 zookeeper, 3 kafka brokers. We have 9 topics, out

Re: High CPU usage for idle kafka server

2015-06-05 Thread Jiangjie Qin
Has this to do with KAFKA-1461? Can you see which thread is taking a lot of cpu? Some jconsole plugin can get that information. Jiangjie (Becket) Qin On 6/5/15, 2:57 PM, pundlik.anuja pundlik.an...@gmail.com wrote: Hi Jay, Good to hear from you. I met you at the kafka meetup at linkedin. -

Re: High CPU usage for idle kafka server

2015-06-05 Thread pundlik.anuja
Hi Jay, Good to hear from you. I met you at the kafka meetup at linkedin. - No, I am running kafka_2.11-0.8.2.1 Are there any logs/ any info that I can provide that will help you understand what could be the issue? Thanks, Anuja On Fri, Jun 5, 2015 at 2:36 PM, Jay Kreps jay.kr...@gmail.com

Re: High CPU usage for idle kafka server

2015-06-05 Thread pundlik.anuja
Hi Otis, How do I check garbage collection on kafka broker? On Thu, Jun 4, 2015 at 1:24 PM, Otis Gospodnetic otis.gospodne...@gmail.com wrote: How's their garbage collection doing? Otis -- Monitoring * Alerting * Anomaly Detection * Centralized Log Management Solr Elasticsearch Support

Re: High CPU usage for idle kafka server

2015-06-05 Thread pundlik.anuja
There are no messages being sent or received. The system is idle. however, there seems to be some GC going on in the kafka broker and some socket reads and writes. It is using approx 500MB of memory. On Fri, Jun 5, 2015 at 1:44 PM, pundlik.anuja pundlik.an...@gmail.com wrote: Hi Otis, How do I