That's a fairly old release - you may want to try 0.8.1 (or wait for 0.8.1.1 which should be out in late April since 0.8.1 has some known bugs)?
That said, I tried locally but wasn't able to reproduce that level of cpu usage. The 0.8 brokers do have higher idle CPU usage and we have a ticket open to reduce that, but what you are seeing is probably different. Can you use hprof on your broker to see where the CPU usage is going? You will need to add this to your command-line: -agentlib:hprof=cpu=samples,depth=15 Thanks, Joel On Fri, Apr 11, 2014 at 06:06:12PM -0000, Saurabh Agarwal (BLOOMBERG/ 731 LEXIN) wrote: > here is the info...kafka version - 0.8.0-beta1, One broker, one topic has > 6000 partitions, others are only 10 or fewers. no replication... > > ----- Original Message ----- > From: jjkosh...@gmail.com > To: SAURABH AGARWAL (BLOOMBERG/ 731 LEXIN), users@kafka.apache.org > At: Apr 11 2014 13:46:04 > > Which version of Kafka are you using? How many brokers? Do you have > 750 partitions per topic? What is the replication factor? > > On Fri, Apr 11, 2014 at 04:06:57PM -0000, Saurabh Agarwal (BLOOMBERG/ 731 > LEXIN) wrote: > > while running kafka broker with 8 topics and 6000 partitions, %cpu usages > > hover around 300% and virtual memory around 60gb. There are no producers > > and consumers running. Why is the idle broker taking 300% cpu and 60gb vm? > > Any insight will greatly be appreciated. > > > > Thanks, > > Saurabh. > > > > ------------------------------------------------------------------------------- > > > > > -------------------------------------------------------------------------------
