Apache Kafka Process showing high CPU (100 to 200+) usage in Linux when Idle

2018-08-04 Thread Abhijith Sreenivasan
Hello We are seeing high CPU usage for the Kafka process. I am using 0.11 version. Has 5 topics out of which 1 was created newly. We attempted to publish message this new topic which did not show up in the consumer, but no errors in the publisher end. Not sure why the message did not show up in co

Re: Apache Kafka Process showing high CPU (100 to 200+) usage in Linux when Idle

2018-08-04 Thread Ted Yu
bq. only one specific node is showing this issue Is controller running on this node ? updating the metrics is expensive. Cheers On Sat, Aug 4, 2018 at 3:00 PM Abhijith Sreenivasan < abhijithonl...@gmail.com> wrote: > Hello > > We are seeing high CPU usage for the Kafka process. I am using 0.11

Re: Apache Kafka Process showing high CPU (100 to 200+) usage in Linux when Idle

2018-08-04 Thread Manjunath N
After you deleted a topic was it a clean delete. Did you verify in zookeeper and kafka logs directory? if not you may need to do some clean up if there are inconsistency in kafka logs dir and zookeeper. did you try to move the replicas assignment to different machines for this topic and see if i

Re: Apache Kafka Process showing high CPU (100 to 200+) usage in Linux when Idle

2018-08-07 Thread Abhijith Sreenivasan
Controller is not running in this node. We found that one of the new producer added was publishing only to a specific partition which was in this node, that explains the high utilization only in this node. The publisher was running in an Async thread invoking the send() asynchronously but the messa