We have had this issue in 0.8.x and at that time we did not investigate
it. Recently we upgraded to 0.9.0.1 and had similar issue which we
investigated and narrowed down to what's explained here
http://mail-archives.apache.org/mod_mbox/kafka-users/201604.mbox/%3C571F23ED.7050405%40gmail.com%3E.
If upgrading to 0.9.0.1 is an option for you, then you might want to do
that and read through that mail for potential ways to get past this issue.
-Jaikiran
On Friday 22 April 2016 03:04 PM, Kafka wrote:
Hi，we use kafka_2.10-0.8.2.1，and our vm machine config is：4 core，8G
our cluster is consist of three brokers，and our broker config is default 2
replica，our broker load often very high once in a while，
load is greater than 1.5 on average core。
we have about 70 topics on this cluster
when we use Top util, we can see 280% cpu unilization, then i use JSTACK, I
found there are 4 threads use cpu most, which show below:
"kafka-network-thread-9092-0" prio=10 tid=0x00007f46c8709000 nid=0x35dd
runnable [0x00007f46b73f2000]
java.lang.Thread.State: RUNNABLE
"kafka-network-thread-9092-1" prio=10 tid=0x00007f46c873c000 nid=0x35de
runnable [0x00007f46b75f4000]
/kafka-network-thread
"kafka-network-thread-9092-2" prio=10 tid=0x00007f46c8756000 nid=0x35df
runnable [0x00007f46b7cfb000]
java.lang.Thread.State: RUNNABLE
"kafka-network-thread-9092-3" prio=10 tid=0x00007f46c876f800 nid=0x35e0
runnable [0x00007f46b5adb000]
java.lang.Thread.State: RUNNABLE
I found one task:https://
<https://issues.apache.org/jira/browse/KAFKA-493>issues.apache.org/jira/browse/KAFKA-493
<http://issues.apache.org/jira/browse/KAFKA-493> concerns this, but I think this
does’t fit me, because we have other clusters that deployed in
the same config’s vm machine, and others are not occurred this.
the brokers config is same,but we add:
replica.fetch.wait.max.ms=100
num.replica.fetchers=2
I want to question does is the main reason, or have other reason that leads to
high cpu load?
