[
https://issues.apache.org/jira/browse/KAFKA-4739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349634#comment-16349634
]
Steve commented on KAFKA-4739:
------------------------------
Hi,I use the Kafka Connect framework,it use the Kafka consumer. The same
problem happened to me .Some consumer threads use 100% cpu even I stop sending
new message.
{code:java}
USER %CPU PRI SCNT WCHAN USER SYSTEM TID TIME
hdfs 0.0 19 - ep_pol - - 32301 00:14:50
hdfs 0.0 19 - futex_ - - 499 00:00:05
hdfs 0.0 19 - futex_ - - 1060 00:00:24
hdfs 91.6 19 - - - - 24105 2-09:48:19
hdfs 0.0 19 - futex_ - - 7111 00:00:39
[hdfs@ha3 root]$ jstack 32111 | grep 5e29 -A 15
"WorkerSinkTask-t-connector-2" daemon prio=10 tid=0x00007fe9c4108800 nid=0x5e29
runnable [0x00007fe97931d000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
- locked <0x00000007482e6388> (a sun.nio.ch.Util$2)
- locked <0x00000007482e6378> (a java.util.Collections$UnmodifiableSet)
- locked <0x00000007482de340> (a sun.nio.ch.EPollSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
at org.apache.kafka.common.network.Selector.select(Selector.java:425)
at org.apache.kafka.common.network.Selector.poll(Selector.java:254)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:270)
at
org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.clientPoll(ConsumerNetworkClient.java:320){code}
I think it was caused by jdk epoll bug, .The bug is still happen in jdk
1.7,1.8.
Kafka client should handle the epoll bug as same as NETTY or other nio
frameworks.:)
[https://github.com/netty/netty/issues/327]
THRIFT-4251
> KafkaConsumer poll going into an infinite loop
> ----------------------------------------------
>
> Key: KAFKA-4739
> URL: https://issues.apache.org/jira/browse/KAFKA-4739
> Project: Kafka
> Issue Type: Bug
> Components: consumer
> Affects Versions: 0.9.0.1
> Reporter: Vipul Singh
> Priority: Major
>
> We are seeing an issue with our kafka consumer where it seems to go into an
> infinite loop while polling, trying to fetch data from kafka. We are seeing
> the heartbeat requests on the broker from the consumer, but nothing else from
> the kafka consumer.
> We enabled debug level logging on the consumer, and see these logs:
> https://gist.github.com/neoeahit/757bff7acdea62656f065f4dcb8974b4
> And this just goes on. The way we have been able to replicate this issue, is
> by restarting the process in multiple successions.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
