Re: [Java New Producer] CPU Usage Spike to 100% when network connection is lost
Also do you know what version you are running we did fix several bugs similar to this against trunk. -Jay On Wed, Sep 17, 2014 at 2:14 PM, Bhavesh Mistry mistry.p.bhav...@gmail.com wrote: Hi Kafka Dev team, I see my CPU spike to 100% when network connection is lost for while. It seems network IO thread are very busy logging following error message. Is this expected behavior ? 2014-09-17 14:06:16.830 [kafka-producer-network-thread] ERROR org.apache.kafka.clients.producer.internals.Sender - Uncaught error in kafka producer I/O thread: java.lang.IllegalStateException: No entry found for node -2 at org.apache.kafka.clients.ClusterConnectionStates.nodeState( ClusterConnectionStates.java:110) at org.apache.kafka.clients.ClusterConnectionStates.disconnected( ClusterConnectionStates.java:99) at org.apache.kafka.clients.NetworkClient.initiateConnect( NetworkClient.java:394) at org.apache.kafka.clients.NetworkClient.maybeUpdateMetadata( NetworkClient.java:380) at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:174) at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:175) at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:115) at java.lang.Thread.run(Thread.java:744) Thanks, Bhavesh
Re: [Java New Producer] CPU Usage Spike to 100% when network connection is lost
HI Jay, I am running trunk producer based on following last commit with timestamp Mon Sep 15 20:34:14 2014 -0700. Please let me know if this timestamp contains the fix. Otherwise, I will file a bug with logs as Neha suggested. commit cf0f5750b39e675cf9a9c6d6394a665366db0f58 Author: Alexis Midon mi...@apache.org Date: Mon Sep 15 20:34:14 2014 -0700 KAFKA-1597 New metrics: ResponseQueueSize and BeingSentResponses; reviewed by Neha Narkhede and Jun Rao Thanks, Bhavesh On Wed, Sep 17, 2014 at 11:22 PM, Jay Kreps jay.kr...@gmail.com wrote: Also do you know what version you are running we did fix several bugs similar to this against trunk. -Jay On Wed, Sep 17, 2014 at 2:14 PM, Bhavesh Mistry mistry.p.bhav...@gmail.com wrote: Hi Kafka Dev team, I see my CPU spike to 100% when network connection is lost for while. It seems network IO thread are very busy logging following error message. Is this expected behavior ? 2014-09-17 14:06:16.830 [kafka-producer-network-thread] ERROR org.apache.kafka.clients.producer.internals.Sender - Uncaught error in kafka producer I/O thread: java.lang.IllegalStateException: No entry found for node -2 at org.apache.kafka.clients.ClusterConnectionStates.nodeState( ClusterConnectionStates.java:110) at org.apache.kafka.clients.ClusterConnectionStates.disconnected( ClusterConnectionStates.java:99) at org.apache.kafka.clients.NetworkClient.initiateConnect( NetworkClient.java:394) at org.apache.kafka.clients.NetworkClient.maybeUpdateMetadata( NetworkClient.java:380) at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:174) at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:175) at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:115) at java.lang.Thread.run(Thread.java:744) Thanks, Bhavesh
Re: [Java New Producer] CPU Usage Spike to 100% when network connection is lost
Hmm, yes, the fix was prior to that, that sounds like a bug. -Jay On Thu, Sep 18, 2014 at 9:39 AM, Bhavesh Mistry mistry.p.bhav...@gmail.com wrote: HI Jay, I am running trunk producer based on following last commit with timestamp Mon Sep 15 20:34:14 2014 -0700. Please let me know if this timestamp contains the fix. Otherwise, I will file a bug with logs as Neha suggested. commit cf0f5750b39e675cf9a9c6d6394a665366db0f58 Author: Alexis Midon mi...@apache.org Date: Mon Sep 15 20:34:14 2014 -0700 KAFKA-1597 New metrics: ResponseQueueSize and BeingSentResponses; reviewed by Neha Narkhede and Jun Rao Thanks, Bhavesh On Wed, Sep 17, 2014 at 11:22 PM, Jay Kreps jay.kr...@gmail.com wrote: Also do you know what version you are running we did fix several bugs similar to this against trunk. -Jay On Wed, Sep 17, 2014 at 2:14 PM, Bhavesh Mistry mistry.p.bhav...@gmail.com wrote: Hi Kafka Dev team, I see my CPU spike to 100% when network connection is lost for while. It seems network IO thread are very busy logging following error message. Is this expected behavior ? 2014-09-17 14:06:16.830 [kafka-producer-network-thread] ERROR org.apache.kafka.clients.producer.internals.Sender - Uncaught error in kafka producer I/O thread: java.lang.IllegalStateException: No entry found for node -2 at org.apache.kafka.clients.ClusterConnectionStates.nodeState( ClusterConnectionStates.java:110) at org.apache.kafka.clients.ClusterConnectionStates.disconnected( ClusterConnectionStates.java:99) at org.apache.kafka.clients.NetworkClient.initiateConnect( NetworkClient.java:394) at org.apache.kafka.clients.NetworkClient.maybeUpdateMetadata( NetworkClient.java:380) at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:174) at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:175) at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:115) at java.lang.Thread.run(Thread.java:744) Thanks, Bhavesh
Re: [Java New Producer] CPU Usage Spike to 100% when network connection is lost
This seems like a problem. Please can you file a JIRA and attach the log4j output there? On Wed, Sep 17, 2014 at 2:14 PM, Bhavesh Mistry mistry.p.bhav...@gmail.com wrote: Hi Kafka Dev team, I see my CPU spike to 100% when network connection is lost for while. It seems network IO thread are very busy logging following error message. Is this expected behavior ? 2014-09-17 14:06:16.830 [kafka-producer-network-thread] ERROR org.apache.kafka.clients.producer.internals.Sender - Uncaught error in kafka producer I/O thread: java.lang.IllegalStateException: No entry found for node -2 at org.apache.kafka.clients.ClusterConnectionStates.nodeState( ClusterConnectionStates.java:110) at org.apache.kafka.clients.ClusterConnectionStates.disconnected( ClusterConnectionStates.java:99) at org.apache.kafka.clients.NetworkClient.initiateConnect( NetworkClient.java:394) at org.apache.kafka.clients.NetworkClient.maybeUpdateMetadata( NetworkClient.java:380) at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:174) at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:175) at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:115) at java.lang.Thread.run(Thread.java:744) Thanks, Bhavesh
