Re: Unreasonably high CPU from Kafka (0.8.2.1)

2015-09-17 Thread Jaikiran Pai
Thanks Brock for that tip! I ran the ps -L -p command and this is trimmed version of it: PID LWP TTY TIME CMD 19524 19550 ?01:11:38 java 19524 19551 ?01:33:55 java 19524 19552 ?01:38:32 java 19524 19557 ?01:30:12 java 19524 is the process id and these

Re: Unreasonably high CPU from Kafka (0.8.2.1)

2015-09-17 Thread Jun Rao
You can enable profiling in tools like jvisualvm and see which methods other select() show up high. Thanks, Jun On Thu, Sep 17, 2015 at 5:58 PM, Jaikiran Pai wrote: > That's a good point, I missed that. Is there any other way to track down > why it would consume this high CPU when it's idle? I

Re: Unreasonably high CPU from Kafka (0.8.2.1)

2015-09-17 Thread Brock Noland
If you are on Linux, you can use the following approach. 1) Get the pid 2) Find the largest consumer of CPU via: ps -L -p 3) Convert the LWP column to hex and then look for that value in the thread dump on the "nid=0x.." field of each thread. Cheers! Brock On Thu, Sep 17, 2015 at 3:58 PM, Jai

Re: Unreasonably high CPU from Kafka (0.8.2.1)

2015-09-17 Thread Jaikiran Pai
That's a good point, I missed that. Is there any other way to track down why it would consume this high CPU when it's idle? I can send the thread dumps (taken at intervals of a few seconds of each other), but each of them has not much interesting (IMO) than these specific repeated stacktraces.

Re: Unreasonably high CPU from Kafka (0.8.2.1)

2015-09-17 Thread Jun Rao
The selector is probably not the issue. If there is no incoming traffic, selector.select(300) won't return until after 300ms. Thanks, Jun On Thu, Sep 17, 2015 at 1:13 PM, Jaikiran Pai wrote: > Sending this to the dev list since the Kafka dev team might have more > inputs on this one. Can someo

Re: Unreasonably high CPU from Kafka (0.8.2.1)

2015-09-17 Thread Jaikiran Pai
Sending this to the dev list since the Kafka dev team might have more inputs on this one. Can someone please take a look at the issue noted below and whether the suggested change makes sense? -Jaikiran On Tuesday 15 September 2015 12:03 AM, Jaikiran Pai wrote: We have been using Kafka for a whi