Hello We are seeing high CPU usage for the Kafka process. I am using 0.11 version. Has 5 topics out of which 1 was created newly. We attempted to publish message this new topic which did not show up in the consumer, but no errors in the publisher end. Not sure why the message did not show up in consumer.
This ran for a couple of days (30K messages) when we noticed 100%+ CPU usage. Tried deleting the topic (config is enabled), it was marked for deletion but after which usage rose to below levels 240%+. We restarted the process many times and disabled the publisher/producer but no difference. After some time (1 or 2 hours) we are getting a "Too many open files" error and process is shutting down. We have 3 nodes with Kafka and 3 other nodes running with ZK, but only one specific node is showing this issue. (where new topic partition is present). Still debugging and this is a prod environment.. please help! Thanks, Abhi top - 17:47:43 up 289 days, 18:54, 2 users, load average: 2.65, 2.72, 2.52 Tasks: 144 total, 1 running, 143 sleeping, 0 stopped, 0 zombie %Cpu(s): 37.5 us, 19.4 sy, 0.0 ni, 37.0 id, 0.0 wa, 0.0 hi, 5.4 si, 0.7 st KiB Mem : 16266464 total, 1431916 free, 5769976 used, 9064572 buff/cache KiB Swap: 0 total, 0 free, 0 used. 9230548 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 32058 root 20 0 5898348 1.078g 15548 S 253.0 6.9 99:03.68 java 10 root 20 0 0 0 0 S 0.3 0.0 921:42.77 rcu_sche
