Andrew Jorgensen created KAFKA-4141:
---------------------------------------
Summary: 2x increase in cp usage on new Producer API
Key: KAFKA-4141
URL: https://issues.apache.org/jira/browse/KAFKA-4141
Project: Kafka
Issue Type: Bug
Reporter: Andrew Jorgensen
We are seeing about a 2x increase in CPU usage for the new kafka producer
compared to the 0.8.0.1 producer. We are currently using gzip compression.
We recently upgraded our kafka server and producer to 0.10.0.1 from 0.8.1.1 and
noticed that the cpu usage for the new producer had increased pretty
significantly compared to the old producer. This has caused us to need more
resources to do the same amount of work we were doing before. I did some quick
profiling and it looks like during sending half of the cpu cycles are sped in
org.apache.kafka.common.record.Compressor.putRecord and the other half is in
org.apache.kafka.common.record.Record.computeChecksum (both are around 5.8% of
the cpu cycles for that method). I know its not apples to apples but the old
producer did not seem to have this overhead or at least it was greatly reduced.
Is this a known performance degradation compared to the old producer?
Here is the trace from the old producer:
!http://imgur.com/1xS34Dl.jpg!
New producer:
!http://imgur.com/0w0G5b1.jpg!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
