[
https://issues.apache.org/jira/browse/KAFKA-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14243223#comment-14243223
]
Fabian Lange commented on KAFKA-1351:
-------------------------------------
Hi,
I was about to create a new issue, but figured this might host the comment
almost as good.
As William outlined here:
http://www.autoletics.com/posts/quick-performance-hotspot-analysis-apache-kafka
the Log.append function
(https://github.com/apache/kafka/blob/7847e9c703f3a0b70519666cdb8a6e4c8e37c3a7/core/src/main/scala/kafka/log/Log.scala#L230)
is by far the biggest contributor to response time.
There are a few reasons for this:
A) it has a long synchronized block. I do not know much about the code, so at
first glance I do not see much of an option to fix something
B) it has a trace logging call. Not sure if that trace call needs to be in
synchronized though.
The trace call is implemented by
https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/utils/Logging.scala#L34
As you might notice this does doe a slow String.format for trace. a log level
which is very likely to be off.
The above mentioned Log4j2 pattern would help there.
However there are more options, like implemented by typesafe:
https://github.com/typesafehub/scalalogging/tree/master/scalalogging-log4j/src/main/scala/com/typesafe/scalalogging/log4j
they use a macro, which would transform the code so that the actual string
format would be moved into the log level checking. I think thats quite nifty,
but maybe too much work to integrate.
So if you would check out that trace call, I would appreciate that :)
> String.format is very expensive in Scala
> ----------------------------------------
>
> Key: KAFKA-1351
> URL: https://issues.apache.org/jira/browse/KAFKA-1351
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 0.7.2, 0.8.0, 0.8.1
> Reporter: Neha Narkhede
> Labels: newbie
> Fix For: 0.8.3
>
> Attachments: KAFKA-1351.patch, KAFKA-1351_2014-04-07_18:02:18.patch,
> KAFKA-1351_2014-04-09_15:40:11.patch
>
>
> As found in KAFKA-1350, logging is causing significant overhead in the
> performance of a Kafka server. There are several info statements that use
> String.format which is particularly expensive. We should investigate adding
> our own version of String.format that merely uses string concatenation under
> the covers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
