Gaurav Narula created KAFKA-15141:
-------------------------------------
Summary: High CPU usage with log4j2
Key: KAFKA-15141
URL: https://issues.apache.org/jira/browse/KAFKA-15141
Project: Kafka
Issue Type: Improvement
Components: core
Affects Versions: 3.4.1, 3.5.0, 3.3.2
Reporter: Gaurav Narula
Assignee: Gaurav Narula
Kafka brokers make use of the [Logging
trait|https://github.com/apache/kafka/blob/1f4cbc5d53259031123b6e9e6bb9a5bbe1e084e8/core/src/main/scala/kafka/utils/Logging.scala#L41]
which instantiates a Logger object for every instantiation of the class using
the trait by default.
When using log4j2 as the logging implementation, the instantiation of a Logger
object requires a stack traversal
[[1]|https://github.com/apache/logging-log4j2/blob/2.x/log4j-api/src/main/java/org/apache/logging/log4j/spi/AbstractLoggerAdapter.java#L121]
and
[[2]|https://github.com/apache/logging-log4j2/blob/83bba1bc322e80e7e95edbebc2383f2724dbe0de/log4j-slf4j-impl/src/main/java/org/apache/logging/slf4j/Log4jLoggerFactory.java#L54].
While LOG4J2-2940 ensures stack is not traversed unless required, the default
{{{}ContextSelector{}}},
[ClassLoaderContextSelector|https://logging.apache.org/log4j/2.x/log4j-core/apidocs/org/apache/logging/log4j/core/selector/ClassLoaderContextSelector.html]
causes a stack traversal.
These stack traversals are frequent and quite CPU intensive and profiling
suggests they consume ~5% CPU time (of the CPU stacks we have profiled on a
sample of clusters). While log4j2 users can potentially avoid this by changing
the context selector in their configuration, it is easy to overlook and the
default configurations results in high CPU usage inadvertently.
An easy fix would be to instantiate the loggers statically for some commonly
instantiated classes in Kafka which make use of the Logging trait.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
