[
https://issues.apache.org/jira/browse/KAFKA-7800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manikumar resolved KAFKA-7800.
------------------------------
Fix Version/s: 2.4.0
Resolution: Fixed
> Extend Admin API to support dynamic application log levels
> ----------------------------------------------------------
>
> Key: KAFKA-7800
> URL: https://issues.apache.org/jira/browse/KAFKA-7800
> Project: Kafka
> Issue Type: New Feature
> Reporter: Stanislav Kozlovski
> Assignee: Stanislav Kozlovski
> Priority: Major
> Fix For: 2.4.0
>
>
> [https://cwiki.apache.org/confluence/display/KAFKA/KIP-412%3A+Extend+Admin+API+to+support+dynamic+application+log+levels]
> Logging is a critical part of any system's infrastructure. It is the most
> direct way of observing what is happening with a system. In the case of
> issues, it helps us diagnose the problem quickly which in turn helps lower
> the
> [MTTR|http://enterprisedevops.org/article/devops-metric-mean-time-to-recovery-mttr-definition-and-reasoning].
> Kafka supports application logging via the log4j library and outputs messages
> in various log levels (TRACE, DEBUG, INFO, WARN, ERROR). Log4j is a rich
> library that supports fine-grained logging configurations (e.g use INFO-level
> logging in {{kafka.server.ReplicaManager}} and use DEBUG-level in
> {{kafka.server.KafkaApis}}).
> This is statically configurable through the
> [log4j.properties|https://github.com/apache/kafka/blob/trunk/config/log4j.properties]
> file which gets read once at broker start-up.
> A problem with this static configuration is that we cannot alter the log
> levels when a problem arises. It is severely impractical to edit a properties
> file and restart all brokers in order to gain visibility of a problem taking
> place in production.
> It would be very useful if we support dynamically altering the log levels at
> runtime without needing to restart the Kafka process.
> Log4j itself supports dynamically altering the log levels in a programmatic
> way and Kafka exposes a [JMX
> API|https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/utils/Log4jController.scala]
> that lets you alter them. This allows users to change the log levels via a
> GUI (jconsole) or a CLI (jmxterm) that uses JMX.
> There is one problem with changing log levels through JMX that we hope to
> address and that is *Ease of Use*:
> * Establishing a connection - Connecting to a remote process via JMX
> requires configuring and exposing multiple JMX ports to the outside world.
> This is a burden on users, as most production deployments may stand behind
> layers of firewalls and have policies against opening ports. This makes
> opening the ports and connections in the middle of an incident even more
> burdensome
> * Security - JMX and tools around it support authentication and
> authorization but it is an additional hassle to set up credentials for
> another system.
> * Manual process - Changing the whole cluster's log level requires manually
> connecting to each broker. In big deployments, this is severely impractical
> and forces users to build tooling around it.
> h4. Proposition
> Ideally, Kafka would support dynamically changing log levels and address all
> of the aforementioned concerns out of the box.
> We propose extending the IncrementalAlterConfig/DescribeConfig Admin API with
> functionality for dynamically altering the broker's log level.
> This approach would also pave the way for even finer-grained logging logic
> (e.g log DEBUG level only for a certain topic) and would allow us to leverage
> the existing *AlterConfigPolicy* for custom user-defined validation of
> log-level changes.
> These log-level changes will be *temporary* and reverted on broker restart -
> we will not persist them anywhere.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
