woosuk.ro created HDFS-17382:
--------------------------------
Summary: Add Apache Log4j Extras Library to Hadoop 3.3 for
Enhanced Log Rolling Capabilities
Key: HDFS-17382
URL: https://issues.apache.org/jira/browse/HDFS-17382
Project: Hadoop HDFS
Issue Type: Improvement
Affects Versions: 3.3.4
Reporter: woosuk.ro
In the current Hadoop 3.3 version, the system relies on Log4j 1.x for logging
purposes. This dependency limits the logging functionality, especially when it
comes to rolling log files such as 'hdfs-audit.log'. Rolling log files is
crucial for managing log size and ensuring logs are rotated out over time to
prevent excessive disk space usage. However, the Log4j 1.x version integrated
within Hadoop lacks the necessary capabilities to efficiently handle log
rolling.
This library extends logging capabilities, including more flexible and
configurable log rolling features. By deploying this library, we can enable
advanced rolling strategies such as time-based rolling, size-based rolling, and
compression of rolled logs, which are not supported by the default Log4j 1.x
setup in Hadoop.
The integration of Apache Log4j Extras into Hadoop will significantly improve
log management by allowing for more sophisticated and configurable log rotation
policies. This enhancement is particularly important for maintaining system
performance and reliability, as well as for compliance with log retention
policies.
Although there are plans to upgrade to Log4j 2 in the forthcoming Hadoop 3.5
version, which will inherently solve these issues by providing enhanced logging
features, there is an immediate need to enable advanced log rolling
capabilities in the current and previous versions of Hadoop.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
