[
https://issues.apache.org/jira/browse/SPARK-47194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yang Jie resolved SPARK-47194.
------------------------------
Resolution: Won't Fix
It seems that the `-Dlog4j2.debug` option may not be working in 2.23.0, perhaps
we should skip this upgrade. I have tested the following scenarios:
1. run `dev/make-distribution.sh --tgz` to build a Spark Client
2. add `log4j2.properties` and `spark-defaults.conf` with the same content as
test case `Verify logging configuration is picked from the provided
SPARK_CONF_DIR/log4j2.properties`
```
log4j2.properties
# This log4j config file is for integration test SparkConfPropagateSuite.
rootLogger.level = debug
rootLogger.appenderRef.stdout.ref = console
appender.console.type = Console
appender.console.name = console
appender.console.target = SYSTEM_ERR
appender.console.layout.type = PatternLayout
appender.console.layout.pattern = %d\{HH:mm:ss.SSS} %p %c:
%maxLen\{%m}{512}%n%ex\{8}%n
```
```
spark-defaults.conf
spark.driver.extraJavaOptions -Dlog4j2.debug
spark.executor.extraJavaOptions -Dlog4j2.debug
spark.kubernetes.executor.deleteOnTermination false
```
3. run `bin/run-example SparkPi`
When using log4j 2.22.1, we can have the following log:
```
...
TRACE StatusLogger DefaultConfiguration cleaning Appenders from 1 LoggerConfigs.
DEBUG StatusLogger Stopped
org.apache.logging.log4j.core.config.DefaultConfiguration@384ad17b OK
TRACE StatusLogger Reregistering MBeans after reconfigure.
Selector=org.apache.logging.log4j.core.selector.ClassLoaderContextSelector@5852c06f
TRACE StatusLogger Reregistering context (1/1): '5ffd2b27'
org.apache.logging.log4j.core.LoggerContext@31190526
TRACE StatusLogger Unregistering but no MBeans found matching
'org.apache.logging.log4j2:type=5ffd2b27'
TRACE StatusLogger Unregistering but no MBeans found matching
'org.apache.logging.log4j2:type=5ffd2b27,component=StatusLogger'
TRACE StatusLogger Unregistering but no MBeans found matching
'org.apache.logging.log4j2:type=5ffd2b27,component=ContextSelector'
TRACE StatusLogger Unregistering but no MBeans found matching
'org.apache.logging.log4j2:type=5ffd2b27,component=Loggers,name=*'
TRACE StatusLogger Unregistering but no MBeans found matching
'org.apache.logging.log4j2:type=5ffd2b27,component=Appenders,name=*'
TRACE StatusLogger Unregistering but no MBeans found matching
'org.apache.logging.log4j2:type=5ffd2b27,component=AsyncAppenders,name=*'
TRACE StatusLogger Unregistering but no MBeans found matching
'org.apache.logging.log4j2:type=5ffd2b27,component=AsyncLoggerRingBuffer'
TRACE StatusLogger Unregistering but no MBeans found matching
'org.apache.logging.log4j2:type=5ffd2b27,component=Loggers,name=*,subtype=RingBuffer'
DEBUG StatusLogger Registering MBean org.apache.logging.log4j2:type=5ffd2b27
DEBUG StatusLogger Registering MBean
org.apache.logging.log4j2:type=5ffd2b27,component=StatusLogger
DEBUG StatusLogger Registering MBean
org.apache.logging.log4j2:type=5ffd2b27,component=ContextSelector
DEBUG StatusLogger Registering MBean
org.apache.logging.log4j2:type=5ffd2b27,component=Loggers,name=
DEBUG StatusLogger Registering MBean
org.apache.logging.log4j2:type=5ffd2b27,component=Appenders,name=console
TRACE StatusLogger Using default SystemClock for timestamps.
DEBUG StatusLogger org.apache.logging.log4j.core.util.SystemClock supports
precise timestamps.
TRACE StatusLogger Using DummyNanoClock for nanosecond timestamps.
DEBUG StatusLogger Reconfiguration complete for context[name=5ffd2b27] at URI
/Users/yangjie01/Tools/4.0/spark-4.0.0-SNAPSHOT-bin-3.3.6/conf/log4j2.properties
(org.apache.logging.log4j.core.LoggerContext@31190526) with optional
ClassLoader: null
DEBUG StatusLogger Shutdown hook enabled. Registering a new one.
...
```
But when using log4j 2.23.0, no logs related to `StatusLogger` are printed.
cc @dongjoon-hyun
> Upgrade log4j2 to 2.23.0
> ------------------------
>
> Key: SPARK-47194
> URL: https://issues.apache.org/jira/browse/SPARK-47194
> Project: Spark
> Issue Type: Sub-task
> Components: Build
> Affects Versions: 4.0.0
> Reporter: Yang Jie
> Priority: Major
> Labels: pull-request-available
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
