Yu Tian created SPARK-44646:
-------------------------------
Summary: Migrate Log4j 2.x in Spark 3.4.1 to Logback
Key: SPARK-44646
URL: https://issues.apache.org/jira/browse/SPARK-44646
Project: Spark
Issue Type: Brainstorming
Components: Build
Affects Versions: 3.4.1
Reporter: Yu Tian
Hi,
We are working on the spark 3.4.1 upgrade from spark 3.1.3, in our logging
system, we are using logback framework, it is working with spark 3.1.3 since it
is using log4j 1.x. However, when we upgrade spark to 3.4.1, based on the
[release notes|https://spark.apache.org/docs/latest/core-migration-guide.html],
spark is migrating from log4j 2.x from log4j 1.x, the way we are replacing the
log4j with logback is causing build failures in spark master start process.
Error: Unable to initialize main class org.apache.spark.deploy.master.Master
Caused by: java.lang.NoClassDefFoundError: org/apache/logging/log4j/core/Filter
In our current approach, we are using log4j-over-slf4j to replace the
log4j-core, it is only applicable to log4j 1.x library. And there is no
log4j-over-slf4j for log4j 2.x out there yet. (please correct me if I am
wrong).
I am also curious that why spark choose to use log4j 2.x instead of using SPI,
which gives the users less flexibility to choose whatever logger implementation
they want to use.
I want to share this issue and see if anyone else has been reported this and if
there is any work-around or alternative solutions for it. Any suggestions are
appreciated, thanks.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
