[
https://issues.apache.org/jira/browse/SPARK-44646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17820343#comment-17820343
]
Rafal Mucha commented on SPARK-44646:
-------------------------------------
Hi [~viirya] ,
Do you have any plans to somehow address this issue? It is quite serious
problem, since it force using log4j2 implementation (log4j-core) with Spark.
Solution is either quite naive approach like
[here|https://github.com/apache/spark/pull/45001] , or on the other hand,
creating separate interfaces in Spark core and developing packages that offer
implementations utilising different logging frameworks - it is spring approach.
Have you had any discussions about that?
Best
> Migrate Log4j 2.x in Spark 3.4.1 to Logback
> -------------------------------------------
>
> Key: SPARK-44646
> URL: https://issues.apache.org/jira/browse/SPARK-44646
> Project: Spark
> Issue Type: Brainstorming
> Components: Build
> Affects Versions: 3.4.1
> Reporter: Yu Tian
> Priority: Major
> Attachments: Screenshot 2023-08-09 at 2.40.12 PM.png
>
>
> Hi,
> We are working on the spark 3.4.1 upgrade from spark 3.1.3, in our logging
> system, we are using logback framework, it is working with spark 3.1.3 since
> it is using log4j 1.x. However, when we upgrade spark to 3.4.1, based on the
> [release
> notes|https://spark.apache.org/docs/latest/core-migration-guide.html], spark
> is migrating from log4j 2.x from log4j 1.x, the way we are replacing the
> log4j with logback is causing build failures in spark master start process.
> Error: Unable to initialize main class org.apache.spark.deploy.master.Master
> Caused by: java.lang.NoClassDefFoundError:
> org/apache/logging/log4j/core/Filter
> In our current approach, we are using log4j-over-slf4j to replace the
> log4j-core, it is only applicable to log4j 1.x library. And there is no
> log4j-over-slf4j for log4j 2.x out there yet. (please correct me if I am
> wrong).
> I am also curious that why spark choose to use log4j 2.x instead of using
> SPI, which gives the users less flexibility to choose whatever logger
> implementation they want to use.
> I want to share this issue and see if anyone else has been reported this and
> if there is any work-around or alternative solutions for it. Any suggestions
> are appreciated, thanks.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
