[ https://issues.apache.org/jira/browse/SPARK-44646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750895#comment-17750895 ]
L. C. Hsieh commented on SPARK-44646: ------------------------------------- I have not used it but maybe you can try https://logging.apache.org/log4j/log4j-2.2/log4j-to-slf4j/index.html. > Migrate Log4j 2.x in Spark 3.4.1 to Logback > ------------------------------------------- > > Key: SPARK-44646 > URL: https://issues.apache.org/jira/browse/SPARK-44646 > Project: Spark > Issue Type: Brainstorming > Components: Build > Affects Versions: 3.4.1 > Reporter: Yu Tian > Priority: Major > > Hi, > We are working on the spark 3.4.1 upgrade from spark 3.1.3, in our logging > system, we are using logback framework, it is working with spark 3.1.3 since > it is using log4j 1.x. However, when we upgrade spark to 3.4.1, based on the > [release > notes|https://spark.apache.org/docs/latest/core-migration-guide.html], spark > is migrating from log4j 2.x from log4j 1.x, the way we are replacing the > log4j with logback is causing build failures in spark master start process. > Error: Unable to initialize main class org.apache.spark.deploy.master.Master > Caused by: java.lang.NoClassDefFoundError: > org/apache/logging/log4j/core/Filter > In our current approach, we are using log4j-over-slf4j to replace the > log4j-core, it is only applicable to log4j 1.x library. And there is no > log4j-over-slf4j for log4j 2.x out there yet. (please correct me if I am > wrong). > I am also curious that why spark choose to use log4j 2.x instead of using > SPI, which gives the users less flexibility to choose whatever logger > implementation they want to use. > I want to share this issue and see if anyone else has been reported this and > if there is any work-around or alternative solutions for it. Any suggestions > are appreciated, thanks. -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org