Yuming Wang created SPARK-43534:
-----------------------------------
Summary: Add log4j-1.2-api and log4j-slf4j2-impl to classpath if
active hadoop-provided
Key: SPARK-43534
URL: https://issues.apache.org/jira/browse/SPARK-43534
Project: Spark
Issue Type: Bug
Components: Build
Affects Versions: 3.4.0
Reporter: Yuming Wang
Build Spark:
{code:sh}
./dev/make-distribution.sh --name provided --tgz -Phive -Phive-thriftserver
-Pyarn -Phadoop-provided
tar -zxf spark-3.5.0-SNAPSHOT-bin-provided.tgz {code}
Copy the fellowing jars to spark-3.5.0-SNAPSHOT-bin-provided/jars/:
{noformat}
guava-14.0.1.jar
hadoop-client-api-3.3.5.jar
hadoop-client-runtime-3.3.5.jar
hadoop-shaded-guava-1.1.1.jar
hadoop-yarn-server-web-proxy-3.3.5.jar
slf4j-api-2.0.7.jar
{noformat}
Add a new log4j2.properties to spark-3.5.0-SNAPSHOT-bin-provided/conf:
{code:none}
rootLogger.level = info
rootLogger.appenderRef.file.ref = File
rootLogger.appenderRef.stderr.ref = console
appender.console.type = Console
appender.console.name = console
appender.console.target = SYSTEM_ERR
appender.console.layout.type = PatternLayout
appender.console.layout.pattern = %d{yy/MM/dd HH:mm:ss,SSS} %p [%t] %c{2}:%L :
%m%n
appender.file.type = RollingFile
appender.file.name = File
appender.file.fileName = /tmp/spark/logs/spark.log
appender.file.filePattern = /tmp/spark/logs/spark.%d{yyyyMMdd-HH}.log
appender.file.append = true
appender.file.layout.type = PatternLayout
appender.file.layout.pattern = %d{yy/MM/dd HH:mm:ss,SSS} %p [%t] %c{2}:%L : %m%n
appender.file.policies.type = Policies
appender.file.policies.time.type = TimeBasedTriggeringPolicy
appender.file.policies.time.interval = 1
appender.file.policies.time.modulate = true
appender.file.policies.size.type = SizeBasedTriggeringPolicy
appender.file.policies.size.size = 256M
appender.file.strategy.type = DefaultRolloverStrategy
appender.file.strategy.max = 100
{code}
Start Spark thriftserver:
{code:java}
sbin/start-thriftserver.sh
{code}
Check the log:
{code:sh}
cat /tmp/spark/logs/spark.log
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
