[
https://issues.apache.org/jira/browse/SPARK-43534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuming Wang updated SPARK-43534:
--------------------------------
Description:
Build Spark:
{code:sh}
./dev/make-distribution.sh --name provided --tgz -Phive -Phive-thriftserver
-Pyarn -Phadoop-provided
tar -zxf spark-3.5.0-SNAPSHOT-bin-provided.tgz {code}
Copy the following jars to spark-3.5.0-SNAPSHOT-bin-provided/jars/:
{noformat}
guava-14.0.1.jar
hadoop-client-api-3.3.5.jar
hadoop-client-runtime-3.3.5.jar
hadoop-shaded-guava-1.1.1.jar
hadoop-yarn-server-web-proxy-3.3.5.jar
slf4j-api-2.0.7.jar
{noformat}
Add a new log4j2.properties to spark-3.5.0-SNAPSHOT-bin-provided/conf:
{code:none}
rootLogger.level = info
rootLogger.appenderRef.file.ref = File
rootLogger.appenderRef.stderr.ref = console
appender.console.type = Console
appender.console.name = console
appender.console.target = SYSTEM_ERR
appender.console.layout.type = PatternLayout
appender.console.layout.pattern = %d{yy/MM/dd HH:mm:ss,SSS} %p [%t] %c{2}:%L :
%m%n
appender.file.type = RollingFile
appender.file.name = File
appender.file.fileName = /tmp/spark/logs/spark.log
appender.file.filePattern = /tmp/spark/logs/spark.%d{yyyyMMdd-HH}.log
appender.file.append = true
appender.file.layout.type = PatternLayout
appender.file.layout.pattern = %d{yy/MM/dd HH:mm:ss,SSS} %p [%t] %c{2}:%L : %m%n
appender.file.policies.type = Policies
appender.file.policies.time.type = TimeBasedTriggeringPolicy
appender.file.policies.time.interval = 1
appender.file.policies.time.modulate = true
appender.file.policies.size.type = SizeBasedTriggeringPolicy
appender.file.policies.size.size = 256M
appender.file.strategy.type = DefaultRolloverStrategy
appender.file.strategy.max = 100
{code}
Start Spark thriftserver:
{code:java}
sbin/start-thriftserver.sh
{code}
Check the log:
{code:sh}
cat /tmp/spark/logs/spark.log
{code}
was:
Build Spark:
{code:sh}
./dev/make-distribution.sh --name provided --tgz -Phive -Phive-thriftserver
-Pyarn -Phadoop-provided
tar -zxf spark-3.5.0-SNAPSHOT-bin-provided.tgz {code}
Copy the fellowing jars to spark-3.5.0-SNAPSHOT-bin-provided/jars/:
{noformat}
guava-14.0.1.jar
hadoop-client-api-3.3.5.jar
hadoop-client-runtime-3.3.5.jar
hadoop-shaded-guava-1.1.1.jar
hadoop-yarn-server-web-proxy-3.3.5.jar
slf4j-api-2.0.7.jar
{noformat}
Add a new log4j2.properties to spark-3.5.0-SNAPSHOT-bin-provided/conf:
{code:none}
rootLogger.level = info
rootLogger.appenderRef.file.ref = File
rootLogger.appenderRef.stderr.ref = console
appender.console.type = Console
appender.console.name = console
appender.console.target = SYSTEM_ERR
appender.console.layout.type = PatternLayout
appender.console.layout.pattern = %d{yy/MM/dd HH:mm:ss,SSS} %p [%t] %c{2}:%L :
%m%n
appender.file.type = RollingFile
appender.file.name = File
appender.file.fileName = /tmp/spark/logs/spark.log
appender.file.filePattern = /tmp/spark/logs/spark.%d{yyyyMMdd-HH}.log
appender.file.append = true
appender.file.layout.type = PatternLayout
appender.file.layout.pattern = %d{yy/MM/dd HH:mm:ss,SSS} %p [%t] %c{2}:%L : %m%n
appender.file.policies.type = Policies
appender.file.policies.time.type = TimeBasedTriggeringPolicy
appender.file.policies.time.interval = 1
appender.file.policies.time.modulate = true
appender.file.policies.size.type = SizeBasedTriggeringPolicy
appender.file.policies.size.size = 256M
appender.file.strategy.type = DefaultRolloverStrategy
appender.file.strategy.max = 100
{code}
Start Spark thriftserver:
{code:java}
sbin/start-thriftserver.sh
{code}
Check the log:
{code:sh}
cat /tmp/spark/logs/spark.log
{code}
> Add log4j-1.2-api and log4j-slf4j2-impl to classpath if active hadoop-provided
> ------------------------------------------------------------------------------
>
> Key: SPARK-43534
> URL: https://issues.apache.org/jira/browse/SPARK-43534
> Project: Spark
> Issue Type: Bug
> Components: Build
> Affects Versions: 3.4.0
> Reporter: Yuming Wang
> Priority: Major
>
> Build Spark:
> {code:sh}
> ./dev/make-distribution.sh --name provided --tgz -Phive -Phive-thriftserver
> -Pyarn -Phadoop-provided
> tar -zxf spark-3.5.0-SNAPSHOT-bin-provided.tgz {code}
> Copy the following jars to spark-3.5.0-SNAPSHOT-bin-provided/jars/:
> {noformat}
> guava-14.0.1.jar
> hadoop-client-api-3.3.5.jar
> hadoop-client-runtime-3.3.5.jar
> hadoop-shaded-guava-1.1.1.jar
> hadoop-yarn-server-web-proxy-3.3.5.jar
> slf4j-api-2.0.7.jar
> {noformat}
> Add a new log4j2.properties to spark-3.5.0-SNAPSHOT-bin-provided/conf:
> {code:none}
> rootLogger.level = info
> rootLogger.appenderRef.file.ref = File
> rootLogger.appenderRef.stderr.ref = console
> appender.console.type = Console
> appender.console.name = console
> appender.console.target = SYSTEM_ERR
> appender.console.layout.type = PatternLayout
> appender.console.layout.pattern = %d{yy/MM/dd HH:mm:ss,SSS} %p [%t] %c{2}:%L
> : %m%n
> appender.file.type = RollingFile
> appender.file.name = File
> appender.file.fileName = /tmp/spark/logs/spark.log
> appender.file.filePattern = /tmp/spark/logs/spark.%d{yyyyMMdd-HH}.log
> appender.file.append = true
> appender.file.layout.type = PatternLayout
> appender.file.layout.pattern = %d{yy/MM/dd HH:mm:ss,SSS} %p [%t] %c{2}:%L :
> %m%n
> appender.file.policies.type = Policies
> appender.file.policies.time.type = TimeBasedTriggeringPolicy
> appender.file.policies.time.interval = 1
> appender.file.policies.time.modulate = true
> appender.file.policies.size.type = SizeBasedTriggeringPolicy
> appender.file.policies.size.size = 256M
> appender.file.strategy.type = DefaultRolloverStrategy
> appender.file.strategy.max = 100
> {code}
> Start Spark thriftserver:
> {code:java}
> sbin/start-thriftserver.sh
> {code}
> Check the log:
> {code:sh}
> cat /tmp/spark/logs/spark.log
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
