[
https://issues.apache.org/jira/browse/SPARK-2481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14062870#comment-14062870
]
Masayoshi TSUZUKI commented on SPARK-2481:
------------------------------------------
spark-env.sh is loaded after these lines are evaluated. (It is loaded in the
line spark-daemon.sh is called)
{code:title=sbin/start-history-server.sh|borderStyle=solid}
if [ $# != 0 ]; then
echo "Using command line arguments for setting the log directory is
deprecated. Please "
echo "set the spark.history.fs.logDirectory configuration option instead."
export SPARK_HISTORY_OPTS="$SPARK_HISTORY_OPTS
-Dspark.history.fs.logDirectory=$1"
fi
{code}
If you want to set SPARK_HISTORY_OPTS in spark-env.sh and use the deprecated
option like above, you can write like following in spark-env.sh:
{code:title=spark-env.sh|borderStyle=solid}
SPARK_HISTORY_OPTS="$SPARK_HISTORY_OPTS -Dspark.history.XX=XX"
{code}
This works fine in my environment.
> The environment variables SPARK_HISTORY_OPTS is covered in
> start-history-server.sh
> ----------------------------------------------------------------------------------
>
> Key: SPARK-2481
> URL: https://issues.apache.org/jira/browse/SPARK-2481
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Reporter: Guoqiang Li
> Assignee: Guoqiang Li
>
> If we have the following code in the conf/spark-env.sh
> {{export SPARK_HISTORY_OPTS="-DSpark.history.XX=XX"}}
> The environment variables SPARK_HISTORY_OPTS is covered in
> [start-history-server.sh|https://github.com/apache/spark/blob/master/sbin/start-history-server.sh]
>
> {code}
> if [ $# != 0 ]; then
> echo "Using command line arguments for setting the log directory is
> deprecated. Please "
> echo "set the spark.history.fs.logDirectory configuration option instead."
> export SPARK_HISTORY_OPTS="$SPARK_HISTORY_OPTS
> -Dspark.history.fs.logDirectory=$1"
> fi
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
