Re: how can I set the log configuration file for spark history server ?

(-dev) Just configure your log4j.properties in $SPARK_HOME/conf (or set a custom $SPARK_CONF_DIR for the history server). On Thu, Dec 8, 2016 at 7:20 PM, John Fang wrote: > ./start-history-server.sh > starting org.apache.spark.deploy.history.HistoryServer, logging

Re: how can I set the log configuration file for spark history server ?

You can update $SPARK_HOME/spark-env.sh by setting the environment variable SPARK_HISTORY_OPTS. See http://spark.apache.org/docs/latest/monitoring.html#spark-configuration-options for options (spark.history.fs.logDirectory) you can set. There is log rotation built in (by time, not size) to the

how can I set the log configuration file for spark history server ?

./start-history-server.sh starting org.apache.spark.deploy.history.HistoryServer, logging to  /home/admin/koala/data/versions/0/SPARK/2.0.2/spark-2.0.2-bin-hadoop2.6/logs/spark-admin-org.apache.spark.deploy.history.HistoryServer-1-v069166214.sqa.zmf.out Then the history will print all log to the