Hi Grzegorz, Sorry for the late response. Unfortunately, if the Master UI doesn't know about your applications (they are "completed" with respect to a different Master), then it can't regenerate the UIs even if the logs exist. You will have to use the history server for that.
How did you start the history server? If you are using Spark <=1.0, you can pass the directory as an argument to the sbin/start-history-server.sh script. Otherwise, you may need to set the following in your conf/spark-env.sh to specify the log directory: export SPARK_HISTORY_OPTS=-Dspark.history.fs.logDirectory=/tmp/spark-events It could also be a permissions thing. Make sure your logs in /tmp/spark-events are accessible by the JVM that runs the history server. Also, there's a chance that "/tmp/spark-events" is interpreted as an HDFS path depending on which Spark version you're running. To resolve any ambiguity, you may set the log path to "file:/tmp/spark-events" instead. But first verify whether they actually exist. Let me know if you get it working, -Andrew 2014-08-19 8:23 GMT-07:00 Grzegorz Białek <grzegorz.bia...@codilime.com>: > Hi, > Is there any way view history of applications statistics in master ui > after restarting master server? I have all logs ing /tmp/spark-events/ but > when I start history server in this directory it says "No Completed > Applications Found". Maybe I could copy this logs to dir used by master > server but I couldn't find any. Or maybe I'm doing something wrong > launching history server. > Do you have any idea how to solve it? > > Thanks, > Grzegorz > > > On Thu, Aug 14, 2014 at 10:53 AM, Grzegorz Białek < > grzegorz.bia...@codilime.com> wrote: > >> Hi, >> >> Thank you both for your answers. Browsing using Master UI works fine. >> Unfortunately History Server shows "No Completed Applications Found" even >> if logs exists under given directory, but using Master UI is enough for me. >> >> Best regards, >> Grzegorz >> >> >> >> On Wed, Aug 13, 2014 at 8:09 PM, Andrew Or <and...@databricks.com> wrote: >> >>> The Spark UI isn't available through the same address; otherwise new >>> applications won't be able to bind to it. Once the old application >>> finishes, the standalone Master renders the after-the-fact application UI >>> and exposes it under a different URL. To see this, go to the Master UI >>> (<master-url>:8080) and click on your application in the "Completed >>> Applications" table. >>> >>> >>> 2014-08-13 10:56 GMT-07:00 Matei Zaharia <matei.zaha...@gmail.com>: >>> >>> Take a look at http://spark.apache.org/docs/latest/monitoring.html -- >>>> you need to launch a history server to serve the logs. >>>> >>>> Matei >>>> >>>> On August 13, 2014 at 2:03:08 AM, grzegorz-bialek ( >>>> grzegorz.bia...@codilime.com) wrote: >>>> >>>> Hi, >>>> I wanted to access Spark web UI after application stops. I set >>>> spark.eventLog.enabled to true and logs are availaible >>>> in JSON format in /tmp/spark-event but web UI isn't available under >>>> address >>>> http://<driver-node>:4040 >>>> I'm running Spark in standalone mode. >>>> >>>> What should I do to access web UI after application ends? >>>> >>>> Thanks, >>>> Grzegorz >>>> >>>> >>>> >>>> -- >>>> View this message in context: >>>> http://apache-spark-user-list.1001560.n3.nabble.com/Viewing-web-UI-after-fact-tp12023.html >>>> Sent from the Apache Spark User List mailing list archive at >>>> Nabble.com. >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >>>> For additional commands, e-mail: user-h...@spark.apache.org >>>> >>>> >>> >> >
