You may want to take a look at this PR: https://github.com/apache/spark/pull/1558
Long story short: while not a terrible idea to show running applications, your particular case should be solved differently. Applications are responsible for calling "SparkContext.stop()" at the end of their run, currently, so you should make sure your code does that even when something goes wrong. If that is done, they'll show up in the History Server. On Thu, Oct 2, 2014 at 11:31 AM, SK <skrishna...@gmail.com> wrote: > Hi, > > Currently the history server provides application details for only the > successfully completed jobs (where the APPLICATION_COMPLETE file is > generated). However, (long-running) jobs that we terminate manually or > failed jobs where the APPLICATION_COMPLETE may not be generated, dont show > up on the history server page. They however do show up on the 4040 interface > as long as they are running. Is it possible to save those logs and load them > up on the history server (even when the APPLICATION_COMPLETE is not > present)? This would allow us troubleshoot the failed and terminated jobs > without holding up the cluster. > > thanks > > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Application-details-for-failed-and-teminated-jobs-tp15627.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > -- Marcelo --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
