[ https://issues.apache.org/jira/browse/SPARK-1132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14537292#comment-14537292 ]
sagar commented on SPARK-1132: ------------------------------ Hi Team, I see the issue is resolved and Fix Version/s: is -1.0.0. Is 1.0.0 is spark version ? Where i can get the spark version 1.0.0. Currently i am getting below error - 15/05/10 08:42:20 INFO deprecation: mapred.tip.id is deprecated. Instead, use mapreduce.task.id 15/05/10 08:42:20 INFO deprecation: mapred.task.id is deprecated. Instead, use mapreduce.task.attempt.id 15/05/10 08:42:20 INFO deprecation: mapred.task.is.map is deprecated. Instead, use mapreduce.task.ismap 15/05/10 08:42:20 INFO deprecation: mapred.task.partition is deprecated. Instead, use mapreduce.task.partition 15/05/10 08:42:20 INFO deprecation: mapred.job.id is deprecated. Instead, use mapreduce.job.id 15/05/10 08:42:20 INFO Executor: Finished task 0.0 in stage 0.0 (TID 0). 1750 bytes result sent to driver 15/05/10 08:42:20 INFO TaskSetManager: Finished task 0.0 in stage 0.0 (TID 0) in 207 ms on localhost (1/1) 15/05/10 08:42:20 INFO TaskSchedulerImpl: Removed TaskSet 0.0, whose tasks have all completed, from pool 15/05/10 08:42:20 INFO DAGScheduler: Stage 0 (count at SparkFilter.java:22) finished in 0.225 s 15/05/10 08:42:20 INFO DAGScheduler: Job 0 finished: count at SparkFilter.java:22, took 0.314437 s 0 15/05/10 08:42:20 ERROR LiveListenerBus: Listener EventLoggingListener threw an exception java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.spark.scheduler.EventLoggingListener$$anonfun$logEvent$3.apply(EventLoggingListener.scala:144) at org.apache.spark.scheduler.EventLoggingListener$$anonfun$logEvent$3.apply(EventLoggingListener.scala:144) at scala.Option.foreach(Option.scala:236) at org.apache.spark.scheduler.EventLoggingListener.logEvent(EventLoggingListener.scala:144) at org.apache.spark.scheduler.EventLoggingListener.onStageCompleted(EventLoggingListener.scala:165) at org.apache.spark.scheduler.SparkListenerBus$class.onPostEvent(SparkListenerBus.scala:32) at org.apache.spark.scheduler.LiveListenerBus.onPostEvent(LiveListenerBus.scala:31) at org.apache.spark.scheduler.LiveListenerBus.onPostEvent(LiveListenerBus.scala:31) at org.apache.spark.util.ListenerBus$class.postToAll(ListenerBus.scala:53) at org.apache.spark.util.AsynchronousListenerBus.postToAll(AsynchronousListenerBus.scala:36) at org.apache.spark.util.AsynchronousListenerBus$$anon$1$$anonfun$run$1.apply$mcV$sp(AsynchronousListenerBus.scala:76) at org.apache.spark.util.AsynchronousListenerBus$$anon$1$$anonfun$run$1.apply(AsynchronousListenerBus.scala:61) at org.apache.spark.util.AsynchronousListenerBus$$anon$1$$anonfun$run$1.apply(AsynchronousListenerBus.scala:61) at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1617) at org.apache.spark.util.AsynchronousListenerBus$$anon$1.run(AsynchronousListenerBus.scala:60) Caused by: java.io.IOException: Filesystem closed at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:792) at org.apache.hadoop.hdfs.DFSOutputStream.flushOrSync(DFSOutputStream.java:1998) at org.apache.hadoop.hdfs.DFSOutputStream.hflush(DFSOutputStream.java:1959) at org.apache.hadoop.fs.FSDataOutputStream.hflush(FSDataOutputStream.java:130) ... 19 more > Persisting Web UI through refactoring the SparkListener interface > ----------------------------------------------------------------- > > Key: SPARK-1132 > URL: https://issues.apache.org/jira/browse/SPARK-1132 > Project: Spark > Issue Type: Improvement > Components: Spark Core, Web UI > Affects Versions: 0.9.0 > Reporter: Andrew Or > Assignee: Andrew Or > Priority: Blocker > Fix For: 1.0.0 > > > This issue is a spin-off from another issue - > https://spark-project.atlassian.net/browse/SPARK-969 > The main issue with the existing Spark Web UI is that its information is lost > as soon as the application terminates. This is the direct result of the > SparkUI being coupled with SparkContext, which is stopped when the > application is finished. > The attached document proposes to tackle this by logging SparkListenerEvents > to persist information displayed on the Web UI. We take this opportunity to > replace the existing format for storing this information, HTML, with one that > is more flexible, JSON. This allows further post-hoc analysis of a particular > Spark application beyond simply reviving the Web UI. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org