Re: Storage history in web UI

@jacek - thanks a lot for the book @joe - looks like the rest api also exposes a few things like /applications/[app-id]/storage/rdd /applications/[app-id]/storage/rdd/[rdd-id] that might perhaps be of interest to you ? http://spark.apache.org/docs/latest/monitoring.html On 9 January 2017 at

Re: Storage history in web UI

Hi, A possible workaround...Use SparkListener and save the results to a custom sink. After all web UI is a mere bag of SparkListeners + excellent visualizations. Jacek On 3 Jan 2017 4:14 p.m., "Joseph Naegele" wrote: Hi all, Is there any way to observe Storage

Storage history in web UI

Hi all, Is there any way to observe Storage history in Spark, i.e. which RDDs were cached and where, etc. after an application completes? It appears the Storage tab in the History Server UI is useless. Thanks --- Joe Naegele Grier Forensics