Re: Storage history in web UI

[Appu K]

@jacek - thanks a lot for the book

@joe - looks like the rest api also exposes a few things like
/applications/[app-id]/storage/rdd
/applications/[app-id]/storage/rdd/[rdd-id]
that might perhaps be of interest to you ?
http://spark.apache.org/docs/latest/monitoring.html





On 9 January 2017 at 12:07:34 AM, Jacek Laskowski (ja...@japila.pl) wrote:

Hi,

A possible workaround...Use SparkListener and save the results to a custom
sink.

After all web UI is a mere bag of SparkListeners + excellent
visualizations.

Jacek

On 3 Jan 2017 4:14 p.m., "Joseph Naegele" 
wrote:

Hi all,

Is there any way to observe Storage history in Spark, i.e. which RDDs were
cached and where, etc. after an application completes? It appears the
Storage tab in the History Server UI is useless.

Thanks
---
Joe Naegele
Grier Forensics



-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org

Re: Storage history in web UI

[Jacek Laskowski]

Hi,

A possible workaround...Use SparkListener and save the results to a custom
sink.

After all web UI is a mere bag of SparkListeners + excellent
visualizations.

Jacek

On 3 Jan 2017 4:14 p.m., "Joseph Naegele" 
wrote:

Hi all,

Is there any way to observe Storage history in Spark, i.e. which RDDs were
cached and where, etc. after an application completes? It appears the
Storage tab in the History Server UI is useless.

Thanks
---
Joe Naegele
Grier Forensics



-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org

Storage history in web UI

[Joseph Naegele]

Hi all,

Is there any way to observe Storage history in Spark, i.e. which RDDs were 
cached and where, etc. after an application completes? It appears the Storage 
tab in the History Server UI is useless.

Thanks
---
Joe Naegele
Grier Forensics



-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org