For that you need SPARK-1537 and the patch to go with it
It is still the spark web UI, it just hands off storage and retrieval of the
history to the underlying Yarn timeline server, rather than through the
filesystem. You'll get to see things as they go along too.
If you do want to try it,
Hi all,
I wonder if anyone has used use MapReduce Job History to show Spark jobs.
I can see my Spark jobs (Spark running on Yarn cluster) on Resource manager
(RM).
I start Spark History server, and then through Spark's web-based user
interface I can monitor the cluster (and track cluster and
