Hi all, I wonder if anyone has used use MapReduce Job History to show Spark jobs.
I can see my Spark jobs (Spark running on Yarn cluster) on Resource manager (RM). I start Spark History server, and then through Spark's web-based user interface I can monitor the cluster (and track cluster and job statistics). Basically Yarn RM gets linked to Spark History server, which enables monitoring. But instead of using Spark History Server , is it possible to see Spark jobs on MapReduce job history ? (in addition to seeing them on RM) (I know through yarn logs -applicationId <app ID> we can get all logs after Spark job has completed, but my concern is to see the logs and completed jobs through common web ui - MapReduce Job History ) Thanks in advance.
