I was trying to profile some Spark jobs and I want to collect Java Flight
Recorder(JFR) files from each executor. I am running my job on a YARN
cluster with several nodes, so I cannot manually collect JRF file for each
run.
MR provides a way to name JFR files generated by each task with taskId.
Hi All,
I am new to Spark and I am trying to understand how preemption works with
Spark on Yarn. My goal is to determine amount of re-work a Spark application
has to do if an executor is preempted.
For my test, I am using a 4 node cluster with Cloudera VM running Spark
1.3.0. I am running
