Hi All, On running Concurrent Spark Jobs (huge number of tasks) with same Spark Context, there is high scheduler delay. We have FAIR schedulingPolicy set and also we tried with different pool for each jobs but still no improvement. What are the tuning ways to improve Scheduler Delay.
Thanks, Prabhu Joseph