Running 30 Spark applications at the same time is slower than one on average

[eab...@163.com]

Hi All,

I have a CDH5.16.2 hadoop cluster with 1+3 nodes(64C/128G, 1NN/RM + 3DN/NM), 
and yarn with 192C/240G. I used the following test scenario:

1.spark app resource with 2G driver memory/2C driver vcore/1 executor nums/2G 
executor memory/2C executor vcore.
2.one spark app will use 5G4C on yarn.
3.first, I only run one spark app takes 40s.
4.Then, I run 30 the same spark app at once, and each spark app takes 80s on 
average.

So, I want to know why the run time gap is so big, and how to optimize?

Thanks