Hi,
More cores without getting memory per core ratio correct can result in more
queuing and hence more contention as was evident from the earlier published
results
I had a bit of discussion with one of the spark experts who stated/claimed
one should have one executor per server and then get
ct: Re: Spark standalone workers, executors and JVMs
Hi Mohammed,
Thanks for your reply. I agree with you, however a single application can use
multiple executors as well, so I am still not clear which option is best. Let
me make an example to be a little more concrete.
Let's say I am only r
Hi Mohammed,
Thanks for your reply. I agree with you, however a single application can
use multiple executors as well, so I am still not clear which option is
best. Let me make an example to be a little more concrete.
Let's say I am only running a single application. Let's assume again that I
The workers and executors run as separate JVM processes in the standalone mode.
The use of multiple workers on a single machine depends on how you will be
using the clusters. If you run multiple Spark applications simultaneously, each
application gets its own its executor. So, for example, if
