An application can have only one executor at each machine or container (YARN). How many thread that each executor have is determined by the parameter "executor-cores". There are also other parameter setting method that you can specify "total- executor-cores" and each executor cores will be determined uncertain. It is strong recommend to use "executor-cores" rather than "total- executor-cores" which may cause workload unbalance of different executors.
2014-10-28 17:56 GMT+08:00 shahab <[email protected]>: > Hi, > > I am running a stand alone Spark cluster, 2 workers each has 2 cores. > I submit one Spakr application to the cluster, and I monitor the execution > process via UI ( both worker-ip:8081 and master-ip:4040) > There I can see that the application is handled by many Executors, in my > case one worker has 10 executors and the other one only one !! > > My question is what is the cardinality relation between executor process > and the submitted Spark application ? I assumed that for each application > there will be one executor process handling the all spark related tasks > (maps, filtering, reduce,...) ?! > > best, > /Shahab >
