Typo. We can not figure a way to increase the number of executor in one node in mesos.
On Wednesday, May 27, 2015, DB Tsai <dbt...@dbtsai.com> wrote: > If with mesos, how do we control the number of executors? In our cluster, > each node only has one executor with very big JVM. Sometimes, if the > executor dies, all the concurrent running tasks will be gone. We would like > to have multiple executors in one node but can not figure out a way to do > it in Yarn. > > On Wednesday, May 27, 2015, Saisai Shao <sai.sai.s...@gmail.com > <javascript:_e(%7B%7D,'cvml','sai.sai.s...@gmail.com');>> wrote: > >> The drive has a heuristic mechanism to decide the number of executors in >> the run-time according the pending tasks. You could enable with >> configuration, you could refer to spark document to find the details. >> >> 2015-05-27 15:00 GMT+08:00 canan chen <ccn...@gmail.com>: >> >>> How does the dynamic allocation works ? I mean does it related >>> with parallelism of my RDD and how does driver know how many executor it >>> needs ? >>> >>> On Wed, May 27, 2015 at 2:49 PM, Saisai Shao <sai.sai.s...@gmail.com> >>> wrote: >>> >>>> It depends on how you use Spark, if you use Spark with Yarn and enable >>>> dynamic allocation, the number of executor is not fixed, will change >>>> dynamically according to the load. >>>> >>>> Thanks >>>> Jerry >>>> >>>> 2015-05-27 14:44 GMT+08:00 canan chen <ccn...@gmail.com>: >>>> >>>>> It seems the executor number is fixed for the standalone mode, not >>>>> sure other modes. >>>>> >>>> >>>> >>> >> > > -- > Sent from my iPhone > -- Sent from my iPhone
