Right, I can override the root pool in configuration file, Thanks Mark. On Wed, Jan 6, 2016 at 8:45 AM, Mark Hamstra <m...@clearstorydata.com> wrote:
> Just configure <pool name="default"> with > <schedulingMode>FAIR</schedulingMode> in fairscheduler.xml (or > in spark.scheduler.allocation.file if you have over-riden the default name > for the config file.) `buildDefaultPool()` will only build the pool named > "default" with the default properties (such as schedulingMode = > DEFAULT_SCHEDULING_MODE -- i.e. SchedulingMode.FAIR) if that pool name is > not already built ( > https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/scheduler/SchedulableBuilder.scala#L90 > ). > > > On Tue, Jan 5, 2016 at 4:15 PM, Jeff Zhang <zjf...@gmail.com> wrote: > >> Sorry, I don't make it clearly. What I want is the default pool is fair >> scheduling. But seems if I want to use fair scheduling now, I have to set >> spark.scheduler.pool explicitly. >> >> On Wed, Jan 6, 2016 at 2:03 AM, Mark Hamstra <m...@clearstorydata.com> >> wrote: >> >>> I don't understand. If you're using fair scheduling and don't set a >>> pool, the default pool will be used. >>> >>> On Tue, Jan 5, 2016 at 1:57 AM, Jeff Zhang <zjf...@gmail.com> wrote: >>> >>>> >>>> It seems currently spark.scheduler.pool must be set as localProperties >>>> (associate with thread). Any reason why spark.scheduler.pool can not be >>>> used globally. My scenario is that I want my thriftserver started with >>>> fair scheduler as the default pool without using set command to set the >>>> pool. Is there anyway to do that ? Or do I miss anything here ? >>>> >>>> -- >>>> Best Regards >>>> >>>> Jeff Zhang >>>> >>> >>> >> >> >> -- >> Best Regards >> >> Jeff Zhang >> > > -- Best Regards Jeff Zhang
