Re: Number of tasks per executor and resource limits

2020-02-21 Thread Charles-François Natali
Perfect, thanks! On Fri, 21 Feb 2020, 00:55 Qian Zhang, wrote: > if we wanted to support multiple tasks per executor, could we leverage the >> mesos containeriser to easily put each task in its own cgroup? > > > If what your custom executor does is similar with default executor (i.e., > launch

Re: Number of tasks per executor and resource limits

2020-02-20 Thread Qian Zhang
> > if we wanted to support multiple tasks per executor, could we leverage the > mesos containeriser to easily put each task in its own cgroup? If what your custom executor does is similar with default executor (i.e., launch each task in a task group as a nested container by calling Mesos agent A

Re: Number of tasks per executor and resource limits

2020-02-20 Thread Vinod Kone
Andrei, Qian: Can one of you answer the above question? On Thu, Feb 20, 2020 at 7:15 PM Charles-François Natali wrote: > Thanks for the quick reply! > > I think we're going to go for one executor per task for now, that's much > simpler. > > Otherwise I was wondering - if we wanted to support mul

Re: Number of tasks per executor and resource limits

2020-02-20 Thread Charles-François Natali
Thanks for the quick reply! I think we're going to go for one executor per task for now, that's much simpler. Otherwise I was wondering - if we wanted to support multiple tasks per executor, could we leverage the mesos containeriser to easily put each task in its own cgroup? Is it just a matte

Re: Number of tasks per executor and resource limits

2020-02-20 Thread Vinod Kone
Hi Charles, We are actually working on a new feature that puts each of the tasks (of the default executor) in its own cgroup so that they can be individually limited. See https://issues.apache.org/jira/browse/MESOS-9916 . For custom executor, you would be on your own to implement the same. Also, y

Number of tasks per executor and resource limits

2020-02-20 Thread Charles-François Natali
Hi, Is there a way to force Mesos to use one executor per task? The reason I would want to do that is for resources limits: for example, when using cgroup to limit CPU and memory, IIUC the containeriser sets limits corresponding to the sum of the resources allocated to the tasks managed by the und