Andrei, Qian: Can one of you answer the above question? On Thu, Feb 20, 2020 at 7:15 PM Charles-François Natali <cf.nat...@gmail.com> wrote:
> Thanks for the quick reply! > > I think we're going to go for one executor per task for now, that's much > simpler. > > Otherwise I was wondering - if we wanted to support multiple tasks per > executor, could we leverage the mesos containeriser to easily put each task > in its own cgroup? > > > > Is it just a matter of setting 'execu > On Thu, 20 Feb 2020, 17:40 Vinod Kone, <vinodk...@apache.org> wrote: > >> Hi Charles, >> >> We are actually working on a new feature that puts each of the tasks (of >> the default executor) in its own cgroup so that they can be individually >> limited. See https://issues.apache.org/jira/browse/MESOS-9916 . For >> custom executor, you would be on your own to implement the same. Also, your >> custom executor / scheduler should be able to limit one task per executor >> if you so desire. >> >> On Thu, Feb 20, 2020 at 6:34 PM Charles-François Natali < >> cf.nat...@gmail.com> wrote: >> >>> Hi, >>> >>> Is there a way to force Mesos to use one executor per task? >>> The reason I would want to do that is for resources limits: for example, >>> when using cgroup to limit CPU and memory, IIUC the containeriser sets >>> limits corresponding to the sum of the resources allocated to the tasks >>> managed by the underlying executor. >>> >>> Which means that for example if a task is allocated 1GB and another 2GB, >>> if they are started by the same executor, the collarbone containeriser will >>> limit the total memory for the two tasks (plus the executor) to 3GB. But, >>> unless I'm missing something, nothing prevents a process (assuming one >>> process per task with e.g. the command executor or a custom executor) from >>> using more than its limit. >>> >>> Obviously it would be possible for the executor to enforce the >>> individual limits itself using cgroup - does the command/default executor >>> do that? - but in our case where we use a custom executor it would be quite >>> painful. >>> >>> Unless I'm missing something? >>> >>> Thanks in advance for suggestions, >>> >>> Charles >>> >>
