Quick clarification: you'll have a single `process::Executor` and queue up all the rmdirs on that, correct? So you'll still tie up a worker thread, but only one of them.
Either way it makes sense to add `process::Executor::execute()`. I'm happy to shepherd that for you Chun, send me a patch! On Mon, Sep 11, 2017 at 7:32 PM, Chun-Hung Hsiao <chhs...@mesosphere.io> wrote: > Hi, > > I'm thinking about extending `process::Executor` with a new `execute()` > interface. > The need of this new interface surfaced when I'm working on > https://issues.apache.org/jira/browse/MESOS-7964 > Summary: > 1. A disk GC might execute multiple `rmdirs` callbacks, and some of them > are heavy duty. We don't want to run them on `GarbageCollectorProcess` so > that it won't block other events of the process. > Currently we do the following: > async(rmdirs).onAny(...); > 2. `async` puts each `rmdir` callback in an actor. When there are many > heavy-duty `rmdirs` callbacks, the actors end up occupying all worker > threads and blocking other actors for minutes. > > Yan suggested me to use `process::Executor` such that: > 1. The `rmdirs` callbacks are not executed on `GarbaceGollectorProcess` > 2. All `rmdirs` callbacks are executed on a single thread > Since the `Executor` class only contains a `defer()` function that returns > a `_Deferred` structure, > I'm doing the following: > executor.defer(rmdirs).operator std::function<Future<Nothing>( > )>()().onAny(...) > > Would it make sense to add another `execute()` function to directly return > a `Future`? > > - Chun-Hung > > -- Benjamin Hindman Founder of Mesosphere and Co-Creator of Apache Mesos Mesosphere Inc. <http://www.mesosphere.io/> Follow us on Twitter: @mesosphere <http://twitter.com/mesosphere>
