Vinod, Thanks for the explanation. Just to clarify, by thread safe, do you also mean multiple threads can call driver.sendStatusUpdate (and other functions)?
Li On Thu, Aug 22, 2013 at 12:34 AM, Vinod Kone <[email protected]> wrote: > (1) Is executor per-host instance or per-task instance? > > Typically you run one "mesos slave" process per host. A slave can have > multiple executors running underneath it. Each executor can have multiple > tasks underneath it. On the receipt of a task, a slave launches a new > executor if no executor with TaskInfo.ExecutorInfo.executor_id is currently > alive. If an executor with that id is alive, the task is just passed to the > executor. On the other hand, if you are utilizing command executor for your > task the slave will always launch a new executor per task. > > (2) If executor is a per-host instance, I would imagine the executor >> realize tasks as thread/process in launchTask(). How would the executor >> track task status then? Since the main thread is doing driver.run(), does >> it need to have another thread check task status and call >> driver.sendStatusUpdate()? Is MesosExecutorDriver thread safe? >> >> > As you can see from above, executor is not per-host. Having said that, > yes, executor could realize running different tasks in different threads or > child processes. The executor could get the exit status of the task from > the child process exit status (for example see Command Executor) or using > shared memory if you are using threads. > > And yes your main thread will be blocked on run(), so you need another > thread to track tasks and status updates. Both the scheduler and executor > drivers are thread safe. You would only get one callback at a time. > > > >> Thanks, >> Li >> >> >> On Wed, Aug 21, 2013 at 2:11 PM, Vinod Kone <[email protected]> wrote: >> >>> Hey Li, >>> >>> While your point about better documentation is duly noted, here are the >>> answers to your specific questions. >>> >>> >>>> (1) Under TaskInfo, it says "Either ExecutorInfo or CommandInfo should >>>> be set", that's the difference? >>>> >>> >>> The difference is as follows: >>> >>> If you set 'ExecutorInfo', the mesos slave launches your custom executor >>> as described in the ExecutorInfo and passes it the task. How the executor >>> runs the task is up to the Executor. >>> >>> Instead, if you set 'CommandInfo' on the TaskInfo, the mesos slave >>> launches an in-built default executor (called 'Command Executor') and >>> passes the shell command to it. You could use this executor if all you want >>> do is run an arbitrary shell commands as your tasks. Note that the command >>> executor exits as soon as the shell command exits. In other words each new >>> task launches a new command executor. >>> >>> Command Executor is also a decent example of how to write an executor >>> that runs an arbitrary command. See src/launcher/executor.cpp >>> >>> >>>> (2) How should I pass task information (command, env, etc) to the >>>> executor? Do I put these in the TaskInfo.ExecutorInfo? Or should I put >>>> these in TaskInfo.ExecutorInfo.data? >>>> >>> >>> Task specific information should be passed via TaskInfo.CommandInfo. >>> Executor specific information should be passed via >>> TaskInfo.ExecutorInfo.CommandInfo. The CommandInfo protobuf has fields for >>> passing environment and command. >>> >>> TaskInfo.ExecutorInfo.data can be used to pass arbitrary blob of data to >>> the executor. The ExecutorInfo will be passed to the executor through the >>> registered() callback. >>> >>> >>> (3) Is framwork/mesos-submit outdated? the api it uses is different from >>>> any other examples >>>> >>>> >>> I believe it is a bit out dated. But there has been some recent effort >>> to resurrect it. Not sure about the current status. >>> >>> >>> >>>> Thanks, >>>> Li >>>> >>>> >>>> >>>> On Wed, Aug 21, 2013 at 10:37 AM, 许立剑 <[email protected]> wrote: >>>> >>>>> See include/mesos/mesos.proto. >>>>> >>>>> 许立剑 >>>>> >>>>> 爱奇艺技术产品中心云平台 >>>>> >>>>> *[image: cid:[email protected]]* >>>>> >>>>> ☎ 15652344221 | QQ: 276583295 | 12F-W061 <http://www.iqiyi.com/> >>>>> >>>>> >>>>> >>>>> 2013/8/21 Li Jin <[email protected]> >>>>> >>>>>> Hello, >>>>>> >>>>>> I am implementing a basic mesos framework(Java) and executor(Python). >>>>>> I find it quite difficult to find information about framework/executor >>>>>> protocol. Here is all the information I found >>>>>> >>>>>> Doc: >>>>>> >>>>>> >>>>>> https://github.com/apache/mesos/blob/master/docs/App-Framework-development-guide.textile >>>>>> >>>>>> This is nice but it doesn't explain things like how to construct >>>>>> TaskInfo >>>>>> >>>>>> Examples: >>>>>> >>>>>> src/examples: >>>>>> These examples are nice to get started, however, these examples don't >>>>>> show some of the basic things I would like to know (they don't pass a >>>>>> command to the executor, for instance) >>>>>> >>>>>> framework/mesos-submit: >>>>>> Mesos-submit actually passed a command to the executor which is nice. >>>>>> However the python api seems quite different from the java one and I find >>>>>> it hard to understand what it's doing and even how to reimplement this in >>>>>> Java >>>>>> >>>>>> In general, I couldn't find doc/example about all the protobuf >>>>>> (TaskInfo, ExecutorInfo, etc) objects and how to use them. >>>>>> >>>>>> I am wondering if there are more for me to look at. If not, I guess >>>>>> my best shot is take a look at the source code or Chronos >>>>>> >>>>>> Thanks, >>>>>> Li >>>>>> >>>>> >>>>> >>>> >>> >> >
<<DA9C7994-A0D6-4183-9C45-1ECA2C91A74F.png>>
