Re: How to make use of a custom executor?

Thank you for this, it's very helpful. One question, I've been launching two tasks via my custom executor (which just launches stress at the moment), and according to the webui, the second task is being registered as killed instead of finished. Why does it do this? Here's

Re: How to make use of a custom executor?

eduler and executor up and > running using the built in examples, but the examples don't show how to > activate the custom executor on the slaves. Manually executing it using the > command executor and launching the custom executor with java -jar seems to > work somewhat, but

How to make use of a custom executor?

n the sandbox will be uploaded to another location. I've got a scheduler and executor up and running using the built in examples, but the examples don't show how to activate the custom executor on the slaves. Manually executing it using the command executor and launching the custom exe

Re: Return a vector value from custom executor

wrote: >> >>> Hi all >>> >>> What is the best practice to have a custom executor to return large size >>> (~200mb) value (e.g. matrix array, or numpy array) ? >>> >>> B.R >>> Bryan >>> >> >> >> >> -- >> Best Regards, >> Haosdent Huang >> > >

Re: Return a vector value from custom executor

st > notify framework the progress of store. > > On Tue, Jul 5, 2016 at 2:17 PM, Bryan Fok > wrote: > >> Hi all >> >> What is the best practice to have a custom executor to return large size >> (~200mb) value (e.g. matrix array, or numpy array) ? >> >> B.R >> Bryan >> > > > > -- > Best Regards, > Haosdent Huang >

Re: Return a vector value from custom executor

I afraid protobuf don't allow so large message size? I think you may store your result in HDFS or other distributed filesystem. And executor just notify framework the progress of store. On Tue, Jul 5, 2016 at 2:17 PM, Bryan Fok wrote: > Hi all > > What is the best practice to

Return a vector value from custom executor

Hi all What is the best practice to have a custom executor to return large size (~200mb) value (e.g. matrix array, or numpy array) ? B.R Bryan

Re: Improvements to container support with custom executor

os' own command executor (MESOS-3004). > > > > I've proposed a solution in this doc. The main idea is that we allow the > custom executor to still run under the host filesystem (so that it does not > have to deal with dependency issues). Mesos will provision the image (the

Re: Improvements to container support with custom executor

ainerizer and want to run a task > (using command executor) under a specified image, we have the same issue for > Mesos' own command executor (MESOS-3004). > > I've proposed a solution in this doc. The main idea is that we allow the > custom executor to still run under th

Re: Improvements to container support with custom executor

ps://docs.google.com/document/d/16hyLVRL0nz-KBts1J5stGyxZPniFPbPbs7R-ZRQVCH4/edit?usp=sharing>. The main idea is that we allow the custom executor to still run under the host filesystem (so that it does not have to deal with dependency issues). Mesos will provision the image (the image request

Improvements to container support with custom executor

(cross-posting for a wider audience) Hi folks, With mesos-0.23.0 it looks like a new dependency made it in for TLS support. While this is fine in theory it actually makes the Docker Containerizer story very difficult to reason about. Here's the situation: Aurora uses a custom Python executor for

Re: Custom executor

To clarify, if you didn't already know, your custom executor should implement the Mesos executor interface. On Wed, Jul 29, 2015 at 10:05 AM, Ondrej Smola wrote: > As Connor already mentioned you can use URL for your artifact/app - there > is whole REST API - artifact store - for w

Re: Custom executor

ol! Will that run as one instance per task, or one scheduler per > slave? > > > -- > *From:* Connor Doyle [connor@gmail.com] > *Sent:* 29 July 2015 17:24 > *To:* user@mesos.apache.org > *Subject:* Re: Custom executor > > You don't ev

Re: Custom executor

ote: > > ah cool! Will that run as one instance per task, or one scheduler per slave? > > > From: Connor Doyle [connor@gmail.com] > Sent: 29 July 2015 17:24 > To: user@mesos.apache.org > Subject: Re: Custom executor > > You don't even have to pre-load the e

RE: Custom executor

ah cool! Will that run as one instance per task, or one scheduler per slave? From: Connor Doyle [connor@gmail.com] Sent: 29 July 2015 17:24 To: user@mesos.apache.org Subject: Re: Custom executor You don't even have to pre-load the executor on the slave

Re: Custom executor

> putting it on the slave, and supplying the path to it in the JSON? > > Thanks! > > Aaron > > From: Ondrej Smola [ondrej.sm...@gmail.com] > Sent: 29 July 2015 10:13 > To: user@mesos.apache.org > Subject: Re: Custom executor > > Hi Aaron, > > custom

RE: Custom executor

Many thanks! :) From: haosdent [haosd...@gmail.com] Sent: 29 July 2015 15:21 To: user@mesos.apache.org Subject: Re: Custom executor Hi, @Aaron Sorry the incorrect conclusion about executor in Marathon, Chronos. According this link http://stackoverflow.com

Re: Custom executor

> executable, putting it on the slave, and supplying the path to it in the > JSON? > > Thanks! > > Aaron > > -- > *From:* Ondrej Smola [ondrej.sm...@gmail.com] > *Sent:* 29 July 2015 10:13 > > *To:* user@mesos.apache.org > *Subj

RE: Custom executor

: Custom executor Hi Aaron, custom executor should be supported by Marathon - i dont use it but from tests in https://github.com/mesosphere/marathon/blob/master/src/test/scala/mesosphere/mesos/TaskBuilderTest.scala#L236 there is a option to specify path to custom executor. https

Re: Custom executor

Hi Aaron, custom executor should be supported by Marathon - i dont use it but from tests in https://github.com/mesosphere/marathon/blob/master/src/test/scala/mesosphere/mesos/TaskBuilderTest.scala#L236 there is a option to specify path to custom executor. https://mesosphere.github.io/marathon

RE: Custom executor

t find a simple solution to this using the existing architecture.. I'd love to know your thoughts though! Thanks, Aaron From: Tim Chen [t...@mesosphere.io] Sent: 28 July 2015 19:01 To: user@mesos.apache.org Subject: Re: Custom executor Can you explain what

RE: Custom executor

Hi All, haosdent says that using a custom executor in Chronos and Marathon would require changing their code, but Sargun Dhilon suggests that this isn't necessary.. Anyone know which is correct? Perhaps with an example? Thanks! Aaron From: haosdent [

Re: Custom executor

Can you explain what your motivations are and what your new custom executor will do? Tim On Tue, Jul 28, 2015 at 5:08 AM, Aaron Carey wrote: > Hi, > > Is it possible to build a custom executor which is not associated with a > particular scheduler framework? I want to be able to wr

Re: Custom executor

Hi, @Araon If you want to develop your custom framework, you could checkout this document https://github.com/apache/mesos/blob/master/docs/app-framework-development-guide.md first. > I want to be able to write a custom executor which is available to multiple schedulers (eg Marathon, Chronos

Re: Custom executor

r executor accessible to multiple frameworks is as simple as multiple frameworks referring to that executor in TaskInfo at launch time. Sent from my iPhone On Jul 28, 2015, at 05:08, Aaron Carey wrote: Hi, Is it possible to build a custom executor which is not associated with a particular scheduler fr

Custom executor

Hi, Is it possible to build a custom executor which is not associated with a particular scheduler framework? I want to be able to write a custom executor which is available to multiple schedulers (eg Marathon, Chronos and our own custom scheduler). Is this possible? I couldn't quite figur

Re: Custom Executor issue, killTask, task LOST

Thanks for the answer David, I ended up doing exactly that. On 23 July 2015 at 17:13, David Greenberg wrote: > It's the responsibility of each executor to shut itself down when it > decides its no longer needed. The idea here is that an executor can stay > running with zero tasks, so that when t

Re: Custom Executor issue, killTask, task LOST

It's the responsibility of each executor to shut itself down when it decides its no longer needed. The idea here is that an executor can stay running with zero tasks, so that when the scheduler decides to launch a task, the scheduler's already initialized, thus saving startup time. To avoid the pr

Custom Executor issue, killTask, task LOST

Hello, *TL;DR*: What is the proper way to kill executor process(tree) from within executor code? I have an executor written in python that I use with Marathon to launch my tasks. I use it to have a control on task status switch STAGING => RUNNING. I launch a long-running subprocess from the `laun