Batch Scheduler with dependency support

2015-05-13 Thread Aaron Carey
Hi All, I was just wondering if anyone out there knew of a good mesos batch scheduler which supports dependencies between tasks? (ie Task B cannot run until Task A is complete) Thanks, Aaron

Batch Scheduler with dependency support

2015-05-13 Thread Jeff Schroeder
> thanks, > Aaron > > -- > *From:* jeffschr...@gmail.com [jeffschr...@gmail.com] on behalf of Jeff > Schroeder [jeffschroe...@computer.org] > *Sent:* 13 May 2015 13:12 > *To:* user@mesos.apache.org > *Subject:* Re: Batch Scheduler with dependency support > > Apa

Re: Batch Scheduler with dependency support

2015-05-13 Thread craig mcmillan
i've been using https://github.com/mesos/chronos :c On 13 May 2015, at 12:06, Aaron Carey wrote: Hi All, I was just wondering if anyone out there knew of a good mesos batch scheduler which supports dependencies between tasks? (ie Task B cannot run until Task A is complete) Thanks, Aaron

RE: Batch Scheduler with dependency support

2015-05-13 Thread Aaron Carey
t: Re: Batch Scheduler with dependency support i've been using https://github.com/mesos/chronos :c On 13 May 2015, at 12:06, Aaron Carey wrote: > Hi All, > > I was just wondering if anyone out there knew of a good mesos batch > scheduler which supports dependencies between tasks? (

Re: Batch Scheduler with dependency support

2015-05-13 Thread craig mcmillan
nks! Aaron From: craig mcmillan [mccraigmccr...@gmail.com] Sent: 13 May 2015 12:09 To: user@mesos.apache.org Subject: Re: Batch Scheduler with dependency support i've been using https://github.com/mesos/chronos :c On 13 May 2015, at 12:06, Aaron Carey wrote: Hi A

Re: Batch Scheduler with dependency support

2015-05-13 Thread Jeff Schroeder
Apache Aurora does this and you can be explicit about the ordering On Wednesday, May 13, 2015, Aaron Carey wrote: > Hi All, > > I was just wondering if anyone out there knew of a good mesos batch > scheduler which supports dependencies between tasks? (ie Task B cannot run > until Task A is comp

RE: Batch Scheduler with dependency support

2015-05-13 Thread Aaron Carey
2015 13:12 To: user@mesos.apache.org Subject: Re: Batch Scheduler with dependency support Apache Aurora does this and you can be explicit about the ordering On Wednesday, May 13, 2015, Aaron Carey mailto:aca...@ilm.com>> wrote: Hi All, I was just wondering if anyone out there knew of a good

RE: Batch Scheduler with dependency support

2015-05-13 Thread Aaron Carey
Thanks Jeff, Any other options around as well? From: jeffschr...@gmail.com [jeffschr...@gmail.com] on behalf of Jeff Schroeder [jeffschroe...@computer.org] Sent: 13 May 2015 14:12 To: user@mesos.apache.org Subject: Batch Scheduler with dependency support It

Re: Batch Scheduler with dependency support

2015-05-13 Thread Jeff Schroeder
hroeder [jeffschroe...@computer.org > ] > *Sent:* 13 May 2015 14:12 > *To:* user@mesos.apache.org > > *Subject:* Batch Scheduler with dependency support > > It does both just as well, along with cron-like functionality. It is > harder to install and takes a bit more understandi

RE: Batch Scheduler with dependency support

2015-05-13 Thread Aaron Carey
Thanks! I hadn't come across that one before :) From: jeffschr...@gmail.com [jeffschr...@gmail.com] on behalf of Jeff Schroeder [jeffschroe...@computer.org] Sent: 13 May 2015 16:39 To: user@mesos.apache.org Subject: Re: Batch Scheduler with dependency su

RE: Batch Scheduler with dependency support

2015-05-13 Thread Andras Kerekes
You might want to have a look at stolos too: https://github.com/sailthru/stolos Andras From: Aaron Carey [mailto:aca...@ilm.com] Sent: Wednesday, May 13, 2015 11:54 AM To: user@mesos.apache.org Subject: RE: Batch Scheduler with dependency support Thanks! I hadn't come a

Re: Batch Scheduler with dependency support

2015-05-13 Thread Tim Chen
stolos > > > > Andras > > > > > > *From:* Aaron Carey [mailto:aca...@ilm.com] > *Sent:* Wednesday, May 13, 2015 11:54 AM > *To:* user@mesos.apache.org > *Subject:* RE: Batch Scheduler with dependency support > > > > Thanks! I hadn't come across that one

Re: Batch Scheduler with dependency support

2015-05-13 Thread Alex Gaudio
gt; >> >> https://github.com/sailthru/stolos >> >> >> >> Andras >> >> >> >> >> >> *From:* Aaron Carey [mailto:aca...@ilm.com] >> *Sent:* Wednesday, May 13, 2015 11:54 AM >> *To:* user@mesos.apache.org >> *Subject:* RE: Bat

Re: Batch Scheduler with dependency support

2015-05-13 Thread Alex Gaudio
t 9:46 AM, Andras Kerekes < >> andras.kere...@ishisystems.com> wrote: >> >>> You might want to have a look at stolos too: >>> >>> >>> >>> https://github.com/sailthru/stolos >>> >>> >>> >>> Andras >>> >

Re: Batch Scheduler with dependency support

2015-05-13 Thread Tim Chen
015 at 9:46 AM, Andras Kerekes < >> andras.kere...@ishisystems.com> wrote: >> >>> You might want to have a look at stolos too: >>> >>> >>> >>> https://github.com/sailthru/stolos >>> >>> >>> >>> Andras >&

Re: Batch Scheduler with dependency support

2015-05-13 Thread Tim St Clair
iginal Message - > From: "Alex Gaudio" > To: user@mesos.apache.org > Sent: Wednesday, May 13, 2015 3:04:20 PM > Subject: Re: Batch Scheduler with dependency support > Hi Tim (and everyone else!), > I am the primary author of Stolos. We use Stolos to run all of our

Re: Batch Scheduler with dependency support

2015-05-13 Thread Tom Arnfeld
amp;HPC to chime in "on-list".  Subtle nudge ;-)  Tim From: "Alex Gaudio" To: user@mesos.apache.org Sent: Wednesday, May 13, 2015 3:04:20 PM Subject: Re: Batch Scheduler with dependency support Hi Tim (and everyone else!), I am the primary author of Stolos.  We use St

Re: Batch Scheduler with dependency support

2015-05-13 Thread Sharma Podila
in HTC&HPC to chime in > "on-list". > > Subtle nudge ;-) > Tim > > -- > > *From: *"Alex Gaudio" > *To: *user@mesos.apache.org > *Sent: *Wednesday, May 13, 2015 3:04:20 PM > > *Subject: *Re: Batch Scheduler with dependency support > >

Re: Batch Scheduler with dependency support

2015-05-13 Thread Alex Gaudio
gt; >> http://pegasus.isi.edu/ , http://ccl.cse.nd.edu/software/makeflow/ >> >> ​​ >> I keep longing for folks with decades of experience in HTC&HPC to chime >> in "on-list". >> >> Subtle nudge ;-) >> Tim >> >> --

RE: Batch Scheduler with dependency support

2015-05-14 Thread Aaron Carey
Thanks Andras! This is very interesting, it comes quite close to what we're looking for, Thanks, Aaron From: Andras Kerekes [andras.kere...@ishisystems.com] Sent: 13 May 2015 17:46 To: user@mesos.apache.org Subject: RE: Batch Scheduler with dependency su

RE: Batch Scheduler with dependency support

2015-05-14 Thread Aaron Carey
erent executables though. Aaron From: Tim Chen [t...@mesosphere.io] Sent: 13 May 2015 19:01 To: user@mesos.apache.org Subject: Re: Batch Scheduler with dependency support How are you running your batch jobs? Is the batch job script/executable an in-house app? Tim On Wed

Re: Batch Scheduler with dependency support

2015-05-14 Thread Douglas Thain
> >>>> How are you running your batch jobs? Is the batch job script/executable >>>> an in-house app? >>>> >>>> Tim >>>> >>>> On Wed, May 13, 2015 at 9:46 AM, Andras Kerekes >>>> wrote: &

RE: [Junk released by User action] Re: Batch Scheduler with dependency support

2015-05-14 Thread Aaron Carey
Subject: [Junk released by User action] Re: Batch Scheduler with dependency support ​I keep longing for folks with decades of experience in HTC&HPC to chime in "on-list". FWIW, I come from that background, but, am not in that space at this time. My prior life was in developing a (n

Re: [Junk released by User action] Re: Batch Scheduler with dependency support

2015-05-14 Thread Sharma Podila
t; *From:* Sharma Podila [spod...@netflix.com] > *Sent:* 13 May 2015 23:49 > *To:* user@mesos.apache.org > *Cc:* Douglas Thain; Brian Bockelman > *Subject:* [Junk released by User action] Re: Batch Scheduler with > dependency support > >​I keep longing for folks with decad

Re: [Junk released by User action] Re: Batch Scheduler with dependency support

2015-05-14 Thread Pablo Cingolani
gt;> >> -- >> *From:* Sharma Podila [spod...@netflix.com] >> *Sent:* 13 May 2015 23:49 >> *To:* user@mesos.apache.org >> *Cc:* Douglas Thain; Brian Bockelman >> *Subject:* [Junk released by User action] Re: Batch Scheduler with >> d