I should also mention that I run Stolos as a Mesos executor and another tool I wrote, Relay.Mesos, as the Mesos Scheduler (that spins up Stolos Executors).
The github links for these tools are: https://github.com/sailthru/relay.mesos https://github.com/sailthru/stolos On Wed, May 13, 2015 at 4:04 PM Alex Gaudio <adgau...@gmail.com> wrote: > Hi Tim (and everyone else!), > > I am the primary author of Stolos. We use Stolos to run all of our batch > jobs on Mesos. The batch jobs are scripts we can run from the > command-line. Scripts range from bash scripts, Spark jobs and R scripts. > > It's a great tool for us because, unlike Chronos, it lets us define a > script as stage in a dependency chain, where the script can run with > different parameters for different dependency contexts. (The closest usage > of this would be to have many Chronos servers, though this does not work in > all cases). > > The tool is a critical component of Sailthru's data science > infrastructure, but I believe we are the only people who use the tool right > now. > > If you are interested in learning more, I'm happy to invest time to talk > more about Stolos, what it does and how we use it! > > Alex > > On Wed, May 13, 2015 at 2:02 PM Tim Chen <t...@mesosphere.io> wrote: > >> How are you running your batch jobs? Is the batch job script/executable >> an in-house app? >> >> Tim >> >> On Wed, May 13, 2015 at 9:46 AM, Andras Kerekes < >> andras.kere...@ishisystems.com> wrote: >> >>> You might want to have a look at stolos too: >>> >>> >>> >>> https://github.com/sailthru/stolos >>> >>> >>> >>> Andras >>> >>> >>> >>> >>> >>> *From:* Aaron Carey [mailto:aca...@ilm.com] >>> *Sent:* Wednesday, May 13, 2015 11:54 AM >>> *To:* user@mesos.apache.org >>> *Subject:* RE: Batch Scheduler with dependency support >>> >>> >>> >>> Thanks! I hadn't come across that one before :) >>> ------------------------------ >>> >>> *From:* jeffschr...@gmail.com [jeffschr...@gmail.com] on behalf of Jeff >>> Schroeder [jeffschroe...@computer.org] >>> *Sent:* 13 May 2015 16:39 >>> *To:* user@mesos.apache.org >>> *Subject:* Re: Batch Scheduler with dependency support >>> >>> Lookup Hubspot's Singularity >>> >>> On Wednesday, May 13, 2015, Aaron Carey <aca...@ilm.com> wrote: >>> >>> Thanks Jeff, >>> >>> Any other options around as well? >>> ------------------------------ >>> >>> *From:* jeffschr...@gmail.com <http://UrlBlockedError.aspx> [ >>> jeffschr...@gmail.com <http://UrlBlockedError.aspx>] on behalf of Jeff >>> Schroeder [jeffschroe...@computer.org <http://UrlBlockedError.aspx>] >>> *Sent:* 13 May 2015 14:12 >>> *To:* user@mesos.apache.org <http://UrlBlockedError.aspx> >>> *Subject:* Batch Scheduler with dependency support >>> >>> It does both just as well, along with cron-like functionality. It is >>> harder to install and takes a bit more understanding however. The official >>> tutorial is a process that loops 100 times and then exits. >>> >>> >>> >>> http://aurora.apache.org/documentation/latest/tutorial/#the-script >>> >>> Aurora is pretty much a superset of most other generic frameworks sans >>> maybe hubspot's singularity. >>> >>> >>> On Wednesday, May 13, 2015, Aaron Carey <aca...@ilm.com >>> <http://UrlBlockedError.aspx>> wrote: >>> >>> I was under the impression Aurora was for long running services? Is it >>> suitable for scheduling one of batch processes too? >>> >>> thanks, >>> Aaron >>> ------------------------------ >>> >>> *From:* jeffschr...@gmail.com [jeffschr...@gmail.com] on behalf of Jeff >>> Schroeder [jeffschroe...@computer.org] >>> *Sent:* 13 May 2015 13:12 >>> *To:* user@mesos.apache.org >>> *Subject:* Re: Batch Scheduler with dependency support >>> >>> Apache Aurora does this and you can be explicit about the ordering >>> >>> On Wednesday, May 13, 2015, Aaron Carey <aca...@ilm.com> wrote: >>> >>> Hi All, >>> >>> I was just wondering if anyone out there knew of a good mesos batch >>> scheduler which supports dependencies between tasks? (ie Task B cannot run >>> until Task A is complete) >>> >>> Thanks, >>> Aaron >>> >>> >>> >>> -- >>> Text by Jeff, typos by iPhone >>> >>> >>> >>> -- >>> Text by Jeff, typos by iPhone >>> >>> >>> >>> -- >>> Text by Jeff, typos by iPhone >>> >> >>
