Re: Moving away from cron jobs to some workflow manager

2018-06-19 Thread Lior Okman
Hi, You could try something like Concourse ( https://concourse-ci.org/ ). It allows you to define a pipeline which is comprised of jobs and the order in which they should be invoked. -- Lior On Tue, Jun 19, 2018 at 9:06 AM, Rabin Yasharzadehe wrote: > Hi all, > > I need some advice, currently

Re: Moving away from cron jobs to some workflow manager

2018-06-19 Thread Steve Litt
On Tue, 19 Jun 2018 09:42:35 +0300 Omer Zak wrote: > For dependency management, you may want to use 'make' or modern Hi Omer, While corresponding with someone offlist, I had another idea maybe as good as using make. I could make a customized installation of the process supervisor part of either

Re: Moving away from cron jobs to some workflow manager

2018-06-19 Thread Steve Litt
On Tue, 19 Jun 2018 12:25:21 +0300 Dimid Duchovny wrote: > Hi Rabin, > > > I'm far from being a linux expert, but isn't dependency between > services handled by systemd? > E.g. https://wiki.archlinux.org/index.php/Systemd/Timers If you drive on that side of the road :-) More seriously, I thin

Re: Moving away from cron jobs to some workflow manager

2018-06-19 Thread Steve Litt
On Tue, 19 Jun 2018 09:42:35 +0300 Omer Zak wrote: > For dependency management, you may want to use 'make' If you can depend on each task to create specific files, yeah, that sounds like a great idea. I should have thought of it. And then you just put it in a loop so things are always being pr

Re: Moving away from cron jobs to some workflow manager

2018-06-19 Thread linux.il
I suggest to check Jenkins (as already suggested) and Rundeck. Vitaly On Tue, Jun 19, 2018 at 9:06 AM, Rabin Yasharzadehe wrote: > Hi all, > > I need some advice, currently I have a huge cron file which schedules > tasks one after anther, and each task is position precisely (with some room > for

Re: Moving away from cron jobs to some workflow manager

2018-06-19 Thread Moish
try GNUbatch On 19/06/2018 09:06, Rabin Yasharzadehe wrote: Hi all, I need some advice, currently I have a huge cron file which schedules tasks one after anther, and each task is pos

Re: Moving away from cron jobs to some workflow manager

2018-06-19 Thread Omer Zak
1. Execution time limits: Ansible has async with polling intervals. I did not research for methods to kill hung tasks. https://docs.ansible.com/ansible/latest/user_guide/playbooks_async.html 2. Dashboard-like functionality According to: https://www.reddit.com/r/ansible/comments/5ksphc/best_web_g

Re: Moving away from cron jobs to some workflow manager

2018-06-19 Thread Ari Becker
Hi Rabin, Did you consider using Jenkins? It may be a little heavyweight, but it should be relatively easy to set up and configure. You can use the same scripts you're using today, the ability to state which jobs run on which nodes, set up dependencies between them, set timeouts, set cron triggers

Re: Moving away from cron jobs to some workflow manager

2018-06-19 Thread Rabin Yasharzadehe
systemd is a complete different tool, which was not designed for this kinda purpose. (maybe in the future it will grow to be something like that ;-) ) I'm looking for something a bit more sophisticated then "go to this machine" and "run this script" and "expect this result" i like to define execut

Re: Moving away from cron jobs to some workflow manager

2018-06-19 Thread Dimid Duchovny
Hi Rabin, I'm far from being a linux expert, but isn't dependency between services handled by systemd? E.g. https://wiki.archlinux.org/index.php/Systemd/Timers HTH 2018-06-19 12:12 GMT+03:00 Moish : > Try GNUbatch. > > > On June 19, 2018 9:42:35 AM GMT+03:00, Omer Zak wrote: > >> For dependen

Re: Moving away from cron jobs to some workflow manager

2018-06-19 Thread Moish
Try GNUbatch. On June 19, 2018 9:42:35 AM GMT+03:00, Omer Zak wrote: >For dependency management, you may want to use 'make' or modern >equivalents ('ant', 'gradle', etc.). >For controlling remote nodes, 'ansible' may be able to do the work. > >--- Omer Zak > > >On Tue, 2018-06-19 at 09:06 +0300

Re: Moving away from cron jobs to some workflow manager

2018-06-19 Thread Rabin Yasharzadehe
I'll have to read the documentation to learn more, but this project seems barely maintained as only minor versions each year or two (last release was 2 years ago), that doesn't give a lot of confidence. but i'll check it out thanks. -- Rabin On Tue, 19 Jun 2018 at 09:43, Marc Volovic wrote: >