Hi Dinuka, Sorry for the late reply. It is great to explore options to integrate Dag capabilities into the job submitter. We already have some form of HTCondor support in Airavata. Can you summarize the difference between what we already have for HTCondor and this Dag feature? I am specifically looking for practical usages instead of technical differences.
Thanks Dimuthu On Sun, Jan 1, 2023 at 10:35 AM Dinuka De Silva <l.dinukadesi...@gmail.com> wrote: > Hi, > > The current implementation of mapping these (Pre, Post, etc. job commands) > to the job scheduler script has assumed the job scheduler script to be a > type of shell script. So, the order of the execution is based on the order > of the commands listed in the script which is as below. > > - Module Commands > - Pre Job Commands > - Job Submitter Command > - Post Job Commands > > The scheduler script of SLURM, FORK, LSF, UGE, and PBS are shell scripts > while HTCondor and maybe some other job schedulers have different file > types. The script grammar in HTCondor does not support appending > shell scripts inside. Now, we are needing to support Pre, Post, and other > commands for HTCondor realizing the current design doesn't support it. > > In HTCondor there's an option [1] to configure pre and post-scripts to be > executed at the worker instance. But then the script has to be Dag and the > pre-script, post-script and job-script are to be separate files. So, I > tried a sample and planning to put this to airavata. > > [1] > https://htcondor.readthedocs.io/en/latest/users-manual/dagman-workflows.html > > Thanks & Regards, > Dinuka > > >
