I think it would be useful, yes, and mostly for the epilog script. In the job script itself, you are creating such files, so some of the proposed use cases are a bit tricky to get right in the way you described them. For example, if you scp these files, you are scp'ing them to their status before scp is run. Something else might happen (e.g. scp warnings) which will be added to the files after the command is run, and those would not be included. Also, the buffers might not have flushed, so the scp'ed version can be incomplete. Even worse for post-processing, which can be covered better with something like the following in the slurm script
program_that_creates_lots_of_output | tee full_output_just_in_case_is_needed.txt | post_processing_script So that the original slurm file will automatically contain the post-processing version, and the "just in case" file will contain the full log. Of course the name of the latter does not need to be hardcoded and can use things like $SLURM_JOB_ID to make it unique for each job. On Mon, Jan 22, 2024 at 12:11 AM Bjørn-Helge Mevik <b.h.me...@usit.uio.no> wrote: > I would find that useful, yes. Especially if the variables were made > available for the Prolog and Epilog scripts. > > -- > Regards, > Bjørn-Helge Mevik, dr. scient, > Department for Research Computing, University of Oslo > >
