This sounds like a good idea but is confusing to me. We use stages so if anything fails in stage 2 we don't waste compute time on anything in stage 3 or higher.
If one uses this needs how does that affect this? Will it run everything to the end despite failures in earlier stages or will it stop any more higher stage jobs from starting if there is a failure on a stage? So for example stage 2 is running and because of needs job x of stage 3 starts. Now stage 2 has a failed job, will no more jobs in stage 3 be started? Ideally if anything fails in stage 2 it kills the jobs started in stage 3 but I guess it doesn't do that. Barry > On Jul 6, 2020, at 1:15 PM, Jacob Faibussowitsch <jacob....@gmail.com> wrote: > >> If we did this in the pipeline they could be significantly faster > We might also consider not locking _all_ stage n+1 jobs behind completion of > stage n. Gitlab supports this via the “needs” command > https://gitlab.com/help/ci/yaml/README.md#needs > <https://gitlab.com/help/ci/yaml/README.md#needs> which allows you to run > jobs immediately after their prerequisite jobs have finished regardless of > stage. For example, I find that I am quite often waiting on stage 2 freebsd > jobs to start in order to advance to stage 3, even after every other stage 2 > job has run its course. > > Best regards, > > Jacob Faibussowitsch > (Jacob Fai - booss - oh - vitch) > Cell: (312) 694-3391 > >> On Jul 6, 2020, at 12:47 PM, Barry Smith <bsm...@petsc.dev >> <mailto:bsm...@petsc.dev>> wrote: >> >> >> We should be more clever than checking just examples but also check source >> code. If, for example, only KSP is changed then tests could skip the >> sys/vec/mat levels saving time. >> >> If we did this in the pipeline they could be significantly faster; all the >> time saved not testing sys to mat for Matt's changes :-) >> >> Is there a fallacy here? >> >> Barry >> >> Need to check changes in include files as well, of course. >> >> >>> On Jul 6, 2020, at 12:13 PM, Jacob Faibussowitsch <jacob....@gmail.com >>> <mailto:jacob....@gmail.com>> wrote: >>> >>> While were on this topic, I always parse git diff —name-only $(git >>> merge-base —fork-point master) when testing. This gives you all the files >>> that are different between your branch and master. You can switch it out >>> for maint if needs be. >>> >>> Best regards, >>> >>> Jacob Faibussowitsch >>> (Jacob Fai - booss - oh - vitch) >>> Cell: (312) 694-3391 >>> >>>> On Jul 6, 2020, at 12:00 PM, Barry Smith <bsm...@petsc.dev >>>> <mailto:bsm...@petsc.dev>> wrote: >>>> >>>> >>>> I couldn't see how Jed can avoid doing the filename hacking since >>>> alltesttargets are the hacked filenames but probably I am missing >>>> something. >>>> >>>> Anyways this patch works for me based on Jed's email. >>>> >>>> Barry >>>> <since.patch> >>>> >>>>> On Jul 6, 2020, at 9:40 AM, Jed Brown <j...@jedbrown.org >>>>> <mailto:j...@jedbrown.org>> wrote: >>>>> >>>>> It'd be possible to hack the file names, but I'd rather not duplicate >>>>> that logic. >>>>> >>>>> My inclination would be to filter similar to argsearch, which requires >>>>> adding the source files to testfiles. Then you could use >>>>> >>>>> make test since=@ >>>>> >>>>> ifdef since >>>>> gitchanged := $(shell git diff --name-only $(since) src/) >>>>> endif >>>>> >>>>> Pierre Jolivet <pierre.joli...@enseeiht.fr >>>>> <mailto:pierre.joli...@enseeiht.fr>> writes: >>>>> >>>>>> Hello, >>>>>> To git/regexp enthusiasts and/or test harness people: is there some >>>>>> script/command lying around to convert a list of modified examples >>>>>> (according to git) into a list understandable by the test system? >>>>>> Thanks for your help, >>>>>> Pierre >>>>>> >>>>>> PS: here is such a list >>>>>> $ git status -uno >>>>>> On branch master >>>>>> Your branch is up to date with 'origin/master'. >>>>>> >>>>>> Changes not staged for commit: >>>>>> (use "git add <file>..." to update what will be committed) >>>>>> (use "git checkout -- <file>..." to discard changes in working directory) >>>>>> >>>>>> modified: config/BuildSystem/config/types.py >>>>>> modified: include/petscsys.h >>>>>> modified: src/dm/label/tutorials/ex1f90.F90 >>>>>> modified: src/ksp/ksp/tests/ex16f.F90 >>>>>> modified: src/ksp/ksp/tutorials/ex5f.F90 >>>>>> modified: src/ksp/ksp/tutorials/ex75f.F90 >>>>>> modified: src/ksp/ksp/tutorials/ex76f.F90 >>>>>> modified: src/ksp/ksp/tutorials/ex77f.F90 >>>>>> modified: src/ksp/ksp/tutorials/ex7f.F90 >>>>>> modified: src/mat/tests/ex196f90.F90 >>>>>> modified: src/sys/tests/ex13f.F90 >>>>>> modified: src/sys/tests/ex47f.F90 >>>>>> modified: src/sys/tutorials/ex17f.F90 >>>>>> modified: src/sys/tutorials/ex2f.F90 >>>>>> modified: src/vec/vec/tutorials/ex11f90.F90 >>>>>> modified: src/vec/vec/tutorials/ex18f.F90 >>>>>> modified: src/vec/vec/tutorials/ex1f.F90 >>>>>> >>>>>> no changes added to commit (use "git add" and/or "git commit -a") >>>> >>> >> >
