On Tue, Jul 28, 2020 at 04:19:56PM +0200, Peter Krempa wrote: > On Tue, Jul 28, 2020 at 15:08:11 +0100, Daniel Berrange wrote: > > On Tue, Jul 28, 2020 at 04:04:39PM +0200, Pavel Hrdina wrote: > > > On Tue, Jul 28, 2020 at 03:48:29PM +0200, Peter Krempa wrote: > > > > On Thu, Jul 16, 2020 at 11:59:38 +0200, Pavel Hrdina wrote: > > > > > Signed-off-by: Pavel Hrdina <[email protected]> > > > > > --- > > > > > build-aux/Makefile.in | 9 +++ > > > > > .../Makefile.nonreentrant | 0 > > > > > build-aux/meson.build | 30 +++++++++ > > > > > build-aux/syntax-check.mk | 62 > > > > > +++++++++---------- > > > > > meson.build | 2 + > > > > > 5 files changed, 72 insertions(+), 31 deletions(-) > > > > > create mode 100644 build-aux/Makefile.in > > > > > rename Makefile.nonreentrant => build-aux/Makefile.nonreentrant > > > > > (100%) > > > > > create mode 100644 build-aux/meson.build > > > > > > > > [...] > > > > > > > > > +make_prog = find_program('make') > > > > > + > > > > > +# There is no way how to pass value to -j using run_target so let's > > > > > use > > > > > +# it without any value to run all tests in parallel. > > > > > +run_target( > > > > > + 'syntax-check', > > > > > + command: [ > > > > > + make_prog, '-C', meson.current_build_dir(), '-j', 'syntax-check', > > > > > + ], > > > > > > > > While I do run syntax check with unlimited '-j'. I don't think it's > > > > entirely cool to impose that on everybody. Specifically overcommiting > > > > the system is not cool. Since meson is automagically scaling can't we > > > > use the meson-detected cpu number here? > > > > > > Unfortunately no, that was the first thing I was trying to figure out > > > by going through meson code as well. It's not ideal I know. > > > > > > Other options are to not use -j at all which is no-go or we can add some > > > code to detect the available number of CPUs. But again it would not > > > reflect the fact if user runs 'ninja -j N'. > > > > In libosinfo we put "syntax-check" as part of the unit tests, rather > > than as a separate meson target. With that you don't need to pass > > -j to syntax-check, because other unit tests are running in parallel > > already, and chances are syntax-check will finish first even when > > serialized. > > Unfortunately it's not even close. > > Serialized syntax-check: > real 0m22.139s > user 0m24.209s > sys 0m6.788s > > test suite: > real 0m4.833s > user 0m12.408s > sys 0m3.918s > > syntax-check with -j == ncpus: (24 thread box) > > real 0m2.099s > user 0m28.558s > sys 0m7.739s > > > As said, I'm a big fan of -jncpus or -j. so I really want to keep it > especially given the data above, but on the other hand I don't want to > set the CI boxes on fire.
So I was trying to figure out what to do with our syntax-check and this
could be one solution:
rc = run_command(
'sed', '-n',
's/^\\(sc_[a-zA-Z0-9_-]*\\):.*/\\1/p',
meson.current_source_dir() / 'syntax-check.mk',
check: true,
)
sc_tests = rc.stdout().strip().split()
This is how syntax-check.mk gets the list of targets to run for
syntax-check target. We can use the same list to define tests like this:
foreach target : sc_tests
rc = run_command(
python3_prog, '-c',
'print("@0@".replace("sc_", ""))'.format(target),
check: true,
env: runutf8,
)
name = rc.stdout().strip()
test(
name,
make_prog,
args: [ '-C', meson.current_build_dir(), target ],
depends: [
potfiles_dep,
],
suite: 'syntax-check',
)
endforeach
It could be simplified if we don't care that all the syntax-check tests
would have 'sc_' prefix.
To use it with meson/ninja would be possible with the following
commands:
ninja test
- will run all tests regardless of the test suite, it is not
possible to specify test suite
meson test
- same as ninja test
meson test --no-suite syntax-check
- this would be equivalent to make check
meson test --suite syntax-check
- this would be equivalent to make syntax-check
In addition we can add `suite: 'check'` to the remaining test() calls to
make `meson test --suite check` available as well.
With this change running `meson test --suite syntax-check` takes on my
machine 3.037s compared to `ninja syntax-check` which takes 2.256s.
Pavel
signature.asc
Description: PGP signature
