On Mon, Dec 6, 2021 at 3:26 PM Vladimir Sementsov-Ogievskiy < vsement...@virtuozzo.com> wrote:
> 06.12.2021 21:37, John Snow wrote: > > > > > > On Fri, Dec 3, 2021 at 7:22 AM Vladimir Sementsov-Ogievskiy < > vsement...@virtuozzo.com <mailto:vsement...@virtuozzo.com>> wrote: > > > > Hi all! > > > > Finally, I can not stand it any longer. So, I'm happy to present > > multiprocessing support for iotests test runner. > > > > testing on tmpfs: > > > > Before: > > > > time check -qcow2 > > ... > > real 12m28.095s > > user 9m53.398s > > sys 2m55.548s > > > > After: > > > > time check -qcow2 -j 12 > > ... > > real 2m12.136s > > user 12m40.948s > > sys 4m7.449s > > > > > > VERY excellent. And this will probably flush a lot more bugs loose, too. > (Which I consider a good thing!) > > Thanks!) > > > We could look into utilizing it for 'make check', but we'll have to be > prepared for a greater risk of race conditions on the CI if we do. But... > it's seriously hard to argue with this kind of optimization, very well done! > > I thought about this too. I think, we can at least passthrought -j flag of > "make -j9 check" to ./check > > I think, CIs mostly call make check without -j flag. But I always call > make -j9 check. And it always upset me that all tests run in parallel > except for iotests. So if it possible to detect that we are called through > "make -j9 check" and use "-j 9" for iotests/check in this case, it would be > good. > > > > > > > Hmm, seems -j 6 should be enough. I have 6 cores, 2 threads per core. > > Anyway, that's so fast! > > > > Vladimir Sementsov-Ogievskiy (3): > > iotests/testrunner.py: add doc string for run_test() > > iotests/testrunner.py: move updating last_elapsed to run_tests > > iotests: check: multiprocessing support > > > > tests/qemu-iotests/check | 4 +- > > tests/qemu-iotests/testrunner.py | 86 > ++++++++++++++++++++++++++++---- > > 2 files changed, 80 insertions(+), 10 deletions(-) > > > > -- > > 2.31.1 > > > > > I'll also now add: Tested-by: John Snow <js...@redhat.com> I tried to find a different workaround in just a few minutes, but that just made it clear that your solution was right. While I had it checked out, I ran it a few times and it looks good to me! (And no new problems from the Python CI stuff, so it looks good to me.)
