Hi all,
Short version:
Install the latest ruby2.5-rubygem-parallel_tests ruby2.5-rubygem-parallel_tests
packages from YaST:Head OBS and enjoy lightning fast "rake test:unit"
in yast2-storage-ng :-)
(The "devel_yast" pattern has been adapted, running "zypper dup" should be
enough.)
Long version:
I have spent some time speeding up the unit tests in yast2-storage-ng package.
The
reason was that running all tests took several minutes, about 3 minutes even on
a quite fast CPU. So after doing even a small change you would have to wait
several minutes to see whether you did not break something by mistake.
The main bottleneck was the sequential start of the tests, so even on a quad
core CPU
only one CPU was actually used.
I have added parallel_test gem [1] support to "rake test:unit" and turned that
on in yast2-storage-ng package. (By default the standard sequential RSpec is
used.)
It uses all available CPUs and scales almost linearly, the speed up is about
`nproc` times.
A similar approach I have used in Travis, originally we used license check,
Rubocop,
yardoc, unit tests, package build, ... everything sequentially in one Travis
job.
Fortunately Travis supports defining multiple jobs which run independently and
are
started in parallel.
I have split the Travis work in yast2-storage-ng into three groups: unit tests,
rubocop and the rest (yardoc, package build...).
If you want to use a similar solution in your package then check the
documentation
I wrote about this: the Travis [2], the unit tests [3].
Some real numbers to see the improvement:
- Running "rake test:unit" locally: from 2:47s to 0:38s (speedup ~4.3x on a quad
core CPU with HT on)
- Similar speed up in "rake osc:build" (it just additionally prepares the
chroot)
- OBS package build: from 323s-505s to 102s-235s
(checked several builds, it highly depends on the speed of the worker)
- Travis speed up: from 8-10 minutes to 3-4 minutes (parallel jobs + parallel
tests)
Enjoy!
[1] https://github.com/grosser/parallel_tests
[2]
http://yastgithubio.readthedocs.io/en/latest/travis-integration/#parallel-build-jobs
[3]
http://yastgithubio.readthedocs.io/en/latest/how-to-write-tests/#parallel-tests
--
Best Regards
Ladislav Slezák
Yast Developer
------------------------------------------------------------------------
SUSE LINUX, s.r.o. e-mail: [email protected]
Lihovarská 1060/12 tel: +420 284 028 960
190 00 Prague 9 fax: +420 284 028 951
Czech Republic http://www.suse.cz/
--
To unsubscribe, e-mail: [email protected]
To contact the owner, e-mail: [email protected]
