We currently have two documentation targets to build:
- 'man' for the man pages;
- 'html' for the web page.
There are two bottlenecks in the process:
1) sphinx runs with a single process;
2) the two targets are serialized.
For (1), we can just add the "-j auto" to sphinx_build and that should
bring some speed gains.
For (2) it's a little trickier because the reason the builds are
serialized is that Sphinx keeps track of changed files, but we still
need a way to tell meson whether a target needs to re-execute, so we
added dependency tracking and timestamp checking for (only) the 'html'
build via the depfile.py extension. Since the sources for both builds
are the same, we made the 'man' build dependent on 'html' so that it
would rebuild when needed.
So patch 1 adds the -j option to Sphinx and patch 2 adds depfile
support to the 'man' build. We can now run the two in parallel (ninja
will take care of that).
On my 16 core machine,
the -j change saves about 20s
the depfile change saves about 10s
===
On master:
$ ../configure --enable-docs ...
$ time make -j$(nproc) html man
GIT ui/keycodemapdb meson tests/fp/berkeley-testfloat-3
tests/fp/berkeley-softfloat-3 dtc
/usr/bin/ninja build.ninja && touch build.ninja.stamp
ninja: no work to do.
/usr/bin/python3 -B .../qemu/meson/meson.py introspect --targets --tests
--benchmarks | /usr/bin/python3 -B scripts/mtest2make.py > Makefile.mtest
GIT ui/keycodemapdb meson tests/fp/berkeley-testfloat-3
tests/fp/berkeley-softfloat-3 dtc
[1/2] Generating docs/QEMU manual with a custom command
[2/2] Generating docs/QEMU man pages with a custom command
make: Nothing to be done for 'man'.
real 0m50.155s
user 0m49.759s
sys 0m0.761s
$ mv docs ../saved-docs
This series:
$ ../configure --enable-docs ...
$ time make -j$(nproc) html man
GIT ui/keycodemapdb meson tests/fp/berkeley-testfloat-3
tests/fp/berkeley-softfloat-3 dtc
/usr/bin/ninja build.ninja && touch build.ninja.stamp
ninja: no work to do.
/usr/bin/python3 -B .../qemu/meson/meson.py introspect --targets --tests
--benchmarks | /usr/bin/python3 -B scripts/mtest2make.py > Makefile.mtest
GIT ui/keycodemapdb meson tests/fp/berkeley-testfloat-3
tests/fp/berkeley-softfloat-3 dtc
[1/2] Generating docs/QEMU man pages with a custom command
[2/2] Generating docs/QEMU manual with a custom command
make: Nothing to be done for 'man'.
real 0m21.708s
user 1m12.317s
sys 0m2.056s
Diff sanity check:
$ diff -rq docs/ ../saved-docs/
Only in ../saved-docs/: docs.d # now manual.dep
Only in ../saved-docs/: docs.stamp # now manual.stamp
Only in docs/: man.dep
Only in docs/: man.p
Only in docs/: man.stamp
Only in docs/: manual.dep
Only in docs/: manual.stamp
Files docs/manual.p/about/build-platforms.doctree and
../saved-docs/manual.p/about/build-platforms.doctree differ
... # sphinx cache files, a bunch of these^
Rebuilding (here I show that a man file and an html file are
unchanged, change their source .rst and rebuild each target):
$ ninja -d explain html
ninja: no work to do.
$ ninja -d explain man
ninja: no work to do.
$ man -l docs/qemu.1 | grep foobar
$ grep foobar docs/manual/system/i386/pc.html
$ vi ../docs/system/target-i386-desc.rst.inc #add the 'foobar' string
$ ninja -d explain man
ninja explain: restat of output docs/man.stamp older than most recent input
docs/system/target-i386-desc.rst.inc (1683122999140339620 vs
1683123032492362281)
ninja explain: docs/man.stamp is dirty
ninja explain: docs/qemu-block-drivers.7 is dirty
ninja explain: docs/qemu-cpu-models.7 is dirty
ninja explain: docs/qemu-ga-ref.7 is dirty
ninja explain: docs/qemu-ga.8 is dirty
ninja explain: docs/qemu-img.1 is dirty
ninja explain: docs/qemu-nbd.8 is dirty
ninja explain: docs/qemu-pr-helper.8 is dirty
ninja explain: docs/qemu-qmp-ref.7 is dirty
ninja explain: docs/qemu-storage-daemon-qmp-ref.7 is dirty
ninja explain: docs/qemu-storage-daemon.1 is dirty
ninja explain: docs/qemu-trace-stap.1 is dirty
ninja explain: docs/qemu.1 is dirty
ninja explain: docs/virtfs-proxy-helper.1 is dirty
[1/1] Generating docs/QEMU man pages with a custom command
$ man -l docs/qemu.1 | grep foobar
The QEMU PC System emulator simulates the following foobar peripherals:
$ grep foobar docs/manual/system/i386/pc.html #html files unchanged
$ ninja -d explain html
ninja explain: restat of output docs/manual.stamp older than most recent input
docs/system/target-i386-desc.rst.inc (1683122995876337403 vs
1683123032492362281)
ninja explain: docs/manual.stamp is dirty
[1/1] Generating docs/QEMU manual with a custom command
$ man -l docs/qemu.1 | grep foobar
The QEMU PC System emulator simulates the following foobar peripherals:
$ grep foobar docs/manual/system/i386/pc.html
<p>The QEMU PC System emulator simulates the following foobar peripherals:</p>
===
Fabiano Rosas (2):
meson: Pass -j option to sphinx
meson: Deserialize the man pages and html builds
docs/meson.build | 48 +++++++++++++++++++++++++-------------
docs/sphinx/dbusdomain.py | 4 ++++
docs/sphinx/fakedbusdoc.py | 5 ++++
docs/sphinx/qmp_lexer.py | 5 ++++
4 files changed, 46 insertions(+), 16 deletions(-)
--
2.35.3
