Re: [PATCH 135/147] meson: sphinx-build
On 10/08/20 21:06, Paolo Bonzini wrote: >>> diff --git a/configure b/configure >>> index 21b9ed2..7e7b4d8 100755 >>> --- a/configure >>> +++ b/configure >>> @@ -7768,7 +7768,6 @@ echo "INSTALL_PROG=$install -c -m 0755" >> >>> $config_host_mak >>> echo "INSTALL_LIB=$install -c -m 0644" >> $config_host_mak >>> echo "PYTHON=$python" >> $config_host_mak >>> echo "SPHINX_BUILD=$sphinx_build" >> $config_host_mak >>> -echo "SPHINX_WERROR=$sphinx_werror" >> $config_host_mak >> Shouldn't we also be deleting the code in configure that >> sets $sphinx_werror if we're no longer using it ? > Yes. > ... It's still used by has_sphinx_build, actually. Paolo
Re: [PATCH 135/147] meson: sphinx-build
Il lun 10 ago 2020, 21:57 Peter Maydell ha scritto: > > Since it's all handled internally by sphinx, I think you only need to > > add the man pages to the dictionary, and get rid of the corresponding > > Texinfo outputs in qapi/meson.build and qga/meson.build? > > The patchset has a fair amount of change to the makefiles: > Makefile | 86 +--- > rules.mak | 14 +- > > > > In other words, it should be just this: > > ...so if it's that simple that would be nice. > Hopefully. Of course you would also have to delete the Texinfo rules in meson.build but that's certainly more self explanatory then the qapi-gen.py bit. Paolo > -- PMM > >
Re: [PATCH 135/147] meson: sphinx-build
On Mon, 10 Aug 2020 at 20:46, Paolo Bonzini wrote: > > On 10/08/20 21:36, Peter Maydell wrote: > >> It should be possible and probably not too hard once I figure out how > >> Sphinx events work. It's a fair request since build_always_stale is > >> inferior and Meson requires no particular magic to include the depfile. > >> Maybe that will win you over. :) > >> > >> I can also leave out sphinx from the initial conversion. > > If we have a working-but-build-always conversion for Sphinx > > I'd be happy to take that and then upgrade it to processing > > the dependencies properly later. > > > > (The thing I'm not really looking forward to is updating > > the qapi-doc-to-rst patchset to Meson...) > > Since it's all handled internally by sphinx, I think you only need to > add the man pages to the dictionary, and get rid of the corresponding > Texinfo outputs in qapi/meson.build and qga/meson.build? The patchset has a fair amount of change to the makefiles: Makefile | 86 +--- rules.mak | 14 +- > In other words, it should be just this: ...so if it's that simple that would be nice. -- PMM
Re: [PATCH 135/147] meson: sphinx-build
On 10/08/20 21:36, Peter Maydell wrote: >> It should be possible and probably not too hard once I figure out how >> Sphinx events work. It's a fair request since build_always_stale is >> inferior and Meson requires no particular magic to include the depfile. >> Maybe that will win you over. :) >> >> I can also leave out sphinx from the initial conversion. > If we have a working-but-build-always conversion for Sphinx > I'd be happy to take that and then upgrade it to processing > the dependencies properly later. > > (The thing I'm not really looking forward to is updating > the qapi-doc-to-rst patchset to Meson...) Since it's all handled internally by sphinx, I think you only need to add the man pages to the dictionary, and get rid of the corresponding Texinfo outputs in qapi/meson.build and qga/meson.build? In other words, it should be just this: diff --git a/qapi/meson.build b/qapi/meson.build index de5b16f..315252f 100644 --- a/qapi/meson.build +++ b/qapi/meson.build @@ -96,7 +96,7 @@ foreach module : qapi_all_modules endforeach qapi_files = custom_target('shared QAPI source files', - output: qapi_util_outputs + qapi_specific_outputs + qapi_nonmodule_outputs + ['qapi-doc.texi'], + output: qapi_util_outputs + qapi_specific_outputs + qapi_nonmodule_outputs, input: [ files('qapi-schema.json') ], command: [ qapi_gen, '-o', 'qapi', '-b', '@INPUT0@' ], depend_files: [ qapi_inputs, qapi_gen_depends ]) @@ -120,5 +120,3 @@ foreach output : qapi_specific_outputs + qapi_nonmodule_outputs specific_ss.add(when: 'CONFIG_SOFTMMU', if_true: qapi_files[i]) i = i + 1 endforeach - -qapi_doc_texi = qapi_files[i] diff --git a/qga/meson.build b/qga/meson.build index 6f6..741b683 100644 --- a/qga/meson.build +++ b/qga/meson.build @@ -16,7 +16,7 @@ qga_qapi_outputs = [ ] qga_qapi_files = custom_target('QGA QAPI files', - output: qga_qapi_outputs + ['qga-qapi-doc.texi'], + output: qga_qapi_outputs, input: 'qapi-schema.json', command: [ qapi_gen, '-o', 'qga', '-p', 'qga-', '@INPUT0@' ], depend_files: qapi_gen_depends) @@ -27,7 +27,6 @@ foreach output: qga_qapi_outputs qga_ss.add(qga_qapi_files[i]) i = i + 1 endforeach -qga_qapi_doc_texi = qga_qapi_files[i] qga_ss.add(files( 'commands.c', Paolo
Re: [PATCH 135/147] meson: sphinx-build
On Mon, 10 Aug 2020 at 20:31, Paolo Bonzini wrote: > > On 10/08/20 21:21, Peter Maydell wrote: > >> Yes, because the Makefile's approach is not maintainable in my opinion; > >> *.rst.inc files were already not included in the Makefile. I'll look > >> into using a Sphinx extension to produce a dependency file. > > > > Yeah, agreed that the makefile approach isn't great. (It lists > > some .rst.inc files but we added more without updating the > > dependencies, I think.) > > > > If Sphinx can be persuaded to output a dependency file that > > would certainly be the nicest approach; I hadn't thought > > of trying that. > > It should be possible and probably not too hard once I figure out how > Sphinx events work. It's a fair request since build_always_stale is > inferior and Meson requires no particular magic to include the depfile. > Maybe that will win you over. :) > > I can also leave out sphinx from the initial conversion. If we have a working-but-build-always conversion for Sphinx I'd be happy to take that and then upgrade it to processing the dependencies properly later. (The thing I'm not really looking forward to is updating the qapi-doc-to-rst patchset to Meson...) thanks -- PMM
Re: [PATCH 135/147] meson: sphinx-build
On 10/08/20 21:21, Peter Maydell wrote: >> Yes, because the Makefile's approach is not maintainable in my opinion; >> *.rst.inc files were already not included in the Makefile. I'll look >> into using a Sphinx extension to produce a dependency file. > > Yeah, agreed that the makefile approach isn't great. (It lists > some .rst.inc files but we added more without updating the > dependencies, I think.) > > If Sphinx can be persuaded to output a dependency file that > would certainly be the nicest approach; I hadn't thought > of trying that. It should be possible and probably not too hard once I figure out how Sphinx events work. It's a fair request since build_always_stale is inferior and Meson requires no particular magic to include the depfile. Maybe that will win you over. :) I can also leave out sphinx from the initial conversion. > It would be nice to note in the commit messages where the > conversion has made this kind of "we're going to do it a > different way" design decision rather than just being > a translation of the makefile logic into Meson. Yes, I'll do that for the final version (to be posted Friday or next Monday). Thanks for running these initial test, it looks encouraging. Paolo
Re: [PATCH 135/147] meson: sphinx-build
On Mon, 10 Aug 2020 at 20:06, Paolo Bonzini wrote: > > On 10/08/20 20:33, Peter Maydell wrote: > > Does "build_always_stale: true" do what I guess it does from the > > name? Does this mean we're discarding the makefile's approach of > > only running sphinx if it's necessary in favour of always running > > half a dozen sphinx invocations every build ? > > Yes, because the Makefile's approach is not maintainable in my opinion; > *.rst.inc files were already not included in the Makefile. I'll look > into using a Sphinx extension to produce a dependency file. Yeah, agreed that the makefile approach isn't great. (It lists some .rst.inc files but we added more without updating the dependencies, I think.) If Sphinx can be persuaded to output a dependency file that would certainly be the nicest approach; I hadn't thought of trying that. It would be nice to note in the commit messages where the conversion has made this kind of "we're going to do it a different way" design decision rather than just being a translation of the makefile logic into Meson. thanks -- PMM
Re: [PATCH 135/147] meson: sphinx-build
On 10/08/20 20:33, Peter Maydell wrote: > On Mon, 10 Aug 2020 at 19:16, Paolo Bonzini wrote: >> >> Signed-off-by: Marc-André Lureau >> Signed-off-by: Paolo Bonzini > >> diff --git a/configure b/configure >> index 21b9ed2..7e7b4d8 100755 >> --- a/configure >> +++ b/configure >> @@ -7768,7 +7768,6 @@ echo "INSTALL_PROG=$install -c -m 0755" >> >> $config_host_mak >> echo "INSTALL_LIB=$install -c -m 0644" >> $config_host_mak >> echo "PYTHON=$python" >> $config_host_mak >> echo "SPHINX_BUILD=$sphinx_build" >> $config_host_mak >> -echo "SPHINX_WERROR=$sphinx_werror" >> $config_host_mak > > Shouldn't we also be deleting the code in configure that > sets $sphinx_werror if we're no longer using it ? Yes. >> +these_man_pages = [] >> +install_dirs = [] >> +foreach page, section : man_pages.get(manual, {}) >> + these_man_pages += page >> + install_dirs += section == '' ? false : get_option('mandir') / section >> +endforeach >> +if these_man_pages.length() > 0 >> + sphinxmans += custom_target(manual + ' man pages', >> + build_always_stale: true, >> + build_by_default: build_docs, >> + output: these_man_pages, >> + install: build_docs, >> + install_dir: install_dirs, >> + command: [SPHINX_ARGS, '-b', 'man', '-d', >> private_dir, >> + input_dir, meson.current_build_dir()]) >> +endif >> + endforeach >> + alias_target('sphinxdocs', sphinxdocs) >> + alias_target('man', sphinxmans) > > Does "build_always_stale: true" do what I guess it does from the > name? Does this mean we're discarding the makefile's approach of > only running sphinx if it's necessary in favour of always running > half a dozen sphinx invocations every build ? Yes, because the Makefile's approach is not maintainable in my opinion; *.rst.inc files were already not included in the Makefile. I'll look into using a Sphinx extension to produce a dependency file. Paolo Paolo
Re: [PATCH 135/147] meson: sphinx-build
On Mon, 10 Aug 2020 at 19:16, Paolo Bonzini wrote: > > Signed-off-by: Marc-André Lureau > Signed-off-by: Paolo Bonzini > diff --git a/configure b/configure > index 21b9ed2..7e7b4d8 100755 > --- a/configure > +++ b/configure > @@ -7768,7 +7768,6 @@ echo "INSTALL_PROG=$install -c -m 0755" >> > $config_host_mak > echo "INSTALL_LIB=$install -c -m 0644" >> $config_host_mak > echo "PYTHON=$python" >> $config_host_mak > echo "SPHINX_BUILD=$sphinx_build" >> $config_host_mak > -echo "SPHINX_WERROR=$sphinx_werror" >> $config_host_mak Shouldn't we also be deleting the code in configure that sets $sphinx_werror if we're no longer using it ? > +these_man_pages = [] > +install_dirs = [] > +foreach page, section : man_pages.get(manual, {}) > + these_man_pages += page > + install_dirs += section == '' ? false : get_option('mandir') / section > +endforeach > +if these_man_pages.length() > 0 > + sphinxmans += custom_target(manual + ' man pages', > + build_always_stale: true, > + build_by_default: build_docs, > + output: these_man_pages, > + install: build_docs, > + install_dir: install_dirs, > + command: [SPHINX_ARGS, '-b', 'man', '-d', > private_dir, > + input_dir, meson.current_build_dir()]) > +endif > + endforeach > + alias_target('sphinxdocs', sphinxdocs) > + alias_target('man', sphinxmans) Does "build_always_stale: true" do what I guess it does from the name? Does this mean we're discarding the makefile's approach of only running sphinx if it's necessary in favour of always running half a dozen sphinx invocations every build ? thanks -- PMM
[PATCH 135/147] meson: sphinx-build
Signed-off-by: Marc-André Lureau Signed-off-by: Paolo Bonzini --- Makefile | 142 +++-- configure | 1 - docs/index.html.in | 4 +- docs/meson.build | 68 + meson.build| 2 + rules.mak | 48 -- 6 files changed, 79 insertions(+), 186 deletions(-) create mode 100644 docs/meson.build diff --git a/Makefile b/Makefile index 2297712..b0207a9 100644 --- a/Makefile +++ b/Makefile @@ -119,36 +119,9 @@ $(call set-vpath, $(SRC_PATH)) LIBS+=-lz $(LIBS_TOOLS) -# Sphinx does not allow building manuals into the same directory as -# the source files, so if we're doing an in-tree QEMU build we must -# build the manuals into a subdirectory (and then install them from -# there for 'make install'). For an out-of-tree build we can just -# use the docs/ subdirectory in the build tree as normal. -ifeq ($(realpath $(SRC_PATH)),$(realpath .)) -MANUAL_BUILDDIR := docs/built -else -MANUAL_BUILDDIR := docs -endif - ifdef BUILD_DOCS -DOCS+=$(MANUAL_BUILDDIR)/system/qemu.1 -DOCS+=$(MANUAL_BUILDDIR)/tools/qemu-img.1 -DOCS+=$(MANUAL_BUILDDIR)/tools/qemu-nbd.8 -DOCS+=$(MANUAL_BUILDDIR)/interop/qemu-ga.8 -ifeq ($(CONFIG_LINUX)$(CONFIG_SECCOMP)$(CONFIG_LIBCAP_NG),yyy) -DOCS+=$(MANUAL_BUILDDIR)/tools/virtiofsd.1 -endif -DOCS+=$(MANUAL_BUILDDIR)/system/qemu-block-drivers.7 DOCS+=docs/interop/qemu-qmp-ref.html docs/interop/qemu-qmp-ref.txt docs/interop/qemu-qmp-ref.7 DOCS+=docs/interop/qemu-ga-ref.html docs/interop/qemu-ga-ref.txt docs/interop/qemu-ga-ref.7 -DOCS+=$(MANUAL_BUILDDIR)/system/qemu-cpu-models.7 -DOCS+=$(MANUAL_BUILDDIR)/index.html -ifdef CONFIG_VIRTFS -DOCS+=$(MANUAL_BUILDDIR)/tools/virtfs-proxy-helper.1 -endif -ifdef CONFIG_TRACE_SYSTEMTAP -DOCS+=$(MANUAL_BUILDDIR)/tools/qemu-trace-stap.1 -endif else DOCS= endif @@ -231,11 +204,6 @@ dist: qemu-$(VERSION).tar.bz2 qemu-%.tar.bz2: $(SRC_PATH)/scripts/make-release "$(SRC_PATH)" "$(patsubst qemu-%.tar.bz2,%,$@)" -define clean-manual = -rm -rf $(MANUAL_BUILDDIR)/$1/_static -rm -f $(MANUAL_BUILDDIR)/$1/objects.inv $(MANUAL_BUILDDIR)/$1/searchindex.js $(MANUAL_BUILDDIR)/$1/*.html -endef - distclean: clean rm -f config-host.mak config-host.h* $(DOCS) rm -f tests/tcg/config-*.mak @@ -251,13 +219,6 @@ distclean: clean rm -f docs/interop/qemu-qmp-ref.txt docs/interop/qemu-ga-ref.txt rm -f docs/interop/qemu-qmp-ref.pdf docs/interop/qemu-ga-ref.pdf rm -f docs/interop/qemu-qmp-ref.html docs/interop/qemu-ga-ref.html - rm -rf .doctrees - $(call clean-manual,devel) - $(call clean-manual,interop) - $(call clean-manual,specs) - $(call clean-manual,system) - $(call clean-manual,tools) - $(call clean-manual,user) rm -Rf .sdk KEYMAPS=da en-gb et fr fr-ch is lt no pt-br sv \ @@ -291,28 +252,8 @@ else BLOBS= endif -# Note that we manually filter-out the non-Sphinx documentation which -# is currently built into the docs/interop directory in the build tree, -# and also any sphinx-built manpages. -define install-manual = -for d in $$(cd $(MANUAL_BUILDDIR) && find $1 -type d); do $(INSTALL_DIR) "$(DESTDIR)$(qemu_docdir)/$$d"; done -for f in $$(cd $(MANUAL_BUILDDIR) && find $1 -type f -a '!' '(' -name '*.[0-9]' -o -name 'qemu-*-qapi.*' -o -name 'qemu-*-ref.*' ')' ); do $(INSTALL_DATA) "$(MANUAL_BUILDDIR)/$$f" "$(DESTDIR)$(qemu_docdir)/$$f"; done -endef - -# Note that we deliberately do not install the "devel" manual: it is -# for QEMU developers, and not interesting to our users. -.PHONY: install-sphinxdocs -install-sphinxdocs: sphinxdocs - $(call install-manual,interop) - $(call install-manual,specs) - $(call install-manual,system) - $(call install-manual,tools) - $(call install-manual,user) - -install-doc: $(DOCS) install-sphinxdocs +install-doc: $(DOCS) $(INSTALL_DIR) "$(DESTDIR)$(qemu_docdir)" - $(INSTALL_DATA) $(MANUAL_BUILDDIR)/index.html "$(DESTDIR)$(qemu_docdir)" - $(INSTALL_DIR) "$(DESTDIR)$(qemu_docdir)/interop" $(INSTALL_DATA) docs/interop/qemu-qmp-ref.html "$(DESTDIR)$(qemu_docdir)/interop" $(INSTALL_DATA) docs/interop/qemu-qmp-ref.txt "$(DESTDIR)$(qemu_docdir)/interop" ifdef CONFIG_POSIX @@ -320,19 +261,7 @@ ifdef CONFIG_POSIX $(INSTALL_DATA) $(MANUAL_BUILDDIR)/system/qemu.1 "$(DESTDIR)$(mandir)/man1" $(INSTALL_DIR) "$(DESTDIR)$(mandir)/man7" $(INSTALL_DATA) docs/interop/qemu-qmp-ref.7 "$(DESTDIR)$(mandir)/man7" - $(INSTALL_DATA) $(MANUAL_BUILDDIR)/system/qemu-block-drivers.7 "$(DESTDIR)$(mandir)/man7" - $(INSTALL_DATA) $(MANUAL_BUILDDIR)/system/qemu-cpu-models.7 "$(DESTDIR)$(mandir)/man7" -ifeq ($(CONFIG_TOOLS),y) - $(INSTALL_DATA) $(MANUAL_BUILDDIR)/tools/qemu-img.1 "$(DESTDIR)$(mandir)/man1" - $(INSTALL_DIR) "$(DESTDIR)$(mandir)/man8" - $(INSTALL_DATA) $(MANUAL_BUILDDIR)/tools/qemu-nbd.8 "$(D