On 2025/06/06 4:35, John Snow wrote:
Hi, I've long been a little confused about the specifics of our build
platform guarantee and how it applies to documentation and testing.
My *current* understanding is that our build platform guarantee applies
to both unit tests and building documentation, but that this requirement
may not be as absolute as I imagine it.
The way I have endeavored to manage the Python tooling in our tree so
far is to preserve, without fail, our ability to perform fully offline
builds on all supported platforms (provided that the right distro repo
packages are available). The Python virtual environment created at
configure time bends over backwards to use system packages *whenever
possible*, and the list of exceptions - notably Meson itself - uses
vendored packages only in very specific cases where it is possible to
vendor such packages. Fetching packages from PyPI is generally offered
only as a convenience for developer workstations to, in general, save
developers from having to know anything about Python. (I think I've done
a good job there, to be honest!)
It is a nice work indeed. I spent substantial time to work on QEMU but
the Sphinx problem mentioned later was the only problem I encountered.
(Notably: Meson is pure python and has no dependencies, so it is
possible to vendor it for offline builds. Tools like Sphinx, however,
have many dependencies and are not so easily vendored. Thus, we have
created a tenuous arrangement where we are allowed to use versions of
Meson that otherwise would break our build platform guarantee.)
Lately, we've had some issues with the wide range of Sphinx versions we
support presenting various cross-platform difficulties. In particular,
Akihiko Odaki has sent patches to bump our Sphinx version to at least
6.2.1, because platforms with Python 3.13.1 can no longer run Sphinx 3.x
at all, so having that be our "default install version" causes issues on
newer platforms.
However, if we take as iron-clad our commitment to the build platform
promise -- *and* guarantee offline/tarball builds as well -- then Debian
12 (as an example) only offers Sphinx 5.3.0 and not newer unless we
allow internet access to fetch Sphinx 6.2.1. This is not a problem for
developer workstations at all, but I am unclear on what problems this
may cause for tarball releases and downstream offline/isolated/
reproducible builds, if any.
In this case, we can (probably) "fix" the issue by continuing to allow
older Sphinx while preferring a newer Sphinx version when it is missing,
but then we lose the ability to make code cleanups and drop a lot of
back-compat crud. If memory serves, there were other issues recently
where older versions of Sphinx behaved differently from newer versions,
causing intermittent failures that were hard to track down.
What I'd like to know is: what precisely are our options in this
scenario? Do we consider it acceptable for some platforms to be unable
to build docs offline? How highly do we value the ability to locally
build docs for any given release?
Before I throw my weight behind any given option, I just want to know
what we consider our non-negotiable obligations to be.
Thanks,
--js
Thank you for detailed explanation. I see a few possible options in this
situation.
Pretending as a lawyer interpreting docs/about/build-platforms.rst as a
law, I think it is allowed to require the Internet access. It says:
> For the purposes of identifying supported software versions available
> on Linux, the project will look at CentOS, Debian, Fedora, openSUSE,
> RHEL, SLES and Ubuntu LTS. Other distros will be assumed to ship
> similar software versions.
>
> For FreeBSD and OpenBSD, decisions will be made based on the contents
> of the respective ports repository, while NetBSD will use the pkgsrc
> repository.
>
> For macOS, `Homebrew`_ will be used, although `MacPorts`_ is expected
> to carry similar versions.
>
> Some build dependencies may follow less conservative rules:
> Python build dependencies
> Some of QEMU's build dependencies are written in Python. Usually
> these are only packaged by distributions for the default Python
> runtime.
> If QEMU bumps its minimum Python version and a non-default runtime
> is required, it may be necessary to fetch python modules from the
> Python Package Index (PyPI) via ``pip``, in order to build QEMU.
So it is allowed to require a non-default runtime that is available on
PyPI but not on Debian.
But it is also allowed to bundle a python module or to keep its version
requirement low to help users just as we do for Meson, of course.
Another possible option is to provide a mechanism to download the
required Python packages; users need the Internet to to prepare the
build platform (e.g., Debian) and QEMU source code anyway, so it makes
sense to require downloading the Python packages at the same time for
the future offline usage.
"pip download" is probably for such a use case, and we can make sure
that nobody get hurt by the Sphinx change or other Python dependency
changes by wiring up it into python/scripts/mkvenv.py.
In summary, there are the following viable options:
(1) Don't bump the Sphinx version.
(2) Require the Internet access when running ./configure --enable-docs.
(3) Bundle Sphinx.
(4) Let users "pip download".
Skipping into the conclusion, I'm for (2). The other options have their
own disadvantages:
(1) may cause intermittent failures with old Sphinx versions, which is
too bad.
(3) is just difficult, according on your description.
(4) is nice, but requires more time and I don't think it's worthwhile to
take the risk of old Sphinx versions until it is implemented.
In contrast, it seems (2) has the minimum risk and acceptable level of
annoyance. The documentation is for humans, and any human using QEMU
should have one computer with the Internet access at least. Perhaps some
users will be forced to change the build flag to stop building the
documentation on an offline computer and to prepare a build environment
on an online computer, but it's acceptable I guess.
Regards,
Akihiko Odaki
