Hello!
This series drops the in-tree version of our python-qemu-qmp package
("qemu.qmp") in favor of the version hosted on PyPI, whose repository is
located at https://gitlab.com/qemu-project/python-qemu-qmp.
GitLab CI: https://gitlab.com/jsnow/qemu/-/pipelines/2335222126
The major effects of this patch series are:
1. qemu.qmp and the python/ directory will be installed to the
configure-time venv (build/pyvenv) by default.
2. qemu.qmp will be installed from PyPI or vendored packages instead of
being utilized directly from the qemu.git tree.
3. "make check-venv" is removed as it is no longer needed.
4. functional test dependencies are installed as-needed when running
those tests.
5. Unfortunately, python3-wheel and python3-setuptools are now required
on the host system if tests are to be executed and >= Python 3.13 is
used.
6. An awful lot of deleted lines of code, and a lot fewer headaches
managing two nearly-identical copies of this source code. O:-)
v8:
- lcitool update patch(es):
- Fixed netbsd VM test
- Half-fixed Haiku VM test;
test fails now for reasons other than this patchset
- New patch; added compatibility for iotests ./check to run outside of
the venv, while tests will still run inside the venv, as a
transitional lubricant for block maintainer workflow (Thanks, Kevin!)
- I really think this is it, any other small change requests I think
should be addressed as follow-up patches at this point. Otherwise,
it's going to get out of hand...
Note 1: The lcitool upgrade causes a failure in check-rust-tools-nightly
(which is allowed to fail), and a separate standalone fix for this issue
has been sent as
https://patchew.org/QEMU/[email protected]/
Note 2: This series does not fix the currently failing check-python-tox
test (also allowed to fail), which is fixed separately in
https://patchew.org/QEMU/[email protected]/
Note 3: This series works around, but does not fix, the fact that we are
still utilizing Debian 11 for our CI infrastructure. That is rectified
in another separate series;
https://patchew.org/QEMU/[email protected]/ - The
only difference this patch series would make to this series is the
version numbers chosen in the 20th patch, which can be trivially updated
later depending on merge order.
v7:
- Fixed OpenBSD VM test
- Fixed a recently failing MacOS test, by...
- Upgraded lcitool to latest version
- Upgraded Fedora from 41 -> 43
- Upgraded Alpine from 3.21 -> 3.23
- Included contrig/plugins/cpp fix for CI testing purposes
- Added minimum versions for setuptools and wheel to pythondeps.toml
v6:
- Added workaround for configure-less VM tests
v5:
- Changed the local package install mechanism as per Paolo's advice
- Changed "file://" mechanism for local installs, following the msys2
test fixes upstream that did the same
- Fix typo in cover letter (thuth)
- Fixed comments around mkvenv invocations; functests group will
*always* use the --online flag, whether or not it was
configured. (danpb)
- Please, let this nightmare be over
v4:
- Rename "checktests" group to "tooling", install it by default
alongside meson at configure-time.
- Remove "check-venv" entirely.
- Update python/README.rst to reflect the new paradigm
- Update many scripts that import qemu.qmp to give new errors when they
are run outside of the configure venv
KNOWN ISSUES IN V4:
- The last patch is an RFC based on Thomas' observations, I don't have
a strong opinion on if we want it or not.
- Removing "make check-venv" causes the VM tests to rebuild more often;
I was not sure how to replace the old conditional behavior in an
equivalent manner. Suggestions, anyone...?
v3:
- Fixed FreeBSD vm test
- Fixed Cirrus macOS test (Needed setuptools as a dep, "file:///" had
one too many forward slashes, which works fine for older
pip/setuptools, but was incompatibly fixed recently.)
- Fixed an issue that prevented installing the local python packages in
an offline, isolated build environment (need --no-isolated-build and
pip environment variable flags)
- Added the python3-wheel and python3-setuptools build dependencies to
lcitool, owing to the above issues
- Ensured that the $download flag (for PyPI access) from configure is
carried over into pytest/meson.build invocations. This defaults to
"enabled", so test dependencies should not have any issue being
fetched on most developer workstations and build environments.
- Fixed several bugs in the mkvenv.py local package installation mechanism.
- Flags and workarounds added for installing local packages offline now
apply *only* to local package installations, and not "normal"
depspecs.
- Reordered patches into more logical "phases" for easier review (I
hope...?)
Known issues: "make check-build" still installs all of the test
dependencies to the pyvenv. I'm not sure if this is desired or not, but
I ran into problems preventing it from doing this without breaking a ton
of CI tests. I'll continue to investigate this, but wanted to send this
v3 out.
v2:
- move "make check-venv" earlier in GitLab CI/CD pipeline, to avoid
re-running configure
- Fix functional tests not using PyPI to fetch pygdbmi package
- Remove pre-requisites which are now merged
--js
John Snow (19):
python/mkvenv: create timestamp file for each group "ensured"
python/mkvenv: bump 'qemu.qmp' dependency for testdeps
python/mkvenv: add 'tooling' and 'functests' dependency groups
python/mkvenv: add mechanism to install local package(s)
meson, mkvenv: add functests custom target
tests: Use configured python to run GitLab iotests
iotests: tolerate being run outside of pyvenv
tests: use "run" script to execute device-crash-test
tests/lcitool: update lcitool to latest version
tests/lcitool: add python3 wheel and setuptools deps for qemu
python: add vendored qemu.qmp package
meson, mkvenv: make functional tests depend on functests group
meson, mkvenv: add qemu.git/python/qemu package to pythondeps.toml
configure: unconditionally install "tooling" group
tests: replace check-venv with vm-venv target
scripts: nudge users to use 'run' script for scripts that import
qemu.qmp
python: delete qemu.qmp
python: update README.rst to reflect qemu.qmp's removal
python: add setuptools and wheel dependencies
Pierrick Bouvier (1):
[do-not-merge] contrib/plugins/cpp: use __has_include
python/README.rst | 52 +-
configure | 2 +-
meson.build | 1 +
.gitlab-ci.d/buildtest.yml | 26 +-
.gitlab-ci.d/cirrus/freebsd-14.vars | 2 +-
.gitlab-ci.d/cirrus/macos-14.vars | 2 +-
.gitlab-ci.d/windows.yml | 2 +
contrib/plugins/cpp.cpp | 261 +++-
python/qemu/qmp/__init__.py | 60 -
python/qemu/qmp/error.py | 53 -
python/qemu/qmp/events.py | 751 -----------
python/qemu/qmp/legacy.py | 339 -----
python/qemu/qmp/message.py | 217 ----
python/qemu/qmp/models.py | 146 ---
python/qemu/qmp/protocol.py | 1101 -----------------
python/qemu/qmp/py.typed | 0
python/qemu/qmp/qmp_client.py | 732 -----------
python/qemu/qmp/qmp_shell.py | 689 -----------
python/qemu/qmp/qmp_tui.py | 665 ----------
python/qemu/qmp/util.py | 150 ---
python/qemu/utils/qom_fuse.py | 1 -
python/scripts/mkvenv.py | 43 +-
python/scripts/vendor.py | 2 +
python/setup.cfg | 31 +-
python/tests/minreqs.txt | 8 +-
python/tests/protocol.py | 596 ---------
python/wheels/qemu_qmp-0.0.5-py3-none-any.whl | Bin 0 -> 72263 bytes
pythondeps.toml | 21 +-
pyvenv/meson.build | 27 +
scripts/compare-machine-types.py | 7 +-
scripts/device-crash-test | 5 +-
scripts/qmp/qemu-ga-client | 13 +-
scripts/qmp/qmp-shell | 13 +-
scripts/qmp/qmp-shell-wrap | 13 +-
scripts/qmp/qom-fuse | 13 +-
scripts/qmp/qom-get | 13 +-
scripts/qmp/qom-list | 13 +-
scripts/qmp/qom-set | 13 +-
scripts/qmp/qom-tree | 13 +-
scripts/qmp_helper.py | 9 +-
scripts/render_block_graph.py | 10 +-
scripts/simplebench/bench_block_job.py | 10 +-
tests/Makefile.include | 22 +-
tests/docker/dockerfiles/alpine.docker | 6 +-
tests/docker/dockerfiles/centos9.docker | 242 ++--
.../dockerfiles/fedora-rust-nightly.docker | 262 ++--
.../dockerfiles/fedora-win64-cross.docker | 162 +--
tests/docker/dockerfiles/fedora.docker | 262 ++--
tests/docker/dockerfiles/opensuse-leap.docker | 1 +
tests/functional/meson.build | 5 +-
tests/lcitool/libvirt-ci | 2 +-
tests/lcitool/mappings.yml | 2 +-
tests/lcitool/projects/qemu.yml | 2 +
tests/lcitool/refresh | 8 +-
tests/migration-stress/guestperf/engine.py | 15 +-
tests/qemu-iotests/testenv.py | 25 +-
tests/vm/Makefile.include | 20 +-
tests/vm/generated/freebsd.json | 2 +
tests/vm/haiku.x86_64 | 2 +
tests/vm/netbsd | 1 +
tests/vm/openbsd | 3 +
61 files changed, 1025 insertions(+), 6144 deletions(-)
delete mode 100644 python/qemu/qmp/__init__.py
delete mode 100644 python/qemu/qmp/error.py
delete mode 100644 python/qemu/qmp/events.py
delete mode 100644 python/qemu/qmp/legacy.py
delete mode 100644 python/qemu/qmp/message.py
delete mode 100644 python/qemu/qmp/models.py
delete mode 100644 python/qemu/qmp/protocol.py
delete mode 100644 python/qemu/qmp/py.typed
delete mode 100644 python/qemu/qmp/qmp_client.py
delete mode 100644 python/qemu/qmp/qmp_shell.py
delete mode 100644 python/qemu/qmp/qmp_tui.py
delete mode 100644 python/qemu/qmp/util.py
delete mode 100644 python/tests/protocol.py
create mode 100644 python/wheels/qemu_qmp-0.0.5-py3-none-any.whl
create mode 100644 pyvenv/meson.build
--
2.53.0
