[PATCH v8 00/31] python: create installable package

2021-05-27 Thread John Snow
dirs-to # [RB] CR [SOB] JS [20] python-add-flake8-to-pipenv # [RB] CR [TB] CR [SOB] JS [21] python-move-mypy-ini-into # [RB] CR [SOB] JS [22] python-add-mypy-to-pipenv # [RB] CR [TB] CR [SOB] JS [23] python-move-isort-cfg-into # [RB] CR [SOB] JS [24] python-qemu-add-isort-to

Re: [PATCH v7 31/31] gitlab: add python linters to CI

2021-05-27 Thread John Snow
On 5/27/21 1:02 PM, Cleber Rosa wrote: Nipick: - make -C python venv-check ACK, made this change.

Re: [PATCH v7 31/31] gitlab: add python linters to CI

2021-05-26 Thread John Snow
On 5/26/21 2:47 PM, Vladimir Sementsov-Ogievskiy wrote: build requisite for PyPI packages in the event that PyPI only has a sdist and not a bdist for a given dependency during installation. i.e. some packages are compiled during installation? Realized I didn't answer this directly. Yes, somet

Re: [PATCH v7 31/31] gitlab: add python linters to CI

2021-05-26 Thread John Snow
On 5/26/21 2:47 PM, Vladimir Sementsov-Ogievskiy wrote: Worth leaving the comment somewhere? (not worth any kind of resending of course) Yeah, I'll explain the package selection in the commit message at least. --js

Re: [PATCH v7 29/31] python: add .gitignore

2021-05-26 Thread John Snow
On 5/26/21 5:18 AM, Vladimir Sementsov-Ogievskiy wrote: 26.05.2021 03:24, John Snow wrote: Ignore *Python* build and package output (build, dist, qemu.egg-info); these files are not created as part of a QEMU build. They are created by running the commands 'python3 setup.py ' when

Re: [PATCH v7 31/31] gitlab: add python linters to CI

2021-05-26 Thread John Snow
On 5/26/21 11:32 AM, Alex Bennée wrote: John Snow writes: Add a python container that contains just enough juice for us to run the python code quality analysis tools. Base this container on fedora, because fedora has very convenient packaging for testing multiple python versions. Add two

Re: [PATCH v7 31/31] gitlab: add python linters to CI

2021-05-26 Thread John Snow
On 5/26/21 5:33 AM, Vladimir Sementsov-Ogievskiy wrote: 26.05.2021 03:24, John Snow wrote: Add a python container that contains just enough juice for us to run the python code quality analysis tools. Base this container on fedora, because fedora has very convenient packaging for testing

Re: [PATCH v7 17/31] python: add pylint to pipenv

2021-05-26 Thread John Snow
On 5/26/21 5:14 AM, Vladimir Sementsov-Ogievskiy wrote: 26.05.2021 03:24, John Snow wrote: We are specifying >= pylint 2.8.x for several reasons: 1. For setup.cfg support, added in pylint 2.5.x 2. To specify a version that has incompatibly dropped     bad-whitespace checks (2.6.x) 3. 2.

[PATCH v7 31/31] gitlab: add python linters to CI

2021-05-25 Thread John Snow
Python 3.10. This test is allowed to fail with a warning. Signed-off-by: John Snow --- .gitlab-ci.d/containers.yml| 5 + .gitlab-ci.yml | 26 ++ tests/docker/dockerfiles/python.docker | 18 ++ 3 files changed, 49

[PATCH v7 26/31] python: add devel package requirements to setuptools

2021-05-25 Thread John Snow
elopment dependencies are not already met. This can be useful for automated build scripts where fetching network packages may be undesirable. Signed-off-by: John Snow Reviewed-by: Cleber Rosa --- python/PACKAGE.rst | 4 python/README.rst | 4 python/Pipfile | 5 +

[PATCH v7 30/31] python: add tox support

2021-05-25 Thread John Snow
ode should be amended to work around the issue. With confidence that the tests pass on 3.6 through 3.10 inclusive, add the appropriate classifiers to setup.cfg to indicate which versions we claim to support. Tox 3.18.0 or above is required to use the 'allowlist_externals' option. Signed-off-

[PATCH v7 29/31] python: add .gitignore

2021-05-25 Thread John Snow
int, et al) Ignore .idea (pycharm) .vscode, and .venv (pipenv et al). Signed-off-by: John Snow --- python/.gitignore | 15 +++ 1 file changed, 15 insertions(+) create mode 100644 python/.gitignore diff --git a/python/.gitignore b/python/.gitignore new file mode 100644 index 0

[PATCH v7 25/31] python/qemu: add qemu package itself to pipenv

2021-05-25 Thread John Snow
he qemu packages are installed in 'editable' mode; all changes made to the python package inside the git tree will be reflected in the installed package without reinstallation. This includes changes made via git pull and so on. Signed-off-by: John Snow Reviewed-by: Cleber Rosa Teste

[PATCH v7 24/31] python/qemu: add isort to pipenv

2021-05-25 Thread John Snow
t; mode) with 'isort -c qemu' from the python root. isort can also be used to fix/rewrite import order automatically by using 'isort qemu'. Signed-off-by: John Snow Reviewed-by: Cleber Rosa --- python/Pipfile | 1 + python/Pipfile.lock | 4 ++-- 2 files changed, 3 inserti

[PATCH v7 22/31] python: add mypy to pipenv

2021-05-25 Thread John Snow
t path detection mechanisms in mypy slightly, and it will fail. See https://github.com/python/mypy/issues/8584 for a decent entry point with more breadcrumbs on the various behaviors that contribute to this subtle difference. Signed-off-by: John Snow Reviewed-by: Cleber Rosa Tested-by: Cleber Rosa

[PATCH v7 18/31] python: move flake8 config to setup.cfg

2021-05-25 Thread John Snow
Update the comment concerning the flake8 exception to match commit 42c0dd12, whose commit message stated: A note on the flake8 exception: flake8 will warn on *any* bare except, but pylint's is context-aware and will suppress the warning if you re-raise the exception. Signed-off-by: John

[PATCH v7 19/31] python: add excluded dirs to flake8 config

2021-05-25 Thread John Snow
uld be excluded anyway. A forthcoming commit canonizes this with a "make venv" command. Signed-off-by: John Snow Reviewed-by: Cleber Rosa --- python/setup.cfg | 2 ++ 1 file changed, 2 insertions(+) diff --git a/python/setup.cfg b/python/setup.cfg index 52a89a0a290..9aeab2bb0d3 1006

[PATCH v7 16/31] python: move pylintrc into setup.cfg

2021-05-25 Thread John Snow
Delete the empty settings now that it's sharing a home with settings for other tools. pylint can now be run from this folder as "pylint qemu". Signed-off-by: John Snow Reviewed-by: Cleber Rosa Tested-by: Cleber Rosa --- python/qemu/machin

[PATCH v7 28/31] python: add Makefile for some common tasks

2021-05-25 Thread John Snow
te miscellaneous python packaging output possibly created by pipenv, pip, or other python packaging utilities make distclean: delete the above, the .venv, and the editable "qemu" package forwarder (qemu.egg-info) if there is one. Signed-off-by: John Snow Reviewed-by: Cleber Ros

[PATCH v7 27/31] python: add avocado-framework and tests

2021-05-25 Thread John Snow
correct arguments. (A forthcoming commit adds the much easier 'make check'.) Signed-off-by: John Snow Reviewed-by: Cleber Rosa Tested-by: Cleber Rosa --- python/README.rst | 2 ++ python/Pipfile.lock| 8 python/avocado.cfg | 10 ++ python/setup.cfg |

[PATCH v7 23/31] python: move .isort.cfg into setup.cfg

2021-05-25 Thread John Snow
Signed-off-by: John Snow Reviewed-by: Cleber Rosa --- python/.isort.cfg | 7 --- python/setup.cfg | 8 2 files changed, 8 insertions(+), 7 deletions(-) delete mode 100644 python/.isort.cfg diff --git a/python/.isort.cfg b/python/.isort.cfg deleted file mode 100644 index

[PATCH v7 14/31] python: Add pipenv support

2021-05-25 Thread John Snow
t of packages that were known to operate correctly. This latter file provides the real value for easy setup of container images and CI environments. Signed-off-by: John Snow Reviewed-by: Cleber Rosa --- python/README.rst | 3 +++ python/Pipfile| 11 +++ 2 files changed, 14 insertio

[PATCH v7 07/31] python/machine: Trim line length to below 80 chars

2021-05-25 Thread John Snow
One more little delinting fix that snuck in. Signed-off-by: John Snow --- python/qemu/machine.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/python/qemu/machine.py b/python/qemu/machine.py index 5d72c4ca369..a8837b36e47 100644 --- a/python/qemu/machine.py +++ b/python

[PATCH v7 20/31] python: Add flake8 to pipenv

2021-05-25 Thread John Snow
flake8 3.5.x does not support the --extend-ignore syntax used in the .flake8 file to gracefully extend default ignores, so 3.6.x is our minimum requirement. There is no known upper bound. flake8 can be run from the python/ directory with no arguments. Signed-off-by: John Snow Reviewed-by

[PATCH v7 09/31] python: create qemu packages

2021-05-25 Thread John Snow
ll of the useful QMP library classes are available directly from "qemu.qmp" instead of "qemu.qmp.qmp". Signed-off-by: John Snow --- python/{qemu => }/.isort.cfg| 0 python/qemu/__init__.py | 11 --- python/qemu/{ => machine}/.flake

[PATCH v7 21/31] python: move mypy.ini into setup.cfg

2021-05-25 Thread John Snow
mypy supports reading its configuration values from a central project configuration file; do so. Signed-off-by: John Snow Reviewed-by: Cleber Rosa --- python/mypy.ini | 4 python/setup.cfg | 5 + 2 files changed, 5 insertions(+), 4 deletions(-) delete mode 100644 python/mypy.ini

[PATCH v7 17/31] python: add pylint to pipenv

2021-05-25 Thread John Snow
x27;consider-using-with' warning that must be disabled in some cases. These pragmas cause warnings themselves in 2.7.x. Signed-off-by: John Snow --- python/Pipfile | 1 + python/Pipfile.lock | 130 2 files changed, 131 insertions(+) cr

[PATCH v7 11/31] python: add VERSION file

2021-05-25 Thread John Snow
.0a1. Signed-off-by: John Snow Reviewed-by: Cleber Rosa --- python/VERSION | 1 + python/setup.cfg | 1 + 2 files changed, 2 insertions(+) create mode 100644 python/VERSION diff --git a/python/VERSION b/python/VERSION new file mode 100644 index 000..c19f3b832b7 --- /dev/null +++ b/pytho

[PATCH v7 12/31] python: add directory structure README.rst files

2021-05-25 Thread John Snow
. They are not designed for inclusion in a published manual. Signed-off-by: John Snow Reviewed-by: Cleber Rosa --- python/README.rst | 41 ++ python/qemu/README.rst | 8 +++ python/qemu/machine/README.rst | 9 python/qemu/qmp

[PATCH v7 04/31] python/console_socket: Add a pylint ignore

2021-05-25 Thread John Snow
We manage cleaning up this resource ourselves. Pylint should shush. Signed-off-by: John Snow Reviewed-by: Cleber Rosa Message-id: 20210517184808.3562549-5-js...@redhat.com Signed-off-by: John Snow --- python/qemu/console_socket.py | 1 + 1 file changed, 1 insertion(+) diff --git a/python

[PATCH v7 15/31] python: add pylint import exceptions

2021-05-25 Thread John Snow
that are too old for features we want in isort and mypy. Oh well. Signed-off-by: John Snow Reviewed-by: Cleber Rosa --- python/qemu/machine/__init__.py | 3 +++ python/qemu/machine/machine.py | 2 +- python/qemu/machine/qtest.py| 2 +- 3 files changed, 5 insertions(+), 2 deletions(-) diff

[PATCH v7 10/31] python: add qemu package installer

2021-05-25 Thread John Snow
his matter: https://github.com/pypa/packaging-problems/issues/256 https://github.com/pypa/pip/issues/6334 https://github.com/pypa/pip/issues/6375 https://github.com/pypa/pip/issues/6434 https://github.com/pypa/pip/issues/6438 Signed-off-by: John Snow --- python/PACKAG

[PATCH v7 13/31] python: add MANIFEST.in

2021-05-25 Thread John Snow
'tox', as well as by the tooling needed to upload packages to PyPI. Exclude the 'README.rst' file -- that's intended as a guidebook to our source tree, not a file that needs to be distributed. Signed-off-by: John Snow --- python/README.rst | 2 ++ python/MANIFEST.in | 3

[PATCH v7 03/31] python/machine: use subprocess.run instead of subprocess.Popen

2021-05-25 Thread John Snow
use run() instead of Popen() -- to assert to pylint that we are not forgetting to close a long-running program. Signed-off-by: John Snow Reviewed-by: Cleber Rosa Tested-by: Cleber Rosa Message-id: 20210517184808.3562549-4-js...@redhat.com Signed-off-by: John Snow --- python/qemu/machine.py

[PATCH v7 08/31] iotests/297: add --namespace-packages to mypy arguments

2021-05-25 Thread John Snow
mypy is kind of weird about how it handles imports. For legacy reasons, it won't load PEP 420 namespaces, because of logic implemented prior to that becoming a standard. So, if you plan on using any, you have to pass --namespace-packages. Alright, fine. Signed-off-by: John Snow Review

[PATCH v7 05/31] python/machine: Disable pylint warning for open() in _pre_launch

2021-05-25 Thread John Snow
call to shutdown(), and therefore also on __exit__. Signed-off-by: John Snow Reviewed-by: Wainer dos Santos Moschetta Reviewed-by: Cleber Rosa Message-id: 20210517184808.3562549-6-js...@redhat.com Signed-off-by: John Snow --- python/qemu/machine.py | 6 +- 1 file changed, 5 insertions(+), 1

[PATCH v7 06/31] python/machine: disable warning for Popen in _launch()

2021-05-25 Thread John Snow
We handle this resource rather meticulously in shutdown/kill/wait/__exit__ et al, through the laborious mechanisms in _do_shutdown(). Quiet this pylint warning here. Signed-off-by: John Snow Reviewed-by: Cleber Rosa Message-id: 20210517184808.3562549-7-js...@redhat.com Signed-off-by: John Snow

[PATCH v7 01/31] python/console_socket: avoid one-letter variable

2021-05-25 Thread John Snow
Fixes pylint warnings. Signed-off-by: John Snow Reviewed-by: Cleber Rosa Reviewed-by: Philippe Mathieu-Daudé Message-id: 20210517184808.3562549-2-js...@redhat.com Signed-off-by: John Snow --- python/qemu/console_socket.py | 10 +- 1 file changed, 5 insertions(+), 5 deletions(-) diff

[PATCH v7 00/31] python: create installable package

2021-05-25 Thread John Snow
ort-to# [RB] CR [SOB] JS [25] python-qemu-add-qemu-package# [RB] CR [TB] CR [SOB] JS [26] python-add-devel-package# [RB] CR [SOB] JS [27] python-add-avocado-framework# [RB] CR [TB] CR [SOB] JS [28] python-add-makefile-for-some# [RB] CR [TB] CR [SOB] JS [29] python-add-

[PATCH v7 02/31] python/machine: use subprocess.DEVNULL instead of open(os.path.devnull)

2021-05-25 Thread John Snow
One less file resource to manage, and it helps quiet some pylint >= 2.8.0 warnings about not using a with-context manager for the open call. Signed-off-by: John Snow Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Cleber Rosa Message-id: 20210517184808.3562549-3-js...@redhat.com Signed-off

Re: [PATCH v6 23/25] python: add .gitignore

2021-05-25 Thread John Snow
On 5/25/21 4:42 PM, Cleber Rosa wrote: On Tue, May 25, 2021 at 04:10:55PM -0400, John Snow wrote: On 5/25/21 3:36 PM, Cleber Rosa wrote: On Wed, May 12, 2021 at 07:12:39PM -0400, John Snow wrote: Ignore *Python* build and package output (build, dist, qemu.egg-info); these files are not

Re: [PATCH v6 24/25] gitlab: add python linters to CI

2021-05-25 Thread John Snow
On 5/25/21 3:55 PM, Cleber Rosa wrote: On Wed, May 12, 2021 at 07:12:40PM -0400, John Snow wrote: Add python3.6 to the fedora container image: we need it to run the linters against that explicit version to make sure we don't break our minimum version promise. Add pipenv so that we can

Re: [PATCH v6 25/25] python: add tox support

2021-05-25 Thread John Snow
On 5/25/21 4:15 PM, Cleber Rosa wrote: On Wed, May 12, 2021 at 07:12:41PM -0400, John Snow wrote: This is intended to be a manually run, non-CI script. Use tox to test the linters against all python versions from 3.6 to 3.9. This will only work if you actually have those versions installed

Re: [PATCH v6 23/25] python: add .gitignore

2021-05-25 Thread John Snow
On 5/25/21 3:36 PM, Cleber Rosa wrote: On Wed, May 12, 2021 at 07:12:39PM -0400, John Snow wrote: Ignore *Python* build and package output (build, dist, qemu.egg-info); these files are not created as part of a QEMU build. Ignore miscellaneous cached python confetti (__pycache__, *.pyc

Re: [PATCH v6 22/25] python: add Makefile for some common tasks

2021-05-25 Thread John Snow
On 5/25/21 3:24 PM, Cleber Rosa wrote: On Wed, May 12, 2021 at 07:12:38PM -0400, John Snow wrote: Add "make venv" to create the pipenv-managed virtual environment that contains our explicitly pinned dependencies. Add "make check" to run the python linters [in the host e

Re: [PATCH v6 20/25] python: add devel package requirements to setuptools

2021-05-25 Thread John Snow
On 5/25/21 12:13 PM, Cleber Rosa wrote: On Wed, May 12, 2021 at 07:12:36PM -0400, John Snow wrote: setuptools doesn't have a formal understanding of development requires, but it has an optional feataures section. Fine; add a "devel" feature and add the requirements to it. To av

Re: [PATCH v6 13/25] python: add excluded dirs to flake8 config

2021-05-25 Thread John Snow
On 5/25/21 11:50 AM, Cleber Rosa wrote: On Wed, May 12, 2021 at 07:12:29PM -0400, John Snow wrote: Instruct flake8 to avoid certain well-known directories created by python tooling that it ought not check. Note that at-present, nothing actually creates a ".venv" directory; but it

Re: [PATCH v6 06/25] python: add directory structure README.rst files

2021-05-25 Thread John Snow
On 5/24/21 10:33 PM, Cleber Rosa wrote: On Wed, May 12, 2021 at 07:12:22PM -0400, John Snow wrote: Add short readmes to python/, python/qemu/, python/qemu/machine, python/qemu/qmp, and python/qemu/utils that explain the directory hierarchy. These readmes are visible when browsing the source on

Re: [PATCH v6 18/25] python/qemu: add isort to pipenv

2021-05-25 Thread John Snow
On 5/25/21 11:56 AM, Cleber Rosa wrote: On Wed, May 12, 2021 at 07:12:34PM -0400, John Snow wrote: isort 5.0.0 through 5.0.4 has a bug that causes it to misinterpret certain "from ..." clauses that are not related to imports. isort < 5.1.1 has a bug where it does not handle

Re: [PATCH v6 07/25] python: add MANIFEST.in

2021-05-25 Thread John Snow
On 5/24/21 10:42 PM, Cleber Rosa wrote: On Wed, May 12, 2021 at 07:12:23PM -0400, John Snow wrote: When creating a source distribution via 'python3 setup.py sdist', the VERSION and PACKAGE.rst files aren't bundled by default. Create a MANIFEST.in file that instructs the build t

Re: [PATCH v2 3/3] qapi: deprecate drive-backup

2021-05-24 Thread John Snow
On 5/24/21 10:06 AM, Vladimir Sementsov-Ogievskiy wrote: 15.05.2021 01:38, John Snow wrote: On 5/6/21 5:57 AM, Kashyap Chamarthy wrote: TODO: We also need to deprecate drive-backup transaction action.. But union members in QAPI doesn't support 'deprecated' feature. I tried to

Re: [PATCH v6 03/25] python: create utils sub-package

2021-05-21 Thread John Snow
On 5/21/21 1:18 PM, Willian Rampazzo wrote: On Wed, May 12, 2021 at 8:13 PM John Snow wrote: Create a space for miscellaneous things that don't belong strictly in "qemu.machine" nor "qemu.qmp" packages. Signed-off-by: John Snow --- python/qemu/machine/_

Re: [PATCH v6 04/25] python: add qemu package installer

2021-05-21 Thread John Snow
On 5/21/21 12:00 AM, Cleber Rosa wrote: On Wed, May 12, 2021 at 07:12:20PM -0400, John Snow wrote: Add setup.cfg and setup.py, necessary for installing a package via pip. Add a ReST document (PACKAGE.rst) explaining the basics of what this package is for and who to contact for more information

Re: [PATCH] hw/ide: Fix crash when plugging a piix3-ide device into the x-remote machine

2021-05-18 Thread John Snow
On 4/28/21 5:32 AM, Thomas Huth wrote: On 28/04/2021 11.24, Stefan Hajnoczi wrote: On Fri, Apr 16, 2021 at 02:52:56PM +0200, Thomas Huth wrote: @@ -158,7 +166,11 @@ static void pci_piix_ide_realize(PCIDevice *dev, Error **errp)   vmstate_register(VMSTATE_IF(dev), 0, &vmstate_ide_pci, d); -

Re: [PATCH] hw/ide: Fix crash when plugging a piix3-ide device into the x-remote machine

2021-05-18 Thread John Snow
On 4/28/21 5:32 AM, Thomas Huth wrote: On 28/04/2021 11.24, Stefan Hajnoczi wrote: On Fri, Apr 16, 2021 at 02:52:56PM +0200, Thomas Huth wrote: @@ -158,7 +166,11 @@ static void pci_piix_ide_realize(PCIDevice *dev, Error **errp)   vmstate_register(VMSTATE_IF(dev), 0, &vmstate_ide_pci, d); -

Re: [PATCH v2 5/6] hw/ide/via: Connect IDE function output IRQs to the ISA function input

2021-05-18 Thread John Snow
On 5/11/21 12:18 AM, Philippe Mathieu-Daudé wrote: To avoid abusing isa_get_irq(NULL) using a hidden ISA bridge under the hood, let the IDE function expose 2 output IRQs, and connect them to the ISA function inputs when creating the south bridge chipset model in vt82c686b_southbridge_init. See

Re: [PATCH v2 4/6] hw/ide/via: Replace magic 2 value by ARRAY_SIZE / MAX_IDE_DEVS

2021-05-18 Thread John Snow
On 5/11/21 12:18 AM, Philippe Mathieu-Daudé wrote: Signed-off-by: Philippe Mathieu-Daudé Acked-by: John Snow --- hw/ide/via.c | 12 ++-- 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/hw/ide/via.c b/hw/ide/via.c index be09912b334..6c667a92130 100644 --- a/hw/ide

Re: [PATCH v2 02/12] hw/ide/Kconfig: Add missing dependency PCI -> IDE_QDEV

2021-05-18 Thread John Snow
ion `pci_ide_create_devs': hw/ide/pci.c:491: undefined reference to `ide_create_drive' Fixes: 8f01b41e109 ("ide: express dependencies with Kconfig") Acked-by: Paolo Bonzini Reviewed-by: Bin Meng Signed-off-by: Philippe Mathieu-Daudé Acked-by: John Snow --- hw/ide/Kconfig | 2 +- 1 fi

Re: [PATCH v2] fdc: check null block pointer before r/w data transfer

2021-05-18 Thread John Snow
On 9/22/20 5:27 AM, P J P wrote: From: Prasad J Pandit While transferring data via fdctrl_read/write_data() routines, check that current drive does not have a null block pointer. Avoid null pointer dereference. -> https://ruhr-uni-bochum.sciebo.de/s/NNWP2GfwzYKeKwE?path=%2Ffdc_nullptr1

Re: [PATCH] fdc: check null block pointer before blk_pwrite

2021-05-18 Thread John Snow
On 8/27/20 7:38 AM, P J P wrote: From: Prasad J Pandit While transferring data via fdctrl_write_data(), check that current drive does not have a null block pointer. Avoid null pointer dereference. Hi PJP. I assume this patch actually covers the exact same thing that the other if cur_drv->bl

Re: [PATCH v6 03/25] python: create utils sub-package

2021-05-18 Thread John Snow
On 5/18/21 11:12 AM, Cleber Rosa wrote: As you mentioned in the previous patch notes, I would not mind a squash here. Either way: Reviewed-by: Cleber Rosa Tested-by: Cleber Rosa I will definitely do so, then. :) Thanks for the reviews thus far! --js

Re: [PATCH v3 10/15] qemu_iotests: extent QMP socket timeout when using valgrind

2021-05-18 Thread John Snow
On 5/18/21 9:58 AM, Emanuele Giuseppe Esposito wrote: So the current plan I have for _qmp_timer is: - As Max suggested, move it in __init__ and check there for the wrapper contents. If we need to block forever (gdb, valgrind), we set it to None. Otherwise to 15 seconds. I think setting it al

Re: [PATCH v2] floppy: remove dead code related to formatting

2021-05-17 Thread John Snow
On 4/27/21 10:28 PM, Alexander Bulekov wrote: fdctrl_format_sector was added in baca51faff ("updated floppy driver: formatting code, disk geometry auto detect (Jocelyn Mayer)") The single callsite is guarded by a check: fdctrl->data_state & FD_STATE_FORMAT However, the only place where the FD_

Re: [PATCH v2] fdc: fix floppy boot for Red Hat Linux 5.2

2021-05-17 Thread John Snow
On 4/27/21 2:10 PM, John Snow wrote: The image size indicates it's an 81 track floppy disk image, which we don't have a listing for in the geometry table. When you force the drive type to 1.44MB, it guesses the reasonably close 18/80. When the drive type is allowed to auto-detect or s

Re: [PATCH v4 0/9] hw/block/fdc: Allow Kconfig-selecting ISA bus/SysBus floppy controllers

2021-05-17 Thread John Snow
On 5/17/21 4:50 PM, Philippe Mathieu-Daudé wrote: On 5/17/21 9:19 PM, John Snow wrote: On 5/17/21 2:39 PM, Philippe Mathieu-Daudé wrote: Missing review: #1 Hi, The floppy disc controllers pulls in irrelevant devices (sysbus in an ISA-only machine, ISA bus + isa devices on a sysbus-only

Re: [PATCH v4 0/9] hw/block/fdc: Allow Kconfig-selecting ISA bus/SysBus floppy controllers

2021-05-17 Thread John Snow
On 5/17/21 2:39 PM, Philippe Mathieu-Daudé wrote: Missing review: #1 Hi, The floppy disc controllers pulls in irrelevant devices (sysbus in an ISA-only machine, ISA bus + isa devices on a sysbus-only machine). This series clean that by extracting each device in its own file, adding the corresp

Re: [PATCH v6 00/25] python: create installable package

2021-05-17 Thread John Snow
On 5/12/21 7:12 PM, John Snow wrote: Based-on: <20210512214642.2803189-1-js...@redhat.com> CI: https://gitlab.com/jsnow/qemu/-/pipelines/302010131 GitLab: https://gitlab.com/jsnow/qemu/-/tree/python-package-mk3 MR: https://gitlab.com/jsnow/qemu/-/merge_requests/4 Patchset updated and r

Re: [PATCH v3 3/8] hw/block/fdc: Extract ISA floppy controllers to fdc-isa.c

2021-05-17 Thread John Snow
On 5/17/21 2:04 PM, John Snow wrote: On 5/17/21 1:49 PM, Philippe Mathieu-Daudé wrote: Some machines use floppy controllers via the SysBus interface, and don't need to pull in all the ISA code. Extract the ISA specific code to a new unit: fdc-isa.c, and add a new Kconfig symbol: &qu

Re: [PATCH v3 3/8] hw/block/fdc: Extract ISA floppy controllers to fdc-isa.c

2021-05-17 Thread John Snow
On 5/17/21 1:49 PM, Philippe Mathieu-Daudé wrote: Some machines use floppy controllers via the SysBus interface, and don't need to pull in all the ISA code. Extract the ISA specific code to a new unit: fdc-isa.c, and add a new Kconfig symbol: "FDC_ISA". Reviewed-by: John Snow

Re: [PATCH] fdc: check drive block device before usage (CVE-2021-20196)

2021-05-17 Thread John Snow
On 5/17/21 7:12 AM, P J P wrote: +-- On Sat, 15 May 2021, Philippe Mathieu-Daudé wrote --+ | This patch misses the qtest companion with the reproducer | provided by Alexander. Do we need a revised patch[-series] including a qtest? OR it can be done at merge time? Thank you. -- - P J P 8685 54

Re: [PATCH 00/10] Python: delint iotests, machine.py and console_socket.py

2021-05-17 Thread John Snow
On 5/12/21 5:46 PM, John Snow wrote: gitlab CI: https://gitlab.com/jsnow/qemu/-/pipelines/301924893 branch: https://gitlab.com/jsnow/qemu/-/commits/python-package-pre-cleanup This series serves as a pre-requisite for packaging the python series and getting the linters running via CI. The first

Re: [PATCH v2 3/3] qapi: deprecate drive-backup

2021-05-14 Thread John Snow
On 5/6/21 5:57 AM, Kashyap Chamarthy wrote: TODO: We also need to deprecate drive-backup transaction action.. But union members in QAPI doesn't support 'deprecated' feature. I tried to dig a bit, but failed :/ Markus, could you please help with it? At least by advice? Oho, I see. OK, I'm not M

Re: [PATCH v2 2/3] docs/interop/bitmaps: use blockdev-backup

2021-05-14 Thread John Snow
On 5/5/21 9:58 AM, Vladimir Sementsov-Ogievskiy wrote: We are going to deprecate drive-backup, so use modern interface here. In examples where target image creation is shown, show blockdev-add as well. If target creation omitted, omit blockdev-add as well. Signed-off-by: Vladimir Sementsov-Ogiev

Re: [PATCH v2 1/3] docs/block-replication: use blockdev-backup

2021-05-14 Thread John Snow
On 5/5/21 9:58 AM, Vladimir Sementsov-Ogievskiy wrote: We are going to deprecate drive-backup, so don't mention it here. Moreover, blockdev-backup seems more correct in the context. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow --- docs/block-replication.tx

Re: [PATCH v3 10/15] qemu_iotests: extent QMP socket timeout when using valgrind

2021-05-14 Thread John Snow
On 5/14/21 4:16 AM, Emanuele Giuseppe Esposito wrote: On 13/05/2021 20:47, John Snow wrote: On 4/14/21 1:03 PM, Emanuele Giuseppe Esposito wrote: As with gdbserver, valgrind delays the test execution, so the default QMP socket timeout timeout too soon. Signed-off-by: Emanuele Giuseppe

Re: [PATCH] fdc: check drive block device before usage (CVE-2021-20196)

2021-05-14 Thread John Snow
On 5/14/21 3:23 PM, Thomas Huth wrote: On 23/01/2021 11.03, P J P wrote: From: Prasad J Pandit While processing ioport command in 'fdctrl_write_dor', device controller may select a drive which is not initialised with a block device. This may result in a NULL pointer dereference. Add checks to

Re: [PATCH 07/10] iotests: use subprocess.run where possible

2021-05-14 Thread John Snow
On 5/12/21 5:46 PM, John Snow wrote: pylint 2.8.x adds warnings whenever we use Popen calls without using 'with', so it's desirable to convert synchronous calls to run() invocations where applicable. (Though, this trades one pylint warning for another due to a pylint bug, whic

Re: [PATCH 03/10] python/machine: use subprocess.run instead of subprocess.Popen

2021-05-14 Thread John Snow
On 5/14/21 10:08 AM, Wainer dos Santos Moschetta wrote: Now it might throw a CalledProcessError given that `check=True`. Shouldn't it capture the exception and (possible) re-throw as an QEMUMachineError? I lied to you again. The existing callers all check for failure explicitly, so in the int

Re: [PATCH 05/10] python/machine: Disable pylint warning for open() in _pre_launch

2021-05-14 Thread John Snow
On 5/14/21 10:42 AM, Wainer dos Santos Moschetta wrote: Hi, On 5/12/21 6:46 PM, John Snow wrote: Shift the open() call later so that the pylint pragma applies *only* to that one open() call. Add a note that suggests why this is safe: the resource is unconditionally cleaned up in _post_shutdown

Re: [PATCH 03/10] python/machine: use subprocess.run instead of subprocess.Popen

2021-05-14 Thread John Snow
On 5/14/21 10:08 AM, Wainer dos Santos Moschetta wrote: Hi, On 5/12/21 6:46 PM, John Snow wrote: use run() instead of Popen() -- to assert to pylint that we are not forgetting to close a long-running program. Signed-off-by: John Snow ---   python/qemu/machine.py | 15 +--   1 file

Re: [PATCH v3 10/15] qemu_iotests: extent QMP socket timeout when using valgrind

2021-05-13 Thread John Snow
On 4/14/21 1:03 PM, Emanuele Giuseppe Esposito wrote: As with gdbserver, valgrind delays the test execution, so the default QMP socket timeout timeout too soon. Signed-off-by: Emanuele Giuseppe Esposito --- python/qemu/machine.py| 2 +- tests/qemu-iotests/iotests.py | 4 ++-- 2 file

Re: [PATCH v3 05/15] qemu-iotests: delay QMP socket timers

2021-05-13 Thread John Snow
On 5/3/21 11:02 AM, Max Reitz wrote: On 30.04.21 23:03, Emanuele Giuseppe Esposito wrote: On 30/04/2021 13:59, Max Reitz wrote: On 14.04.21 19:03, Emanuele Giuseppe Esposito wrote: Attaching a gdbserver implies that the qmp socket should wait indefinitely for an answer from QEMU. Signed-off

Re: [PATCH v3 02/15] python: qemu: pass the wrapper field from QEMUQtestmachine to QEMUMachine

2021-05-13 Thread John Snow
On 4/14/21 1:03 PM, Emanuele Giuseppe Esposito wrote: Signed-off-by: Emanuele Giuseppe Esposito Assuming it doesn't make the linter explode, which I didn't run: Reviewed-by: John Snow --- python/qemu/qtest.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --gi

Re: [PATCH v3 01/15] python: qemu: add timer parameter for qmp.accept socket

2021-05-13 Thread John Snow
unded blocking waits? I assume this came up in development or you'd not have added it. Reviewed-by: John Snow Signed-off-by: Emanuele Giuseppe Esposito --- python/qemu/machine.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/python/qemu/machine.py b/python/qemu/

[PATCH v6 24/25] gitlab: add python linters to CI

2021-05-12 Thread John Snow
Add python3.6 to the fedora container image: we need it to run the linters against that explicit version to make sure we don't break our minimum version promise. Add pipenv so that we can fetch precise versions of pip packages we need to guarantee test reproducability. Signed-off-by: John

[PATCH v6 25/25] python: add tox support

2021-05-12 Thread John Snow
o work around the issue. Signed-off-by: John Snow --- python/README.rst | 2 ++ python/.gitignore | 1 + python/Makefile | 7 ++- python/setup.cfg | 1 + python/tox.ini| 13 + 5 files changed, 23 insertions(+), 1 deletion(-) create mode 100644 python/tox.ini diff -

[PATCH v6 22/25] python: add Makefile for some common tasks

2021-05-12 Thread John Snow
te miscellaneous python packaging output possibly created by pipenv, pip, or other python packaging utilities make distclean: delete the above, the .venv, and the editable "qemu" package forwarder (qemu.egg-info) if there is one. Signed-off-by: John Snow --- python/README.rst | 3 +++

[PATCH v6 18/25] python/qemu: add isort to pipenv

2021-05-12 Thread John Snow
-c qemu' from the python root. Signed-off-by: John Snow --- python/Pipfile | 1 + python/Pipfile.lock | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/python/Pipfile b/python/Pipfile index bb2e34b381d..fb7c8d142ee 100644 --- a/python/Pipfile +++ b/python/Pipfile @@

[PATCH v6 19/25] python/qemu: add qemu package itself to pipenv

2021-05-12 Thread John Snow
he qemu packages are installed in 'editable' mode; all changes made to the python package inside the git tree will be reflected in the installed package without reinstallation. This includes changes made via git pull and so on. Signed-off-by: John Snow Reviewed-by: Cleber Rosa Teste

[PATCH v6 17/25] python: move .isort.cfg into setup.cfg

2021-05-12 Thread John Snow
Signed-off-by: John Snow Reviewed-by: Cleber Rosa --- python/.isort.cfg | 7 --- python/setup.cfg | 8 2 files changed, 8 insertions(+), 7 deletions(-) delete mode 100644 python/.isort.cfg diff --git a/python/.isort.cfg b/python/.isort.cfg deleted file mode 100644 index

[PATCH v6 16/25] python: add mypy to pipenv

2021-05-12 Thread John Snow
t path detection mechanisms in mypy slightly, and it will fail. See https://github.com/python/mypy/issues/8584 for a decent entry point with more breadcrumbs on the various behaviors that contribute to this subtle difference. Signed-off-by: John Snow Reviewed-by: Cleber Rosa Tested-by: Cleber Rosa

[PATCH v6 23/25] python: add .gitignore

2021-05-12 Thread John Snow
Ignore *Python* build and package output (build, dist, qemu.egg-info); these files are not created as part of a QEMU build. Ignore miscellaneous cached python confetti (__pycache__, *.pyc, .mypy_cache). Ignore .idea (pycharm) .vscode, and .venv (pipenv et al). Signed-off-by: John Snow

[PATCH v6 21/25] python: add avocado-framework and tests

2021-05-12 Thread John Snow
correct arguments. (A forthcoming commit adds the much easier 'make check'.) Signed-off-by: John Snow --- python/README.rst | 2 + python/Pipfile.lock| 104 ++--- python/avocado.cfg | 10 python/setup.cfg | 1 + python/test

[PATCH v6 20/25] python: add devel package requirements to setuptools

2021-05-12 Thread John Snow
elopment dependencies are not already met. This can be useful for automated build scripts where fetching network packages may be undesirable. Signed-off-by: John Snow --- python/Pipfile | 5 + python/Pipfile.lock | 14 +- python/setup.cfg| 9 + 3 files cha

[PATCH v6 15/25] python: move mypy.ini into setup.cfg

2021-05-12 Thread John Snow
mypy supports reading its configuration values from a central project configuration file; do so. Signed-off-by: John Snow --- python/mypy.ini | 4 python/setup.cfg | 5 + 2 files changed, 5 insertions(+), 4 deletions(-) delete mode 100644 python/mypy.ini diff --git a/python/mypy.ini

[PATCH v6 10/25] python: move pylintrc into setup.cfg

2021-05-12 Thread John Snow
Delete the empty settings now that it's sharing a home with settings for other tools. pylint can now be run from this folder as "pylint qemu". Signed-off-by: John Snow Reviewed-by: Cleber Rosa Tested-by: Cleber Rosa --- python/qemu/machin

[PATCH v6 09/25] python: add pylint import exceptions

2021-05-12 Thread John Snow
that are too old for features we want in isort and mypy. Oh well. Signed-off-by: John Snow Reviewed-by: Cleber Rosa --- python/qemu/machine/__init__.py | 3 +++ python/qemu/machine/machine.py | 2 +- python/qemu/machine/qtest.py| 2 +- 3 files changed, 5 insertions(+), 2 deletions(-) diff

[PATCH v6 14/25] python: Add flake8 to pipenv

2021-05-12 Thread John Snow
flake8 3.5.x does not support the --extend-ignore syntax used in the .flake8 file to gracefully extend default ignores, so 3.6.x is our minimum requirement. There is no known upper bound. flake8 can be run from the python/ directory with no arguments. Signed-off-by: John Snow Reviewed-by

[PATCH v6 06/25] python: add directory structure README.rst files

2021-05-12 Thread John Snow
. They are not designed for inclusion in a published manual. Signed-off-by: John Snow --- python/README.rst | 41 ++ python/qemu/README.rst | 8 +++ python/qemu/machine/README.rst | 9 python/qemu/qmp/README.rst | 9

[PATCH v6 13/25] python: add excluded dirs to flake8 config

2021-05-12 Thread John Snow
uld be excluded anyway. A forthcoming commit canonizes this with a "make venv" command. Signed-off-by: John Snow --- python/setup.cfg | 2 ++ 1 file changed, 2 insertions(+) diff --git a/python/setup.cfg b/python/setup.cfg index 9ecb2902006..f21a1c42fc0 100644 --- a/python/setup.c

<    9   10   11   12   13   14   15   16   17   18   >