Re: [PATCH v2 03/15] net/vhost-vdpa: Fix device compatibility check

2021-10-13 Thread Jason Wang
在 2021/10/8 下午9:34, Kevin Wolf 写道: vhost-vdpa works only with specific devices. At startup, it second guesses what the command line option handling will do and error out if it thinks a non-virtio device will attach to them. This second guessing is not only ugly, it can lead to wrong error

Re: [PATCH v2 02/15] net/vhost-user: Fix device compatibility check

2021-10-13 Thread Jason Wang
在 2021/10/8 下午9:34, Kevin Wolf 写道: vhost-user works only with specific devices. At startup, it second guesses what the command line option handling will do and error out if it thinks a non-virtio device will attach to them. This second guessing is not only ugly, it can lead to wrong error

Re: [PATCH v2 01/15] net: Introduce NetClientInfo.check_peer_type()

2021-10-13 Thread Jason Wang
在 2021/10/8 下午9:34, Kevin Wolf 写道: Some network backends (vhost-user and vhost-vdpa) work only with specific devices. At startup, they second guess what the command line option handling will do and error out if they think a non-virtio device will attach to them. This second guessing is not

[PATCH v4 2/8] python/machine: Handle QMP errors on close more meticulously

2021-10-13 Thread John Snow
To use the AQMP backend, Machine just needs to be a little more diligent about what happens when closing a QMP connection. The operation is no longer a freebie in the async world; it may return errors encountered in the async bottom half on incoming message receipt, etc. (AQMP's disconnect,

[PATCH v4 7/8] python/aqmp: Create sync QMP wrapper for iotests

2021-10-13 Thread John Snow
This is a wrapper around the async QMPClient that mimics the old, synchronous QEMUMonitorProtocol class. It is designed to be interchangeable with the old implementation. It does not, however, attempt to mimic Exception compatibility. Signed-off-by: John Snow Acked-by: Hanna Reitz ---

[PATCH v4 8/8] python, iotests: replace qmp with aqmp

2021-10-13 Thread John Snow
Swap out the synchronous QEMUMonitorProtocol from qemu.qmp with the sync wrapper from qemu.aqmp instead. Add an escape hatch in the form of the environment variable QEMU_PYTHON_LEGACY_QMP which allows you to cajole QEMUMachine into using the old implementation, proving that both implementations

[PATCH v4 4/8] iotests: Accommodate async QMP Exception classes

2021-10-13 Thread John Snow
(But continue to support the old ones for now, too.) There are very few cases of any user of QEMUMachine or a subclass thereof relying on a QMP Exception type. If you'd like to check for yourself, you want to grep for all of the derivatives of QMPError, excluding 'AQMPError' and its derivatives.

[PATCH v4 3/8] python/aqmp: Remove scary message

2021-10-13 Thread John Snow
The scary message interferes with the iotests output. Coincidentally, if iotests works by removing this, then it's good evidence that we don't really need to scare people away from using it. Signed-off-by: John Snow Acked-by: Hanna Reitz --- python/qemu/aqmp/__init__.py | 12 1

[PATCH v4 5/8] iotests: Conditionally silence certain AQMP errors

2021-10-13 Thread John Snow
AQMP likes to be very chatty about errors it encounters. In general, this is good because it allows us to get good diagnostic information for otherwise complex async failures. For example, during a failed QMP connection attempt, we might see: +ERROR:qemu.aqmp.qmp_client.qemub-2536319:Negotiation

[PATCH v4 6/8] iotests/300: avoid abnormal shutdown race condition

2021-10-13 Thread John Snow
Wait for the destination VM to close itself instead of racing to shut it down first, which produces different error log messages from AQMP depending on precisely when we tried to shut it down. (For example: We may try to issue 'quit' immediately prior to the target VM closing its QMP socket,

[PATCH v4 0/8] Switch iotests to using Async QMP

2021-10-13 Thread John Snow
GitLab: https://gitlab.com/jsnow/qemu/-/commits/python-aqmp-iotest-wrapper CI: https://gitlab.com/jsnow/qemu/-/pipelines/387972757 Hiya, This series continues where the last two AQMP series left off and adds a synchronous 'legacy' wrapper around the new AQMP interface, then drops it straight

[PATCH v4 1/8] python/machine: remove has_quit argument

2021-10-13 Thread John Snow
If we spy on the QMP commands instead, we don't need callers to remember to pass it. Seems like a fair trade-off. The one slightly weird bit is overloading this instance variable for wait(), where we use it to mean "don't issue the qmp 'quit' command". This means that wait() will "fail" if the

iotest 030 SIGSEGV

2021-10-13 Thread John Snow
In trying to replace the QMP library backend, I have now twice stumbled upon a SIGSEGV in iotest 030 in the last three weeks or so. I didn't have debug symbols on at the time, so I've got only this stack trace: (gdb) thread apply all bt Thread 8 (Thread 0x7f0a6b8c4640 (LWP 1873554)): #0

Re: [PATCH v2 09/15] softmmu/qdev-monitor: add error handling in qdev_set_id

2021-10-13 Thread Eric Blake
On Wed, Oct 13, 2021 at 03:10:38PM +0200, Damien Hedde wrote: > > > @@ -691,7 +703,13 @@ DeviceState *qdev_device_add(QemuOpts *opts, Error > > > **errp) > > > } > > > } > > > -qdev_set_id(dev, g_strdup(qemu_opts_id(opts))); > > > +/* > > > + * set dev's parent and

Re: [PATCH v2 00/15] qdev: Add JSON -device

2021-10-13 Thread Kevin Wolf
Am 13.10.2021 um 17:30 hat Michael S. Tsirkin geschrieben: > On Fri, Oct 08, 2021 at 03:34:27PM +0200, Kevin Wolf wrote: > > It's still a long way until we'll have QAPIfied devices, but there are > > some improvements that we can already make now to make the future switch > > easier. > > > > One

Re: [PATCH v3 0/7] Switch iotests to using Async QMP

2021-10-13 Thread John Snow
On Wed, Oct 13, 2021 at 10:49 AM Hanna Reitz wrote: > On 13.10.21 16:00, John Snow wrote: > > > > > > On Wed, Oct 13, 2021 at 8:51 AM John Snow wrote: > > > > > > > > On Wed, Oct 13, 2021 at 4:45 AM Hanna Reitz > wrote: > > > > On 13.10.21 00:34, John Snow wrote: > > >

Re: [PATCH 09/13] iotests: split linters.py out from 297

2021-10-13 Thread Hanna Reitz
On 13.10.21 17:07, John Snow wrote: On Wed, Oct 13, 2021 at 7:50 AM Hanna Reitz wrote: On 04.10.21 23:04, John Snow wrote: > Now, 297 is just the iotests-specific incantations and linters.py is as > minimal as I can think to make it. The only remaining element in here

Re: [PATCH v2 00/15] qdev: Add JSON -device

2021-10-13 Thread Michael S. Tsirkin
On Fri, Oct 08, 2021 at 03:34:27PM +0200, Kevin Wolf wrote: > It's still a long way until we'll have QAPIfied devices, but there are > some improvements that we can already make now to make the future switch > easier. > > One important part of this is having code paths without QemuOpts, which >

Re: [PATCH 10/13] iotests/linters: Add entry point for linting via Python CI

2021-10-13 Thread John Snow
On Wed, Oct 13, 2021 at 8:11 AM Hanna Reitz wrote: > On 04.10.21 23:05, John Snow wrote: > > We need at least a tiny little shim here to join test file discovery > > with test invocation. This logic could conceivably be hosted somewhere > > in python/, but I felt it was strictly the least-rude

Re: [PATCH 09/13] iotests: split linters.py out from 297

2021-10-13 Thread John Snow
On Wed, Oct 13, 2021 at 7:50 AM Hanna Reitz wrote: > On 04.10.21 23:04, John Snow wrote: > > Now, 297 is just the iotests-specific incantations and linters.py is as > > minimal as I can think to make it. The only remaining element in here > > that ought to be configuration and not code is the

Re: [PATCH v3 0/7] Switch iotests to using Async QMP

2021-10-13 Thread Hanna Reitz
On 13.10.21 16:00, John Snow wrote: On Wed, Oct 13, 2021 at 8:51 AM John Snow wrote: On Wed, Oct 13, 2021 at 4:45 AM Hanna Reitz wrote: On 13.10.21 00:34, John Snow wrote: > Based-on: <20211012214152.802483-1-js...@redhat.com> >            [PULL 00/10] Python

Re: [PATCH 05/13] iotests/297: Create main() function

2021-10-13 Thread John Snow
On Wed, Oct 13, 2021 at 7:03 AM Hanna Reitz wrote: > On 04.10.21 23:04, John Snow wrote: > > Instead of running "run_linters" directly, create a main() function that > > will be responsible for environment setup, leaving run_linters() > > responsible only for execution of the linters. > > > >

Re: [PATCH 02/13] iotests/297: Split mypy configuration out into mypy.ini

2021-10-13 Thread John Snow
On Wed, Oct 13, 2021 at 6:53 AM Hanna Reitz wrote: > On 04.10.21 23:04, John Snow wrote: > > More separation of code and configuration. > > > > Signed-off-by: John Snow > > --- > > tests/qemu-iotests/297 | 14 +- > > tests/qemu-iotests/mypy.ini | 12 > > 2

Re: [PATCH v3 7/7] python, iotests: replace qmp with aqmp

2021-10-13 Thread Eric Blake
On Tue, Oct 12, 2021 at 06:34:45PM -0400, John Snow wrote: > Swap out the synchronous QEMUMonitorProtocol from qemu.qmp with the sync > wrapper from qemu.aqmp instead. > > Add an escape hatch in the form of the environment variable > QEMU_PYTHON_LEGACY_QMP which allows you to cajole QEMUMachine

Re: [PATCH v3 0/7] Switch iotests to using Async QMP

2021-10-13 Thread John Snow
On Wed, Oct 13, 2021 at 8:51 AM John Snow wrote: > > > On Wed, Oct 13, 2021 at 4:45 AM Hanna Reitz wrote: > >> On 13.10.21 00:34, John Snow wrote: >> > Based-on: <20211012214152.802483-1-js...@redhat.com> >> >[PULL 00/10] Python patches >> > GitLab: >>

Re: [PATCH v2 08/15] qdev: Make DeviceState.id independent of QemuOpts

2021-10-13 Thread Damien Hedde
On 10/8/21 15:34, Kevin Wolf wrote: DeviceState.id is a pointer to a string that is stored in the QemuOpts object DeviceState.opts and freed together with it. We want to create devices without going through QemuOpts in the future, so make this a separately allocated string. Signed-off-by:

Re: [PATCH v2 02/15] net/vhost-user: Fix device compatibility check

2021-10-13 Thread Damien Hedde
On 10/8/21 15:34, Kevin Wolf wrote: vhost-user works only with specific devices. At startup, it second guesses what the command line option handling will do and error out if it thinks a non-virtio device will attach to them. This second guessing is not only ugly, it can lead to wrong error

Re: [PATCH v2 03/15] net/vhost-vdpa: Fix device compatibility check

2021-10-13 Thread Damien Hedde
On 10/8/21 15:34, Kevin Wolf wrote: vhost-vdpa works only with specific devices. At startup, it second guesses what the command line option handling will do and error out if it thinks a non-virtio device will attach to them. This second guessing is not only ugly, it can lead to wrong error

Re: [PATCH v2 01/15] net: Introduce NetClientInfo.check_peer_type()

2021-10-13 Thread Damien Hedde
On 10/8/21 15:34, Kevin Wolf wrote: Some network backends (vhost-user and vhost-vdpa) work only with specific devices. At startup, they second guess what the command line option handling will do and error out if they think a non-virtio device will attach to them. This second guessing is not

Re: [PATCH v2 09/15] softmmu/qdev-monitor: add error handling in qdev_set_id

2021-10-13 Thread Damien Hedde
On 10/11/21 23:00, Eric Blake wrote: On Fri, Oct 08, 2021 at 03:34:36PM +0200, Kevin Wolf wrote: From: Damien Hedde qdev_set_id() is mostly used when the user adds a device (using -device cli option or device_add qmp command). This commit adds an error parameter to handle the case where

Re: [PATCH v3 0/7] Switch iotests to using Async QMP

2021-10-13 Thread John Snow
On Wed, Oct 13, 2021 at 4:45 AM Hanna Reitz wrote: > On 13.10.21 00:34, John Snow wrote: > > Based-on: <20211012214152.802483-1-js...@redhat.com> > >[PULL 00/10] Python patches > > GitLab: > https://gitlab.com/jsnow/qemu/-/commits/python-aqmp-iotest-wrapper > > CI:

Re: [PATCH 13/13] iotests: [RFC] drop iotest 297

2021-10-13 Thread Hanna Reitz
On 04.10.21 23:05, John Snow wrote: (This is highlighting a what-if, which might make it clear why any special infrastructure is still required at all. It's not intended to actually be merged at this step -- running JUST the iotest linters from e.g. 'make check' is not yet accommodated, so

Re: [PATCH 12/13] python: Add iotest linters to test suite

2021-10-13 Thread Hanna Reitz
On 04.10.21 23:05, John Snow wrote: Run mypy and pylint on the iotests files directly from the Python CI test infrastructure. This ensures that any accidental breakages to the qemu.[qmp|aqmp|machine|utils] packages will be caught by that test suite. It also ensures that these linters are run

Re: [PATCH 11/13] iotests/linters: Add workaround for mypy bug #9852

2021-10-13 Thread Hanna Reitz
On 04.10.21 23:05, John Snow wrote: This one is insidious: if you write an import as "from {namespace} import {subpackage}" as mirror-top-perms (now) does, mypy will fail on every-other invocation *if* the package being imported is a typed, installed, namespace-scoped package. Upsettingly,

Re: [PATCH 10/13] iotests/linters: Add entry point for linting via Python CI

2021-10-13 Thread Hanna Reitz
On 04.10.21 23:05, John Snow wrote: We need at least a tiny little shim here to join test file discovery with test invocation. This logic could conceivably be hosted somewhere in python/, but I felt it was strictly the least-rude thing to keep the test logic here in iotests/, even if this small

Re: [PATCH 09/13] iotests: split linters.py out from 297

2021-10-13 Thread Hanna Reitz
On 04.10.21 23:04, John Snow wrote: Now, 297 is just the iotests-specific incantations and linters.py is as minimal as I can think to make it. The only remaining element in here that ought to be configuration and not code is the list of skip files, Yeah... but they're still numerous enough

Re: [PATCH 08/13] iotests/297: refactor run_[mypy|pylint] as generic execution shim

2021-10-13 Thread Hanna Reitz
On 04.10.21 23:04, John Snow wrote: There's virtually nothing special here anymore; we can combine these into a single, rather generic function. Signed-off-by: John Snow --- tests/qemu-iotests/297 | 46 +++--- 1 file changed, 25 insertions(+), 21

Re: [PATCH 07/13] iotests/297: Split run_linters apart into run_pylint and run_mypy

2021-10-13 Thread Hanna Reitz
On 04.10.21 23:04, John Snow wrote: Signed-off-by: John Snow --- Note, this patch really ought to be squashed with the next one, Yes, it should be. but I am performing a move known as "Hedging my bets." It's easier to squash than de-squash :) True.  Still, should be squashed. ;)

Re: [PATCH 06/13] iotests/297: Separate environment setup from test execution

2021-10-13 Thread Hanna Reitz
On 04.10.21 23:04, John Snow wrote: Move environment setup into main(), leaving pure test execution behind in run_linters(). Signed-off-by: John Snow --- tests/qemu-iotests/297 | 23 ++- 1 file changed, 14 insertions(+), 9 deletions(-) Reviewed-by: Hanna Reitz

Re: [PATCH 05/13] iotests/297: Create main() function

2021-10-13 Thread Hanna Reitz
On 04.10.21 23:04, John Snow wrote: Instead of running "run_linters" directly, create a main() function that will be responsible for environment setup, leaving run_linters() responsible only for execution of the linters. (That environment setup will be moved over in forthcoming commits.)

Re: [PATCH 03/13] iotests/297: Add get_files() function

2021-10-13 Thread Hanna Reitz
On 04.10.21 23:04, John Snow wrote: Split out file discovery into its own method to begin separating out configuration/setup and test execution. Signed-off-by: John Snow --- tests/qemu-iotests/297 | 9 +++-- 1 file changed, 7 insertions(+), 2 deletions(-) Reviewed-by: Hanna Reitz

Re: [PATCH 02/13] iotests/297: Split mypy configuration out into mypy.ini

2021-10-13 Thread Hanna Reitz
On 04.10.21 23:04, John Snow wrote: More separation of code and configuration. Signed-off-by: John Snow --- tests/qemu-iotests/297 | 14 +- tests/qemu-iotests/mypy.ini | 12 2 files changed, 13 insertions(+), 13 deletions(-) create mode 100644

Re: [PATCH 01/13] iotests/297: Move pylint config into pylintrc

2021-10-13 Thread Hanna Reitz
On 04.10.21 23:04, John Snow wrote: Move --score=n and --notes=XXX,FIXME into pylintrc. This pulls configuration out of code, which I think is probably a good thing in general. Signed-off-by: John Snow --- tests/qemu-iotests/297 | 4 +--- tests/qemu-iotests/pylintrc | 16

Re: [PATCH] block/vpc: Add a sanity check that fixed-size images have the right type

2021-10-13 Thread Hanna Reitz
On 12.10.21 10:27, Thomas Huth wrote: The code in vpc.c uses BDRVVPCState->footer.type in various places to decide whether the image is a fixed-size (VHD_FIXED) or a dynamic (VHD_DYNAMIC) image. However, we never check that this field really contains VHD_FIXED if we detected a fixed size image

Re: [PATCH 04/15] pcie: Add callback preceding SR-IOV VFs update

2021-10-13 Thread Michael S. Tsirkin
On Tue, Oct 12, 2021 at 06:06:46PM +0200, Lukasz Maniak wrote: > On Tue, Oct 12, 2021 at 03:25:12AM -0400, Michael S. Tsirkin wrote: > > On Thu, Oct 07, 2021 at 06:23:55PM +0200, Lukasz Maniak wrote: > > > PCIe devices implementing SR-IOV may need to perform certain actions > > > before the VFs

Re: [PATCH v3 0/7] Switch iotests to using Async QMP

2021-10-13 Thread Hanna Reitz
On 13.10.21 00:34, John Snow wrote: Based-on: <20211012214152.802483-1-js...@redhat.com> [PULL 00/10] Python patches GitLab: https://gitlab.com/jsnow/qemu/-/commits/python-aqmp-iotest-wrapper CI: https://gitlab.com/jsnow/qemu/-/pipelines/387210591 Hiya, This series continues where

Re: [PATCH v3 6/7] python/aqmp: Create sync QMP wrapper for iotests

2021-10-13 Thread Hanna Reitz
On 13.10.21 00:34, John Snow wrote: This is a wrapper around the async QMPClient that mimics the old, synchronous QEMUMonitorProtocol class. It is designed to be interchangeable with the old implementation. It does not, however, attempt to mimic Exception compatibility. Signed-off-by: John

Re: [PATCH v3 5/7] iotests: Conditionally silence certain AQMP errors

2021-10-13 Thread Hanna Reitz
On 13.10.21 00:34, John Snow wrote: AQMP likes to be very chatty about errors it encounters. In general, this is good because it allows us to get good diagnostic information for otherwise complex async failures. For example, during a failed QMP connection attempt, we might see:

Re: [PATCH v3 4/7] iotests: Accommodate async QMP Exception classes

2021-10-13 Thread Hanna Reitz
On 13.10.21 00:34, John Snow wrote: (But continue to support the old ones for now, too.) There are very few cases of any user of QEMUMachine or a subclass thereof relying on a QMP Exception type. If you'd like to check for yourself, you want to grep for all of the derivatives of QMPError,

Re: [PATCH v3 3/7] python/aqmp: Remove scary message

2021-10-13 Thread Hanna Reitz
On 13.10.21 00:34, John Snow wrote: The scary message interferes with the iotests output. Coincidentally, if iotests works by removing this, then it's good evidence that we don't really need to scare people away from using it. Signed-off-by: John Snow --- python/qemu/aqmp/__init__.py | 12

Re: [PATCH v3 2/7] python/machine: Handle QMP errors on close more meticulously

2021-10-13 Thread Hanna Reitz
On 13.10.21 00:34, John Snow wrote: To use the AQMP backend, Machine just needs to be a little more diligent about what happens when closing a QMP connection. The operation is no longer a freebie in the async world; it may return errors encountered in the async bottom half on incoming message