Re: [PATCH v2 4/7] configure: Add nice hint to Python failure message

2023-02-09 Thread Thomas Huth
On 10/02/2023 01.31, John Snow wrote: If we begin requiring Python 3.7+, a few platforms are going to need to install an additional package. This is at least mildly annoying to the user (and I hate negative attention), so solve the user's problem for them before they get a chance to become

Re: [PATCH v2 3/7] configure: Look for auxiliary Python installations

2023-02-09 Thread Thomas Huth
On 10/02/2023 01.31, John Snow wrote: At the moment, we look for just "python3" and "python", which is good enough almost all of the time. But ... if you are on a platform that uses an older Python by default and only offers a newer Python as an option, you'll have to specify

Re: [PATCH v2 6/7] CI: Stop building docs on centos8

2023-02-09 Thread Philippe Mathieu-Daudé
On 10/2/23 01:31, John Snow wrote: CentOS 8 does not ship with a sphinx new enough for our purposes (It necessarily uses Python 3.6), so drop this from this build. We can resume building docs on CentOS 9 if we wish, but we also currently test and build docs on Fedora, Ubuntu, Alpine and Debian.

[PATCH v2 0/7] Python: Drop support for Python 3.6

2023-02-09 Thread John Snow
Howdy, this series increases our minimum python version to 3.7. CI: https://gitlab.com/jsnow/qemu/-/pipelines/771780626 (All green!) GL: https://gitlab.com/jsnow/qemu/-/commits/python-require-37 Patches 1 and 2 are loose pre-requisites; I'd like to merge them into qemu.git within the week

[PATCH v2 2/7] python: drop pipenv

2023-02-09 Thread John Snow
The pipenv tool was nice in theory, but in practice it's just too hard to update selectively, and it makes using it a pain. The qemu.qmp repo dropped pipenv support a while back and it's been functioning just fine, so I'm backporting that change here to qemu.git. Signed-off-by: John Snow ---

[PATCH v2 7/7] Python: Drop support for Python 3.6

2023-02-09 Thread John Snow
Python 3.6 was EOL 2021-12-31. Newer versions of upstream libraries have begun dropping support for this version and it is becoming more cumbersome to support. Avocado-framework and qemu.qmp each have their own reasons for wanting to drop Python 3.6, but won't until QEMU does. Versions of Python

[PATCH v2 3/7] configure: Look for auxiliary Python installations

2023-02-09 Thread John Snow
At the moment, we look for just "python3" and "python", which is good enough almost all of the time. But ... if you are on a platform that uses an older Python by default and only offers a newer Python as an option, you'll have to specify --python=/usr/bin/foo every time. We can be kind and

[PATCH v2 1/7] python: support pylint 2.16

2023-02-09 Thread John Snow
Pylint 2.16 adds a few new checks that cause the optional check-tox CI job to fail. 1. The superfluous-parens check seems to be a bit more aggressive, 2. broad-exception-raised is new; it discourages "raise Exception". Fix these minor issues and turn the lights green. Signed-off-by: John Snow

[PATCH v2 5/7] DO-NOT-MERGE: testing: Add Python >= 3.7 to Centos, OpenSuSE

2023-02-09 Thread John Snow
This is just a proof-of-concept patch, as these files are lcitool generated. The real fix will involve updating the lcitool configuration and updating these files that way. This is just to prove that bumping our dependency works. Signed-off-by: John Snow ---

[PATCH v2 6/7] CI: Stop building docs on centos8

2023-02-09 Thread John Snow
CentOS 8 does not ship with a sphinx new enough for our purposes (It necessarily uses Python 3.6), so drop this from this build. We can resume building docs on CentOS 9 if we wish, but we also currently test and build docs on Fedora, Ubuntu, Alpine and Debian. Signed-off-by: John Snow ---

[PATCH v2 4/7] configure: Add nice hint to Python failure message

2023-02-09 Thread John Snow
If we begin requiring Python 3.7+, a few platforms are going to need to install an additional package. This is at least mildly annoying to the user (and I hate negative attention), so solve the user's problem for them before they get a chance to become irritated while searching on Google for how

Re: [PATCH 6/7] CI: Stop building docs on centos8

2023-02-09 Thread John Snow
On Thu, Feb 9, 2023 at 5:43 PM Paolo Bonzini wrote: > > > > Il gio 9 feb 2023, 16:40 John Snow ha scritto: >> >> CentOS 8 does not ship with a sphinx new enough for our purposes (It >> necessarily uses Python 3.6), so drop this from this build. We can >> resume building docs on CentOS 9 if we

Re: [PATCH 6/7] CI: Stop building docs on centos8

2023-02-09 Thread Paolo Bonzini
Il gio 9 feb 2023, 16:40 John Snow ha scritto: > CentOS 8 does not ship with a sphinx new enough for our purposes (It > necessarily uses Python 3.6), so drop this from this build. We can > resume building docs on CentOS 9 if we wish, but we also currently test > and build docs on Fedora, Ubuntu,

Re: [PATCH 7/7] Python: Drop support for Python 3.6

2023-02-09 Thread John Snow
On Thu, Feb 9, 2023 at 1:03 PM Daniel P. Berrangé wrote: > > On Thu, Feb 09, 2023 at 11:40:57AM -0500, John Snow wrote: > > On Thu, Feb 9, 2023 at 11:27 AM Daniel P. Berrangé > > wrote: > > > > > > On Thu, Feb 09, 2023 at 10:40:34AM -0500, John Snow wrote: > > > > Python 3.6 was EOL 2021-12-31.

Re: [PATCH 1/3] migration: In case of postcopy, the memory ends in res_postcopy_only

2023-02-09 Thread Juan Quintela
Vladimir Sementsov-Ogievskiy wrote: > On 08.02.23 16:57, Juan Quintela wrote: >> So remove last assignation of res_compatible. > > I hoped for some description when asked to split it out :) > >> Signed-off-by: Juan Quintela >> --- >> migration/ram.c | 2 +- >> 1 file changed, 1 insertion(+),

Re: [PATCH 7/7] Python: Drop support for Python 3.6

2023-02-09 Thread Daniel P . Berrangé
On Thu, Feb 09, 2023 at 11:40:57AM -0500, John Snow wrote: > On Thu, Feb 9, 2023 at 11:27 AM Daniel P. Berrangé > wrote: > > > > On Thu, Feb 09, 2023 at 10:40:34AM -0500, John Snow wrote: > > > Python 3.6 was EOL 2021-12-31. Newer versions of upstream libraries have > > > begun dropping support

Re: [PATCH 1/7] python: support pylint 2.16

2023-02-09 Thread Beraldo Leal
On Thu, Feb 09, 2023 at 10:40:28AM -0500, John Snow wrote: > Pylint 2.16 adds a few new checks that cause the optional check-tox CI > job to fail. > > 1. The superfluous-parens check seems to be a bit more aggressive, > 2. broad-exception-raised is new; it discourages "raise Exception". > > Fix

Re: [PATCH 1/3] migration: In case of postcopy, the memory ends in res_postcopy_only

2023-02-09 Thread Vladimir Sementsov-Ogievskiy
On 08.02.23 16:57, Juan Quintela wrote: So remove last assignation of res_compatible. I hoped for some description when asked to split it out :) Signed-off-by: Juan Quintela --- migration/ram.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/migration/ram.c

Re: [PATCH] block/file-posix: don't use functions calling AIO_WAIT_WHILE in worker threads

2023-02-09 Thread Kevin Wolf
Am 09.02.2023 um 16:45 hat Emanuele Giuseppe Esposito geschrieben: > When calling bdrv_getlength() in handle_aiocb_write_zeroes(), the > function creates a new coroutine and then waits that it finishes using > AIO_WAIT_WHILE. > The problem is that this function could also run in a worker thread, >

Re: [PATCH 7/7] Python: Drop support for Python 3.6

2023-02-09 Thread John Snow
On Thu, Feb 9, 2023 at 11:27 AM Daniel P. Berrangé wrote: > > On Thu, Feb 09, 2023 at 10:40:34AM -0500, John Snow wrote: > > Python 3.6 was EOL 2021-12-31. Newer versions of upstream libraries have > > begun dropping support for this version and it is becoming more > > cumbersome to support.

Re: [PATCH 7/7] Python: Drop support for Python 3.6

2023-02-09 Thread Daniel P . Berrangé
On Thu, Feb 09, 2023 at 10:40:34AM -0500, John Snow wrote: > Python 3.6 was EOL 2021-12-31. Newer versions of upstream libraries have > begun dropping support for this version and it is becoming more > cumbersome to support. Avocado-framework and qemu.qmp each have their > own reasons for wanting

Re: [PATCH 5/7] testing: Add Python >= 3.7 to Centos, OpenSuSE

2023-02-09 Thread John Snow
On Thu, Feb 9, 2023 at 10:56 AM Thomas Huth wrote: > > On 09/02/2023 16.40, John Snow wrote: > > This is just a proof-of-concept patch, as these files are lcitool > > generated. The real fix will involve updating the lcitool configuration > > and updating these files that way. > > I think it

Re: [PATCH] block/file-posix: don't use functions calling AIO_WAIT_WHILE in worker threads

2023-02-09 Thread Philippe Mathieu-Daudé
On 9/2/23 16:45, Emanuele Giuseppe Esposito wrote: When calling bdrv_getlength() in handle_aiocb_write_zeroes(), the function creates a new coroutine and then waits that it finishes using AIO_WAIT_WHILE. The problem is that this function could also run in a worker thread, that has a different

Re: [PATCH 6/7] CI: Stop building docs on centos8

2023-02-09 Thread Philippe Mathieu-Daudé
On 9/2/23 16:40, John Snow wrote: CentOS 8 does not ship with a sphinx new enough for our purposes (It necessarily uses Python 3.6), so drop this from this build. We can resume building docs on CentOS 9 if we wish, but we also currently test and build docs on Fedora, Ubuntu, Alpine and Debian.

Re: [PATCH 1/7] python: support pylint 2.16

2023-02-09 Thread Philippe Mathieu-Daudé
On 9/2/23 16:40, John Snow wrote: Pylint 2.16 adds a few new checks that cause the optional check-tox CI job to fail. 1. The superfluous-parens check seems to be a bit more aggressive, 2. broad-exception-raised is new; it discourages "raise Exception". Fix these minor issues and turn the

Re: [PATCH 6/7] CI: Stop building docs on centos8

2023-02-09 Thread Thomas Huth
On 09/02/2023 16.40, John Snow wrote: CentOS 8 does not ship with a sphinx new enough for our purposes (It necessarily uses Python 3.6), so drop this from this build. We can resume building docs on CentOS 9 if we wish, but we also currently test and build docs on Fedora, Ubuntu, Alpine and

Re: [PATCH 5/7] testing: Add Python >= 3.7 to Centos, OpenSuSE

2023-02-09 Thread Thomas Huth
On 09/02/2023 16.40, John Snow wrote: This is just a proof-of-concept patch, as these files are lcitool generated. The real fix will involve updating the lcitool configuration and updating these files that way. I think it would be good to have a RFC or DONOTMERGE in the patch title, so that

[PATCH] block/file-posix: don't use functions calling AIO_WAIT_WHILE in worker threads

2023-02-09 Thread Emanuele Giuseppe Esposito
When calling bdrv_getlength() in handle_aiocb_write_zeroes(), the function creates a new coroutine and then waits that it finishes using AIO_WAIT_WHILE. The problem is that this function could also run in a worker thread, that has a different AioContext from main loop and iothreads, therefore in

[PATCH 5/7] testing: Add Python >= 3.7 to Centos, OpenSuSE

2023-02-09 Thread John Snow
This is just a proof-of-concept patch, as these files are lcitool generated. The real fix will involve updating the lcitool configuration and updating these files that way. Note that this requires OpenSuSE Leap 15.3; 15.2 won't cut it. This is just to prove that bumping our dependency works.

[PATCH 3/7] configure: Look for auxiliary Python installations

2023-02-09 Thread John Snow
At the moment, we look for just "python3" and "python", which is good enough almost all of the time. But ... if you are on a platform that uses an older Python by default and only offers a newer Python as an option, you'll have to specify --python=/usr/bin/foo every time. We can be kind and

[PATCH 4/7] configure: Add nice hint to Python failure message

2023-02-09 Thread John Snow
If we begin requiring Python 3.7+, a few platforms are going to need to install an additional package. This is at least mildly annoying to the user (and I hate negative attention), so solve the user's problem for them before they get a chance to become irritated while searching on Google for how

[PATCH 2/7] Python: drop pipenv

2023-02-09 Thread John Snow
The pipenv tool was nice in theory, but in practice it's just too hard to update selectively, and it makes using it a pain. The qemu.qmp repo dropped pipenv support a while back and it's been functioning just fine, so I'm backporting that change here to qemu.git. Signed-off-by: John Snow ---

[PATCH 6/7] CI: Stop building docs on centos8

2023-02-09 Thread John Snow
CentOS 8 does not ship with a sphinx new enough for our purposes (It necessarily uses Python 3.6), so drop this from this build. We can resume building docs on CentOS 9 if we wish, but we also currently test and build docs on Fedora, Ubuntu, Alpine and Debian. Signed-off-by: John Snow ---

[PATCH 7/7] Python: Drop support for Python 3.6

2023-02-09 Thread John Snow
Python 3.6 was EOL 2021-12-31. Newer versions of upstream libraries have begun dropping support for this version and it is becoming more cumbersome to support. Avocado-framework and qemu.qmp each have their own reasons for wanting to drop Python 3.6. Since it is safe to under our supported

[PATCH 0/7] Python: Drop support for Python 3.6

2023-02-09 Thread John Snow
Howdy, this series increases our minimum python version to 3.7. CI: https://gitlab.com/jsnow/qemu/-/pipelines/771780626 (All green!) GL: https://gitlab.com/jsnow/qemu/-/commits/python-require-37 Patches 1 and 2 are loose pre-requisites; I'd like to merge them into qemu.git within the week

[PATCH 1/7] python: support pylint 2.16

2023-02-09 Thread John Snow
Pylint 2.16 adds a few new checks that cause the optional check-tox CI job to fail. 1. The superfluous-parens check seems to be a bit more aggressive, 2. broad-exception-raised is new; it discourages "raise Exception". Fix these minor issues and turn the lights green. Signed-off-by: John Snow

[PULL 6/6] iotests/detect-zeroes-registered-buf: add new test

2023-02-09 Thread Stefan Hajnoczi
This regression test demonstrates that detect-zeroes works with registered buffers. Bug details: https://gitlab.com/qemu-project/qemu/-/issues/1404 Reviewed-by: Eric Blake Reviewed-by: Hanna Czenczek Signed-off-by: Stefan Hajnoczi Message-Id: <20230207203719.242926-5-stefa...@redhat.com> ---

[PULL 4/6] qemu-io: use BdrvRequestFlags instead of int

2023-02-09 Thread Stefan Hajnoczi
The block layer APIs use BdrvRequestFlags while qemu-io code uses int. Although the code compiles and runs fine, BdrvRequestFlags is clearer because it differentiates between other types of flags like bdrv_open() flags. This is purely refactoring. Reviewed-by: Eric Blake Reviewed-by: Hanna

[PULL 1/6] vhost-user-fs: Back up vqs before cleaning up vhost_dev

2023-02-09 Thread Stefan Hajnoczi
From: Akihiko Odaki vhost_dev_cleanup() clears vhost_dev so back up its vqs member to free the memory pointed by the member. Fixes: 98fc1ada4c ("virtio: add vhost-user-fs base device") Signed-off-by: Akihiko Odaki Signed-off-by: Stefan Hajnoczi Message-Id:

[PULL 5/6] qemu-io: add -r option to register I/O buffer

2023-02-09 Thread Stefan Hajnoczi
The blk_register_buf() API is an optimization hint that allows some block drivers to avoid I/O buffer housekeeping or bounce buffers. Add an -r option to register the I/O buffer so that qemu-io can be used to test the blk_register_buf() API. The next commit will add a test that uses the new

[PULL 3/6] block: fix detect-zeroes= with BDRV_REQ_REGISTERED_BUF

2023-02-09 Thread Stefan Hajnoczi
When a write request is converted into a write zeroes request by the detect-zeroes= feature, it is no longer associated with an I/O buffer. The BDRV_REQ_REGISTERED_BUF flag doesn't make sense without an I/O buffer and must be cleared because bdrv_co_do_pwrite_zeroes() fails with -EINVAL when it's

[PULL 2/6] virtio-blk: add missing AioContext lock

2023-02-09 Thread Stefan Hajnoczi
From: Emanuele Giuseppe Esposito virtio_blk_update_config() calls blk_get_geometry and blk_getlength, and both functions eventually end up calling bdrv_poll_co when not running in a coroutine: - blk_getlength is a co_wrapper_mixed function - blk_get_geometry calls bdrv_get_geometry ->

[PULL 0/6] Block patches

2023-02-09 Thread Stefan Hajnoczi
The following changes since commit 417296c8d8588f782018d01a317f88957e9786d6: tests/qtest/netdev-socket: Raise connection timeout to 60 seconds (2023-02-09 11:23:53 +) are available in the Git repository at: https://gitlab.com/stefanha/qemu.git tags/block-pull-request for you to fetch

Re: [PULL 03/30] migration: Split save_live_pending() into state_pending_*

2023-02-09 Thread Juan Quintela
Avihai Horon wrote: > On 07/02/2023 2:56, Juan Quintela wrote: >> External email: Use caution opening links or attachments >> >> >> We split the function into to: >> >> - state_pending_estimate: We estimate the remaining state size without >>stopping the machine. >> >> - state pending_exact:

Re: [PATCH v4 0/4] block: fix detect-zeroes= with BDRV_REQ_REGISTERED_BUF

2023-02-09 Thread Stefan Hajnoczi
On Tue, Feb 07, 2023 at 03:37:15PM -0500, Stefan Hajnoczi wrote: > v4: > - Add 'r' to read_f() getopt() call [Hanna] > - Fix qemu_io_alloc() and friends buf and len with qemuio_misalign [Hanna] > - Fix qemu_iovec_destroy()/qemu_io_free() ordering in aio_write_done() [Hanna] > - Add mutually

Re: [PATCH] virtio-blk: add missing AioContext lock

2023-02-09 Thread Stefan Hajnoczi
On Wed, Feb 08, 2023 at 06:11:48AM -0500, Emanuele Giuseppe Esposito wrote: > virtio_blk_update_config() calls blk_get_geometry and blk_getlength, > and both functions eventually end up calling bdrv_poll_co when not > running in a coroutine: > - blk_getlength is a co_wrapper_mixed function > -

Re: [PATCH] virtio-blk: add missing AioContext lock

2023-02-09 Thread Stefan Hajnoczi
On Wed, Feb 08, 2023 at 06:11:48AM -0500, Emanuele Giuseppe Esposito wrote: > virtio_blk_update_config() calls blk_get_geometry and blk_getlength, > and both functions eventually end up calling bdrv_poll_co when not > running in a coroutine: > - blk_getlength is a co_wrapper_mixed function > -

Re: rainier-bmc machine dumping core with latest qemu

2023-02-09 Thread Ninad Palsule
Hi Emanuele, On 2/9/23 5:24 AM, Emanuele Giuseppe Esposito wrote: Adding Kevin too. I can't reproduce your issue. I tried the exact steps showed in your mail, both with BLD_VERSION=20230205025034 (not anymore latest compose) and BLD_VERSION=20230209025037 (currently latest), and the VM boots

RE: [PULL 28/30] multifd: Fix flush of zero copy page send request

2023-02-09 Thread Duan, Zhenzhong
Hi Juan, >-Original Message- >From: Juan Quintela >Sent: Tuesday, February 7, 2023 8:57 AM >To: qemu-de...@nongnu.org >Cc: qemu-block@nongnu.org; Stefan Berger ; >Stefan Hajnoczi ; Halil Pasic ; >John Snow ; David Hildenbrand ; >Fam Zheng ; Thomas Huth ; Daniel P. >Berrangé ; Laurent

Re: [PULL 28/30] multifd: Fix flush of zero copy page send request

2023-02-09 Thread Juan Quintela
"Duan, Zhenzhong" wrote: > Hi Juan, > >>-Original Message- >>From: Juan Quintela >>Sent: Tuesday, February 7, 2023 8:57 AM >>To: qemu-de...@nongnu.org >>Cc: qemu-block@nongnu.org; Stefan Berger ; >>Stefan Hajnoczi ; Halil Pasic ; >>John Snow ; David Hildenbrand ; >>Fam Zheng ; Thomas

Re: rainier-bmc machine dumping core with latest qemu

2023-02-09 Thread Emanuele Giuseppe Esposito
Adding Kevin too. I can't reproduce your issue. I tried the exact steps showed in your mail, both with BLD_VERSION=20230205025034 (not anymore latest compose) and BLD_VERSION=20230209025037 (currently latest), and the VM boots till the login screen. Talking with Kevin it looks like

Re: [PATCH 5/7] hw/ide/piix: Use generic ide_init_ioport()

2023-02-09 Thread Bernhard Beschow
On Wed, Feb 8, 2023 at 1:08 AM Philippe Mathieu-Daudé wrote: > TYPE_PIIX3_IDE is a PCI function inheriting from QOM > TYPE_PCI_DEVICE. To be able to call the ISA specific > ide_init_ioport_isa(), we call this function passing > a NULL ISADevice argument. Remove this hack by calling > the