Re: [PATCH v10 2/3] block/nbd: nbd reconnect

2019-10-16 Thread Eric Blake
On 10/9/19 3:41 AM, Vladimir Sementsov-Ogievskiy wrote: Implement reconnect. To achieve this: 1. add new modes: connecting-wait: means, that reconnecting is in progress, and there were small number of reconnect attempts, so all requests are waiting for the connection.

Re: [PATCH v8 0/8] Add Qemu to SeaBIOS LCHS interface

2019-10-16 Thread no-reply
Patchew URL: https://patchew.org/QEMU/20191016164145.115898-1-sam...@google.com/ Hi, This series seems to have some coding style problems. See output below for more information: Subject: [PATCH v8 0/8] Add Qemu to SeaBIOS LCHS interface Type: series Message-id:

[PATCH 09/10] image-fuzzer: Run using python3

2019-10-16 Thread Eduardo Habkost
image-fuzzer is now supposed to be ready to run using Python 3. Remove the __future__ imports and change the interpreter line to "#!/usr/bin/env python3". Signed-off-by: Eduardo Habkost --- tests/image-fuzzer/qcow2/__init__.py | 1 - tests/image-fuzzer/qcow2/layout.py | 1 -

[PATCH 05/10] image-fuzzer: Use %r for all fiels at Field.__repr__()

2019-10-16 Thread Eduardo Habkost
This makes the formatting code simpler, and safer if we change the type of self.value from str to bytes. Signed-off-by: Eduardo Habkost --- tests/image-fuzzer/qcow2/layout.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/image-fuzzer/qcow2/layout.py

[PATCH 04/10] image-fuzzer: Use io.StringIO

2019-10-16 Thread Eduardo Habkost
StringIO.StringIO is not available on Python 3, but io.StringIO is available on both Python 2 and 3. io.StringIO is slightly different from the Python 2 StringIO module, though, so we need bytes coming from subprocess.Popen() to be explicitly decoded. Signed-off-by: Eduardo Habkost ---

[PATCH 03/10] image-fuzzer: Explicitly use integer division operator

2019-10-16 Thread Eduardo Habkost
Most of the division expressions in image-fuzzer assume integer division. Use the // operator to keep the same behavior when we move to Python 3. Signed-off-by: Eduardo Habkost --- tests/image-fuzzer/qcow2/fuzz.py | 12 - tests/image-fuzzer/qcow2/layout.py | 40

[PATCH 10/10] image-fuzzer: Use errors parameter of subprocess.Popen()

2019-10-16 Thread Eduardo Habkost
Instead of manually encoding stderr and stdout output, use `errors` parameter of subprocess.Popen(). This will make process.communicate() return unicode strings instead of bytes objects. Signed-off-by: Eduardo Habkost --- tests/image-fuzzer/runner.py | 11 --- 1 file changed, 4

[PATCH 08/10] image-fuzzer: Encode file name and file format to bytes

2019-10-16 Thread Eduardo Habkost
Callers of create_image() will pass strings as arguments, but the Image class will expect bytes objects to be provided. Encode them inside create_image(). Signed-off-by: Eduardo Habkost --- tests/image-fuzzer/qcow2/layout.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git

[PATCH 07/10] image-fuzzer: Use bytes constant for field values

2019-10-16 Thread Eduardo Habkost
Field values are supposed to be bytes objects, not unicode strings. Change two constants that were declared as strings. Signed-off-by: Eduardo Habkost --- tests/image-fuzzer/qcow2/layout.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git

[PATCH 02/10] image-fuzzer: Write bytes instead of string to image file

2019-10-16 Thread Eduardo Habkost
This is necessary for Python 3 compatibility. Signed-off-by: Eduardo Habkost --- tests/image-fuzzer/qcow2/layout.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/image-fuzzer/qcow2/layout.py b/tests/image-fuzzer/qcow2/layout.py index c57418fa15..fe273d4143 100644

[PATCH 06/10] image-fuzzer: Return bytes objects on string fuzzing functions

2019-10-16 Thread Eduardo Habkost
No caller of fuzzer functions is interested in unicode string values, so replace them with bytes sequences. Signed-off-by: Eduardo Habkost --- tests/image-fuzzer/qcow2/fuzz.py | 42 1 file changed, 21 insertions(+), 21 deletions(-) diff --git

[PATCH 01/10] image-fuzzer: Open image files in binary mode

2019-10-16 Thread Eduardo Habkost
This probably never caused problems because on Linux there's no actual newline conversion happening, but on Python 3 the binary/text distinction is stronger and we must explicitly open the image file in binary mode. Signed-off-by: Eduardo Habkost --- tests/image-fuzzer/qcow2/layout.py | 2 +- 1

[PATCH 00/10] image-fuzzer: Port to Python 3

2019-10-16 Thread Eduardo Habkost
This series ports image-fuzzer to Python 3. Eduardo Habkost (10): image-fuzzer: Open image files in binary mode image-fuzzer: Write bytes instead of string to image file image-fuzzer: Explicitly use integer division operator image-fuzzer: Use io.StringIO image-fuzzer: Use %r for all

Re: [PATCH v4 0/4] qcow2: advanced compression options

2019-10-16 Thread no-reply
Patchew URL: https://patchew.org/QEMU/157124-882302-1-git-send-email-andrey.shinkev...@virtuozzo.com/ Hi, This series failed the docker-mingw@fedora build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally.

Re: [PATCH v4 0/4] qcow2: advanced compression options

2019-10-16 Thread no-reply
Patchew URL: https://patchew.org/QEMU/157124-882302-1-git-send-email-andrey.shinkev...@virtuozzo.com/ Hi, This series failed the docker-quick@centos7 build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally.

[PATCH v2 1/6] block/block-copy: allocate buffer in block_copy_with_bounce_buffer

2019-10-16 Thread Vladimir Sementsov-Ogievskiy
Move bounce_buffer allocation block_copy_with_bounce_buffer. This commit simplifies further work on implementing copying by larger chunks (of different size) and further asynchronous handling of block_copy iterations (with help of block/aio_task API). Allocation works fast, a lot faster than disk

[PATCH v2 2/6] block/block-copy: limit copy_range_size to 16 MiB

2019-10-16 Thread Vladimir Sementsov-Ogievskiy
Large copy range may imply memory allocation and large io effort, so using 2G copy range request may be bad idea. Let's limit it to 16 MiB. It also helps the following patch to refactor copy-with-offload fallback to copy-with-bounce-buffer. Note, that total memory usage of backup is still not

[PATCH v2 5/6] block/block-copy: add memory limit

2019-10-16 Thread Vladimir Sementsov-Ogievskiy
Currently total allocation for parallel requests to block-copy instance is unlimited. Let's limit it to 128 MiB. For now block-copy is used only in backup, so actually we limit total allocation for backup job. Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/block/block-copy.h | 3 +++

[PATCH v2 0/6] block-copy: memory limit

2019-10-16 Thread Vladimir Sementsov-Ogievskiy
I'm going to bring block-status driven, async copying process to block-copy, to make it fast. The first step is to limit memory usage of backup, here is it. v2: [mostly by Max's comments] Now based on master (Thank you Max!) 01: add Max's r-b 02: add Max's r-b 03: - refactor block_copy_do_copy

[PATCH v2 3/6] block/block-copy: refactor copying

2019-10-16 Thread Vladimir Sementsov-Ogievskiy
Merge copying code into one function block_copy_do_copy, which only calls bdrv_ io functions and don't do any synchronization (like dirty bitmap set/reset). Refactor block_copy() function so that it takes full decision about size of chunk to be copied and does all the synchronization (checking

[PATCH v2 6/6] block/block-copy: increase buffered copy request

2019-10-16 Thread Vladimir Sementsov-Ogievskiy
No reason to limit buffered copy to one cluster. Let's allow up to 1 MiB. Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/block/block-copy.h | 2 +- block/block-copy.c | 48 +- 2 files changed, 33 insertions(+), 17 deletions(-) diff --git

[PATCH v2 4/6] util: introduce SharedResource

2019-10-16 Thread Vladimir Sementsov-Ogievskiy
Introduce an API for some shared splittable resource, like memory. It's going to be used by backup. Backup uses both read/write io and copy_range. copy_range may consume memory implictly, so the new API is abstract: it doesn't allocate any real memory by but only hands out tickets. The idea is

[PATCH v8 8/8] hd-geo-test: Add tests for lchs override

2019-10-16 Thread Sam Eiderman
From: Sam Eiderman Add QTest tests to check the logical geometry override option. The tests in hd-geo-test are out of date - they only test IDE and do not test interesting MBRs. Creating qcow2 disks with specific size and MBR layout is currently unused - we only use a default empty MBR.

[PATCH v8 3/8] bootdevice: Add interface to gather LCHS

2019-10-16 Thread Sam Eiderman
From: Sam Eiderman Add an interface to provide direct logical CHS values for boot devices. We will use this interface in the next commits. Reviewed-by: Karl Heubaum Reviewed-by: Arbel Moshe Signed-off-by: Sam Eiderman Signed-off-by: Sam Eiderman --- bootdevice.c| 55

[PATCH v8 2/8] block: Support providing LCHS from user

2019-10-16 Thread Sam Eiderman
From: Sam Eiderman Add logical geometry variables to BlockConf. A user can now supply "lcyls", "lheads" & "lsecs" for any HD device that supports CHS ("cyls", "heads", "secs"). These devices include: * ide-hd * scsi-hd * virtio-blk-pci In future commits we will use the provided

[PATCH v8 5/8] bootdevice: Gather LCHS from all relevant devices

2019-10-16 Thread Sam Eiderman
From: Sam Eiderman Relevant devices are: * ide-hd (and ide-cd, ide-drive) * scsi-hd (and scsi-cd, scsi-disk, scsi-block) * virtio-blk-pci We do not call del_boot_device_lchs() for ide-* since we don't need to - IDE block devices do not support unplugging. Reviewed-by: Karl Heubaum

[PATCH v8 4/8] scsi: Propagate unrealize() callback to scsi-hd

2019-10-16 Thread Sam Eiderman
From: Sam Eiderman We will need to add LCHS removal logic to scsi-hd's unrealize() in the next commit. Reviewed-by: Karl Heubaum Reviewed-by: Arbel Moshe Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Sam Eiderman Signed-off-by: Sam Eiderman --- hw/scsi/scsi-bus.c | 16

[PATCH v8 6/8] bootdevice: Refactor get_boot_devices_list

2019-10-16 Thread Sam Eiderman
From: Sam Eiderman Move device name construction to a separate function. We will reuse this function in the following commit to pass logical CHS parameters through fw_cfg much like we currently pass bootindex. Reviewed-by: Karl Heubaum Reviewed-by: Arbel Moshe Reviewed-by: Philippe

[PATCH v8 7/8] bootdevice: FW_CFG interface for LCHS values

2019-10-16 Thread Sam Eiderman
From: Sam Eiderman Using fw_cfg, supply logical CHS values directly from QEMU to the BIOS. Non-standard logical geometries break under QEMU. A virtual disk which contains an operating system which depends on logical geometries (consistent values being reported from BIOS INT13 AH=08) will most

[PATCH v8 1/1] hd-geo-test: Add tests for lchs override

2019-10-16 Thread Sam Eiderman
From: Sam Eiderman Add QTest tests to check the logical geometry override option. The tests in hd-geo-test are out of date - they only test IDE and do not test interesting MBRs. Creating qcow2 disks with specific size and MBR layout is currently unused - we only use a default empty MBR.

Re: [SeaBIOS] Re: [PATCH v7 7/8] bootdevice: FW_CFG interface for LCHS values

2019-10-16 Thread Philippe Mathieu-Daudé
On 10/16/19 5:19 PM, Sam Eiderman wrote: Sure! Philippe withdrew his R-b on 7/8, as I explained 7/8 is fine (only need to remove a bad comment) the problem was in the tests 8/8 - should I include the original R/b? I withdrew it because John was preparing his pull request, and I needed more

[PATCH v8 1/8] block: Refactor macros - fix tabbing

2019-10-16 Thread Sam Eiderman
From: Sam Eiderman Fixing tabbing in block related macros. Reviewed-by: Karl Heubaum Reviewed-by: Arbel Moshe Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Sam Eiderman Signed-off-by: Sam Eiderman --- hw/ide/qdev.c| 2 +- include/hw/block/block.h | 16 2

[PATCH v8 0/8] Add Qemu to SeaBIOS LCHS interface

2019-10-16 Thread Sam Eiderman
v1: Non-standard logical geometries break under QEMU. A virtual disk which contains an operating system which depends on logical geometries (consistent values being reported from BIOS INT13 AH=08) will most likely break under QEMU/SeaBIOS if it has non-standard logical geometries - for example

[PATCH v4 0/4] qcow2: advanced compression options

2019-10-16 Thread Andrey Shinkevich
New enhancements for writing compressed data to QCOW2 image. v4: The 'compression' support at the block generic layer has been accumulated in the separate patch 1/4. A little code refactoring was made. v3: Instead of introducing multiple key options for many drivers, the

[PATCH v4 3/4] tests/qemu-iotests: add case to write compressed data of multiple clusters

2019-10-16 Thread Andrey Shinkevich
Add the test case to the iotest #214 that checks possibility of writing compressed data of more than one cluster size. Signed-off-by: Andrey Shinkevich --- tests/qemu-iotests/214 | 35 +++ tests/qemu-iotests/214.out | 15 +++ 2 files changed, 50

[PATCH v4 2/4] qcow2: Allow writing compressed data of multiple clusters

2019-10-16 Thread Andrey Shinkevich
QEMU currently supports writing compressed data of the size equal to one cluster. This patch allows writing QCOW2 compressed data that exceed one cluster. Now, we split buffered data into separate clusters and write them compressed using the existing functionality. Suggested-by: Pavel Butsykin

[PATCH v4 1/4] block: support compressed write at generic layer

2019-10-16 Thread Andrey Shinkevich
To inform the block layer about writing all the data compressed, we introduce the 'compress' command line option. Based on that option, the written data will be aligned by the cluster size at the generic layer. Suggested-by: Roman Kagan Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by:

[PATCH v4 4/4] tests/qemu-iotests: add case for block-stream compress

2019-10-16 Thread Andrey Shinkevich
Add a case to the iotest #030 that tests the 'compress' option for a block-stream job. Signed-off-by: Andrey Shinkevich --- tests/qemu-iotests/030 | 51 +- tests/qemu-iotests/030.out | 4 ++-- 2 files changed, 52 insertions(+), 3 deletions(-)

Re: [SeaBIOS] Re: [PATCH v7 7/8] bootdevice: FW_CFG interface for LCHS values

2019-10-16 Thread Sam Eiderman
Sure! Philippe withdrew his R-b on 7/8, as I explained 7/8 is fine (only need to remove a bad comment) the problem was in the tests 8/8 - should I include the original R/b? I guess all other 1-6 are fine to add R/b... On Wed, Oct 16, 2019 at 6:07 PM John Snow wrote: > > > > On 10/16/19 10:55

Re: [SeaBIOS] Re: [PATCH v7 7/8] bootdevice: FW_CFG interface for LCHS values

2019-10-16 Thread John Snow
On 10/16/19 10:55 AM, Sam Eiderman wrote: > Thanks for the detailed comment Laszlo, > > Indeed my e-mail has changed and I only received replies to the > commits where I added this new mail in the S-o-b section, should of > added in all of them. > > So as you said it, the problem was actually

Re: [SeaBIOS] Re: [PATCH v7 7/8] bootdevice: FW_CFG interface for LCHS values

2019-10-16 Thread Sam Eiderman
Thanks for the detailed comment Laszlo, Indeed my e-mail has changed and I only received replies to the commits where I added this new mail in the S-o-b section, should of added in all of them. So as you said it, the problem was actually in using qfw_cfg_get_u32 which assumes the value is

Re: [SeaBIOS] Re: [PATCH v7 7/8] bootdevice: FW_CFG interface for LCHS values

2019-10-16 Thread Philippe Mathieu-Daudé
On 10/16/19 2:14 PM, Laszlo Ersek wrote: Hi Sam, On 10/16/19 13:02, Sam Eiderman wrote: Gentle Ping, Philippe, John? Just wondering if the series is okay, as Gerd pointed out this series is a blocker for the corresponding changes in SeaBIOS for v 1.13 The QEMU series is still not merged,

Re: [SeaBIOS] Re: [PATCH v7 7/8] bootdevice: FW_CFG interface for LCHS values

2019-10-16 Thread Laszlo Ersek
Hi Sam, On 10/16/19 13:02, Sam Eiderman wrote: > Gentle Ping, > > Philippe, John? > > Just wondering if the series is okay, as Gerd pointed out this series > is a blocker for the corresponding changes in SeaBIOS for v 1.13 The QEMU series is still not merged, due to a bug in the last patch

Re: [PATCH v3 2/4] blkdebug: Allow taking/unsharing permissions

2019-10-16 Thread Vladimir Sementsov-Ogievskiy
14.10.2019 18:39, Max Reitz wrote: > Sometimes it is useful to be able to add a node to the block graph that > takes or unshare a certain set of permissions for debugging purposes. > This patch adds this capability to blkdebug. > > (Note that you cannot make blkdebug release or share permissions

Re: [PATCH v3 0/5] qcow2: advanced compression options

2019-10-16 Thread no-reply
Patchew URL: https://patchew.org/QEMU/1571163625-642312-1-git-send-email-andrey.shinkev...@virtuozzo.com/ Hi, This series failed the docker-mingw@fedora build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally.

Re: [SeaBIOS] [PATCH v7 7/8] bootdevice: FW_CFG interface for LCHS values

2019-10-16 Thread Sam Eiderman
Gentle Ping, Philippe, John? Just wondering if the series is okay, as Gerd pointed out this series is a blocker for the corresponding changes in SeaBIOS for v 1.13 Sam On Tue, Oct 8, 2019 at 2:51 PM Sam Eiderman wrote: > > > > On Tue, Oct 8, 2019, 13:34 Philippe Mathieu-Daudé wrote: >> >>

Re: [PATCH v3 0/5] qcow2: advanced compression options

2019-10-16 Thread no-reply
Patchew URL: https://patchew.org/QEMU/1571163625-642312-1-git-send-email-andrey.shinkev...@virtuozzo.com/ Hi, This series failed the docker-quick@centos7 build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally.

Re: [PATCH v2 00/21] iotests: Allow ./check -o data_file

2019-10-16 Thread Max Reitz
On 16.10.19 02:19, no-re...@patchew.org wrote: > Patchew URL: > https://patchew.org/QEMU/20191015142729.18123-1-mre...@redhat.com/ > > > > Hi, > > This series seems to have some coding style problems. See output below for > more information: > > Subject: [PATCH v2 00/21] iotests: Allow

Re: [RFC PATCH 00/23] Add subcluster allocation to qcow2

2019-10-16 Thread Alberto Garcia
On Tue 15 Oct 2019 06:05:23 PM CEST, Eric Blake wrote: >> 6356 5548 4740 3932 3124 2316 15 8 7 0 >> >> <-> <-> >>

Re: [PATCH v2 00/20] nvme: support NVMe v1.3d, SGLs and multiple namespaces

2019-10-16 Thread Fam Zheng
On Tue, 10/15 12:38, Klaus Jensen wrote: > Hi, > > (Quick note to Fam): most of this series is irrelevant to you as the > maintainer of the nvme block driver, but patch "nvme: add support for > scatter gather lists" touches block/nvme.c due to changes in the shared > NvmeCmd struct. Yeah, that

Re: [PATCH v3 0/5] qcow2: advanced compression options

2019-10-16 Thread no-reply
Patchew URL: https://patchew.org/QEMU/1571163625-642312-1-git-send-email-andrey.shinkev...@virtuozzo.com/ Hi, This series failed the docker-quick@centos7 build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally.