[Qemu-block] [PATCH 2/2] tests/ide-test: test case for crash when processing short PRDs

2018-06-19 Thread Amol Surati
From: John Snow Related Bug: https://bugs.launchpad.net/qemu/+bug/1777315 Signed-off-by: Amol Surati --- tests/ide-test.c | 28 1 file changed, 28 insertions(+) diff --git a/tests/ide-test.c b/tests/ide-test.c index f39431b1a9..382c29a174 100644 ---

[Qemu-block] [PATCH 1/2] ide/hw/core: fix crash on processing a partial-sector-size DMA xfer

2018-06-19 Thread Amol Surati
Fixes: https://bugs.launchpad.net/qemu/+bug/1777315 QEMU's short PRD policy applies to a DMA transfer of size < 512 bytes. But it fails to consider transfers which are >= 512 bytes, but are not a multiple of 512 bytes. Such transfers are not subject to the short PRD policy. They end up violating

Re: [Qemu-block] [RFC 1/1] ide: bug #1777315: io_buffer_size and sg.size can represent partial sector sizes

2018-06-19 Thread Amol Surati
On Wed, Jun 20, 2018 at 06:23:19AM +0530, Amol Surati wrote: > On Tue, Jun 19, 2018 at 05:43:52PM -0400, John Snow wrote: > > > > > > On 06/19/2018 05:26 PM, Amol Surati wrote: > > > On Tue, Jun 19, 2018 at 08:04:03PM +0530, Amol Surati wrote: > > >> On Tue, Jun 19, 2018 at 09:45:15AM -0400,

Re: [Qemu-block] [RFC 1/1] ide: bug #1777315: io_buffer_size and sg.size can represent partial sector sizes

2018-06-19 Thread Amol Surati
On Tue, Jun 19, 2018 at 05:43:52PM -0400, John Snow wrote: > > > On 06/19/2018 05:26 PM, Amol Surati wrote: > > On Tue, Jun 19, 2018 at 08:04:03PM +0530, Amol Surati wrote: > >> On Tue, Jun 19, 2018 at 09:45:15AM -0400, John Snow wrote: > >>> > >>> > >>> On 06/19/2018 04:53 AM, Kevin Wolf wrote:

Re: [Qemu-block] [Qemu-devel] [PATCH] [RFC v2] aio: properly bubble up errors from initialization

2018-06-19 Thread Nishanth Aravamudan
On 19.06.2018 [15:35:57 -0700], Nishanth Aravamudan wrote: > On 19.06.2018 [13:14:51 -0700], Nishanth Aravamudan wrote: > > On 19.06.2018 [14:35:33 -0500], Eric Blake wrote: > > > On 06/15/2018 12:47 PM, Nishanth Aravamudan via Qemu-devel wrote: > > > > > > > } else if

Re: [Qemu-block] [Qemu-devel] [PATCH] [RFC v2] aio: properly bubble up errors from initialization

2018-06-19 Thread Nishanth Aravamudan
On 19.06.2018 [13:14:51 -0700], Nishanth Aravamudan wrote: > On 19.06.2018 [14:35:33 -0500], Eric Blake wrote: > > On 06/15/2018 12:47 PM, Nishanth Aravamudan via Qemu-devel wrote: > > > } else if (s->use_linux_aio) { > > > +int rc; > > > +rc =

Re: [Qemu-block] [PATCH 2/6] nbd: allow authorization with nbd-server-start QMP command

2018-06-19 Thread Daniel P . Berrangé
On Tue, Jun 19, 2018 at 03:10:12PM -0500, Eric Blake wrote: > On 06/15/2018 10:50 AM, Daniel P. Berrangé wrote: > > From: "Daniel P. Berrange" > > > > As with the previous patch to qemu-nbd, the nbd-server-start QMP command > > also needs to be able to specify authorization when enabling TLS

Re: [Qemu-block] [RFC 1/1] ide: bug #1777315: io_buffer_size and sg.size can represent partial sector sizes

2018-06-19 Thread John Snow
On 06/19/2018 05:26 PM, Amol Surati wrote: > On Tue, Jun 19, 2018 at 08:04:03PM +0530, Amol Surati wrote: >> On Tue, Jun 19, 2018 at 09:45:15AM -0400, John Snow wrote: >>> >>> >>> On 06/19/2018 04:53 AM, Kevin Wolf wrote: Am 19.06.2018 um 06:01 hat Amol Surati geschrieben: > On Mon,

Re: [Qemu-block] [RFC 1/1] ide: bug #1777315: io_buffer_size and sg.size can represent partial sector sizes

2018-06-19 Thread Amol Surati
On Tue, Jun 19, 2018 at 08:04:03PM +0530, Amol Surati wrote: > On Tue, Jun 19, 2018 at 09:45:15AM -0400, John Snow wrote: > > > > > > On 06/19/2018 04:53 AM, Kevin Wolf wrote: > > > Am 19.06.2018 um 06:01 hat Amol Surati geschrieben: > > >> On Mon, Jun 18, 2018 at 08:14:10PM -0400, John Snow

Re: [Qemu-block] [PATCH v5 3/6] nbd/server: add nbd_meta_empty_or_pattern helper

2018-06-19 Thread Eric Blake
On 06/09/2018 10:17 AM, Vladimir Sementsov-Ogievskiy wrote: Add nbd_meta_pattern() and nbd_meta_empty_or_pattern() helpers for metadata query parsing. nbd_meta_pattern() will be reused for "qemu" s/for/for the/ namespace in following patches. Signed-off-by: Vladimir Sementsov-Ogievskiy ---

Re: [Qemu-block] [Qemu-devel] [PATCH] [RFC v2] aio: properly bubble up errors from initialization

2018-06-19 Thread Nishanth Aravamudan
On 19.06.2018 [14:35:33 -0500], Eric Blake wrote: > On 06/15/2018 12:47 PM, Nishanth Aravamudan via Qemu-devel wrote: > > laio_init() can fail for a couple of reasons, which will lead to a NULL > > pointer dereference in laio_attach_aio_context(). > > > > To solve this, add a

Re: [Qemu-block] [PATCH 2/6] nbd: allow authorization with nbd-server-start QMP command

2018-06-19 Thread Eric Blake
On 06/15/2018 10:50 AM, Daniel P. Berrangé wrote: From: "Daniel P. Berrange" As with the previous patch to qemu-nbd, the nbd-server-start QMP command also needs to be able to specify authorization when enabling TLS encryption. First the client must create a QAuthZ object instance using the

Re: [Qemu-block] [PATCH 1/6] qemu-nbd: add support for authorization of TLS clients

2018-06-19 Thread Eric Blake
On 06/15/2018 10:50 AM, Daniel P. Berrangé wrote: From: "Daniel P. Berrange" Currently any client which can complete the TLS handshake is able to use the NBD server. The server admin can turn on the 'verify-peer' option for the x509 creds to require the client to provide a x509 certificate.

Re: [Qemu-block] [Qemu-devel] [PATCH] [RFC v2] aio: properly bubble up errors from initialization

2018-06-19 Thread Eric Blake
On 06/15/2018 12:47 PM, Nishanth Aravamudan via Qemu-devel wrote: laio_init() can fail for a couple of reasons, which will lead to a NULL pointer dereference in laio_attach_aio_context(). To solve this, add a aio_setup_linux_aio() function which is called before aio_get_linux_aio() where it is

Re: [Qemu-block] [Qemu-devel] [PULL 08/26] qobject: Move block-specific qdict code to block-qdict.c

2018-06-19 Thread Eric Blake
On 06/15/2018 09:20 AM, Kevin Wolf wrote: From: Markus Armbruster Pure code motion, except for two brace placements and a comment tweaked to appease checkpatch. Signed-off-by: Markus Armbruster Reviewed-by: Kevin Wolf Signed-off-by: Kevin Wolf --- qobject/block-qdict.c | 640

Re: [Qemu-block] [PATCH v5 2/6] nbd/server: refactor NBDExportMetaContexts

2018-06-19 Thread Eric Blake
On 06/09/2018 10:17 AM, Vladimir Sementsov-Ogievskiy wrote: Use NBDExport pointer instead of just export name: there no needs to s/no needs/is no need/ store duplicated name in the struct, moreover, NBDExport will be used further. Signed-off-by: Vladimir Sementsov-Ogievskiy ---

Re: [Qemu-block] [Qemu-devel] [PATCH 6/7] block/qcow2-refcount: fix out-of-file L1 entries to be zero

2018-06-19 Thread Eric Blake
On 06/19/2018 01:34 PM, Vladimir Sementsov-Ogievskiy wrote: Zero out corrupted L1 table entry, which reference L2 table out of underlying file. Zero L1 table entry means that "the L2 table and all clusters described by this L2 table are unallocated." Signed-off-by: Vladimir Sementsov-Ogievskiy

Re: [Qemu-block] [PATCH v6 5/6] iotests: Add new test 214 for max compressed cluster offset

2018-06-19 Thread Eric Blake
On 04/26/2018 07:10 AM, Alberto Garcia wrote: On Thu 26 Apr 2018 04:51:28 AM CEST, Eric Blake wrote: If you have a capable file system (tmpfs is good, ext4 not so much; run ./check with TEST_DIR pointing to a good location so as not to skip the test), it's actually possible to create a qcow2

Re: [Qemu-block] [Qemu-devel] [PATCH v6 0/6] minor qcow2 compression improvements

2018-06-19 Thread Eric Blake
ping On 04/25/2018 09:51 PM, Eric Blake wrote: Even though v5 was posted earlier today, it was worth a respin: - 2/6: add R-b [Berto] - 4/6, 6/6: improve commit messages [Max] - 5/6: new patch, with an iotests proving that 4/6 is a bug fix [Max] The new test is rather slow (nearly 90 seconds

Re: [Qemu-block] [Qemu-devel] [PATCH 3/7] block/qcow2-refcount: check_refcounts_l2: refactor compressed case

2018-06-19 Thread Eric Blake
On 06/19/2018 01:34 PM, Vladimir Sementsov-Ogievskiy wrote: Separate offset and size of compressed cluster. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2-refcount.c | 15 ++- 1 file changed, 10 insertions(+), 5 deletions(-) Hmm, I wonder if this duplicates my

Re: [Qemu-block] [Qemu-devel] [PATCH 2/7] block/qcow2-refcount: avoid eating RAM

2018-06-19 Thread Eric Blake
On 06/19/2018 01:34 PM, Vladimir Sementsov-Ogievskiy wrote: qcow2_inc_refcounts_imrt() (through realloc_refcount_array()) can eat unpredicted amount of memory on corrupted table entries, which are s/unpredicted/an unpredictable/ referencing regions far beyond the end of file. Prevent this,

Re: [Qemu-block] [PATCH v5 1/6] nbd/server: fix trace

2018-06-19 Thread Eric Blake
On 06/09/2018 10:17 AM, Vladimir Sementsov-Ogievskiy wrote: Return code = 1 doesn't mean that we parsed base:allocation. Use correct traces in both -parsed and -skipped cases. Signed-off-by: Vladimir Sementsov-Ogievskiy --- nbd/server.c | 9 +++-- 1 file changed, 7 insertions(+), 2

[Qemu-block] [PATCH 6/7] block/qcow2-refcount: fix out-of-file L1 entries to be zero

2018-06-19 Thread Vladimir Sementsov-Ogievskiy
Zero out corrupted L1 table entry, which reference L2 table out of underlying file. Zero L1 table entry means that "the L2 table and all clusters described by this L2 table are unallocated." Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2-refcount.c | 37

[Qemu-block] [PATCH 0/7] qcow2 check improvements

2018-06-19 Thread Vladimir Sementsov-Ogievskiy
Hi all! We've faced the following problem: after host fs corruption, vm images becomes invalid. And which is interesting, starting qemu-img check on them led to allocating of the whole RAM and then killing qemu-img by OOM Killer. This was due to corrupted l2 entries, which referenced clusters

[Qemu-block] [PATCH 3/7] block/qcow2-refcount: check_refcounts_l2: refactor compressed case

2018-06-19 Thread Vladimir Sementsov-Ogievskiy
Separate offset and size of compressed cluster. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2-refcount.c | 15 ++- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c index 28d21bedc3..42167b7040 100644 ---

[Qemu-block] [PATCH 5/7] block/qcow2-refcount: check_refcounts_l2: split fix_l2_entry_to_zero

2018-06-19 Thread Vladimir Sementsov-Ogievskiy
Split entry repairing to separate function, to be reused later. Note: entry in in-memory l2 table (local variable in check_refcounts_l2) is not updated after this patch. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2-refcount.c | 147

[Qemu-block] [PATCH 7/7] block/qcow2-refcount: fix out-of-file L2 entries to be read-as-zero

2018-06-19 Thread Vladimir Sementsov-Ogievskiy
Rewrite corrupted L2 table entry, which reference space out of underlying file. Make this L2 table entry read-as-all-zeros without any allocation. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2-refcount.c | 32 1 file changed, 32 insertions(+)

[Qemu-block] [PATCH 2/7] block/qcow2-refcount: avoid eating RAM

2018-06-19 Thread Vladimir Sementsov-Ogievskiy
qcow2_inc_refcounts_imrt() (through realloc_refcount_array()) can eat unpredicted amount of memory on corrupted table entries, which are referencing regions far beyond the end of file. Prevent this, by skipping such regions from further processing. Signed-off-by: Vladimir Sementsov-Ogievskiy

[Qemu-block] [PATCH 1/7] block/qcow2-refcount: fix check_oflag_copied

2018-06-19 Thread Vladimir Sementsov-Ogievskiy
Increase corruptions_fixed only after successful fix. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2-refcount.c | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c index 18c729aa27..f9d095aa2d 100644 ---

[Qemu-block] [PATCH 4/7] block/qcow2-refcount: check_refcounts_l2: reduce ignored overlaps

2018-06-19 Thread Vladimir Sementsov-Ogievskiy
Reduce number of structures ignored in overlap check: when checking active table ignore active tables, when checking inactive table ignore inactive ones. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2-refcount.c | 16 +--- 1 file changed, 9 insertions(+), 7 deletions(-)

Re: [Qemu-block] [Qemu-devel] [PULL 00/35] Block layer patches

2018-06-19 Thread Peter Maydell
On 18 June 2018 at 17:44, Kevin Wolf wrote: > The following changes since commit 2ef2f16781af9dee6ba6517755e9073ba5799fa2: > > Merge remote-tracking branch > 'remotes/dgilbert/tags/pull-migration-20180615a' into staging (2018-06-15 > 18:13:35 +0100) > > are available in the git repository at:

Re: [Qemu-block] [RFC 1/1] ide: bug #1777315: io_buffer_size and sg.size can represent partial sector sizes

2018-06-19 Thread Amol Surati
On Tue, Jun 19, 2018 at 09:45:15AM -0400, John Snow wrote: > > > On 06/19/2018 04:53 AM, Kevin Wolf wrote: > > Am 19.06.2018 um 06:01 hat Amol Surati geschrieben: > >> On Mon, Jun 18, 2018 at 08:14:10PM -0400, John Snow wrote: > >>> > >>> > >>> On 06/18/2018 02:02 PM, Amol Surati wrote: >

Re: [Qemu-block] [RFC PATCH 06/10] block: Allow changing the backing file on reopen

2018-06-19 Thread Alberto Garcia
>> Wait, I think the description I gave is inaccurate: >> >> commit_complete() calls bdrv_drop_intermediate(), and that updates the >> backing image name (c->role->update_filename()). If we're doing this in >> an intermediate node then it needs to be reopened in read-write mode, >> while keeping

Re: [Qemu-block] [RFC 1/1] ide: bug #1777315: io_buffer_size and sg.size can represent partial sector sizes

2018-06-19 Thread John Snow
On 06/19/2018 04:53 AM, Kevin Wolf wrote: > Am 19.06.2018 um 06:01 hat Amol Surati geschrieben: >> On Mon, Jun 18, 2018 at 08:14:10PM -0400, John Snow wrote: >>> >>> >>> On 06/18/2018 02:02 PM, Amol Surati wrote: On Mon, Jun 18, 2018 at 12:05:15AM +0530, Amol Surati wrote: > This patch

[Qemu-block] [PATCH v5 7/8] hw/block/fdc: Always apply block configuration to block driver

2018-06-19 Thread Ari Sundholm
This allows the block driver to use the block configuration of the new floppy device. One use for this information is to set request limits using this information. Signed-off-by: Ari Sundholm --- hw/block/fdc.c | 4 1 file changed, 4 insertions(+) diff --git a/hw/block/fdc.c

[Qemu-block] [PATCH v5 8/8] block: Add blklogwrites

2018-06-19 Thread Ari Sundholm
From: Aapo Vienamo Implements a block device write logging system, similar to Linux kernel device mapper dm-log-writes. The write operations that are performed on a block device are logged to a file or another block device. The write log format is identical to the dm-log-writes format.

[Qemu-block] [PATCH v5 1/8] block: Move two block permission constants to the relevant enum

2018-06-19 Thread Ari Sundholm
This allows using the two constants outside of block.c, which will happen in a subsequent patch. Signed-off-by: Ari Sundholm --- block.c | 6 -- include/block/block.h | 7 +++ 2 files changed, 7 insertions(+), 6 deletions(-) diff --git a/block.c b/block.c index

[Qemu-block] [PATCH v5 5/8] hw/block/virtio-blk: Always apply block configuration to block driver

2018-06-19 Thread Ari Sundholm
This allows the block driver to use the block configuration of the new VirtIO block device. One use for this information is to set request limits using this information. Signed-off-by: Ari Sundholm --- hw/block/virtio-blk.c | 1 + 1 file changed, 1 insertion(+) diff --git

[Qemu-block] [PATCH v5 2/8] block: Add a mechanism for passing a block driver a block configuration

2018-06-19 Thread Ari Sundholm
A block driver may need to know about the block configuration, most critically the sector sizes, of a block backend for alignment purposes or for some other reason. It doesn't seem like qemu has an existing mechanism for a block backend to convey the required information to the relevant block

[Qemu-block] [PATCH v5 0/8] New block driver: blklogwrites

2018-06-19 Thread Ari Sundholm
This patch series adds a new block driver, blklogwrites, to QEMU. The driver is given two block devices: a raw device backed by an image or a host block device, and a log device, typically backed by a file, on which writes to the raw device are logged. The logging format used is the same as in

[Qemu-block] [PATCH v5 4/8] hw/ide/qdev: Always apply block configuration to block driver

2018-06-19 Thread Ari Sundholm
This allows the block driver to use the block configuration of the new IDE device. One use for this information is to set request limits using this information. Signed-off-by: Ari Sundholm --- hw/ide/qdev.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/hw/ide/qdev.c b/hw/ide/qdev.c

[Qemu-block] [PATCH v5 6/8] hw/block/nvme: Always apply block configuration to block driver

2018-06-19 Thread Ari Sundholm
This allows the block driver to use the block configuration of the new NVMe device. One use for this information is to set request limits using this information. Signed-off-by: Ari Sundholm --- hw/block/nvme.c | 1 + 1 file changed, 1 insertion(+) diff --git a/hw/block/nvme.c b/hw/block/nvme.c

Re: [Qemu-block] [RFC PATCH 06/10] block: Allow changing the backing file on reopen

2018-06-19 Thread Kevin Wolf
Am 19.06.2018 um 14:27 hat Alberto Garcia geschrieben: > On Mon 18 Jun 2018 06:12:29 PM CEST, Kevin Wolf wrote: > >> >> This patch allows the user to change the backing file of an image > >> >> that is being reopened. Here's what it does: > >> >> > >> >> - In bdrv_reopen_queue_child(): if the

Re: [Qemu-block] [PATCH 5/6] vnc: allow specifying a custom authorization object name

2018-06-19 Thread Daniel P . Berrangé
On Fri, Jun 15, 2018 at 04:51:02PM +0100, Daniel P. Berrangé wrote: > From: "Daniel P. Berrange" > > The VNC server has historically had support for ACLs to check both the > SASL username and the TLS x509 distinguished name. The VNC server was > responsible for creating the initial ACL, and the

Re: [Qemu-block] [PATCH 6/6] monitor: deprecate acl_show, acl_reset, acl_policy, acl_add, acl_remove

2018-06-19 Thread Daniel P . Berrangé
On Tue, Jun 19, 2018 at 01:31:40PM +0100, Dr. David Alan Gilbert wrote: > * Daniel P. Berrangé (berra...@redhat.com) wrote: > > The various ACL related commands are obsolete now that the QAuthZ > > framework for authorization is fully integrated throughout QEMU network > > services. Mark it as

Re: [Qemu-block] [PATCH 6/6] monitor: deprecate acl_show, acl_reset, acl_policy, acl_add, acl_remove

2018-06-19 Thread Dr. David Alan Gilbert
* Daniel P. Berrangé (berra...@redhat.com) wrote: > The various ACL related commands are obsolete now that the QAuthZ > framework for authorization is fully integrated throughout QEMU network > services. Mark it as deprecated with no replacement to be provided. > > Signed-off-by: Daniel P.

Re: [Qemu-block] [RFC PATCH 06/10] block: Allow changing the backing file on reopen

2018-06-19 Thread Alberto Garcia
On Mon 18 Jun 2018 06:12:29 PM CEST, Kevin Wolf wrote: >> >> This patch allows the user to change the backing file of an image >> >> that is being reopened. Here's what it does: >> >> >> >> - In bdrv_reopen_queue_child(): if the 'backing' option points to an >> >>image different from the

Re: [Qemu-block] [PATCH v4 10/10] block/blklogwrites: Use the block device logical sector size when logging writes

2018-06-19 Thread Alberto Garcia
On Mon 18 Jun 2018 05:53:50 PM CEST, Ari Sundholm wrote: > On 06/18/2018 06:36 PM, Alberto Garcia wrote: >> On Fri 08 Jun 2018 02:32:28 PM CEST, Ari Sundholm wrote: >>> The guest OS may perform writes which are aligned to the logical >>> sector size instead of the physical one, so logging at this

Re: [Qemu-block] [RFC 1/1] ide: bug #1777315: io_buffer_size and sg.size can represent partial sector sizes

2018-06-19 Thread Kevin Wolf
Am 19.06.2018 um 06:01 hat Amol Surati geschrieben: > On Mon, Jun 18, 2018 at 08:14:10PM -0400, John Snow wrote: > > > > > > On 06/18/2018 02:02 PM, Amol Surati wrote: > > > On Mon, Jun 18, 2018 at 12:05:15AM +0530, Amol Surati wrote: > > >> This patch fixes the assumption that io_buffer_size is

Re: [Qemu-block] [Qemu-devel] [PATCH v2 00/10] block: Try to create well typed json:{} filenames

2018-06-19 Thread Markus Armbruster
Max Reitz writes: > See cover letter of v1 here: > http://lists.nongnu.org/archive/html/qemu-block/2018-05/msg00290.html > > > This series depends on Markus's series "block: Configuration fixes and > rbd authentication": > > Based-on: <20180607062559.16127-1-arm...@redhat.com> PATCH 06+07