Re: [Qemu-block] [Qemu-devel] [PATCH 06/13] qcrypto-luks: implement more rigorous header checking

2019-08-26 Thread Maxim Levitsky
On Mon, 2019-08-26 at 08:31 -0500, Eric Blake wrote: > On 8/25/19 11:08 AM, Maxim Levitsky wrote: > > > > > I'd do a separate check for stripes and active fields, and then give a > > > > specific error message for each. That way if this does ever trigger > &

Re: [Qemu-block] [PATCH] block: posix: Always allocate the first block

2019-08-25 Thread Maxim Levitsky
On Sun, 2019-08-25 at 22:51 +0300, Nir Soffer wrote: > On Sun, Aug 25, 2019 at 10:44 AM Maxim Levitsky wrote: > > On Sat, 2019-08-17 at 00:21 +0300, Nir Soffer wrote: > > > When creating an image with preallocation "off" or "falloc", the first > > >

Re: [Qemu-block] [Qemu-devel] [PATCH 05/13] qcrypto-luks: clear the masterkey and password before freeing them always

2019-08-25 Thread Maxim Levitsky
On Sun, 2019-08-25 at 18:31 +0300, Maxim Levitsky wrote: > On Thu, 2019-08-22 at 13:56 +0300, Maxim Levitsky wrote: > > On Thu, 2019-08-22 at 11:49 +0100, Daniel P. Berrangé wrote: > > > On Tue, Aug 20, 2019 at 08:12:51PM +0200, Max Reitz wrote: > > > > On 14.08.

Re: [Qemu-block] [Qemu-devel] [PATCH 00/13] RFC: luks/encrypted qcow2 key management

2019-08-25 Thread Maxim Levitsky
On Thu, 2019-08-22 at 12:35 +0100, Daniel P. Berrangé wrote: > On Wed, Aug 14, 2019 at 11:22:06PM +0300, Maxim Levitsky wrote: > > Hi! > > > > This patch series implements key management for luks based encryption > > It supports both raw luks images and qcow2 encrypte

Re: [Qemu-block] [Qemu-devel] [PATCH 12/13] qemu-img: implement key management

2019-08-25 Thread Maxim Levitsky
On Thu, 2019-08-22 at 16:42 +0200, Max Reitz wrote: > On 22.08.19 13:32, Daniel P. Berrangé wrote: > > On Tue, Aug 20, 2019 at 08:29:55PM +0200, Max Reitz wrote: > > > On 14.08.19 22:22, Maxim Levitsky wrote: > > > > Signed-off-by: Maxim Levitsky > > >

Re: [Qemu-block] [Qemu-devel] [PATCH 09/13] qcrypto-luks: implement the encryption key management

2019-08-25 Thread Maxim Levitsky
On Thu, 2019-08-22 at 12:27 +0100, Daniel P. Berrangé wrote: > On Wed, Aug 14, 2019 at 11:22:15PM +0300, Maxim Levitsky wrote: > > Signed-off-by: Maxim Levitsky > > --- > > crypto/block-luks.c | 374 +++- > > 1 file changed, 37

Re: [Qemu-block] [Qemu-devel] [PATCH 07/13] block: add manage-encryption command (qmp and blockdev)

2019-08-25 Thread Maxim Levitsky
On Thu, 2019-08-22 at 16:07 +0200, Markus Armbruster wrote: > Maxim Levitsky writes: > > > On Wed, 2019-08-21 at 13:47 +0200, Markus Armbruster wrote: > > > Maxim Levitsky writes: > > > > > > > This adds: > > > > > > > >

Re: [Qemu-block] [Qemu-devel] [PATCH 06/13] qcrypto-luks: implement more rigorous header checking

2019-08-25 Thread Maxim Levitsky
On Sun, 2019-08-25 at 18:40 +0300, Maxim Levitsky wrote: > On Thu, 2019-08-22 at 12:04 +0100, Daniel P. Berrangé wrote: > > On Wed, Aug 14, 2019 at 11:22:12PM +0300, Maxim Levitsky wrote: > > > Check that keyslots don't overlap with the data, > > > and check that keysl

Re: [Qemu-block] [Qemu-devel] [PATCH 06/13] qcrypto-luks: implement more rigorous header checking

2019-08-25 Thread Maxim Levitsky
On Thu, 2019-08-22 at 12:04 +0100, Daniel P. Berrangé wrote: > On Wed, Aug 14, 2019 at 11:22:12PM +0300, Maxim Levitsky wrote: > > Check that keyslots don't overlap with the data, > > and check that keyslots don't overlap with each other. > > (this is done using naiv

Re: [Qemu-block] [Qemu-devel] [PATCH 05/13] qcrypto-luks: clear the masterkey and password before freeing them always

2019-08-25 Thread Maxim Levitsky
On Thu, 2019-08-22 at 13:56 +0300, Maxim Levitsky wrote: > On Thu, 2019-08-22 at 11:49 +0100, Daniel P. Berrangé wrote: > > On Tue, Aug 20, 2019 at 08:12:51PM +0200, Max Reitz wrote: > > > On 14.08.19 22:22, Maxim Levitsky wrote: > > > > While there are other place

Re: [Qemu-block] [PATCH 04/13] qcrypto-luks: refactoring: simplify the math used for keyslot locations

2019-08-25 Thread Maxim Levitsky
On Thu, 2019-08-22 at 11:47 +0100, Daniel P. Berrangé wrote: > On Wed, Aug 14, 2019 at 11:22:10PM +0300, Maxim Levitsky wrote: > > Signed-off-by: Maxim Levitsky > > --- > > crypto/block-luks.c | 64 +++-- > > 1 file changed, 38

Re: [Qemu-block] [Qemu-devel] [PATCH 03/13] qcrypto-luks: refactoring: extract load/store/check/parse header functions

2019-08-25 Thread Maxim Levitsky
On Thu, 2019-08-22 at 11:34 +0100, Daniel P. Berrangé wrote: > On Thu, Aug 22, 2019 at 01:43:05AM +0300, Maxim Levitsky wrote: > > On Tue, 2019-08-20 at 20:01 +0200, Max Reitz wrote: > > > On 14.08.19 22:22, Maxim Levitsky wrote: > > > > With upcoming key management

Re: [Qemu-block] [Qemu-devel] [PATCH 03/13] qcrypto-luks: refactoring: extract load/store/check/parse header functions

2019-08-25 Thread Maxim Levitsky
On Thu, 2019-08-22 at 11:38 +0100, Daniel P. Berrangé wrote: > On Wed, Aug 14, 2019 at 11:22:09PM +0300, Maxim Levitsky wrote: > > With upcoming key management, the header will > > need to be stored after the image is created. > > > > Extracting load header isn't

Re: [Qemu-block] [Qemu-devel] [PATCH 02/13] qcrypto-luks: misc refactoring

2019-08-25 Thread Maxim Levitsky
On Thu, 2019-08-22 at 16:32 +0200, Max Reitz wrote: > On 22.08.19 01:59, Maxim Levitsky wrote: > > On Tue, 2019-08-20 at 19:36 +0200, Max Reitz wrote: > > > On 14.08.19 22:22, Maxim Levitsky wrote: > > > > This is also a preparation for key read/write/erase functions

Re: [Qemu-block] [PATCH] block: posix: Always allocate the first block

2019-08-25 Thread Maxim Levitsky
o kind of a backward compat hack which might be one day removed. [...] Best regards, Maxim Levitsky

[Qemu-block] [PATCH 1/2] block/nvme: add support for write zeros

2019-08-25 Thread Maxim Levitsky
Signed-off-by: Maxim Levitsky --- block/nvme.c | 72 +++- block/trace-events | 1 + include/block/nvme.h | 19 +++- 3 files changed, 90 insertions(+), 2 deletions(-) diff --git a/block/nvme.c b/block/nvme.c index 5be3a39b63..f8bd11e19a

[Qemu-block] [PATCH 2/2] block/nvme: add support for discard

2019-08-25 Thread Maxim Levitsky
Signed-off-by: Maxim Levitsky --- block/nvme.c | 83 ++ block/trace-events | 2 ++ 2 files changed, 85 insertions(+) diff --git a/block/nvme.c b/block/nvme.c index f8bd11e19a..dd041f39c9 100644 --- a/block/nvme.c +++ b/block/nvme.c @@ -112,6

[Qemu-block] [PATCH 0/2] block/nvme: add support for write zeros and discard

2019-08-25 Thread Maxim Levitsky
This is the second part of the patches I prepared for this driver back when I worked on mdev-nvme. Best regards, Maxim Levitsky Maxim Levitsky (2): block/nvme: add support for write zeros block/nvme: add support for discard block/nvme.c | 155

Re: [Qemu-block] [Qemu-devel] [PATCH 01/13] block-crypto: misc refactoring

2019-08-22 Thread Maxim Levitsky
On Thu, 2019-08-22 at 16:34 +0200, Max Reitz wrote: > On 22.08.19 02:05, Maxim Levitsky wrote: > > On Tue, 2019-08-20 at 18:38 +0200, Max Reitz wrote: > > > On 14.08.19 22:22, Maxim Levitsky wrote: > > > > * rename the write_func to create_write_func, > > >

Re: [Qemu-block] [PATCH 08/13] qcrypto: add the plumbing for encryption management

2019-08-22 Thread Maxim Levitsky
On Thu, 2019-08-22 at 12:16 +0100, Daniel P. Berrangé wrote: > On Wed, Aug 14, 2019 at 11:22:14PM +0300, Maxim Levitsky wrote: > > This adds qcrypto_block_manage_encryption, which > > is thin wrapper around manage_encryption of the crypto driver > > which is also adde

Re: [Qemu-block] [PATCH 10/13] block/crypto: implement the encryption key management

2019-08-22 Thread Maxim Levitsky
On Thu, 2019-08-22 at 12:29 +0100, Daniel P. Berrangé wrote: > On Wed, Aug 14, 2019 at 11:22:16PM +0300, Maxim Levitsky wrote: > > This implements the encryption key management > > using the generic code in qcrypto layer > > > > This code adds another 'write_func'

Re: [Qemu-block] [Qemu-devel] [PATCH 02/13] qcrypto-luks: misc refactoring

2019-08-22 Thread Maxim Levitsky
On Thu, 2019-08-22 at 12:10 +0100, Daniel P. Berrangé wrote: > On Thu, Aug 22, 2019 at 02:04:28PM +0300, Maxim Levitsky wrote: > > On Thu, 2019-08-22 at 11:29 +0100, Daniel P. Berrangé wrote: > > > On Thu, Aug 15, 2019 at 05:40:11PM -0400, John Snow wrote: > > > > &g

Re: [Qemu-block] [PATCH 02/13] qcrypto-luks: misc refactoring

2019-08-22 Thread Maxim Levitsky
On Thu, 2019-08-22 at 11:29 +0100, Daniel P. Berrangé wrote: > On Thu, Aug 15, 2019 at 05:40:11PM -0400, John Snow wrote: > > > > > > On 8/14/19 4:22 PM, Maxim Levitsky wrote: > > > This is also a preparation for key read/write/erase functions > > > >

Re: [Qemu-block] [Qemu-devel] [PATCH 03/13] qcrypto-luks: refactoring: extract load/store/check/parse header functions

2019-08-22 Thread Maxim Levitsky
On Thu, 2019-08-22 at 11:32 +0100, Daniel P. Berrangé wrote: > On Thu, Aug 22, 2019 at 01:43:05AM +0300, Maxim Levitsky wrote: > > On Tue, 2019-08-20 at 20:01 +0200, Max Reitz wrote: > > > On 14.08.19 22:22, Maxim Levitsky wrote: > > > > With upcoming key management

Re: [Qemu-block] [Qemu-devel] [PATCH 05/13] qcrypto-luks: clear the masterkey and password before freeing them always

2019-08-22 Thread Maxim Levitsky
On Thu, 2019-08-22 at 11:49 +0100, Daniel P. Berrangé wrote: > On Tue, Aug 20, 2019 at 08:12:51PM +0200, Max Reitz wrote: > > On 14.08.19 22:22, Maxim Levitsky wrote: > > > While there are other places where these are still stored in memory, > > > this is still one less

Re: [Qemu-block] [Qemu-devel] [PATCH 01/13] block-crypto: misc refactoring

2019-08-21 Thread Maxim Levitsky
On Wed, 2019-08-21 at 16:39 +0100, Daniel P. Berrangé wrote: > On Wed, Aug 14, 2019 at 11:22:07PM +0300, Maxim Levitsky wrote: > > * rename the write_func to create_write_func, > > and init_func to create_init_func > > this is preparation for other write_func that will &g

Re: [Qemu-block] [Qemu-devel] [PATCH 01/13] block-crypto: misc refactoring

2019-08-21 Thread Maxim Levitsky
On Tue, 2019-08-20 at 18:38 +0200, Max Reitz wrote: > On 14.08.19 22:22, Maxim Levitsky wrote: > > * rename the write_func to create_write_func, > > and init_func to create_init_func > > this is preparation for other write_func that will > > be used to

Re: [Qemu-block] [Qemu-devel] [PATCH 02/13] qcrypto-luks: misc refactoring

2019-08-21 Thread Maxim Levitsky
On Tue, 2019-08-20 at 19:36 +0200, Max Reitz wrote: > On 14.08.19 22:22, Maxim Levitsky wrote: > > This is also a preparation for key read/write/erase functions > > > > * use master key len from the header > > * prefer to use crypto params in the QCryptoBlockL

Re: [Qemu-block] [Qemu-devel] [PATCH 05/13] qcrypto-luks: clear the masterkey and password before freeing them always

2019-08-21 Thread Maxim Levitsky
On Thu, 2019-08-22 at 02:01 +0300, Nir Soffer wrote: > On Wed, Aug 14, 2019, 23:23 Maxim Levitsky wrote: > > > While there are other places where these are still stored in memory, > > this is still one less key material area that can be sniffed with > > vari

Re: [Qemu-block] [Qemu-devel] [PATCH 03/13] qcrypto-luks: refactoring: extract load/store/check/parse header functions

2019-08-21 Thread Maxim Levitsky
On Tue, 2019-08-20 at 20:01 +0200, Max Reitz wrote: > On 14.08.19 22:22, Maxim Levitsky wrote: > > With upcoming key management, the header will > > need to be stored after the image is created. > > > > Extracting load header isn't strictly needed, but > >

Re: [Qemu-block] [Qemu-devel] [PATCH 05/13] qcrypto-luks: clear the masterkey and password before freeing them always

2019-08-21 Thread Maxim Levitsky
On Tue, 2019-08-20 at 20:12 +0200, Max Reitz wrote: > On 14.08.19 22:22, Maxim Levitsky wrote: > > While there are other places where these are still stored in memory, > > this is still one less key material area that can be sniffed with > > various side channel attacks >

Re: [Qemu-block] [Qemu-devel] [PATCH 12/13] qemu-img: implement key management

2019-08-21 Thread Maxim Levitsky
On Tue, 2019-08-20 at 20:29 +0200, Max Reitz wrote: > On 14.08.19 22:22, Maxim Levitsky wrote: > > Signed-off-by: Maxim Levitsky > > --- > > block/crypto.c | 16 ++ > > block/crypto.h | 3 + > > qemu-img-cmds.hx | 13

Re: [Qemu-block] [Qemu-devel] [PATCH 07/13] block: add manage-encryption command (qmp and blockdev)

2019-08-21 Thread Maxim Levitsky
On Tue, 2019-08-20 at 20:27 +0200, Max Reitz wrote: > On 14.08.19 22:22, Maxim Levitsky wrote: > > This adds: > > > > * x-blockdev-update-encryption and x-blockdev-erase-encryption qmp commands > > Both commands take the QCryptoKeyManageOptions > > the x-bl

Re: [Qemu-block] [Qemu-devel] [PATCH 07/13] block: add manage-encryption command (qmp and blockdev)

2019-08-21 Thread Maxim Levitsky
On Wed, 2019-08-21 at 13:47 +0200, Markus Armbruster wrote: > Maxim Levitsky writes: > > > This adds: > > > > * x-blockdev-update-encryption and x-blockdev-erase-encryption qmp commands > > Both commands take the QCryptoKeyManageOptions > > the x-blockde

Re: [Qemu-block] [Qemu-devel] [PATCH 00/13] RFC: luks/encrypted qcow2 key management

2019-08-21 Thread Maxim Levitsky
On Tue, 2019-08-20 at 19:59 +0200, Max Reitz wrote: > On 14.08.19 22:22, Maxim Levitsky wrote: > > [...] > > > Testing. This was lightly tested with manual testing and with few iotests > > that I prepared. > > I haven't yet tested fully the write sharing behavior

Re: [Qemu-block] [Qemu-devel] [PATCH 00/13] RFC: luks/encrypted qcow2 key management

2019-08-21 Thread Maxim Levitsky
On Wed, 2019-08-21 at 13:31 +0200, Markus Armbruster wrote: > Maxim Levitsky writes: > > > On Thu, 2019-08-15 at 10:00 -0500, Eric Blake wrote: > > > On 8/15/19 9:44 AM, Maxim Levitsky wrote: > > > > > > > > > >

Re: [Qemu-block] [PATCH 02/13] qcrypto-luks: misc refactoring

2019-08-19 Thread Maxim Levitsky
On Thu, 2019-08-15 at 17:40 -0400, John Snow wrote: > > On 8/14/19 4:22 PM, Maxim Levitsky wrote: > > This is also a preparation for key read/write/erase functions > > > > This is a matter of taste and I am not usually reviewing LUKS patches > (So don't take me

Re: [Qemu-block] [Qemu-devel] [PATCH 00/13] RFC: luks/encrypted qcow2 key management

2019-08-19 Thread Maxim Levitsky
On Thu, 2019-08-15 at 10:00 -0500, Eric Blake wrote: > On 8/15/19 9:44 AM, Maxim Levitsky wrote: > > > > > > Does the idea of a union type with a default value for the > > > > > discriminator > > > > > help? Maybe we have a discrim

Re: [Qemu-block] [Qemu-devel] [PATCH 00/13] RFC: luks/encrypted qcow2 key management

2019-08-15 Thread Maxim Levitsky
On Thu, 2019-08-15 at 16:18 +0200, Markus Armbruster wrote: > Kevin Wolf writes: > > > Am 14.08.2019 um 23:08 hat Eric Blake geschrieben: > > > On 8/14/19 3:22 PM, Maxim Levitsky wrote: > > > > > > > This is an issue that was raised today on I

Re: [Qemu-block] [PATCH 00/13] RFC: luks/encrypted qcow2 key management

2019-08-15 Thread Maxim Levitsky
On Wed, 2019-08-14 at 16:08 -0500, Eric Blake wrote: > On 8/14/19 3:22 PM, Maxim Levitsky wrote: > > > This is an issue that was raised today on IRC with Kevin Wolf. Really thanks > > for the idea! > > > > We agreed that this new qmp interface should take the same

Re: [Qemu-block] [Qemu-devel] [PATCH 2/2] qapi: deprecate implicit filters

2019-08-14 Thread Maxim Levitsky
ago; do we want to amend the QAPI > schema specification to allow commands to return with "Warning" strings, > or "Deprecated" stings to allow in-band deprecation notices for cases > like these? > > example: > > { "return": {}, > "deprecated": True, > "warning": "Omitting filter-node-name parameter is deprecated, it will > be required in the future" > } > > There's no "error" key, so this should be recognized as success by > compatible clients, but they'll definitely see the extra information. > > Part of my motivation is to facilitate a more aggressive deprecation of > legacy features by ensuring that we are able to rigorously notify users > through any means that they need to adjust their scripts. > > --js > This is a very good idea IMHO. Best regards, Maxim Levitsky

[Qemu-block] [PATCH 12/13] qemu-img: implement key management

2019-08-14 Thread Maxim Levitsky
Signed-off-by: Maxim Levitsky --- block/crypto.c | 16 ++ block/crypto.h | 3 + qemu-img-cmds.hx | 13 + qemu-img.c | 140 +++ 4 files changed, 172 insertions(+) diff --git a/block/crypto.c b/block/crypto.c index 415b6db041

[Qemu-block] [PATCH 13/13] iotests : add tests for encryption key management

2019-08-14 Thread Maxim Levitsky
Signed-off-by: Maxim Levitsky --- tests/qemu-iotests/257 | 197 ++ tests/qemu-iotests/257.out | 96 +++ tests/qemu-iotests/258 | 95 +++ tests/qemu-iotests/258.out | 30 + tests/qemu-iotests/259

[Qemu-block] [PATCH 10/13] block/crypto: implement the encryption key management

2019-08-14 Thread Maxim Levitsky
, we have it, and should use it instead. Signed-off-by: Maxim Levitsky --- block/crypto.c | 96 -- 1 file changed, 93 insertions(+), 3 deletions(-) diff --git a/block/crypto.c b/block/crypto.c index 42a3f0898b..415b6db041 100644 --- a/block/crypto.c

[Qemu-block] [PATCH 08/13] qcrypto: add the plumbing for encryption management

2019-08-14 Thread Maxim Levitsky
This adds qcrypto_block_manage_encryption, which is thin wrapper around manage_encryption of the crypto driver which is also added Signed-off-by: Maxim Levitsky --- crypto/block.c | 29 + crypto/blockpriv.h | 9 + include/crypto/block.h | 27

[Qemu-block] [PATCH 07/13] block: add manage-encryption command (qmp and blockdev)

2019-08-14 Thread Maxim Levitsky
BlockDriverState and not BdrvChild, for the ease of use from the qmp code. It is not expected that this function will be used by anything but qmp and qemu-img code Signed-off-by: Maxim Levitsky --- block/block-backend.c | 9 block/io.c | 24

[Qemu-block] [PATCH 11/13] block/qcow2: implement the encryption key managment

2019-08-14 Thread Maxim Levitsky
This is the main purpose of the patchset, to enaable us to manage luks like header, embedded in the qcow2 image, which standard cryptosetup tools don't support. Signed-off-by: Maxim Levitsky --- block/qcow2.c | 27 +++ 1 file changed, 27 insertions(+) diff --git a/block

[Qemu-block] [PATCH 09/13] qcrypto-luks: implement the encryption key management

2019-08-14 Thread Maxim Levitsky
Signed-off-by: Maxim Levitsky --- crypto/block-luks.c | 374 +++- 1 file changed, 373 insertions(+), 1 deletion(-) diff --git a/crypto/block-luks.c b/crypto/block-luks.c index 1997e92fe1..2c33643b52 100644 --- a/crypto/block-luks.c +++ b/crypto/block

[Qemu-block] [PATCH 03/13] qcrypto-luks: refactoring: extract load/store/check/parse header functions

2019-08-14 Thread Maxim Levitsky
is no longer endian swapped in place, to prevent (mostly theoretical races I think) races where someone could see the header in the process of beeing byteswapped. Signed-off-by: Maxim Levitsky --- crypto/block-luks.c | 756 ++-- 1 file changed, 440

[Qemu-block] [PATCH 02/13] qcrypto-luks: misc refactoring

2019-08-14 Thread Maxim Levitsky
in the QCryptoBlockLUKS Signed-off-by: Maxim Levitsky --- crypto/block-luks.c | 213 ++-- 1 file changed, 105 insertions(+), 108 deletions(-) diff --git a/crypto/block-luks.c b/crypto/block-luks.c index 409ab50f20..48213abde7 100644 --- a/crypto/block-luks.c +++ b

[Qemu-block] [PATCH 06/13] qcrypto-luks: implement more rigorous header checking

2019-08-14 Thread Maxim Levitsky
Check that keyslots don't overlap with the data, and check that keyslots don't overlap with each other. (this is done using naive O(n^2) nested loops, but since there are just 8 keyslots, this doens't really matter. Signed-off-by: Maxim Levitsky --- crypto/block-luks.c | 42

[Qemu-block] [PATCH 00/13] RFC: luks/encrypted qcow2 key management

2019-08-14 Thread Maxim Levitsky
to add. Best regards, Maxim Levitsky Maxim Levitsky (13): block-crypto: misc refactoring qcrypto-luks: misc refactoring qcrypto-luks: refactoring: extract load/store/check/parse header functions qcrypto-luks: refactoring: simplify the math used for keyslot locations qcrypto

[Qemu-block] [PATCH 05/13] qcrypto-luks: clear the masterkey and password before freeing them always

2019-08-14 Thread Maxim Levitsky
While there are other places where these are still stored in memory, this is still one less key material area that can be sniffed with various side channel attacks Signed-off-by: Maxim Levitsky --- crypto/block-luks.c | 52 ++--- 1 file changed, 44

[Qemu-block] [PATCH 04/13] qcrypto-luks: refactoring: simplify the math used for keyslot locations

2019-08-14 Thread Maxim Levitsky
Signed-off-by: Maxim Levitsky --- crypto/block-luks.c | 64 +++-- 1 file changed, 38 insertions(+), 26 deletions(-) diff --git a/crypto/block-luks.c b/crypto/block-luks.c index 6bb369f3b4..e1a4df94b7 100644 --- a/crypto/block-luks.c +++ b/crypto/block

[Qemu-block] [PATCH 01/13] block-crypto: misc refactoring

2019-08-14 Thread Maxim Levitsky
* rename the write_func to create_write_func, and init_func to create_init_func this is preparation for other write_func that will be used to update the encryption keys. No functional changes Signed-off-by: Maxim Levitsky --- block/crypto.c | 15 --- 1 file changed, 8

Re: [Qemu-block] [PATCH for-4.1?] nvme: Limit blkshift to 12 (for 4 kB blocks)

2019-07-30 Thread Maxim Levitsky
K page size that might work, but again, I don't think any hardware vendor dared yet to sell devices with sector size > 4K. Reviewed-by: Maxim Levitsky Best regards, Maxim Levitsky > > Reported-by: Peter Maydell > Suggested-by: Maxim Levitsky > Signed-off-by: Max Reitz >

Re: [Qemu-block] [PULL 2/5] block/nvme: support larger that 512 bytes sector devices

2019-07-29 Thread Maxim Levitsky
On Mon, 2019-07-29 at 15:25 +0200, Max Reitz wrote: > On 29.07.19 15:16, Peter Maydell wrote: > > On Mon, 22 Jul 2019 at 18:26, Max Reitz wrote: > > > > > > From: Maxim Levitsky > > > > > > Currently the driver hardcodes the sector size to 512,

Re: [Qemu-block] [PULL 2/5] block/nvme: support larger that 512 bytes sector devices

2019-07-29 Thread Maxim Levitsky
On Mon, 2019-07-29 at 14:16 +0100, Peter Maydell wrote: > On Mon, 22 Jul 2019 at 18:26, Max Reitz wrote: > > > > From: Maxim Levitsky > > > > Currently the driver hardcodes the sector size to 512, > > and doesn't check the underlying device. Fix that. >

Re: [Qemu-block] [Qemu-devel] [PATCH v2 10/11] iotests: Test convert -n to pre-filled image

2019-07-25 Thread Maxim Levitsky
": 67090432, "depth": 0, "zero": true, "data": > false}] > + > +=== -n to a non-zero image === > + > +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 > +Formatting 'TEST_DIR/t.IMGFMT.orig', fmt=IMGFMT size=67108864 > +wrote 65536/65536 bytes at offset 0 > +64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > +Images are identical. > *** done Reviewed-by: Maxim Levitsky Best regards, Maxim Levitsky

Re: [Qemu-block] [Qemu-devel] [PATCH v2 09/11] iotests: Convert to preallocated encrypted qcow2

2019-07-25 Thread Maxim Levitsky
sts/qemu-iotests/188.out > @@ -15,4 +15,8 @@ read 16777216/16777216 bytes at offset 0 > > == verify open failure with wrong password == > qemu-io: can't open: Invalid password, cannot unlock any keyslot > + > +== verify that has_zero_init returns false when preallocating == > +Formatting 'TEST_DIR/t.IMGFMT.orig', fmt=IMGFMT size=16777216 > +Images are identical. > *** done Reviewed-by: Maxim Levitsky Best regards, Maxim Levitsky

Re: [Qemu-block] [Qemu-devel] [PATCH v2 08/11] vhdx: Fix .bdrv_has_zero_init()

2019-07-25 Thread Maxim Levitsky
rowing and static are preallocated this makes sense. Its a bit amusing and not surprising that the the spec for this format is in .docx. I took a quick look to get a rough impression of the file format. Reviewed-by: Maxim Levitsky Best regards, Maxim Levitsky

Re: [Qemu-block] [Qemu-devel] [PATCH v2 07/11] vdi: Fix .bdrv_has_zero_init()

2019-07-25 Thread Maxim Levitsky
eallocated this makes sense. I see that the code when it allocates a new block at the end of the file, actually zeroes it out, so most likely this is right. Reviewed-by: Maxim Levitsky Best regards, Maxim Levitsky

Re: [Qemu-block] [Qemu-devel] [PATCH v2 06/11] qcow2: Fix .bdrv_has_zero_init()

2019-07-25 Thread Maxim Levitsky
int64_t pos) > { > @@ -5186,7 +5213,7 @@ BlockDriver bdrv_qcow2 = { > .bdrv_child_perm = bdrv_format_default_perms, > .bdrv_co_create_opts = qcow2_co_create_opts, > .bdrv_co_create = qcow2_co_create, > -.bdrv_has_zero_init = bdrv_has_zero_

Re: [Qemu-block] [Qemu-devel] [PATCH v2 04/11] block: Implement .bdrv_has_zero_init_truncate()

2019-07-25 Thread Maxim Levitsky
cate = sd_co_truncate, > diff --git a/block/ssh.c b/block/ssh.c > index 501933b855..84d01e892b 100644 > --- a/block/ssh.c > +++ b/block/ssh.c > @@ -1390,6 +1390,7 @@ static BlockDriver bdrv_ssh = { > .bdrv_co_create_opts = ssh_co_create_opts, > .bdrv_close = ssh_close, > .bdrv_has_zero_init = ssh_has_zero_init, > +.bdrv_has_zero_init_truncate = ssh_has_zero_init, > .bdrv_co_readv= ssh_co_readv, > .bdrv_co_writev = ssh_co_writev, > .bdrv_getlength = ssh_getlength, Reviewed-by: Maxim Levitsky Best regards, Maxim Levitsky

Re: [Qemu-block] [Qemu-devel] [PATCH v2 05/11] block: Use bdrv_has_zero_init_truncate()

2019-07-25 Thread Maxim Levitsky
has_zero_init(bs->file->bs) == 0) { > +if (bdrv_has_zero_init_truncate(bs->file->bs) == 0) { > use_zero_buffers = true; > > /* zero fill the front, if any */ Reviewed-by: Maxim Levitsky Best regards, Maxim Levitsky

Re: [Qemu-block] [Qemu-devel] [PATCH v2 03/11] block: Add bdrv_has_zero_init_truncate()

2019-07-25 Thread Maxim Levitsky
{ > +return bs->drv->bdrv_has_zero_init_truncate(bs); > +} > +if (bs->file && bs->drv->is_filter) { > +return bdrv_has_zero_init_truncate(bs->file->bs); > +} > + > + /* safe default */ > +return 0; > +} > + > bool bdrv_unallocated_blocks_are_zero(BlockDriverState *bs) > { > BlockDriverInfo bdi; This looks like a very correct change, even for the sake of clarifying the scope of bdrv_has_zero_init Reviewed-by: Maxim Levitsky Best regards, Maxim Levitsky

Re: [Qemu-block] [Qemu-devel] [PATCH v2 02/11] mirror: Fix bdrv_has_zero_init() use

2019-07-25 Thread Maxim Levitsky
YNC_MODE_NONE, MIRROR_OPEN_BACKING_CHAIN, > + MIRROR_SYNC_MODE_NONE, MIRROR_OPEN_BACKING_CHAIN, false, > BLOCKDEV_ON_ERROR_REPORT, BLOCKDEV_ON_ERROR_REPORT, > false, "filter_node", MIRROR_COPY_MODE_BACKGROUND, > _abort); >From my limited understanding of this code, it looks ok to me. Still to be very sure, I sort of suggest still to check that nobody relies on target zeroing when non in full sync mode, to avoid breaking the users For example, QMP reference states that MIRROR_SYNC_MODE_TOP copies data in the topmost image to the destination. If there is only the topmost image, I could image the caller assume that target is identical to the source. Reviewed-by: Maxim Levitsky Best regards, Maxim Levitsky

Re: [Qemu-block] [Qemu-devel] [PATCH v2 01/11] qemu-img: Fix bdrv_has_zero_init() use in convert

2019-07-25 Thread Maxim Levitsky
+} > > if (!s->has_zero_init && !s->target_has_backing && > bdrv_can_write_zeroes_with_unmap(blk_bs(s->target))) > @@ -2423,6 +2426,8 @@ static int img_convert(int argc, char **argv) > } > } > > + s.target_is_

Re: [Qemu-block] [PATCH v3] block/rbd: add preallocation support

2019-07-23 Thread Maxim Levitsky
ap_create(BlockDriverState *bs, > @@ -1256,6 +1425,11 @@ static QemuOptsList qemu_rbd_create_opts = { > .type = QEMU_OPT_SIZE, > .help = "RBD object size" > }, > + { > +.name = BLOCK_OPT_PREALLOC, > +.type = QEMU_OPT_STRING, > +.help = "Preallocation mode (allowed values: off, full)" > +}, > { > .name = "password-secret", > .type = QEMU_OPT_STRING, > diff --git a/qapi/block-core.json b/qapi/block-core.json > index 0d43d4f37c..ff55171f8d 100644 > --- a/qapi/block-core.json > +++ b/qapi/block-core.json > @@ -4346,13 +4346,16 @@ > # point to a snapshot. > # @size Size of the virtual disk in bytes > # @cluster-size RBD object size > +# @preallocationPreallocation mode for the new image (since: 4.2) > +# (default: off; allowed values: off, full) > # > # Since: 2.12 > ## > { 'struct': 'BlockdevCreateOptionsRbd', >'data': { 'location': 'BlockdevOptionsRbd', > 'size': 'size', > -'*cluster-size' : 'size' } } > +'*cluster-size' : 'size', > +'*preallocation': 'PreallocMode' } } > > ## > # @BlockdevVmdkSubformat: I think I don't see anything obviously wrong, but note that I don't know ceph yet, thus I might have missed something. So: Reviewed-by: Maxim Levitsky Best regards, Maxim Levitsky

Re: [Qemu-block] [Qemu-trivial] [Qemu-devel] [PATCH 2/2] qemu-img: better error message when opening a backing file fails

2019-07-22 Thread Maxim Levitsky
On Mon, 2019-07-22 at 10:15 +0100, Daniel P. Berrangé wrote: > On Sun, Jul 21, 2019 at 09:15:08PM +0300, Maxim Levitsky wrote: > > Currently we print message like that: > > > > " > > new_file.qcow2 : error message > > " > > > > However the e

Re: [Qemu-block] [PATCH 2/2] qemu-img: better error message when opening a backing file fails

2019-07-22 Thread Maxim Levitsky
On Mon, 2019-07-22 at 11:41 +0200, Kevin Wolf wrote: > Am 21.07.2019 um 20:15 hat Maxim Levitsky geschrieben: > > Currently we print message like that: > > > > " > > new_file.qcow2 : error message > > " > > > > However the error

[Qemu-block] [PATCH 0/2] RFC: Trivial error message fixes for luks format

2019-07-21 Thread Maxim Levitsky
These are attempts to improve a bit error message based on bunch of luks related bugzillas assigned to me. Feel free to reject these if you think that it doesn't make the messages better. Best regards, Maxim Levitsky Maxim Levitsky (2): LUKS: better error message when creating too

[Qemu-block] [PATCH 1/2] LUKS: better error message when creating too large files

2019-07-21 Thread Maxim Levitsky
, detect -EFBIG and in this case present a better error message, which is what this patch does The new error message is: qemu-img: error creating test.luks: The requested file size is too large Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1534898 Signed-off-by: Maxim Levitsky --- block

[Qemu-block] [PATCH 2/2] qemu-img: better error message when opening a backing file fails

2019-07-21 Thread Maxim Levitsky
slot Could not open backing image to determine size. Error message after: qemu-img: error creating sn.qcow2: \ json:{ "encrypt.key-secret": "sec1", "driver": "qcow2", "file": { "driver": "file", "filename"

Re: [Qemu-block] [PATCH v4 0/3] Few bugfixes for userspace nvme driver

2019-07-21 Thread Maxim Levitsky
On Fri, 2019-07-19 at 11:51 +0200, Max Reitz wrote: > On 16.07.19 18:30, Maxim Levitsky wrote: > > This is reduced version of patch series for userspace nvme driver, > > that only includes the bugfixes I made. > > > > Best regards, > > Maxim Levitsky > &g

Re: [Qemu-block] [PATCH v5] LUKS: support preallocation

2019-07-21 Thread Maxim Levitsky
On Fri, 2019-07-19 at 12:28 +0200, Max Reitz wrote: > On 16.07.19 18:19, Maxim Levitsky wrote: > > preallocation=off and preallocation=metadata > > both allocate luks header only, and preallocation=falloc/full > > is passed to underlying file. > > > > F

Re: [Qemu-block] [H v5 0/3] Few bugfixes for userspace nvme driver

2019-07-16 Thread Maxim Levitsky
On Tue, 2019-07-16 at 19:29 +0300, Maxim Levitsky wrote: > This is reduced version of patch series for userspace nvme driver, > that only includes the bugfixes I made. > > Best regards, > Maxim Levitsky > > Maxim Levitsky (3): > block/nvme: fix doorbell stride

[Qemu-block] [PATCH v4 3/3] block/nvme: don't touch the completion entries

2019-07-16 Thread Maxim Levitsky
Completion entries are meant to be only read by the host and written by the device. The driver is supposed to scan the completions from the last point where it left, and until it sees a completion with non flipped phase bit. Signed-off-by: Maxim Levitsky Reviewed-by: Max Reitz --- block

[Qemu-block] [PATCH v4 1/3] block/nvme: fix doorbell stride

2019-07-16 Thread Maxim Levitsky
Fix the math involving non standard doorbell stride Signed-off-by: Maxim Levitsky Reviewed-by: Max Reitz --- block/nvme.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/nvme.c b/block/nvme.c index 9896b7f7c6..82fdefccd6 100644 --- a/block/nvme.c +++ b/block/nvme.c

[Qemu-block] [PATCH v4 2/3] block/nvme: support larger that 512 bytes sector devices

2019-07-16 Thread Maxim Levitsky
Currently the driver hardcodes the sector size to 512, and doesn't check the underlying device. Fix that. Also fail if underlying nvme device is formatted with metadata as this needs special support. Signed-off-by: Maxim Levitsky --- block/nvme.c | 45

[Qemu-block] [PATCH v4 0/3] Few bugfixes for userspace nvme driver

2019-07-16 Thread Maxim Levitsky
This is reduced version of patch series for userspace nvme driver, that only includes the bugfixes I made. Best regards, Maxim Levitsky Maxim Levitsky (3): block/nvme: fix doorbell stride block/nvme: support larger that 512 bytes sector devices block/nvme: don't touch

[Qemu-block] [H v5 0/3] Few bugfixes for userspace nvme driver

2019-07-16 Thread Maxim Levitsky
This is reduced version of patch series for userspace nvme driver, that only includes the bugfixes I made. Best regards, Maxim Levitsky Maxim Levitsky (3): block/nvme: fix doorbell stride block/nvme: support larger that 512 bytes sector devices block/nvme: don't touch

[Qemu-block] [PATCH v5] LUKS: support preallocation

2019-07-16 Thread Maxim Levitsky
preallocation=off and preallocation=metadata both allocate luks header only, and preallocation=falloc/full is passed to underlying file. Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1534951 Signed-off-by: Maxim Levitsky --- This is hopefully a revision without coding style violations

Re: [Qemu-block] [PATCH 2/7] block: Add blk_truncate_for_formatting()

2019-07-16 Thread Maxim Levitsky
On Tue, 2019-07-16 at 16:08 +0300, Maxim Levitsky wrote: > On Fri, 2019-07-12 at 19:35 +0200, Max Reitz wrote: > > Signed-off-by: Max Reitz > > --- > > include/sysemu/block-backend.h | 12 > > block/block-backend.c | 54

Re: [Qemu-block] [PATCH 1/7] block/nbd: Fix hang in .bdrv_close()

2019-07-16 Thread Maxim Levitsky
ueue_wakeup) Since we instead do aio_poll, that never happens. The patch itself looks fine, so Reviewed-by: Maxim Levitsky Best regards, Maxim Levitsky > > Suggested-by: Kevin Wolf > Signed-off-by: Max Reitz > --- > block/nbd.c | 14 +- > 1 file changed, 13

Re: [Qemu-block] [PATCH 4/7] block: Generic file creation fallback

2019-07-16 Thread Maxim Levitsky
opts) { > -error_setg(errp, "Protocol driver '%s' does not support image > creation", > - proto_drv->format_name); > -return; > -} > - > create_opts = qemu_opts_append(create_opts, drv->create_opts); > -

Re: [Qemu-block] [PATCH 3/7] block: Use blk_truncate_for_formatting()

2019-07-16 Thread Maxim Levitsky
rv_qed_co_create(BlockdevCreateOptions *opts, > l1_size = header.cluster_size * header.table_size; > > /* File must start empty and grow, check truncate is supported */ > -ret = blk_truncate(blk, 0, PREALLOC_MODE_OFF, errp); > +ret = blk_truncate_for_formatting(blk, 0, errp); > if (ret < 0) { > goto out; > } Reviewed-by: Maxim Levitsky Best regards, Maxim Levitsky

Re: [Qemu-block] [PATCH 5/7] file-posix: Drop hdev_co_create_opts()

2019-07-16 Thread Maxim Levitsky
drv_co_create_opts = hdev_co_create_opts, > -.create_opts = _create_opts, > .mutable_opts= mutable_opts, > .bdrv_co_invalidate_cache = raw_co_invalidate_cache, > > @@ -3659,8 +3594,6 @@ static BlockDriver bdrv_host_cdrom = { > .bdrv_reopen_prepare = raw_reopen_prepare, > .bdrv_reopen_commit = raw_reopen_commit, > .bdrv_reopen_abort = raw_reopen_abort, > -.bdrv_co_create_opts = hdev_co_create_opts, > -.create_opts= _create_opts, > .mutable_opts = mutable_opts, > > .bdrv_co_preadv = raw_co_preadv, Reviewed-by: Maxim Levitsky Best regards, Maxim Levitsky

Re: [Qemu-block] [PATCH 2/7] block: Add blk_truncate_for_formatting()

2019-07-16 Thread Maxim Levitsky
size or it can accept truncate ending up in bigger file that it asked for. As we once discussed on IRC, the fact that truncate on a block device 'succeeds', despite not really beeing able to change the block device size, causes other issues, like not beeing able to use preallocation=full when c

Re: [Qemu-block] [PATCH 6/7] iscsi: Drop iscsi_co_create_opts()

2019-07-16 Thread Maxim Levitsky
.bdrv_close = iscsi_close, > -.bdrv_co_create_opts= iscsi_co_create_opts, > -.create_opts= _create_opts, > .bdrv_reopen_prepare= iscsi_reopen_prepare, > .bdrv_reopen_commit = iscsi_reopen_commit, > .bdrv_co_invalidate_cache = iscsi_co_invalidate_cache, Well, in theory the original code did not zero the first sector, like what the generic code will do now, but this is OK due to the same reasons the original zeroing code was added. Reviewed-by: Maxim Levitsky Best regards, Maxim Levitsky

Re: [Qemu-block] [PATCH v4] LUKS: support preallocation

2019-07-16 Thread Maxim Levitsky
On Tue, 2019-07-16 at 14:41 +0200, Max Reitz wrote: > On 16.07.19 10:15, Maxim Levitsky wrote: > > preallocation=off and preallocation=metadata > > both allocate luks header only, and preallocation=falloc/full > > is passed to underlying file. > > > > F

[Qemu-block] [PATCH v4] LUKS: support preallocation

2019-07-16 Thread Maxim Levitsky
preallocation=off and preallocation=metadata both allocate luks header only, and preallocation=falloc/full is passed to underlying file. Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1534951 Signed-off-by: Maxim Levitsky --- block/crypto.c | 29 ++--- qapi

Re: [Qemu-block] [PATCH v3] LUKS: support preallocation

2019-07-15 Thread Maxim Levitsky
On Mon, 2019-07-15 at 10:30 +0200, Stefano Garzarella wrote: > On Sun, Jul 14, 2019 at 05:51:51PM +0300, Maxim Levitsky wrote: > > On Thu, 2019-07-11 at 18:27 +0200, Stefano Garzarella wrote: > > > On Thu, Jul 11, 2019 at 06:09:40PM +0300, Maxim Levitsky wrote: > &g

Re: [Qemu-block] [PATCH v3] LUKS: support preallocation

2019-07-14 Thread Maxim Levitsky
On Thu, 2019-07-11 at 18:27 +0200, Stefano Garzarella wrote: > On Thu, Jul 11, 2019 at 06:09:40PM +0300, Maxim Levitsky wrote: > > preallocation=off and preallocation=metadata > > both allocate luks header only, and preallocation=falloc/full > > is passed to underlying file.

[Qemu-block] [PATCH v3] LUKS: support preallocation

2019-07-11 Thread Maxim Levitsky
preallocation=off and preallocation=metadata both allocate luks header only, and preallocation=falloc/full is passed to underlying file. Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1534951 Signed-off-by: Maxim Levitsky --- Note that QMP support was only compile tested, since I am still

Re: [Qemu-block] [PATCH v2] LUKS: support preallocation in qemu-img

2019-07-11 Thread Maxim Levitsky
On Thu, 2019-07-11 at 15:43 +0200, Max Reitz wrote: > On 11.07.19 11:11, Maxim Levitsky wrote: > > preallocation=off and preallocation=metadata > > both allocate luks header only, and preallocation=falloc/full > > is passed to underlying file. > > > > F

Re: [Qemu-block] [PATCH] doc: Preallocation does not require writing zeroes

2019-07-11 Thread Maxim Levitsky
reallocation, and it looks like these are the only mentions that need to be fixed. So, thank you very much, and Reviewed-by: Maxim Levitsky Best regards, Maxim Levitsky

[Qemu-block] [PATCH v2] LUKS: support preallocation in qemu-img

2019-07-11 Thread Maxim Levitsky
preallocation=off and preallocation=metadata both allocate luks header only, and preallocation=falloc/full is passed to underlying file. Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1534951 Signed-off-by: Maxim Levitsky --- block/crypto.c | 25 ++--- 1 file changed

Re: [Qemu-block] [PATCH] LUKS: support preallocation in qemu-img

2019-07-11 Thread Maxim Levitsky
On Wed, 2019-07-10 at 23:52 +0200, Max Reitz wrote: > On 10.07.19 23:24, Max Reitz wrote: > > On 10.07.19 19:03, Maxim Levitsky wrote: > > > preallocation=off and preallocation=metadata > > > both allocate luks header only, and preallocation=falloc/full > &g

Re: [Qemu-block] [Qemu-devel] [PATCH] nvme: Set number of queues later in nvme_init()

2019-07-10 Thread Maxim Levitsky
gt; /* Set up admin queue. */ > > s->queues = g_new(NVMeQueuePair *, 1); > > -s->nr_queues = 1; > > s->queues[0] = nvme_create_queue_pair(bs, 0, NVME_QUEUE_SIZE, errp); > > if (!s->queues[0]) { > > ret = -EINVAL; > > goto out; > > } > > +s->nr_queues = 1; > > QEMU_BUILD_BUG_ON(NVME_QUEUE_SIZE & 0xF000); > > s->regs->aqa = cpu_to_le32((NVME_QUEUE_SIZE << 16) | NVME_QUEUE_SIZE); > > s->regs->asq = cpu_to_le64(s->queues[0]->sq.iova); > > > > Reviewed-by: Maxim Levitsky Best regards, Maxim Levitsky

<    4   5   6   7   8   9   10   >