Re: [PATCH] block/rbd: Add support for rbd image encryption

[Ilya Dryomov]

On Sun, Jun 27, 2021 at 1:09 PM Or Ozeri  wrote:
>
> Should I still leave the encryption format part of the state, and just not 
> report it? or should I also remove it from the state?

I'd remove it, reverting to the previous version of the patch
except instead of fetching the size with rbd_get_size() and storing
it in a local variable keep using s->image_size.

Thanks,

Ilya

RE: [PATCH] block/rbd: Add support for rbd image encryption

[Or Ozeri]

Should I still leave the encryption format part of the state, and just not report it? or should I also remove it from the state?-"Ilya Dryomov" <idryo...@gmail.com> wrote: -To: "Or Ozeri" <o...@il.ibm.com>From: "Ilya Dryomov" <idryo...@gmail.com>Date: 06/27/2021 02:00PMCc: qemu-de...@nongnu.org, qemu-block@nongnu.org, kw...@redhat.com, "Mykola Golub" <to.my.troc...@gmail.com>, "Danny Harnik" <dan...@il.ibm.com>, "Daniel P. Berrangé" <berra...@redhat.com>Subject: [EXTERNAL] Re: [PATCH] block/rbd: Add support for rbd image encryptionOn Sun, Jun 27, 2021 at 10:44 AM Or Ozeri <o...@il.ibm.com> wrote:>> Ilya,>> I fixed according to your suggestions, except for the passphrase zeroing.> Looks like it's not a one-liner, but rather a long list of ifdefs to try and cover all possible platforms/compilers (this is what I've seen they do in k5-int.h).> I didn't want to copy this into rbd.c.> I guess that the right solution would be adding a qemu utility function (not sure where exactly), but anyways perhaps this, like the changes I previously made to raw_format.c, should be made in different patch.Hi Or,Yes, given that there doesn't seem to be an existing straightforwardAPI for it, I don't think it is a blocker.  Just something to keep inmind.You also implemented one change that I didn't suggest -- storingthe encryption status in BDRVRBDState.  BTW it is a good practiceto include the version in the subject (e.g. [PATCH v3] ...) anda per-version changelog in the description.The way the encryption format is reported in this patch actually begsmore questions because now I think we need to differentiate between anencrypted image for which the encryption profile has been loaded andone for which it hasn't been loaded and probably also report the rawsize...The previous approach of reporting the encryption format only for "raw"images was a good starting point IMO.  I'd keep the bit that switchesfrom rbd_get_size() to s->image_size and drop everything else for now.>> Thanks,> Or>> -"Or Ozeri" <o...@il.ibm.com> wrote: -> To: qemu-de...@nongnu.org> From: "Or Ozeri" <o...@il.ibm.com>> Date: 06/27/2021 11:31AM> Cc: qemu-block@nongnu.org, o...@il.ibm.com, kw...@redhat.com, to.my.troc...@gmail.com, dan...@il.ibm.com, berra...@redhat.com, idryo...@gmail.com> Subject: [PATCH] block/rbd: Add support for rbd image encryption>> Starting from ceph Pacific, RBD has built-in support for image-level encryption.> Currently supported formats are LUKS version 1 and 2.>> There are 2 new relevant librbd APIs for controlling encryption, both expect an> open image context:>> rbd_encryption_format: formats an image (i.e. writes the LUKS header)> rbd_encryption_load: loads encryptor/decryptor to the image IO stack>> This commit extends the qemu rbd driver API to support the above.>> Signed-off-by: Or Ozeri <o...@il.ibm.com>> --->  block/rbd.c          | 380 ++->  qapi/block-core.json | 110 ->  2 files changed, 484 insertions(+), 6 deletions(-)>> diff --git a/block/rbd.c b/block/rbd.c> index f098a89c7b..dadecaf3da 100644> --- a/block/rbd.c> +++ b/block/rbd.c> @@ -73,6 +73,18 @@>  #define LIBRBD_USE_IOVEC 0>  #endif>> +#define RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN 8> +> +static const char rbd_luks_header_verification[> +        RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN] = {> +    'L', 'U', 'K', 'S', 0xBA, 0xBE, 0, 1> +};> +> +static const char rbd_luks2_header_verification[> +        RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN] = {> +    'L', 'U', 'K', 'S', 0xBA, 0xBE, 0, 2> +};> +>  typedef enum {>      RBD_AIO_READ,>      RBD_AIO_WRITE,> @@ -106,6 +118,7 @@ typedef struct BDRVRBDState {>      char *snap;>      char *namespace;>      uint64_t image_size;> +    ImageInfoSpecificRbd image_info;>  } BDRVRBDState;>>  static int qemu_rbd_connect(rados_t *cluster, rados_ioctx_t *io_ctx,> @@ -341,6 +354,207 @@ static void qemu_rbd_memset(RADOSCB *rcb, int64_t offs)>      }>  }>> +#ifdef LIBRBD_SUPPORTS_ENCRYPTION> +static int qemu_rbd_convert_luks_options(> +        RbdEncryptionOptionsLUKSBase *luks_opts,> +        char **passphrase,> +        size_t *passphrase_len,> +        Error **errp)> +{> +    return qcrypto_secret_lookup(luks_opts->key_secret, (uint8_t **)passphrase,> +                                 passphrase_len, errp);> +}> +> +static int qemu_rbd_convert_luks_create_options(> +        RbdEncryptionCreateOptionsLUKSBase *luks_opts,> +        rbd_encryption_algorithm_t *alg,> +        char **passphrase,> +        size_t *passphrase_len

Re: [PATCH] block/rbd: Add support for rbd image encryption

[Ilya Dryomov]

On Sun, Jun 27, 2021 at 10:44 AM Or Ozeri  wrote:
>
> Ilya,
>
> I fixed according to your suggestions, except for the passphrase zeroing.
> Looks like it's not a one-liner, but rather a long list of ifdefs to try and 
> cover all possible platforms/compilers (this is what I've seen they do in 
> k5-int.h).
> I didn't want to copy this into rbd.c.
> I guess that the right solution would be adding a qemu utility function (not 
> sure where exactly), but anyways perhaps this, like the changes I previously 
> made to raw_format.c, should be made in different patch.

Hi Or,

Yes, given that there doesn't seem to be an existing straightforward
API for it, I don't think it is a blocker.  Just something to keep in
mind.

You also implemented one change that I didn't suggest -- storing
the encryption status in BDRVRBDState.  BTW it is a good practice
to include the version in the subject (e.g. [PATCH v3] ...) and
a per-version changelog in the description.

The way the encryption format is reported in this patch actually begs
more questions because now I think we need to differentiate between an
encrypted image for which the encryption profile has been loaded and
one for which it hasn't been loaded and probably also report the raw
size...

The previous approach of reporting the encryption format only for "raw"
images was a good starting point IMO.  I'd keep the bit that switches
from rbd_get_size() to s->image_size and drop everything else for now.

>
> Thanks,
> Or
>
> -"Or Ozeri"  wrote: -
> To: qemu-de...@nongnu.org
> From: "Or Ozeri" 
> Date: 06/27/2021 11:31AM
> Cc: qemu-block@nongnu.org, o...@il.ibm.com, kw...@redhat.com, 
> to.my.troc...@gmail.com, dan...@il.ibm.com, berra...@redhat.com, 
> idryo...@gmail.com
> Subject: [PATCH] block/rbd: Add support for rbd image encryption
>
> Starting from ceph Pacific, RBD has built-in support for image-level 
> encryption.
> Currently supported formats are LUKS version 1 and 2.
>
> There are 2 new relevant librbd APIs for controlling encryption, both expect 
> an
> open image context:
>
> rbd_encryption_format: formats an image (i.e. writes the LUKS header)
> rbd_encryption_load: loads encryptor/decryptor to the image IO stack
>
> This commit extends the qemu rbd driver API to support the above.
>
> Signed-off-by: Or Ozeri 
> ---
>  block/rbd.c  | 380 ++-
>  qapi/block-core.json | 110 -
>  2 files changed, 484 insertions(+), 6 deletions(-)
>
> diff --git a/block/rbd.c b/block/rbd.c
> index f098a89c7b..dadecaf3da 100644
> --- a/block/rbd.c
> +++ b/block/rbd.c
> @@ -73,6 +73,18 @@
>  #define LIBRBD_USE_IOVEC 0
>  #endif
>
> +#define RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN 8
> +
> +static const char rbd_luks_header_verification[
> +RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN] = {
> +'L', 'U', 'K', 'S', 0xBA, 0xBE, 0, 1
> +};
> +
> +static const char rbd_luks2_header_verification[
> +RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN] = {
> +'L', 'U', 'K', 'S', 0xBA, 0xBE, 0, 2
> +};
> +
>  typedef enum {
>  RBD_AIO_READ,
>  RBD_AIO_WRITE,
> @@ -106,6 +118,7 @@ typedef struct BDRVRBDState {
>  char *snap;
>  char *namespace;
>  uint64_t image_size;
> +ImageInfoSpecificRbd image_info;
>  } BDRVRBDState;
>
>  static int qemu_rbd_connect(rados_t *cluster, rados_ioctx_t *io_ctx,
> @@ -341,6 +354,207 @@ static void qemu_rbd_memset(RADOSCB *rcb, int64_t offs)
>  }
>  }
>
> +#ifdef LIBRBD_SUPPORTS_ENCRYPTION
> +static int qemu_rbd_convert_luks_options(
> +RbdEncryptionOptionsLUKSBase *luks_opts,
> +char **passphrase,
> +size_t *passphrase_len,
> +Error **errp)
> +{
> +return qcrypto_secret_lookup(luks_opts->key_secret, (uint8_t 
> **)passphrase,
> + passphrase_len, errp);
> +}
> +
> +static int qemu_rbd_convert_luks_create_options(
> +RbdEncryptionCreateOptionsLUKSBase *luks_opts,
> +rbd_encryption_algorithm_t *alg,
> +char **passphrase,
> +size_t *passphrase_len,
> +Error **errp)
> +{
> +int r = 0;
> +
> +r = qemu_rbd_convert_luks_options(
> +qapi_RbdEncryptionCreateOptionsLUKSBase_base(luks_opts),
> +passphrase, passphrase_len, errp);
> +if (r < 0) {
> +return r;
> +}
> +
> +if (luks_opts->has_cipher_alg) {
> +switch (luks_opts->cipher_alg) {
> +case QCRYPTO_CIPHER_ALG_AES_128: {
> +*alg = RBD_ENCRYPTION_ALGORITHM_AES128;
> +break;
> +}
&g

Re: [PATCH] block/rbd: Add support for rbd image encryption

[Or Ozeri]

Ilya,I fixed according to your suggestions, except for the passphrase zeroing.Looks like it's not a one-liner, but rather a long list of ifdefs to try and cover all possible platforms/compilers (this is what I've seen they do in k5-int.h).I didn't want to copy this into rbd.c.I guess that the right solution would be adding a qemu utility function (not sure where exactly), but anyways perhaps this, like the changes I previously made to raw_format.c, should be made in different patch.Thanks,Or-"Or Ozeri"  wrote: -To: qemu-de...@nongnu.orgFrom: "Or Ozeri" Date: 06/27/2021 11:31AMCc: qemu-block@nongnu.org, o...@il.ibm.com, kw...@redhat.com, to.my.troc...@gmail.com, dan...@il.ibm.com, berra...@redhat.com, idryo...@gmail.comSubject: [PATCH] block/rbd: Add support for rbd image encryptionStarting from ceph Pacific, RBD has built-in support for image-level encryption.Currently supported formats are LUKS version 1 and 2.There are 2 new relevant librbd APIs for controlling encryption, both expect anopen image context:rbd_encryption_format: formats an image (i.e. writes the LUKS header)rbd_encryption_load: loads encryptor/decryptor to the image IO stackThis commit extends the qemu rbd driver API to support the above.Signed-off-by: Or Ozeri --- block/rbd.c          | 380 ++- qapi/block-core.json | 110 - 2 files changed, 484 insertions(+), 6 deletions(-)diff --git a/block/rbd.c b/block/rbd.cindex f098a89c7b..dadecaf3da 100644--- a/block/rbd.c+++ b/block/rbd.c@@ -73,6 +73,18 @@ #define LIBRBD_USE_IOVEC 0 #endif +#define RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN 8++static const char rbd_luks_header_verification[+        RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN] = {+    'L', 'U', 'K', 'S', 0xBA, 0xBE, 0, 1+};++static const char rbd_luks2_header_verification[+        RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN] = {+    'L', 'U', 'K', 'S', 0xBA, 0xBE, 0, 2+};+ typedef enum {     RBD_AIO_READ,     RBD_AIO_WRITE,@@ -106,6 +118,7 @@ typedef struct BDRVRBDState {     char *snap;     char *namespace;     uint64_t image_size;+    ImageInfoSpecificRbd image_info; } BDRVRBDState;  static int qemu_rbd_connect(rados_t *cluster, rados_ioctx_t *io_ctx,@@ -341,6 +354,207 @@ static void qemu_rbd_memset(RADOSCB *rcb, int64_t offs)     } } +#ifdef LIBRBD_SUPPORTS_ENCRYPTION+static int qemu_rbd_convert_luks_options(+        RbdEncryptionOptionsLUKSBase *luks_opts,+        char **passphrase,+        size_t *passphrase_len,+        Error **errp)+{+    return qcrypto_secret_lookup(luks_opts->key_secret, (uint8_t **)passphrase,+                                 passphrase_len, errp);+}++static int qemu_rbd_convert_luks_create_options(+        RbdEncryptionCreateOptionsLUKSBase *luks_opts,+        rbd_encryption_algorithm_t *alg,+        char **passphrase,+        size_t *passphrase_len,+        Error **errp)+{+    int r = 0;++    r = qemu_rbd_convert_luks_options(+            qapi_RbdEncryptionCreateOptionsLUKSBase_base(luks_opts),+            passphrase, passphrase_len, errp);+    if (r < 0) {+        return r;+    }++    if (luks_opts->has_cipher_alg) {+        switch (luks_opts->cipher_alg) {+            case QCRYPTO_CIPHER_ALG_AES_128: {+                *alg = RBD_ENCRYPTION_ALGORITHM_AES128;+                break;+            }+            case QCRYPTO_CIPHER_ALG_AES_256: {+                *alg = RBD_ENCRYPTION_ALGORITHM_AES256;+                break;+            }+            default: {+                r = -ENOTSUP;+                error_setg_errno(errp, -r, "unknown encryption algorithm: %u",+                                 luks_opts->cipher_alg);+                return r;+            }+        }+    } else {+        /* default alg */+        *alg = RBD_ENCRYPTION_ALGORITHM_AES256;+    }++    return 0;+}++static int qemu_rbd_encryption_format(rbd_image_t image,+                                      RbdEncryptionCreateOptions *encrypt,+                                      Error **errp)+{+    int r = 0;+    g_autofree char *passphrase = NULL;+    size_t passphrase_len;+    rbd_encryption_format_t format;+    rbd_encryption_options_t opts;+    rbd_encryption_luks1_format_options_t luks_opts;+    rbd_encryption_luks2_format_options_t luks2_opts;+    size_t opts_size;+    uint64_t raw_size, effective_size;++    r = rbd_get_size(image, _size);+    if (r < 0) {+        error_setg_errno(errp, -r, "cannot get raw image size");+        return r;+    }++    switch (encrypt->format) {+        case RBD_IMAGE_ENCRYPTION_FORMAT_LUKS: {+            memset(_opts, 0, sizeof(luks_opts));+            format = RBD_ENCRYPTION_FORMAT_LUKS1;+            opts = _opts;+            opts_size = sizeof(luks_opts);+            r = qemu_rbd_convert_luks_create_options(+                    qapi_RbdEncryptionCreateOptionsLUKS_base(>u.luks),+                    _opts.alg, , _len, errp);+            if (r < 0) {+                return r;+            }+     

[PATCH] block/rbd: Add support for rbd image encryption

[Or Ozeri]

Starting from ceph Pacific, RBD has built-in support for image-level encryption.
Currently supported formats are LUKS version 1 and 2.

There are 2 new relevant librbd APIs for controlling encryption, both expect an
open image context:

rbd_encryption_format: formats an image (i.e. writes the LUKS header)
rbd_encryption_load: loads encryptor/decryptor to the image IO stack

This commit extends the qemu rbd driver API to support the above.

Signed-off-by: Or Ozeri 
---
 block/rbd.c  | 380 ++-
 qapi/block-core.json | 110 -
 2 files changed, 484 insertions(+), 6 deletions(-)

diff --git a/block/rbd.c b/block/rbd.c
index f098a89c7b..dadecaf3da 100644
--- a/block/rbd.c
+++ b/block/rbd.c
@@ -73,6 +73,18 @@
 #define LIBRBD_USE_IOVEC 0
 #endif
 
+#define RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN 8
+
+static const char rbd_luks_header_verification[
+RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN] = {
+'L', 'U', 'K', 'S', 0xBA, 0xBE, 0, 1
+};
+
+static const char rbd_luks2_header_verification[
+RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN] = {
+'L', 'U', 'K', 'S', 0xBA, 0xBE, 0, 2
+};
+
 typedef enum {
 RBD_AIO_READ,
 RBD_AIO_WRITE,
@@ -106,6 +118,7 @@ typedef struct BDRVRBDState {
 char *snap;
 char *namespace;
 uint64_t image_size;
+ImageInfoSpecificRbd image_info;
 } BDRVRBDState;
 
 static int qemu_rbd_connect(rados_t *cluster, rados_ioctx_t *io_ctx,
@@ -341,6 +354,207 @@ static void qemu_rbd_memset(RADOSCB *rcb, int64_t offs)
 }
 }
 
+#ifdef LIBRBD_SUPPORTS_ENCRYPTION
+static int qemu_rbd_convert_luks_options(
+RbdEncryptionOptionsLUKSBase *luks_opts,
+char **passphrase,
+size_t *passphrase_len,
+Error **errp)
+{
+return qcrypto_secret_lookup(luks_opts->key_secret, (uint8_t **)passphrase,
+ passphrase_len, errp);
+}
+
+static int qemu_rbd_convert_luks_create_options(
+RbdEncryptionCreateOptionsLUKSBase *luks_opts,
+rbd_encryption_algorithm_t *alg,
+char **passphrase,
+size_t *passphrase_len,
+Error **errp)
+{
+int r = 0;
+
+r = qemu_rbd_convert_luks_options(
+qapi_RbdEncryptionCreateOptionsLUKSBase_base(luks_opts),
+passphrase, passphrase_len, errp);
+if (r < 0) {
+return r;
+}
+
+if (luks_opts->has_cipher_alg) {
+switch (luks_opts->cipher_alg) {
+case QCRYPTO_CIPHER_ALG_AES_128: {
+*alg = RBD_ENCRYPTION_ALGORITHM_AES128;
+break;
+}
+case QCRYPTO_CIPHER_ALG_AES_256: {
+*alg = RBD_ENCRYPTION_ALGORITHM_AES256;
+break;
+}
+default: {
+r = -ENOTSUP;
+error_setg_errno(errp, -r, "unknown encryption algorithm: %u",
+ luks_opts->cipher_alg);
+return r;
+}
+}
+} else {
+/* default alg */
+*alg = RBD_ENCRYPTION_ALGORITHM_AES256;
+}
+
+return 0;
+}
+
+static int qemu_rbd_encryption_format(rbd_image_t image,
+  RbdEncryptionCreateOptions *encrypt,
+  Error **errp)
+{
+int r = 0;
+g_autofree char *passphrase = NULL;
+size_t passphrase_len;
+rbd_encryption_format_t format;
+rbd_encryption_options_t opts;
+rbd_encryption_luks1_format_options_t luks_opts;
+rbd_encryption_luks2_format_options_t luks2_opts;
+size_t opts_size;
+uint64_t raw_size, effective_size;
+
+r = rbd_get_size(image, _size);
+if (r < 0) {
+error_setg_errno(errp, -r, "cannot get raw image size");
+return r;
+}
+
+switch (encrypt->format) {
+case RBD_IMAGE_ENCRYPTION_FORMAT_LUKS: {
+memset(_opts, 0, sizeof(luks_opts));
+format = RBD_ENCRYPTION_FORMAT_LUKS1;
+opts = _opts;
+opts_size = sizeof(luks_opts);
+r = qemu_rbd_convert_luks_create_options(
+qapi_RbdEncryptionCreateOptionsLUKS_base(>u.luks),
+_opts.alg, , _len, errp);
+if (r < 0) {
+return r;
+}
+luks_opts.passphrase = passphrase;
+luks_opts.passphrase_size = passphrase_len;
+break;
+}
+case RBD_IMAGE_ENCRYPTION_FORMAT_LUKS2: {
+memset(_opts, 0, sizeof(luks2_opts));
+format = RBD_ENCRYPTION_FORMAT_LUKS2;
+opts = _opts;
+opts_size = sizeof(luks2_opts);
+r = qemu_rbd_convert_luks_create_options(
+qapi_RbdEncryptionCreateOptionsLUKS2_base(
+>u.luks2),
+_opts.alg, , _len, errp);
+if (r < 0) {
+return r;
+}
+luks2_opts.passphrase = passphrase;
+

Re: [PATCH] block/rbd: Add support for rbd image encryption

[Ilya Dryomov]

On Mon, Jun 21, 2021 at 4:49 PM Or Ozeri  wrote:
>
> Starting from ceph Pacific, RBD has built-in support for image-level 
> encryption.
> Currently supported formats are LUKS version 1 and 2.
>
> There are 2 new relevant librbd APIs for controlling encryption, both expect 
> an
> open image context:
>
> rbd_encryption_format: formats an image (i.e. writes the LUKS header)
> rbd_encryption_load: loads encryptor/decryptor to the image IO stack
>
> This commit extends the qemu rbd driver API to support the above.
>
> Signed-off-by: Or Ozeri 
> ---
>  block/rbd.c  | 367 ++-
>  qapi/block-core.json | 110 -
>  2 files changed, 471 insertions(+), 6 deletions(-)
>
> diff --git a/block/rbd.c b/block/rbd.c
> index f098a89c7b..7e282a8e94 100644
> --- a/block/rbd.c
> +++ b/block/rbd.c
> @@ -73,6 +73,18 @@
>  #define LIBRBD_USE_IOVEC 0
>  #endif
>
> +#define RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN 8
> +
> +static const char rbd_luks_header_verification[
> +RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN] = {
> +'L', 'U', 'K', 'S', 0xBA, 0xBE, 0, 1
> +};
> +
> +static const char rbd_luks2_header_verification[
> +RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN] = {
> +'L', 'U', 'K', 'S', 0xBA, 0xBE, 0, 2
> +};
> +
>  typedef enum {
>  RBD_AIO_READ,
>  RBD_AIO_WRITE,
> @@ -341,6 +353,202 @@ static void qemu_rbd_memset(RADOSCB *rcb, int64_t offs)
>  }
>  }
>
> +#ifdef LIBRBD_SUPPORTS_ENCRYPTION
> +static int qemu_rbd_convert_luks_options(
> +RbdEncryptionOptionsLUKSBase *luks_opts,
> +char **passphrase,
> +size_t *passphrase_len,
> +Error **errp)
> +{
> +return qcrypto_secret_lookup(
> +luks_opts->key_secret, (uint8_t **)passphrase, passphrase_len, 
> errp);

Hi Or,

This looks good to me, just a few nits to consider:

Why wrap after the paren?  The second line ends up going over 80
columns.  I'd wrap between passphrase and passphrase_len.

> +}
> +
> +static int qemu_rbd_convert_luks_create_options(
> +RbdEncryptionCreateOptionsLUKSBase *luks_opts,
> +rbd_encryption_algorithm_t *alg,
> +char **passphrase,
> +size_t *passphrase_len,
> +Error **errp)
> +{
> +int r = 0;
> +
> +r = qemu_rbd_convert_luks_options(
> +qapi_RbdEncryptionCreateOptionsLUKSBase_base(luks_opts),
> +passphrase, passphrase_len, errp);
> +if (r < 0) {
> +return r;
> +}
> +
> +if (luks_opts->has_cipher_alg) {
> +switch (luks_opts->cipher_alg) {
> +case QCRYPTO_CIPHER_ALG_AES_128: {
> +*alg = RBD_ENCRYPTION_ALGORITHM_AES128;
> +break;
> +}
> +case QCRYPTO_CIPHER_ALG_AES_256: {
> +*alg = RBD_ENCRYPTION_ALGORITHM_AES256;
> +break;
> +}
> +default: {
> +r = -ENOTSUP;
> +error_setg_errno(errp, -r, "unknown encryption algorithm: 
> %u",
> + luks_opts->cipher_alg);
> +return r;
> +}
> +}
> +} else {
> +/* default alg */
> +*alg = RBD_ENCRYPTION_ALGORITHM_AES256;
> +}
> +
> +return 0;
> +}
> +
> +static int qemu_rbd_encryption_format(rbd_image_t image,
> +  RbdEncryptionCreateOptions *encrypt,
> +  Error **errp)
> +{
> +int r = 0;
> +g_autofree char *passphrase = NULL;

Should the passphrase be zeroed before being freed to avoid it lying
around in memory?  A quick grep didn't turn up a memzero_explicit or
memset_s like pattern in other places though...

> +size_t passphrase_len;
> +rbd_encryption_format_t format;
> +rbd_encryption_options_t opts;
> +rbd_encryption_luks1_format_options_t luks_opts;
> +rbd_encryption_luks2_format_options_t luks2_opts;
> +size_t opts_size;
> +uint64_t raw_size, effective_size;
> +
> +r = rbd_get_size(image, _size);
> +if (r < 0) {
> +error_setg_errno(errp, -r, "cannot get raw image size");
> +return r;
> +}
> +
> +switch (encrypt->format) {
> +case RBD_IMAGE_ENCRYPTION_FORMAT_LUKS: {
> +memset(_opts, 0, sizeof(luks_opts));
> +format = RBD_ENCRYPTION_FORMAT_LUKS1;
> +opts = _opts;
> +opts_size = sizeof(luks_opts);
> +r = qemu_rbd_convert_luks_create_options(
> +
> qapi_RbdEncryptionCreateOptionsLUKS_base(>u.luks),
> +_opts.alg, , _len, errp);
> +if (r < 0) {
> +return r;
> +}
> +luks_opts.passphrase = passphrase;
> +luks_opts.passphrase_size = passphrase_len;
> +break;
> +}
> +case RBD_IMAGE_ENCRYPTION_FORMAT_LUKS2: {
> +memset(_opts, 0, sizeof(luks2_opts));
> +format = 

[PATCH] block/rbd: Add support for rbd image encryption

[Or Ozeri]

Starting from ceph Pacific, RBD has built-in support for image-level encryption.
Currently supported formats are LUKS version 1 and 2.

There are 2 new relevant librbd APIs for controlling encryption, both expect an
open image context:

rbd_encryption_format: formats an image (i.e. writes the LUKS header)
rbd_encryption_load: loads encryptor/decryptor to the image IO stack

This commit extends the qemu rbd driver API to support the above.

Signed-off-by: Or Ozeri 
---
 block/rbd.c  | 367 ++-
 qapi/block-core.json | 110 -
 2 files changed, 471 insertions(+), 6 deletions(-)

diff --git a/block/rbd.c b/block/rbd.c
index f098a89c7b..7e282a8e94 100644
--- a/block/rbd.c
+++ b/block/rbd.c
@@ -73,6 +73,18 @@
 #define LIBRBD_USE_IOVEC 0
 #endif
 
+#define RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN 8
+
+static const char rbd_luks_header_verification[
+RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN] = {
+'L', 'U', 'K', 'S', 0xBA, 0xBE, 0, 1
+};
+
+static const char rbd_luks2_header_verification[
+RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN] = {
+'L', 'U', 'K', 'S', 0xBA, 0xBE, 0, 2
+};
+
 typedef enum {
 RBD_AIO_READ,
 RBD_AIO_WRITE,
@@ -341,6 +353,202 @@ static void qemu_rbd_memset(RADOSCB *rcb, int64_t offs)
 }
 }
 
+#ifdef LIBRBD_SUPPORTS_ENCRYPTION
+static int qemu_rbd_convert_luks_options(
+RbdEncryptionOptionsLUKSBase *luks_opts,
+char **passphrase,
+size_t *passphrase_len,
+Error **errp)
+{
+return qcrypto_secret_lookup(
+luks_opts->key_secret, (uint8_t **)passphrase, passphrase_len, 
errp);
+}
+
+static int qemu_rbd_convert_luks_create_options(
+RbdEncryptionCreateOptionsLUKSBase *luks_opts,
+rbd_encryption_algorithm_t *alg,
+char **passphrase,
+size_t *passphrase_len,
+Error **errp)
+{
+int r = 0;
+
+r = qemu_rbd_convert_luks_options(
+qapi_RbdEncryptionCreateOptionsLUKSBase_base(luks_opts),
+passphrase, passphrase_len, errp);
+if (r < 0) {
+return r;
+}
+
+if (luks_opts->has_cipher_alg) {
+switch (luks_opts->cipher_alg) {
+case QCRYPTO_CIPHER_ALG_AES_128: {
+*alg = RBD_ENCRYPTION_ALGORITHM_AES128;
+break;
+}
+case QCRYPTO_CIPHER_ALG_AES_256: {
+*alg = RBD_ENCRYPTION_ALGORITHM_AES256;
+break;
+}
+default: {
+r = -ENOTSUP;
+error_setg_errno(errp, -r, "unknown encryption algorithm: %u",
+ luks_opts->cipher_alg);
+return r;
+}
+}
+} else {
+/* default alg */
+*alg = RBD_ENCRYPTION_ALGORITHM_AES256;
+}
+
+return 0;
+}
+
+static int qemu_rbd_encryption_format(rbd_image_t image,
+  RbdEncryptionCreateOptions *encrypt,
+  Error **errp)
+{
+int r = 0;
+g_autofree char *passphrase = NULL;
+size_t passphrase_len;
+rbd_encryption_format_t format;
+rbd_encryption_options_t opts;
+rbd_encryption_luks1_format_options_t luks_opts;
+rbd_encryption_luks2_format_options_t luks2_opts;
+size_t opts_size;
+uint64_t raw_size, effective_size;
+
+r = rbd_get_size(image, _size);
+if (r < 0) {
+error_setg_errno(errp, -r, "cannot get raw image size");
+return r;
+}
+
+switch (encrypt->format) {
+case RBD_IMAGE_ENCRYPTION_FORMAT_LUKS: {
+memset(_opts, 0, sizeof(luks_opts));
+format = RBD_ENCRYPTION_FORMAT_LUKS1;
+opts = _opts;
+opts_size = sizeof(luks_opts);
+r = qemu_rbd_convert_luks_create_options(
+qapi_RbdEncryptionCreateOptionsLUKS_base(>u.luks),
+_opts.alg, , _len, errp);
+if (r < 0) {
+return r;
+}
+luks_opts.passphrase = passphrase;
+luks_opts.passphrase_size = passphrase_len;
+break;
+}
+case RBD_IMAGE_ENCRYPTION_FORMAT_LUKS2: {
+memset(_opts, 0, sizeof(luks2_opts));
+format = RBD_ENCRYPTION_FORMAT_LUKS2;
+opts = _opts;
+opts_size = sizeof(luks2_opts);
+r = qemu_rbd_convert_luks_create_options(
+qapi_RbdEncryptionCreateOptionsLUKS2_base(
+>u.luks2),
+_opts.alg, , _len, errp);
+if (r < 0) {
+return r;
+}
+luks2_opts.passphrase = passphrase;
+luks2_opts.passphrase_size = passphrase_len;
+break;
+}
+default: {
+r = -ENOTSUP;
+error_setg_errno(
+errp, -r, "unknown image encryption format: %u",
+encrypt->format);
+return 

Re: [PATCH] block/rbd: Add support for rbd image encryption

[no-reply]

Patchew URL: https://patchew.org/QEMU/20210621142103.1417408-1-...@il.ibm.com/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Type: series
Message-id: 20210621142103.1417408-1-...@il.ibm.com
Subject: [PATCH] block/rbd: Add support for rbd image encryption

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
 - [tag update]  
patchew/20210614083800.1166166-1-richard.hender...@linaro.org -> 
patchew/20210614083800.1166166-1-richard.hender...@linaro.org
 * [new tag] patchew/20210621142103.1417408-1-...@il.ibm.com -> 
patchew/20210621142103.1417408-1-...@il.ibm.com
Switched to a new branch 'test'
19faaee block/rbd: Add support for rbd image encryption

=== OUTPUT BEGIN ===
ERROR: "(foo**)" should be "(foo **)"
#60: FILE: block/rbd.c:374:
+luks_opts->key_secret, (uint8_t**)passphrase, passphrase_len, 
errp);

total: 1 errors, 0 warnings, 601 lines checked

Commit 19faaee12b6d (block/rbd: Add support for rbd image encryption) has style 
problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
=== OUTPUT END ===

Test command exited with code: 1


The full log is available at
http://patchew.org/logs/20210621142103.1417408-1-...@il.ibm.com/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-de...@redhat.com

[PATCH] block/rbd: Add support for rbd image encryption

[Or Ozeri]

Starting from ceph Pacific, RBD has built-in support for image-level encryption.
Currently supported formats are LUKS version 1 and 2.

There are 2 new relevant librbd APIs for controlling encryption, both expect an
open image context:

rbd_encryption_format: formats an image (i.e. writes the LUKS header)
rbd_encryption_load: loads encryptor/decryptor to the image IO stack

This commit extends the qemu rbd driver API to support the above.

Signed-off-by: Or Ozeri 
---
 block/rbd.c  | 367 ++-
 qapi/block-core.json | 110 -
 2 files changed, 471 insertions(+), 6 deletions(-)

diff --git a/block/rbd.c b/block/rbd.c
index f098a89c7b..be1419a9bd 100644
--- a/block/rbd.c
+++ b/block/rbd.c
@@ -73,6 +73,18 @@
 #define LIBRBD_USE_IOVEC 0
 #endif
 
+#define RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN 8
+
+static const char rbd_luks_header_verification[
+RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN] = {
+'L', 'U', 'K', 'S', 0xBA, 0xBE, 0, 1
+};
+
+static const char rbd_luks2_header_verification[
+RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN] = {
+'L', 'U', 'K', 'S', 0xBA, 0xBE, 0, 2
+};
+
 typedef enum {
 RBD_AIO_READ,
 RBD_AIO_WRITE,
@@ -341,6 +353,202 @@ static void qemu_rbd_memset(RADOSCB *rcb, int64_t offs)
 }
 }
 
+#ifdef LIBRBD_SUPPORTS_ENCRYPTION
+static int qemu_rbd_convert_luks_options(
+RbdEncryptionOptionsLUKSBase *luks_opts,
+char **passphrase,
+size_t *passphrase_len,
+Error **errp)
+{
+return qcrypto_secret_lookup(
+luks_opts->key_secret, (uint8_t**)passphrase, passphrase_len, 
errp);
+}
+
+static int qemu_rbd_convert_luks_create_options(
+RbdEncryptionCreateOptionsLUKSBase *luks_opts,
+rbd_encryption_algorithm_t *alg,
+char **passphrase,
+size_t *passphrase_len,
+Error **errp)
+{
+int r = 0;
+
+r = qemu_rbd_convert_luks_options(
+qapi_RbdEncryptionCreateOptionsLUKSBase_base(luks_opts),
+passphrase, passphrase_len, errp);
+if (r < 0) {
+return r;
+}
+
+if (luks_opts->has_cipher_alg) {
+switch (luks_opts->cipher_alg) {
+case QCRYPTO_CIPHER_ALG_AES_128: {
+*alg = RBD_ENCRYPTION_ALGORITHM_AES128;
+break;
+}
+case QCRYPTO_CIPHER_ALG_AES_256: {
+*alg = RBD_ENCRYPTION_ALGORITHM_AES256;
+break;
+}
+default: {
+r = -ENOTSUP;
+error_setg_errno(errp, -r, "unknown encryption algorithm: %u",
+ luks_opts->cipher_alg);
+return r;
+}
+}
+} else {
+/* default alg */
+*alg = RBD_ENCRYPTION_ALGORITHM_AES256;
+}
+
+return 0;
+}
+
+static int qemu_rbd_encryption_format(rbd_image_t image,
+  RbdEncryptionCreateOptions *encrypt,
+  Error **errp)
+{
+int r = 0;
+g_autofree char *passphrase = NULL;
+size_t passphrase_len;
+rbd_encryption_format_t format;
+rbd_encryption_options_t opts;
+rbd_encryption_luks1_format_options_t luks_opts;
+rbd_encryption_luks2_format_options_t luks2_opts;
+size_t opts_size;
+uint64_t raw_size, effective_size;
+
+r = rbd_get_size(image, _size);
+if (r < 0) {
+error_setg_errno(errp, -r, "cannot get raw image size");
+return r;
+}
+
+switch (encrypt->format) {
+case RBD_IMAGE_ENCRYPTION_FORMAT_LUKS: {
+memset(_opts, 0, sizeof(luks_opts));
+format = RBD_ENCRYPTION_FORMAT_LUKS1;
+opts = _opts;
+opts_size = sizeof(luks_opts);
+r = qemu_rbd_convert_luks_create_options(
+qapi_RbdEncryptionCreateOptionsLUKS_base(>u.luks),
+_opts.alg, , _len, errp);
+if (r < 0) {
+return r;
+}
+luks_opts.passphrase = passphrase;
+luks_opts.passphrase_size = passphrase_len;
+break;
+}
+case RBD_IMAGE_ENCRYPTION_FORMAT_LUKS2: {
+memset(_opts, 0, sizeof(luks2_opts));
+format = RBD_ENCRYPTION_FORMAT_LUKS2;
+opts = _opts;
+opts_size = sizeof(luks2_opts);
+r = qemu_rbd_convert_luks_create_options(
+qapi_RbdEncryptionCreateOptionsLUKS2_base(
+>u.luks2),
+_opts.alg, , _len, errp);
+if (r < 0) {
+return r;
+}
+luks2_opts.passphrase = passphrase;
+luks2_opts.passphrase_size = passphrase_len;
+break;
+}
+default: {
+r = -ENOTSUP;
+error_setg_errno(
+errp, -r, "unknown image encryption format: %u",
+encrypt->format);
+return r;

Re: [PATCH] block/rbd: Add support for rbd image encryption

[Ilya Dryomov]

On Mon, Jun 21, 2021 at 1:27 PM Daniel P. Berrangé  wrote:
>
> On Mon, Jun 21, 2021 at 01:23:46PM +0200, Ilya Dryomov wrote:
> > On Mon, Jun 21, 2021 at 1:04 PM Daniel P. Berrangé  
> > wrote:
> > >
> > > On Mon, Jun 21, 2021 at 12:59:37PM +0200, Ilya Dryomov wrote:
> > > > On Mon, Jun 21, 2021 at 10:32 AM Daniel P. Berrangé 
> > > >  wrote:
> > > > >
> > > > > On Sat, Jun 19, 2021 at 09:44:32PM +0200, Ilya Dryomov wrote:
> > > > > > On Thu, Jun 17, 2021 at 6:05 PM Or Ozeri  wrote:
> > > > > > >
> > > > > > > Starting from ceph Pacific, RBD has built-in support for 
> > > > > > > image-level encryption.
> > > > > > > Currently supported formats are LUKS version 1 and 2.
> > > > > > >
> > > > > > > There are 2 new relevant librbd APIs for controlling encryption, 
> > > > > > > both expect an
> > > > > > > open image context:
> > > > > > >
> > > > > > > rbd_encryption_format: formats an image (i.e. writes the LUKS 
> > > > > > > header)
> > > > > > > rbd_encryption_load: loads encryptor/decryptor to the image IO 
> > > > > > > stack
> > > > > > >
> > > > > > > This commit extends the qemu rbd driver API to support the above.
> > > > > > >
> > > > > > > Signed-off-by: Or Ozeri 
> > > > > > > ---
> > > > > > >  block/raw-format.c   |   7 +
> > > > > > >  block/rbd.c  | 371 
> > > > > > > ++-
> > > > > > >  qapi/block-core.json | 110 -
> > > > > > >  3 files changed, 482 insertions(+), 6 deletions(-)
> > > > >
> > > > >
> > > > > > > diff --git a/block/rbd.c b/block/rbd.c
> > > > > > > index f098a89c7b..183b17cd84 100644
> > > > > > > --- a/block/rbd.c
> > > > > > > +++ b/block/rbd.c
> > > > > > > @@ -73,6 +73,18 @@
> > > > > > >  #define LIBRBD_USE_IOVEC 0
> > > > > > >  #endif
> > > > > > >
> > > > > > > +#define RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN 8
> > > > > > > +
> > > > > > > +static const char rbd_luks_header_verification[
> > > > > > > +RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN] = {
> > > > > > > +'L', 'U', 'K', 'S', 0xBA, 0xBE, 0, 1
> > > > > > > +};
> > > > > > > +
> > > > > > > +static const char rbd_luks2_header_verification[
> > > > > > > +RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN] = {
> > > > > > > +'L', 'U', 'K', 'S', 0xBA, 0xBE, 0, 2
> > > > > > > +};
> > > > > > > +
> > > > > > >  typedef enum {
> > > > > > >  RBD_AIO_READ,
> > > > > > >  RBD_AIO_WRITE,
> > > > > > > @@ -341,6 +353,206 @@ static void qemu_rbd_memset(RADOSCB *rcb, 
> > > > > > > int64_t offs)
> > > > > > >  }
> > > > > > >  }
> > > > > > >
> > > > > > > +#ifdef LIBRBD_SUPPORTS_ENCRYPTION
> > > > > > > +static int qemu_rbd_convert_luks_options(
> > > > > > > +RbdEncryptionOptionsLUKSBase *luks_opts,
> > > > > > > +char **passphrase,
> > > > > > > +Error **errp)
> > > > > > > +{
> > > > > > > +int r = 0;
> > > > > > > +
> > > > > > > +if (!luks_opts->has_key_secret) {
> > > > > > > +r = -EINVAL;
> > > > > > > +error_setg_errno(errp, -r, "missing encrypt.key-secret");
> > > > > > > +return r;
> > > > > > > +}
> > > > > >
> > > > > > Why is key-secret optional?
> > > > >
> > > > > It doesn't look like it is handled correctly here, but we need to
> > > > > be able to run 'qemu-img info ' and get information back
> > > > > on the size of the image, and whether or not it is encrypted,
> > > > > without having to supply a passphrase upfront. So it is right that
> > > > > key-secret be optional, but also we shouldn't return an fatal
> > > > > error like this.
> > > >
> > > > Hi Daniel,
> > > >
> > > > The key-secret lives inside RbdEncryptionOptions (or
> > > > RbdEncryptionCreateOptions) which are already optional:
> > > >
> > > > '*encrypt': 'RbdEncryptionOptions'
> > > >
> > > > '*encrypt' :'RbdEncryptionCreateOptions'
> > > >
> > > > The image is opened as usual and then, if "encrypt" is specified,
> > > > the encryption profile is loaded (or created and laid down).  It does
> > > > not make sense to attempt to load or create the encryption profile
> > > > without the passphrase -- it would always fail.
> > >
> > > Ah, that sounds like it is probably ok then.
> > >
> > >
> > > > > Only if BDRV_O_NO_IO is NOT set, should this error be reported
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > > >  static int64_t qemu_rbd_getlength(BlockDriverState *bs)
> > > > > > >  {
> > > > > > >  BDRVRBDState *s = bs->opaque;
> > > > > > > @@ -1243,6 +1589,22 @@ static QemuOptsList qemu_rbd_create_opts = 
> > > > > > > {
> > > > > > >  .type = QEMU_OPT_STRING,
> > > > > > >  .help = "ID of secret providing the password",
> > > > > > >  },
> > > > > > > +{
> > > > > > > +.name = "encrypt.format",
> > > > > > > +.type = QEMU_OPT_STRING,
> > > > > > > +.help = "Encrypt the image, format choices: 'luks', 
> > > > > > > 'luks2'",
> > > > > >
> > > > > > I think it should be "luks1" and 

Re: [PATCH] block/rbd: Add support for rbd image encryption

[Daniel P . Berrangé]

On Mon, Jun 21, 2021 at 01:23:46PM +0200, Ilya Dryomov wrote:
> On Mon, Jun 21, 2021 at 1:04 PM Daniel P. Berrangé  
> wrote:
> >
> > On Mon, Jun 21, 2021 at 12:59:37PM +0200, Ilya Dryomov wrote:
> > > On Mon, Jun 21, 2021 at 10:32 AM Daniel P. Berrangé  
> > > wrote:
> > > >
> > > > On Sat, Jun 19, 2021 at 09:44:32PM +0200, Ilya Dryomov wrote:
> > > > > On Thu, Jun 17, 2021 at 6:05 PM Or Ozeri  wrote:
> > > > > >
> > > > > > Starting from ceph Pacific, RBD has built-in support for 
> > > > > > image-level encryption.
> > > > > > Currently supported formats are LUKS version 1 and 2.
> > > > > >
> > > > > > There are 2 new relevant librbd APIs for controlling encryption, 
> > > > > > both expect an
> > > > > > open image context:
> > > > > >
> > > > > > rbd_encryption_format: formats an image (i.e. writes the LUKS 
> > > > > > header)
> > > > > > rbd_encryption_load: loads encryptor/decryptor to the image IO stack
> > > > > >
> > > > > > This commit extends the qemu rbd driver API to support the above.
> > > > > >
> > > > > > Signed-off-by: Or Ozeri 
> > > > > > ---
> > > > > >  block/raw-format.c   |   7 +
> > > > > >  block/rbd.c  | 371 
> > > > > > ++-
> > > > > >  qapi/block-core.json | 110 -
> > > > > >  3 files changed, 482 insertions(+), 6 deletions(-)
> > > >
> > > >
> > > > > > diff --git a/block/rbd.c b/block/rbd.c
> > > > > > index f098a89c7b..183b17cd84 100644
> > > > > > --- a/block/rbd.c
> > > > > > +++ b/block/rbd.c
> > > > > > @@ -73,6 +73,18 @@
> > > > > >  #define LIBRBD_USE_IOVEC 0
> > > > > >  #endif
> > > > > >
> > > > > > +#define RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN 8
> > > > > > +
> > > > > > +static const char rbd_luks_header_verification[
> > > > > > +RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN] = {
> > > > > > +'L', 'U', 'K', 'S', 0xBA, 0xBE, 0, 1
> > > > > > +};
> > > > > > +
> > > > > > +static const char rbd_luks2_header_verification[
> > > > > > +RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN] = {
> > > > > > +'L', 'U', 'K', 'S', 0xBA, 0xBE, 0, 2
> > > > > > +};
> > > > > > +
> > > > > >  typedef enum {
> > > > > >  RBD_AIO_READ,
> > > > > >  RBD_AIO_WRITE,
> > > > > > @@ -341,6 +353,206 @@ static void qemu_rbd_memset(RADOSCB *rcb, 
> > > > > > int64_t offs)
> > > > > >  }
> > > > > >  }
> > > > > >
> > > > > > +#ifdef LIBRBD_SUPPORTS_ENCRYPTION
> > > > > > +static int qemu_rbd_convert_luks_options(
> > > > > > +RbdEncryptionOptionsLUKSBase *luks_opts,
> > > > > > +char **passphrase,
> > > > > > +Error **errp)
> > > > > > +{
> > > > > > +int r = 0;
> > > > > > +
> > > > > > +if (!luks_opts->has_key_secret) {
> > > > > > +r = -EINVAL;
> > > > > > +error_setg_errno(errp, -r, "missing encrypt.key-secret");
> > > > > > +return r;
> > > > > > +}
> > > > >
> > > > > Why is key-secret optional?
> > > >
> > > > It doesn't look like it is handled correctly here, but we need to
> > > > be able to run 'qemu-img info ' and get information back
> > > > on the size of the image, and whether or not it is encrypted,
> > > > without having to supply a passphrase upfront. So it is right that
> > > > key-secret be optional, but also we shouldn't return an fatal
> > > > error like this.
> > >
> > > Hi Daniel,
> > >
> > > The key-secret lives inside RbdEncryptionOptions (or
> > > RbdEncryptionCreateOptions) which are already optional:
> > >
> > > '*encrypt': 'RbdEncryptionOptions'
> > >
> > > '*encrypt' :'RbdEncryptionCreateOptions'
> > >
> > > The image is opened as usual and then, if "encrypt" is specified,
> > > the encryption profile is loaded (or created and laid down).  It does
> > > not make sense to attempt to load or create the encryption profile
> > > without the passphrase -- it would always fail.
> >
> > Ah, that sounds like it is probably ok then.
> >
> >
> > > > Only if BDRV_O_NO_IO is NOT set, should this error be reported
> > > >
> > > >
> > > >
> > > >
> > > > > >  static int64_t qemu_rbd_getlength(BlockDriverState *bs)
> > > > > >  {
> > > > > >  BDRVRBDState *s = bs->opaque;
> > > > > > @@ -1243,6 +1589,22 @@ static QemuOptsList qemu_rbd_create_opts = {
> > > > > >  .type = QEMU_OPT_STRING,
> > > > > >  .help = "ID of secret providing the password",
> > > > > >  },
> > > > > > +{
> > > > > > +.name = "encrypt.format",
> > > > > > +.type = QEMU_OPT_STRING,
> > > > > > +.help = "Encrypt the image, format choices: 'luks', 
> > > > > > 'luks2'",
> > > > >
> > > > > I think it should be "luks1" and "luks2" to match rbd/librbd.h and
> > > > > "rbd encryption format" command.
> > > >
> > > > No, it should stay "luks" not "luks1", to match the existing QEMU
> > > > terminology for its LUKS v1 encryption support.
> > >
> > > If you insist on following the QEMU nomenclature here it's fine with
> > > me but I want to point out that 

Re: [PATCH] block/rbd: Add support for rbd image encryption

[Ilya Dryomov]

On Mon, Jun 21, 2021 at 1:04 PM Daniel P. Berrangé  wrote:
>
> On Mon, Jun 21, 2021 at 12:59:37PM +0200, Ilya Dryomov wrote:
> > On Mon, Jun 21, 2021 at 10:32 AM Daniel P. Berrangé  
> > wrote:
> > >
> > > On Sat, Jun 19, 2021 at 09:44:32PM +0200, Ilya Dryomov wrote:
> > > > On Thu, Jun 17, 2021 at 6:05 PM Or Ozeri  wrote:
> > > > >
> > > > > Starting from ceph Pacific, RBD has built-in support for image-level 
> > > > > encryption.
> > > > > Currently supported formats are LUKS version 1 and 2.
> > > > >
> > > > > There are 2 new relevant librbd APIs for controlling encryption, both 
> > > > > expect an
> > > > > open image context:
> > > > >
> > > > > rbd_encryption_format: formats an image (i.e. writes the LUKS header)
> > > > > rbd_encryption_load: loads encryptor/decryptor to the image IO stack
> > > > >
> > > > > This commit extends the qemu rbd driver API to support the above.
> > > > >
> > > > > Signed-off-by: Or Ozeri 
> > > > > ---
> > > > >  block/raw-format.c   |   7 +
> > > > >  block/rbd.c  | 371 
> > > > > ++-
> > > > >  qapi/block-core.json | 110 -
> > > > >  3 files changed, 482 insertions(+), 6 deletions(-)
> > >
> > >
> > > > > diff --git a/block/rbd.c b/block/rbd.c
> > > > > index f098a89c7b..183b17cd84 100644
> > > > > --- a/block/rbd.c
> > > > > +++ b/block/rbd.c
> > > > > @@ -73,6 +73,18 @@
> > > > >  #define LIBRBD_USE_IOVEC 0
> > > > >  #endif
> > > > >
> > > > > +#define RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN 8
> > > > > +
> > > > > +static const char rbd_luks_header_verification[
> > > > > +RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN] = {
> > > > > +'L', 'U', 'K', 'S', 0xBA, 0xBE, 0, 1
> > > > > +};
> > > > > +
> > > > > +static const char rbd_luks2_header_verification[
> > > > > +RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN] = {
> > > > > +'L', 'U', 'K', 'S', 0xBA, 0xBE, 0, 2
> > > > > +};
> > > > > +
> > > > >  typedef enum {
> > > > >  RBD_AIO_READ,
> > > > >  RBD_AIO_WRITE,
> > > > > @@ -341,6 +353,206 @@ static void qemu_rbd_memset(RADOSCB *rcb, 
> > > > > int64_t offs)
> > > > >  }
> > > > >  }
> > > > >
> > > > > +#ifdef LIBRBD_SUPPORTS_ENCRYPTION
> > > > > +static int qemu_rbd_convert_luks_options(
> > > > > +RbdEncryptionOptionsLUKSBase *luks_opts,
> > > > > +char **passphrase,
> > > > > +Error **errp)
> > > > > +{
> > > > > +int r = 0;
> > > > > +
> > > > > +if (!luks_opts->has_key_secret) {
> > > > > +r = -EINVAL;
> > > > > +error_setg_errno(errp, -r, "missing encrypt.key-secret");
> > > > > +return r;
> > > > > +}
> > > >
> > > > Why is key-secret optional?
> > >
> > > It doesn't look like it is handled correctly here, but we need to
> > > be able to run 'qemu-img info ' and get information back
> > > on the size of the image, and whether or not it is encrypted,
> > > without having to supply a passphrase upfront. So it is right that
> > > key-secret be optional, but also we shouldn't return an fatal
> > > error like this.
> >
> > Hi Daniel,
> >
> > The key-secret lives inside RbdEncryptionOptions (or
> > RbdEncryptionCreateOptions) which are already optional:
> >
> > '*encrypt': 'RbdEncryptionOptions'
> >
> > '*encrypt' :'RbdEncryptionCreateOptions'
> >
> > The image is opened as usual and then, if "encrypt" is specified,
> > the encryption profile is loaded (or created and laid down).  It does
> > not make sense to attempt to load or create the encryption profile
> > without the passphrase -- it would always fail.
>
> Ah, that sounds like it is probably ok then.
>
>
> > > Only if BDRV_O_NO_IO is NOT set, should this error be reported
> > >
> > >
> > >
> > >
> > > > >  static int64_t qemu_rbd_getlength(BlockDriverState *bs)
> > > > >  {
> > > > >  BDRVRBDState *s = bs->opaque;
> > > > > @@ -1243,6 +1589,22 @@ static QemuOptsList qemu_rbd_create_opts = {
> > > > >  .type = QEMU_OPT_STRING,
> > > > >  .help = "ID of secret providing the password",
> > > > >  },
> > > > > +{
> > > > > +.name = "encrypt.format",
> > > > > +.type = QEMU_OPT_STRING,
> > > > > +.help = "Encrypt the image, format choices: 'luks', 
> > > > > 'luks2'",
> > > >
> > > > I think it should be "luks1" and "luks2" to match rbd/librbd.h and
> > > > "rbd encryption format" command.
> > >
> > > No, it should stay "luks" not "luks1", to match the existing QEMU
> > > terminology for its LUKS v1 encryption support.
> >
> > If you insist on following the QEMU nomenclature here it's fine with
> > me but I want to point out that encryption-formatted clones won't be
> > interoperable with QEMU LUKS driver or dm-crypt so making the names
> > match QEMU instead of librbd and rbd CLI seems a bit misleading.
>
> In what way is it not interoperable ?
>
> If we don't specify any 'encrypt' option, does the guest see the
> raw LUKS header + payload, or is the 

Re: [PATCH] block/rbd: Add support for rbd image encryption

[Daniel P . Berrangé]

On Mon, Jun 21, 2021 at 12:59:37PM +0200, Ilya Dryomov wrote:
> On Mon, Jun 21, 2021 at 10:32 AM Daniel P. Berrangé  
> wrote:
> >
> > On Sat, Jun 19, 2021 at 09:44:32PM +0200, Ilya Dryomov wrote:
> > > On Thu, Jun 17, 2021 at 6:05 PM Or Ozeri  wrote:
> > > >
> > > > Starting from ceph Pacific, RBD has built-in support for image-level 
> > > > encryption.
> > > > Currently supported formats are LUKS version 1 and 2.
> > > >
> > > > There are 2 new relevant librbd APIs for controlling encryption, both 
> > > > expect an
> > > > open image context:
> > > >
> > > > rbd_encryption_format: formats an image (i.e. writes the LUKS header)
> > > > rbd_encryption_load: loads encryptor/decryptor to the image IO stack
> > > >
> > > > This commit extends the qemu rbd driver API to support the above.
> > > >
> > > > Signed-off-by: Or Ozeri 
> > > > ---
> > > >  block/raw-format.c   |   7 +
> > > >  block/rbd.c  | 371 ++-
> > > >  qapi/block-core.json | 110 -
> > > >  3 files changed, 482 insertions(+), 6 deletions(-)
> >
> >
> > > > diff --git a/block/rbd.c b/block/rbd.c
> > > > index f098a89c7b..183b17cd84 100644
> > > > --- a/block/rbd.c
> > > > +++ b/block/rbd.c
> > > > @@ -73,6 +73,18 @@
> > > >  #define LIBRBD_USE_IOVEC 0
> > > >  #endif
> > > >
> > > > +#define RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN 8
> > > > +
> > > > +static const char rbd_luks_header_verification[
> > > > +RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN] = {
> > > > +'L', 'U', 'K', 'S', 0xBA, 0xBE, 0, 1
> > > > +};
> > > > +
> > > > +static const char rbd_luks2_header_verification[
> > > > +RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN] = {
> > > > +'L', 'U', 'K', 'S', 0xBA, 0xBE, 0, 2
> > > > +};
> > > > +
> > > >  typedef enum {
> > > >  RBD_AIO_READ,
> > > >  RBD_AIO_WRITE,
> > > > @@ -341,6 +353,206 @@ static void qemu_rbd_memset(RADOSCB *rcb, int64_t 
> > > > offs)
> > > >  }
> > > >  }
> > > >
> > > > +#ifdef LIBRBD_SUPPORTS_ENCRYPTION
> > > > +static int qemu_rbd_convert_luks_options(
> > > > +RbdEncryptionOptionsLUKSBase *luks_opts,
> > > > +char **passphrase,
> > > > +Error **errp)
> > > > +{
> > > > +int r = 0;
> > > > +
> > > > +if (!luks_opts->has_key_secret) {
> > > > +r = -EINVAL;
> > > > +error_setg_errno(errp, -r, "missing encrypt.key-secret");
> > > > +return r;
> > > > +}
> > >
> > > Why is key-secret optional?
> >
> > It doesn't look like it is handled correctly here, but we need to
> > be able to run 'qemu-img info ' and get information back
> > on the size of the image, and whether or not it is encrypted,
> > without having to supply a passphrase upfront. So it is right that
> > key-secret be optional, but also we shouldn't return an fatal
> > error like this.
> 
> Hi Daniel,
> 
> The key-secret lives inside RbdEncryptionOptions (or
> RbdEncryptionCreateOptions) which are already optional:
> 
> '*encrypt': 'RbdEncryptionOptions'
> 
> '*encrypt' :'RbdEncryptionCreateOptions'
> 
> The image is opened as usual and then, if "encrypt" is specified,
> the encryption profile is loaded (or created and laid down).  It does
> not make sense to attempt to load or create the encryption profile
> without the passphrase -- it would always fail.

Ah, that sounds like it is probably ok then.


> > Only if BDRV_O_NO_IO is NOT set, should this error be reported
> >
> >
> >
> >
> > > >  static int64_t qemu_rbd_getlength(BlockDriverState *bs)
> > > >  {
> > > >  BDRVRBDState *s = bs->opaque;
> > > > @@ -1243,6 +1589,22 @@ static QemuOptsList qemu_rbd_create_opts = {
> > > >  .type = QEMU_OPT_STRING,
> > > >  .help = "ID of secret providing the password",
> > > >  },
> > > > +{
> > > > +.name = "encrypt.format",
> > > > +.type = QEMU_OPT_STRING,
> > > > +.help = "Encrypt the image, format choices: 'luks', 
> > > > 'luks2'",
> > >
> > > I think it should be "luks1" and "luks2" to match rbd/librbd.h and
> > > "rbd encryption format" command.
> >
> > No, it should stay "luks" not "luks1", to match the existing QEMU
> > terminology for its LUKS v1 encryption support.
> 
> If you insist on following the QEMU nomenclature here it's fine with
> me but I want to point out that encryption-formatted clones won't be
> interoperable with QEMU LUKS driver or dm-crypt so making the names
> match QEMU instead of librbd and rbd CLI seems a bit misleading.

In what way is it not interoperable ?

If we don't specify any 'encrypt' option, does the guest see the
raw LUKS header + payload, or is the header completely hidden
and only ever accessible RBD ?


> > > > +##
> > > > +# @RbdEncryptionCreateOptionsLUKSBase:
> > > > +#
> > > > +# @cipher-alg: The encryption algorithm
> > > > +#
> > > > +# Since: 6.1
> > > > +##
> > > > +{ 'struct': 'RbdEncryptionCreateOptionsLUKSBase',
> > > > +  'base': 

Re: [PATCH] block/rbd: Add support for rbd image encryption

[Ilya Dryomov]

On Mon, Jun 21, 2021 at 10:32 AM Daniel P. Berrangé  wrote:
>
> On Sat, Jun 19, 2021 at 09:44:32PM +0200, Ilya Dryomov wrote:
> > On Thu, Jun 17, 2021 at 6:05 PM Or Ozeri  wrote:
> > >
> > > Starting from ceph Pacific, RBD has built-in support for image-level 
> > > encryption.
> > > Currently supported formats are LUKS version 1 and 2.
> > >
> > > There are 2 new relevant librbd APIs for controlling encryption, both 
> > > expect an
> > > open image context:
> > >
> > > rbd_encryption_format: formats an image (i.e. writes the LUKS header)
> > > rbd_encryption_load: loads encryptor/decryptor to the image IO stack
> > >
> > > This commit extends the qemu rbd driver API to support the above.
> > >
> > > Signed-off-by: Or Ozeri 
> > > ---
> > >  block/raw-format.c   |   7 +
> > >  block/rbd.c  | 371 ++-
> > >  qapi/block-core.json | 110 -
> > >  3 files changed, 482 insertions(+), 6 deletions(-)
>
>
> > > diff --git a/block/rbd.c b/block/rbd.c
> > > index f098a89c7b..183b17cd84 100644
> > > --- a/block/rbd.c
> > > +++ b/block/rbd.c
> > > @@ -73,6 +73,18 @@
> > >  #define LIBRBD_USE_IOVEC 0
> > >  #endif
> > >
> > > +#define RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN 8
> > > +
> > > +static const char rbd_luks_header_verification[
> > > +RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN] = {
> > > +'L', 'U', 'K', 'S', 0xBA, 0xBE, 0, 1
> > > +};
> > > +
> > > +static const char rbd_luks2_header_verification[
> > > +RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN] = {
> > > +'L', 'U', 'K', 'S', 0xBA, 0xBE, 0, 2
> > > +};
> > > +
> > >  typedef enum {
> > >  RBD_AIO_READ,
> > >  RBD_AIO_WRITE,
> > > @@ -341,6 +353,206 @@ static void qemu_rbd_memset(RADOSCB *rcb, int64_t 
> > > offs)
> > >  }
> > >  }
> > >
> > > +#ifdef LIBRBD_SUPPORTS_ENCRYPTION
> > > +static int qemu_rbd_convert_luks_options(
> > > +RbdEncryptionOptionsLUKSBase *luks_opts,
> > > +char **passphrase,
> > > +Error **errp)
> > > +{
> > > +int r = 0;
> > > +
> > > +if (!luks_opts->has_key_secret) {
> > > +r = -EINVAL;
> > > +error_setg_errno(errp, -r, "missing encrypt.key-secret");
> > > +return r;
> > > +}
> >
> > Why is key-secret optional?
>
> It doesn't look like it is handled correctly here, but we need to
> be able to run 'qemu-img info ' and get information back
> on the size of the image, and whether or not it is encrypted,
> without having to supply a passphrase upfront. So it is right that
> key-secret be optional, but also we shouldn't return an fatal
> error like this.

Hi Daniel,

The key-secret lives inside RbdEncryptionOptions (or
RbdEncryptionCreateOptions) which are already optional:

'*encrypt': 'RbdEncryptionOptions'

'*encrypt' :'RbdEncryptionCreateOptions'

The image is opened as usual and then, if "encrypt" is specified,
the encryption profile is loaded (or created and laid down).  It does
not make sense to attempt to load or create the encryption profile
without the passphrase -- it would always fail.

"qemu-img info" can get the size, etc without loading the encryption
profile (i.e. ->bdrv_get_info and ->bdrv_get_specific_info don't need
it).  But note that once the encryption profile is loaded, the size
changes because librbd subtracts the LUKS header overhead.  So

$ qemu-img info --image-opts driver=rbd,pool=foo,image=bar

and

$ qemu-img info --object secret,file=passphrase.txt,id=sec0
--image-opts 
driver=rbd,pool=foo,image=bar,encrypt.format=luks2,encrypt.key-secret=sec0

would give different results.

>
> Only if BDRV_O_NO_IO is NOT set, should this error be reported
>
>
>
>
> > >  static int64_t qemu_rbd_getlength(BlockDriverState *bs)
> > >  {
> > >  BDRVRBDState *s = bs->opaque;
> > > @@ -1243,6 +1589,22 @@ static QemuOptsList qemu_rbd_create_opts = {
> > >  .type = QEMU_OPT_STRING,
> > >  .help = "ID of secret providing the password",
> > >  },
> > > +{
> > > +.name = "encrypt.format",
> > > +.type = QEMU_OPT_STRING,
> > > +.help = "Encrypt the image, format choices: 'luks', 'luks2'",
> >
> > I think it should be "luks1" and "luks2" to match rbd/librbd.h and
> > "rbd encryption format" command.
>
> No, it should stay "luks" not "luks1", to match the existing QEMU
> terminology for its LUKS v1 encryption support.

If you insist on following the QEMU nomenclature here it's fine with
me but I want to point out that encryption-formatted clones won't be
interoperable with QEMU LUKS driver or dm-crypt so making the names
match QEMU instead of librbd and rbd CLI seems a bit misleading.

>
>
> > > @@ -3609,6 +3622,94 @@
> > >  { 'enum': 'RbdAuthMode',
> > >'data': [ 'cephx', 'none' ] }
> > >
> > > +##
> > > +# @RbdImageEncryptionFormat:
> > > +#
> > > +# Since: 6.1
> > > +##
> > > +{ 'enum': 'RbdImageEncryptionFormat',
> > > +  'data': [ 'luks', 'luks2' ] }
> >
> > Ditto 

Re: [PATCH] block/rbd: Add support for rbd image encryption

[Daniel P . Berrangé]

On Sat, Jun 19, 2021 at 09:44:32PM +0200, Ilya Dryomov wrote:
> On Thu, Jun 17, 2021 at 6:05 PM Or Ozeri  wrote:
> >
> > Starting from ceph Pacific, RBD has built-in support for image-level 
> > encryption.
> > Currently supported formats are LUKS version 1 and 2.
> >
> > There are 2 new relevant librbd APIs for controlling encryption, both 
> > expect an
> > open image context:
> >
> > rbd_encryption_format: formats an image (i.e. writes the LUKS header)
> > rbd_encryption_load: loads encryptor/decryptor to the image IO stack
> >
> > This commit extends the qemu rbd driver API to support the above.
> >
> > Signed-off-by: Or Ozeri 
> > ---
> >  block/raw-format.c   |   7 +
> >  block/rbd.c  | 371 ++-
> >  qapi/block-core.json | 110 -
> >  3 files changed, 482 insertions(+), 6 deletions(-)


> > diff --git a/block/rbd.c b/block/rbd.c
> > index f098a89c7b..183b17cd84 100644
> > --- a/block/rbd.c
> > +++ b/block/rbd.c
> > @@ -73,6 +73,18 @@
> >  #define LIBRBD_USE_IOVEC 0
> >  #endif
> >
> > +#define RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN 8
> > +
> > +static const char rbd_luks_header_verification[
> > +RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN] = {
> > +'L', 'U', 'K', 'S', 0xBA, 0xBE, 0, 1
> > +};
> > +
> > +static const char rbd_luks2_header_verification[
> > +RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN] = {
> > +'L', 'U', 'K', 'S', 0xBA, 0xBE, 0, 2
> > +};
> > +
> >  typedef enum {
> >  RBD_AIO_READ,
> >  RBD_AIO_WRITE,
> > @@ -341,6 +353,206 @@ static void qemu_rbd_memset(RADOSCB *rcb, int64_t 
> > offs)
> >  }
> >  }
> >
> > +#ifdef LIBRBD_SUPPORTS_ENCRYPTION
> > +static int qemu_rbd_convert_luks_options(
> > +RbdEncryptionOptionsLUKSBase *luks_opts,
> > +char **passphrase,
> > +Error **errp)
> > +{
> > +int r = 0;
> > +
> > +if (!luks_opts->has_key_secret) {
> > +r = -EINVAL;
> > +error_setg_errno(errp, -r, "missing encrypt.key-secret");
> > +return r;
> > +}
> 
> Why is key-secret optional?

It doesn't look like it is handled correctly here, but we need to
be able to run 'qemu-img info ' and get information back
on the size of the image, and whether or not it is encrypted,
without having to supply a passphrase upfront. So it is right that
key-secret be optional, but also we shouldn't return an fatal
error like this.

Only if BDRV_O_NO_IO is NOT set, should this error be reported




> >  static int64_t qemu_rbd_getlength(BlockDriverState *bs)
> >  {
> >  BDRVRBDState *s = bs->opaque;
> > @@ -1243,6 +1589,22 @@ static QemuOptsList qemu_rbd_create_opts = {
> >  .type = QEMU_OPT_STRING,
> >  .help = "ID of secret providing the password",
> >  },
> > +{
> > +.name = "encrypt.format",
> > +.type = QEMU_OPT_STRING,
> > +.help = "Encrypt the image, format choices: 'luks', 'luks2'",
> 
> I think it should be "luks1" and "luks2" to match rbd/librbd.h and
> "rbd encryption format" command.

No, it should stay "luks" not "luks1", to match the existing QEMU
terminology for its LUKS v1 encryption support.


> > @@ -3609,6 +3622,94 @@
> >  { 'enum': 'RbdAuthMode',
> >'data': [ 'cephx', 'none' ] }
> >
> > +##
> > +# @RbdImageEncryptionFormat:
> > +#
> > +# Since: 6.1
> > +##
> > +{ 'enum': 'RbdImageEncryptionFormat',
> > +  'data': [ 'luks', 'luks2' ] }
> 
> Ditto -- "luks1" and "luks2".

No, the patch is correct as is.

> > +# @RbdEncryptionOptionsLUKSBase:
> > +#
> > +# @key-secret: ID of a QCryptoSecret object providing a passphrase
> > +#  for unlocking the encryption
> > +#
> > +# Since: 6.1
> > +##
> > +{ 'struct': 'RbdEncryptionOptionsLUKSBase',
> > +  'data': { '*key-secret': 'str' }}
> 
> When would we not need a passphrase?  I think it should be required.

When running 'qemu-img info'

> > +##
> > +# @RbdEncryptionCreateOptionsLUKSBase:
> > +#
> > +# @cipher-alg: The encryption algorithm
> > +#
> > +# Since: 6.1
> > +##
> > +{ 'struct': 'RbdEncryptionCreateOptionsLUKSBase',
> > +  'base': 'RbdEncryptionOptionsLUKSBase',
> > +  'data': { '*cipher-alg': 'QCryptoCipherAlgorithm'}}
> 
> Why QCryptoCipherAlgorithm instead of just enumerating the two
> algorithms that librbd supports?  An early failure when parsing
> seems better than failing in qemu_rbd_convert_luks_create_options()
> and having to clean up the newly created image.

We don't want to duplicate algorithm names that already have
a defined enum data type.

> 
> > +
> > +##
> > +# @RbdEncryptionOptionsLUKS:
> > +#
> > +# Since: 6.1
> > +##
> > +{ 'struct': 'RbdEncryptionOptionsLUKS',
> > +  'base': 'RbdEncryptionOptionsLUKSBase',
> > +  'data': {}}
> > +
> > +##
> > +# @RbdEncryptionOptionsLUKS2:
> > +#
> > +# Since: 6.1
> > +##
> > +{ 'struct': 'RbdEncryptionOptionsLUKS2',
> > +  'base': 'RbdEncryptionOptionsLUKSBase',
> > +  'data': {}}
> > +
> > +##
> > +# @RbdEncryptionCreateOptionsLUKS:
> > 

RE: [PATCH] block/rbd: Add support for rbd image encryption

[Or Ozeri]

Thanks for your review!Regarding the last point:I currently don't have any plans for adding new luks2-only options, but they do exist. See in cryptsetup, for example the sector size which is configurable in luks2 only.In the librbd API we followed the same spirit and split luks1 and luks2 option structs.Same goes for passphrase (or "key-secret", as it called in qemu). In cryptsetup this is sort-of optional (there are other ways to authenticate othar than a passphrase, like a keyfile or a token).-Ilya Dryomov  wrote: -To: Or Ozeri From: Ilya Dryomov Date: 06/19/2021 10:44PMCc: qemu-de...@nongnu.org, qemu-block@nongnu.org, kw...@redhat.com, Mykola Golub , Danny Harnik , berra...@redhat.comSubject: [EXTERNAL] Re: [PATCH] block/rbd: Add support for rbd image encryptionOn Thu, Jun 17, 2021 at 6:05 PM Or Ozeri  wrote:>> Starting from ceph Pacific, RBD has built-in support for image-level encryption.> Currently supported formats are LUKS version 1 and 2.>> There are 2 new relevant librbd APIs for controlling encryption, both expect an> open image context:>> rbd_encryption_format: formats an image (i.e. writes the LUKS header)> rbd_encryption_load: loads encryptor/decryptor to the image IO stack>> This commit extends the qemu rbd driver API to support the above.>> Signed-off-by: Or Ozeri > --->  block/raw-format.c   |   7 +>  block/rbd.c          | 371 ++->  qapi/block-core.json | 110 ->  3 files changed, 482 insertions(+), 6 deletions(-)>> diff --git a/block/raw-format.c b/block/raw-format.c> index 7717578ed6..f6e70e2356 100644> --- a/block/raw-format.c> +++ b/block/raw-format.c> @@ -369,6 +369,12 @@ static int raw_get_info(BlockDriverState *bs, BlockDriverInfo *bdi)>      return bdrv_get_info(bs->file->bs, bdi);>  }>> +static ImageInfoSpecific *raw_get_specific_info(BlockDriverState *bs,> +                                                Error **errp)> +{> +    return bdrv_get_specific_info(bs->file->bs, errp);> +}> +>  static void raw_refresh_limits(BlockDriverState *bs, Error **errp)>  {>      if (bs->probed) {> @@ -603,6 +609,7 @@ BlockDriver bdrv_raw = {>      .has_variable_length  = true,>      .bdrv_measure         = _measure,>      .bdrv_get_info        = _get_info,> +    .bdrv_get_specific_info = _get_specific_info,Hi Or,This feels a bit contentious to me.AFAIU ImageInfoSpecific is for format-specfic information.  "raw"is a format and passing the request down the stack this way resultsin a somewhat confusing output such as    $ qemu-img info rbd:foo/bar    image: json:{"driver": "raw", "file": {"pool": "foo", "image":"bar", "driver": "rbd", "namespace": ""}}    file format: raw    ...    Format specific information:       I think this should be broken out into its own patch and get separateacks.>      .bdrv_refresh_limits  = _refresh_limits,>      .bdrv_probe_blocksizes = _probe_blocksizes,>      .bdrv_probe_geometry  = _probe_geometry,> diff --git a/block/rbd.c b/block/rbd.c> index f098a89c7b..183b17cd84 100644> --- a/block/rbd.c> +++ b/block/rbd.c> @@ -73,6 +73,18 @@>  #define LIBRBD_USE_IOVEC 0>  #endif>> +#define RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN 8> +> +static const char rbd_luks_header_verification[> +        RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN] = {> +    'L', 'U', 'K', 'S', 0xBA, 0xBE, 0, 1> +};> +> +static const char rbd_luks2_header_verification[> +        RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN] = {> +    'L', 'U', 'K', 'S', 0xBA, 0xBE, 0, 2> +};> +>  typedef enum {>      RBD_AIO_READ,>      RBD_AIO_WRITE,> @@ -341,6 +353,206 @@ static void qemu_rbd_memset(RADOSCB *rcb, int64_t offs)>      }>  }>> +#ifdef LIBRBD_SUPPORTS_ENCRYPTION> +static int qemu_rbd_convert_luks_options(> +        RbdEncryptionOptionsLUKSBase *luks_opts,> +        char **passphrase,> +        Error **errp)> +{> +    int r = 0;> +> +    if (!luks_opts->has_key_secret) {> +        r = -EINVAL;> +        error_setg_errno(errp, -r, "missing encrypt.key-secret");> +        return r;> +    }Why is key-secret optional?> +> +    *passphrase = qcrypto_secret_lookup_as_utf8(luks_opts->key_secret, errp);> +    if (!*passphrase) {> +        return -EIO;> +    }> +> +    return 0;> +}> +> +static int qemu_rbd_convert_luks_create_options(> +        RbdEncryptionCreateOptionsLUKSBase *luks_opts,> +        rbd_encryption_algorithm_t *alg,> +        char **passphrase,> +        Error **errp)> +{> +    int r = 0;> +> +    r = qemu_rbd_convert_luks_options(> +            qapi_RbdEncryptionCreateOptionsLUKSBase_base(luks_opts),> +            passphrase, errp);> +    if (r < 0) {> +        return r;> +    }> +> +    if (luks_opts->has_cipher_alg) {> +        switch (luks_opts->cipher_alg) {> +            case QCRYPTO_CIPHER_ALG_AES_128: {> +                *alg = RBD_ENCRYPTION_ALGORITHM_AES128;> +                break;> +           

Re: [PATCH] block/rbd: Add support for rbd image encryption

[Ilya Dryomov]

On Sat, Jun 19, 2021 at 9:44 PM Ilya Dryomov  wrote:
>
> On Thu, Jun 17, 2021 at 6:05 PM Or Ozeri  wrote:
> >
> > Starting from ceph Pacific, RBD has built-in support for image-level 
> > encryption.
> > Currently supported formats are LUKS version 1 and 2.
> >
> > There are 2 new relevant librbd APIs for controlling encryption, both 
> > expect an
> > open image context:
> >
> > rbd_encryption_format: formats an image (i.e. writes the LUKS header)
> > rbd_encryption_load: loads encryptor/decryptor to the image IO stack
> >
> > This commit extends the qemu rbd driver API to support the above.
> >
> > Signed-off-by: Or Ozeri 
> > ---
> >  block/raw-format.c   |   7 +
> >  block/rbd.c  | 371 ++-
> >  qapi/block-core.json | 110 -
> >  3 files changed, 482 insertions(+), 6 deletions(-)
> >
> > diff --git a/block/raw-format.c b/block/raw-format.c
> > index 7717578ed6..f6e70e2356 100644
> > --- a/block/raw-format.c
> > +++ b/block/raw-format.c
> > @@ -369,6 +369,12 @@ static int raw_get_info(BlockDriverState *bs, 
> > BlockDriverInfo *bdi)
> >  return bdrv_get_info(bs->file->bs, bdi);
> >  }
> >
> > +static ImageInfoSpecific *raw_get_specific_info(BlockDriverState *bs,
> > +Error **errp)
> > +{
> > +return bdrv_get_specific_info(bs->file->bs, errp);
> > +}
> > +
> >  static void raw_refresh_limits(BlockDriverState *bs, Error **errp)
> >  {
> >  if (bs->probed) {
> > @@ -603,6 +609,7 @@ BlockDriver bdrv_raw = {
> >  .has_variable_length  = true,
> >  .bdrv_measure = _measure,
> >  .bdrv_get_info= _get_info,
> > +.bdrv_get_specific_info = _get_specific_info,
>
> Hi Or,
>
> This feels a bit contentious to me.
>
> AFAIU ImageInfoSpecific is for format-specfic information.  "raw"
> is a format and passing the request down the stack this way results
> in a somewhat confusing output such as
>
> $ qemu-img info rbd:foo/bar
> image: json:{"driver": "raw", "file": {"pool": "foo", "image":
> "bar", "driver": "rbd", "namespace": ""}}
> file format: raw
> ...
> Format specific information:
>
>
> I think this should be broken out into its own patch and get separate
> acks.
>
> >  .bdrv_refresh_limits  = _refresh_limits,
> >  .bdrv_probe_blocksizes = _probe_blocksizes,
> >  .bdrv_probe_geometry  = _probe_geometry,
> > diff --git a/block/rbd.c b/block/rbd.c
> > index f098a89c7b..183b17cd84 100644
> > --- a/block/rbd.c
> > +++ b/block/rbd.c
> > @@ -73,6 +73,18 @@
> >  #define LIBRBD_USE_IOVEC 0
> >  #endif
> >
> > +#define RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN 8
> > +
> > +static const char rbd_luks_header_verification[
> > +RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN] = {
> > +'L', 'U', 'K', 'S', 0xBA, 0xBE, 0, 1
> > +};
> > +
> > +static const char rbd_luks2_header_verification[
> > +RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN] = {
> > +'L', 'U', 'K', 'S', 0xBA, 0xBE, 0, 2
> > +};
> > +
> >  typedef enum {
> >  RBD_AIO_READ,
> >  RBD_AIO_WRITE,
> > @@ -341,6 +353,206 @@ static void qemu_rbd_memset(RADOSCB *rcb, int64_t 
> > offs)
> >  }
> >  }
> >
> > +#ifdef LIBRBD_SUPPORTS_ENCRYPTION
> > +static int qemu_rbd_convert_luks_options(
> > +RbdEncryptionOptionsLUKSBase *luks_opts,
> > +char **passphrase,
> > +Error **errp)
> > +{
> > +int r = 0;
> > +
> > +if (!luks_opts->has_key_secret) {
> > +r = -EINVAL;
> > +error_setg_errno(errp, -r, "missing encrypt.key-secret");
> > +return r;
> > +}
>
> Why is key-secret optional?
>
> > +
> > +*passphrase = qcrypto_secret_lookup_as_utf8(luks_opts->key_secret, 
> > errp);
> > +if (!*passphrase) {
> > +return -EIO;
> > +}
> > +
> > +return 0;
> > +}
> > +
> > +static int qemu_rbd_convert_luks_create_options(
> > +RbdEncryptionCreateOptionsLUKSBase *luks_opts,
> > +rbd_encryption_algorithm_t *alg,
> > +char **passphrase,
> > +Error **errp)
> > +{
> > +int r = 0;
> > +
> > +r = qemu_rbd_convert_luks_options(
> > +qapi_RbdEncryptionCreateOptionsLUKSBase_base(luks_opts),
> > +passphrase, errp);
> > +if (r < 0) {
> > +return r;
> > +}
> > +
> > +if (luks_opts->has_cipher_alg) {
> > +switch (luks_opts->cipher_alg) {
> > +case QCRYPTO_CIPHER_ALG_AES_128: {
> > +*alg = RBD_ENCRYPTION_ALGORITHM_AES128;
> > +break;
> > +}
> > +case QCRYPTO_CIPHER_ALG_AES_256: {
> > +*alg = RBD_ENCRYPTION_ALGORITHM_AES256;
> > +break;
> > +}
> > +default: {
> > +r = -ENOTSUP;
> > +error_setg_errno(errp, -r, "unknown encryption algorithm: 
> > %u",
> > + luks_opts->cipher_alg);
> > +return r;
> > +}
> > + 

Re: [PATCH] block/rbd: Add support for rbd image encryption

[Ilya Dryomov]

On Thu, Jun 17, 2021 at 6:05 PM Or Ozeri  wrote:
>
> Starting from ceph Pacific, RBD has built-in support for image-level 
> encryption.
> Currently supported formats are LUKS version 1 and 2.
>
> There are 2 new relevant librbd APIs for controlling encryption, both expect 
> an
> open image context:
>
> rbd_encryption_format: formats an image (i.e. writes the LUKS header)
> rbd_encryption_load: loads encryptor/decryptor to the image IO stack
>
> This commit extends the qemu rbd driver API to support the above.
>
> Signed-off-by: Or Ozeri 
> ---
>  block/raw-format.c   |   7 +
>  block/rbd.c  | 371 ++-
>  qapi/block-core.json | 110 -
>  3 files changed, 482 insertions(+), 6 deletions(-)
>
> diff --git a/block/raw-format.c b/block/raw-format.c
> index 7717578ed6..f6e70e2356 100644
> --- a/block/raw-format.c
> +++ b/block/raw-format.c
> @@ -369,6 +369,12 @@ static int raw_get_info(BlockDriverState *bs, 
> BlockDriverInfo *bdi)
>  return bdrv_get_info(bs->file->bs, bdi);
>  }
>
> +static ImageInfoSpecific *raw_get_specific_info(BlockDriverState *bs,
> +Error **errp)
> +{
> +return bdrv_get_specific_info(bs->file->bs, errp);
> +}
> +
>  static void raw_refresh_limits(BlockDriverState *bs, Error **errp)
>  {
>  if (bs->probed) {
> @@ -603,6 +609,7 @@ BlockDriver bdrv_raw = {
>  .has_variable_length  = true,
>  .bdrv_measure = _measure,
>  .bdrv_get_info= _get_info,
> +.bdrv_get_specific_info = _get_specific_info,

Hi Or,

This feels a bit contentious to me.

AFAIU ImageInfoSpecific is for format-specfic information.  "raw"
is a format and passing the request down the stack this way results
in a somewhat confusing output such as

$ qemu-img info rbd:foo/bar
image: json:{"driver": "raw", "file": {"pool": "foo", "image":
"bar", "driver": "rbd", "namespace": ""}}
file format: raw
...
Format specific information:
   

I think this should be broken out into its own patch and get separate
acks.

>  .bdrv_refresh_limits  = _refresh_limits,
>  .bdrv_probe_blocksizes = _probe_blocksizes,
>  .bdrv_probe_geometry  = _probe_geometry,
> diff --git a/block/rbd.c b/block/rbd.c
> index f098a89c7b..183b17cd84 100644
> --- a/block/rbd.c
> +++ b/block/rbd.c
> @@ -73,6 +73,18 @@
>  #define LIBRBD_USE_IOVEC 0
>  #endif
>
> +#define RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN 8
> +
> +static const char rbd_luks_header_verification[
> +RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN] = {
> +'L', 'U', 'K', 'S', 0xBA, 0xBE, 0, 1
> +};
> +
> +static const char rbd_luks2_header_verification[
> +RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN] = {
> +'L', 'U', 'K', 'S', 0xBA, 0xBE, 0, 2
> +};
> +
>  typedef enum {
>  RBD_AIO_READ,
>  RBD_AIO_WRITE,
> @@ -341,6 +353,206 @@ static void qemu_rbd_memset(RADOSCB *rcb, int64_t offs)
>  }
>  }
>
> +#ifdef LIBRBD_SUPPORTS_ENCRYPTION
> +static int qemu_rbd_convert_luks_options(
> +RbdEncryptionOptionsLUKSBase *luks_opts,
> +char **passphrase,
> +Error **errp)
> +{
> +int r = 0;
> +
> +if (!luks_opts->has_key_secret) {
> +r = -EINVAL;
> +error_setg_errno(errp, -r, "missing encrypt.key-secret");
> +return r;
> +}

Why is key-secret optional?

> +
> +*passphrase = qcrypto_secret_lookup_as_utf8(luks_opts->key_secret, errp);
> +if (!*passphrase) {
> +return -EIO;
> +}
> +
> +return 0;
> +}
> +
> +static int qemu_rbd_convert_luks_create_options(
> +RbdEncryptionCreateOptionsLUKSBase *luks_opts,
> +rbd_encryption_algorithm_t *alg,
> +char **passphrase,
> +Error **errp)
> +{
> +int r = 0;
> +
> +r = qemu_rbd_convert_luks_options(
> +qapi_RbdEncryptionCreateOptionsLUKSBase_base(luks_opts),
> +passphrase, errp);
> +if (r < 0) {
> +return r;
> +}
> +
> +if (luks_opts->has_cipher_alg) {
> +switch (luks_opts->cipher_alg) {
> +case QCRYPTO_CIPHER_ALG_AES_128: {
> +*alg = RBD_ENCRYPTION_ALGORITHM_AES128;
> +break;
> +}
> +case QCRYPTO_CIPHER_ALG_AES_256: {
> +*alg = RBD_ENCRYPTION_ALGORITHM_AES256;
> +break;
> +}
> +default: {
> +r = -ENOTSUP;
> +error_setg_errno(errp, -r, "unknown encryption algorithm: 
> %u",
> + luks_opts->cipher_alg);
> +return r;
> +}
> +}
> +} else {
> +/* default alg */
> +*alg = RBD_ENCRYPTION_ALGORITHM_AES256;
> +}
> +
> +return 0;
> +}
> +
> +static int qemu_rbd_encryption_format(rbd_image_t image,
> +  RbdEncryptionCreateOptions *encrypt,
> +  Error **errp)
> +{
> +int r = 0;
> 

[PATCH] block/rbd: Add support for rbd image encryption

[Or Ozeri]

Starting from ceph Pacific, RBD has built-in support for image-level encryption.
Currently supported formats are LUKS version 1 and 2.

There are 2 new relevant librbd APIs for controlling encryption, both expect an
open image context:

rbd_encryption_format: formats an image (i.e. writes the LUKS header)
rbd_encryption_load: loads encryptor/decryptor to the image IO stack

This commit extends the qemu rbd driver API to support the above.

Signed-off-by: Or Ozeri 
---
 block/raw-format.c   |   7 +
 block/rbd.c  | 371 ++-
 qapi/block-core.json | 110 -
 3 files changed, 482 insertions(+), 6 deletions(-)

diff --git a/block/raw-format.c b/block/raw-format.c
index 7717578ed6..f6e70e2356 100644
--- a/block/raw-format.c
+++ b/block/raw-format.c
@@ -369,6 +369,12 @@ static int raw_get_info(BlockDriverState *bs, 
BlockDriverInfo *bdi)
 return bdrv_get_info(bs->file->bs, bdi);
 }
 
+static ImageInfoSpecific *raw_get_specific_info(BlockDriverState *bs,
+Error **errp)
+{
+return bdrv_get_specific_info(bs->file->bs, errp);
+}
+
 static void raw_refresh_limits(BlockDriverState *bs, Error **errp)
 {
 if (bs->probed) {
@@ -603,6 +609,7 @@ BlockDriver bdrv_raw = {
 .has_variable_length  = true,
 .bdrv_measure = _measure,
 .bdrv_get_info= _get_info,
+.bdrv_get_specific_info = _get_specific_info,
 .bdrv_refresh_limits  = _refresh_limits,
 .bdrv_probe_blocksizes = _probe_blocksizes,
 .bdrv_probe_geometry  = _probe_geometry,
diff --git a/block/rbd.c b/block/rbd.c
index f098a89c7b..183b17cd84 100644
--- a/block/rbd.c
+++ b/block/rbd.c
@@ -73,6 +73,18 @@
 #define LIBRBD_USE_IOVEC 0
 #endif
 
+#define RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN 8
+
+static const char rbd_luks_header_verification[
+RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN] = {
+'L', 'U', 'K', 'S', 0xBA, 0xBE, 0, 1
+};
+
+static const char rbd_luks2_header_verification[
+RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN] = {
+'L', 'U', 'K', 'S', 0xBA, 0xBE, 0, 2
+};
+
 typedef enum {
 RBD_AIO_READ,
 RBD_AIO_WRITE,
@@ -341,6 +353,206 @@ static void qemu_rbd_memset(RADOSCB *rcb, int64_t offs)
 }
 }
 
+#ifdef LIBRBD_SUPPORTS_ENCRYPTION
+static int qemu_rbd_convert_luks_options(
+RbdEncryptionOptionsLUKSBase *luks_opts,
+char **passphrase,
+Error **errp)
+{
+int r = 0;
+
+if (!luks_opts->has_key_secret) {
+r = -EINVAL;
+error_setg_errno(errp, -r, "missing encrypt.key-secret");
+return r;
+}
+
+*passphrase = qcrypto_secret_lookup_as_utf8(luks_opts->key_secret, errp);
+if (!*passphrase) {
+return -EIO;
+}
+
+return 0;
+}
+
+static int qemu_rbd_convert_luks_create_options(
+RbdEncryptionCreateOptionsLUKSBase *luks_opts,
+rbd_encryption_algorithm_t *alg,
+char **passphrase,
+Error **errp)
+{
+int r = 0;
+
+r = qemu_rbd_convert_luks_options(
+qapi_RbdEncryptionCreateOptionsLUKSBase_base(luks_opts),
+passphrase, errp);
+if (r < 0) {
+return r;
+}
+
+if (luks_opts->has_cipher_alg) {
+switch (luks_opts->cipher_alg) {
+case QCRYPTO_CIPHER_ALG_AES_128: {
+*alg = RBD_ENCRYPTION_ALGORITHM_AES128;
+break;
+}
+case QCRYPTO_CIPHER_ALG_AES_256: {
+*alg = RBD_ENCRYPTION_ALGORITHM_AES256;
+break;
+}
+default: {
+r = -ENOTSUP;
+error_setg_errno(errp, -r, "unknown encryption algorithm: %u",
+ luks_opts->cipher_alg);
+return r;
+}
+}
+} else {
+/* default alg */
+*alg = RBD_ENCRYPTION_ALGORITHM_AES256;
+}
+
+return 0;
+}
+
+static int qemu_rbd_encryption_format(rbd_image_t image,
+  RbdEncryptionCreateOptions *encrypt,
+  Error **errp)
+{
+int r = 0;
+g_autofree char *passphrase = NULL;
+g_autofree rbd_encryption_options_t opts = NULL;
+rbd_encryption_format_t format;
+rbd_image_info_t info;
+size_t opts_size;
+uint64_t raw_size;
+
+r = rbd_stat(image, , sizeof(info));
+if (r < 0) {
+error_setg_errno(errp, -r, "cannot get raw image size");
+return r;
+}
+raw_size = info.size;
+
+switch (encrypt->format) {
+case RBD_IMAGE_ENCRYPTION_FORMAT_LUKS: {
+rbd_encryption_luks1_format_options_t *luks_opts =
+g_new0(rbd_encryption_luks1_format_options_t, 1);
+format = RBD_ENCRYPTION_FORMAT_LUKS1;
+opts = luks_opts;
+opts_size = sizeof(rbd_encryption_luks1_format_options_t);
+r = qemu_rbd_convert_luks_create_options(
+

RE: [PATCH] block/rbd: Add support for rbd image encryption

[Or Ozeri]

Thanks Daniel!I prepared a modified patch addressing all of your suggestions (including resizing after formatting to increase the image size).The only thing I'm not sure about is your last point regarding reporting image is encrypted.When I followed the flow of "qemu-img info" on an "rbd:pool/image" I saw that this information is extracted from the root BlockDriverState.In our case, the root BlockDriverState comes from:BlockDriver bdrv_raw = {    .format_name          = "raw",...The RBD driver is a child of this root raw driver.Given this situation, it is not clear to me how can I set:bs->drv->format_name="luks2", bs->encrypted=trueOn the root BlockDriverState.Any advice?-"Daniel P. Berrangé"  wrote: -To: Or Ozeri From: "Daniel P. Berrangé" Date: 05/04/2021 05:47PMCc: qemu-de...@nongnu.org, kw...@redhat.com, to.my.troc...@gmail.com, qemu-block@nongnu.org, dan...@il.ibm.comSubject: [EXTERNAL] Re: [PATCH] block/rbd: Add support for rbd image encryptionOn Sun, May 02, 2021 at 10:36:17AM +0300, Or Ozeri wrote:> Starting from ceph Pacific, RBD has built-in support for image-level encryption.> Currently supported formats are LUKS version 1 and 2.> > There are 2 new relevant librbd APIs for controlling encryption, both expect an> open image context:> > rbd_encryption_format: formats an image (i.e. writes the LUKS header)> rbd_encryption_load: loads encryptor/decryptor to the image IO stack> > This commit extends the qemu rbd driver API to support the above.> > Signed-off-by: Or Ozeri > --->  block/rbd.c          | 230 ++->  qapi/block-core.json |  61 >  2 files changed, 287 insertions(+), 4 deletions(-)> > diff --git a/block/rbd.c b/block/rbd.c> index f098a89c7b..1239e97889 100644> --- a/block/rbd.c> +++ b/block/rbd.c> @@ -108,6 +108,13 @@ typedef struct BDRVRBDState {>      uint64_t image_size;>  } BDRVRBDState;>  > +#ifdef LIBRBD_SUPPORTS_ENCRYPTION> +typedef int (*RbdEncryptionFunc)(rbd_image_t image,> +                                 rbd_encryption_format_t format,> +                                 rbd_encryption_options_t opts,> +                                 size_t opts_size);> +#endif> +>  static int qemu_rbd_connect(rados_t *cluster, rados_ioctx_t *io_ctx,>                              BlockdevOptionsRbd *opts, bool cache,>                              const char *keypairs, const char *secretid,> @@ -341,6 +348,115 @@ static void qemu_rbd_memset(RADOSCB *rcb, int64_t offs)>      }>  }>  > +#ifdef LIBRBD_SUPPORTS_ENCRYPTION> +static int qemu_rbd_convert_luks_options(> +        RbdEncryptionOptionsLUKSBase *luks_opts,> +        rbd_encryption_algorithm_t *alg,> +        char** passphrase,> +        Error **errp)> +{> +    int r = 0;> +> +    if (!luks_opts->has_passphrase_secret) {> +        r = -EINVAL;> +        error_setg_errno(errp, -r, "missing encrypt.passphrase-secret");> +        return r;> +    }> +> +    *passphrase = qcrypto_secret_lookup_as_utf8(luks_opts->passphrase_secret,> +                                                errp);> +    if (!*passphrase) {> +        return -EIO;> +    }> +> +    if (luks_opts->has_cipher_alg) {> +        switch (luks_opts->cipher_alg) {> +            case RBD_ENCRYPTION_ALGORITHM_AES_128: {> +                *alg = RBD_ENCRYPTION_ALGORITHM_AES128;> +                break;> +            }> +            case RBD_ENCRYPTION_ALGORITHM_AES_256: {> +                *alg = RBD_ENCRYPTION_ALGORITHM_AES256;> +                break;> +            }> +            default: {> +                r = -ENOTSUP;> +                error_setg_errno(errp, -r, "unknown encryption algorithm: %u",> +                                 luks_opts->cipher_alg);> +                return r;> +            }> +        }> +    } else {> +        /* default alg */> +        *alg = RBD_ENCRYPTION_ALGORITHM_AES256;> +    }> +> +    return 0;> +}> +> +static int qemu_rbd_apply_encryption_function(rbd_image_t image,> +                                              RbdEncryptionSpec* spec,> +                                              RbdEncryptionFunc func,> +                                              Error **errp)> +{> +    int r = 0;> +    g_autofree char *passphrase = NULL;> +    g_autofree rbd_encryption_options_t opts = NULL;> +    rbd_encryption_format_t format;> +    size_t opts_size;> +> +    switch (spec->format) {> +        case RBD_IMAGE_ENCRYPTION_FORMAT_LUKS1: {> +            rbd_encryption_luks1_format_options_t *luks1_opts => +                    g_new0(rbd_encryption_luks1_format_options_t, 1);> +            format = RBD_ENCRYPTION_FORMAT_LUKS1;> +            opts = luks1_opts;> +            opts_size = sizeof(rbd_encryption_luks1_format_options_t);> +            r = qemu_rbd_convert_luks_options(> +                    qapi_RbdEncryptionOptionsLUKS1_base(>u.luks1),> +                    _opts->alg, , errp);> +            if (passphrase) {> +  

[PATCH] block/rbd: Add support for rbd image encryption

[Or Ozeri]

Starting from ceph Pacific, RBD has built-in support for image-level encryption.
Currently supported formats are LUKS version 1 and 2.

There are 2 new relevant librbd APIs for controlling encryption, both expect an
open image context:

rbd_encryption_format: formats an image (i.e. writes the LUKS header)
rbd_encryption_load: loads encryptor/decryptor to the image IO stack

This commit extends the qemu rbd driver API to support the above.

Signed-off-by: Or Ozeri 
---
 block/raw-format.c   |   7 +
 block/rbd.c  | 371 ++-
 qapi/block-core.json | 110 -
 3 files changed, 482 insertions(+), 6 deletions(-)

diff --git a/block/raw-format.c b/block/raw-format.c
index 7717578ed6..f6e70e2356 100644
--- a/block/raw-format.c
+++ b/block/raw-format.c
@@ -369,6 +369,12 @@ static int raw_get_info(BlockDriverState *bs, 
BlockDriverInfo *bdi)
 return bdrv_get_info(bs->file->bs, bdi);
 }
 
+static ImageInfoSpecific *raw_get_specific_info(BlockDriverState *bs,
+Error **errp)
+{
+return bdrv_get_specific_info(bs->file->bs, errp);
+}
+
 static void raw_refresh_limits(BlockDriverState *bs, Error **errp)
 {
 if (bs->probed) {
@@ -603,6 +609,7 @@ BlockDriver bdrv_raw = {
 .has_variable_length  = true,
 .bdrv_measure = _measure,
 .bdrv_get_info= _get_info,
+.bdrv_get_specific_info = _get_specific_info,
 .bdrv_refresh_limits  = _refresh_limits,
 .bdrv_probe_blocksizes = _probe_blocksizes,
 .bdrv_probe_geometry  = _probe_geometry,
diff --git a/block/rbd.c b/block/rbd.c
index f098a89c7b..183b17cd84 100644
--- a/block/rbd.c
+++ b/block/rbd.c
@@ -73,6 +73,18 @@
 #define LIBRBD_USE_IOVEC 0
 #endif
 
+#define RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN 8
+
+static const char rbd_luks_header_verification[
+RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN] = {
+'L', 'U', 'K', 'S', 0xBA, 0xBE, 0, 1
+};
+
+static const char rbd_luks2_header_verification[
+RBD_ENCRYPTION_LUKS_HEADER_VERIFICATION_LEN] = {
+'L', 'U', 'K', 'S', 0xBA, 0xBE, 0, 2
+};
+
 typedef enum {
 RBD_AIO_READ,
 RBD_AIO_WRITE,
@@ -341,6 +353,206 @@ static void qemu_rbd_memset(RADOSCB *rcb, int64_t offs)
 }
 }
 
+#ifdef LIBRBD_SUPPORTS_ENCRYPTION
+static int qemu_rbd_convert_luks_options(
+RbdEncryptionOptionsLUKSBase *luks_opts,
+char **passphrase,
+Error **errp)
+{
+int r = 0;
+
+if (!luks_opts->has_key_secret) {
+r = -EINVAL;
+error_setg_errno(errp, -r, "missing encrypt.key-secret");
+return r;
+}
+
+*passphrase = qcrypto_secret_lookup_as_utf8(luks_opts->key_secret, errp);
+if (!*passphrase) {
+return -EIO;
+}
+
+return 0;
+}
+
+static int qemu_rbd_convert_luks_create_options(
+RbdEncryptionCreateOptionsLUKSBase *luks_opts,
+rbd_encryption_algorithm_t *alg,
+char **passphrase,
+Error **errp)
+{
+int r = 0;
+
+r = qemu_rbd_convert_luks_options(
+qapi_RbdEncryptionCreateOptionsLUKSBase_base(luks_opts),
+passphrase, errp);
+if (r < 0) {
+return r;
+}
+
+if (luks_opts->has_cipher_alg) {
+switch (luks_opts->cipher_alg) {
+case QCRYPTO_CIPHER_ALG_AES_128: {
+*alg = RBD_ENCRYPTION_ALGORITHM_AES128;
+break;
+}
+case QCRYPTO_CIPHER_ALG_AES_256: {
+*alg = RBD_ENCRYPTION_ALGORITHM_AES256;
+break;
+}
+default: {
+r = -ENOTSUP;
+error_setg_errno(errp, -r, "unknown encryption algorithm: %u",
+ luks_opts->cipher_alg);
+return r;
+}
+}
+} else {
+/* default alg */
+*alg = RBD_ENCRYPTION_ALGORITHM_AES256;
+}
+
+return 0;
+}
+
+static int qemu_rbd_encryption_format(rbd_image_t image,
+  RbdEncryptionCreateOptions *encrypt,
+  Error **errp)
+{
+int r = 0;
+g_autofree char *passphrase = NULL;
+g_autofree rbd_encryption_options_t opts = NULL;
+rbd_encryption_format_t format;
+rbd_image_info_t info;
+size_t opts_size;
+uint64_t raw_size;
+
+r = rbd_stat(image, , sizeof(info));
+if (r < 0) {
+error_setg_errno(errp, -r, "cannot get raw image size");
+return r;
+}
+raw_size = info.size;
+
+switch (encrypt->format) {
+case RBD_IMAGE_ENCRYPTION_FORMAT_LUKS: {
+rbd_encryption_luks1_format_options_t *luks_opts =
+g_new0(rbd_encryption_luks1_format_options_t, 1);
+format = RBD_ENCRYPTION_FORMAT_LUKS1;
+opts = luks_opts;
+opts_size = sizeof(rbd_encryption_luks1_format_options_t);
+r = qemu_rbd_convert_luks_create_options(
+

Re: [PATCH] block/rbd: Add support for rbd image encryption

[Daniel P . Berrangé]

On Wed, May 05, 2021 at 03:32:09PM +, Or Ozeri wrote:
>Thanks Daniel!
>I prepared a modified patch addressing all of your suggestions (including
>resizing after formatting to increase the image size).
>The only thing I'm not sure about is your last point regarding reporting
>image is encrypted.
>When I followed the flow of "qemu-img info" on an "rbd:pool/image" I saw
>that this information is extracted from the root BlockDriverState.
>In our case, the root BlockDriverState comes from:
>BlockDriver bdrv_raw = {
>    .format_name          = "raw",
>...
>The RBD driver is a child of this root raw driver.
>Given this situation, it is not clear to me how can I set:
>bs->drv->format_name="luks2", bs->encrypted=true
>On the root BlockDriverState.
>Any advice?


In the QAPI schema  block-core.json there is a "ImageInfoSpecific"
union that gives format specific information. IIUC, you should
need to implement that for RBD to provide extra info. I can't
remember whether qemu-img info will automagically print this or
if extra printing code is needed too.

>-"Daniel P. Berrangé" <[1]berra...@redhat.com> wrote: -
>To: Or Ozeri <[2]o...@il.ibm.com>
>From: "Daniel P. Berrangé" <[3]berra...@redhat.com>
>Date: 05/04/2021 05:47PM
>Cc: [4]qemu-de...@nongnu.org, [5]kw...@redhat.com,
>[6]to.my.troc...@gmail.com, [7]qemu-block@nongnu.org, [8]dan...@il.ibm.com
>Subject: [EXTERNAL] Re: [PATCH] block/rbd: Add support for rbd image
>encryption
> 
>On Sun, May 02, 2021 at 10:36:17AM +0300, Or Ozeri wrote:
>> Starting from ceph Pacific, RBD has built-in support for image-level
>encryption.
>> Currently supported formats are LUKS version 1 and 2.
>>
>> There are 2 new relevant librbd APIs for controlling encryption, both
>expect an
>> open image context:
>>
>> rbd_encryption_format: formats an image (i.e. writes the LUKS header)
>> rbd_encryption_load: loads encryptor/decryptor to the image IO stack
> 
>>
>> This commit extends the qemu rbd driver API to support the above.
>>
>> Signed-off-by: Or Ozeri <[9]o...@il.ibm.com>
>> ---
>>  block/rbd.c          | 230 ++-
>>  qapi/block-core.json |  61 
>>  2 files changed, 287 insertions(+), 4 deletions(-)
>>
>> diff --git a/block/rbd.c b/block/rbd.c
>> index f098a89c7b..1239e97889 100644
>> --- a/block/rbd.c
>> +++ b/block/rbd.c
>> @@ -108,6 +108,13 @@ typedef struct BDRVRBDState {
>>      uint64_t image_size;
>>  } BDRVRBDState;
>>  
>> +#ifdef LIBRBD_SUPPORTS_ENCRYPTION
>> +typedef int (*RbdEncryptionFunc)(rbd_image_t image,
>> +                                 rbd_encryption_format_t format,
>> +                                 rbd_encryption_options_t opts,
>> +                                 size_t opts_size);
>> +#endif
>> +
>>  static int qemu_rbd_connect(rados_t *cluster, rados_ioctx_t *io_ctx,
>>                              BlockdevOptionsRbd *opts, bool cache,
>>                              const char *keypairs, const char *secretid,
>> @@ -341,6 +348,115 @@ static void qemu_rbd_memset(RADOSCB *rcb, int64_t
>offs)
>>      }
>>  }
>>  
>> +#ifdef LIBRBD_SUPPORTS_ENCRYPTION
>> +static int qemu_rbd_convert_luks_options(
>> +        RbdEncryptionOptionsLUKSBase *luks_opts,
>> +        rbd_encryption_algorithm_t *alg,
>> +        char** passphrase,
>> +        Error **errp)
>> +{
>> +    int r = 0;
>> +
>> +    if (!luks_opts->has_passphrase_secret) {
>> +        r = -EINVAL;
>> +        error_setg_errno(errp, -r, "missing
>encrypt.passphrase-secret");
>> +        return r;
>> +    }
>> +
>> +    *passphrase =
>qcrypto_secret_lookup_as_utf8(luks_opts->passphrase_secret,
>> +                                                errp);
>> +    if (!*passphrase) {
>> +        return -EIO;
>> +    }
>> +
>> +    if (luks_opts->has_cipher_alg) {
>> +        switch (luks_opts->cipher_alg) {
>> +            case RBD_ENCRYPTION_ALGORITHM_AES_128: {
>> +                *alg = RBD_ENCRYPTION_ALGORITHM_AES128;
>> +                break

Re: [PATCH] block/rbd: Add support for rbd image encryption

[Daniel P . Berrangé]

On Sun, May 02, 2021 at 10:36:17AM +0300, Or Ozeri wrote:
> Starting from ceph Pacific, RBD has built-in support for image-level 
> encryption.
> Currently supported formats are LUKS version 1 and 2.
> 
> There are 2 new relevant librbd APIs for controlling encryption, both expect 
> an
> open image context:
> 
> rbd_encryption_format: formats an image (i.e. writes the LUKS header)
> rbd_encryption_load: loads encryptor/decryptor to the image IO stack

> 
> This commit extends the qemu rbd driver API to support the above.
> 
> Signed-off-by: Or Ozeri 
> ---
>  block/rbd.c  | 230 ++-
>  qapi/block-core.json |  61 
>  2 files changed, 287 insertions(+), 4 deletions(-)
> 
> diff --git a/block/rbd.c b/block/rbd.c
> index f098a89c7b..1239e97889 100644
> --- a/block/rbd.c
> +++ b/block/rbd.c
> @@ -108,6 +108,13 @@ typedef struct BDRVRBDState {
>  uint64_t image_size;
>  } BDRVRBDState;
>  
> +#ifdef LIBRBD_SUPPORTS_ENCRYPTION
> +typedef int (*RbdEncryptionFunc)(rbd_image_t image,
> + rbd_encryption_format_t format,
> + rbd_encryption_options_t opts,
> + size_t opts_size);
> +#endif
> +
>  static int qemu_rbd_connect(rados_t *cluster, rados_ioctx_t *io_ctx,
>  BlockdevOptionsRbd *opts, bool cache,
>  const char *keypairs, const char *secretid,
> @@ -341,6 +348,115 @@ static void qemu_rbd_memset(RADOSCB *rcb, int64_t offs)
>  }
>  }
>  
> +#ifdef LIBRBD_SUPPORTS_ENCRYPTION
> +static int qemu_rbd_convert_luks_options(
> +RbdEncryptionOptionsLUKSBase *luks_opts,
> +rbd_encryption_algorithm_t *alg,
> +char** passphrase,
> +Error **errp)
> +{
> +int r = 0;
> +
> +if (!luks_opts->has_passphrase_secret) {
> +r = -EINVAL;
> +error_setg_errno(errp, -r, "missing encrypt.passphrase-secret");
> +return r;
> +}
> +
> +*passphrase = qcrypto_secret_lookup_as_utf8(luks_opts->passphrase_secret,
> +errp);
> +if (!*passphrase) {
> +return -EIO;
> +}
> +
> +if (luks_opts->has_cipher_alg) {
> +switch (luks_opts->cipher_alg) {
> +case RBD_ENCRYPTION_ALGORITHM_AES_128: {
> +*alg = RBD_ENCRYPTION_ALGORITHM_AES128;
> +break;
> +}
> +case RBD_ENCRYPTION_ALGORITHM_AES_256: {
> +*alg = RBD_ENCRYPTION_ALGORITHM_AES256;
> +break;
> +}
> +default: {
> +r = -ENOTSUP;
> +error_setg_errno(errp, -r, "unknown encryption algorithm: 
> %u",
> + luks_opts->cipher_alg);
> +return r;
> +}
> +}
> +} else {
> +/* default alg */
> +*alg = RBD_ENCRYPTION_ALGORITHM_AES256;
> +}
> +
> +return 0;
> +}
> +
> +static int qemu_rbd_apply_encryption_function(rbd_image_t image,
> +  RbdEncryptionSpec* spec,
> +  RbdEncryptionFunc func,
> +  Error **errp)
> +{
> +int r = 0;
> +g_autofree char *passphrase = NULL;
> +g_autofree rbd_encryption_options_t opts = NULL;
> +rbd_encryption_format_t format;
> +size_t opts_size;
> +
> +switch (spec->format) {
> +case RBD_IMAGE_ENCRYPTION_FORMAT_LUKS1: {
> +rbd_encryption_luks1_format_options_t *luks1_opts =
> +g_new0(rbd_encryption_luks1_format_options_t, 1);
> +format = RBD_ENCRYPTION_FORMAT_LUKS1;
> +opts = luks1_opts;
> +opts_size = sizeof(rbd_encryption_luks1_format_options_t);
> +r = qemu_rbd_convert_luks_options(
> +qapi_RbdEncryptionOptionsLUKS1_base(>u.luks1),
> +_opts->alg, , errp);
> +if (passphrase) {
> +luks1_opts->passphrase = passphrase;
> +luks1_opts->passphrase_size = strlen(passphrase);
> +}
> +break;
> +}
> +case RBD_IMAGE_ENCRYPTION_FORMAT_LUKS2: {
> +rbd_encryption_luks2_format_options_t *luks2_opts =
> +g_new0(rbd_encryption_luks2_format_options_t, 1);
> +format = RBD_ENCRYPTION_FORMAT_LUKS2;
> +opts = luks2_opts;
> +opts_size = sizeof(rbd_encryption_luks2_format_options_t);
> +r = qemu_rbd_convert_luks_options(
> +qapi_RbdEncryptionOptionsLUKS2_base(>u.luks2),
> +_opts->alg, , errp);
> +if (passphrase) {
> +luks2_opts->passphrase = passphrase;
> +luks2_opts->passphrase_size = strlen(passphrase);
> +}
> +break;
> +}
> +default: {
> + 

Re: [PATCH] block/rbd: Add support for rbd image encryption

[no-reply]

Patchew URL: https://patchew.org/QEMU/20210502073617.2978836-1-...@il.ibm.com/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Type: series
Message-id: 20210502073617.2978836-1-...@il.ibm.com
Subject: [PATCH] block/rbd: Add support for rbd image encryption

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
   8f860d2..53c5433  master -> master
 * [new tag] patchew/20210502073617.2978836-1-...@il.ibm.com -> 
patchew/20210502073617.2978836-1-...@il.ibm.com
Switched to a new branch 'test'
11af1d7 block/rbd: Add support for rbd image encryption

=== OUTPUT BEGIN ===
ERROR: "foo** bar" should be "foo **bar"
#51: FILE: block/rbd.c:355:
+char** passphrase,

ERROR: "foo* bar" should be "foo *bar"
#94: FILE: block/rbd.c:398:
+  RbdEncryptionSpec* spec,

total: 2 errors, 0 warnings, 385 lines checked

Commit 11af1d704fbe (block/rbd: Add support for rbd image encryption) has style 
problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
=== OUTPUT END ===

Test command exited with code: 1


The full log is available at
http://patchew.org/logs/20210502073617.2978836-1-...@il.ibm.com/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-de...@redhat.com

[PATCH] block/rbd: Add support for rbd image encryption

[Or Ozeri]

Starting from ceph Pacific, RBD has built-in support for image-level encryption.
Currently supported formats are LUKS version 1 and 2.

There are 2 new relevant librbd APIs for controlling encryption, both expect an
open image context:

rbd_encryption_format: formats an image (i.e. writes the LUKS header)
rbd_encryption_load: loads encryptor/decryptor to the image IO stack

This commit extends the qemu rbd driver API to support the above.

Signed-off-by: Or Ozeri 
---
 block/rbd.c  | 230 ++-
 qapi/block-core.json |  61 
 2 files changed, 287 insertions(+), 4 deletions(-)

diff --git a/block/rbd.c b/block/rbd.c
index f098a89c7b..1239e97889 100644
--- a/block/rbd.c
+++ b/block/rbd.c
@@ -108,6 +108,13 @@ typedef struct BDRVRBDState {
 uint64_t image_size;
 } BDRVRBDState;
 
+#ifdef LIBRBD_SUPPORTS_ENCRYPTION
+typedef int (*RbdEncryptionFunc)(rbd_image_t image,
+ rbd_encryption_format_t format,
+ rbd_encryption_options_t opts,
+ size_t opts_size);
+#endif
+
 static int qemu_rbd_connect(rados_t *cluster, rados_ioctx_t *io_ctx,
 BlockdevOptionsRbd *opts, bool cache,
 const char *keypairs, const char *secretid,
@@ -341,6 +348,115 @@ static void qemu_rbd_memset(RADOSCB *rcb, int64_t offs)
 }
 }
 
+#ifdef LIBRBD_SUPPORTS_ENCRYPTION
+static int qemu_rbd_convert_luks_options(
+RbdEncryptionOptionsLUKSBase *luks_opts,
+rbd_encryption_algorithm_t *alg,
+char** passphrase,
+Error **errp)
+{
+int r = 0;
+
+if (!luks_opts->has_passphrase_secret) {
+r = -EINVAL;
+error_setg_errno(errp, -r, "missing encrypt.passphrase-secret");
+return r;
+}
+
+*passphrase = qcrypto_secret_lookup_as_utf8(luks_opts->passphrase_secret,
+errp);
+if (!*passphrase) {
+return -EIO;
+}
+
+if (luks_opts->has_cipher_alg) {
+switch (luks_opts->cipher_alg) {
+case RBD_ENCRYPTION_ALGORITHM_AES_128: {
+*alg = RBD_ENCRYPTION_ALGORITHM_AES128;
+break;
+}
+case RBD_ENCRYPTION_ALGORITHM_AES_256: {
+*alg = RBD_ENCRYPTION_ALGORITHM_AES256;
+break;
+}
+default: {
+r = -ENOTSUP;
+error_setg_errno(errp, -r, "unknown encryption algorithm: %u",
+ luks_opts->cipher_alg);
+return r;
+}
+}
+} else {
+/* default alg */
+*alg = RBD_ENCRYPTION_ALGORITHM_AES256;
+}
+
+return 0;
+}
+
+static int qemu_rbd_apply_encryption_function(rbd_image_t image,
+  RbdEncryptionSpec* spec,
+  RbdEncryptionFunc func,
+  Error **errp)
+{
+int r = 0;
+g_autofree char *passphrase = NULL;
+g_autofree rbd_encryption_options_t opts = NULL;
+rbd_encryption_format_t format;
+size_t opts_size;
+
+switch (spec->format) {
+case RBD_IMAGE_ENCRYPTION_FORMAT_LUKS1: {
+rbd_encryption_luks1_format_options_t *luks1_opts =
+g_new0(rbd_encryption_luks1_format_options_t, 1);
+format = RBD_ENCRYPTION_FORMAT_LUKS1;
+opts = luks1_opts;
+opts_size = sizeof(rbd_encryption_luks1_format_options_t);
+r = qemu_rbd_convert_luks_options(
+qapi_RbdEncryptionOptionsLUKS1_base(>u.luks1),
+_opts->alg, , errp);
+if (passphrase) {
+luks1_opts->passphrase = passphrase;
+luks1_opts->passphrase_size = strlen(passphrase);
+}
+break;
+}
+case RBD_IMAGE_ENCRYPTION_FORMAT_LUKS2: {
+rbd_encryption_luks2_format_options_t *luks2_opts =
+g_new0(rbd_encryption_luks2_format_options_t, 1);
+format = RBD_ENCRYPTION_FORMAT_LUKS2;
+opts = luks2_opts;
+opts_size = sizeof(rbd_encryption_luks2_format_options_t);
+r = qemu_rbd_convert_luks_options(
+qapi_RbdEncryptionOptionsLUKS2_base(>u.luks2),
+_opts->alg, , errp);
+if (passphrase) {
+luks2_opts->passphrase = passphrase;
+luks2_opts->passphrase_size = strlen(passphrase);
+}
+break;
+}
+default: {
+r = -ENOTSUP;
+error_setg_errno(
+errp, -r, "unknown image encryption format: %u",
+spec->format);
+}
+}
+
+if (r < 0) {
+return r;
+}
+
+r = func(image, format, opts, opts_size);
+if (r < 0) {
+error_setg_errno(errp, -r,