Re: [PATCH 0/5] qemu: Don't log spurious errors on qemuMonitorDelObject

2020-03-19 Thread Michal Prívozník
On 18. 3. 2020 12:40, Peter Krempa wrote:
> qemuMonitorDelObject is often used in cleanup cases so we need to
> control whether to log errors.
> 
> First patch actually prevents one of the spurious calls in cases we know
> it would be pointless.
> 
> Peter Krempa (5):
>   qemuDomainChangeEjectableMedia: Don't always remove managed PR daemon
>   qemuMonitorJSON(Add|Del)Object: Refactor cleanup
>   qemuMonitorJSONCheckError: Use g_autofree
>   qemuMonitorJSONCheckError: Allow suppressing of error reporting
>   qemu: Suppress error reporting from qemuMonitorDelObject in cleanup
> paths
> 
>  src/qemu/qemu_block.c| 10 ++---
>  src/qemu/qemu_driver.c   |  2 +-
>  src/qemu/qemu_hotplug.c  | 34 +---
>  src/qemu/qemu_monitor.c  |  5 ++-
>  src/qemu/qemu_monitor.h  |  3 +-
>  src/qemu/qemu_monitor_json.c | 77 ++--
>  src/qemu/qemu_monitor_json.h |  3 +-
>  7 files changed, 71 insertions(+), 63 deletions(-)
> 

Reviewed-by: Michal Privoznik 

Michal



Re: qemu:///embed and isolation from global components

2020-03-19 Thread Andrea Bolognani
On Wed, 2020-03-18 at 18:01 +0100, Michal Prívozník wrote:
> On 18. 3. 2020 16:47, Andrea Bolognani wrote:
> > if I use either one of
> > 
> >   
> > 
> >   
> > 
> >   
> > 
> >   
> > 
> > both qemu:///embed instances try to use the same paths:
> > 
> >   /dev/hugepages/libvirt/qemu/$domid-$domname/
> > 
> >   /dev/shm/libvirt/qemu/$domid-$domname/
> 
> This is because qemu driver uses virDomainDefGetShortName() to construct
> the path which is not embed driver aware. In fact, we use it on a few
> other places:
> 
> libvirt.git $ git grep -C0 virDomainDefGetShortName  -- src/qemu/ \
>  | wc -l
> 15
> 
> so we probably have more broken plaes. I will investigate and post a
> patch. Probably something among those lines that fixed machined name
> generation.

I'm not sure changing virDomainDefGetShortName() to include the embed
root hash all the time, which I assume is what you had in mind, is a
good enough solution.

That would work very well for /dev/shm and friends, but if the path
you're building is *inside* the embed root, then, you just added 15
characters to the path of any file. This wouldn't be too bad if not
for the fact that the path to a UNIX socket can only be up to 108
characters long, which is limiting enough already...

Can we make the whole thing smart enough that the embed root hash
will appear in paths that are outside of the embed root, but not in
those that are inside it?

-- 
Andrea Bolognani / Red Hat / Virtualization



Re: qemu:///embed and isolation from global components

2020-03-19 Thread Daniel P . Berrangé
On Thu, Mar 19, 2020 at 10:21:39AM +0100, Andrea Bolognani wrote:
> On Wed, 2020-03-18 at 18:01 +0100, Michal Prívozník wrote:
> > On 18. 3. 2020 16:47, Andrea Bolognani wrote:
> > > if I use either one of
> > > 
> > >   
> > > 
> > >   
> > > 
> > >   
> > > 
> > >   
> > > 
> > > both qemu:///embed instances try to use the same paths:
> > > 
> > >   /dev/hugepages/libvirt/qemu/$domid-$domname/
> > > 
> > >   /dev/shm/libvirt/qemu/$domid-$domname/
> > 
> > This is because qemu driver uses virDomainDefGetShortName() to construct
> > the path which is not embed driver aware. In fact, we use it on a few
> > other places:
> > 
> > libvirt.git $ git grep -C0 virDomainDefGetShortName  -- src/qemu/ \
> >  | wc -l
> > 15
> > 
> > so we probably have more broken plaes. I will investigate and post a
> > patch. Probably something among those lines that fixed machined name
> > generation.
> 
> I'm not sure changing virDomainDefGetShortName() to include the embed
> root hash all the time, which I assume is what you had in mind, is a
> good enough solution.
> 
> That would work very well for /dev/shm and friends, but if the path
> you're building is *inside* the embed root, then, you just added 15
> characters to the path of any file. This wouldn't be too bad if not
> for the fact that the path to a UNIX socket can only be up to 108
> characters long, which is limiting enough already...
> 
> Can we make the whole thing smart enough that the embed root hash
> will appear in paths that are outside of the embed root, but not in
> those that are inside it?

This is probably a matter of having 2 methods for building a unique
name. One for use cases interacting with global resources, and one
for use cases interacting with isolated resources. Then we'll need
to make sure the driver calls the right one in each context.

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|



[PATCH] tests: switch away from HAVE_SOCKETPAIR

2020-03-19 Thread Pino Toscano
Since the removal of gnulib, HAVE_SOCKETPAIR is no more defined, making
these two tests effectively skipped.

Use the same strategy used in other generic library bits, i.e. exclude
the socketpair usage on Windows.

Semi-related change in virnetdaemontest.c to make it build: since
virutil.h does not include unistd.h anymore, we need to include it.

Signed-off-by: Pino Toscano 
---
 tests/virnetdaemontest.c   | 4 +++-
 tests/virnetserverclienttest.c | 2 +-
 2 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/tests/virnetdaemontest.c b/tests/virnetdaemontest.c
index 825487f0a1..09d268627c 100644
--- a/tests/virnetdaemontest.c
+++ b/tests/virnetdaemontest.c
@@ -18,13 +18,15 @@
 
 #include 
 
+#include 
+
 #include "testutils.h"
 #include "virerror.h"
 #include "rpc/virnetdaemon.h"
 
 #define VIR_FROM_THIS VIR_FROM_RPC
 
-#if defined(HAVE_SOCKETPAIR) && defined(WITH_YAJL)
+#if !defined(WIN32) && defined(WITH_YAJL)
 struct testClientPriv {
 int magic;
 };
diff --git a/tests/virnetserverclienttest.c b/tests/virnetserverclienttest.c
index a9a56c48d5..668fd02a1e 100644
--- a/tests/virnetserverclienttest.c
+++ b/tests/virnetserverclienttest.c
@@ -24,7 +24,7 @@
 
 #define VIR_FROM_THIS VIR_FROM_RPC
 
-#ifdef HAVE_SOCKETPAIR
+#ifndef WIN32
 
 static void *
 testClientNew(virNetServerClientPtr client G_GNUC_UNUSED,
-- 
2.25.1



Re: [PATCH] tests: switch away from HAVE_SOCKETPAIR

2020-03-19 Thread Daniel P . Berrangé
On Thu, Mar 19, 2020 at 12:05:17PM +0100, Pino Toscano wrote:
> Since the removal of gnulib, HAVE_SOCKETPAIR is no more defined, making
> these two tests effectively skipped.

Doh, I had compared the config.h file before & after gnulib removal
to identify  HAVE_* macros that no longer existed, but guess I
missed this.

> 
> Use the same strategy used in other generic library bits, i.e. exclude
> the socketpair usage on Windows.
> 
> Semi-related change in virnetdaemontest.c to make it build: since
> virutil.h does not include unistd.h anymore, we need to include it.
> 
> Signed-off-by: Pino Toscano 
> ---
>  tests/virnetdaemontest.c   | 4 +++-
>  tests/virnetserverclienttest.c | 2 +-
>  2 files changed, 4 insertions(+), 2 deletions(-)

Reviewed-by: Daniel P. Berrangé 


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|



Re: [PATCH 1/1] conf: qemu 9pfs: add 'multidevs' option

2020-03-19 Thread Ján Tomko

On a Tuesday in 2020, Christian Schoenebeck wrote:

Introduce new 'multidevs' option for filesystem.

 


I don't like the 'multidevs' name, but cannot think of anything
beter.

'collisions' maybe?


   
   
 

This option prevents misbheaviours on guest if a 9pfs export
contains multiple devices, due to the potential file ID collisions
this otherwise may cause.

Signed-off-by: Christian Schoenebeck 
---
docs/formatdomain.html.in | 47 ++-
docs/schemas/domaincommon.rng | 10 
src/conf/domain_conf.c| 30 ++
src/conf/domain_conf.h| 13 ++
src/qemu/qemu_command.c   |  7 ++
5 files changed, 106 insertions(+), 1 deletion(-)


Please split the XML changes from the qemu driver changes.

Also missing:
* qemu_capabilities addition
* qemuDomainDeviceDefValidateFS in qemu_domain.c - check for the capability,
  reject this setting for virtiofs
* qemuxml2xmltest addition
* qemuxml2argvtest addition

(no changes required for virschematest - it checks all the XML files in
the directories used by the above tests against the schema)



diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in
index 594146009d..13c506988b 100644
--- a/docs/formatdomain.html.in
+++ b/docs/formatdomain.html.in
@@ -3967,7 +3967,7 @@


  
-  
+  



@@ -4084,13 +4084,58 @@



+  
  Since 5.2.0, the filesystem element
  has an optional attribute model with supported values
  "virtio-transitional", "virtio-non-transitional", or "virtio".
  See Virtio transitional devices
  for more details.
+  
+


Unrelated change that can be split out.


+  
+  The filesystem element has an optional attribute multidevs
+  which specifies how to deal with a filesystem export containing more than
+  one device, in order to avoid file ID collisions on guest when using 9pfs
+  (since 6.2.0, requires QEMU 4.2).
+  This attribute is not available for virtiofs. The possible values are:
+  
+
+
+default
+
+Use QEMU's default setting (which currently is warn).
+
+remap
+
+This setting allows guest to access multiple devices per export without
+encountering misbehaviours. Inode numbers from host are automatically
+remapped on guest to actively prevent file ID collisions if guest
+accesses one export containing multiple devices.
+
+forbid
+
+Only allow to access one device per export by guest. Attempts to access
+additional devices on the same export will cause the individual
+filesystem access by guest to fail with an error and being logged 
(once)
+as error on host side.
+
+warn
+
+This setting resembles the behaviour of 9pfs prior to QEMU 4.2, that is
+no action is performed to prevent any potential file ID collisions if 
an
+export contains multiple devices, with the only exception: a warning is
+logged (once) on host side now. This setting may lead to misbehaviours
+on guest side if more than one device is exported per export, due to 
the
+potential file ID collisions this may cause on guest side in that case.
+
+
+
  




+  
+  The filesystem element may contain the following 
subelements:
+  
+


And so can this one.


  driver
  
The optional driver element allows specifying further details
@@ -25422,6 +25449,9 @@ virDomainFSDefFormat(virBufferPtr buf,
virBufferAsprintf(buf, " model='%s'",
  virDomainFSModelTypeToString(def->model));
}
+if (def->multidevs) {
+virBufferAsprintf(buf, " multidevs='%s'", multidevs);
+}


make syntax-check complains here:
Curly brackets around single-line body:
../src/conf/domain_conf.c:25452-25454:
if (def->multidevs) {
virBufferAsprintf(buf, " multidevs='%s'", multidevs);
}

Jano


signature.asc
Description: PGP signature


Re: [PATCH 3/6] qemuDomainBuildNamespace: Make @devPath const

2020-03-19 Thread Pavel Mores
On Wed, Mar 18, 2020 at 06:32:13PM +0100, Michal Privoznik wrote:
> The @devPath variable is not modifiable. It merely just points to
> string containing path where private devtmpfs is being
> constructed. Make it const so it doesn't look weird that it's not
> freed.
> 
> Signed-off-by: Michal Privoznik 
> ---
>  src/qemu/qemu_domain.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c
> index 2724607311..a4c07fffcb 100644
> --- a/src/qemu/qemu_domain.c
> +++ b/src/qemu/qemu_domain.c
> @@ -15176,7 +15176,7 @@ qemuDomainBuildNamespace(virQEMUDriverConfigPtr cfg,
>   virDomainObjPtr vm)
>  {
>  struct qemuDomainCreateDeviceData data;
> -char *devPath = NULL;
> +const char *devPath = NULL;
>  char **devMountsPath = NULL, **devMountsSavePath = NULL;
>  size_t ndevMountsPath = 0, i;
>  int ret = -1;
> -- 
> 2.24.1
> 

Reviewed-by: Pavel Mores 



Re: [PATCH 4/6] virfile: Handle directories in virFileBindMountDevice()

2020-03-19 Thread Pavel Mores
On Wed, Mar 18, 2020 at 06:32:14PM +0100, Michal Privoznik wrote:
> The @src is not always a file. It may also be a directory (for
> instance qemuDomainCreateDeviceRecursive() assumes that) - even
> though it doesn't happen usually. Anyway, mount() can mount only
> a dir onto a dir and a file onto a file.
> 
> Signed-off-by: Michal Privoznik 
> ---
>  src/util/virfile.c | 13 +++--
>  1 file changed, 11 insertions(+), 2 deletions(-)
> 
> diff --git a/src/util/virfile.c b/src/util/virfile.c
> index 0f31554323..9c175b041f 100644
> --- a/src/util/virfile.c
> +++ b/src/util/virfile.c
> @@ -3743,8 +3743,17 @@ int
>  virFileBindMountDevice(const char *src,
> const char *dst)
>  {
> -if (virFileTouch(dst, 0666) < 0)
> -return -1;
> +if (!virFileExists(dst)) {
> +if (virFileIsDir(src)) {
> +if (virFileMakePath(dst)) {

Personally, I'd consider

if (virFileMakePath(dst) < 0)

clearer here for a reader who's not familiar with the function and what exactly
it returns.

Reviewed-by: Pavel Mores 

> +virReportSystemError(errno, _("Unable to make dir %s"), dst);
> +return -1;
> +}
> +} else {
> +if (virFileTouch(dst, 0666) < 0)
> +return -1;
> +}
> +}
>  
>  if (mount(src, dst, "none", MS_BIND, NULL) < 0) {
>  virReportSystemError(errno, _("Failed to bind %s on to %s"), src,
> -- 
> 2.24.1
> 



Re: [PATCH 1/6] qemuDomainCreateDeviceRecursive: Report error if mkdir() fails

2020-03-19 Thread Pavel Mores
On Wed, Mar 18, 2020 at 06:32:11PM +0100, Michal Privoznik wrote:
> The virFileMakePathWithMode() which is our recursive version of
> mkdir() fails, it simply just returns a negative value with errno
> set. No error is reported (as compared to virFileTouch() for
> instance).
> 
> Signed-off-by: Michal Privoznik 
> ---
>  src/qemu/qemu_domain.c | 6 +-
>  1 file changed, 5 insertions(+), 1 deletion(-)
> 
> diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c
> index 0e2252f6cf..48bf5ae559 100644
> --- a/src/qemu/qemu_domain.c
> +++ b/src/qemu/qemu_domain.c
> @@ -14643,8 +14643,12 @@ qemuDomainCreateDeviceRecursive(const char *device,
>   * proper owner and mode. Bind mount only after that. */
>  } else if (isDir) {
>  if (create &&
> -virFileMakePathWithMode(devicePath, sb.st_mode) < 0)
> +virFileMakePathWithMode(devicePath, sb.st_mode) < 0) {
> +virReportSystemError(errno,
> + _("Unable to make dir %s"),
> + devicePath);
>  goto cleanup;
> +}
>  } else {
>  virReportError(VIR_ERR_OPERATION_UNSUPPORTED,
> _("unsupported device type %s 0%o"),
> -- 
> 2.24.1
> 

Reviewed-by: Pavel Mores 



Re: [PATCH 1/1] conf: qemu 9pfs: add 'multidevs' option

2020-03-19 Thread Daniel P . Berrangé
On Thu, Mar 19, 2020 at 04:57:41PM +0100, Christian Schoenebeck wrote:
> On Donnerstag, 19. März 2020 14:10:26 CET Ján Tomko wrote:
> > On a Tuesday in 2020, Christian Schoenebeck wrote:
> > >Introduce new 'multidevs' option for filesystem.
> > >
> > >  
> > 
> > I don't like the 'multidevs' name, but cannot think of anything
> > beter.
> > 
> > 'collisions' maybe?
> 
> Not sure if 'collisions' is better, e.g. collisions='remap' sounds scary. :)
> And which collision would that be? At least IMO 'multidevs' is less ambigious.
> I have no problem though to change it to whatever name you might come up 
> with. 
> Just keep the resulting key-value pair set in mind:
> 
> multidevs='default'
> multidevs='remap'
> multidevs='forbid'
> multidevs='warn'
> 
> vs.
> 
> collisions='default'
> collisions='remap' <- probably misleading what 'remap' means in this case
> collisions='forbid'
> collisions='warn' <- wrong, it warns about multiple devices, not about file 
> ID 
> collisions.
> 
> So different key-name might also require different value-names.
> 
> Another option would be the long form 'multi-devices=...'

I tried to come up with names when this was posted to QEMU, but didn't
think of much better than multidevs, so I think that's acceptable for
libvirt usage.

"collisions" isn't better enough to justify different naming from QEMU


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|



Re: [PATCH 1/1] conf: qemu 9pfs: add 'multidevs' option

2020-03-19 Thread Christian Schoenebeck
On Donnerstag, 19. März 2020 14:10:26 CET Ján Tomko wrote:
> On a Tuesday in 2020, Christian Schoenebeck wrote:
> >Introduce new 'multidevs' option for filesystem.
> >
> >  
> 
> I don't like the 'multidevs' name, but cannot think of anything
> beter.
> 
> 'collisions' maybe?

Not sure if 'collisions' is better, e.g. collisions='remap' sounds scary. :)
And which collision would that be? At least IMO 'multidevs' is less ambigious.
I have no problem though to change it to whatever name you might come up with. 
Just keep the resulting key-value pair set in mind:

multidevs='default'
multidevs='remap'
multidevs='forbid'
multidevs='warn'

vs.

collisions='default'
collisions='remap' <- probably misleading what 'remap' means in this case
collisions='forbid'
collisions='warn' <- wrong, it warns about multiple devices, not about file ID 
collisions.

So different key-name might also require different value-names.

Another option would be the long form 'multi-devices=...'

> >
> >
> >  
> >  
> >
> >This option prevents misbheaviours on guest if a 9pfs export
> >contains multiple devices, due to the potential file ID collisions
> >this otherwise may cause.
> >
> >Signed-off-by: Christian Schoenebeck 
> >---
> >
> > docs/formatdomain.html.in | 47 ++-
> > docs/schemas/domaincommon.rng | 10 
> > src/conf/domain_conf.c| 30 ++
> > src/conf/domain_conf.h| 13 ++
> > src/qemu/qemu_command.c   |  7 ++
> > 5 files changed, 106 insertions(+), 1 deletion(-)
> 
> Please split the XML changes from the qemu driver changes.

Ok

> Also missing:
> * qemu_capabilities addition

AFAICS the common procedure is to add new capabilities always to the end of 
the enum list. So I guess I will do that as well.

> * qemuDomainDeviceDefValidateFS in qemu_domain.c - check for the capability,
> reject this setting for virtiofs

Good to know where that check is supposed to go to, thanks!

> * qemuxml2xmltest addition
> * qemuxml2argvtest addition

Ok, I have to read up how those tests work. AFAICS I need to add xml files to 
their data subdirs.

Separate patches required for those 2 tests?

> (no changes required for virschematest - it checks all the XML files in
> the directories used by the above tests against the schema)
> 
> >diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in
> >index 594146009d..13c506988b 100644
> >--- a/docs/formatdomain.html.in
> >+++ b/docs/formatdomain.html.in
> >@@ -3967,7 +3967,7 @@
> >
> > 
> > 
> >   
> >   
> >
> >-  
> >+   >multidevs='remap'>>
> > 
> > 
> > 
> >
> >@@ -4084,13 +4084,58 @@
> >
> > 
> > 
> >
> >+  
> >
> >   Since 5.2.0, the filesystem element
> >   has an optional attribute model with supported values
> >   "virtio-transitional", "virtio-non-transitional", or "virtio".
> >   See Virtio transitional
> >   devices
> >   for more details.
> >
> >+  
> >+
> 
> Unrelated change that can be split out.

Ok, I'll make that the 1st preparatory patch then including ...

> >+  
> >+  The filesystem element has an optional attribute
> >multidevs +  which specifies how to deal with a
> >filesystem export containing more than +  one device, in order to
> >avoid file ID collisions on guest when using 9pfs +  ( >class="since">since 6.2.0, requires QEMU 4.2). +  This
> >attribute is not available for virtiofs. The possible values are: + 
> >
> >+
> >+
> >+default
> >+
> >+Use QEMU's default setting (which currently is warn).
> >+
> >+remap
> >+
> >+This setting allows guest to access multiple devices per export
> >without +encountering misbehaviours. Inode numbers from host are
> >automatically +remapped on guest to actively prevent file ID
> >collisions if guest +accesses one export containing multiple
> >devices.
> >+
> >+forbid
> >+
> >+Only allow to access one device per export by guest. Attempts to
> >access +additional devices on the same export will cause the
> >individual +filesystem access by guest to fail with an error and
> >being logged (once) +as error on host side.
> >+
> >+warn
> >+
> >+This setting resembles the behaviour of 9pfs prior to QEMU 4.2,
> >that is +no action is performed to prevent any potential file ID
> >collisions if an +export contains multiple devices, with the only
> >exception: a warning is +logged (once) on host side now. This
> >setting may lead to misbehaviours +on guest side if more than one
> >

Re: [PATCH 1/1] conf: qemu 9pfs: add 'multidevs' option

2020-03-19 Thread Ján Tomko

On a Thursday in 2020, Christian Schoenebeck wrote:

On Donnerstag, 19. März 2020 14:10:26 CET Ján Tomko wrote:

On a Tuesday in 2020, Christian Schoenebeck wrote:
>Introduce new 'multidevs' option for filesystem.
>
>  

I don't like the 'multidevs' name, but cannot think of anything
beter.

'collisions' maybe?


Not sure if 'collisions' is better, e.g. collisions='remap' sounds scary. :)
And which collision would that be? At least IMO 'multidevs' is less ambigious.
I have no problem though to change it to whatever name you might come up with.
Just keep the resulting key-value pair set in mind:

multidevs='default'
multidevs='remap'
multidevs='forbid'
multidevs='warn'

vs.

collisions='default'
collisions='remap' <- probably misleading what 'remap' means in this case
collisions='forbid'
collisions='warn' <- wrong, it warns about multiple devices, not about file ID
collisions.

So different key-name might also require different value-names.

Another option would be the long form 'multi-devices=...'


Okay, let's leave it at 'multidevs'.




>
>
>
>  
>
>This option prevents misbheaviours on guest if a 9pfs export
>contains multiple devices, due to the potential file ID collisions
>this otherwise may cause.
>
>Signed-off-by: Christian Schoenebeck 
>---
>
> docs/formatdomain.html.in | 47 ++-
> docs/schemas/domaincommon.rng | 10 
> src/conf/domain_conf.c| 30 ++
> src/conf/domain_conf.h| 13 ++
> src/qemu/qemu_command.c   |  7 ++
> 5 files changed, 106 insertions(+), 1 deletion(-)

Please split the XML changes from the qemu driver changes.


Ok


Also missing:
* qemu_capabilities addition


AFAICS the common procedure is to add new capabilities always to the end of
the enum list. So I guess I will do that as well.


* qemuDomainDeviceDefValidateFS in qemu_domain.c - check for the capability,
reject this setting for virtiofs


Good to know where that check is supposed to go to, thanks!


* qemuxml2xmltest addition
* qemuxml2argvtest addition


Ok, I have to read up how those tests work. AFAICS I need to add xml files to
their data subdirs.

Separate patches required for those 2 tests?


Usually xml2xmltest is in the same test as the XML parser/formatter
and xml2argvtest is a part of the qemu driver patch.

Jano


signature.asc
Description: PGP signature


[PATCH] qemu: Don't crash when getting targets for a multipath

2020-03-19 Thread Michal Privoznik
In one of my previous commits I've introduced code that creates
all devices for given (possible) multipath target. But I've made
a mistake there - the code accesses src->path without checking if
the disk source is local. Note that the path is NULL if the
source is not local.

Fixes: a30078cb832646177defd256e77c632905f1e6d0
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1814947

Signed-off-by: Michal Privoznik 
---
 src/qemu/qemu_domain.c | 16 
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c
index 0e2252f6cf..ef17829235 100644
--- a/src/qemu/qemu_domain.c
+++ b/src/qemu/qemu_domain.c
@@ -15862,19 +15862,19 @@ qemuDomainNamespaceSetupDisk(virDomainObjPtr vm,
 }
 
 tmpPath = g_strdup(next->path);
+
+if (virDevMapperGetTargets(next->path, &targetPaths) < 0 &&
+errno != ENOSYS && errno != EBADF) {
+virReportSystemError(errno,
+ _("Unable to get devmapper targets for 
%s"),
+ next->path);
+return -1;
+}
 }
 
 if (virStringListAdd(&paths, tmpPath) < 0)
 return -1;
 
-if (virDevMapperGetTargets(next->path, &targetPaths) < 0 &&
-errno != ENOSYS && errno != EBADF) {
-virReportSystemError(errno,
- _("Unable to get devmapper targets for %s"),
- next->path);
-return -1;
-}
-
 if (virStringListMerge(&paths, &targetPaths) < 0)
 return -1;
 }
-- 
2.24.1



[PATCH 2/4] qemu: block: Extract logic decision when to use a separate 'raw' layer for slice

2020-03-19 Thread Peter Krempa
Introduce qemuBlockStorageSourceNeedsStorageSliceLayer which will hold
the decision logic and fix all places that open-code it.

Signed-off-by: Peter Krempa 
---
 src/qemu/qemu_block.c  | 24 +---
 src/qemu/qemu_block.h  |  3 +++
 src/qemu/qemu_domain.c |  3 +--
 3 files changed, 25 insertions(+), 5 deletions(-)

diff --git a/src/qemu/qemu_block.c b/src/qemu/qemu_block.c
index 4ed17b6df3..b5b34ab441 100644
--- a/src/qemu/qemu_block.c
+++ b/src/qemu/qemu_block.c
@@ -1446,8 +1446,7 @@ 
qemuBlockStorageSourceGetBlockdevProps(virStorageSourcePtr src,
 g_autoptr(virJSONValue) props = NULL;
 const char *storagenode = src->nodestorage;

-if (src->sliceStorage &&
-src->format != VIR_STORAGE_FILE_RAW)
+if (qemuBlockStorageSourceNeedsStorageSliceLayer(src))
 storagenode = src->sliceStorage->nodename;

 if (!(props = qemuBlockStorageSourceGetBlockdevFormatProps(src)))
@@ -1568,7 +1567,7 @@ 
qemuBlockStorageSourceAttachPrepareBlockdev(virStorageSourcePtr src,
 data->storageNodeName = src->nodestorage;
 data->formatNodeName = src->nodeformat;

-if (src->sliceStorage && src->format != VIR_STORAGE_FILE_RAW) {
+if (qemuBlockStorageSourceNeedsStorageSliceLayer(src)) {
 if (!(data->storageSliceProps = 
qemuBlockStorageSourceGetBlockdevStorageSliceProps(src)))
 return NULL;

@@ -3308,3 +3307,22 @@ qemuBlockReopenReadOnly(virDomainObjPtr vm,

 return 0;
 }
+
+/**
+ * qemuBlockStorageSourceNeedSliceLayer:
+ * @src: source to inspect
+ *
+ * Returns true if @src requires an extra 'raw' layer for handling of the 
storage
+ * slice.
+ */
+bool
+qemuBlockStorageSourceNeedsStorageSliceLayer(const virStorageSource *src)
+{
+if (!src->sliceStorage)
+return false;
+
+if (src->format != VIR_STORAGE_FILE_RAW)
+return true;
+
+return false;
+}
diff --git a/src/qemu/qemu_block.h b/src/qemu/qemu_block.h
index 75b25bfea5..28475b25c1 100644
--- a/src/qemu/qemu_block.h
+++ b/src/qemu/qemu_block.h
@@ -254,3 +254,6 @@ int
 qemuBlockReopenReadOnly(virDomainObjPtr vm,
 virStorageSourcePtr src,
 qemuDomainAsyncJob asyncJob);
+
+bool
+qemuBlockStorageSourceNeedsStorageSliceLayer(const virStorageSource *src);
diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c
index 0e2252f6cf..7dda986e3a 100644
--- a/src/qemu/qemu_domain.c
+++ b/src/qemu/qemu_domain.c
@@ -16590,8 +16590,7 @@ 
qemuDomainPrepareStorageSourceBlockdev(virDomainDiskDefPtr disk,
 src->nodestorage = g_strdup_printf("libvirt-%u-storage", src->id);
 src->nodeformat = g_strdup_printf("libvirt-%u-format", src->id);

-if (src->sliceStorage &&
-src->format != VIR_STORAGE_FILE_RAW)
+if (qemuBlockStorageSourceNeedsStorageSliceLayer(src))
 src->sliceStorage->nodename = g_strdup_printf("libvirt-%u-slice-sto", 
src->id);

 if (qemuDomainValidateStorageSource(src, priv->qemuCaps) < 0)
-- 
2.24.1



[PATCH 4/4] qemu: block: Split up formatting of JSON props for 'raw' and 'luks' drivers

2020-03-19 Thread Peter Krempa
qemuBlockStorageSourceGetFormatRawProps aggregated both formats but
since we now have props specific for either of those formats it's
unwanted to aggregate the code such way. Split out the 'luks' props
formatter into qemuBlockStorageSourceGetFormatLUKSProps.

The wrong separation demonstrates istself on formatting of the 'size'
and 'offset' attributes for the 'luks' driver which does not conform
to the qapi schema.

https://bugzilla.redhat.com/show_bug.cgi?id=1814975

Signed-off-by: Peter Krempa 
---
 src/qemu/qemu_block.c | 42 ---
 .../disk-slices.x86_64-latest.args|  2 +-
 2 files changed, 29 insertions(+), 15 deletions(-)

diff --git a/src/qemu/qemu_block.c b/src/qemu/qemu_block.c
index ee15167215..5c85ddd44c 100644
--- a/src/qemu/qemu_block.c
+++ b/src/qemu/qemu_block.c
@@ -1200,24 +1200,32 @@ 
qemuBlockStorageSourceGetBackendProps(virStorageSourcePtr src,


 static int
-qemuBlockStorageSourceGetFormatRawProps(virStorageSourcePtr src,
-virJSONValuePtr props)
+qemuBlockStorageSourceGetFormatLUKSProps(virStorageSourcePtr src,
+ virJSONValuePtr props)
 {
 qemuDomainStorageSourcePrivatePtr srcPriv = 
QEMU_DOMAIN_STORAGE_SOURCE_PRIVATE(src);
-const char *driver = "raw";
-const char *secretalias = NULL;

-if (src->encryption &&
-src->encryption->format == VIR_STORAGE_ENCRYPTION_FORMAT_LUKS &&
-srcPriv &&
-srcPriv->encinfo) {
-driver = "luks";
-secretalias = srcPriv->encinfo->s.aes.alias;
+if (!srcPriv || !srcPriv->encinfo || !srcPriv->encinfo->s.aes.alias) {
+virReportError(VIR_ERR_INTERNAL_ERROR, "%s",
+   _("missing secret info for 'luks' driver"));
+return -1;
 }

 if (virJSONValueObjectAdd(props,
-  "s:driver", driver,
-  "S:key-secret", secretalias, NULL) < 0)
+  "s:driver", "luks",
+  "s:key-secret", srcPriv->encinfo->s.aes.alias,
+  NULL) < 0)
+return -1;
+
+return 0;
+}
+
+
+static int
+qemuBlockStorageSourceGetFormatRawProps(virStorageSourcePtr src,
+virJSONValuePtr props)
+{
+if (virJSONValueObjectAdd(props, "s:driver", "raw", NULL) < 0)
 return -1;

 /* Currently only storage slices are supported. We'll have to calculate
@@ -1371,8 +1379,14 @@ 
qemuBlockStorageSourceGetBlockdevFormatProps(virStorageSourcePtr src)
 /* The fat layer is emulated by the storage access layer, so we need to
  * put a raw layer on top */
 case VIR_STORAGE_FILE_RAW:
-if (qemuBlockStorageSourceGetFormatRawProps(src, props) < 0)
-return NULL;
+if (src->encryption &&
+src->encryption->format == VIR_STORAGE_ENCRYPTION_FORMAT_LUKS) {
+if (qemuBlockStorageSourceGetFormatLUKSProps(src, props) < 0)
+return NULL;
+} else {
+if (qemuBlockStorageSourceGetFormatRawProps(src, props) < 0)
+return NULL;
+}
 break;

 case VIR_STORAGE_FILE_QCOW2:
diff --git a/tests/qemuxml2argvdata/disk-slices.x86_64-latest.args 
b/tests/qemuxml2argvdata/disk-slices.x86_64-latest.args
index 63bdaa58be..869b4c0af0 100644
--- a/tests/qemuxml2argvdata/disk-slices.x86_64-latest.args
+++ b/tests/qemuxml2argvdata/disk-slices.x86_64-latest.args
@@ -56,7 +56,7 @@ keyid=masterKey0,iv=AAECAwQFBgcICQoLDA0ODw==,format=base64 \
 "size":321,"file":"libvirt-1-storage","auto-read-only":true,\
 "discard":"unmap"}' \
 -blockdev '{"node-name":"libvirt-1-format","read-only":false,"driver":"luks",\
-"key-secret":"libvirt-1-format-encryption-secret0","offset":1234,"size":321,\
+"key-secret":"libvirt-1-format-encryption-secret0",\
 "file":"libvirt-1-slice-sto"}' \
 -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=libvirt-1-format,\
 id=virtio-disk2 \
-- 
2.24.1



[PATCH 0/4] qemu: Fix usage of 'slice' and 'luks'

2020-03-19 Thread Peter Krempa
We special cased some behaviour and it broke on combination with other
special case.

Peter Krempa (4):
  qemuxml2argvdata/disk-slices: Add test case for 'luks' encryption
  qemu: block: Extract logic decision when to use a separate 'raw' layer
for slice
  qemuBlockStorageSourceNeedsStorageSliceLayer: Deal with 'luks' files
  qemu: block: Split up formatting of JSON props for 'raw' and 'luks'
drivers

 src/qemu/qemu_block.c | 70 ++-
 src/qemu/qemu_block.h |  3 +
 src/qemu/qemu_domain.c|  3 +-
 .../disk-slices.x86_64-latest.args| 41 +++
 tests/qemuxml2argvdata/disk-slices.xml| 13 
 .../disk-slices.x86_64-latest.xml | 16 -
 6 files changed, 112 insertions(+), 34 deletions(-)

-- 
2.24.1



[PATCH 3/4] qemuBlockStorageSourceNeedsStorageSliceLayer: Deal with 'luks' files

2020-03-19 Thread Peter Krempa
The 'luks' driver in qemu is as any other non-raw format driver and thus
doesn't support the properties for 'slice'. Since libvirt considers
luks files to be raw+encryption we need to special case them when
dealing with the slice.

https://bugzilla.redhat.com/show_bug.cgi?id=1814975

Signed-off-by: Peter Krempa 
---
 src/qemu/qemu_block.c | 4 
 tests/qemuxml2argvdata/disk-slices.x86_64-latest.args | 5 -
 2 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/src/qemu/qemu_block.c b/src/qemu/qemu_block.c
index b5b34ab441..ee15167215 100644
--- a/src/qemu/qemu_block.c
+++ b/src/qemu/qemu_block.c
@@ -3324,5 +3324,9 @@ qemuBlockStorageSourceNeedsStorageSliceLayer(const 
virStorageSource *src)
 if (src->format != VIR_STORAGE_FILE_RAW)
 return true;

+if (src->encryption &&
+src->encryption->format == VIR_STORAGE_ENCRYPTION_FORMAT_LUKS)
+return true;
+
 return false;
 }
diff --git a/tests/qemuxml2argvdata/disk-slices.x86_64-latest.args 
b/tests/qemuxml2argvdata/disk-slices.x86_64-latest.args
index 75485c17a7..63bdaa58be 100644
--- a/tests/qemuxml2argvdata/disk-slices.x86_64-latest.args
+++ b/tests/qemuxml2argvdata/disk-slices.x86_64-latest.args
@@ -52,9 +52,12 @@ 
data=9eao5F8qtkGt+seB1HYivWIxbtwUu6MQtg1zpj/oDtUsPr1q8wBYM91uEHCn6j/1,\
 keyid=masterKey0,iv=AAECAwQFBgcICQoLDA0ODw==,format=base64 \
 -blockdev '{"driver":"file","filename":"/var/lib/libvirt/images/luks.img",\
 "node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \
+-blockdev '{"driver":"raw","node-name":"libvirt-1-slice-sto","offset":1234,\
+"size":321,"file":"libvirt-1-storage","auto-read-only":true,\
+"discard":"unmap"}' \
 -blockdev '{"node-name":"libvirt-1-format","read-only":false,"driver":"luks",\
 "key-secret":"libvirt-1-format-encryption-secret0","offset":1234,"size":321,\
-"file":"libvirt-1-storage"}' \
+"file":"libvirt-1-slice-sto"}' \
 -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=libvirt-1-format,\
 id=virtio-disk2 \
 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 \
-- 
2.24.1



[PATCH 1/4] qemuxml2argvdata/disk-slices: Add test case for 'luks' encryption

2020-03-19 Thread Peter Krempa
Since libvirt handles the luks encryption in a weird special way
(raw+encryption) we should really test that case with slices as well.

Signed-off-by: Peter Krempa 
---
 .../disk-slices.x86_64-latest.args| 38 ---
 tests/qemuxml2argvdata/disk-slices.xml| 13 +++
 .../disk-slices.x86_64-latest.xml | 16 +++-
 3 files changed, 52 insertions(+), 15 deletions(-)

diff --git a/tests/qemuxml2argvdata/disk-slices.x86_64-latest.args 
b/tests/qemuxml2argvdata/disk-slices.x86_64-latest.args
index 49daa293a4..75485c17a7 100644
--- a/tests/qemuxml2argvdata/disk-slices.x86_64-latest.args
+++ b/tests/qemuxml2argvdata/disk-slices.x86_64-latest.args
@@ -29,25 +29,35 @@ file=/tmp/lib/domain--1-QEMUGuest1/master-key.aes \
 -boot strict=on \
 -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 \
 -blockdev '{"driver":"file","filename":"/var/lib/libvirt/images/raw.img",\
-"node-name":"libvirt-3-storage","auto-read-only":true,"discard":"unmap"}' \
--blockdev '{"node-name":"libvirt-3-format","read-only":false,"driver":"raw",\
-"offset":1234,"size":321,"file":"libvirt-3-storage"}' \
--device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x2,drive=libvirt-3-format,\
+"node-name":"libvirt-4-storage","auto-read-only":true,"discard":"unmap"}' \
+-blockdev '{"node-name":"libvirt-4-format","read-only":false,"driver":"raw",\
+"offset":1234,"size":321,"file":"libvirt-4-storage"}' \
+-device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x2,drive=libvirt-4-format,\
 id=virtio-disk0,bootindex=1 \
 -blockdev '{"driver":"file","filename":"/var/lib/libvirt/images/raw.img",\
-"node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"}' \
--blockdev '{"driver":"raw","node-name":"libvirt-2-slice-sto","offset":9876,\
-"size":123456789,"file":"libvirt-2-storage","auto-read-only":true,\
+"node-name":"libvirt-3-storage","auto-read-only":true,"discard":"unmap"}' \
+-blockdev '{"driver":"raw","node-name":"libvirt-3-slice-sto","offset":9876,\
+"size":123456789,"file":"libvirt-3-storage","auto-read-only":true,\
 "discard":"unmap"}' \
--blockdev '{"node-name":"libvirt-2-format","read-only":true,"driver":"qcow2",\
-"file":"libvirt-2-slice-sto","backing":null}' \
+-blockdev '{"node-name":"libvirt-3-format","read-only":true,"driver":"qcow2",\
+"file":"libvirt-3-slice-sto","backing":null}' \
 -blockdev 
'{"driver":"file","filename":"/var/lib/libvirt/images/overlay.qcow2",\
-"node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \
--blockdev '{"node-name":"libvirt-1-format","read-only":false,"driver":"qcow2",\
-"file":"libvirt-1-storage","backing":"libvirt-2-format"}' \
--device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x3,drive=libvirt-1-format,\
+"node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"}' \
+-blockdev '{"node-name":"libvirt-2-format","read-only":false,"driver":"qcow2",\
+"file":"libvirt-2-storage","backing":"libvirt-3-format"}' \
+-device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x3,drive=libvirt-2-format,\
 id=virtio-disk1 \
--device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4 \
+-object secret,id=libvirt-1-format-encryption-secret0,\
+data=9eao5F8qtkGt+seB1HYivWIxbtwUu6MQtg1zpj/oDtUsPr1q8wBYM91uEHCn6j/1,\
+keyid=masterKey0,iv=AAECAwQFBgcICQoLDA0ODw==,format=base64 \
+-blockdev '{"driver":"file","filename":"/var/lib/libvirt/images/luks.img",\
+"node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \
+-blockdev '{"node-name":"libvirt-1-format","read-only":false,"driver":"luks",\
+"key-secret":"libvirt-1-format-encryption-secret0","offset":1234,"size":321,\
+"file":"libvirt-1-storage"}' \
+-device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=libvirt-1-format,\
+id=virtio-disk2 \
+-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 \
 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,\
 resourcecontrol=deny \
 -msg timestamp=on
diff --git a/tests/qemuxml2argvdata/disk-slices.xml 
b/tests/qemuxml2argvdata/disk-slices.xml
index cff7cc7ee4..5c6f29d154 100644
--- a/tests/qemuxml2argvdata/disk-slices.xml
+++ b/tests/qemuxml2argvdata/disk-slices.xml
@@ -38,6 +38,19 @@
   
   
 
+
+  
+  
+
+  
+
+
+  
+
+  
+  
+  
+
 
 
 
diff --git a/tests/qemuxml2xmloutdata/disk-slices.x86_64-latest.xml 
b/tests/qemuxml2xmloutdata/disk-slices.x86_64-latest.xml
index 65590df417..4f4abcda1f 100644
--- a/tests/qemuxml2xmloutdata/disk-slices.x86_64-latest.xml
+++ b/tests/qemuxml2xmloutdata/disk-slices.x86_64-latest.xml
@@ -43,6 +43,20 @@
   
   
 
+
+  
+  
+
+  
+
+
+  
+
+  
+  
+  
+  
+
 
   
 
@@ -50,7 +64,7 @@
 
 
 
-  
+  
 
   
 
-- 
2.24.1



Re: [PATCH] qemu: Don't crash when getting targets for a multipath

2020-03-19 Thread Peter Krempa
On Thu, Mar 19, 2020 at 17:22:37 +0100, Michal Privoznik wrote:
> In one of my previous commits I've introduced code that creates
> all devices for given (possible) multipath target. But I've made
> a mistake there - the code accesses src->path without checking if
> the disk source is local. Note that the path is NULL if the
> source is not local.

Well next->path 'may' be NULL if it's not local, but in this case it is
local because NVMe disks are local, but they don't have the path.

> 
> Fixes: a30078cb832646177defd256e77c632905f1e6d0
> Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1814947
> 
> Signed-off-by: Michal Privoznik 
> ---
>  src/qemu/qemu_domain.c | 16 
>  1 file changed, 8 insertions(+), 8 deletions(-)

Once the commit message makes sense:

Reviewed-by: Peter Krempa 

> 
> diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c
> index 0e2252f6cf..ef17829235 100644
> --- a/src/qemu/qemu_domain.c
> +++ b/src/qemu/qemu_domain.c
> @@ -15862,19 +15862,19 @@ qemuDomainNamespaceSetupDisk(virDomainObjPtr vm,
>  }
>  
>  tmpPath = g_strdup(next->path);
> +
> +if (virDevMapperGetTargets(next->path, &targetPaths) < 0 &&
> +errno != ENOSYS && errno != EBADF) {
> +virReportSystemError(errno,
> + _("Unable to get devmapper targets for 
> %s"),
> + next->path);
> +return -1;
> +}
>  }
>  
>  if (virStringListAdd(&paths, tmpPath) < 0)
>  return -1;
>  
> -if (virDevMapperGetTargets(next->path, &targetPaths) < 0 &&
> -errno != ENOSYS && errno != EBADF) {
> -virReportSystemError(errno,
> - _("Unable to get devmapper targets for %s"),
> - next->path);
> -return -1;
> -}
> -
>  if (virStringListMerge(&paths, &targetPaths) < 0)
>  return -1;

Is this supposed to go after 'tmpPath' in the list? Otherwise you might
want to move it toghether with the call to virDevMapperGetTargets.



Re: [PATCH] qemu: Don't crash when getting targets for a multipath

2020-03-19 Thread Ján Tomko

On a Thursday in 2020, Michal Privoznik wrote:

In one of my previous commits I've introduced code that creates
all devices for given (possible) multipath target. But I've made
a mistake there - the code accesses src->path without checking if
the disk source is local. Note that the path is NULL if the
source is not local.

Fixes: a30078cb832646177defd256e77c632905f1e6d0
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1814947

Signed-off-by: Michal Privoznik 
---
src/qemu/qemu_domain.c | 16 
1 file changed, 8 insertions(+), 8 deletions(-)



Reviewed-by: Ján Tomko 

Jano


signature.asc
Description: PGP signature


[PATCH 0/2] Fix wrong copy of encryption 'usage' string

2020-03-19 Thread Peter Krempa
Copying the pointer caused double-free when clearing the domain object.

Peter Krempa (2):
  virSecretLookupDefCopy: Remove return value
  virStorageEncryptionSecretCopy: Properly copy internals

 src/util/virsecret.c| 3 +--
 src/util/virsecret.h| 4 ++--
 src/util/virstorageencryption.c | 8 +++-
 src/util/virstoragefile.c   | 3 +--
 4 files changed, 7 insertions(+), 11 deletions(-)

-- 
2.24.1



[PATCH 2/2] virStorageEncryptionSecretCopy: Properly copy internals

2020-03-19 Thread Peter Krempa
virStorageEncryptionSecretPtr may have a string inside it, thus we must
copy the string too. Use virSecretLookupDefCopy to do that.

Likely caused by 756b46ddd24.

https://bugzilla.redhat.com/show_bug.cgi?id=1814923

Signed-off-by: Peter Krempa 
---
 src/util/virstorageencryption.c | 8 +++-
 1 file changed, 3 insertions(+), 5 deletions(-)

diff --git a/src/util/virstorageencryption.c b/src/util/virstorageencryption.c
index 74836d4a00..6765fdc23a 100644
--- a/src/util/virstorageencryption.c
+++ b/src/util/virstorageencryption.c
@@ -85,12 +85,10 @@ virStorageEncryptionFree(virStorageEncryptionPtr enc)
 static virStorageEncryptionSecretPtr
 virStorageEncryptionSecretCopy(const virStorageEncryptionSecret *src)
 {
-virStorageEncryptionSecretPtr ret;
-
-if (VIR_ALLOC(ret) < 0)
-return NULL;
+virStorageEncryptionSecretPtr ret = g_new0(virStorageEncryptionSecret, 1);

-memcpy(ret, src, sizeof(*src));
+ret->type = src->type;
+virSecretLookupDefCopy(&ret->seclookupdef, &src->seclookupdef);

 return ret;
 }
-- 
2.24.1



[PATCH 1/2] virSecretLookupDefCopy: Remove return value

2020-03-19 Thread Peter Krempa
The function always returns succes so there's no need for a return
value.

Signed-off-by: Peter Krempa 
---
 src/util/virsecret.c  | 3 +--
 src/util/virsecret.h  | 4 ++--
 src/util/virstoragefile.c | 3 +--
 3 files changed, 4 insertions(+), 6 deletions(-)

diff --git a/src/util/virsecret.c b/src/util/virsecret.c
index f44d964198..54d6bbcb7c 100644
--- a/src/util/virsecret.c
+++ b/src/util/virsecret.c
@@ -48,7 +48,7 @@ virSecretLookupDefClear(virSecretLookupTypeDefPtr def)
 }


-int
+void
 virSecretLookupDefCopy(virSecretLookupTypeDefPtr dst,
const virSecretLookupTypeDef *src)
 {
@@ -58,7 +58,6 @@ virSecretLookupDefCopy(virSecretLookupTypeDefPtr dst,
 } else if (dst->type == VIR_SECRET_LOOKUP_TYPE_USAGE) {
 dst->u.usage = g_strdup(src->u.usage);
 }
-return 0;
 }


diff --git a/src/util/virsecret.h b/src/util/virsecret.h
index 8c49cfbc89..cfdf2b6e29 100644
--- a/src/util/virsecret.h
+++ b/src/util/virsecret.h
@@ -48,8 +48,8 @@ struct _virSecretLookupTypeDef {
 };

 void virSecretLookupDefClear(virSecretLookupTypeDefPtr def);
-int virSecretLookupDefCopy(virSecretLookupTypeDefPtr dst,
-   const virSecretLookupTypeDef *src);
+void virSecretLookupDefCopy(virSecretLookupTypeDefPtr dst,
+const virSecretLookupTypeDef *src);
 int virSecretLookupParseSecret(xmlNodePtr secretnode,
virSecretLookupTypeDefPtr def);
 void virSecretLookupFormatSecret(virBufferPtr buf,
diff --git a/src/util/virstoragefile.c b/src/util/virstoragefile.c
index 870c40f446..e723cc9410 100644
--- a/src/util/virstoragefile.c
+++ b/src/util/virstoragefile.c
@@ -1792,8 +1792,7 @@ virStorageAuthDefCopy(const virStorageAuthDef *src)
 authdef->secrettype = g_strdup(src->secrettype);
 authdef->authType = src->authType;

-if (virSecretLookupDefCopy(&authdef->seclookupdef, &src->seclookupdef) < 0)
-return NULL;
+virSecretLookupDefCopy(&authdef->seclookupdef, &src->seclookupdef);

 return g_steal_pointer(&authdef);
 }
-- 
2.24.1



Re: [PATCH 0/4] qemu: Fix usage of 'slice' and 'luks'

2020-03-19 Thread Ján Tomko

On a Thursday in 2020, Peter Krempa wrote:

We special cased some behaviour and it broke on combination with other
special case.

Peter Krempa (4):
 qemuxml2argvdata/disk-slices: Add test case for 'luks' encryption
 qemu: block: Extract logic decision when to use a separate 'raw' layer
   for slice
 qemuBlockStorageSourceNeedsStorageSliceLayer: Deal with 'luks' files
 qemu: block: Split up formatting of JSON props for 'raw' and 'luks'
   drivers

src/qemu/qemu_block.c | 70 ++-
src/qemu/qemu_block.h |  3 +
src/qemu/qemu_domain.c|  3 +-
.../disk-slices.x86_64-latest.args| 41 +++
tests/qemuxml2argvdata/disk-slices.xml| 13 
.../disk-slices.x86_64-latest.xml | 16 -
6 files changed, 112 insertions(+), 34 deletions(-)



Reviewed-by: Ján Tomko 

Jano


signature.asc
Description: PGP signature


[PATCH] Minor doc fix

2020-03-19 Thread Sebastian Mitterle
1. Fix link to knowledge base article
2. Use  to make sure kbase.html has page title

Signed-off-by Sebastian Mitterle 
---
 docs/formatbackup.html.in | 2 +-
 docs/kbase.html.in| 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/docs/formatbackup.html.in b/docs/formatbackup.html.in
index 543d913072..55acd13ddc 100644
--- a/docs/formatbackup.html.in
+++ b/docs/formatbackup.html.in
@@ -14,7 +14,7 @@
   description of the actions to perform, as well as an optional
   second XML document describing a
   checkpoint to create at the same point in time. See
-  also a comparison between
+  also a comparison between
   the various state capture APIs.
 
 
diff --git a/docs/kbase.html.in b/docs/kbase.html.in
index 7055f4fda4..c586e0f676 100644
--- a/docs/kbase.html.in
+++ b/docs/kbase.html.in
@@ -2,7 +2,7 @@
 
 http://www.w3.org/1999/xhtml";>
   
-Knowledge base
+Knowledge base
 
 
   
-- 
2.21.0



Re: [PATCH 2/2] virStorageEncryptionSecretCopy: Properly copy internals

2020-03-19 Thread Ján Tomko

On a Thursday in 2020, Peter Krempa wrote:

virStorageEncryptionSecretPtr may have a string inside it, thus we must
copy the string too. Use virSecretLookupDefCopy to do that.

Likely caused by 756b46ddd24.


Please remove the period from the end.

Also, at the times of that commit, memcpy was sufficient:
struct _virStorageEncryptionSecret {
int type; /* virStorageEncryptionSecretType */
unsigned char uuid[VIR_UUID_BUFLEN];
};

So this actually fixes commit 47e88b33b which switched to using
virSecretLookupDef.



https://bugzilla.redhat.com/show_bug.cgi?id=1814923

Signed-off-by: Peter Krempa 
---
src/util/virstorageencryption.c | 8 +++-
1 file changed, 3 insertions(+), 5 deletions(-)



signature.asc
Description: PGP signature


Re: [PATCH 0/2] Fix wrong copy of encryption 'usage' string

2020-03-19 Thread Ján Tomko

On a Thursday in 2020, Peter Krempa wrote:

Copying the pointer caused double-free when clearing the domain object.

Peter Krempa (2):
 virSecretLookupDefCopy: Remove return value
 virStorageEncryptionSecretCopy: Properly copy internals

src/util/virsecret.c| 3 +--
src/util/virsecret.h| 4 ++--
src/util/virstorageencryption.c | 8 +++-
src/util/virstoragefile.c   | 3 +--
4 files changed, 7 insertions(+), 11 deletions(-)



Reviewed-by: Ján Tomko 

Jano


signature.asc
Description: PGP signature


Re: [PATCH 0/2] Fix wrong copy of encryption 'usage' string

2020-03-19 Thread Michal Prívozník
On 19. 3. 2020 17:48, Peter Krempa wrote:
> Copying the pointer caused double-free when clearing the domain object.
> 
> Peter Krempa (2):
>   virSecretLookupDefCopy: Remove return value
>   virStorageEncryptionSecretCopy: Properly copy internals
> 
>  src/util/virsecret.c| 3 +--
>  src/util/virsecret.h| 4 ++--
>  src/util/virstorageencryption.c | 8 +++-
>  src/util/virstoragefile.c   | 3 +--
>  4 files changed, 7 insertions(+), 11 deletions(-)
> 

Reviewed-by: Michal Privoznik 

Michal



Re: [PATCH 2/2] virStorageEncryptionSecretCopy: Properly copy internals

2020-03-19 Thread Peter Krempa
On Thu, Mar 19, 2020 at 18:14:15 +0100, Ján Tomko wrote:
> On a Thursday in 2020, Peter Krempa wrote:
> > virStorageEncryptionSecretPtr may have a string inside it, thus we must
> > copy the string too. Use virSecretLookupDefCopy to do that.
> > 
> > Likely caused by 756b46ddd24.
> 
> Please remove the period from the end.
> 
> Also, at the times of that commit, memcpy was sufficient:
> struct _virStorageEncryptionSecret {
> int type; /* virStorageEncryptionSecretType */
> unsigned char uuid[VIR_UUID_BUFLEN];
> };
> 
> So this actually fixes commit 47e88b33b which switched to using
> virSecretLookupDef.

Oh, okay, I didn't look as closely into the history. I'll update the
problematic commit.



Re: [PATCH] qemu: Don't crash when getting targets for a multipath

2020-03-19 Thread Michal Prívozník
On 19. 3. 2020 17:43, Peter Krempa wrote:
> On Thu, Mar 19, 2020 at 17:22:37 +0100, Michal Privoznik wrote:
>> In one of my previous commits I've introduced code that creates
>> all devices for given (possible) multipath target. But I've made
>> a mistake there - the code accesses src->path without checking if
>> the disk source is local. Note that the path is NULL if the
>> source is not local.
> 
> Well next->path 'may' be NULL if it's not local, but in this case it is
> local because NVMe disks are local, but they don't have the path.

Local as in virStorageSourceIsLocalStorage() == true. And for NVMe we
return false exactly because src->path has to be NULL (it can't be set
to any meaningfull path).

How about this formulation:

Note that the path is NULL if the
source is not local as viewed by
virStorageSourceIsLocalStorage().

> 
>>
>> Fixes: a30078cb832646177defd256e77c632905f1e6d0
>> Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1814947
>>
>> Signed-off-by: Michal Privoznik 
>> ---
>>  src/qemu/qemu_domain.c | 16 
>>  1 file changed, 8 insertions(+), 8 deletions(-)
> 
> Once the commit message makes sense:
> 
> Reviewed-by: Peter Krempa 
> 
>>
>> diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c
>> index 0e2252f6cf..ef17829235 100644
>> --- a/src/qemu/qemu_domain.c
>> +++ b/src/qemu/qemu_domain.c
>> @@ -15862,19 +15862,19 @@ qemuDomainNamespaceSetupDisk(virDomainObjPtr vm,
>>  }
>>  
>>  tmpPath = g_strdup(next->path);
>> +
>> +if (virDevMapperGetTargets(next->path, &targetPaths) < 0 &&
>> +errno != ENOSYS && errno != EBADF) {
>> +virReportSystemError(errno,
>> + _("Unable to get devmapper targets for 
>> %s"),
>> + next->path);
>> +return -1;
>> +}
>>  }
>>  
>>  if (virStringListAdd(&paths, tmpPath) < 0)
>>  return -1;
>>  
>> -if (virDevMapperGetTargets(next->path, &targetPaths) < 0 &&
>> -errno != ENOSYS && errno != EBADF) {
>> -virReportSystemError(errno,
>> - _("Unable to get devmapper targets for 
>> %s"),
>> - next->path);
>> -return -1;
>> -}
>> -
>>  if (virStringListMerge(&paths, &targetPaths) < 0)
>>  return -1;
> 
> Is this supposed to go after 'tmpPath' in the list? Otherwise you might
> want to move it toghether with the call to virDevMapperGetTargets.
> 

Good point, the order doesn't matter.

Michal



Re: [PATCH 2/2] virStorageEncryptionSecretCopy: Properly copy internals

2020-03-19 Thread Ján Tomko

On a Thursday in 2020, Peter Krempa wrote:

On Thu, Mar 19, 2020 at 18:14:15 +0100, Ján Tomko wrote:

On a Thursday in 2020, Peter Krempa wrote:
> virStorageEncryptionSecretPtr may have a string inside it, thus we must
> copy the string too. Use virSecretLookupDefCopy to do that.
>
> Likely caused by 756b46ddd24.

Please remove the period from the end.

Also, at the times of that commit, memcpy was sufficient:
struct _virStorageEncryptionSecret {
int type; /* virStorageEncryptionSecretType */
unsigned char uuid[VIR_UUID_BUFLEN];
};

So this actually fixes commit 47e88b33b which switched to using
virSecretLookupDef.


Oh, okay, I didn't look as closely into the history. I'll update the
problematic commit.



Well, 756b46ddd24 is also problematic - something like this is easy to
overlook.

Jano


signature.asc
Description: PGP signature


Re: [PATCH] qemu: Don't crash when getting targets for a multipath

2020-03-19 Thread Peter Krempa
On Thu, Mar 19, 2020 at 18:39:58 +0100, Michal Privoznik wrote:
> On 19. 3. 2020 17:43, Peter Krempa wrote:
> > On Thu, Mar 19, 2020 at 17:22:37 +0100, Michal Privoznik wrote:
> >> In one of my previous commits I've introduced code that creates
> >> all devices for given (possible) multipath target. But I've made
> >> a mistake there - the code accesses src->path without checking if
> >> the disk source is local. Note that the path is NULL if the
> >> source is not local.
> > 
> > Well next->path 'may' be NULL if it's not local, but in this case it is
> > local because NVMe disks are local, but they don't have the path.
> 
> Local as in virStorageSourceIsLocalStorage() == true. And for NVMe we
> return false exactly because src->path has to be NULL (it can't be set
> to any meaningfull path).

Yes. The meaning of virStorageSourceIsLocalStorage shifted after NVMe
disks were added.

> 
> How about this formulation:
> 
> Note that the path is NULL if the
> source is not local as viewed by
> virStorageSourceIsLocalStorage().

That is still misleading, because network disks can have non-null path.

I'd prefer:

'next->path' is NULL/doesn't make sense for VIR_STORAGE_TYPE_NVME



Re: [PATCH 2/6] qemuDomainBuildNamespace: Try harder to remove temp directories

2020-03-19 Thread Pavel Mores
On Wed, Mar 18, 2020 at 06:32:12PM +0100, Michal Privoznik wrote:
> If building namespace fails somewhere in the middle (that is some
> files exists under devMountsSavePath[i]), then plain rmdir() is
> not enough to remove dir. Umount the temp location and use
> virFileDeleteTree() to remove the directory.
> 
> Signed-off-by: Michal Privoznik 
> ---
>  src/qemu/qemu_domain.c | 5 -
>  1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c
> index 48bf5ae559..2724607311 100644
> --- a/src/qemu/qemu_domain.c
> +++ b/src/qemu/qemu_domain.c
> @@ -15311,9 +15311,12 @@ qemuDomainBuildNamespace(virQEMUDriverConfigPtr cfg,
>  ret = 0;
>   cleanup:
>  for (i = 0; i < ndevMountsPath; i++) {
> +#if defined(__linux__)
> +umount(devMountsSavePath[i]);
> +#endif /* defined(__linux__) */
>  /* The path can be either a regular file or a dir. */
>  if (virFileIsDir(devMountsSavePath[i]))
> -rmdir(devMountsSavePath[i]);
> +virFileDeleteTree(devMountsSavePath[i]);
>  else
>  unlink(devMountsSavePath[i]);
>  }
> -- 
> 2.24.1
> 

Reviewed-by: Pavel Mores 



Re: [PULL v2 00/11] Bitmaps patches

2020-03-19 Thread Peter Maydell
On Wed, 18 Mar 2020 at 20:24, John Snow  wrote:
>
> The following changes since commit d649689a8ecb2e276cc20d3af6d416e3c299cb17:
>
>   Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into 
> staging (2020-03-17 18:33:05 +)
>
> are available in the Git repository at:
>
>   https://github.com/jnsnow/qemu.git tags/bitmaps-pull-request
>
> for you to fetch changes up to 2d00cbd8e222a4adc08f415c399e84590ee8ff9a:
>
>   block/qcow2-bitmap: use bdrv_dirty_bitmap_next_dirty (2020-03-18 14:03:46 
> -0400)
>
> 
> Pull request
>
> 


Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/5.0
for any user-visible changes.

-- PMM




Re: [PULL v2 00/11] Bitmaps patches

2020-03-19 Thread John Snow



On 3/19/20 1:57 PM, Peter Maydell wrote:
> On Wed, 18 Mar 2020 at 20:24, John Snow  wrote:
>>
>> The following changes since commit d649689a8ecb2e276cc20d3af6d416e3c299cb17:
>>
>>   Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into 
>> staging (2020-03-17 18:33:05 +)
>>
>> are available in the Git repository at:
>>
>>   https://github.com/jnsnow/qemu.git tags/bitmaps-pull-request
>>
>> for you to fetch changes up to 2d00cbd8e222a4adc08f415c399e84590ee8ff9a:
>>
>>   block/qcow2-bitmap: use bdrv_dirty_bitmap_next_dirty (2020-03-18 14:03:46 
>> -0400)
>>
>> 
>> Pull request
>>
>> 
> 
> 
> Applied, thanks.
> 

Wonderful, thanks!

> Please update the changelog at https://wiki.qemu.org/ChangeLog/5.0
> for any user-visible changes.
> 
> -- PMM

Will do.



Re: [PATCH] Minor doc fix

2020-03-19 Thread Ján Tomko

On a Thursday in 2020, Sebastian Mitterle wrote:

1. Fix link to knowledge base article
2. Use  to make sure kbase.html has page title


These are actually two fixes.
I've split them up using these bullet points as commit messages.



Signed-off-by Sebastian Mitterle 


You can use 'git commit -s' to add your sign-off line.


---
docs/formatbackup.html.in | 2 +-
docs/kbase.html.in| 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)



Reviewed-by: Ján Tomko 
and pushed

Jano


signature.asc
Description: PGP signature


Re: [PATCH 5/6] virprocess: Passthru error from virProcessRunInForkHelper

2020-03-19 Thread Pavel Mores
On Wed, Mar 18, 2020 at 06:32:15PM +0100, Michal Privoznik wrote:
> When running a function in a forked child, so far the only thing
> we could report is exit status of the child and the error
> message. However, it may be beneficial to the caller to know the
> actual error that happened in the child.
> 
> Signed-off-by: Michal Privoznik 
> ---
>  build-aux/syntax-check.mk |  2 +-
>  src/util/virprocess.c | 51 ---
>  tests/commandtest.c   | 43 +
>  3 files changed, 91 insertions(+), 5 deletions(-)
> 
> diff --git a/build-aux/syntax-check.mk b/build-aux/syntax-check.mk
> index 3020921be8..2a38c03ba9 100644
> --- a/build-aux/syntax-check.mk
> +++ b/build-aux/syntax-check.mk
> @@ -1987,7 +1987,7 @@ exclude_file_name_regexp--sc_flags_usage = \
>  exclude_file_name_regexp--sc_libvirt_unmarked_diagnostics = \
>^(src/rpc/gendispatch\.pl$$|tests/)
>  
> -exclude_file_name_regexp--sc_po_check = ^(docs/|src/rpc/gendispatch\.pl$$)
> +exclude_file_name_regexp--sc_po_check = 
> ^(docs/|src/rpc/gendispatch\.pl$$|tests/commandtest.c$$)
>  
>  exclude_file_name_regexp--sc_prohibit_VIR_ERR_NO_MEMORY = \
>
> ^(build-aux/syntax-check\.mk|include/libvirt/virterror\.h|src/remote/remote_daemon_dispatch\.c|src/util/virerror\.c|docs/internals/oomtesting\.html\.in)$$
> diff --git a/src/util/virprocess.c b/src/util/virprocess.c
> index 24135070b7..e8674402f9 100644
> --- a/src/util/virprocess.c
> +++ b/src/util/virprocess.c
> @@ -1126,6 +1126,23 @@ virProcessRunInMountNamespace(pid_t pid G_GNUC_UNUSED,
>  
>  
>  #ifndef WIN32
> +typedef struct {
> +int code;
> +int domain;
> +char message[VIR_ERROR_MAX_LENGTH];
> +virErrorLevel level;
> +char str1[VIR_ERROR_MAX_LENGTH];
> +char str2[VIR_ERROR_MAX_LENGTH];
> +/* str3 doesn't seem to be used. Ignore it. */
> +int int1;
> +int int2;
> +} errorData;
> +
> +typedef union {
> +errorData data;
> +char bindata[sizeof(errorData)];
> +} errorDataBin;
> +
>  static int
>  virProcessRunInForkHelper(int errfd,
>pid_t ppid,
> @@ -1134,9 +1151,19 @@ virProcessRunInForkHelper(int errfd,
>  {
>  if (cb(ppid, opaque) < 0) {
>  virErrorPtr err = virGetLastError();
> +errorDataBin bin = { 0 };
> +
>  if (err) {
> -size_t len = strlen(err->message) + 1;
> -ignore_value(safewrite(errfd, err->message, len));
> +bin.data.code = err->code;
> +bin.data.domain = err->domain;
> +ignore_value(virStrcpy(bin.data.message, err->message, 
> sizeof(bin.data.message)));
> +bin.data.level = err->level;
> +ignore_value(virStrcpy(bin.data.str1, err->str1, 
> sizeof(bin.data.str1)));
> +ignore_value(virStrcpy(bin.data.str2, err->str2, 
> sizeof(bin.data.str2)));
> +bin.data.int1 = err->int1;
> +bin.data.int2 = err->int2;
> +
> +ignore_value(safewrite(errfd, bin.bindata, sizeof(bin)));
>  }
>  
>  return -1;
> @@ -1188,16 +1215,32 @@ virProcessRunInFork(virProcessForkCallback cb,
>  } else {
>  int status;
>  g_autofree char *buf = NULL;
> +errorDataBin bin;
> +int nread;
>  
>  VIR_FORCE_CLOSE(errfd[1]);
> -ignore_value(virFileReadHeaderFD(errfd[0], 1024, &buf));
> +nread = virFileReadHeaderFD(errfd[0], sizeof(bin), &buf);
>  ret = virProcessWait(child, &status, false);
>  if (ret == 0) {
>  ret = status == EXIT_CANCELED ? -1 : status;
>  if (ret) {
>  virReportError(VIR_ERR_INTERNAL_ERROR,
> _("child reported (status=%d): %s"),
> -   status, NULLSTR(buf));
> +   status, NULLSTR(bin.data.message));
> +
> +if (nread == sizeof(bin)) {
> +memcpy(bin.bindata, buf, sizeof(bin));
> +virRaiseErrorFull(__FILE__, __FUNCTION__, __LINE__,
> +  bin.data.domain,
> +  bin.data.code,
> +  bin.data.level,
> +  bin.data.str1,
> +  bin.data.str2,
> +  NULL,
> +  bin.data.int1,
> +  bin.data.int2,
> +  "%s", bin.data.message);
> +}
>  }
>  }
>  }
> diff --git a/tests/commandtest.c b/tests/commandtest.c
> index a64aa9ad33..f4a2c67c05 100644
> --- a/tests/commandtest.c
> +++ b/tests/commandtest.c
> @@ -1257,6 +1257,48 @@ static int test27(const void *unused G_GNUC_UNUSED)
>  }
>  
>  
> +static int
> +test28Callback(pid_t pid G_GNUC_UNUSED,
> +   void *opaque G_GNUC_UNUSED)
> +{
> +virR

Re: libvirt-devaddr: a new library for device address assignment

2020-03-19 Thread Laine Stump
TL;DR - I'm not as anti-XML as the proposal seems to be, but also not 
pro-XML. I also (after thinking about it) understand the advantage of 
putting this in a separate library. So yeah, let's go it!


On 3/13/20 6:47 AM, Daniel P. Berrangé wrote:

On Fri, Mar 13, 2020 at 11:23:44AM +0200, Dan Kenigsberg wrote:

On Wed, 4 Mar 2020, 14:51 Daniel P. Berrangé,  wrote:

We've been doing alot of refactoring of code in recent times, and also
have plans for significant infrastructure changes. We still need to
spend time delivering interesting features to users / applications.
This mail is to introduce an idea for a solution to an specific
area applications have had long term pain with libvirt's current
"mechanism, not policy" approach - device addressing. This is a way
for us to show brand new ideas & approaches for what the libvirt
project can deliver in terms of management APIs.

To set expectations straight: I have written no code for this yet,
merely identified the gap & conceptual solution.


The device addressing problem
=

One of the key jobs libvirt does when processing a new domain XML
configuration is to assign addresses to all devices that are present.
This involves adding various device controllers (PCI bridges, PCI root
ports, IDE/SCSI buses, USB controllers, etc) if they are not already
present, and then assigning PCI, USB, IDE, SCSI, etc, addresses to each
device so they are associated with controllers. When libvirt spawns a
QEMU guest, it will pass full address information to QEMU.

Libvirt, as a general rule, aims to avoid defining and implementing
policy around expansion of guest configuration / defaults, however, it
is inescapable in the case of device addressing due to the need to
guarantee a stable hardware ABI to make live migration and save/restore
to disk work.  The policy that libvirt has implemented for device
addressing is, as much as possible, the same as the addressing scheme
QEMU would apply itself.

While libvirt succeeds in its goal of providing a stable hardware API,
the addressing scheme used is not well suited to all deployment
scenarios of QEMU. This is an inevitable result of having a specific
assignment policy implemented in libvirt which has to trade off mutually
incompatible use cases/goals.

When the libvirt addressing policy is not been sufficient, management
applications are forced to take on address assignment themselves,
which is a massive non-trivial job with many subtle problems to
consider.

Places where libvirt's addressing is insufficient for PCI include

  * Setting up multiple guest NUMA nodes and associating devices to
specific nodes
  * Pre-emptive creation of extra PCIe root ports, to allow for later
device hotplug on PCIe topologies
  * Determining whether to place a device on a PCI or PCIe bridge
  * Controlling whether a device is placed into a hotpluggable slot
  * Controlling whether a PCIe root port supports hotplug or not
  * Determining whether to places all devices on distinct slots or
buses, vs grouping them all into functions on the same slot
  * Ability to expand the device addressing without being on the
hypervisor host

(I don't understand the last bullet point)

I'm not sure if this is still the case, but at some point in time
there was a desire from KubeVirt to be able to expand the users'
configuration when loaded in KubeVirt, filling in various defaults
for devices. This would run when the end user YAML/JSON config
was first posted to the k8s API for storage, some arbitrary amount
of time later the config gets chosen to run on a virtualization
host at which point it is turned into libvirt domain XML.



If I recall the discussion properly, the context was that we wanted 
kubevirt to remember all the stuff like PCI addresses, MAC addresses, 
exact machinetype to be "backfilled" from libvirt into the Kubevirt 
config, but for them that's a one-way street. So having all these things 
set by a separate API (even in a separate library) would definitely be 
an advantage for them, as long as all the same info was available at 
that time (e.g. you really need to know the machinetypes supported by 
the specific qemu that is going to be used in order to set the exact 
machinetype)







Libvirt wishes to avoid implementing many different address assignment
policies. It also wishes to keep the domain XML as a representation
of the virtual hardware, not add a bunch of properties to it which
merely serve as tunable input parameters for device addressing
algorithms.

There is thus a dilemma here. Management applications increasingly
need fine grained control over device addressing, while libvirt
doesn't want to expose fine grained policy controls via the XML.


The new libvirt-devaddr API
===

The way out of this is to define a brand new virt management API
which tackles this specific problem in a way that addresses all the
problems mgmt apps have with device addressing and explicitly
pr

Re: [PATCH 5/6] virprocess: Passthru error from virProcessRunInForkHelper

2020-03-19 Thread Ján Tomko

On a Wednesday in 2020, Michal Privoznik wrote:

When running a function in a forked child, so far the only thing
we could report is exit status of the child and the error
message. However, it may be beneficial to the caller to know the
actual error that happened in the child.

Signed-off-by: Michal Privoznik 
---
build-aux/syntax-check.mk |  2 +-
src/util/virprocess.c | 51 ---
tests/commandtest.c   | 43 +
3 files changed, 91 insertions(+), 5 deletions(-)

diff --git a/build-aux/syntax-check.mk b/build-aux/syntax-check.mk
index 3020921be8..2a38c03ba9 100644
--- a/build-aux/syntax-check.mk
+++ b/build-aux/syntax-check.mk
@@ -1987,7 +1987,7 @@ exclude_file_name_regexp--sc_flags_usage = \
exclude_file_name_regexp--sc_libvirt_unmarked_diagnostics = \
  ^(src/rpc/gendispatch\.pl$$|tests/)

-exclude_file_name_regexp--sc_po_check = ^(docs/|src/rpc/gendispatch\.pl$$)
+exclude_file_name_regexp--sc_po_check = 
^(docs/|src/rpc/gendispatch\.pl$$|tests/commandtest.c$$)

exclude_file_name_regexp--sc_prohibit_VIR_ERR_NO_MEMORY = \
  
^(build-aux/syntax-check\.mk|include/libvirt/virterror\.h|src/remote/remote_daemon_dispatch\.c|src/util/virerror\.c|docs/internals/oomtesting\.html\.in)$$
diff --git a/src/util/virprocess.c b/src/util/virprocess.c
index 24135070b7..e8674402f9 100644
--- a/src/util/virprocess.c
+++ b/src/util/virprocess.c
@@ -1126,6 +1126,23 @@ virProcessRunInMountNamespace(pid_t pid G_GNUC_UNUSED,


#ifndef WIN32
+typedef struct {
+int code;
+int domain;
+char message[VIR_ERROR_MAX_LENGTH];
+virErrorLevel level;
+char str1[VIR_ERROR_MAX_LENGTH];
+char str2[VIR_ERROR_MAX_LENGTH];
+/* str3 doesn't seem to be used. Ignore it. */
+int int1;
+int int2;
+} errorData;
+
+typedef union {
+errorData data;
+char bindata[sizeof(errorData)];
+} errorDataBin;
+
static int
virProcessRunInForkHelper(int errfd,
  pid_t ppid,
@@ -1134,9 +1151,19 @@ virProcessRunInForkHelper(int errfd,
{
if (cb(ppid, opaque) < 0) {
virErrorPtr err = virGetLastError();
+errorDataBin bin = { 0 };
+


Consider allocating this on the heap instead of a 3K+ stack variable.

Jano


if (err) {
-size_t len = strlen(err->message) + 1;
-ignore_value(safewrite(errfd, err->message, len));
+bin.data.code = err->code;
+bin.data.domain = err->domain;
+ignore_value(virStrcpy(bin.data.message, err->message, 
sizeof(bin.data.message)));
+bin.data.level = err->level;
+ignore_value(virStrcpy(bin.data.str1, err->str1, 
sizeof(bin.data.str1)));
+ignore_value(virStrcpy(bin.data.str2, err->str2, 
sizeof(bin.data.str2)));
+bin.data.int1 = err->int1;
+bin.data.int2 = err->int2;
+
+ignore_value(safewrite(errfd, bin.bindata, sizeof(bin)));
}

return -1;
@@ -1188,16 +1215,32 @@ virProcessRunInFork(virProcessForkCallback cb,
} else {
int status;
g_autofree char *buf = NULL;
+errorDataBin bin;


Same here.

Jano


signature.asc
Description: PGP signature


[PATCH 6/6] tests: virstoragetest: validate that array deflattening works for gluster

2020-03-19 Thread Peter Krempa
Validate that we are able to parse back the dotted syntax arrays we were
generating in the pre-blockdev era.

Signed-off-by: Peter Krempa 
---
 tests/virstoragetest.c | 18 ++
 1 file changed, 18 insertions(+)

diff --git a/tests/virstoragetest.c b/tests/virstoragetest.c
index c5954d..547118951e 100644
--- a/tests/virstoragetest.c
+++ b/tests/virstoragetest.c
@@ -1387,6 +1387,24 @@ mymain(void)
 "  \n"
 "  \n"
 "\n");
+TEST_BACKING_PARSE("json:{\"driver\": \"raw\","
+ "\"file\": {\"server.0.host\": \"A.A.A.A\","
+"\"server.1.host\": \"B.B.B.B\","
+"\"server.2.host\": \"C.C.C.C\","
+"\"driver\": \"gluster\","
+"\"path\": \"raw\","
+"\"server.0.type\": \"tcp\","
+"\"server.1.type\": \"tcp\","
+"\"server.2.type\": \"tcp\","
+"\"server.0.port\": \"24007\","
+"\"server.1.port\": \"24007\","
+"\"server.2.port\": \"24007\","
+"\"volume\": \"vol1\"}}",
+   "\n"
+   "  \n"
+   "  \n"
+   "  \n"
+   "\n");
 TEST_BACKING_PARSE("json:{\"file\":{\"driver\":\"nbd\","
"\"path\":\"/path/to/socket\""
   "}"
-- 
2.24.1



[PATCH 3/6] util: json: Extract deflattening of keys into a separate function

2020-03-19 Thread Peter Krempa
Extract the code so that there's a clean separation once we'll want do
do other steps.

Signed-off-by: Peter Krempa 
---
 src/util/virjson.c | 29 -
 1 file changed, 20 insertions(+), 9 deletions(-)

diff --git a/src/util/virjson.c b/src/util/virjson.c
index f308927fa0..65d6521789 100644
--- a/src/util/virjson.c
+++ b/src/util/virjson.c
@@ -2049,6 +2049,10 @@ virJSONStringReformat(const char *jsonstr,
 }


+static virJSONValuePtr
+virJSONValueObjectDeflattenKeys(virJSONValuePtr json);
+
+
 static int
 virJSONValueObjectDeflattenWorker(const char *key,
   virJSONValuePtr value,
@@ -2064,7 +2068,7 @@ virJSONValueObjectDeflattenWorker(const char *key,
 if (!strchr(key, '.')) {

 if (virJSONValueIsObject(value))
-newval = virJSONValueObjectDeflatten(value);
+newval = virJSONValueObjectDeflattenKeys(value);
 else
 newval = virJSONValueCopy(value);

@@ -2113,6 +2117,20 @@ virJSONValueObjectDeflattenWorker(const char *key,
 }


+static virJSONValuePtr
+virJSONValueObjectDeflattenKeys(virJSONValuePtr json)
+{
+g_autoptr(virJSONValue) deflattened = virJSONValueNewObject();
+
+if (virJSONValueObjectForeachKeyValue(json,
+  virJSONValueObjectDeflattenWorker,
+  deflattened) < 0)
+return NULL;
+
+return g_steal_pointer(&deflattened);
+}
+
+
 /**
  * virJSONValueObjectDeflatten:
  *
@@ -2128,12 +2146,5 @@ virJSONValueObjectDeflattenWorker(const char *key,
 virJSONValuePtr
 virJSONValueObjectDeflatten(virJSONValuePtr json)
 {
-g_autoptr(virJSONValue) deflattened = virJSONValueNewObject();
-
-if (virJSONValueObjectForeachKeyValue(json,
-  virJSONValueObjectDeflattenWorker,
-  deflattened) < 0)
-return NULL;
-
-return g_steal_pointer(&deflattened);
+return virJSONValueObjectDeflattenKeys(json);
 }
-- 
2.24.1



[PATCH 0/6] Parse back some legacy backing store strings

2020-03-19 Thread Peter Krempa
See patch 4 for code and 6 for what we try to parse back.

Peter Krempa (6):
  virBitmapNewEmpty: Use g_new0 to allocate and remove error checking
  virJSONValueObjectDeflattenWorker: Refactor cleanup
  util: json: Extract deflattening of keys into a separate function
  virjson: Deflatten arrays generated by the json->commandline generator
  jsontest: Add test cases for deflattening of arrays
  tests: virstoragetest: validate that array deflattening works for
gluster

 src/util/virbitmap.c  |  14 +--
 src/util/virhostcpu.c |   6 +-
 src/util/virjson.c| 105 ++
 src/util/virtpm.c |   3 +-
 tests/virbitmaptest.c |   8 +-
 .../deflatten-dotted-array-in.json|  27 +
 .../deflatten-dotted-array-out.json   |  43 +++
 tests/virjsontest.c   |   1 +
 tests/virstoragetest.c|  18 +++
 9 files changed, 182 insertions(+), 43 deletions(-)
 create mode 100644 tests/virjsondata/deflatten-dotted-array-in.json
 create mode 100644 tests/virjsondata/deflatten-dotted-array-out.json

-- 
2.24.1



[PATCH 1/6] virBitmapNewEmpty: Use g_new0 to allocate and remove error checking

2020-03-19 Thread Peter Krempa
virBitmapNewEmpty can't fail now so we can make it obvious and fix all
callers.

Signed-off-by: Peter Krempa 
---
 src/util/virbitmap.c  | 14 +++---
 src/util/virhostcpu.c |  6 ++
 src/util/virtpm.c |  3 +--
 tests/virbitmaptest.c |  8 ++--
 4 files changed, 8 insertions(+), 23 deletions(-)

diff --git a/src/util/virbitmap.c b/src/util/virbitmap.c
index baf77fe556..d38a2dd7e9 100644
--- a/src/util/virbitmap.c
+++ b/src/util/virbitmap.c
@@ -110,17 +110,12 @@ virBitmapNew(size_t size)
  *
  * Allocate an empty bitmap. It can be used with self-expanding APIs.
  *
- * Returns a pointer to the allocated bitmap or NULL if memory cannot be
- * allocated. Reports libvirt errors.
+ * Returns a pointer to the allocated bitmap.
  */
 virBitmapPtr
 virBitmapNewEmpty(void)
 {
-virBitmapPtr ret;
-
-ignore_value(VIR_ALLOC(ret));
-
-return ret;
+return g_new0(virBitmap, 1);
 }


@@ -610,16 +605,13 @@ virBitmapParse(const char *str,
 virBitmapPtr
 virBitmapParseUnlimited(const char *str)
 {
-virBitmapPtr bitmap;
+virBitmapPtr bitmap = virBitmapNewEmpty();
 bool neg = false;
 const char *cur = str;
 char *tmp;
 size_t i;
 int start, last;

-if (!(bitmap = virBitmapNewEmpty()))
-return NULL;
-
 if (!str)
 goto error;

diff --git a/src/util/virhostcpu.c b/src/util/virhostcpu.c
index 67d3e3308b..41033b7473 100644
--- a/src/util/virhostcpu.c
+++ b/src/util/virhostcpu.c
@@ -336,8 +336,7 @@ virHostCPUParseNode(const char *node,
 goto cleanup;

 /* enumerate sockets in the node */
-if (!(sockets_map = virBitmapNewEmpty()))
-goto cleanup;
+sockets_map = virBitmapNewEmpty();

 while ((direrr = virDirRead(cpudir, &cpudirent, node)) > 0) {
 if (sscanf(cpudirent->d_name, "cpu%u", &cpu) != 1)
@@ -373,8 +372,7 @@ virHostCPUParseNode(const char *node,
 goto cleanup;

 for (i = 0; i < sock_max; i++)
-if (!(cores_maps[i] = virBitmapNewEmpty()))
-goto cleanup;
+cores_maps[i] = virBitmapNewEmpty();

 /* Iterate over all CPUs in the node, in ascending order */
 for (cpu = 0; cpu < npresent_cpus; cpu++) {
diff --git a/src/util/virtpm.c b/src/util/virtpm.c
index 505a4230dd..c734bf941a 100644
--- a/src/util/virtpm.c
+++ b/src/util/virtpm.c
@@ -183,8 +183,7 @@ virTPMExecGetCaps(virCommandPtr cmd,
 if (virCommandRun(cmd, &exitstatus) < 0)
 return NULL;

-if (!(bitmap = virBitmapNewEmpty()))
-return NULL;
+bitmap = virBitmapNewEmpty();

 /* older version does not support --print-capabilties -- that's fine */
 if (exitstatus != 0) {
diff --git a/tests/virbitmaptest.c b/tests/virbitmaptest.c
index 2808d9c880..1c7dc1d610 100644
--- a/tests/virbitmaptest.c
+++ b/tests/virbitmaptest.c
@@ -193,8 +193,7 @@ test4(const void *data G_GNUC_UNUSED)

 /* 0. empty set */

-if (!(bitmap = virBitmapNewEmpty()))
-goto error;
+bitmap = virBitmapNewEmpty();

 if (virBitmapNextSetBit(bitmap, -1) != -1)
 goto error;
@@ -632,12 +631,9 @@ test11(const void *opaque)
 static int
 test12(const void *opaque G_GNUC_UNUSED)
 {
-virBitmapPtr map = NULL;
+virBitmapPtr map = virBitmapNewEmpty();
 int ret = -1;

-if (!(map = virBitmapNewEmpty()))
-return -1;
-
 TEST_MAP(0, "");

 if (virBitmapSetBitExpand(map, 128) < 0)
-- 
2.24.1



[PATCH 2/6] virJSONValueObjectDeflattenWorker: Refactor cleanup

2020-03-19 Thread Peter Krempa
Use automatic memory handling to remove the cleanup section.

Signed-off-by: Peter Krempa 
---
 src/util/virjson.c | 27 +++
 1 file changed, 11 insertions(+), 16 deletions(-)

diff --git a/src/util/virjson.c b/src/util/virjson.c
index 2d7368b0b6..f308927fa0 100644
--- a/src/util/virjson.c
+++ b/src/util/virjson.c
@@ -2055,11 +2055,10 @@ virJSONValueObjectDeflattenWorker(const char *key,
   void *opaque)
 {
 virJSONValuePtr retobj = opaque;
-virJSONValuePtr newval = NULL;
+g_autoptr(virJSONValue) newval = NULL;
 virJSONValuePtr existobj;
-char **tokens = NULL;
+VIR_AUTOSTRINGLIST tokens = NULL;
 size_t ntokens = 0;
-int ret = -1;

 /* non-nested keys only need to be copied */
 if (!strchr(key, '.')) {
@@ -2075,46 +2074,42 @@ virJSONValueObjectDeflattenWorker(const char *key,
 if (virJSONValueObjectHasKey(retobj, key)) {
 virReportError(VIR_ERR_INVALID_ARG,
_("can't deflatten colliding key '%s'"), key);
-goto cleanup;
+return -1;
 }

 if (virJSONValueObjectAppend(retobj, key, newval) < 0)
-goto cleanup;
+return -1;
+
+newval = NULL;

 return 0;
 }

 if (!(tokens = virStringSplitCount(key, ".", 2, &ntokens)))
-goto cleanup;
+return -1;

 if (ntokens != 2) {
 virReportError(VIR_ERR_INVALID_ARG,
_("invalid nested value key '%s'"), key);
-goto cleanup;
+return -1;
 }

 if (!(existobj = virJSONValueObjectGet(retobj, tokens[0]))) {
 existobj = virJSONValueNewObject();

 if (virJSONValueObjectAppend(retobj, tokens[0], existobj) < 0)
-goto cleanup;
+return -1;

 } else {
 if (!virJSONValueIsObject(existobj)) {
 virReportError(VIR_ERR_INVALID_ARG, "%s",
_("mixing nested objects and values is forbidden in 
"
  "JSON deflattening"));
-goto cleanup;
+return -1;
 }
 }

-ret = virJSONValueObjectDeflattenWorker(tokens[1], value, existobj);
-
- cleanup:
-virStringListFreeCount(tokens, ntokens);
-virJSONValueFree(newval);
-
-return ret;
+return virJSONValueObjectDeflattenWorker(tokens[1], value, existobj);
 }


-- 
2.24.1



[PATCH 4/6] virjson: Deflatten arrays generated by the json->commandline generator

2020-03-19 Thread Peter Krempa
For the few instances where we'd generate an array in dotted syntax we
should be able to parse it back. Add another step in deflattening of the
dotted syntax which reconstructs the arrays so that the backing store
parser can parse it.

https://bugzilla.redhat.com/show_bug.cgi?id=1466177

Signed-off-by: Peter Krempa 
---
 src/util/virjson.c | 61 +-
 1 file changed, 60 insertions(+), 1 deletion(-)

diff --git a/src/util/virjson.c b/src/util/virjson.c
index 65d6521789..be472d49e4 100644
--- a/src/util/virjson.c
+++ b/src/util/virjson.c
@@ -2131,6 +2131,58 @@ virJSONValueObjectDeflattenKeys(virJSONValuePtr json)
 }


+/**
+ * virJSONValueObjectDeflattenArrays:
+ *
+ * Reconstruct JSON arrays from objects which only have sequential numeric
+ * keys starting from 0.
+ */
+static void
+virJSONValueObjectDeflattenArrays(virJSONValuePtr json)
+{
+g_autofree virJSONValuePtr *arraymembers = NULL;
+virJSONObjectPtr obj;
+size_t i;
+
+if (!json ||
+json->type != VIR_JSON_TYPE_OBJECT)
+return;
+
+obj = &json->data.object;
+
+arraymembers = g_new0(virJSONValuePtr, obj->npairs);
+
+for (i = 0; i < obj->npairs; i++)
+virJSONValueObjectDeflattenArrays(obj->pairs[i].value);
+
+for (i = 0; i < obj->npairs; i++) {
+virJSONObjectPairPtr pair = obj->pairs + i;
+unsigned int keynum;
+
+if (virStrToLong_uip(pair->key, NULL, 10, &keynum) < 0)
+return;
+
+if (keynum >= obj->npairs)
+return;
+
+if (arraymembers[keynum])
+return;
+
+arraymembers[keynum] = pair->value;
+}
+
+for (i = 0; i < obj->npairs; i++)
+g_free(obj->pairs[i].key);
+
+g_free(json->data.object.pairs);
+
+i = obj->npairs;
+json->type = VIR_JSON_TYPE_ARRAY;
+json->data.array.nvalues = i;
+json->data.array.values = g_steal_pointer(&arraymembers);
+}
+
+
 /**
  * virJSONValueObjectDeflatten:
  *
@@ -2146,5 +2198,12 @@ virJSONValueObjectDeflattenKeys(virJSONValuePtr json)
 virJSONValuePtr
 virJSONValueObjectDeflatten(virJSONValuePtr json)
 {
-return virJSONValueObjectDeflattenKeys(json);
+virJSONValuePtr deflattened;
+
+if (!(deflattened = virJSONValueObjectDeflattenKeys(json)))
+return NULL;
+
+virJSONValueObjectDeflattenArrays(deflattened);
+
+return deflattened;
 }
-- 
2.24.1



[PATCH 5/6] jsontest: Add test cases for deflattening of arrays

2020-03-19 Thread Peter Krempa
Signed-off-by: Peter Krempa 
---
 .../deflatten-dotted-array-in.json| 27 
 .../deflatten-dotted-array-out.json   | 43 +++
 tests/virjsontest.c   |  1 +
 3 files changed, 71 insertions(+)
 create mode 100644 tests/virjsondata/deflatten-dotted-array-in.json
 create mode 100644 tests/virjsondata/deflatten-dotted-array-out.json

diff --git a/tests/virjsondata/deflatten-dotted-array-in.json 
b/tests/virjsondata/deflatten-dotted-array-in.json
new file mode 100644
index 00..06486a8f38
--- /dev/null
+++ b/tests/virjsondata/deflatten-dotted-array-in.json
@@ -0,0 +1,27 @@
+{
+"valid": {
+"0": "test",
+"1": { "something": 123 },
+"2": true
+},
+"outoforder": {
+"1": { "something": 123 },
+"2": true,
+"0": "test"
+},
+"invalid-overflow": {
+"1": { "something": 123 },
+"2": true,
+"4": "test"
+},
+"invalid-strings": {
+"1": { "something": 123 },
+"2": true,
+"test": "test"
+},
+"nestedkeys": {
+"test.0.test": 123,
+"test.2.test": 123,
+"test.1.test": 123
+}
+}
diff --git a/tests/virjsondata/deflatten-dotted-array-out.json 
b/tests/virjsondata/deflatten-dotted-array-out.json
new file mode 100644
index 00..b32b4b14a9
--- /dev/null
+++ b/tests/virjsondata/deflatten-dotted-array-out.json
@@ -0,0 +1,43 @@
+{
+  "valid": [
+"test",
+{
+  "something": 123
+},
+true
+  ],
+  "outoforder": [
+"test",
+{
+  "something": 123
+},
+true
+  ],
+  "invalid-overflow": {
+"1": {
+  "something": 123
+},
+"2": true,
+"4": "test"
+  },
+  "invalid-strings": {
+"1": {
+  "something": 123
+},
+"2": true,
+"test": "test"
+  },
+  "nestedkeys": {
+"test": [
+  {
+"test": 123
+  },
+  {
+"test": 123
+  },
+  {
+"test": 123
+  }
+]
+  }
+}
diff --git a/tests/virjsontest.c b/tests/virjsontest.c
index 9da0f9c90d..77ca6b449b 100644
--- a/tests/virjsontest.c
+++ b/tests/virjsontest.c
@@ -609,6 +609,7 @@ mymain(void)
 DO_TEST_DEFLATTEN("concat", true);
 DO_TEST_DEFLATTEN("concat-double-key", false);
 DO_TEST_DEFLATTEN("qemu-sheepdog", true);
+DO_TEST_DEFLATTEN("dotted-array", true);

 return (ret == 0) ? EXIT_SUCCESS : EXIT_FAILURE;
 }
-- 
2.24.1



[PATCH v1 1/4] qemu_domain.c: do not launch ppc64 guests with APIC-EOI setting

2020-03-19 Thread Daniel Henrique Barboza
The "" feature, although it's not available for pseries,
can be declared in the domain XML of ppc64 guests without errors.
But setting its 'eoi' attribute will break QEMU. For "":

qemu-kvm: Expected key=value format, found +kvm_pv_eoi

A similar error happens with eoi='off'.

One can argue that it's better to simply forbid launching ppc64
guests with "" declared in the ppc64 XML - it is a feature that
the architecture doesn't support and this would make it clearer
about it. This is sensible, but there are ppc64 guests that are running
with "" declared in the domain (and A LOT of guests running with
"" for that matter, probably reminiscent of x86 templates that
were reused for ppc64) that will stop working if we go this route.

A more subtle approach is to detect if the 'eoi' element is being set
for ppc64 guests, and warn the user about it with a better error
message than the one QEMU provides. This is the new error message
when any value is set for the 'eoi' element:

error: unsupported configuration: The 'eoi' attribute of the 'apic'
feature is not supported for architecture 'ppc64' or machine type
'pseries'.

Signed-off-by: Daniel Henrique Barboza 
---
 src/qemu/qemu_domain.c | 20 +++-
 1 file changed, 19 insertions(+), 1 deletion(-)

diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c
index edc8ba2ddb..381b62b96a 100644
--- a/src/qemu/qemu_domain.c
+++ b/src/qemu/qemu_domain.c
@@ -5262,8 +5262,26 @@ qemuDomainDefValidateFeatures(const virDomainDef *def,
 }
 break;
 
-case VIR_DOMAIN_FEATURE_ACPI:
 case VIR_DOMAIN_FEATURE_APIC:
+/* the  declaration is harmless for ppc64, but
+ * its 'eoi' attribute isn't. To detect if 'eoi' was
+ * present in the XML we can check the tristate switch
+ * of def->apic_eoi */
+if (def->features[i] != VIR_TRISTATE_SWITCH_ABSENT &&
+def->apic_eoi != VIR_TRISTATE_SWITCH_ABSENT &&
+ARCH_IS_PPC64(def->os.arch)) {
+virReportError(VIR_ERR_CONFIG_UNSUPPORTED,
+   _("The 'eoi' attribute of the '%s' feature "
+ "is not supported for architecture '%s' or "
+ "machine type '%s'"),
+ featureName,
+ virArchToString(def->os.arch),
+ def->os.machine);
+ return -1;
+}
+break;
+
+case VIR_DOMAIN_FEATURE_ACPI:
 case VIR_DOMAIN_FEATURE_PAE:
 case VIR_DOMAIN_FEATURE_HAP:
 case VIR_DOMAIN_FEATURE_VIRIDIAN:
-- 
2.25.1




[PATCH v1 3/4] qemu_domain.c: do not launch ppc64 guests with 'pmu' setting

2020-03-19 Thread Daniel Henrique Barboza
The Perfomance Monitoring Unit (PMU) feature is not available for
the Power architecture. The "" feature will always have a value
'on' or 'off' after saving the domain XML, and both will be rejected
by QEMU when launching. This is the error message for
"":

qemu-kvm: can't apply global host-powerpc64-cpu.pmu=on: Property '.pmu' not 
found

A similar error message is thrown for "".

This patch prevents the pseries guest from launching with any
pmu setting with a more informative error message:

error: unsupported configuration: The 'pmu' feature is not
supported for architecture 'ppc64' or machine type 'pseries'

Signed-off-by: Daniel Henrique Barboza 
---
 src/qemu/qemu_domain.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c
index c67abec778..2e5f987a04 100644
--- a/src/qemu/qemu_domain.c
+++ b/src/qemu/qemu_domain.c
@@ -5282,6 +5282,7 @@ qemuDomainDefValidateFeatures(const virDomainDef *def,
 break;
 
 case VIR_DOMAIN_FEATURE_PVSPINLOCK:
+case VIR_DOMAIN_FEATURE_PMU:
 if (def->features[i] != VIR_TRISTATE_SWITCH_ABSENT &&
 ARCH_IS_PPC64(def->os.arch)) {
 virReportError(VIR_ERR_CONFIG_UNSUPPORTED,
@@ -5301,7 +5302,6 @@ qemuDomainDefValidateFeatures(const virDomainDef *def,
 case VIR_DOMAIN_FEATURE_PRIVNET:
 case VIR_DOMAIN_FEATURE_HYPERV:
 case VIR_DOMAIN_FEATURE_CAPABILITIES:
-case VIR_DOMAIN_FEATURE_PMU:
 case VIR_DOMAIN_FEATURE_MSRS:
 case VIR_DOMAIN_FEATURE_LAST:
 break;
-- 
2.25.1




[PATCH v1 2/4] qemu_domain.c: do not launch ppc64 guests with 'pvspinlock' setting

2020-03-19 Thread Daniel Henrique Barboza
PowerPC does not support the 'pvspinlock' feature. The ""
declaration will always have a value 'on' or 'off', and both will
break QEMU when launching. This is the error message for
"":

qemu-kvm: Expected key=value format, found +kvm_pv_unhalt

A similar error message is thrown for "".

This patch prevents the pseries guest from launching with any
pvspinlock setting with a more informative error message:

error: unsupported configuration: The 'pvspinlock' feature is not
supported for architecture 'ppc64' or machine type 'pseries'

Signed-off-by: Daniel Henrique Barboza 
---
 src/qemu/qemu_domain.c | 14 +-
 1 file changed, 13 insertions(+), 1 deletion(-)

diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c
index 381b62b96a..c67abec778 100644
--- a/src/qemu/qemu_domain.c
+++ b/src/qemu/qemu_domain.c
@@ -5281,13 +5281,25 @@ qemuDomainDefValidateFeatures(const virDomainDef *def,
 }
 break;
 
+case VIR_DOMAIN_FEATURE_PVSPINLOCK:
+if (def->features[i] != VIR_TRISTATE_SWITCH_ABSENT &&
+ARCH_IS_PPC64(def->os.arch)) {
+virReportError(VIR_ERR_CONFIG_UNSUPPORTED,
+   _("The '%s' feature is not supported for "
+ "architecture '%s' or machine type '%s'"),
+ featureName,
+ virArchToString(def->os.arch),
+ def->os.machine);
+ return -1;
+}
+break;
+
 case VIR_DOMAIN_FEATURE_ACPI:
 case VIR_DOMAIN_FEATURE_PAE:
 case VIR_DOMAIN_FEATURE_HAP:
 case VIR_DOMAIN_FEATURE_VIRIDIAN:
 case VIR_DOMAIN_FEATURE_PRIVNET:
 case VIR_DOMAIN_FEATURE_HYPERV:
-case VIR_DOMAIN_FEATURE_PVSPINLOCK:
 case VIR_DOMAIN_FEATURE_CAPABILITIES:
 case VIR_DOMAIN_FEATURE_PMU:
 case VIR_DOMAIN_FEATURE_MSRS:
-- 
2.25.1




[PATCH v1 4/4] qemu_domain.c: do not launch ppc64 guests with Hyperv settings

2020-03-19 Thread Daniel Henrique Barboza
Hyperv features aren't supported in QEMU for ppc64. The 
declaration in the XML by itself is benign, but any of its 14 current
features will break QEMU with an error like this:

qemu-kvm: Expected key=value format, found hv_relaxed

This is a more extreme case than the one for apic eoi because we
would need an extra 'switch' statement, with all current Hyperv
features in the body of qemuDomainDefValidateFeatures(), to
check if the user attempted to activate any of them. It's easier to
simply fail to launch with any 'hyperv' declaration in the XML for
ppc64 guests.

A fair disclaimer about Windows and PowerPC: the last Windows version
that ran in the architecture is the hall of famer Windows NT 4.0,
launched in 1996 and with end of extended support for the Server
version in 2004 [1]. I am acknowledging that there might be Windows
NT 4.0 users running in PowerPC, but not enough people running it
under KVM/QEMU to justify Libvirt allowing 'hyperv' to exist in the
domain XML of ppc64 domains.

[1] https://en.wikipedia.org/wiki/Windows_NT_4.0

Signed-off-by: Daniel Henrique Barboza 
---
 src/qemu/qemu_domain.c | 13 -
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c
index 2e5f987a04..ba458bfde9 100644
--- a/src/qemu/qemu_domain.c
+++ b/src/qemu/qemu_domain.c
@@ -5295,12 +5295,23 @@ qemuDomainDefValidateFeatures(const virDomainDef *def,
 }
 break;
 
+case VIR_DOMAIN_FEATURE_HYPERV:
+if (def->features[i] != VIR_TRISTATE_SWITCH_ABSENT &&
+ARCH_IS_PPC64(def->os.arch)) {
+virReportError(VIR_ERR_CONFIG_UNSUPPORTED,
+   _("Hyperv features are not supported for "
+ "architecture '%s' or machine type '%s'"),
+ virArchToString(def->os.arch),
+ def->os.machine);
+ return -1;
+}
+break;
+
 case VIR_DOMAIN_FEATURE_ACPI:
 case VIR_DOMAIN_FEATURE_PAE:
 case VIR_DOMAIN_FEATURE_HAP:
 case VIR_DOMAIN_FEATURE_VIRIDIAN:
 case VIR_DOMAIN_FEATURE_PRIVNET:
-case VIR_DOMAIN_FEATURE_HYPERV:
 case VIR_DOMAIN_FEATURE_CAPABILITIES:
 case VIR_DOMAIN_FEATURE_MSRS:
 case VIR_DOMAIN_FEATURE_LAST:
-- 
2.25.1




[PATCH v1 0/4] Prevent ppc64 guest launch with broken features

2020-03-19 Thread Daniel Henrique Barboza
Hi,

This work was intended to fix a bug with APIC-EOI setting
only (patch 1). I decided to take a closer look and ended up
handling  more cases.

I consider the first 3 patches to be straightforward: those
are conditions that QEMU will complain about and I'm simply
refusing to launch while giving a better error message than
the one QEMU provides.

Patch 4 is more debatable.  declaration in
the XML is benign, but any of the 14 features being declared
will cause QEMU errors. Instead of putting more code (i.e.
a switch with 14 features in the middle of the code) to
prevent any of those features to be enabled, I made a call
to simply refuse to launch with any  setting in
the XML. This is not because I have a beef with Windows
or MS or anything. This is about making an educated guess that
it's highly unlikable that anyone is running ppc64 guests
with  declared in the XML, and it's not worth all
the code to support it. But I'm all ears to hear otherwise.


After this series, these are the features that aren't supported by
ppc64 but can be declared in a ppc64 guest without resulting
in a failed launch:


   
   (without 'eoi' setting)
  
  
  



Daniel Henrique Barboza (4):
  qemu_domain.c: do not launch ppc64 guests with APIC-EOI setting
  qemu_domain.c: do not launch ppc64 guests with 'pvspinlock' setting
  qemu_domain.c: do not launch ppc64 guests with 'pmu' setting
  qemu_domain.c: do not launch ppc64 guests with Hyperv settings

 src/qemu/qemu_domain.c | 49 ++
 1 file changed, 45 insertions(+), 4 deletions(-)

-- 
2.25.1




listen type socket support for rdp graphics

2020-03-19 Thread Vasiliy Tolstov
Hi! How hard implement listen type socket support for rdp graphics? VNC and
SPICE already have such options?
And does this support for rdp available for qemu driver? (I need to access
qemu vm via rdp)
Thanks!

-- 
Vasiliy Tolstov,
e-mail: v.tols...@selfip.ru