Re: [Qemu-block] [Qemu-devel] [PATCH] Replacing (and removing) get_ticks_per_sec() function with NANOSECONDS_PER_SECOND Signed-off-by: Rutuja Shah

2016-03-10 Thread rutuja shah 
Thanks Eric. I will keep these points in mind while sending patches.
Regards
Rutuja Shah


On Fri, Mar 11, 2016 at 2:40 AM, Eric Blake  wrote:
> On 03/10/2016 12:30 PM, rutu.shah...@gmail.com wrote:
>> From: Rutuja Shah 
>>
>
> Your commit message body was botched, cramming everything into the
> subject line.  Be sure you have a one-line summary (preferably shorter
> than 60 characters), then a blank line, before the rest of your
> description and S-o-b.  Also, it's good to say "why" in the commit body,
> not just "what".  Something like:
>
> maint: Drop unused get_ticks_per_sec()
>
> Replace the use of get_ticks_per_sec() with NANOSECONDS_PER_SECOND,
> because...
>
> Signed-off-by: Rutuja Shah 
>
> --
> Eric Blake   eblake redhat com+1-919-301-3266
> Libvirt virtualization library http://libvirt.org
>

Re: [Qemu-block] [PATCH v12 1/3] Add new block driver interface to add/delete a BDS's child

2016-03-10 Thread Changlong Xie 

On 03/10/2016 10:57 PM, Alberto Garcia wrote:

On Thu 10 Mar 2016 03:49:39 AM CET, Changlong Xie wrote:

From: Wen Congyang 

In some cases, we want to take a quorum child offline, and take
another child online.

Signed-off-by: Wen Congyang 
Signed-off-by: zhanghailiang 
Signed-off-by: Gonglei 
Signed-off-by: Changlong Xie 
Reviewed-by: Max Reitz 



+if (!tmp) {
+error_setg(errp, "The node %s does not have child named %s",
+   bdrv_get_device_or_node_name(parent_bs),
+   bdrv_get_device_or_node_name(child->bs));
+return;


I think it should be "does not have a child" or "does not have any
child".

With that corrected,


Ok, will fix in next version.

Thanks
-Xie



Reviewed-by: Alberto Garcia 

Berto


.

Re: [Qemu-block] [Qemu-devel] [PATCH] Replacing (and removing) get_ticks_per_sec() function with NANOSECONDS_PER_SECOND Signed-off-by: Rutuja Shah

2016-03-10 Thread Eric Blake 
On 03/10/2016 12:30 PM, rutu.shah...@gmail.com wrote:
> From: Rutuja Shah 
> 

Your commit message body was botched, cramming everything into the
subject line.  Be sure you have a one-line summary (preferably shorter
than 60 characters), then a blank line, before the rest of your
description and S-o-b.  Also, it's good to say "why" in the commit body,
not just "what".  Something like:

maint: Drop unused get_ticks_per_sec()

Replace the use of get_ticks_per_sec() with NANOSECONDS_PER_SECOND,
because...

Signed-off-by: Rutuja Shah 

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

[Qemu-block] [PATCH] Replacing (and removing) get_ticks_per_sec() function with NANOSECONDS_PER_SECOND Signed-off-by: Rutuja Shah

2016-03-10 Thread rutu . shah . 26 
From: Rutuja Shah 

---
 audio/audio.c |  2 +-
 audio/noaudio.c   |  4 ++--
 audio/spiceaudio.c|  2 +-
 audio/wavaudio.c  |  2 +-
 backends/baum.c   |  2 +-
 block/qed.c   |  2 +-
 cpus.c|  6 +++---
 hw/acpi/core.c|  4 ++--
 hw/arm/omap1.c| 14 +++---
 hw/arm/spitz.c|  2 +-
 hw/arm/stellaris.c|  2 +-
 hw/arm/strongarm.c|  2 +-
 hw/audio/adlib.c  |  2 +-
 hw/audio/sb16.c   |  4 ++--
 hw/block/fdc.c|  2 +-
 hw/block/pflash_cfi02.c   |  4 ++--
 hw/bt/hci-csr.c   |  4 ++--
 hw/char/cadence_uart.c|  4 ++--
 hw/char/serial.c  |  6 +++---
 hw/display/vga.c  |  6 +++---
 hw/dma/rc4030.c   |  2 +-
 hw/ide/core.c |  2 +-
 hw/input/hid.c|  2 +-
 hw/input/tsc2005.c|  2 +-
 hw/input/tsc210x.c|  2 +-
 hw/intc/i8259.c   |  2 +-
 hw/misc/arm_sysctl.c  |  2 +-
 hw/misc/macio/cuda.c  | 16 
 hw/misc/macio/macio.c |  2 +-
 hw/net/dp8393x.c  |  2 +-
 hw/ppc/ppc.c  | 18 +-
 hw/ppc/ppc405_uc.c|  4 ++--
 hw/ppc/ppc_booke.c|  2 +-
 hw/sd/sdhci-internal.h|  2 +-
 hw/sparc64/sun4u.c|  4 ++--
 hw/timer/i8254.c  |  4 ++--
 hw/timer/i8254_common.c   |  6 +++---
 hw/timer/mc146818rtc.c|  6 +++---
 hw/timer/omap_gptimer.c   |  2 +-
 hw/timer/omap_synctimer.c |  2 +-
 hw/timer/pl031.c  | 10 +-
 hw/timer/pxa2xx_timer.c   | 14 +++---
 hw/usb/hcd-ehci.c |  4 ++--
 hw/usb/hcd-musb.c |  2 +-
 hw/usb/hcd-ohci.c | 10 +-
 hw/usb/hcd-uhci.c |  6 +++---
 hw/usb/tusb6010.c |  4 ++--
 hw/watchdog/wdt_diag288.c |  2 +-
 hw/watchdog/wdt_ib700.c   |  2 +-
 include/hw/acpi/acpi.h|  2 +-
 include/qemu/timer.h  |  9 ++---
 monitor.c |  4 ++--
 target-ppc/kvm.c  |  4 ++--
 53 files changed, 114 insertions(+), 119 deletions(-)

diff --git a/audio/audio.c b/audio/audio.c
index e841532..8c2c495 100644
--- a/audio/audio.c
+++ b/audio/audio.c
@@ -1870,7 +1870,7 @@ static void audio_init (void)
 conf.period.ticks = 1;
 } else {
 conf.period.ticks =
-muldiv64 (1, get_ticks_per_sec (), conf.period.hertz);
+muldiv64 (1, NANOSECONDS_PER_SECOND, conf.period.hertz);
 }
 
 e = qemu_add_vm_change_state_handler (audio_vm_change_state_handler, s);
diff --git a/audio/noaudio.c b/audio/noaudio.c
index 09588b9..931aa03 100644
--- a/audio/noaudio.c
+++ b/audio/noaudio.c
@@ -49,7 +49,7 @@ static int no_run_out (HWVoiceOut *hw, int live)
 
 now = qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
 ticks = now - no->old_ticks;
-bytes = muldiv64 (ticks, hw->info.bytes_per_second, get_ticks_per_sec ());
+bytes = muldiv64 (ticks, hw->info.bytes_per_second, 
NANOSECONDS_PER_SECOND);
 bytes = audio_MIN (bytes, INT_MAX);
 samples = bytes >> hw->info.shift;
 
@@ -106,7 +106,7 @@ static int no_run_in (HWVoiceIn *hw)
 int64_t now = qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
 int64_t ticks = now - no->old_ticks;
 int64_t bytes =
-muldiv64 (ticks, hw->info.bytes_per_second, get_ticks_per_sec ());
+muldiv64 (ticks, hw->info.bytes_per_second, 
NANOSECONDS_PER_SECOND);
 
 no->old_ticks = now;
 bytes = audio_MIN (bytes, INT_MAX);
diff --git a/audio/spiceaudio.c b/audio/spiceaudio.c
index 297fd41..a098057 100644
--- a/audio/spiceaudio.c
+++ b/audio/spiceaudio.c
@@ -104,7 +104,7 @@ static int rate_get_samples (struct audio_pcm_info *info, 
SpiceRateCtl *rate)
 
 now = qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
 ticks = now - rate->start_ticks;
-bytes = muldiv64 (ticks, info->bytes_per_second, get_ticks_per_sec ());
+bytes = muldiv64 (ticks, info->bytes_per_second, NANOSECONDS_PER_SECOND);
 samples = (bytes - rate->bytes_sent) >> info->shift;
 if (samples < 0 || samples > 65536) {
 error_report("Resetting rate control (%" PRId64 " samples)", samples);
diff --git a/audio/wavaudio.c b/audio/wavaudio.c
index 343b1a1..1991487 100644
--- a/audio/wavaudio.c
+++ b/audio/wavaudio.c
@@ -51,7 +51,7 @@ static int wav_run_out (HWVoiceOut *hw, int live)
 int64_t now = qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
 int64_t ticks = now - wav->old_ticks;
 int64_t bytes =
-muldiv64 (ticks, hw->info.bytes_per_second, get_ticks_per_sec ());
+muldiv64 (ticks, hw->info.bytes_per_second, NANOSECONDS_PER_SECOND);
 
 if (bytes > INT_MAX) {
 samples = INT_MAX >> hw->info.shift;
diff --git a/backends/baum.c b/backends/baum.c
index c11320e..20b49f2 100644
--- a/backends/baum.c
+++ b/backends/baum.c
@@ -336,7 +336,7 @@ static int baum_eat_packet(BaumDriverState *baum, const 
uint8_t *buf, int len)
 
 /* Allow 100ms to complete the DisplayData packet */
 timer_mod(baum->cellCoun

[Qemu-block] [PATCH] Replaced get_ticks_per_sec calls with NANOSECONDS_PER_SECOND

2016-03-10 Thread rutuja shah 
Hi,
As there are no callers to get_ticks_per_sec() function, definition of
this function could be removed completely?
---
 backends/baum.c   |  2 +-
 block/qed.c   |  2 +-
 cpus.c|  6 +++---
 hw/acpi/core.c|  4 ++--
 hw/arm/omap1.c| 14 +++---
 hw/arm/spitz.c|  2 +-
 hw/arm/stellaris.c|  2 +-
 hw/arm/strongarm.c|  2 +-
 hw/block/fdc.c|  2 +-
 hw/block/pflash_cfi02.c   |  4 ++--
 hw/bt/hci-csr.c   |  4 ++--
 hw/char/cadence_uart.c|  4 ++--
 hw/char/serial.c  |  6 +++---
 hw/display/vga.c  |  6 +++---
 hw/dma/rc4030.c   |  2 +-
 hw/ide/core.c |  2 +-
 hw/input/hid.c|  2 +-
 hw/input/tsc2005.c|  2 +-
 hw/input/tsc210x.c|  2 +-
 hw/intc/i8259.c   |  2 +-
 hw/misc/arm_sysctl.c  |  2 +-
 hw/misc/macio/cuda.c  | 16 
 hw/misc/macio/macio.c |  2 +-
 hw/net/dp8393x.c  |  2 +-
 hw/ppc/ppc.c  | 18 +-
 hw/ppc/ppc405_uc.c|  4 ++--
 hw/ppc/ppc_booke.c|  2 +-
 hw/sd/sdhci-internal.h|  2 +-
 hw/sparc64/sun4u.c|  4 ++--
 hw/timer/i8254.c  |  4 ++--
 hw/timer/i8254_common.c   |  6 +++---
 hw/timer/mc146818rtc.c|  6 +++---
 hw/timer/omap_gptimer.c   |  2 +-
 hw/timer/omap_synctimer.c |  2 +-
 hw/timer/pl031.c  | 10 +-
 hw/timer/pxa2xx_timer.c   | 14 +++---
 hw/usb/hcd-ehci.c |  4 ++--
 hw/usb/hcd-musb.c |  2 +-
 hw/usb/hcd-ohci.c | 10 +-
 hw/usb/hcd-uhci.c |  6 +++---
 hw/usb/tusb6010.c |  4 ++--
 hw/watchdog/wdt_diag288.c |  2 +-
 hw/watchdog/wdt_ib700.c   |  2 +-
 include/hw/acpi/acpi.h|  2 +-
 include/qemu/timer.h  |  4 ++--
 monitor.c |  4 ++--
 target-ppc/kvm.c  |  4 ++--
 47 files changed, 106 insertions(+), 106 deletions(-)

diff --git a/backends/baum.c b/backends/baum.c
index c11320e..20b49f2 100644
--- a/backends/baum.c
+++ b/backends/baum.c
@@ -336,7 +336,7 @@ static int baum_eat_packet(BaumDriverState *baum,
const uint8_t *buf, int len)

 /* Allow 100ms to complete the DisplayData packet */
 timer_mod(baum->cellCount_timer,
qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL) +
-   get_ticks_per_sec() / 10);
+   NANOSECONDS_PER_SECOND / 10);
 for (i = 0; i < baum->x * baum->y ; i++) {
 EAT(c);
 cells[i] = c;
diff --git a/block/qed.c b/block/qed.c
index 404be1e..6fa7e1f 100644
--- a/block/qed.c
+++ b/block/qed.c
@@ -345,7 +345,7 @@ static void qed_start_need_check_timer(BDRVQEDState *s)
  * migration.
  */
 timer_mod(s->need_check_timer, qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL) +
-   get_ticks_per_sec() * QED_NEED_CHECK_TIMEOUT);
+   NANOSECONDS_PER_SECOND * QED_NEED_CHECK_TIMEOUT);
 }

 /* It's okay to call this multiple times or when no timer is started */
diff --git a/cpus.c b/cpus.c
index bc774e2..17be5de 100644
--- a/cpus.c
+++ b/cpus.c
@@ -275,7 +275,7 @@ void cpu_disable_ticks(void)
fairly approximate, so ignore small variation.
When the guest is idle real and virtual time will be aligned in
the IO wait loop.  */
-#define ICOUNT_WOBBLE (get_ticks_per_sec() / 10)
+#define ICOUNT_WOBBLE (NANOSECONDS_PER_SECOND / 10)

 static void icount_adjust(void)
 {
@@ -326,7 +326,7 @@ static void icount_adjust_vm(void *opaque)
 {
 timer_mod(icount_vm_timer,
qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL) +
-   get_ticks_per_sec() / 10);
+   NANOSECONDS_PER_SECOND / 10);
 icount_adjust();
 }

@@ -665,7 +665,7 @@ void configure_icount(QemuOpts *opts, Error **errp)
 icount_adjust_vm, NULL);
 timer_mod(icount_vm_timer,
qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL) +
-   get_ticks_per_sec() / 10);
+   NANOSECONDS_PER_SECOND / 10);
 }

 /***/
diff --git a/hw/acpi/core.c b/hw/acpi/core.c
index 3d9e5c4..5ea3b3b 100644
--- a/hw/acpi/core.c
+++ b/hw/acpi/core.c
@@ -389,7 +389,7 @@ uint16_t acpi_pm1_evt_get_sts(ACPIREGS *ar)
acpi_pm_tmr_update function uses ns for setting the timer. */
 int64_t d = qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
 if (d >= muldiv64(ar->tmr.overflow_time,
-  get_ticks_per_sec(), PM_TIMER_FREQUENCY)) {
+  NANOSECONDS_PER_SECOND, PM_TIMER_FREQUENCY)) {
 ar->pm1.evt.sts |= ACPI_BITMASK_TIMER_STATUS;
 }
 return ar->pm1.evt.sts;
@@ -483,7 +483,7 @@ void acpi_pm_tmr_update(ACPIREGS *ar, bool enable)

 /* schedule a timer interruption if needed */
 if (enable) {
-expire_time = muldiv64(ar->tmr.overflow_time, get_ticks_per_sec(),
+expire_time = muldiv64(ar->tmr.overflow_time, NANOSECONDS_PER_SECOND,

Re: [Qemu-block] [PATCH v2 2/4] monitor: Separate QUORUM_REPORT_BAD events according to the node name

2016-03-10 Thread Eric Blake 
On 03/10/2016 04:55 AM, Alberto Garcia wrote:
> The QUORUM_REPORT_BAD event is emitted whenever there's an I/O error
> in a child of a Quorum device. This event is emitted at a maximum rate
> of 1 per second. This means that an error in one of the children will
> mask errors in the other children if they happen within the same 1
> second interval.
> 
> This patch modifies qapi_event_throttle_equal() so QUORUM_REPORT_BAD
> events are kept separately if they come from different children.
> 
> Signed-off-by: Alberto Garcia 
> Reviewed-by: Max Reitz 
> ---
>  monitor.c | 9 +
>  1 file changed, 9 insertions(+)
> 
> diff --git a/monitor.c b/monitor.c
> index e99ca8c..c9fe862 100644
> --- a/monitor.c
> +++ b/monitor.c
> @@ -572,6 +572,10 @@ static unsigned int qapi_event_throttle_hash(const void 
> *key)
>  hash += g_str_hash(qdict_get_str(evstate->data, "id"));
>  }
>  
> +if (evstate->event == QAPI_EVENT_QUORUM_REPORT_BAD) {
> +hash += g_str_hash(qdict_get_str(evstate->data, "node-name"));

Is ^= any better than += when computing hashes, so that carry bits
aren't weakening the distribution of bits?  But as long as the
computation is consistent, I'm not too worried, since you were copying
the line above.

Reviewed-by: Eric Blake 

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

[Qemu-block] [PATCH v3 10/10] vnc: allow specifying a custom ACL object name

2016-03-10 Thread Daniel P. Berrange 
The VNC server has historically had support for ACLs to check
both the SASL username and the TLS x509 distinguished name.
The VNC server was responsible for creating the initial ACL,
and the client app was then responsible for populating it with
rules using the HMP 'acl_add' command.

This is not satisfactory for a variety of reasons. There is
no way to populate the ACLs from the command line, users are
forced to use the HMP. With multiple network services all
supporting TLS and ACLs now, it is desirable to be able to
define a single ACL that is referenced by all services.

To address these limitations, two new options are added to the
VNC server CLI. The 'tls-acl' option takes the ID of a QAuthZ
object to use for checking TLS x509 distinguished names, and
the 'sasl-acl' option takes the ID of another object to use for
checking SASL usernames.

In this example, we setup two ACLs. The first allows any client
with a certificate issued by the 'RedHat' organization in the
'London' locality. The second ACL allows clients with either
the 'j...@redhat.com' or  'f...@redhat.com' kerberos usernames.
Both ACLs must pass for the user to be allowed.

$QEMU -object tls-creds-x509,id=tls0,dir=/home/berrange/qemutls,\
  endpoint=server,verify-peer=yes \
  -object authz-simple,id=acl0,policy=deny,\
  rules.0.match=O=RedHat,,L=London,rules.0.policy=allow \
  -object authz-simple,id=acl0,policy=deny,\
  rules.0.match=f...@redhat.com,rules.0.policy=allow \
  rules.0.match=j...@redhat.com,rules.0.policy=allow \
  -vnc 0.0.0.0:1,tls-creds=tls0,tls-acl=tlsacl0,
   sasl,sasl-acl=saslacl0 \
  ...other QEMU args...

Signed-off-by: Daniel P. Berrange 
---
 ui/vnc.c | 73 
 1 file changed, 60 insertions(+), 13 deletions(-)

diff --git a/ui/vnc.c b/ui/vnc.c
index 324512d..7090f0b 100644
--- a/ui/vnc.c
+++ b/ui/vnc.c
@@ -3261,6 +3261,12 @@ static QemuOptsList qemu_vnc_opts = {
 .name = "acl",
 .type = QEMU_OPT_BOOL,
 },{
+.name = "tls-acl",
+.type = QEMU_OPT_STRING,
+},{
+.name = "sasl-acl",
+.type = QEMU_OPT_STRING,
+},{
 .name = "lossy",
 .type = QEMU_OPT_BOOL,
 },{
@@ -3483,6 +3489,10 @@ void vnc_display_open(const char *id, Error **errp)
 int saslErr;
 #endif
 int acl = 0;
+const char *tlsacl;
+#ifdef CONFIG_VNC_SASL
+const char *saslacl;
+#endif
 int lock_key_sync = 1;
 
 if (!vs) {
@@ -3670,6 +3680,21 @@ void vnc_display_open(const char *id, Error **errp)
 }
 }
 acl = qemu_opt_get_bool(opts, "acl", false);
+tlsacl = qemu_opt_get(opts, "tls-acl");
+if (acl && tlsacl) {
+error_setg(errp, "'acl' option is mutually exclusive with the "
+   "'tls-acl' options");
+goto fail;
+}
+
+#ifdef CONFIG_VNC_SASL
+saslacl = qemu_opt_get(opts, "sasl-acl");
+if (acl && saslacl) {
+error_setg(errp, "'acl' option is mutually exclusive with the "
+   "'sasl-acl' options");
+goto fail;
+}
+#endif
 
 share = qemu_opt_get(opts, "share");
 if (share) {
@@ -3699,7 +3724,9 @@ void vnc_display_open(const char *id, Error **errp)
 vs->non_adaptive = true;
 }
 
-if (acl) {
+if (tlsacl) {
+vs->tlsaclname = g_strdup(tlsacl);
+} else if (acl) {
 if (strcmp(vs->id, "default") == 0) {
 vs->tlsaclname = g_strdup("vnc.x509dname");
 } else {
@@ -3710,19 +3737,39 @@ void vnc_display_open(const char *id, Error **errp)
   &error_abort);
 }
 #ifdef CONFIG_VNC_SASL
-if (acl && sasl) {
-char *aclname;
+if (sasl) {
+if (saslacl) {
+Object *container, *acl;
+container = object_get_objects_root();
+acl = object_resolve_path_component(container, saslacl);
+if (!acl) {
+error_setg(errp, "Cannot find ACL %s", saslacl);
+goto fail;
+}
 
-if (strcmp(vs->id, "default") == 0) {
-aclname = g_strdup("vnc.username");
-} else {
-aclname = g_strdup_printf("vnc.%s.username", vs->id);
-}
-vs->sasl.acl =
-QAUTHZ(qauthz_simple_new(aclname,
- QAUTHZ_SIMPLE_POLICY_DENY,
- &error_abort));
-g_free(aclname);
+if (!object_dynamic_cast(acl, TYPE_QAUTHZ)) {
+error_setg(errp, "Object '%s' is not a QAuthZ subclass",
+   saslacl);
+goto fail;
+}
+vs->sasl.acl = QAUTHZ(acl);
+} else if (acl) {
+char *aclname;
+
+if (strcmp(vs->id, "default") == 0) {
+aclname = g_strdup("vnc.username");
+

[Qemu-block] [PATCH v3 09/10] chardev: add support for ACLs for TLS clients

2016-03-10 Thread Daniel P. Berrange 
Currently any client which can complete the TLS handshake
is able to use a chardev server. The server admin can turn
on the 'verify-peer' option for the x509 creds to require
the client to provide a x509 certificate. This means the
client will have to acquire a certificate from the CA before
they are permitted to use the chardev server. This is still
a fairly weak bar.

This adds a 'tls-acl=ACL-ID' option to the socket chardev
backend which takes the ID of a previously added 'QAuthZ'
object instance. This ACL will be used to validate the client's
x509 distinguished name. Clients failing the ACL will not be
permitted to use the chardev server.

For example to setup an ACL that only allows connection from
a client whose x509 certificate distinguished name contains
'CN=fred', you would use:

  $QEMU -object tls-creds-x509,id=tls0,dir=/home/berrange/qemutls,\
endpoint=server,verify-peer=yes \
-object authz-simple,id=acl0,policy=deny,\
rules.0.match=*CN=fred,rules.0.policy=allow \
-chardev socket,host=127.0.0.1,port=9000,server,\
 tls-creds=tls0,tls-acl=acl0 \
...other qemud args...

Signed-off-by: Daniel P. Berrange 
---
 qapi-schema.json |  2 ++
 qemu-char.c  | 11 ++-
 2 files changed, 12 insertions(+), 1 deletion(-)

diff --git a/qapi-schema.json b/qapi-schema.json
index b6769de..a6a7205 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -3202,6 +3202,7 @@
 # @addr: socket address to listen on (server=true)
 #or connect to (server=false)
 # @tls-creds: #optional the ID of the TLS credentials object (since 2.6)
+# @tls-acl: #optional the ID of the QAuthZ authorization object (since 2.6)
 # @server: #optional create server socket (default: true)
 # @wait: #optional wait for incoming connection on server
 #sockets (default: false).
@@ -3217,6 +3218,7 @@
 ##
 { 'struct': 'ChardevSocket', 'data': { 'addr'   : 'SocketAddress',
  '*tls-creds'  : 'str',
+ '*tls-acl': 'str',
  '*server': 'bool',
  '*wait'  : 'bool',
  '*nodelay'   : 'bool',
diff --git a/qemu-char.c b/qemu-char.c
index e0147f3..9533e7e 100644
--- a/qemu-char.c
+++ b/qemu-char.c
@@ -2533,6 +2533,7 @@ typedef struct {
 QIOChannelSocket *listen_ioc;
 guint listen_tag;
 QCryptoTLSCreds *tls_creds;
+char *tls_acl;
 int connected;
 int max_size;
 int do_telnetopt;
@@ -2963,7 +2964,7 @@ static void tcp_chr_tls_init(CharDriverState *chr)
 if (s->is_listen) {
 tioc = qio_channel_tls_new_server(
 s->ioc, s->tls_creds,
-NULL, /* XXX Use an ACL */
+s->tls_acl,
 &err);
 } else {
 tioc = qio_channel_tls_new_client(
@@ -3084,6 +3085,7 @@ static void tcp_chr_close(CharDriverState *chr)
 if (s->tls_creds) {
 object_unref(OBJECT(s->tls_creds));
 }
+g_free(s->tls_acl);
 if (s->write_msgfds_num) {
 g_free(s->write_msgfds);
 }
@@ -3623,6 +3625,7 @@ static void qemu_chr_parse_socket(QemuOpts *opts, 
ChardevBackend *backend,
 const char *host = qemu_opt_get(opts, "host");
 const char *port = qemu_opt_get(opts, "port");
 const char *tls_creds = qemu_opt_get(opts, "tls-creds");
+const char *tls_acl = qemu_opt_get(opts, "tls-acl");
 SocketAddress *addr;
 ChardevSocket *sock;
 
@@ -3656,6 +3659,7 @@ static void qemu_chr_parse_socket(QemuOpts *opts, 
ChardevBackend *backend,
 sock->has_reconnect = true;
 sock->reconnect = reconnect;
 sock->tls_creds = g_strdup(tls_creds);
+sock->tls_acl = g_strdup(tls_acl);
 
 addr = g_new0(SocketAddress, 1);
 if (path) {
@@ -4094,6 +4098,9 @@ QemuOptsList qemu_chardev_opts = {
 .name = "tls-creds",
 .type = QEMU_OPT_STRING,
 },{
+.name = "tls-acl",
+.type = QEMU_OPT_STRING,
+},{
 .name = "width",
 .type = QEMU_OPT_NUMBER,
 },{
@@ -4341,6 +4348,7 @@ static CharDriverState *qmp_chardev_open_socket(const 
char *id,
 }
 }
 }
+s->tls_acl = g_strdup(sock->tls_acl);
 
 qapi_copy_SocketAddress(&s->addr, sock->addr);
 
@@ -4386,6 +4394,7 @@ static CharDriverState *qmp_chardev_open_socket(const 
char *id,
 if (s->tls_creds) {
 object_unref(OBJECT(s->tls_creds));
 }
+g_free(s->tls_acl);
 g_free(s);
 qemu_chr_free_common(chr);
 return NULL;
-- 
2.5.0

[Qemu-block] [PATCH v3 06/10] acl: delete existing ACL implementation

2016-03-10 Thread Daniel P. Berrange 
The 'qemu_acl' type was a previous non-QOM based attempt to
provide an authorization facility in QEMU. Because it is
non-QOM based it cannot be created via the command line and
requires special monitor commands to manipulate it.

The new QAuthZ and QAuthZSimple QOM classes provide a superset
of the functionality in qemu_acl, so the latter can now be
deleted. The HMP 'acl_*' monitor commands are converted to
use the new QAuthZSimple data type instead in order to provide
backwards compatibility, but their use is discouraged.

Signed-off-by: Daniel P. Berrange 
---
 Makefile   |   6 +-
 crypto/tlssession.c|  28 --
 include/qemu/acl.h |  74 
 monitor.c  | 161 ++-
 tests/Makefile |   2 +-
 tests/test-crypto-tlssession.c |  13 +--
 tests/test-io-channel-tls.c|  14 +--
 ui/vnc-auth-sasl.c |   2 +-
 ui/vnc-auth-sasl.h |   4 +-
 ui/vnc.c   |  11 ++-
 util/Makefile.objs |   1 -
 util/acl.c | 188 -
 12 files changed, 156 insertions(+), 348 deletions(-)
 delete mode 100644 include/qemu/acl.h
 delete mode 100644 util/acl.c

diff --git a/Makefile b/Makefile
index 60ad13e..8f7ffd3 100644
--- a/Makefile
+++ b/Makefile
@@ -235,9 +235,9 @@ util/module.o-cflags = 
-D'CONFIG_BLOCK_MODULES=$(block-modules)'
 
 qemu-img.o: qemu-img-cmds.h
 
-qemu-img$(EXESUF): qemu-img.o $(block-obj-y) $(crypto-obj-y) $(io-obj-y) 
$(qom-obj-y) libqemuutil.a libqemustub.a
-qemu-nbd$(EXESUF): qemu-nbd.o $(block-obj-y) $(crypto-obj-y) $(io-obj-y) 
$(qom-obj-y) libqemuutil.a libqemustub.a
-qemu-io$(EXESUF): qemu-io.o $(block-obj-y) $(crypto-obj-y) $(io-obj-y) 
$(qom-obj-y) libqemuutil.a libqemustub.a
+qemu-img$(EXESUF): qemu-img.o $(block-obj-y) $(crypto-obj-y) $(io-obj-y) 
$(qom-obj-y) $(util-qom-obj-y) libqemuutil.a libqemustub.a
+qemu-nbd$(EXESUF): qemu-nbd.o $(block-obj-y) $(crypto-obj-y) $(io-obj-y) 
$(qom-obj-y) $(util-qom-obj-y) libqemuutil.a libqemustub.a
+qemu-io$(EXESUF): qemu-io.o $(block-obj-y) $(crypto-obj-y) $(io-obj-y) 
$(qom-obj-y) $(util-qom-obj-y) libqemuutil.a libqemustub.a
 
 qemu-bridge-helper$(EXESUF): qemu-bridge-helper.o
 
diff --git a/crypto/tlssession.c b/crypto/tlssession.c
index e0d9658..26e8097 100644
--- a/crypto/tlssession.c
+++ b/crypto/tlssession.c
@@ -22,7 +22,7 @@
 #include "crypto/tlssession.h"
 #include "crypto/tlscredsanon.h"
 #include "crypto/tlscredsx509.h"
-#include "qemu/acl.h"
+#include "qemu/authz.h"
 #include "trace.h"
 
 #ifdef CONFIG_GNUTLS
@@ -207,6 +207,7 @@ qcrypto_tls_session_check_certificate(QCryptoTLSSession 
*session,
 unsigned int nCerts, i;
 time_t now;
 gnutls_x509_crt_t cert = NULL;
+Error *err = NULL;
 
 now = time(NULL);
 if (now == ((time_t)-1)) {
@@ -295,16 +296,33 @@ qcrypto_tls_session_check_certificate(QCryptoTLSSession 
*session,
 goto error;
 }
 if (session->aclname) {
-qemu_acl *acl = qemu_acl_find(session->aclname);
-int allow;
-if (!acl) {
+QAuthZ *acl;
+Object *obj;
+Object *container;
+bool allow;
+
+container = object_get_objects_root();
+obj = object_resolve_path_component(container,
+session->aclname);
+if (!obj) {
 error_setg(errp, "Cannot find ACL %s",
session->aclname);
 goto error;
 }
 
-allow = qemu_acl_party_is_allowed(acl, session->peername);
+if (!object_dynamic_cast(obj, TYPE_QAUTHZ)) {
+error_setg(errp, "Object '%s' is not a QAuthZ subclass",
+   session->aclname);
+goto error;
+}
 
+acl = QAUTHZ(obj);
+
+allow = qauthz_is_allowed(acl, session->peername, &err);
+if (err) {
+error_propagate(errp, err);
+goto error;
+}
 if (!allow) {
 error_setg(errp, "TLS x509 ACL check for %s is denied",
session->peername);
diff --git a/include/qemu/acl.h b/include/qemu/acl.h
deleted file mode 100644
index 116487e..000
--- a/include/qemu/acl.h
+++ /dev/null
@@ -1,74 +0,0 @@
-/*
- * QEMU access control list management
- *
- * Copyright (C) 2009 Red Hat, Inc
- *
- * Permission is hereby granted, free of charge, to any person obtaining a copy
- * of this software and associated documentation files (the "Software"), to 
deal
- * in the Software without restriction, including without limitation the rights
- * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
- * copies of

[Qemu-block] [PATCH v3 02/10] qapi: allow QmpInputVisitor to auto-cast types

2016-03-10 Thread Daniel P. Berrange 
Currently the QmpInputVisitor assumes that all scalar
values are directly represented as their final types.
ie it assumes an 'int' is using QInt, and a 'bool' is
using QBool.

This extends it so that QString is optionally permitted
for any of the non-string scalar types. This behaviour
is turned on by requesting the 'autocast' flag in the
constructor.

This makes it possible to use QmpInputVisitor with a
QDict produced from QemuOpts, where everything is in
string format.

Signed-off-by: Daniel P. Berrange 
---
 include/qapi/qmp-input-visitor.h |   3 +
 qapi/qmp-input-visitor.c |  96 +++-
 tests/test-qmp-input-visitor.c   | 115 ++-
 3 files changed, 196 insertions(+), 18 deletions(-)

diff --git a/include/qapi/qmp-input-visitor.h b/include/qapi/qmp-input-visitor.h
index 3ed499c..c25cb7c 100644
--- a/include/qapi/qmp-input-visitor.h
+++ b/include/qapi/qmp-input-visitor.h
@@ -21,6 +21,9 @@ typedef struct QmpInputVisitor QmpInputVisitor;
 
 QmpInputVisitor *qmp_input_visitor_new(QObject *obj);
 QmpInputVisitor *qmp_input_visitor_new_strict(QObject *obj);
+QmpInputVisitor *qmp_input_visitor_new_full(QObject *obj,
+bool strict,
+bool autocast);
 
 void qmp_input_visitor_cleanup(QmpInputVisitor *v);
 
diff --git a/qapi/qmp-input-visitor.c b/qapi/qmp-input-visitor.c
index e659832..59d2165 100644
--- a/qapi/qmp-input-visitor.c
+++ b/qapi/qmp-input-visitor.c
@@ -35,6 +35,7 @@ struct QmpInputVisitor
 StackObject stack[QIV_STACK_SIZE];
 int nb_stack;
 bool strict;
+bool autocast;
 };
 
 static QmpInputVisitor *to_qiv(Visitor *v)
@@ -217,15 +218,26 @@ static void qmp_input_type_int64(Visitor *v, const char 
*name, int64_t *obj,
  Error **errp)
 {
 QmpInputVisitor *qiv = to_qiv(v);
-QInt *qint = qobject_to_qint(qmp_input_get_object(qiv, name, true));
+QObject *qobj = qmp_input_get_object(qiv, name, true);
+QInt *qint;
+QString *qstr;
 
-if (!qint) {
-error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
-   "integer");
+qint = qobject_to_qint(qobj);
+if (qint) {
+*obj = qint_get_int(qint);
 return;
 }
 
-*obj = qint_get_int(qint);
+qstr = qobject_to_qstring(qobj);
+if (qstr && qstr->string && qiv->autocast) {
+errno = 0;
+if (qemu_strtoll(qstr->string, NULL, 10, obj) == 0) {
+return;
+}
+}
+
+error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
+   "integer");
 }
 
 static void qmp_input_type_uint64(Visitor *v, const char *name, uint64_t *obj,
@@ -233,30 +245,61 @@ static void qmp_input_type_uint64(Visitor *v, const char 
*name, uint64_t *obj,
 {
 /* FIXME: qobject_to_qint mishandles values over INT64_MAX */
 QmpInputVisitor *qiv = to_qiv(v);
-QInt *qint = qobject_to_qint(qmp_input_get_object(qiv, name, true));
+QObject *qobj = qmp_input_get_object(qiv, name, true);
+QInt *qint;
+QString *qstr;
 
-if (!qint) {
-error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
-   "integer");
+qint = qobject_to_qint(qobj);
+if (qint) {
+*obj = qint_get_int(qint);
 return;
 }
 
-*obj = qint_get_int(qint);
+qstr = qobject_to_qstring(qobj);
+if (qstr && qstr->string && qiv->autocast) {
+errno = 0;
+if (qemu_strtoull(qstr->string, NULL, 10, obj) == 0) {
+return;
+}
+}
+
+error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
+   "integer");
 }
 
 static void qmp_input_type_bool(Visitor *v, const char *name, bool *obj,
 Error **errp)
 {
 QmpInputVisitor *qiv = to_qiv(v);
-QBool *qbool = qobject_to_qbool(qmp_input_get_object(qiv, name, true));
+QObject *qobj = qmp_input_get_object(qiv, name, true);
+QBool *qbool;
+QString *qstr;
 
-if (!qbool) {
-error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
-   "boolean");
+qbool = qobject_to_qbool(qobj);
+if (qbool) {
+*obj = qbool_get_bool(qbool);
 return;
 }
 
-*obj = qbool_get_bool(qbool);
+
+qstr = qobject_to_qstring(qobj);
+if (qstr && qstr->string && qiv->autocast) {
+if (!strcasecmp(qstr->string, "on") ||
+!strcasecmp(qstr->string, "yes") ||
+!strcasecmp(qstr->string, "true")) {
+*obj = true;
+return;
+}
+if (!strcasecmp(qstr->string, "off") ||
+!strcasecmp(qstr->string, "no") ||
+!strcasecmp(qstr->string, "false")) {
+*obj = false;
+return;
+}
+}
+
+error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
+   "boolean");
 }
 
 static void qmp_input_type_s

[Qemu-block] [PATCH v3 08/10] nbd: allow an ACL to be set with nbd-server-start QMP command

2016-03-10 Thread Daniel P. Berrange 
As with the previous patch to qemu-nbd, the nbd-server-start
QMP command also needs to be able to specify an ACL when
enabling TLS encryption.

First the client must create a QAuthZ object instance using
the 'object-add' command:

   {
 'execute': 'object-add',
 'arguments': {
   'qom-type': 'authz-simple',
   'id': 'tls0',
   'parameters': {
 'policy': 'deny',
 'rules': [
   {
 'match': '*CN=fred',
 'policy': 'allow'
   }
 ]
   }
 }
   }

They can then reference this in the new 'tls-acl' parameter
when executing the 'nbd-server-start' command.

   {
 'execute': 'nbd-server-start',
 'arguments': {
   'addr': {
   'type': 'inet',
   'host': '127.0.0.1',
   'port': '9000'
   },
   'tls-creds': 'tls0',
   'tls-acl': 'tlsacl0'
 }
   }

Signed-off-by: Daniel P. Berrange 
---
 blockdev-nbd.c  | 10 +-
 hmp.c   |  2 +-
 qapi/block.json |  4 +++-
 qmp-commands.hx |  2 +-
 4 files changed, 14 insertions(+), 4 deletions(-)

diff --git a/blockdev-nbd.c b/blockdev-nbd.c
index 12cae0e..ae5335e 100644
--- a/blockdev-nbd.c
+++ b/blockdev-nbd.c
@@ -24,6 +24,7 @@ typedef struct NBDServerData {
 QIOChannelSocket *listen_ioc;
 int watch;
 QCryptoTLSCreds *tlscreds;
+char *tlsacl;
 } NBDServerData;
 
 static NBDServerData *nbd_server;
@@ -45,7 +46,8 @@ static gboolean nbd_accept(QIOChannel *ioc, GIOCondition 
condition,
 }
 
 nbd_client_new(NULL, cioc,
-   nbd_server->tlscreds, NULL,
+   nbd_server->tlscreds,
+   nbd_server->tlsacl,
nbd_client_put);
 object_unref(OBJECT(cioc));
 return TRUE;
@@ -65,6 +67,7 @@ static void nbd_server_free(NBDServerData *server)
 if (server->tlscreds) {
 object_unref(OBJECT(server->tlscreds));
 }
+g_free(server->tlsacl);
 
 g_free(server);
 }
@@ -101,6 +104,7 @@ static QCryptoTLSCreds *nbd_get_tls_creds(const char *id, 
Error **errp)
 
 void qmp_nbd_server_start(SocketAddress *addr,
   bool has_tls_creds, const char *tls_creds,
+  bool has_tls_acl, const char *tls_acl,
   Error **errp)
 {
 if (nbd_server) {
@@ -128,6 +132,10 @@ void qmp_nbd_server_start(SocketAddress *addr,
 }
 }
 
+if (has_tls_acl) {
+nbd_server->tlsacl = g_strdup(tls_acl);
+}
+
 nbd_server->watch = qio_channel_add_watch(
 QIO_CHANNEL(nbd_server->listen_ioc),
 G_IO_IN,
diff --git a/hmp.c b/hmp.c
index 7a98726..20703fd 100644
--- a/hmp.c
+++ b/hmp.c
@@ -1802,7 +1802,7 @@ void hmp_nbd_server_start(Monitor *mon, const QDict 
*qdict)
 goto exit;
 }
 
-qmp_nbd_server_start(addr, false, NULL, &local_err);
+qmp_nbd_server_start(addr, false, NULL, false, NULL, &local_err);
 qapi_free_SocketAddress(addr);
 if (local_err != NULL) {
 goto exit;
diff --git a/qapi/block.json b/qapi/block.json
index 58e6b30..6b209e1 100644
--- a/qapi/block.json
+++ b/qapi/block.json
@@ -147,6 +147,7 @@
 #
 # @addr: Address on which to listen.
 # @tls-creds: (optional) ID of the TLS credentials object. Since 2.6
+# @tls-acl: (optional) ID of the QAuthZ authorization object. Since 2.6
 #
 # Returns: error if the server is already running.
 #
@@ -154,7 +155,8 @@
 ##
 { 'command': 'nbd-server-start',
   'data': { 'addr': 'SocketAddress',
-'*tls-creds': 'str'} }
+'*tls-creds': 'str',
+'*tls-acl': 'str'} }
 
 ##
 # @nbd-server-add:
diff --git a/qmp-commands.hx b/qmp-commands.hx
index b629673..7a3fa26 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -3859,7 +3859,7 @@ EQMP
 
 {
 .name   = "nbd-server-start",
-.args_type  = "addr:q,tls-creds:s?",
+.args_type  = "addr:q,tls-creds:s?,tls-acl:s?",
 .mhandler.cmd_new = qmp_marshal_nbd_server_start,
 },
 {
-- 
2.5.0

[Qemu-block] [PATCH v3 01/10] qdict: implement a qdict_crumple method for un-flattening a dict

2016-03-10 Thread Daniel P. Berrange 
The qdict_flatten() method will take a dict whose elements are
further nested dicts/lists and flatten them by concatenating
keys.

The qdict_crumple() method aims to do the reverse, taking a flat
qdict, and turning it into a set of nested dicts/lists. It will
apply nesting based on the key name, with a '.' indicating a
new level in the hierarchy. If the keys in the nested structure
are all numeric, it will create a list, otherwise it will create
a dict.

If the keys are a mixture of numeric and non-numeric, or the
numeric keys are not in strictly ascending order, an error will
be reported.

As an example, a flat dict containing

 {
   'foo.0.bar': 'one',
   'foo.0.wizz': '1',
   'foo.1.bar': 'two',
   'foo.1.wizz': '2'
 }

will get turned into a dict with one element 'foo' whose
value is a list. The list elements will each in turn be
dicts.

 {
   'foo' => [
 { 'bar': 'one', 'wizz': '1' }
 { 'bar': 'two', 'wizz': '2' }
   ],
 }

If the key is intended to contain a literal '.', then it must
be escaped as '..'. ie a flat dict

  {
 'foo..bar': 'wizz',
 'bar.foo..bar': 'eek',
 'bar.hello': 'world'
  }

Will end up as

  {
 'foo.bar': 'wizz',
 'bar': {
'foo.bar': 'eek',
'hello': 'world'
 }
  }

The intent of this function is that it allows a set of QemuOpts
to be turned into a nested data structure that mirrors the nested
used when the same object is defined over QMP.

Signed-off-by: Daniel P. Berrange 
---
 include/qapi/qmp/qdict.h |   1 +
 qobject/qdict.c  | 267 +++
 tests/check-qdict.c  | 143 +
 3 files changed, 411 insertions(+)

diff --git a/include/qapi/qmp/qdict.h b/include/qapi/qmp/qdict.h
index 71b8eb0..8a3ac13 100644
--- a/include/qapi/qmp/qdict.h
+++ b/include/qapi/qmp/qdict.h
@@ -73,6 +73,7 @@ void qdict_flatten(QDict *qdict);
 void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start);
 void qdict_array_split(QDict *src, QList **dst);
 int qdict_array_entries(QDict *src, const char *subqdict);
+QObject *qdict_crumple(QDict *src, bool recursive, Error **errp);
 
 void qdict_join(QDict *dest, QDict *src, bool overwrite);
 
diff --git a/qobject/qdict.c b/qobject/qdict.c
index 9833bd0..3a01fcc 100644
--- a/qobject/qdict.c
+++ b/qobject/qdict.c
@@ -682,6 +682,273 @@ void qdict_array_split(QDict *src, QList **dst)
 }
 }
 
+
+/**
+ * qdict_split_flat_key:
+ *
+ * Given a flattened key such as 'foo.0.bar', split it
+ * into two parts at the first '.' separator. Allows
+ * double dot ('..') to escape the normal separator.
+ *
+ * eg
+ *'foo.0.bar' -> prefix='foo' and suffix='0.bar'
+ *'foo..0.bar' -> prefix='foo.0' and suffix='bar'
+ *
+ * The '..' sequence will be unescaped in the returned
+ * 'prefix' string. The 'suffix' string will be left
+ * in escaped format, so it can be fed back into the
+ * qdict_split_flat_key() key as the input later.
+ */
+static void qdict_split_flat_key(const char *key, char **prefix, char **suffix)
+{
+const char *separator;
+size_t i, j;
+
+/* Find first '.' separator, but if there is a pair '..'
+ * that acts as an escape, so skip over '..' */
+separator = NULL;
+do {
+if (separator) {
+separator += 2;
+} else {
+separator = key;
+}
+separator = strchr(separator, '.');
+} while (separator && *(separator + 1) == '.');
+
+if (separator) {
+*prefix = g_strndup(key,
+separator - key);
+*suffix = g_strdup(separator + 1);
+} else {
+*prefix = g_strdup(key);
+*suffix = NULL;
+}
+
+/* Unescape the '..' sequence into '.' */
+for (i = 0, j = 0; (*prefix)[i] != '\0'; i++, j++) {
+if ((*prefix)[i] == '.' &&
+(*prefix)[i + 1] == '.') {
+i++;
+}
+(*prefix)[j] = (*prefix)[i];
+}
+(*prefix)[j] = '\0';
+}
+
+
+/**
+ * qdict_list_size:
+ * @maybe_List: dict that may be only list elements
+ *
+ * Determine whether all keys in @maybe_list are
+ * valid list elements. They they are all valid,
+ * then this returns the number of elements. If
+ * they all look like non-numeric keys, then returns
+ * zero. If there is a mix of numeric and non-numeric
+ * keys, then an error is set as it is both a list
+ * and a dict at once.
+ *
+ * Returns: number of list elemets, 0 if a dict, -1 on error
+ */
+static ssize_t qdict_list_size(QDict *maybe_list, Error **errp)
+{
+const QDictEntry *entry, *next;
+ssize_t len = 0;
+ssize_t max = -1;
+int is_list = -1;
+int64_t val;
+
+entry = qdict_first(maybe_list);
+while (entry != NULL) {
+next = qdict_next(maybe_list, entry);
+
+if (qemu_strtoll(entry->key, NULL, 10, &val) == 0) {
+if (is_list == -1) {
+is_list = 1;
+} else if (!is_list) {
+error_setg(errp,
+   "Key '%s' is

[Qemu-block] [PATCH v3 04/10] util: add QAuthZ object as an authorization base class

2016-03-10 Thread Daniel P. Berrange 
The current qemu_acl module provides a simple access control
list facility inside QEMU, which is used via a set of monitor
commands acl_show, acl_policy, acl_add, acl_remove & acl_reset.

Note there is no ability to create ACLs - the network services
(eg VNC server) were expected to create ACLs that they want to
check.

There is also no way to define ACLs on the command line, nor
potentially integrate with external authorization systems like
polkit, pam, ldap lookup, etc.

The QAuthZ object defines a minimal abstract QOM class that can
be subclassed for creating different authorization providers.

Signed-off-by: Daniel P. Berrange 
---
 MAINTAINERS  |  7 +
 Makefile |  1 +
 Makefile.objs|  2 ++
 Makefile.target  |  2 ++
 include/qemu/authz.h | 81 
 util/Makefile.objs   |  2 ++
 util/authz.c | 46 +
 7 files changed, 141 insertions(+)
 create mode 100644 include/qemu/authz.h
 create mode 100644 util/authz.c

diff --git a/MAINTAINERS b/MAINTAINERS
index dc0aa54..73bc431 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1301,6 +1301,13 @@ F: include/qemu/throttle.h
 F: util/throttle.c
 L: qemu-block@nongnu.org
 
+Authorization
+M: Daniel P. Berrange 
+S: Maintained
+F: util/authz*
+F: include/qemu/authz*
+F: tests/test-authz-*
+
 Usermode Emulation
 --
 Overall
diff --git a/Makefile b/Makefile
index 70e3ebc..903dc35 100644
--- a/Makefile
+++ b/Makefile
@@ -150,6 +150,7 @@ endif
 dummy := $(call unnest-vars,, \
 stub-obj-y \
 util-obj-y \
+util-qom-obj-y \
 qga-obj-y \
 ivshmem-client-obj-y \
 ivshmem-server-obj-y \
diff --git a/Makefile.objs b/Makefile.objs
index fbcaa74..8bc9a77 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -4,6 +4,8 @@ stub-obj-y = stubs/
 util-obj-y = util/ qobject/ qapi/
 util-obj-y += qmp-introspect.o qapi-types.o qapi-visit.o qapi-event.o
 
+util-qom-obj-y += util/
+
 ###
 # block-obj-y is code used by both qemu system emulation and qemu-img
 
diff --git a/Makefile.target b/Makefile.target
index 34ddb7e..9728f86 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -171,6 +171,7 @@ include $(SRC_PATH)/Makefile.objs
 dummy := $(call unnest-vars,,target-obj-y)
 target-obj-y-save := $(target-obj-y)
 dummy := $(call unnest-vars,.., \
+   util-qom-obj-y \
block-obj-y \
block-obj-m \
crypto-obj-y \
@@ -183,6 +184,7 @@ target-obj-y := $(target-obj-y-save)
 all-obj-y += $(common-obj-y)
 all-obj-y += $(target-obj-y)
 all-obj-y += $(qom-obj-y)
+all-obj-y += $(util-qom-obj-y)
 all-obj-$(CONFIG_SOFTMMU) += $(block-obj-y)
 all-obj-$(CONFIG_USER_ONLY) += $(crypto-aes-obj-y)
 all-obj-$(CONFIG_SOFTMMU) += $(crypto-obj-y)
diff --git a/include/qemu/authz.h b/include/qemu/authz.h
new file mode 100644
index 000..89fa6da
--- /dev/null
+++ b/include/qemu/authz.h
@@ -0,0 +1,81 @@
+/*
+ * QEMU authorization framework
+ *
+ * Copyright (c) 2016 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see .
+ *
+ */
+
+#ifndef QAUTHZ_H__
+#define QAUTHZ_H__
+
+#include "qemu-common.h"
+#include "qapi/error.h"
+#include "qom/object.h"
+
+
+#define TYPE_QAUTHZ "authz"
+
+#define QAUTHZ_CLASS(klass) \
+ OBJECT_CLASS_CHECK(QAuthZClass, (klass), \
+TYPE_QAUTHZ)
+#define QAUTHZ_GET_CLASS(obj) \
+ OBJECT_GET_CLASS(QAuthZClass, (obj), \
+  TYPE_QAUTHZ)
+#define QAUTHZ(obj) \
+ INTERFACE_CHECK(QAuthZ, (obj), \
+ TYPE_QAUTHZ)
+
+typedef struct QAuthZ QAuthZ;
+typedef struct QAuthZClass QAuthZClass;
+
+/**
+ * QAuthZ:
+ *
+ * The QAuthZ class defines an API contract to be used
+ * for providing an authorization driver for network
+ * services.
+ */
+
+struct QAuthZ {
+Object parent_obj;
+};
+
+
+struct QAuthZClass {
+ObjectClass parent_class;
+
+bool (*is_allowed)(QAuthZ *authz,
+   const char *identity,
+   Error **errp);
+};
+
+
+/**
+ * qauthz_is_allowed:
+ * @authz: the authorization object
+ * @identity: the user identity to authorize
+ * @errp: pointer to a NULL initialized error object
+ *
+ * Check if a user

[Qemu-block] [PATCH v3 03/10] qom: support arbitrary non-scalar properties with -object

2016-03-10 Thread Daniel P. Berrange 
The current -object command line syntax only allows for
creation of objects with scalar properties, or a list
with a fixed scalar element type. Objects which have
properties that are represented as structs in the QAPI
schema cannot be created using -object.

This is a design limitation of the way the OptsVisitor
is written. It simply iterates over the QemuOpts values
as a flat list. The support for lists is enabled by
allowing the same key to be repeated in the opts string.

It is not practical to extend the OptsVisitor to support
more complex data structures while also maintaining
the existing list handling behaviour that is relied upon
by other areas of QEMU.

Fortunately there is no existing object that implements
the UserCreatable interface that relies on the list
handling behaviour, so it is possible to swap out the
OptsVisitor for a different visitor implementation, so
-object supports non-scalar properties, thus leaving
other users of OptsVisitor unaffected.

The previously added qdict_crumple() method is able to
take a qdict containing a flat set of properties and
turn that into a arbitrarily nested set of dicts and
lists. By combining qemu_opts_to_qdict and qdict_crumple()
together, we can turn the opt string into a data structure
that is practically identical to that passed over QMP
when defining an object. The only difference is that all
the scalar values are represented as strings, rather than
strings, ints and bools. This is sufficient to let us
replace the OptsVisitor with the QMPInputVisitor for
use with -object.

Thus -object can now support non-scalar properties,
for example the QMP object

  {
"execute": "object-add",
"arguments": {
  "qom-type": "demo",
  "id": "demo0",
  "parameters": {
"foo": [
  { "bar": "one", "wizz": "1" },
  { "bar": "two", "wizz": "2" }
]
  }
}
  }

Would be creatable via the CLI now using

$QEMU \
  -object demo,id=demo0,\
  foo.0.bar=one,foo.0.wizz=1,\
  foo.1.bar=two,foo.1.wizz=2

This is also wired up to work for the 'object_add' command
in the HMP monitor with the same syntax.

  (hmp) object_add demo,id=demo0,\
   foo.0.bar=one,foo.0.wizz=1,\
   foo.1.bar=two,foo.1.wizz=2

Signed-off-by: Daniel P. Berrange 
---
 hmp.c  |  18 +--
 qom/object_interfaces.c|  20 ++-
 tests/check-qom-proplist.c | 295 -
 3 files changed, 313 insertions(+), 20 deletions(-)

diff --git a/hmp.c b/hmp.c
index 5b6084a..7a98726 100644
--- a/hmp.c
+++ b/hmp.c
@@ -25,7 +25,7 @@
 #include "qemu/sockets.h"
 #include "monitor/monitor.h"
 #include "monitor/qdev.h"
-#include "qapi/opts-visitor.h"
+#include "qapi/qmp-input-visitor.h"
 #include "qapi/qmp/qerror.h"
 #include "qapi/string-output-visitor.h"
 #include "qapi/util.h"
@@ -1673,20 +1673,12 @@ void hmp_netdev_del(Monitor *mon, const QDict *qdict)
 void hmp_object_add(Monitor *mon, const QDict *qdict)
 {
 Error *err = NULL;
-QemuOpts *opts;
-OptsVisitor *ov;
+QmpInputVisitor *qiv;
 Object *obj = NULL;
 
-opts = qemu_opts_from_qdict(qemu_find_opts("object"), qdict, &err);
-if (err) {
-hmp_handle_error(mon, &err);
-return;
-}
-
-ov = opts_visitor_new(opts);
-obj = user_creatable_add(qdict, opts_get_visitor(ov), &err);
-opts_visitor_cleanup(ov);
-qemu_opts_del(opts);
+qiv = qmp_input_visitor_new_full((QObject *)qdict, true, true);
+obj = user_creatable_add(qdict, qmp_input_get_visitor(qiv), &err);
+qmp_input_visitor_cleanup(qiv);
 
 if (err) {
 hmp_handle_error(mon, &err);
diff --git a/qom/object_interfaces.c b/qom/object_interfaces.c
index c2f6e29..9c41730 100644
--- a/qom/object_interfaces.c
+++ b/qom/object_interfaces.c
@@ -1,9 +1,9 @@
 #include "qemu/osdep.h"
 #include "qom/object_interfaces.h"
 #include "qemu/module.h"
+#include "qemu/option.h"
 #include "qapi-visit.h"
-#include "qapi/qmp-output-visitor.h"
-#include "qapi/opts-visitor.h"
+#include "qapi/qmp-input-visitor.h"
 
 void user_creatable_complete(Object *obj, Error **errp)
 {
@@ -120,6 +120,7 @@ Object *user_creatable_add_type(const char *type, const 
char *id,
 obj = object_new(type);
 if (qdict) {
 for (e = qdict_first(qdict); e; e = qdict_next(qdict, e)) {
+
 object_property_set(obj, v, e->key, &local_err);
 if (local_err) {
 goto out;
@@ -151,15 +152,22 @@ out:
 
 Object *user_creatable_add_opts(QemuOpts *opts, Error **errp)
 {
-OptsVisitor *ov;
+QmpInputVisitor *qiv;
 QDict *pdict;
+QObject *pobj;
 Object *obj = NULL;
 
-ov = opts_visitor_new(opts);
 pdict = qemu_opts_to_qdict(opts, NULL);
+pobj = qdict_crumple(pdict, true, errp);
+if (!pobj) {
+goto cleanup;
+}
+qiv = qmp_input_visitor_new_full(pobj, true, true);
 
-obj = user_creatable_add(pdict, opts_get_visitor(ov), errp);
-opts_v

[Qemu-block] [PATCH v3 07/10] qemu-nbd: add support for ACLs for TLS clients

2016-03-10 Thread Daniel P. Berrange 
Currently any client which can complete the TLS handshake
is able to use the NBD server. The server admin can turn
on the 'verify-peer' option for the x509 creds to require
the client to provide a x509 certificate. This means the
client will have to acquire a certificate from the CA before
they are permitted to use the NBD server. This is still a
fairly weak bar.

This adds a '--tls-acl ACL-ID' option to the qemu-nbd command
which takes the ID of a previously added 'QAuthZ' object
instance. This ACL will be used to validate the client's
x509 distinguished name. Clients failing the ACL will not be
permitted to use the NBD server.

For example to setup an ACL that only allows connection from
a client whose x509 certificate distinguished name contains
'CN=fred', you would use:

  qemu-nbd -object tls-creds-x509,id=tls0,dir=/home/berrange/qemutls,\
   endpoint=server,verify-peer=yes \
   -object authz-simple,id=acl0,policy=deny,\
   rules.0.match=*CN=fred,rules.0.policy=allow \
   -tls-creds tls0 \
   -tls-acl acl0
   other qemu-nbd args...

Signed-off-by: Daniel P. Berrange 
---
 qemu-nbd.c| 13 -
 qemu-nbd.texi |  4 
 2 files changed, 16 insertions(+), 1 deletion(-)

diff --git a/qemu-nbd.c b/qemu-nbd.c
index a5c1d95..d70960f 100644
--- a/qemu-nbd.c
+++ b/qemu-nbd.c
@@ -42,6 +42,7 @@
 #define QEMU_NBD_OPT_OBJECT260
 #define QEMU_NBD_OPT_TLSCREDS  261
 #define QEMU_NBD_OPT_IMAGE_OPTS262
+#define QEMU_NBD_OPT_TLSACL263
 
 static NBDExport *exp;
 static bool newproto;
@@ -55,6 +56,7 @@ static int nb_fds;
 static QIOChannelSocket *server_ioc;
 static int server_watch = -1;
 static QCryptoTLSCreds *tlscreds;
+static const char *tlsacl;
 
 static void usage(const char *name)
 {
@@ -344,7 +346,7 @@ static gboolean nbd_accept(QIOChannel *ioc, GIOCondition 
cond, gpointer opaque)
 nb_fds++;
 nbd_update_server_watch();
 nbd_client_new(newproto ? NULL : exp, cioc,
-   tlscreds, NULL, nbd_client_closed);
+   tlscreds, tlsacl, nbd_client_closed);
 object_unref(OBJECT(cioc));
 
 return TRUE;
@@ -488,6 +490,7 @@ int main(int argc, char **argv)
 { "export-name", required_argument, NULL, 'x' },
 { "tls-creds", required_argument, NULL, QEMU_NBD_OPT_TLSCREDS },
 { "image-opts", no_argument, NULL, QEMU_NBD_OPT_IMAGE_OPTS },
+{ "tls-acl", no_argument, NULL, QEMU_NBD_OPT_TLSACL },
 { NULL, 0, NULL, 0 }
 };
 int ch;
@@ -689,6 +692,9 @@ int main(int argc, char **argv)
 case QEMU_NBD_OPT_IMAGE_OPTS:
 imageOpts = true;
 break;
+case QEMU_NBD_OPT_TLSACL:
+tlsacl = optarg;
+break;
 }
 }
 
@@ -725,6 +731,11 @@ int main(int argc, char **argv)
  error_get_pretty(local_err));
 exit(EXIT_FAILURE);
 }
+} else {
+if (tlsacl) {
+error_report("--tls-acl is not permitted without --tls-creds");
+exit(EXIT_FAILURE);
+}
 }
 
 if (disconnect) {
diff --git a/qemu-nbd.texi b/qemu-nbd.texi
index 9f23343..69f32cb 100644
--- a/qemu-nbd.texi
+++ b/qemu-nbd.texi
@@ -86,6 +86,10 @@ the new style NBD protocol negotiation
 Enable mandatory TLS encryption for the server by setting the ID
 of the TLS credentials object previously created with the --object
 option.
+@item --tls-acl=ID
+Specify the ID of a qauthz object previously created with the
+--object option. This will be used to authorize users who
+connect against their x509 distinguish name.
 @item -v, --verbose
 Display extra debugging information
 @item -h, --help
-- 
2.5.0

[Qemu-block] [PATCH v3 05/10] util: add QAuthZSimple object type for a simple access control list

2016-03-10 Thread Daniel P. Berrange 
Add a QAuthZSimple object type that implements the QAuthZ
interface. This simple built-in implementation maintains
a trivial access control list with a sequence of match
rules and a final default policy. This replicates the
functionality currently provided by the qemu_acl module.

To create an instance of this object via the QMP monitor,
the syntax used would be

  {
"execute": "object-add",
"arguments": {
  "qom-type": "authz-simple",
  "id": "auth0",
  "parameters": {
"rules": [
   { "match": "fred", "policy": "allow" },
   { "match": "bob", "policy": "allow" },
   { "match": "danb", "policy": "deny" },
   { "match": "dan*", "policy": "allow" }
],
"policy": "deny"
  }
}
  }

Or via the -object command line

  $QEMU \
 -object authz-simple,id=acl0,policy=deny,\
 match.0.name=fred,match.0.policy=allow, \
 match.1.name=bob,match.1.policy=allow, \
 match.2.name=danb,match.2.policy=deny, \
 match.3.name=dan*,match.3.policy=allow

This sets up an authorization rule that allows 'fred',
'bob' and anyone whose name starts with 'dan', except
for 'danb'. Everyone unmatched is denied.

Signed-off-by: Daniel P. Berrange 
---
 Makefile|   2 +-
 include/qemu/authz-simple.h | 107 ++
 qapi-schema.json|   6 +-
 qapi/util.json  |  31 ++
 tests/.gitignore|   1 +
 tests/Makefile  |   3 +
 tests/test-authz-simple.c   | 156 +++
 util/Makefile.objs  |   1 +
 util/authz-simple.c | 256 
 9 files changed, 561 insertions(+), 2 deletions(-)
 create mode 100644 include/qemu/authz-simple.h
 create mode 100644 qapi/util.json
 create mode 100644 tests/test-authz-simple.c
 create mode 100644 util/authz-simple.c

diff --git a/Makefile b/Makefile
index 903dc35..60ad13e 100644
--- a/Makefile
+++ b/Makefile
@@ -274,7 +274,7 @@ qapi-modules = $(SRC_PATH)/qapi-schema.json 
$(SRC_PATH)/qapi/common.json \
$(SRC_PATH)/qapi/block.json $(SRC_PATH)/qapi/block-core.json \
$(SRC_PATH)/qapi/event.json $(SRC_PATH)/qapi/introspect.json \
$(SRC_PATH)/qapi/crypto.json $(SRC_PATH)/qapi/rocker.json \
-   $(SRC_PATH)/qapi/trace.json
+   $(SRC_PATH)/qapi/trace.json $(SRC_PATH)/qapi/util.json
 
 qapi-types.c qapi-types.h :\
 $(qapi-modules) $(SRC_PATH)/scripts/qapi-types.py $(qapi-py)
diff --git a/include/qemu/authz-simple.h b/include/qemu/authz-simple.h
new file mode 100644
index 000..74c09e3
--- /dev/null
+++ b/include/qemu/authz-simple.h
@@ -0,0 +1,107 @@
+/*
+ * QEMU simple authorization driver
+ *
+ * Copyright (c) 2016 Red Hat, Inc.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see .
+ *
+ */
+
+#ifndef QAUTHZ_SIMPLE_H__
+#define QAUTHZ_SIMPLE_H__
+
+#include "qemu/authz.h"
+
+
+#define TYPE_QAUTHZ_SIMPLE "authz-simple"
+
+#define QAUTHZ_SIMPLE_CLASS(klass) \
+ OBJECT_CLASS_CHECK(QAuthZSimpleClass, (klass), \
+TYPE_QAUTHZ_SIMPLE)
+#define QAUTHZ_SIMPLE_GET_CLASS(obj) \
+ OBJECT_GET_CLASS(QAuthZSimpleClass, (obj), \
+  TYPE_QAUTHZ_SIMPLE)
+#define QAUTHZ_SIMPLE(obj) \
+ INTERFACE_CHECK(QAuthZSimple, (obj), \
+ TYPE_QAUTHZ_SIMPLE)
+
+typedef struct QAuthZSimple QAuthZSimple;
+typedef struct QAuthZSimpleClass QAuthZSimpleClass;
+
+
+/**
+ * QAuthZSimple:
+ *
+ * This authorization driver provides a simple mechanism
+ * for granting access by matching user names against a
+ * list of globs. Each match rule has an associated policy
+ * and a catch all policy applies if no rule matches
+ *
+ * To create an instace of this class via QMP:
+ *
+ *  {
+ *"execute": "object-add",
+ *"arguments": {
+ *  "qom-type": "authz-simple",
+ *  "id": "auth0",
+ *  "parameters": {
+ *"rules": [
+ *   { "match": "fred", "policy": "allow" },
+ *   { "match": "bob", "policy": "allow" },
+ *   { "match": "danb", "policy": "deny" },
+ *   { "match": "dan*", "policy": "allow" }
+ *],
+ *"policy": "deny"
+ *  }
+ *}
+ *  }
+ *
+ * Or via the CLI:
+ *
+ *   $QEMU  \
+ *-object authz

[Qemu-block] [PATCH v3 00/10] Provide a QOM-based authorization API

2016-03-10 Thread Daniel P. Berrange 
Many years ago I was responsible for adding the 'qemu_acl' type
and associated HMP commands. Looking back at it now, it is quite
a poor facility with a couple of bad limitations. First, the
responsibility for creating the ACLs was left with the QEMU network
service (VNC server was only thing ever doing it). This meant you
could not share ACLs across multiple services. Second, there was
no way to populate ACLs on the command line, you had no choice but
to use the HMP commands. Third, the API was hardcoded around the
idea of an in-QEMU implementation, leaving no scope for plugging
in alternative implementations backed by, for exmaple, LDAP or PAM.

This series introduces a much better authorization API design
to QEMU that addresses all these problems, and maintains back
compatibility. It of course is based on the QOM framework, so
that immediately gives us ability to create objects via the
CLI, HMP or QMP. There is an abstract base clss "QAuthZ" which
defines the basic API for QEMU network services to use, and a
specific implementation "QAuthZ" simple which replicates the
functionality of 'qemu_acl'. It is thus possible to add other
impls, without changing any other part of QEMU in the future.
Finally, the user is responsible for creating the ACL objects,
so they can have one ACL associated with all their TLS enabled
network services.

There was only one small problem with this, specifically the
-object CLI arg and HMP 'object_add' command had no way to let
the user specify non-scalar properties for objects. eg if an
object had a property which is a list of structs, you are out
of luck if you want to create it without using QMP.

Thus the first three patches do some work around QAPI / QOM
to make it possible to specify non-scalar properties with
the -object CLI arg and HMP 'object_add' command. See the
respective patches for illustration of the syntax used.

The patches 4 and 5 introduce the new base class and specific
implementation.

Patch 6 kills the old qemu_acl code, updating any existing
callers of it to use the QAuthZSimple QOM class instead.

Patches 7-10 add support for associating ACLs with the
network services supporting TLS encryption (NBD, chardev
and VNC).

Aside from the outstanding migration TLS patches, this series
wraps up the feature based work I have for TLS in this release
cycle.

Changed in v3:

 - Created separate qdict_list_size method (Max)
 - Added unit tests for case of empty dict (Max)
 - Fix variable names to use underscore separator (Max)
 - Fix potential free of uninitialized variables (Max)
 - Use QObject APIs for casts, instead of C type casts (Max)

Changed in v2:

 - Adapt to changes in qapi visitor APIs
 - Add a 'bool recursive' flag to qdict_crumple (Max)
 - Fix memory leaks in qdict_crumple (Max)
 - Split out key splitting code from qdict_crumple (Max)
 - Use saner variable names in qdict_crumple (Max)
 - Added some tests for bad inputs to qdict_crumple

Daniel P. Berrange (10):
  qdict: implement a qdict_crumple method for un-flattening a dict
  qapi: allow QmpInputVisitor to auto-cast types
  qom: support arbitrary non-scalar properties with -object
  util: add QAuthZ object as an authorization base class
  util: add QAuthZSimple object type for a simple access control list
  acl: delete existing ACL implementation
  qemu-nbd: add support for ACLs for TLS clients
  nbd: allow an ACL to be set with nbd-server-start QMP command
  chardev: add support for ACLs for TLS clients
  vnc: allow specifying a custom ACL object name

 MAINTAINERS  |   7 +
 Makefile |   9 +-
 Makefile.objs|   2 +
 Makefile.target  |   2 +
 blockdev-nbd.c   |  10 +-
 crypto/tlssession.c  |  28 +++-
 hmp.c|  20 +--
 include/qapi/qmp-input-visitor.h |   3 +
 include/qapi/qmp/qdict.h |   1 +
 include/qemu/acl.h   |  74 --
 include/qemu/authz-simple.h  | 107 ++
 include/qemu/authz.h |  81 +++
 monitor.c| 161 +
 qapi-schema.json |   8 +-
 qapi/block.json  |   4 +-
 qapi/qmp-input-visitor.c |  96 +++--
 qapi/util.json   |  31 
 qemu-char.c  |  11 +-
 qemu-nbd.c   |  13 +-
 qemu-nbd.texi|   4 +
 qmp-commands.hx  |   2 +-
 qobject/qdict.c  | 267 +++
 qom/object_interfaces.c  |  20 ++-
 tests/.gitignore |   1 +
 tests/Makefile   |   5 +-
 tests/check-qdict.c  | 143 +++
 tests/check-qom-proplist.c   | 295 ++-
 tests/test-authz-simple.c| 156 +
 tests/test-crypto-tlssession.c   |  13 +-
 tests/test-io-channel-tls.c  |  14 +-
 tests/test-qmp-input

[Qemu-block] [PATCH v3] block/gluster: add support for SEEK_DATA/SEEK_HOLE

2016-03-10 Thread Niels de Vos 
GlusterFS 3.8 contains support for SEEK_DATA and SEEK_HOLE. This makes
it possible to detect sparse areas in files.

Signed-off-by: Niels de Vos 

---
Tested by compiling and running "qemu-img map gluster://..." with a
build of the current master branch of glusterfs. Using a Fedora cloud
image (in raw format) shows many SEEK procudure calls going back and
forth over the network. The output of "qemu map" matches the output when
run against the image on the local filesystem.

v2 based on feedback from Jeff Cody:
- Replace compile time detection by runtime detection
- Update return pointer (new argument) for .bdrv_co_get_block_status
---
 block/gluster.c | 182 
 1 file changed, 182 insertions(+)

diff --git a/block/gluster.c b/block/gluster.c
index 65077a0..a4f0628 100644
--- a/block/gluster.c
+++ b/block/gluster.c
@@ -23,6 +23,7 @@ typedef struct GlusterAIOCB {
 typedef struct BDRVGlusterState {
 struct glfs *glfs;
 struct glfs_fd *fd;
+bool supports_seek_data;
 } BDRVGlusterState;
 
 typedef struct GlusterConf {
@@ -286,6 +287,28 @@ static void qemu_gluster_parse_flags(int bdrv_flags, int 
*open_flags)
 }
 }
 
+/*
+ * Do SEEK_DATA/HOLE to detect if it is functional. Older broken versions of
+ * gfapi incorrectly return the current offset when SEEK_DATA/HOLE is used.
+ * - Corrected versions return -1 and set errno to EINVAL.
+ * - Versions that support SEEK_DATA/HOLE correctly, will return -1 and set
+ *   errno to ENXIO when SEEK_DATA is called with a position of EOF.
+ */
+static bool qemu_gluster_test_seek(struct glfs_fd *fd)
+{
+off_t ret, eof;
+
+eof = glfs_lseek(fd, 0, SEEK_END);
+if (eof < 0) {
+/* this should never occur */
+return false;
+}
+
+/* this should always fail with ENXIO if SEEK_DATA is supported */
+ret = glfs_lseek(fd, eof, SEEK_DATA);
+return (ret < 0) && (errno == ENXIO);
+}
+
 static int qemu_gluster_open(BlockDriverState *bs,  QDict *options,
  int bdrv_flags, Error **errp)
 {
@@ -320,6 +343,8 @@ static int qemu_gluster_open(BlockDriverState *bs,  QDict 
*options,
 ret = -errno;
 }
 
+s->supports_seek_data = qemu_gluster_test_seek(s->fd);
+
 out:
 qemu_opts_del(opts);
 qemu_gluster_gconf_free(gconf);
@@ -677,6 +702,159 @@ static int qemu_gluster_has_zero_init(BlockDriverState 
*bs)
 return 0;
 }
 
+/*
+ * Find allocation range in @bs around offset @start.
+ * May change underlying file descriptor's file offset.
+ * If @start is not in a hole, store @start in @data, and the
+ * beginning of the next hole in @hole, and return 0.
+ * If @start is in a non-trailing hole, store @start in @hole and the
+ * beginning of the next non-hole in @data, and return 0.
+ * If @start is in a trailing hole or beyond EOF, return -ENXIO.
+ * If we can't find out, return a negative errno other than -ENXIO.
+ *
+ * (Shamefully copied from raw-posix.c, only miniscule adaptions.)
+ */
+static int find_allocation(BlockDriverState *bs, off_t start,
+   off_t *data, off_t *hole)
+{
+BDRVGlusterState *s = bs->opaque;
+off_t offs;
+
+if (!s->supports_seek_data) {
+return -ENOTSUP;
+}
+
+/*
+ * SEEK_DATA cases:
+ * D1. offs == start: start is in data
+ * D2. offs > start: start is in a hole, next data at offs
+ * D3. offs < 0, errno = ENXIO: either start is in a trailing hole
+ *  or start is beyond EOF
+ * If the latter happens, the file has been truncated behind
+ * our back since we opened it.  All bets are off then.
+ * Treating like a trailing hole is simplest.
+ * D4. offs < 0, errno != ENXIO: we learned nothing
+ */
+offs = glfs_lseek(s->fd, start, SEEK_DATA);
+if (offs < 0) {
+return -errno;  /* D3 or D4 */
+}
+assert(offs >= start);
+
+if (offs > start) {
+/* D2: in hole, next data at offs */
+*hole = start;
+*data = offs;
+return 0;
+}
+
+/* D1: in data, end not yet known */
+
+/*
+ * SEEK_HOLE cases:
+ * H1. offs == start: start is in a hole
+ * If this happens here, a hole has been dug behind our back
+ * since the previous lseek().
+ * H2. offs > start: either start is in data, next hole at offs,
+ *   or start is in trailing hole, EOF at offs
+ * Linux treats trailing holes like any other hole: offs ==
+ * start.  Solaris seeks to EOF instead: offs > start (blech).
+ * If that happens here, a hole has been dug behind our back
+ * since the previous lseek().
+ * H3. offs < 0, errno = ENXIO: start is beyond EOF
+ * If this happens, the file has been truncated behind our
+ * back since we opened it.  Treat it like a trailing hole.
+ * H4. offs < 0, errno != ENXIO: we learned nothing
+ * Pretend we know nothing at

Re: [Qemu-block] [PATCH] Replaced get_ticks_per_sec calls with NANOSECONDS_PER_SECOND

2016-03-10 Thread Paolo Bonzini 


On 10/03/2016 18:22, rutuja shah wrote:
> Hi,
> As there are no callers to get_ticks_per_sec() function, definition of
> this function could be removed completely?

Yes, please.

> diff --git a/backends/baum.c b/backends/baum.c
> index c11320e..20b49f2 100644
> --- a/backends/baum.c
> +++ b/backends/baum.c
> @@ -336,7 +336,7 @@ static int baum_eat_packet(BaumDriverState *baum,
> const uint8_t *buf, int len)
> 
>  /* Allow 100ms to complete the DisplayData packet */
>  timer_mod(baum->cellCount_timer,
> qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL) +
> -   get_ticks_per_sec() / 10);
> +   NANOSECONDS_PER_SECOND / 10);

There is a problem with your patch, probably because you're using gmail;
lines are wrapped (for example the timer_mod line shows as two lines).

You can use the smtp.gmail.com server and git-send-email to avoid this
issue.

Thanks,

Paolo

Re: [Qemu-block] [PATCH v12 1/3] Add new block driver interface to add/delete a BDS's child

2016-03-10 Thread Alberto Garcia 
On Thu 10 Mar 2016 03:49:39 AM CET, Changlong Xie wrote:
> From: Wen Congyang 
>
> In some cases, we want to take a quorum child offline, and take
> another child online.
>
> Signed-off-by: Wen Congyang 
> Signed-off-by: zhanghailiang 
> Signed-off-by: Gonglei 
> Signed-off-by: Changlong Xie 
> Reviewed-by: Max Reitz 

> +if (!tmp) {
> +error_setg(errp, "The node %s does not have child named %s",
> +   bdrv_get_device_or_node_name(parent_bs),
> +   bdrv_get_device_or_node_name(child->bs));
> +return;

I think it should be "does not have a child" or "does not have any
child".

With that corrected,

Reviewed-by: Alberto Garcia 

Berto

[Qemu-block] [PATCH v2 1/4] quorum: Fix crash in quorum_aio_cb()

2016-03-10 Thread Alberto Garcia 
quorum_aio_cb() emits the QUORUM_REPORT_BAD event if there's
an I/O error in a Quorum child. However sacb->aiocb must be
correctly initialized for this to happen. read_quorum_children() and
read_fifo_child() are not doing this, which results in a QEMU crash.

Signed-off-by: Alberto Garcia 
Reviewed-by: Max Reitz 
---
 block/quorum.c | 12 +++-
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/block/quorum.c b/block/quorum.c
index 11cc60b..73ff309 100644
--- a/block/quorum.c
+++ b/block/quorum.c
@@ -648,8 +648,9 @@ static BlockAIOCB *read_quorum_children(QuorumAIOCB *acb)
 }
 
 for (i = 0; i < s->num_children; i++) {
-bdrv_aio_readv(s->children[i]->bs, acb->sector_num, &acb->qcrs[i].qiov,
-   acb->nb_sectors, quorum_aio_cb, &acb->qcrs[i]);
+acb->qcrs[i].aiocb = bdrv_aio_readv(s->children[i]->bs, 
acb->sector_num,
+&acb->qcrs[i].qiov, 
acb->nb_sectors,
+quorum_aio_cb, &acb->qcrs[i]);
 }
 
 return &acb->common;
@@ -664,9 +665,10 @@ static BlockAIOCB *read_fifo_child(QuorumAIOCB *acb)
 qemu_iovec_init(&acb->qcrs[acb->child_iter].qiov, acb->qiov->niov);
 qemu_iovec_clone(&acb->qcrs[acb->child_iter].qiov, acb->qiov,
  acb->qcrs[acb->child_iter].buf);
-bdrv_aio_readv(s->children[acb->child_iter]->bs, acb->sector_num,
-   &acb->qcrs[acb->child_iter].qiov, acb->nb_sectors,
-   quorum_aio_cb, &acb->qcrs[acb->child_iter]);
+acb->qcrs[acb->child_iter].aiocb =
+bdrv_aio_readv(s->children[acb->child_iter]->bs, acb->sector_num,
+   &acb->qcrs[acb->child_iter].qiov, acb->nb_sectors,
+   quorum_aio_cb, &acb->qcrs[acb->child_iter]);
 
 return &acb->common;
 }
-- 
2.7.0

[Qemu-block] [PATCH v2 4/4] iotests: Add test for QMP event rates

2016-03-10 Thread Alberto Garcia 
This test verifies that the rate-limited QMP events are emitted at a
maximum rate of 1 per second as defined in monitor_qapi_event_conf in
monitor.c

It also checks that QUORUM_REPORT_BAD events generated from different
nodes are kept in separate queues so they don't mask each other.

Signed-off-by: Alberto Garcia 
Reviewed-by: Max Reitz 
---
 tests/qemu-iotests/146 | 129 +
 tests/qemu-iotests/146.out |   5 ++
 tests/qemu-iotests/group   |   1 +
 3 files changed, 135 insertions(+)
 create mode 100644 tests/qemu-iotests/146
 create mode 100644 tests/qemu-iotests/146.out

diff --git a/tests/qemu-iotests/146 b/tests/qemu-iotests/146
new file mode 100644
index 000..30bc379
--- /dev/null
+++ b/tests/qemu-iotests/146
@@ -0,0 +1,129 @@
+#!/usr/bin/env python
+#
+# Test the rate limit of QMP events
+#
+# Copyright (C) 2016 Igalia, S.L.
+# Author: Alberto Garcia 
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see .
+#
+
+import os
+import iotests
+
+imgs = (os.path.join(iotests.test_dir, 'quorum0.img'),
+os.path.join(iotests.test_dir, 'quorum1.img'),
+os.path.join(iotests.test_dir, 'quorum2.img'))
+
+img_conf = (os.path.join(iotests.test_dir, 'quorum0.conf'),
+os.path.join(iotests.test_dir, 'quorum1.conf'),
+os.path.join(iotests.test_dir, 'quorum2.conf'))
+
+event_rate = 10
+sector_size = 512
+offset = 10
+
+class TestQuorumEvents(iotests.QMPTestCase):
+
+def create_blkdebug_file(self, blkdebug_file, bad_sector):
+file = open(blkdebug_file, 'w')
+file.write('''
+[inject-error]
+event = "read_aio"
+errno = "5"
+sector = "%d"
+''' % bad_sector)
+file.close()
+
+def setUp(self):
+driveopts = ['driver=quorum', 'vote-threshold=2']
+for i in range(len(imgs)):
+iotests.qemu_img('create', '-f', iotests.imgfmt, imgs[i], '1M')
+self.create_blkdebug_file(img_conf[i], i + offset)
+driveopts.append('children.%d.driver=%s' % (i, iotests.imgfmt))
+driveopts.append('children.%d.file.driver=blkdebug' % i)
+driveopts.append('children.%d.file.config=%s' % (i, img_conf[i]))
+driveopts.append('children.%d.file.image.filename=%s' % (i, 
imgs[i]))
+driveopts.append('children.%d.node-name=img%d' % (i, i))
+self.vm = iotests.VM()
+self.vm.add_drive(None, opts = ','.join(driveopts))
+self.vm.launch()
+
+def tearDown(self):
+self.vm.shutdown()
+for i in range(len(imgs)):
+os.remove(imgs[i])
+os.remove(img_conf[i])
+
+def do_check_event(self, node, sector = 0):
+if node == None:
+self.assertEqual(self.vm.get_qmp_event(), None)
+return
+
+for event in self.vm.get_qmp_events(wait=True):
+if event['event'] == 'QUORUM_REPORT_BAD':
+self.assert_qmp(event, 'data/node-name', node)
+self.assert_qmp(event, 'data/sector-num', sector)
+
+def testQuorum(self):
+if not 'quorum' in iotests.qemu_img_pipe('--help'):
+return
+
+# Generate an error and get an event
+self.vm.hmp_qemu_io("drive0", "aio_read %d %d" %
+(offset * sector_size, sector_size))
+self.vm.qtest("clock_step 10")
+self.do_check_event('img0', offset)
+
+# I/O errors in the same child: only one event is emitted
+delay = 10
+for i in range(3):
+self.vm.hmp_qemu_io("drive0", "aio_read %d %d" %
+(offset * sector_size, sector_size))
+self.vm.qtest("clock_step %d" % delay)
+self.do_check_event(None)
+
+# Wait enough so the event is finally emitted
+self.vm.qtest("clock_step %d" % (2 * event_rate))
+self.do_check_event('img0', offset)
+
+# I/O errors in the same child: all events are emitted
+delay = 2 * event_rate
+for i in range(3):
+self.vm.hmp_qemu_io("drive0", "aio_read %d %d" %
+(offset * sector_size, sector_size))
+self.vm.qtest("clock_step %d" % delay)
+self.do_check_event('img0', offset)
+
+# I/O errors in different children: all events are emitted
+delay = 10
+for i in range(len(imgs)):
+self.vm.hmp_qe

[Qemu-block] [PATCH v2 0/4] Separate QUORUM_REPORT_BAD events according to their node name

2016-03-10 Thread Alberto Garcia 
QUORUM_REPORT_BAD events are limited to a maximum rate of 1 per
second. While this is not a problem in itself, this means that an
error in one a Quorum child will mask errors in the other children if
they happen within the same 1 second interval.

This series fixes that problem by separating these events in different
queues if they come from different nodes. Once we add the 'type' field
to QUORUM_REPORT_BAD we will also be able to classify them according
to the type if we want.

In addition to the above, this series also fixes a crash that happens
if there's an I/O error in one of the children. This is serious enough
so I'll send the patch to fix this crash to qemu-stable as well.

Regards,

Berto

v2:
- Patch 3: Rename clock_type to event_clock_type [Max]

v1: https://lists.gnu.org/archive/html/qemu-devel/2016-03/msg02161.html
- Original version

Alberto Garcia (4):
  quorum: Fix crash in quorum_aio_cb()
  monitor: Separate QUORUM_REPORT_BAD events according to the node name
  monitor: Use QEMU_CLOCK_VIRTUAL for the event queue in qtest mode
  iotests: Add test for QMP event rates

 block/quorum.c |  12 +++--
 monitor.c  |  22 ++--
 tests/qemu-iotests/146 | 129 +
 tests/qemu-iotests/146.out |   5 ++
 tests/qemu-iotests/group   |   1 +
 5 files changed, 161 insertions(+), 8 deletions(-)
 create mode 100644 tests/qemu-iotests/146
 create mode 100644 tests/qemu-iotests/146.out

-- 
2.7.0

[Qemu-block] [PATCH v2 3/4] monitor: Use QEMU_CLOCK_VIRTUAL for the event queue in qtest mode

2016-03-10 Thread Alberto Garcia 
This allows us to perform tests on the monitor queues to verify that
the rate limits are enforced.

Signed-off-by: Alberto Garcia 
Reviewed-by: Max Reitz 
---
 monitor.c | 13 ++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/monitor.c b/monitor.c
index c9fe862..894f862 100644
--- a/monitor.c
+++ b/monitor.c
@@ -76,6 +76,7 @@
 #include "qapi-event.h"
 #include "qmp-introspect.h"
 #include "sysemu/block-backend.h"
+#include "sysemu/qtest.h"
 
 /* for hmp_info_irq/pic */
 #if defined(TARGET_SPARC)
@@ -232,6 +233,8 @@ static const mon_cmd_t qmp_cmds[];
 
 Monitor *cur_mon;
 
+static QEMUClockType event_clock_type = QEMU_CLOCK_REALTIME;
+
 static void monitor_command_cb(void *opaque, const char *cmdline,
void *readline_opaque);
 
@@ -513,7 +516,7 @@ monitor_qapi_event_queue(QAPIEvent event, QDict *qdict, 
Error **errp)
  * monitor_qapi_event_handler() in evconf->rate ns.  Any
  * events arriving before then will be delayed until then.
  */
-int64_t now = qemu_clock_get_ns(QEMU_CLOCK_REALTIME);
+int64_t now = qemu_clock_get_ns(event_clock_type);
 
 monitor_qapi_event_emit(event, qdict);
 
@@ -522,7 +525,7 @@ monitor_qapi_event_queue(QAPIEvent event, QDict *qdict, 
Error **errp)
 evstate->data = data;
 QINCREF(evstate->data);
 evstate->qdict = NULL;
-evstate->timer = timer_new_ns(QEMU_CLOCK_REALTIME,
+evstate->timer = timer_new_ns(event_clock_type,
   monitor_qapi_event_handler,
   evstate);
 g_hash_table_add(monitor_qapi_event_state, evstate);
@@ -547,7 +550,7 @@ static void monitor_qapi_event_handler(void *opaque)
 qemu_mutex_lock(&monitor_lock);
 
 if (evstate->qdict) {
-int64_t now = qemu_clock_get_ns(QEMU_CLOCK_REALTIME);
+int64_t now = qemu_clock_get_ns(event_clock_type);
 
 monitor_qapi_event_emit(evstate->event, evstate->qdict);
 QDECREF(evstate->qdict);
@@ -603,6 +606,10 @@ static gboolean qapi_event_throttle_equal(const void *a, 
const void *b)
 
 static void monitor_qapi_event_init(void)
 {
+if (qtest_enabled()) {
+event_clock_type = QEMU_CLOCK_VIRTUAL;
+}
+
 monitor_qapi_event_state = g_hash_table_new(qapi_event_throttle_hash,
 qapi_event_throttle_equal);
 qmp_event_set_func_emit(monitor_qapi_event_queue);
-- 
2.7.0

[Qemu-block] [PATCH v2 2/4] monitor: Separate QUORUM_REPORT_BAD events according to the node name

2016-03-10 Thread Alberto Garcia 
The QUORUM_REPORT_BAD event is emitted whenever there's an I/O error
in a child of a Quorum device. This event is emitted at a maximum rate
of 1 per second. This means that an error in one of the children will
mask errors in the other children if they happen within the same 1
second interval.

This patch modifies qapi_event_throttle_equal() so QUORUM_REPORT_BAD
events are kept separately if they come from different children.

Signed-off-by: Alberto Garcia 
Reviewed-by: Max Reitz 
---
 monitor.c | 9 +
 1 file changed, 9 insertions(+)

diff --git a/monitor.c b/monitor.c
index e99ca8c..c9fe862 100644
--- a/monitor.c
+++ b/monitor.c
@@ -572,6 +572,10 @@ static unsigned int qapi_event_throttle_hash(const void 
*key)
 hash += g_str_hash(qdict_get_str(evstate->data, "id"));
 }
 
+if (evstate->event == QAPI_EVENT_QUORUM_REPORT_BAD) {
+hash += g_str_hash(qdict_get_str(evstate->data, "node-name"));
+}
+
 return hash;
 }
 
@@ -589,6 +593,11 @@ static gboolean qapi_event_throttle_equal(const void *a, 
const void *b)
qdict_get_str(evb->data, "id"));
 }
 
+if (eva->event == QAPI_EVENT_QUORUM_REPORT_BAD) {
+return !strcmp(qdict_get_str(eva->data, "node-name"),
+   qdict_get_str(evb->data, "node-name"));
+}
+
 return TRUE;
 }
 
-- 
2.7.0

Re: [Qemu-block] [PATCH 1/4] quorum: Fix crash in quorum_aio_cb()

2016-03-10 Thread Alberto Garcia 
On Wed 09 Mar 2016 05:54:55 PM CET, Max Reitz  wrote:
> On 09.03.2016 17:11, Alberto Garcia wrote:
>> quorum_aio_cb() emits the QUORUM_REPORT_BAD event if there's
>> an I/O error in a Quorum child. However sacb->aiocb must be
>> correctly initialized for this to happen. read_quorum_children() and
>> read_fifo_child() are not doing this, which results in a QEMU crash.
>> 
>> Signed-off-by: Alberto Garcia 
>> ---
>>  block/quorum.c | 12 +++-
>>  1 file changed, 7 insertions(+), 5 deletions(-)
>
> Reviewed-by: Max Reitz 
>
> Although I'm wondering whether we could have just used acb->common.bs
> instead of sacb->aiocb->bs in quorum_aio_cb(). I guess that
> sacb->aiocb is supposed to be equal to &acb->common.

acb->common.bs is the Quorum BDS, sacb->aiocb->bs is the child BDS.

Berto