Re: [Qemu-devel] I want to recover my QEMU password.

[Stefan Weil]

Am 11.09.2015 um 17:38 schrieb SkyKiDS:
> Hi, there!
>
> Long long time ago, in 2010, I am a member of QEMU wiki, just had an
> account, and contributed some news or corrected mis-spells.
> But now, I forgot my password of the site, so hope to recover my password.
> Anyone can help me?
>
> My account is SkyKiDS . my registered email maybe nub...@gmail.com or this.
>
> Any help will be appreciated. Thx. :)

There are two accounts: SkyKIDS and Skykids. Are both yours?
Skykids has no contributions, so I think that account could be
removed.

It looks like there is no mailing function enabled for the wiki,
so I cannot send an e-mail to you using the wiki, nor
can you request a new password per e-mail.

You'll need help from an administrator...

Stefan

Re: [Qemu-devel] [PATCH 04/29] Introduce QDict

[Luiz Capitulino]

On Fri, 11 Sep 2015 20:22:38 -0400
Programmingkid  wrote:

> Could you make a tutorial on how to use the QDict type?

There are several examples in tests/check-qdict.c.

[Qemu-devel] [PATCH 04/29] Introduce QDict

[Programmingkid]

Could you make a tutorial on how to use the QDict type?

Re: [Qemu-devel] [PATCH V3 2/2] sdhci: Split sdhci.h for public and internal device usage

[Alistair Francis]

On Thu, Sep 10, 2015 at 4:25 AM, Sai Pavan Boddu
 wrote:
> Split sdhci.h into sdhci-common.h and sdhci.h separating common
> register declarations and object declarations. And move sdhci.h to
> include/hw/sd/.
>
> Signed-off-by: Sai Pavan Boddu 
> ---
> Changes for V2:
> Create new area in includes for sd. And move sdhci.h to same.
> Changes for V3:
> Split the headers to public and common.
> ---
>  hw/sd/{sdhci.h => sdhci-common.h} | 71 ++
>  hw/sd/sdhci.c |  3 +-
>  include/hw/sd/sdhci.h | 91 
> +++

I don't think the names of these files make sense. The one in the
include directory should be the 'common' or 'public' header. While the
other one should be the 'internal' one.

Thanks,

Alistair

>  3 files changed, 96 insertions(+), 69 deletions(-)
>  rename hw/sd/{sdhci.h => sdhci-common.h} (75%)
>  create mode 100644 include/hw/sd/sdhci.h
>
> diff --git a/hw/sd/sdhci.h b/hw/sd/sdhci-common.h
> similarity index 75%
> rename from hw/sd/sdhci.h
> rename to hw/sd/sdhci-common.h
> index a45593f..2f83236 100644
> --- a/hw/sd/sdhci.h
> +++ b/hw/sd/sdhci-common.h
> @@ -21,14 +21,10 @@
>   * You should have received a copy of the GNU General Public License along
>   * with this program; if not, see .
>   */
> +#ifndef SDHCI_COMMON_H
> +#define SDHCI_COMMON_H
>
> -#ifndef SDHCI_H
> -#define SDHCI_H
> -
> -#include "qemu-common.h"
> -#include "hw/pci/pci.h"
> -#include "hw/sysbus.h"
> -#include "hw/sd/sd.h"
> +#include "hw/sd/sdhci.h"
>
>  /* R/W SDMA System Address register 0x0 */
>  #define SDHC_SYSAD 0x00
> @@ -231,65 +227,6 @@ enum {
>  sdhc_gap_write  = 2   /* SDHC stopped at block gap during write 
> operation */
>  };
>
> -/* SD/MMC host controller state */
> -typedef struct SDHCIState {
> -union {
> -PCIDevice pcidev;
> -SysBusDevice busdev;
> -};
> -SDState *card;
> -MemoryRegion iomem;
> -
> -QEMUTimer *insert_timer;   /* timer for 'changing' sd card. */
> -QEMUTimer *transfer_timer;
> -qemu_irq eject_cb;
> -qemu_irq ro_cb;
> -qemu_irq irq;
> -
> -uint32_t sdmasysad;/* SDMA System Address register */
> -uint16_t blksize;  /* Host DMA Buff Boundary and Transfer BlkSize 
> Reg */
> -uint16_t blkcnt;   /* Blocks count for current transfer */
> -uint32_t argument; /* Command Argument Register */
> -uint16_t trnmod;   /* Transfer Mode Setting Register */
> -uint16_t cmdreg;   /* Command Register */
> -uint32_t rspreg[4];/* Response Registers 0-3 */
> -uint32_t prnsts;   /* Present State Register */
> -uint8_t  hostctl;  /* Host Control Register */
> -uint8_t  pwrcon;   /* Power control Register */
> -uint8_t  blkgap;   /* Block Gap Control Register */
> -uint8_t  wakcon;   /* WakeUp Control Register */
> -uint16_t clkcon;   /* Clock control Register */
> -uint8_t  timeoutcon;   /* Timeout Control Register */
> -uint8_t  admaerr;  /* ADMA Error Status Register */
> -uint16_t norintsts;/* Normal Interrupt Status Register */
> -uint16_t errintsts;/* Error Interrupt Status Register */
> -uint16_t norintstsen;  /* Normal Interrupt Status Enable Register */
> -uint16_t errintstsen;  /* Error Interrupt Status Enable Register */
> -uint16_t norintsigen;  /* Normal Interrupt Signal Enable Register */
> -uint16_t errintsigen;  /* Error Interrupt Signal Enable Register */
> -uint16_t acmd12errsts; /* Auto CMD12 error status register */
> -uint64_t admasysaddr;  /* ADMA System Address Register */
> -
> -uint32_t capareg;  /* Capabilities Register */
> -uint32_t maxcurr;  /* Maximum Current Capabilities Register */
> -uint8_t  *fifo_buffer; /* SD host i/o FIFO buffer */
> -uint32_t buf_maxsz;
> -uint16_t data_count;   /* current element in FIFO buffer */
> -uint8_t  stopped_state;/* Current SDHC state */
> -/* Buffer Data Port Register - virtual access point to R and W buffers */
> -/* Software Reset Register - always reads as 0 */
> -/* Force Event Auto CMD12 Error Interrupt Reg - write only */
> -/* Force Event Error Interrupt Register- write only */
> -/* RO Host Controller Version Register always reads as 0x2401 */
> -} SDHCIState;
> -
>  extern const VMStateDescription sdhci_vmstate;
>
> -#define TYPE_PCI_SDHCI "sdhci-pci"
> -#define PCI_SDHCI(obj) OBJECT_CHECK(SDHCIState, (obj), TYPE_PCI_SDHCI)
> -
> -#define TYPE_SYSBUS_SDHCI "generic-sdhci"
> -#define SYSBUS_SDHCI(obj)   \
> - OBJECT_CHECK(SDHCIState, (obj), TYPE_SYSBUS_SDHCI)
> -
> -#endif /* SDHCI_H */
> +#endif
> diff --git a/hw/sd/sdhci.c b/hw/sd/sdhci.c
> index e63367b..a95e6e0 100644
> --- a/hw/sd/sdhci.c
> +++ b/hw/sd/sdhci.c
> @@ -28,8 +28,7 @@
>  #include "sysemu/dma.h"
>  #include "qemu/timer.h"
>  #include "q

Re: [Qemu-devel] [PATCH V3 1/2] hw/sd : move sd.h to include/hw/sd/

[Alistair Francis]

On Thu, Sep 10, 2015 at 4:25 AM, Sai Pavan Boddu
 wrote:
> Create new folder in includes for sd headers.

I'd change the commit message to something like:

sd.h: Move sd.h to include/hw/sd/

Create a sd directory under include/hw/ and move sd.h to that folder.

>
> Signed-off-by: Sai Pavan Boddu 

Otherwise:

Reviewed-by: Alistair Francis 

Thanks,

Alistair

> ---
> Changes for V3:
> None.
> ---
>  hw/sd/milkymist-memcard.c | 2 +-
>  hw/sd/omap_mmc.c  | 2 +-
>  hw/sd/pl181.c | 2 +-
>  hw/sd/pxa2xx_mmci.c   | 2 +-
>  hw/sd/sd.c| 2 +-
>  hw/sd/sdhci.h | 2 +-
>  hw/sd/ssi-sd.c| 2 +-
>  include/hw/{ => sd}/sd.h  | 0
>  8 files changed, 7 insertions(+), 7 deletions(-)
>  rename include/hw/{ => sd}/sd.h (100%)
>
> diff --git a/hw/sd/milkymist-memcard.c b/hw/sd/milkymist-memcard.c
> index 2209ef1..b430d56 100644
> --- a/hw/sd/milkymist-memcard.c
> +++ b/hw/sd/milkymist-memcard.c
> @@ -28,7 +28,7 @@
>  #include "qemu/error-report.h"
>  #include "sysemu/block-backend.h"
>  #include "sysemu/blockdev.h"
> -#include "hw/sd.h"
> +#include "hw/sd/sd.h"
>
>  enum {
>  ENABLE_CMD_TX   = (1<<0),
> diff --git a/hw/sd/omap_mmc.c b/hw/sd/omap_mmc.c
> index 35d8033..5bc4719 100644
> --- a/hw/sd/omap_mmc.c
> +++ b/hw/sd/omap_mmc.c
> @@ -18,7 +18,7 @@
>   */
>  #include "hw/hw.h"
>  #include "hw/arm/omap.h"
> -#include "hw/sd.h"
> +#include "hw/sd/sd.h"
>
>  struct omap_mmc_s {
>  qemu_irq irq;
> diff --git a/hw/sd/pl181.c b/hw/sd/pl181.c
> index 11fcd47..ddd9b6f 100644
> --- a/hw/sd/pl181.c
> +++ b/hw/sd/pl181.c
> @@ -10,7 +10,7 @@
>  #include "sysemu/block-backend.h"
>  #include "sysemu/blockdev.h"
>  #include "hw/sysbus.h"
> -#include "hw/sd.h"
> +#include "hw/sd/sd.h"
>
>  //#define DEBUG_PL181 1
>
> diff --git a/hw/sd/pxa2xx_mmci.c b/hw/sd/pxa2xx_mmci.c
> index d1fe6d5..b217080 100644
> --- a/hw/sd/pxa2xx_mmci.c
> +++ b/hw/sd/pxa2xx_mmci.c
> @@ -12,7 +12,7 @@
>
>  #include "hw/hw.h"
>  #include "hw/arm/pxa.h"
> -#include "hw/sd.h"
> +#include "hw/sd/sd.h"
>  #include "hw/qdev.h"
>
>  struct PXA2xxMMCIState {
> diff --git a/hw/sd/sd.c b/hw/sd/sd.c
> index a1ff465..0787e33 100644
> --- a/hw/sd/sd.c
> +++ b/hw/sd/sd.c
> @@ -31,7 +31,7 @@
>
>  #include "hw/hw.h"
>  #include "sysemu/block-backend.h"
> -#include "hw/sd.h"
> +#include "hw/sd/sd.h"
>  #include "qemu/bitmap.h"
>
>  //#define DEBUG_SD 1
> diff --git a/hw/sd/sdhci.h b/hw/sd/sdhci.h
> index 3352d23..a45593f 100644
> --- a/hw/sd/sdhci.h
> +++ b/hw/sd/sdhci.h
> @@ -28,7 +28,7 @@
>  #include "qemu-common.h"
>  #include "hw/pci/pci.h"
>  #include "hw/sysbus.h"
> -#include "hw/sd.h"
> +#include "hw/sd/sd.h"
>
>  /* R/W SDMA System Address register 0x0 */
>  #define SDHC_SYSAD 0x00
> diff --git a/hw/sd/ssi-sd.c b/hw/sd/ssi-sd.c
> index e4b2d4f..c49ff62 100644
> --- a/hw/sd/ssi-sd.c
> +++ b/hw/sd/ssi-sd.c
> @@ -13,7 +13,7 @@
>  #include "sysemu/block-backend.h"
>  #include "sysemu/blockdev.h"
>  #include "hw/ssi.h"
> -#include "hw/sd.h"
> +#include "hw/sd/sd.h"
>
>  //#define DEBUG_SSI_SD 1
>
> diff --git a/include/hw/sd.h b/include/hw/sd/sd.h
> similarity index 100%
> rename from include/hw/sd.h
> rename to include/hw/sd/sd.h
> --
> 1.9.1
>
>

Re: [Qemu-devel] [PATCH] pulseaudio: reduce 24s recording latency

[Marc-André Lureau]

Hi

On Fri, Sep 11, 2015 at 9:03 PM, Volker Rümelin  wrote:
> Current code doesn't provide pulseaudio buffer attributes for
> recording. Without buffer attributes pulseaudio uses a default
> buffer of 4MB. 4MB is approximately 24s 16bit stereo audio
> data at 44.1kHz.

Why isn't the buffer processed as soon as some data is available?


-- 
Marc-André Lureau

Re: [Qemu-devel] [PATCH v4 16/38] block: Add BlockBackendRootState

[Eric Blake]

On 07/20/2015 11:45 AM, Max Reitz wrote:
> This structure will store some of the state of the root BDS if the BDS
> tree is removed, so that state can be restored once a new BDS tree is
> inserted.
> 
> Signed-off-by: Max Reitz 
> ---
>  block/block-backend.c  | 37 +
>  include/block/block_int.h  | 10 ++
>  include/qemu/typedefs.h|  1 +
>  include/sysemu/block-backend.h |  2 ++
>  4 files changed, 50 insertions(+)

Reviewed-by: Eric Blake 

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH] Add a few argument filters to the seccomp sandbox

[Namsun Ch'o]

> On Thursday, September 10, 2015 03:48:52 PM Daniel P. Berrange wrote:
> > On Wed, Sep 09, 2015 at 09:55:33PM -0400, namn...@safe-mail.net wrote:
> > > +
> > > +/* shmget */
> > > +rc = seccomp_rule_add(ctx, SCMP_ACT_ALLOW, SCMP_SYS(shmget), 2,
> > > +SCMP_A0(SCMP_CMP_EQ, IPC_PRIVATE),
> > > +SCMP_A2(SCMP_CMP_EQ, IP_CREAT|0777));
> > 
> > I'm not familiar with semantics of these seccomp rules, but is this
> > saying that the second arg must be exactly equal to IP_CREAT|0777 ?
> 
> Yes.
> 
> > If the app passes IP_CREAT|0600, would that be permitted instead ?
> > The latter is what I see gtk2 source code passing for mode.
> 
> It wouldn't match the rule as written above, if it doesn't match any other 
> configured rules it would hit the default filter action.
> 
> -- 
> paul moore
> security @ redhat

I'd already replied to that but in a new top-level thread because I had an
updated patch. Did I misunderstand the "contributing a patch" FAQ? Anyway,
I added the IP_CREAT|0600 rule to the v2 patch:

https://lists.nongnu.org/archive/html/qemu-devel/2015-09/msg02953.html

[Qemu-devel] [PATCH] pulseaudio: reduce 24s recording latency

[Volker Rümelin]

Current code doesn't provide pulseaudio buffer attributes for
recording. Without buffer attributes pulseaudio uses a default
buffer of 4MB. 4MB is approximately 24s 16bit stereo audio
data at 44.1kHz.

This patch fixes bug #1492649

|Signed-off-by: Volker Rümelin |
---
 audio/paaudio.c | 9 -
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/audio/paaudio.c b/audio/paaudio.c
index fea6071..c1169d4 100644
--- a/audio/paaudio.c
+++ b/audio/paaudio.c
@@ -608,6 +608,7 @@ static int qpa_init_in(HWVoiceIn *hw, struct audsettings 
*as, void *drv_opaque)
 {
 int error;
 pa_sample_spec ss;
+pa_buffer_attr ba;
 struct audsettings obt_as = *as;
 PAVoiceIn *pa = (PAVoiceIn *) hw;
 paaudio *g = pa->g = drv_opaque;
@@ -616,6 +617,12 @@ static int qpa_init_in(HWVoiceIn *hw, struct audsettings 
*as, void *drv_opaque)
 ss.channels = as->nchannels;
 ss.rate = as->freq;
 
+ba.fragsize = pa_frame_size (&ss) * g->conf.samples;
+ba.maxlength = 5 * ba.fragsize;
+ba.tlength = -1;
+ba.prebuf = -1;
+ba.minreq = -1;
+
 obt_as.fmt = pa_to_audfmt (ss.format, &obt_as.endianness);
 
 pa->stream = qpa_simple_new (
@@ -625,7 +632,7 @@ static int qpa_init_in(HWVoiceIn *hw, struct audsettings 
*as, void *drv_opaque)
 g->conf.source,
 &ss,
 NULL,   /* channel map */
-NULL,   /* buffering attributes */
+&ba,/* buffering attributes */
 &error
 );
 if (!pa->stream) {
-- 
1.8.4.5

[Qemu-devel] I want to recover my QEMU password.

[SkyKiDS]

Hi, there!

Long long time ago, in 2010, I am a member of QEMU wiki, just had an
account, and contributed some news or corrected mis-spells.
But now, I forgot my password of the site, so hope to recover my password.
Anyone can help me?

My account is SkyKiDS . my registered email maybe nub...@gmail.com or this.

Any help will be appreciated. Thx. :)





-- 
하늘을 향해 쏴라, SkyKiDS!

Re: [Qemu-devel] [PATCH] Add a few argument filters to the seccomp sandbox

[Paul Moore]

On Thursday, September 10, 2015 03:48:52 PM Daniel P. Berrange wrote:
> On Wed, Sep 09, 2015 at 09:55:33PM -0400, namn...@safe-mail.net wrote:
> > +
> > +/* shmget */
> > +rc = seccomp_rule_add(ctx, SCMP_ACT_ALLOW, SCMP_SYS(shmget), 2,
> > +SCMP_A0(SCMP_CMP_EQ, IPC_PRIVATE),
> > +SCMP_A2(SCMP_CMP_EQ, IP_CREAT|0777));
> 
> I'm not familiar with semantics of these seccomp rules, but is this
> saying that the second arg must be exactly equal to IP_CREAT|0777 ?

Yes.

> If the app passes IP_CREAT|0600, would that be permitted instead ?
> The latter is what I see gtk2 source code passing for mode.

It wouldn't match the rule as written above, if it doesn't match any other 
configured rules it would hit the default filter action.

-- 
paul moore
security @ redhat

Re: [Qemu-devel] [PATCH v5 11/14] block/backup: support block job transactions

[Eric Blake]

On 09/07/2015 01:34 AM, Fam Zheng wrote:
> From: Stefan Hajnoczi 
> 
> Join the transaction when the 'transactional-cancel' QMP argument is
> true.
> 
> This ensures that the sync bitmap is not thrown away if another block
> job in the transaction is cancelled or fails.  This is critical so
> incremental backup with multiple disks can be retried in case of
> cancellation/failure.
> 
> Signed-off-by: Stefan Hajnoczi 
> Signed-off-by: Fam Zheng 
> ---

Interface review:

> +void qmp_blockdev_backup(const char *device, const char *target,
> + enum MirrorSyncMode sync,
> + bool has_speed, int64_t speed,
> + bool has_on_source_error,
> + BlockdevOnError on_source_error,
> + bool has_on_target_error,
> + BlockdevOnError on_target_error,
> + bool has_transactional_cancel,
> + bool transactional_cancel,
> + Error **errp)
> +{
> +if (has_transactional_cancel && transactional_cancel) {
> +error_setg(errp, "Transactional cancel can only be used in the "
> +   "'transaction' command");
> +return;
> +}

Hmm. It might be nicer if we had two separate qapi structs:

# used in 'blockdev-backup'
{ 'struct':'BlockdevBackup', 'data': { device ... on-target-error } }

# used in 'transaction'
{ 'struct':'BlockdevBackupTxn', 'base':'BlockdevBackup',
  'data': { 'transactional-cancel':'bool' } }

so that the user can't abuse the boolean from the wrong context.

[Side notes...
Furthermore, with pending changes coming down the qapi pipeline, we will
generate the C struct so that you can safely do
'(BlockdevBackup*)blockdev_backup_txn' to go from the child back to the
base class.
https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg02583.html

> +
> +do_blockdev_backup(device, target, sync, has_speed, speed,
> +   has_on_source_error, on_source_error,
> +   has_on_target_error, on_target_error,
> +   NULL, errp);
> +}

And with other changes coming down the pipe, you could write a function as:

void do_blockdev_backup(BlockdevBackup *args, Error **errp)

by adding 'box':true' to 'blockdev-backup' and make it a bit easier to
pass around all the data without breaking it into multiple parameters.
https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg02599.html

But we're not there yet - it may have to be a simplification we apply
after the fact. :)
...]

> +++ b/qapi/block-core.json
> @@ -736,6 +736,11 @@
>  #   default 'report' (no limitations, since this applies to
>  #   a different block device than @device).
>  #
> +# @transactional-cancel: #optional whether failure or cancellation of other
> +#block jobs with @transactional-cancel true in the 
> same
> +#transaction causes the whole group to cancel.
> +#(Since 2.5)

Mention default false.

> +#
>  # Note that @on-source-error and @on-target-error only affect background I/O.
>  # If an error occurs during a guest write request, the device's rerror/werror
>  # actions will be used.
> @@ -747,7 +752,8 @@
>  'sync': 'MirrorSyncMode', '*mode': 'NewImageMode',
>  '*speed': 'int', '*bitmap': 'str',
>  '*on-source-error': 'BlockdevOnError',
> -'*on-target-error': 'BlockdevOnError' } }
> +'*on-target-error': 'BlockdevOnError',
> +'*transactional-cancel': 'bool' } }

See my above comments about the idea of using a parent and child class,
rather than exposing this outside of transaction, especially since you
didn't document that it can't be used outside of a transaction.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v6 20/26] qapi: Fix output visitor to return qnull() instead of NULL

[Eric Blake]

On 09/11/2015 02:43 PM, Eric Blake wrote:

>> +++ b/tests/test-qmp-output-visitor.c
>> @@ -485,7 +485,7 @@ static void test_visitor_out_empty(TestOutputVisitorData 
>> *data,
>>  QObject *arg;
>>  
>>  arg = qmp_output_get_qobject(data->qov);
>> -g_assert(!arg);
>> +g_assert(qobject_type(arg) == QTYPE_QNULL);
>>  }
> 
> Missing
>  qobject_decref(arg);
> 
> Ultimately, the global qnull_ starts life with refcnt 1, and after this
> test should be back to 1.  But it is coming back as 3, so even with a
> qobject_decref, that would get it back down to 2.  So we are leaking a
> reference to qnull somewhere.
> 
> I'm still investigating, and may be able to find the patch

Squash this in, and you can have:
Reviewed-by: Eric Blake 

diff --git i/qapi/qmp-output-visitor.c w/qapi/qmp-output-visitor.c
index 2d6083e..b96849e 100644
--- i/qapi/qmp-output-visitor.c
+++ w/qapi/qmp-output-visitor.c
@@ -66,11 +66,7 @@ static QObject *qmp_output_first(QmpOutputVisitor *qov)
 {
 QStackEntry *e = QTAILQ_LAST(&qov->stack, QStack);

-if (!e) {
-return qnull();
-}
-
-return e->value;
+return e ? e->value : NULL;
 }

 static QObject *qmp_output_last(QmpOutputVisitor *qov)
@@ -190,6 +186,8 @@ QObject *qmp_output_get_qobject(QmpOutputVisitor *qov)
 QObject *obj = qmp_output_first(qov);
 if (obj) {
 qobject_incref(obj);
+} else {
+obj = qnull();
 }
 return obj;
 }
diff --git i/tests/test-qmp-output-visitor.c
w/tests/test-qmp-output-visitor.c
index 256bffd..8bd48f4 100644
--- i/tests/test-qmp-output-visitor.c
+++ w/tests/test-qmp-output-visitor.c
@@ -486,6 +486,9 @@ static void
test_visitor_out_empty(TestOutputVisitorData *data,

 arg = qmp_output_get_qobject(data->qov);
 g_assert(qobject_type(arg) == QTYPE_QNULL);
+/* Check that qnull reference counting is sane */
+g_assert(arg->refcnt == 2);
+qobject_decref(arg);
 }

 static void init_native_list(UserDefNativeListUnion *cvalue)


-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v6 20/26] qapi: Fix output visitor to return qnull() instead of NULL

[Eric Blake]

On 09/11/2015 01:18 PM, Markus Armbruster wrote:
> Before commit 1d10b44, it crashed.  Since then, it returns NULL, with
> a FIXME comment.  The FIXME is valid: code that assumes QObject *
> can't be null exists.  I'm not aware of a way to feed this problematic
> return value to code that actually chokes on null in the current code,
> but the next few commits will create one.
> 
> Commit 481b002 solved a very similar problem by introducing a special
> null QObject.  Use it here, and drop the FIXME.
> 
> Update the test accordingly.
> 
> Signed-off-by: Markus Armbruster 
> ---
>  qapi/qmp-output-visitor.c   | 3 +--
>  tests/test-qmp-output-visitor.c | 2 +-
>  2 files changed, 2 insertions(+), 3 deletions(-)
> 

> +++ b/tests/test-qmp-output-visitor.c
> @@ -485,7 +485,7 @@ static void test_visitor_out_empty(TestOutputVisitorData 
> *data,
>  QObject *arg;
>  
>  arg = qmp_output_get_qobject(data->qov);
> -g_assert(!arg);
> +g_assert(qobject_type(arg) == QTYPE_QNULL);
>  }

Missing
 qobject_decref(arg);

Ultimately, the global qnull_ starts life with refcnt 1, and after this
test should be back to 1.  But it is coming back as 3, so even with a
qobject_decref, that would get it back down to 2.  So we are leaking a
reference to qnull somewhere.

I'm still investigating, and may be able to find the patch

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

[Qemu-devel] [PULL 15/23] qcow2: Fix memory leak in qcow2_update_options() error path

[Kevin Wolf]

Signed-off-by: Kevin Wolf 
Reviewed-by: Eric Blake 
Reviewed-by: Max Reitz 
---
 block/qcow2.c | 12 ++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/block/qcow2.c b/block/qcow2.c
index c61d996..374a56d 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -597,8 +597,8 @@ static int qcow2_update_options(BlockDriverState *bs, QDict 
*options,
 const char *opt_overlap_check, *opt_overlap_check_template;
 int overlap_check_template = 0;
 uint64_t l2_cache_size, refcount_cache_size;
-Qcow2Cache *l2_table_cache;
-Qcow2Cache *refcount_block_cache;
+Qcow2Cache *l2_table_cache = NULL;
+Qcow2Cache *refcount_block_cache = NULL;
 uint64_t cache_clean_interval;
 bool use_lazy_refcounts;
 int i;
@@ -735,6 +735,14 @@ static int qcow2_update_options(BlockDriverState *bs, 
QDict *options,
 
 ret = 0;
 fail:
+if (ret < 0) {
+if (l2_table_cache) {
+qcow2_cache_destroy(bs, l2_table_cache);
+}
+if (refcount_block_cache) {
+qcow2_cache_destroy(bs, refcount_block_cache);
+}
+}
 qemu_opts_del(opts);
 opts = NULL;
 
-- 
1.8.3.1

[Qemu-devel] [PULL 11/23] qcow2: Factor out qcow2_update_options()

[Kevin Wolf]

Eventually we want to be able to change options at runtime. As a first
step towards that goal, separate some option handling code from the
general initialisation code in qcow2_open().

Signed-off-by: Kevin Wolf 
Reviewed-by: Eric Blake 
Reviewed-by: Max Reitz 
---
 block/qcow2.c | 135 +-
 1 file changed, 76 insertions(+), 59 deletions(-)

diff --git a/block/qcow2.c b/block/qcow2.c
index 7f06d37..4dd0699 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -589,6 +589,80 @@ static void read_cache_sizes(BlockDriverState *bs, 
QemuOpts *opts,
 }
 }
 
+static int qcow2_update_options(BlockDriverState *bs, QemuOpts *opts,
+int flags, Error **errp)
+{
+BDRVQcow2State *s = bs->opaque;
+const char *opt_overlap_check, *opt_overlap_check_template;
+int overlap_check_template = 0;
+int i;
+int ret;
+
+s->use_lazy_refcounts = qemu_opt_get_bool(opts, QCOW2_OPT_LAZY_REFCOUNTS,
+(s->compatible_features & QCOW2_COMPAT_LAZY_REFCOUNTS));
+
+s->discard_passthrough[QCOW2_DISCARD_NEVER] = false;
+s->discard_passthrough[QCOW2_DISCARD_ALWAYS] = true;
+s->discard_passthrough[QCOW2_DISCARD_REQUEST] =
+qemu_opt_get_bool(opts, QCOW2_OPT_DISCARD_REQUEST,
+  flags & BDRV_O_UNMAP);
+s->discard_passthrough[QCOW2_DISCARD_SNAPSHOT] =
+qemu_opt_get_bool(opts, QCOW2_OPT_DISCARD_SNAPSHOT, true);
+s->discard_passthrough[QCOW2_DISCARD_OTHER] =
+qemu_opt_get_bool(opts, QCOW2_OPT_DISCARD_OTHER, false);
+
+opt_overlap_check = qemu_opt_get(opts, QCOW2_OPT_OVERLAP);
+opt_overlap_check_template = qemu_opt_get(opts, 
QCOW2_OPT_OVERLAP_TEMPLATE);
+if (opt_overlap_check_template && opt_overlap_check &&
+strcmp(opt_overlap_check_template, opt_overlap_check))
+{
+error_setg(errp, "Conflicting values for qcow2 options '"
+   QCOW2_OPT_OVERLAP "' ('%s') and '" 
QCOW2_OPT_OVERLAP_TEMPLATE
+   "' ('%s')", opt_overlap_check, opt_overlap_check_template);
+ret = -EINVAL;
+goto fail;
+}
+if (!opt_overlap_check) {
+opt_overlap_check = opt_overlap_check_template ?: "cached";
+}
+
+if (!strcmp(opt_overlap_check, "none")) {
+overlap_check_template = 0;
+} else if (!strcmp(opt_overlap_check, "constant")) {
+overlap_check_template = QCOW2_OL_CONSTANT;
+} else if (!strcmp(opt_overlap_check, "cached")) {
+overlap_check_template = QCOW2_OL_CACHED;
+} else if (!strcmp(opt_overlap_check, "all")) {
+overlap_check_template = QCOW2_OL_ALL;
+} else {
+error_setg(errp, "Unsupported value '%s' for qcow2 option "
+   "'overlap-check'. Allowed are any of the following: "
+   "none, constant, cached, all", opt_overlap_check);
+ret = -EINVAL;
+goto fail;
+}
+
+s->overlap_check = 0;
+for (i = 0; i < QCOW2_OL_MAX_BITNR; i++) {
+/* overlap-check defines a template bitmask, but every flag may be
+ * overwritten through the associated boolean option */
+s->overlap_check |=
+qemu_opt_get_bool(opts, overlap_bool_option_names[i],
+  overlap_check_template & (1 << i)) << i;
+}
+
+if (s->use_lazy_refcounts && s->qcow_version < 3) {
+error_setg(errp, "Lazy refcounts require a qcow2 image with at least "
+   "qemu 1.1 compatibility level");
+ret = -EINVAL;
+goto fail;
+}
+
+ret = 0;
+fail:
+return ret;
+}
+
 static int qcow2_open(BlockDriverState *bs, QDict *options, int flags,
   Error **errp)
 {
@@ -600,8 +674,6 @@ static int qcow2_open(BlockDriverState *bs, QDict *options, 
int flags,
 Error *local_err = NULL;
 uint64_t ext_end;
 uint64_t l1_vm_state_index;
-const char *opt_overlap_check, *opt_overlap_check_template;
-int overlap_check_template = 0;
 uint64_t l2_cache_size, refcount_cache_size;
 uint64_t cache_clean_interval;
 
@@ -992,69 +1064,14 @@ static int qcow2_open(BlockDriverState *bs, QDict 
*options, int flags,
 }
 
 /* Enable lazy_refcounts according to image and command line options */
-s->use_lazy_refcounts = qemu_opt_get_bool(opts, QCOW2_OPT_LAZY_REFCOUNTS,
-(s->compatible_features & QCOW2_COMPAT_LAZY_REFCOUNTS));
-
-s->discard_passthrough[QCOW2_DISCARD_NEVER] = false;
-s->discard_passthrough[QCOW2_DISCARD_ALWAYS] = true;
-s->discard_passthrough[QCOW2_DISCARD_REQUEST] =
-qemu_opt_get_bool(opts, QCOW2_OPT_DISCARD_REQUEST,
-  flags & BDRV_O_UNMAP);
-s->discard_passthrough[QCOW2_DISCARD_SNAPSHOT] =
-qemu_opt_get_bool(opts, QCOW2_OPT_DISCARD_SNAPSHOT, true);
-s->discard_passthrough[QCOW2_DISCARD_OTHER] =
-qemu_opt_get_bool(opts, QCOW2_OPT_DISCARD_OTHER, false);
-
-opt_overlap_check = qemu_opt_get(opts, QCOW2_OPT_OVERLAP);

[Qemu-devel] [PULL 01/23] block: Always pass NULL as drv for bdrv_open()

[Kevin Wolf]

From: Max Reitz 

Change all callers of bdrv_open() to pass the driver name in the options
QDict instead of passing its BlockDriver pointer.

Signed-off-by: Max Reitz 
Reviewed-by: Alberto Garcia 
Signed-off-by: Kevin Wolf 
---
 block.c   | 24 ++--
 block/qcow2.c | 16 -
 block/vvfat.c |  8 +--
 blockdev.c| 72 +++
 4 files changed, 57 insertions(+), 63 deletions(-)

diff --git a/block.c b/block.c
index cb5d7ae..d0b9101 100644
--- a/block.c
+++ b/block.c
@@ -1385,11 +1385,13 @@ int bdrv_append_temp_snapshot(BlockDriverState *bs, int 
flags, Error **errp)
   qstring_from_str("file"));
 qdict_put(snapshot_options, "file.filename",
   qstring_from_str(tmp_filename));
+qdict_put(snapshot_options, "driver",
+  qstring_from_str("qcow2"));
 
 bs_snapshot = bdrv_new();
 
 ret = bdrv_open(&bs_snapshot, NULL, NULL, snapshot_options,
-flags, &bdrv_qcow2, &local_err);
+flags, NULL, &local_err);
 if (ret < 0) {
 error_propagate(errp, local_err);
 goto out;
@@ -3739,7 +3741,6 @@ void bdrv_img_create(const char *filename, const char 
*fmt,
 const char *backing_fmt, *backing_file;
 int64_t size;
 BlockDriver *drv, *proto_drv;
-BlockDriver *backing_drv = NULL;
 Error *local_err = NULL;
 int ret = 0;
 
@@ -3813,14 +3814,6 @@ void bdrv_img_create(const char *filename, const char 
*fmt,
 }
 
 backing_fmt = qemu_opt_get(opts, BLOCK_OPT_BACKING_FMT);
-if (backing_fmt) {
-backing_drv = bdrv_find_format(backing_fmt);
-if (!backing_drv) {
-error_setg(errp, "Unknown backing file format '%s'",
-   backing_fmt);
-goto out;
-}
-}
 
 // The size for the image must always be specified, with one exception:
 // If we are using a backing file, we can obtain the size from there
@@ -3831,6 +3824,7 @@ void bdrv_img_create(const char *filename, const char 
*fmt,
 char *full_backing = g_new0(char, PATH_MAX);
 int64_t size;
 int back_flags;
+QDict *backing_options = NULL;
 
 bdrv_get_full_backing_filename_from_filename(filename, 
backing_file,
  full_backing, 
PATH_MAX,
@@ -3844,9 +3838,15 @@ void bdrv_img_create(const char *filename, const char 
*fmt,
 back_flags =
 flags & ~(BDRV_O_RDWR | BDRV_O_SNAPSHOT | BDRV_O_NO_BACKING);
 
+if (backing_fmt) {
+backing_options = qdict_new();
+qdict_put(backing_options, "driver",
+  qstring_from_str(backing_fmt));
+}
+
 bs = NULL;
-ret = bdrv_open(&bs, full_backing, NULL, NULL, back_flags,
-backing_drv, &local_err);
+ret = bdrv_open(&bs, full_backing, NULL, backing_options,
+back_flags, NULL, &local_err);
 g_free(full_backing);
 if (ret < 0) {
 goto out;
diff --git a/block/qcow2.c b/block/qcow2.c
index ea34ae2..867b43b 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -1873,8 +1873,10 @@ static int qcow2_create2(const char *filename, int64_t 
total_size,
  QemuOpts *opts, int version, int refcount_order,
  Error **errp)
 {
-/* Calculate cluster_bits */
 int cluster_bits;
+QDict *options;
+
+/* Calculate cluster_bits */
 cluster_bits = ctz32(cluster_size);
 if (cluster_bits < MIN_CLUSTER_BITS || cluster_bits > MAX_CLUSTER_BITS ||
 (1 << cluster_bits) != cluster_size)
@@ -2032,9 +2034,11 @@ static int qcow2_create2(const char *filename, int64_t 
total_size,
  * refcount of the cluster that is occupied by the header and the refcount
  * table)
  */
-ret = bdrv_open(&bs, filename, NULL, NULL,
+options = qdict_new();
+qdict_put(options, "driver", qstring_from_str("qcow2"));
+ret = bdrv_open(&bs, filename, NULL, options,
 BDRV_O_RDWR | BDRV_O_CACHE_WB | BDRV_O_NO_FLUSH,
-&bdrv_qcow2, &local_err);
+NULL, &local_err);
 if (ret < 0) {
 error_propagate(errp, local_err);
 goto out;
@@ -2084,9 +2088,11 @@ static int qcow2_create2(const char *filename, int64_t 
total_size,
 bs = NULL;
 
 /* Reopen the image without BDRV_O_NO_FLUSH to flush it before returning */
-ret = bdrv_open(&bs, filename, NULL, NULL,
+options = qdict_new();
+qdict_put(options, "driver", qstring_from_str("qcow2"));
+ret = bdrv_open(&bs, filename, NULL, options,
 BDRV_O_RDWR | BDRV_O_CACHE_WB | BDRV_O_NO_BACKING,
-&bdrv_qcow2, &local_err);
+NULL, &local_err);
 if (local_err) {
 error_propagate(errp, local_e

[Qemu-devel] [PULL 04/23] block: Drop drv parameter from bdrv_fill_options()

[Kevin Wolf]

From: Max Reitz 

Now that this parameter is effectively unused, we can drop it and change
the function accordingly.

Signed-off-by: Max Reitz 
Reviewed-by: Alberto Garcia 
Signed-off-by: Kevin Wolf 
---
 block.c | 59 ++-
 1 file changed, 22 insertions(+), 37 deletions(-)

diff --git a/block.c b/block.c
index 52fa7f8..7c61555 100644
--- a/block.c
+++ b/block.c
@@ -996,13 +996,13 @@ static QDict *parse_json_filename(const char *filename, 
Error **errp)
  * block driver has been specified explicitly.
  */
 static int bdrv_fill_options(QDict **options, const char **pfilename,
- int *flags, BlockDriver *drv, Error **errp)
+ int *flags, Error **errp)
 {
 const char *filename = *pfilename;
 const char *drvname;
 bool protocol = *flags & BDRV_O_PROTOCOL;
 bool parse_filename = false;
-BlockDriver *tmp_drv;
+BlockDriver *drv = NULL;
 Error *local_err = NULL;
 
 /* Parse json: pseudo-protocol */
@@ -1021,15 +1021,15 @@ static int bdrv_fill_options(QDict **options, const 
char **pfilename,
 }
 
 drvname = qdict_get_try_str(*options, "driver");
-
-/* If the user has explicitly specified the driver, this choice should
- * override the BDRV_O_PROTOCOL flag */
-tmp_drv = drv;
-if (!tmp_drv && drvname) {
-tmp_drv = bdrv_find_format(drvname);
-}
-if (tmp_drv) {
-protocol = tmp_drv->bdrv_file_open;
+if (drvname) {
+drv = bdrv_find_format(drvname);
+if (!drv) {
+error_setg(errp, "Unknown driver '%s'", drvname);
+return -ENOENT;
+}
+/* If the user has explicitly specified the driver, this choice should
+ * override the BDRV_O_PROTOCOL flag */
+protocol = drv->bdrv_file_open;
 }
 
 if (protocol) {
@@ -1053,33 +1053,18 @@ static int bdrv_fill_options(QDict **options, const 
char **pfilename,
 /* Find the right block driver */
 filename = qdict_get_try_str(*options, "filename");
 
-if (drv) {
-if (drvname) {
-error_setg(errp, "Driver specified twice");
-return -EINVAL;
-}
-drvname = drv->format_name;
-qdict_put(*options, "driver", qstring_from_str(drvname));
-} else {
-if (!drvname && protocol) {
-if (filename) {
-drv = bdrv_find_protocol(filename, parse_filename, errp);
-if (!drv) {
-return -EINVAL;
-}
-
-drvname = drv->format_name;
-qdict_put(*options, "driver", qstring_from_str(drvname));
-} else {
-error_setg(errp, "Must specify either driver or file");
-return -EINVAL;
-}
-} else if (drvname) {
-drv = bdrv_find_format(drvname);
+if (!drvname && protocol) {
+if (filename) {
+drv = bdrv_find_protocol(filename, parse_filename, errp);
 if (!drv) {
-error_setg(errp, "Unknown driver '%s'", drvname);
-return -ENOENT;
+return -EINVAL;
 }
+
+drvname = drv->format_name;
+qdict_put(*options, "driver", qstring_from_str(drvname));
+} else {
+error_setg(errp, "Must specify either driver or file");
+return -EINVAL;
 }
 }
 
@@ -1474,7 +1459,7 @@ static int bdrv_open_inherit(BlockDriverState **pbs, 
const char *filename,
 flags = child_role->inherit_flags(parent->open_flags);
 }
 
-ret = bdrv_fill_options(&options, &filename, &flags, NULL, &local_err);
+ret = bdrv_fill_options(&options, &filename, &flags, &local_err);
 if (local_err) {
 goto fail;
 }
-- 
1.8.3.1

[Qemu-devel] [PULL 03/23] block: Drop drv parameter from bdrv_open_inherit()

[Kevin Wolf]

From: Max Reitz 

Now that this parameter is effectively unused, we can drop it and just
pass NULL to bdrv_fill_options().

Signed-off-by: Max Reitz 
Reviewed-by: Alberto Garcia 
Signed-off-by: Kevin Wolf 
---
 block.c | 17 +++--
 1 file changed, 7 insertions(+), 10 deletions(-)

diff --git a/block.c b/block.c
index 0e1b4b4..52fa7f8 100644
--- a/block.c
+++ b/block.c
@@ -85,8 +85,7 @@ static QLIST_HEAD(, BlockDriver) bdrv_drivers =
 static int bdrv_open_inherit(BlockDriverState **pbs, const char *filename,
  const char *reference, QDict *options, int flags,
  BlockDriverState *parent,
- const BdrvChildRole *child_role,
- BlockDriver *drv, Error **errp);
+ const BdrvChildRole *child_role, Error **errp);
 
 static void bdrv_dirty_bitmap_truncate(BlockDriverState *bs);
 /* If non-zero, use only whitelisted block drivers */
@@ -1227,8 +1226,7 @@ int bdrv_open_backing_file(BlockDriverState *bs, QDict 
*options, Error **errp)
 assert(bs->backing_hd == NULL);
 ret = bdrv_open_inherit(&backing_hd,
 *backing_filename ? backing_filename : NULL,
-NULL, options, 0, bs, &child_backing,
-NULL, &local_err);
+NULL, options, 0, bs, &child_backing, &local_err);
 if (ret < 0) {
 bdrv_unref(backing_hd);
 backing_hd = NULL;
@@ -1291,7 +1289,7 @@ BdrvChild *bdrv_open_child(const char *filename,
 
 bs = NULL;
 ret = bdrv_open_inherit(&bs, filename, reference, image_options, 0,
-parent, child_role, NULL, errp);
+parent, child_role, errp);
 if (ret < 0) {
 goto done;
 }
@@ -1422,11 +1420,11 @@ out:
 static int bdrv_open_inherit(BlockDriverState **pbs, const char *filename,
  const char *reference, QDict *options, int flags,
  BlockDriverState *parent,
- const BdrvChildRole *child_role,
- BlockDriver *drv, Error **errp)
+ const BdrvChildRole *child_role, Error **errp)
 {
 int ret;
 BlockDriverState *file = NULL, *bs;
+BlockDriver *drv = NULL;
 const char *drvname;
 Error *local_err = NULL;
 int snapshot_flags = 0;
@@ -1476,13 +1474,12 @@ static int bdrv_open_inherit(BlockDriverState **pbs, 
const char *filename,
 flags = child_role->inherit_flags(parent->open_flags);
 }
 
-ret = bdrv_fill_options(&options, &filename, &flags, drv, &local_err);
+ret = bdrv_fill_options(&options, &filename, &flags, NULL, &local_err);
 if (local_err) {
 goto fail;
 }
 
 /* Find the right image format driver */
-drv = NULL;
 drvname = qdict_get_try_str(options, "driver");
 if (drvname) {
 drv = bdrv_find_format(drvname);
@@ -1640,7 +1637,7 @@ int bdrv_open(BlockDriverState **pbs, const char 
*filename,
   const char *reference, QDict *options, int flags, Error **errp)
 {
 return bdrv_open_inherit(pbs, filename, reference, options, flags, NULL,
- NULL, NULL, errp);
+ NULL, errp);
 }
 
 typedef struct BlockReopenQueueEntry {
-- 
1.8.3.1

[Qemu-devel] [PATCH v2 3/3] pc: Introduce pc-*-2.5 machine classes

[Eduardo Habkost]

Signed-off-by: Eduardo Habkost 
---
 hw/i386/pc_piix.c| 14 +-
 hw/i386/pc_q35.c | 12 +++-
 include/hw/compat.h  |  3 +++
 include/hw/i386/pc.h |  4 
 4 files changed, 31 insertions(+), 2 deletions(-)

diff --git a/hw/i386/pc_piix.c b/hw/i386/pc_piix.c
index 2d9df7c..caa4edc 100644
--- a/hw/i386/pc_piix.c
+++ b/hw/i386/pc_piix.c
@@ -464,7 +464,7 @@ static void pc_i440fx_machine_options(MachineClass *m)
 m->default_display = "std";
 }
 
-static void pc_i440fx_2_4_machine_options(MachineClass *m)
+static void pc_i440fx_2_5_machine_options(MachineClass *m)
 {
 PCMachineClass *pcmc = PC_MACHINE_CLASS(m);
 pc_i440fx_machine_options(m);
@@ -473,6 +473,18 @@ static void pc_i440fx_2_4_machine_options(MachineClass *m)
 m->is_default = 1;
 }
 
+DEFINE_I440FX_MACHINE(v2_5, "pc-i440fx-2.5", NULL,
+  pc_i440fx_2_5_machine_options);
+
+
+static void pc_i440fx_2_4_machine_options(MachineClass *m)
+{
+pc_i440fx_2_5_machine_options(m);
+m->alias = NULL;
+m->is_default = 0;
+SET_MACHINE_COMPAT(m, PC_COMPAT_2_4);
+}
+
 DEFINE_I440FX_MACHINE(v2_4, "pc-i440fx-2.4", NULL,
   pc_i440fx_2_4_machine_options)
 
diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c
index 966eedb..506b6bf 100644
--- a/hw/i386/pc_q35.c
+++ b/hw/i386/pc_q35.c
@@ -370,7 +370,7 @@ static void pc_q35_machine_options(MachineClass *m)
 m->no_tco = 0;
 }
 
-static void pc_q35_2_4_machine_options(MachineClass *m)
+static void pc_q35_2_5_machine_options(MachineClass *m)
 {
 PCMachineClass *pcmc = PC_MACHINE_CLASS(m);
 pc_q35_machine_options(m);
@@ -378,6 +378,16 @@ static void pc_q35_2_4_machine_options(MachineClass *m)
 m->alias = "q35";
 }
 
+DEFINE_Q35_MACHINE(v2_5, "pc-q35-2.5", NULL,
+   pc_q35_2_5_machine_options);
+
+static void pc_q35_2_4_machine_options(MachineClass *m)
+{
+pc_q35_2_5_machine_options(m);
+m->alias = NULL;
+SET_MACHINE_COMPAT(m, PC_COMPAT_2_4);
+}
+
 DEFINE_Q35_MACHINE(v2_4, "pc-q35-2.4", NULL,
pc_q35_2_4_machine_options);
 
diff --git a/include/hw/compat.h b/include/hw/compat.h
index 94c8097..095de5d 100644
--- a/include/hw/compat.h
+++ b/include/hw/compat.h
@@ -1,6 +1,9 @@
 #ifndef HW_COMPAT_H
 #define HW_COMPAT_H
 
+#define HW_COMPAT_2_4 \
+/* empty */
+
 #define HW_COMPAT_2_3 \
 {\
 .driver   = "virtio-blk-pci",\
diff --git a/include/hw/i386/pc.h b/include/hw/i386/pc.h
index 3e002c9..6896328 100644
--- a/include/hw/i386/pc.h
+++ b/include/hw/i386/pc.h
@@ -291,7 +291,11 @@ int e820_add_entry(uint64_t, uint64_t, uint32_t);
 int e820_get_num_entries(void);
 bool e820_get_entry(int, uint32_t, uint64_t *, uint64_t *);
 
+#define PC_COMPAT_2_4 \
+HW_COMPAT_2_4
+
 #define PC_COMPAT_2_3 \
+PC_COMPAT_2_4 \
 HW_COMPAT_2_3 \
 {\
 .driver   = TYPE_X86_CPU,\
-- 
2.1.0

[Qemu-devel] [PATCH v2 2/3] q35: Move options common to all classes to pc_i440fx_machine_options()

[Eduardo Habkost]

The existing default_machine_opts and default_display settings will
still apply to future machine classes. So it makes sense to move them to
pc_i440fx_machine_options() instead of keeping them in a
version-specific machine_options function.

Signed-off-by: Eduardo Habkost 
---
 hw/i386/pc_piix.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/hw/i386/pc_piix.c b/hw/i386/pc_piix.c
index 3f925b2..2d9df7c 100644
--- a/hw/i386/pc_piix.c
+++ b/hw/i386/pc_piix.c
@@ -460,6 +460,8 @@ static void pc_i440fx_machine_options(MachineClass *m)
 m->family = "pc_piix";
 m->desc = "Standard PC (i440FX + PIIX, 1996)";
 m->hot_add_cpu = pc_hot_add_cpu;
+m->default_machine_opts = "firmware=bios-256k.bin";
+m->default_display = "std";
 }
 
 static void pc_i440fx_2_4_machine_options(MachineClass *m)
@@ -467,8 +469,6 @@ static void pc_i440fx_2_4_machine_options(MachineClass *m)
 PCMachineClass *pcmc = PC_MACHINE_CLASS(m);
 pc_i440fx_machine_options(m);
 pcmc->broken_reserved_end = true;
-m->default_machine_opts = "firmware=bios-256k.bin";
-m->default_display = "std";
 m->alias = "pc";
 m->is_default = 1;
 }
-- 
2.1.0

[Qemu-devel] [PATCH v2 1/3] q35: Move options common to all classes to pc_q35_machine_options()

[Eduardo Habkost]

The existing default_machine_opts, default_display, no_floppy, and
no_tco settings will still apply to future machine classes. So it makes
sense to move them to pc_q35_machine_options() instead of keeping them
in a version-specific machine_options function.

Signed-off-by: Eduardo Habkost 
---
 hw/i386/pc_q35.c | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c
index 11601ab..966eedb 100644
--- a/hw/i386/pc_q35.c
+++ b/hw/i386/pc_q35.c
@@ -364,6 +364,10 @@ static void pc_q35_machine_options(MachineClass *m)
 m->desc = "Standard PC (Q35 + ICH9, 2009)";
 m->hot_add_cpu = pc_hot_add_cpu;
 m->units_per_default_bus = 1;
+m->default_machine_opts = "firmware=bios-256k.bin";
+m->default_display = "std";
+m->no_floppy = 1;
+m->no_tco = 0;
 }
 
 static void pc_q35_2_4_machine_options(MachineClass *m)
@@ -371,10 +375,6 @@ static void pc_q35_2_4_machine_options(MachineClass *m)
 PCMachineClass *pcmc = PC_MACHINE_CLASS(m);
 pc_q35_machine_options(m);
 pcmc->broken_reserved_end = true;
-m->default_machine_opts = "firmware=bios-256k.bin";
-m->default_display = "std";
-m->no_floppy = 1;
-m->no_tco = 0;
 m->alias = "q35";
 }
 
-- 
2.1.0

[Qemu-devel] [PATCH v2 0/3] pc-*-2.5 machine-types

[Eduardo Habkost]

This introduces the pc-q35-2.5 and pc-i440fx-2.5 machine-types.

Changes v1 -> v2:
* Trivial rebase on top of latest qemu.git master

Eduardo Habkost (3):
  q35: Move options common to all classes to pc_q35_machine_options()
  q35: Move options common to all classes to pc_i440fx_machine_options()
  pc: Introduce pc-*-2.5 machine classes

 hw/i386/pc_piix.c| 18 +++---
 hw/i386/pc_q35.c | 20 +++-
 include/hw/compat.h  |  3 +++
 include/hw/i386/pc.h |  4 
 4 files changed, 37 insertions(+), 8 deletions(-)

-- 
2.1.0

Re: [Qemu-devel] [PATCH v5 04/14] backup: Extract dirty bitmap handling as a separate function

[Eric Blake]

On 09/07/2015 01:34 AM, Fam Zheng wrote:
> This will be reused by the coming new transactional completion code.
> 
> Signed-off-by: Fam Zheng 
> Reviewed-by: John Snow 
> ---
>  block/backup.c | 26 --
>  1 file changed, 16 insertions(+), 10 deletions(-)
> 
> diff --git a/block/backup.c b/block/backup.c
> index 965654d..9776d9c 100644
> --- a/block/backup.c
> +++ b/block/backup.c
> @@ -210,6 +210,21 @@ static void backup_iostatus_reset(BlockJob *job)
>  
>  bdrv_iostatus_reset(s->target);
>  }
> +static void backup_cleanup_sync_bitmap(BackupBlockJob *job, int ret)

Blank lines between functions?

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH 0/4] target-i386: Don't try to enable unsupported TCG features by default

[Eduardo Habkost]

On Fri, Sep 11, 2015 at 12:32:35PM -0700, Richard Henderson wrote:
> On 09/11/2015 12:25 PM, Eduardo Habkost wrote:
> > This series makes QEMU stop trying to enable an unsupported feature by 
> > default
> > in TCG mode, avoiding the warnings shown by the (now enabled by default)
> > "check" mode.
> > 
> > To do that, I cchanged the kvm_default_features code to be based on QOM
> > properties, and added a new "type" field to allow per-CPU-model KVM 
> > defaults to
> > be specified.
> > 
> > Before this series:
> > 
> >   $ qemu-system-x86_64 -S
> >   warning: TCG doesn't support requested feature: CPUID.01H:EDX.de [bit 2]
> >   warning: TCG doesn't support requested feature: CPUID.8001H:EDX [bit 
> > 2]
> > 
> > After this series:
> > 
> >   $ ./x86_64-softmmu/qemu-system-x86_64 -S
> >   [no warning output]
> 
> Wouldn't it be easier simply to implement this bit?
> It seems like we're simply creating a compatibility mess otherwise.

I don't think the compatibility mess is worse than the one we already
have with x2apic. I even think the new type/property/value system is
cleaner than the existing kvm_default_*features arrays.

About implementing DE in TCG: I really don't think it is easier, but if
somebody wants to implement it, it would be welcome.

-- 
Eduardo

Re: [Qemu-devel] [PATCH qemu v2 2/2] spapr_pci: Remove constraints about VFIO-PCI devices

[Alex Williamson]

On Wed, 2015-09-09 at 20:43 -0600, Alex Williamson wrote:
> On Thu, 2015-09-03 at 14:40 +1000, Alexey Kardashevskiy wrote:
> > So far there were 2 limitations enforced on an emulated PHB
> > regarding VFIO:
> > 1) only one IOMMU group per IOMMU container was allowed and
> > the spapr-pci-vfio-host-bridge device has an IOMMU ID property for this;
> > 2) only one IOMMU container per PHB was allowed as a group
> > can only be attached to one container.
> > 
> > However these are not really necessary so we are removing them here.
> > 
> > This deprecates IOMMU group ID and changes vfio_container_do_ioctl()
> > not to receive it. Instead of getting a container from a group ID,
> > this calls ioctl() for every container attached to the PHB address space.
> > This allows multiple containers on the same PHB which means multiple
> > groups per PHB. Note that this will lead to every H_PUT_TCE&etc call
> > to be passed to every container separately which will affect
> > the performance. For 32bit devices it is still recommended to put
> > every group to a separate PHB.
> > 
> > Since the existing VFIO code is already trying to share a container for
> > multiple groups, just removing a group id from
> > the vfio_container_do_ioctl() allows the existing code to share
> > a container if it is supported by the host kernel.
> > 
> > This replaces the check for a group id to be set correctly with
> > the check that it is not set.
> > 
> > This removes casts to sPAPRPHBVFIOState as none of sPAPRPHBVFIOState
> > members is accessed here.
> > 
> > Signed-off-by: Alexey Kardashevskiy 
> > ---
> >  hw/ppc/spapr_pci.c  | 10 +-
> >  hw/ppc/spapr_pci_vfio.c | 17 ++---
> >  hw/vfio/common.c| 22 ++
> >  include/hw/vfio/vfio.h  |  3 +--
> >  4 files changed, 18 insertions(+), 34 deletions(-)
> > 
> > diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
> > index 4e289cb..1b7559d 100644
> > --- a/hw/ppc/spapr_pci.c
> > +++ b/hw/ppc/spapr_pci.c
> > @@ -810,11 +810,6 @@ static int 
> > spapr_phb_dma_capabilities_update(sPAPRPHBState *sphb)
> >  pci_for_each_device(bus, pci_bus_num(bus), spapr_phb_walk_devices, 
> > sphb);
> >  
> >  if (sphb->vfio_num) {
> > -if (sphb->iommugroupid == -1) {
> > -error_report("Wrong IOMMU group ID %d", sphb->iommugroupid);
> > -return -1;
> > -}
> > -
> >  ret = spapr_phb_vfio_dma_capabilities_update(sphb);
> >  if (ret) {
> >  error_report("Unable to get DMA32 info from VFIO");
> > @@ -1282,6 +1277,11 @@ static void spapr_phb_realize(DeviceState *dev, 
> > Error **errp)
> >  PCIBus *bus;
> >  uint64_t msi_window_size = 4096;
> >  
> > +if ((sphb->iommugroupid != -1) &&
> > +object_dynamic_cast(OBJECT(sphb), 
> > TYPE_SPAPR_PCI_VFIO_HOST_BRIDGE)) {
> > +error_report("Warning: iommugroupid is deprecated and will be 
> > ignored");
> > +}
> > +
> >  if (sphb->index != (uint32_t)-1) {
> >  hwaddr windows_base;
> >  
> > diff --git a/hw/ppc/spapr_pci_vfio.c b/hw/ppc/spapr_pci_vfio.c
> > index f94d8a4..48137d5 100644
> > --- a/hw/ppc/spapr_pci_vfio.c
> > +++ b/hw/ppc/spapr_pci_vfio.c
> > @@ -35,7 +35,7 @@ int spapr_phb_vfio_dma_capabilities_update(sPAPRPHBState 
> > *sphb)
> >  struct vfio_iommu_spapr_tce_info info = { .argsz = sizeof(info) };
> >  int ret;
> >  
> > -ret = vfio_container_ioctl(&sphb->iommu_as, sphb->iommugroupid,
> > +ret = vfio_container_ioctl(&sphb->iommu_as,
> > VFIO_IOMMU_SPAPR_TCE_GET_INFO, &info);
> >  if (ret) {
> >  return ret;
> > @@ -54,8 +54,7 @@ void spapr_phb_vfio_eeh_reenable(sPAPRPHBState *sphb)
> >  .op= VFIO_EEH_PE_ENABLE
> >  };
> >  
> > -vfio_container_ioctl(&sphb->iommu_as,
> > - sphb->iommugroupid, VFIO_EEH_PE_OP, &op);
> > +vfio_container_ioctl(&sphb->iommu_as, VFIO_EEH_PE_OP, &op);
> >  }
> >  
> >  int spapr_phb_vfio_eeh_set_option(sPAPRPHBState *sphb,
> > @@ -81,8 +80,7 @@ int spapr_phb_vfio_eeh_set_option(sPAPRPHBState *sphb,
> >  return RTAS_OUT_PARAM_ERROR;
> >  }
> >  
> > -ret = vfio_container_ioctl(&sphb->iommu_as, sphb->iommugroupid,
> > -   VFIO_EEH_PE_OP, &op);
> > +ret = vfio_container_ioctl(&sphb->iommu_as, VFIO_EEH_PE_OP, &op);
> >  if (ret < 0) {
> >  return RTAS_OUT_HW_ERROR;
> >  }
> > @@ -96,8 +94,7 @@ int spapr_phb_vfio_eeh_get_state(sPAPRPHBState *sphb, int 
> > *state)
> >  int ret;
> >  
> >  op.op = VFIO_EEH_PE_GET_STATE;
> > -ret = vfio_container_ioctl(&sphb->iommu_as, sphb->iommugroupid,
> > -   VFIO_EEH_PE_OP, &op);
> > +ret = vfio_container_ioctl(&sphb->iommu_as, VFIO_EEH_PE_OP, &op);
> >  if (ret < 0) {
> >  return RTAS_OUT_PARAM_ERROR;
> >  }
> > @@ -170,8 +167,7 @@ int spapr_phb_vfio_eeh_reset(sPAPRPHBState *sphb, int 
> > option)
> >

Re: [Qemu-devel] [PATCH v5 09/14] block: Add block job transactions

[Eric Blake]

On 09/07/2015 01:34 AM, Fam Zheng wrote:
> Sometimes block jobs must execute as a transaction group.  Finishing
> jobs wait until all other jobs are ready to complete successfully.
> Failure or cancellation of one job cancels the other jobs in the group.
> 
> Signed-off-by: Stefan Hajnoczi 
> [Rewrite the implementation which is now contained in block_job_completed.
> --Fam]
> Signed-off-by: Fam Zheng 
> ---
>  blockjob.c   | 134 
> ++-
>  include/block/block.h|   1 +
>  include/block/blockjob.h |  38 ++
>  3 files changed, 171 insertions(+), 2 deletions(-)

> +static void block_job_completed_txn_success(BlockJob *job)
> +{
> +AioContext *ctx;
> +BlockJobTxn *txn = job->txn;
> +BlockJob *other_job, *next;
> +/*
> + * Successful completion, see if there are other running jobs in this
> + * txn.
> + **/

Unusual **/


-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

[Qemu-devel] [PULL 19/23] qemu-iotests: More qcow2 reopen tests

[Kevin Wolf]

Signed-off-by: Kevin Wolf 
Reviewed-by: Max Reitz 
---
 tests/qemu-iotests/137 | 145 +
 tests/qemu-iotests/137.out |  42 +
 tests/qemu-iotests/group   |   1 +
 3 files changed, 188 insertions(+)
 create mode 100755 tests/qemu-iotests/137
 create mode 100644 tests/qemu-iotests/137.out

diff --git a/tests/qemu-iotests/137 b/tests/qemu-iotests/137
new file mode 100755
index 000..9a6597c
--- /dev/null
+++ b/tests/qemu-iotests/137
@@ -0,0 +1,145 @@
+#!/bin/bash
+#
+# Test qcow2 reopen
+#
+# Copyright (C) 2015 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see .
+#
+
+# creator
+owner=kw...@redhat.com
+
+seq="$(basename $0)"
+echo "QA output created by $seq"
+
+here="$PWD"
+tmp=/tmp/$$
+status=1   # failure is the default!
+
+_cleanup()
+{
+_cleanup_test_img
+}
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+. ./common.qemu
+
+_supported_fmt qcow2
+_supported_proto generic
+_supported_os Linux
+
+
+_make_test_img 64M
+
+echo === Try setting valid values for all options ===
+echo
+
+# Try all options and then check that all of the basic I/O operations still
+# work on this image.
+$QEMU_IO \
+-c "reopen -o lazy-refcounts=on,pass-discard-request=on" \
+-c "reopen -o lazy-refcounts=off,pass-discard-request=off" \
+-c "reopen -o pass-discard-snapshot=on,pass-discard-other=on" \
+-c "reopen -o pass-discard-snapshot=off,pass-discard-other=off" \
+-c "reopen -o overlap-check=all" \
+-c "reopen -o overlap-check=none" \
+-c "reopen -o overlap-check=cached" \
+-c "reopen -o overlap-check=constant" \
+-c "reopen -o overlap-check.template=all" \
+-c "reopen -o overlap-check.template=none" \
+-c "reopen -o overlap-check.template=cached" \
+-c "reopen -o overlap-check.template=constant" \
+-c "reopen -o overlap-check.main-header=on" \
+-c "reopen -o overlap-check.main-header=off" \
+-c "reopen -o overlap-check.active-l1=on" \
+-c "reopen -o overlap-check.active-l1=off" \
+-c "reopen -o overlap-check.active-l2=on" \
+-c "reopen -o overlap-check.active-l2=off" \
+-c "reopen -o overlap-check.refcount-table=on" \
+-c "reopen -o overlap-check.refcount-table=off" \
+-c "reopen -o overlap-check.refcount-block=on" \
+-c "reopen -o overlap-check.refcount-block=off" \
+-c "reopen -o overlap-check.snapshot-table=on" \
+-c "reopen -o overlap-check.snapshot-table=off" \
+-c "reopen -o overlap-check.inactive-l1=on" \
+-c "reopen -o overlap-check.inactive-l1=off" \
+-c "reopen -o overlap-check.inactive-l2=on" \
+-c "reopen -o overlap-check.inactive-l2=off" \
+-c "reopen -o cache-size=1M" \
+-c "reopen -o l2-cache-size=512k" \
+-c "reopen -o refcount-cache-size=128k" \
+-c "reopen -o cache-clean-interval=5" \
+-c "reopen -o cache-clean-interval=0" \
+-c "reopen -o cache-clean-interval=10" \
+\
+-c "write -P 55 0 32M" \
+-c "read -P 55 0 32M" \
+-c "discard 0 32M" \
+-c "write -z 0 32M" \
+-c "read -P 0 0 32M" \
+\
+"$TEST_IMG" | _filter_qemu_io
+
+
+echo
+echo === Try setting some invalid values ===
+echo
+
+$QEMU_IO \
+-c "reopen -o lazy-refcounts=42" \
+-c "reopen -o cache-size=1M,l2-cache-size=64k,refcount-cache-size=64k" \
+-c "reopen -o cache-size=1M,l2-cache-size=2M" \
+-c "reopen -o cache-size=1M,refcount-cache-size=2M" \
+-c "reopen -o l2-cache-size=256T" \
+-c "reopen -o refcount-cache-size=256T" \
+-c "reopen -o overlap-check=constant,overlap-check.template=all" \
+-c "reopen -o overlap-check=blubb" \
+-c "reopen -o overlap-check.template=blubb" \
+-c "reopen -o cache-clean-interval=-1" \
+"$TEST_IMG" | _filter_qemu_io
+
+echo
+echo === Test transaction semantics ===
+echo
+
+# Whether lazy-refcounts was actually enabled can easily be tested: Check if
+# the dirty bit is set after a crash
+$QEMU_IO \
+-c "reopen -o lazy-refcounts=on,overlap-check=blubb" \
+-c "write -P 0x5a 0 512" \
+-c "sigraise $(kill -l KILL)" \
+"$TEST_IMG" 2>&1 | _filter_qemu_io
+
+# The dirty bit must not be set
+$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+
+# Similarly we can test whether corruption detection has been enabled:
+# Create L1/L2, overwrite first e

[Qemu-devel] [PULL 20/23] qcow2: Make size_to_clusters() return uint64_t

[Kevin Wolf]

From: Max Reitz 

Sadly, some images may have more clusters than what can be represented
using a plain int. We should be prepared for that case (in
qcow2_check_refcounts() we actually were trying to catch that case, but
since size_to_clusters() truncated the returned value, that check never
did anything useful).

Cc: qemu-stable 
Signed-off-by: Max Reitz 
Signed-off-by: Kevin Wolf 
---
 block/qcow2-cluster.c  | 28 ++--
 block/qcow2-refcount.c | 10 +++---
 block/qcow2.h  |  6 +++---
 3 files changed, 28 insertions(+), 16 deletions(-)

diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
index 412ee27..6ede629 100644
--- a/block/qcow2-cluster.c
+++ b/block/qcow2-cluster.c
@@ -298,7 +298,7 @@ fail:
  * as contiguous. (This allows it, for example, to stop at the first compressed
  * cluster which may require a different handling)
  */
-static int count_contiguous_clusters(uint64_t nb_clusters, int cluster_size,
+static int count_contiguous_clusters(int nb_clusters, int cluster_size,
 uint64_t *l2_table, uint64_t stop_flags)
 {
 int i;
@@ -321,7 +321,7 @@ static int count_contiguous_clusters(uint64_t nb_clusters, 
int cluster_size,
return i;
 }
 
-static int count_contiguous_free_clusters(uint64_t nb_clusters, uint64_t 
*l2_table)
+static int count_contiguous_free_clusters(int nb_clusters, uint64_t *l2_table)
 {
 int i;
 
@@ -495,6 +495,7 @@ int qcow2_get_cluster_offset(BlockDriverState *bs, uint64_t 
offset,
 if (nb_needed > nb_available) {
 nb_needed = nb_available;
 }
+assert(nb_needed <= INT_MAX);
 
 *cluster_offset = 0;
 
@@ -530,6 +531,8 @@ int qcow2_get_cluster_offset(BlockDriverState *bs, uint64_t 
offset,
 
 l2_index = (offset >> s->cluster_bits) & (s->l2_size - 1);
 *cluster_offset = be64_to_cpu(l2_table[l2_index]);
+
+/* nb_needed <= INT_MAX, thus nb_clusters <= INT_MAX, too */
 nb_clusters = size_to_clusters(s, nb_needed << 9);
 
 ret = qcow2_get_cluster_type(*cluster_offset);
@@ -960,7 +963,7 @@ static int handle_copied(BlockDriverState *bs, uint64_t 
guest_offset,
 int l2_index;
 uint64_t cluster_offset;
 uint64_t *l2_table;
-unsigned int nb_clusters;
+uint64_t nb_clusters;
 unsigned int keep_clusters;
 int ret;
 
@@ -979,6 +982,7 @@ static int handle_copied(BlockDriverState *bs, uint64_t 
guest_offset,
 
 l2_index = offset_to_l2_index(s, guest_offset);
 nb_clusters = MIN(nb_clusters, s->l2_size - l2_index);
+assert(nb_clusters <= INT_MAX);
 
 /* Find L2 entry for the first involved cluster */
 ret = get_cluster_table(bs, guest_offset, &l2_table, &l2_index);
@@ -1061,7 +1065,7 @@ out:
  * restarted, but the whole request should not be failed.
  */
 static int do_alloc_cluster_offset(BlockDriverState *bs, uint64_t guest_offset,
-uint64_t *host_offset, unsigned int *nb_clusters)
+   uint64_t *host_offset, uint64_t 
*nb_clusters)
 {
 BDRVQcow2State *s = bs->opaque;
 
@@ -1079,7 +1083,7 @@ static int do_alloc_cluster_offset(BlockDriverState *bs, 
uint64_t guest_offset,
 *host_offset = cluster_offset;
 return 0;
 } else {
-int ret = qcow2_alloc_clusters_at(bs, *host_offset, *nb_clusters);
+int64_t ret = qcow2_alloc_clusters_at(bs, *host_offset, *nb_clusters);
 if (ret < 0) {
 return ret;
 }
@@ -1115,7 +1119,7 @@ static int handle_alloc(BlockDriverState *bs, uint64_t 
guest_offset,
 int l2_index;
 uint64_t *l2_table;
 uint64_t entry;
-unsigned int nb_clusters;
+uint64_t nb_clusters;
 int ret;
 
 uint64_t alloc_cluster_offset;
@@ -1133,6 +1137,7 @@ static int handle_alloc(BlockDriverState *bs, uint64_t 
guest_offset,
 
 l2_index = offset_to_l2_index(s, guest_offset);
 nb_clusters = MIN(nb_clusters, s->l2_size - l2_index);
+assert(nb_clusters <= INT_MAX);
 
 /* Find L2 entry for the first involved cluster */
 ret = get_cluster_table(bs, guest_offset, &l2_table, &l2_index);
@@ -1426,7 +1431,8 @@ int qcow2_decompress_cluster(BlockDriverState *bs, 
uint64_t cluster_offset)
  * clusters.
  */
 static int discard_single_l2(BlockDriverState *bs, uint64_t offset,
-unsigned int nb_clusters, enum qcow2_discard_type type, bool full_discard)
+ uint64_t nb_clusters, enum qcow2_discard_type 
type,
+ bool full_discard)
 {
 BDRVQcow2State *s = bs->opaque;
 uint64_t *l2_table;
@@ -1441,6 +1447,7 @@ static int discard_single_l2(BlockDriverState *bs, 
uint64_t offset,
 
 /* Limit nb_clusters to one L2 table */
 nb_clusters = MIN(nb_clusters, s->l2_size - l2_index);
+assert(nb_clusters <= INT_MAX);
 
 for (i = 0; i < nb_clusters; i++) {
 uint64_t old_l2_entry;
@@ -1503,7 +1510,7 @@ int qcow2_discard_clusters(BlockDriverState *bs, uint64_t 
offset,
 {
 BDRVQcow2State *s = bs->opaque;
 uint64_t end_offset;
-unsigned int

[Qemu-devel] [PULL 17/23] qcow2: Support updating driver-specific options in reopen

[Kevin Wolf]

For updating the cache sizes, disabling lazy refcounts and updating the
clean_cache_timer there is a bit more to do than just changing the
variables, but otherwise we're all set for changing options during
bdrv_reopen().

Just implement the missing pieces and hook the functions up in
bdrv_reopen().

Signed-off-by: Kevin Wolf 
Reviewed-by: Max Reitz 
---
 block/qcow2.c | 81 ---
 1 file changed, 72 insertions(+), 9 deletions(-)

diff --git a/block/qcow2.c b/block/qcow2.c
index 64ba3cb..56ad808 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -649,7 +649,24 @@ static int qcow2_update_options_prepare(BlockDriverState 
*bs,
 goto fail;
 }
 
-/* alloc L2 table/refcount block cache */
+/* alloc new L2 table/refcount block cache, flush old one */
+if (s->l2_table_cache) {
+ret = qcow2_cache_flush(bs, s->l2_table_cache);
+if (ret) {
+error_setg_errno(errp, -ret, "Failed to flush the L2 table cache");
+goto fail;
+}
+}
+
+if (s->refcount_block_cache) {
+ret = qcow2_cache_flush(bs, s->refcount_block_cache);
+if (ret) {
+error_setg_errno(errp, -ret,
+ "Failed to flush the refcount block cache");
+goto fail;
+}
+}
+
 r->l2_table_cache = qcow2_cache_create(bs, l2_cache_size);
 r->refcount_block_cache = qcow2_cache_create(bs, refcount_cache_size);
 if (r->l2_table_cache == NULL || r->refcount_block_cache == NULL) {
@@ -660,14 +677,15 @@ static int qcow2_update_options_prepare(BlockDriverState 
*bs,
 
 /* New interval for cache cleanup timer */
 r->cache_clean_interval =
-qemu_opt_get_number(opts, QCOW2_OPT_CACHE_CLEAN_INTERVAL, 0);
+qemu_opt_get_number(opts, QCOW2_OPT_CACHE_CLEAN_INTERVAL,
+s->cache_clean_interval);
 if (r->cache_clean_interval > UINT_MAX) {
 error_setg(errp, "Cache clean interval too big");
 ret = -EINVAL;
 goto fail;
 }
 
-/* Enable lazy_refcounts according to image and command line options */
+/* lazy-refcounts; flush if going from enabled to disabled */
 r->use_lazy_refcounts = qemu_opt_get_bool(opts, QCOW2_OPT_LAZY_REFCOUNTS,
 (s->compatible_features & QCOW2_COMPAT_LAZY_REFCOUNTS));
 if (r->use_lazy_refcounts && s->qcow_version < 3) {
@@ -677,6 +695,14 @@ static int qcow2_update_options_prepare(BlockDriverState 
*bs,
 goto fail;
 }
 
+if (s->use_lazy_refcounts && !r->use_lazy_refcounts) {
+ret = qcow2_mark_clean(bs);
+if (ret < 0) {
+error_setg_errno(errp, -ret, "Failed to disable lazy refcounts");
+goto fail;
+}
+}
+
 /* Overlap check options */
 opt_overlap_check = qemu_opt_get(opts, QCOW2_OPT_OVERLAP);
 opt_overlap_check_template = qemu_opt_get(opts, 
QCOW2_OPT_OVERLAP_TEMPLATE);
@@ -741,6 +767,12 @@ static void qcow2_update_options_commit(BlockDriverState 
*bs,
 BDRVQcow2State *s = bs->opaque;
 int i;
 
+if (s->l2_table_cache) {
+qcow2_cache_destroy(bs, s->l2_table_cache);
+}
+if (s->refcount_block_cache) {
+qcow2_cache_destroy(bs, s->refcount_block_cache);
+}
 s->l2_table_cache = r->l2_table_cache;
 s->refcount_block_cache = r->refcount_block_cache;
 
@@ -751,8 +783,11 @@ static void qcow2_update_options_commit(BlockDriverState 
*bs,
 s->discard_passthrough[i] = r->discard_passthrough[i];
 }
 
-s->cache_clean_interval = r->cache_clean_interval;
-cache_clean_timer_init(bs, bdrv_get_aio_context(bs));
+if (s->cache_clean_interval != r->cache_clean_interval) {
+cache_clean_timer_del(bs);
+s->cache_clean_interval = r->cache_clean_interval;
+cache_clean_timer_init(bs, bdrv_get_aio_context(bs));
+}
 }
 
 static void qcow2_update_options_abort(BlockDriverState *bs,
@@ -1199,26 +1234,52 @@ static int qcow2_set_key(BlockDriverState *bs, const 
char *key)
 return 0;
 }
 
-/* We have no actual commit/abort logic for qcow2, but we need to write out any
- * unwritten data if we reopen read-only. */
 static int qcow2_reopen_prepare(BDRVReopenState *state,
 BlockReopenQueue *queue, Error **errp)
 {
+Qcow2ReopenState *r;
 int ret;
 
+r = g_new0(Qcow2ReopenState, 1);
+state->opaque = r;
+
+ret = qcow2_update_options_prepare(state->bs, r, state->options,
+   state->flags, errp);
+if (ret < 0) {
+goto fail;
+}
+
+/* We need to write out any unwritten data if we reopen read-only. */
 if ((state->flags & BDRV_O_RDWR) == 0) {
 ret = bdrv_flush(state->bs);
 if (ret < 0) {
-return ret;
+goto fail;
 }
 
 ret = qcow2_mark_clean(state->bs);
 if (ret < 0) {
-return ret;
+goto fail;
 }
 }
 
 return 0;

[Qemu-devel] [PULL 18/23] qemu-iotests: Reopen qcow2 with lazy-refcounts change

[Kevin Wolf]

Signed-off-by: Kevin Wolf 
Reviewed-by: Max Reitz 
---
 tests/qemu-iotests/039 | 27 +++
 tests/qemu-iotests/039.out | 18 ++
 2 files changed, 45 insertions(+)

diff --git a/tests/qemu-iotests/039 b/tests/qemu-iotests/039
index 617f397..9e9b379 100755
--- a/tests/qemu-iotests/039
+++ b/tests/qemu-iotests/039
@@ -147,6 +147,33 @@ $PYTHON qcow2.py "$TEST_IMG".base dump-header | grep 
incompatible_features
 _check_test_img
 TEST_IMG="$TEST_IMG".base _check_test_img
 
+echo
+echo "== Changing lazy_refcounts setting at runtime =="
+
+IMGOPTS="compat=1.1,lazy_refcounts=off"
+_make_test_img $size
+
+$QEMU_IO -c "reopen -o lazy-refcounts=on" \
+ -c "write -P 0x5a 0 512" \
+ -c "sigraise $(kill -l KILL)" "$TEST_IMG" 2>&1 \
+| _filter_qemu_io
+
+# The dirty bit must be set
+$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_check_test_img
+
+IMGOPTS="compat=1.1,lazy_refcounts=on"
+_make_test_img $size
+
+$QEMU_IO -c "reopen -o lazy-refcounts=off" \
+ -c "write -P 0x5a 0 512" \
+ -c "sigraise $(kill -l KILL)" "$TEST_IMG" 2>&1 \
+| _filter_qemu_io
+
+# The dirty bit must not be set
+$PYTHON qcow2.py "$TEST_IMG" dump-header | grep incompatible_features
+_check_test_img
+
 
 # success, all done
 echo "*** done"
diff --git a/tests/qemu-iotests/039.out b/tests/qemu-iotests/039.out
index d8c5a44..03a31c5 100644
--- a/tests/qemu-iotests/039.out
+++ b/tests/qemu-iotests/039.out
@@ -74,4 +74,22 @@ incompatible_features 0x0
 incompatible_features 0x0
 No errors were found on the image.
 No errors were found on the image.
+
+== Changing lazy_refcounts setting at runtime ==
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728
+wrote 512/512 bytes at offset 0
+512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+./common.config: Killed  ( exec "$QEMU_IO_PROG" 
$QEMU_IO_OPTIONS "$@" )
+incompatible_features 0x1
+ERROR cluster 5 refcount=0 reference=1
+ERROR OFLAG_COPIED data cluster: l2_entry=8005 refcount=0
+
+2 errors were found on the image.
+Data may be corrupted, or further writes to the image may corrupt it.
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728
+wrote 512/512 bytes at offset 0
+512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+./common.config: Killed  ( exec "$QEMU_IO_PROG" 
$QEMU_IO_OPTIONS "$@" )
+incompatible_features 0x0
+No errors were found on the image.
 *** done
-- 
1.8.3.1

[Qemu-devel] [PULL 21/23] iotests: Add test for checking large image files

[Kevin Wolf]

From: Max Reitz 

Add a test for checking a qcow2 file with a multiple of 2^32 clusters.

Signed-off-by: Max Reitz 
Reviewed-by: Eric Blake 
Signed-off-by: Kevin Wolf 
---
 tests/qemu-iotests/138 | 73 ++
 tests/qemu-iotests/138.out |  9 ++
 tests/qemu-iotests/group   |  1 +
 3 files changed, 83 insertions(+)
 create mode 100755 tests/qemu-iotests/138
 create mode 100644 tests/qemu-iotests/138.out

diff --git a/tests/qemu-iotests/138 b/tests/qemu-iotests/138
new file mode 100755
index 000..a5c3464
--- /dev/null
+++ b/tests/qemu-iotests/138
@@ -0,0 +1,73 @@
+#!/bin/bash
+#
+# General test case for qcow2's image check
+#
+# Copyright (C) 2015 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see .
+#
+
+# creator
+owner=mre...@redhat.com
+
+seq="$(basename $0)"
+echo "QA output created by $seq"
+
+here="$PWD"
+tmp=/tmp/$$
+status=1   # failure is the default!
+
+_cleanup()
+{
+   _cleanup_test_img
+}
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+# This tests qocw2-specific low-level functionality
+_supported_fmt qcow2
+_supported_proto file
+_supported_os Linux
+
+echo
+echo '=== Check on an image with a multiple of 2^32 clusters ==='
+echo
+
+IMGOPTS=$(_optstr_add "$IMGOPTS" "cluster_size=512") \
+_make_test_img 512
+
+# Allocate L2 table
+$QEMU_IO -c 'write 0 512' "$TEST_IMG" | _filter_qemu_io
+
+# Put the data cluster at a multiple of 2 TB, resulting in the image apparently
+# having a multiple of 2^32 clusters
+# (To be more specific: It is at 32 PB)
+poke_file "$TEST_IMG" 2048 "\x80\x80\x00\x00\x00\x00\x00\x00"
+
+# An offset of 32 PB results in qemu-img check having to allocate an in-memory
+# refcount table of 128 TB (16 bit refcounts, 512 byte clusters).
+# This should be generally too much for any system and thus fail.
+# What this test is checking is that the qcow2 driver actually tries to 
allocate
+# such a large amount of memory (and is consequently aborting) instead of 
having
+# truncated the cluster count somewhere (which would result in much less memory
+# being allocated and then a segfault occurring).
+_check_test_img
+
+# success, all done
+echo "*** done"
+rm -f $seq.full
+status=0
diff --git a/tests/qemu-iotests/138.out b/tests/qemu-iotests/138.out
new file mode 100644
index 000..3fe911f
--- /dev/null
+++ b/tests/qemu-iotests/138.out
@@ -0,0 +1,9 @@
+QA output created by 138
+
+=== Check on an image with a multiple of 2^32 clusters ===
+
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=512
+wrote 512/512 bytes at offset 0
+512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+qemu-img: Check failed: Cannot allocate memory
+*** done
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index 3a6a8f0..439b1d2 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -135,3 +135,4 @@
 134 rw auto quick
 135 rw auto
 137 rw auto
+138 rw auto quick
-- 
1.8.3.1

[Qemu-devel] [PULL 13/23] qcow2: Move rest of option handling to qcow2_update_options()

[Kevin Wolf]

With this commit, the handling of driver-specific options in
qcow2_open() is completely separated out into qcow2_update_options().

Signed-off-by: Kevin Wolf 
Reviewed-by: Max Reitz 
---
 block/qcow2.c | 134 +-
 1 file changed, 68 insertions(+), 66 deletions(-)

diff --git a/block/qcow2.c b/block/qcow2.c
index be6b3c2..cf6992e 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -589,15 +589,77 @@ static void read_cache_sizes(BlockDriverState *bs, 
QemuOpts *opts,
 }
 }
 
-static int qcow2_update_options(BlockDriverState *bs, QemuOpts *opts,
+static int qcow2_update_options(BlockDriverState *bs, QDict *options,
 int flags, Error **errp)
 {
 BDRVQcow2State *s = bs->opaque;
+QemuOpts *opts = NULL;
 const char *opt_overlap_check, *opt_overlap_check_template;
 int overlap_check_template = 0;
+uint64_t l2_cache_size, refcount_cache_size;
+uint64_t cache_clean_interval;
 int i;
+Error *local_err = NULL;
 int ret;
 
+opts = qemu_opts_create(&qcow2_runtime_opts, NULL, 0, &error_abort);
+qemu_opts_absorb_qdict(opts, options, &local_err);
+if (local_err) {
+error_propagate(errp, local_err);
+ret = -EINVAL;
+goto fail;
+}
+
+/* get L2 table/refcount block cache size from command line options */
+read_cache_sizes(bs, opts, &l2_cache_size, &refcount_cache_size,
+ &local_err);
+if (local_err) {
+error_propagate(errp, local_err);
+ret = -EINVAL;
+goto fail;
+}
+
+l2_cache_size /= s->cluster_size;
+if (l2_cache_size < MIN_L2_CACHE_SIZE) {
+l2_cache_size = MIN_L2_CACHE_SIZE;
+}
+if (l2_cache_size > INT_MAX) {
+error_setg(errp, "L2 cache size too big");
+ret = -EINVAL;
+goto fail;
+}
+
+refcount_cache_size /= s->cluster_size;
+if (refcount_cache_size < MIN_REFCOUNT_CACHE_SIZE) {
+refcount_cache_size = MIN_REFCOUNT_CACHE_SIZE;
+}
+if (refcount_cache_size > INT_MAX) {
+error_setg(errp, "Refcount cache size too big");
+ret = -EINVAL;
+goto fail;
+}
+
+/* alloc L2 table/refcount block cache */
+s->l2_table_cache = qcow2_cache_create(bs, l2_cache_size);
+s->refcount_block_cache = qcow2_cache_create(bs, refcount_cache_size);
+if (s->l2_table_cache == NULL || s->refcount_block_cache == NULL) {
+error_setg(errp, "Could not allocate metadata caches");
+ret = -ENOMEM;
+goto fail;
+}
+
+/* New interval for cache cleanup timer */
+cache_clean_interval =
+qemu_opt_get_number(opts, QCOW2_OPT_CACHE_CLEAN_INTERVAL, 0);
+if (cache_clean_interval > UINT_MAX) {
+error_setg(errp, "Cache clean interval too big");
+ret = -EINVAL;
+goto fail;
+}
+s->cache_clean_interval = cache_clean_interval;
+cache_clean_timer_init(bs, bdrv_get_aio_context(bs));
+
+/* Enable lazy_refcounts according to image and command line options */
 s->use_lazy_refcounts = qemu_opt_get_bool(opts, QCOW2_OPT_LAZY_REFCOUNTS,
 (s->compatible_features & QCOW2_COMPAT_LAZY_REFCOUNTS));
 
@@ -660,6 +722,9 @@ static int qcow2_update_options(BlockDriverState *bs, 
QemuOpts *opts,
 
 ret = 0;
 fail:
+qemu_opts_del(opts);
+opts = NULL;
+
 return ret;
 }
 
@@ -670,12 +735,9 @@ static int qcow2_open(BlockDriverState *bs, QDict 
*options, int flags,
 unsigned int len, i;
 int ret = 0;
 QCowHeader header;
-QemuOpts *opts = NULL;
 Error *local_err = NULL;
 uint64_t ext_end;
 uint64_t l1_vm_state_index;
-uint64_t l2_cache_size, refcount_cache_size;
-uint64_t cache_clean_interval;
 
 ret = bdrv_pread(bs->file, 0, &header, sizeof(header));
 if (ret < 0) {
@@ -923,71 +985,12 @@ static int qcow2_open(BlockDriverState *bs, QDict 
*options, int flags,
 }
 }
 
-/* get L2 table/refcount block cache size from command line options */
-opts = qemu_opts_create(&qcow2_runtime_opts, NULL, 0, &error_abort);
-qemu_opts_absorb_qdict(opts, options, &local_err);
-if (local_err) {
-error_propagate(errp, local_err);
-ret = -EINVAL;
-goto fail;
-}
-
-read_cache_sizes(bs, opts, &l2_cache_size, &refcount_cache_size,
- &local_err);
-if (local_err) {
-error_propagate(errp, local_err);
-ret = -EINVAL;
-goto fail;
-}
-
-l2_cache_size /= s->cluster_size;
-if (l2_cache_size < MIN_L2_CACHE_SIZE) {
-l2_cache_size = MIN_L2_CACHE_SIZE;
-}
-if (l2_cache_size > INT_MAX) {
-error_setg(errp, "L2 cache size too big");
-ret = -EINVAL;
-goto fail;
-}
-
-refcount_cache_size /= s->cluster_size;
-if (refcount_cache_size < MIN_REFCOUNT_CACHE_SIZE) {
-refcount_cache_size = MIN_REFCOUNT_CACHE_SIZE;
-}
-if (refcount_cache_size > INT_MAX) {
-error_setg

[Qemu-devel] [PULL 16/23] qcow2: Make qcow2_update_options() suitable for transactions

[Kevin Wolf]

Before we can allow updating options at runtime with bdrv_reopen(), we
need to split the function into prepare/commit/abort parts.

Signed-off-by: Kevin Wolf 
Reviewed-by: Max Reitz 
---
 block/qcow2.c | 113 +-
 1 file changed, 73 insertions(+), 40 deletions(-)

diff --git a/block/qcow2.c b/block/qcow2.c
index 374a56d..64ba3cb 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -589,18 +589,25 @@ static void read_cache_sizes(BlockDriverState *bs, 
QemuOpts *opts,
 }
 }
 
-static int qcow2_update_options(BlockDriverState *bs, QDict *options,
-int flags, Error **errp)
+typedef struct Qcow2ReopenState {
+Qcow2Cache *l2_table_cache;
+Qcow2Cache *refcount_block_cache;
+bool use_lazy_refcounts;
+int overlap_check;
+bool discard_passthrough[QCOW2_DISCARD_MAX];
+uint64_t cache_clean_interval;
+} Qcow2ReopenState;
+
+static int qcow2_update_options_prepare(BlockDriverState *bs,
+Qcow2ReopenState *r,
+QDict *options, int flags,
+Error **errp)
 {
 BDRVQcow2State *s = bs->opaque;
 QemuOpts *opts = NULL;
 const char *opt_overlap_check, *opt_overlap_check_template;
 int overlap_check_template = 0;
 uint64_t l2_cache_size, refcount_cache_size;
-Qcow2Cache *l2_table_cache = NULL;
-Qcow2Cache *refcount_block_cache = NULL;
-uint64_t cache_clean_interval;
-bool use_lazy_refcounts;
 int i;
 Error *local_err = NULL;
 int ret;
@@ -643,27 +650,27 @@ static int qcow2_update_options(BlockDriverState *bs, 
QDict *options,
 }
 
 /* alloc L2 table/refcount block cache */
-l2_table_cache = qcow2_cache_create(bs, l2_cache_size);
-refcount_block_cache = qcow2_cache_create(bs, refcount_cache_size);
-if (l2_table_cache == NULL || refcount_block_cache == NULL) {
+r->l2_table_cache = qcow2_cache_create(bs, l2_cache_size);
+r->refcount_block_cache = qcow2_cache_create(bs, refcount_cache_size);
+if (r->l2_table_cache == NULL || r->refcount_block_cache == NULL) {
 error_setg(errp, "Could not allocate metadata caches");
 ret = -ENOMEM;
 goto fail;
 }
 
 /* New interval for cache cleanup timer */
-cache_clean_interval =
+r->cache_clean_interval =
 qemu_opt_get_number(opts, QCOW2_OPT_CACHE_CLEAN_INTERVAL, 0);
-if (cache_clean_interval > UINT_MAX) {
+if (r->cache_clean_interval > UINT_MAX) {
 error_setg(errp, "Cache clean interval too big");
 ret = -EINVAL;
 goto fail;
 }
 
 /* Enable lazy_refcounts according to image and command line options */
-use_lazy_refcounts = qemu_opt_get_bool(opts, QCOW2_OPT_LAZY_REFCOUNTS,
+r->use_lazy_refcounts = qemu_opt_get_bool(opts, QCOW2_OPT_LAZY_REFCOUNTS,
 (s->compatible_features & QCOW2_COMPAT_LAZY_REFCOUNTS));
-if (use_lazy_refcounts && s->qcow_version < 3) {
+if (r->use_lazy_refcounts && s->qcow_version < 3) {
 error_setg(errp, "Lazy refcounts require a qcow2 image with at least "
"qemu 1.1 compatibility level");
 ret = -EINVAL;
@@ -702,49 +709,75 @@ static int qcow2_update_options(BlockDriverState *bs, 
QDict *options,
 goto fail;
 }
 
-/*
- * Start updating fields in BDRVQcow2State.
- * After this point no failure is allowed any more.
- */
-s->overlap_check = 0;
+r->overlap_check = 0;
 for (i = 0; i < QCOW2_OL_MAX_BITNR; i++) {
 /* overlap-check defines a template bitmask, but every flag may be
  * overwritten through the associated boolean option */
-s->overlap_check |=
+r->overlap_check |=
 qemu_opt_get_bool(opts, overlap_bool_option_names[i],
   overlap_check_template & (1 << i)) << i;
 }
 
-s->l2_table_cache = l2_table_cache;
-s->refcount_block_cache = refcount_block_cache;
-
-s->use_lazy_refcounts = use_lazy_refcounts;
-
-s->discard_passthrough[QCOW2_DISCARD_NEVER] = false;
-s->discard_passthrough[QCOW2_DISCARD_ALWAYS] = true;
-s->discard_passthrough[QCOW2_DISCARD_REQUEST] =
+r->discard_passthrough[QCOW2_DISCARD_NEVER] = false;
+r->discard_passthrough[QCOW2_DISCARD_ALWAYS] = true;
+r->discard_passthrough[QCOW2_DISCARD_REQUEST] =
 qemu_opt_get_bool(opts, QCOW2_OPT_DISCARD_REQUEST,
   flags & BDRV_O_UNMAP);
-s->discard_passthrough[QCOW2_DISCARD_SNAPSHOT] =
+r->discard_passthrough[QCOW2_DISCARD_SNAPSHOT] =
 qemu_opt_get_bool(opts, QCOW2_OPT_DISCARD_SNAPSHOT, true);
-s->discard_passthrough[QCOW2_DISCARD_OTHER] =
+r->discard_passthrough[QCOW2_DISCARD_OTHER] =
 qemu_opt_get_bool(opts, QCOW2_OPT_DISCARD_OTHER, false);
 
-s->cache_clean_interval = cache_clean_interval;
-cache_clean_timer_init(bs, bdrv_get_aio_context(bs));
-
 ret = 0;

[Qemu-devel] [PULL 12/23] qcow2: Move qcow2_update_options() call up

[Kevin Wolf]

qcow2_update_options() only updates some variables in BDRVQcowState and
doesn't really depend on other parts of it being initialised yet, so it
can be moved so that it immediately follows the other half of option
handling code in qcow2_open().

Signed-off-by: Kevin Wolf 
Reviewed-by: Eric Blake 
Reviewed-by: Max Reitz 
---
 block/qcow2.c | 18 +-
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/block/qcow2.c b/block/qcow2.c
index 4dd0699..be6b3c2 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -979,6 +979,15 @@ static int qcow2_open(BlockDriverState *bs, QDict 
*options, int flags,
 s->cache_clean_interval = cache_clean_interval;
 cache_clean_timer_init(bs, bdrv_get_aio_context(bs));
 
+/* Enable lazy_refcounts according to image and command line options */
+ret = qcow2_update_options(bs, opts, flags, errp);
+if (ret < 0) {
+goto fail;
+}
+
+qemu_opts_del(opts);
+opts = NULL;
+
 s->cluster_cache = g_malloc(s->cluster_size);
 /* one more sector for decompressed data alignment */
 s->cluster_data = qemu_try_blockalign(bs->file, QCOW_MAX_CRYPT_CLUSTERS
@@ -1063,15 +1072,6 @@ static int qcow2_open(BlockDriverState *bs, QDict 
*options, int flags,
 }
 }
 
-/* Enable lazy_refcounts according to image and command line options */
-ret = qcow2_update_options(bs, opts, flags, errp);
-if (ret < 0) {
-goto fail;
-}
-
-qemu_opts_del(opts);
-opts = NULL;
-
 #ifdef DEBUG_ALLOC
 {
 BdrvCheckResult result = {0};
-- 
1.8.3.1

[Qemu-devel] [PULL 23/23] qcow2: Make qcow2_alloc_bytes() more explicit

[Kevin Wolf]

From: Max Reitz 

In case of -EAGAIN returned by update_refcount(), we should discard the
cluster offset we were trying to allocate and request a new one, because
in theory that old offset might now be taken by a refcount block.

In practice, this was not the case due to update_refcount() generally
returning strictly monotonic increasing cluster offsets. However, this
behavior is not set in stone, and it is also not obvious when looking at
qcow2_alloc_bytes() alone, so we should not rely on it.

Reported-by: Kevin Wolf 
Signed-off-by: Max Reitz 
Signed-off-by: Kevin Wolf 
---
 block/qcow2-refcount.c | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c
index 5f67798..3579c4d 100644
--- a/block/qcow2-refcount.c
+++ b/block/qcow2-refcount.c
@@ -949,11 +949,17 @@ int64_t qcow2_alloc_bytes(BlockDriverState *bs, int size)
 
 if (!offset || ROUND_UP(offset, s->cluster_size) != new_cluster) {
 offset = new_cluster;
+free_in_cluster = s->cluster_size;
+} else {
+free_in_cluster += s->cluster_size;
 }
 }
 
 assert(offset);
 ret = update_refcount(bs, offset, size, 1, false, QCOW2_DISCARD_NEVER);
+if (ret < 0) {
+offset = 0;
+}
 } while (ret == -EAGAIN);
 if (ret < 0) {
 return ret;
-- 
1.8.3.1

[Qemu-devel] [PULL 06/23] qcow2: Rename BDRVQcowState to BDRVQcow2State

[Kevin Wolf]

BDRVQcowState is already used by qcow1, and gdb is always confused which
one to use. Rename the qcow2 one so they can be distinguished.

Signed-off-by: Kevin Wolf 
Reviewed-by: Max Reitz 
Reviewed-by: Stefan Hajnoczi 
Reviewed-by: Alberto Garcia 
---
 block/qcow2-cache.c| 14 -
 block/qcow2-cluster.c  | 48 +++
 block/qcow2-refcount.c | 58 ++---
 block/qcow2-snapshot.c | 20 ++---
 block/qcow2.c  | 78 +-
 block/qcow2.h  | 22 +++---
 6 files changed, 120 insertions(+), 120 deletions(-)

diff --git a/block/qcow2-cache.c b/block/qcow2-cache.c
index 046f5b8..7b14c5c 100644
--- a/block/qcow2-cache.c
+++ b/block/qcow2-cache.c
@@ -55,14 +55,14 @@ struct Qcow2Cache {
 static inline void *qcow2_cache_get_table_addr(BlockDriverState *bs,
 Qcow2Cache *c, int table)
 {
-BDRVQcowState *s = bs->opaque;
+BDRVQcow2State *s = bs->opaque;
 return (uint8_t *) c->table_array + (size_t) table * s->cluster_size;
 }
 
 static inline int qcow2_cache_get_table_idx(BlockDriverState *bs,
   Qcow2Cache *c, void *table)
 {
-BDRVQcowState *s = bs->opaque;
+BDRVQcow2State *s = bs->opaque;
 ptrdiff_t table_offset = (uint8_t *) table - (uint8_t *) c->table_array;
 int idx = table_offset / s->cluster_size;
 assert(idx >= 0 && idx < c->size && table_offset % s->cluster_size == 0);
@@ -73,7 +73,7 @@ static void qcow2_cache_table_release(BlockDriverState *bs, 
Qcow2Cache *c,
   int i, int num_tables)
 {
 #if QEMU_MADV_DONTNEED != QEMU_MADV_INVALID
-BDRVQcowState *s = bs->opaque;
+BDRVQcow2State *s = bs->opaque;
 void *t = qcow2_cache_get_table_addr(bs, c, i);
 int align = getpagesize();
 size_t mem_size = (size_t) s->cluster_size * num_tables;
@@ -121,7 +121,7 @@ void qcow2_cache_clean_unused(BlockDriverState *bs, 
Qcow2Cache *c)
 
 Qcow2Cache *qcow2_cache_create(BlockDriverState *bs, int num_tables)
 {
-BDRVQcowState *s = bs->opaque;
+BDRVQcow2State *s = bs->opaque;
 Qcow2Cache *c;
 
 c = g_new0(Qcow2Cache, 1);
@@ -172,7 +172,7 @@ static int qcow2_cache_flush_dependency(BlockDriverState 
*bs, Qcow2Cache *c)
 
 static int qcow2_cache_entry_flush(BlockDriverState *bs, Qcow2Cache *c, int i)
 {
-BDRVQcowState *s = bs->opaque;
+BDRVQcow2State *s = bs->opaque;
 int ret = 0;
 
 if (!c->entries[i].dirty || !c->entries[i].offset) {
@@ -229,7 +229,7 @@ static int qcow2_cache_entry_flush(BlockDriverState *bs, 
Qcow2Cache *c, int i)
 
 int qcow2_cache_flush(BlockDriverState *bs, Qcow2Cache *c)
 {
-BDRVQcowState *s = bs->opaque;
+BDRVQcow2State *s = bs->opaque;
 int result = 0;
 int ret;
 int i;
@@ -306,7 +306,7 @@ int qcow2_cache_empty(BlockDriverState *bs, Qcow2Cache *c)
 static int qcow2_cache_do_get(BlockDriverState *bs, Qcow2Cache *c,
 uint64_t offset, void **table, bool read_from_disk)
 {
-BDRVQcowState *s = bs->opaque;
+BDRVQcow2State *s = bs->opaque;
 int i;
 int ret;
 int lookup_index;
diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
index 61309ae..412ee27 100644
--- a/block/qcow2-cluster.c
+++ b/block/qcow2-cluster.c
@@ -32,7 +32,7 @@
 int qcow2_grow_l1_table(BlockDriverState *bs, uint64_t min_size,
 bool exact_size)
 {
-BDRVQcowState *s = bs->opaque;
+BDRVQcow2State *s = bs->opaque;
 int new_l1_size2, ret, i;
 uint64_t *new_l1_table;
 int64_t old_l1_table_offset, old_l1_size;
@@ -148,7 +148,7 @@ int qcow2_grow_l1_table(BlockDriverState *bs, uint64_t 
min_size,
 static int l2_load(BlockDriverState *bs, uint64_t l2_offset,
 uint64_t **l2_table)
 {
-BDRVQcowState *s = bs->opaque;
+BDRVQcow2State *s = bs->opaque;
 int ret;
 
 ret = qcow2_cache_get(bs, s->l2_table_cache, l2_offset, (void**) l2_table);
@@ -163,7 +163,7 @@ static int l2_load(BlockDriverState *bs, uint64_t l2_offset,
 #define L1_ENTRIES_PER_SECTOR (512 / 8)
 int qcow2_write_l1_entry(BlockDriverState *bs, int l1_index)
 {
-BDRVQcowState *s = bs->opaque;
+BDRVQcow2State *s = bs->opaque;
 uint64_t buf[L1_ENTRIES_PER_SECTOR] = { 0 };
 int l1_start_index;
 int i, ret;
@@ -203,7 +203,7 @@ int qcow2_write_l1_entry(BlockDriverState *bs, int l1_index)
 
 static int l2_allocate(BlockDriverState *bs, int l1_index, uint64_t **table)
 {
-BDRVQcowState *s = bs->opaque;
+BDRVQcow2State *s = bs->opaque;
 uint64_t old_l2_offset;
 uint64_t *l2_table = NULL;
 int64_t l2_offset;
@@ -339,7 +339,7 @@ static int count_contiguous_free_clusters(uint64_t 
nb_clusters, uint64_t *l2_tab
 /* The crypt function is compatible with the linux cryptoloop
algorithm for < 4 GB images. NOTE: out_buf == in_buf is
supported */
-int qcow2_encrypt_sectors(BDRVQcowState *s, int64_t sector_num,
+int qcow2_encrypt_sectors(BDRVQcow2State *s, int64_t sector_num,
 

[Qemu-devel] [PULL 09/23] qemu-io: Add command 'reopen'

[Kevin Wolf]

Signed-off-by: Kevin Wolf 
Reviewed-by: Max Reitz 
---
 qemu-io-cmds.c | 90 ++
 1 file changed, 90 insertions(+)

diff --git a/qemu-io-cmds.c b/qemu-io-cmds.c
index 53477e1..d6572a8 100644
--- a/qemu-io-cmds.c
+++ b/qemu-io-cmds.c
@@ -1979,6 +1979,95 @@ static const cmdinfo_t map_cmd = {
.oneline= "prints the allocated areas of a file",
 };
 
+static void reopen_help(void)
+{
+printf(
+"\n"
+" Changes the open options of an already opened image\n"
+"\n"
+" Example:\n"
+" 'reopen -o lazy-refcounts=on' - activates lazy refcount writeback on a qcow2 
image\n"
+"\n"
+" -r, -- Reopen the image read-only\n"
+" -c, -- Change the cache mode to the given value\n"
+" -o, -- Changes block driver options (cf. 'open' command)\n"
+"\n");
+}
+
+static int reopen_f(BlockBackend *blk, int argc, char **argv);
+
+static QemuOptsList reopen_opts = {
+.name = "reopen",
+.merge_lists = true,
+.head = QTAILQ_HEAD_INITIALIZER(reopen_opts.head),
+.desc = {
+/* no elements => accept any params */
+{ /* end of list */ }
+},
+};
+
+static const cmdinfo_t reopen_cmd = {
+   .name   = "reopen",
+   .argmin = 0,
+   .argmax = -1,
+   .cfunc  = reopen_f,
+   .args   = "[-r] [-c cache] [-o options]",
+   .oneline= "reopens an image with new options",
+   .help   = reopen_help,
+};
+
+static int reopen_f(BlockBackend *blk, int argc, char **argv)
+{
+BlockDriverState *bs = blk_bs(blk);
+QemuOpts *qopts;
+QDict *opts;
+int c;
+int flags = bs->open_flags;
+
+BlockReopenQueue *brq;
+Error *local_err = NULL;
+
+while ((c = getopt(argc, argv, "c:o:r")) != -1) {
+switch (c) {
+case 'c':
+if (bdrv_parse_cache_flags(optarg, &flags) < 0) {
+error_report("Invalid cache option: %s", optarg);
+return 0;
+}
+break;
+case 'o':
+if (!qemu_opts_parse_noisily(&reopen_opts, optarg, 0)) {
+qemu_opts_reset(&reopen_opts);
+return 0;
+}
+break;
+case 'r':
+flags &= ~BDRV_O_RDWR;
+break;
+default:
+qemu_opts_reset(&reopen_opts);
+return qemuio_command_usage(&reopen_cmd);
+}
+}
+
+if (optind != argc) {
+qemu_opts_reset(&reopen_opts);
+return qemuio_command_usage(&reopen_cmd);
+}
+
+qopts = qemu_opts_find(&reopen_opts, NULL);
+opts = qopts ? qemu_opts_to_qdict(qopts, NULL) : NULL;
+qemu_opts_reset(&reopen_opts);
+
+brq = bdrv_reopen_queue(NULL, bs, opts, flags);
+bdrv_reopen_multiple(brq, &local_err);
+if (local_err) {
+error_report_err(local_err);
+}
+
+return 0;
+}
+
 static int break_f(BlockBackend *blk, int argc, char **argv)
 {
 int ret;
@@ -2266,6 +2355,7 @@ static void __attribute((constructor)) 
init_qemuio_commands(void)
 qemuio_add_command(&discard_cmd);
 qemuio_add_command(&alloc_cmd);
 qemuio_add_command(&map_cmd);
+qemuio_add_command(&reopen_cmd);
 qemuio_add_command(&break_cmd);
 qemuio_add_command(&remove_break_cmd);
 qemuio_add_command(&resume_cmd);
-- 
1.8.3.1

[Qemu-devel] [PULL 22/23] vmdk: Fix next_cluster_sector for compressed write

[Kevin Wolf]

From: Radoslav Gerganov 

When the VMDK is streamOptimized (or compressed), the
next_cluster_sector must not be incremented by a fixed number of
sectors. Instead of this, it must be rounded up to the next consecutive
sector. Fixing this results in much smaller compressed images.

Signed-off-by: Radoslav Gerganov 
Reviewed-by: Fam Zheng 
Signed-off-by: Kevin Wolf 
---
 block/vmdk.c | 8 ++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/block/vmdk.c b/block/vmdk.c
index 7bdc3d0..be0d640 100644
--- a/block/vmdk.c
+++ b/block/vmdk.c
@@ -1324,8 +1324,12 @@ static int vmdk_write_extent(VmdkExtent *extent, int64_t 
cluster_offset,
 
 write_end_sector = DIV_ROUND_UP(write_offset + write_len, 
BDRV_SECTOR_SIZE);
 
-extent->next_cluster_sector = MAX(extent->next_cluster_sector,
-  write_end_sector);
+if (extent->compressed) {
+extent->next_cluster_sector = write_end_sector;
+} else {
+extent->next_cluster_sector = MAX(extent->next_cluster_sector,
+  write_end_sector);
+}
 
 if (ret != write_len) {
 ret = ret < 0 ? ret : -EIO;
-- 
1.8.3.1

[Qemu-devel] [PULL 14/23] qcow2: Leave s unchanged on qcow2_update_options() failure

[Kevin Wolf]

On return, either all new options should be applied to BDRVQcowState (on
success), or all of the old settings should be preserved (on failure).

Signed-off-by: Kevin Wolf 
Reviewed-by: Max Reitz 
---
 block/qcow2.c | 57 +++--
 1 file changed, 35 insertions(+), 22 deletions(-)

diff --git a/block/qcow2.c b/block/qcow2.c
index cf6992e..c61d996 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -597,7 +597,10 @@ static int qcow2_update_options(BlockDriverState *bs, 
QDict *options,
 const char *opt_overlap_check, *opt_overlap_check_template;
 int overlap_check_template = 0;
 uint64_t l2_cache_size, refcount_cache_size;
+Qcow2Cache *l2_table_cache;
+Qcow2Cache *refcount_block_cache;
 uint64_t cache_clean_interval;
+bool use_lazy_refcounts;
 int i;
 Error *local_err = NULL;
 int ret;
@@ -640,9 +643,9 @@ static int qcow2_update_options(BlockDriverState *bs, QDict 
*options,
 }
 
 /* alloc L2 table/refcount block cache */
-s->l2_table_cache = qcow2_cache_create(bs, l2_cache_size);
-s->refcount_block_cache = qcow2_cache_create(bs, refcount_cache_size);
-if (s->l2_table_cache == NULL || s->refcount_block_cache == NULL) {
+l2_table_cache = qcow2_cache_create(bs, l2_cache_size);
+refcount_block_cache = qcow2_cache_create(bs, refcount_cache_size);
+if (l2_table_cache == NULL || refcount_block_cache == NULL) {
 error_setg(errp, "Could not allocate metadata caches");
 ret = -ENOMEM;
 goto fail;
@@ -656,23 +659,18 @@ static int qcow2_update_options(BlockDriverState *bs, 
QDict *options,
 ret = -EINVAL;
 goto fail;
 }
-s->cache_clean_interval = cache_clean_interval;
-cache_clean_timer_init(bs, bdrv_get_aio_context(bs));
 
 /* Enable lazy_refcounts according to image and command line options */
-s->use_lazy_refcounts = qemu_opt_get_bool(opts, QCOW2_OPT_LAZY_REFCOUNTS,
+use_lazy_refcounts = qemu_opt_get_bool(opts, QCOW2_OPT_LAZY_REFCOUNTS,
 (s->compatible_features & QCOW2_COMPAT_LAZY_REFCOUNTS));
+if (use_lazy_refcounts && s->qcow_version < 3) {
+error_setg(errp, "Lazy refcounts require a qcow2 image with at least "
+   "qemu 1.1 compatibility level");
+ret = -EINVAL;
+goto fail;
+}
 
-s->discard_passthrough[QCOW2_DISCARD_NEVER] = false;
-s->discard_passthrough[QCOW2_DISCARD_ALWAYS] = true;
-s->discard_passthrough[QCOW2_DISCARD_REQUEST] =
-qemu_opt_get_bool(opts, QCOW2_OPT_DISCARD_REQUEST,
-  flags & BDRV_O_UNMAP);
-s->discard_passthrough[QCOW2_DISCARD_SNAPSHOT] =
-qemu_opt_get_bool(opts, QCOW2_OPT_DISCARD_SNAPSHOT, true);
-s->discard_passthrough[QCOW2_DISCARD_OTHER] =
-qemu_opt_get_bool(opts, QCOW2_OPT_DISCARD_OTHER, false);
-
+/* Overlap check options */
 opt_overlap_check = qemu_opt_get(opts, QCOW2_OPT_OVERLAP);
 opt_overlap_check_template = qemu_opt_get(opts, 
QCOW2_OPT_OVERLAP_TEMPLATE);
 if (opt_overlap_check_template && opt_overlap_check &&
@@ -704,6 +702,10 @@ static int qcow2_update_options(BlockDriverState *bs, 
QDict *options,
 goto fail;
 }
 
+/*
+ * Start updating fields in BDRVQcow2State.
+ * After this point no failure is allowed any more.
+ */
 s->overlap_check = 0;
 for (i = 0; i < QCOW2_OL_MAX_BITNR; i++) {
 /* overlap-check defines a template bitmask, but every flag may be
@@ -713,12 +715,23 @@ static int qcow2_update_options(BlockDriverState *bs, 
QDict *options,
   overlap_check_template & (1 << i)) << i;
 }
 
-if (s->use_lazy_refcounts && s->qcow_version < 3) {
-error_setg(errp, "Lazy refcounts require a qcow2 image with at least "
-   "qemu 1.1 compatibility level");
-ret = -EINVAL;
-goto fail;
-}
+s->l2_table_cache = l2_table_cache;
+s->refcount_block_cache = refcount_block_cache;
+
+s->use_lazy_refcounts = use_lazy_refcounts;
+
+s->discard_passthrough[QCOW2_DISCARD_NEVER] = false;
+s->discard_passthrough[QCOW2_DISCARD_ALWAYS] = true;
+s->discard_passthrough[QCOW2_DISCARD_REQUEST] =
+qemu_opt_get_bool(opts, QCOW2_OPT_DISCARD_REQUEST,
+  flags & BDRV_O_UNMAP);
+s->discard_passthrough[QCOW2_DISCARD_SNAPSHOT] =
+qemu_opt_get_bool(opts, QCOW2_OPT_DISCARD_SNAPSHOT, true);
+s->discard_passthrough[QCOW2_DISCARD_OTHER] =
+qemu_opt_get_bool(opts, QCOW2_OPT_DISCARD_OTHER, false);
+
+s->cache_clean_interval = cache_clean_interval;
+cache_clean_timer_init(bs, bdrv_get_aio_context(bs));
 
 ret = 0;
 fail:
-- 
1.8.3.1

[Qemu-devel] [PULL 10/23] qcow2: Improve error message

[Kevin Wolf]

Eric says that "any" sounds better than "either", and my non-native
feeling says the same, so let's change it.

Suggested-by: Eric Blake 
Signed-off-by: Kevin Wolf 
Reviewed-by: Max Reitz 
---
 block/qcow2.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/block/qcow2.c b/block/qcow2.c
index 9b09e01..7f06d37 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -1030,7 +1030,7 @@ static int qcow2_open(BlockDriverState *bs, QDict 
*options, int flags,
 overlap_check_template = QCOW2_OL_ALL;
 } else {
 error_setg(errp, "Unsupported value '%s' for qcow2 option "
-   "'overlap-check'. Allowed are either of the following: "
+   "'overlap-check'. Allowed are any of the following: "
"none, constant, cached, all", opt_overlap_check);
 ret = -EINVAL;
 goto fail;
-- 
1.8.3.1

[Qemu-devel] [PULL 08/23] qemu-io: Remove duplicate 'open' error message

[Kevin Wolf]

qemu_opts_parse_noisily() already prints an error message with the exact
reason why the parsing failed. No need to add another less specific one.

Signed-off-by: Kevin Wolf 
Reviewed-by: Max Reitz 
---
 qemu-io.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/qemu-io.c b/qemu-io.c
index f1e3a67..269f17c 100644
--- a/qemu-io.c
+++ b/qemu-io.c
@@ -156,7 +156,6 @@ static int open_f(BlockBackend *blk, int argc, char **argv)
 break;
 case 'o':
 if (!qemu_opts_parse_noisily(&empty_opts, optarg, false)) {
-printf("could not parse option list -- %s\n", optarg);
 qemu_opts_reset(&empty_opts);
 return 0;
 }
-- 
1.8.3.1

[Qemu-devel] [PULL 07/23] block: Allow specifying driver-specific options to reopen

[Kevin Wolf]

Signed-off-by: Kevin Wolf 
Reviewed-by: Max Reitz 
---
 block.c   | 42 +++---
 block/commit.c|  4 ++--
 include/block/block.h |  4 +++-
 3 files changed, 44 insertions(+), 6 deletions(-)

diff --git a/block.c b/block.c
index 3de83e6..6268e37 100644
--- a/block.c
+++ b/block.c
@@ -1636,6 +1636,9 @@ typedef struct BlockReopenQueueEntry {
  *
  * bs is the BlockDriverState to add to the reopen queue.
  *
+ * options contains the changed options for the associated bs
+ * (the BlockReopenQueue takes ownership)
+ *
  * flags contains the open flags for the associated bs
  *
  * returns a pointer to bs_queue, which is either the newly allocated
@@ -1643,18 +1646,28 @@ typedef struct BlockReopenQueueEntry {
  *
  */
 BlockReopenQueue *bdrv_reopen_queue(BlockReopenQueue *bs_queue,
-BlockDriverState *bs, int flags)
+BlockDriverState *bs,
+QDict *options, int flags)
 {
 assert(bs != NULL);
 
 BlockReopenQueueEntry *bs_entry;
 BdrvChild *child;
+QDict *old_options;
 
 if (bs_queue == NULL) {
 bs_queue = g_new0(BlockReopenQueue, 1);
 QSIMPLEQ_INIT(bs_queue);
 }
 
+if (!options) {
+options = qdict_new();
+}
+
+old_options = qdict_clone_shallow(bs->options);
+qdict_join(options, old_options, false);
+QDECREF(old_options);
+
 /* bdrv_open() masks this flag out */
 flags &= ~BDRV_O_PROTOCOL;
 
@@ -1666,13 +1679,15 @@ BlockReopenQueue *bdrv_reopen_queue(BlockReopenQueue 
*bs_queue,
 }
 
 child_flags = child->role->inherit_flags(flags);
-bdrv_reopen_queue(bs_queue, child->bs, child_flags);
+/* TODO Pass down child flags (backing.*, extents.*, ...) */
+bdrv_reopen_queue(bs_queue, child->bs, NULL, child_flags);
 }
 
 bs_entry = g_new0(BlockReopenQueueEntry, 1);
 QSIMPLEQ_INSERT_TAIL(bs_queue, bs_entry, entry);
 
 bs_entry->state.bs = bs;
+bs_entry->state.options = options;
 bs_entry->state.flags = flags;
 
 return bs_queue;
@@ -1725,6 +1740,7 @@ cleanup:
 if (ret && bs_entry->prepared) {
 bdrv_reopen_abort(&bs_entry->state);
 }
+QDECREF(bs_entry->state.options);
 g_free(bs_entry);
 }
 g_free(bs_queue);
@@ -1737,7 +1753,7 @@ int bdrv_reopen(BlockDriverState *bs, int bdrv_flags, 
Error **errp)
 {
 int ret = -1;
 Error *local_err = NULL;
-BlockReopenQueue *queue = bdrv_reopen_queue(NULL, bs, bdrv_flags);
+BlockReopenQueue *queue = bdrv_reopen_queue(NULL, bs, NULL, bdrv_flags);
 
 ret = bdrv_reopen_multiple(queue, &local_err);
 if (local_err != NULL) {
@@ -1813,6 +1829,26 @@ int bdrv_reopen_prepare(BDRVReopenState *reopen_state, 
BlockReopenQueue *queue,
 goto error;
 }
 
+/* Options that are not handled are only okay if they are unchanged
+ * compared to the old state. It is expected that some options are only
+ * used for the initial open, but not reopen (e.g. filename) */
+if (qdict_size(reopen_state->options)) {
+const QDictEntry *entry = qdict_first(reopen_state->options);
+
+do {
+QString *new_obj = qobject_to_qstring(entry->value);
+const char *new = qstring_get_str(new_obj);
+const char *old = qdict_get_try_str(reopen_state->bs->options,
+entry->key);
+
+if (!old || strcmp(new, old)) {
+error_setg(errp, "Cannot change the option '%s'", entry->key);
+ret = -EINVAL;
+goto error;
+}
+} while ((entry = qdict_next(reopen_state->options, entry)));
+}
+
 ret = 0;
 
 error:
diff --git a/block/commit.c b/block/commit.c
index 7312a5b..d12e26f 100644
--- a/block/commit.c
+++ b/block/commit.c
@@ -236,11 +236,11 @@ void commit_start(BlockDriverState *bs, BlockDriverState 
*base,
 
 /* convert base & overlay_bs to r/w, if necessary */
 if (!(orig_base_flags & BDRV_O_RDWR)) {
-reopen_queue = bdrv_reopen_queue(reopen_queue, base,
+reopen_queue = bdrv_reopen_queue(reopen_queue, base, NULL,
  orig_base_flags | BDRV_O_RDWR);
 }
 if (!(orig_overlay_flags & BDRV_O_RDWR)) {
-reopen_queue = bdrv_reopen_queue(reopen_queue, overlay_bs,
+reopen_queue = bdrv_reopen_queue(reopen_queue, overlay_bs, NULL,
  orig_overlay_flags | BDRV_O_RDWR);
 }
 if (reopen_queue) {
diff --git a/include/block/block.h b/include/block/block.h
index e539194..ef67353 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -147,6 +147,7 @@ typedef QSIMPLEQ_HEAD(BlockReopenQueue, 
BlockReopenQueueEntry) BlockReopenQueue;
 typedef struct BDRVReopenState {
 BlockDriverState *bs;
 int flags;
+QDict *options;
 void *opaque;
 } BDRVR

[Qemu-devel] [PULL 05/23] block: Drop bdrv_find_whitelisted_format()

[Kevin Wolf]

From: Max Reitz 

It is unused by now, so we can drop it.

Signed-off-by: Max Reitz 
Reviewed-by: Alberto Garcia 
Signed-off-by: Kevin Wolf 
---
 block.c   | 7 ---
 include/block/block.h | 2 --
 2 files changed, 9 deletions(-)

diff --git a/block.c b/block.c
index 7c61555..3de83e6 100644
--- a/block.c
+++ b/block.c
@@ -313,13 +313,6 @@ static int bdrv_is_whitelisted(BlockDriver *drv, bool 
read_only)
 return 0;
 }
 
-BlockDriver *bdrv_find_whitelisted_format(const char *format_name,
-  bool read_only)
-{
-BlockDriver *drv = bdrv_find_format(format_name);
-return drv && bdrv_is_whitelisted(drv, read_only) ? drv : NULL;
-}
-
 typedef struct CreateCo {
 BlockDriver *drv;
 char *filename;
diff --git a/include/block/block.h b/include/block/block.h
index ab4518c..e539194 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -193,8 +193,6 @@ BlockDriver *bdrv_find_protocol(const char *filename,
 bool allow_protocol_prefix,
 Error **errp);
 BlockDriver *bdrv_find_format(const char *format_name);
-BlockDriver *bdrv_find_whitelisted_format(const char *format_name,
-  bool readonly);
 int bdrv_create(BlockDriver *drv, const char* filename,
 QemuOpts *opts, Error **errp);
 int bdrv_create_file(const char *filename, QemuOpts *opts, Error **errp);
-- 
1.8.3.1

[Qemu-devel] [PULL 02/23] block: Drop drv parameter from bdrv_open()

[Kevin Wolf]

From: Max Reitz 

Now that this parameter is effectively unused, we can drop it and just
pass NULL on to bdrv_open_inherit().

Signed-off-by: Max Reitz 
Reviewed-by: Alberto Garcia 
Signed-off-by: Kevin Wolf 
---
 block.c   | 9 -
 block/block-backend.c | 2 +-
 block/parallels.c | 2 +-
 block/qcow.c  | 2 +-
 block/qcow2.c | 6 +++---
 block/qed.c   | 2 +-
 block/sheepdog.c  | 5 ++---
 block/vdi.c   | 2 +-
 block/vhdx.c  | 2 +-
 block/vmdk.c  | 7 +++
 block/vpc.c   | 2 +-
 block/vvfat.c | 2 +-
 blockdev.c| 8 
 include/block/block.h | 3 +--
 14 files changed, 25 insertions(+), 29 deletions(-)

diff --git a/block.c b/block.c
index d0b9101..0e1b4b4 100644
--- a/block.c
+++ b/block.c
@@ -1391,7 +1391,7 @@ int bdrv_append_temp_snapshot(BlockDriverState *bs, int 
flags, Error **errp)
 bs_snapshot = bdrv_new();
 
 ret = bdrv_open(&bs_snapshot, NULL, NULL, snapshot_options,
-flags, NULL, &local_err);
+flags, &local_err);
 if (ret < 0) {
 error_propagate(errp, local_err);
 goto out;
@@ -1637,11 +1637,10 @@ close_and_fail:
 }
 
 int bdrv_open(BlockDriverState **pbs, const char *filename,
-  const char *reference, QDict *options, int flags,
-  BlockDriver *drv, Error **errp)
+  const char *reference, QDict *options, int flags, Error **errp)
 {
 return bdrv_open_inherit(pbs, filename, reference, options, flags, NULL,
- NULL, drv, errp);
+ NULL, NULL, errp);
 }
 
 typedef struct BlockReopenQueueEntry {
@@ -3846,7 +3845,7 @@ void bdrv_img_create(const char *filename, const char 
*fmt,
 
 bs = NULL;
 ret = bdrv_open(&bs, full_backing, NULL, backing_options,
-back_flags, NULL, &local_err);
+back_flags, &local_err);
 g_free(full_backing);
 if (ret < 0) {
 goto out;
diff --git a/block/block-backend.c b/block/block-backend.c
index aee8a12..c2e8732 100644
--- a/block/block-backend.c
+++ b/block/block-backend.c
@@ -126,7 +126,7 @@ BlockBackend *blk_new_open(const char *name, const char 
*filename,
 return NULL;
 }
 
-ret = bdrv_open(&blk->bs, filename, reference, options, flags, NULL, errp);
+ret = bdrv_open(&blk->bs, filename, reference, options, flags, errp);
 if (ret < 0) {
 blk_unref(blk);
 return NULL;
diff --git a/block/parallels.c b/block/parallels.c
index 046b568..5cd6ec3 100644
--- a/block/parallels.c
+++ b/block/parallels.c
@@ -476,7 +476,7 @@ static int parallels_create(const char *filename, QemuOpts 
*opts, Error **errp)
 
 file = NULL;
 ret = bdrv_open(&file, filename, NULL, NULL,
-BDRV_O_RDWR | BDRV_O_PROTOCOL, NULL, &local_err);
+BDRV_O_RDWR | BDRV_O_PROTOCOL, &local_err);
 if (ret < 0) {
 error_propagate(errp, local_err);
 return ret;
diff --git a/block/qcow.c b/block/qcow.c
index 01fba54..6e35db1 100644
--- a/block/qcow.c
+++ b/block/qcow.c
@@ -793,7 +793,7 @@ static int qcow_create(const char *filename, QemuOpts 
*opts, Error **errp)
 
 qcow_bs = NULL;
 ret = bdrv_open(&qcow_bs, filename, NULL, NULL,
-BDRV_O_RDWR | BDRV_O_PROTOCOL, NULL, &local_err);
+BDRV_O_RDWR | BDRV_O_PROTOCOL, &local_err);
 if (ret < 0) {
 error_propagate(errp, local_err);
 goto cleanup;
diff --git a/block/qcow2.c b/block/qcow2.c
index 867b43b..a707d8d 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -1975,7 +1975,7 @@ static int qcow2_create2(const char *filename, int64_t 
total_size,
 
 bs = NULL;
 ret = bdrv_open(&bs, filename, NULL, NULL, BDRV_O_RDWR | BDRV_O_PROTOCOL,
-NULL, &local_err);
+&local_err);
 if (ret < 0) {
 error_propagate(errp, local_err);
 return ret;
@@ -2038,7 +2038,7 @@ static int qcow2_create2(const char *filename, int64_t 
total_size,
 qdict_put(options, "driver", qstring_from_str("qcow2"));
 ret = bdrv_open(&bs, filename, NULL, options,
 BDRV_O_RDWR | BDRV_O_CACHE_WB | BDRV_O_NO_FLUSH,
-NULL, &local_err);
+&local_err);
 if (ret < 0) {
 error_propagate(errp, local_err);
 goto out;
@@ -2092,7 +2092,7 @@ static int qcow2_create2(const char *filename, int64_t 
total_size,
 qdict_put(options, "driver", qstring_from_str("qcow2"));
 ret = bdrv_open(&bs, filename, NULL, options,
 BDRV_O_RDWR | BDRV_O_CACHE_WB | BDRV_O_NO_BACKING,
-NULL, &local_err);
+&local_err);
 if (local_err) {
 error_propagate(errp, local_err);
 goto out;
diff --git a/block/qed.c b/block/qed.c
index 954ed00..a7ff1d9 100644
--- a/block/qed.c
+++ b/bl

[Qemu-devel] [PULL 00/23] Block layer patches

[Kevin Wolf]

The following changes since commit 30c38c90bd3f1bb105ebc069ac1821067c980b7c:

  scripts/qemu-gdb: Add brief comment describing usage (2015-09-11 17:14:50 
+0100)

are available in the git repository at:

  git://repo.or.cz/qemu/kevin.git tags/for-upstream

for you to fetch changes up to 1fcbcc93872953d08cd35830d1169fed19196290:

  qcow2: Make qcow2_alloc_bytes() more explicit (2015-09-11 20:03:02 +0200)


Block layer patches


Kevin Wolf (14):
  qcow2: Rename BDRVQcowState to BDRVQcow2State
  block: Allow specifying driver-specific options to reopen
  qemu-io: Remove duplicate 'open' error message
  qemu-io: Add command 'reopen'
  qcow2: Improve error message
  qcow2: Factor out qcow2_update_options()
  qcow2: Move qcow2_update_options() call up
  qcow2: Move rest of option handling to qcow2_update_options()
  qcow2: Leave s unchanged on qcow2_update_options() failure
  qcow2: Fix memory leak in qcow2_update_options() error path
  qcow2: Make qcow2_update_options() suitable for transactions
  qcow2: Support updating driver-specific options in reopen
  qemu-iotests: Reopen qcow2 with lazy-refcounts change
  qemu-iotests: More qcow2 reopen tests

Max Reitz (8):
  block: Always pass NULL as drv for bdrv_open()
  block: Drop drv parameter from bdrv_open()
  block: Drop drv parameter from bdrv_open_inherit()
  block: Drop drv parameter from bdrv_fill_options()
  block: Drop bdrv_find_whitelisted_format()
  qcow2: Make size_to_clusters() return uint64_t
  iotests: Add test for checking large image files
  qcow2: Make qcow2_alloc_bytes() more explicit

Radoslav Gerganov (1):
  vmdk: Fix next_cluster_sector for compressed write

 block.c| 150 +++---
 block/block-backend.c  |   2 +-
 block/commit.c |   4 +-
 block/parallels.c  |   2 +-
 block/qcow.c   |   2 +-
 block/qcow2-cache.c|  14 +-
 block/qcow2-cluster.c  |  76 +++
 block/qcow2-refcount.c |  74 ---
 block/qcow2-snapshot.c |  20 +-
 block/qcow2.c  | 486 +
 block/qcow2.h  |  26 +--
 block/qed.c|   2 +-
 block/sheepdog.c   |   5 +-
 block/vdi.c|   2 +-
 block/vhdx.c   |   2 +-
 block/vmdk.c   |  15 +-
 block/vpc.c|   2 +-
 block/vvfat.c  |   8 +-
 blockdev.c |  72 +++
 include/block/block.h  |   9 +-
 qemu-io-cmds.c |  90 +
 qemu-io.c  |   1 -
 tests/qemu-iotests/039 |  27 +++
 tests/qemu-iotests/039.out |  18 ++
 tests/qemu-iotests/137 | 145 ++
 tests/qemu-iotests/137.out |  42 
 tests/qemu-iotests/138 |  73 +++
 tests/qemu-iotests/138.out |   9 +
 tests/qemu-iotests/group   |   2 +
 29 files changed, 972 insertions(+), 408 deletions(-)
 create mode 100755 tests/qemu-iotests/137
 create mode 100644 tests/qemu-iotests/137.out
 create mode 100755 tests/qemu-iotests/138
 create mode 100644 tests/qemu-iotests/138.out

[Qemu-devel] [PATCH 2/4] target-i386: Convert kvm_default_*features to property/value pairs

[Eduardo Habkost]

Convert the kvm_default_features and kvm_default_unset_features arrays
into a simple list of property/value pairs that will be applied to
X86CPU objects when using KVM.

Signed-off-by: Eduardo Habkost 
---
 hw/i386/pc_piix.c |  8 ++---
 hw/i386/pc_q35.c  |  4 +--
 target-i386/cpu.c | 87 ---
 target-i386/cpu.h | 11 +--
 4 files changed, 65 insertions(+), 45 deletions(-)

diff --git a/hw/i386/pc_piix.c b/hw/i386/pc_piix.c
index 3f925b2..e79ef0d 100644
--- a/hw/i386/pc_piix.c
+++ b/hw/i386/pc_piix.c
@@ -325,7 +325,7 @@ static void pc_compat_2_1(MachineState *machine)
 
 pc_compat_2_2(machine);
 smbios_uuid_encoded = false;
-x86_cpu_compat_kvm_no_autodisable(FEAT_8000_0001_ECX, CPUID_EXT3_SVM);
+x86_cpu_change_kvm_default("svm", NULL);
 pcms->enforce_aligned_dimm = false;
 }
 
@@ -361,7 +361,7 @@ static void pc_compat_1_7(MachineState *machine)
 gigabyte_align = false;
 option_rom_has_mr = true;
 legacy_acpi_table_size = 6414;
-x86_cpu_compat_kvm_no_autoenable(FEAT_1_ECX, CPUID_EXT_X2APIC);
+x86_cpu_change_kvm_default("x2apic", NULL);
 }
 
 static void pc_compat_1_6(MachineState *machine)
@@ -391,7 +391,7 @@ static void pc_compat_1_3(MachineState *machine)
 static void pc_compat_1_2(MachineState *machine)
 {
 pc_compat_1_3(machine);
-x86_cpu_compat_kvm_no_autoenable(FEAT_KVM, 1 << KVM_FEATURE_PV_EOI);
+x86_cpu_change_kvm_default("kvm-pv-eoi", NULL);
 }
 
 /* PC compat function for pc-0.10 to pc-0.13 */
@@ -414,7 +414,7 @@ static void pc_init_isa(MachineState *machine)
 if (!machine->cpu_model) {
 machine->cpu_model = "486";
 }
-x86_cpu_compat_kvm_no_autoenable(FEAT_KVM, 1 << KVM_FEATURE_PV_EOI);
+x86_cpu_change_kvm_default("kvm-pv-eoi", NULL);
 enable_compat_apic_id_mode();
 pc_init1(machine, TYPE_I440FX_PCI_HOST_BRIDGE, TYPE_I440FX_PCI_DEVICE);
 }
diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c
index 11601ab..c17d9df 100644
--- a/hw/i386/pc_q35.c
+++ b/hw/i386/pc_q35.c
@@ -309,7 +309,7 @@ static void pc_compat_2_1(MachineState *machine)
 pc_compat_2_2(machine);
 pcms->enforce_aligned_dimm = false;
 smbios_uuid_encoded = false;
-x86_cpu_compat_kvm_no_autodisable(FEAT_8000_0001_ECX, CPUID_EXT3_SVM);
+x86_cpu_change_kvm_default("svm", NULL);
 }
 
 static void pc_compat_2_0(MachineState *machine)
@@ -326,7 +326,7 @@ static void pc_compat_1_7(MachineState *machine)
 smbios_defaults = false;
 gigabyte_align = false;
 option_rom_has_mr = true;
-x86_cpu_compat_kvm_no_autoenable(FEAT_1_ECX, CPUID_EXT_X2APIC);
+x86_cpu_change_kvm_default("x2apic", NULL);
 }
 
 static void pc_compat_1_6(MachineState *machine)
diff --git a/target-i386/cpu.c b/target-i386/cpu.c
index d505743..9a4bfa3 100644
--- a/target-i386/cpu.c
+++ b/target-i386/cpu.c
@@ -478,38 +478,6 @@ const char *get_register_name_32(unsigned int reg)
 return x86_reg_info_32[reg].name;
 }
 
-/* KVM-specific features that are automatically added to all CPU models
- * when KVM is enabled.
- */
-static uint32_t kvm_default_features[FEATURE_WORDS] = {
-[FEAT_KVM] = (1 << KVM_FEATURE_CLOCKSOURCE) |
-(1 << KVM_FEATURE_NOP_IO_DELAY) |
-(1 << KVM_FEATURE_CLOCKSOURCE2) |
-(1 << KVM_FEATURE_ASYNC_PF) |
-(1 << KVM_FEATURE_STEAL_TIME) |
-(1 << KVM_FEATURE_PV_EOI) |
-(1 << KVM_FEATURE_CLOCKSOURCE_STABLE_BIT),
-[FEAT_1_ECX] = CPUID_EXT_X2APIC,
-};
-
-/* Features that are not added by default to any CPU model when KVM is enabled.
- */
-static uint32_t kvm_default_unset_features[FEATURE_WORDS] = {
-[FEAT_1_EDX] = CPUID_ACPI,
-[FEAT_1_ECX] = CPUID_EXT_MONITOR,
-[FEAT_8000_0001_ECX] = CPUID_EXT3_SVM,
-};
-
-void x86_cpu_compat_kvm_no_autoenable(FeatureWord w, uint32_t features)
-{
-kvm_default_features[w] &= ~features;
-}
-
-void x86_cpu_compat_kvm_no_autodisable(FeatureWord w, uint32_t features)
-{
-kvm_default_unset_features[w] &= ~features;
-}
-
 /*
  * Returns the set of feature flags that are supported and migratable by
  * QEMU, for a given FeatureWord.
@@ -1398,6 +1366,43 @@ static X86CPUDefinition builtin_x86_defs[] = {
 },
 };
 
+typedef struct PropValue {
+const char *prop, *value;
+} PropValue;
+
+/* KVM-specific features that are automatically added/removed
+ * from all CPU models when KVM is enabled.
+ */
+static PropValue kvm_default_props[] = {
+{ "kvmclock", "on" },
+{ "kvm-nopiodelay", "on" },
+{ "kvm-asyncpf", "on" },
+{ "kvm-steal-time", "on" },
+{ "kvm-pv-eoi", "on" },
+{ "kvmclock-stable-bit", "on" },
+{ "x2apic", "on" },
+{ "acpi", "off" },
+{ "monitor", "off" },
+{ "svm", "off" },
+{ NULL, NULL },
+};
+
+void x86_cpu_change_kvm_default(const char *prop, const char *value)
+{
+PropValue *pv;
+for (pv = kvm_default_props; pv->prop; pv++) {
+if (!strcmp(pv->prop, prop)) {
+pv->value = value;
+break;
+}
+}

[Qemu-devel] [PATCH 0/4] target-i386: Don't try to enable unsupported TCG features by default

[Eduardo Habkost]

This series makes QEMU stop trying to enable an unsupported feature by default
in TCG mode, avoiding the warnings shown by the (now enabled by default)
"check" mode.

To do that, I cchanged the kvm_default_features code to be based on QOM
properties, and added a new "type" field to allow per-CPU-model KVM defaults to
be specified.

Before this series:

  $ qemu-system-x86_64 -S
  warning: TCG doesn't support requested feature: CPUID.01H:EDX.de [bit 2]
  warning: TCG doesn't support requested feature: CPUID.8001H:EDX [bit 2]

After this series:

  $ ./x86_64-softmmu/qemu-system-x86_64 -S
  [no warning output]

Eduardo Habkost (4):
  target-i386: Expand PPRO_FEATURES on qemu64/qemu32 definitions
  target-i386: Convert kvm_default_*features to property/value pairs
  target-i386: Add optional class name to kvm_default_props
  target-i386: Disable DE on qemu64 & qemu32 on TCG

 hw/i386/pc_piix.c |   8 ++---
 hw/i386/pc_q35.c  |   4 +--
 target-i386/cpu.c | 104 ++
 target-i386/cpu.h |  12 +--
 4 files changed, 81 insertions(+), 47 deletions(-)

-- 
2.1.0

[Qemu-devel] [PATCH 4/4] target-i386: Disable DE on qemu64 & qemu32 on TCG

[Eduardo Habkost]

We don't want to make QEMU show CPUID feature warnings by default when
running in TCG mode, so disable the flag in the default qemu32 and
qemu64 CPU models, and add it to kvm_default_props so it gets enabled
when running KVM.

We don't need any compat code for this because:
* DE was already enabled in KVM mode, and is kept enabled
* DE was already disabled in TCG mode, and is kept disabled

Signed-off-by: Eduardo Habkost 
---
 target-i386/cpu.c | 6 --
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/target-i386/cpu.c b/target-i386/cpu.c
index ec9eca2..1617e48 100644
--- a/target-i386/cpu.c
+++ b/target-i386/cpu.c
@@ -669,7 +669,7 @@ static X86CPUDefinition builtin_x86_defs[] = {
 .model = 6,
 .stepping = 3,
 .features[FEAT_1_EDX] =
-CPUID_FP87 | CPUID_DE | CPUID_PSE | CPUID_TSC |
+CPUID_FP87 | CPUID_PSE | CPUID_TSC |
 CPUID_MSR | CPUID_MCE | CPUID_CX8 | CPUID_PGE | CPUID_CMOV |
 CPUID_PAT | CPUID_FXSR | CPUID_MMX | CPUID_SSE | CPUID_SSE2 |
 CPUID_PAE | CPUID_SEP | CPUID_APIC |
@@ -775,7 +775,7 @@ static X86CPUDefinition builtin_x86_defs[] = {
 .model = 6,
 .stepping = 3,
 .features[FEAT_1_EDX] =
-CPUID_FP87 | CPUID_DE | CPUID_PSE | CPUID_TSC |
+CPUID_FP87 | CPUID_PSE | CPUID_TSC |
 CPUID_MSR | CPUID_MCE | CPUID_CX8 | CPUID_PGE | CPUID_CMOV |
 CPUID_PAT | CPUID_FXSR | CPUID_MMX | CPUID_SSE | CPUID_SSE2 |
 CPUID_PAE | CPUID_SEP | CPUID_APIC,
@@ -1385,6 +1385,8 @@ static PropValue kvm_default_props[] = {
 { NULL, "acpi", "off" },
 { NULL, "monitor", "off" },
 { NULL, "svm", "off" },
+{ X86_CPU_TYPE_NAME("qemu64"), "de", "on" },
+{ X86_CPU_TYPE_NAME("qemu32"), "de", "on" },
 { NULL, NULL },
 };
 
-- 
2.1.0

[Qemu-devel] [PATCH v6 19/26] qapi: Improve built-in type documentation

[Markus Armbruster]

Clarify how they map to JSON.  Add how they map to C.  Fix the
reference to StringInputVisitor.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
---
 docs/qapi-code-gen.txt | 29 ++---
 1 file changed, 18 insertions(+), 11 deletions(-)

diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 147638f..35a4a0f 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -140,17 +140,24 @@ must have a value that forms a struct name.
 
 === Built-in Types ===
 
-The following types are built-in to the parser:
-  'str' - arbitrary UTF-8 string
-  'int' - 64-bit signed integer (although the C code may place further
-  restrictions on acceptable range)
-  'number' - floating point number
-  'bool' - JSON value of true or false
-  'int8', 'int16', 'int32', 'int64' - like 'int', but enforce maximum
-  bit size
-  'uint8', 'uint16', 'uint32', 'uint64' - unsigned counterparts
-  'size' - like 'uint64', but allows scaled suffix from command line
-   visitor
+The following types are predefined, and map to C as follows:
+
+  SchemaC  JSON
+  str   char * any JSON string, UTF-8
+  numberdouble any JSON number
+  int   int64_ta JSON number without fractional part
+   that fits into the C integer type
+  int8  int8_t likewise
+  int16 int16_tlikewise
+  int32 int32_tlikewise
+  int64 int64_tlikewise
+  uint8 uint8_tlikewise
+  uint16uint16_t   likewise
+  uint32uint32_t   likewise
+  uint64uint64_t   likewise
+  size  uint64_t   like uint64_t, except StringInputVisitor
+   accepts size suffixes
+  bool  bool   JSON true or false
 
 
 === Includes ===
-- 
2.4.3

[Qemu-devel] [PATCH 1/4] target-i386: Expand PPRO_FEATURES on qemu64/qemu32 definitions

[Eduardo Habkost]

The qemu64 and qemu32 CPU models won't be Pentium Pro supersets anymore
because the DE feature will be removed. Copy PPRO_FEATURES into their
definitions so we can change the CPU models later.

Signed-off-by: Eduardo Habkost 
---
 target-i386/cpu.c | 10 --
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/target-i386/cpu.c b/target-i386/cpu.c
index 32fee00..d505743 100644
--- a/target-i386/cpu.c
+++ b/target-i386/cpu.c
@@ -701,7 +701,10 @@ static X86CPUDefinition builtin_x86_defs[] = {
 .model = 6,
 .stepping = 3,
 .features[FEAT_1_EDX] =
-PPRO_FEATURES |
+CPUID_FP87 | CPUID_DE | CPUID_PSE | CPUID_TSC |
+CPUID_MSR | CPUID_MCE | CPUID_CX8 | CPUID_PGE | CPUID_CMOV |
+CPUID_PAT | CPUID_FXSR | CPUID_MMX | CPUID_SSE | CPUID_SSE2 |
+CPUID_PAE | CPUID_SEP | CPUID_APIC |
 CPUID_MTRR | CPUID_CLFLUSH | CPUID_MCA |
 CPUID_PSE36,
 .features[FEAT_1_ECX] =
@@ -804,7 +807,10 @@ static X86CPUDefinition builtin_x86_defs[] = {
 .model = 6,
 .stepping = 3,
 .features[FEAT_1_EDX] =
-PPRO_FEATURES,
+CPUID_FP87 | CPUID_DE | CPUID_PSE | CPUID_TSC |
+CPUID_MSR | CPUID_MCE | CPUID_CX8 | CPUID_PGE | CPUID_CMOV |
+CPUID_PAT | CPUID_FXSR | CPUID_MMX | CPUID_SSE | CPUID_SSE2 |
+CPUID_PAE | CPUID_SEP | CPUID_APIC,
 .features[FEAT_1_ECX] =
 CPUID_EXT_SSE3 | CPUID_EXT_POPCNT,
 .xlevel = 0x8004,
-- 
2.1.0

Re: [Qemu-devel] [PATCH 0/4] target-i386: Don't try to enable unsupported TCG features by default

[Richard Henderson]

On 09/11/2015 12:25 PM, Eduardo Habkost wrote:
> This series makes QEMU stop trying to enable an unsupported feature by default
> in TCG mode, avoiding the warnings shown by the (now enabled by default)
> "check" mode.
> 
> To do that, I cchanged the kvm_default_features code to be based on QOM
> properties, and added a new "type" field to allow per-CPU-model KVM defaults 
> to
> be specified.
> 
> Before this series:
> 
>   $ qemu-system-x86_64 -S
>   warning: TCG doesn't support requested feature: CPUID.01H:EDX.de [bit 2]
>   warning: TCG doesn't support requested feature: CPUID.8001H:EDX [bit 2]
> 
> After this series:
> 
>   $ ./x86_64-softmmu/qemu-system-x86_64 -S
>   [no warning output]

Wouldn't it be easier simply to implement this bit?
It seems like we're simply creating a compatibility mess otherwise.


r~

[Qemu-devel] [PATCH v6 22/26] qom: Don't use 'gen': false for qom-get, qom-set, object-add

[Markus Armbruster]

With the previous commit, the generated marshalers just work, and save
us a bit of handwritten code.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
---
 include/monitor/monitor.h |  3 ---
 qapi-schema.json  |  9 +++--
 qmp-commands.hx   |  6 +++---
 qmp.c | 21 +++--
 scripts/qapi.py   |  1 +
 5 files changed, 14 insertions(+), 26 deletions(-)

diff --git a/include/monitor/monitor.h b/include/monitor/monitor.h
index 9aff47e..bc6cb6d 100644
--- a/include/monitor/monitor.h
+++ b/include/monitor/monitor.h
@@ -42,9 +42,6 @@ void monitor_read_command(Monitor *mon, int show_prompt);
 int monitor_read_password(Monitor *mon, ReadLineFunc *readline_func,
   void *opaque);
 
-void qmp_qom_set(QDict *qdict, QObject **ret, Error **errp);
-void qmp_qom_get(QDict *qdict, QObject **ret, Error **errp);
-void qmp_object_add(QDict *qdict, QObject **ret, Error **errp);
 void object_add(const char *type, const char *id, const QDict *qdict,
 Visitor *v, Error **errp);
 
diff --git a/qapi-schema.json b/qapi-schema.json
index 4060b78..a11596a 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -1695,8 +1695,7 @@
 ##
 { 'command': 'qom-get',
   'data': { 'path': 'str', 'property': 'str' },
-  'returns': '**',
-  'gen': false }
+  'returns': 'any' }
 
 ##
 # @qom-set:
@@ -1713,8 +1712,7 @@
 # Since: 1.2
 ##
 { 'command': 'qom-set',
-  'data': { 'path': 'str', 'property': 'str', 'value': '**' },
-  'gen': false }
+  'data': { 'path': 'str', 'property': 'str', 'value': 'any' } }
 
 ##
 # @set_password:
@@ -2124,8 +2122,7 @@
 # Since: 2.0
 ##
 { 'command': 'object-add',
-  'data': {'qom-type': 'str', 'id': 'str', '*props': '**'},
-  'gen': false }
+  'data': {'qom-type': 'str', 'id': 'str', '*props': 'any'} }
 
 ##
 # @object-del:
diff --git a/qmp-commands.hx b/qmp-commands.hx
index 4640a3d..7c74e20 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -953,7 +953,7 @@ EQMP
 {
 .name   = "object-add",
 .args_type  = "qom-type:s,id:s,props:q?",
-.mhandler.cmd_new = qmp_object_add,
+.mhandler.cmd_new = qmp_marshal_object_add,
 },
 
 SQMP
@@ -3575,13 +3575,13 @@ EQMP
 {
 .name   = "qom-set",
.args_type  = "path:s,property:s,value:q",
-   .mhandler.cmd_new = qmp_qom_set,
+.mhandler.cmd_new = qmp_marshal_qom_set,
 },
 
 {
 .name   = "qom-get",
.args_type  = "path:s,property:s",
-   .mhandler.cmd_new = qmp_qom_get,
+.mhandler.cmd_new = qmp_marshal_qom_get,
 },
 
 {
diff --git a/qmp.c b/qmp.c
index f31c6dc..b8b8b5b 100644
--- a/qmp.c
+++ b/qmp.c
@@ -228,12 +228,9 @@ ObjectPropertyInfoList *qmp_qom_list(const char *path, 
Error **errp)
 return props;
 }
 
-/* FIXME: teach qapi about how to pass through Visitors */
-void qmp_qom_set(QDict *qdict, QObject **ret, Error **errp)
+void qmp_qom_set(const char *path, const char *property, QObject *value,
+ Error **errp)
 {
-const char *path = qdict_get_str(qdict, "path");
-const char *property = qdict_get_str(qdict, "property");
-QObject *value = qdict_get(qdict, "value");
 Object *obj;
 
 obj = object_resolve_path(path, NULL);
@@ -246,20 +243,18 @@ void qmp_qom_set(QDict *qdict, QObject **ret, Error 
**errp)
 object_property_set_qobject(obj, value, property, errp);
 }
 
-void qmp_qom_get(QDict *qdict, QObject **ret, Error **errp)
+QObject *qmp_qom_get(const char *path, const char *property, Error **errp)
 {
-const char *path = qdict_get_str(qdict, "path");
-const char *property = qdict_get_str(qdict, "property");
 Object *obj;
 
 obj = object_resolve_path(path, NULL);
 if (!obj) {
 error_set(errp, ERROR_CLASS_DEVICE_NOT_FOUND,
   "Device '%s' not found", path);
-return;
+return NULL;
 }
 
-*ret = object_property_get_qobject(obj, property, errp);
+return object_property_get_qobject(obj, property, errp);
 }
 
 void qmp_set_password(const char *protocol, const char *password,
@@ -655,11 +650,9 @@ out:
 object_unref(obj);
 }
 
-void qmp_object_add(QDict *qdict, QObject **ret, Error **errp)
+void qmp_object_add(const char *type, const char *id,
+bool has_props, QObject *props, Error **errp)
 {
-const char *type = qdict_get_str(qdict, "qom-type");
-const char *id = qdict_get_str(qdict, "id");
-QObject *props = qdict_get(qdict, "props");
 const QDict *pdict = NULL;
 QmpInputVisitor *qiv;
 
diff --git a/scripts/qapi.py b/scripts/qapi.py
index e8c0ea4..415f782 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -40,6 +40,7 @@ builtin_types = {
 returns_whitelist = [
 # From QMP:
 'human-monitor-command',
+'qom-get',
 'query-migrate-cache-size',
 'query-tpm-models',
 'query-tpm-types',
-- 
2.4.3

[Qemu-devel] [PATCH v6 13/26] qapi: Clean up after recent conversions to QAPISchemaVisitor

[Markus Armbruster]

Generate just 'FOO' instead of 'struct FOO' when possible.

Drop helper functions that are now unused.

Make pep8 and pylint reasonably happy.

Rename generate_FOO() functions to gen_FOO() for consistency.

Use more consistent and sensible variable names.

Consistently use c_ for mapping keys when their value is a C
identifier or type.

Simplify gen_enum() and gen_visit_union()

Consistently use single quotes for C text string literals.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
---
 docs/qapi-code-gen.txt   |   2 +-
 scripts/qapi-commands.py | 140 ++-
 scripts/qapi-event.py| 122 -
 scripts/qapi-types.py|  79 ++
 scripts/qapi-visit.py| 127 ++
 scripts/qapi.py  | 131 +---
 6 files changed, 274 insertions(+), 327 deletions(-)

diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 7c500b6..e85f113 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -590,7 +590,7 @@ Example:
 UserDefOne *value;
 uint64_t padding;
 };
-struct UserDefOneList *next;
+UserDefOneList *next;
 };
 
 void qapi_free_UserDefOneList(UserDefOneList *obj);
diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index cbff356..0501582 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -15,20 +15,22 @@
 from qapi import *
 import re
 
-def generate_command_decl(name, args, ret_type):
-arglist=""
-if args:
-for memb in args.members:
-argtype = memb.type.c_type(is_param=True)
+
+def gen_command_decl(name, arg_type, ret_type):
+argstr = ''
+if arg_type:
+for memb in arg_type.members:
 if memb.optional:
-arglist += "bool has_%s, " % c_name(memb.name)
-arglist += "%s %s, " % (argtype, c_name(memb.name))
+argstr += 'bool has_%s, ' % c_name(memb.name)
+argstr += '%s %s, ' % (memb.type.c_type(is_param=True),
+   c_name(memb.name))
 return mcgen('''
-%(ret_type)s qmp_%(name)s(%(args)sError **errp);
+%(c_type)s qmp_%(c_name)s(%(args)sError **errp);
 ''',
- ret_type=(ret_type and ret_type.c_type()) or 'void',
- name=c_name(name),
- args=arglist)
+ c_type=(ret_type and ret_type.c_type()) or 'void',
+ c_name=c_name(name),
+ args=argstr)
+
 
 def gen_err_check(err):
 if not err:
@@ -40,37 +42,42 @@ if (%(err)s) {
 ''',
  err=err)
 
-def gen_sync_call(name, args, ret_type):
-ret = ""
-arglist=""
-retval=""
-if ret_type:
-retval = "retval = "
-if args:
-for memb in args.members:
+
+def gen_call(name, arg_type, ret_type):
+ret = ''
+
+argstr = ''
+if arg_type:
+for memb in arg_type.members:
 if memb.optional:
-arglist += "has_%s, " % c_name(memb.name)
-arglist += "%s, " % c_name(memb.name)
+argstr += 'has_%s, ' % c_name(memb.name)
+argstr += '%s, ' % c_name(memb.name)
+
+lhs = ''
+if ret_type:
+lhs = 'retval = '
+
 push_indent()
 ret = mcgen('''
-%(retval)sqmp_%(name)s(%(args)s&local_err);
+%(lhs)sqmp_%(c_name)s(%(args)s&local_err);
 ''',
-name=c_name(name), args=arglist, retval=retval)
+c_name=c_name(name), args=argstr, lhs=lhs)
 if ret_type:
 ret += gen_err_check('local_err')
 ret += mcgen('''
 
 qmp_marshal_output_%(c_name)s(retval, ret, &local_err);
 ''',
-c_name=c_name(name))
+ c_name=c_name(name))
 pop_indent()
 return ret
 
-def gen_visitor_input_containers_decl(args):
-ret = ""
+
+def gen_visitor_input_containers_decl(arg_type):
+ret = ''
 
 push_indent()
-if args:
+if arg_type:
 ret += mcgen('''
 QmpInputVisitor *mi = qmp_input_visitor_new_strict(QOBJECT(args));
 QapiDeallocVisitor *md;
@@ -80,17 +87,18 @@ Visitor *v;
 
 return ret
 
-def gen_visitor_input_vars_decl(args):
-ret = ""
+
+def gen_visitor_input_vars_decl(arg_type):
+ret = ''
 push_indent()
 
-if args:
-for memb in args.members:
+if arg_type:
+for memb in arg_type.members:
 if memb.optional:
 ret += mcgen('''
-bool has_%(argname)s = false;
+bool has_%(c_name)s = false;
 ''',
- argname=c_name(memb.name))
+ c_name=c_name(memb.name))
 ret += mcgen('''
 %(c_type)s %(c_name)s = %(c_null)s;
 ''',
@@ -101,19 +109,20 @@ bool has_%(argname)s = false;
 pop_indent()
 return ret
 
-def gen_visitor_input_block(args, dealloc=False):
-ret = ""
+
+def gen_visitor_input_block(arg_type, de

[Qemu-devel] [PATCH 3/4] target-i386: Add optional class name to kvm_default_props

[Eduardo Habkost]

This will allow us to define class-specific KVM defaults, instead of
defaults that apply to all CPU models.

If this table is starting to look like the global properties tables,
that's not a coincidence: in the future, we might convert this to
accelerator-specific code that simply register global properties when
the machine is initialized.

Signed-off-by: Eduardo Habkost 
---
 hw/i386/pc_piix.c |  8 
 hw/i386/pc_q35.c  |  4 ++--
 target-i386/cpu.c | 29 +
 target-i386/cpu.h |  3 ++-
 4 files changed, 25 insertions(+), 19 deletions(-)

diff --git a/hw/i386/pc_piix.c b/hw/i386/pc_piix.c
index e79ef0d..ad98735 100644
--- a/hw/i386/pc_piix.c
+++ b/hw/i386/pc_piix.c
@@ -325,7 +325,7 @@ static void pc_compat_2_1(MachineState *machine)
 
 pc_compat_2_2(machine);
 smbios_uuid_encoded = false;
-x86_cpu_change_kvm_default("svm", NULL);
+x86_cpu_change_kvm_default(NULL, "svm", NULL);
 pcms->enforce_aligned_dimm = false;
 }
 
@@ -361,7 +361,7 @@ static void pc_compat_1_7(MachineState *machine)
 gigabyte_align = false;
 option_rom_has_mr = true;
 legacy_acpi_table_size = 6414;
-x86_cpu_change_kvm_default("x2apic", NULL);
+x86_cpu_change_kvm_default(NULL, "x2apic", NULL);
 }
 
 static void pc_compat_1_6(MachineState *machine)
@@ -391,7 +391,7 @@ static void pc_compat_1_3(MachineState *machine)
 static void pc_compat_1_2(MachineState *machine)
 {
 pc_compat_1_3(machine);
-x86_cpu_change_kvm_default("kvm-pv-eoi", NULL);
+x86_cpu_change_kvm_default(NULL, "kvm-pv-eoi", NULL);
 }
 
 /* PC compat function for pc-0.10 to pc-0.13 */
@@ -414,7 +414,7 @@ static void pc_init_isa(MachineState *machine)
 if (!machine->cpu_model) {
 machine->cpu_model = "486";
 }
-x86_cpu_change_kvm_default("kvm-pv-eoi", NULL);
+x86_cpu_change_kvm_default(NULL, "kvm-pv-eoi", NULL);
 enable_compat_apic_id_mode();
 pc_init1(machine, TYPE_I440FX_PCI_HOST_BRIDGE, TYPE_I440FX_PCI_DEVICE);
 }
diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c
index c17d9df..94d417e 100644
--- a/hw/i386/pc_q35.c
+++ b/hw/i386/pc_q35.c
@@ -309,7 +309,7 @@ static void pc_compat_2_1(MachineState *machine)
 pc_compat_2_2(machine);
 pcms->enforce_aligned_dimm = false;
 smbios_uuid_encoded = false;
-x86_cpu_change_kvm_default("svm", NULL);
+x86_cpu_change_kvm_default(NULL, "svm", NULL);
 }
 
 static void pc_compat_2_0(MachineState *machine)
@@ -326,7 +326,7 @@ static void pc_compat_1_7(MachineState *machine)
 smbios_defaults = false;
 gigabyte_align = false;
 option_rom_has_mr = true;
-x86_cpu_change_kvm_default("x2apic", NULL);
+x86_cpu_change_kvm_default(NULL, "x2apic", NULL);
 }
 
 static void pc_compat_1_6(MachineState *machine)
diff --git a/target-i386/cpu.c b/target-i386/cpu.c
index 9a4bfa3..ec9eca2 100644
--- a/target-i386/cpu.c
+++ b/target-i386/cpu.c
@@ -1367,6 +1367,7 @@ static X86CPUDefinition builtin_x86_defs[] = {
 };
 
 typedef struct PropValue {
+const char *type;
 const char *prop, *value;
 } PropValue;
 
@@ -1374,24 +1375,25 @@ typedef struct PropValue {
  * from all CPU models when KVM is enabled.
  */
 static PropValue kvm_default_props[] = {
-{ "kvmclock", "on" },
-{ "kvm-nopiodelay", "on" },
-{ "kvm-asyncpf", "on" },
-{ "kvm-steal-time", "on" },
-{ "kvm-pv-eoi", "on" },
-{ "kvmclock-stable-bit", "on" },
-{ "x2apic", "on" },
-{ "acpi", "off" },
-{ "monitor", "off" },
-{ "svm", "off" },
+{ NULL, "kvmclock", "on" },
+{ NULL, "kvm-nopiodelay", "on" },
+{ NULL, "kvm-asyncpf", "on" },
+{ NULL, "kvm-steal-time", "on" },
+{ NULL, "kvm-pv-eoi", "on" },
+{ NULL, "kvmclock-stable-bit", "on" },
+{ NULL, "x2apic", "on" },
+{ NULL, "acpi", "off" },
+{ NULL, "monitor", "off" },
+{ NULL, "svm", "off" },
 { NULL, NULL },
 };
 
-void x86_cpu_change_kvm_default(const char *prop, const char *value)
+void x86_cpu_change_kvm_default(const char *type, const char *prop,
+const char *value)
 {
 PropValue *pv;
 for (pv = kvm_default_props; pv->prop; pv++) {
-if (!strcmp(pv->prop, prop)) {
+if (!g_strcmp0(pv->type, type) && !strcmp(pv->prop, prop)) {
 pv->value = value;
 break;
 }
@@ -2079,6 +2081,9 @@ static void x86_cpu_apply_props(X86CPU *cpu, PropValue 
*props)
 if (!pv->value) {
 continue;
 }
+if (pv->type && !object_dynamic_cast(OBJECT(cpu), pv->type)) {
+continue;
+}
 object_property_parse(OBJECT(cpu), pv->value, pv->prop,
   &error_abort);
 }
diff --git a/target-i386/cpu.h b/target-i386/cpu.h
index f7c733b..c77d455 100644
--- a/target-i386/cpu.h
+++ b/target-i386/cpu.h
@@ -1340,7 +1340,8 @@ void cpu_report_tpr_access(CPUX86State *env, TPRAccess 
access);
  * It is valid to call this funciton only for properties that
  * are already present in the kvm_de

[Qemu-devel] [PATCH v6 21/26] qapi: Introduce a first class 'any' type

[Markus Armbruster]

It's first class, because unlike '**', it actually works, i.e. doesn't
require 'gen': false.

'**' will go away next.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
---
 docs/qapi-code-gen.txt |  1 +
 include/qapi/visitor-impl.h|  2 +
 include/qapi/visitor.h |  1 +
 qapi/qapi-dealloc-visitor.c|  9 
 qapi/qapi-visit-core.c |  6 +++
 qapi/qmp-input-visitor.c   | 11 +
 qapi/qmp-output-visitor.c  |  9 
 scripts/qapi-types.py  |  1 +
 scripts/qapi.py|  9 ++--
 tests/Makefile |  3 +-
 tests/qapi-schema/args-any.err |  1 +
 .../{flat-union-base-star.exit => args-any.exit}   |  0
 tests/qapi-schema/args-any.json|  2 +
 .../{flat-union-base-star.out => args-any.out} |  0
 tests/qapi-schema/flat-union-base-any.err  |  1 +
 tests/qapi-schema/flat-union-base-any.exit |  1 +
 ...ion-base-star.json => flat-union-base-any.json} |  2 +-
 tests/qapi-schema/flat-union-base-any.out  |  0
 tests/qapi-schema/flat-union-base-star.err |  1 -
 tests/qapi-schema/qapi-schema-test.json|  5 +-
 tests/qapi-schema/qapi-schema-test.out |  9 +++-
 tests/qapi-schema/type-bypass.out  |  4 +-
 tests/test-qmp-commands.c  |  5 ++
 tests/test-qmp-input-visitor.c | 45 ++
 tests/test-qmp-output-visitor.c| 53 ++
 25 files changed, 171 insertions(+), 10 deletions(-)
 create mode 100644 tests/qapi-schema/args-any.err
 rename tests/qapi-schema/{flat-union-base-star.exit => args-any.exit} (100%)
 create mode 100644 tests/qapi-schema/args-any.json
 rename tests/qapi-schema/{flat-union-base-star.out => args-any.out} (100%)
 create mode 100644 tests/qapi-schema/flat-union-base-any.err
 create mode 100644 tests/qapi-schema/flat-union-base-any.exit
 rename tests/qapi-schema/{flat-union-base-star.json => 
flat-union-base-any.json} (95%)
 create mode 100644 tests/qapi-schema/flat-union-base-any.out
 delete mode 100644 tests/qapi-schema/flat-union-base-star.err

diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 35a4a0f..c713e3a 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -158,6 +158,7 @@ The following types are predefined, and map to C as follows:
   size  uint64_t   like uint64_t, except StringInputVisitor
accepts size suffixes
   bool  bool   JSON true or false
+  any   QObject *  any JSON value
 
 
 === Includes ===
diff --git a/include/qapi/visitor-impl.h b/include/qapi/visitor-impl.h
index f4a2f74..8c0ba57 100644
--- a/include/qapi/visitor-impl.h
+++ b/include/qapi/visitor-impl.h
@@ -40,6 +40,8 @@ struct Visitor
 void (*type_str)(Visitor *v, char **obj, const char *name, Error **errp);
 void (*type_number)(Visitor *v, double *obj, const char *name,
 Error **errp);
+void (*type_any)(Visitor *v, QObject **obj, const char *name,
+ Error **errp);
 
 /* May be NULL */
 void (*optional)(Visitor *v, bool *present, const char *name,
diff --git a/include/qapi/visitor.h b/include/qapi/visitor.h
index 00ba104..cfc19a6 100644
--- a/include/qapi/visitor.h
+++ b/include/qapi/visitor.h
@@ -58,6 +58,7 @@ void visit_type_size(Visitor *v, uint64_t *obj, const char 
*name, Error **errp);
 void visit_type_bool(Visitor *v, bool *obj, const char *name, Error **errp);
 void visit_type_str(Visitor *v, char **obj, const char *name, Error **errp);
 void visit_type_number(Visitor *v, double *obj, const char *name, Error 
**errp);
+void visit_type_any(Visitor *v, QObject **obj, const char *name, Error **errp);
 bool visit_start_union(Visitor *v, bool data_present, Error **errp);
 void visit_end_union(Visitor *v, bool data_present, Error **errp);
 
diff --git a/qapi/qapi-dealloc-visitor.c b/qapi/qapi-dealloc-visitor.c
index d7f92c5..737deab 100644
--- a/qapi/qapi-dealloc-visitor.c
+++ b/qapi/qapi-dealloc-visitor.c
@@ -151,6 +151,14 @@ static void qapi_dealloc_type_number(Visitor *v, double 
*obj, const char *name,
 {
 }
 
+static void qapi_dealloc_type_anything(Visitor *v, QObject **obj,
+   const char *name, Error **errp)
+{
+if (obj) {
+qobject_decref(*obj);
+}
+}
+
 static void qapi_dealloc_type_size(Visitor *v, uint64_t *obj, const char *name,
Error **errp)
 {
@@ -216,6 +224,7 @@ QapiDeallocVisitor *qapi_dealloc_visitor_new(void)
 v->visitor.type_bool = qapi_dealloc_type_bool;
 v->visitor.type_str = qapi_dealloc_type_str;
 v->visitor.type_number = qapi_dealloc_type_number;
+v->visitor.type_any = qapi_dealloc_type_anything;

[Qemu-devel] [PATCH v6 17/26] qapi: De-duplicate parameter list generation

[Markus Armbruster]

Generated qapi-event.[ch] lose line breaks.  No change otherwise.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
---
 scripts/qapi-commands.py | 11 ++-
 scripts/qapi-event.py| 18 +++---
 scripts/qapi.py  | 16 
 3 files changed, 21 insertions(+), 24 deletions(-)

diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index c68659a..833768e 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -17,19 +17,12 @@ import re
 
 
 def gen_command_decl(name, arg_type, ret_type):
-argstr = ''
-if arg_type:
-for memb in arg_type.members:
-if memb.optional:
-argstr += 'bool has_%s, ' % c_name(memb.name)
-argstr += '%s %s, ' % (memb.type.c_type(is_param=True),
-   c_name(memb.name))
 return mcgen('''
-%(c_type)s qmp_%(c_name)s(%(args)sError **errp);
+%(c_type)s qmp_%(c_name)s(%(params)s);
 ''',
  c_type=(ret_type and ret_type.c_type()) or 'void',
  c_name=c_name(name),
- args=argstr)
+ params=gen_params(arg_type, 'Error **errp'))
 
 
 def gen_err_check(err):
diff --git a/scripts/qapi-event.py b/scripts/qapi-event.py
index 5873a05..d15fad9 100644
--- a/scripts/qapi-event.py
+++ b/scripts/qapi-event.py
@@ -15,21 +15,9 @@ from qapi import *
 
 
 def gen_event_send_proto(name, arg_type):
-api_name = "void qapi_event_send_%s(" % c_name(name).lower()
-l = len(api_name)
-
-if arg_type:
-for m in arg_type.members:
-if m.optional:
-api_name += "bool has_%s,\n" % c_name(m.name)
-api_name += "".ljust(l)
-
-api_name += "%s %s,\n" % (m.type.c_type(is_param=True),
-  c_name(m.name))
-api_name += "".ljust(l)
-
-api_name += "Error **errp)"
-return api_name
+return 'void qapi_event_send_%(c_name)s(%(param)s)' % {
+'c_name': c_name(name.lower()),
+'param': gen_params(arg_type, 'Error **errp')}
 
 
 def gen_event_send_decl(name, arg_type):
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 6b11036..a4a5996 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -1456,6 +1456,22 @@ extern const char *const %(c_name)s_lookup[];
  c_name=c_name(name))
 return ret
 
+def gen_params(arg_type, extra):
+if not arg_type:
+return extra
+assert not arg_type.variants
+ret = ''
+sep = ''
+for memb in arg_type.members:
+ret += sep
+sep = ', '
+if memb.optional:
+ret += 'bool has_%s, ' % c_name(memb.name)
+ret += '%s %s' % (memb.type.c_type(is_param=True), c_name(memb.name))
+if extra:
+ret += sep + extra
+return ret
+
 #
 # Common command line parsing
 #
-- 
2.4.3

[Qemu-devel] [PATCH v6 26/26] qapi-introspect: Hide type names

[Markus Armbruster]

To eliminate the temptation for clients to look up types by name
(which are not ABI), replace all type names by meaningless strings.

Reduces output of query-schema by 13 out of 85KiB.

As a debugging aid, provide option -u to suppress the hiding.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
---
 docs/qapi-code-gen.txt | 36 +---
 qapi/introspect.json   | 12 
 scripts/qapi-introspect.py | 41 +++--
 3 files changed, 56 insertions(+), 33 deletions(-)

diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 458b217..01e292b 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -522,13 +522,16 @@ additional variant members depending on the value of 
meta-type.
 Each SchemaInfo object describes a wire ABI entity of a certain
 meta-type: a command, event or one of several kinds of type.
 
-SchemaInfo for entities defined in the QAPI schema have the same name
-as in the schema.  This is the case for all commands and events, and
-most types.
+SchemaInfo for commands and events have the same name as in the QAPI
+schema.
 
 Command and event names are part of the wire ABI, but type names are
-not.  Therefore, looking up a type by its name in the QAPI schema is
-wrong.  Look up the command or event, then follow references by name.
+not.  Therefore, the SchemaInfo for types have auto-generated
+meaningless names.  For readability, the examples in this section use
+meaningful type names instead.
+
+To examine a type, start with a command or event using it, then follow
+references by name.
 
 QAPI schema definitions not reachable that way are omitted.
 
@@ -559,8 +562,7 @@ object type without members.  The event may not have a data 
member on
 the wire then.
 
 Each command or event defined with dictionary-valued 'data' in the
-QAPI schema implicitly defines an object type called ":obj-NAME-arg",
-where NAME is the command or event's name.
+QAPI schema implicitly defines an object type.
 
 Example: the SchemaInfo for EVENT_C from section Events
 
@@ -615,12 +617,9 @@ Note that base types are "flattened": its members are 
included in the
 
 A simple union implicitly defines an enumeration type for its implicit
 discriminator (called "type" on the wire, see section Union types).
-Such a type's name is made by appending "Kind" to the simple union's
-name.
 
 A simple union implicitly defines an object type for each of its
-variants.  The type's name is ":obj-NAME-wrapper", where NAME is the
-name of the name of the variant's type.
+variants.
 
 Example: the SchemaInfo for simple union BlockdevOptions from section
 Union types
@@ -651,8 +650,7 @@ Example: the SchemaInfo for BlockRef from section Alternate 
types
 
 The SchemaInfo for an array type has meta-type "array", and variant
 member "element-type", which names the array's element type.  Array
-types are implicitly defined.  An array type's name is made by
-appending "List" to its element type's name.
+types are implicitly defined.
 
 Example: the SchemaInfo for ['str']
 
@@ -1059,13 +1057,13 @@ Example:
 [Uninteresting stuff omitted...]
 
 const char example_qmp_schema_json[] = "["
-"{\"arg-type\": \":empty\", \"meta-type\": \"event\", \"name\": 
\"MY_EVENT\"}, "
+"{\"arg-type\": \"0\", \"meta-type\": \"event\", \"name\": 
\"MY_EVENT\"}, "
+"{\"arg-type\": \"1\", \"meta-type\": \"command\", \"name\": 
\"my-command\", \"ret-type\": \"2\"}, "
+"{\"members\": [], \"meta-type\": \"object\", \"name\": \"0\"}, "
+"{\"members\": [{\"name\": \"arg1\", \"type\": \"2\"}], \"meta-type\": 
\"object\", \"name\": \"1\"}, "
+"{\"members\": [{\"name\": \"integer\", \"type\": \"int\"}, {\"name\": 
\"string\", \"type\": \"str\"}], \"meta-type\": \"object\", \"name\": \"2\"}, "
 "{\"json-type\": \"int\", \"meta-type\": \"builtin\", \"name\": 
\"int\"}, "
-"{\"json-type\": \"string\", \"meta-type\": \"builtin\", \"name\": 
\"str\"}, "
-"{\"members\": [], \"meta-type\": \"object\", \"name\": \":empty\"}, "
-"{\"members\": [{\"name\": \"arg1\", \"type\": \"UserDefOne\"}], 
\"meta-type\": \"object\", \"name\": \":obj-my-command-arg\"}, "
-"{\"members\": [{\"name\": \"integer\", \"type\": \"int\"}, {\"name\": 
\"string\", \"type\": \"str\"}], \"meta-type\": \"object\", \"name\": 
\"UserDefOne\"}, "
-"{\"arg-type\": \":obj-my-command-arg\", \"meta-type\": \"command\", 
\"name\": \"my-command\", \"ret-type\": \"UserDefOne\"}]";
+"{\"json-type\": \"string\", \"meta-type\": \"builtin\", \"name\": 
\"str\"}]";
 $ cat qapi-generated/example-qmp-introspect.h
 [Uninteresting stuff omitted...]
 
diff --git a/qapi/introspect.json b/qapi/introspect.json
index 9c8ad53..cc50dc6 100644
--- a/qapi/introspect.json
+++ b/qapi/introspect.json
@@ -75,17 +75,13 @@
 # @SchemaInfo
 #
 # @name: the entity's name, inherited from @base.
-# Entities defined in the QAPI sche

[Qemu-devel] [PATCH v6 25/26] qapi: New QMP command query-qmp-schema for QMP introspection

[Markus Armbruster]

qapi/introspect.json defines the introspection schema.  It's designed
for QMP introspection, but should do for similar uses, such as QGA.

The introspection schema does not reflect all the rules and
restrictions that apply to QAPI schemata.  A valid QAPI schema has an
introspection value conforming to the introspection schema, but the
converse is not true.

Introspection lowers away a number of schema details, and makes
implicit things explicit:

* The built-in types are declared with their JSON type.

  All integer types are mapped to 'int', because how many bits we use
  internally is an implementation detail.  It could be pressed into
  external interface service as very approximate range information,
  but that's a bad idea.  If we need range information, we better do
  it properly.

* Implicit type definitions are made explicit, and given
  auto-generated names:

  - Array types, named by appending "List" to the name of their
element type, like in generated C.

  - The enumeration types implicitly defined by simple union types,
named by appending "Kind" to the name of their simple union type,
like in generated C.

  - Types that don't occur in generated C.  Their names start with ':'
so they don't clash with the user's names.

* All type references are by name.

* The struct and union types are generalized into an object type.

* Base types are flattened.

* Commands take a single argument and return a single result.

  Dictionary argument or list result is an implicit type definition.

  The empty object type is used when a command takes no arguments or
  produces no results.

  The argument is always of object type, but the introspection schema
  doesn't reflect that.

  The 'gen': false directive is omitted as implementation detail.

  The 'success-response' directive is omitted as well for now, even
  though it's not an implementation detail, because it's not used by
  QMP.

* Events carry a single data value.

  Implicit type definition and empty object type use, just like for
  commands.

  The value is of object type, but the introspection schema doesn't
  reflect that.

* Types not used by commands or events are omitted.

  Indirect use counts as use.

* Optional members have a default, which can only be null right now

  Instead of a mandatory "optional" flag, we have an optional default.
  No default means mandatory, default null means optional without
  default value.  Non-null is available for optional with default
  (possible future extension).

* Clients should *not* look up types by name, because type names are
  not ABI.  Look up the command or event you're interested in, then
  follow the references.

  TODO Should we hide the type names to eliminate the temptation?

New generator scripts/qapi-introspect.py computes an introspection
value for its input, and generates a C variable holding it.

It can generate awfully long lines.  Marked TODO.

A new test-qmp-input-visitor test case feeds its result for both
tests/qapi-schema/qapi-schema-test.json and qapi-schema.json to a
QmpInputVisitor to verify it actually conforms to the schema.

New QMP command query-qmp-schema takes its return value from that
variable.  Its reply is some 85KiBytes for me right now.

If this turns out to be too much, we have a couple of options:

* We can use shorter names in the JSON.  Not the QMP style.

* Optionally return the sub-schema for commands and events given as
  arguments.

  Right now qmp_query_schema() sends the string literal computed by
  qmp-introspect.py.  To compute sub-schema at run time, we'd have to
  duplicate parts of qapi-introspect.py in C.  Unattractive.

* Let clients cache the output of query-qmp-schema.

  It changes only on QEMU upgrades, i.e. rarely.  Provide a command
  query-qmp-schema-hash.  Clients can have a cache indexed by hash,
  and re-query the schema only when they don't have it cached.  Even
  simpler: put the hash in the QMP greeting.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
---
 .gitignore  |   1 +
 Makefile|   9 +-
 Makefile.objs   |   4 +-
 docs/qapi-code-gen.txt  | 237 +++-
 monitor.c   |  16 ++
 qapi-schema.json|   3 +
 qapi/introspect.json| 277 
 qmp-commands.hx |  17 ++
 scripts/qapi-introspect.py  | 184 
 scripts/qapi.py |  13 +-
 tests/.gitignore|   1 +
 tests/Makefile  |  10 +-
 tests/qapi-schema/alternate-good.out|   1 +
 tests/qapi-schema/args-member-array.out |   1 +
 tests/qapi-schema/comments.out  |   1 +
 tests/qapi-schema/empty.out |   1 +

[Qemu-devel] [PATCH v6 11/26] qapi-event: Convert to QAPISchemaVisitor, fixing data with base

[Markus Armbruster]

Fixes events whose data is struct with base to include the struct's
base members.  Test case is qapi-schema-test.json's event
__org.qemu_x-command:

{ 'event': '__ORG.QEMU_X-EVENT', 'data': '__org.qemu_x-Struct' }

{ 'struct': '__org.qemu_x-Struct', 'base': '__org.qemu_x-Base',
  'data': { '__org.qemu_x-member2': 'str' } }

{ 'struct': '__org.qemu_x-Base',
  'data': { '__org.qemu_x-member1': '__org.qemu_x-Enum' } }

Patch's effect on generated qapi_event_send___org_qemu_x_event():

-void qapi_event_send___org_qemu_x_event(const char *__org_qemu_x_member2,
+void qapi_event_send___org_qemu_x_event(__org_qemu_x_Enum 
__org_qemu_x_member1,
+const char *__org_qemu_x_member2,
 Error **errp)
 {
 QDict *qmp;
@@ -224,6 +225,10 @@ void qapi_event_send___org_qemu_x_event(
 goto clean;
 }

+visit_type___org_qemu_x_Enum(v, &__org_qemu_x_member1, 
"__org.qemu_x-member1", &local_err);
+if (local_err) {
+goto clean;
+}
 visit_type_str(v, (char **)&__org_qemu_x_member2, 
"__org.qemu_x-member2", &local_err);
 if (local_err) {
 goto clean;

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
---
 scripts/qapi-event.py   | 89 +
 tests/qapi-schema/qapi-schema-test.json |  3 --
 2 files changed, 47 insertions(+), 45 deletions(-)

diff --git a/scripts/qapi-event.py b/scripts/qapi-event.py
index 537da17..d19acda 100644
--- a/scripts/qapi-event.py
+++ b/scripts/qapi-event.py
@@ -2,14 +2,15 @@
 # QAPI event generator
 #
 # Copyright (c) 2014 Wenchao Xia
+# Copyright (c) 2015 Red Hat Inc.
 #
 # Authors:
 #  Wenchao Xia 
+#  Markus Armbruster 
 #
 # This work is licensed under the terms of the GNU GPL, version 2.
 # See the COPYING file in the top-level directory.
 
-from ordereddict import OrderedDict
 from qapi import *
 
 def _generate_event_api_name(event_name, params):
@@ -17,13 +18,13 @@ def _generate_event_api_name(event_name, params):
 l = len(api_name)
 
 if params:
-for argname, argentry, optional in parse_args(params):
-if optional:
-api_name += "bool has_%s,\n" % c_name(argname)
+for m in params.members:
+if m.optional:
+api_name += "bool has_%s,\n" % c_name(m.name)
 api_name += "".ljust(l)
 
-api_name += "%s %s,\n" % (c_type(argentry, is_param=True),
-  c_name(argname))
+api_name += "%s %s,\n" % (m.type.c_type(is_param=True),
+  c_name(m.name))
 api_name += "".ljust(l)
 
 api_name += "Error **errp)"
@@ -51,7 +52,7 @@ def generate_event_implement(api_name, event_name, params):
 """,
 api_name = api_name)
 
-if params:
+if params and params.members:
 ret += mcgen("""
 QmpOutputVisitor *qov;
 Visitor *v;
@@ -72,7 +73,7 @@ def generate_event_implement(api_name, event_name, params):
  event_name = event_name)
 
 # step 3: visit the params if params != None
-if params:
+if params and params.members:
 ret += mcgen("""
 qov = qmp_output_visitor_new();
 g_assert(qov);
@@ -89,15 +90,15 @@ def generate_event_implement(api_name, event_name, params):
 """,
 event_name = event_name)
 
-for argname, argentry, optional in parse_args(params):
-if optional:
+for memb in params.members:
+if memb.optional:
 ret += mcgen("""
 if (has_%(var)s) {
 """,
- var = c_name(argname))
+ var=c_name(memb.name))
 push_indent()
 
-if argentry == "str":
+if memb.type.name == "str":
 var_type = "(char **)"
 else:
 var_type = ""
@@ -109,11 +110,11 @@ def generate_event_implement(api_name, event_name, 
params):
 }
 """,
  var_type = var_type,
- var = c_name(argname),
- type = type_name(argentry),
- name = argname)
+ var=c_name(memb.name),
+ type=memb.type.c_name(),
+ name=memb.name)
 
-if optional:
+if memb.optional:
 pop_indent()
 ret += mcgen("""
 }
@@ -140,7 +141,7 @@ def generate_event_implement(api_name, event_name, params):
  event_enum_value = c_enum_const(event_enum_name, event_name))
 
 # step 5: clean up
-if params:
+if params and params.members:
 ret += mcgen("""
  clean:
 qmp_output_visitor_cleanup(qov);
@@ -153,6 +154,30 @@ def generate_event_implement(api_name, event_name, params):
 
 return ret
 
+
+class QAPI

Re: [Qemu-devel] [PATCH] q35: Remove old machine versions

[Markus Armbruster]

Eduardo Habkost  writes:

> Ping?
>
> So, what's the reason we are still keeping those old machines in the
> code?

Inertia?

[Qemu-devel] [PATCH v6 16/26] qapi: Rename qmp_marshal_input_FOO() to qmp_marshal_FOO()

[Markus Armbruster]

These functions marshal both input and output.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
---
 docs/qapi-code-gen.txt|   4 +-
 docs/writing-qmp-commands.txt |   8 +-
 monitor.c |   2 +-
 qmp-commands.hx   | 242 +-
 qmp.c |   6 +-
 scripts/qapi-commands.py  |   4 +-
 6 files changed, 133 insertions(+), 133 deletions(-)

diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index e85f113..09d15dc 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -736,7 +736,7 @@ Example:
 qapi_dealloc_visitor_cleanup(md);
 }
 
-static void qmp_marshal_input_my_command(QDict *args, QObject **ret, Error 
**errp)
+static void qmp_marshal_my_command(QDict *args, QObject **ret, Error 
**errp)
 {
 Error *local_err = NULL;
 UserDefOne *retval;
@@ -769,7 +769,7 @@ Example:
 
 static void qmp_init_marshal(void)
 {
-qmp_register_command("my-command", qmp_marshal_input_my_command, 
QCO_NO_OPTIONS);
+qmp_register_command("my-command", qmp_marshal_my_command, 
QCO_NO_OPTIONS);
 }
 
 qapi_init(qmp_init_marshal);
diff --git a/docs/writing-qmp-commands.txt b/docs/writing-qmp-commands.txt
index f7693ca..8647cac 100644
--- a/docs/writing-qmp-commands.txt
+++ b/docs/writing-qmp-commands.txt
@@ -127,7 +127,7 @@ following at the bottom:
 {
 .name   = "hello-world",
 .args_type  = "",
-.mhandler.cmd_new = qmp_marshal_input_hello_world,
+.mhandler.cmd_new = qmp_marshal_hello_world,
 },
 
 You're done. Now build qemu, run it as suggested in the "Testing" section,
@@ -179,7 +179,7 @@ The last step is to update the qmp-commands.hx file:
 {
 .name   = "hello-world",
 .args_type  = "message:s?",
-.mhandler.cmd_new = qmp_marshal_input_hello_world,
+.mhandler.cmd_new = qmp_marshal_hello_world,
 },
 
 Notice that the "args_type" member got our "message" argument. The character
@@ -461,7 +461,7 @@ The last step is to add the correspoding entry in the 
qmp-commands.hx file:
 {
 .name   = "query-alarm-clock",
 .args_type  = "",
-.mhandler.cmd_new = qmp_marshal_input_query_alarm_clock,
+.mhandler.cmd_new = qmp_marshal_query_alarm_clock,
 },
 
 Time to test the new command. Build qemu, run it as described in the "Testing"
@@ -607,7 +607,7 @@ To test this you have to add the corresponding 
qmp-commands.hx entry:
 {
 .name   = "query-alarm-methods",
 .args_type  = "",
-.mhandler.cmd_new = qmp_marshal_input_query_alarm_methods,
+.mhandler.cmd_new = qmp_marshal_query_alarm_methods,
 },
 
 Now Build qemu, run it as explained in the "Testing" section and try our new
diff --git a/monitor.c b/monitor.c
index 5455ab9..e083716 100644
--- a/monitor.c
+++ b/monitor.c
@@ -5142,7 +5142,7 @@ static QObject *get_qmp_greeting(void)
 {
 QObject *ver = NULL;
 
-qmp_marshal_input_query_version(NULL, &ver, NULL);
+qmp_marshal_query_version(NULL, &ver, NULL);
 return qobject_from_jsonf("{'QMP':{'version': %p,'capabilities': 
[]}}",ver);
 }
 
diff --git a/qmp-commands.hx b/qmp-commands.hx
index 9848fd8..4640a3d 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -63,7 +63,7 @@ EQMP
 {
 .name   = "quit",
 .args_type  = "",
-.mhandler.cmd_new = qmp_marshal_input_quit,
+.mhandler.cmd_new = qmp_marshal_quit,
 },
 
 SQMP
@@ -84,7 +84,7 @@ EQMP
 {
 .name   = "eject",
 .args_type  = "force:-f,device:B",
-.mhandler.cmd_new = qmp_marshal_input_eject,
+.mhandler.cmd_new = qmp_marshal_eject,
 },
 
 SQMP
@@ -110,7 +110,7 @@ EQMP
 {
 .name   = "change",
 .args_type  = "device:B,target:F,arg:s?",
-.mhandler.cmd_new = qmp_marshal_input_change,
+.mhandler.cmd_new = qmp_marshal_change,
 },
 
 SQMP
@@ -146,7 +146,7 @@ EQMP
 {
 .name   = "screendump",
 .args_type  = "filename:F",
-.mhandler.cmd_new = qmp_marshal_input_screendump,
+.mhandler.cmd_new = qmp_marshal_screendump,
 },
 
 SQMP
@@ -169,7 +169,7 @@ EQMP
 {
 .name   = "stop",
 .args_type  = "",
-.mhandler.cmd_new = qmp_marshal_input_stop,
+.mhandler.cmd_new = qmp_marshal_stop,
 },
 
 SQMP
@@ -190,7 +190,7 @@ EQMP
 {
 .name   = "cont",
 .args_type  = "",
-.mhandler.cmd_new = qmp_marshal_input_cont,
+.mhandler.cmd_new = qmp_marshal_cont,
 },
 
 SQMP
@@ -211,7 +211,7 @@ EQMP
 {
 .name   = "system_wakeup",
 .args_type  = "",
-.mhandler.cmd_new = qmp_marshal_input_system_wakeup,
+.mhandler.cmd_new = qmp_marshal_system_wakeup,
 },
 
 SQMP
@@ -232,7 +232,7 @@ EQMP
 {
 .name   = "system_reset",
 .args_type  = "",
- 

[Qemu-devel] [PATCH v6 08/26] qapi-commands: Convert to QAPISchemaVisitor

[Markus Armbruster]

Output unchanged apart from reordering and white-space.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
---
 scripts/qapi-commands.py | 159 +++
 scripts/qapi.py  |   2 +-
 2 files changed, 91 insertions(+), 70 deletions(-)

diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index 12bdc4c..a68517a 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -12,21 +12,22 @@
 # This work is licensed under the terms of the GNU GPL, version 2.
 # See the COPYING file in the top-level directory.
 
-from ordereddict import OrderedDict
 from qapi import *
 import re
 
 def generate_command_decl(name, args, ret_type):
 arglist=""
-for argname, argtype, optional in parse_args(args):
-argtype = c_type(argtype, is_param=True)
-if optional:
-arglist += "bool has_%s, " % c_name(argname)
-arglist += "%s %s, " % (argtype, c_name(argname))
+if args:
+for memb in args.members:
+argtype = memb.type.c_type(is_param=True)
+if memb.optional:
+arglist += "bool has_%s, " % c_name(memb.name)
+arglist += "%s %s, " % (argtype, c_name(memb.name))
 return mcgen('''
 %(ret_type)s qmp_%(name)s(%(args)sError **errp);
 ''',
- ret_type=c_type(ret_type), name=c_name(name),
+ ret_type=(ret_type and ret_type.c_type()) or 'void',
+ name=c_name(name),
  args=arglist)
 
 def gen_err_check(err):
@@ -45,10 +46,11 @@ def gen_sync_call(name, args, ret_type):
 retval=""
 if ret_type:
 retval = "retval = "
-for argname, argtype, optional in parse_args(args):
-if optional:
-arglist += "has_%s, " % c_name(argname)
-arglist += "%s, " % (c_name(argname))
+if args:
+for memb in args.members:
+if memb.optional:
+arglist += "has_%s, " % c_name(memb.name)
+arglist += "%s, " % c_name(memb.name)
 push_indent()
 ret = mcgen('''
 %(retval)sqmp_%(name)s(%(args)s&local_err);
@@ -68,7 +70,7 @@ def gen_visitor_input_containers_decl(args):
 ret = ""
 
 push_indent()
-if len(args) > 0:
+if args:
 ret += mcgen('''
 QmpInputVisitor *mi = qmp_input_visitor_new_strict(QOBJECT(args));
 QapiDeallocVisitor *md;
@@ -81,22 +83,26 @@ Visitor *v;
 def gen_visitor_input_vars_decl(args):
 ret = ""
 push_indent()
-for argname, argtype, optional in parse_args(args):
-if optional:
-ret += mcgen('''
+
+if args:
+for memb in args.members:
+if memb.optional:
+ret += mcgen('''
 bool has_%(argname)s = false;
 ''',
- argname=c_name(argname))
-if is_c_ptr(argtype):
-ret += mcgen('''
+ argname=c_name(memb.name))
+if is_c_ptr(memb.type.c_type()):
+ret += mcgen('''
 %(argtype)s %(argname)s = NULL;
 ''',
- argname=c_name(argname), argtype=c_type(argtype))
-else:
-ret += mcgen('''
+ argname=c_name(memb.name),
+ argtype=memb.type.c_type())
+else:
+ret += mcgen('''
 %(argtype)s %(argname)s = {0};
 ''',
- argname=c_name(argname), argtype=c_type(argtype))
+ argname=c_name(memb.name),
+ argtype=memb.type.c_type())
 
 pop_indent()
 return ret
@@ -106,7 +112,7 @@ def gen_visitor_input_block(args, dealloc=False):
 errparg = '&local_err'
 errarg = 'local_err'
 
-if len(args) == 0:
+if not args:
 return ret
 
 push_indent()
@@ -124,25 +130,26 @@ v = qapi_dealloc_get_visitor(md);
 v = qmp_input_get_visitor(mi);
 ''')
 
-for argname, argtype, optional in parse_args(args):
-if optional:
+for memb in args.members:
+if memb.optional:
 ret += mcgen('''
 visit_optional(v, &has_%(c_name)s, "%(name)s", %(errp)s);
 ''',
- c_name=c_name(argname), name=argname, errp=errparg)
+ c_name=c_name(memb.name), name=memb.name,
+ errp=errparg)
 ret += gen_err_check(errarg)
 ret += mcgen('''
 if (has_%(c_name)s) {
 ''',
- c_name=c_name(argname))
+ c_name=c_name(memb.name))
 push_indent()
 ret += mcgen('''
 visit_type_%(visitor)s(v, &%(c_name)s, "%(name)s", %(errp)s);
 ''',
- c_name=c_name(argname), name=argname, argtype=argtype,
- visitor=type_name(argtype), errp=errparg)
+ c_name=c_name(memb.name), name=memb.name,
+ visitor=memb.type.c_name(), errp=errparg)
 ret += gen_err_check(errarg)
-if optional:
+if memb.optional:

[Qemu-devel] [PATCH v6 15/26] qapi-commands: Rearrange code

[Markus Armbruster]

Rename gen_marshal_input() to gen_marshal(), because the generated
function marshals both arguments and results.

Rename gen_visitor_input_containers_decl() to gen_marshal_vars(), and
move the other variable declarations there, too.

Rename gen_visitor_input_block() to gen_marshal_input_visit(), and
rearrange its code slightly.

Rename gen_marshal_input_decl() to gen_marshal_proto(), because the
result isn't a full declaration, unlike gen_command_decl()'s.

New gen_marshal_decl() actually returns a full declaration.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
---
 scripts/qapi-commands.py | 87 ++--
 1 file changed, 39 insertions(+), 48 deletions(-)

diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index 0501582..6a52c88 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -59,6 +59,7 @@ def gen_call(name, arg_type, ret_type):
 
 push_indent()
 ret = mcgen('''
+
 %(lhs)sqmp_%(c_name)s(%(args)s&local_err);
 ''',
 c_name=c_name(name), args=argstr, lhs=lhs)
@@ -73,26 +74,26 @@ qmp_marshal_output_%(c_name)s(retval, ret, &local_err);
 return ret
 
 
-def gen_visitor_input_containers_decl(arg_type):
-ret = ''
+def gen_marshal_vars(arg_type, ret_type):
+ret = mcgen('''
+Error *local_err = NULL;
+''')
 
 push_indent()
+
+if ret_type:
+ret += mcgen('''
+%(c_type)s retval;
+''',
+ c_type=ret_type.c_type())
+
 if arg_type:
 ret += mcgen('''
 QmpInputVisitor *mi = qmp_input_visitor_new_strict(QOBJECT(args));
 QapiDeallocVisitor *md;
 Visitor *v;
 ''')
-pop_indent()
 
-return ret
-
-
-def gen_visitor_input_vars_decl(arg_type):
-ret = ''
-push_indent()
-
-if arg_type:
 for memb in arg_type.members:
 if memb.optional:
 ret += mcgen('''
@@ -105,15 +106,19 @@ bool has_%(c_name)s = false;
  c_name=c_name(memb.name),
  c_type=memb.type.c_type(),
  c_null=memb.type.c_null())
+ret += '\n'
+else:
+ret += mcgen('''
+
+(void)args;
+''')
 
 pop_indent()
 return ret
 
 
-def gen_visitor_input_block(arg_type, dealloc=False):
+def gen_marshal_input_visit(arg_type, dealloc=False):
 ret = ''
-errparg = '&local_err'
-errarg = 'local_err'
 
 if not arg_type:
 return ret
@@ -129,6 +134,8 @@ md = qapi_dealloc_visitor_new();
 v = qapi_dealloc_get_visitor(md);
 ''')
 else:
+errparg = '&local_err'
+errarg = 'local_err'
 ret += mcgen('''
 v = qmp_input_get_visitor(mi);
 ''')
@@ -167,10 +174,7 @@ qapi_dealloc_visitor_cleanup(md);
 
 
 def gen_marshal_output(name, ret_type):
-if not ret_type:
-return ''
-
-ret = mcgen('''
+return mcgen('''
 
 static void qmp_marshal_output_%(c_cmd_name)s(%(c_type)s ret_in, QObject 
**ret_out, Error **errp)
 {
@@ -195,47 +199,34 @@ out:
 qapi_dealloc_visitor_cleanup(md);
 }
 ''',
-c_type=ret_type.c_type(), c_cmd_name=c_name(name),
-c_name=ret_type.c_name())
+ c_type=ret_type.c_type(), c_cmd_name=c_name(name),
+ c_name=ret_type.c_name())
 
-return ret
 
-
-def gen_marshal_input_decl(name):
+def gen_marshal_proto(name):
 ret = 'void qmp_marshal_input_%s(QDict *args, QObject **ret, Error 
**errp)' % c_name(name)
 if not middle_mode:
 ret = 'static ' + ret
 return ret
 
 
-def gen_marshal_input(name, arg_type, ret_type):
-hdr = gen_marshal_input_decl(name)
+def gen_marshal_decl(name):
+return mcgen('''
+%(proto)s;
+''',
+ proto=gen_marshal_proto(name))
 
+
+def gen_marshal(name, arg_type, ret_type):
 ret = mcgen('''
 
-%(header)s
+%(proto)s
 {
-Error *local_err = NULL;
 ''',
-header=hdr)
-
-if ret_type:
-ret += mcgen('''
-%(c_type)s retval;
-''',
- c_type=ret_type.c_type())
-
-if arg_type:
-ret += gen_visitor_input_containers_decl(arg_type)
-ret += gen_visitor_input_vars_decl(arg_type) + '\n'
-ret += gen_visitor_input_block(arg_type) + '\n'
-else:
-ret += mcgen('''
-
-(void)args;
-
-''')
+proto=gen_marshal_proto(name))
 
+ret += gen_marshal_vars(arg_type, ret_type)
+ret += gen_marshal_input_visit(arg_type)
 ret += gen_call(name, arg_type, ret_type)
 
 if re.search('^ *goto out;', ret, re.MULTILINE):
@@ -246,7 +237,7 @@ out:
 ret += mcgen('''
 error_propagate(errp, local_err);
 ''')
-ret += gen_visitor_input_block(arg_type, dealloc=True)
+ret += gen_marshal_input_visit(arg_type, dealloc=True)
 ret += mcgen('''
 }
 ''')
@@ -307,8 +298,8 @@ class QAPISchemaGenCommandVisitor(QAPISchemaVisitor):
 if ret_type:
 self.defn += gen_marshal_output(name, ret_type)
 if middle_mode:
-self.decl += gen_marshal_input_decl(nam

[Qemu-devel] [PATCH v6 20/26] qapi: Fix output visitor to return qnull() instead of NULL

[Markus Armbruster]

Before commit 1d10b44, it crashed.  Since then, it returns NULL, with
a FIXME comment.  The FIXME is valid: code that assumes QObject *
can't be null exists.  I'm not aware of a way to feed this problematic
return value to code that actually chokes on null in the current code,
but the next few commits will create one.

Commit 481b002 solved a very similar problem by introducing a special
null QObject.  Use it here, and drop the FIXME.

Update the test accordingly.

Signed-off-by: Markus Armbruster 
---
 qapi/qmp-output-visitor.c   | 3 +--
 tests/test-qmp-output-visitor.c | 2 +-
 2 files changed, 2 insertions(+), 3 deletions(-)

diff --git a/qapi/qmp-output-visitor.c b/qapi/qmp-output-visitor.c
index efc19d5..2d6083e 100644
--- a/qapi/qmp-output-visitor.c
+++ b/qapi/qmp-output-visitor.c
@@ -66,9 +66,8 @@ static QObject *qmp_output_first(QmpOutputVisitor *qov)
 {
 QStackEntry *e = QTAILQ_LAST(&qov->stack, QStack);
 
-/* FIXME - find a better way to deal with NULL values */
 if (!e) {
-return NULL;
+return qnull();
 }
 
 return e->value;
diff --git a/tests/test-qmp-output-visitor.c b/tests/test-qmp-output-visitor.c
index 338ada0..256bffd 100644
--- a/tests/test-qmp-output-visitor.c
+++ b/tests/test-qmp-output-visitor.c
@@ -485,7 +485,7 @@ static void test_visitor_out_empty(TestOutputVisitorData 
*data,
 QObject *arg;
 
 arg = qmp_output_get_qobject(data->qov);
-g_assert(!arg);
+g_assert(qobject_type(arg) == QTYPE_QNULL);
 }
 
 static void init_native_list(UserDefNativeListUnion *cvalue)
-- 
2.4.3

[Qemu-devel] [PATCH v6 05/26] tests/qapi-schema: Convert test harness to QAPISchemaVisitor

[Markus Armbruster]

The old code prints the result of parsing (list of expression
dictionaries), and partial results of semantic analysis (list of enum
dictionaries, list of struct dictionaries).

The new code prints a trace of a schema visit, i.e. what the back-ends
are going to use.  Built-in and array types are omitted, because
they're boring.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
Reviewed-by: Daniel P. Berrange 
---
 tests/qapi-schema/alternate-good.out|  15 +-
 tests/qapi-schema/args-member-array.out |  13 +-
 tests/qapi-schema/comments.out  |   4 +-
 tests/qapi-schema/empty.out |   3 -
 tests/qapi-schema/enum-empty.out|   4 +-
 tests/qapi-schema/event-case.out|   4 +-
 tests/qapi-schema/flat-union-reverse-define.out |  21 ++-
 tests/qapi-schema/ident-with-escape.out |   7 +-
 tests/qapi-schema/include-relpath.out   |   4 +-
 tests/qapi-schema/include-repetition.out|   4 +-
 tests/qapi-schema/include-simple.out|   4 +-
 tests/qapi-schema/indented-expr.out |   7 +-
 tests/qapi-schema/qapi-schema-test.out  | 214 ++--
 tests/qapi-schema/returns-int.out   |   5 +-
 tests/qapi-schema/test-qapi.py  |  45 -
 tests/qapi-schema/type-bypass.out   |   7 +-
 16 files changed, 245 insertions(+), 116 deletions(-)

diff --git a/tests/qapi-schema/alternate-good.out 
b/tests/qapi-schema/alternate-good.out
index 99848ee..3d765ff 100644
--- a/tests/qapi-schema/alternate-good.out
+++ b/tests/qapi-schema/alternate-good.out
@@ -1,6 +1,9 @@
-[OrderedDict([('struct', 'Data'), ('data', OrderedDict([('*number', 'int'), 
('*name', 'str')]))]),
- OrderedDict([('enum', 'Enum'), ('data', ['hello', 'world'])]),
- OrderedDict([('alternate', 'Alt'), ('data', OrderedDict([('value', 'int'), 
('string', 'Enum'), ('struct', 'Data')]))])]
-[{'enum_name': 'Enum', 'enum_values': ['hello', 'world']},
- {'enum_name': 'AltKind', 'enum_values': None}]
-[OrderedDict([('struct', 'Data'), ('data', OrderedDict([('*number', 'int'), 
('*name', 'str')]))])]
+alternate Alt
+case value: int
+case string: Enum
+case struct: Data
+enum AltKind ['value', 'string', 'struct']
+object Data
+member number: int optional=True
+member name: str optional=True
+enum Enum ['hello', 'world']
diff --git a/tests/qapi-schema/args-member-array.out 
b/tests/qapi-schema/args-member-array.out
index c39fa25..b67384c 100644
--- a/tests/qapi-schema/args-member-array.out
+++ b/tests/qapi-schema/args-member-array.out
@@ -1,5 +1,8 @@
-[OrderedDict([('enum', 'abc'), ('data', ['a', 'b', 'c'])]),
- OrderedDict([('struct', 'def'), ('data', OrderedDict([('array', ['abc'])]))]),
- OrderedDict([('command', 'okay'), ('data', OrderedDict([('member1', ['int']), 
('member2', ['def'])]))])]
-[{'enum_name': 'abc', 'enum_values': ['a', 'b', 'c']}]
-[OrderedDict([('struct', 'def'), ('data', OrderedDict([('array', ['abc'])]))])]
+object :obj-okay-arg
+member member1: intList optional=False
+member member2: defList optional=False
+enum abc ['a', 'b', 'c']
+object def
+member array: abcList optional=False
+command okay :obj-okay-arg -> None
+   gen=True success_response=True
diff --git a/tests/qapi-schema/comments.out b/tests/qapi-schema/comments.out
index 4ce3dcf..6161b90 100644
--- a/tests/qapi-schema/comments.out
+++ b/tests/qapi-schema/comments.out
@@ -1,3 +1 @@
-[OrderedDict([('enum', 'Status'), ('data', ['good', 'bad', 'ugly'])])]
-[{'enum_name': 'Status', 'enum_values': ['good', 'bad', 'ugly']}]
-[]
+enum Status ['good', 'bad', 'ugly']
diff --git a/tests/qapi-schema/empty.out b/tests/qapi-schema/empty.out
index b7f89a4..e69de29 100644
--- a/tests/qapi-schema/empty.out
+++ b/tests/qapi-schema/empty.out
@@ -1,3 +0,0 @@
-[]
-[]
-[]
diff --git a/tests/qapi-schema/enum-empty.out b/tests/qapi-schema/enum-empty.out
index 3b75c16..e09b00f 100644
--- a/tests/qapi-schema/enum-empty.out
+++ b/tests/qapi-schema/enum-empty.out
@@ -1,3 +1 @@
-[OrderedDict([('enum', 'MyEnum'), ('data', [])])]
-[{'enum_name': 'MyEnum', 'enum_values': []}]
-[]
+enum MyEnum []
diff --git a/tests/qapi-schema/event-case.out b/tests/qapi-schema/event-case.out
index 3764bc7..b5ae4c2 100644
--- a/tests/qapi-schema/event-case.out
+++ b/tests/qapi-schema/event-case.out
@@ -1,3 +1 @@
-[OrderedDict([('event', 'oops')])]
-[]
-[]
+event oops None
diff --git a/tests/qapi-schema/flat-union-reverse-define.out 
b/tests/qapi-schema/flat-union-reverse-define.out
index 1ed7b8a..477fb31 100644
--- a/tests/qapi-schema/flat-union-reverse-define.out
+++ b/tests/qapi-schema/flat-union-reverse-define.out
@@ -1,9 +1,12 @@
-[OrderedDict([('union', 'TestUnion'), ('base', 'TestBase'), ('discriminator', 
'enum1'), ('data', OrderedDict([('value1', 'TestTypeA'), ('value2', 
'TestTypeB')]))]),
- OrderedDict([('struct', 'TestBase'), ('data', OrderedDict([('enum1', 
'TestEnum')]))]),
- OrderedDict([('enum', 'TestEnum'), 

[Qemu-devel] [PATCH v6 24/26] qapi: Pseudo-type '**' is now unused, drop it

[Markus Armbruster]

'gen': false needs to stay for now, because netdev_add is still using
it.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
---
 docs/qapi-code-gen.txt| 18 ++
 scripts/qapi.py   | 20 
 tests/Makefile|  2 +-
 tests/qapi-schema/type-bypass-no-gen.err  |  1 -
 tests/qapi-schema/type-bypass-no-gen.exit |  1 -
 tests/qapi-schema/type-bypass-no-gen.json |  2 --
 tests/qapi-schema/type-bypass-no-gen.out  |  0
 tests/qapi-schema/type-bypass.err |  0
 tests/qapi-schema/type-bypass.exit|  1 -
 tests/qapi-schema/type-bypass.json|  2 --
 tests/qapi-schema/type-bypass.out |  4 
 11 files changed, 11 insertions(+), 40 deletions(-)
 delete mode 100644 tests/qapi-schema/type-bypass-no-gen.err
 delete mode 100644 tests/qapi-schema/type-bypass-no-gen.exit
 delete mode 100644 tests/qapi-schema/type-bypass-no-gen.json
 delete mode 100644 tests/qapi-schema/type-bypass-no-gen.out
 delete mode 100644 tests/qapi-schema/type-bypass.err
 delete mode 100644 tests/qapi-schema/type-bypass.exit
 delete mode 100644 tests/qapi-schema/type-bypass.json
 delete mode 100644 tests/qapi-schema/type-bypass.out

diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index a5fccd4..ce32d74 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -111,10 +111,7 @@ and field names within a type, should be all lower case 
with words
 separated by a hyphen.  However, some existing older commands and
 complex types use underscore; when extending such expressions,
 consistency is preferred over blindly avoiding underscore.  Event
-names should be ALL_CAPS with words separated by underscore.  The
-special string '**' appears for some commands that manually perform
-their own type checking rather than relying on the type-safe code
-produced by the qapi code generators.
+names should be ALL_CAPS with words separated by underscore.
 
 Any name (command, event, type, field, or enum value) beginning with
 "x-" is marked experimental, and may be withdrawn or changed
@@ -453,14 +450,11 @@ which would validate this Client JSON Protocol 
transaction:
  <= { "return": [ { "value": "one" }, { } ] }
 
 In rare cases, QAPI cannot express a type-safe representation of a
-corresponding Client JSON Protocol command.  In these cases, if the
-command expression includes the key 'gen' with boolean value false,
-then the 'data' or 'returns' member that intends to bypass generated
-type-safety and do its own manual validation should use an inline
-dictionary definition, with a value of '**' rather than a valid type
-name for the keys that the generated code will not validate.  Please
-try to avoid adding new commands that rely on this, and instead use
-type-safe unions.  For an example of bypass usage:
+corresponding Client JSON Protocol command.  You then have to suppress
+generation of a marshalling function by including a key 'gen' with
+boolean value false, and instead write your own function.  Please try
+to avoid adding new commands that rely on this, and instead use
+type-safe unions.  For an example of this usage:
 
  { 'command': 'netdev_add',
'data': {'type': 'str', 'id': 'str'},
diff --git a/scripts/qapi.py b/scripts/qapi.py
index 415f782..60da124 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -428,15 +428,12 @@ def is_enum(name):
 
 def check_type(expr_info, source, value, allow_array = False,
allow_dict = False, allow_optional = False,
-   allow_star = False, allow_metas = []):
+   allow_metas = []):
 global all_names
 
 if value is None:
 return
 
-if allow_star and value == '**':
-return
-
 # Check if array type for value is okay
 if isinstance(value, list):
 if not allow_array:
@@ -450,10 +447,6 @@ def check_type(expr_info, source, value, allow_array = 
False,
 
 # Check if type name for value is okay
 if isinstance(value, str):
-if value == '**':
-raise QAPIExprError(expr_info,
-"%s uses '**' but did not request 'gen':false"
-% source)
 if not value in all_names:
 raise QAPIExprError(expr_info,
 "%s uses unknown type '%s'"
@@ -479,7 +472,7 @@ def check_type(expr_info, source, value, allow_array = 
False,
 # Todo: allow dictionaries to represent default values of
 # an optional argument.
 check_type(expr_info, "Member '%s' of %s" % (key, source), arg,
-   allow_array=True, allow_star=allow_star,
+   allow_array=True,
allow_metas=['built-in', 'union', 'alternate', 'struct',
 'enum'])
 
@@ -499,18 +492,16 @@ def check_member_clash(expr_info, base_name, data, source 
= ""):
 
 def check_command(expr, expr_info):
 name = expr['command']
-allow_st

[Qemu-devel] [PATCH v6 14/26] qapi-visit: Rearrange code a bit

[Markus Armbruster]

Move gen_visit_decl() to a better place.  Inline
generate_visit_struct_body().

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
---
 scripts/qapi-visit.py | 50 --
 1 file changed, 20 insertions(+), 30 deletions(-)

diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index 2ac244b..d776948 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -19,6 +19,16 @@ implicit_structs_seen = set()
 struct_fields_seen = set()
 
 
+def gen_visit_decl(name, scalar=False):
+c_type = c_name(name) + ' *'
+if not scalar:
+c_type += '*'
+return mcgen('''
+void visit_type_%(c_name)s(Visitor *m, %(c_type)sobj, const char *name, Error 
**errp);
+''',
+ c_name=c_name(name), c_type=c_type)
+
+
 def gen_visit_implicit_struct(typ):
 if typ in implicit_structs_seen:
 return ''
@@ -117,12 +127,17 @@ out:
 return ret
 
 
-def gen_visit_struct_body(name):
+def gen_visit_struct(name, base, members):
+ret = gen_visit_struct_fields(name, base, members)
+
 # FIXME: if *obj is NULL on entry, and visit_start_struct() assigns to
 # *obj, but then visit_type_FOO_fields() fails, we should clean up *obj
 # rather than leaving it non-NULL. As currently written, the caller must
 # call qapi_free_FOO() to avoid a memory leak of the partial FOO.
-ret = mcgen('''
+ret += mcgen('''
+
+void visit_type_%(c_name)s(Visitor *m, %(c_name)s **obj, const char *name, 
Error **errp)
+{
 Error *err = NULL;
 
 visit_start_struct(m, (void **)obj, "%(name)s", name, sizeof(%(c_name)s), 
&err);
@@ -133,25 +148,10 @@ def gen_visit_struct_body(name):
 visit_end_struct(m, &err);
 }
 error_propagate(errp, err);
-''',
-name=name, c_name=c_name(name))
-return ret
-
-
-def gen_visit_struct(name, base, members):
-ret = gen_visit_struct_fields(name, base, members)
-ret += mcgen('''
-
-void visit_type_%(c_name)s(Visitor *m, %(c_name)s **obj, const char *name, 
Error **errp)
-{
-''',
- c_name=c_name(name))
-
-ret += gen_visit_struct_body(name)
-
-ret += mcgen('''
 }
-''')
+''',
+ name=name, c_name=c_name(name))
+
 return ret
 
 
@@ -342,16 +342,6 @@ out:
 return ret
 
 
-def gen_visit_decl(name, scalar=False):
-c_type = c_name(name) + ' *'
-if not scalar:
-c_type += '*'
-return mcgen('''
-void visit_type_%(c_name)s(Visitor *m, %(c_type)sobj, const char *name, Error 
**errp);
-''',
- c_name=c_name(name), c_type=c_type)
-
-
 class QAPISchemaGenVisitVisitor(QAPISchemaVisitor):
 def __init__(self):
 self.decl = None
-- 
2.4.3

[Qemu-devel] [PATCH v6 18/26] qapi-commands: De-duplicate output marshaling functions

[Markus Armbruster]

gen_marshal_output() uses its parameter name only for name of the
generated function.  Name it after the type being marshaled instead of
its caller, and drop duplicates.

Saves 7 copies of qmp_marshal_output_int() in qemu-ga, and one copy of
qmp_marshal_output_str() in qemu-system-*.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
---
 docs/qapi-code-gen.txt   |  4 ++--
 scripts/qapi-commands.py | 17 ++---
 2 files changed, 12 insertions(+), 9 deletions(-)

diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 09d15dc..147638f 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -713,7 +713,7 @@ Example:
 $ cat qapi-generated/example-qmp-marshal.c
 [Uninteresting stuff omitted...]
 
-static void qmp_marshal_output_my_command(UserDefOne *ret_in, QObject 
**ret_out, Error **errp)
+static void qmp_marshal_output_UserDefOne(UserDefOne *ret_in, QObject 
**ret_out, Error **errp)
 {
 Error *local_err = NULL;
 QmpOutputVisitor *mo = qmp_output_visitor_new();
@@ -756,7 +756,7 @@ Example:
 goto out;
 }
 
-qmp_marshal_output_my_command(retval, ret, &local_err);
+qmp_marshal_output_UserDefOne(retval, ret, &local_err);
 
 out:
 error_propagate(errp, local_err);
diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index 833768e..810a897 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -62,7 +62,7 @@ def gen_call(name, arg_type, ret_type):
 
 qmp_marshal_output_%(c_name)s(retval, ret, &local_err);
 ''',
- c_name=c_name(name))
+ c_name=ret_type.c_name())
 pop_indent()
 return ret
 
@@ -166,10 +166,10 @@ qapi_dealloc_visitor_cleanup(md);
 return ret
 
 
-def gen_marshal_output(name, ret_type):
+def gen_marshal_output(ret_type):
 return mcgen('''
 
-static void qmp_marshal_output_%(c_cmd_name)s(%(c_type)s ret_in, QObject 
**ret_out, Error **errp)
+static void qmp_marshal_output_%(c_name)s(%(c_type)s ret_in, QObject 
**ret_out, Error **errp)
 {
 Error *local_err = NULL;
 QmpOutputVisitor *mo = qmp_output_visitor_new();
@@ -192,8 +192,7 @@ out:
 qapi_dealloc_visitor_cleanup(md);
 }
 ''',
- c_type=ret_type.c_type(), c_cmd_name=c_name(name),
- c_name=ret_type.c_name())
+ c_type=ret_type.c_type(), c_name=ret_type.c_name())
 
 
 def gen_marshal_proto(name):
@@ -272,24 +271,28 @@ class QAPISchemaGenCommandVisitor(QAPISchemaVisitor):
 self.decl = None
 self.defn = None
 self._regy = None
+self._visited_ret_types = None
 
 def visit_begin(self, schema):
 self.decl = ''
 self.defn = ''
 self._regy = ''
+self._visited_ret_types = set()
 
 def visit_end(self):
 if not middle_mode:
 self.defn += gen_registry(self._regy)
 self._regy = None
+self._visited_ret_types = None
 
 def visit_command(self, name, info, arg_type, ret_type,
   gen, success_response):
 if not gen:
 return
 self.decl += gen_command_decl(name, arg_type, ret_type)
-if ret_type:
-self.defn += gen_marshal_output(name, ret_type)
+if ret_type and ret_type not in self._visited_ret_types:
+self._visited_ret_types.add(ret_type)
+self.defn += gen_marshal_output(ret_type)
 if middle_mode:
 self.decl += gen_marshal_decl(name)
 self.defn += gen_marshal(name, arg_type, ret_type)
-- 
2.4.3

[Qemu-devel] [PATCH v6 06/26] qapi-types: Convert to QAPISchemaVisitor, fixing flat unions

[Markus Armbruster]

Fixes flat unions to get the base's base members.  Test case is from
commit 2fc0043, in qapi-schema-test.json:

{ 'union': 'UserDefFlatUnion',
  'base': 'UserDefUnionBase',
  'discriminator': 'enum1',
  'data': { 'value1' : 'UserDefA',
'value2' : 'UserDefB',
'value3' : 'UserDefB' } }

{ 'struct': 'UserDefUnionBase',
  'base': 'UserDefZero',
  'data': { 'string': 'str', 'enum1': 'EnumOne' } }

{ 'struct': 'UserDefZero',
  'data': { 'integer': 'int' } }

Patch's effect on UserDefFlatUnion:

 struct UserDefFlatUnion {
 /* Members inherited from UserDefUnionBase: */
+int64_t integer;
 char *string;
 EnumOne enum1;
 /* Own members: */
 union { /* union tag is @enum1 */
 void *data;
 UserDefA *value1;
 UserDefB *value2;
 UserDefB *value3;
 };
 };

Flat union visitors remain broken.  They'll be fixed next.

Two ugly special cases for simple unions now stand out like sore
thumbs:

1. The type tag is named 'type' everywhere, except in generated C,
   where it's 'kind'.

2. QAPISchema lowers simple unions to semantically equivalent flat
   unions.  However, the C generated for a simple unions differs from
   the C generated for its equivalent flat union, and we therefore
   need special code to preserve that pointless difference for now.

Mark both TODO.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
---
 docs/qapi-code-gen.txt  |  51 +++---
 scripts/qapi-types.py   | 284 ++--
 scripts/qapi.py |  10 +-
 tests/qapi-schema/qapi-schema-test.json |   4 +-
 4 files changed, 164 insertions(+), 185 deletions(-)

diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index ff16df2..710defc 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -537,22 +537,6 @@ Example:
 $ cat qapi-generated/example-qapi-types.c
 [Uninteresting stuff omitted...]
 
-void qapi_free_UserDefOneList(UserDefOneList *obj)
-{
-QapiDeallocVisitor *md;
-Visitor *v;
-
-if (!obj) {
-return;
-}
-
-md = qapi_dealloc_visitor_new();
-v = qapi_dealloc_get_visitor(md);
-visit_type_UserDefOneList(v, &obj, NULL, NULL);
-qapi_dealloc_visitor_cleanup(md);
-}
-
-
 void qapi_free_UserDefOne(UserDefOne *obj)
 {
 QapiDeallocVisitor *md;
@@ -567,6 +551,21 @@ Example:
 visit_type_UserDefOne(v, &obj, NULL, NULL);
 qapi_dealloc_visitor_cleanup(md);
 }
+
+void qapi_free_UserDefOneList(UserDefOneList *obj)
+{
+QapiDeallocVisitor *md;
+Visitor *v;
+
+if (!obj) {
+return;
+}
+
+md = qapi_dealloc_visitor_new();
+v = qapi_dealloc_get_visitor(md);
+visit_type_UserDefOneList(v, &obj, NULL, NULL);
+qapi_dealloc_visitor_cleanup(md);
+}
 $ cat qapi-generated/example-qapi-types.h
 [Uninteresting stuff omitted...]
 
@@ -577,24 +576,24 @@ Example:
 
 typedef struct UserDefOne UserDefOne;
 
-typedef struct UserDefOneList {
+typedef struct UserDefOneList UserDefOneList;
+
+struct UserDefOne {
+int64_t integer;
+char *string;
+};
+
+void qapi_free_UserDefOne(UserDefOne *obj);
+
+struct UserDefOneList {
 union {
 UserDefOne *value;
 uint64_t padding;
 };
 struct UserDefOneList *next;
-} UserDefOneList;
-
-
-[Functions on built-in types omitted...]
-
-struct UserDefOne {
-int64_t integer;
-char *string;
 };
 
 void qapi_free_UserDefOneList(UserDefOneList *obj);
-void qapi_free_UserDefOne(UserDefOne *obj);
 
 #endif
 
diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
index d162ca2..9c422d1 100644
--- a/scripts/qapi-types.py
+++ b/scripts/qapi-types.py
@@ -2,88 +2,67 @@
 # QAPI types generator
 #
 # Copyright IBM, Corp. 2011
+# Copyright (c) 2013-2015 Red Hat Inc.
 #
 # Authors:
 #  Anthony Liguori 
+#  Markus Armbruster 
 #
 # This work is licensed under the terms of the GNU GPL, version 2.
 # See the COPYING file in the top-level directory.
 
-from ordereddict import OrderedDict
 from qapi import *
 
-def generate_fwd_builtin(name):
-return mcgen('''
-
-typedef struct %(name)sList {
-union {
-%(type)s value;
-uint64_t padding;
-};
-struct %(name)sList *next;
-} %(name)sList;
-''',
- type=c_type(name),
- name=name)
-
-def generate_fwd_struct(name):
+def gen_fwd_object_or_array(name):
 return mcgen('''
 
 typedef struct %(name)s %(name)s;
-
-typedef struct %(name)sList {
-union {
-%(name)s *value;
-uint64_t padding;
-};
-struct %(name)sList *next;
-} %(name)sList;
 ''',
  name=c_name(name))
 
-def generate_fwd_enum_struct(name):
+def ge

[Qemu-devel] [PATCH v6 23/26] qapi-schema: Fix up misleading specification of netdev_add

[Markus Armbruster]

It doesn't take a 'props' argument, let alone one in the format
"NAME=VALUE,..."

The bogus arguments specification doesn't matter due to 'gen': false.
Clean it up to be incomplete rather than wrong, and document the
incompleteness.

While there, improve netdev_add usage example in the manual: add a
device option to show how it's done.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
---
 docs/qapi-code-gen.txt |  2 +-
 qapi-schema.json   | 13 +++--
 qmp-commands.hx|  4 +++-
 3 files changed, 11 insertions(+), 8 deletions(-)

diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index c713e3a..a5fccd4 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -463,7 +463,7 @@ try to avoid adding new commands that rely on this, and 
instead use
 type-safe unions.  For an example of bypass usage:
 
  { 'command': 'netdev_add',
-   'data': {'type': 'str', 'id': 'str', '*props': '**'},
+   'data': {'type': 'str', 'id': 'str'},
'gen': false }
 
 Normally, the QAPI schema is used to describe synchronous exchanges,
diff --git a/qapi-schema.json b/qapi-schema.json
index a11596a..056f4e9 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -2076,11 +2076,12 @@
 #
 # @id: the name of the new network backend
 #
-# @props: #optional a list of properties to be passed to the backend in
-# the format 'name=value', like 'ifname=tap0,script=no'
+# Additional arguments depend on the type.
 #
-# Notes: The semantics of @props is not well defined.  Future commands will be
-#introduced that provide stronger typing for backend creation.
+# TODO This command effectively bypasses QAPI completely due to its
+# "additional arguments" business.  It shouldn't have been added to
+# the schema in this form.  It should be qapified properly, or
+# replaced by a properly qapified command.
 #
 # Since: 0.14.0
 #
@@ -2088,8 +2089,8 @@
 #  If @type is not a valid network backend, DeviceNotFound
 ##
 { 'command': 'netdev_add',
-  'data': {'type': 'str', 'id': 'str', '*props': '**'},
-  'gen': false }
+  'data': {'type': 'str', 'id': 'str'},
+  'gen': false }# so we can get the additional arguments
 
 ##
 # @netdev_del:
diff --git a/qmp-commands.hx b/qmp-commands.hx
index 7c74e20..5a54406 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -917,7 +917,9 @@ Arguments:
 
 Example:
 
--> { "execute": "netdev_add", "arguments": { "type": "user", "id": "netdev1" } 
}
+-> { "execute": "netdev_add",
+ "arguments": { "type": "user", "id": "netdev1",
+"dnssearch": "example.org" } }
 <- { "return": {} }
 
 Note: The supported device options are the same ones supported by the '-netdev'
-- 
2.4.3

[Qemu-devel] [PATCH v6 00/26] qapi: QMP introspection

[Markus Armbruster]

Series needs my "[PATCH for-2.4 0/2] qom: Fix misuse of Error API" to
pass tests.  It's in Andreas's qom-next (currently at commit de1da05),
so I based on that.

You can fetch it from my git://repo.or.cz/qemu/armbru.git branch
qapi-introspect.

* PATCH 01-18 basically replace the intermediate representation.  The
  replacement isn't complete, but fully functional.  See PATCH 02 for
  rationale and future work.

* PATCH 19-24 replace the '**' type bypass by a proper top type called
  'any'.

* PATCH 25-26 are the QMP introspection.

v6:
* Straightforward rebase, R-bys retained.
* Generated code is unchanged.
* PATCH 02:
  - Fix typo in commit message [Eric]
* PATCH 06-09,16-17
  - Review scaffolding patches, drop along with their reverts
* PATCH 20
  - New, fixes a latent bug uncovered by this series
* PATCH 24 (was PATCH 30):
  - Rename qmp_query_schema() to qmp_query_qmp_schema() for
consistency with the QMP command name [Eric]
  - Consistent spacing in introspect.json member comments [Eric]
  - Squash in old PATCH 31

v5:
* Straightforward rebase, R-bys retained.
* Generated code is unchanged unless noted otherwise.
* PATCH 02-05,10-12,15,19-21,26,30,31:
  - Make pep8 happy [Dan]
* PATCH 02,03,10-12,15,19,24,26,29-32:
  - Rename private instance variables to start with '_' [Eric]
* PATCH 02,10,11,30:
  - Really fix name of simple unions' implicit tag [Eric], and
simplify _gen_variants
Changes generated code, obviously.
* PATCH 02,10:
  - Delay definition of simple_union_type() until its first use
* PATCH 02:
  - Trim trailing whitespace [Dan]
  - Commit message tweak
* PATCH 10,11:
  - Explain simple union TODOs better
* PATCH 22,27,30:
  - Fix tab damage [Eric]
* PATCH 30:
  - Rename query-schema to query-qmp-schema [Michael]
Changes generated code, obviously.
  - Documentation improvements [Michael, Eric]
  - Make to_json() robust for key containing '"' (should never happen)
[Eric]
* PATCH 32:
  - Documentation improvements [Eric]
  - Trim trailing whitespace

v4:
* Straightforward rebase, R-bys retained unless noted otherwise.
* PATCH 02:
  - Add a TODO on catching name collisions [Eric]
  - Unify how QAPISchemaObjectType.__init__() asserts "either None or
isinstance" [Eric]
  - Drop bogus "not implemented" comment in simple_union_type() [Eric]
  - Use iteritems() where appropriate [Eric]
  - Drop dead conditional in _def_command() [Eric]
* PATCH 03:
  - Define json_type() always
Just to be neat; it only gets called when it was defined before.
* PATCH 04,10-12,15,30:
  - Give visit_begin() parameter schema from the start instead of
adding it in PATCH 30 [Eric]
* PATCH 07,16:
  - Touch up commit message's hints for reviewers [Eric]
* PATCH 10+11:
  - Reword a comment more clearly [Eric]
* PATCH 11:
  - Fix test_validate_fail_union_flat_no_discrim() [Eric]
* PATCH 18:
  - Drop dead commit message paragraph [Eric]
* PATCH 26:
  - Cover ['any'] in qapi-schema-test.json [Eric]
* PATCH 30:
  - Rework commit message
  - New docs/qapi-code-gen.txt section "Client JSON Protocol
introspection"
  - Fix docs/qapi-code-gen.txt pastos [Eric]
  - Rearrange and document qapi/introspect.json
The forward references are intentional.  I picked this order to
make the file more readable.
  - Fill in proper qmp-commands.hx documentation
  - Uniform spacing in generated code
  - Put the commit message's TODO on long lines into code, too
  - Drop _gen_json()'s useless default argument [Eric]
  - Fix name of simple unions' implicit tag [Eric]
  - Adjust white-space to make pylint happier
  - Fix indentation in test-qmp-input-strict.c [Eric]
* PATCH 31:
  - Commit message tweak
  - Update docs/qapi-code-gen.txt
  - R-by dropped to make sure the doc gets reviewed.
* PATCH 32:
  - New option -u to suppress the type name hiding (debugging aid)
Easier than generating comments.  We can still generate them
later.
  - Update docs/qapi-code-gen.txt and qapi/introspect.json comments
  - Use %d to format integers [Eric]
  - R-by dropped to make sure the doc gets reviewed.
* This change log is again long, but compared to last time, the
  changes are rather local, and most of them are to the last three
  patches.  I think we're converging.

v3:
* Straightforward rebase, R-bys retained unless noted otherwise.  Due
  to my widespread changes, you may prefer to review from scratch
  anyway.
* Old PATCH 01-19 split off into separate non-RFC series "[PATCH
  00/26] qapi: Another round of fixes and cleanups"
* PATCH 02:
  - Commit message spelling fix [Eric]
  - Clarify patch's non-effect in commit message.
  - Assert enumeration constants are unique.
  - Assert object's base is an object without variants [Eric].
  - Drop a few superfluous check() recursions.
  - Use arg_type, ret_type instead of args, ret for commands [Eric],
and arg_type instead of data for events.
  - Lower simple variants to flat ones as described on
qapi-code-gen.txt.  Replace QAPISchemaObjectType's .flat by
.s

[Qemu-devel] [PATCH v6 07/26] qapi-visit: Convert to QAPISchemaVisitor, fixing bugs

[Markus Armbruster]

Fixes flat unions to visit the base's base members (the previous
commit merely added them to the struct).  Same test case.

Patch's effect on visit_type_UserDefFlatUnion():

 static void visit_type_UserDefFlatUnion_fields(Visitor *m, 
UserDefFlatUnion **obj, Error **errp)
 {
 Error *err = NULL;

+visit_type_int(m, &(*obj)->integer, "integer", &err);
+if (err) {
+goto out;
+}
 visit_type_str(m, &(*obj)->string, "string", &err);
 if (err) {
 goto out;

Test cases updated for the bug fix.

Fixes alternates to generate a visitor for their implicit enumeration
type.  None of them are currently used, obviously.  Example:
block-core.json's BlockdevRef now generates
visit_type_BlockdevRefKind().

The previous commit's two ugly special cases exist here, too.  Mark
both TODO.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
---
 scripts/qapi-visit.py   | 270 +---
 tests/qapi-schema/qapi-schema-test.json |   3 -
 tests/test-qmp-input-strict.c   |   4 +-
 tests/test-qmp-input-visitor.c  |   4 +-
 4 files changed, 117 insertions(+), 164 deletions(-)

diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index c493964..cfebcb7 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -12,7 +12,6 @@
 # This work is licensed under the terms of the GNU GPL, version 2.
 # See the COPYING file in the top-level directory.
 
-from ordereddict import OrderedDict
 from qapi import *
 import re
 
@@ -24,13 +23,13 @@ def generate_visit_implicit_struct(type):
 return ''
 implicit_structs_seen.add(type)
 ret = ''
-if type not in struct_fields_seen:
+if type.name not in struct_fields_seen:
 # Need a forward declaration
 ret += mcgen('''
 
 static void visit_type_%(c_type)s_fields(Visitor *m, %(c_type)s **obj, Error 
**errp);
 ''',
- c_type=type_name(type))
+ c_type=type.c_name())
 
 ret += mcgen('''
 
@@ -46,7 +45,7 @@ static void visit_type_implicit_%(c_type)s(Visitor *m, 
%(c_type)s **obj, Error *
 error_propagate(errp, err);
 }
 ''',
- c_type=type_name(type))
+ c_type=type.c_name())
 return ret
 
 def generate_visit_struct_fields(name, members, base = None):
@@ -74,24 +73,24 @@ if (err) {
 goto out;
 }
 ''',
- type=type_name(base), c_name=c_name('base'))
+ type=base.c_name(), c_name=c_name('base'))
 
-for argname, argentry, optional in parse_args(members):
-if optional:
+for memb in members:
+if memb.optional:
 ret += mcgen('''
 visit_optional(m, &(*obj)->has_%(c_name)s, "%(name)s", &err);
 if (!err && (*obj)->has_%(c_name)s) {
 ''',
- c_name=c_name(argname), name=argname)
+ c_name=c_name(memb.name), name=memb.name)
 push_indent()
 
 ret += mcgen('''
 visit_type_%(type)s(m, &(*obj)->%(c_name)s, "%(name)s", &err);
 ''',
- type=type_name(argentry), c_name=c_name(argname),
- name=argname)
+ type=memb.type.c_name(), c_name=c_name(memb.name),
+ name=memb.name)
 
-if optional:
+if memb.optional:
 pop_indent()
 ret += mcgen('''
 }
@@ -136,12 +135,7 @@ def generate_visit_struct_body(name):
 
 return ret
 
-def generate_visit_struct(expr):
-
-name = expr['struct']
-members = expr['data']
-base = expr.get('base')
-
+def gen_visit_struct(name, base, members):
 ret = generate_visit_struct_fields(name, members, base)
 
 ret += mcgen('''
@@ -158,10 +152,10 @@ void visit_type_%(name)s(Visitor *m, %(name)s **obj, 
const char *name, Error **e
 ''')
 return ret
 
-def generate_visit_list(name):
+def gen_visit_list(name, element_type):
 return mcgen('''
 
-void visit_type_%(name)sList(Visitor *m, %(name)sList **obj, const char *name, 
Error **errp)
+void visit_type_%(name)s(Visitor *m, %(name)s **obj, const char *name, Error 
**errp)
 {
 Error *err = NULL;
 GenericList *i, **prev;
@@ -174,8 +168,8 @@ void visit_type_%(name)sList(Visitor *m, %(name)sList 
**obj, const char *name, E
 for (prev = (GenericList **)obj;
  !err && (i = visit_next_list(m, prev, &err)) != NULL;
  prev = &i) {
-%(name)sList *native_i = (%(name)sList *)i;
-visit_type_%(name)s(m, &native_i->value, NULL, &err);
+%(name)s *native_i = (%(name)s *)i;
+visit_type_%(c_elt_type)s(m, &native_i->value, NULL, &err);
 }
 
 error_propagate(errp, err);
@@ -185,7 +179,8 @@ out:
 error_propagate(errp, err);
 }
 ''',
-name=type_name(name))
+ name=c_name(name),
+ c_elt_type=element_type.c_name())
 
 def generate_visit_enum(name):
 return mcgen('''
@@ -197,7 +192,7 @@ void visit_type_%(c_name)s(Vi

[Qemu-devel] [PATCH v6 10/26] qapi-event: Eliminate global variable event_enum_value

[Markus Armbruster]

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
---
 scripts/qapi-event.py | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/scripts/qapi-event.py b/scripts/qapi-event.py
index aed45d6..537da17 100644
--- a/scripts/qapi-event.py
+++ b/scripts/qapi-event.py
@@ -137,7 +137,7 @@ def generate_event_implement(api_name, event_name, params):
 emit(%(event_enum_value)s, qmp, &local_err);
 
 """,
- event_enum_value = event_enum_value)
+ event_enum_value = c_enum_const(event_enum_name, event_name))
 
 # step 5: clean up
 if params:
@@ -223,7 +223,6 @@ for expr in exprs:
 fdecl.write(ret)
 
 # We need an enum value per event
-event_enum_value = c_enum_const(event_enum_name, event_name)
 ret = generate_event_implement(api_name, event_name, params)
 fdef.write(ret)
 
-- 
2.4.3

[Qemu-devel] [PATCH v6 09/26] qapi: De-duplicate enum code generation

[Markus Armbruster]

Duplicated in commit 21cd70d.  Yes, we can't import qapi-types, but
that's no excuse.  Move the helpers from qapi-types.py to qapi.py, and
replace the duplicates in qapi-event.py.

The generated event enumeration type's lookup table becomes
const-correct (see commit 2e4450f), and uses explicit indexes instead
of relying on order (see commit 912ae9c).

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
---
 docs/qapi-code-gen.txt |  9 ---
 scripts/qapi-event.py  | 67 +++---
 scripts/qapi-types.py  | 55 -
 scripts/qapi.py| 55 +
 4 files changed, 64 insertions(+), 122 deletions(-)

diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index 710defc..7c500b6 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -821,9 +821,9 @@ Example:
 QDECREF(qmp);
 }
 
-const char *example_QAPIEvent_lookup[] = {
-"MY_EVENT",
-NULL,
+const char *const example_QAPIEvent_lookup[] = {
+[EXAMPLE_QAPI_EVENT_MY_EVENT] = "MY_EVENT",
+[EXAMPLE_QAPI_EVENT_MAX] = NULL,
 };
 $ cat qapi-generated/example-qapi-event.h
 [Uninteresting stuff omitted...]
@@ -838,10 +838,11 @@ Example:
 
 void qapi_event_send_my_event(Error **errp);
 
-extern const char *example_QAPIEvent_lookup[];
 typedef enum example_QAPIEvent {
 EXAMPLE_QAPI_EVENT_MY_EVENT = 0,
 EXAMPLE_QAPI_EVENT_MAX = 1,
 } example_QAPIEvent;
 
+extern const char *const example_QAPIEvent_lookup[];
+
 #endif
diff --git a/scripts/qapi-event.py b/scripts/qapi-event.py
index aec2d32..aed45d6 100644
--- a/scripts/qapi-event.py
+++ b/scripts/qapi-event.py
@@ -153,63 +153,6 @@ def generate_event_implement(api_name, event_name, params):
 
 return ret
 
-
-# Following are the functions that generate an enum type for all defined
-# events, similar to qapi-types.py. Here we already have enum name and
-# values which were generated before and recorded in event_enum_*. It also
-# works around the issue that "import qapi-types" can't work.
-
-def generate_event_enum_decl(event_enum_name, event_enum_values):
-lookup_decl = mcgen('''
-
-extern const char *%(event_enum_name)s_lookup[];
-''',
-event_enum_name = event_enum_name)
-
-enum_decl = mcgen('''
-typedef enum %(event_enum_name)s {
-''',
-  event_enum_name = event_enum_name)
-
-# append automatically generated _MAX value
-enum_max_value = c_enum_const(event_enum_name, "MAX")
-enum_values = event_enum_values + [ enum_max_value ]
-
-i = 0
-for value in enum_values:
-enum_decl += mcgen('''
-%(value)s = %(i)d,
-''',
- value = value,
- i = i)
-i += 1
-
-enum_decl += mcgen('''
-} %(event_enum_name)s;
-''',
-   event_enum_name = event_enum_name)
-
-return lookup_decl + enum_decl
-
-def generate_event_enum_lookup(event_enum_name, event_enum_strings):
-ret = mcgen('''
-
-const char *%(event_enum_name)s_lookup[] = {
-''',
-event_enum_name = event_enum_name)
-
-for string in event_enum_strings:
-ret += mcgen('''
-"%(string)s",
-''',
- string = string)
-
-ret += mcgen('''
-NULL,
-};
-''')
-return ret
-
 (input_file, output_dir, do_c, do_h, prefix, dummy) = parse_command_line()
 
 c_comment = '''
@@ -266,8 +209,7 @@ fdecl.write(mcgen('''
 exprs = QAPISchema(input_file).get_exprs()
 
 event_enum_name = c_name(prefix + "QAPIEvent", protect=False)
-event_enum_values = []
-event_enum_strings = []
+event_names = []
 
 for expr in exprs:
 if expr.has_key('event'):
@@ -286,12 +228,11 @@ for expr in exprs:
 fdef.write(ret)
 
 # Record it, and generate enum later
-event_enum_values.append(event_enum_value)
-event_enum_strings.append(event_name)
+event_names.append(event_name)
 
-ret = generate_event_enum_decl(event_enum_name, event_enum_values)
+ret = generate_enum(event_enum_name, event_names)
 fdecl.write(ret)
-ret = generate_event_enum_lookup(event_enum_name, event_enum_strings)
+ret = generate_enum_lookup(event_enum_name, event_names)
 fdef.write(ret)
 
 close_output(fdef, fdecl)
diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
index 9c422d1..bc20136 100644
--- a/scripts/qapi-types.py
+++ b/scripts/qapi-types.py
@@ -80,61 +80,6 @@ struct %(name)s {
 
 return ret
 
-def generate_enum_lookup(name, values):
-ret = mcgen('''
-
-const char *const %(name)s_lookup[] = {
-''',
-name=c_name(name))
-for value in values:
-index = c_enum_const(name, value)
-ret += mcgen('''
-[%(index)s] = "%(value)s",
-''',
- index = index, value = value)
-
-max_index = c_enum_const(name, 'MAX')
-ret += mcgen('''
-[%(max_index)s] = NULL,
-};
-''',
-max_index=max_ind

[Qemu-devel] [PATCH v6 04/26] qapi: New QAPISchemaVisitor

[Markus Armbruster]

The visitor will help keeping the code generation code simple and
reasonably separated from QAPISchema details.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
Reviewed-by: Daniel P. Berrange 
---
 scripts/qapi.py | 63 +
 1 file changed, 63 insertions(+)

diff --git a/scripts/qapi.py b/scripts/qapi.py
index 0f6d475..fd52a9a 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -768,6 +768,39 @@ class QAPISchemaEntity(object):
 def check(self, schema):
 pass
 
+def visit(self, visitor):
+pass
+
+
+class QAPISchemaVisitor(object):
+def visit_begin(self, schema):
+pass
+
+def visit_end(self):
+pass
+
+def visit_builtin_type(self, name, info, json_type):
+pass
+
+def visit_enum_type(self, name, info, values):
+pass
+
+def visit_array_type(self, name, info, element_type):
+pass
+
+def visit_object_type(self, name, info, base, members, variants):
+pass
+
+def visit_alternate_type(self, name, info, variants):
+pass
+
+def visit_command(self, name, info, arg_type, ret_type,
+  gen, success_response):
+pass
+
+def visit_event(self, name, info, arg_type):
+pass
+
 
 class QAPISchemaType(QAPISchemaEntity):
 def c_type(self, is_param=False):
@@ -814,6 +847,9 @@ class QAPISchemaBuiltinType(QAPISchemaType):
 def json_type(self):
 return self._json_type_name
 
+def visit(self, visitor):
+visitor.visit_builtin_type(self.name, self.info, self.json_type())
+
 
 class QAPISchemaEnumType(QAPISchemaType):
 def __init__(self, name, info, values):
@@ -834,6 +870,9 @@ class QAPISchemaEnumType(QAPISchemaType):
 def json_type(self):
 return 'string'
 
+def visit(self, visitor):
+visitor.visit_enum_type(self.name, self.info, self.values)
+
 
 class QAPISchemaArrayType(QAPISchemaType):
 def __init__(self, name, info, element_type):
@@ -849,6 +888,9 @@ class QAPISchemaArrayType(QAPISchemaType):
 def json_type(self):
 return 'array'
 
+def visit(self, visitor):
+visitor.visit_array_type(self.name, self.info, self.element_type)
+
 
 class QAPISchemaObjectType(QAPISchemaType):
 def __init__(self, name, info, base, local_members, variants):
@@ -897,6 +939,10 @@ class QAPISchemaObjectType(QAPISchemaType):
 def json_type(self):
 return 'object'
 
+def visit(self, visitor):
+visitor.visit_object_type(self.name, self.info,
+  self.base, self.local_members, self.variants)
+
 
 class QAPISchemaObjectTypeMember(object):
 def __init__(self, name, typ, optional):
@@ -964,6 +1010,9 @@ class QAPISchemaAlternateType(QAPISchemaType):
 def json_type(self):
 return 'value'
 
+def visit(self, visitor):
+visitor.visit_alternate_type(self.name, self.info, self.variants)
+
 
 class QAPISchemaCommand(QAPISchemaEntity):
 def __init__(self, name, info, arg_type, ret_type, gen, success_response):
@@ -986,6 +1035,11 @@ class QAPISchemaCommand(QAPISchemaEntity):
 self.ret_type = schema.lookup_type(self._ret_type_name)
 assert isinstance(self.ret_type, QAPISchemaType)
 
+def visit(self, visitor):
+visitor.visit_command(self.name, self.info,
+  self.arg_type, self.ret_type,
+  self.gen, self.success_response)
+
 
 class QAPISchemaEvent(QAPISchemaEntity):
 def __init__(self, name, info, arg_type):
@@ -1000,6 +1054,9 @@ class QAPISchemaEvent(QAPISchemaEntity):
 assert isinstance(self.arg_type, QAPISchemaObjectType)
 assert not self.arg_type.variants   # not implemented
 
+def visit(self, visitor):
+visitor.visit_event(self.name, self.info, self.arg_type)
+
 
 class QAPISchema(object):
 def __init__(self, fname):
@@ -1196,6 +1253,12 @@ class QAPISchema(object):
 for ent in self._entity_dict.values():
 ent.check(self)
 
+def visit(self, visitor):
+visitor.visit_begin(self)
+for name in sorted(self._entity_dict.keys()):
+self._entity_dict[name].visit(visitor)
+visitor.visit_end()
+
 
 #
 # Code generation helpers
-- 
2.4.3

[Qemu-devel] [PATCH v6 12/26] qapi: Replace dirty is_c_ptr() by method c_null()

[Markus Armbruster]

is_c_ptr() looks whether the end of the C text for the type looks like
a pointer.  Works, but is fragile.

We now have a better tool: use QAPISchemaType method c_null().  The
initializers for non-pointers become prettier: 0, false or the
enumeration constant with the value 0 instead of {0}.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
---
 scripts/qapi-commands.py | 16 +---
 scripts/qapi.py  |  3 ---
 2 files changed, 5 insertions(+), 14 deletions(-)

diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index a68517a..cbff356 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -91,18 +91,12 @@ def gen_visitor_input_vars_decl(args):
 bool has_%(argname)s = false;
 ''',
  argname=c_name(memb.name))
-if is_c_ptr(memb.type.c_type()):
-ret += mcgen('''
-%(argtype)s %(argname)s = NULL;
+ret += mcgen('''
+%(c_type)s %(c_name)s = %(c_null)s;
 ''',
- argname=c_name(memb.name),
- argtype=memb.type.c_type())
-else:
-ret += mcgen('''
-%(argtype)s %(argname)s = {0};
-''',
- argname=c_name(memb.name),
- argtype=memb.type.c_type())
+ c_name=c_name(memb.name),
+ c_type=memb.type.c_type(),
+ c_null=memb.type.c_null())
 
 pop_indent()
 return ret
diff --git a/scripts/qapi.py b/scripts/qapi.py
index e1268f4..0f9537d 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -1430,9 +1430,6 @@ def c_type(value, is_param=False):
 assert isinstance(value, str) and value != ""
 return c_name(value) + pointer_suffix
 
-def is_c_ptr(value):
-return value.endswith(pointer_suffix)
-
 def genindent(count):
 ret = ""
 for i in range(count):
-- 
2.4.3

[Qemu-devel] [PATCH v6 02/26] qapi: New QAPISchema intermediate reperesentation

[Markus Armbruster]

The QAPI code generators work with a syntax tree (nested dictionaries)
plus a few symbol tables (also dictionaries) on the side.

They have clearly outgrown these simple data structures.  There's lots
of rummaging around in dictionaries, and information is recomputed on
the fly.  For the work I'm going to do, I want more clearly defined
and more convenient interfaces.

Going forward, I also want less coupling between the back-ends and the
syntax tree, to make messing with the syntax easier.

Create a bunch of classes to represent QAPI schemata.

Have the QAPISchema initializer call the parser, then walk the syntax
tree to create the new internal representation, and finally perform
semantic analysis.

Shortcut: the semantic analysis still relies on existing check_exprs()
to do the actual semantic checking.  All this code needs to move into
the classes.  Mark as TODO.

Simple unions are lowered to flat unions.  Flat unions and structs are
represented as a more general object type.

Catching name collisions in generated code would be nice.  Mark as
TODO.

We generate array types eagerly, even though most of them aren't used.
Mark as TODO.

Nothing uses the new intermediate representation just yet, thus no
change to generated files.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
Reviewed-by: Daniel P. Berrange 
---
 scripts/qapi-commands.py   |   2 +-
 scripts/qapi-event.py  |   2 +-
 scripts/qapi-types.py  |   2 +-
 scripts/qapi-visit.py  |   2 +-
 scripts/qapi.py| 379 -
 tests/qapi-schema/test-qapi.py |   2 +-
 6 files changed, 375 insertions(+), 14 deletions(-)

diff --git a/scripts/qapi-commands.py b/scripts/qapi-commands.py
index 890ce5d..12bdc4c 100644
--- a/scripts/qapi-commands.py
+++ b/scripts/qapi-commands.py
@@ -272,7 +272,7 @@ for o, a in opts:
 if o in ("-m", "--middle"):
 middle_mode = True
 
-exprs = parse_schema(input_file)
+exprs = QAPISchema(input_file).get_exprs()
 commands = filter(lambda expr: expr.has_key('command'), exprs)
 commands = filter(lambda expr: not expr.has_key('gen'), commands)
 
diff --git a/scripts/qapi-event.py b/scripts/qapi-event.py
index 7f238df..aec2d32 100644
--- a/scripts/qapi-event.py
+++ b/scripts/qapi-event.py
@@ -263,7 +263,7 @@ fdecl.write(mcgen('''
 ''',
   prefix=prefix))
 
-exprs = parse_schema(input_file)
+exprs = QAPISchema(input_file).get_exprs()
 
 event_enum_name = c_name(prefix + "QAPIEvent", protect=False)
 event_enum_values = []
diff --git a/scripts/qapi-types.py b/scripts/qapi-types.py
index f2428f3..d162ca2 100644
--- a/scripts/qapi-types.py
+++ b/scripts/qapi-types.py
@@ -336,7 +336,7 @@ fdecl.write(mcgen('''
 #include 
 '''))
 
-exprs = parse_schema(input_file)
+exprs = QAPISchema(input_file).get_exprs()
 
 fdecl.write(guardstart("QAPI_TYPES_BUILTIN_STRUCT_DECL"))
 for typename in builtin_types.keys():
diff --git a/scripts/qapi-visit.py b/scripts/qapi-visit.py
index 3cd662b..c493964 100644
--- a/scripts/qapi-visit.py
+++ b/scripts/qapi-visit.py
@@ -446,7 +446,7 @@ fdecl.write(mcgen('''
 ''',
   prefix=prefix))
 
-exprs = parse_schema(input_file)
+exprs = QAPISchema(input_file).get_exprs()
 
 # to avoid header dependency hell, we always generate declarations
 # for built-in types in our header files and simply guard them
diff --git a/scripts/qapi.py b/scripts/qapi.py
index a0165dd..6c7d37b 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -302,6 +302,8 @@ class QAPISchemaParser(object):
 
 #
 # Semantic analysis of schema expressions
+# TODO fold into QAPISchema
+# TODO catching name collisions in generated code would be nice
 #
 
 def find_base_fields(base):
@@ -747,15 +749,374 @@ def check_exprs(exprs):
 else:
 assert False, 'unexpected meta type'
 
-return map(lambda expr_elem: expr_elem['expr'], exprs)
-
-def parse_schema(fname):
-try:
-schema = QAPISchemaParser(open(fname, "r"))
-return check_exprs(schema.exprs)
-except (QAPISchemaError, QAPIExprError), e:
-print >>sys.stderr, e
-exit(1)
+return exprs
+
+
+#
+# Schema compiler frontend
+#
+
+class QAPISchemaEntity(object):
+def __init__(self, name, info):
+assert isinstance(name, str)
+self.name = name
+self.info = info
+
+def check(self, schema):
+pass
+
+
+class QAPISchemaType(QAPISchemaEntity):
+pass
+
+
+class QAPISchemaBuiltinType(QAPISchemaType):
+def __init__(self, name):
+QAPISchemaType.__init__(self, name, None)
+
+
+class QAPISchemaEnumType(QAPISchemaType):
+def __init__(self, name, info, values):
+QAPISchemaType.__init__(self, name, info)
+for v in values:
+assert isinstance(v, str)
+self.values = values
+
+def check(self, schema):
+assert len(set(self.values)) == len(self.values)
+
+
+class QAPISchemaArrayType(QAPISchemaType):
+def __init__(self, name, info, eleme

[Qemu-devel] [PATCH v6 03/26] qapi: QAPISchema code generation helper methods

[Markus Armbruster]

New methods c_name(), c_type(), c_null(), json_type(),
alternate_qtype().

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
Reviewed-by: Daniel P. Berrange 
---
 scripts/qapi.py | 93 -
 1 file changed, 86 insertions(+), 7 deletions(-)

diff --git a/scripts/qapi.py b/scripts/qapi.py
index 6c7d37b..0f6d475 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -762,17 +762,57 @@ class QAPISchemaEntity(object):
 self.name = name
 self.info = info
 
+def c_name(self):
+return c_name(self.name)
+
 def check(self, schema):
 pass
 
 
 class QAPISchemaType(QAPISchemaEntity):
-pass
+def c_type(self, is_param=False):
+return c_name(self.name) + pointer_suffix
+
+def c_null(self):
+return 'NULL'
+
+def json_type(self):
+pass
+
+def alternate_qtype(self):
+json2qtype = {
+'string':  'QTYPE_QSTRING',
+'number':  'QTYPE_QFLOAT',
+'int': 'QTYPE_QINT',
+'boolean': 'QTYPE_QBOOL',
+'object':  'QTYPE_QDICT'
+}
+return json2qtype.get(self.json_type())
 
 
 class QAPISchemaBuiltinType(QAPISchemaType):
-def __init__(self, name):
+def __init__(self, name, json_type, c_type, c_null):
 QAPISchemaType.__init__(self, name, None)
+assert not c_type or isinstance(c_type, str)
+assert json_type in ('string', 'number', 'int', 'boolean', 'null',
+ 'value')
+self._json_type_name = json_type
+self._c_type_name = c_type
+self._c_null_val = c_null
+
+def c_name(self):
+return self.name
+
+def c_type(self, is_param=False):
+if is_param and self.name == 'str':
+return 'const ' + self._c_type_name
+return self._c_type_name
+
+def c_null(self):
+return self._c_null_val
+
+def json_type(self):
+return self._json_type_name
 
 
 class QAPISchemaEnumType(QAPISchemaType):
@@ -785,6 +825,15 @@ class QAPISchemaEnumType(QAPISchemaType):
 def check(self, schema):
 assert len(set(self.values)) == len(self.values)
 
+def c_type(self, is_param=False):
+return c_name(self.name)
+
+def c_null(self):
+return c_enum_const(self.name, (self.values + ['MAX'])[0])
+
+def json_type(self):
+return 'string'
+
 
 class QAPISchemaArrayType(QAPISchemaType):
 def __init__(self, name, info, element_type):
@@ -797,6 +846,9 @@ class QAPISchemaArrayType(QAPISchemaType):
 self.element_type = schema.lookup_type(self._element_type_name)
 assert self.element_type
 
+def json_type(self):
+return 'array'
+
 
 class QAPISchemaObjectType(QAPISchemaType):
 def __init__(self, name, info, base, local_members, variants):
@@ -834,6 +886,17 @@ class QAPISchemaObjectType(QAPISchemaType):
 self.variants.check(schema, members, seen)
 self.members = members
 
+def c_name(self):
+assert self.info
+return QAPISchemaType.c_name(self)
+
+def c_type(self, is_param=False):
+assert self.info
+return QAPISchemaType.c_type(self)
+
+def json_type(self):
+return 'object'
+
 
 class QAPISchemaObjectTypeMember(object):
 def __init__(self, name, typ, optional):
@@ -898,6 +961,9 @@ class QAPISchemaAlternateType(QAPISchemaType):
 def check(self, schema):
 self.variants.check(schema, [], {})
 
+def json_type(self):
+return 'value'
+
 
 class QAPISchemaCommand(QAPISchemaEntity):
 def __init__(self, name, info, arg_type, ret_type, gen, success_response):
@@ -963,15 +1029,28 @@ class QAPISchema(object):
 def lookup_type(self, name):
 return self.lookup_entity(name, QAPISchemaType)
 
-def _def_builtin_type(self, name):
-self._def_entity(QAPISchemaBuiltinType(name))
+def _def_builtin_type(self, name, json_type, c_type, c_null):
+self._def_entity(QAPISchemaBuiltinType(name, json_type,
+   c_type, c_null))
 if name != '**':
 self._make_array_type(name) # TODO really needed?
 
 def _def_predefineds(self):
-for t in ['str', 'number', 'int', 'int8', 'int16', 'int32', 'int64',
-  'uint8', 'uint16', 'uint32', 'uint64', 'size', 'bool', '**']:
-self._def_builtin_type(t)
+for t in [('str','string',  'char' + pointer_suffix, 'NULL'),
+  ('number', 'number',  'double',   '0'),
+  ('int','int', 'int64_t',  '0'),
+  ('int8',   'int', 'int8_t',   '0'),
+  ('int16',  'int', 'int16_t',  '0'),
+  ('int32',  'int', 'int32_t',  '0'),
+  ('int64',  'int', 'int64_t',  '0'),
+  ('uint8',  'int', 'uint8_t',  '0'),
+  ('uint16', 'int', 'uint16_t', '0'),
+ 

[Qemu-devel] [PATCH v6 01/26] qapi: Rename class QAPISchema to QAPISchemaParser

[Markus Armbruster]

I want to name a new class QAPISchema.

While there, make it a new-style class.

Signed-off-by: Markus Armbruster 
Reviewed-by: Eric Blake 
Reviewed-by: Daniel P. Berrange 
---
 scripts/qapi.py | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/scripts/qapi.py b/scripts/qapi.py
index 817d824..a0165dd 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -103,7 +103,7 @@ class QAPIExprError(Exception):
 return error_path(self.info['parent']) + \
 "%s:%d: %s" % (self.info['file'], self.info['line'], self.msg)
 
-class QAPISchema:
+class QAPISchemaParser(object):
 
 def __init__(self, fp, previously_included = [], incl_info = None):
 abs_fname = os.path.abspath(fp.name)
@@ -149,8 +149,8 @@ class QAPISchema:
 except IOError, e:
 raise QAPIExprError(expr_info,
 '%s: %s' % (e.strerror, include))
-exprs_include = QAPISchema(fobj, previously_included,
-   expr_info)
+exprs_include = QAPISchemaParser(fobj, previously_included,
+ expr_info)
 self.exprs.extend(exprs_include.exprs)
 else:
 expr_elem = {'expr': expr,
@@ -751,7 +751,7 @@ def check_exprs(exprs):
 
 def parse_schema(fname):
 try:
-schema = QAPISchema(open(fname, "r"))
+schema = QAPISchemaParser(open(fname, "r"))
 return check_exprs(schema.exprs)
 except (QAPISchemaError, QAPIExprError), e:
 print >>sys.stderr, e
-- 
2.4.3

Re: [Qemu-devel] [PATCH 19/23] userfaultfd: activate syscall

[Dr. David Alan Gilbert]

* Bharata B Rao (bhar...@linux.vnet.ibm.com) wrote:
> (cc trimmed since this looks like an issue that is contained within QEMU)
> 
> On Tue, Sep 08, 2015 at 03:13:56PM +0100, Dr. David Alan Gilbert wrote:
> > * Bharata B Rao (bhar...@linux.vnet.ibm.com) wrote:
> > > On Tue, Sep 08, 2015 at 01:46:52PM +0100, Dr. David Alan Gilbert wrote:
> > > > * Bharata B Rao (bhar...@linux.vnet.ibm.com) wrote:
> > > > > On Tue, Sep 08, 2015 at 09:59:47AM +0100, Dr. David Alan Gilbert 
> > > > > wrote:
> > > > > > * Bharata B Rao (bhar...@linux.vnet.ibm.com) wrote:
> > > > > > > In fact I had successfully done postcopy migration of sPAPR guest 
> > > > > > > with
> > > > > > > this setup.
> > > > > > 
> > > > > > Interesting - I'd not got that far myself on power; I was hitting a 
> > > > > > problem
> > > > > > loading htab ( htab_load() bad index 2113929216 (14848+0 entries) 
> > > > > > in htab stream (htab_shift=25) )
> > > > > > 
> > > > > > Did you have to make any changes to the qemu code to get that happy?
> > > > > 
> > > > > I should have mentioned that I tried only QEMU driven migration within
> > > > > the same host using wp3-postcopy branch of your tree. I don't see the
> > > > > above issue.
> > > > > 
> > > > > (qemu) info migrate
> > > > > capabilities: xbzrle: off rdma-pin-all: off auto-converge: off 
> > > > > zero-blocks: off compress: off x-postcopy-ram: on 
> > > > > Migration status: completed
> > > > > total time: 39432 milliseconds
> > > > > downtime: 162 milliseconds
> > > > > setup: 14 milliseconds
> > > > > transferred ram: 1297209 kbytes
> > > > > throughput: 270.72 mbps
> > > > > remaining ram: 0 kbytes
> > > > > total ram: 4194560 kbytes
> > > > > duplicate: 734015 pages
> > > > > skipped: 0 pages
> > > > > normal: 318469 pages
> > > > > normal bytes: 1273876 kbytes
> > > > > dirty sync count: 4
> > > > > 
> > > > > I will try migration between different hosts soon and check.
> > > > 
> > > > I hit that on the same host; are you sure you've switched into postcopy 
> > > > mode;
> > > > i.e. issued a migrate_start_postcopy before the end of migration?
> > > 
> > > Sorry I was following your discussion with Li in this thread
> > > 
> > > https://www.marc.info/?l=qemu-devel&m=143035620026744&w=4
> > > 
> > > and it wasn't obvious to me that anything apart from turning on the
> > > x-postcopy-ram capability was required :(
> > 
> > OK.
> > 
> > > So I do see the problem now.
> > > 
> > > At the source
> > > -
> > > Error reading data from KVM HTAB fd: Bad file descriptor
> > > Segmentation fault
> > > 
> > > At the target
> > > -
> > > htab_load() bad index 2113929216 (14336+0 entries) in htab stream 
> > > (htab_shift=25)
> > > qemu-system-ppc64: error while loading state section id 56(spapr/htab)
> > > qemu-system-ppc64: postcopy_ram_listen_thread: loadvm failed: -22
> > > qemu-system-ppc64: VQ 0 size 0x100 Guest index 0x0 inconsistent with Host 
> > > index 0x1f: delta 0xffe1
> > > qemu-system-ppc64: error while loading state for instance 0x0 of device 
> > > 'pci@8002000:00.0/virtio-net'
> > > *** Error in `./ppc64-softmmu/qemu-system-ppc64': corrupted double-linked 
> > > list: 0x0100241234a0 ***
> > > === Backtrace: =
> > > /lib64/power8/libc.so.6Segmentation fault
> > 
> > Good - my current world has got rid of the segfaults/corruption in the 
> > cleanup on power - but those
> > are only after it stumbled over the htab problem.
> > 
> > I don't know the innards of power/htab, so if you've got any pointers on 
> > what upset it
> > I'd be happy for some pointers.
>  
> When migrate_start_postcopy is issued, for HTAB, the SaveStateEntry
> save_live_iterate call is coming after save_live_complete. In case of HTAB,
> the spapr->htab_fd is closed when HTAB saving is completed in
> save_live_complete handler. When save_live_iterate call comes after this,
> we end up accessing an invalid fd resulting in the migration failure
> we are seeing here.

Ah interesting.

> - With postcopy migration, is it expected to get a save_live_iterate
>   call after save_live_complete ? IIUC, save_live_complete signals the
>   completion of the saving. Is save_live_iterate handler expected to
>   handle this condition ?

OK, thanks for spotting that; Power's htab was the only other iterative
device running at the same time.
I'll change it to avoid calling save_live_iterate during postcopy on devices
that haven't declared a save_live_complete_postcopy handler.

> 
> I am able to get past this failure and get migration to complete successfully
> by this below hack where I teach save_live_iterate handler to ignore
> the requests after save_live_complete has been called.

Excellent; but as stated, I think it's better if I fix it and then the
individual devices don't need to change.

Thanks again,

Dave

> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index 2f8155d..550e234 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -1236,6 +1236,11 @@ static int htab_sav

Re: [Qemu-devel] [PATCH v5 01/14] qapi: Add transaction support to block-dirty-bitmap operations

[Eric Blake]

On 09/07/2015 01:34 AM, Fam Zheng wrote:
> From: John Snow 
> 
> This adds two qmp commands to transactions.
> 
> block-dirty-bitmap-add allows you to create a bitmap simultaneously
> alongside a new full backup to accomplish a clean synchronization
> point.
> 
> block-dirty-bitmap-clear allows you to reset a bitmap back to as-if
> it were new, which can also be used alongside a full backup to
> accomplish a clean synchronization point.
> 
> Signed-off-by: Fam Zheng 
> Signed-off-by: John Snow 
> Reviewed-by: Max Reitz 
> Reviewed-by: Stefan Hajnoczi 
> Signed-off-by: Stefan Hajnoczi 
> Signed-off-by: Fam Zheng 

Odd to have a double S-o-B.

Reviewed-by: Eric Blake 

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

[Qemu-devel] [PATCH v2 2/4] qga: do not override configuration verbosity

[marcandre . lureau]

From: Marc-André Lureau 

Move the default verbosity settings before loading the configuration
file, or it will overwrite it. Found thanks to writing qga tests :)

Signed-off-by: Marc-André Lureau 
---
 qga/main.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/qga/main.c b/qga/main.c
index 18e1e1d..aa6a063 100644
--- a/qga/main.c
+++ b/qga/main.c
@@ -1083,8 +1083,6 @@ static void config_parse(GAConfig *config, int argc, char 
**argv)
 { NULL, 0, NULL, 0 }
 };
 
-config->log_level = G_LOG_LEVEL_ERROR | G_LOG_LEVEL_CRITICAL;
-
 while ((ch = getopt_long(argc, argv, sopt, lopt, &opt_ind)) != -1) {
 switch (ch) {
 case 'm':
@@ -1332,6 +1330,8 @@ int main(int argc, char **argv)
 GAState *s = g_new0(GAState, 1);
 GAConfig *config = g_new0(GAConfig, 1);
 
+config->log_level = G_LOG_LEVEL_ERROR | G_LOG_LEVEL_CRITICAL;
+
 module_call_init(MODULE_INIT_QAPI);
 
 init_dfl_pathnames();
-- 
2.4.3

[Qemu-devel] [PATCH v2 4/4] tests: add a local test for guest agent

[marcandre . lureau]

From: Marc-André Lureau 

Add some local guest agent tests (as it is better than nothing) only
when CONFIG_LINUX.

They can be run inside or outside a VM, when run inside a VM, they will
do a bit more side effects, such as freezing/thawing the FS or changing
the time.

A better test would involve a managed VM (or container), but it might be
better to leave that off to autotest/avocado.

Signed-off-by: Marc-André Lureau 
---
 tests/Makefile   |   3 +
 tests/test-qga.c | 791 +++
 2 files changed, 794 insertions(+)
 create mode 100644 tests/test-qga.c

diff --git a/tests/Makefile b/tests/Makefile
index 34c6136..d5837a4 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -76,6 +76,7 @@ check-unit-y += tests/test-write-threshold$(EXESUF)
 gcov-files-test-write-threshold-y = block/write-threshold.c
 check-unit-$(CONFIG_GNUTLS_HASH) += tests/test-crypto-hash$(EXESUF)
 check-unit-y += tests/test-crypto-cipher$(EXESUF)
+check-unit-$(CONFIG_LINUX) += tests/test-qga$(EXESUF)
 
 check-block-$(CONFIG_POSIX) += tests/qemu-iotests-quick.sh
 
@@ -432,6 +433,8 @@ endif
 qtest-obj-y = tests/libqtest.o libqemuutil.a libqemustub.a
 $(check-qtest-y): $(qtest-obj-y)
 
+tests/test-qga: tests/test-qga.o $(qtest-obj-y)
+
 .PHONY: check-help
 check-help:
@echo "Regression testing targets:"
diff --git a/tests/test-qga.c b/tests/test-qga.c
new file mode 100644
index 000..cfee134
--- /dev/null
+++ b/tests/test-qga.c
@@ -0,0 +1,791 @@
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include "libqtest.h"
+#include "config-host.h"
+
+typedef struct {
+char *test_dir;
+GMainLoop *loop;
+int fd;
+GPid pid;
+} TestFixture;
+
+static int connect_qga(char *path)
+{
+int s, ret, len, i = 0;
+struct sockaddr_un remote;
+
+s = socket(AF_UNIX, SOCK_STREAM, 0);
+g_assert(s != -1);
+
+remote.sun_family = AF_UNIX;
+do {
+strcpy(remote.sun_path, path);
+len = strlen(remote.sun_path) + sizeof(remote.sun_family);
+ret = connect(s, (struct sockaddr *)&remote, len);
+if (ret == -1) {
+g_usleep(G_USEC_PER_SEC);
+}
+if (i++ == 5) {
+return -1;
+}
+} while (ret == -1);
+
+return s;
+}
+
+static void qga_watch(GPid pid, gint status, gpointer user_data)
+{
+TestFixture *fixture = user_data;
+
+g_assert_cmpint(status, ==, 0);
+g_main_loop_quit(fixture->loop);
+}
+
+static void
+fixture_setup(TestFixture *fixture, gconstpointer data)
+{
+const gchar *extra_arg = data;
+GError *error = NULL;
+gchar *cwd, *path, *cmd, **argv = NULL;
+
+fixture->loop = g_main_loop_new(NULL, FALSE);
+
+fixture->test_dir = g_strdup("/tmp/qgatest.XX");
+g_assert_nonnull(g_mkdtemp(fixture->test_dir));
+
+path = g_build_filename(fixture->test_dir, "sock", NULL);
+cwd = g_get_current_dir();
+cmd = g_strdup_printf("%s%cqemu-ga -m unix-listen -t %s -p %s %s %s",
+  cwd, G_DIR_SEPARATOR,
+  fixture->test_dir, path,
+  getenv("QTEST_LOG") ? "-v" : "",
+  extra_arg ?: "");
+g_shell_parse_argv(cmd, NULL, &argv, &error);
+g_assert_no_error(error);
+
+g_spawn_async(fixture->test_dir, argv, NULL,
+  G_SPAWN_SEARCH_PATH|G_SPAWN_DO_NOT_REAP_CHILD,
+  NULL, NULL, &fixture->pid, &error);
+g_assert_no_error(error);
+
+g_child_watch_add(fixture->pid, qga_watch, fixture);
+
+fixture->fd = connect_qga(path);
+
+g_strfreev(argv);
+g_free(cmd);
+g_free(cwd);
+g_free(path);
+}
+
+static void
+fixture_tear_down(TestFixture *fixture, gconstpointer data)
+{
+gchar *tmp;
+
+kill(fixture->pid, SIGTERM);
+
+g_main_loop_run(fixture->loop);
+g_main_loop_unref(fixture->loop);
+
+g_spawn_close_pid(fixture->pid);
+
+tmp = g_build_filename(fixture->test_dir, "foo", NULL);
+g_unlink(tmp);
+g_free(tmp);
+
+tmp = g_build_filename(fixture->test_dir, "qga.state", NULL);
+g_unlink(tmp);
+g_free(tmp);
+
+tmp = g_build_filename(fixture->test_dir, "sock", NULL);
+g_unlink(tmp);
+g_free(tmp);
+
+g_rmdir(fixture->test_dir);
+g_free(fixture->test_dir);
+}
+
+static void qmp_assertion_message_error(const char *domain,
+const char *file,
+int line,
+const char *func,
+const char *expr,
+QDict  *dict)
+{
+const char *class, *desc;
+char *s;
+QDict *error;
+
+error = qdict_get_qdict(dict, "error");
+class = qdict_get_try_str(error, "class");
+desc = qdict_get_try_str(error, "desc");
+
+s = g_strdup_printf("assertion failed %s: %s %s"

[Qemu-devel] [PATCH v2 0/4] qga: add local tests on linux

[marcandre . lureau]

From: Marc-André Lureau 

v1->v2:
- do not loop forever when connection to qga (Michael Roth)
- add some config file tests
- fix config verbosity
- swallow virt-what stderr
- only compile and check on Linux

Marc-André Lureau (4):
  qga: add QGA_CONF environment variable
  qga: do not override configuration verbosity
  qtest: add a few fd-level qmp helpers
  tests: add a local test for guest agent

 qga/main.c   |   7 +-
 tests/Makefile   |   3 +
 tests/libqtest.c |  45 +++-
 tests/libqtest.h |   7 +
 tests/test-qga.c | 791 +++
 5 files changed, 846 insertions(+), 7 deletions(-)
 create mode 100644 tests/test-qga.c

-- 
2.4.3

[Qemu-devel] [PATCH v2 1/4] qga: add QGA_CONF environment variable

[marcandre . lureau]

From: Marc-André Lureau 

Having a environment variable allows to override default configuration
path, useful for testing. Note that this can't easily be an argument,
since loading config is done before parsing the arguments.

Signed-off-by: Marc-André Lureau 
---
 qga/main.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/qga/main.c b/qga/main.c
index d8e063a..18e1e1d 100644
--- a/qga/main.c
+++ b/qga/main.c
@@ -945,10 +945,11 @@ static void config_load(GAConfig *config)
 {
 GError *gerr = NULL;
 GKeyFile *keyfile;
+const char *conf = g_getenv("QGA_CONF") ?: QGA_CONF_DEFAULT;
 
 /* read system config */
 keyfile = g_key_file_new();
-if (!g_key_file_load_from_file(keyfile, QGA_CONF_DEFAULT, 0, &gerr)) {
+if (!g_key_file_load_from_file(keyfile, conf, 0, &gerr)) {
 goto end;
 }
 if (g_key_file_has_key(keyfile, "general", "daemon", NULL)) {
-- 
2.4.3

[Qemu-devel] [PATCH v2 3/4] qtest: add a few fd-level qmp helpers

[marcandre . lureau]

From: Marc-André Lureau 

Add a few functions to interact with qmp via a simple fd.

Signed-off-by: Marc-André Lureau 
Reviewed-by: Michael Roth 
---
 tests/libqtest.c | 45 +
 tests/libqtest.h |  7 +++
 2 files changed, 48 insertions(+), 4 deletions(-)

diff --git a/tests/libqtest.c b/tests/libqtest.c
index e5188e0..11e541d 100644
--- a/tests/libqtest.c
+++ b/tests/libqtest.c
@@ -357,7 +357,7 @@ static void qmp_response(JSONMessageParser *parser, QList 
*tokens)
 qmp->response = (QDict *)obj;
 }
 
-QDict *qtest_qmp_receive(QTestState *s)
+QDict *qmp_fd_receive(int fd)
 {
 QMPResponseParser qmp;
 bool log = getenv("QTEST_LOG") != NULL;
@@ -368,7 +368,7 @@ QDict *qtest_qmp_receive(QTestState *s)
 ssize_t len;
 char c;
 
-len = read(s->qmp_fd, &c, 1);
+len = read(fd, &c, 1);
 if (len == -1 && errno == EINTR) {
 continue;
 }
@@ -388,12 +388,17 @@ QDict *qtest_qmp_receive(QTestState *s)
 return qmp.response;
 }
 
+QDict *qtest_qmp_receive(QTestState *s)
+{
+return qmp_fd_receive(s->qmp_fd);
+}
+
 /**
  * Allow users to send a message without waiting for the reply,
  * in the case that they choose to discard all replies up until
  * a particular EVENT is received.
  */
-void qtest_async_qmpv(QTestState *s, const char *fmt, va_list ap)
+void qmp_fd_sendv(int fd, const char *fmt, va_list ap)
 {
 va_list ap_copy;
 QObject *qobj;
@@ -417,13 +422,25 @@ void qtest_async_qmpv(QTestState *s, const char *fmt, 
va_list ap)
 fprintf(stderr, "%s", str);
 }
 /* Send QMP request */
-socket_send(s->qmp_fd, str, size);
+socket_send(fd, str, size);
 
 QDECREF(qstr);
 qobject_decref(qobj);
 }
 }
 
+void qtest_async_qmpv(QTestState *s, const char *fmt, va_list ap)
+{
+qmp_fd_sendv(s->qmp_fd, fmt, ap);
+}
+
+QDict *qmp_fdv(int fd, const char *fmt, va_list ap)
+{
+qmp_fd_sendv(fd, fmt, ap);
+
+return qmp_fd_receive(fd);
+}
+
 QDict *qtest_qmpv(QTestState *s, const char *fmt, va_list ap)
 {
 qtest_async_qmpv(s, fmt, ap);
@@ -432,6 +449,26 @@ QDict *qtest_qmpv(QTestState *s, const char *fmt, va_list 
ap)
 return qtest_qmp_receive(s);
 }
 
+QDict *qmp_fd(int fd, const char *fmt, ...)
+{
+va_list ap;
+QDict *response;
+
+va_start(ap, fmt);
+response = qmp_fdv(fd, fmt, ap);
+va_end(ap);
+return response;
+}
+
+void qmp_fd_send(int fd, const char *fmt, ...)
+{
+va_list ap;
+
+va_start(ap, fmt);
+qmp_fd_sendv(fd, fmt, ap);
+va_end(ap);
+}
+
 QDict *qtest_qmp(QTestState *s, const char *fmt, ...)
 {
 va_list ap;
diff --git a/tests/libqtest.h b/tests/libqtest.h
index ec42031..ecd9872 100644
--- a/tests/libqtest.h
+++ b/tests/libqtest.h
@@ -818,4 +818,11 @@ static inline int64_t clock_set(int64_t val)
  */
 bool qtest_big_endian(void);
 
+
+QDict *qmp_fd_receive(int fd);
+void qmp_fd_sendv(int fd, const char *fmt, va_list ap);
+void qmp_fd_send(int fd, const char *fmt, ...);
+QDict *qmp_fdv(int fd, const char *fmt, va_list ap);
+QDict *qmp_fd(int fd, const char *fmt, ...);
+
 #endif
-- 
2.4.3

Re: [Qemu-devel] [PATCH v5 09/14] block: Add block job transactions

[Max Reitz]

On 07.09.2015 09:34, Fam Zheng wrote:
> Sometimes block jobs must execute as a transaction group.  Finishing
> jobs wait until all other jobs are ready to complete successfully.
> Failure or cancellation of one job cancels the other jobs in the group.
> 
> Signed-off-by: Stefan Hajnoczi 
> [Rewrite the implementation which is now contained in block_job_completed.
> --Fam]
> Signed-off-by: Fam Zheng 
> ---
>  blockjob.c   | 134 
> ++-
>  include/block/block.h|   1 +
>  include/block/blockjob.h |  38 ++
>  3 files changed, 171 insertions(+), 2 deletions(-)
> 
> diff --git a/blockjob.c b/blockjob.c
> index 36c18e0..1b4eb20 100644
> --- a/blockjob.c
> +++ b/blockjob.c
> @@ -36,6 +36,19 @@
>  #include "qemu/timer.h"
>  #include "qapi-event.h"
>  
> +/* Transactional group of block jobs */
> +struct BlockJobTxn {
> +
> +/* Is this txn being cancelled? */
> +bool aborting;
> +
> +/* List of jobs */
> +QLIST_HEAD(, BlockJob) jobs;
> +
> +/* Reference count */
> +int refcnt;
> +};
> +
>  void *block_job_create(const BlockJobDriver *driver, BlockDriverState *bs,
> int64_t speed, BlockCompletionFunc *cb,
> void *opaque, Error **errp)
> @@ -90,6 +103,85 @@ void block_job_unref(BlockJob *job)
>  }
>  }
>  
> +static void block_job_completed_single(BlockJob *job)
> +{
> +if (!job->ret) {
> +if (job->driver->commit) {
> +job->driver->commit(job);
> +}
> +} else {
> +if (job->driver->abort) {
> +job->driver->abort(job);
> +}
> +}
> +job->cb(job->opaque, job->ret);
> +block_job_unref(job);
> +}
> +
> +static void block_job_completed_txn_abort(BlockJob *job)
> +{
> +AioContext *ctx;
> +BlockJobTxn *txn = job->txn;
> +BlockJob *other_job, *next;
> +
> +if (txn->aborting) {
> +/*
> + * We are cancelled by another job, which will handle everything.
> + */

After having reviewed patch 11, I think we need to call
block_job_txn_unref(txn) here.

> +return;
> +}
> +txn->aborting = true;
> +/* We are the first failed job. Cancel other jobs. */
> +QLIST_FOREACH(other_job, &txn->jobs, txn_list) {
> +ctx = bdrv_get_aio_context(other_job->bs);
> +aio_context_acquire(ctx);
> +}
> +QLIST_FOREACH(other_job, &txn->jobs, txn_list) {
> +if (other_job == job || other_job->completed) {
> +/* Other jobs are "effectively" cancelled by us, set the status 
> for
> + * them; this job, however, may or may not be cancelled, 
> depending
> + * on the caller, so leave it. */
> +if (other_job != job) {
> +other_job->cancelled = true;
> +}
> +continue;
> +}
> +block_job_cancel_sync(other_job);
> +assert(other_job->completed);
> +}
> +QLIST_FOREACH_SAFE(other_job, &txn->jobs, txn_list, next) {
> +ctx = bdrv_get_aio_context(other_job->bs);
> +block_job_completed_single(other_job);
> +aio_context_release(ctx);
> +}
> +block_job_txn_unref(txn);
> +}
> +
> +static void block_job_completed_txn_success(BlockJob *job)
> +{
> +AioContext *ctx;
> +BlockJobTxn *txn = job->txn;
> +BlockJob *other_job, *next;
> +/*
> + * Successful completion, see if there are other running jobs in this
> + * txn.
> + **/
> +QLIST_FOREACH(other_job, &txn->jobs, txn_list) {
> +if (!other_job->completed) {

Same here.

Max

> +return;
> +}
> +}
> +/* We are the last completed job, commit the transaction. */
> +QLIST_FOREACH_SAFE(other_job, &txn->jobs, txn_list, next) {
> +ctx = bdrv_get_aio_context(other_job->bs);
> +aio_context_acquire(ctx);
> +assert(other_job->ret == 0);
> +block_job_completed_single(other_job);
> +aio_context_release(ctx);
> +}
> +block_job_txn_unref(txn);
> +}
> +
>  void block_job_completed(BlockJob *job, int ret)
>  {
>  BlockDriverState *bs = job->bs;
> @@ -98,8 +190,13 @@ void block_job_completed(BlockJob *job, int ret)
>  assert(!job->completed);
>  job->completed = true;
>  job->ret = ret;
> -job->cb(job->opaque, ret);
> -block_job_unref(job);
> +if (!job->txn) {
> +block_job_completed_single(job);
> +} else if (ret < 0 || block_job_is_cancelled(job)) {
> +block_job_completed_txn_abort(job);
> +} else {
> +block_job_completed_txn_success(job);
> +}
>  }
>  
>  void block_job_set_speed(BlockJob *job, int64_t speed, Error **errp)
> @@ -398,3 +495,36 @@ void block_job_defer_to_main_loop(BlockJob *job,
>  
>  qemu_bh_schedule(data->bh);
>  }
> +
> +BlockJobTxn *block_job_txn_new(void)
> +{
> +BlockJobTxn *txn = g_new0(BlockJobTxn, 1);
> +QLIST_INIT(&txn->jobs);
> +txn->refcnt = 1;
> +return txn;
> +

Re: [Qemu-devel] [PATCH v5 11/14] block/backup: support block job transactions

[Max Reitz]

On 07.09.2015 09:34, Fam Zheng wrote:
> From: Stefan Hajnoczi 
> 
> Join the transaction when the 'transactional-cancel' QMP argument is
> true.
> 
> This ensures that the sync bitmap is not thrown away if another block
> job in the transaction is cancelled or fails.  This is critical so
> incremental backup with multiple disks can be retried in case of
> cancellation/failure.
> 
> Signed-off-by: Stefan Hajnoczi 
> Signed-off-by: Fam Zheng 
> ---
>  block/backup.c|  25 +++--
>  blockdev.c| 139 
> --
>  hmp.c |   2 +-
>  include/block/block_int.h |   3 +-
>  qapi/block-core.json  |  16 +-
>  5 files changed, 148 insertions(+), 37 deletions(-)
> 
> diff --git a/block/backup.c b/block/backup.c
> index 9776d9c..3a3dccc 100644
> --- a/block/backup.c
> +++ b/block/backup.c

[snip]

> @@ -545,6 +559,7 @@ void backup_start(BlockDriverState *bs, BlockDriverState 
> *target,
> sync_bitmap : NULL;
>  job->common.len = len;
>  job->common.co = qemu_coroutine_create(backup_run);
> +block_job_txn_add_job(txn, &job->common);

You're calling this for every single job, so the reference count of txn
will probably be job count + 1. However, block_job_txn_unref() is only
called for one block job by block_job_completed_txn_{abort,success}(). I
think you need to call block_job_txn_unref() for every completed block
job, otherwise it will be leaked if there is more than a single job in
the txn.

Max

>  qemu_coroutine_enter(job->common.co, job);
>  return;
> 



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH] q35: Remove old machine versions

[Eduardo Habkost]

Ping?

So, what's the reason we are still keeping those old machines in the
code?


On Tue, Aug 18, 2015 at 04:11:42PM -0700, Eduardo Habkost wrote:
> Migration with q35 was not possible before commit
> 04329029a8c539eb5f75dcb6d8b016f0c53a031a, because q35 unconditionally creates
> an ich9-ahci device, that was marked as unmigratable. So all q35 machines
> before pc-q35-2.4 were unmigratable, and there's no point in keeping
> compatibility code for them.
> 
> Remove all old pc-q35 machine classes and keep only pc-q35-2.4.
> 
> Signed-off-by: Eduardo Habkost 
> ---
>  hw/i386/pc_q35.c | 153 
> ---
>  1 file changed, 153 deletions(-)
> 
> diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c
> index 4ee653e..e482f2f 100644
> --- a/hw/i386/pc_q35.c
> +++ b/hw/i386/pc_q35.c
> @@ -272,60 +272,6 @@ static void pc_q35_init(MachineState *machine)
>  }
>  }
>  
> -static void pc_compat_2_3(MachineState *machine)
> -{
> -PCMachineState *pcms = PC_MACHINE(machine);
> -savevm_skip_section_footers();
> -if (kvm_enabled()) {
> -pcms->smm = ON_OFF_AUTO_OFF;
> -}
> -global_state_set_optional();
> -savevm_skip_configuration();
> -}
> -
> -static void pc_compat_2_2(MachineState *machine)
> -{
> -pc_compat_2_3(machine);
> -machine->suppress_vmdesc = true;
> -}
> -
> -static void pc_compat_2_1(MachineState *machine)
> -{
> -PCMachineState *pcms = PC_MACHINE(machine);
> -
> -pc_compat_2_2(machine);
> -pcms->enforce_aligned_dimm = false;
> -x86_cpu_compat_kvm_no_autodisable(FEAT_8000_0001_ECX, CPUID_EXT3_SVM);
> -}
> -
> -static void pc_compat_2_0(MachineState *machine)
> -{
> -pc_compat_2_1(machine);
> -}
> -
> -static void pc_compat_1_7(MachineState *machine)
> -{
> -pc_compat_2_0(machine);
> -option_rom_has_mr = true;
> -x86_cpu_compat_kvm_no_autoenable(FEAT_1_ECX, CPUID_EXT_X2APIC);
> -}
> -
> -static void pc_compat_1_6(MachineState *machine)
> -{
> -pc_compat_1_7(machine);
> -rom_file_has_mr = false;
> -}
> -
> -static void pc_compat_1_5(MachineState *machine)
> -{
> -pc_compat_1_6(machine);
> -}
> -
> -static void pc_compat_1_4(MachineState *machine)
> -{
> -pc_compat_1_5(machine);
> -}
> -
>  #define DEFINE_Q35_MACHINE(suffix, name, compatfn, optionfn) \
>  static void pc_init_##suffix(MachineState *machine) \
>  { \
> @@ -358,102 +304,3 @@ static void pc_q35_2_4_machine_options(MachineClass *m)
>  
>  DEFINE_Q35_MACHINE(v2_4, "pc-q35-2.4", NULL,
> pc_q35_2_4_machine_options);
> -
> -
> -static void pc_q35_2_3_machine_options(MachineClass *m)
> -{
> -pc_q35_2_4_machine_options(m);
> -m->no_floppy = 0;
> -m->no_tco = 1;
> -m->alias = NULL;
> -SET_MACHINE_COMPAT(m, PC_COMPAT_2_3);
> -}
> -
> -DEFINE_Q35_MACHINE(v2_3, "pc-q35-2.3", pc_compat_2_3,
> -   pc_q35_2_3_machine_options);
> -
> -
> -static void pc_q35_2_2_machine_options(MachineClass *m)
> -{
> -PCMachineClass *pcmc = PC_MACHINE_CLASS(m);
> -pc_q35_2_3_machine_options(m);
> -SET_MACHINE_COMPAT(m, PC_COMPAT_2_2);
> -pcmc->rsdp_in_ram = false;
> -}
> -
> -DEFINE_Q35_MACHINE(v2_2, "pc-q35-2.2", pc_compat_2_2,
> -   pc_q35_2_2_machine_options);
> -
> -
> -static void pc_q35_2_1_machine_options(MachineClass *m)
> -{
> -PCMachineClass *pcmc = PC_MACHINE_CLASS(m);
> -pc_q35_2_2_machine_options(m);
> -m->default_display = NULL;
> -SET_MACHINE_COMPAT(m, PC_COMPAT_2_1);
> -pcmc->smbios_uuid_encoded = false;
> -}
> -
> -DEFINE_Q35_MACHINE(v2_1, "pc-q35-2.1", pc_compat_2_1,
> -   pc_q35_2_1_machine_options);
> -
> -
> -static void pc_q35_2_0_machine_options(MachineClass *m)
> -{
> -PCMachineClass *pcmc = PC_MACHINE_CLASS(m);
> -pc_q35_2_1_machine_options(m);
> -SET_MACHINE_COMPAT(m, PC_COMPAT_2_0);
> -pcmc->has_reserved_memory = false;
> -pcmc->smbios_legacy_mode = true;
> -pcmc->acpi_data_size = 0x1;
> -}
> -
> -DEFINE_Q35_MACHINE(v2_0, "pc-q35-2.0", pc_compat_2_0,
> -   pc_q35_2_0_machine_options);
> -
> -
> -static void pc_q35_1_7_machine_options(MachineClass *m)
> -{
> -PCMachineClass *pcmc = PC_MACHINE_CLASS(m);
> -pc_q35_2_0_machine_options(m);
> -m->default_machine_opts = NULL;
> -SET_MACHINE_COMPAT(m, PC_COMPAT_1_7);
> -pcmc->smbios_defaults = false;
> -pcmc->gigabyte_align = false;
> -}
> -
> -DEFINE_Q35_MACHINE(v1_7, "pc-q35-1.7", pc_compat_1_7,
> -   pc_q35_1_7_machine_options);
> -
> -
> -static void pc_q35_1_6_machine_options(MachineClass *m)
> -{
> -PCMachineClass *pcmc = PC_MACHINE_CLASS(m);
> -pc_q35_machine_options(m);
> -SET_MACHINE_COMPAT(m, PC_COMPAT_1_6);
> -pcmc->has_acpi_build = false;
> -}
> -
> -DEFINE_Q35_MACHINE(v1_6, "pc-q35-1.6", pc_compat_1_6,
> -   pc_q35_1_6_machine_options);
> -
> -
> -static void pc_q35_1_5_machine_options(MachineClass *m)
> -{
> -pc_q35_1_6_m

Re: [Qemu-devel] [PATCH v5 10/14] blockdev: make BlockJobTxn available to qmp 'transaction'

[Max Reitz]

On 07.09.2015 09:34, Fam Zheng wrote:
> From: Stefan Hajnoczi 
> 
> Provide a BlockJobTxn to actions executed in a qmp 'transaction'
> command.  This allows actions to make their block jobs either complete
> as a group or fail/cancel together.
> 
> The next patch adds the first user.
> 
> Signed-off-by: Stefan Hajnoczi 
> Reviewed-by: Fam Zheng 
> Reviewed-by: John Snow 

You can vouch for yourself, but can you vouch for John being fine with
the change from v4? ;-)

> Signed-off-by: Fam Zheng 
> ---
>  blockdev.c | 6 ++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/blockdev.c b/blockdev.c
> index 4ebded8..3d5e5cd 100644
> --- a/blockdev.c
> +++ b/blockdev.c
> @@ -1285,6 +1285,7 @@ typedef struct BlkActionOps {
>  struct BlkActionState {
>  TransactionAction *action;
>  const BlkActionOps *ops;
> +BlockJobTxn *block_job_txn;
>  QSIMPLEQ_ENTRY(BlkActionState) entry;
>  };
>  
> @@ -1889,12 +1890,15 @@ static const BlkActionOps actions[] = {
>  void qmp_transaction(TransactionActionList *dev_list, Error **errp)
>  {
>  TransactionActionList *dev_entry = dev_list;
> +BlockJobTxn *block_job_txn;
>  BlkActionState *state, *next;
>  Error *local_err = NULL;
>  
>  QSIMPLEQ_HEAD(snap_bdrv_states, BlkActionState) snap_bdrv_states;
>  QSIMPLEQ_INIT(&snap_bdrv_states);
>  
> +block_job_txn = block_job_txn_new();
> +
>  /* drain all i/o before any operations */
>  bdrv_drain_all();
>  
> @@ -1914,6 +1918,7 @@ void qmp_transaction(TransactionActionList *dev_list, 
> Error **errp)
>  state = g_malloc0(ops->instance_size);
>  state->ops = ops;
>  state->action = dev_info;
> +state->block_job_txn = block_job_txn;
>  QSIMPLEQ_INSERT_TAIL(&snap_bdrv_states, state, entry);
>  
>  state->ops->prepare(state, &local_err);
> @@ -1940,6 +1945,7 @@ delete_and_fail:
>  }
>  }
>  exit:
> +block_job_txn_unref(block_job_txn);
>  QSIMPLEQ_FOREACH_SAFE(state, &snap_bdrv_states, entry, next) {
>  if (state->ops->clean) {
>  state->ops->clean(state);
> 

I'd put the unref() call after the FOREACH loop, since the txn is
referenced by every state->block_job_txn, so the state->ops->clean()
operation may still try to access that field.

Max



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v5 09/14] block: Add block job transactions

[Max Reitz]

On 07.09.2015 09:34, Fam Zheng wrote:
> Sometimes block jobs must execute as a transaction group.  Finishing
> jobs wait until all other jobs are ready to complete successfully.
> Failure or cancellation of one job cancels the other jobs in the group.
> 
> Signed-off-by: Stefan Hajnoczi 
> [Rewrite the implementation which is now contained in block_job_completed.
> --Fam]
> Signed-off-by: Fam Zheng 
> ---
>  blockjob.c   | 134 
> ++-
>  include/block/block.h|   1 +
>  include/block/blockjob.h |  38 ++
>  3 files changed, 171 insertions(+), 2 deletions(-)
> 

[snip]

> diff --git a/include/block/block.h b/include/block/block.h
> index e619b99..f80d984 100644
> --- a/include/block/block.h
> +++ b/include/block/block.h

[snip]

> @@ -395,4 +398,39 @@ void block_job_defer_to_main_loop(BlockJob *job,
>BlockJobDeferToMainLoopFn *fn,
>void *opaque);
>  
> +/**
> + * block_job_txn_new:
> + *
> + * Allocate and return a new block job transaction.  Jobs can be added to the
> + * transaction using block_job_txn_add_job().
> + *
> + * The transaction is automatically freed when the last job completes or is
> + * cancelled.
> + *
> + * All jobs in the transaction either complete successfully or fail/cancel 
> as a
> + * group.  Jobs wait for each other before completing.  Cancelling one job
> + * cancels all jobs in the transaction.
> + */
> +BlockJobTxn *block_job_txn_new(void);
> +
> +/**
> + * block_job_txn_unref:
> + *
> + * Release a reference that was previously acquired with 
> block_job_txn_add_job
> + * or block_job_txn_new. If it's the last reference to the object, it will be
> + * freed.
> + */
> +void block_job_txn_unref(BlockJobTxn *txn);
> +
> +/**
> + * block_job_txn_add_job:
> + * @txn: The transaction (may be NULL)
> + * @job: Job to add to the transaction
> + *
> + * Add @job to the transaction.  The @job must not already be in a 
> transaction.
> + * The block job driver must call block_job_txn_prepare_to_complete() before
> + * final cleanup and completion.

This comment still needs a fixup (block_job_txn_prepare_to_complete()
does not exist).

Looks good other than that.

Max

> + */
> +void block_job_txn_add_job(BlockJobTxn *txn, BlockJob *job);
> +
>  #endif
> 




signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v5 07/14] blockjob: Add "completed" and "ret" in BlockJob

[Max Reitz]

On 07.09.2015 09:34, Fam Zheng wrote:
> They are set when block_job_completed is called.
> 
> Signed-off-by: Fam Zheng 
> Reviewed-by: John Snow 
> ---
>  blockjob.c   | 3 +++
>  include/block/blockjob.h | 9 +
>  2 files changed, 12 insertions(+)

Reviewed-by: Max Reitz 



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v5 08/14] blockjob: Simplify block_job_finish_sync

[Max Reitz]

On 07.09.2015 09:34, Fam Zheng wrote:
> With job->completed and job->ret to replace BlockFinishData.
> 
> Signed-off-by: Fam Zheng 
> ---
>  blockjob.c | 27 ++-
>  1 file changed, 6 insertions(+), 21 deletions(-)

Reviewed-by: Max Reitz 



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v5 06/14] blockjob: Add .commit and .abort block job actions

[Max Reitz]

On 07.09.2015 09:34, Fam Zheng wrote:
> Signed-off-by: Fam Zheng 
> ---
>  include/block/blockjob.h | 18 ++
>  1 file changed, 18 insertions(+)

Reviewed-by: Max Reitz 



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v5 05/14] blockjob: Introduce reference count

[Max Reitz]

On 07.09.2015 09:34, Fam Zheng wrote:
> So that block_job_complete_sync can be simplified.
> 
> Signed-off-by: Fam Zheng 
> ---
>  block/mirror.c   |  2 +-
>  blockjob.c   | 22 ++
>  include/block/blockjob.h | 18 +++---
>  3 files changed, 30 insertions(+), 12 deletions(-)

Reviewed-by: Max Reitz 



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v5 04/14] backup: Extract dirty bitmap handling as a separate function

[Max Reitz]

On 07.09.2015 09:34, Fam Zheng wrote:
> This will be reused by the coming new transactional completion code.
> 
> Signed-off-by: Fam Zheng 
> Reviewed-by: John Snow 
> ---
>  block/backup.c | 26 --
>  1 file changed, 16 insertions(+), 10 deletions(-)

Reviewed-by: Max Reitz 



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v3 3/4] block: add a 'blockdev-snapshot' QMP command

[Max Reitz]

On 10.09.2015 15:39, Alberto Garcia wrote:
> One of the limitations of the 'blockdev-snapshot-sync' command is that
> it does not allow passing BlockdevOptions to the newly created
> snapshots, so they are always opened using the default values.
> 
> Extending the command to allow passing options is not a practical
> solution because there is overlap between those options and some of
> the existing parameters of the command.
> 
> This patch introduces a new 'blockdev-snapshot' command with a simpler
> interface: it just takes two references to existing block devices that
> will be used as the source and target for the snapshot.
> 
> Since the main difference between the two commands is that one of them
> creates and opens the target image, while the other uses an already
> opened one, the bulk of the implementation is shared.
> 
> Signed-off-by: Alberto Garcia 
> ---
>  blockdev.c   | 163 
> ---
>  qapi-schema.json |   2 +
>  qapi/block-core.json |  26 
>  qmp-commands.hx  |  29 +
>  4 files changed, 160 insertions(+), 60 deletions(-)

Reviewed-by: Max Reitz 



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v3 4/4] block: add tests for the 'blockdev-snapshot' command

[Eric Blake]

On 09/10/2015 07:39 AM, Alberto Garcia wrote:
> Signed-off-by: Alberto Garcia 
> ---
>  tests/qemu-iotests/085 | 97 
> +++---
>  tests/qemu-iotests/085.out | 34 +++-
>  2 files changed, 123 insertions(+), 8 deletions(-)
> 

>  
> +# ${1}: unique identifier for the snapshot filename
> +# ${2}: true: ignore backing images (default); false: don't ignore them
> +function add_snapshot_image()
> +{
> +base_image="${TEST_DIR}/$((${1}-1))-${snapshot_virt0}"
> +snapshot_file="${TEST_DIR}/${1}-${snapshot_virt0}"
> +_make_test_img -b "${base_image}" "$size"
> +mv "${TEST_IMG}" "${snapshot_file}"
> +cmd="{ 'execute': 'blockdev-add', 'arguments':
> +   { 'options':
> + { 'driver': 'qcow2', 'node-name': 'snap_"${1}"',
> +   'ignore-backing': "${2:-true}", 'file':

Needs to be reworked to use 'backing':'' instead of 'ignore-backing'.

But overall looks like a sane set of tests to cover both positive and
negative expected behavior.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v3 3/4] block: add a 'blockdev-snapshot' QMP command

[Eric Blake]

On 09/10/2015 07:39 AM, Alberto Garcia wrote:
> One of the limitations of the 'blockdev-snapshot-sync' command is that
> it does not allow passing BlockdevOptions to the newly created
> snapshots, so they are always opened using the default values.
> 
> Extending the command to allow passing options is not a practical
> solution because there is overlap between those options and some of
> the existing parameters of the command.
> 
> This patch introduces a new 'blockdev-snapshot' command with a simpler
> interface: it just takes two references to existing block devices that
> will be used as the source and target for the snapshot.
> 
> Since the main difference between the two commands is that one of them
> creates and opens the target image, while the other uses an already
> opened one, the bulk of the implementation is shared.
> 
> Signed-off-by: Alberto Garcia 
> ---
>  blockdev.c   | 163 
> ---
>  qapi-schema.json |   2 +
>  qapi/block-core.json |  26 
>  qmp-commands.hx  |  29 +
>  4 files changed, 160 insertions(+), 60 deletions(-)
> 
> diff --git a/blockdev.c b/blockdev.c
> index 6b787c1..78cfb79 100644
> --- a/blockdev.c
> +++ b/blockdev.c
> @@ -1183,6 +1183,18 @@ void qmp_blockdev_snapshot_sync(bool has_device, const 
> char *device,
> &snapshot, errp);
>  }
>  
> +void qmp_blockdev_snapshot(const char *device, const char *snapshot,
> +   Error **errp)
> +{
> +BlockdevSnapshot snapshot_data = {
> +.device = (char *) device,
> +.snapshot = (char *) snapshot
> +};

Hmm. Sounds like you'd love to use my pending 'box':true qapi glue to
make this function have the saner signature of

void qmp_blockdev_snapshot(BlockdevSnapshot *arg, Error **errp);

rather than having to rebuild the struct yourself (with an annoying cast
to lose const) :)
https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg02599.html

But no need to hold this series up waiting for the qapi review queue to
flush. We can simplify later.

>  
> -options = qdict_new();
> -if (has_snapshot_node_name) {
> -qdict_put(options, "node-name",
> -  qstring_from_str(snapshot_node_name));
> +if (snapshot_node_name && bdrv_find_node(snapshot_node_name)) {
> +error_setg(errp, "New snapshot node name already existing");

Pre-existing, but s/existing/exists/ while you are reindenting this.


> +++ b/qapi/block-core.json
> @@ -705,6 +705,19 @@
>  '*format': 'str', '*mode': 'NewImageMode' } }
>  
>  ##
> +# @BlockdevSnapshot
> +#
> +# @device: device or node name to generate the snapshot from.
> +#
> +# @snapshot: reference to the existing block device that will be used
> +#for the snapshot.

Maybe mention that it must NOT have a current backing file, and point to
the "backing":"" trick to get it that way.

> +Create a snapshot, by installing 'device' as the backing image of
> +'snapshot'. Additionally, if 'device' is associated with a block
> +device, the block device changes to using 'snapshot' as its new active
> +image.

Still didn't answer the question from the earlier review of whether the
blockdev-snapshot-sync behavior of specifying the node name of an active
layer in order to not pivot the block device to the snapshot still makes
sense to support in the blockdev-snapshot case.  But we could always add
an optional boolean flag later if someone comes up for a use case where
they'd need to create a snapshot of the active layer without the block
device pivoting, so I don't think it should hold up this patch.

> +
> +Arguments:
> +
> +- "device": snapshot source (json-string)
> +- "snapshot": snapshot target (json-string)
> +
> +Example:
> +
> +-> { "execute": "blockdev-snapshot", "arguments": { "device": "ide-hd0",
> +"snapshot": "node1534" } 
> }
> +<- { "return": {} }

Maybe enhance the example to show the preliminary blockdev-add that
created node1534?

I've pointed out some potential wording improvements, but think they are
minor enough that you can have:

Reviewed-by: Eric Blake 

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature