date:20190709

John Snow  writes:

> Depending on what a user is trying to accomplish, there might be a few
> bitmap cleanup actions that occur when an operation is finished that
> could be useful.
>
> I am proposing three:
> - NEVER: The bitmap is never synchronized against what was copied.
> - ALWAYS: The bitmap is always synchronized, even on failures.
> - ON-SUCCESS: The bitmap is synchronized only on success.
>
> The existing incremental backup modes use 'on-success' semantics,
> so add just that one for right now.
>
> Signed-off-by: John Snow 
> Reviewed-by: Max Reitz 
> ---
>  qapi/block-core.json | 14 ++
>  1 file changed, 14 insertions(+)
>
> diff --git a/qapi/block-core.json b/qapi/block-core.json
> index 0af3866015..0c853d00bd 100644
> --- a/qapi/block-core.json
> +++ b/qapi/block-core.json
> @@ -1134,6 +1134,20 @@
>  { 'enum': 'MirrorSyncMode',
>'data': ['top', 'full', 'none', 'incremental'] }
>  
> +##
> +# @BitmapSyncMode:
> +#
> +# An enumeration of possible behaviors for the synchronization of a bitmap
> +# when used for data copy operations.
> +#
> +# @on-success: The bitmap is only synced when the operation is successful.
> +#  This is the behavior always used for 'INCREMENTAL' backups.
> +#
> +# Since: 4.2
> +##
> +{ 'enum': 'BitmapSyncMode',
> +  'data': ['on-success'] }
> +
>  ##
>  # @MirrorCopyMode:
>  #

Reviewed-by: Markus Armbruster

Re: [Qemu-devel] [PATCH v4 01/18] qapi/block-core: Introduce BackupCommon

John Snow  writes:

> On 7/9/19 7:25 PM, John Snow wrote:
>> drive-backup and blockdev-backup have an awful lot of things in common
>> that are the same. Let's fix that.
>> 
>> I don't deduplicate 'target', because the semantics actually did change
>> between each structure. Leave that one alone so it can be documented
>> separately.
>
> Sorry Markus, I forgot to adjust this. Pretend I wrote:
>
> Where documentation was not identical, use the most up-to-date version.
> For "speed", use Blockdev-Backup's version. For "sync", use
> Drive-Backup's version.

With that:
Reviewed-by: Markus Armbruster 

[...]

Re: [Qemu-devel] [PATCH 02/11] audio: basic support for multi backend audio

"Kővágó, Zoltán"  writes:

> Audio functions no longer access glob_audio_state, instead they get an
> AudioState as a parameter.  This is required in order to support
> multiple backends.
>
> glob_audio_state is also gone, and replaced with a tailq so we can store
> more than one states.
>
> Signed-off-by: Kővágó, Zoltán 
> ---
[...]
> diff --git a/hmp-commands.hx b/hmp-commands.hx
> index bfa5681dd2..23196da3fe 100644
> --- a/hmp-commands.hx
> +++ b/hmp-commands.hx
> @@ -819,16 +819,17 @@ ETEXI
>  
>  {
>  .name   = "wavcapture",
> -.args_type  = "path:F,freq:i?,bits:i?,nchannels:i?",
> -.params = "path [frequency [bits [channels]]]",
> +.args_type  = "path:F,freq:i?,bits:i?,nchannels:i?,audiodev:s?",
> +.params = "path [frequency [bits [channels [audiodev",
>  .help   = "capture audio to a wave file (default frequency=44100 
> bits=16 channels=2)",
>  .cmd= hmp_wavcapture,
>  },
>  STEXI
> -@item wavcapture @var{filename} [@var{frequency} [@var{bits} 
> [@var{channels}]]]
> +@item wavcapture @var{filename} [@var{frequency} [@var{bits} [@var{channels} 
> [@var{audiodev}
>  @findex wavcapture
> -Capture audio into @var{filename}. Using sample rate @var{frequency}
> -bits per sample @var{bits} and number of channels @var{channels}.
> +Capture audio into @var{filename} from @var{audiodev}. Using sample rate
> +@var{frequency} bits per sample @var{bits} and number of channels
> +@var{channels}.
>  
>  Defaults:
>  @itemize @minus
   @item Sample rate = 44100 Hz - CD quality
   @item Bits = 16
   @item Number of channels = 2 - Stereo
   @end itemize
   ETEXI

Defaults for the other optional arguments are listed here.  Why not for
@audiodev?

> diff --git a/qemu-options.hx b/qemu-options.hx
> index 9621e934c0..0111055aa4 100644
> --- a/qemu-options.hx
> +++ b/qemu-options.hx
> @@ -1978,6 +1978,11 @@ can help the device and guest to keep up and not lose 
> events in case
>  events are arriving in bulk.  Possible causes for the latter are flaky
>  network connections, or scripts for automated testing.
>  
> +@item audiodev=@var{audiodev}
> +
> +Use the specified @var{audiodev} when the VNC client requests audio
> +transmission.
> +

What's the default?

>  @end table
>  ETEXI

Re: [Qemu-devel] [BUG] VM abort after migration

2019-07-09 Thread Jason Wang



On 2019/7/10 上午11:36, Longpeng (Mike) wrote:

在 2019/7/10 11:25, Jason Wang 写道:

On 2019/7/8 下午5:47, Dr. David Alan Gilbert wrote:

* longpeng (longpe...@huawei.com) wrote:

Hi guys,

We found a qemu core in our testing environment, the assertion
'assert(bus->irq_count[i] == 0)' in pcibus_reset() was triggered and
the bus->irq_count[i] is '-1'.

Through analysis, it was happened after VM migration and we think
it was caused by the following sequence:

*Migration Source*
1. save bus pci.0 state, including irq_count[x] ( =0 , old )
2. save E1000:
     e1000_pre_save
  e1000_mit_timer
   set_interrupt_cause
    pci_set_irq --> update pci_dev->irq_state to 1 and
    update bus->irq_count[x] to 1 ( new )
  the irq_state sent to dest.

*Migration Dest*
1. Receive the irq_count[x] of pci.0 is 0 , but the irq_state of e1000 is 1.
2. If the e1000 need change irqline , it would call to pci_irq_handler(),
    the irq_state maybe change to 0 and bus->irq_count[x] will become
    -1 in this situation.
3. do VM reboot then the assertion will be triggered.

We also found some guys faced the similar problem:
[1] https://lists.gnu.org/archive/html/qemu-devel/2016-11/msg02525.html
[2] https://bugs.launchpad.net/qemu/+bug/1702621

Is there some patches to fix this problem ?

I don't remember any.


Can we save pcibus state after all the pci devs are saved ?

Does this problem only happen with e1000? I think so.
If it's only e1000 I think we should fix it - I think once the VM is
stopped for doing the device migration it shouldn't be raising
interrupts.


I wonder maybe we can simply fix this by no setting ICS on pre_save() but
scheduling mit timer unconditionally in post_load().


I also think this is a bug of e1000 because we find more cores with the same
frame thease days.

I'm not familiar with e1000 so hope someone could fix it, thanks. :)



Draft a path in attachment, please test.

Thanks



Thanks



Dave


Thanks,
Longpeng(Mike)

--
Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

.

>From afe9258486672d76d7bf133ac9032a0d457bcd0b Mon Sep 17 00:00:00 2001
From: Jason Wang 
Date: Wed, 10 Jul 2019 11:52:53 +0800
Subject: [PATCH] e1000: don't raise interrupt in pre_save()

We should not raise any interrupt after VM has been stopped but this
is what e1000 currently did when mit timer is active in
pre_save(). Fixing this by scheduling a timer in post_load() which can
make sure the interrupt was raised when VM is running.

Signed-off-by: Jason Wang 
---
 hw/net/e1000.c | 8 ++--
 1 file changed, 2 insertions(+), 6 deletions(-)

diff --git a/hw/net/e1000.c b/hw/net/e1000.c
index 1dc1466332..a023ceb27c 100644
--- a/hw/net/e1000.c
+++ b/hw/net/e1000.c
@@ -1381,11 +1381,6 @@ static int e1000_pre_save(void *opaque)
 E1000State *s = opaque;
 NetClientState *nc = qemu_get_queue(s->nic);
 
-/* If the mitigation timer is active, emulate a timeout now. */
-if (s->mit_timer_on) {
-e1000_mit_timer(s);
-}
-
 /*
  * If link is down and auto-negotiation is supported and ongoing,
  * complete auto-negotiation immediately. This allows us to look
@@ -1423,7 +1418,8 @@ static int e1000_post_load(void *opaque, int version_id)
 s->mit_irq_level = false;
 }
 s->mit_ide = 0;
-s->mit_timer_on = false;
+s->mit_timer_on = true;
+timer_mod(s->mit_timer, qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL) + 1);
 
 /* nc.link_down can't be migrated, so infer link_down according
  * to link status bit in mac_reg[STATUS].
-- 
2.19.1

Re: [Qemu-devel] [PATCH V2] net/colo-compare.c: Fix memory leak and code style issue.

2019-07-09 Thread Jason Wang




On 2019/7/4 下午4:36, Zhang Chen wrote:

From: Zhang Chen 

This patch to fix the origin "char *data" menory leak, code style issue
and add necessary check here.
Reported-by: Coverity (CID 1402785)

Signed-off-by: Zhang Chen 
---
  net/colo-compare.c | 28 +---
  1 file changed, 21 insertions(+), 7 deletions(-)

diff --git a/net/colo-compare.c b/net/colo-compare.c
index 909dd6c6eb..ed349f5f6a 100644
--- a/net/colo-compare.c
+++ b/net/colo-compare.c
@@ -127,6 +127,17 @@ static int compare_chr_send(CompareState *s,
  uint32_t vnet_hdr_len,
  bool notify_remote_frame);
  
+static bool packet_matches_str(const char *str,

+   uint8_t *buf,
+   uint32_t packet_len)
+{
+if (packet_len <= strlen(str)) {
+return false;
+}
+
+return !memcmp(str, buf, strlen(str) + 1);



This assumes buf is NULL terminated (you pass notify_rs->buf) which is 
not correct I think?


Thanks



+}
+
  static void notify_remote_frame(CompareState *s)
  {
  char msg[] = "DO_CHECKPOINT";
@@ -1008,21 +1019,24 @@ static void compare_notify_rs_finalize(SocketReadState 
*notify_rs)
  {
  CompareState *s = container_of(notify_rs, CompareState, notify_rs);
  
-/* Get Xen colo-frame's notify and handle the message */

-char *data = g_memdup(notify_rs->buf, notify_rs->packet_len);
-char msg[] = "COLO_COMPARE_GET_XEN_INIT";
+const char msg[] = "COLO_COMPARE_GET_XEN_INIT";
  int ret;
  
-if (!strcmp(data, "COLO_USERSPACE_PROXY_INIT")) {

+if (packet_matches_str("COLO_USERSPACE_PROXY_INIT",
+   notify_rs->buf,
+   notify_rs->packet_len)) {
  ret = compare_chr_send(s, (uint8_t *)msg, strlen(msg), 0, true);
  if (ret < 0) {
  error_report("Notify Xen COLO-frame INIT failed");
  }
-}
-
-if (!strcmp(data, "COLO_CHECKPOINT")) {
+} else if (packet_matches_str("COLO_CHECKPOINT",
+  notify_rs->buf,
+  notify_rs->packet_len)) {
  /* colo-compare do checkpoint, flush pri packet and remove sec packet 
*/
  g_queue_foreach(>conn_list, colo_flush_packets, s);
+} else {
+error_report("COLO compare got unsupported instruction '%s'",
+ (char *)notify_rs->buf);
  }
  }

Re: [Qemu-devel] [BUG] VM abort after migration

2019-07-09 Thread Longpeng (Mike)

在 2019/7/10 11:25, Jason Wang 写道:
> 
> On 2019/7/8 下午5:47, Dr. David Alan Gilbert wrote:
>> * longpeng (longpe...@huawei.com) wrote:
>>> Hi guys,
>>>
>>> We found a qemu core in our testing environment, the assertion
>>> 'assert(bus->irq_count[i] == 0)' in pcibus_reset() was triggered and
>>> the bus->irq_count[i] is '-1'.
>>>
>>> Through analysis, it was happened after VM migration and we think
>>> it was caused by the following sequence:
>>>
>>> *Migration Source*
>>> 1. save bus pci.0 state, including irq_count[x] ( =0 , old )
>>> 2. save E1000:
>>>     e1000_pre_save
>>>  e1000_mit_timer
>>>   set_interrupt_cause
>>>    pci_set_irq --> update pci_dev->irq_state to 1 and
>>>    update bus->irq_count[x] to 1 ( new )
>>>  the irq_state sent to dest.
>>>
>>> *Migration Dest*
>>> 1. Receive the irq_count[x] of pci.0 is 0 , but the irq_state of e1000 is 1.
>>> 2. If the e1000 need change irqline , it would call to pci_irq_handler(),
>>>    the irq_state maybe change to 0 and bus->irq_count[x] will become
>>>    -1 in this situation.
>>> 3. do VM reboot then the assertion will be triggered.
>>>
>>> We also found some guys faced the similar problem:
>>> [1] https://lists.gnu.org/archive/html/qemu-devel/2016-11/msg02525.html
>>> [2] https://bugs.launchpad.net/qemu/+bug/1702621
>>>
>>> Is there some patches to fix this problem ?
>> I don't remember any.
>>
>>> Can we save pcibus state after all the pci devs are saved ?
>> Does this problem only happen with e1000? I think so.
>> If it's only e1000 I think we should fix it - I think once the VM is
>> stopped for doing the device migration it shouldn't be raising
>> interrupts.
> 
> 
> I wonder maybe we can simply fix this by no setting ICS on pre_save() but
> scheduling mit timer unconditionally in post_load().
>
I also think this is a bug of e1000 because we find more cores with the same
frame thease days.

I'm not familiar with e1000 so hope someone could fix it, thanks. :)

> Thanks
> 
> 
>>
>> Dave
>>
>>> Thanks,
>>> Longpeng(Mike)
>> -- 
>> Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK
> 
> .
> 

-- 
Regards,
Longpeng(Mike)

Re: [Qemu-devel] [BUG] VM abort after migration

2019-07-09 Thread Jason Wang




On 2019/7/8 下午5:47, Dr. David Alan Gilbert wrote:

* longpeng (longpe...@huawei.com) wrote:

Hi guys,

We found a qemu core in our testing environment, the assertion
'assert(bus->irq_count[i] == 0)' in pcibus_reset() was triggered and
the bus->irq_count[i] is '-1'.

Through analysis, it was happened after VM migration and we think
it was caused by the following sequence:

*Migration Source*
1. save bus pci.0 state, including irq_count[x] ( =0 , old )
2. save E1000:
e1000_pre_save
 e1000_mit_timer
  set_interrupt_cause
   pci_set_irq --> update pci_dev->irq_state to 1 and
   update bus->irq_count[x] to 1 ( new )
 the irq_state sent to dest.

*Migration Dest*
1. Receive the irq_count[x] of pci.0 is 0 , but the irq_state of e1000 is 1.
2. If the e1000 need change irqline , it would call to pci_irq_handler(),
   the irq_state maybe change to 0 and bus->irq_count[x] will become
   -1 in this situation.
3. do VM reboot then the assertion will be triggered.

We also found some guys faced the similar problem:
[1] https://lists.gnu.org/archive/html/qemu-devel/2016-11/msg02525.html
[2] https://bugs.launchpad.net/qemu/+bug/1702621

Is there some patches to fix this problem ?

I don't remember any.


Can we save pcibus state after all the pci devs are saved ?

Does this problem only happen with e1000? I think so.
If it's only e1000 I think we should fix it - I think once the VM is
stopped for doing the device migration it shouldn't be raising
interrupts.



I wonder maybe we can simply fix this by no setting ICS on pre_save() 
but scheduling mit timer unconditionally in post_load().


Thanks




Dave


Thanks,
Longpeng(Mike)

--
Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-devel] [PATCH v3] block/rbd: implement .bdrv_get_allocated_file_size callback

2019-07-09 Thread Jason Dillaman

On Tue, Jul 9, 2019 at 11:32 AM Max Reitz  wrote:
>
> On 09.07.19 15:09, Stefano Garzarella wrote:
> > On Tue, Jul 09, 2019 at 08:55:19AM -0400, Jason Dillaman wrote:
> >> On Tue, Jul 9, 2019 at 5:45 AM Max Reitz  wrote:
> >>>
> >>> On 09.07.19 10:55, Max Reitz wrote:
>  On 09.07.19 05:08, Jason Dillaman wrote:
> > On Fri, Jul 5, 2019 at 6:43 AM Stefano Garzarella  
> > wrote:
> >>
> >> On Fri, Jul 05, 2019 at 11:58:43AM +0200, Max Reitz wrote:
> >>> On 05.07.19 11:32, Stefano Garzarella wrote:
>  This patch allows 'qemu-img info' to show the 'disk size' for
>  the RBD images that have the fast-diff feature enabled.
> 
>  If this feature is enabled, we use the rbd_diff_iterate2() API
>  to calculate the allocated size for the image.
> 
>  Signed-off-by: Stefano Garzarella 
>  ---
>  v3:
>    - return -ENOTSUP instead of -1 when fast-diff is not available
>  [John, Jason]
>  v2:
>    - calculate the actual usage only if the fast-diff feature is
>  enabled [Jason]
>  ---
>   block/rbd.c | 54 
>  +
>   1 file changed, 54 insertions(+)
> >>>
> >>> Well, the librbd documentation is non-existing as always, but while
> >>> googling, I at least found that libvirt has exactly the same code.  
> >>> So I
> >>> suppose it must be quite correct, then.
> >>>
> >>
> >> While I wrote this code I took a look at libvirt implementation and 
> >> also
> >> at the "rbd" tool in the ceph repository: compute_image_disk_usage() in
> >> src/tools/rbd/action/DiskUsage.cc
> >>
>  diff --git a/block/rbd.c b/block/rbd.c
>  index 59757b3120..b6bed683e5 100644
>  --- a/block/rbd.c
>  +++ b/block/rbd.c
>  @@ -1084,6 +1084,59 @@ static int64_t 
>  qemu_rbd_getlength(BlockDriverState *bs)
>   return info.size;
>   }
> 
>  +static int rbd_allocated_size_cb(uint64_t offset, size_t len, int 
>  exists,
>  + void *arg)
>  +{
>  +int64_t *alloc_size = (int64_t *) arg;
>  +
>  +if (exists) {
>  +(*alloc_size) += len;
>  +}
>  +
>  +return 0;
>  +}
>  +
>  +static int64_t qemu_rbd_get_allocated_file_size(BlockDriverState 
>  *bs)
>  +{
>  +BDRVRBDState *s = bs->opaque;
>  +uint64_t flags, features;
>  +int64_t alloc_size = 0;
>  +int r;
>  +
>  +r = rbd_get_flags(s->image, );
>  +if (r < 0) {
>  +return r;
>  +}
>  +
>  +r = rbd_get_features(s->image, );
>  +if (r < 0) {
>  +return r;
>  +}
>  +
>  +/*
>  + * We use rbd_diff_iterate2() only if the RBD image have 
>  fast-diff
>  + * feature enabled. If it is disabled, rbd_diff_iterate2() 
>  could be
>  + * very slow on a big image.
>  + */
>  +if (!(features & RBD_FEATURE_FAST_DIFF) ||
>  +(flags & RBD_FLAG_FAST_DIFF_INVALID)) {
>  +return -ENOTSUP;
>  +}
>  +
>  +/*
>  + * rbd_diff_iterate2(), if the source snapshot name is NULL, 
>  invokes
>  + * the callback on all allocated regions of the image.
>  + */
>  +r = rbd_diff_iterate2(s->image, NULL, 0,
>  +  bs->total_sectors * BDRV_SECTOR_SIZE, 0, 
>  1,
>  +  _allocated_size_cb, _size);
> >>>
> >>> But I have a question.  This is basically block_status, right?  So it
> >>> gives us information on which areas are allocated and which are not.
> >>> The result thus gives us a lower bound on the allocation size, but is 
> >>> it
> >>> really exactly the allocation size?
> >>>
> >>> There are two things I’m concerned about:
> >>>
> >>> 1. What about metadata?
> >>
> >> Good question, I don't think it includes the size used by metadata and 
> >> I
> >> don't know if there is a way to know it. I'll check better.
> >
> > It does not include the size of metadata, the "rbd_diff_iterate2"
> > function is literally just looking for touched data blocks within the
> > RBD image.
> >
> >>>
> >>> 2. If you have multiple snapshots, this will only report the overall
> >>> allocation information, right?  So say there is something like this:
> >>>
> >>> (“A” means an allocated MB, “-” is an unallocated MB)
> >>>
> >>> Snapshot 1: ---
> >>> Snapshot 2: --A
> >>> Snapshot

Re: [Qemu-devel] [PATCH v4 01/18] qapi/block-core: Introduce BackupCommon




On 7/9/19 7:25 PM, John Snow wrote:
> drive-backup and blockdev-backup have an awful lot of things in common
> that are the same. Let's fix that.
> 
> I don't deduplicate 'target', because the semantics actually did change
> between each structure. Leave that one alone so it can be documented
> separately.

Sorry Markus, I forgot to adjust this. Pretend I wrote:

Where documentation was not identical, use the most up-to-date version.
For "speed", use Blockdev-Backup's version. For "sync", use
Drive-Backup's version.

> 
> Signed-off-by: John Snow 
> Reviewed-by: Max Reitz 
> ---
>  qapi/block-core.json | 103 ++-
>  1 file changed, 33 insertions(+), 70 deletions(-)
> 
> diff --git a/qapi/block-core.json b/qapi/block-core.json
> index 0d43d4f37c..0af3866015 100644
> --- a/qapi/block-core.json
> +++ b/qapi/block-core.json
> @@ -1315,32 +1315,23 @@
>'data': { 'node': 'str', 'overlay': 'str' } }
>  
>  ##
> -# @DriveBackup:
> +# @BackupCommon:
>  #
>  # @job-id: identifier for the newly-created block job. If
>  #  omitted, the device name will be used. (Since 2.7)
>  #
>  # @device: the device name or node-name of a root node which should be 
> copied.
>  #
> -# @target: the target of the new image. If the file exists, or if it
> -#  is a device, the existing file/device will be used as the new
> -#  destination.  If it does not exist, a new file will be created.
> -#
> -# @format: the format of the new destination, default is to
> -#  probe if @mode is 'existing', else the format of the source
> -#
>  # @sync: what parts of the disk image should be copied to the destination
>  #(all the disk, only the sectors allocated in the topmost image, 
> from a
>  #dirty bitmap, or only new I/O).
>  #
> -# @mode: whether and how QEMU should create a new image, default is
> -#'absolute-paths'.
> -#
> -# @speed: the maximum speed, in bytes per second
> +# @speed: the maximum speed, in bytes per second. The default is 0,
> +# for unlimited.
>  #
>  # @bitmap: the name of dirty bitmap if sync is "incremental".
>  #  Must be present if sync is "incremental", must NOT be present
> -#  otherwise. (Since 2.4)
> +#  otherwise. (Since 2.4 (drive-backup), 3.1 (blockdev-backup))
>  #
>  # @compress: true to compress data, if the target format supports it.
>  #(default: false) (since 2.8)
> @@ -1370,75 +1361,47 @@
>  # I/O.  If an error occurs during a guest write request, the device's
>  # rerror/werror actions will be used.
>  #
> +# Since: 4.2
> +##
> +{ 'struct': 'BackupCommon',
> +  'data': { '*job-id': 'str', 'device': 'str',
> +'sync': 'MirrorSyncMode', '*speed': 'int',
> +'*bitmap': 'str', '*compress': 'bool',
> +'*on-source-error': 'BlockdevOnError',
> +'*on-target-error': 'BlockdevOnError',
> +'*auto-finalize': 'bool', '*auto-dismiss': 'bool' } }
> +
> +##
> +# @DriveBackup:
> +#
> +# @target: the target of the new image. If the file exists, or if it
> +#  is a device, the existing file/device will be used as the new
> +#  destination.  If it does not exist, a new file will be created.
> +#
> +# @format: the format of the new destination, default is to
> +#  probe if @mode is 'existing', else the format of the source
> +#
> +# @mode: whether and how QEMU should create a new image, default is
> +#'absolute-paths'.
> +#
>  # Since: 1.6
>  ##
>  { 'struct': 'DriveBackup',
> -  'data': { '*job-id': 'str', 'device': 'str', 'target': 'str',
> -'*format': 'str', 'sync': 'MirrorSyncMode',
> -'*mode': 'NewImageMode', '*speed': 'int',
> -'*bitmap': 'str', '*compress': 'bool',
> -'*on-source-error': 'BlockdevOnError',
> -'*on-target-error': 'BlockdevOnError',
> -'*auto-finalize': 'bool', '*auto-dismiss': 'bool' } }
> +  'base': 'BackupCommon',
> +  'data': { 'target': 'str',
> +'*format': 'str',
> +'*mode': 'NewImageMode' } }
>  
>  ##
>  # @BlockdevBackup:
>  #
> -# @job-id: identifier for the newly-created block job. If
> -#  omitted, the device name will be used. (Since 2.7)
> -#
> -# @device: the device name or node-name of a root node which should be 
> copied.
> -#
>  # @target: the device name or node-name of the backup target node.
>  #
> -# @sync: what parts of the disk image should be copied to the destination
> -#(all the disk, only the sectors allocated in the topmost image, or
> -#only new I/O).
> -#
> -# @speed: the maximum speed, in bytes per second. The default is 0,
> -# for unlimited.
> -#
> -# @bitmap: the name of dirty bitmap if sync is "incremental".
> -#  Must be present if sync is "incremental", must NOT be present
> -#  otherwise. (Since 3.1)
> -#
> -# @compress: true to compress data, if the target format supports it.
> -#

[Qemu-devel] [PATCH 3/8] iotests/257: Refactor backup helpers

This test needs support for non-bitmap backups and missing or
unspecified bitmap sync modes, so rewrite the helpers to be a little
more generic.

Signed-off-by: John Snow 
---
 tests/qemu-iotests/257 |  46 +
 tests/qemu-iotests/257.out | 192 ++---
 2 files changed, 124 insertions(+), 114 deletions(-)

diff --git a/tests/qemu-iotests/257 b/tests/qemu-iotests/257
index 2ff4aa8695..2eb4f26c28 100755
--- a/tests/qemu-iotests/257
+++ b/tests/qemu-iotests/257
@@ -208,6 +208,14 @@ def get_bitmap(bitmaps, drivename, name, recording=None):
 return bitmap
 return None
 
+def blockdev_backup(vm, device, target, sync, **kwargs):
+result = vm.qmp_log('blockdev-backup',
+device=device,
+target=target,
+sync=sync,
+**kwargs)
+return result
+
 def reference_backup(drive, n, filepath):
 log("--- Reference Backup #{:d} ---\n".format(n))
 target_id = "ref_target_{:d}".format(n)
@@ -215,24 +223,26 @@ def reference_backup(drive, n, filepath):
 target_drive = Drive(filepath, vm=drive.vm)
 
 target_drive.create_target(target_id, drive.fmt, drive.size)
-drive.vm.qmp_log("blockdev-backup",
- job_id=job_id, device=drive.name,
- target=target_id, sync="full")
+blockdev_backup(drive.vm, drive.name, target_id, "full", job_id=job_id)
 drive.vm.run_job(job_id, auto_dismiss=True)
 log('')
 
-def bitmap_backup(drive, n, filepath, bitmap, bitmap_mode):
-log("--- Bitmap Backup #{:d} ---\n".format(n))
-target_id = "bitmap_target_{:d}".format(n)
-job_id = "bitmap_backup_{:d}".format(n)
+def backup(drive, n, filepath, bitmap, bitmap_mode, sync='bitmap'):
+log("--- Test Backup #{:d} ---\n".format(n))
+target_id = "backup_target_{:d}".format(n)
+job_id = "backup_{:d}".format(n)
 target_drive = Drive(filepath, vm=drive.vm)
 
 target_drive.create_target(target_id, drive.fmt, drive.size)
-drive.vm.qmp_log("blockdev-backup", job_id=job_id, device=drive.name,
- target=target_id, sync="bitmap",
- bitmap_mode=bitmap_mode,
- bitmap=bitmap,
- auto_finalize=False)
+
+kwargs = {
+'job_id': job_id,
+'auto_finalize': False,
+'bitmap': bitmap,
+'bitmap_mode': bitmap_mode,
+}
+kwargs = {key: val for key, val in kwargs.items() if val is not None}
+blockdev_backup(drive.vm, drive.name, target_id, sync, **kwargs)
 return job_id
 
 def perform_writes(drive, n):
@@ -264,7 +274,7 @@ def compare_images(image, reference, baseimg=None, 
expected_match=True):
 "OK!" if ret == expected_ret else "ERROR!"),
 filters=[iotests.filter_testfiles])
 
-def test_bitmap_sync(bsync_mode, failure=None):
+def test_bitmap_sync(bsync_mode, msync_mode='bitmap', failure=None):
 """
 Test bitmap backup routines.
 
@@ -292,7 +302,7 @@ def test_bitmap_sync(bsync_mode, failure=None):
  fbackup0, fbackup1, fbackup2), \
  iotests.VM() as vm:
 
-mode = "Bitmap Sync Mode {:s}".format(bsync_mode)
+mode = "Mode {:s}; Bitmap Sync {:s}".format(msync_mode, bsync_mode)
 preposition = "with" if failure else "without"
 cond = "{:s} {:s}".format(preposition,
   "{:s} failure".format(failure) if failure
@@ -363,12 +373,12 @@ def test_bitmap_sync(bsync_mode, failure=None):
 ebitmap.compare(bitmap)
 reference_backup(drive0, 1, fbackup1)
 
-# 1 - Bitmap Backup (Optional induced failure)
+# 1 - Test Backup (w/ Optional induced failure)
 if failure == 'intermediate':
 # Activate blkdebug induced failure for second-to-next read
 log(vm.hmp_qemu_io(drive0.name, 'flush'))
 log('')
-job = bitmap_backup(drive0, 1, bsync1, "bitmap0", bsync_mode)
+job = backup(drive0, 1, bsync1, "bitmap0", bsync_mode, sync=msync_mode)
 
 def _callback():
 """Issue writes while the job is open to test bitmap divergence."""
@@ -409,7 +419,7 @@ def test_bitmap_sync(bsync_mode, failure=None):
 reference_backup(drive0, 2, fbackup2)
 
 # 2 - Bitmap Backup (In failure modes, this is a recovery.)
-job = bitmap_backup(drive0, 2, bsync2, "bitmap0", bsync_mode)
+job = backup(drive0, 2, bsync2, "bitmap0", bsync_mode)
 vm.run_job(job, auto_dismiss=True, auto_finalize=False)
 bitmaps = query_bitmaps(vm)
 log(bitmaps, indent=2)
@@ -443,7 +453,7 @@ def test_bitmap_sync(bsync_mode, failure=None):
 def main():
 for bsync_mode in ("never", "on-success", "always"):
 for failure in ("simulated", "intermediate", None):
-test_bitmap_sync(bsync_mode, failure)
+test_bitmap_sync(bsync_mode, "bitmap", failure)
 
 if __name__ ==

[Qemu-devel] [PATCH 7/8] block/backup: support bitmap sync modes for non-bitmap backups

Accept bitmaps and sync policies for the other backup modes.
This allows us to do things like create a bitmap synced to a full backup
without a transaction, or start a resumable backup process.

Some combinations don't make sense, though:

- NEVER policy combined with any non-BITMAP mode doesn't do anything,
  because the bitmap isn't used for input or output.
  It's harmless, but is almost certainly never what the user wanted.

- sync=NONE is more questionable. It can't use on-success because this
  job never completes with success anyway, and the resulting artifact
  of 'always' is suspect: because we start with a full bitmap and only
  copy out segments that get written to, the final output bitmap will
  always be ... a fully set bitmap.

  Maybe there's contexts in which bitmaps make sense for sync=none,
  but not without more severe changes to the current job, and omitting
  it here doesn't prevent us from adding it later.

Signed-off-by: John Snow 
---
 block/backup.c   |  8 +---
 blockdev.c   | 22 ++
 qapi/block-core.json |  6 --
 3 files changed, 27 insertions(+), 9 deletions(-)

diff --git a/block/backup.c b/block/backup.c
index 38c4a688c6..b94ed595ba 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -602,7 +602,7 @@ BlockJob *backup_job_create(const char *job_id, 
BlockDriverState *bs,
 return NULL;
 }
 
-if (sync_mode == MIRROR_SYNC_MODE_BITMAP) {
+if (sync_bitmap) {
 /* If we need to write to this bitmap, check that we can: */
 if (bitmap_mode != BITMAP_SYNC_MODE_NEVER &&
 bdrv_dirty_bitmap_check(sync_bitmap, BDRV_BITMAP_DEFAULT, errp)) {
@@ -613,12 +613,6 @@ BlockJob *backup_job_create(const char *job_id, 
BlockDriverState *bs,
 if (bdrv_dirty_bitmap_create_successor(bs, sync_bitmap, errp) < 0) {
 return NULL;
 }
-} else if (sync_bitmap) {
-error_setg(errp,
-   "a bitmap was given to backup_job_create, "
-   "but it received an incompatible sync_mode (%s)",
-   MirrorSyncMode_str(sync_mode));
-return NULL;
 }
 
 len = bdrv_getlength(bs);
diff --git a/blockdev.c b/blockdev.c
index f0b7da53b0..bc152f8e0d 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -3502,6 +3502,28 @@ static BlockJob *do_backup_common(BackupCommon *backup,
 if (bdrv_dirty_bitmap_check(bmap, BDRV_BITMAP_ALLOW_RO, errp)) {
 return NULL;
 }
+
+/* This does not produce a useful bitmap artifact: */
+if (backup->sync == MIRROR_SYNC_MODE_NONE) {
+error_setg(errp, "sync mode '%s' does not produce meaningful 
bitmap"
+   " outputs", MirrorSyncMode_str(backup->sync));
+return NULL;
+}
+
+/* If the bitmap isn't used for input or output, this is useless: */
+if (backup->bitmap_mode == BITMAP_SYNC_MODE_NEVER &&
+backup->sync != MIRROR_SYNC_MODE_BITMAP) {
+error_setg(errp, "Bitmap sync mode '%s' has no meaningful effect"
+   " when combined with sync mode '%s'",
+   BitmapSyncMode_str(backup->bitmap_mode),
+   MirrorSyncMode_str(backup->sync));
+return NULL;
+}
+}
+
+if (!backup->has_bitmap && backup->has_bitmap_mode) {
+error_setg(errp, "Cannot specify Bitmap sync mode without a bitmap");
+return NULL;
 }
 
 if (!backup->auto_finalize) {
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 5a578806c5..099e4f37b2 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -1352,13 +1352,15 @@
 # @speed: the maximum speed, in bytes per second. The default is 0,
 # for unlimited.
 #
-# @bitmap: the name of a dirty bitmap if sync is "bitmap" or "incremental".
+# @bitmap: The name of a dirty bitmap to use.
 #  Must be present if sync is "bitmap" or "incremental".
+#  Can be present if sync is "full" or "top".
 #  Must not be present otherwise.
 #  (Since 2.4 (drive-backup), 3.1 (blockdev-backup))
 #
 # @bitmap-mode: Specifies the type of data the bitmap should contain after
-#   the operation concludes. Must be present if sync is "bitmap".
+#   the operation concludes.
+#   Must be present if a bitmap was provided,
 #   Must NOT be present otherwise. (Since 4.2)
 #
 # @compress: true to compress data, if the target format supports it.
-- 
2.21.0

[Qemu-devel] [PATCH 8/8] iotests/257: test traditional sync modes

Signed-off-by: John Snow 
---
 tests/qemu-iotests/257 |   31 +
 tests/qemu-iotests/257.out | 3089 
 2 files changed, 3120 insertions(+)

diff --git a/tests/qemu-iotests/257 b/tests/qemu-iotests/257
index de8707cb19..8de1c4da19 100755
--- a/tests/qemu-iotests/257
+++ b/tests/qemu-iotests/257
@@ -288,6 +288,12 @@ def test_bitmap_sync(bsync_mode, msync_mode='bitmap', 
failure=None):
   Bitmaps are always synchronized, regardless of failure.
   (Partial images must be kept.)
 
+:param msync_mode: The mirror sync mode to use for the first backup.
+   Can be any one of:
+- bitmap: Backups based on bitmap manifest.
+- full:   Full backups.
+- top:Full backups of the top layer only.
+
 :param failure: Is the (optional) failure mode, and can be any of:
 - None: No failure. Test the normative path. Default.
 - simulated:Cancel the job right before it completes.
@@ -410,6 +416,11 @@ def test_bitmap_sync(bsync_mode, msync_mode='bitmap', 
failure=None):
 if bsync_mode == 'always' and failure == 'intermediate':
 # We manage to copy one sector (one bit) before the error.
 ebitmap.clear_bit(ebitmap.first_bit)
+if msync_mode in ('full', 'top'):
+# These modes return all bits set except what was 
copied/skipped
+fail_bit = ebitmap.first_bit
+ebitmap.clear()
+ebitmap.dirty_bits(range(fail_bit, SIZE // GRANULARITY))
 ebitmap.compare(get_bitmap(bitmaps, drive0.device, 'bitmap0'))
 
 # 2 - Writes and Reference Backup
@@ -501,6 +512,21 @@ def test_backup_api():
 'bitmap404': ['on-success', 'always', 'never', None],
 'bitmap0':   [None],
 },
+'full': {
+None:['on-success', 'always', 'never'],
+'bitmap404': ['on-success', 'always', 'never', None],
+'bitmap0':   ['never', None],
+},
+'top': {
+None:['on-success', 'always', 'never'],
+'bitmap404': ['on-success', 'always', 'never', None],
+'bitmap0':   ['never', None],
+},
+'none': {
+None:['on-success', 'always', 'never'],
+'bitmap404': ['on-success', 'always', 'never', None],
+'bitmap0':   ['on-success', 'always', 'never', None],
+}
 }
 
 for sync_mode, config in error_cases.items():
@@ -522,6 +548,11 @@ def main():
 for failure in ("simulated", "intermediate", None):
 test_bitmap_sync(bsync_mode, "bitmap", failure)
 
+for sync_mode in ('full', 'top'):
+for bsync_mode in ('on-success', 'always'):
+for failure in ('simulated', 'intermediate', None):
+test_bitmap_sync(bsync_mode, sync_mode, failure)
+
 test_backup_api()
 
 if __name__ == '__main__':
diff --git a/tests/qemu-iotests/257.out b/tests/qemu-iotests/257.out
index 43f2e0f9c9..304e33ab73 100644
--- a/tests/qemu-iotests/257.out
+++ b/tests/qemu-iotests/257.out
@@ -2246,6 +2246,3002 @@ qemu_img compare "TEST_DIR/PID-bsync2" 
"TEST_DIR/PID-fbackup2" ==> Identical, OK
 qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fbackup2" ==> Identical, OK!
 
 
+=== Mode full; Bitmap Sync on-success with simulated failure ===
+
+--- Preparing image & VM ---
+
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": 
{"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+{"execute": "device_add", "arguments": {"drive": "drive0", "driver": 
"scsi-hd", "id": "device0", "share-rw": true}}
+{"return": {}}
+
+--- Write #0 ---
+
+write -P0x49 0x000 0x1
+{"return": ""}
+write -P0x6c 0x010 0x1
+{"return": ""}
+write -P0x6f 0x200 0x1
+{"return": ""}
+write -P0x76 0x3ff 0x1
+{"return": ""}
+{
+  "bitmaps": {
+"device0": []
+  }
+}
+
+--- Reference Backup #0 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id": 
"ref_backup_0", "sync": "full", "target": "ref_target_0"}}
+{"return": {}}
+{"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, 
"speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": 
{"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Add Bitmap ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, 
"name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+
+--- Write #1 ---
+
+write -P0x65 0x000 0x1
+{"return": ""}
+write -P0x77 0x00f8000 0x1
+{"return": ""}
+write -P0x72 0x2008000 0x1
+{"return": ""}
+write -P0x69 0x3fe 0x1

[Qemu-devel] [PATCH 2/8] iotests/257: add EmulatedBitmap class

Represent a bitmap with an object that we can mark and clear bits in.
This makes it easier to manage partial writes when we don't write a
full group's worth of patterns before an error.

Signed-off-by: John Snow 
---
 tests/qemu-iotests/257 | 125 +
 1 file changed, 76 insertions(+), 49 deletions(-)

diff --git a/tests/qemu-iotests/257 b/tests/qemu-iotests/257
index f576a35a5c..2ff4aa8695 100755
--- a/tests/qemu-iotests/257
+++ b/tests/qemu-iotests/257
@@ -85,6 +85,60 @@ GROUPS = [
 Pattern('0xdd', 0x3fc)]), # New; leaving a gap to the right
 ]
 
+
+class EmulatedBitmap:
+def __init__(self, granularity=GRANULARITY):
+self._bits = set()
+self.groups = set()
+self.granularity = granularity
+
+def dirty_bits(self, bits):
+self._bits |= set(bits)
+
+def dirty_group(self, n):
+self.dirty_bits(GROUPS[n].bits(self.granularity))
+
+def clear(self):
+self._bits = set()
+
+def clear_bits(self, bits):
+self._bits = self._bits - set(bits)
+
+def clear_bit(self, bit):
+self.clear_bits({bit})
+
+def clear_group(self, n):
+self.clear_bits(GROUPS[n].bits(self.granularity))
+
+@property
+def first_bit(self):
+return sorted(self.bits)[0]
+
+@property
+def bits(self):
+return self._bits
+
+@property
+def count(self):
+return len(self.bits)
+
+def compare(self, qmp_bitmap):
+"""
+Print a nice human-readable message checking that a bitmap as reported
+by the QMP interface has as many bits set as we expect it to.
+"""
+
+name = qmp_bitmap.get('name', '(anonymous)')
+log("= Checking Bitmap {:s} =".format(name))
+
+want = self.count
+have = qmp_bitmap['count'] // qmp_bitmap['granularity']
+
+log("expecting {:d} dirty sectors; have {:d}. {:s}".format(
+want, have, "OK!" if want == have else "ERROR!"))
+log('')
+
+
 class Drive:
 """Represents, vaguely, a drive attached to a VM.
 Includes format, graph, and device information."""
@@ -195,27 +249,6 @@ def perform_writes(drive, n):
 log('')
 return bitmaps
 
-def calculate_bits(groups=None):
-"""Calculate how many bits we expect to see dirtied."""
-if groups:
-bits = set.union(*(GROUPS[group].bits(GRANULARITY) for group in 
groups))
-return len(bits)
-return 0
-
-def bitmap_comparison(bitmap, groups=None, want=0):
-"""
-Print a nice human-readable message checking that this bitmap has as
-many bits set as we expect it to.
-"""
-log("= Checking Bitmap {:s} =".format(bitmap.get('name', '(anonymous)')))
-
-if groups:
-want = calculate_bits(groups)
-have = bitmap['count'] // bitmap['granularity']
-
-log("expecting {:d} dirty sectors; have {:d}. {:s}".format(
-want, have, "OK!" if want == have else "ERROR!"))
-log('')
 
 def compare_images(image, reference, baseimg=None, expected_match=True):
 """
@@ -321,12 +354,13 @@ def test_bitmap_sync(bsync_mode, failure=None):
 vm.qmp_log("block-dirty-bitmap-add", node=drive0.name,
name="bitmap0", granularity=GRANULARITY)
 log('')
+ebitmap = EmulatedBitmap()
 
 # 1 - Writes and Reference Backup
 bitmaps = perform_writes(drive0, 1)
-dirty_groups = {1}
+ebitmap.dirty_group(1)
 bitmap = get_bitmap(bitmaps, drive0.device, 'bitmap0')
-bitmap_comparison(bitmap, groups=dirty_groups)
+ebitmap.compare(bitmap)
 reference_backup(drive0, 1, fbackup1)
 
 # 1 - Bitmap Backup (Optional induced failure)
@@ -342,54 +376,47 @@ def test_bitmap_sync(bsync_mode, failure=None):
 log('')
 bitmaps = perform_writes(drive0, 2)
 # Named bitmap (static, should be unchanged)
-bitmap_comparison(get_bitmap(bitmaps, drive0.device, 'bitmap0'),
-  groups=dirty_groups)
+ebitmap.compare(get_bitmap(bitmaps, drive0.device, 'bitmap0'))
 # Anonymous bitmap (dynamic, shows new writes)
-bitmap_comparison(get_bitmap(bitmaps, drive0.device, '',
- recording=True), groups={2})
-dirty_groups.add(2)
+anonymous = EmulatedBitmap()
+anonymous.dirty_group(2)
+anonymous.compare(get_bitmap(bitmaps, drive0.device, '',
+ recording=True))
+
+# Simulate the order in which this will happen:
+# group 1 gets cleared first, then group two gets written.
+if ((bsync_mode == 'on-success' and not failure) or
+(bsync_mode == 'always')):
+ebitmap.clear_group(1)
+ebitmap.dirty_group(2)
 
 vm.run_job(job, auto_dismiss=True, auto_finalize=False,
pre_finalize=_callback,

[Qemu-devel] [PATCH 1/8] iotests/257: add Pattern class

Just kidding, this is easier to manage with a full class instead of a
namedtuple.

Signed-off-by: John Snow 
---
 tests/qemu-iotests/257 | 58 +++---
 1 file changed, 32 insertions(+), 26 deletions(-)

diff --git a/tests/qemu-iotests/257 b/tests/qemu-iotests/257
index 75a651c7c3..f576a35a5c 100755
--- a/tests/qemu-iotests/257
+++ b/tests/qemu-iotests/257
@@ -19,7 +19,6 @@
 #
 # owner=js...@redhat.com
 
-from collections import namedtuple
 import math
 import os
 
@@ -29,10 +28,18 @@ from iotests import log, qemu_img
 SIZE = 64 * 1024 * 1024
 GRANULARITY = 64 * 1024
 
-Pattern = namedtuple('Pattern', ['byte', 'offset', 'size'])
-def mkpattern(byte, offset, size=GRANULARITY):
-"""Constructor for Pattern() with default size"""
-return Pattern(byte, offset, size)
+
+class Pattern:
+def __init__(self, byte, offset, size=GRANULARITY):
+self.byte = byte
+self.offset = offset
+self.size = size
+
+def bits(self, granularity):
+lower = math.floor(self.offset / granularity)
+upper = math.floor((self.offset + self.size - 1) / granularity)
+return set(range(lower, upper + 1))
+
 
 class PatternGroup:
 """Grouping of Pattern objects. Initialize with an iterable of Patterns."""
@@ -43,40 +50,39 @@ class PatternGroup:
 """Calculate the unique bits dirtied by this pattern grouping"""
 res = set()
 for pattern in self.patterns:
-lower = math.floor(pattern.offset / granularity)
-upper = math.floor((pattern.offset + pattern.size - 1) / 
granularity)
-res = res | set(range(lower, upper + 1))
+res = res | pattern.bits(granularity)
 return res
 
+
 GROUPS = [
 PatternGroup([
 # Batch 0: 4 clusters
-mkpattern('0x49', 0x000),
-mkpattern('0x6c', 0x010),   # 1M
-mkpattern('0x6f', 0x200),   # 32M
-mkpattern('0x76', 0x3ff)]), # 64M - 64K
+Pattern('0x49', 0x000),
+Pattern('0x6c', 0x010),   # 1M
+Pattern('0x6f', 0x200),   # 32M
+Pattern('0x76', 0x3ff)]), # 64M - 64K
 PatternGroup([
 # Batch 1: 6 clusters (3 new)
-mkpattern('0x65', 0x000),   # Full overwrite
-mkpattern('0x77', 0x00f8000),   # Partial-left (1M-32K)
-mkpattern('0x72', 0x2008000),   # Partial-right (32M+32K)
-mkpattern('0x69', 0x3fe)]), # Adjacent-left (64M - 128K)
+Pattern('0x65', 0x000),   # Full overwrite
+Pattern('0x77', 0x00f8000),   # Partial-left (1M-32K)
+Pattern('0x72', 0x2008000),   # Partial-right (32M+32K)
+Pattern('0x69', 0x3fe)]), # Adjacent-left (64M - 128K)
 PatternGroup([
 # Batch 2: 7 clusters (3 new)
-mkpattern('0x74', 0x001),   # Adjacent-right
-mkpattern('0x69', 0x00e8000),   # Partial-left  (1M-96K)
-mkpattern('0x6e', 0x2018000),   # Partial-right (32M+96K)
-mkpattern('0x67', 0x3fe,
-  2*GRANULARITY)]), # Overwrite [(64M-128K)-64M)
+Pattern('0x74', 0x001),   # Adjacent-right
+Pattern('0x69', 0x00e8000),   # Partial-left  (1M-96K)
+Pattern('0x6e', 0x2018000),   # Partial-right (32M+96K)
+Pattern('0x67', 0x3fe,
+2*GRANULARITY)]), # Overwrite [(64M-128K)-64M)
 PatternGroup([
 # Batch 3: 8 clusters (5 new)
 # Carefully chosen such that nothing re-dirties the one cluster
 # that copies out successfully before failure in Group #1.
-mkpattern('0xaa', 0x001,
-  3*GRANULARITY),   # Overwrite and 2x Adjacent-right
-mkpattern('0xbb', 0x00d8000),   # Partial-left (1M-160K)
-mkpattern('0xcc', 0x2028000),   # Partial-right (32M+160K)
-mkpattern('0xdd', 0x3fc)]), # New; leaving a gap to the right
+Pattern('0xaa', 0x001,
+3*GRANULARITY),   # Overwrite and 2x Adjacent-right
+Pattern('0xbb', 0x00d8000),   # Partial-left (1M-160K)
+Pattern('0xcc', 0x2028000),   # Partial-right (32M+160K)
+Pattern('0xdd', 0x3fc)]), # New; leaving a gap to the right
 ]
 
 class Drive:
-- 
2.21.0

[Qemu-devel] [PATCH 5/8] iotests/257: test API failures

Signed-off-by: John Snow 
---
 tests/qemu-iotests/257 | 69 +++
 tests/qemu-iotests/257.out | 85 ++
 2 files changed, 154 insertions(+)

diff --git a/tests/qemu-iotests/257 b/tests/qemu-iotests/257
index 2eb4f26c28..de8707cb19 100755
--- a/tests/qemu-iotests/257
+++ b/tests/qemu-iotests/257
@@ -450,10 +450,79 @@ def test_bitmap_sync(bsync_mode, msync_mode='bitmap', 
failure=None):
 compare_images(img_path, fbackup2)
 log('')
 
+def test_backup_api():
+"""
+"""
+with iotests.FilePaths(['img', 'bsync1']) as \
+ (img_path, backup_path), \
+ iotests.VM() as vm:
+
+log("\n=== API failure tests ===\n")
+log('--- Preparing image & VM ---\n')
+drive0 = Drive(img_path, vm=vm)
+drive0.img_create(iotests.imgfmt, SIZE)
+vm.add_device("{},id=scsi0".format(iotests.get_virtio_scsi_device()))
+vm.launch()
+
+file_config = {
+'driver': 'file',
+'filename': drive0.path
+}
+
+vm.qmp_log('blockdev-add',
+   filters=[iotests.filter_qmp_testfiles],
+   node_name="drive0",
+   driver=drive0.fmt,
+   file=file_config)
+drive0.node = 'drive0'
+drive0.device = 'device0'
+vm.qmp_log("device_add", id=drive0.device,
+   drive=drive0.name, driver="scsi-hd")
+log('')
+
+target0 = Drive(backup_path, vm=vm)
+target0.create_target("backup_target", drive0.fmt, drive0.size)
+log('')
+
+vm.qmp_log("block-dirty-bitmap-add", node=drive0.name,
+   name="bitmap0", granularity=GRANULARITY)
+log('')
+
+log('-- Testing invalid QMP commands --\n')
+
+error_cases = {
+'incremental': {
+None:['on-success', 'always', 'never', None],
+'bitmap404': ['on-success', 'always', 'never', None],
+'bitmap0':   ['always', 'never']
+},
+'bitmap': {
+None:['on-success', 'always', 'never', None],
+'bitmap404': ['on-success', 'always', 'never', None],
+'bitmap0':   [None],
+},
+}
+
+for sync_mode, config in error_cases.items():
+log("-- Sync mode {:s} tests --\n".format(sync_mode))
+for bitmap, policies in config.items():
+for policy in policies:
+kwargs = { 'job_id': 'api_job' }
+if bitmap is not None:
+kwargs['bitmap'] = bitmap
+if policy is not None:
+kwargs['bitmap-mode'] = policy
+blockdev_backup(drive0.vm, drive0.name, "backup_target",
+sync_mode, **kwargs)
+log('')
+
+
 def main():
 for bsync_mode in ("never", "on-success", "always"):
 for failure in ("simulated", "intermediate", None):
 test_bitmap_sync(bsync_mode, "bitmap", failure)
 
+test_backup_api()
+
 if __name__ == '__main__':
 iotests.script_main(main, supported_fmts=['qcow2'])
diff --git a/tests/qemu-iotests/257.out b/tests/qemu-iotests/257.out
index 0abc96acd3..43f2e0f9c9 100644
--- a/tests/qemu-iotests/257.out
+++ b/tests/qemu-iotests/257.out
@@ -2245,3 +2245,88 @@ qemu_img compare "TEST_DIR/PID-bsync1" 
"TEST_DIR/PID-fbackup1" ==> Identical, OK
 qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fbackup2" ==> Identical, 
OK!
 qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fbackup2" ==> Identical, OK!
 
+
+=== API failure tests ===
+
+--- Preparing image & VM ---
+
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": 
{"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+{"execute": "device_add", "arguments": {"drive": "drive0", "driver": 
"scsi-hd", "id": "device0"}}
+{"return": {}}
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, 
"name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+
+-- Testing invalid QMP commands --
+
+-- Sync mode incremental tests --
+
+{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "on-success", 
"device": "drive0", "job-id": "api_job", "sync": "incremental", "target": 
"backup_target"}}
+{"error": {"class": "GenericError", "desc": "must provide a valid bitmap name 
for 'incremental' sync mode"}}
+
+{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "always", 
"device": "drive0", "job-id": "api_job", "sync": "incremental", "target": 
"backup_target"}}
+{"error": {"class": "GenericError", "desc": "must provide a valid bitmap name 
for 'incremental' sync mode"}}
+
+{"execute":

[Qemu-devel] [PATCH 0/8] bitmaps: allow bitmaps to be used with full and top

Requires: <20190709232550.10724-1-js...@redhat.com>
[PATCH v4 00/18] bitmaps: introduce 'bitmap' sync mode

This follows the previous series which adds the 'bitmap' sync mode
and uses it to add interactions with bitmaps to the 'full' and 'top'
modes to blockdev-backup and drive-backup.

Why?
 on-success: Can conveniently synchronize a bitmap to a full backup.
 Allows for transactionless anchor backups.
 Allows us to attempt an anchor backup without damaging
   our bitmap until the backup is successful.
 Allows for transactional, ungrouped anchor backups.
 always: Allows us to resume full/top style backups with a later
 invocation to sync=bitmap. Neat!

Summary:
1-3: Refactor iotest 257 to accommodate this;
4-5: Augment 257 to test trivial failure cases
6-7: Implement feature
8: Test new feature

John Snow (8):
  iotests/257: add Pattern class
  iotests/257: add EmulatedBitmap class
  iotests/257: Refactor backup helpers
  block/backup: hoist bitmap check into QMP interface
  iotests/257: test API failures
  block/backup: issue progress updates for skipped regions
  block/backup: support bitmap sync modes for non-bitmap backups
  iotests/257: test traditional sync modes

 block/backup.c |   22 +-
 blockdev.c |   32 +
 qapi/block-core.json   |6 +-
 tests/qemu-iotests/257 |  329 +++-
 tests/qemu-iotests/257.out | 3366 +++-
 5 files changed, 3548 insertions(+), 207 deletions(-)

-- 
2.21.0

[Qemu-devel] [PATCH 6/8] block/backup: issue progress updates for skipped regions

The way bitmap backups work is by starting at 75% if it needs
to copy just 25% of the disk.

The way sync=top currently works, however, is to start at 0% and then
never update the progress if it doesn't copy a region. If it needs to
copy 25% of the disk, we'll finish at 25%.

Update the progress when we skip regions.

Signed-off-by: John Snow 
---
 block/backup.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/block/backup.c b/block/backup.c
index a64b768e24..38c4a688c6 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -417,6 +417,7 @@ static int coroutine_fn backup_loop(BackupBlockJob *job)
 if (job->sync_mode == MIRROR_SYNC_MODE_TOP &&
 bdrv_is_unallocated_range(bs, offset, job->cluster_size))
 {
+job_progress_update(>common.job, job->cluster_size);
 bdrv_reset_dirty_bitmap(job->copy_bitmap, offset,
 job->cluster_size);
 continue;
-- 
2.21.0

[Qemu-devel] [PATCH 4/8] block/backup: hoist bitmap check into QMP interface

This is nicer to do in the unified QMP interface that we have now,
because it lets us use the right terminology back at the user.

Signed-off-by: John Snow 
---
 block/backup.c | 13 -
 blockdev.c | 10 ++
 2 files changed, 14 insertions(+), 9 deletions(-)

diff --git a/block/backup.c b/block/backup.c
index e2729cf6fa..a64b768e24 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -566,6 +566,10 @@ BlockJob *backup_job_create(const char *job_id, 
BlockDriverState *bs,
 assert(bs);
 assert(target);
 
+/* QMP interface protects us from these cases */
+assert(sync_mode != MIRROR_SYNC_MODE_INCREMENTAL);
+assert(sync_bitmap || sync_mode != MIRROR_SYNC_MODE_BITMAP);
+
 if (bs == target) {
 error_setg(errp, "Source and target cannot be the same");
 return NULL;
@@ -597,16 +601,7 @@ BlockJob *backup_job_create(const char *job_id, 
BlockDriverState *bs,
 return NULL;
 }
 
-/* QMP interface should have handled translating this to bitmap mode */
-assert(sync_mode != MIRROR_SYNC_MODE_INCREMENTAL);
-
 if (sync_mode == MIRROR_SYNC_MODE_BITMAP) {
-if (!sync_bitmap) {
-error_setg(errp, "must provide a valid bitmap name for "
-   "'%s' sync mode", MirrorSyncMode_str(sync_mode));
-return NULL;
-}
-
 /* If we need to write to this bitmap, check that we can: */
 if (bitmap_mode != BITMAP_SYNC_MODE_NEVER &&
 bdrv_dirty_bitmap_check(sync_bitmap, BDRV_BITMAP_DEFAULT, errp)) {
diff --git a/blockdev.c b/blockdev.c
index 3e30bc2ca7..f0b7da53b0 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -3464,6 +3464,16 @@ static BlockJob *do_backup_common(BackupCommon *backup,
 return NULL;
 }
 
+if ((backup->sync == MIRROR_SYNC_MODE_BITMAP) ||
+(backup->sync == MIRROR_SYNC_MODE_INCREMENTAL)) {
+/* done before desugaring 'incremental' to print the right message */
+if (!backup->has_bitmap) {
+error_setg(errp, "must provide a valid bitmap name for "
+   "'%s' sync mode", MirrorSyncMode_str(backup->sync));
+return NULL;
+}
+}
+
 if (backup->sync == MIRROR_SYNC_MODE_INCREMENTAL) {
 if (backup->has_bitmap_mode &&
 backup->bitmap_mode != BITMAP_SYNC_MODE_ON_SUCCESS) {
-- 
2.21.0

[Qemu-devel] [PATCH] configure: remove obsoleted $sparc_cpu variable

2019-07-09 Thread Carlo Marcelo Arenas Belón

9b9c37c364 ("tcg-sparc: Assume v9 cpu always, i.e. force v8plus in
32-bit mode.", 2012-09-21) removed the need for this variable and
most of the references to it, but this one.

Remove defunct code, no effect or functionality change expected.

Signed-off-by: Carlo Marcelo Arenas Belón 
---
 configure | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/configure b/configure
index 4983c8b533..7518f201ab 100755
--- a/configure
+++ b/configure
@@ -6378,9 +6378,6 @@ if test "$vnc" = "yes" ; then
 echo "VNC JPEG support  $vnc_jpeg"
 echo "VNC PNG support   $vnc_png"
 fi
-if test -n "$sparc_cpu"; then
-echo "Target Sparc Arch $sparc_cpu"
-fi
 echo "xen support   $xen"
 if test "$xen" = "yes" ; then
   echo "xen ctrl version  $xen_ctrl_version"
-- 
2.22.0

[Qemu-devel] [PATCH v4 17/18] iotests: add test 257 for bitmap-mode backups

Signed-off-by: John Snow 
---
 tests/qemu-iotests/257 |  416 +++
 tests/qemu-iotests/257.out | 2247 
 tests/qemu-iotests/group   |1 +
 3 files changed, 2664 insertions(+)
 create mode 100755 tests/qemu-iotests/257
 create mode 100644 tests/qemu-iotests/257.out

diff --git a/tests/qemu-iotests/257 b/tests/qemu-iotests/257
new file mode 100755
index 00..75a651c7c3
--- /dev/null
+++ b/tests/qemu-iotests/257
@@ -0,0 +1,416 @@
+#!/usr/bin/env python
+#
+# Test bitmap-sync backups (incremental, differential, and partials)
+#
+# Copyright (c) 2019 John Snow for Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see .
+#
+# owner=js...@redhat.com
+
+from collections import namedtuple
+import math
+import os
+
+import iotests
+from iotests import log, qemu_img
+
+SIZE = 64 * 1024 * 1024
+GRANULARITY = 64 * 1024
+
+Pattern = namedtuple('Pattern', ['byte', 'offset', 'size'])
+def mkpattern(byte, offset, size=GRANULARITY):
+"""Constructor for Pattern() with default size"""
+return Pattern(byte, offset, size)
+
+class PatternGroup:
+"""Grouping of Pattern objects. Initialize with an iterable of Patterns."""
+def __init__(self, patterns):
+self.patterns = patterns
+
+def bits(self, granularity):
+"""Calculate the unique bits dirtied by this pattern grouping"""
+res = set()
+for pattern in self.patterns:
+lower = math.floor(pattern.offset / granularity)
+upper = math.floor((pattern.offset + pattern.size - 1) / 
granularity)
+res = res | set(range(lower, upper + 1))
+return res
+
+GROUPS = [
+PatternGroup([
+# Batch 0: 4 clusters
+mkpattern('0x49', 0x000),
+mkpattern('0x6c', 0x010),   # 1M
+mkpattern('0x6f', 0x200),   # 32M
+mkpattern('0x76', 0x3ff)]), # 64M - 64K
+PatternGroup([
+# Batch 1: 6 clusters (3 new)
+mkpattern('0x65', 0x000),   # Full overwrite
+mkpattern('0x77', 0x00f8000),   # Partial-left (1M-32K)
+mkpattern('0x72', 0x2008000),   # Partial-right (32M+32K)
+mkpattern('0x69', 0x3fe)]), # Adjacent-left (64M - 128K)
+PatternGroup([
+# Batch 2: 7 clusters (3 new)
+mkpattern('0x74', 0x001),   # Adjacent-right
+mkpattern('0x69', 0x00e8000),   # Partial-left  (1M-96K)
+mkpattern('0x6e', 0x2018000),   # Partial-right (32M+96K)
+mkpattern('0x67', 0x3fe,
+  2*GRANULARITY)]), # Overwrite [(64M-128K)-64M)
+PatternGroup([
+# Batch 3: 8 clusters (5 new)
+# Carefully chosen such that nothing re-dirties the one cluster
+# that copies out successfully before failure in Group #1.
+mkpattern('0xaa', 0x001,
+  3*GRANULARITY),   # Overwrite and 2x Adjacent-right
+mkpattern('0xbb', 0x00d8000),   # Partial-left (1M-160K)
+mkpattern('0xcc', 0x2028000),   # Partial-right (32M+160K)
+mkpattern('0xdd', 0x3fc)]), # New; leaving a gap to the right
+]
+
+class Drive:
+"""Represents, vaguely, a drive attached to a VM.
+Includes format, graph, and device information."""
+
+def __init__(self, path, vm=None):
+self.path = path
+self.vm = vm
+self.fmt = None
+self.size = None
+self.node = None
+self.device = None
+
+@property
+def name(self):
+return self.node or self.device
+
+def img_create(self, fmt, size):
+self.fmt = fmt
+self.size = size
+iotests.qemu_img_create('-f', self.fmt, self.path, str(self.size))
+
+def create_target(self, name, fmt, size):
+basename = os.path.basename(self.path)
+file_node_name = "file_{}".format(basename)
+vm = self.vm
+
+log(vm.command('blockdev-create', job_id='bdc-file-job',
+   options={
+   'driver': 'file',
+   'filename': self.path,
+   'size': 0,
+   }))
+vm.run_job('bdc-file-job')
+log(vm.command('blockdev-add', driver='file',
+   node_name=file_node_name, filename=self.path))
+
+log(vm.command('blockdev-create', job_id='bdc-fmt-job',
+   options={
+

[Qemu-devel] [PATCH v4 11/18] block/backup: upgrade copy_bitmap to BdrvDirtyBitmap

This simplifies some interface matters; namely the initialization and
(later) merging the manifest back into the sync_bitmap if it was
provided.

Signed-off-by: John Snow 
---
 block/backup.c | 81 ++
 1 file changed, 42 insertions(+), 39 deletions(-)

diff --git a/block/backup.c b/block/backup.c
index efd0dcd2e7..fb1b39c44e 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -38,7 +38,10 @@ typedef struct CowRequest {
 typedef struct BackupBlockJob {
 BlockJob common;
 BlockBackend *target;
+
 BdrvDirtyBitmap *sync_bitmap;
+BdrvDirtyBitmap *copy_bitmap;
+
 MirrorSyncMode sync_mode;
 BitmapSyncMode bitmap_mode;
 BlockdevOnError on_source_error;
@@ -51,7 +54,6 @@ typedef struct BackupBlockJob {
 NotifierWithReturn before_write;
 QLIST_HEAD(, CowRequest) inflight_reqs;
 
-HBitmap *copy_bitmap;
 bool use_copy_range;
 int64_t copy_range_size;
 
@@ -113,7 +115,7 @@ static int coroutine_fn 
backup_cow_with_bounce_buffer(BackupBlockJob *job,
 int write_flags = job->serialize_target_writes ? BDRV_REQ_SERIALISING : 0;
 
 assert(QEMU_IS_ALIGNED(start, job->cluster_size));
-hbitmap_reset(job->copy_bitmap, start, job->cluster_size);
+bdrv_reset_dirty_bitmap(job->copy_bitmap, start, job->cluster_size);
 nbytes = MIN(job->cluster_size, job->len - start);
 if (!*bounce_buffer) {
 *bounce_buffer = blk_blockalign(blk, job->cluster_size);
@@ -146,7 +148,7 @@ static int coroutine_fn 
backup_cow_with_bounce_buffer(BackupBlockJob *job,
 
 return nbytes;
 fail:
-hbitmap_set(job->copy_bitmap, start, job->cluster_size);
+bdrv_set_dirty_bitmap(job->copy_bitmap, start, job->cluster_size);
 return ret;
 
 }
@@ -169,12 +171,14 @@ static int coroutine_fn 
backup_cow_with_offload(BackupBlockJob *job,
 assert(QEMU_IS_ALIGNED(start, job->cluster_size));
 nbytes = MIN(job->copy_range_size, end - start);
 nr_clusters = DIV_ROUND_UP(nbytes, job->cluster_size);
-hbitmap_reset(job->copy_bitmap, start, job->cluster_size * nr_clusters);
+bdrv_reset_dirty_bitmap(job->copy_bitmap, start,
+job->cluster_size * nr_clusters);
 ret = blk_co_copy_range(blk, start, job->target, start, nbytes,
 read_flags, write_flags);
 if (ret < 0) {
 trace_backup_do_cow_copy_range_fail(job, start, ret);
-hbitmap_set(job->copy_bitmap, start, job->cluster_size * nr_clusters);
+bdrv_set_dirty_bitmap(job->copy_bitmap, start,
+  job->cluster_size * nr_clusters);
 return ret;
 }
 
@@ -202,7 +206,7 @@ static int coroutine_fn backup_do_cow(BackupBlockJob *job,
 cow_request_begin(_request, job, start, end);
 
 while (start < end) {
-if (!hbitmap_get(job->copy_bitmap, start)) {
+if (!bdrv_dirty_bitmap_get(job->copy_bitmap, start)) {
 trace_backup_do_cow_skip(job, start);
 start += job->cluster_size;
 continue; /* already copied */
@@ -298,14 +302,16 @@ static void backup_abort(Job *job)
 static void backup_clean(Job *job)
 {
 BackupBlockJob *s = container_of(job, BackupBlockJob, common.job);
+BlockDriverState *bs = blk_bs(s->common.blk);
+
+if (s->copy_bitmap) {
+bdrv_release_dirty_bitmap(bs, s->copy_bitmap);
+s->copy_bitmap = NULL;
+}
+
 assert(s->target);
 blk_unref(s->target);
 s->target = NULL;
-
-if (s->copy_bitmap) {
-hbitmap_free(s->copy_bitmap);
-s->copy_bitmap = NULL;
-}
 }
 
 void backup_do_checkpoint(BlockJob *job, Error **errp)
@@ -320,7 +326,7 @@ void backup_do_checkpoint(BlockJob *job, Error **errp)
 return;
 }
 
-hbitmap_set(backup_job->copy_bitmap, 0, backup_job->len);
+bdrv_set_dirty_bitmap(backup_job->copy_bitmap, 0, backup_job->len);
 }
 
 static void backup_drain(BlockJob *job)
@@ -389,59 +395,52 @@ static bool bdrv_is_unallocated_range(BlockDriverState 
*bs,
 
 static int coroutine_fn backup_loop(BackupBlockJob *job)
 {
-int ret;
 bool error_is_read;
 int64_t offset;
-HBitmapIter hbi;
+BdrvDirtyBitmapIter *bdbi;
 BlockDriverState *bs = blk_bs(job->common.blk);
+int ret = 0;
 
-hbitmap_iter_init(, job->copy_bitmap, 0);
-while ((offset = hbitmap_iter_next()) != -1) {
+bdbi = bdrv_dirty_iter_new(job->copy_bitmap);
+while ((offset = bdrv_dirty_iter_next(bdbi)) != -1) {
 if (job->sync_mode == MIRROR_SYNC_MODE_TOP &&
 bdrv_is_unallocated_range(bs, offset, job->cluster_size))
 {
-hbitmap_reset(job->copy_bitmap, offset, job->cluster_size);
+bdrv_reset_dirty_bitmap(job->copy_bitmap, offset,
+job->cluster_size);
 continue;
 }
 
 do {
 if (yield_and_check(job)) {
-return 0;
+goto out;
 }
 ret = backup_do_cow(job,

[Qemu-devel] [PATCH v4 15/18] iotests: teach FilePath to produce multiple paths

Use "FilePaths" instead of "FilePath" to request multiple files be
cleaned up after we leave that object's scope.

This is not crucial; but it saves a little typing.

Signed-off-by: John Snow 
Reviewed-by: Max Reitz 
---
 tests/qemu-iotests/iotests.py | 34 --
 1 file changed, 24 insertions(+), 10 deletions(-)

diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index c544659ecb..6135c9663d 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -358,31 +358,45 @@ class Timeout:
 def timeout(self, signum, frame):
 raise Exception(self.errmsg)
 
+def file_pattern(name):
+return "{0}-{1}".format(os.getpid(), name)
 
-class FilePath(object):
-'''An auto-generated filename that cleans itself up.
+class FilePaths(object):
+"""
+FilePaths is an auto-generated filename that cleans itself up.
 
 Use this context manager to generate filenames and ensure that the file
 gets deleted::
 
-with TestFilePath('test.img') as img_path:
+with FilePaths(['test.img']) as img_path:
 qemu_img('create', img_path, '1G')
 # migration_sock_path is automatically deleted
-'''
-def __init__(self, name):
-filename = '{0}-{1}'.format(os.getpid(), name)
-self.path = os.path.join(test_dir, filename)
+"""
+def __init__(self, names):
+self.paths = []
+for name in names:
+self.paths.append(os.path.join(test_dir, file_pattern(name)))
 
 def __enter__(self):
-return self.path
+return self.paths
 
 def __exit__(self, exc_type, exc_val, exc_tb):
 try:
-os.remove(self.path)
+for path in self.paths:
+os.remove(path)
 except OSError:
 pass
 return False
 
+class FilePath(FilePaths):
+"""
+FilePath is a specialization of FilePaths that takes a single filename.
+"""
+def __init__(self, name):
+super(FilePath, self).__init__([name])
+
+def __enter__(self):
+return self.paths[0]
 
 def file_path_remover():
 for path in reversed(file_path_remover.paths):
@@ -407,7 +421,7 @@ def file_path(*names):
 
 paths = []
 for name in names:
-filename = '{0}-{1}'.format(os.getpid(), name)
+filename = file_pattern(name)
 path = os.path.join(test_dir, filename)
 file_path_remover.paths.append(path)
 paths.append(path)
-- 
2.21.0

Re: [Qemu-devel] [PATCH v2 0/5] tests/docker: add podman support

Patchew URL: 
https://patchew.org/QEMU/20190709194330.837-1-marcandre.lur...@redhat.com/



Hi,

This series failed the asan build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
make docker-image-fedora V=1 NETWORK=1
time make docker-test-debug@fedora TARGET_LIST=x86_64-softmmu J=14 NETWORK=1
=== TEST SCRIPT END ===

PASS 10 test-char /char/serial
PASS 11 test-char /char/hotswap
=
==7981==ERROR: AddressSanitizer: stack-use-after-scope on address 
0x7ffd93a51870 at pc 0x56164f628de6 bp 0x7ffd93a50eb0 sp 0x7ffd93a50ea8
READ of size 8 at 0x7ffd93a51870 thread T0
PASS 12 test-char /char/websocket
#0 0x56164f628de5 in char_socket_server_test 
/tmp/qemu-test/src/tests/test-char.c:762:50
---
  Right alloca redzone:cb
  Shadow gap:  cc
==7981==ABORTING
ERROR - too few tests run (expected 34, got 12)
make: *** [/tmp/qemu-test/src/tests/Makefile.include:902: check-unit] Error 1
make: *** Waiting for unfinished jobs
PASS 1 endianness-test /x86_64/endianness/pc
---
PASS 1 fdc-test /x86_64/fdc/cmos
PASS 2 fdc-test /x86_64/fdc/no_media_on_start
PASS 3 fdc-test /x86_64/fdc/read_without_media
==8008==WARNING: ASan doesn't fully support makecontext/swapcontext functions 
and may produce false positives in some cases!
PASS 4 fdc-test /x86_64/fdc/media_change
PASS 5 fdc-test /x86_64/fdc/sense_interrupt
PASS 6 fdc-test /x86_64/fdc/relative_seek
---
PASS 12 fdc-test /x86_64/fdc/read_no_dma_19
PASS 13 fdc-test /x86_64/fdc/fuzz-registers
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  
QTEST_QEMU_BINARY=x86_64-softmmu/qemu-system-x86_64 QTEST_QEMU_IMG=qemu-img 
tests/ide-test -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl 
--test-name="ide-test" 
==8020==WARNING: ASan doesn't fully support makecontext/swapcontext functions 
and may produce false positives in some cases!
PASS 1 ide-test /x86_64/ide/identify
==8026==WARNING: ASan doesn't fully support makecontext/swapcontext functions 
and may produce false positives in some cases!
PASS 2 ide-test /x86_64/ide/flush
==8032==WARNING: ASan doesn't fully support makecontext/swapcontext functions 
and may produce false positives in some cases!
PASS 3 ide-test /x86_64/ide/bmdma/simple_rw
==8038==WARNING: ASan doesn't fully support makecontext/swapcontext functions 
and may produce false positives in some cases!
PASS 4 ide-test /x86_64/ide/bmdma/trim
==8044==WARNING: ASan doesn't fully support makecontext/swapcontext functions 
and may produce false positives in some cases!
PASS 5 ide-test /x86_64/ide/bmdma/short_prdt
==8050==WARNING: ASan doesn't fully support makecontext/swapcontext functions 
and may produce false positives in some cases!
PASS 6 ide-test /x86_64/ide/bmdma/one_sector_short_prdt
==8056==WARNING: ASan doesn't fully support makecontext/swapcontext functions 
and may produce false positives in some cases!
PASS 7 ide-test /x86_64/ide/bmdma/long_prdt
==8062==WARNING: ASan doesn't fully support makecontext/swapcontext functions 
and may produce false positives in some cases!
==8062==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 
0x7fffee1e4000; bottom 0x7f9c59526000; size: 0x006394cbe000 (427698151424)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
PASS 8 ide-test /x86_64/ide/bmdma/no_busmaster
PASS 9 ide-test /x86_64/ide/flush/nodev
==8073==WARNING: ASan doesn't fully support makecontext/swapcontext functions 
and may produce false positives in some cases!
PASS 10 ide-test /x86_64/ide/flush/empty_drive
==8078==WARNING: ASan doesn't fully support makecontext/swapcontext functions 
and may produce false positives in some cases!
PASS 11 ide-test /x86_64/ide/flush/retry_pci
==8084==WARNING: ASan doesn't fully support makecontext/swapcontext functions 
and may produce false positives in some cases!
PASS 12 ide-test /x86_64/ide/flush/retry_isa
==8091==WARNING: ASan doesn't fully support makecontext/swapcontext functions 
and may produce false positives in some cases!
PASS 13 ide-test /x86_64/ide/cdrom/pio
==8097==WARNING: ASan doesn't fully support makecontext/swapcontext functions 
and may produce false positives in some cases!
PASS 14 ide-test /x86_64/ide/cdrom/pio_large
==8103==WARNING: ASan doesn't fully support makecontext/swapcontext functions 
and may produce false positives in some cases!
PASS 15 ide-test /x86_64/ide/cdrom/dma
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  
QTEST_QEMU_BINARY=x86_64-softmmu/qemu-system-x86_64 QTEST_QEMU_IMG=qemu-img 
tests/ahci-test -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl 
--test-name="ahci-test" 
==8117==WARNING: ASan doesn't fully support makecontext/swapcontext functions 
and may produce false positives in some cases!
PASS 1 ahci-test /x86_64/ahci/sanity
==8123==WARNING: ASan doesn't

[Qemu-devel] [PATCH v4 12/18] block/backup: add 'always' bitmap sync policy

This adds an "always" policy for bitmap synchronization. Regardless of if
the job succeeds or fails, the bitmap is *always* synchronized. This means
that for backups that fail part-way through, the bitmap retains a record of
which sectors need to be copied out to accomplish a new backup using the
old, partial result.

In effect, this allows us to "resume" a failed backup; however the new backup
will be from the new point in time, so it isn't a "resume" as much as it is
an "incremental retry." This can be useful in the case of extremely large
backups that fail considerably through the operation and we'd like to not waste
the work that was already performed.

Signed-off-by: John Snow 
Reviewed-by: Max Reitz 
---
 block/backup.c   | 27 +++
 qapi/block-core.json |  5 -
 2 files changed, 23 insertions(+), 9 deletions(-)

diff --git a/block/backup.c b/block/backup.c
index fb1b39c44e..acfe87b756 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -268,18 +268,29 @@ static void backup_cleanup_sync_bitmap(BackupBlockJob 
*job, int ret)
 {
 BdrvDirtyBitmap *bm;
 BlockDriverState *bs = blk_bs(job->common.blk);
+bool sync = (((ret == 0) || (job->bitmap_mode == BITMAP_SYNC_MODE_ALWAYS)) 
\
+ && (job->bitmap_mode != BITMAP_SYNC_MODE_NEVER));
 
-if (ret < 0 || job->bitmap_mode == BITMAP_SYNC_MODE_NEVER) {
+if (sync) {
 /*
- * Failure, or we don't want to synchronize the bitmap.
- * Merge the successor back into the parent, delete nothing.
+ * We succeeded, or we always intended to sync the bitmap.
+ * Delete this bitmap and install the child.
  */
-bm = bdrv_reclaim_dirty_bitmap(bs, job->sync_bitmap, NULL);
-assert(bm);
-} else {
-/* Everything is fine, delete this bitmap and install the backup. */
 bm = bdrv_dirty_bitmap_abdicate(bs, job->sync_bitmap, NULL);
-assert(bm);
+} else {
+/*
+ * We failed, or we never intended to sync the bitmap anyway.
+ * Merge the successor back into the parent, keeping all data.
+ */
+bm = bdrv_reclaim_dirty_bitmap(bs, job->sync_bitmap, NULL);
+}
+
+assert(bm);
+
+if (ret < 0 && job->bitmap_mode == BITMAP_SYNC_MODE_ALWAYS) {
+/* If we failed and synced, merge in the bits we didn't copy: */
+bdrv_dirty_bitmap_merge_internal(bm, job->copy_bitmap,
+ NULL, true);
 }
 }
 
diff --git a/qapi/block-core.json b/qapi/block-core.json
index b1a98e..5a578806c5 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -1149,10 +1149,13 @@
 # @never: The bitmap is never synchronized with the operation, and is
 # treated solely as a read-only manifest of blocks to copy.
 #
+# @always: The bitmap is always synchronized with the operation,
+#  regardless of whether or not the operation was successful.
+#
 # Since: 4.2
 ##
 { 'enum': 'BitmapSyncMode',
-  'data': ['on-success', 'never'] }
+  'data': ['on-success', 'never', 'always'] }
 
 ##
 # @MirrorCopyMode:
-- 
2.21.0

[Qemu-devel] [PATCH v4 18/18] block/backup: loosen restriction on readonly bitmaps

With the "never" sync policy, we actually can utilize readonly bitmaps
now. Loosen the check at the QMP level, and tighten it based on
provided arguments down at the job creation level instead.

Signed-off-by: John Snow 
Reviewed-by: Max Reitz 
---
 block/backup.c | 6 ++
 blockdev.c | 2 +-
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/block/backup.c b/block/backup.c
index acfe87b756..e2729cf6fa 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -607,6 +607,12 @@ BlockJob *backup_job_create(const char *job_id, 
BlockDriverState *bs,
 return NULL;
 }
 
+/* If we need to write to this bitmap, check that we can: */
+if (bitmap_mode != BITMAP_SYNC_MODE_NEVER &&
+bdrv_dirty_bitmap_check(sync_bitmap, BDRV_BITMAP_DEFAULT, errp)) {
+return NULL;
+}
+
 /* Create a new bitmap, and freeze/disable this one. */
 if (bdrv_dirty_bitmap_create_successor(bs, sync_bitmap, errp) < 0) {
 return NULL;
diff --git a/blockdev.c b/blockdev.c
index 5dfaa976c9..3e30bc2ca7 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -3489,7 +3489,7 @@ static BlockJob *do_backup_common(BackupCommon *backup,
"when providing a bitmap");
 return NULL;
 }
-if (bdrv_dirty_bitmap_check(bmap, BDRV_BITMAP_DEFAULT, errp)) {
+if (bdrv_dirty_bitmap_check(bmap, BDRV_BITMAP_ALLOW_RO, errp)) {
 return NULL;
 }
 }
-- 
2.21.0

[Qemu-devel] [PATCH v4 03/18] blockdev-backup: utilize do_backup_common

Signed-off-by: John Snow 
Reviewed-by: Max Reitz 
---
 blockdev.c | 65 +-
 1 file changed, 6 insertions(+), 59 deletions(-)

diff --git a/blockdev.c b/blockdev.c
index 5bc8ecd087..77365d8166 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -3624,78 +3624,25 @@ BlockJob *do_blockdev_backup(BlockdevBackup *backup, 
JobTxn *txn,
 {
 BlockDriverState *bs;
 BlockDriverState *target_bs;
-Error *local_err = NULL;
-BdrvDirtyBitmap *bmap = NULL;
 AioContext *aio_context;
-BlockJob *job = NULL;
-int job_flags = JOB_DEFAULT;
-int ret;
-
-if (!backup->has_speed) {
-backup->speed = 0;
-}
-if (!backup->has_on_source_error) {
-backup->on_source_error = BLOCKDEV_ON_ERROR_REPORT;
-}
-if (!backup->has_on_target_error) {
-backup->on_target_error = BLOCKDEV_ON_ERROR_REPORT;
-}
-if (!backup->has_job_id) {
-backup->job_id = NULL;
-}
-if (!backup->has_auto_finalize) {
-backup->auto_finalize = true;
-}
-if (!backup->has_auto_dismiss) {
-backup->auto_dismiss = true;
-}
-if (!backup->has_compress) {
-backup->compress = false;
-}
+BlockJob *job;
 
 bs = bdrv_lookup_bs(backup->device, backup->device, errp);
 if (!bs) {
 return NULL;
 }
 
-aio_context = bdrv_get_aio_context(bs);
-aio_context_acquire(aio_context);
-
 target_bs = bdrv_lookup_bs(backup->target, backup->target, errp);
 if (!target_bs) {
-goto out;
+return NULL;
 }
 
-ret = bdrv_try_set_aio_context(target_bs, aio_context, errp);
-if (ret < 0) {
-goto out;
-}
+aio_context = bdrv_get_aio_context(bs);
+aio_context_acquire(aio_context);
 
-if (backup->has_bitmap) {
-bmap = bdrv_find_dirty_bitmap(bs, backup->bitmap);
-if (!bmap) {
-error_setg(errp, "Bitmap '%s' could not be found", backup->bitmap);
-goto out;
-}
-if (bdrv_dirty_bitmap_check(bmap, BDRV_BITMAP_DEFAULT, errp)) {
-goto out;
-}
-}
+job = do_backup_common(qapi_BlockdevBackup_base(backup),
+   bs, target_bs, aio_context, txn, errp);
 
-if (!backup->auto_finalize) {
-job_flags |= JOB_MANUAL_FINALIZE;
-}
-if (!backup->auto_dismiss) {
-job_flags |= JOB_MANUAL_DISMISS;
-}
-job = backup_job_create(backup->job_id, bs, target_bs, backup->speed,
-backup->sync, bmap, backup->compress,
-backup->on_source_error, backup->on_target_error,
-job_flags, NULL, NULL, txn, _err);
-if (local_err != NULL) {
-error_propagate(errp, local_err);
-}
-out:
 aio_context_release(aio_context);
 return job;
 }
-- 
2.21.0

[Qemu-devel] [PATCH v4 02/18] drive-backup: create do_backup_common

Create a common core that comprises the actual meat of what the backup API
boundary needs to do, and then switch drive-backup to use it.

Signed-off-by: John Snow 
Reviewed-by: Max Reitz 
---
 blockdev.c | 122 +
 1 file changed, 67 insertions(+), 55 deletions(-)

diff --git a/blockdev.c b/blockdev.c
index 4d141e9a1f..5bc8ecd087 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -3425,6 +3425,70 @@ out:
 aio_context_release(aio_context);
 }
 
+/* Common QMP interface for drive-backup and blockdev-backup */
+static BlockJob *do_backup_common(BackupCommon *backup,
+  BlockDriverState *bs,
+  BlockDriverState *target_bs,
+  AioContext *aio_context,
+  JobTxn *txn, Error **errp)
+{
+BlockJob *job = NULL;
+BdrvDirtyBitmap *bmap = NULL;
+int job_flags = JOB_DEFAULT;
+int ret;
+
+if (!backup->has_speed) {
+backup->speed = 0;
+}
+if (!backup->has_on_source_error) {
+backup->on_source_error = BLOCKDEV_ON_ERROR_REPORT;
+}
+if (!backup->has_on_target_error) {
+backup->on_target_error = BLOCKDEV_ON_ERROR_REPORT;
+}
+if (!backup->has_job_id) {
+backup->job_id = NULL;
+}
+if (!backup->has_auto_finalize) {
+backup->auto_finalize = true;
+}
+if (!backup->has_auto_dismiss) {
+backup->auto_dismiss = true;
+}
+if (!backup->has_compress) {
+backup->compress = false;
+}
+
+ret = bdrv_try_set_aio_context(target_bs, aio_context, errp);
+if (ret < 0) {
+return NULL;
+}
+
+if (backup->has_bitmap) {
+bmap = bdrv_find_dirty_bitmap(bs, backup->bitmap);
+if (!bmap) {
+error_setg(errp, "Bitmap '%s' could not be found", backup->bitmap);
+return NULL;
+}
+if (bdrv_dirty_bitmap_check(bmap, BDRV_BITMAP_DEFAULT, errp)) {
+return NULL;
+}
+}
+
+if (!backup->auto_finalize) {
+job_flags |= JOB_MANUAL_FINALIZE;
+}
+if (!backup->auto_dismiss) {
+job_flags |= JOB_MANUAL_DISMISS;
+}
+
+job = backup_job_create(backup->job_id, bs, target_bs, backup->speed,
+backup->sync, bmap, backup->compress,
+backup->on_source_error, backup->on_target_error,
+job_flags, NULL, NULL, txn, errp);
+return job;
+}
+
 static BlockJob *do_drive_backup(DriveBackup *backup, JobTxn *txn,
  Error **errp)
 {
@@ -3432,39 +3496,16 @@ static BlockJob *do_drive_backup(DriveBackup *backup, 
JobTxn *txn,
 BlockDriverState *target_bs;
 BlockDriverState *source = NULL;
 BlockJob *job = NULL;
-BdrvDirtyBitmap *bmap = NULL;
 AioContext *aio_context;
 QDict *options = NULL;
 Error *local_err = NULL;
-int flags, job_flags = JOB_DEFAULT;
+int flags;
 int64_t size;
 bool set_backing_hd = false;
-int ret;
 
-if (!backup->has_speed) {
-backup->speed = 0;
-}
-if (!backup->has_on_source_error) {
-backup->on_source_error = BLOCKDEV_ON_ERROR_REPORT;
-}
-if (!backup->has_on_target_error) {
-backup->on_target_error = BLOCKDEV_ON_ERROR_REPORT;
-}
 if (!backup->has_mode) {
 backup->mode = NEW_IMAGE_MODE_ABSOLUTE_PATHS;
 }
-if (!backup->has_job_id) {
-backup->job_id = NULL;
-}
-if (!backup->has_auto_finalize) {
-backup->auto_finalize = true;
-}
-if (!backup->has_auto_dismiss) {
-backup->auto_dismiss = true;
-}
-if (!backup->has_compress) {
-backup->compress = false;
-}
 
 bs = bdrv_lookup_bs(backup->device, backup->device, errp);
 if (!bs) {
@@ -3541,12 +3582,6 @@ static BlockJob *do_drive_backup(DriveBackup *backup, 
JobTxn *txn,
 goto out;
 }
 
-ret = bdrv_try_set_aio_context(target_bs, aio_context, errp);
-if (ret < 0) {
-bdrv_unref(target_bs);
-goto out;
-}
-
 if (set_backing_hd) {
 bdrv_set_backing_hd(target_bs, source, _err);
 if (local_err) {
@@ -3554,31 +3589,8 @@ static BlockJob *do_drive_backup(DriveBackup *backup, 
JobTxn *txn,
 }
 }
 
-if (backup->has_bitmap) {
-bmap = bdrv_find_dirty_bitmap(bs, backup->bitmap);
-if (!bmap) {
-error_setg(errp, "Bitmap '%s' could not be found", backup->bitmap);
-goto unref;
-}
-if (bdrv_dirty_bitmap_check(bmap, BDRV_BITMAP_DEFAULT, errp)) {
-goto unref;
-}
-}
-if (!backup->auto_finalize) {
-job_flags |= JOB_MANUAL_FINALIZE;
-}
-if (!backup->auto_dismiss) {
-job_flags |= JOB_MANUAL_DISMISS;
-}
-
-job = backup_job_create(backup->job_id, bs, target_bs, backup->speed,
-backup->sync, bmap, backup->compress,
-

[Qemu-devel] [PATCH v4 09/18] block/dirty-bitmap: add bdrv_dirty_bitmap_merge_internal

I'm surprised it didn't come up sooner, but sometimes we have a +busy
bitmap as a source. This is dangerous from the QMP API, but if we are
the owner that marked the bitmap busy, it's safe to merge it using it as
a read only source.

It is not safe in the general case to allow users to read from in-use
bitmaps, so create an internal variant that foregoes the safety
checking.

Signed-off-by: John Snow 
---
 block/dirty-bitmap.c  | 54 +++
 include/block/block_int.h |  3 +++
 2 files changed, 52 insertions(+), 5 deletions(-)

diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 95a9c2a5d8..7881fea684 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -810,6 +810,12 @@ bool bdrv_dirty_bitmap_next_dirty_area(BdrvDirtyBitmap 
*bitmap,
 return hbitmap_next_dirty_area(bitmap->bitmap, offset, bytes);
 }
 
+/**
+ * bdrv_merge_dirty_bitmap: merge src into dest.
+ * Ensures permissions on bitmaps are reasonable; use for public API.
+ *
+ * @backup: If provided, make a copy of dest here prior to merge.
+ */
 void bdrv_merge_dirty_bitmap(BdrvDirtyBitmap *dest, const BdrvDirtyBitmap *src,
  HBitmap **backup, Error **errp)
 {
@@ -833,6 +839,42 @@ void bdrv_merge_dirty_bitmap(BdrvDirtyBitmap *dest, const 
BdrvDirtyBitmap *src,
 goto out;
 }
 
+ret = bdrv_dirty_bitmap_merge_internal(dest, src, backup, false);
+assert(ret);
+
+out:
+qemu_mutex_unlock(dest->mutex);
+if (src->mutex != dest->mutex) {
+qemu_mutex_unlock(src->mutex);
+}
+}
+
+/**
+ * bdrv_dirty_bitmap_merge_internal: merge src into dest.
+ * Does NOT check bitmap permissions; not suitable for use as public API.
+ *
+ * @backup: If provided, make a copy of dest here prior to merge.
+ * @lock: If true, lock and unlock bitmaps on the way in/out.
+ * returns true if the merge succeeded; false if unattempted.
+ */
+bool bdrv_dirty_bitmap_merge_internal(BdrvDirtyBitmap *dest,
+  const BdrvDirtyBitmap *src,
+  HBitmap **backup,
+  bool lock)
+{
+bool ret;
+
+assert(!bdrv_dirty_bitmap_readonly(dest));
+assert(!bdrv_dirty_bitmap_inconsistent(dest));
+assert(!bdrv_dirty_bitmap_inconsistent(src));
+
+if (lock) {
+qemu_mutex_lock(dest->mutex);
+if (src->mutex != dest->mutex) {
+qemu_mutex_lock(src->mutex);
+}
+}
+
 if (backup) {
 *backup = dest->bitmap;
 dest->bitmap = hbitmap_alloc(dest->size, hbitmap_granularity(*backup));
@@ -840,11 +882,13 @@ void bdrv_merge_dirty_bitmap(BdrvDirtyBitmap *dest, const 
BdrvDirtyBitmap *src,
 } else {
 ret = hbitmap_merge(dest->bitmap, src->bitmap, dest->bitmap);
 }
-assert(ret);
 
-out:
-qemu_mutex_unlock(dest->mutex);
-if (src->mutex != dest->mutex) {
-qemu_mutex_unlock(src->mutex);
+if (lock) {
+qemu_mutex_unlock(dest->mutex);
+if (src->mutex != dest->mutex) {
+qemu_mutex_unlock(src->mutex);
+}
 }
+
+return ret;
 }
diff --git a/include/block/block_int.h b/include/block/block_int.h
index e1f2aa627e..83ffdf4950 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -1238,6 +1238,9 @@ void bdrv_set_dirty(BlockDriverState *bs, int64_t offset, 
int64_t bytes);
 
 void bdrv_clear_dirty_bitmap(BdrvDirtyBitmap *bitmap, HBitmap **out);
 void bdrv_restore_dirty_bitmap(BdrvDirtyBitmap *bitmap, HBitmap *backup);
+bool bdrv_dirty_bitmap_merge_internal(BdrvDirtyBitmap *dest,
+  const BdrvDirtyBitmap *src,
+  HBitmap **backup, bool lock);
 
 void bdrv_inc_in_flight(BlockDriverState *bs);
 void bdrv_dec_in_flight(BlockDriverState *bs);
-- 
2.21.0

[Qemu-devel] [PATCH v4 04/18] qapi: add BitmapSyncMode enum

Depending on what a user is trying to accomplish, there might be a few
bitmap cleanup actions that occur when an operation is finished that
could be useful.

I am proposing three:
- NEVER: The bitmap is never synchronized against what was copied.
- ALWAYS: The bitmap is always synchronized, even on failures.
- ON-SUCCESS: The bitmap is synchronized only on success.

The existing incremental backup modes use 'on-success' semantics,
so add just that one for right now.

Signed-off-by: John Snow 
Reviewed-by: Max Reitz 
---
 qapi/block-core.json | 14 ++
 1 file changed, 14 insertions(+)

diff --git a/qapi/block-core.json b/qapi/block-core.json
index 0af3866015..0c853d00bd 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -1134,6 +1134,20 @@
 { 'enum': 'MirrorSyncMode',
   'data': ['top', 'full', 'none', 'incremental'] }
 
+##
+# @BitmapSyncMode:
+#
+# An enumeration of possible behaviors for the synchronization of a bitmap
+# when used for data copy operations.
+#
+# @on-success: The bitmap is only synced when the operation is successful.
+#  This is the behavior always used for 'INCREMENTAL' backups.
+#
+# Since: 4.2
+##
+{ 'enum': 'BitmapSyncMode',
+  'data': ['on-success'] }
+
 ##
 # @MirrorCopyMode:
 #
-- 
2.21.0

[Qemu-devel] [PATCH v4 05/18] block/backup: Add mirror sync mode 'bitmap'

We don't need or want a new sync mode for simple differences in
semantics.  Create a new mode simply named "BITMAP" that is designed to
make use of the new Bitmap Sync Mode field.

Because the only bitmap sync mode is 'on-success', this adds no new
functionality to the backup job (yet). The old incremental backup mode
is maintained as a syntactic sugar for sync=bitmap, mode=on-success.

Add all of the plumbing necessary to support this new instruction.

Signed-off-by: John Snow 
Reviewed-by: Max Reitz 
---
 block/backup.c| 20 
 block/mirror.c|  6 --
 block/replication.c   |  2 +-
 blockdev.c| 25 +++--
 include/block/block_int.h |  4 +++-
 qapi/block-core.json  | 21 +++--
 6 files changed, 58 insertions(+), 20 deletions(-)

diff --git a/block/backup.c b/block/backup.c
index 715e1d3be8..996941fa61 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -38,9 +38,9 @@ typedef struct CowRequest {
 typedef struct BackupBlockJob {
 BlockJob common;
 BlockBackend *target;
-/* bitmap for sync=incremental */
 BdrvDirtyBitmap *sync_bitmap;
 MirrorSyncMode sync_mode;
+BitmapSyncMode bitmap_mode;
 BlockdevOnError on_source_error;
 BlockdevOnError on_target_error;
 CoRwlock flush_rwlock;
@@ -452,7 +452,7 @@ static int coroutine_fn backup_run(Job *job, Error **errp)
 
 job_progress_set_remaining(job, s->len);
 
-if (s->sync_mode == MIRROR_SYNC_MODE_INCREMENTAL) {
+if (s->sync_mode == MIRROR_SYNC_MODE_BITMAP) {
 backup_incremental_init_copy_bitmap(s);
 } else {
 hbitmap_set(s->copy_bitmap, 0, s->len);
@@ -536,6 +536,7 @@ static int64_t 
backup_calculate_cluster_size(BlockDriverState *target,
 BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
   BlockDriverState *target, int64_t speed,
   MirrorSyncMode sync_mode, BdrvDirtyBitmap *sync_bitmap,
+  BitmapSyncMode bitmap_mode,
   bool compress,
   BlockdevOnError on_source_error,
   BlockdevOnError on_target_error,
@@ -583,10 +584,13 @@ BlockJob *backup_job_create(const char *job_id, 
BlockDriverState *bs,
 return NULL;
 }
 
-if (sync_mode == MIRROR_SYNC_MODE_INCREMENTAL) {
+/* QMP interface should have handled translating this to bitmap mode */
+assert(sync_mode != MIRROR_SYNC_MODE_INCREMENTAL);
+
+if (sync_mode == MIRROR_SYNC_MODE_BITMAP) {
 if (!sync_bitmap) {
 error_setg(errp, "must provide a valid bitmap name for "
- "\"incremental\" sync mode");
+   "'%s' sync mode", MirrorSyncMode_str(sync_mode));
 return NULL;
 }
 
@@ -596,8 +600,8 @@ BlockJob *backup_job_create(const char *job_id, 
BlockDriverState *bs,
 }
 } else if (sync_bitmap) {
 error_setg(errp,
-   "a sync_bitmap was provided to backup_run, "
-   "but received an incompatible sync_mode (%s)",
+   "a bitmap was given to backup_job_create, "
+   "but it received an incompatible sync_mode (%s)",
MirrorSyncMode_str(sync_mode));
 return NULL;
 }
@@ -639,8 +643,8 @@ BlockJob *backup_job_create(const char *job_id, 
BlockDriverState *bs,
 job->on_source_error = on_source_error;
 job->on_target_error = on_target_error;
 job->sync_mode = sync_mode;
-job->sync_bitmap = sync_mode == MIRROR_SYNC_MODE_INCREMENTAL ?
-   sync_bitmap : NULL;
+job->sync_bitmap = sync_bitmap;
+job->bitmap_mode = bitmap_mode;
 job->compress = compress;
 
 /* Detect image-fleecing (and similar) schemes */
diff --git a/block/mirror.c b/block/mirror.c
index 2fcec70e35..75c8f38c6a 100644
--- a/block/mirror.c
+++ b/block/mirror.c
@@ -1717,8 +1717,10 @@ void mirror_start(const char *job_id, BlockDriverState 
*bs,
 bool is_none_mode;
 BlockDriverState *base;
 
-if (mode == MIRROR_SYNC_MODE_INCREMENTAL) {
-error_setg(errp, "Sync mode 'incremental' not supported");
+if ((mode == MIRROR_SYNC_MODE_INCREMENTAL) ||
+(mode == MIRROR_SYNC_MODE_BITMAP)) {
+error_setg(errp, "Sync mode '%s' not supported",
+   MirrorSyncMode_str(mode));
 return;
 }
 is_none_mode = mode == MIRROR_SYNC_MODE_NONE;
diff --git a/block/replication.c b/block/replication.c
index 23b2993d74..936b2f8b5a 100644
--- a/block/replication.c
+++ b/block/replication.c
@@ -543,7 +543,7 @@ static void replication_start(ReplicationState *rs, 
ReplicationMode mode,
 
 s->backup_job = backup_job_create(
 NULL, s->secondary_disk->bs, 
s->hidden_disk->bs,
-0, MIRROR_SYNC_MODE_NONE, NULL, false,
+0, MIRROR_SYNC_MODE_NONE, NULL, 0, false,

[Qemu-devel] [PATCH v4 16/18] iotests: Add virtio-scsi device helper

Seems that it comes up enough.

Signed-off-by: John Snow 
Reviewed-by: Max Reitz 
---
 tests/qemu-iotests/040| 6 +-
 tests/qemu-iotests/093| 6 ++
 tests/qemu-iotests/139| 7 ++-
 tests/qemu-iotests/238| 5 +
 tests/qemu-iotests/iotests.py | 4 
 5 files changed, 10 insertions(+), 18 deletions(-)

diff --git a/tests/qemu-iotests/040 b/tests/qemu-iotests/040
index b81133a474..657b37103c 100755
--- a/tests/qemu-iotests/040
+++ b/tests/qemu-iotests/040
@@ -85,11 +85,7 @@ class TestSingleDrive(ImageCommitTestCase):
 qemu_io('-f', 'raw', '-c', 'write -P 0xab 0 524288', backing_img)
 qemu_io('-f', iotests.imgfmt, '-c', 'write -P 0xef 524288 524288', 
mid_img)
 self.vm = iotests.VM().add_drive(test_img, 
"node-name=top,backing.node-name=mid,backing.backing.node-name=base", 
interface="none")
-if iotests.qemu_default_machine == 's390-ccw-virtio':
-self.vm.add_device("virtio-scsi-ccw")
-else:
-self.vm.add_device("virtio-scsi-pci")
-
+self.vm.add_device(iotests.get_virtio_scsi_device())
 self.vm.add_device("scsi-hd,id=scsi0,drive=drive0")
 self.vm.launch()
 
diff --git a/tests/qemu-iotests/093 b/tests/qemu-iotests/093
index d88fbc182e..46153220f8 100755
--- a/tests/qemu-iotests/093
+++ b/tests/qemu-iotests/093
@@ -366,10 +366,8 @@ class ThrottleTestGroupNames(iotests.QMPTestCase):
 class ThrottleTestRemovableMedia(iotests.QMPTestCase):
 def setUp(self):
 self.vm = iotests.VM()
-if iotests.qemu_default_machine == 's390-ccw-virtio':
-self.vm.add_device("virtio-scsi-ccw,id=virtio-scsi")
-else:
-self.vm.add_device("virtio-scsi-pci,id=virtio-scsi")
+self.vm.add_device("{},id=virtio-scsi".format(
+iotests.get_virtio_scsi_device()))
 self.vm.launch()
 
 def tearDown(self):
diff --git a/tests/qemu-iotests/139 b/tests/qemu-iotests/139
index 933b45121a..2176ea51ba 100755
--- a/tests/qemu-iotests/139
+++ b/tests/qemu-iotests/139
@@ -35,11 +35,8 @@ class TestBlockdevDel(iotests.QMPTestCase):
 def setUp(self):
 iotests.qemu_img('create', '-f', iotests.imgfmt, base_img, '1M')
 self.vm = iotests.VM()
-if iotests.qemu_default_machine == 's390-ccw-virtio':
-self.vm.add_device("virtio-scsi-ccw,id=virtio-scsi")
-else:
-self.vm.add_device("virtio-scsi-pci,id=virtio-scsi")
-
+self.vm.add_device("{},id=virtio-scsi".format(
+iotests.get_virtio_scsi_device()))
 self.vm.launch()
 
 def tearDown(self):
diff --git a/tests/qemu-iotests/238 b/tests/qemu-iotests/238
index 1c0a46fa90..387a77b2cd 100755
--- a/tests/qemu-iotests/238
+++ b/tests/qemu-iotests/238
@@ -23,10 +23,7 @@ import os
 import iotests
 from iotests import log
 
-if iotests.qemu_default_machine == 's390-ccw-virtio':
-virtio_scsi_device = 'virtio-scsi-ccw'
-else:
-virtio_scsi_device = 'virtio-scsi-pci'
+virtio_scsi_device = iotests.get_virtio_scsi_device()
 
 vm = iotests.VM()
 vm.launch()
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index 6135c9663d..8ae7bc353e 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -164,6 +164,10 @@ def qemu_io_silent(*args):
  (-exitcode, ' '.join(args)))
 return exitcode
 
+def get_virtio_scsi_device():
+if qemu_default_machine == 's390-ccw-virtio':
+return 'virtio-scsi-ccw'
+return 'virtio-scsi-pci'
 
 class QemuIoInteractive:
 def __init__(self, *args):
-- 
2.21.0

[Qemu-devel] [PATCH v4 13/18] iotests: add testing shim for script-style python tests

Because the new-style python tests don't use the iotests.main() test
launcher, we don't turn on the debugger logging for these scripts
when invoked via ./check -d.

Refactor the launcher shim into new and old style shims so that they
share environmental configuration.

Two cleanup notes: debug was not actually used as a global, and there
was no reason to create a class in an inner scope just to achieve
default variables; we can simply create an instance of the runner with
the values we want instead.

Signed-off-by: John Snow 
Reviewed-by: Max Reitz 
---
 tests/qemu-iotests/iotests.py | 40 +++
 1 file changed, 26 insertions(+), 14 deletions(-)

diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index 3ecef5bc90..fcad957d63 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -61,7 +61,6 @@ cachemode = os.environ.get('CACHEMODE')
 qemu_default_machine = os.environ.get('QEMU_DEFAULT_MACHINE')
 
 socket_scm_helper = os.environ.get('SOCKET_SCM_HELPER', 'socket_scm_helper')
-debug = False
 
 luks_default_secret_object = 'secret,id=keysec0,data=' + \
  os.environ.get('IMGKEYSECRET', '')
@@ -834,11 +833,22 @@ def skip_if_unsupported(required_formats=[], 
read_only=False):
 return func_wrapper
 return skip_test_decorator
 
-def main(supported_fmts=[], supported_oses=['linux'], supported_cache_modes=[],
- unsupported_fmts=[]):
-'''Run tests'''
+def execute_unittest(output, verbosity, debug):
+runner = unittest.TextTestRunner(stream=output, descriptions=True,
+ verbosity=verbosity)
+try:
+# unittest.main() will use sys.exit(); so expect a SystemExit
+# exception
+unittest.main(testRunner=runner)
+finally:
+if not debug:
+sys.stderr.write(re.sub(r'Ran (\d+) tests? in [\d.]+s',
+r'Ran \1 tests', output.getvalue()))
 
-global debug
+def execute_test(test_function=None,
+ supported_fmts=[], supported_oses=['linux'],
+ supported_cache_modes=[], unsupported_fmts=[]):
+"""Run either unittest or script-style tests."""
 
 # We are using TEST_DIR and QEMU_DEFAULT_MACHINE as proxies to
 # indicate that we're not being run via "check". There may be
@@ -870,13 +880,15 @@ def main(supported_fmts=[], supported_oses=['linux'], 
supported_cache_modes=[],
 
 logging.basicConfig(level=(logging.DEBUG if debug else logging.WARN))
 
-class MyTestRunner(unittest.TextTestRunner):
-def __init__(self, stream=output, descriptions=True, 
verbosity=verbosity):
-unittest.TextTestRunner.__init__(self, stream, descriptions, 
verbosity)
+if not test_function:
+execute_unittest(output, verbosity, debug)
+else:
+test_function()
 
-# unittest.main() will use sys.exit() so expect a SystemExit exception
-try:
-unittest.main(testRunner=MyTestRunner)
-finally:
-if not debug:
-sys.stderr.write(re.sub(r'Ran (\d+) tests? in [\d.]+s', r'Ran \1 
tests', output.getvalue()))
+def script_main(test_function, *args, **kwargs):
+"""Run script-style tests outside of the unittest framework"""
+execute_test(test_function, *args, **kwargs)
+
+def main(*args, **kwargs):
+"""Run tests using the unittest framework"""
+execute_test(None, *args, **kwargs)
-- 
2.21.0

[Qemu-devel] [PATCH v4 06/18] block/backup: add 'never' policy to bitmap sync mode

This adds a "never" policy for bitmap synchronization. Regardless of if
the job succeeds or fails, we never update the bitmap. This can be used
to perform differential backups, or simply to avoid the job modifying a
bitmap.

Signed-off-by: John Snow 
Reviewed-by: Max Reitz 
---
 block/backup.c   | 7 +--
 qapi/block-core.json | 5 -
 2 files changed, 9 insertions(+), 3 deletions(-)

diff --git a/block/backup.c b/block/backup.c
index 996941fa61..efd0dcd2e7 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -265,8 +265,11 @@ static void backup_cleanup_sync_bitmap(BackupBlockJob 
*job, int ret)
 BdrvDirtyBitmap *bm;
 BlockDriverState *bs = blk_bs(job->common.blk);
 
-if (ret < 0) {
-/* Merge the successor back into the parent, delete nothing. */
+if (ret < 0 || job->bitmap_mode == BITMAP_SYNC_MODE_NEVER) {
+/*
+ * Failure, or we don't want to synchronize the bitmap.
+ * Merge the successor back into the parent, delete nothing.
+ */
 bm = bdrv_reclaim_dirty_bitmap(bs, job->sync_bitmap, NULL);
 assert(bm);
 } else {
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 99dcd5f099..b1a98e 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -1146,10 +1146,13 @@
 # @on-success: The bitmap is only synced when the operation is successful.
 #  This is the behavior always used for 'INCREMENTAL' backups.
 #
+# @never: The bitmap is never synchronized with the operation, and is
+# treated solely as a read-only manifest of blocks to copy.
+#
 # Since: 4.2
 ##
 { 'enum': 'BitmapSyncMode',
-  'data': ['on-success'] }
+  'data': ['on-success', 'never'] }
 
 ##
 # @MirrorCopyMode:
-- 
2.21.0

[Qemu-devel] [PATCH v4 10/18] block/dirty-bitmap: add bdrv_dirty_bitmap_get

Add a public interface for get. While we're at it,
rename "bdrv_get_dirty_bitmap_locked" to "bdrv_dirty_bitmap_get_locked".

(There are more functions to rename to the bdrv_dirty_bitmap_VERB form,
but they will wait until the conclusion of this series.)

Signed-off-by: John Snow 
Reviewed-by: Max Reitz 
---
 block/dirty-bitmap.c | 19 ---
 block/mirror.c   |  2 +-
 include/block/dirty-bitmap.h |  4 ++--
 migration/block.c|  5 ++---
 nbd/server.c |  2 +-
 5 files changed, 18 insertions(+), 14 deletions(-)

diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 7881fea684..75a5daf116 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -509,14 +509,19 @@ BlockDirtyInfoList 
*bdrv_query_dirty_bitmaps(BlockDriverState *bs)
 }
 
 /* Called within bdrv_dirty_bitmap_lock..unlock */
-bool bdrv_get_dirty_locked(BlockDriverState *bs, BdrvDirtyBitmap *bitmap,
-   int64_t offset)
+bool bdrv_dirty_bitmap_get_locked(BdrvDirtyBitmap *bitmap, int64_t offset)
 {
-if (bitmap) {
-return hbitmap_get(bitmap->bitmap, offset);
-} else {
-return false;
-}
+return hbitmap_get(bitmap->bitmap, offset);
+}
+
+bool bdrv_dirty_bitmap_get(BdrvDirtyBitmap *bitmap, int64_t offset)
+{
+bool ret;
+bdrv_dirty_bitmap_lock(bitmap);
+ret = bdrv_dirty_bitmap_get_locked(bitmap, offset);
+bdrv_dirty_bitmap_unlock(bitmap);
+
+return ret;
 }
 
 /**
diff --git a/block/mirror.c b/block/mirror.c
index 75c8f38c6a..63c3ead094 100644
--- a/block/mirror.c
+++ b/block/mirror.c
@@ -476,7 +476,7 @@ static uint64_t coroutine_fn 
mirror_iteration(MirrorBlockJob *s)
 int64_t next_offset = offset + nb_chunks * s->granularity;
 int64_t next_chunk = next_offset / s->granularity;
 if (next_offset >= s->bdev_length ||
-!bdrv_get_dirty_locked(source, s->dirty_bitmap, next_offset)) {
+!bdrv_dirty_bitmap_get_locked(s->dirty_bitmap, next_offset)) {
 break;
 }
 if (test_bit(next_chunk, s->in_flight_bitmap)) {
diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
index 62682eb865..0120ef3f05 100644
--- a/include/block/dirty-bitmap.h
+++ b/include/block/dirty-bitmap.h
@@ -84,12 +84,12 @@ void bdrv_dirty_bitmap_set_busy(BdrvDirtyBitmap *bitmap, 
bool busy);
 void bdrv_merge_dirty_bitmap(BdrvDirtyBitmap *dest, const BdrvDirtyBitmap *src,
  HBitmap **backup, Error **errp);
 void bdrv_dirty_bitmap_set_migration(BdrvDirtyBitmap *bitmap, bool migration);
+bool bdrv_dirty_bitmap_get(BdrvDirtyBitmap *bitmap, int64_t offset);
 
 /* Functions that require manual locking.  */
 void bdrv_dirty_bitmap_lock(BdrvDirtyBitmap *bitmap);
 void bdrv_dirty_bitmap_unlock(BdrvDirtyBitmap *bitmap);
-bool bdrv_get_dirty_locked(BlockDriverState *bs, BdrvDirtyBitmap *bitmap,
-   int64_t offset);
+bool bdrv_dirty_bitmap_get_locked(BdrvDirtyBitmap *bitmap, int64_t offset);
 void bdrv_set_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap,
   int64_t offset, int64_t bytes);
 void bdrv_reset_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap,
diff --git a/migration/block.c b/migration/block.c
index 91f98ef44a..a5b60456ae 100644
--- a/migration/block.c
+++ b/migration/block.c
@@ -520,7 +520,6 @@ static int mig_save_device_dirty(QEMUFile *f, 
BlkMigDevState *bmds,
  int is_async)
 {
 BlkMigBlock *blk;
-BlockDriverState *bs = blk_bs(bmds->blk);
 int64_t total_sectors = bmds->total_sectors;
 int64_t sector;
 int nr_sectors;
@@ -535,8 +534,8 @@ static int mig_save_device_dirty(QEMUFile *f, 
BlkMigDevState *bmds,
 blk_mig_unlock();
 }
 bdrv_dirty_bitmap_lock(bmds->dirty_bitmap);
-if (bdrv_get_dirty_locked(bs, bmds->dirty_bitmap,
-  sector * BDRV_SECTOR_SIZE)) {
+if (bdrv_dirty_bitmap_get_locked(bmds->dirty_bitmap,
+ sector * BDRV_SECTOR_SIZE)) {
 if (total_sectors - sector < BDRV_SECTORS_PER_DIRTY_CHUNK) {
 nr_sectors = total_sectors - sector;
 } else {
diff --git a/nbd/server.c b/nbd/server.c
index 10faedcfc5..fbd51b48a7 100644
--- a/nbd/server.c
+++ b/nbd/server.c
@@ -2003,7 +2003,7 @@ static unsigned int bitmap_to_extents(BdrvDirtyBitmap 
*bitmap, uint64_t offset,
 bdrv_dirty_bitmap_lock(bitmap);
 
 it = bdrv_dirty_iter_new(bitmap);
-dirty = bdrv_get_dirty_locked(NULL, bitmap, offset);
+dirty = bdrv_dirty_bitmap_get_locked(bitmap, offset);
 
 assert(begin < overall_end && nb_extents);
 while (begin < overall_end && i < nb_extents) {
-- 
2.21.0

[Qemu-devel] [PATCH v4 14/18] iotests: teach run_job to cancel pending jobs

run_job can cancel pending jobs to simulate failure. This lets us use
the pending callback to issue test commands while the job is open, but
then still have the job fail in the end.

Signed-off-by: John Snow 
Reviewed-by: Max Reitz 
---
 tests/qemu-iotests/iotests.py | 22 --
 1 file changed, 20 insertions(+), 2 deletions(-)

diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index fcad957d63..c544659ecb 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -541,7 +541,22 @@ class VM(qtest.QEMUQtestMachine):
 
 # Returns None on success, and an error string on failure
 def run_job(self, job, auto_finalize=True, auto_dismiss=False,
-pre_finalize=None, wait=60.0):
+pre_finalize=None, cancel=False, wait=60.0):
+"""
+run_job moves a job from creation through to dismissal.
+
+:param job: String. ID of recently-launched job
+:param auto_finalize: Bool. True if the job was launched with
+  auto_finalize. Defaults to True.
+:param auto_dismiss: Bool. True if the job was launched with
+ auto_dismiss=True. Defaults to False.
+:param pre_finalize: Callback. A callable that takes no arguments to be
+ invoked prior to issuing job-finalize, if any.
+:param cancel: Bool. When true, cancels the job after the pre_finalize
+   callback.
+:param wait: Float. Timeout value specifying how long to wait for any
+ event, in seconds. Defaults to 60.0.
+"""
 match_device = {'data': {'device': job}}
 match_id = {'data': {'id': job}}
 events = [
@@ -568,7 +583,10 @@ class VM(qtest.QEMUQtestMachine):
 elif status == 'pending' and not auto_finalize:
 if pre_finalize:
 pre_finalize()
-self.qmp_log('job-finalize', id=job)
+if cancel:
+self.qmp_log('job-cancel', id=job)
+else:
+self.qmp_log('job-finalize', id=job)
 elif status == 'concluded' and not auto_dismiss:
 self.qmp_log('job-dismiss', id=job)
 elif status == 'null':
-- 
2.21.0

[Qemu-devel] [PATCH v4 01/18] qapi/block-core: Introduce BackupCommon

drive-backup and blockdev-backup have an awful lot of things in common
that are the same. Let's fix that.

I don't deduplicate 'target', because the semantics actually did change
between each structure. Leave that one alone so it can be documented
separately.

Signed-off-by: John Snow 
Reviewed-by: Max Reitz 
---
 qapi/block-core.json | 103 ++-
 1 file changed, 33 insertions(+), 70 deletions(-)

diff --git a/qapi/block-core.json b/qapi/block-core.json
index 0d43d4f37c..0af3866015 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -1315,32 +1315,23 @@
   'data': { 'node': 'str', 'overlay': 'str' } }
 
 ##
-# @DriveBackup:
+# @BackupCommon:
 #
 # @job-id: identifier for the newly-created block job. If
 #  omitted, the device name will be used. (Since 2.7)
 #
 # @device: the device name or node-name of a root node which should be copied.
 #
-# @target: the target of the new image. If the file exists, or if it
-#  is a device, the existing file/device will be used as the new
-#  destination.  If it does not exist, a new file will be created.
-#
-# @format: the format of the new destination, default is to
-#  probe if @mode is 'existing', else the format of the source
-#
 # @sync: what parts of the disk image should be copied to the destination
 #(all the disk, only the sectors allocated in the topmost image, from a
 #dirty bitmap, or only new I/O).
 #
-# @mode: whether and how QEMU should create a new image, default is
-#'absolute-paths'.
-#
-# @speed: the maximum speed, in bytes per second
+# @speed: the maximum speed, in bytes per second. The default is 0,
+# for unlimited.
 #
 # @bitmap: the name of dirty bitmap if sync is "incremental".
 #  Must be present if sync is "incremental", must NOT be present
-#  otherwise. (Since 2.4)
+#  otherwise. (Since 2.4 (drive-backup), 3.1 (blockdev-backup))
 #
 # @compress: true to compress data, if the target format supports it.
 #(default: false) (since 2.8)
@@ -1370,75 +1361,47 @@
 # I/O.  If an error occurs during a guest write request, the device's
 # rerror/werror actions will be used.
 #
+# Since: 4.2
+##
+{ 'struct': 'BackupCommon',
+  'data': { '*job-id': 'str', 'device': 'str',
+'sync': 'MirrorSyncMode', '*speed': 'int',
+'*bitmap': 'str', '*compress': 'bool',
+'*on-source-error': 'BlockdevOnError',
+'*on-target-error': 'BlockdevOnError',
+'*auto-finalize': 'bool', '*auto-dismiss': 'bool' } }
+
+##
+# @DriveBackup:
+#
+# @target: the target of the new image. If the file exists, or if it
+#  is a device, the existing file/device will be used as the new
+#  destination.  If it does not exist, a new file will be created.
+#
+# @format: the format of the new destination, default is to
+#  probe if @mode is 'existing', else the format of the source
+#
+# @mode: whether and how QEMU should create a new image, default is
+#'absolute-paths'.
+#
 # Since: 1.6
 ##
 { 'struct': 'DriveBackup',
-  'data': { '*job-id': 'str', 'device': 'str', 'target': 'str',
-'*format': 'str', 'sync': 'MirrorSyncMode',
-'*mode': 'NewImageMode', '*speed': 'int',
-'*bitmap': 'str', '*compress': 'bool',
-'*on-source-error': 'BlockdevOnError',
-'*on-target-error': 'BlockdevOnError',
-'*auto-finalize': 'bool', '*auto-dismiss': 'bool' } }
+  'base': 'BackupCommon',
+  'data': { 'target': 'str',
+'*format': 'str',
+'*mode': 'NewImageMode' } }
 
 ##
 # @BlockdevBackup:
 #
-# @job-id: identifier for the newly-created block job. If
-#  omitted, the device name will be used. (Since 2.7)
-#
-# @device: the device name or node-name of a root node which should be copied.
-#
 # @target: the device name or node-name of the backup target node.
 #
-# @sync: what parts of the disk image should be copied to the destination
-#(all the disk, only the sectors allocated in the topmost image, or
-#only new I/O).
-#
-# @speed: the maximum speed, in bytes per second. The default is 0,
-# for unlimited.
-#
-# @bitmap: the name of dirty bitmap if sync is "incremental".
-#  Must be present if sync is "incremental", must NOT be present
-#  otherwise. (Since 3.1)
-#
-# @compress: true to compress data, if the target format supports it.
-#(default: false) (since 2.8)
-#
-# @on-source-error: the action to take on an error on the source,
-#   default 'report'.  'stop' and 'enospc' can only be used
-#   if the block device supports io-status (see BlockInfo).
-#
-# @on-target-error: the action to take on an error on the target,
-#   default 'report' (no limitations, since this applies to
-#   a different block device than @device).
-#
-# @auto-finalize: When false, this

[Qemu-devel] [PATCH v4 00/18] bitmaps: introduce 'bitmap' sync mode

This series adds a new "BITMAP" sync mode that is meant to replace the
existing "INCREMENTAL" sync mode.

This mode can have its behavior modified by issuing any of three bitmap sync
modes, passed as arguments to the job.

The three bitmap sync modes are:
- ON-SUCCESS: This is an alias for the old incremental mode. The bitmap is
  conditionally synchronized based on the return code of the job
  upon completion.
- NEVER: This is, effectively, the differential backup mode. It never clears
 the bitmap, as the name suggests.
- ALWAYS: Here is the new, exciting thing. The bitmap is always synchronized,
  even on failure. On success, this is identical to incremental, but
  on failure it clears only the bits that were copied successfully.
  This can be used to "resume" incremental backups from later points
  in times.

I wrote this series by accident on my way to implement incremental mode
for mirror, but this happened first -- the problem is that Mirror mode
uses its existing modes in a very particular way; and this was the best
way to add bitmap support into the mirror job properly.

Summary:
- 01-03: refactor blockdev-backup and drive-backup to share more interface code
- 04-05: add the new 'bitmap' sync mode with sync policy 'conditional',
 which is functionally identical to 'incremental' sync mode.
- 06:add sync policy 'never' ("Differential" backups.)
- 07-11: rework some merging code to facilite patch 12;
- 12:add sync policy 'always' ("Resumable" backups)
- 13-16: test infrastructure changes to support patch 16:
- 17:new iotest!
- 18:minor policy loosening as a QOL improvement

Future work:
 - Update bitmaps.rst to explain these. (WIP, it's hard, sorry!)
 - Add these modes to Mirror. (Done*, but needs tests.)
 - Allow the use of bitmaps and bitmap sync modes with non-BITMAP modes;
   This will allow for resumable/re-tryable full backups.

===
V4:
===

[] : patches are identical
[] : number of functional differences between upstream/downstream patch
[down] : patch is downstream-only
The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively

001/18:[] [--] 'qapi/block-core: Introduce BackupCommon'
002/18:[] [--] 'drive-backup: create do_backup_common'
003/18:[] [--] 'blockdev-backup: utilize do_backup_common'
004/18:[] [--] 'qapi: add BitmapSyncMode enum'
005/18:[] [--] 'block/backup: Add mirror sync mode 'bitmap''
006/18:[] [--] 'block/backup: add 'never' policy to bitmap sync mode'
007/18:[] [--] 'hbitmap: Fix merge when b is empty, and result is not an 
alias of a'
008/18:[] [--] 'hbitmap: enable merging across granularities'
009/18:[0004] [FC] 'block/dirty-bitmap: add bdrv_dirty_bitmap_merge_internal'
010/18:[] [--] 'block/dirty-bitmap: add bdrv_dirty_bitmap_get'
011/18:[0008] [FC] 'block/backup: upgrade copy_bitmap to BdrvDirtyBitmap'
012/18:[] [--] 'block/backup: add 'always' bitmap sync policy'
013/18:[] [--] 'iotests: add testing shim for script-style python tests'
014/18:[] [--] 'iotests: teach run_job to cancel pending jobs'
015/18:[] [--] 'iotests: teach FilePath to produce multiple paths'
016/18:[] [--] 'iotests: Add virtio-scsi device helper'
017/18:[0063] [FC] 'iotests: add test 257 for bitmap-mode backups'
018/18:[] [--] 'block/backup: loosen restriction on readonly bitmaps'

Changes:
009: Added assertions.
011: Moved copy bitmap to source node.
017: Rework get_bitmap to tolerate multiple anonymous bitmaps
 Update test output to accommodate the same.

===
V3:
===

Changes:
001: Made suggested doc fixes.
 Changed 'since' to 4.2.
002: Added bds and aio_context to backup_common
 Removed accidental extraneous unref on target_bs
 Removed local_err propagation
003: Fallout from #002; hoist aio_context acquisition up into do_blockdev_backup
004: 'conditional' --> 'on-success'
005: Rediscover the lost stanza that ensures a bitmap mode was given
 Fallout from 2, 3, 4.
006: Block comment fix for patchew
 Fallout from #4
009: Fix assert() style issue. Why'd they let a macro be lowercase like that?
 Probably to make specifically my life difficult.
010: Fix style issue {
011: Fix long lines
 rename "bs" --> "target_bs" where appropriate
 Free copy_bitmap from the right node
012: Multiline comment changes for patchew
 Fallout from #4
015: Fix long line for patchew
 Reinstate that second newline that Max likes
017: Fallout from #4.

===
V2:
===

Changes:
004: Fixed typo
 Change @conditional docstring
005: Moved desugaring code into blockdev.c, facilitated by patches 1-3.
006: Change @never docstring slightly.
007: Merge will clear the target bitmap when both components bitmaps are empty,
 and the target bitmap is not an alias of either component bitmap.
008: Check orig_size (logical size) instead of size (actual size) to enable
 cross-granularity merging.
 Fix the

[Qemu-devel] [PATCH v4 07/18] hbitmap: Fix merge when b is empty, and result is not an alias of a

Nobody calls the function like this currently, but we neither prohibit
or cope with this behavior. I decided to make the function cope with it.

Signed-off-by: John Snow 
Reviewed-by: Max Reitz 
---
 util/hbitmap.c | 13 ++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/util/hbitmap.c b/util/hbitmap.c
index 7905212a8b..3b6acae42b 100644
--- a/util/hbitmap.c
+++ b/util/hbitmap.c
@@ -781,8 +781,9 @@ bool hbitmap_can_merge(const HBitmap *a, const HBitmap *b)
 }
 
 /**
- * Given HBitmaps A and B, let A := A (BITOR) B.
- * Bitmap B will not be modified.
+ * Given HBitmaps A and B, let R := A (BITOR) B.
+ * Bitmaps A and B will not be modified,
+ * except when bitmap R is an alias of A or B.
  *
  * @return true if the merge was successful,
  * false if it was not attempted.
@@ -797,7 +798,13 @@ bool hbitmap_merge(const HBitmap *a, const HBitmap *b, 
HBitmap *result)
 }
 assert(hbitmap_can_merge(b, result));
 
-if (hbitmap_count(b) == 0) {
+if ((!hbitmap_count(a) && result == b) ||
+(!hbitmap_count(b) && result == a)) {
+return true;
+}
+
+if (!hbitmap_count(a) && !hbitmap_count(b)) {
+hbitmap_reset_all(result);
 return true;
 }
 
-- 
2.21.0

[Qemu-devel] [PATCH v4 08/18] hbitmap: enable merging across granularities

Signed-off-by: John Snow 
Reviewed-by: Max Reitz 
---
 util/hbitmap.c | 36 +++-
 1 file changed, 35 insertions(+), 1 deletion(-)

diff --git a/util/hbitmap.c b/util/hbitmap.c
index 3b6acae42b..306bc4876d 100644
--- a/util/hbitmap.c
+++ b/util/hbitmap.c
@@ -777,7 +777,27 @@ void hbitmap_truncate(HBitmap *hb, uint64_t size)
 
 bool hbitmap_can_merge(const HBitmap *a, const HBitmap *b)
 {
-return (a->size == b->size) && (a->granularity == b->granularity);
+return (a->orig_size == b->orig_size);
+}
+
+/**
+ * hbitmap_sparse_merge: performs dst = dst | src
+ * works with differing granularities.
+ * best used when src is sparsely populated.
+ */
+static void hbitmap_sparse_merge(HBitmap *dst, const HBitmap *src)
+{
+uint64_t offset = 0;
+uint64_t count = src->orig_size;
+
+while (hbitmap_next_dirty_area(src, , )) {
+hbitmap_set(dst, offset, count);
+offset += count;
+if (offset >= src->orig_size) {
+break;
+}
+count = src->orig_size - offset;
+}
 }
 
 /**
@@ -808,10 +828,24 @@ bool hbitmap_merge(const HBitmap *a, const HBitmap *b, 
HBitmap *result)
 return true;
 }
 
+if (a->granularity != b->granularity) {
+if ((a != result) && (b != result)) {
+hbitmap_reset_all(result);
+}
+if (a != result) {
+hbitmap_sparse_merge(result, a);
+}
+if (b != result) {
+hbitmap_sparse_merge(result, b);
+}
+return true;
+}
+
 /* This merge is O(size), as BITS_PER_LONG and HBITMAP_LEVELS are constant.
  * It may be possible to improve running times for sparsely populated maps
  * by using hbitmap_iter_next, but this is suboptimal for dense maps.
  */
+assert(a->size == b->size);
 for (i = HBITMAP_LEVELS - 1; i >= 0; i--) {
 for (j = 0; j < a->sizes[i]; j++) {
 result->levels[i][j] = a->levels[i][j] | b->levels[i][j];
-- 
2.21.0

Re: [Qemu-devel] [PATCH v3 17/18] iotests: add test 257 for bitmap-mode backups




On 7/9/19 2:49 PM, Max Reitz wrote:
> On 05.07.19 22:16, John Snow wrote:
>> Signed-off-by: John Snow 
>> ---
>>  tests/qemu-iotests/257 |  409 +++
>>  tests/qemu-iotests/257.out | 2199 
>>  tests/qemu-iotests/group   |1 +
>>  3 files changed, 2609 insertions(+)
>>  create mode 100755 tests/qemu-iotests/257
>>  create mode 100644 tests/qemu-iotests/257.out
> 
> Reviewed-by: Max Reitz 
> 

Thanks for all the reviews so far :)

Despite my grumpiness I do rather like the way this turned out.

--js

Re: [Qemu-devel] [qemu-s390x] [PATCH for-4.1 1/2] s390x/pci: add some fallthrough annotations

2019-07-09 Thread Collin Walling


On 7/8/19 9:23 AM, Christian Borntraeger wrote:



On 08.07.19 14:54, Cornelia Huck wrote:

According to the comment, the bits are supposed to accumulate.

Reported-by: Stefan Weil 
Fixes: 5d1abf234462 ("s390x/pci: enforce zPCI state checking")
Signed-off-by: Cornelia Huck 


This patch does not change behaviour, so it is certainly not wrong.

So lets have a look at if the bug report was actually a real bug or
just a missing annotation.


---
  hw/s390x/s390-pci-inst.c | 2 ++
  1 file changed, 2 insertions(+)

diff --git a/hw/s390x/s390-pci-inst.c b/hw/s390x/s390-pci-inst.c
index 61f30b8e55d2..00235148bed7 100644
--- a/hw/s390x/s390-pci-inst.c
+++ b/hw/s390x/s390-pci-inst.c
@@ -1209,8 +1209,10 @@ int stpcifc_service_call(S390CPU *cpu, uint8_t r1, 
uint64_t fiba, uint8_t ar,
   * FH Enabled bit is set to one in states of ENABLED, BLOCKED or ERROR. */
  case ZPCI_FS_ERROR:
  fib.fc |= 0x20;
+/* fallthrough */


This is correct, in case of an error we are also blocked.



Agreed. This is definitely correct based on our architecture.


  case ZPCI_FS_BLOCKED:
  fib.fc |= 0x40;
+/* fallthrough */


I think this is also correct, but  it would be good if Collin could verify.



I failed to find anything to support setting the function control
enabled bit when the function state is in error / blocked. I'm
assuming this might be some QEMU hack to get things working? I'll have
to dive further to understand why this was done this way, as it doesn't
align with how the s390x architecture is documented. It's confusing.

Functionally, this doesn't change anything... so I'll at least give it
an ACK for now and investigate this further.


  case ZPCI_FS_ENABLED:
  fib.fc |= 0x80;
  if (pbdev->iommu->enabled) {






Acked-by: Collin Walling 

Side note: is there somewhere that I could access this bug report? :)

Re: [Qemu-devel] [PATCH 00/11] Multiple simultaneous audio backends

Patchew URL: 
https://patchew.org/QEMU/cover.1562695780.git.dirty.ice...@gmail.com/



Hi,

This series failed the asan build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
make docker-image-fedora V=1 NETWORK=1
time make docker-test-debug@fedora TARGET_LIST=x86_64-softmmu J=14 NETWORK=1
=== TEST SCRIPT END ===

PASS 5 test-qmp-cmds /qmp/dispatch_cmd_success_response
PASS 6 test-qmp-cmds /qmp/dealloc_types
PASS 7 test-qmp-cmds /qmp/dealloc_partial
==9597==WARNING: ASan doesn't fully support makecontext/swapcontext functions 
and may produce false positives in some cases!
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  
tests/test-string-input-visitor -m=quick -k --tap < /dev/null | 
./scripts/tap-driver.pl --test-name="test-string-input-visitor" 
PASS 4 fdc-test /x86_64/fdc/media_change
PASS 5 fdc-test /x86_64/fdc/sense_interrupt
---
PASS 32 test-opts-visitor /visitor/opts/range/beyond
PASS 33 test-opts-visitor /visitor/opts/dict/unvisited
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  
tests/test-coroutine -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl 
--test-name="test-coroutine" 
==9635==WARNING: ASan doesn't fully support makecontext/swapcontext functions 
and may produce false positives in some cases!
==9635==WARNING: ASan is ignoring requested __asan_handle_no_return: stack top: 
0x7ffc4c709000; bottom 0x7f5cdbaf8000; size: 0x009f70c11000 (684791500800)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
PASS 1 test-coroutine /basic/no-dangling-access
---
PASS 12 test-aio /aio/event/flush
PASS 13 test-aio /aio/event/wait/no-flush-cb
PASS 11 fdc-test /x86_64/fdc/read_no_dma_18
==9652==WARNING: ASan doesn't fully support makecontext/swapcontext functions 
and may produce false positives in some cases!
PASS 14 test-aio /aio/timer/schedule
PASS 15 test-aio /aio/coroutine/queue-chaining
PASS 16 test-aio /aio-gsource/flush
---
PASS 27 test-aio /aio-gsource/event/wait/no-flush-cb
PASS 28 test-aio /aio-gsource/timer/schedule
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  
tests/test-aio-multithread -m=quick -k --tap < /dev/null | 
./scripts/tap-driver.pl --test-name="test-aio-multithread" 
==9658==WARNING: ASan doesn't fully support makecontext/swapcontext functions 
and may produce false positives in some cases!
PASS 1 test-aio-multithread /aio/multi/lifecycle
PASS 2 test-aio-multithread /aio/multi/schedule
PASS 12 fdc-test /x86_64/fdc/read_no_dma_19
PASS 13 fdc-test /x86_64/fdc/fuzz-registers
PASS 3 test-aio-multithread /aio/multi/mutex/contended
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  
QTEST_QEMU_BINARY=x86_64-softmmu/qemu-system-x86_64 QTEST_QEMU_IMG=qemu-img 
tests/ide-test -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl 
--test-name="ide-test" 
==9686==WARNING: ASan doesn't fully support makecontext/swapcontext functions 
and may produce false positives in some cases!
PASS 1 ide-test /x86_64/ide/identify
==9692==WARNING: ASan doesn't fully support makecontext/swapcontext functions 
and may produce false positives in some cases!
PASS 2 ide-test /x86_64/ide/flush
==9698==WARNING: ASan doesn't fully support makecontext/swapcontext functions 
and may produce false positives in some cases!
PASS 3 ide-test /x86_64/ide/bmdma/simple_rw
PASS 4 test-aio-multithread /aio/multi/mutex/handoff
==9704==WARNING: ASan doesn't fully support makecontext/swapcontext functions 
and may produce false positives in some cases!
PASS 4 ide-test /x86_64/ide/bmdma/trim
PASS 5 test-aio-multithread /aio/multi/mutex/mcs
==9715==WARNING: ASan doesn't fully support makecontext/swapcontext functions 
and may produce false positives in some cases!
PASS 5 ide-test /x86_64/ide/bmdma/short_prdt
PASS 6 test-aio-multithread /aio/multi/mutex/pthread
==9726==WARNING: ASan doesn't fully support makecontext/swapcontext functions 
and may produce false positives in some cases!
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  
tests/test-throttle -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl 
--test-name="test-throttle" 
PASS 1 test-throttle /throttle/leak_bucket
PASS 2 test-throttle /throttle/compute_wait
---
PASS 6 test-throttle /throttle/detach_attach
PASS 7 test-throttle /throttle/config_functions
PASS 8 test-throttle /throttle/accounting
==9733==WARNING: ASan doesn't fully support makecontext/swapcontext functions 
and may produce false positives in some cases!
PASS 9 test-throttle /throttle/groups
PASS 10 test-throttle /throttle/config/enabled
PASS 11 test-throttle /throttle/config/conflicting
---
PASS 15 test-throttle /throttle/config/iops_size
PASS 6 ide-test /x86_64/ide/bmdma/one_sector_short_prdt
MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))}  
tests/test-thread-pool -m=quick -k --tap < /dev/null | ./scripts/tap-driver.pl

Re: [Qemu-devel] [PATCH 00/11] Multiple simultaneous audio backends

Patchew URL: 
https://patchew.org/QEMU/cover.1562695780.git.dirty.ice...@gmail.com/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Type: series
Message-id: cover.1562695780.git.dirty.ice...@gmail.com
Subject: [Qemu-devel] [PATCH 00/11] Multiple simultaneous audio backends

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
 * [new tag] patchew/20190709213651.77315-1-mrol...@gmail.com -> 
patchew/20190709213651.77315-1-mrol...@gmail.com
Switched to a new branch 'test'
3080c9b audio: use size_t where makes sense
c4faae9 audio: remove read and write pcm_ops
8d1cbe2 paaudio: fix playback glitches
dbeb913 audio: do not run each backend in audio_run
ec4e34e audio: remove audio_MIN, audio_MAX
b36085a paaudio: properly disconnect streams in fini_*
db53be0 paaudio: do not move stream when sink/source name is specified
052ab5c audio: audiodev= parameters no longer optional when -audiodev present
c0cf5a5 audio: add audiodev properties to frontends
15c571f audio: basic support for multi backend audio
88dfd06 audio: reduce glob_audio_state usage

=== OUTPUT BEGIN ===
1/11 Checking commit 88dfd065d232 (audio: reduce glob_audio_state usage)
2/11 Checking commit 15c571f4acb7 (audio: basic support for multi backend audio)
ERROR: space prohibited between function name and open parenthesis '('
#168: FILE: audio/audio.c:1526:
+QLIST_INSERT_HEAD (>state->card_head, card, entries);

total: 1 errors, 0 warnings, 366 lines checked

Patch 2/11 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

3/11 Checking commit c0cf5a5363ac (audio: add audiodev properties to frontends)
4/11 Checking commit 052ab5cd93bd (audio: audiodev= parameters no longer 
optional when -audiodev present)
5/11 Checking commit db53be05092d (paaudio: do not move stream when sink/source 
name is specified)
6/11 Checking commit b36085a0bf72 (paaudio: properly disconnect streams in 
fini_*)
7/11 Checking commit ec4e34eade3d (audio: remove audio_MIN, audio_MAX)
ERROR: space prohibited between function name and open parenthesis '('
#22: FILE: audio/alsaaudio.c:637:
+int len = MIN (alsa->pending, left_till_end_samples);

ERROR: space prohibited between function name and open parenthesis '('
#31: FILE: audio/alsaaudio.c:700:
+decr = MIN (live, avail);

ERROR: space prohibited between function name and open parenthesis '('
#40: FILE: audio/alsaaudio.c:918:
+decr = MIN (dead, avail);

ERROR: space prohibited between function name and open parenthesis '('
#53: FILE: audio/audio.c:536:
+m = MIN (m, sw->total_hw_samples_acquired);

ERROR: space prohibited between function name and open parenthesis '('
#62: FILE: audio/audio.c:556:
+int len = MIN (left, live);

ERROR: space prohibited between function name and open parenthesis '('
#70: FILE: audio/audio.c:563:
+int samples_to_clip = MIN (len, samples_till_end_of_buf);

ERROR: space prohibited between function name and open parenthesis '('
#79: FILE: audio/audio.c:617:
+swlim = MIN (swlim, samples);

ERROR: space prohibited between function name and open parenthesis '('
#88: FILE: audio/audio.c:665:
+m = MIN (m, sw->total_hw_samples_mixed);

ERROR: space prohibited between function name and open parenthesis '('
#97: FILE: audio/audio.c:728:
+swlim = MIN (swlim, samples);

ERROR: space prohibited between function name and open parenthesis '('
#106: FILE: audio/audio.c:740:
+blck = MIN (dead, left);

ERROR: space prohibited between function name and open parenthesis '('
#115: FILE: audio/audio.c:1032:
+int to_write = MIN (till_end_of_hw, n);

ERROR: space prohibited between function name and open parenthesis '('
#124: FILE: audio/audio.c:1050:
+n = MIN (samples, hw->samples - rpos);

ERROR: space prohibited between function name and open parenthesis '('
#133: FILE: audio/audio.c:1206:
+int to_capture = MIN (live, left);

ERROR: space prohibited between function name and open parenthesis '('
#174: FILE: audio/coreaudio.c:416:
+decr = MIN (core->decr, live);

ERROR: space prohibited between function name and open parenthesis '('
#187: FILE: audio/dsoundaudio.c:710:
+len = MIN (len, dead);

ERROR: space prohibited between function name and open parenthesis '('
#205: FILE: audio/noaudio.c:59:
+decr = MIN (live, samples);

ERROR: space prohibited between function name and open parenthesis '('
#214: FILE: audio/noaudio.c:114:
+bytes = MIN (bytes, INT_MAX);

ERROR: space prohibited between function name and open parenthesis '('
#217: FILE:

Re: [Qemu-devel] Should memory hotplug work with vhost-user backends?

2019-07-09 Thread Raphael Norwitz

On Jul 3, 2019, at 11:57 AM, Michael S. Tsirkin
mailto:m...@redhat.com>> wrote:

On Tue, Jul 02, 2019 at 10:08:54PM +, Raphael Norwitz wrote:
For background I am trying to work around a ram slot limit imposed by the
vhost-user protocol. We are having trouble reconciling the comment here: https:
//github.com/qemu/qemu/blob/master/hw/virtio/vhost-user.c#L333
that “For
non-vring specific requests, like VHOST_USER_SET_MEM_TABLE., we just need to
send it once the first time” and the high level implementation of memory
hot-add, which calls set_mem_table every time a VM hot adds memory.

IIUC the comment refers to multiple virtqueue. It is trying to say that
we do not need to send VHOST_USER_SET_MEM_TABLE for each
virtqueue.

A few questions:

What exactly is the check `if (vhost_user_one_time_request(msg->hdr.request) &&
dev->vq_index != 0)` for?

Some backends register multiple dev instances per backend: one for each
virtqueue. This check avoids sending VHOST_USER_SET_MEM_TABLE more than
once in this case.

In the message for commit
b931bfbf042983f311b3b09894d8030b2755a638, which introduced the check, I see it
says “non-vring specific messages[, which should] be sent only once” and gives
VHOST_USER_SET_MEM_TABLE as an example one such message. The
`vhost_user_one_time_request()` call clearly checks whether this type of
message is the kind of message is supposed to be sent once of which
VHOST_USER_SET_MEM_TABLE is one. Why, then, does this commit add the check if
`dev->vq_index != 0`? It seems like there is a latent assumption that after the
first call dev->vq_index should be set to some value greater than one, however
for many cases such as vhost-user-scsi devices we can see this is clearly not
the case
https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_qemu_qemu_blob_master_hw_scsi_vhost-2Duser-2Dscsi.c-23=DwIDaQ=s883GpUCOChKOHiocYtGcg=In4gmR1pGzKB8G5p6LUrWqkSMec2L5EtXZow_FZNJZk=JQ2hn-RhTfyhZ0Fxq5mzERaoVzb7cT8vcI7Xv20J3yk=aIeMP5ifKxF3no26c_M5x1rnVH0fhoFU2iRG9BPRNBk=
L95. Is this check then ‘broken’ for such devices?

I think vhost-scsi has a single instance per backend, that is
why vq_index is 0.

If this check is indeed broken for such devices, and set_mem_table call is only
supposed to be run once for such devices, is the ability to call it multiple
times technically a bug for devices such as vhost-user-scsci devices? If so,
this would imply that the existing ability to hot add memory to vhost-user-scsi
devices is by extension technically a bug/unintended behavior. Is this the
case?

Thanks,

Raphael

I don't think that is the case. It's possible that memory hotplug has
bugs with vhost-user, but I don't think it's anything fundamental.

--
MST

Thanks for the responses Stefan and MST, I’ll be looking into it and posting an
RFC when I'm ready.

Re: [Qemu-devel] [PATCH v24 0/7] QEMU AVR 8 bit cores

Hi Philippe.

I am working on it.

On Tue, Jul 2, 2019 at 6:55 PM Philippe Mathieu-Daudé 
wrote:

> Hi Michael,
>
> On 6/28/19 2:01 PM, Michael Rolnik wrote:
> > This series of patches adds 8bit AVR cores to QEMU.
> > All instruction, except BREAK/DES/SPM/SPMX, are implemented. Not fully
> tested yet.
> > However I was able to execute simple code with functions. e.g fibonacci
> calculation.
> > This series of patches include a non real, sample board.
> > No fuses support yet. PC is set to 0 at reset.
>
> I see in this thread you test some binary:
> https://lists.gnu.org/archive/html/qemu-devel/2019-06/msg02291.html
>
>
> https://github.com/seharris/qemu-avr-tests/blob/master/free-rtos/Demo/AVR_ATMega2560_GCC/demo.elf
>
> Can you add a test that uses your binary and check it does something?
> That would help maintaining your work over time.
>
> Avocado tests can be quite simple, i.e.:
> https://lists.gnu.org/archive/html/qemu-devel/2019-06/msg06515.html
>
> Since you added support for the USART, you can also add some
> assembler instructions to use it to this test:
>
>
> https://git.qemu.org/?p=qemu.git;a=blob;f=tests/boot-serial-test.c;h=24852d4c7d0b3fc08fb0dab35f32372a0b2c46db;hb=HEAD
>
> Thanks!
>
> Phil.
>


-- 
Best Regards,
Michael Rolnik

[Qemu-devel] [PATCH v25 4/7] target/avr: Add instruction translation

This includes:
- TCG translations for each instruction

Signed-off-by: Michael Rolnik 
---
 target/avr/translate.c | 2888 
 1 file changed, 2888 insertions(+)
 create mode 100644 target/avr/translate.c

diff --git a/target/avr/translate.c b/target/avr/translate.c
new file mode 100644
index 00..951a48067e
--- /dev/null
+++ b/target/avr/translate.c
@@ -0,0 +1,2888 @@
+/*
+ * QEMU AVR CPU
+ *
+ * Copyright (c) 2016 Michael Rolnik
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see
+ * 
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/qemu-print.h"
+#include "tcg/tcg.h"
+#include "cpu.h"
+#include "exec/exec-all.h"
+#include "disas/disas.h"
+#include "tcg-op.h"
+#include "exec/cpu_ldst.h"
+#include "exec/helper-proto.h"
+#include "exec/helper-gen.h"
+#include "exec/log.h"
+#include "exec/gdbstub.h"
+#include "exec/translator.h"
+
+/*
+ *  Define if you want a BREAK instruction translated to a breakpoint
+ *  Active debugging connection is assumed
+ *  This is for
+ *  https://github.com/seharris/qemu-avr-tests/tree/master/instruction-tests
+ *  tests
+ */
+#undef BREAKPOINT_ON_BREAK
+
+static TCGv cpu_pc;
+
+static TCGv cpu_Cf;
+static TCGv cpu_Zf;
+static TCGv cpu_Nf;
+static TCGv cpu_Vf;
+static TCGv cpu_Sf;
+static TCGv cpu_Hf;
+static TCGv cpu_Tf;
+static TCGv cpu_If;
+
+static TCGv cpu_rampD;
+static TCGv cpu_rampX;
+static TCGv cpu_rampY;
+static TCGv cpu_rampZ;
+
+static TCGv cpu_r[NO_CPU_REGISTERS];
+static TCGv cpu_eind;
+static TCGv cpu_sp;
+
+static TCGv cpu_skip;
+
+static const char reg_names[NO_CPU_REGISTERS][8] = {
+"r0",  "r1",  "r2",  "r3",  "r4",  "r5",  "r6",  "r7",
+"r8",  "r9",  "r10", "r11", "r12", "r13", "r14", "r15",
+"r16", "r17", "r18", "r19", "r20", "r21", "r22", "r23",
+"r24", "r25", "r26", "r27", "r28", "r29", "r30", "r31",
+};
+#define REG(x) (cpu_r[x])
+
+enum {
+DISAS_EXIT   = DISAS_TARGET_0,  /* We want return to the cpu main loop.  */
+DISAS_LOOKUP = DISAS_TARGET_1,  /* We have a variable condition exit.  */
+DISAS_CHAIN  = DISAS_TARGET_2,  /* We have a single condition exit.  */
+};
+
+typedef struct DisasContext DisasContext;
+
+/* This is the state at translation time. */
+struct DisasContext {
+TranslationBlock *tb;
+
+CPUAVRState *env;
+CPUState *cs;
+
+target_long npc;
+uint32_t opcode;
+
+/* Routine used to access memory */
+int memidx;
+int bstate;
+int singlestep;
+
+TCGv skip_var0;
+TCGv skip_var1;
+TCGCond skip_cond;
+bool free_skip_var0;
+};
+
+static int to_A(DisasContext *ctx, int indx) { return 16 + (indx % 16); }
+static int to_B(DisasContext *ctx, int indx) { return 16 + (indx % 8); }
+static int to_C(DisasContext *ctx, int indx) { return 24 + (indx % 4) * 2; }
+static int to_D(DisasContext *ctx, int indx) { return (indx % 16) * 2; }
+
+static uint16_t next_word(DisasContext *ctx)
+{
+return cpu_lduw_code(ctx->env, ctx->npc++ * 2);
+}
+
+static int append_16(DisasContext *ctx, int x)
+{
+return x << 16 | next_word(ctx);
+}
+
+static bool decode_insn(DisasContext *ctx, uint16_t insn);
+#include "decode_insn.inc.c"
+
+static bool avr_have_feature(DisasContext *ctx, int feature)
+{
+if (!avr_feature(ctx->env, feature)) {
+gen_helper_unsupported(cpu_env);
+ctx->bstate = DISAS_NORETURN;
+return false;
+}
+return true;
+}
+
+static void gen_goto_tb(DisasContext *ctx, int n, target_ulong dest)
+{
+TranslationBlock *tb = ctx->tb;
+
+if (ctx->singlestep == 0) {
+tcg_gen_goto_tb(n);
+tcg_gen_movi_i32(cpu_pc, dest);
+tcg_gen_exit_tb(tb, n);
+} else {
+tcg_gen_movi_i32(cpu_pc, dest);
+gen_helper_debug(cpu_env);
+tcg_gen_exit_tb(NULL, 0);
+}
+ctx->bstate = DISAS_NORETURN;
+}
+
+#include "exec/gen-icount.h"
+
+static void gen_add_CHf(TCGv R, TCGv Rd, TCGv Rr)
+{
+TCGv t1 = tcg_temp_new_i32();
+TCGv t2 = tcg_temp_new_i32();
+TCGv t3 = tcg_temp_new_i32();
+
+tcg_gen_and_tl(t1, Rd, Rr); /* t1 = Rd & Rr */
+tcg_gen_andc_tl(t2, Rd, R); /* t2 = Rd & ~R */
+tcg_gen_andc_tl(t3, Rr, R); /* t3 = Rr & ~R */
+tcg_gen_or_tl(t1, t1, t2); /* t1 = t1 | t2 | t3 */
+tcg_gen_or_tl(t1, t1, t3);
+
+tcg_gen_shri_tl(cpu_Cf, t1, 7); /* Cf = t1(7) */
+tcg_gen_shri_tl(cpu_Hf, t1, 3); /* Hf =

[Qemu-devel] [PATCH v25 6/7] target/avr: Add example board configuration

From: Sarah Harris 

A simple board setup that configures an AVR CPU to run a given firmware image.
This is all that's useful to implement without peripheral emulation as AVR CPUs 
include a lot of on-board peripherals.

Signed-off-by: Michael Rolnik 
---
 hw/Kconfig   |   1 +
 hw/avr/Kconfig   |   4 +
 hw/avr/Makefile.objs |   1 +
 hw/avr/sample.c  | 217 +++
 4 files changed, 223 insertions(+)
 create mode 100644 hw/avr/Kconfig
 create mode 100644 hw/avr/Makefile.objs
 create mode 100644 hw/avr/sample.c

diff --git a/hw/Kconfig b/hw/Kconfig
index 195f541e50..1f25636855 100644
--- a/hw/Kconfig
+++ b/hw/Kconfig
@@ -42,6 +42,7 @@ source watchdog/Kconfig
 # arch Kconfig
 source arm/Kconfig
 source alpha/Kconfig
+source avr/Kconfig
 source cris/Kconfig
 source hppa/Kconfig
 source i386/Kconfig
diff --git a/hw/avr/Kconfig b/hw/avr/Kconfig
new file mode 100644
index 00..c6ca8fe775
--- /dev/null
+++ b/hw/avr/Kconfig
@@ -0,0 +1,4 @@
+config AVR_SAMPLE
+bool
+select AVR_TIMER16
+select AVR_USART
diff --git a/hw/avr/Makefile.objs b/hw/avr/Makefile.objs
new file mode 100644
index 00..626b7064b3
--- /dev/null
+++ b/hw/avr/Makefile.objs
@@ -0,0 +1 @@
+obj-y += sample.o
diff --git a/hw/avr/sample.c b/hw/avr/sample.c
new file mode 100644
index 00..e4cb548a33
--- /dev/null
+++ b/hw/avr/sample.c
@@ -0,0 +1,217 @@
+/*
+ * QEMU AVR CPU
+ *
+ * Copyright (c) 2016 Michael Rolnik
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see
+ * 
+ */
+
+/*
+ *  NOTE:
+ *  This is not a real AVR board, this is an example!
+ *  The CPU is an approximation of an ATmega2560, but is missing various
+ *  built-in peripherals.
+ *
+ *  This example board loads provided binary file into flash memory and
+ *  executes it from 0x address in the code memory space.
+ *
+ *  Currently used for AVR CPU validation
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "qemu-common.h"
+#include "cpu.h"
+#include "hw/hw.h"
+#include "sysemu/sysemu.h"
+#include "sysemu/qtest.h"
+#include "ui/console.h"
+#include "hw/boards.h"
+#include "hw/loader.h"
+#include "qemu/error-report.h"
+#include "exec/address-spaces.h"
+#include "include/hw/sysbus.h"
+#include "include/hw/char/avr_usart.h"
+#include "include/hw/timer/avr_timer16.h"
+#include "elf.h"
+
+#define SIZE_FLASH 0x0004
+#define SIZE_SRAM 0x2200
+/*
+ * Size of additional "external" memory, as if the AVR were configured to use
+ * an external RAM chip.
+ * Note that the configuration registers that normally enable this feature are
+ * unimplemented.
+ */
+#define SIZE_EXMEM 0x
+
+/* Offsets of periphals in emulated memory space (i.e. not host addresses)  */
+#define PRR0 0x64
+#define PRR1 0x65
+#define USART_BASE 0xc0
+#define USART_PRR PRR0
+#define USART_PRR_MASK 0b0010
+#define TIMER1_BASE 0x80
+#define TIMER1_IMSK_BASE 0x6f
+#define TIMER1_IFR_BASE 0x36
+#define TIMER1_PRR PRR0
+#define TIMER1_PRR_MASK 0b0100
+
+/* Interrupt numbers used by peripherals */
+#define USART_RXC_IRQ 24
+#define USART_DRE_IRQ 25
+#define USART_TXC_IRQ 26
+
+#define TIMER1_CAPT_IRQ 15
+#define TIMER1_COMPA_IRQ 16
+#define TIMER1_COMPB_IRQ 17
+#define TIMER1_COMPC_IRQ 18
+#define TIMER1_OVF_IRQ 19
+
+typedef struct {
+MachineClass parent;
+} SampleMachineClass;
+
+typedef struct {
+MachineState parent;
+MemoryRegion *ram;
+MemoryRegion *flash;
+AVRUsartState *usart0;
+AVRTimer16State *timer1;
+} SampleMachineState;
+
+#define TYPE_SAMPLE_MACHINE MACHINE_TYPE_NAME("sample")
+
+#define SAMPLE_MACHINE(obj) \
+OBJECT_CHECK(SampleMachineState, obj, TYPE_SAMPLE_MACHINE)
+#define SAMPLE_MACHINE_GET_CLASS(obj) \
+OBJECT_GET_CLASS(SampleMachineClass, obj, TYPE_SAMPLE_MACHINE)
+#define SAMPLE_MACHINE_CLASS(klass) \
+OBJECT_CLASS_CHECK(SampleMachineClass, klass, TYPE_SAMPLE_MACHINE)
+
+static void sample_init(MachineState *machine)
+{
+SampleMachineState *sms = SAMPLE_MACHINE(machine);
+MemoryRegion *system_memory = get_system_memory();
+AVRCPU *cpu;
+const char *firmware = NULL;
+const char *filename;
+int bytes_loaded;
+SysBusDevice *busdev;
+DeviceState *cpudev;
+
+system_memory = get_system_memory();
+sms->ram = g_new(MemoryRegion, 1);
+sms->flash =

[Qemu-devel] [PATCH v25 5/7] target/avr: Add limited support for USART and 16 bit timer peripherals

From: Sarah Harris 

These were designed to facilitate testing but should provide enough function to 
be useful in other contexts.
Only a subset of the functions of each peripheral is implemented, mainly due to 
the lack of a standard way to handle electrical connections (like GPIO pins).

Signed-off-by: Michael Rolnik 
---
 hw/char/Kconfig|   3 +
 hw/char/Makefile.objs  |   1 +
 hw/char/avr_usart.c| 316 ++
 hw/timer/Kconfig   |   3 +
 hw/timer/Makefile.objs |   1 +
 hw/timer/avr_timer16.c | 587 +
 include/hw/char/avr_usart.h|  99 ++
 include/hw/timer/avr_timer16.h |  99 ++
 8 files changed, 1109 insertions(+)
 create mode 100644 hw/char/avr_usart.c
 create mode 100644 hw/timer/avr_timer16.c
 create mode 100644 include/hw/char/avr_usart.h
 create mode 100644 include/hw/timer/avr_timer16.h

diff --git a/hw/char/Kconfig b/hw/char/Kconfig
index 40e7a8b8bb..331b20983f 100644
--- a/hw/char/Kconfig
+++ b/hw/char/Kconfig
@@ -46,3 +46,6 @@ config SCLPCONSOLE
 
 config TERMINAL3270
 bool
+
+config AVR_USART
+bool
diff --git a/hw/char/Makefile.objs b/hw/char/Makefile.objs
index 02d8a66925..09ed50f1d0 100644
--- a/hw/char/Makefile.objs
+++ b/hw/char/Makefile.objs
@@ -21,6 +21,7 @@ obj-$(CONFIG_PSERIES) += spapr_vty.o
 obj-$(CONFIG_DIGIC) += digic-uart.o
 obj-$(CONFIG_STM32F2XX_USART) += stm32f2xx_usart.o
 obj-$(CONFIG_RASPI) += bcm2835_aux.o
+obj-$(CONFIG_AVR_USART) += avr_usart.o
 
 common-obj-$(CONFIG_CMSDK_APB_UART) += cmsdk-apb-uart.o
 common-obj-$(CONFIG_ETRAXFS) += etraxfs_ser.o
diff --git a/hw/char/avr_usart.c b/hw/char/avr_usart.c
new file mode 100644
index 00..26c711336b
--- /dev/null
+++ b/hw/char/avr_usart.c
@@ -0,0 +1,316 @@
+/*
+ * AVR USART
+ *
+ * Copyright (c) 2018 University of Kent
+ * Author: Sarah Harris
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to 
deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING 
FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+#include "qemu/osdep.h"
+#include "hw/char/avr_usart.h"
+#include "qemu/log.h"
+
+static int avr_usart_can_receive(void *opaque)
+{
+AVRUsartState *usart = opaque;
+
+if (usart->data_valid || !(usart->csrb & USART_CSRB_RXEN)) {
+return 0;
+}
+return 1;
+}
+
+static void avr_usart_receive(void *opaque, const uint8_t *buffer, int size)
+{
+AVRUsartState *usart = opaque;
+assert(size == 1);
+assert(!usart->data_valid);
+usart->data = buffer[0];
+usart->data_valid = true;
+usart->csra |= USART_CSRA_RXC;
+if (usart->csrb & USART_CSRB_RXCIE) {
+qemu_set_irq(usart->rxc_irq, 1);
+}
+}
+
+static void update_char_mask(AVRUsartState *usart)
+{
+uint8_t mode = ((usart->csrc & USART_CSRC_CSZ0) ? 1 : 0) |
+((usart->csrc & USART_CSRC_CSZ1) ? 2 : 0) |
+((usart->csrb & USART_CSRB_CSZ2) ? 4 : 0);
+switch (mode) {
+case 0:
+usart->char_mask = 0b1;
+break;
+case 1:
+usart->char_mask = 0b11;
+break;
+case 2:
+usart->char_mask = 0b111;
+break;
+case 3:
+usart->char_mask = 0b;
+break;
+case 4:
+/* Fallthrough. */
+case 5:
+/* Fallthrough. */
+case 6:
+qemu_log_mask(
+LOG_GUEST_ERROR,
+"%s: Reserved character size 0x%x\n",
+__func__,
+mode);
+break;
+case 7:
+qemu_log_mask(
+LOG_GUEST_ERROR,
+"%s: Nine bit character size not supported (forcing eight)\n",
+__func__);
+usart->char_mask = 0b;
+break;
+default:
+assert(0);
+}
+}
+
+static void avr_usart_reset(DeviceState *dev)
+{
+AVRUsartState *usart = AVR_USART(dev);
+usart->data_valid = false;
+usart->csra = 0b0010;
+usart->csrb = 0b;
+usart->csrc = 0b0110;
+usart->brrl = 0;
+usart->brrh = 0;
+update_char_mask(usart);
+

[Qemu-devel] [PATCH v25 1/7] target/avr: Add outward facing interfaces and core CPU logic

From: Sarah Harris 

This includes:
- CPU data structures
- object model classes and functions
- migration functions
- GDB hooks

Signed-off-by: Michael Rolnik 
---
 gdb-xml/avr-cpu.xml|  49 
 target/avr/cpu-param.h |  37 +++
 target/avr/cpu.c   | 584 +
 target/avr/cpu.h   | 283 
 target/avr/gdbstub.c   |  85 ++
 target/avr/machine.c   | 123 +
 6 files changed, 1161 insertions(+)
 create mode 100644 gdb-xml/avr-cpu.xml
 create mode 100644 target/avr/cpu-param.h
 create mode 100644 target/avr/cpu.c
 create mode 100644 target/avr/cpu.h
 create mode 100644 target/avr/gdbstub.c
 create mode 100644 target/avr/machine.c

diff --git a/gdb-xml/avr-cpu.xml b/gdb-xml/avr-cpu.xml
new file mode 100644
index 00..c4747f5b40
--- /dev/null
+++ b/gdb-xml/avr-cpu.xml
@@ -0,0 +1,49 @@
+
+
+
+
+
+
+
+  
+  
+  
+  
+  
+  
+  
+  
+  
+  
+  
+  
+  
+  
+  
+  
+  
+  
+  
+  
+  
+  
+  
+  
+  
+  
+  
+  
+  
+  
+  
+  
+  
+  
+  
+
diff --git a/target/avr/cpu-param.h b/target/avr/cpu-param.h
new file mode 100644
index 00..5bbf985726
--- /dev/null
+++ b/target/avr/cpu-param.h
@@ -0,0 +1,37 @@
+/*
+ * QEMU AVR CPU
+ *
+ * Copyright (c) 2016 Michael Rolnik
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see
+ * 
+ */
+
+#ifndef AVR_CPU_PARAM_H
+#define AVR_CPU_PARAM_H 1
+
+#define TARGET_LONG_BITS 32
+/*
+ * TARGET_PAGE_BITS cannot be more than 8 bits because
+ * 1.  all IO registers occupy [0x .. 0x00ff] address range, and they
+ * should be implemented as a device and not memory
+ * 2.  SRAM starts at the address 0x0100
+ */
+#define TARGET_PAGE_BITS 8
+#define TARGET_PHYS_ADDR_SPACE_BITS 24
+#define TARGET_VIRT_ADDR_SPACE_BITS 24
+#define NB_MMU_MODES 2
+
+
+#endif
diff --git a/target/avr/cpu.c b/target/avr/cpu.c
new file mode 100644
index 00..ac85508156
--- /dev/null
+++ b/target/avr/cpu.c
@@ -0,0 +1,584 @@
+/*
+ * QEMU AVR CPU
+ *
+ * Copyright (c) 2016 Michael Rolnik
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see
+ * 
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/qemu-print.h"
+#include "qemu/log.h"
+#include "cpu.h"
+#include "exec/exec-all.h"
+#include "qapi/error.h"
+#include "hw/qdev-properties.h"
+#include "migration/vmstate.h"
+
+static void avr_cpu_set_pc(CPUState *cs, vaddr value)
+{
+AVRCPU *cpu = AVR_CPU(cs);
+
+cpu->env.pc_w = value / 2; /* internally PC points to words */
+}
+
+static bool avr_cpu_has_work(CPUState *cs)
+{
+AVRCPU *cpu = AVR_CPU(cs);
+CPUAVRState *env = >env;
+
+return (cs->interrupt_request & (CPU_INTERRUPT_HARD | CPU_INTERRUPT_RESET))
+&& cpu_interrupts_enabled(env);
+}
+
+static void avr_cpu_synchronize_from_tb(CPUState *cs, TranslationBlock *tb)
+{
+AVRCPU *cpu = AVR_CPU(cs);
+CPUAVRState *env = >env;
+
+env->pc_w = tb->pc / 2; /* internally PC points to words */
+}
+
+static void avr_cpu_reset(CPUState *cs)
+{
+AVRCPU *cpu = AVR_CPU(cs);
+AVRCPUClass *mcc = AVR_CPU_GET_CLASS(cpu);
+CPUAVRState *env = >env;
+
+mcc->parent_reset(cs);
+
+env->pc_w = 0;
+env->sregI = 1;
+env->sregC = 0;
+env->sregZ = 0;
+env->sregN = 0;
+env->sregV = 0;
+env->sregS = 0;
+env->sregH = 0;
+env->sregT = 0;
+
+env->rampD = 0;
+env->rampX = 0;
+env->rampY = 0;
+env->rampZ = 0;
+env->eind = 0;
+env->sp = 0;
+
+env->skip = 0;
+
+memset(env->r, 0, sizeof(env->r));
+
+tlb_flush(cs);
+}
+
+static void avr_cpu_disas_set_info(CPUState *cpu, disassemble_info *info)
+{
+info->mach = bfd_arch_avr;
+info->print_insn = NULL;
+}
+
+static void avr_cpu_realizefn(DeviceState *dev, Error **errp)
+{
+

[Qemu-devel] [PATCH v25 2/7] target/avr: Add instruction helpers

From: Sarah Harris 

Stubs for unimplemented instructions and helpers for instructions that need to 
interact with QEMU.
SPM and WDR are unimplemented because they require emulation of complex 
peripherals.
The implementation of SLEEP is very limited due to the lack of peripherals to 
generate wake interrupts.
Memory access instructions are implemented here because some address ranges 
actually refer to CPU registers.

Signed-off-by: Michael Rolnik 
---
 target/avr/helper.c | 354 
 target/avr/helper.h |  29 
 2 files changed, 383 insertions(+)
 create mode 100644 target/avr/helper.c
 create mode 100644 target/avr/helper.h

diff --git a/target/avr/helper.c b/target/avr/helper.c
new file mode 100644
index 00..4d4eaed5ea
--- /dev/null
+++ b/target/avr/helper.c
@@ -0,0 +1,354 @@
+/*
+ * QEMU AVR CPU
+ *
+ * Copyright (c) 2016 Michael Rolnik
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see
+ * 
+ */
+
+#include "qemu/osdep.h"
+
+#include "cpu.h"
+#include "hw/irq.h"
+#include "hw/sysbus.h"
+#include "sysemu/sysemu.h"
+#include "exec/exec-all.h"
+#include "exec/cpu_ldst.h"
+#include "exec/helper-proto.h"
+#include "exec/ioport.h"
+#include "qemu/host-utils.h"
+#include "qemu/error-report.h"
+
+bool avr_cpu_exec_interrupt(CPUState *cs, int interrupt_request)
+{
+bool ret = false;
+CPUClass *cc = CPU_GET_CLASS(cs);
+AVRCPU *cpu = AVR_CPU(cs);
+CPUAVRState *env = >env;
+
+if (interrupt_request & CPU_INTERRUPT_RESET) {
+if (cpu_interrupts_enabled(env)) {
+cs->exception_index = EXCP_RESET;
+cc->do_interrupt(cs);
+
+cs->interrupt_request &= ~CPU_INTERRUPT_RESET;
+
+ret = true;
+}
+}
+if (interrupt_request & CPU_INTERRUPT_HARD) {
+if (cpu_interrupts_enabled(env) && env->intsrc != 0) {
+int index = ctz32(env->intsrc);
+cs->exception_index = EXCP_INT(index);
+cc->do_interrupt(cs);
+
+env->intsrc &= env->intsrc - 1; /* clear the interrupt */
+cs->interrupt_request &= ~CPU_INTERRUPT_HARD;
+
+ret = true;
+}
+}
+return ret;
+}
+
+void avr_cpu_do_interrupt(CPUState *cs)
+{
+AVRCPU *cpu = AVR_CPU(cs);
+CPUAVRState *env = >env;
+
+uint32_t ret = env->pc_w;
+int vector = 0;
+int size = avr_feature(env, AVR_FEATURE_JMP_CALL) ? 2 : 1;
+int base = 0;
+
+if (cs->exception_index == EXCP_RESET) {
+vector = 0;
+} else if (env->intsrc != 0) {
+vector = ctz32(env->intsrc) + 1;
+}
+
+if (avr_feature(env, AVR_FEATURE_3_BYTE_PC)) {
+cpu_stb_data(env, env->sp--, (ret & 0xff));
+cpu_stb_data(env, env->sp--, (ret & 0x00ff00) >> 8);
+cpu_stb_data(env, env->sp--, (ret & 0xff) >> 16);
+} else if (avr_feature(env, AVR_FEATURE_2_BYTE_PC)) {
+cpu_stb_data(env, env->sp--, (ret & 0xff));
+cpu_stb_data(env, env->sp--, (ret & 0x00ff00) >> 8);
+} else {
+cpu_stb_data(env, env->sp--, (ret & 0xff));
+}
+
+env->pc_w = base + vector * size;
+env->sregI = 0; /* clear Global Interrupt Flag */
+
+cs->exception_index = -1;
+}
+
+int avr_cpu_memory_rw_debug(CPUState *cs, vaddr addr, uint8_t *buf,
+int len, bool is_write)
+{
+return cpu_memory_rw_debug(cs, addr, buf, len, is_write);
+}
+
+hwaddr avr_cpu_get_phys_page_debug(CPUState *cs, vaddr addr)
+{
+return addr; /* I assume 1:1 address correspondance */
+}
+
+int avr_cpu_handle_mmu_fault(
+CPUState *cs, vaddr address, int size, int rw, int mmu_idx)
+{
+/* currently it's assumed that this will never happen */
+cs->exception_index = EXCP_DEBUG;
+cpu_dump_state(cs, stderr, 0);
+return 1;
+}
+
+bool avr_cpu_tlb_fill(CPUState *cs, vaddr address, int size,
+MMUAccessType access_type, int mmu_idx,
+bool probe, uintptr_t retaddr)
+{
+int prot = 0;
+MemTxAttrs attrs = {};
+uint32_t paddr;
+
+address &= TARGET_PAGE_MASK;
+
+if (mmu_idx == MMU_CODE_IDX) {
+/* access to code in flash */
+paddr = OFFSET_CODE + address;
+prot = PAGE_READ | PAGE_EXEC;
+if (paddr + TARGET_PAGE_SIZE > OFFSET_DATA) {
+error_report("execution left flash memory");

[Qemu-devel] [PATCH v25 7/7] target/avr: Register AVR support with the rest of QEMU, the build system, and the MAINTAINERS file

From: Sarah Harris 

Signed-off-by: Michael Rolnik 
---
 MAINTAINERS |  6 ++
 arch_init.c |  2 ++
 configure   |  7 +++
 default-configs/avr-softmmu.mak |  5 +
 hw/avr/sample.c |  2 +-
 include/disas/dis-asm.h |  6 ++
 include/sysemu/arch_init.h  |  1 +
 qapi/common.json|  3 ++-
 target/avr/Makefile.objs| 33 +
 target/avr/cpu.c| 13 -
 target/avr/cpu.h|  3 ---
 tests/machine-none-test.c   |  1 +
 12 files changed, 68 insertions(+), 14 deletions(-)
 create mode 100644 default-configs/avr-softmmu.mak
 create mode 100644 target/avr/Makefile.objs

diff --git a/MAINTAINERS b/MAINTAINERS
index cc9636b43a..934ad5739b 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -163,6 +163,12 @@ S: Maintained
 F: hw/arm/smmu*
 F: include/hw/arm/smmu*
 
+AVR TCG CPUs
+M: Michael Rolnik 
+S: Maintained
+F: target/avr/
+F: hw/avr/
+
 CRIS TCG CPUs
 M: Edgar E. Iglesias 
 S: Maintained
diff --git a/arch_init.c b/arch_init.c
index 74b0708634..413ad7acfd 100644
--- a/arch_init.c
+++ b/arch_init.c
@@ -85,6 +85,8 @@ int graphic_depth = 32;
 #define QEMU_ARCH QEMU_ARCH_UNICORE32
 #elif defined(TARGET_XTENSA)
 #define QEMU_ARCH QEMU_ARCH_XTENSA
+#elif defined(TARGET_AVR)
+#define QEMU_ARCH QEMU_ARCH_AVR
 #endif
 
 const uint32_t arch_type = QEMU_ARCH;
diff --git a/configure b/configure
index 4983c8b533..ab8ebba100 100755
--- a/configure
+++ b/configure
@@ -7503,6 +7503,10 @@ case "$target_name" in
 target_compiler=$cross_cc_aarch64
 eval "target_compiler_cflags=\$cross_cc_cflags_${target_name}"
   ;;
+  avr)
+   gdb_xml_files="avr-cpu.xml"
+target_compiler=$cross_cc_avr
+  ;;
   cris)
 target_compiler=$cross_cc_cris
   ;;
@@ -7780,6 +7784,9 @@ for i in $ARCH $TARGET_BASE_ARCH ; do
   disas_config "ARM_A64"
 fi
   ;;
+  avr)
+disas_config "AVR"
+  ;;
   cris)
 disas_config "CRIS"
   ;;
diff --git a/default-configs/avr-softmmu.mak b/default-configs/avr-softmmu.mak
new file mode 100644
index 00..d1e1c28118
--- /dev/null
+++ b/default-configs/avr-softmmu.mak
@@ -0,0 +1,5 @@
+# Default configuration for avr-softmmu
+
+# Boards:
+#
+CONFIG_AVR_SAMPLE=y
diff --git a/hw/avr/sample.c b/hw/avr/sample.c
index e4cb548a33..de97fe4ae7 100644
--- a/hw/avr/sample.c
+++ b/hw/avr/sample.c
@@ -197,7 +197,7 @@ static void sample_class_init(ObjectClass *oc, void *data)
 mc->default_cpus = 1;
 mc->min_cpus = mc->default_cpus;
 mc->max_cpus = mc->default_cpus;
-mc->default_cpu_type = AVR_CPU_TYPE_NAME("avr6"); /* ATmega2560. */
+mc->default_cpu_type = "avr6-avr-cpu"; /* ATmega2560. */
 mc->is_default = 1;
 }
 
diff --git a/include/disas/dis-asm.h b/include/disas/dis-asm.h
index e9c7dd8eb4..8bedce17ac 100644
--- a/include/disas/dis-asm.h
+++ b/include/disas/dis-asm.h
@@ -211,6 +211,12 @@ enum bfd_architecture
 #define bfd_mach_m32r  0  /* backwards compatibility */
   bfd_arch_mn10200,/* Matsushita MN10200 */
   bfd_arch_mn10300,/* Matsushita MN10300 */
+  bfd_arch_avr,   /* Atmel AVR microcontrollers.  */
+#define bfd_mach_avr1  1
+#define bfd_mach_avr2  2
+#define bfd_mach_avr3  3
+#define bfd_mach_avr4  4
+#define bfd_mach_avr5  5
   bfd_arch_cris,   /* Axis CRIS */
 #define bfd_mach_cris_v0_v10   255
 #define bfd_mach_cris_v32  32
diff --git a/include/sysemu/arch_init.h b/include/sysemu/arch_init.h
index 10cbafe970..aff57bfe61 100644
--- a/include/sysemu/arch_init.h
+++ b/include/sysemu/arch_init.h
@@ -25,6 +25,7 @@ enum {
 QEMU_ARCH_NIOS2 = (1 << 17),
 QEMU_ARCH_HPPA = (1 << 18),
 QEMU_ARCH_RISCV = (1 << 19),
+QEMU_ARCH_AVR = (1 << 20),
 };
 
 extern const uint32_t arch_type;
diff --git a/qapi/common.json b/qapi/common.json
index 99d313ef3b..6866c3e81d 100644
--- a/qapi/common.json
+++ b/qapi/common.json
@@ -183,11 +183,12 @@
 #is true even for "qemu-system-x86_64".
 #
 # ppcemb: dropped in 3.1
+# avr: added in 4.1
 #
 # Since: 3.0
 ##
 { 'enum' : 'SysEmuTarget',
-  'data' : [ 'aarch64', 'alpha', 'arm', 'cris', 'hppa', 'i386', 'lm32',
+  'data' : [ 'aarch64', 'alpha', 'arm', 'avr', 'cris', 'hppa', 'i386', 'lm32',
  'm68k', 'microblaze', 'microblazeel', 'mips', 'mips64',
  'mips64el', 'mipsel', 'moxie', 'nios2', 'or1k', 'ppc',
  'ppc64', 'riscv32', 'riscv64', 's390x', 'sh4',
diff --git a/target/avr/Makefile.objs b/target/avr/Makefile.objs
new file mode 100644
index 00..1034d87525
--- /dev/null
+++ b/target/avr/Makefile.objs
@@ -0,0 +1,33 @@
+#
+#  QEMU AVR CPU
+#
+#  Copyright (c) 2016 Michael Rolnik
+#
+#  This library is free software; you can redistribute it and/or
+#  modify it under the terms of the GNU Lesser General Public
+#  License as published by the Free Software Foundation; either
+#  version 2.1 of the License, or (at your option) any later

[Qemu-devel] [PATCH v25 3/7] target/avr: Add instruction decoding

This includes:
- encoding of all 16 bit instructions
- encoding of all 32 bit instructions

Signed-off-by: Michael Rolnik 
---
 target/avr/insn.decode | 175 +
 1 file changed, 175 insertions(+)
 create mode 100644 target/avr/insn.decode

diff --git a/target/avr/insn.decode b/target/avr/insn.decode
new file mode 100644
index 00..6b387762c6
--- /dev/null
+++ b/target/avr/insn.decode
@@ -0,0 +1,175 @@
+#
+#   A = [16 .. 31]
+#   B = [16 .. 23]
+#   C = [24, 26, 28, 30]
+#   D = [0, 2, 4, 6, 8, .. 30]
+
+%rd 4:5
+%rr 9:1 0:4
+
+_rr  rd rr
+_imm rd imm
+
+@op_rd_rr    .. . . _rr  rd=%rd rr=%rr
+ADD  11 . . @op_rd_rr
+ADC 0001 11 . . @op_rd_rr
+AND 0010 00 . . @op_rd_rr
+CP  0001 01 . . @op_rd_rr
+CPC  01 . . @op_rd_rr
+CPSE0001 00 . . @op_rd_rr
+EOR 0010 01 . . @op_rd_rr
+MOV 0010 11 . . @op_rd_rr
+MUL 1001 11 . . @op_rd_rr
+OR  0010 10 . . @op_rd_rr
+SBC  10 . . @op_rd_rr
+SUB 0001 10 . . @op_rd_rr
+
+
+%rd_c   4:2 !function=to_C
+%imm6   6:2 0:4
+
+@op_rd_imm6   .. .. _imm rd=%rd_c imm=%imm6
+ADIW1001 0110 .. .. @op_rd_imm6
+SBIW1001 0111 .. .. @op_rd_imm6
+
+
+%rd_a   4:4 !function=to_A
+%rr_a   0:4 !function=to_A
+%rd_d   4:4 !function=to_D
+%rr_d   0:4 !function=to_D
+%imm8   8:4 0:4
+
+@op_rd_imm8     _imm rd=%rd_a imm=%imm8
+ANDI0111    @op_rd_imm8
+CPI 0011    @op_rd_imm8
+LDI 1110    @op_rd_imm8
+ORI 0110    @op_rd_imm8
+SBCI0100    @op_rd_imm8
+SUBI0101    @op_rd_imm8
+
+
+@op_rd   ... rd:5 
+ASR 1001 010 . 0101 @op_rd
+COM 1001 010 .  @op_rd
+DEC 1001 010 . 1010 @op_rd
+ELPM2   1001 000 . 0110 @op_rd
+ELPMX   1001 000 . 0111 @op_rd
+INC 1001 010 . 0011 @op_rd
+LDX11001 000 . 1100 @op_rd
+LDX21001 000 . 1101 @op_rd
+LDX31001 000 . 1110 @op_rd
+LDY21001 000 . 1001 @op_rd
+LDY31001 000 . 1010 @op_rd
+LDZ21001 000 . 0001 @op_rd
+LDZ31001 000 . 0010 @op_rd
+LPM21001 000 . 0100 @op_rd
+LPMX1001 000 . 0101 @op_rd
+LSR 1001 010 . 0110 @op_rd
+NEG 1001 010 . 0001 @op_rd
+POP 1001 000 .  @op_rd
+PUSH1001 001 .  @op_rd
+ROR 1001 010 . 0111 @op_rd
+STY21001 001 . 1001 @op_rd
+STY31001 001 . 1010 @op_rd
+STZ21001 001 . 0001 @op_rd
+STZ31001 001 . 0010 @op_rd
+SWAP1001 010 . 0010 @op_rd
+
+
+@op_bit   . bit:3 
+BCLR1001 0100 1 ... 1000@op_bit
+BSET1001 0100 0 ... 1000@op_bit
+
+
+@op_rd_bit   ... rd:5 . bit:3
+BLD  100 . 0 ...@op_rd_bit
+BST  101 . 0 ...@op_rd_bit
+
+
+@op_bit_imm  .. imm:s7 bit:3
+BRBC 01 ... ... @op_bit_imm
+BRBS 00 ... ... @op_bit_imm
+
+
+BREAK   1001 0101 1001 1000
+EICALL  1001 0101 0001 1001
+EIJMP   1001 0100 0001 1001
+ELPM1   1001 0101 1101 1000
+ICALL   1001 0101  1001
+IJMP1001 0100  1001
+LPM11001 0101 1100 1000
+NOP    
+RET 1001 0101  1000
+RETI1001 0101 0001 1000
+SLEEP   1001 0101 1000 1000
+SPM 1001 0101 1110 1000
+SPMX1001 0101  1000
+WDR 1001 0101 1010 1000
+
+
+@op_reg_bit   reg:5 bit:3
+CBI 1001 1000 . ... @op_reg_bit
+SBI 1001 1010 . ... @op_reg_bit
+SBIC1001 1001 . ... @op_reg_bit
+SBIS1001 1011 . ... @op_reg_bit
+
+
+DES 1001 0100 imm:4 1011
+
+
+%rd_b   4:3

[Qemu-devel] [PATCH v25 0/7] QEMU AVR 8 bit cores

This series of patches adds 8bit AVR cores to QEMU.
All instruction, except BREAK/DES/SPM/SPMX, are implemented. Not fully tested 
yet.
However I was able to execute simple code with functions. e.g fibonacci 
calculation.
This series of patches include a non real, sample board.
No fuses support yet. PC is set to 0 at reset.

the patches include the following
1. just a basic 8bit AVR CPU, without instruction decoding or translation
2. CPU features which allow define the following 8bit AVR cores
 avr1
 avr2 avr25
 avr3 avr31 avr35
 avr4
 avr5 avr51
 avr6
 xmega2 xmega4 xmega5 xmega6 xmega7
3. a definition of sample machine with SRAM, FLASH and CPU which allows to 
execute simple code
4. encoding for all AVR instructions
5. interrupt handling
6. helpers for IN, OUT, SLEEP, WBR & unsupported instructions
7. a decoder which given an opcode decides what istruction it is
8. translation of AVR instruction into TCG
9. all features together

changes since v3
1. rampD/X/Y/Z registers are encoded as 0x00ff (instead of 0x00ff) for 
faster address manipulaton
2. ffs changed to ctz32
3. duplicate code removed at avr_cpu_do_interrupt
4. using andc instead of not + and
5. fixing V flag calculation in varios instructions
6. freeing local variables in PUSH
7. tcg_const_local_i32 -> tcg_const_i32
8. using sextract32 instead of my implementation
9. fixing BLD instruction
10.xor(r) instead of 0xff - r at COM
11.fixing MULS/MULSU not to modify inputs' content
12.using SUB for NEG
13.fixing tcg_gen_qemu_ld/st call in XCH

changes since v4
1. target is now defined as big endian in order to optimize push_ret/pop_ret
2. all style warnings are fixed
3. adding cpu_set/get_sreg functions
4. simplifying gen_goto_tb as there is no real paging
5. env->pc -> env->pc_w
6. making flag dump more compact
7. more spacing
8. renaming CODE/DATA_INDEX -> MMU_CODE/DATA_IDX
9. removing avr_set_feature
10. SPL/SPH set bug fix
11. switching stb_phys to cpu_stb_data
12. cleaning up avr_decode
13. saving sreg, rampD/X/Y/Z, eind in HW format (savevm)
14. saving CPU features (savevm)

changes since v5
1. BLD bug fix
2. decoder generator is added

chages since v6
1. using cpu_get_sreg/cpu_set_sreg in 
avr_cpu_gdb_read_register/avr_cpu_gdb_write_register
2. configure the target as little endian because otherwise GDB does not work
3. fixing and testing gen_push_ret/gen_pop_ret

changes since v7
1. folding back v6
2. logging at helper_outb and helper_inb are done for non supported yet 
registers only
3. MAINTAINERS updated

changes since v8
1. removing hw/avr from hw/Makefile.obj as it should not be built for all
2. making linux compilable
3. testing on
a. Mac, Apple LLVM version 7.0.0
b. Ubuntu 12.04, gcc 4.9.2
c. Fedora 23, gcc 5.3.1
4. folding back some patches
5. translation bug fixes for ORI, CPI, XOR instructions
6. propper handling of cpu register writes though memory

changes since v9
1. removing forward declarations of static functions
2. disabling debug prints
3. switching to case range instead of if else if ...
4. LD/ST IN/OUT accessing CPU maintainder registers are not routed to any device
5. commenst about sample board and sample IO device added
6. sample board description is more descriptive now
7. memory_region_allocate_system_memory is used to create RAM
8. now there are helper_fullrd & helper_fullwr when LD/ST try to access 
registers

changes since v10
1. movig back fullwr & fullrd into the commit where outb and inb were introduced
2. changing tlb_fill function signature
3. adding empty line between functions
4. adding newline on the last line of the file
5. using tb->flags to generae full access ST/LD instructions
6. fixing SBRC bug
7. folding back 10th commit
8. whenever a new file is introduced it's added to Makefile.objs

changes since v11
1. updating to v2.7.0-rc
2. removing assignment to env->fullacc from gen_intermediate_code

changes since v12
1. fixing spacing
2. fixing get/put_segment functions
3. removing target-avr/machine.h file
4. VMSTATE_SINGLE_TEST -> VMSTATE_SINGLE
5. comment spelling
6. removing hw/avr/sample_io.c
7. char const* -> const char*
8. proper ram allocation
9. fixing breakpoint functionality.
10.env1 -> env
11.fixing avr_cpu_gdb_write_register & avr_cpu_gdb_read_register functions
12.any cpu is removed
12.feature bits are not saved into vm state

changes since v13
1. rebasing to v2.7.0-rc1

changes since v14
1. I made self review with git gui tool. (I did not know such a thing exists)
2. removing all double/tripple spaces
3. removing comment reference to SampleIO
4. folding back some changes, so there is not deleted lines in my code
5. moving avr configuration, within configure file, before chris

changes since v15
1. removing IO registers cache from CPU
2. implementing CBI/SBI as read(helper_inb), modify, write(helper_outb)
3. implementing CBIC/SBIC as read(helper_inb), check, branch
4. adding missing tcg_temp_free_i32 for tcg_const_i32

changes since v16
1.

Re: [Qemu-devel] [PATCH v2 5/5] tcg: Release mmap_lock on translation fault

2019-07-09 Thread Alex Bennée



Richard Henderson  writes:

> Turn helper_retaddr into a multi-state flag that may now also
> indicate when we're performing a read on behalf of the translator.
> In this case, release the mmap_lock before the longjmp back to
> the main cpu loop, and thereby avoid a failing assert therein.
>
> Fixes: https://bugs.launchpad.net/qemu/+bug/1832353
> Signed-off-by: Richard Henderson 

Reviewed-by: Alex Bennée 

with check-tcg this time

Tested-by: Alex Bennée 

> ---
>  include/exec/cpu_ldst_useronly_template.h | 20 +--
>  accel/tcg/user-exec.c | 66 ---
>  2 files changed, 63 insertions(+), 23 deletions(-)
>
> diff --git a/include/exec/cpu_ldst_useronly_template.h 
> b/include/exec/cpu_ldst_useronly_template.h
> index d663826ac2..2378f2958c 100644
> --- a/include/exec/cpu_ldst_useronly_template.h
> +++ b/include/exec/cpu_ldst_useronly_template.h
> @@ -64,12 +64,18 @@
>  static inline RES_TYPE
>  glue(glue(cpu_ld, USUFFIX), MEMSUFFIX)(CPUArchState *env, abi_ptr ptr)
>  {
> -#if !defined(CODE_ACCESS)
> +#ifdef CODE_ACCESS
> +RES_TYPE ret;
> +set_helper_retaddr(1);
> +ret = glue(glue(ld, USUFFIX), _p)(g2h(ptr));
> +clear_helper_retaddr();
> +return ret;
> +#else
>  trace_guest_mem_before_exec(
>  env_cpu(env), ptr,
>  trace_mem_build_info(SHIFT, false, MO_TE, false));
> -#endif
>  return glue(glue(ld, USUFFIX), _p)(g2h(ptr));
> +#endif
>  }
>
>  #ifndef CODE_ACCESS
> @@ -90,12 +96,18 @@ glue(glue(glue(cpu_ld, USUFFIX), MEMSUFFIX), 
> _ra)(CPUArchState *env,
>  static inline int
>  glue(glue(cpu_lds, SUFFIX), MEMSUFFIX)(CPUArchState *env, abi_ptr ptr)
>  {
> -#if !defined(CODE_ACCESS)
> +#ifdef CODE_ACCESS
> +int ret;
> +set_helper_retaddr(1);
> +ret = glue(glue(lds, SUFFIX), _p)(g2h(ptr));
> +clear_helper_retaddr();
> +return ret;
> +#else
>  trace_guest_mem_before_exec(
>  env_cpu(env), ptr,
>  trace_mem_build_info(SHIFT, true, MO_TE, false));
> -#endif
>  return glue(glue(lds, SUFFIX), _p)(g2h(ptr));
> +#endif
>  }
>
>  #ifndef CODE_ACCESS
> diff --git a/accel/tcg/user-exec.c b/accel/tcg/user-exec.c
> index 4384b59a4d..897d1571c4 100644
> --- a/accel/tcg/user-exec.c
> +++ b/accel/tcg/user-exec.c
> @@ -64,27 +64,56 @@ static inline int handle_cpu_signal(uintptr_t pc, 
> siginfo_t *info,
>  CPUState *cpu = current_cpu;
>  CPUClass *cc;
>  unsigned long address = (unsigned long)info->si_addr;
> -MMUAccessType access_type;
> +MMUAccessType access_type = is_write ? MMU_DATA_STORE : MMU_DATA_LOAD;
>
> -/* We must handle PC addresses from two different sources:
> - * a call return address and a signal frame address.
> - *
> - * Within cpu_restore_state_from_tb we assume the former and adjust
> - * the address by -GETPC_ADJ so that the address is within the call
> - * insn so that addr does not accidentally match the beginning of the
> - * next guest insn.
> - *
> - * However, when the PC comes from the signal frame, it points to
> - * the actual faulting host insn and not a call insn.  Subtracting
> - * GETPC_ADJ in that case may accidentally match the previous guest insn.
> - *
> - * So for the later case, adjust forward to compensate for what
> - * will be done later by cpu_restore_state_from_tb.
> - */
> -if (helper_retaddr) {
> +switch (helper_retaddr) {
> +default:
> +/*
> + * Fault during host memory operation within a helper function.
> + * The helper's host return address, saved here, gives us a
> + * pointer into the generated code that will unwind to the
> + * correct guest pc.
> + */
>  pc = helper_retaddr;
> -} else {
> +break;
> +
> +case 0:
> +/*
> + * Fault during host memory operation within generated code.
> + * (Or, a unrelated bug within qemu, but we can't tell from here).
> + *
> + * We take the host pc from the signal frame.  However, we cannot
> + * use that value directly.  Within cpu_restore_state_from_tb, we
> + * assume PC comes from GETPC(), as used by the helper functions,
> + * so we adjust the address by -GETPC_ADJ to form an address that
> + * is within the call insn, so that the address does not 
> accidentially
> + * match the beginning of the next guest insn.  However, when the
> + * pc comes from the signal frame it points to the actual faulting
> + * host memory insn and not the return from a call insn.
> + *
> + * Therefore, adjust to compensate for what will be done later
> + * by cpu_restore_state_from_tb.
> + */
>  pc += GETPC_ADJ;
> +break;
> +
> +case 1:
> +/*
> + * Fault during host read for translation, or loosely, "execution".
> + *
> + * The guest pc is already pointing to the start of the TB for which

[Qemu-devel] [PATCH 2/4] raw: Recognize zoned backing devices

The purpose of this patch is to recognize a zoned block device (ZBD)
when it is opened as a raw file. The new code initializes the zoned
model propery introduced by the previous commit.

This commit is Linux-specific and in essence it checks the Zoned
Block Device Model (None/Host-managed/Host-aware) value in sysfs on
the host.

In order to avoid code duplication in file-posix.c, a common helper
function is added to read value of a sysfs entry under
/sys/block//queue. Now, the existing function that reads the
"max_segments" value and the the new function that reads "zoned"
value share the same helper code.

Signed-off-by: Dmitry Fomichev 
---
 block/file-posix.c | 69 ++
 block/raw-format.c |  8 ++
 2 files changed, 66 insertions(+), 11 deletions(-)

diff --git a/block/file-posix.c b/block/file-posix.c
index ab05b51a66..dd81dc3301 100644
--- a/block/file-posix.c
+++ b/block/file-posix.c
@@ -1055,33 +1055,53 @@ static int 
hdev_get_max_transfer_length(BlockDriverState *bs, int fd)
 #endif
 }
 
-static int hdev_get_max_segments(const struct stat *st)
+static int hdev_read_blk_queue_entry(const struct stat *st, const char *key,
+char *buf, int buf_len)
 {
 #ifdef CONFIG_LINUX
-char buf[32];
-const char *end;
 char *sysfspath;
 int ret;
 int fd = -1;
-long max_segments;
 
-sysfspath = g_strdup_printf("/sys/dev/block/%u:%u/queue/max_segments",
-major(st->st_rdev), minor(st->st_rdev));
+sysfspath = g_strdup_printf("/sys/dev/block/%u:%u/queue/%s",
+major(st->st_rdev), minor(st->st_rdev), key);
 fd = open(sysfspath, O_RDONLY);
 if (fd == -1) {
 ret = -errno;
 goto out;
 }
 do {
-ret = read(fd, buf, sizeof(buf) - 1);
+ret = read(fd, buf, buf_len - 1);
 } while (ret == -1 && errno == EINTR);
 if (ret < 0) {
 ret = -errno;
-goto out;
 } else if (ret == 0) {
 ret = -EIO;
+}
+out:
+if (fd != -1) {
+close(fd);
+}
+g_free(sysfspath);
+return ret;
+#else
+return -ENOTSUP;
+#endif
+}
+
+static int hdev_get_max_segments(const struct stat *st)
+{
+#ifdef CONFIG_LINUX
+char buf[32];
+const char *end;
+int ret;
+long max_segments;
+
+ret = hdev_read_blk_queue_entry(st, "max_segments", buf, sizeof(buf));
+if (ret < 0) {
 goto out;
 }
+
 buf[ret] = 0;
 /* The file is ended with '\n', pass 'end' to accept that. */
 ret = qemu_strtol(buf, , 10, _segments);
@@ -1090,10 +1110,33 @@ static int hdev_get_max_segments(const struct stat *st)
 }
 
 out:
-if (fd != -1) {
-close(fd);
+return ret;
+#else
+return -ENOTSUP;
+#endif
+}
+
+static int hdev_get_zoned_model(const struct stat *st)
+{
+#ifdef CONFIG_LINUX
+char buf[32];
+int ret;
+
+ret = hdev_read_blk_queue_entry(st, "zoned", buf, sizeof(buf));
+if (ret < 0) {
+ret = BLK_ZONED_NONE;
+goto out;
 }
-g_free(sysfspath);
+
+buf[ret - 1] = 0;
+ret = BLK_ZONED_NONE;
+if (strcmp(buf, "host-managed") == 0) {
+ret = BLK_ZONED_HM;
+} else if (strcmp(buf, "host-aware") == 0) {
+ret = BLK_ZONED_HA;
+}
+
+out:
 return ret;
 #else
 return -ENOTSUP;
@@ -1116,6 +1159,10 @@ static void raw_refresh_limits(BlockDriverState *bs, 
Error **errp)
 bs->bl.max_transfer = MIN(bs->bl.max_transfer,
   ret * getpagesize());
 }
+ret = hdev_get_zoned_model();
+if (ret >= 0) {
+bs->bl.zoned_model = ret;
+}
 }
 }
 
diff --git a/block/raw-format.c b/block/raw-format.c
index bffd424dd0..12c2a3f95d 100644
--- a/block/raw-format.c
+++ b/block/raw-format.c
@@ -369,6 +369,13 @@ static void raw_refresh_limits(BlockDriverState *bs, Error 
**errp)
 }
 }
 
+static void raw_get_zoned_info(BlockDriverState *bs)
+{
+if (!bs->probed) {
+bs->bl.zoned_model = bs->file->bs->bl.zoned_model;
+}
+}
+
 static int coroutine_fn raw_co_truncate(BlockDriverState *bs, int64_t offset,
 PreallocMode prealloc, Error **errp)
 {
@@ -572,6 +579,7 @@ BlockDriver bdrv_raw = {
 .bdrv_co_ioctl= _co_ioctl,
 .create_opts  = _create_opts,
 .bdrv_has_zero_init   = _has_zero_init,
+.bdrv_get_zoned_info  = _get_zoned_info,
 .strong_runtime_opts  = raw_strong_runtime_opts,
 .mutable_opts = mutable_opts,
 };
-- 
2.21.0

[Qemu-devel] [PATCH 4/4] hw/scsi: Don't realize zoned block devices for virtio-scsi legacy drivers

From: Shin'ichiro Kawasaki 

Prevent scsi-hd and scsi-disk drivers from attaching a zoned block device
because it will appear as a regular block device at the guest and will
most certainly cause problems.

The functionality to support ZBDs is not planned for scsi-hd and
scsi-disk legacy drivers. It is supported via scsi-generic driver already
and discussion is ongoing for scsi-block driver.

Signed-off-by: Shin'ichiro Kawasaki 
---
 hw/scsi/scsi-disk.c | 5 +
 1 file changed, 5 insertions(+)

diff --git a/hw/scsi/scsi-disk.c b/hw/scsi/scsi-disk.c
index ed7295bfd7..80682a61fb 100644
--- a/hw/scsi/scsi-disk.c
+++ b/hw/scsi/scsi-disk.c
@@ -2401,6 +2401,11 @@ static void scsi_hd_realize(SCSIDevice *dev, Error 
**errp)
  * backend will be issued in scsi_realize
  */
 if (s->qdev.conf.blk) {
+if (blk_is_zoned(s->qdev.conf.blk)) {
+error_setg(errp, "zoned block devices are not supported");
+return;
+}
+
 ctx = blk_get_aio_context(s->qdev.conf.blk);
 aio_context_acquire(ctx);
 blkconf_blocksizes(>qdev.conf);
-- 
2.21.0

[Qemu-devel] [PATCH 1/4] block: Add zoned device model property

This commit adds Zoned Device Model as defined in T10 ZBC and
T13 ZAC standards, along with some useful accessor functions.

Since the default value of zero in the zoned model field corresponds
to a regular (non-zoned) block device, there is no functionality
change with this commit.

Signed-off-by: Dmitry Fomichev 
---
 block.c| 14 ++
 block/block-backend.c  | 20 
 include/block/block.h  |  9 +
 include/block/block_int.h  |  4 
 include/sysemu/block-backend.h |  2 ++
 5 files changed, 49 insertions(+)

diff --git a/block.c b/block.c
index c139540f2b..396d999f00 100644
--- a/block.c
+++ b/block.c
@@ -4649,6 +4649,20 @@ void bdrv_get_geometry(BlockDriverState *bs, uint64_t 
*nb_sectors_ptr)
 *nb_sectors_ptr = nb_sectors < 0 ? 0 : nb_sectors;
 }
 
+uint8_t bdrv_get_zoned_model(BlockDriverState *bs)
+{
+if (bs->drv->bdrv_get_zoned_info) {
+bs->drv->bdrv_get_zoned_info(bs);
+}
+
+return bs->bl.zoned_model;
+}
+
+uint8_t bdrv_is_zoned(BlockDriverState *bs)
+{
+return bdrv_get_zoned_model(bs) != BLK_ZONED_NONE;
+}
+
 bool bdrv_is_sg(BlockDriverState *bs)
 {
 return bs->sg;
diff --git a/block/block-backend.c b/block/block-backend.c
index a8d160fd5d..34723f3655 100644
--- a/block/block-backend.c
+++ b/block/block-backend.c
@@ -1438,6 +1438,15 @@ int64_t blk_nb_sectors(BlockBackend *blk)
 return bdrv_nb_sectors(blk_bs(blk));
 }
 
+uint8_t blk_get_zoned_model(BlockBackend *blk)
+{
+if (!blk_is_available(blk)) {
+return BLK_ZONED_NONE;
+}
+
+return bdrv_get_zoned_model(blk_bs(blk));
+}
+
 BlockAIOCB *blk_aio_preadv(BlockBackend *blk, int64_t offset,
QEMUIOVector *qiov, BdrvRequestFlags flags,
BlockCompletionFunc *cb, void *opaque)
@@ -1705,6 +1714,17 @@ bool blk_is_sg(BlockBackend *blk)
 return bdrv_is_sg(bs);
 }
 
+bool blk_is_zoned(BlockBackend *blk)
+{
+BlockDriverState *bs = blk_bs(blk);
+
+if (!bs) {
+return false;
+}
+
+return bdrv_is_zoned(bs);
+}
+
 bool blk_enable_write_cache(BlockBackend *blk)
 {
 return blk->enable_write_cache;
diff --git a/include/block/block.h b/include/block/block.h
index 734c9d2f76..7aa096afcc 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -278,6 +278,13 @@ enum {
 
 char *bdrv_perm_names(uint64_t perm);
 
+/* Known zoned device models */
+enum blk_zoned_model {
+BLK_ZONED_NONE, /* Regular block device */
+BLK_ZONED_HA,   /* Host-aware zoned block device */
+BLK_ZONED_HM,   /* Host-managed zoned block device */
+};
+
 /* disk I/O throttling */
 void bdrv_init(void);
 void bdrv_init_with_whitelist(void);
@@ -354,6 +361,8 @@ int64_t bdrv_get_allocated_file_size(BlockDriverState *bs);
 BlockMeasureInfo *bdrv_measure(BlockDriver *drv, QemuOpts *opts,
BlockDriverState *in_bs, Error **errp);
 void bdrv_get_geometry(BlockDriverState *bs, uint64_t *nb_sectors_ptr);
+uint8_t bdrv_get_zoned_model(BlockDriverState *bs);
+uint8_t bdrv_is_zoned(BlockDriverState *bs);
 void bdrv_refresh_limits(BlockDriverState *bs, Error **errp);
 int bdrv_commit(BlockDriverState *bs);
 int bdrv_change_backing_file(BlockDriverState *bs,
diff --git a/include/block/block_int.h b/include/block/block_int.h
index d6415b53c1..8f35591803 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -416,6 +416,7 @@ struct BlockDriver {
 bool (*bdrv_debug_is_suspended)(BlockDriverState *bs, const char *tag);
 
 void (*bdrv_refresh_limits)(BlockDriverState *bs, Error **errp);
+void (*bdrv_get_zoned_info)(BlockDriverState *bs);
 
 /*
  * Returns 1 if newly created images are guaranteed to contain only
@@ -614,6 +615,9 @@ typedef struct BlockLimits {
 
 /* maximum number of iovec elements */
 int max_iov;
+
+/* Zoned device model. Zero value indicates a regular block device */
+uint8_t zoned_model;
 } BlockLimits;
 
 typedef struct BdrvOpBlocker BdrvOpBlocker;
diff --git a/include/sysemu/block-backend.h b/include/sysemu/block-backend.h
index 733c4957eb..65b6341218 100644
--- a/include/sysemu/block-backend.h
+++ b/include/sysemu/block-backend.h
@@ -156,6 +156,7 @@ int blk_pwrite(BlockBackend *blk, int64_t offset, const 
void *buf, int bytes,
 int64_t blk_getlength(BlockBackend *blk);
 void blk_get_geometry(BlockBackend *blk, uint64_t *nb_sectors_ptr);
 int64_t blk_nb_sectors(BlockBackend *blk);
+uint8_t blk_get_zoned_model(BlockBackend *blk);
 BlockAIOCB *blk_aio_preadv(BlockBackend *blk, int64_t offset,
QEMUIOVector *qiov, BdrvRequestFlags flags,
BlockCompletionFunc *cb, void *opaque);
@@ -189,6 +190,7 @@ void blk_error_action(BlockBackend *blk, BlockErrorAction 
action,
   bool is_read, int error);
 bool blk_is_read_only(BlockBackend *blk);
 bool blk_is_sg(BlockBackend *blk);
+bool blk_is_zoned(BlockBackend *blk);

[Qemu-devel] [PATCH 3/4] virtio-blk: Don't realize zoned block devices

Prevent virtio-blk code from attaching a zoned block device because
it will otherwise appear as a reqular block device at the guest and
that will most certainly cause problems.

The functionality to support ZBDs via virtio-blk should be pretty
useful and there are some attempts underway to get it implemented,
but such work will inevitably lead to some modifications in virtio
protocol spec. Therefore, this activity is considered a more
long-term effort.

So for now, we just don't allow zoned block devices to work via
virtio-blk.

Signed-off-by: Dmitry Fomichev 
---
 hw/block/virtio-blk.c | 5 +
 1 file changed, 5 insertions(+)

diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
index cbb3729158..c11e028308 100644
--- a/hw/block/virtio-blk.c
+++ b/hw/block/virtio-blk.c
@@ -1140,6 +1140,11 @@ static void virtio_blk_device_realize(DeviceState *dev, 
Error **errp)
 return;
 }
 
+if (blk_is_zoned(conf->conf.blk)) {
+error_setg(errp, "zoned block devices are not supported");
+return;
+}
+
 if (virtio_has_feature(s->host_features, VIRTIO_BLK_F_DISCARD) &&
 (!conf->max_discard_sectors ||
  conf->max_discard_sectors > BDRV_REQUEST_MAX_SECTORS)) {
-- 
2.21.0

[Qemu-devel] [PATCH 0/4] virtio: handle zoned backing devices

Currently, attaching zoned block devices (i.e. storage devices
compliant to ZAC/ZBC standards) using several virtio methods doesn't
work - the zoned devices appear as regular block devices at the guest.
This may cause unexpected i/o errors and, potentially, some data
corruption.

To be more precise, attaching a zoned device via virtio-pci-blk,
virtio-scsi-pci/scsi-disk or virtio-scsi-pci/scsi-hd demonstrates the
above behavior. A simple fix is needed to make
virtio-scsi-pci/scsi-block work and this is covered by a different
patch. The virtio-scsi-pci/scsi-generic method appears to handle zoned
devices without problems.

This patchset adds code to check if the backing device that is being
opened is zoned. If this is the case, the new code prohibits
virtualization of the device for all use cases lacking proper zoned
support.

Considering that there is still a couple of different working ways
to virtualize a ZBD, this patchset provides a reasonable short-term
solution for this problem. What about long term?

It appears to be beneficial to add proper ZBD support to virtio-blk.
In order to support this use case properly, some virtio-blk protocol
changes will be necessary. They are needed to allow the host code to
propagate some ZBD properties that are required for virtio guest
driver to configure the guest block device as ZBD, such as zoned
device model, zone size and the total number of zones. Further, some
support needs to be added for REPORT ZONES command as well as for zone
operations, such as OPEN ZONE, CLOSE ZONE, FINISH ZONE and RESET ZONE.

These additions to the protocol are relatively straightforward, but
they need to be approved by the virtio TC and the whole process may
take some time.

ZBD support for virtio-scsi-pci/scsi-disk and virtio-scsi-pci/scsi-hd
does not seem as necessary. Users will be expected to attach zoned
block devices via virtio-scsi-pci/scsi-block instead.

This patchset contains some Linux-specific code. This code is
necessary to obtain Zoned Block Device model value from Linux sysfs.
Dmitry Fomichev (3):
  block: Add zoned device model property
  raw: Recognize zoned backing devices
  virtio-blk: Don't realize zoned block devices

Shin'ichiro Kawasaki (1):
  hw/scsi: Don't realize zoned block devices for virtio-scsi legacy
drivers

 block.c| 14 +++
 block/block-backend.c  | 20 ++
 block/file-posix.c | 69 --
 block/raw-format.c |  8 
 hw/block/virtio-blk.c  |  5 +++
 hw/scsi/scsi-disk.c|  5 +++
 include/block/block.h  |  9 +
 include/block/block_int.h  |  4 ++
 include/sysemu/block-backend.h |  2 +
 9 files changed, 125 insertions(+), 11 deletions(-)

-- 
2.21.0

[Qemu-devel] [PATCH v2 1/5] docker.py: add podman support

Add a --engine option to select either docker, podman or auto.

Among other advantages, podman allows to run rootless & daemonless
containers, fortunately sharing compatible CLI with docker.

Signed-off-by: Marc-André Lureau 
---
 tests/docker/docker.py | 43 +-
 1 file changed, 38 insertions(+), 5 deletions(-)

diff --git a/tests/docker/docker.py b/tests/docker/docker.py
index 53a8c9c801..1f59a78b10 100755
--- a/tests/docker/docker.py
+++ b/tests/docker/docker.py
@@ -20,6 +20,7 @@ import hashlib
 import atexit
 import uuid
 import argparse
+import enum
 import tempfile
 import re
 import signal
@@ -38,6 +39,26 @@ FILTERED_ENV_NAMES = ['ftp_proxy', 'http_proxy', 
'https_proxy']
 
 DEVNULL = open(os.devnull, 'wb')
 
+class EngineEnum(enum.IntEnum):
+AUTO = 1
+DOCKER = 2
+PODMAN = 3
+
+def __str__(self):
+return self.name.lower()
+
+def __repr__(self):
+return str(self)
+
+@staticmethod
+def argparse(s):
+try:
+return EngineEnum[s.upper()]
+except KeyError:
+return s
+
+
+USE_ENGINE = EngineEnum.AUTO
 
 def _text_checksum(text):
 """Calculate a digest string unique to the text content"""
@@ -48,9 +69,14 @@ def _file_checksum(filename):
 return _text_checksum(open(filename, 'rb').read())
 
 
-def _guess_docker_command():
-""" Guess a working docker command or raise exception if not found"""
-commands = [["docker"], ["sudo", "-n", "docker"]]
+def _guess_engine_command():
+""" Guess a working engine command or raise exception if not found"""
+commands = []
+
+if USE_ENGINE in [EngineEnum.AUTO, EngineEnum.PODMAN]:
+commands += [["podman"]]
+if USE_ENGINE in [EngineEnum.AUTO, EngineEnum.DOCKER]:
+commands += [["docker"], ["sudo", "-n", "docker"]]
 for cmd in commands:
 try:
 # docker version will return the client details in stdout
@@ -61,7 +87,7 @@ def _guess_docker_command():
 except OSError:
 pass
 commands_txt = "\n".join(["  " + " ".join(x) for x in commands])
-raise Exception("Cannot find working docker command. Tried:\n%s" %
+raise Exception("Cannot find working engine command. Tried:\n%s" %
 commands_txt)
 
 
@@ -190,7 +216,7 @@ def _dockerfile_preprocess(df):
 class Docker(object):
 """ Running Docker commands """
 def __init__(self):
-self._command = _guess_docker_command()
+self._command = _guess_engine_command()
 self._instances = []
 atexit.register(self._kill_instances)
 signal.signal(signal.SIGTERM, self._kill_instances)
@@ -502,6 +528,8 @@ class ProbeCommand(SubCommand):
 print("yes")
 elif docker._command[0] == "sudo":
 print("sudo")
+elif docker._command[0] == "podman":
+print("podman")
 except Exception:
 print("no")
 
@@ -597,9 +625,13 @@ class CheckCommand(SubCommand):
 
 
 def main():
+global USE_ENGINE
+
 parser = argparse.ArgumentParser(description="A Docker helper",
  usage="%s  ..." %
  os.path.basename(sys.argv[0]))
+parser.add_argument("--engine", type=EngineEnum.argparse, 
choices=list(EngineEnum),
+help="specify which container engine to use")
 subparsers = parser.add_subparsers(title="subcommands", help=None)
 for cls in SubCommand.__subclasses__():
 cmd = cls()
@@ -608,6 +640,7 @@ def main():
 cmd.args(subp)
 subp.set_defaults(cmdobj=cmd)
 args, argv = parser.parse_known_args()
+USE_ENGINE = args.engine
 return args.cmdobj.run(args, argv)
 
 
-- 
2.22.0.214.g8dca754b1e

[Qemu-devel] [PATCH v2 4/5] test-char: skip tcp tests if ipv4 check failed

Signed-off-by: Marc-André Lureau 
---
 tests/Makefile.include |  2 +-
 tests/test-char.c  | 17 +
 2 files changed, 14 insertions(+), 5 deletions(-)

diff --git a/tests/Makefile.include b/tests/Makefile.include
index a983dd32da..2bddebaf4b 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -523,7 +523,7 @@ tests/check-qlit$(EXESUF): tests/check-qlit.o 
$(test-util-obj-y)
 tests/check-qom-interface$(EXESUF): tests/check-qom-interface.o 
$(test-qom-obj-y)
 tests/check-qom-proplist$(EXESUF): tests/check-qom-proplist.o $(test-qom-obj-y)
 
-tests/test-char$(EXESUF): tests/test-char.o $(test-util-obj-y) $(qtest-obj-y) 
$(test-io-obj-y) $(chardev-obj-y)
+tests/test-char$(EXESUF): tests/test-char.o $(test-util-obj-y) $(qtest-obj-y) 
$(test-io-obj-y) $(chardev-obj-y) tests/socket-helpers.o
 tests/test-coroutine$(EXESUF): tests/test-coroutine.o $(test-block-obj-y)
 tests/test-aio$(EXESUF): tests/test-aio.o $(test-block-obj-y)
 tests/test-aio-multithread$(EXESUF): tests/test-aio-multithread.o 
$(test-block-obj-y)
diff --git a/tests/test-char.c b/tests/test-char.c
index f9440cdcfd..2dde620afc 100644
--- a/tests/test-char.c
+++ b/tests/test-char.c
@@ -15,6 +15,7 @@
 #include "io/channel-socket.h"
 #include "qapi/qobject-input-visitor.h"
 #include "qapi/qapi-visit-sockets.h"
+#include "socket-helpers.h"
 
 static bool quit;
 
@@ -1356,11 +1357,17 @@ static void char_hotswap_test(void)
 
 int main(int argc, char **argv)
 {
+bool has_ipv4, has_ipv6;
+
 qemu_init_main_loop(_abort);
 socket_init();
 
 g_test_init(, , NULL);
 
+if (socket_check_protocol_support(_ipv4, _ipv6) < 0) {
+return -1;
+}
+
 module_call_init(MODULE_INIT_QOM);
 qemu_add_opts(_chardev_opts);
 
@@ -1438,10 +1445,12 @@ int main(int argc, char **argv)
 g_test_add_data_func("/char/socket/client/wait-conn-fdpass/" # name, \
   ##name, char_socket_client_test)
 
-SOCKET_SERVER_TEST(tcp, );
-SOCKET_CLIENT_TEST(tcp, );
-g_test_add_data_func("/char/socket/server/two-clients/tcp", ,
- char_socket_server_two_clients_test);
+if (has_ipv4) {
+SOCKET_SERVER_TEST(tcp, );
+SOCKET_CLIENT_TEST(tcp, );
+g_test_add_data_func("/char/socket/server/two-clients/tcp", ,
+ char_socket_server_two_clients_test);
+}
 #ifndef WIN32
 SOCKET_SERVER_TEST(unix, );
 SOCKET_CLIENT_TEST(unix, );
-- 
2.22.0.214.g8dca754b1e

[Qemu-devel] [PATCH v2 2/5] tests/docker: add podman support

Allow to specify the container engine to run with ENGINE variable.

By default, ENGINE=auto and will select either podman or docker.

With current podman, we have to use a uidmap trick in order to be able
to rw-share the ccache directory with the container user.

With a user 1000, the default mapping is:
1000 (host) -> 0 (container).

So write access to /var/tmp/ccache ends will end with permission
denied error.

With "--uidmap 1000:0:1 --uidmap 0:1:1000", the mapping is:
1000 (host) -> 0 (container, 1st namespace) -> 1000 (container, 2nd namespace).

(the rest is mumbo jumbo to avoid holes in the range of UIDs)

A future podman version may have an option such as --userns-keep-uid.
Thanks to Debarshi Ray for the help!

Cc: Debarshi Ray 
Signed-off-by: Marc-André Lureau 
---
 Makefile  |  2 +-
 tests/docker/Makefile.include | 17 ++---
 2 files changed, 15 insertions(+), 4 deletions(-)

diff --git a/Makefile b/Makefile
index c63de4e36c..2bcf9bc674 100644
--- a/Makefile
+++ b/Makefile
@@ -1152,7 +1152,7 @@ endif
@echo  ''
@echo  'Test targets:'
@echo  '  check   - Run all tests (check-help for details)'
-   @echo  '  docker  - Help about targets running tests inside 
Docker containers'
+   @echo  '  docker  - Help about targets running tests inside 
containers'
@echo  '  vm-help - Help about targets running tests inside VM'
@echo  ''
@echo  'Documentation targets:'
diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index aaf5396b85..0abd2ab0c9 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -17,7 +17,9 @@ DOCKER_TESTS := $(notdir $(shell \
 
 DOCKER_TOOLS := travis
 
-DOCKER_SCRIPT=$(SRC_PATH)/tests/docker/docker.py
+ENGINE := auto
+
+DOCKER_SCRIPT=$(SRC_PATH)/tests/docker/docker.py --engine $(ENGINE)
 
 TESTS ?= %
 IMAGES ?= %
@@ -146,7 +148,7 @@ $(foreach i,$(filter-out 
$(DOCKER_PARTIAL_IMAGES),$(DOCKER_IMAGES) $(DOCKER_DEPR
 )
 
 docker:
-   @echo 'Build QEMU and run tests inside Docker containers'
+   @echo 'Build QEMU and run tests inside Docker or Podman containers'
@echo
@echo 'Available targets:'
@echo
@@ -193,6 +195,14 @@ endif
@echo 'EXECUTABLE=Include executable in image.'
@echo 'EXTRA_FILES=" [... ]"'
@echo ' Include extra files in image.'
+   @echo 'ENGINE=auto/docker/podman'
+   @echo ' Specify which container engine to run.'
+
+UID=$(shell id -u)
+UID1=$(shell expr $(UID) + 1)
+ifeq ($(shell $(DOCKER_SCRIPT) probe),podman)
+PODMAN=1
+endif
 
 # This rule if for directly running against an arbitrary docker target.
 # It is called by the expanded docker targets (e.g. make
@@ -212,7 +222,8 @@ docker-run: docker-qemu-src
"  COPYING $(EXECUTABLE) to $(IMAGE)"))
$(call quiet-command,   \
$(DOCKER_SCRIPT) run\
-   $(if $(NOUSER),,-u $(shell id -u))  \
+   $(if $(NOUSER),,-u $(UID)   \
+   $(if $(PODMAN),--uidmap $(UID):0:1 --uidmap 
0:1:$(UID) --uidmap $(UID1):$(UID1):64536)) \
--security-opt seccomp=unconfined   \
$(if $V,,--rm)  \
$(if $(DEBUG),-ti,) \
-- 
2.22.0.214.g8dca754b1e

[Qemu-devel] [PATCH v2 5/5] test: skip tests if socket_check_protocol_support() failed

Skip the tests if socket_check_protocol_support() failed, but do run
g_test_run() to keep TAP harness happy.

Signed-off-by: Marc-André Lureau 
---
 tests/test-char.c  | 4 +++-
 tests/test-io-channel-socket.c | 4 +++-
 tests/test-util-sockets.c  | 4 +++-
 3 files changed, 9 insertions(+), 3 deletions(-)

diff --git a/tests/test-char.c b/tests/test-char.c
index 2dde620afc..b56e43c1eb 100644
--- a/tests/test-char.c
+++ b/tests/test-char.c
@@ -1365,7 +1365,8 @@ int main(int argc, char **argv)
 g_test_init(, , NULL);
 
 if (socket_check_protocol_support(_ipv4, _ipv6) < 0) {
-return -1;
+g_printerr("socket_check_protocol_support() failed\n");
+goto end;
 }
 
 module_call_init(MODULE_INIT_QOM);
@@ -1465,5 +1466,6 @@ int main(int argc, char **argv)
 g_test_add_func("/char/hotswap", char_hotswap_test);
 g_test_add_func("/char/websocket", char_websock_test);
 
+end:
 return g_test_run();
 }
diff --git a/tests/test-io-channel-socket.c b/tests/test-io-channel-socket.c
index d2053c464c..d172f3070f 100644
--- a/tests/test-io-channel-socket.c
+++ b/tests/test-io-channel-socket.c
@@ -566,7 +566,8 @@ int main(int argc, char **argv)
  * with either IPv4 or IPv6 disabled.
  */
 if (socket_check_protocol_support(_ipv4, _ipv6) < 0) {
-return 1;
+g_printerr("socket_check_protocol_support() failed\n");
+goto end;
 }
 
 if (has_ipv4) {
@@ -595,5 +596,6 @@ int main(int argc, char **argv)
 test_io_channel_unix_listen_cleanup);
 #endif /* _WIN32 */
 
+end:
 return g_test_run();
 }
diff --git a/tests/test-util-sockets.c b/tests/test-util-sockets.c
index f1ebffee5a..e2a3a8a093 100644
--- a/tests/test-util-sockets.c
+++ b/tests/test-util-sockets.c
@@ -242,7 +242,8 @@ int main(int argc, char **argv)
  * with either IPv4 or IPv6 disabled.
  */
 if (socket_check_protocol_support(_ipv4, _ipv6) < 0) {
-return 1;
+g_printerr("socket_check_protocol_support() failed\n");
+goto end;
 }
 
 if (has_ipv4) {
@@ -264,5 +265,6 @@ int main(int argc, char **argv)
 test_socket_fd_pass_num_nocli);
 }
 
+end:
 return g_test_run();
 }
-- 
2.22.0.214.g8dca754b1e

[Qemu-devel] [PATCH v2 0/5] tests/docker: add podman support

Hi,

podman allows to run containers in a similar fashion as docker, but
without daemon or root privileges. Thank you podman!

There is a weird issue with getaddrinfo(), that I reported upstream
https://github.com/containers/libpod/issues/3535. For now, it is
worked around with extra socket_check_protocol_support() checks.

thanks

v2:
- add socket_check_protocol_support() to test-char
- keep TAP harness happy when socket_check_protocol_support() fails
- removed bad AI_ADDRCONFIG patch
- rebased

Marc-André Lureau (5):
  docker.py: add podman support
  tests/docker: add podman support
  tests: specify the address family when checking bind
  test-char: skip tcp tests if ipv4 check failed
  test: skip tests if socket_check_protocol_support() failed

 Makefile   |  2 +-
 tests/Makefile.include |  2 +-
 tests/docker/Makefile.include  | 17 +++---
 tests/docker/docker.py | 43 ++
 tests/socket-helpers.c | 17 ++
 tests/socket-helpers.h | 11 -
 tests/test-char.c  | 19 +++
 tests/test-io-channel-socket.c |  4 +++-
 tests/test-util-sockets.c  |  4 +++-
 9 files changed, 88 insertions(+), 31 deletions(-)

-- 
2.22.0.214.g8dca754b1e

[Qemu-devel] [PATCH v2 3/5] tests: specify the address family when checking bind

getaddrinfo() may succeed with PF_UNSPEC, but fail when more specific.

(this allows to skip some tests that would fail under podman)

Signed-off-by: Marc-André Lureau 
---
 tests/socket-helpers.c | 17 +
 tests/socket-helpers.h | 11 ---
 2 files changed, 13 insertions(+), 15 deletions(-)

diff --git a/tests/socket-helpers.c b/tests/socket-helpers.c
index 8112763f5b..19a51e887e 100644
--- a/tests/socket-helpers.c
+++ b/tests/socket-helpers.c
@@ -30,7 +30,16 @@
 # define EAI_ADDRFAMILY 0
 #endif
 
-int socket_can_bind_connect(const char *hostname)
+/*
+ * @hostname: a DNS name or numeric IP address
+ *
+ * Check whether it is possible to bind & connect to ports
+ * on the DNS name or IP address @hostname. If an IP address
+ * is used, it must not be a wildcard address.
+ *
+ * Returns 0 on success, -1 on error with errno set
+ */
+static int socket_can_bind_connect(const char *hostname, int family)
 {
 int lfd = -1, cfd = -1, afd = -1;
 struct addrinfo ai, *res = NULL;
@@ -44,7 +53,7 @@ int socket_can_bind_connect(const char *hostname)
 
 memset(, 0, sizeof(ai));
 ai.ai_flags = AI_CANONNAME | AI_ADDRCONFIG;
-ai.ai_family = AF_UNSPEC;
+ai.ai_family = family;
 ai.ai_socktype = SOCK_STREAM;
 
 /* lookup */
@@ -129,7 +138,7 @@ int socket_check_protocol_support(bool *has_ipv4, bool 
*has_ipv6)
 {
 *has_ipv4 = *has_ipv6 = false;
 
-if (socket_can_bind_connect("127.0.0.1") < 0) {
+if (socket_can_bind_connect("127.0.0.1", PF_INET) < 0) {
 if (errno != EADDRNOTAVAIL) {
 return -1;
 }
@@ -137,7 +146,7 @@ int socket_check_protocol_support(bool *has_ipv4, bool 
*has_ipv6)
 *has_ipv4 = true;
 }
 
-if (socket_can_bind_connect("::1") < 0) {
+if (socket_can_bind_connect("::1", PF_INET6) < 0) {
 if (errno != EADDRNOTAVAIL) {
 return -1;
 }
diff --git a/tests/socket-helpers.h b/tests/socket-helpers.h
index 9de0e6b151..512a004811 100644
--- a/tests/socket-helpers.h
+++ b/tests/socket-helpers.h
@@ -20,17 +20,6 @@
 #ifndef TESTS_SOCKET_HELPERS_H
 #define TESTS_SOCKET_HELPERS_H
 
-/*
- * @hostname: a DNS name or numeric IP address
- *
- * Check whether it is possible to bind & connect to ports
- * on the DNS name or IP address @hostname. If an IP address
- * is used, it must not be a wildcard address.
- *
- * Returns 0 on success, -1 on error with errno set
- */
-int socket_can_bind_connect(const char *hostname);
-
 /*
  * @has_ipv4: set to true on return if IPv4 is available
  * @has_ipv6: set to true on return if IPv6 is available
-- 
2.22.0.214.g8dca754b1e

Re: [Qemu-devel] RFC: Why does target/m68k RTE insn. use gen_exception

2019-07-09 Thread Richard Henderson

On 7/9/19 6:58 PM, Lucien Murray-Pitts wrote:
> Any suggestions on how to obtain pc_next from the "m68k_cpu_do_interrupt(
> CPUState *cs)" ?

I did have a suggestion.  It was fairly detailed.

https://lists.gnu.org/archive/html/qemu-devel/2019-06/msg06522.html


r~

[Qemu-devel] [ANNOUNCE] QEMU 4.1.0-rc0 is now available

2019-07-09 Thread Michael Roth

Hello,

On behalf of the QEMU Team, I'd like to announce the availability of the
first release candidate for the QEMU 4.1 release.  This release is meant
for testing purposes and should not be used in a production environment.

  http://download.qemu-project.org/qemu-4.1.0-rc0.tar.xz
  http://download.qemu-project.org/qemu-4.1.0-rc0.tar.xz.sig

You can help improve the quality of the QEMU 4.1 release by testing this
release and reporting bugs on Launchpad:

  https://bugs.launchpad.net/qemu/

The release plan, as well a documented known issues for release
candidates, are available at:

  http://wiki.qemu.org/Planning/4.1

Please add entries to the ChangeLog for the 4.1 release below:

  http://wiki.qemu.org/ChangeLog/4.1

Thank you to everyone involved!

Re: [Qemu-devel] [PATCH v3 17/18] iotests: add test 257 for bitmap-mode backups

On 05.07.19 22:16, John Snow wrote:
> Signed-off-by: John Snow 
> ---
>  tests/qemu-iotests/257 |  409 +++
>  tests/qemu-iotests/257.out | 2199 
>  tests/qemu-iotests/group   |1 +
>  3 files changed, 2609 insertions(+)
>  create mode 100755 tests/qemu-iotests/257
>  create mode 100644 tests/qemu-iotests/257.out

Reviewed-by: Max Reitz 



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v3 12/18] block/backup: add 'always' bitmap sync policy

On 05.07.19 22:16, John Snow wrote:
> This adds an "always" policy for bitmap synchronization. Regardless of if
> the job succeeds or fails, the bitmap is *always* synchronized. This means
> that for backups that fail part-way through, the bitmap retains a record of
> which sectors need to be copied out to accomplish a new backup using the
> old, partial result.
> 
> In effect, this allows us to "resume" a failed backup; however the new backup
> will be from the new point in time, so it isn't a "resume" as much as it is
> an "incremental retry." This can be useful in the case of extremely large
> backups that fail considerably through the operation and we'd like to not 
> waste
> the work that was already performed.
> 
> Signed-off-by: John Snow 
> ---
>  block/backup.c   | 27 +++
>  qapi/block-core.json |  5 -
>  2 files changed, 23 insertions(+), 9 deletions(-)

Reviewed-by: Max Reitz 



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v3 2/9] hw/block/pflash_cfi01: Use the correct READ_ARRAY value

2019-07-09 Thread Philippe Mathieu-Daudé

On 7/9/19 7:10 PM, Dr. David Alan Gilbert wrote:
> * Philippe Mathieu-Daudé (phi...@redhat.com) wrote:
>> Hi David,
>>
>> On 7/9/19 12:30 PM, Dr. David Alan Gilbert wrote:
>>> * Philippe Mathieu-Daudé (phi...@redhat.com) wrote:
 In the "Read Array Flowchart" the command has a value of 0xFF.

 In the document [*] the "Read Array Flowchart", the READ_ARRAY
 command has a value of 0xff.

 Use the correct value in the pflash model.

 There is no change of behavior in the guest, because:
 - when the guest were sending 0xFF, the reset_flash label
   was setting the command value as 0x00
 - 0x00 was used internally for READ_ARRAY

 To keep migration behaving correctly, we have to increase
 the VMState version. When migrating from an older version,
 we use the correct command value.
>>>
>>> The problem is that incrementing the version will break backwards
>>> compatibility; so you won't be able to migrate this back to an older
>>> QEMU version; so for example a q35/uefi with this won't be able
>>> to migrate backwards to a 4.0.0 or older qemu.
>>>
>>> So instead of bumping the version_id you probably need to wire
>>> the behaviour to a machine type and then on your new type
>>> wire a subsection containing a flag; the reception of that subsection
>>> tells you to use the new/correct semantics.
>>
>> I'm starting to understand VMState subsections, but it might be overkill
>> for this change...
>>
>>   Subsections
>>   ---
>>
>>   The most common structure change is adding new data, e.g. when adding
>>   a newer form of device, or adding that state that you previously
>>   forgot to migrate.  This is best solved using a subsection.
>>
>> This is not the case here, the field is already present and migrated.
>>
>> It seems I can use a simple pre_save hook, always migrating the
>> READ_ARRAY using the incorrect value:
>>
>> -- >8 --
>> --- a/hw/block/pflash_cfi01.c
>> +++ b/hw/block/pflash_cfi01.c
>> @@ -97,12 +97,29 @@ struct PFlashCFI01 {
>>  bool incorrect_read_array_command;
>>  };
>>
>> +static int pflash_pre_save(void *opaque)
>> +{
>> +PFlashCFI01 *s = opaque;
>> +
>> +/*
>> + * Previous to QEMU v4.1 an incorrect value of 0x00 was used for the
>> + * READ_ARRAY command. To preserve migrating to these older version,
>> + * always migrate the READ_ARRAY command as 0x00.
>> + */
>> +if (s->cmd == 0xff) {
>> +s->cmd = 0x00;
>> +}
>> +
>> +return 0;
>> +}
> 
> Be careful what happens if migration fails and you continue on the
> source - is that OK - or are you going to have to flip that back somehow
> (in a post_save).

Hmm OK...

> 
> Another way to do the same is to have a dummy field; tmp_cmd, and the
> tmp_cmd is the thing that's actually migrated and filled by pre_save
> (or use VMSTATE_WITH_TMP )
> 
> 
>>  static int pflash_post_load(void *opaque, int version_id);
>>
>>  static const VMStateDescription vmstate_pflash = {
>>  .name = "pflash_cfi01",
>>  .version_id = 1,
>>  .minimum_version_id = 1,
>> +.pre_save = pflash_pre_save,
>>  .post_load = pflash_post_load,
>>  .fields = (VMStateField[]) {
>>  VMSTATE_UINT8(wcycle, PFlashCFI01),
>> @@ -1001,5 +1018,14 @@ static int pflash_post_load(void *opaque, int
>> version_id)
>>  pfl->vmstate = qemu_add_vm_change_state_handler(postload_update_cb,
>>  pfl);
>>  }
>> +
>> +/*
>> + * Previous to QEMU v4.1 an incorrect value of 0x00 was used for the
>> + * READ_ARRAY command.
>> + */
>> +if (pfl->cmd == 0x00) {
>> +pfl->cmd = 0xff;
>> +}
>> +
>>  return 0;
>>  }
>> ---
>>
>> Being simpler and less intrusive (no new property in hw/core/machine.c),
>> is this acceptable?
> 
> From the migration point of view yes; I don't know enough about pflash
> to say if it makes sense;  for example could there ever be a 00 command
> really used and then you'd have to distinguish that somehow?

Well, I think this change is simpler than it looks.

Currently the code is (what will run on older guest):

static uint32_t pflash_read(PFlashCFI01 *pfl, hwaddr offset,
int width, int be)
{
switch (pfl->cmd) {
default:
/* This should never happen : reset state & treat it as a read*/
DPRINTF("%s: unknown command state: %x\n", __func__, pfl->cmd);
pfl->wcycle = 0;
pfl->cmd = 0;
/* fall through to read code */
case 0x00:
/* Flash area read */
ret = pflash_data_read(pfl, offset, width, be);
break;

and:

static void pflash_write(PFlashCFI01 *pfl, hwaddr offset,
 uint32_t value, int width, int be)
{
switch (pfl->wcycle) {
case 0:
/* read mode */
switch (cmd) {
case 0x00: /* ??? */
goto reset_flash;
case 0xff: /* Read array mode */
DPRINTF("%s: Read array mode\n",

Re: [Qemu-devel] [PATCH v3 05/18] block/backup: Add mirror sync mode 'bitmap'

On 05.07.19 22:16, John Snow wrote:
> We don't need or want a new sync mode for simple differences in
> semantics.  Create a new mode simply named "BITMAP" that is designed to
> make use of the new Bitmap Sync Mode field.
> 
> Because the only bitmap sync mode is 'on-success', this adds no new
> functionality to the backup job (yet). The old incremental backup mode
> is maintained as a syntactic sugar for sync=bitmap, mode=on-success.
> 
> Add all of the plumbing necessary to support this new instruction.
> 
> Signed-off-by: John Snow 
> ---
>  block/backup.c| 20 
>  block/mirror.c|  6 --
>  block/replication.c   |  2 +-
>  blockdev.c| 25 +++--
>  include/block/block_int.h |  4 +++-
>  qapi/block-core.json  | 21 +++--
>  6 files changed, 58 insertions(+), 20 deletions(-)

Reviewed-by: Max Reitz 



signature.asc
Description: OpenPGP digital signature

[Qemu-devel] [PATCH 07/11] audio: remove audio_MIN, audio_MAX

There's already a MIN and MAX macro in include/qemu/osdep.h, use them
instead.

Signed-off-by: Kővágó, Zoltán 
---
 audio/audio.h | 17 -
 audio/alsaaudio.c |  6 +++---
 audio/audio.c | 20 ++--
 audio/coreaudio.c |  2 +-
 audio/dsoundaudio.c   |  2 +-
 audio/noaudio.c   | 10 +-
 audio/ossaudio.c  |  6 +++---
 audio/paaudio.c   | 12 ++--
 audio/sdlaudio.c  |  6 +++---
 audio/spiceaudio.c| 10 +-
 audio/wavaudio.c  |  4 ++--
 hw/audio/ac97.c   | 10 +-
 hw/audio/adlib.c  |  4 ++--
 hw/audio/cs4231a.c|  4 ++--
 hw/audio/es1370.c |  6 +++---
 hw/audio/gus.c|  6 +++---
 hw/audio/hda-codec.c  | 16 
 hw/audio/milkymist-ac97.c |  8 
 hw/audio/pcspk.c  |  2 +-
 hw/audio/sb16.c   |  2 +-
 hw/audio/wm8750.c |  4 ++--
 21 files changed, 70 insertions(+), 87 deletions(-)

diff --git a/audio/audio.h b/audio/audio.h
index c0722a5cda..4a95758516 100644
--- a/audio/audio.h
+++ b/audio/audio.h
@@ -146,23 +146,6 @@ static inline void *advance (void *p, int incr)
 return (d + incr);
 }
 
-#ifdef __GNUC__
-#define audio_MIN(a, b) ( __extension__ ({  \
-__typeof (a) ta = a;\
-__typeof (b) tb = b;\
-((ta)>(tb)?(tb):(ta));  \
-}))
-
-#define audio_MAX(a, b) ( __extension__ ({  \
-__typeof (a) ta = a;\
-__typeof (b) tb = b;\
-((ta)<(tb)?(tb):(ta));  \
-}))
-#else
-#define audio_MIN(a, b) ((a)>(b)?(b):(a))
-#define audio_MAX(a, b) ((a)<(b)?(b):(a))
-#endif
-
 int wav_start_capture(AudioState *state, CaptureState *s, const char *path,
   int freq, int bits, int nchannels);
 
diff --git a/audio/alsaaudio.c b/audio/alsaaudio.c
index 3745c823ad..6b9e0f06af 100644
--- a/audio/alsaaudio.c
+++ b/audio/alsaaudio.c
@@ -634,7 +634,7 @@ static void alsa_write_pending (ALSAVoiceOut *alsa)
 
 while (alsa->pending) {
 int left_till_end_samples = hw->samples - alsa->wpos;
-int len = audio_MIN (alsa->pending, left_till_end_samples);
+int len = MIN (alsa->pending, left_till_end_samples);
 char *src = advance (alsa->pcm_buf, alsa->wpos << hw->info.shift);
 
 while (len) {
@@ -697,7 +697,7 @@ static int alsa_run_out (HWVoiceOut *hw, int live)
 return 0;
 }
 
-decr = audio_MIN (live, avail);
+decr = MIN (live, avail);
 decr = audio_pcm_hw_clip_out (hw, alsa->pcm_buf, decr, alsa->pending);
 alsa->pending += decr;
 alsa_write_pending (alsa);
@@ -915,7 +915,7 @@ static int alsa_run_in (HWVoiceIn *hw)
 }
 }
 
-decr = audio_MIN (dead, avail);
+decr = MIN (dead, avail);
 if (!decr) {
 return 0;
 }
diff --git a/audio/audio.c b/audio/audio.c
index 82dd0e3e13..6bf30ac9b3 100644
--- a/audio/audio.c
+++ b/audio/audio.c
@@ -533,7 +533,7 @@ static int audio_pcm_hw_find_min_in (HWVoiceIn *hw)
 
 for (sw = hw->sw_head.lh_first; sw; sw = sw->entries.le_next) {
 if (sw->active) {
-m = audio_MIN (m, sw->total_hw_samples_acquired);
+m = MIN (m, sw->total_hw_samples_acquired);
 }
 }
 return m;
@@ -553,14 +553,14 @@ int audio_pcm_hw_clip_out (HWVoiceOut *hw, void *pcm_buf,
int live, int pending)
 {
 int left = hw->samples - pending;
-int len = audio_MIN (left, live);
+int len = MIN (left, live);
 int clipped = 0;
 
 while (len) {
 struct st_sample *src = hw->mix_buf + hw->rpos;
 uint8_t *dst = advance (pcm_buf, hw->rpos << hw->info.shift);
 int samples_till_end_of_buf = hw->samples - hw->rpos;
-int samples_to_clip = audio_MIN (len, samples_till_end_of_buf);
+int samples_to_clip = MIN (len, samples_till_end_of_buf);
 
 hw->clip (dst, src, samples_to_clip);
 
@@ -614,7 +614,7 @@ int audio_pcm_sw_read (SWVoiceIn *sw, void *buf, int size)
 }
 
 swlim = (live * sw->ratio) >> 32;
-swlim = audio_MIN (swlim, samples);
+swlim = MIN (swlim, samples);
 
 while (swlim) {
 src = hw->conv_buf + rpos;
@@ -662,7 +662,7 @@ static int audio_pcm_hw_find_min_out (HWVoiceOut *hw, int 
*nb_livep)
 
 for (sw = hw->sw_head.lh_first; sw; sw = sw->entries.le_next) {
 if (sw->active || !sw->empty) {
-m = audio_MIN (m, sw->total_hw_samples_mixed);
+m = MIN (m, sw->total_hw_samples_mixed);
 nb_live += 1;
 }
 }
@@ -725,7 +725,7 @@ int audio_pcm_sw_write (SWVoiceOut *sw, void *buf, int size)
 
 dead = hwsamples - live;
 swlim = ((int64_t) dead << 32) / sw->ratio;
-swlim = audio_MIN (swlim, samples);
+swlim = MIN (swlim, samples);
 if (swlim) {
 sw->conv (sw->buf, buf, swlim);
 
@@ -737,7 +737,7 @@

[Qemu-devel] [PATCH 11/11] audio: use size_t where makes sense

Signed-off-by: Kővágó, Zoltán 
---
 audio/audio.h   |   4 +-
 audio/audio_int.h   |  26 +++
 audio/audio_template.h  |  14 ++--
 audio/mixeng.h  |   9 +--
 audio/rate_template.h   |   2 +-
 include/sysemu/replay.h |   4 +-
 audio/alsaaudio.c   |  26 +++
 audio/audio.c   | 156 
 audio/coreaudio.c   |  10 +--
 audio/dsoundaudio.c |  11 +--
 audio/noaudio.c |  16 ++---
 audio/ossaudio.c|  45 ++--
 audio/paaudio.c |  44 ++--
 audio/sdlaudio.c|  20 +++---
 audio/spiceaudio.c  |  12 ++--
 audio/wavaudio.c|   8 +--
 replay/replay-audio.c   |  16 ++---
 replay/replay.c |   2 +-
 18 files changed, 212 insertions(+), 213 deletions(-)

diff --git a/audio/audio.h b/audio/audio.h
index 4a95758516..2db27bba7b 100644
--- a/audio/audio.h
+++ b/audio/audio.h
@@ -112,7 +112,7 @@ SWVoiceOut *AUD_open_out (
 );
 
 void AUD_close_out (QEMUSoundCard *card, SWVoiceOut *sw);
-int  AUD_write (SWVoiceOut *sw, void *pcm_buf, int size);
+size_t AUD_write (SWVoiceOut *sw, void *pcm_buf, size_t size);
 int  AUD_get_buffer_size_out (SWVoiceOut *sw);
 void AUD_set_active_out (SWVoiceOut *sw, int on);
 int  AUD_is_active_out (SWVoiceOut *sw);
@@ -133,7 +133,7 @@ SWVoiceIn *AUD_open_in (
 );
 
 void AUD_close_in (QEMUSoundCard *card, SWVoiceIn *sw);
-int  AUD_read (SWVoiceIn *sw, void *pcm_buf, int size);
+size_t AUD_read (SWVoiceIn *sw, void *pcm_buf, size_t size);
 void AUD_set_active_in (SWVoiceIn *sw, int on);
 int  AUD_is_active_in (SWVoiceIn *sw);
 
diff --git a/audio/audio_int.h b/audio/audio_int.h
index 003b7ab8cc..a674c5374a 100644
--- a/audio/audio_int.h
+++ b/audio/audio_int.h
@@ -61,12 +61,12 @@ typedef struct HWVoiceOut {
 
 f_sample *clip;
 
-int rpos;
+size_t rpos;
 uint64_t ts_helper;
 
 struct st_sample *mix_buf;
 
-int samples;
+size_t samples;
 QLIST_HEAD (sw_out_listhead, SWVoiceOut) sw_head;
 QLIST_HEAD (sw_cap_listhead, SWVoiceCap) cap_head;
 int ctl_caps;
@@ -82,13 +82,13 @@ typedef struct HWVoiceIn {
 
 t_sample *conv;
 
-int wpos;
-int total_samples_captured;
+size_t wpos;
+size_t total_samples_captured;
 uint64_t ts_helper;
 
 struct st_sample *conv_buf;
 
-int samples;
+size_t samples;
 QLIST_HEAD (sw_in_listhead, SWVoiceIn) sw_head;
 int ctl_caps;
 struct audio_pcm_ops *pcm_ops;
@@ -103,7 +103,7 @@ struct SWVoiceOut {
 int64_t ratio;
 struct st_sample *buf;
 void *rate;
-int total_hw_samples_mixed;
+size_t total_hw_samples_mixed;
 int active;
 int empty;
 HWVoiceOut *hw;
@@ -120,7 +120,7 @@ struct SWVoiceIn {
 struct audio_pcm_info info;
 int64_t ratio;
 void *rate;
-int total_hw_samples_acquired;
+size_t total_hw_samples_acquired;
 struct st_sample *buf;
 f_sample *clip;
 HWVoiceIn *hw;
@@ -149,12 +149,12 @@ struct audio_driver {
 struct audio_pcm_ops {
 int  (*init_out)(HWVoiceOut *hw, struct audsettings *as, void *drv_opaque);
 void (*fini_out)(HWVoiceOut *hw);
-int  (*run_out) (HWVoiceOut *hw, int live);
+size_t (*run_out)(HWVoiceOut *hw, size_t live);
 int  (*ctl_out) (HWVoiceOut *hw, int cmd, ...);
 
 int  (*init_in) (HWVoiceIn *hw, struct audsettings *as, void *drv_opaque);
 void (*fini_in) (HWVoiceIn *hw);
-int  (*run_in)  (HWVoiceIn *hw);
+size_t (*run_in)(HWVoiceIn *hw);
 int  (*ctl_in)  (HWVoiceIn *hw, int cmd, ...);
 };
 
@@ -208,10 +208,10 @@ audio_driver *audio_driver_lookup(const char *name);
 void audio_pcm_init_info (struct audio_pcm_info *info, struct audsettings *as);
 void audio_pcm_info_clear_buf (struct audio_pcm_info *info, void *buf, int 
len);
 
-int  audio_pcm_hw_get_live_in (HWVoiceIn *hw);
+size_t audio_pcm_hw_get_live_in(HWVoiceIn *hw);
 
-int audio_pcm_hw_clip_out (HWVoiceOut *hw, void *pcm_buf,
-   int live, int pending);
+size_t audio_pcm_hw_clip_out(HWVoiceOut *hw, void *pcm_buf,
+ size_t live, size_t pending);
 
 int audio_bug (const char *funcname, int cond);
 void *audio_calloc (const char *funcname, int nmemb, size_t size);
@@ -224,7 +224,7 @@ void audio_run(AudioState *s, const char *msg);
 
 #define VOICE_VOLUME_CAP (1 << VOICE_VOLUME)
 
-static inline int audio_ring_dist (int dst, int src, int len)
+static inline size_t audio_ring_dist(size_t dst, size_t src, size_t len)
 {
 return (dst >= src) ? (dst - src) : (len - src + dst);
 }
diff --git a/audio/audio_template.h b/audio/audio_template.h
index 54f07338e7..2562bf5f00 100644
--- a/audio/audio_template.h
+++ b/audio/audio_template.h
@@ -75,16 +75,16 @@ static void glue (audio_pcm_hw_free_resources_, TYPE) (HW 
*hw)
 HWBUF = NULL;
 }
 
-static int glue (audio_pcm_hw_alloc_resources_, TYPE) (HW *hw)
+static bool glue(audio_pcm_hw_alloc_resources_, TYPE)(HW *hw)
 {
 HWBUF = audio_calloc(__func__, hw->samples,

[Qemu-devel] [PATCH 02/11] audio: basic support for multi backend audio

Audio functions no longer access glob_audio_state, instead they get an
AudioState as a parameter.  This is required in order to support
multiple backends.

glob_audio_state is also gone, and replaced with a tailq so we can store
more than one states.

Signed-off-by: Kővágó, Zoltán 
---
 audio/audio.h  |  12 +++--
 audio/audio_int.h  |   2 +
 audio/audio_template.h |   2 +-
 ui/vnc.h   |   2 +
 audio/audio.c  | 102 +++--
 audio/wavcapture.c |   6 +--
 monitor/misc.c |  12 -
 ui/vnc.c   |  15 +-
 hmp-commands.hx|  11 +++--
 qemu-options.hx|   5 ++
 10 files changed, 131 insertions(+), 38 deletions(-)

diff --git a/audio/audio.h b/audio/audio.h
index 64b0f761bc..ad2457f4de 100644
--- a/audio/audio.h
+++ b/audio/audio.h
@@ -78,8 +78,10 @@ typedef struct SWVoiceOut SWVoiceOut;
 typedef struct CaptureVoiceOut CaptureVoiceOut;
 typedef struct SWVoiceIn SWVoiceIn;
 
+typedef struct AudioState AudioState;
 typedef struct QEMUSoundCard {
 char *name;
+AudioState *state;
 QLIST_ENTRY (QEMUSoundCard) entries;
 } QEMUSoundCard;
 
@@ -92,7 +94,8 @@ void AUD_log (const char *cap, const char *fmt, ...) 
GCC_FMT_ATTR(2, 3);
 
 void AUD_register_card (const char *name, QEMUSoundCard *card);
 void AUD_remove_card (QEMUSoundCard *card);
-CaptureVoiceOut *AUD_add_capture (
+CaptureVoiceOut *AUD_add_capture(
+AudioState *s,
 struct audsettings *as,
 struct audio_capture_ops *ops,
 void *opaque
@@ -160,8 +163,8 @@ static inline void *advance (void *p, int incr)
 #define audio_MAX(a, b) ((a)<(b)?(b):(a))
 #endif
 
-int wav_start_capture (CaptureState *s, const char *path, int freq,
-   int bits, int nchannels);
+int wav_start_capture(AudioState *state, CaptureState *s, const char *path,
+  int freq, int bits, int nchannels);
 
 bool audio_is_cleaning_up(void);
 void audio_cleanup(void);
@@ -175,4 +178,7 @@ void audio_parse_option(const char *opt);
 void audio_init_audiodevs(void);
 void audio_legacy_help(void);
 
+AudioState *audio_state_by_name(const char *name);
+const char *audio_get_id(QEMUSoundCard *card);
+
 #endif /* QEMU_AUDIO_H */
diff --git a/audio/audio_int.h b/audio/audio_int.h
index 8164696b2c..9f01f6ad00 100644
--- a/audio/audio_int.h
+++ b/audio/audio_int.h
@@ -196,6 +196,8 @@ typedef struct AudioState {
 
 bool timer_running;
 uint64_t timer_last;
+
+QTAILQ_ENTRY(AudioState) list;
 } AudioState;
 
 extern const struct mixeng_volume nominal_volume;
diff --git a/audio/audio_template.h b/audio/audio_template.h
index c721fed75d..54f07338e7 100644
--- a/audio/audio_template.h
+++ b/audio/audio_template.h
@@ -428,7 +428,7 @@ SW *glue (AUD_open_, TYPE) (
 struct audsettings *as
 )
 {
-AudioState *s = _audio_state;
+AudioState *s = card->state;
 AudiodevPerDirectionOptions *pdo = glue(audio_get_pdo_, TYPE)(s->dev);
 
 if (audio_bug(__func__, !card || !name || !callback_fn || !as)) {
diff --git a/ui/vnc.h b/ui/vnc.h
index 2f84db3142..6f54653455 100644
--- a/ui/vnc.h
+++ b/ui/vnc.h
@@ -183,6 +183,8 @@ struct VncDisplay
 #ifdef CONFIG_VNC_SASL
 VncDisplaySASL sasl;
 #endif
+
+AudioState *audio_state;
 };
 
 typedef struct VncTight {
diff --git a/audio/audio.c b/audio/audio.c
index 8d2f580788..e9dd7c8b32 100644
--- a/audio/audio.c
+++ b/audio/audio.c
@@ -87,7 +87,8 @@ audio_driver *audio_driver_lookup(const char *name)
 return NULL;
 }
 
-static AudioState glob_audio_state;
+static QTAILQ_HEAD(AudioStateHead, AudioState) audio_states =
+QTAILQ_HEAD_INITIALIZER(audio_states);
 
 const struct mixeng_volume nominal_volume = {
 .mute = 0,
@@ -1236,11 +1237,14 @@ static void audio_run_capture (AudioState *s)
 
 void audio_run (const char *msg)
 {
-AudioState *s = _audio_state;
+AudioState *s;
+
+QTAILQ_FOREACH(s, _states, list) {
+audio_run_out(s);
+audio_run_in(s);
+audio_run_capture(s);
+}
 
-audio_run_out (s);
-audio_run_in (s);
-audio_run_capture (s);
 #ifdef DEBUG_POLL
 {
 static double prevtime;
@@ -1304,13 +1308,11 @@ bool audio_is_cleaning_up(void)
 return is_cleaning_up;
 }
 
-void audio_cleanup(void)
+static void free_audio_state(AudioState *s)
 {
-AudioState *s = _audio_state;
 HWVoiceOut *hwo, *hwon;
 HWVoiceIn *hwi, *hwin;
 
-is_cleaning_up = true;
 QLIST_FOREACH_SAFE(hwo, >hw_head_out, entries, hwon) {
 SWVoiceCap *sc;
 
@@ -1347,6 +1349,17 @@ void audio_cleanup(void)
 qapi_free_Audiodev(s->dev);
 s->dev = NULL;
 }
+g_free(s);
+}
+
+void audio_cleanup(void)
+{
+is_cleaning_up = true;
+while (!QTAILQ_EMPTY(_states)) {
+AudioState *s = QTAILQ_FIRST(_states);
+QTAILQ_REMOVE(_states, s, list);
+free_audio_state(s);
+}
 }
 
 static const VMStateDescription vmstate_audio = {
@@ -1373,28 +1386,33 @@ static AudiodevListEntry

[Qemu-devel] [PATCH] qdev: Collect HMP handlers command handlers in qdev-monitor.c

Move hmp_device_add(), hmp_device_del() from monitor/hmp-cmds.c to
qdev-monitor.c, where they are covered by MAINTAINERS section "QOM",
just like qapi/qdev.json.  hmp_info_qtree() and hmp_info_qdm() are
already there.

Signed-off-by: Markus Armbruster 
---
This is a follow-up to my "Move QOM, qdev, machine core and dump code"
series, merged in commit 374f63f6810.

 monitor/hmp-cmds.c | 19 ---
 qdev-monitor.c | 18 ++
 2 files changed, 18 insertions(+), 19 deletions(-)

diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c
index 99ceb0846b..550d363374 100644
--- a/monitor/hmp-cmds.c
+++ b/monitor/hmp-cmds.c
@@ -25,7 +25,6 @@
 #include "qemu/timer.h"
 #include "qemu/sockets.h"
 #include "monitor/monitor-internal.h"
-#include "monitor/qdev.h"
 #include "qapi/error.h"
 #include "qapi/clone-visitor.h"
 #include "qapi/opts-visitor.h"
@@ -35,7 +34,6 @@
 #include "qapi/qapi-commands-migration.h"
 #include "qapi/qapi-commands-misc.h"
 #include "qapi/qapi-commands-net.h"
-#include "qapi/qapi-commands-qdev.h"
 #include "qapi/qapi-commands-rocker.h"
 #include "qapi/qapi-commands-run-state.h"
 #include "qapi/qapi-commands-tpm.h"
@@ -2181,23 +2179,6 @@ void hmp_migrate(Monitor *mon, const QDict *qdict)
 }
 }
 
-void hmp_device_add(Monitor *mon, const QDict *qdict)
-{
-Error *err = NULL;
-
-qmp_device_add((QDict *)qdict, NULL, );
-hmp_handle_error(mon, );
-}
-
-void hmp_device_del(Monitor *mon, const QDict *qdict)
-{
-const char *id = qdict_get_str(qdict, "id");
-Error *err = NULL;
-
-qmp_device_del(id, );
-hmp_handle_error(mon, );
-}
-
 void hmp_netdev_add(Monitor *mon, const QDict *qdict)
 {
 Error *err = NULL;
diff --git a/qdev-monitor.c b/qdev-monitor.c
index 58222c2211..cd6299e92b 100644
--- a/qdev-monitor.c
+++ b/qdev-monitor.c
@@ -20,6 +20,7 @@
 #include "qemu/osdep.h"
 #include "hw/qdev.h"
 #include "hw/sysbus.h"
+#include "monitor/hmp.h"
 #include "monitor/monitor.h"
 #include "monitor/qdev.h"
 #include "sysemu/arch_init.h"
@@ -844,6 +845,23 @@ void qmp_device_del(const char *id, Error **errp)
 }
 }
 
+void hmp_device_add(Monitor *mon, const QDict *qdict)
+{
+Error *err = NULL;
+
+qmp_device_add((QDict *)qdict, NULL, );
+hmp_handle_error(mon, );
+}
+
+void hmp_device_del(Monitor *mon, const QDict *qdict)
+{
+const char *id = qdict_get_str(qdict, "id");
+Error *err = NULL;
+
+qmp_device_del(id, );
+hmp_handle_error(mon, );
+}
+
 BlockBackend *blk_by_qdev_id(const char *id, Error **errp)
 {
 DeviceState *dev;
-- 
2.21.0

[Qemu-devel] [PATCH 04/11] audio: audiodev= parameters no longer optional when -audiodev present

This means you should probably stop using -soundhw (as it doesn't allow
you to specify any options) and add the device manually with -device.
The exception is pcspk, it's currently not possible to manually add it.
To use it with audiodev, use something like this:

-audiodev id=foo,... -global isa-pcspk.audiodev=foo -soundhw pcspk

Signed-off-by: Kővágó, Zoltán 
---
 audio/audio.c   |  24 ++--
 audio/paaudio.c | 329 +++-
 2 files changed, 203 insertions(+), 150 deletions(-)

diff --git a/audio/audio.c b/audio/audio.c
index e9dd7c8b32..82dd0e3e13 100644
--- a/audio/audio.c
+++ b/audio/audio.c
@@ -101,6 +101,8 @@ const struct mixeng_volume nominal_volume = {
 #endif
 };
 
+static bool legacy_config = true;
+
 #ifdef AUDIO_IS_FLAWLESS_AND_NO_CHECKS_ARE_REQURIED
 #error No its not
 #else
@@ -1392,7 +1394,7 @@ static AudiodevListEntry *audiodev_find(
  * if dev == NULL => legacy implicit initialization, return the already created
  *   state or create a new one
  */
-static AudioState *audio_init(Audiodev *dev)
+static AudioState *audio_init(Audiodev *dev, const char *name)
 {
 static bool atexit_registered;
 size_t i;
@@ -1406,12 +1408,13 @@ static AudioState *audio_init(Audiodev *dev)
 
 if (dev) {
 /* -audiodev option */
+legacy_config = false;
 drvname = AudiodevDriver_str(dev->driver);
 } else if (!QTAILQ_EMPTY(_states)) {
-/*
- * todo: check for -audiodev once we have normal audiodev selection
- * support
- */
+if (!legacy_config) {
+dolog("You must specify an audiodev= for the device %s\n", name);
+exit(1);
+}
 return QTAILQ_FIRST(_states);
 } else {
 /* legacy implicit initialization */
@@ -1518,7 +1521,7 @@ void audio_free_audiodev_list(AudiodevListHead *head)
 void AUD_register_card (const char *name, QEMUSoundCard *card)
 {
 if (!card->state) {
-card->state = audio_init(NULL);
+card->state = audio_init(NULL, name);
 }
 
 card->name = g_strdup (name);
@@ -1544,8 +1547,11 @@ CaptureVoiceOut *AUD_add_capture(
 struct capture_callback *cb;
 
 if (!s) {
-/* todo: remove when we have normal audiodev selection support */
-s = audio_init(NULL);
+if (!legacy_config) {
+dolog("You must specify audiodev when trying to capture\n");
+return NULL;
+}
+s = audio_init(NULL, NULL);
 }
 
 if (audio_validate_settings (as)) {
@@ -1776,7 +1782,7 @@ void audio_init_audiodevs(void)
 AudiodevListEntry *e;
 
 QSIMPLEQ_FOREACH(e, , next) {
-audio_init(e->dev);
+audio_init(e->dev, NULL);
 }
 }
 
diff --git a/audio/paaudio.c b/audio/paaudio.c
index 5fc886bb33..cc3a34c2ea 100644
--- a/audio/paaudio.c
+++ b/audio/paaudio.c
@@ -11,10 +11,21 @@
 #include "audio_int.h"
 #include "audio_pt_int.h"
 
-typedef struct {
-Audiodev *dev;
+typedef struct PAConnection {
+char *server;
+int refcount;
+QTAILQ_ENTRY(PAConnection) list;
+
 pa_threaded_mainloop *mainloop;
 pa_context *context;
+} PAConnection;
+
+static QTAILQ_HEAD(PAConnectionHead, PAConnection) pa_conns =
+QTAILQ_HEAD_INITIALIZER(pa_conns);
+
+typedef struct {
+Audiodev *dev;
+PAConnection *conn;
 } paaudio;
 
 typedef struct {
@@ -45,7 +56,7 @@ typedef struct {
 int samples;
 } PAVoiceIn;
 
-static void qpa_audio_fini(void *opaque);
+static void qpa_conn_fini(PAConnection *c);
 
 static void GCC_FMT_ATTR (2, 3) qpa_logerr (int err, const char *fmt, ...)
 {
@@ -108,11 +119,11 @@ static inline int PA_STREAM_IS_GOOD(pa_stream_state_t x)
 
 static int qpa_simple_read (PAVoiceIn *p, void *data, size_t length, int 
*rerror)
 {
-paaudio *g = p->g;
+PAConnection *c = p->g->conn;
 
-pa_threaded_mainloop_lock (g->mainloop);
+pa_threaded_mainloop_lock(c->mainloop);
 
-CHECK_DEAD_GOTO (g, p->stream, rerror, unlock_and_fail);
+CHECK_DEAD_GOTO(c, p->stream, rerror, unlock_and_fail);
 
 while (length > 0) {
 size_t l;
@@ -121,11 +132,11 @@ static int qpa_simple_read (PAVoiceIn *p, void *data, 
size_t length, int *rerror
 int r;
 
 r = pa_stream_peek (p->stream, >read_data, >read_length);
-CHECK_SUCCESS_GOTO (g, rerror, r == 0, unlock_and_fail);
+CHECK_SUCCESS_GOTO(c, rerror, r == 0, unlock_and_fail);
 
 if (!p->read_data) {
-pa_threaded_mainloop_wait (g->mainloop);
-CHECK_DEAD_GOTO (g, p->stream, rerror, unlock_and_fail);
+pa_threaded_mainloop_wait(c->mainloop);
+CHECK_DEAD_GOTO(c, p->stream, rerror, unlock_and_fail);
 } else {
 p->read_index = 0;
 }
@@ -148,53 +159,53 @@ static int qpa_simple_read (PAVoiceIn *p, void *data, 
size_t length, int *rerror
 p->read_length = 0;
 p->read_index = 0;
 
-

[Qemu-devel] [PATCH 03/11] audio: add audiodev properties to frontends

Finally add audiodev= options to audio frontends so users can specify
which backend to use when multiple backends exist.  Not specifying an
audiodev= option currently causes the first audiodev to be used, this is
fixed in the next commit.

Example usage: -audiodev pa,id=foo -device AC97,audiodev=foo

Signed-off-by: Kővágó, Zoltán 
---
 audio/audio.h|  3 ++
 include/hw/qdev-properties.h |  3 ++
 hw/audio/ac97.c  |  1 +
 hw/audio/adlib.c |  1 +
 hw/audio/cs4231a.c   |  1 +
 hw/audio/es1370.c|  7 +++-
 hw/audio/gus.c   |  1 +
 hw/audio/hda-codec.c |  1 +
 hw/audio/milkymist-ac97.c|  6 
 hw/audio/pcspk.c |  1 +
 hw/audio/pl041.c |  1 +
 hw/audio/sb16.c  |  1 +
 hw/audio/wm8750.c|  6 
 hw/core/qdev-properties-system.c | 57 
 hw/usb/dev-audio.c   |  1 +
 15 files changed, 90 insertions(+), 1 deletion(-)

diff --git a/audio/audio.h b/audio/audio.h
index ad2457f4de..c0722a5cda 100644
--- a/audio/audio.h
+++ b/audio/audio.h
@@ -181,4 +181,7 @@ void audio_legacy_help(void);
 AudioState *audio_state_by_name(const char *name);
 const char *audio_get_id(QEMUSoundCard *card);
 
+#define DEFINE_AUDIO_PROPERTIES(_s, _f) \
+DEFINE_PROP_AUDIODEV("audiodev", _s, _f)
+
 #endif /* QEMU_AUDIO_H */
diff --git a/include/hw/qdev-properties.h b/include/hw/qdev-properties.h
index 1eae5ab056..de0b2c8423 100644
--- a/include/hw/qdev-properties.h
+++ b/include/hw/qdev-properties.h
@@ -35,6 +35,7 @@ extern const PropertyInfo qdev_prop_blocksize;
 extern const PropertyInfo qdev_prop_pci_host_devaddr;
 extern const PropertyInfo qdev_prop_uuid;
 extern const PropertyInfo qdev_prop_arraylen;
+extern const PropertyInfo qdev_prop_audiodev;
 extern const PropertyInfo qdev_prop_link;
 extern const PropertyInfo qdev_prop_off_auto_pcibar;
 extern const PropertyInfo qdev_prop_pcie_link_speed;
@@ -236,6 +237,8 @@ extern const PropertyInfo qdev_prop_pcie_link_width;
 + type_check(QemuUUID, typeof_field(_state, _field)),  \
 .set_default = true,   \
 }
+#define DEFINE_PROP_AUDIODEV(_n, _s, _f) \
+DEFINE_PROP(_n, _s, _f, qdev_prop_audiodev, QEMUSoundCard)
 
 #define DEFINE_PROP_END_OF_LIST()   \
 {}
diff --git a/hw/audio/ac97.c b/hw/audio/ac97.c
index fb98da2678..0d8e524233 100644
--- a/hw/audio/ac97.c
+++ b/hw/audio/ac97.c
@@ -1409,6 +1409,7 @@ static int ac97_init (PCIBus *bus)
 }
 
 static Property ac97_properties[] = {
+DEFINE_AUDIO_PROPERTIES(AC97LinkState, card),
 DEFINE_PROP_UINT32 ("use_broken_id", AC97LinkState, use_broken_id, 0),
 DEFINE_PROP_END_OF_LIST (),
 };
diff --git a/hw/audio/adlib.c b/hw/audio/adlib.c
index 7dd9a89b89..df2e781788 100644
--- a/hw/audio/adlib.c
+++ b/hw/audio/adlib.c
@@ -299,6 +299,7 @@ static void adlib_realizefn (DeviceState *dev, Error **errp)
 }
 
 static Property adlib_properties[] = {
+DEFINE_AUDIO_PROPERTIES(AdlibState, card),
 DEFINE_PROP_UINT32 ("iobase",  AdlibState, port, 0x220),
 DEFINE_PROP_UINT32 ("freq",AdlibState, freq,  44100),
 DEFINE_PROP_END_OF_LIST (),
diff --git a/hw/audio/cs4231a.c b/hw/audio/cs4231a.c
index 7216b41cc1..e3ea830b47 100644
--- a/hw/audio/cs4231a.c
+++ b/hw/audio/cs4231a.c
@@ -689,6 +689,7 @@ static int cs4231a_init (ISABus *bus)
 }
 
 static Property cs4231a_properties[] = {
+DEFINE_AUDIO_PROPERTIES(CSState, card),
 DEFINE_PROP_UINT32 ("iobase",  CSState, port, 0x534),
 DEFINE_PROP_UINT32 ("irq", CSState, irq,  9),
 DEFINE_PROP_UINT32 ("dma", CSState, dma,  3),
diff --git a/hw/audio/es1370.c b/hw/audio/es1370.c
index 260c142b70..7589671d20 100644
--- a/hw/audio/es1370.c
+++ b/hw/audio/es1370.c
@@ -887,6 +887,11 @@ static int es1370_init (PCIBus *bus)
 return 0;
 }
 
+static Property es1370_properties[] = {
+DEFINE_AUDIO_PROPERTIES(ES1370State, card),
+DEFINE_PROP_END_OF_LIST(),
+};
+
 static void es1370_class_init (ObjectClass *klass, void *data)
 {
 DeviceClass *dc = DEVICE_CLASS (klass);
@@ -903,6 +908,7 @@ static void es1370_class_init (ObjectClass *klass, void 
*data)
 dc->desc = "ENSONIQ AudioPCI ES1370";
 dc->vmsd = _es1370;
 dc->reset = es1370_on_reset;
+dc->props = es1370_properties;
 }
 
 static const TypeInfo es1370_info = {
@@ -923,4 +929,3 @@ static void es1370_register_types (void)
 }
 
 type_init (es1370_register_types)
-
diff --git a/hw/audio/gus.c b/hw/audio/gus.c
index 9ab51631d9..566864bc9e 100644
--- a/hw/audio/gus.c
+++ b/hw/audio/gus.c
@@ -297,6 +297,7 @@ static int GUS_init (ISABus *bus)
 }
 
 static Property gus_properties[] = {
+DEFINE_AUDIO_PROPERTIES(GUSState, card),
 DEFINE_PROP_UINT32 ("freq",GUSState, freq,44100),
 DEFINE_PROP_UINT32 ("iobase",  GUSState, port,0x240),
 DEFINE_PROP_UINT32 ("irq",

[Qemu-devel] [PATCH] tcg/aarch64: Fix output of extract2 opcodes

2019-07-09 Thread Richard Henderson

The aarch64 argument ordering for the operands is big-endian,
whereas the tcg argument ordering is little-endian.  Use REG0
so that we honor the rZ constraints.

Fixes: 464c2969d5d
Reported-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 tcg/aarch64/tcg-target.inc.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tcg/aarch64/tcg-target.inc.c b/tcg/aarch64/tcg-target.inc.c
index b0f8106642..0713448bf5 100644
--- a/tcg/aarch64/tcg-target.inc.c
+++ b/tcg/aarch64/tcg-target.inc.c
@@ -2226,7 +2226,7 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc,
 
 case INDEX_op_extract2_i64:
 case INDEX_op_extract2_i32:
-tcg_out_extr(s, ext, a0, a1, a2, args[3]);
+tcg_out_extr(s, ext, a0, REG0(2), REG0(1), args[3]);
 break;
 
 case INDEX_op_add2_i32:
-- 
2.17.1

[Qemu-devel] [PATCH 01/11] audio: reduce glob_audio_state usage

Remove glob_audio_state from functions, where possible without breaking
the API.  This means that most static functions in audio.c now take an
AudioState pointer instead of implicitly using glob_audio_state.  Also
included a pointer in SWVoice*, HWVoice* structs, so that functions
dealing them can know the audio state without having to pass it around
separately.

This is required in order to support multiple simultaneous audio
backends (added in a later commit).

Signed-off-by: Kővágó, Zoltán 
---
 audio/audio_int.h  |  8 ++
 audio/audio_template.h | 46 
 audio/audio.c  | 59 +++---
 3 files changed, 57 insertions(+), 56 deletions(-)

diff --git a/audio/audio_int.h b/audio/audio_int.h
index 3f14842709..8164696b2c 100644
--- a/audio/audio_int.h
+++ b/audio/audio_int.h
@@ -49,9 +49,11 @@ struct audio_pcm_info {
 int swap_endianness;
 };
 
+typedef struct AudioState AudioState;
 typedef struct SWVoiceCap SWVoiceCap;
 
 typedef struct HWVoiceOut {
+AudioState *s;
 int enabled;
 int poll_mode;
 int pending_disable;
@@ -73,6 +75,7 @@ typedef struct HWVoiceOut {
 } HWVoiceOut;
 
 typedef struct HWVoiceIn {
+AudioState *s;
 int enabled;
 int poll_mode;
 struct audio_pcm_info info;
@@ -94,6 +97,7 @@ typedef struct HWVoiceIn {
 
 struct SWVoiceOut {
 QEMUSoundCard *card;
+AudioState *s;
 struct audio_pcm_info info;
 t_sample *conv;
 int64_t ratio;
@@ -111,6 +115,7 @@ struct SWVoiceOut {
 
 struct SWVoiceIn {
 QEMUSoundCard *card;
+AudioState *s;
 int active;
 struct audio_pcm_info info;
 int64_t ratio;
@@ -188,6 +193,9 @@ typedef struct AudioState {
 int nb_hw_voices_in;
 int vm_running;
 int64_t period_ticks;
+
+bool timer_running;
+uint64_t timer_last;
 } AudioState;
 
 extern const struct mixeng_volume nominal_volume;
diff --git a/audio/audio_template.h b/audio/audio_template.h
index 1232bb54db..c721fed75d 100644
--- a/audio/audio_template.h
+++ b/audio/audio_template.h
@@ -36,9 +36,9 @@
 #define HWBUF hw->conv_buf
 #endif
 
-static void glue (audio_init_nb_voices_, TYPE) (struct audio_driver *drv)
+static void glue(audio_init_nb_voices_, TYPE)(AudioState *s,
+  struct audio_driver *drv)
 {
-AudioState *s = _audio_state;
 int max_voices = glue (drv->max_voices_, TYPE);
 int voice_size = glue (drv->voice_size_, TYPE);
 
@@ -183,8 +183,8 @@ static void glue (audio_pcm_hw_del_sw_, TYPE) (SW *sw)
 
 static void glue (audio_pcm_hw_gc_, TYPE) (HW **hwp)
 {
-AudioState *s = _audio_state;
 HW *hw = *hwp;
+AudioState *s = hw->s;
 
 if (!hw->sw_head.lh_first) {
 #ifdef DAC
@@ -199,15 +199,14 @@ static void glue (audio_pcm_hw_gc_, TYPE) (HW **hwp)
 }
 }
 
-static HW *glue (audio_pcm_hw_find_any_, TYPE) (HW *hw)
+static HW *glue(audio_pcm_hw_find_any_, TYPE)(AudioState *s, HW *hw)
 {
-AudioState *s = _audio_state;
 return hw ? hw->entries.le_next : glue (s->hw_head_, TYPE).lh_first;
 }
 
-static HW *glue (audio_pcm_hw_find_any_enabled_, TYPE) (HW *hw)
+static HW *glue(audio_pcm_hw_find_any_enabled_, TYPE)(AudioState *s, HW *hw)
 {
-while ((hw = glue (audio_pcm_hw_find_any_, TYPE) (hw))) {
+while ((hw = glue(audio_pcm_hw_find_any_, TYPE)(s, hw))) {
 if (hw->enabled) {
 return hw;
 }
@@ -215,12 +214,10 @@ static HW *glue (audio_pcm_hw_find_any_enabled_, TYPE) 
(HW *hw)
 return NULL;
 }
 
-static HW *glue (audio_pcm_hw_find_specific_, TYPE) (
-HW *hw,
-struct audsettings *as
-)
+static HW *glue(audio_pcm_hw_find_specific_, TYPE)(AudioState *s, HW *hw,
+   struct audsettings *as)
 {
-while ((hw = glue (audio_pcm_hw_find_any_, TYPE) (hw))) {
+while ((hw = glue(audio_pcm_hw_find_any_, TYPE)(s, hw))) {
 if (audio_pcm_info_eq (>info, as)) {
 return hw;
 }
@@ -228,10 +225,10 @@ static HW *glue (audio_pcm_hw_find_specific_, TYPE) (
 return NULL;
 }
 
-static HW *glue (audio_pcm_hw_add_new_, TYPE) (struct audsettings *as)
+static HW *glue(audio_pcm_hw_add_new_, TYPE)(AudioState *s,
+ struct audsettings *as)
 {
 HW *hw;
-AudioState *s = _audio_state;
 struct audio_driver *drv = s->drv;
 
 if (!glue (s->nb_hw_voices_, TYPE)) {
@@ -255,6 +252,7 @@ static HW *glue (audio_pcm_hw_add_new_, TYPE) (struct 
audsettings *as)
 return NULL;
 }
 
+hw->s = s;
 hw->pcm_ops = drv->pcm_ops;
 hw->ctl_caps = drv->ctl_caps;
 
@@ -328,33 +326,33 @@ AudiodevPerDirectionOptions *glue(audio_get_pdo_, 
TYPE)(Audiodev *dev)
 abort();
 }
 
-static HW *glue (audio_pcm_hw_add_, TYPE) (struct audsettings *as)
+static HW *glue(audio_pcm_hw_add_, TYPE)(AudioState *s, struct audsettings *as)
 {
 HW *hw;
-AudioState *s = _audio_state;
 AudiodevPerDirectionOptions *pdo =

[Qemu-devel] [PATCH 10/11] audio: remove read and write pcm_ops

They just called audio_pcm_sw_read/write anyway, so it makes no sense
to have them too.  (The noaudio's read is the only exception, but it
should work with the generic code too.)

Signed-off-by: Kővágó, Zoltán 
---
 audio/audio_int.h   |  5 -
 audio/alsaaudio.c   | 12 
 audio/audio.c   |  8 
 audio/coreaudio.c   |  6 --
 audio/dsoundaudio.c | 12 
 audio/noaudio.c | 19 ---
 audio/ossaudio.c| 12 
 audio/paaudio.c | 12 
 audio/sdlaudio.c|  6 --
 audio/spiceaudio.c  | 12 
 audio/wavaudio.c|  6 --
 11 files changed, 4 insertions(+), 106 deletions(-)

diff --git a/audio/audio_int.h b/audio/audio_int.h
index 7e00c1332e..003b7ab8cc 100644
--- a/audio/audio_int.h
+++ b/audio/audio_int.h
@@ -150,13 +150,11 @@ struct audio_pcm_ops {
 int  (*init_out)(HWVoiceOut *hw, struct audsettings *as, void *drv_opaque);
 void (*fini_out)(HWVoiceOut *hw);
 int  (*run_out) (HWVoiceOut *hw, int live);
-int  (*write)   (SWVoiceOut *sw, void *buf, int size);
 int  (*ctl_out) (HWVoiceOut *hw, int cmd, ...);
 
 int  (*init_in) (HWVoiceIn *hw, struct audsettings *as, void *drv_opaque);
 void (*fini_in) (HWVoiceIn *hw);
 int  (*run_in)  (HWVoiceIn *hw);
-int  (*read)(SWVoiceIn *sw, void *buf, int size);
 int  (*ctl_in)  (HWVoiceIn *hw, int cmd, ...);
 };
 
@@ -210,11 +208,8 @@ audio_driver *audio_driver_lookup(const char *name);
 void audio_pcm_init_info (struct audio_pcm_info *info, struct audsettings *as);
 void audio_pcm_info_clear_buf (struct audio_pcm_info *info, void *buf, int 
len);
 
-int  audio_pcm_sw_write (SWVoiceOut *sw, void *buf, int len);
 int  audio_pcm_hw_get_live_in (HWVoiceIn *hw);
 
-int  audio_pcm_sw_read (SWVoiceIn *sw, void *buf, int len);
-
 int audio_pcm_hw_clip_out (HWVoiceOut *hw, void *pcm_buf,
int live, int pending);
 
diff --git a/audio/alsaaudio.c b/audio/alsaaudio.c
index 3daa7c8f8f..e9e3a4819c 100644
--- a/audio/alsaaudio.c
+++ b/audio/alsaaudio.c
@@ -270,11 +270,6 @@ static int alsa_poll_in (HWVoiceIn *hw)
 return alsa_poll_helper (alsa->handle, >pollhlp, POLLIN);
 }
 
-static int alsa_write (SWVoiceOut *sw, void *buf, int len)
-{
-return audio_pcm_sw_write (sw, buf, len);
-}
-
 static snd_pcm_format_t aud_to_alsafmt (AudioFormat fmt, int endianness)
 {
 switch (fmt) {
@@ -988,11 +983,6 @@ static int alsa_run_in (HWVoiceIn *hw)
 return read_samples;
 }
 
-static int alsa_read (SWVoiceIn *sw, void *buf, int size)
-{
-return audio_pcm_sw_read (sw, buf, size);
-}
-
 static int alsa_ctl_in (HWVoiceIn *hw, int cmd, ...)
 {
 ALSAVoiceIn *alsa = (ALSAVoiceIn *) hw;
@@ -1076,13 +1066,11 @@ static struct audio_pcm_ops alsa_pcm_ops = {
 .init_out = alsa_init_out,
 .fini_out = alsa_fini_out,
 .run_out  = alsa_run_out,
-.write= alsa_write,
 .ctl_out  = alsa_ctl_out,
 
 .init_in  = alsa_init_in,
 .fini_in  = alsa_fini_in,
 .run_in   = alsa_run_in,
-.read = alsa_read,
 .ctl_in   = alsa_ctl_in,
 };
 
diff --git a/audio/audio.c b/audio/audio.c
index d73cc086b6..b79f56fe64 100644
--- a/audio/audio.c
+++ b/audio/audio.c
@@ -594,7 +594,7 @@ static int audio_pcm_sw_get_rpos_in (SWVoiceIn *sw)
 }
 }
 
-int audio_pcm_sw_read (SWVoiceIn *sw, void *buf, int size)
+static int audio_pcm_sw_read(SWVoiceIn *sw, void *buf, int size)
 {
 HWVoiceIn *hw = sw->hw;
 int samples, live, ret = 0, swlim, isamp, osamp, rpos, total = 0;
@@ -696,7 +696,7 @@ static int audio_pcm_hw_get_live_out (HWVoiceOut *hw, int 
*nb_live)
 /*
  * Soft voice (playback)
  */
-int audio_pcm_sw_write (SWVoiceOut *sw, void *buf, int size)
+static int audio_pcm_sw_write(SWVoiceOut *sw, void *buf, int size)
 {
 int hwsamples, samples, isamp, osamp, wpos, live, dead, left, swlim, blck;
 int ret = 0, pos = 0, total = 0;
@@ -854,7 +854,7 @@ int AUD_write (SWVoiceOut *sw, void *buf, int size)
 return 0;
 }
 
-return sw->hw->pcm_ops->write(sw, buf, size);
+return audio_pcm_sw_write(sw, buf, size);
 }
 
 int AUD_read (SWVoiceIn *sw, void *buf, int size)
@@ -869,7 +869,7 @@ int AUD_read (SWVoiceIn *sw, void *buf, int size)
 return 0;
 }
 
-return sw->hw->pcm_ops->read(sw, buf, size);
+return audio_pcm_sw_read(sw, buf, size);
 }
 
 int AUD_get_buffer_size_out (SWVoiceOut *sw)
diff --git a/audio/coreaudio.c b/audio/coreaudio.c
index f0ab4014a8..091fe84a34 100644
--- a/audio/coreaudio.c
+++ b/audio/coreaudio.c
@@ -489,11 +489,6 @@ static OSStatus audioDeviceIOProc(
 return 0;
 }
 
-static int coreaudio_write (SWVoiceOut *sw, void *buf, int len)
-{
-return audio_pcm_sw_write (sw, buf, len);
-}
-
 static int coreaudio_init_out(HWVoiceOut *hw, struct audsettings *as,
   void *drv_opaque)
 {
@@ -692,7 +687,6 @@ static struct audio_pcm_ops coreaudio_pcm_ops = {
 .init_out = coreaudio_init_out,
 .fini_out =

[Qemu-devel] [PATCH 05/11] paaudio: do not move stream when sink/source name is specified

Unless we disable stream moving, pulseaudio can easily move the stream
on connect, effectively ignoring the source/sink specified by the user.

Signed-off-by: Kővágó, Zoltán 
---
 audio/paaudio.c | 5 +
 1 file changed, 5 insertions(+)

diff --git a/audio/paaudio.c b/audio/paaudio.c
index cc3a34c2ea..24d98b344a 100644
--- a/audio/paaudio.c
+++ b/audio/paaudio.c
@@ -517,6 +517,11 @@ static pa_stream *qpa_simple_new (
 #endif
 | PA_STREAM_AUTO_TIMING_UPDATE;
 
+if (dev) {
+/* don't move the stream if the user specified a sink/source */
+flags |= PA_STREAM_DONT_MOVE;
+}
+
 if (dir == PA_STREAM_PLAYBACK) {
 r = pa_stream_connect_playback(stream, dev, attr, flags, NULL, NULL);
 } else {
-- 
2.22.0

[Qemu-devel] [PATCH 06/11] paaudio: properly disconnect streams in fini_*

Currently this needs a workaround due to bug #247 in pulseaudio.

Signed-off-by: Kővágó, Zoltán 
---
 audio/paaudio.c | 25 +++--
 1 file changed, 23 insertions(+), 2 deletions(-)

diff --git a/audio/paaudio.c b/audio/paaudio.c
index 24d98b344a..490bcd770e 100644
--- a/audio/paaudio.c
+++ b/audio/paaudio.c
@@ -685,6 +685,27 @@ static int qpa_init_in(HWVoiceIn *hw, struct audsettings 
*as, void *drv_opaque)
 return -1;
 }
 
+static void qpa_simple_disconnect(PAConnection *c, pa_stream *stream)
+{
+int err;
+
+pa_threaded_mainloop_lock(c->mainloop);
+/*
+ * wait until actually connects. workaround pa bug #247
+ * https://gitlab.freedesktop.org/pulseaudio/pulseaudio/issues/247
+ */
+while (pa_stream_get_state(stream) == PA_STREAM_CREATING) {
+pa_threaded_mainloop_wait(c->mainloop);
+}
+
+err = pa_stream_disconnect(stream);
+if (err != 0) {
+dolog("Failed to dissconnect! err=%d\n", err);
+}
+pa_stream_unref(stream);
+pa_threaded_mainloop_unlock(c->mainloop);
+}
+
 static void qpa_fini_out (HWVoiceOut *hw)
 {
 void *ret;
@@ -696,7 +717,7 @@ static void qpa_fini_out (HWVoiceOut *hw)
 audio_pt_join(>pt, , __func__);
 
 if (pa->stream) {
-pa_stream_unref (pa->stream);
+qpa_simple_disconnect(pa->g->conn, pa->stream);
 pa->stream = NULL;
 }
 
@@ -716,7 +737,7 @@ static void qpa_fini_in (HWVoiceIn *hw)
 audio_pt_join(>pt, , __func__);
 
 if (pa->stream) {
-pa_stream_unref (pa->stream);
+qpa_simple_disconnect(pa->g->conn, pa->stream);
 pa->stream = NULL;
 }
 
-- 
2.22.0

[Qemu-devel] [PATCH 08/11] audio: do not run each backend in audio_run

audio_run is called manually by alsa and oss backends when polling.
In this case only the requesting backend should be run, not all of them.

Signed-off-by: Kővágó, Zoltán 
---
 audio/audio_int.h |  2 +-
 audio/alsaaudio.c |  7 +--
 audio/audio.c | 14 +-
 audio/ossaudio.c  | 12 ++--
 4 files changed, 17 insertions(+), 18 deletions(-)

diff --git a/audio/audio_int.h b/audio/audio_int.h
index 9f01f6ad00..7e00c1332e 100644
--- a/audio/audio_int.h
+++ b/audio/audio_int.h
@@ -221,7 +221,7 @@ int audio_pcm_hw_clip_out (HWVoiceOut *hw, void *pcm_buf,
 int audio_bug (const char *funcname, int cond);
 void *audio_calloc (const char *funcname, int nmemb, size_t size);
 
-void audio_run (const char *msg);
+void audio_run(AudioState *s, const char *msg);
 
 #define VOICE_ENABLE 1
 #define VOICE_DISABLE 2
diff --git a/audio/alsaaudio.c b/audio/alsaaudio.c
index 6b9e0f06af..3daa7c8f8f 100644
--- a/audio/alsaaudio.c
+++ b/audio/alsaaudio.c
@@ -39,6 +39,7 @@ struct pollhlp {
 struct pollfd *pfds;
 int count;
 int mask;
+AudioState *s;
 };
 
 typedef struct ALSAVoiceOut {
@@ -199,11 +200,11 @@ static void alsa_poll_handler (void *opaque)
 break;
 
 case SND_PCM_STATE_PREPARED:
-audio_run ("alsa run (prepared)");
+audio_run(hlp->s, "alsa run (prepared)");
 break;
 
 case SND_PCM_STATE_RUNNING:
-audio_run ("alsa run (running)");
+audio_run(hlp->s, "alsa run (running)");
 break;
 
 default:
@@ -749,6 +750,7 @@ static int alsa_init_out(HWVoiceOut *hw, struct audsettings 
*as,
 return -1;
 }
 
+alsa->pollhlp.s = hw->s;
 alsa->handle = handle;
 alsa->dev = dev;
 return 0;
@@ -850,6 +852,7 @@ static int alsa_init_in(HWVoiceIn *hw, struct audsettings 
*as, void *drv_opaque)
 return -1;
 }
 
+alsa->pollhlp.s = hw->s;
 alsa->handle = handle;
 alsa->dev = dev;
 return 0;
diff --git a/audio/audio.c b/audio/audio.c
index 6bf30ac9b3..d73cc086b6 100644
--- a/audio/audio.c
+++ b/audio/audio.c
@@ -835,7 +835,7 @@ static void audio_timer (void *opaque)
 }
 s->timer_last = now;
 
-audio_run("timer");
+audio_run(s, "timer");
 audio_reset_timer(s);
 }
 
@@ -1237,15 +1237,11 @@ static void audio_run_capture (AudioState *s)
 }
 }
 
-void audio_run (const char *msg)
+void audio_run(AudioState *s, const char *msg)
 {
-AudioState *s;
-
-QTAILQ_FOREACH(s, _states, list) {
-audio_run_out(s);
-audio_run_in(s);
-audio_run_capture(s);
-}
+audio_run_out(s);
+audio_run_in(s);
+audio_run_capture(s);
 
 #ifdef DEBUG_POLL
 {
diff --git a/audio/ossaudio.c b/audio/ossaudio.c
index 29139ef1f5..456fba7480 100644
--- a/audio/ossaudio.c
+++ b/audio/ossaudio.c
@@ -110,28 +110,28 @@ static void oss_anal_close (int *fdp)
 
 static void oss_helper_poll_out (void *opaque)
 {
-(void) opaque;
-audio_run ("oss_poll_out");
+AudioState *s = opaque;
+audio_run(s, "oss_poll_out");
 }
 
 static void oss_helper_poll_in (void *opaque)
 {
-(void) opaque;
-audio_run ("oss_poll_in");
+AudioState *s = opaque;
+audio_run(s, "oss_poll_in");
 }
 
 static void oss_poll_out (HWVoiceOut *hw)
 {
 OSSVoiceOut *oss = (OSSVoiceOut *) hw;
 
-qemu_set_fd_handler (oss->fd, NULL, oss_helper_poll_out, NULL);
+qemu_set_fd_handler(oss->fd, NULL, oss_helper_poll_out, hw->s);
 }
 
 static void oss_poll_in (HWVoiceIn *hw)
 {
 OSSVoiceIn *oss = (OSSVoiceIn *) hw;
 
-qemu_set_fd_handler (oss->fd, oss_helper_poll_in, NULL, NULL);
+qemu_set_fd_handler(oss->fd, oss_helper_poll_in, NULL, hw->s);
 }
 
 static int oss_write (SWVoiceOut *sw, void *buf, int len)
-- 
2.22.0

[Qemu-devel] [PATCH 09/11] paaudio: fix playback glitches

Pulseaudio normally assumes that when the server wants it, the client
can generate the audio samples and send it right away.  Unfortunately
this is not the case with QEMU -- it's up to the emulated system when
does it generate the samples.  Buffering the samples and sending them
from a background thread is just a workaround, that doesn't work too
well.  Instead enable pa's compatibility support and let pa worry about
the details.

Signed-off-by: Kővágó, Zoltán 
---
 audio/paaudio.c | 6 ++
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/audio/paaudio.c b/audio/paaudio.c
index 9d46f11b0a..d320d2e453 100644
--- a/audio/paaudio.c
+++ b/audio/paaudio.c
@@ -512,10 +512,8 @@ static pa_stream *qpa_simple_new (
 
 flags =
 PA_STREAM_INTERPOLATE_TIMING
-#ifdef PA_STREAM_ADJUST_LATENCY
-| PA_STREAM_ADJUST_LATENCY
-#endif
-| PA_STREAM_AUTO_TIMING_UPDATE;
+| PA_STREAM_AUTO_TIMING_UPDATE
+| PA_STREAM_EARLY_REQUESTS;
 
 if (dev) {
 /* don't move the stream if the user specified a sink/source */
-- 
2.22.0

Re: [Qemu-devel] [PATCH 1/2] target/arm: Use extract2 for EXTR

2019-07-09 Thread Richard Henderson

On 7/9/19 6:40 PM, Peter Maydell wrote:
> On Tue, 14 May 2019 at 02:11, Richard Henderson
>  wrote:
>>
>> This is, after all, how we implement extract2 in tcg/aarch64.
>>
>> Reviewed-by: Peter Maydell 
>> Signed-off-by: Richard Henderson 
>> ---
>>  target/arm/translate-a64.c | 38 --
>>  1 file changed, 20 insertions(+), 18 deletions(-)
> 
> It turns out that we have a regression in booting at least
> some Linux kernels with TCG on aarch64 hosts (the same
> config works fine on x86-64 hosts). Git bisect points to
> this commit (80ac954c369e7e61bd1ed0) as the cause...

Bisect would finger that one, since this second commit is the only method by
which an extract2 operation would be emitted by the aarch64 target + aarch64
host combination.

(The other place that extract2 might be used are deposits, but aarch64 host has
all of those covered with the bfi instruction.)

Fix for tcg/aarch64 coming up...


r~

[Qemu-devel] [PATCH 00/11] Multiple simultaneous audio backends

Hello,

This is my next set of audio patches. This series makes it possible to use
multiple audio backends in a single qemu instance, and contains a few random
cleanup/fix patches.

Regards,
Zoltan

Kővágó, Zoltán (11):
  audio: reduce glob_audio_state usage
  audio: basic support for multi backend audio
  audio: add audiodev properties to frontends
  audio: audiodev= parameters no longer optional when -audiodev present
  paaudio: do not move stream when sink/source name is specified
  paaudio: properly disconnect streams in fini_*
  audio: remove audio_MIN, audio_MAX
  audio: do not run each backend in audio_run
  paaudio: fix playback glitches
  audio: remove read and write pcm_ops
  audio: use size_t where makes sense

 audio/audio.h|  36 +--
 audio/audio_int.h|  43 ++--
 audio/audio_template.h   |  62 +++--
 audio/mixeng.h   |   9 +-
 audio/rate_template.h|   2 +-
 include/hw/qdev-properties.h |   3 +
 include/sysemu/replay.h  |   4 +-
 ui/vnc.h |   2 +
 audio/alsaaudio.c|  49 ++--
 audio/audio.c| 339 ++---
 audio/coreaudio.c|  18 +-
 audio/dsoundaudio.c  |  25 +-
 audio/noaudio.c  |  39 +--
 audio/ossaudio.c |  75 +++---
 audio/paaudio.c  | 421 ++-
 audio/sdlaudio.c |  30 +--
 audio/spiceaudio.c   |  34 +--
 audio/wavaudio.c |  18 +-
 audio/wavcapture.c   |   6 +-
 hw/audio/ac97.c  |  11 +-
 hw/audio/adlib.c |   5 +-
 hw/audio/cs4231a.c   |   5 +-
 hw/audio/es1370.c|  13 +-
 hw/audio/gus.c   |   7 +-
 hw/audio/hda-codec.c |  17 +-
 hw/audio/milkymist-ac97.c|  14 +-
 hw/audio/pcspk.c |   3 +-
 hw/audio/pl041.c |   1 +
 hw/audio/sb16.c  |   3 +-
 hw/audio/wm8750.c|  10 +-
 hw/core/qdev-properties-system.c |  57 +
 hw/usb/dev-audio.c   |   1 +
 monitor/misc.c   |  12 +-
 replay/replay-audio.c|  16 +-
 replay/replay.c  |   2 +-
 ui/vnc.c |  15 +-
 hmp-commands.hx  |  11 +-
 qemu-options.hx  |   5 +
 38 files changed, 781 insertions(+), 642 deletions(-)

-- 
2.22.0

Re: [Qemu-devel] [PATCH v3 06/18] block/backup: add 'never' policy to bitmap sync mode

On 05.07.19 22:16, John Snow wrote:
> This adds a "never" policy for bitmap synchronization. Regardless of if
> the job succeeds or fails, we never update the bitmap. This can be used
> to perform differential backups, or simply to avoid the job modifying a
> bitmap.
> 
> Signed-off-by: John Snow 
> ---
>  block/backup.c   | 7 +--
>  qapi/block-core.json | 5 -
>  2 files changed, 9 insertions(+), 3 deletions(-)

Reviewed-by: Max Reitz 



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v3 04/18] qapi: add BitmapSyncMode enum

On 05.07.19 22:16, John Snow wrote:
> Depending on what a user is trying to accomplish, there might be a few
> bitmap cleanup actions that occur when an operation is finished that
> could be useful.
> 
> I am proposing three:
> - NEVER: The bitmap is never synchronized against what was copied.
> - ALWAYS: The bitmap is always synchronized, even on failures.
> - ON-SUCCESS: The bitmap is synchronized only on success.
> 
> The existing incremental backup modes use 'on-success' semantics,
> so add just that one for right now.
> 
> Signed-off-by: John Snow 
> ---
>  qapi/block-core.json | 14 ++
>  1 file changed, 14 insertions(+)

Reviewed-by: Max Reitz 



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v3 03/18] blockdev-backup: utilize do_backup_common

On 05.07.19 22:16, John Snow wrote:
> Signed-off-by: John Snow 
> ---
>  blockdev.c | 65 +-
>  1 file changed, 6 insertions(+), 59 deletions(-)

Reviewed-by: Max Reitz 



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v3 02/18] drive-backup: create do_backup_common

On 05.07.19 22:16, John Snow wrote:
> Create a common core that comprises the actual meat of what the backup API
> boundary needs to do, and then switch drive-backup to use it.
> 
> Signed-off-by: John Snow 
> ---
>  blockdev.c | 122 +
>  1 file changed, 67 insertions(+), 55 deletions(-)

Reviewed-by: Max Reitz 



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH for-4.1] pl031: Correctly migrate state when using -rtc clock=host

Patchew URL: 
https://patchew.org/QEMU/20190709143912.28905-1-peter.mayd...@linaro.org/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Message-id: 20190709143912.28905-1-peter.mayd...@linaro.org
Subject: [Qemu-devel] [PATCH for-4.1] pl031: Correctly migrate state when using 
-rtc clock=host
Type: series

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

Switched to a new branch 'test'
3c76578 pl031: Correctly migrate state when using -rtc clock=host

=== OUTPUT BEGIN ===
ERROR: spaces required around that '*' (ctx:VxV)
#156: FILE: hw/timer/pl031.c:300:
+.subsections = (const VMStateDescription*[]) {
 ^

total: 1 errors, 0 warnings, 146 lines checked

Commit 3c76578384e3 (pl031: Correctly migrate state when using -rtc clock=host) 
has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
=== OUTPUT END ===

Test command exited with code: 1


The full log is available at
http://patchew.org/logs/20190709143912.28905-1-peter.mayd...@linaro.org/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-de...@redhat.com

Re: [Qemu-devel] [PATCH 3/4] target/i386: kvm: Save nested-state only in case vCPU have set VMXON region


On 7/5/2019 2:06 PM, Liran Alon wrote:

Having (nested_state->hdr.vmx.vmxon_pa != -1ull) signals that vCPU have set
at some point in time a VMXON region. Note that even though when vCPU enters
SMM mode it temporarily exit VMX operation, KVM still reports (vmxon_pa != 
-1ull).
Therefore, this field can be used as a reliable indicator on when we require to


s/on when we require/for when we are required/

Also, note that you have commit message lines are longer than 76 
characters (longer than 80, for that matter).


But aside from those nits:

Reviewed-by: Maran Wilson 

Thanks,
-Maran


send VMX nested-state as part of migration stream.

Reviewed-by: Joao Martins 
Signed-off-by: Liran Alon 
---
  target/i386/machine.c | 5 ++---
  1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/target/i386/machine.c b/target/i386/machine.c
index 851b249d1a39..20bda9f80154 100644
--- a/target/i386/machine.c
+++ b/target/i386/machine.c
@@ -997,9 +997,8 @@ static bool vmx_nested_state_needed(void *opaque)
  {
  struct kvm_nested_state *nested_state = opaque;
  
-return ((nested_state->format == KVM_STATE_NESTED_FORMAT_VMX) &&

-((nested_state->hdr.vmx.vmxon_pa != -1ull) ||
- (nested_state->hdr.vmx.smm.flags & KVM_STATE_NESTED_SMM_VMXON)));
+return (nested_state->format == KVM_STATE_NESTED_FORMAT_VMX) &&
+   (nested_state->hdr.vmx.vmxon_pa != -1ull);
  }
  
  static const VMStateDescription vmstate_vmx_nested_state = {

Re: [Qemu-devel] [PATCH 2/4] target/i386: kvm: Init nested-state for vCPU exposed with SVM


On 7/5/2019 2:06 PM, Liran Alon wrote:

Reviewed-by: Joao Martins 
Signed-off-by: Liran Alon 
---
  target/i386/cpu.h | 5 +
  target/i386/kvm.c | 2 ++
  2 files changed, 7 insertions(+)

diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 93345792f4cb..cdb0e43676a9 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -1867,6 +1867,11 @@ static inline bool cpu_has_vmx(CPUX86State *env)
  return env->features[FEAT_1_ECX] & CPUID_EXT_VMX;
  }
  
+static inline bool cpu_has_svm(CPUX86State *env)

+{
+return env->features[FEAT_8000_0001_ECX] & CPUID_EXT3_SVM;
+}
+
  /* fpu_helper.c */
  void update_fp_status(CPUX86State *env);
  void update_mxcsr_status(CPUX86State *env);
diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index b57f873ec9e8..4e2c8652168f 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -1721,6 +1721,8 @@ int kvm_arch_init_vcpu(CPUState *cs)
  env->nested_state->format = KVM_STATE_NESTED_FORMAT_VMX;
  vmx_hdr->vmxon_pa = -1ull;
  vmx_hdr->vmcs12_pa = -1ull;
+} else if (cpu_has_svm(env)) {
+env->nested_state->format = KVM_STATE_NESTED_FORMAT_SVM;
  }
  }
  


Reviewed-by: Maran Wilson 

Thanks,
-Maran

Re: [Qemu-devel] [PATCH 1/4] target/i386: kvm: Init nested-state for VMX when vCPU expose VMX


On 7/5/2019 2:06 PM, Liran Alon wrote:

Reviewed-by: Joao Martins 
Signed-off-by: Liran Alon 
---
  target/i386/kvm.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index e4b4f5756a34..b57f873ec9e8 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -1714,7 +1714,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
  
  env->nested_state->size = max_nested_state_len;
  
-if (IS_INTEL_CPU(env)) {

+if (cpu_has_vmx(env)) {
  struct kvm_vmx_nested_state_hdr *vmx_hdr =
  >nested_state->hdr.vmx;
  


Reviewed-by: Maran Wilson 

Thanks,
-Maran

Re: [Qemu-devel] [PATCH 4/4] target/i386: kvm: Demand nested migration kernel capabilities only when vCPU may have enabled VMX