Re: [Qemu-devel] [PATCH 6/6 v3] linux-user/syscall.c: Switch all macros which are not defined in tilegx

[Chen Gang S]

On 2/21/15 11:55, Chris Metcalf wrote:
> On 2/19/2015 7:07 PM, Chen Gang S wrote:
>> +#ifdef TARGET_NR_chown /* not on tilegx */
> 
> I would omit all these comments.  The same will be true for arm64, 
> microblaze, or any other kernel architecture submitted after the tile 
> architecture went in.
> 

OK, if no any additional reply within 2 days, I shall send patch v4.

Thanks.
-- 
Chen Gang

Open, share, and attitude like air, water, and life which God blessed

Re: [Qemu-devel] [PATCH] target-tilegx: Finish decoding the first TB block.

[Chen Gang S]

On 2/21/15 11:05, Chen Gang S wrote:
> At present finish decoding the first TB block: _start. Always let y1 and
> x1 pipe at the last which may content jump instructions.
> 
> The related disassembly code is below which is the same as the objdump:
> 
>   y0: 500bfdb4  move r52, r54
>   y2: 02080760  ld r1, r54
>   y1: 1c064000  fnop
>   x0: 51483000  fnop
>   x1: 180f86c6  addi r12, r54, -16
>   x0: 403f8336  andi r54, r12, -8
>   x1: 286af0068000  lnk r13
>   y0: 500bf005  move r5, r0
>   y2: 03f80760  st r54, r63
>   y1: 040046c6  addi r12, r54, 8
>   y0: 500bfff7  move r55, r63
>   y2: 03f804c0  st r12, r63
>   y1: 04004681  addi r2, r52, 8
>   x0: 40110d86  addi r6, r54, 16
>   x1: 07e0  moveli r0, -1
>   x0: 7000  shl16insli r0, r0, -1
>   x1: 07e18000  moveli r3, 0
>   x0: 7ffa8000  shl16insli r0, r0, -88
>   x1: 38618000  shl16insli r3, r3, 0
>   x0: 500cd000  and r0, r0, r13
>   x1: 387740618000  shl16insli r3, r3, 3816
>   x0: 1fcc  moveli r12, 0
>   x1: 280668618000  add r3, r3, r13
>   x0: 730c  shl16insli r12, r12, 0
>   x1: 07e2  moveli r4, 0
>   x0: 7039030c  shl16insli r12, r12, 912
>   x1: 3882  shl16insli r4, r4, 0
>   x0: 500cd30c  and r12, r12, r13
>   x1: 38818082  shl16insli r4, r4, 4144
>   x0: 500cd104  and r4, r4, r13
>   x1: 286a7180  jr r12
>

Oh, sorry, it has no 'and' operation, it should be 'add' (3 places need
be modified). It is a printing typo issue.

[...] 
> +case 0x5000ULL:
> +switch (TILEGX_CODE_X0_18(bundle)) {
> +/* add Dest, SrcA, SrcB */
> +case 0x000cULL:
> +rdest = (unsigned char)(bundle & TILEGX_DATA_REGISTER);
> +rsrc = (unsigned char)((bundle >> 6) & TILEGX_DATA_REGISTER);
> +rsrcb = (unsigned char)((bundle >> 12) & TILEGX_DATA_REGISTER);
> +qemu_log("and r%d, r%d, r%d", rdest, rsrc, rsrcb);

It should be print "add", not "and".

Thanks.
-- 
Chen Gang

Open, share, and attitude like air, water, and life which God blessed

Re: [Qemu-devel] [PATCH 6/6 v3] linux-user/syscall.c: Switch all macros which are not defined in tilegx

[Chris Metcalf]

On 2/19/2015 7:07 PM, Chen Gang S wrote:

+#ifdef TARGET_NR_chown /* not on tilegx */


I would omit all these comments.  The same will be true for arm64, microblaze, 
or any other kernel architecture submitted after the tile architecture went in.

--
Chris Metcalf, EZChip Semiconductor
http://www.ezchip.com

Re: [Qemu-devel] unable to set SATA serial with a spaces

[Alan Latteri]

 The left padding is important and necessary to keep for my particular 
application.  This is broken in libvirt, but works fine with direct Qemu 
invocation.  

Thank you for the help. 
Alan

> On Feb 20, 2015, at 1:28 AM, Markus Armbruster  wrote:
> 
> Alan Latteri mailto:a...@instinctual.tv>> writes:
> 
>> Yes, left pad.  This is a strace of the authorization program via VirtualBox
>> data[96]=["\0\200\0\24ABCD1234\0\0\0\0\0\0\0\0"…]
>> 
>> then the same thing using the serial setting no spaces in QEMU.
>> 
>> data[96]=["\0\200\0\024ABCD1234\0\0\0\0\0\0\0\0"…]
>> 
>> The spaces matter here.
>> 
>> 
>> I tried setting the spaces manually running QEMU from the command
>> line, but that doesn’t work.
>> 
>> [root@X ~]# /usr/libexec/qemu-kvm -name test1 -S -machine
>> rhel6.0.0,accel=kvm,usb=off -m 1024 -realtime mlock=off -smp
>> 1,sockets=1,cores=1,threads=1 -uuid
>> 372419e1-ca68-408f-b809-04ce54450e60 -no-user-config -nodefaults
>> -chardev
>> socket,id=charmonitor,path=/var/lib/libvirt/qemu/test1.monitor,server,nowait
>> -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc
>> -no-shutdown -boot strict=on -device
>> piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device
>> ahci,id=ahci0,bus=pci.0,addr=0xd -drive
>> file=/var/lib/libvirt/images/372419e1-ca68-408f-b809-04ce54450e60-0.img,if=none,id=drive-sata0-0-0,format=qcow2,serial=
>> ABCD1234,cache=none -device
>> ide-hd,bus=ahci0.0,drive=drive-sata0-0-0,id=sata0-0-0,bootindex=1
>> 
>> qemu-kvm: -drive
>> file=/var/lib/libvirt/images/372419e1-ca68-408f-b809-04ce54450e60-0.img,if=none,id=drive-sata0-0-0,format=qcow2,serial=:
>> could not open disk image ABCD1234,cache=none: Could not open file: No
>> such file or directory
>> 
>> 
>> tried with quotes around the serial value and also using escpare characters.
>> 
>> No go.
> 
> Quoting appears to work for me:
> 
> $ echo -e 'info qtree\nq' | qemu-system-x86_64 -nodefaults -S -display none 
> -monitor stdio -drive if=none,id=drv0,file=tmp.qcow2,serial='
> ABCD1234' -device ide-hd,drive=drv0 | grep serial
>serial = "ABCD1234"
> 
> Aside: the above uses legacy syntax.  The preferred place for serial is
> -device, like this:
> 
>-drive if=none,id=drv0,file=tmp.qcow2
>-device ide-hd,drive=drv0,serial='ABCD1234'

[Qemu-devel] [PATCH] target-tilegx: Finish decoding the first TB block.

[Chen Gang S]

At present finish decoding the first TB block: _start. Always let y1 and
x1 pipe at the last which may content jump instructions.

The related disassembly code is below which is the same as the objdump:

  y0: 500bfdb4  move r52, r54
  y2: 02080760  ld r1, r54
  y1: 1c064000  fnop
  x0: 51483000  fnop
  x1: 180f86c6  addi r12, r54, -16
  x0: 403f8336  andi r54, r12, -8
  x1: 286af0068000  lnk r13
  y0: 500bf005  move r5, r0
  y2: 03f80760  st r54, r63
  y1: 040046c6  addi r12, r54, 8
  y0: 500bfff7  move r55, r63
  y2: 03f804c0  st r12, r63
  y1: 04004681  addi r2, r52, 8
  x0: 40110d86  addi r6, r54, 16
  x1: 07e0  moveli r0, -1
  x0: 7000  shl16insli r0, r0, -1
  x1: 07e18000  moveli r3, 0
  x0: 7ffa8000  shl16insli r0, r0, -88
  x1: 38618000  shl16insli r3, r3, 0
  x0: 500cd000  and r0, r0, r13
  x1: 387740618000  shl16insli r3, r3, 3816
  x0: 1fcc  moveli r12, 0
  x1: 280668618000  add r3, r3, r13
  x0: 730c  shl16insli r12, r12, 0
  x1: 07e2  moveli r4, 0
  x0: 7039030c  shl16insli r12, r12, 912
  x1: 3882  shl16insli r4, r4, 0
  x0: 500cd30c  and r12, r12, r13
  x1: 38818082  shl16insli r4, r4, 4144
  x0: 500cd104  and r4, r4, r13
  x1: 286a7180  jr r12

Signed-off-by: Chen Gang 
---
 target-tilegx/cpu-qom.h   |   2 +
 target-tilegx/cpu.c   |   4 -
 target-tilegx/cpu.h   |   1 +
 target-tilegx/translate.c | 437 +-
 4 files changed, 433 insertions(+), 11 deletions(-)

diff --git a/target-tilegx/cpu-qom.h b/target-tilegx/cpu-qom.h
index e15a8b8..866a77d 100644
--- a/target-tilegx/cpu-qom.h
+++ b/target-tilegx/cpu-qom.h
@@ -69,4 +69,6 @@ static inline TilegxCPU *tilegx_env_get_cpu(CPUTLState *env)
 
 #define ENV_GET_CPU(e) CPU(tilegx_env_get_cpu(e))
 
+#define ENV_OFFSET offsetof(TilegxCPU, env)
+
 #endif
diff --git a/target-tilegx/cpu.c b/target-tilegx/cpu.c
index 3dd66b5..a10cc24 100644
--- a/target-tilegx/cpu.c
+++ b/target-tilegx/cpu.c
@@ -69,10 +69,6 @@ static void tilegx_cpu_realizefn(DeviceState *dev, Error 
**errp)
 mcc->parent_realize(dev, errp);
 }
 
-static void tilegx_tcg_init(void)
-{
-}
-
 static void tilegx_cpu_initfn(Object *obj)
 {
 CPUState *cs = CPU(obj);
diff --git a/target-tilegx/cpu.h b/target-tilegx/cpu.h
index 09a2b26..439c14f 100644
--- a/target-tilegx/cpu.h
+++ b/target-tilegx/cpu.h
@@ -54,6 +54,7 @@ typedef struct CPUTLState {
 
 #include "exec/cpu-all.h"
 
+void tilegx_tcg_init(void);
 int cpu_tilegx_exec(CPUTLState *s);
 int cpu_tilegx_signal_handler(int host_signum, void *pinfo, void *puc);
 
diff --git a/target-tilegx/translate.c b/target-tilegx/translate.c
index 5131fa7..254f439 100644
--- a/target-tilegx/translate.c
+++ b/target-tilegx/translate.c
@@ -25,17 +25,440 @@
 #include "exec/cpu_ldst.h"
 #include "exec/helper-gen.h"
 
+#define TILEGX_BUNDLE_SIZE8  /* Each bundle size in bytes */
+#define TILEGX_BUNDLE_INSNS   3  /* Maximized insns per bundle */
+#define TILEGX_BUNDLE_OPCS10 /* Assume maximized opcs per bundle */
+
+/* Check Bundle whether is Y type, else is X type */
+#define TILEGX_BUNDLE_TYPE_MASK   0xc000ULL
+#define TILEGX_BUNDLE_TYPE_Y(bundle)  ((bundle) & TILEGX_BUNDLE_TYPE_MASK)
+#define TILEGX_BUNDLE_TYPE_X(bundle)  (!TILEGX_BUNDLE_TYPE_Y(bundle))
+
+/* Bundle pipe mask, still remain the bundle type */
+#define TILEGX_PIPE_X0(bundle)((bundle) & 0x7fffULL)
+#define TILEGX_PIPE_X1(bundle)((bundle) & 0x3fff8000ULL)
+#define TILEGX_PIPE_Y0(bundle)((bundle) & 0x780fULL)
+#define TILEGX_PIPE_Y1(bundle)((bundle) & 0x3c078000ULL)
+#define TILEGX_PIPE_Y2(bundle)((bundle) & 0x03f807f0ULL)
+
+/* Code mask */
+#define TILEGX_CODE_X0(bundle)((bundle) & 0x7000ULL)
+#define TILEGX_CODE_X0_18(bundle) ((bundle) & 0x0ffcULL)
+#define TILEGX_CODE_X0_20(bundle) ((bundle) & 0x0ff0ULL)
+#define TILEGX_CODE_X1(bundle)((bundle) & 0x3800ULL)
+#define TILEGX_CODE_X1_49(bundle) ((bundle) & 0x07feULL)
+#define TILEGX_CODE_X1_49_43(bundle)  ((bundle) & 0x0001f800ULL)
+#define TILEGX_CODE_X1_51(bundle) ((bundle) & 0x07f8ULL)
+#define TILEGX_CODE_Y0(bundle)((bundle) & 0x7800ULL)
+#define TILEGX_CODE_Y0_E(bundle)  ((bundle) & 0x000cULL)
+#define TILEGX_CODE_Y1(bundle)((bundle) & 0x3c00ULL)
+#define TILEGX_CODE_Y1_E(bundle)  ((bundle) & 0x0006ULL)
+#define TILEGX_CODE_Y2(bundle)((bundle) & 0x02000400ULL)
+/* No Y2_E */
+
+/* (F)Nop operation, only have effect within their own pipe */
+#define TILEGX_OPCX0_FNOP 0x51483000ULL

Re: [Qemu-devel] [PATCH v3] Makefile.target: set icon for binary file on Mac OS X

[Peter Maydell]

On 21 February 2015 at 02:25, Paolo Bonzini  wrote:
> I honestly don't know which versions are supported.  10.5 was released
> in 2007 and it is enough for Darwin/PPC, so it's probably okay.

Yes, I think 10.5 is the earliest we currently claim to care about.
(In practice I'm not sure when anybody last tried to build on 10.5.)

I have a vague recollection that somebody said they were in the
habit of cross-compiling for OSX from Linux. I guess if these
tools aren't in their cross toolchain they'll let us know :-)

-- PMM

Re: [Qemu-devel] [PATCH RFC v3 05/14] block: add meta bitmaps

[John Snow]

On 02/19/2015 06:43 AM, Vladimir Sementsov-Ogievskiy wrote:

On 19.02.2015 02:45, John Snow wrote:



On 02/18/2015 09:00 AM, Vladimir Sementsov-Ogievskiy wrote:

Meta bitmap is a 'dirty bitmap' for the BdrvDirtyBitmap. It tracks
changes (set/unset) of this BdrvDirtyBitmap. It is needed for live
migration of block dirty bitmaps.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
  block.c   | 40 
  include/block/block.h |  5 +
  2 files changed, 45 insertions(+)

diff --git a/block.c b/block.c
index a127fd2..aaa08b8 100644
--- a/block.c
+++ b/block.c
@@ -58,9 +58,15 @@
   * (3) successor is set: frozen mode.
   * A frozen bitmap cannot be renamed, deleted, anonymized,
cleared, set,
   * or enabled. A frozen bitmap can only abdicate() or reclaim().
+ *
+ * Meta bitmap:
+ * Meta bitmap is a 'dirty bitmap' for the BdrvDirtyBitmap. It
tracks changes
+ * (set/unset) of this BdrvDirtyBitmap. It is needed for live
migration of
+ * block dirty bitmaps.
   */
  struct BdrvDirtyBitmap {
  HBitmap *bitmap;/* Dirty sector bitmap
implementation */
+HBitmap *meta_bitmap;   /* Meta bitmap */
  BdrvDirtyBitmap *successor; /* Anonymous child; implies frozen
status */
  char *name; /* Optional non-empty unique ID */
  int64_t size;   /* Size of the bitmap (Number of
sectors) */
@@ -5398,6 +5404,31 @@ void
bdrv_dirty_bitmap_make_anon(BdrvDirtyBitmap *bitmap)
  bitmap->name = NULL;
  }

+HBitmap *bdrv_create_meta_bitmap(BdrvDirtyBitmap *bitmap,
+uint64_t granularity)
+{
+uint64_t sector_granularity;
+
+assert((granularity & (granularity - 1)) == 0);
+
+granularity *= 8 * bdrv_dirty_bitmap_granularity(bitmap);
+sector_granularity = granularity >> BDRV_SECTOR_BITS;
+assert(sector_granularity);
+


The maths here could use a comment, I think.

the "granularity" field here actually describes the desired
serialization buffer size; e.g. CHUNK_SIZE (1 << 20) or 1 MiB. This
parameter should be renamed to explain what it's actually for.
Something like "chunk_size" and a comment explaining that it is in bytes.

...

That said, let's talk about the default chunk size you're using in
correlation with this function.

a CHUNK_SIZE of 1MiB here is going to lead us to, if we have a bitmap
with the default granularity of 128 sectors\64KiB bytes, a granularity
for the meta_bitmap of one billion sectors (1 << 30) or 512GiB.

That's going to be bigger than most drives entirely, which will
generally lead us to only using a single "chunk" per drive. Which
means we won't really get a lot of mileage out of the bulk/dirty
phases most of the time.

It's wild to think about that the first 1,000,000,000 sectors or
512,000,000,000 bytes will all be represented by the first single bit
in this bitmap. If a single hair on the drive changes, we resend the
_entire_ bitmap, possibly over and over again. Will we ever make
progress? Should we investigate a smaller chunk size?

Here's some quick mappings of chunk size (bytes) to effective
meta_bitmap byte granularities, assuming the meta_bitmap is tracking a
bitmap with the default granularity of 64KiB:

(1 << 20) 1MiB   -- 512GiB  // This is too high of a granularity
(1 << 17) 128KiB --  64GiB
(1 << 15) 32KiB  --  16GiB
(1 << 11) 2KiB   --   1GiB
(1 << 10) 1KiB   -- 512MiB
(1 << 9)  512B   -- 256MiB
(1 << 8)  256B   -- 128MiB
(1 << 5)  32 B   --  16MiB  // This is too small of a chunk size.
(1 << 1)   1 B   --   1MiB

We want to make the chunk sends efficient, but we also want to make
sure that the dirty phase doesn't resend more data than it needs to,
so we need to strike a balance here, no?

I think arguments could be made for most granularities between 128MiB
through 1GiB. Anything outside of that is too lopsided, IMO.

What are your thoughts on this?

Ok, interesting thing to discuss.

My thoughts:
* the chunk size for block-migration is 1mb, than the bitmap (64kb
granularity) for this chunk is 16bit=2bytes long. It's an intuitive
reason for choosing the chunk size about 2 bytes. But in this case the
data/metadata ratio is very bad (about 20bytes for the header of the
chunk). So, taking the nearest value with adequate ratio gives (IMHO)
'1kb -- 512mb': 20b/1k ~ 2%. Or 512b => 4%.

* for ndb+mirror migration scheme the default chunk is 64kb instead of
1mb. So the bitmap is more smaller. But the same reason of data/metadata
ratio leads to 1kb chunk for dirty bitmap migration.

So, what about default to 1kb and additional parameter for migration
(migration capabilities) to give the user a possibility of chose?



That sounds good to me. I'm less sure about the parameter, though. In 
practice I doubt most people will be attempting a live migration of data 
sets big enough to need to alter the default. For cases with such large 
data sets, it almost certainly makes more sense to perform the migration 
via some other method

Re: [Qemu-devel] [PATCH v14 19/19] docs: incremental backup documentation

[John Snow]

On 02/20/2015 06:55 PM, Eric Blake wrote:

On 02/20/2015 04:07 PM, John Snow wrote:

Signed-off-by: John Snow 
---
  docs/bitmaps.md | 253 
  1 file changed, 253 insertions(+)
  create mode 100644 docs/bitmaps.md

diff --git a/docs/bitmaps.md b/docs/bitmaps.md
new file mode 100644
index 000..7cda146
--- /dev/null
+++ b/docs/bitmaps.md
@@ -0,0 +1,253 @@
+# Dirty Bitmaps


No copyright/license?  Might be good to add.



OK.


Also, I'm a fan of reviewing docs before code (does the design make
sense in isolation, and then did we implement it) rather than last in
the series (we may have faithfully documented our code, but that
includes baking in any design flaws that reviewers are now blind to
because of reading the code)



Sorry, I definitely did write this *after* and I selfishly put it at the 
end of the series to avoid disrupting the patch numbers from the 
previous revision.



+* To create a new bitmap that tracks changes in 32KiB segments:
+
+```json
+{ "execute": "block-dirty-bitmap-add",
+  "arguments": {
+"node": "drive0",
+"name": "bitmap0",
+"granularity": "32768"


s/"32768"/32768/ (we are using a JSON integer, not string)


+  }
+}
+```
+
+### Deletion
+
+* Can be performed on a disabled bitmap, but not a frozen one.
+
+* Because bitmaps are only unique to the node to which they are attached,
+you must specify the node/drive name here, too.
+
+```json
+{ "execute": "block-dirty-bitmap-remove",
+  "arguments": {
+"node": "drive0",
+"name": "bitmap0",
+  }


No trailing commas in JSON.


+}
+```
+
+### Enable/Disable:
+
+* Not very useful in current cases, but potentially useful for debugging in the
+future where we'd like to see what information changed only in a specific
+time period:
+
+* To enable (which is, again, the default state after add)
+
+```json
+{ "execute": "block-dirty-bitmap-enable",
+  "arguments": {
+"node": "drive0",
+"name": "bitmap0",
+  }


and again.


+}
+```
+
+* To disable:
+
+```json
+{ "execute": "block-dirty-bitmap-disable",
+  "arguments": {
+"node": "drive0",
+"name": "bitmap0",
+  }
+}
+```


and again. Also, maybe swap these two, since a bitmap defaults to
enabled (that is, you would logically use disable first, then enable to
re-enable, when testing these out, if there is no way to create an
already-disabled bitmap).


+
+### Resetting
+
+* Resetting a bitmap will clear all information it holds.
+* An incremental backup created from an empty bitmap will copy no data,
+as if nothing has changed.
+
+```json
+{ "execute": "block-dirty-bitmap-clear",
+  "arguments": {
+"node": "drive0",
+"name": "bitmap0",
+  }


I'll quit pointing out broken trailing JSON commas, on the assumption
that you'll fix all of them rather than stopping here :)



Yeah, sorry! I have discovered today that I am *awful* at writing out 
json by hand. I promise the commands work otherwise!


I will fix up the documentation here; if this winds up being the *ONLY* 
thing wrong with V14, I would prefer this doc be dropped from this 
series and I will just send a v2 for the doc by itself afterwards.



+}
+```
+
+## Transactions
+
+### Justification
+Bitmaps can be safely modified when the VM is paused or halted by using
+the basic QMP commands. For instance, you might perform the following actions:
+
+1. Boot the VM in a paused state.
+2. Create a full drive backup of drive0
+3. Create a new bitmap attached to drive0
+4. resume execution of the VM
+5. Incremental backups are ready to be created.


Consistency on using trailing '.'?


+
+At this point, the bitmap and drive backup would be correctly in sync,
+and incremental backups made from this point forward would be correctly aligned
+to the full drive backup.
+
+This is not particularly useful if we decide we want to start incremental
+backups after the VM has been running for a while, which will allow us to
+perform actions like the following:
+
+1. Boot the VM and begin execution
+2. Using transactions, perform the following operations:
+* Create 'bitmap0'
+* Create a full drive backup of drive0
+3. Incremental backups are now ready to be created.


It's unclear whether point 2 is describing two separate transactions, or
one transaction with two steps.  If it's just one, I'd word it:

Using a single transaction, perform the following operations:




OK.


+
+The star of the show.
+
+**Nota Bene!** Only incremental backups of entire drives are supported for now.
+So despite the fact that you can attach a bitmap to any arbitrary node, they 
are
+only currently useful when attached to the root node. This is because
+drive-backup only supports drives/devices instead of arbitrary nodes.


Maybe it's time to think about adding a node-backup :)  But doesn't have
to be this series.




I will crawl before I walk. And certainly before I start running and 
flying :)


Thanks,
--JS

Re: [Qemu-devel] [PATCH v14 19/19] docs: incremental backup documentation

[Eric Blake]

On 02/20/2015 04:07 PM, John Snow wrote:
> Signed-off-by: John Snow 
> ---
>  docs/bitmaps.md | 253 
> 
>  1 file changed, 253 insertions(+)
>  create mode 100644 docs/bitmaps.md
> 
> diff --git a/docs/bitmaps.md b/docs/bitmaps.md
> new file mode 100644
> index 000..7cda146
> --- /dev/null
> +++ b/docs/bitmaps.md
> @@ -0,0 +1,253 @@
> +# Dirty Bitmaps

No copyright/license?  Might be good to add.

Also, I'm a fan of reviewing docs before code (does the design make
sense in isolation, and then did we implement it) rather than last in
the series (we may have faithfully documented our code, but that
includes baking in any design flaws that reviewers are now blind to
because of reading the code)

> +* To create a new bitmap that tracks changes in 32KiB segments:
> +
> +```json
> +{ "execute": "block-dirty-bitmap-add",
> +  "arguments": {
> +"node": "drive0",
> +"name": "bitmap0",
> +"granularity": "32768"

s/"32768"/32768/ (we are using a JSON integer, not string)

> +  }
> +}
> +```
> +
> +### Deletion
> +
> +* Can be performed on a disabled bitmap, but not a frozen one.
> +
> +* Because bitmaps are only unique to the node to which they are attached,
> +you must specify the node/drive name here, too.
> +
> +```json
> +{ "execute": "block-dirty-bitmap-remove",
> +  "arguments": {
> +"node": "drive0",
> +"name": "bitmap0",
> +  }

No trailing commas in JSON.

> +}
> +```
> +
> +### Enable/Disable:
> +
> +* Not very useful in current cases, but potentially useful for debugging in 
> the
> +future where we'd like to see what information changed only in a specific
> +time period:
> +
> +* To enable (which is, again, the default state after add)
> +
> +```json
> +{ "execute": "block-dirty-bitmap-enable",
> +  "arguments": {
> +"node": "drive0",
> +"name": "bitmap0",
> +  }

and again.

> +}
> +```
> +
> +* To disable:
> +
> +```json
> +{ "execute": "block-dirty-bitmap-disable",
> +  "arguments": {
> +"node": "drive0",
> +"name": "bitmap0",
> +  }
> +}
> +```

and again. Also, maybe swap these two, since a bitmap defaults to
enabled (that is, you would logically use disable first, then enable to
re-enable, when testing these out, if there is no way to create an
already-disabled bitmap).

> +
> +### Resetting
> +
> +* Resetting a bitmap will clear all information it holds.
> +* An incremental backup created from an empty bitmap will copy no data,
> +as if nothing has changed.
> +
> +```json
> +{ "execute": "block-dirty-bitmap-clear",
> +  "arguments": {
> +"node": "drive0",
> +"name": "bitmap0",
> +  }

I'll quit pointing out broken trailing JSON commas, on the assumption
that you'll fix all of them rather than stopping here :)

> +}
> +```
> +
> +## Transactions
> +
> +### Justification
> +Bitmaps can be safely modified when the VM is paused or halted by using
> +the basic QMP commands. For instance, you might perform the following 
> actions:
> +
> +1. Boot the VM in a paused state.
> +2. Create a full drive backup of drive0
> +3. Create a new bitmap attached to drive0
> +4. resume execution of the VM
> +5. Incremental backups are ready to be created.

Consistency on using trailing '.'?

> +
> +At this point, the bitmap and drive backup would be correctly in sync,
> +and incremental backups made from this point forward would be correctly 
> aligned
> +to the full drive backup.
> +
> +This is not particularly useful if we decide we want to start incremental
> +backups after the VM has been running for a while, which will allow us to
> +perform actions like the following:
> +
> +1. Boot the VM and begin execution
> +2. Using transactions, perform the following operations:
> +* Create 'bitmap0'
> +* Create a full drive backup of drive0
> +3. Incremental backups are now ready to be created.

It's unclear whether point 2 is describing two separate transactions, or
one transaction with two steps.  If it's just one, I'd word it:

Using a single transaction, perform the following operations:


> +
> +The star of the show.
> +
> +**Nota Bene!** Only incremental backups of entire drives are supported for 
> now.
> +So despite the fact that you can attach a bitmap to any arbitrary node, they 
> are
> +only currently useful when attached to the root node. This is because
> +drive-backup only supports drives/devices instead of arbitrary nodes.

Maybe it's time to think about adding a node-backup :)  But doesn't have
to be this series.



-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

[Qemu-devel] [PATCH v14 19/19] docs: incremental backup documentation

[John Snow]

Signed-off-by: John Snow 
---
 docs/bitmaps.md | 253 
 1 file changed, 253 insertions(+)
 create mode 100644 docs/bitmaps.md

diff --git a/docs/bitmaps.md b/docs/bitmaps.md
new file mode 100644
index 000..7cda146
--- /dev/null
+++ b/docs/bitmaps.md
@@ -0,0 +1,253 @@
+# Dirty Bitmaps
+
+* Dirty bitmaps can be created at any time and attached to any node
+(not just complete drives.)
+
+## Dirty Bitmap Names
+
+* A dirty bitmap's name is unique to the node, but bitmaps attached to 
different
+nodes can share the same name.
+
+## Bitmap Modes
+
+* A Bitmap can be "enabled" (tracking writes, the default) or "disabled"
+(read-only, I/O is ignored.)
+
+* A Bitmap can be "frozen," which means that it is currently in-use by a backup
+operation and cannot be deleted, enabled, disabled, renamed, written to, reset,
+etc.
+
+## Basic QMP Usage
+
+### Supported Commands ###
+
+* block-dirty-bitmap-add
+* block-dirty-bitmap-remove
+* block-dirty-bitmap-enable
+* block-dirty-bitmap-disable
+* block-dirty-bitmap-clear
+
+### Creation
+
+* To create a new bitmap, enabled, on the drive with id=drive0:
+
+```json
+{ "execute": "block-dirty-bitmap-add",
+  "arguments": {
+"node": "drive0",
+"name": "bitmap0"
+  }
+}
+```
+
+* To create a new bitmap that tracks changes in 32KiB segments:
+
+```json
+{ "execute": "block-dirty-bitmap-add",
+  "arguments": {
+"node": "drive0",
+"name": "bitmap0",
+"granularity": "32768"
+  }
+}
+```
+
+### Deletion
+
+* Can be performed on a disabled bitmap, but not a frozen one.
+
+* Because bitmaps are only unique to the node to which they are attached,
+you must specify the node/drive name here, too.
+
+```json
+{ "execute": "block-dirty-bitmap-remove",
+  "arguments": {
+"node": "drive0",
+"name": "bitmap0",
+  }
+}
+```
+
+### Enable/Disable:
+
+* Not very useful in current cases, but potentially useful for debugging in the
+future where we'd like to see what information changed only in a specific
+time period:
+
+* To enable (which is, again, the default state after add)
+
+```json
+{ "execute": "block-dirty-bitmap-enable",
+  "arguments": {
+"node": "drive0",
+"name": "bitmap0",
+  }
+}
+```
+
+* To disable:
+
+```json
+{ "execute": "block-dirty-bitmap-disable",
+  "arguments": {
+"node": "drive0",
+"name": "bitmap0",
+  }
+}
+```
+
+### Resetting
+
+* Resetting a bitmap will clear all information it holds.
+* An incremental backup created from an empty bitmap will copy no data,
+as if nothing has changed.
+
+```json
+{ "execute": "block-dirty-bitmap-clear",
+  "arguments": {
+"node": "drive0",
+"name": "bitmap0",
+  }
+}
+```
+
+## Transactions
+
+### Justification
+Bitmaps can be safely modified when the VM is paused or halted by using
+the basic QMP commands. For instance, you might perform the following actions:
+
+1. Boot the VM in a paused state.
+2. Create a full drive backup of drive0
+3. Create a new bitmap attached to drive0
+4. resume execution of the VM
+5. Incremental backups are ready to be created.
+
+At this point, the bitmap and drive backup would be correctly in sync,
+and incremental backups made from this point forward would be correctly aligned
+to the full drive backup.
+
+This is not particularly useful if we decide we want to start incremental
+backups after the VM has been running for a while, which will allow us to
+perform actions like the following:
+
+1. Boot the VM and begin execution
+2. Using transactions, perform the following operations:
+* Create 'bitmap0'
+* Create a full drive backup of drive0
+3. Incremental backups are now ready to be created.
+
+### Supported Bitmap Transactions
+
+* block-dirty-bitmap-add
+* block-dirty-bitmap-enable
+* block-dirty-bitmap-disable
+* block-dirty-bitmap-clear
+
+The usages are identical to their respective QMP commands, but see below
+for examples.
+
+### Example: New Incremental Backup
+
+As outlined in the justification, perhaps we want to create a new incremental
+backup chain attached to a drive.
+
+```json
+{ "execute": "transaction",
+  "arguments": {
+"actions": [
+  {"type": "block-dirty-bitmap-add",
+   "data": {"node": "drive0", "name": "bitmap0"} },
+  {"type": "drive-backup",
+   "data": {"device": "drive0", "target": "/path/to/full_backup.img",
+"sync": "full", "format": "qcow2"} }
+]
+  }
+}
+```
+
+### Example: New Incremental Backup Anchor Point
+
+Maybe we just want to create a new full backup with an existing bitmap and
+want to reset the bitmap to track the new chain.
+
+```json
+{ "execute": "transaction",
+  "arguments": {
+"actions": [
+  {"type": "block-dirty-bitmap-clear",
+   "data": {"node": "drive0", "name": "bitmap0"} },
+  {"type": "drive-backup",
+   "data": {"device": "drive0", "target": "/path/to/new_full_backup.img",
+"sync": "full", "format": "qcow2"} }
+]
+  }
+}
+```
+
+## Incremental Ba

[Qemu-devel] [PATCH v14 16/19] iotests: add transactional incremental backup test

[John Snow]

Reviewed-by: Max Reitz 
Signed-off-by: John Snow 
---
 tests/qemu-iotests/112 | 49 ++
 tests/qemu-iotests/112.out |  4 ++--
 2 files changed, 51 insertions(+), 2 deletions(-)

diff --git a/tests/qemu-iotests/112 b/tests/qemu-iotests/112
index da8f0e0..59eb1b1 100644
--- a/tests/qemu-iotests/112
+++ b/tests/qemu-iotests/112
@@ -180,6 +180,55 @@ class TestIncrementalBackup(iotests.QMPTestCase):
 return True
 
 
+def test_incremental_transaction(self):
+'''Test: Verify backups made from transactionally created bitmaps.
+
+Create a bitmap "before" VM execution begins, then create a second
+bitmap AFTER writes have already occurred. Use transactions to create
+a full backup and synchronize both bitmaps to this backup.
+Create an incremental backup through both bitmaps and verify that
+both backups match the full backup.
+'''
+bitmap0 = self.add_bitmap('bitmap0', 'drive0')
+self.hmp_io_writes('drive0', (('0xab', 0, 512),
+  ('0xfe', '16M', '256k'),
+  ('0x64', '32736k', '64k')))
+bitmap1 = self.add_bitmap('bitmap1', 'drive0')
+
+result = self.vm.qmp('transaction', actions=[
+{
+'type': 'block-dirty-bitmap-clear',
+'data': { 'node': 'drive0',
+  'name': 'bitmap0' },
+},
+{
+'type': 'block-dirty-bitmap-clear',
+'data': { 'node': 'drive0',
+  'name': 'bitmap1' },
+},
+{
+'type': 'drive-backup',
+'data': { 'device': 'drive0',
+  'sync': 'full',
+  'format': iotests.imgfmt,
+  'target': self.full_bak },
+}
+])
+self.assert_qmp(result, 'return', {})
+self.wait_until_completed()
+self.files.append(self.full_bak)
+self.check_full_backup()
+
+self.hmp_io_writes('drive0', (('0x9a', 0, 512),
+  ('0x55', '8M', '352k'),
+  ('0x78', '15872k', '1M')))
+# Both bitmaps should be in sync and create fully valid
+# incremental backups
+res1 = self.create_incremental(bitmap0)
+res2 = self.create_incremental(bitmap1)
+self.assertTrue(res1 and res2)
+
+
 def test_sync_dirty_bitmap_missing(self):
 self.assert_no_active_block_jobs()
 self.files.append(self.foo_img)
diff --git a/tests/qemu-iotests/112.out b/tests/qemu-iotests/112.out
index 8d7e996..89968f3 100644
--- a/tests/qemu-iotests/112.out
+++ b/tests/qemu-iotests/112.out
@@ -1,5 +1,5 @@
-...
+
 --
-Ran 3 tests
+Ran 4 tests
 
 OK
-- 
1.9.3

[Qemu-devel] [PATCH v14 18/19] block: Resize bitmaps on bdrv_truncate

[John Snow]

Signed-off-by: John Snow 
---
 block.c| 20 
 include/block/block.h  |  1 +
 include/qemu/hbitmap.h | 10 ++
 util/hbitmap.c | 47 +++
 4 files changed, 78 insertions(+)

diff --git a/block.c b/block.c
index f3a6dd4..59a8ec9 100644
--- a/block.c
+++ b/block.c
@@ -3514,6 +3514,7 @@ int bdrv_truncate(BlockDriverState *bs, int64_t offset)
 ret = drv->bdrv_truncate(bs, offset);
 if (ret == 0) {
 ret = refresh_total_sectors(bs, offset >> BDRV_SECTOR_BITS);
+bdrv_dirty_bitmap_truncate(bs);
 if (bs->blk) {
 blk_dev_resize_cb(bs->blk);
 }
@@ -5524,6 +5525,25 @@ BdrvDirtyBitmap 
*bdrv_reclaim_dirty_bitmap(BlockDriverState *bs,
 return parent;
 }
 
+static void dirty_bitmap_truncate(BdrvDirtyBitmap *bitmap, uint64_t size)
+{
+assert(!bdrv_dirty_bitmap_frozen(bitmap));
+hbitmap_truncate(bitmap->bitmap, size);
+}
+
+void bdrv_dirty_bitmap_truncate(BlockDriverState *bs)
+{
+BdrvDirtyBitmap *bitmap;
+uint64_t size = bdrv_nb_sectors(bs);
+
+QLIST_FOREACH(bitmap, &bs->dirty_bitmaps, list) {
+if (bdrv_dirty_bitmap_frozen(bitmap)) {
+continue;
+}
+dirty_bitmap_truncate(bitmap, size);
+}
+}
+
 void bdrv_release_dirty_bitmap(BlockDriverState *bs, BdrvDirtyBitmap *bitmap)
 {
 BdrvDirtyBitmap *bm, *next;
diff --git a/include/block/block.h b/include/block/block.h
index f6a50ae..aa6912d 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -442,6 +442,7 @@ BdrvDirtyBitmap *bdrv_create_dirty_bitmap(BlockDriverState 
*bs,
   uint32_t granularity,
   const char *name,
   Error **errp);
+void bdrv_dirty_bitmap_truncate(BlockDriverState *bs);
 int bdrv_dirty_bitmap_create_successor(BlockDriverState *bs,
BdrvDirtyBitmap *bitmap,
Error **errp);
diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h
index c19c1cb..a75157e 100644
--- a/include/qemu/hbitmap.h
+++ b/include/qemu/hbitmap.h
@@ -65,6 +65,16 @@ struct HBitmapIter {
 HBitmap *hbitmap_alloc(uint64_t size, int granularity);
 
 /**
+ * hbitmap_truncate:
+ * @hb: The bitmap to change the size of.
+ * @size: The number of elements to change the bitmap to accommodate.
+ *
+ * truncate or grow an existing bitmap to accommodate a new number of elements.
+ * This may invalidate existing HBitmapIterators.
+ */
+void hbitmap_truncate(HBitmap *hb, uint64_t size);
+
+/**
  * hbitmap_merge:
  * @a: The bitmap to store the result in.
  * @b: The bitmap to merge into @a.
diff --git a/util/hbitmap.c b/util/hbitmap.c
index 962ff29..d17b850 100644
--- a/util/hbitmap.c
+++ b/util/hbitmap.c
@@ -90,6 +90,9 @@ struct HBitmap {
  * bitmap will still allocate HBITMAP_LEVELS arrays.
  */
 unsigned long *levels[HBITMAP_LEVELS];
+
+/* The length of each levels[] array. */
+uint64_t sizes[HBITMAP_LEVELS];
 };
 
 /* Advance hbi to the next nonzero word and return it.  hbi->pos
@@ -384,6 +387,7 @@ HBitmap *hbitmap_alloc(uint64_t size, int granularity)
 hb->granularity = granularity;
 for (i = HBITMAP_LEVELS; i-- > 0; ) {
 size = MAX((size + BITS_PER_LONG - 1) >> BITS_PER_LEVEL, 1);
+hb->sizes[i] = size;
 hb->levels[i] = g_new0(unsigned long, size);
 }
 
@@ -396,6 +400,49 @@ HBitmap *hbitmap_alloc(uint64_t size, int granularity)
 return hb;
 }
 
+void hbitmap_truncate(HBitmap *hb, uint64_t size)
+{
+bool truncate;
+unsigned i;
+uint64_t num_elements = size;
+
+/* Size comes in as logical elements, adjust for granularity. */
+size = (size + (1ULL << hb->granularity) - 1) >> hb->granularity;
+assert(size <= ((uint64_t)1 << HBITMAP_LOG_MAX_SIZE));
+truncate = size < hb->size;
+
+if (size == hb->size) {
+/* A hard day's work */
+return;
+}
+
+hb->size = size;
+for (i = HBITMAP_LEVELS; i-- > 0; ) {
+size = MAX((size + BITS_PER_LONG - 1) >> BITS_PER_LEVEL, 1);
+if (hb->sizes[i] == size) {
+continue;
+}
+hb->sizes[i] = size;
+hb->levels[i] = g_realloc_n(hb->levels[i], size, sizeof(unsigned 
long));
+}
+assert(size == 1);
+
+/* Clear out any "extra space" we may have that the user didn't request:
+ * It may have garbage data in it, now. */
+if (truncate) {
+/* Due to granularity fuzziness, we may accidentally reset some of
+ * the last bits that are actually valid. So, record the current value,
+ * reset the "dead range," then re-set the one sector we care about. */
+bool status = hbitmap_get(hb, num_elements - 1);
+uint64_t fix_count = (hb->size << hb->granularity) - num_elements;
+hbitmap_reset(hb, num_elements, fix_count);
+if (status) {
+  

[Qemu-devel] [PATCH v14 17/19] iotests: add incremental backup failure recovery test

[John Snow]

To test the failure case, we modify iotests.py to allow us to specify
that we'd like to allow failures when we wait for block job events.

Reviewed-by: Max Reitz 
Signed-off-by: John Snow 
---
 tests/qemu-iotests/112| 57 ++-
 tests/qemu-iotests/112.out|  4 +--
 tests/qemu-iotests/iotests.py |  6 +++--
 3 files changed, 62 insertions(+), 5 deletions(-)

diff --git a/tests/qemu-iotests/112 b/tests/qemu-iotests/112
index 59eb1b1..da0bf6d 100644
--- a/tests/qemu-iotests/112
+++ b/tests/qemu-iotests/112
@@ -135,7 +135,11 @@ class TestIncrementalBackup(iotests.QMPTestCase):
  mode='existing')
 self.assert_qmp(result, 'return', {})
 
-event = self.wait_until_completed(bitmap.node, check_offset=validate)
+event = self.wait_until_completed(bitmap.node, check_offset=validate,
+  allow_failures=(not validate))
+if 'error' in event['data']:
+bitmap.del_target()
+return False
 if validate:
 return self.check_incremental(target)
 
@@ -229,6 +233,57 @@ class TestIncrementalBackup(iotests.QMPTestCase):
 self.assertTrue(res1 and res2)
 
 
+def test_incremental_failure(self):
+'''Test: Verify backups made after a failure are correct.
+
+Simulate a failure during an incremental backup block job,
+emulate additional writes, then create another incremental backup
+afterwards and verify that the backup created is correct.
+'''
+
+# Create a blkdebug interface to this img as 'drive1'
+result = self.vm.qmp('blockdev-add', options={
+'id': 'drive1',
+'driver': iotests.imgfmt,
+'file': {
+'driver': 'blkdebug',
+'image': {
+'driver': 'file',
+'filename': self.test_img
+},
+'set-state': [{
+'event': 'flush_to_disk',
+'state': 1,
+'new_state': 2
+}],
+'inject-error': [{
+'event': 'read_aio',
+'errno': 5,
+'state': 2,
+'immediately': False,
+'once': True
+}],
+}
+})
+self.assert_qmp(result, 'return', {})
+
+self.create_full_backup()
+self.add_bitmap('bitmap0', 'drive1')
+# Note: at this point, during a normal execution,
+# Assume that the VM resumes and begins issuing IO requests here.
+
+self.hmp_io_writes('drive1', (('0xab', 0, 512),
+  ('0xfe', '16M', '256k'),
+  ('0x64', '32736k', '64k')))
+
+result = self.create_incremental(validate=False)
+self.assertFalse(result)
+self.hmp_io_writes('drive1', (('0x9a', 0, 512),
+  ('0x55', '8M', '352k'),
+  ('0x78', '15872k', '1M')))
+self.create_incremental()
+
+
 def test_sync_dirty_bitmap_missing(self):
 self.assert_no_active_block_jobs()
 self.files.append(self.foo_img)
diff --git a/tests/qemu-iotests/112.out b/tests/qemu-iotests/112.out
index 89968f3..914e373 100644
--- a/tests/qemu-iotests/112.out
+++ b/tests/qemu-iotests/112.out
@@ -1,5 +1,5 @@
-
+.
 --
-Ran 4 tests
+Ran 5 tests
 
 OK
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index 241b5ee..cf5faac 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -258,14 +258,16 @@ class QMPTestCase(unittest.TestCase):
 self.assert_no_active_block_jobs()
 return result
 
-def wait_until_completed(self, drive='drive0', check_offset=True):
+def wait_until_completed(self, drive='drive0', check_offset=True,
+ allow_failures=False):
 '''Wait for a block job to finish, returning the event'''
 completed = False
 while not completed:
 for event in self.vm.get_qmp_events(wait=True):
 if event['event'] == 'BLOCK_JOB_COMPLETED':
 self.assert_qmp(event, 'data/device', drive)
-self.assert_qmp_absent(event, 'data/error')
+if not allow_failures:
+self.assert_qmp_absent(event, 'data/error')
 if check_offset:
 self.assert_qmp(event, 'data/offset', 
event['data']['len'])
 completed = True
-- 
1.9.3

[Qemu-devel] [PATCH v14 07/19] block: Add bitmap successors

[John Snow]

A bitmap successor is an anonymous BdrvDirtyBitmap that is intended to
be created just prior to a sensitive operation (e.g. Incremental Backup)
that can either succeed or fail, but during the course of which we still
want a bitmap tracking writes.

On creating a successor, we "freeze" the parent bitmap which prevents
its deletion, enabling, anonymization, or creating a bitmap with the
same name.

On success, the parent bitmap can "abdicate" responsibility to the
successor, which will inherit its name. The successor will have been
tracking writes during the course of the backup operation. The parent
will be safely deleted.

On failure, we can "reclaim" the successor from the parent, unifying
them such that the resulting bitmap describes all writes occurring since
the last successful backup, for instance. Reclamation will thaw the
parent, but not explicitly re-enable it.

BdrvDirtyBitmap operations that target a single bitmap are protected
by assertions that the bitmap is not frozen and/or disabled.

BdrvDirtyBitmap operations that target a group of bitmaps, such as
bdrv_{set,reset}_dirty will ignore frozen/disabled drives with a
conditional instead.

QMP transactions that enable/disable bitmaps have extra error checking
surrounding them that prevent modifying bitmaps that are frozen.

Signed-off-by: John Snow 
Reviewed-by: Max Reitz 
---
 block.c   | 104 +-
 blockdev.c|  22 +++
 include/block/block.h |  10 +
 qapi/block-core.json  |   3 ++
 4 files changed, 138 insertions(+), 1 deletion(-)

diff --git a/block.c b/block.c
index 1fa2ed5..665ae69 100644
--- a/block.c
+++ b/block.c
@@ -51,8 +51,17 @@
 #include 
 #endif
 
+/**
+ * A BdrvDirtyBitmap can be in three possible states:
+ * (1) successor is false and disabled is false: full r/w mode
+ * (2) successor is false and disabled is true: read only mode ("disabled")
+ * (3) successor is set: frozen mode.
+ * A frozen bitmap cannot be renamed, deleted, anonymized, cleared, set,
+ * or enabled. A frozen bitmap can only abdicate() or reclaim().
+ */
 struct BdrvDirtyBitmap {
 HBitmap *bitmap;
+BdrvDirtyBitmap *successor;
 char *name;
 bool disabled;
 QLIST_ENTRY(BdrvDirtyBitmap) list;
@@ -5383,6 +5392,7 @@ BdrvDirtyBitmap *bdrv_find_dirty_bitmap(BlockDriverState 
*bs, const char *name)
 
 void bdrv_dirty_bitmap_make_anon(BlockDriverState *bs, BdrvDirtyBitmap *bitmap)
 {
+assert(!bdrv_dirty_bitmap_frozen(bitmap));
 g_free(bitmap->name);
 bitmap->name = NULL;
 }
@@ -5418,9 +5428,98 @@ BdrvDirtyBitmap 
*bdrv_create_dirty_bitmap(BlockDriverState *bs,
 return bitmap;
 }
 
+bool bdrv_dirty_bitmap_frozen(BdrvDirtyBitmap *bitmap)
+{
+return bitmap->successor;
+}
+
 bool bdrv_dirty_bitmap_enabled(BdrvDirtyBitmap *bitmap)
 {
-return !bitmap->disabled;
+return !(bitmap->disabled || bitmap->successor);
+}
+
+/**
+ * Create a successor bitmap destined to replace this bitmap after an 
operation.
+ * Requires that the bitmap is not frozen and has no successor.
+ */
+int bdrv_dirty_bitmap_create_successor(BlockDriverState *bs,
+   BdrvDirtyBitmap *bitmap, Error **errp)
+{
+uint64_t granularity;
+BdrvDirtyBitmap *child;
+
+if (bdrv_dirty_bitmap_frozen(bitmap)) {
+error_setg(errp, "Cannot create a successor for a bitmap that is "
+   "currently frozen");
+return -1;
+}
+assert(!bitmap->successor);
+
+/* Create an anonymous successor */
+granularity = bdrv_dirty_bitmap_granularity(bitmap);
+child = bdrv_create_dirty_bitmap(bs, granularity, NULL, errp);
+if (!child) {
+return -1;
+}
+
+/* Successor will be on or off based on our current state. */
+child->disabled = bitmap->disabled;
+
+/* Install the successor and freeze the parent */
+bitmap->successor = child;
+return 0;
+}
+
+/**
+ * For a bitmap with a successor, yield our name to the successor,
+ * Delete the old bitmap, and return a handle to the new bitmap.
+ */
+BdrvDirtyBitmap *bdrv_dirty_bitmap_abdicate(BlockDriverState *bs,
+BdrvDirtyBitmap *bitmap,
+Error **errp)
+{
+char *name;
+BdrvDirtyBitmap *successor = bitmap->successor;
+
+if (successor == NULL) {
+error_setg(errp, "Cannot relinquish control if "
+   "there's no successor present");
+return NULL;
+}
+
+name = bitmap->name;
+bitmap->name = NULL;
+successor->name = name;
+bitmap->successor = NULL;
+bdrv_release_dirty_bitmap(bs, bitmap);
+
+return successor;
+}
+
+/**
+ * In cases of failure where we can no longer safely delete the parent,
+ * We may wish to re-join the parent and child/successor.
+ * The merged parent will be un-frozen, but not explicitly re-enabled.
+ */
+BdrvDirtyBitmap *bdrv_reclaim_dirty_bitmap(BlockDriverState *bs,

[Qemu-devel] [PATCH v14 15/19] iotests: add simple incremental backup case

[John Snow]

Reviewed-by: Max Reitz 
Signed-off-by: John Snow 
---
 tests/qemu-iotests/112 | 122 +
 tests/qemu-iotests/112.out |   4 +-
 2 files changed, 124 insertions(+), 2 deletions(-)

diff --git a/tests/qemu-iotests/112 b/tests/qemu-iotests/112
index 7985cd1..da8f0e0 100644
--- a/tests/qemu-iotests/112
+++ b/tests/qemu-iotests/112
@@ -28,6 +28,36 @@ def io_write_patterns(img, patterns):
 for pattern in patterns:
 iotests.qemu_io('-c', 'write -P%s %s %s' % pattern, img)
 
+class Bitmap:
+def __init__(self, name, node):
+self.name = name
+self.node = node
+self.pattern = os.path.join(iotests.test_dir.replace('%', '%%'),
+'%s.backup.%%i.img' % name)
+self.num = 0
+self.backups = list()
+
+def new_target(self, num=None):
+if num is None:
+num = self.num
+self.num = num + 1
+target = self.pattern % num
+self.backups.append(target)
+return target
+
+def last_target(self):
+return self.backups[-1]
+
+def del_target(self):
+os.remove(self.backups.pop())
+self.num -= 1
+
+def __del__(self):
+for backup in self.backups:
+try:
+os.remove(backup)
+except OSError:
+pass
 
 class TestIncrementalBackup(iotests.QMPTestCase):
 def setUp(self):
@@ -58,6 +88,98 @@ class TestIncrementalBackup(iotests.QMPTestCase):
 iotests.qemu_img('create', '-f', fmt, img, size)
 self.files.append(img)
 
+
+def create_full_backup(self, drive='drive0'):
+res = self.vm.qmp('drive-backup', device=drive,
+  sync='full', format=iotests.imgfmt,
+  target=self.full_bak)
+self.assert_qmp(res, 'return', {})
+self.wait_until_completed(drive)
+self.check_full_backup()
+self.files.append(self.full_bak)
+
+
+def check_full_backup(self):
+self.assertTrue(iotests.compare_images(self.test_img, self.full_bak))
+
+
+def add_bitmap(self, name, node='drive0'):
+bitmap = Bitmap(name, node)
+self.bitmaps.append(bitmap)
+result = self.vm.qmp('block-dirty-bitmap-add', node=bitmap.node,
+ name=bitmap.name)
+self.assert_qmp(result, 'return', {})
+return bitmap
+
+
+def create_incremental(self, bitmap=None, num=None,
+   parent=None, parentFormat=None, validate=True):
+if bitmap is None:
+bitmap = self.bitmaps[-1]
+
+# If this is the first incremental backup for a bitmap,
+# use the full backup as a backing image. Otherwise, use
+# the last incremental backup.
+if parent is None:
+if bitmap.num == 0:
+parent = self.full_bak
+else:
+parent = bitmap.last_target()
+
+target = bitmap.new_target(num)
+self.img_create(target, iotests.imgfmt, parent=parent)
+
+result = self.vm.qmp('drive-backup', device=bitmap.node,
+ sync='dirty-bitmap', bitmap=bitmap.name,
+ format=iotests.imgfmt, target=target,
+ mode='existing')
+self.assert_qmp(result, 'return', {})
+
+event = self.wait_until_completed(bitmap.node, check_offset=validate)
+if validate:
+return self.check_incremental(target)
+
+
+def check_incremental(self, target=None):
+if target is None:
+target = self.bitmaps[-1].last_target()
+self.assertTrue(iotests.compare_images(self.test_img, target))
+return True
+
+
+def hmp_io_writes(self, drive, patterns):
+for pattern in patterns:
+self.vm.hmp_qemu_io(drive, 'write -P%s %s %s' % pattern)
+self.vm.hmp_qemu_io(drive, 'flush')
+
+
+def test_incremental_simple(self):
+'''
+Test: Create and verify three incremental backups.
+
+Create a bitmap and a full backup before VM execution begins,
+then create a series of three incremental backups "during execution,"
+i.e.; after IO requests begin modifying the drive.
+'''
+self.create_full_backup()
+self.add_bitmap('bitmap0', 'drive0')
+
+# Sanity: Create a "hollow" incremental backup
+self.create_incremental()
+# Three writes: One complete overwrite, one new segment,
+# and one partial overlap.
+self.hmp_io_writes('drive0', (('0xab', 0, 512),
+  ('0xfe', '16M', '256k'),
+  ('0x64', '32736k', '64k')))
+self.create_incremental()
+# Three more writes, one of each kind, like above
+self.hmp_io_writes('drive0', (('0x9a', 0, 512),
+  ('0x55', '8M', '352k'),
+  

[Qemu-devel] [PATCH v14 06/19] qmp: Add block-dirty-bitmap-enable and block-dirty-bitmap-disable

[John Snow]

This allows to put the dirty bitmap into a disabled state where it is
read only. A disabled bitmap will ignore any attempts to set or reset
any of its bits, but can otherwise be renamed, deleted, or re-enabled.

It will be used before backup or writing to persistent file.

Signed-off-by: Fam Zheng 
Signed-off-by: John Snow 
Reviewed-by: Max Reitz 
---
 block.c   | 25 +
 blockdev.c| 40 
 include/block/block.h |  3 +++
 qapi/block-core.json  | 28 
 qmp-commands.hx   | 10 ++
 5 files changed, 106 insertions(+)

diff --git a/block.c b/block.c
index 8b40e38..1fa2ed5 100644
--- a/block.c
+++ b/block.c
@@ -54,6 +54,7 @@
 struct BdrvDirtyBitmap {
 HBitmap *bitmap;
 char *name;
+bool disabled;
 QLIST_ENTRY(BdrvDirtyBitmap) list;
 };
 
@@ -5412,10 +5413,16 @@ BdrvDirtyBitmap 
*bdrv_create_dirty_bitmap(BlockDriverState *bs,
 bitmap = g_new0(BdrvDirtyBitmap, 1);
 bitmap->bitmap = hbitmap_alloc(bitmap_size, ffs(sector_granularity) - 1);
 bitmap->name = g_strdup(name);
+bitmap->disabled = false;
 QLIST_INSERT_HEAD(&bs->dirty_bitmaps, bitmap, list);
 return bitmap;
 }
 
+bool bdrv_dirty_bitmap_enabled(BdrvDirtyBitmap *bitmap)
+{
+return !bitmap->disabled;
+}
+
 void bdrv_release_dirty_bitmap(BlockDriverState *bs, BdrvDirtyBitmap *bitmap)
 {
 BdrvDirtyBitmap *bm, *next;
@@ -5430,6 +5437,16 @@ void bdrv_release_dirty_bitmap(BlockDriverState *bs, 
BdrvDirtyBitmap *bitmap)
 }
 }
 
+void bdrv_disable_dirty_bitmap(BdrvDirtyBitmap *bitmap)
+{
+bitmap->disabled = true;
+}
+
+void bdrv_enable_dirty_bitmap(BdrvDirtyBitmap *bitmap)
+{
+bitmap->disabled = false;
+}
+
 BlockDirtyInfoList *bdrv_query_dirty_bitmaps(BlockDriverState *bs)
 {
 BdrvDirtyBitmap *bm;
@@ -5494,12 +5511,14 @@ void bdrv_dirty_iter_init(BlockDriverState *bs,
 void bdrv_set_dirty_bitmap(BlockDriverState *bs, BdrvDirtyBitmap *bitmap,
int64_t cur_sector, int nr_sectors)
 {
+assert(bdrv_dirty_bitmap_enabled(bitmap));
 hbitmap_set(bitmap->bitmap, cur_sector, nr_sectors);
 }
 
 void bdrv_reset_dirty_bitmap(BlockDriverState *bs, BdrvDirtyBitmap *bitmap,
  int64_t cur_sector, int nr_sectors)
 {
+assert(bdrv_dirty_bitmap_enabled(bitmap));
 hbitmap_reset(bitmap->bitmap, cur_sector, nr_sectors);
 }
 
@@ -5508,6 +5527,9 @@ static void bdrv_set_dirty(BlockDriverState *bs, int64_t 
cur_sector,
 {
 BdrvDirtyBitmap *bitmap;
 QLIST_FOREACH(bitmap, &bs->dirty_bitmaps, list) {
+if (!bdrv_dirty_bitmap_enabled(bitmap)) {
+continue;
+}
 hbitmap_set(bitmap->bitmap, cur_sector, nr_sectors);
 }
 }
@@ -5517,6 +5539,9 @@ static void bdrv_reset_dirty(BlockDriverState *bs, 
int64_t cur_sector,
 {
 BdrvDirtyBitmap *bitmap;
 QLIST_FOREACH(bitmap, &bs->dirty_bitmaps, list) {
+if (!bdrv_dirty_bitmap_enabled(bitmap)) {
+continue;
+}
 hbitmap_reset(bitmap->bitmap, cur_sector, nr_sectors);
 }
 }
diff --git a/blockdev.c b/blockdev.c
index 8842e39..d3599ac 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -2053,6 +2053,46 @@ void qmp_block_dirty_bitmap_remove(const char *node, 
const char *name,
 aio_context_release(aio_context);
 }
 
+void qmp_block_dirty_bitmap_enable(const char *node, const char *name,
+   Error **errp)
+{
+AioContext *aio_context;
+BdrvDirtyBitmap *bitmap;
+BlockDriverState *bs;
+
+bitmap = block_dirty_bitmap_lookup(node, name, &bs, errp);
+if (!bitmap) {
+return;
+}
+
+aio_context = bdrv_get_aio_context(bs);
+aio_context_acquire(aio_context);
+
+bdrv_enable_dirty_bitmap(bitmap);
+
+aio_context_release(aio_context);
+}
+
+void qmp_block_dirty_bitmap_disable(const char *node, const char *name,
+Error **errp)
+{
+AioContext *aio_context;
+BdrvDirtyBitmap *bitmap;
+BlockDriverState *bs;
+
+bitmap = block_dirty_bitmap_lookup(node, name, &bs, errp);
+if (!bitmap) {
+return;
+}
+
+aio_context = bdrv_get_aio_context(bs);
+aio_context_acquire(aio_context);
+
+bdrv_disable_dirty_bitmap(bitmap);
+
+aio_context_release(aio_context);
+}
+
 int do_drive_del(Monitor *mon, const QDict *qdict, QObject **ret_data)
 {
 const char *id = qdict_get_str(qdict, "id");
diff --git a/include/block/block.h b/include/block/block.h
index 41b8418..ae7e2ab 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -446,9 +446,12 @@ BdrvDirtyBitmap *bdrv_find_dirty_bitmap(BlockDriverState 
*bs,
 const char *name);
 void bdrv_dirty_bitmap_make_anon(BlockDriverState *bs, BdrvDirtyBitmap 
*bitmap);
 void bdrv_release_dirty_bitmap(BlockDriverState *bs, BdrvDirtyBitmap *bitmap);
+void bdrv_disable_dirty_bitmap(BdrvDirtyBitmap *bitmap);
+void bdrv_enable

[Qemu-devel] [PATCH v14 10/19] qapi: Add transaction support to block-dirty-bitmap operations

[John Snow]

This adds four qmp commands to transactions.

Users can stop a dirty bitmap, start backup of it, and start another
dirty bitmap atomically, so that the dirty bitmap is tracked
incrementally and we don't miss any write.

For starting a new incremental backup chain, users can also chain
together a bitmap clear and a full block backup.

Signed-off-by: Fam Zheng 
Signed-off-by: John Snow 
---
 blockdev.c   | 170 +++
 qapi-schema.json |  10 +++-
 2 files changed, 179 insertions(+), 1 deletion(-)

diff --git a/blockdev.c b/blockdev.c
index 8422e94..7fbcb98 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -1675,6 +1675,153 @@ static void blockdev_backup_clean(BlkTransactionState 
*common)
 }
 }
 
+typedef struct BlockDirtyBitmapState {
+BlkTransactionState common;
+BdrvDirtyBitmap *bitmap;
+BlockDriverState *bs;
+AioContext *aio_context;
+bool prepared;
+} BlockDirtyBitmapState;
+
+static void block_dirty_bitmap_add_prepare(BlkTransactionState *common,
+   Error **errp)
+{
+BlockDirtyBitmapAdd *action;
+BlockDirtyBitmapState *state = DO_UPCAST(BlockDirtyBitmapState,
+ common, common);
+
+action = common->action->block_dirty_bitmap_add;
+/* AIO context taken within qmp_block_dirty_bitmap_add */
+qmp_block_dirty_bitmap_add(action->node, action->name,
+   action->has_granularity, action->granularity,
+   errp);
+
+if (*errp == NULL) {
+state->prepared = true;
+}
+}
+
+static void block_dirty_bitmap_add_abort(BlkTransactionState *common)
+{
+BlockDirtyBitmapAdd *action;
+   BlockDirtyBitmapState *state = DO_UPCAST(BlockDirtyBitmapState,
+ common, common);
+
+action = common->action->block_dirty_bitmap_add;
+/* Should not fail meaningfully: IF the bitmap was added via .prepare(),
+ * then the node reference and bitmap name must have been valid.
+ * THUS: any failure here could only indicate the lack of a bitmap at all.
+ */
+if (state->prepared) {
+qmp_block_dirty_bitmap_remove(action->node, action->name, NULL);
+}
+}
+
+/**
+ * Enable and Disable re-use the same preparation.
+ */
+static void block_dirty_bitmap_toggle_prepare(BlkTransactionState *common,
+  Error **errp)
+{
+BlockDirtyBitmapState *state = DO_UPCAST(BlockDirtyBitmapState,
+ common, common);
+BlockDirtyBitmap *action;
+BlockDriverState *bs;
+
+/* We may be used by either enable or disable;
+ * We use the "enable" member of the union here,
+ * but "disable" should be functionally equivalent: */
+action = common->action->block_dirty_bitmap_enable;
+assert(action == common->action->block_dirty_bitmap_disable);
+
+state->bitmap = block_dirty_bitmap_lookup(action->node,
+  action->name,
+  &bs,
+  errp);
+if (!state->bitmap) {
+return;
+}
+
+if (bdrv_dirty_bitmap_frozen(state->bitmap)) {
+error_setg(errp, "Cannot modify a frozen bitmap");
+return;
+}
+
+/* AioContext is released in .clean() */
+state->aio_context = bdrv_get_aio_context(bs);
+aio_context_acquire(state->aio_context);
+}
+
+static void block_dirty_bitmap_enable_commit(BlkTransactionState *common)
+{
+BlockDirtyBitmapState *state = DO_UPCAST(BlockDirtyBitmapState,
+ common, common);
+bdrv_enable_dirty_bitmap(state->bitmap);
+}
+
+static void block_dirty_bitmap_disable_commit(BlkTransactionState *common)
+{
+BlockDirtyBitmapState *state = DO_UPCAST(BlockDirtyBitmapState,
+ common, common);
+bdrv_disable_dirty_bitmap(state->bitmap);
+}
+
+static void block_dirty_bitmap_toggle_clean(BlkTransactionState *common)
+{
+BlockDirtyBitmapState *state = DO_UPCAST(BlockDirtyBitmapState,
+ common, common);
+
+if (state->aio_context) {
+aio_context_release(state->aio_context);
+}
+}
+
+static void block_dirty_bitmap_clear_prepare(BlkTransactionState *common,
+ Error **errp)
+{
+BlockDirtyBitmapState *state = DO_UPCAST(BlockDirtyBitmapState,
+ common, common);
+BlockDirtyBitmap *action;
+
+action = common->action->block_dirty_bitmap_clear;
+state->bitmap = block_dirty_bitmap_lookup(action->node,
+  action->name,
+  &state->bs,
+  errp);
+if (!state->bitmap) {
+return;
+ 

[Qemu-devel] [PATCH v14 11/19] qmp: Add dirty bitmap status fields in query-block

[John Snow]

Adds the "disabled" and "frozen" status booleans.

Signed-off-by: Fam Zheng 
Signed-off-by: John Snow 
Reviewed-by: Max Reitz 
---
 block.c  | 2 ++
 qapi/block-core.json | 7 ++-
 2 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/block.c b/block.c
index e2da4cb..a7f56f4 100644
--- a/block.c
+++ b/block.c
@@ -5564,6 +5564,8 @@ BlockDirtyInfoList 
*bdrv_query_dirty_bitmaps(BlockDriverState *bs)
 info->granularity = bdrv_dirty_bitmap_granularity(bm);
 info->has_name = !!bm->name;
 info->name = g_strdup(bm->name);
+info->disabled = bm->disabled;
+info->frozen = bdrv_dirty_bitmap_frozen(bm);
 entry->value = info;
 *plist = entry;
 plist = &entry->next;
diff --git a/qapi/block-core.json b/qapi/block-core.json
index bfdb293..02bf2b6 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -332,10 +332,15 @@
 #
 # @granularity: granularity of the dirty bitmap in bytes (since 1.4)
 #
+# @disabled: whether the dirty bitmap is disabled (Since 2.3)
+#
+# @frozen: whether the dirty bitmap is frozen (Since 2.3)
+#
 # Since: 1.3
 ##
 { 'type': 'BlockDirtyInfo',
-  'data': {'*name': 'str', 'count': 'int', 'granularity': 'uint32'} }
+  'data': {'*name': 'str', 'count': 'int', 'granularity': 'uint32',
+   'disabled': 'bool', 'frozen': 'bool'} }
 
 ##
 # @BlockInfo:
-- 
1.9.3

[Qemu-devel] [PATCH v14 12/19] block: add BdrvDirtyBitmap documentation

[John Snow]

Signed-off-by: John Snow 
Reviewed-by: Max Reitz 
---
 block.c | 10 +-
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/block.c b/block.c
index a7f56f4..c1c11aa 100644
--- a/block.c
+++ b/block.c
@@ -60,11 +60,11 @@
  * or enabled. A frozen bitmap can only abdicate() or reclaim().
  */
 struct BdrvDirtyBitmap {
-HBitmap *bitmap;
-BdrvDirtyBitmap *successor;
-int64_t size;
-char *name;
-bool disabled;
+HBitmap *bitmap;/* Dirty sector bitmap implementation */
+BdrvDirtyBitmap *successor; /* Anonymous child; implies frozen status */
+char *name; /* Optional non-empty unique ID */
+int64_t size;   /* Size of the bitmap (Number of sectors) */
+bool disabled;  /* Bitmap is read-only */
 QLIST_ENTRY(BdrvDirtyBitmap) list;
 };
 
-- 
1.9.3

[Qemu-devel] [PATCH v14 00/19] block: incremental backup series

[John Snow]

This series requires: [PATCH v3] blkdebug: fix "once" rule

Salutations, weary traveler. Might I interest you in some
incremental backups? Only slightly used, just like new!

This patchset enables the in-memory part of the incremental backup
feature. Bitmap persistence and bitmap migration will be addressed
in future patchsets by Vladimir Sementsov-Ogievskiy.

This series and most patches were originally by Fam Zheng, though
I have corrupted them far beyond their original presentation.

Highlights (since v11):
 - Renamed "node-ref" parameter to "node" in all QMP patches.
   (A rose by any other name? ...)
 - Fuller, more robust iotests.
 - Fixed progress percentages for incremental backup jobs.
 - This patch relies upon a blkdebug fix for the iotests.
 - The ability to resize bitmaps (as part of bdrv_truncate)
   has been included.
 - Documentation for the user-perspective has been added.
   Developers are left to wonder what they did to deserve this.

V14:
 - 04: Edited description.
 - 10: Fixed the add transaction. I hope.
 - 18: Added a patch that allows bitmaps to be resized via
   bdrv_truncate().
 - 19: Added a documentation file.

V13:
 - 02: New patch, trying to iron out type issues.
 - 03: Patch no longer includes type fixes.
 - 04: Patch no longer includes tpye fixes.
 - 07: Error message fixes.
 - 08: Removed superfluous empty lines
 - Progress updates occur before cancellation check.
 - 09: Fixed a critical bug where I added an extra '-' to docs.
 - Corrected clear semantics (checks for disabled status)
 - More full stops and newlines removed from error_setg.
 - 10: Error message cleanup.
 - disabled status check added to clear transaction
 - Added version documentation for new transactions
 - Refiddled the add transaction to not delete erroneously
   if we abort after an unsuccessful prepare
 - 13: Joined lines where possible.
 - 15: Removed unnecessary imports
 - "%s" changed to "%i"
 - 'is 0' replaced by '== 0'
 - removed .add_args(["-S"]) and hmp("c") and added a docstring.
 - Check for '%' in dirnames.
 - 16: removed hmp("c"), added a doctsring.
 - xx  Dropped blkdebug patch, was #16.
 - 17: Removed hmp("c"), added a docstring.
 - Moved all blkdebug config into QMP, because Max asked very nicely.

V12:
 - Changed authorship from Fam Zheng to John Snow on most patches
 - 02: Fix the error_setg leak in bdrv_dirty_bitmap_lookup
 - Fix error phrasing in bdrv_dirty_bitmap_lookup
 - Renamed "node-ref" to "node" for QMP commands.
 - 03: Granularity helper no longer requires a BDS argument.
 - 04: Return early if the second bitmap is empty.
 - 05: Renamed the 'enabled' field to 'disabled to emphasize what the
   default operating state is.
 - We now guard against bit sets or resets with the bitmap is
   disabled, making it a more pure "read only" mode.
 - Some documentation phrasing changes.
 - 06: Removed explicit "frozen" state in favor of an implicit one.
   A successor present implies a frozen state.
 - Updated all functions that target a single bitmap to use
   assertions that the bitmap they are trying to modify is not
   frozen/disabled.
 - Functions that target multiple bitmaps use only a conditional,
   and will silently skip disabled bitmaps.
 - thaw() function removed. It is implicitly handled in reclaim
   and abdicate.
 - Added check for return code of hbitmap_merge.
 - Functions now check against enable OR disable when in frozen
   state, for consistency and simplicity.
 - Add "frozen" state documentation to remove/enable/disable
   QMP commands.
 - 07: Some documentation for bdrv_set_dirty_iter.
 - Move function calls outside of assert()
 - Cleanup the unused successor if we do not start the backup
 - Version documentation added for dirty-bitmap to block-core.json
 - Job progress is now reported for incremental backup jobs.
 - 08: bdrv_dirty_bitmap_clear is now in its own patch, here.
 - bdrv_dirty_bitmap_clear no longer takes a BDS argument.
 - 09: Added a transaction for bdrv_dirty_bitmap_clear.
 - 10: Change 'enabled' field to 'disabled' field, to match
   above decision in patch 05.
 - 12: Removed extraneous BDS arguments from most bitmap functions.
 - 13-15: New set of iotests.
 - 16: blkdebug fix, already posted upstream.
 - 17: Final iotest, testing failure case.

Fam Zheng (1):
  qapi: Add optional field "name" to block dirty bitmap

John Snow (18):
  qmp: Ensure consistent granularity type
  qmp: Add block-dirty-bitmap-add and block-dirty-bitmap-remove
  block: Introduce bdrv_dirty_bitmap_granularity()
  hbitmap: add hbitmap_merge
  qmp: Add block-dirty-bitmap-enable and block-dirty-bitmap-disable
  block: Add bitmap successors
  qmp: Add support of "dirty-bitmap" sync mode for drive-backup
  qmp: add block-dirty-bitmap-clear
  qapi: Add transaction support to block-dirty-bitmap operations
  qmp: Add dirty bitmap status fiel

[Qemu-devel] [PATCH v14 14/19] iotests: add invalid input incremental backup tests

[John Snow]

Signed-off-by: John Snow 
Reviewed-by: Max Reitz 
---
 tests/qemu-iotests/112 | 89 ++
 tests/qemu-iotests/112.out |  5 +++
 tests/qemu-iotests/group   |  1 +
 3 files changed, 95 insertions(+)
 create mode 100644 tests/qemu-iotests/112
 create mode 100644 tests/qemu-iotests/112.out

diff --git a/tests/qemu-iotests/112 b/tests/qemu-iotests/112
new file mode 100644
index 000..7985cd1
--- /dev/null
+++ b/tests/qemu-iotests/112
@@ -0,0 +1,89 @@
+#!/usr/bin/env python
+#
+# Tests for incremental drive-backup
+#
+# Copyright (C) 2015 John Snow for Red Hat, Inc.
+#
+# Based on 056.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see .
+#
+
+import os
+import iotests
+
+
+def io_write_patterns(img, patterns):
+for pattern in patterns:
+iotests.qemu_io('-c', 'write -P%s %s %s' % pattern, img)
+
+
+class TestIncrementalBackup(iotests.QMPTestCase):
+def setUp(self):
+self.bitmaps = list()
+self.files = list()
+self.vm = iotests.VM()
+self.test_img = os.path.join(iotests.test_dir, 'base.img')
+self.full_bak = os.path.join(iotests.test_dir, 'backup.img')
+self.foo_img = os.path.join(iotests.test_dir, 'foo.bar')
+self.img_create(self.test_img, iotests.imgfmt)
+self.vm.add_drive(self.test_img)
+# Create a base image with a distinctive patterning
+io_write_patterns(self.test_img, (('0x41', 0, 512),
+  ('0xd5', '1M', '32k'),
+  ('0xdc', '32M', '124k')))
+self.vm.launch()
+
+
+def img_create(self, img, fmt=iotests.imgfmt, size='64M',
+   parent=None, parentFormat=None):
+plist = list()
+if parent:
+if parentFormat is None:
+parentFormat = fmt
+iotests.qemu_img('create', '-f', fmt, img, size,
+ '-b', parent, '-F', parentFormat)
+else:
+iotests.qemu_img('create', '-f', fmt, img, size)
+self.files.append(img)
+
+def test_sync_dirty_bitmap_missing(self):
+self.assert_no_active_block_jobs()
+self.files.append(self.foo_img)
+result = self.vm.qmp('drive-backup', device='drive0',
+ sync='dirty-bitmap', format=iotests.imgfmt,
+ target=self.foo_img)
+self.assert_qmp(result, 'error/class', 'GenericError')
+
+
+def test_sync_dirty_bitmap_not_found(self):
+self.assert_no_active_block_jobs()
+self.files.append(self.foo_img)
+result = self.vm.qmp('drive-backup', device='drive0',
+ sync='dirty-bitmap', bitmap='unknown',
+ format=iotests.imgfmt, target=self.foo_img)
+self.assert_qmp(result, 'error/class', 'GenericError')
+
+
+def tearDown(self):
+self.vm.shutdown()
+for filename in self.files:
+try:
+os.remove(filename)
+except OSError:
+pass
+
+
+if __name__ == '__main__':
+iotests.main(supported_fmts=['qcow2'])
diff --git a/tests/qemu-iotests/112.out b/tests/qemu-iotests/112.out
new file mode 100644
index 000..fbc63e6
--- /dev/null
+++ b/tests/qemu-iotests/112.out
@@ -0,0 +1,5 @@
+..
+--
+Ran 2 tests
+
+OK
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index 4b2b93b..b4ddf1b 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -114,6 +114,7 @@
 109 rw auto
 110 rw auto backing quick
 111 rw auto quick
+112 rw auto backing
 113 rw auto quick
 114 rw auto quick
 116 rw auto quick
-- 
1.9.3

[Qemu-devel] [PATCH v14 09/19] qmp: add block-dirty-bitmap-clear

[John Snow]

Add bdrv_clear_dirty_bitmap and a matching QMP command,
qmp_block_dirty_bitmap_clear that enables a user to reset
the bitmap attached to a drive.

This allows us to reset a bitmap in the event of a full
drive backup.

Reviewed-by: Max Reitz 
Signed-off-by: John Snow 
---
 block.c   |  8 
 blockdev.c| 37 +
 include/block/block.h |  1 +
 qapi/block-core.json  | 14 ++
 qmp-commands.hx   | 24 
 5 files changed, 84 insertions(+)

diff --git a/block.c b/block.c
index 6e3f817..e2da4cb 100644
--- a/block.c
+++ b/block.c
@@ -62,6 +62,7 @@
 struct BdrvDirtyBitmap {
 HBitmap *bitmap;
 BdrvDirtyBitmap *successor;
+int64_t size;
 char *name;
 bool disabled;
 QLIST_ENTRY(BdrvDirtyBitmap) list;
@@ -5422,6 +5423,7 @@ BdrvDirtyBitmap 
*bdrv_create_dirty_bitmap(BlockDriverState *bs,
 }
 bitmap = g_new0(BdrvDirtyBitmap, 1);
 bitmap->bitmap = hbitmap_alloc(bitmap_size, ffs(sector_granularity) - 1);
+bitmap->size = bitmap_size;
 bitmap->name = g_strdup(name);
 bitmap->disabled = false;
 QLIST_INSERT_HEAD(&bs->dirty_bitmaps, bitmap, list);
@@ -5624,6 +5626,12 @@ void bdrv_reset_dirty_bitmap(BlockDriverState *bs, 
BdrvDirtyBitmap *bitmap,
 hbitmap_reset(bitmap->bitmap, cur_sector, nr_sectors);
 }
 
+void bdrv_clear_dirty_bitmap(BdrvDirtyBitmap *bitmap)
+{
+assert(bdrv_dirty_bitmap_enabled(bitmap));
+hbitmap_reset(bitmap->bitmap, 0, bitmap->size);
+}
+
 static void bdrv_set_dirty(BlockDriverState *bs, int64_t cur_sector,
int nr_sectors)
 {
diff --git a/blockdev.c b/blockdev.c
index 6990986..8422e94 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -2116,6 +2116,43 @@ void qmp_block_dirty_bitmap_disable(const char *node, 
const char *name,
 aio_context_release(aio_context);
 }
 
+/**
+ * Completely clear a bitmap, for the purposes of synchronizing a bitmap
+ * immediately after a full backup operation.
+ */
+void qmp_block_dirty_bitmap_clear(const char *node, const char *name,
+  Error **errp)
+{
+AioContext *aio_context;
+BdrvDirtyBitmap *bitmap;
+BlockDriverState *bs;
+
+bitmap = block_dirty_bitmap_lookup(node, name, &bs, errp);
+if (!bitmap) {
+return;
+}
+
+aio_context = bdrv_get_aio_context(bs);
+aio_context_acquire(aio_context);
+
+if (bdrv_dirty_bitmap_frozen(bitmap)) {
+error_setg(errp,
+   "Bitmap '%s' is currently frozen and cannot be modified",
+   name);
+goto out;
+} else if (!bdrv_dirty_bitmap_enabled(bitmap)) {
+error_setg(errp,
+   "Bitmap '%s' is currently disabled and cannot be cleared",
+   name);
+goto out;
+}
+
+bdrv_clear_dirty_bitmap(bitmap);
+
+ out:
+aio_context_release(aio_context);
+}
+
 int do_drive_del(Monitor *mon, const QDict *qdict, QObject **ret_data)
 {
 const char *id = qdict_get_str(qdict, "id");
diff --git a/include/block/block.h b/include/block/block.h
index be35b57..67227a2 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -467,6 +467,7 @@ void bdrv_set_dirty_bitmap(BlockDriverState *bs, 
BdrvDirtyBitmap *bitmap,
int64_t cur_sector, int nr_sectors);
 void bdrv_reset_dirty_bitmap(BlockDriverState *bs, BdrvDirtyBitmap *bitmap,
  int64_t cur_sector, int nr_sectors);
+void bdrv_clear_dirty_bitmap(BdrvDirtyBitmap *bitmap);
 void bdrv_dirty_iter_init(BlockDriverState *bs,
   BdrvDirtyBitmap *bitmap, struct HBitmapIter *hbi);
 void bdrv_set_dirty_iter(struct HBitmapIter *hbi, int64_t offset);
diff --git a/qapi/block-core.json b/qapi/block-core.json
index dca83c5..bfdb293 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -1050,6 +1050,20 @@
   'data': 'BlockDirtyBitmap' }
 
 ##
+# @block-dirty-bitmap-clear
+#
+# Clear (reset) a dirty bitmap on the device
+#
+# Returns: nothing on success
+#  If @node is not a valid block device, DeviceNotFound
+#  If @name is not found, GenericError with an explanation
+#
+# Since 2.3
+##
+{ 'command': 'block-dirty-bitmap-clear',
+  'data': 'BlockDirtyBitmap' }
+
+##
 # @block_set_io_throttle:
 #
 # Change I/O throttle limits for a block drive.
diff --git a/qmp-commands.hx b/qmp-commands.hx
index 5aa3845..21416fa 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -1264,6 +1264,11 @@ EQMP
 .args_type  = "node:B,name:s",
 .mhandler.cmd_new = qmp_marshal_input_block_dirty_bitmap_disable,
 },
+{
+.name   = "block-dirty-bitmap-clear",
+.args_type  = "node:B,name:s",
+.mhandler.cmd_new = qmp_marshal_input_block_dirty_bitmap_clear,
+},
 
 SQMP
 
@@ -1303,6 +1308,25 @@ Example:
   "name": "bitmap0" } }
 <- { "return": {} }
 
+block-dirty-bitmap-clear
+

[Qemu-devel] [PATCH v14 04/19] block: Introduce bdrv_dirty_bitmap_granularity()

[John Snow]

This returns the granularity (in bytes) of dirty bitmap,
which matches the QMP interface and the existing query
interface.

Signed-off-by: John Snow 
Reviewed-by: Max Reitz 
---
 block.c   | 8 ++--
 include/block/block.h | 1 +
 2 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/block.c b/block.c
index 60dbdbd..8b40e38 100644
--- a/block.c
+++ b/block.c
@@ -5440,8 +5440,7 @@ BlockDirtyInfoList 
*bdrv_query_dirty_bitmaps(BlockDriverState *bs)
 BlockDirtyInfo *info = g_new0(BlockDirtyInfo, 1);
 BlockDirtyInfoList *entry = g_new0(BlockDirtyInfoList, 1);
 info->count = bdrv_get_dirty_count(bs, bm);
-info->granularity =
-((uint32_t) BDRV_SECTOR_SIZE << hbitmap_granularity(bm->bitmap));
+info->granularity = bdrv_dirty_bitmap_granularity(bm);
 info->has_name = !!bm->name;
 info->name = g_strdup(bm->name);
 entry->value = info;
@@ -5481,6 +5480,11 @@ uint32_t 
bdrv_get_default_bitmap_granularity(BlockDriverState *bs)
 return granularity;
 }
 
+uint32_t bdrv_dirty_bitmap_granularity(BdrvDirtyBitmap *bitmap)
+{
+return BDRV_SECTOR_SIZE << hbitmap_granularity(bitmap->bitmap);
+}
+
 void bdrv_dirty_iter_init(BlockDriverState *bs,
   BdrvDirtyBitmap *bitmap, HBitmapIter *hbi)
 {
diff --git a/include/block/block.h b/include/block/block.h
index 3589132..41b8418 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -448,6 +448,7 @@ void bdrv_dirty_bitmap_make_anon(BlockDriverState *bs, 
BdrvDirtyBitmap *bitmap);
 void bdrv_release_dirty_bitmap(BlockDriverState *bs, BdrvDirtyBitmap *bitmap);
 BlockDirtyInfoList *bdrv_query_dirty_bitmaps(BlockDriverState *bs);
 uint32_t bdrv_get_default_bitmap_granularity(BlockDriverState *bs);
+uint32_t bdrv_dirty_bitmap_granularity(BdrvDirtyBitmap *bitmap);
 int bdrv_get_dirty(BlockDriverState *bs, BdrvDirtyBitmap *bitmap, int64_t 
sector);
 void bdrv_set_dirty_bitmap(BlockDriverState *bs, BdrvDirtyBitmap *bitmap,
int64_t cur_sector, int nr_sectors);
-- 
1.9.3

[Qemu-devel] [PATCH v14 13/19] block: Ensure consistent bitmap function prototypes

[John Snow]

We often don't need the BlockDriverState for functions
that operate on bitmaps. Remove it.

Signed-off-by: John Snow 
Reviewed-by: Max Reitz 
---
 block.c   | 13 ++---
 block/backup.c|  2 +-
 block/mirror.c| 26 ++
 blockdev.c|  2 +-
 include/block/block.h | 11 +--
 migration/block.c |  7 +++
 6 files changed, 26 insertions(+), 35 deletions(-)

diff --git a/block.c b/block.c
index c1c11aa..f3a6dd4 100644
--- a/block.c
+++ b/block.c
@@ -5391,7 +5391,7 @@ BdrvDirtyBitmap *bdrv_find_dirty_bitmap(BlockDriverState 
*bs, const char *name)
 return NULL;
 }
 
-void bdrv_dirty_bitmap_make_anon(BlockDriverState *bs, BdrvDirtyBitmap *bitmap)
+void bdrv_dirty_bitmap_make_anon(BdrvDirtyBitmap *bitmap)
 {
 assert(!bdrv_dirty_bitmap_frozen(bitmap));
 g_free(bitmap->name);
@@ -5560,7 +5560,7 @@ BlockDirtyInfoList 
*bdrv_query_dirty_bitmaps(BlockDriverState *bs)
 QLIST_FOREACH(bm, &bs->dirty_bitmaps, list) {
 BlockDirtyInfo *info = g_new0(BlockDirtyInfo, 1);
 BlockDirtyInfoList *entry = g_new0(BlockDirtyInfoList, 1);
-info->count = bdrv_get_dirty_count(bs, bm);
+info->count = bdrv_get_dirty_count(bm);
 info->granularity = bdrv_dirty_bitmap_granularity(bm);
 info->has_name = !!bm->name;
 info->name = g_strdup(bm->name);
@@ -5608,20 +5608,19 @@ uint32_t bdrv_dirty_bitmap_granularity(BdrvDirtyBitmap 
*bitmap)
 return BDRV_SECTOR_SIZE << hbitmap_granularity(bitmap->bitmap);
 }
 
-void bdrv_dirty_iter_init(BlockDriverState *bs,
-  BdrvDirtyBitmap *bitmap, HBitmapIter *hbi)
+void bdrv_dirty_iter_init(BdrvDirtyBitmap *bitmap, HBitmapIter *hbi)
 {
 hbitmap_iter_init(hbi, bitmap->bitmap, 0);
 }
 
-void bdrv_set_dirty_bitmap(BlockDriverState *bs, BdrvDirtyBitmap *bitmap,
+void bdrv_set_dirty_bitmap(BdrvDirtyBitmap *bitmap,
int64_t cur_sector, int nr_sectors)
 {
 assert(bdrv_dirty_bitmap_enabled(bitmap));
 hbitmap_set(bitmap->bitmap, cur_sector, nr_sectors);
 }
 
-void bdrv_reset_dirty_bitmap(BlockDriverState *bs, BdrvDirtyBitmap *bitmap,
+void bdrv_reset_dirty_bitmap(BdrvDirtyBitmap *bitmap,
  int64_t cur_sector, int nr_sectors)
 {
 assert(bdrv_dirty_bitmap_enabled(bitmap));
@@ -5667,7 +5666,7 @@ void bdrv_set_dirty_iter(HBitmapIter *hbi, int64_t offset)
 hbitmap_iter_init(hbi, hbi->hb, offset);
 }
 
-int64_t bdrv_get_dirty_count(BlockDriverState *bs, BdrvDirtyBitmap *bitmap)
+int64_t bdrv_get_dirty_count(BdrvDirtyBitmap *bitmap)
 {
 return hbitmap_count(bitmap->bitmap);
 }
diff --git a/block/backup.c b/block/backup.c
index d46c0a3..41bd9af 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -313,7 +313,7 @@ static void coroutine_fn backup_run(void *opaque)
 int64_t last_cluster = -1;
 bool polyrhythmic;
 
-bdrv_dirty_iter_init(bs, job->sync_bitmap, &hbi);
+bdrv_dirty_iter_init(job->sync_bitmap, &hbi);
 /* Does the granularity happen to match our backup cluster size? */
 polyrhythmic = (bdrv_dirty_bitmap_granularity(job->sync_bitmap) !=
 BACKUP_CLUSTER_SIZE);
diff --git a/block/mirror.c b/block/mirror.c
index f89eccf..dcd6f65 100644
--- a/block/mirror.c
+++ b/block/mirror.c
@@ -125,11 +125,9 @@ static void mirror_write_complete(void *opaque, int ret)
 MirrorOp *op = opaque;
 MirrorBlockJob *s = op->s;
 if (ret < 0) {
-BlockDriverState *source = s->common.bs;
 BlockErrorAction action;
 
-bdrv_set_dirty_bitmap(source, s->dirty_bitmap, op->sector_num,
-  op->nb_sectors);
+bdrv_set_dirty_bitmap(s->dirty_bitmap, op->sector_num, op->nb_sectors);
 action = mirror_error_action(s, false, -ret);
 if (action == BLOCK_ERROR_ACTION_REPORT && s->ret >= 0) {
 s->ret = ret;
@@ -143,11 +141,9 @@ static void mirror_read_complete(void *opaque, int ret)
 MirrorOp *op = opaque;
 MirrorBlockJob *s = op->s;
 if (ret < 0) {
-BlockDriverState *source = s->common.bs;
 BlockErrorAction action;
 
-bdrv_set_dirty_bitmap(source, s->dirty_bitmap, op->sector_num,
-  op->nb_sectors);
+bdrv_set_dirty_bitmap(s->dirty_bitmap, op->sector_num, op->nb_sectors);
 action = mirror_error_action(s, true, -ret);
 if (action == BLOCK_ERROR_ACTION_REPORT && s->ret >= 0) {
 s->ret = ret;
@@ -170,10 +166,9 @@ static uint64_t coroutine_fn 
mirror_iteration(MirrorBlockJob *s)
 
 s->sector_num = hbitmap_iter_next(&s->hbi);
 if (s->sector_num < 0) {
-bdrv_dirty_iter_init(source, s->dirty_bitmap, &s->hbi);
+bdrv_dirty_iter_init(s->dirty_bitmap, &s->hbi);
 s->sector_num = hbitmap_iter_next(&s->hbi);
-trace_mirror_restart_iter(s,
-  bdrv_get_dirty_count(source, 
s->dirty_bitmap));
+  

[Qemu-devel] [PATCH v14 08/19] qmp: Add support of "dirty-bitmap" sync mode for drive-backup

[John Snow]

For "dirty-bitmap" sync mode, the block job will iterate through the
given dirty bitmap to decide if a sector needs backup (backup all the
dirty clusters and skip clean ones), just as allocation conditions of
"top" sync mode.

Signed-off-by: Fam Zheng 
Signed-off-by: John Snow 
Reviewed-by: Max Reitz 
---
 block.c   |   9 +++
 block/backup.c| 149 +++---
 block/mirror.c|   4 ++
 blockdev.c|  18 +-
 hmp.c |   3 +-
 include/block/block.h |   1 +
 include/block/block_int.h |   2 +
 qapi/block-core.json  |  13 ++--
 qmp-commands.hx   |   7 ++-
 9 files changed, 172 insertions(+), 34 deletions(-)

diff --git a/block.c b/block.c
index 665ae69..6e3f817 100644
--- a/block.c
+++ b/block.c
@@ -5648,6 +5648,15 @@ static void bdrv_reset_dirty(BlockDriverState *bs, 
int64_t cur_sector,
 }
 }
 
+/**
+ * Advance an HBitmapIter to an arbitrary offset.
+ */
+void bdrv_set_dirty_iter(HBitmapIter *hbi, int64_t offset)
+{
+assert(hbi->hb);
+hbitmap_iter_init(hbi, hbi->hb, offset);
+}
+
 int64_t bdrv_get_dirty_count(BlockDriverState *bs, BdrvDirtyBitmap *bitmap)
 {
 return hbitmap_count(bitmap->bitmap);
diff --git a/block/backup.c b/block/backup.c
index 1c535b1..d46c0a3 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -37,6 +37,8 @@ typedef struct CowRequest {
 typedef struct BackupBlockJob {
 BlockJob common;
 BlockDriverState *target;
+/* bitmap for sync=dirty-bitmap */
+BdrvDirtyBitmap *sync_bitmap;
 MirrorSyncMode sync_mode;
 RateLimit limit;
 BlockdevOnError on_source_error;
@@ -242,6 +244,31 @@ static void backup_complete(BlockJob *job, void *opaque)
 g_free(data);
 }
 
+static bool coroutine_fn yield_and_check(BackupBlockJob *job)
+{
+if (block_job_is_cancelled(&job->common)) {
+return true;
+}
+
+/* we need to yield so that qemu_aio_flush() returns.
+ * (without, VM does not reboot)
+ */
+if (job->common.speed) {
+uint64_t delay_ns = ratelimit_calculate_delay(&job->limit,
+  job->sectors_read);
+job->sectors_read = 0;
+block_job_sleep_ns(&job->common, QEMU_CLOCK_REALTIME, delay_ns);
+} else {
+block_job_sleep_ns(&job->common, QEMU_CLOCK_REALTIME, 0);
+}
+
+if (block_job_is_cancelled(&job->common)) {
+return true;
+}
+
+return false;
+}
+
 static void coroutine_fn backup_run(void *opaque)
 {
 BackupBlockJob *job = opaque;
@@ -254,13 +281,13 @@ static void coroutine_fn backup_run(void *opaque)
 };
 int64_t start, end;
 int ret = 0;
+bool error_is_read;
 
 QLIST_INIT(&job->inflight_reqs);
 qemu_co_rwlock_init(&job->flush_rwlock);
 
 start = 0;
-end = DIV_ROUND_UP(job->common.len / BDRV_SECTOR_SIZE,
-   BACKUP_SECTORS_PER_CLUSTER);
+end = DIV_ROUND_UP(job->common.len, BACKUP_CLUSTER_SIZE);
 
 job->bitmap = hbitmap_alloc(end, 0);
 
@@ -278,28 +305,62 @@ static void coroutine_fn backup_run(void *opaque)
 qemu_coroutine_yield();
 job->common.busy = true;
 }
+} else if (job->sync_mode == MIRROR_SYNC_MODE_DIRTY_BITMAP) {
+/* Dirty Bitmap sync has a slightly different iteration method */
+HBitmapIter hbi;
+int64_t sector;
+int64_t cluster;
+int64_t last_cluster = -1;
+bool polyrhythmic;
+
+bdrv_dirty_iter_init(bs, job->sync_bitmap, &hbi);
+/* Does the granularity happen to match our backup cluster size? */
+polyrhythmic = (bdrv_dirty_bitmap_granularity(job->sync_bitmap) !=
+BACKUP_CLUSTER_SIZE);
+
+/* Find the next dirty /sector/ and copy that /cluster/ */
+while ((sector = hbitmap_iter_next(&hbi)) != -1) {
+cluster = sector / BACKUP_SECTORS_PER_CLUSTER;
+
+/* Fake progress updates for any clusters we skipped,
+ * excluding this current cluster. */
+if (cluster != last_cluster + 1) {
+job->common.offset += ((cluster - last_cluster - 1) *
+   BACKUP_CLUSTER_SIZE);
+}
+
+if (yield_and_check(job)) {
+goto leave;
+}
+
+do {
+ret = backup_do_cow(bs, cluster * BACKUP_SECTORS_PER_CLUSTER,
+BACKUP_SECTORS_PER_CLUSTER, 
&error_is_read);
+if ((ret < 0) &&
+backup_error_action(job, error_is_read, -ret) ==
+BLOCK_ERROR_ACTION_REPORT) {
+goto leave;
+}
+} while (ret < 0);
+
+/* Advance (or rewind) our iterator if we need to. */
+if (polyrhythmic) {
+bdrv_set_dirty_iter(&hbi,
+(cluster + 1) * 
BACKUP_SECTORS_PER_CLUSTER);

[Qemu-devel] [PATCH v14 01/19] qapi: Add optional field "name" to block dirty bitmap

[John Snow]

From: Fam Zheng 

This field will be set for user created dirty bitmap. Also pass in an
error pointer to bdrv_create_dirty_bitmap, so when a name is already
taken on this BDS, it can report an error message. This is not global
check, two BDSes can have dirty bitmap with a common name.

Implemented bdrv_find_dirty_bitmap to find a dirty bitmap by name, will
be used later when other QMP commands want to reference dirty bitmap by
name.

Add bdrv_dirty_bitmap_make_anon. This unsets the name of dirty bitmap.

Signed-off-by: Fam Zheng 
Signed-off-by: John Snow 
Reviewed-by: Max Reitz 
Reviewed-by: Eric Blake 
---
 block.c   | 32 +++-
 block/mirror.c|  2 +-
 include/block/block.h |  7 ++-
 migration/block.c |  2 +-
 qapi/block-core.json  |  4 +++-
 5 files changed, 42 insertions(+), 5 deletions(-)

diff --git a/block.c b/block.c
index 210fd5f..c9c2359 100644
--- a/block.c
+++ b/block.c
@@ -53,6 +53,7 @@
 
 struct BdrvDirtyBitmap {
 HBitmap *bitmap;
+char *name;
 QLIST_ENTRY(BdrvDirtyBitmap) list;
 };
 
@@ -5366,7 +5367,28 @@ bool bdrv_qiov_is_aligned(BlockDriverState *bs, 
QEMUIOVector *qiov)
 return true;
 }
 
-BdrvDirtyBitmap *bdrv_create_dirty_bitmap(BlockDriverState *bs, int 
granularity,
+BdrvDirtyBitmap *bdrv_find_dirty_bitmap(BlockDriverState *bs, const char *name)
+{
+BdrvDirtyBitmap *bm;
+
+assert(name);
+QLIST_FOREACH(bm, &bs->dirty_bitmaps, list) {
+if (bm->name && !strcmp(name, bm->name)) {
+return bm;
+}
+}
+return NULL;
+}
+
+void bdrv_dirty_bitmap_make_anon(BlockDriverState *bs, BdrvDirtyBitmap *bitmap)
+{
+g_free(bitmap->name);
+bitmap->name = NULL;
+}
+
+BdrvDirtyBitmap *bdrv_create_dirty_bitmap(BlockDriverState *bs,
+  int granularity,
+  const char *name,
   Error **errp)
 {
 int64_t bitmap_size;
@@ -5374,6 +5396,10 @@ BdrvDirtyBitmap 
*bdrv_create_dirty_bitmap(BlockDriverState *bs, int granularity,
 
 assert((granularity & (granularity - 1)) == 0);
 
+if (name && bdrv_find_dirty_bitmap(bs, name)) {
+error_setg(errp, "Bitmap already exists: %s", name);
+return NULL;
+}
 granularity >>= BDRV_SECTOR_BITS;
 assert(granularity);
 bitmap_size = bdrv_nb_sectors(bs);
@@ -5384,6 +5410,7 @@ BdrvDirtyBitmap 
*bdrv_create_dirty_bitmap(BlockDriverState *bs, int granularity,
 }
 bitmap = g_new0(BdrvDirtyBitmap, 1);
 bitmap->bitmap = hbitmap_alloc(bitmap_size, ffs(granularity) - 1);
+bitmap->name = g_strdup(name);
 QLIST_INSERT_HEAD(&bs->dirty_bitmaps, bitmap, list);
 return bitmap;
 }
@@ -5395,6 +5422,7 @@ void bdrv_release_dirty_bitmap(BlockDriverState *bs, 
BdrvDirtyBitmap *bitmap)
 if (bm == bitmap) {
 QLIST_REMOVE(bitmap, list);
 hbitmap_free(bitmap->bitmap);
+g_free(bitmap->name);
 g_free(bitmap);
 return;
 }
@@ -5413,6 +5441,8 @@ BlockDirtyInfoList 
*bdrv_query_dirty_bitmaps(BlockDriverState *bs)
 info->count = bdrv_get_dirty_count(bs, bm);
 info->granularity =
 ((int64_t) BDRV_SECTOR_SIZE << hbitmap_granularity(bm->bitmap));
+info->has_name = !!bm->name;
+info->name = g_strdup(bm->name);
 entry->value = info;
 *plist = entry;
 plist = &entry->next;
diff --git a/block/mirror.c b/block/mirror.c
index 4056164..f073ad7 100644
--- a/block/mirror.c
+++ b/block/mirror.c
@@ -703,7 +703,7 @@ static void mirror_start_job(BlockDriverState *bs, 
BlockDriverState *target,
 s->granularity = granularity;
 s->buf_size = MAX(buf_size, granularity);
 
-s->dirty_bitmap = bdrv_create_dirty_bitmap(bs, granularity, errp);
+s->dirty_bitmap = bdrv_create_dirty_bitmap(bs, granularity, NULL, errp);
 if (!s->dirty_bitmap) {
 return;
 }
diff --git a/include/block/block.h b/include/block/block.h
index 321295e..0988b77 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -438,8 +438,13 @@ bool bdrv_qiov_is_aligned(BlockDriverState *bs, 
QEMUIOVector *qiov);
 
 struct HBitmapIter;
 typedef struct BdrvDirtyBitmap BdrvDirtyBitmap;
-BdrvDirtyBitmap *bdrv_create_dirty_bitmap(BlockDriverState *bs, int 
granularity,
+BdrvDirtyBitmap *bdrv_create_dirty_bitmap(BlockDriverState *bs,
+  int granularity,
+  const char *name,
   Error **errp);
+BdrvDirtyBitmap *bdrv_find_dirty_bitmap(BlockDriverState *bs,
+const char *name);
+void bdrv_dirty_bitmap_make_anon(BlockDriverState *bs, BdrvDirtyBitmap 
*bitmap);
 void bdrv_release_dirty_bitmap(BlockDriverState *bs, BdrvDirtyBitmap *bitmap);
 BlockDirtyInfoList *bdrv_query_dirty_bitmaps(BlockDriverState *bs);
 int bdrv_get_dirty(BlockDriverState *

[Qemu-devel] [PATCH v14 03/19] qmp: Add block-dirty-bitmap-add and block-dirty-bitmap-remove

[John Snow]

The new command pair is added to manage a user created dirty bitmap. The
dirty bitmap's name is mandatory and must be unique for the same device,
but different devices can have bitmaps with the same names.

The granularity is an optional field. If it is not specified, we will
choose a default granularity based on the cluster size if available,
clamped to between 4K and 64K to mirror how the 'mirror' code was
already choosing granularity. If we do not have cluster size info
available, we choose 64K. This code has been factored out into a helper
shared with block/mirror.

This patch also introduces the 'block_dirty_bitmap_lookup' helper,
which takes a device name and a dirty bitmap name and validates the
lookup, returning NULL and setting errp if there is a problem with
either field. This helper will be re-used in future patches in this
series.

The types added to block-core.json will be re-used in future patches
in this series, see:
'qapi: Add transaction support to block-dirty-bitmap-{add, enable, disable}'

Signed-off-by: John Snow 
Reviewed-by: Max Reitz 
---
 block.c   |  20 ++
 block/mirror.c|  10 +
 blockdev.c| 100 ++
 include/block/block.h |   1 +
 qapi/block-core.json  |  55 +++
 qmp-commands.hx   |  51 +
 6 files changed, 228 insertions(+), 9 deletions(-)

diff --git a/block.c b/block.c
index 6e08b26..60dbdbd 100644
--- a/block.c
+++ b/block.c
@@ -5461,6 +5461,26 @@ int bdrv_get_dirty(BlockDriverState *bs, BdrvDirtyBitmap 
*bitmap, int64_t sector
 }
 }
 
+/**
+ * Chooses a default granularity based on the existing cluster size,
+ * but clamped between [4K, 64K]. Defaults to 64K in the case that there
+ * is no cluster size information available.
+ */
+uint32_t bdrv_get_default_bitmap_granularity(BlockDriverState *bs)
+{
+BlockDriverInfo bdi;
+uint32_t granularity;
+
+if (bdrv_get_info(bs, &bdi) >= 0 && bdi.cluster_size > 0) {
+granularity = MAX(4096, bdi.cluster_size);
+granularity = MIN(65536, granularity);
+} else {
+granularity = 65536;
+}
+
+return granularity;
+}
+
 void bdrv_dirty_iter_init(BlockDriverState *bs,
   BdrvDirtyBitmap *bitmap, HBitmapIter *hbi)
 {
diff --git a/block/mirror.c b/block/mirror.c
index f8aac33..1cb700e 100644
--- a/block/mirror.c
+++ b/block/mirror.c
@@ -668,15 +668,7 @@ static void mirror_start_job(BlockDriverState *bs, 
BlockDriverState *target,
 MirrorBlockJob *s;
 
 if (granularity == 0) {
-/* Choose the default granularity based on the target file's cluster
- * size, clamped between 4k and 64k.  */
-BlockDriverInfo bdi;
-if (bdrv_get_info(target, &bdi) >= 0 && bdi.cluster_size != 0) {
-granularity = MAX(4096, bdi.cluster_size);
-granularity = MIN(65536, granularity);
-} else {
-granularity = 65536;
-}
+granularity = bdrv_get_default_bitmap_granularity(target);
 }
 
 assert ((granularity & (granularity - 1)) == 0);
diff --git a/blockdev.c b/blockdev.c
index 7d34960..8842e39 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -1173,6 +1173,48 @@ out_aio_context:
 return NULL;
 }
 
+/**
+ * Return a dirty bitmap (if present), after validating
+ * the node reference and bitmap names. Returns NULL on error,
+ * including when the BDS and/or bitmap is not found.
+ */
+static BdrvDirtyBitmap *block_dirty_bitmap_lookup(const char *node,
+  const char *name,
+  BlockDriverState **pbs,
+  Error **errp)
+{
+BlockDriverState *bs;
+BdrvDirtyBitmap *bitmap;
+
+if (!node) {
+error_setg(errp, "Node cannot be NULL");
+return NULL;
+}
+if (!name) {
+error_setg(errp, "Bitmap name cannot be NULL");
+return NULL;
+}
+
+bs = bdrv_lookup_bs(node, node, NULL);
+if (!bs) {
+error_setg(errp, "Node '%s' not found", node);
+return NULL;
+}
+
+/* If caller provided a BDS*, provide the result of that lookup, too. */
+if (pbs) {
+*pbs = bs;
+}
+
+bitmap = bdrv_find_dirty_bitmap(bs, name);
+if (!bitmap) {
+error_setg(errp, "Dirty bitmap '%s' not found", name);
+return NULL;
+}
+
+return bitmap;
+}
+
 /* New and old BlockDriverState structs for atomic group operations */
 
 typedef struct BlkTransactionState BlkTransactionState;
@@ -1953,6 +1995,64 @@ void qmp_block_set_io_throttle(const char *device, 
int64_t bps, int64_t bps_rd,
 aio_context_release(aio_context);
 }
 
+void qmp_block_dirty_bitmap_add(const char *node, const char *name,
+bool has_granularity, uint32_t granularity,
+Error **errp)
+{
+AioContext *aio_context;
+Bl

[Qemu-devel] [PATCH v14 02/19] qmp: Ensure consistent granularity type

[John Snow]

We treat this field with a variety of different types everywhere
in the code. Now it's just uint32_t.

Reviewed-by: Eric Blake 
Reviewed-by: Max Reitz 
Signed-off-by: John Snow 
---
 block.c   | 11 ++-
 block/mirror.c|  4 ++--
 include/block/block.h |  2 +-
 include/block/block_int.h |  2 +-
 qapi/block-core.json  |  2 +-
 5 files changed, 11 insertions(+), 10 deletions(-)

diff --git a/block.c b/block.c
index c9c2359..6e08b26 100644
--- a/block.c
+++ b/block.c
@@ -5387,12 +5387,13 @@ void bdrv_dirty_bitmap_make_anon(BlockDriverState *bs, 
BdrvDirtyBitmap *bitmap)
 }
 
 BdrvDirtyBitmap *bdrv_create_dirty_bitmap(BlockDriverState *bs,
-  int granularity,
+  uint32_t granularity,
   const char *name,
   Error **errp)
 {
 int64_t bitmap_size;
 BdrvDirtyBitmap *bitmap;
+uint32_t sector_granularity;
 
 assert((granularity & (granularity - 1)) == 0);
 
@@ -5400,8 +5401,8 @@ BdrvDirtyBitmap 
*bdrv_create_dirty_bitmap(BlockDriverState *bs,
 error_setg(errp, "Bitmap already exists: %s", name);
 return NULL;
 }
-granularity >>= BDRV_SECTOR_BITS;
-assert(granularity);
+sector_granularity = granularity >> BDRV_SECTOR_BITS;
+assert(sector_granularity);
 bitmap_size = bdrv_nb_sectors(bs);
 if (bitmap_size < 0) {
 error_setg_errno(errp, -bitmap_size, "could not get length of device");
@@ -5409,7 +5410,7 @@ BdrvDirtyBitmap 
*bdrv_create_dirty_bitmap(BlockDriverState *bs,
 return NULL;
 }
 bitmap = g_new0(BdrvDirtyBitmap, 1);
-bitmap->bitmap = hbitmap_alloc(bitmap_size, ffs(granularity) - 1);
+bitmap->bitmap = hbitmap_alloc(bitmap_size, ffs(sector_granularity) - 1);
 bitmap->name = g_strdup(name);
 QLIST_INSERT_HEAD(&bs->dirty_bitmaps, bitmap, list);
 return bitmap;
@@ -5440,7 +5441,7 @@ BlockDirtyInfoList 
*bdrv_query_dirty_bitmaps(BlockDriverState *bs)
 BlockDirtyInfoList *entry = g_new0(BlockDirtyInfoList, 1);
 info->count = bdrv_get_dirty_count(bs, bm);
 info->granularity =
-((int64_t) BDRV_SECTOR_SIZE << hbitmap_granularity(bm->bitmap));
+((uint32_t) BDRV_SECTOR_SIZE << hbitmap_granularity(bm->bitmap));
 info->has_name = !!bm->name;
 info->name = g_strdup(bm->name);
 entry->value = info;
diff --git a/block/mirror.c b/block/mirror.c
index f073ad7..f8aac33 100644
--- a/block/mirror.c
+++ b/block/mirror.c
@@ -656,7 +656,7 @@ static const BlockJobDriver commit_active_job_driver = {
 
 static void mirror_start_job(BlockDriverState *bs, BlockDriverState *target,
  const char *replaces,
- int64_t speed, int64_t granularity,
+ int64_t speed, uint32_t granularity,
  int64_t buf_size,
  BlockdevOnError on_source_error,
  BlockdevOnError on_target_error,
@@ -717,7 +717,7 @@ static void mirror_start_job(BlockDriverState *bs, 
BlockDriverState *target,
 
 void mirror_start(BlockDriverState *bs, BlockDriverState *target,
   const char *replaces,
-  int64_t speed, int64_t granularity, int64_t buf_size,
+  int64_t speed, uint32_t granularity, int64_t buf_size,
   MirrorSyncMode mode, BlockdevOnError on_source_error,
   BlockdevOnError on_target_error,
   BlockCompletionFunc *cb,
diff --git a/include/block/block.h b/include/block/block.h
index 0988b77..abdf8d6 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -439,7 +439,7 @@ bool bdrv_qiov_is_aligned(BlockDriverState *bs, 
QEMUIOVector *qiov);
 struct HBitmapIter;
 typedef struct BdrvDirtyBitmap BdrvDirtyBitmap;
 BdrvDirtyBitmap *bdrv_create_dirty_bitmap(BlockDriverState *bs,
-  int granularity,
+  uint32_t granularity,
   const char *name,
   Error **errp);
 BdrvDirtyBitmap *bdrv_find_dirty_bitmap(BlockDriverState *bs,
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 7ad1950..be99ec0 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -576,7 +576,7 @@ void commit_active_start(BlockDriverState *bs, 
BlockDriverState *base,
  */
 void mirror_start(BlockDriverState *bs, BlockDriverState *target,
   const char *replaces,
-  int64_t speed, int64_t granularity, int64_t buf_size,
+  int64_t speed, uint32_t granularity, int64_t buf_size,
   MirrorSyncMode mode, BlockdevOnError on_source_error,
   BlockdevOnError on_target_error,
   BlockCompletionF

[Qemu-devel] [PATCH v14 05/19] hbitmap: add hbitmap_merge

[John Snow]

We add a bitmap merge operation to assist in error cases
where we wish to combine two bitmaps together.

This is algorithmically O(bits) provided HBITMAP_LEVELS remains
constant. For a full bitmap on a 64bit machine:
sum(bits/64^k, k, 0, HBITMAP_LEVELS) ~= 1.01587 * bits

We may be able to improve running speed for particularly sparse
bitmaps by using iterators, but the running time for dense maps
will be worse.

We present the simpler solution first, and we can refine it later
if needed.

Signed-off-by: John Snow 
Reviewed-by: Max Reitz 
Reviewed-by: Stefan Hajnoczi 
---
 include/qemu/hbitmap.h | 11 +++
 util/hbitmap.c | 32 
 2 files changed, 43 insertions(+)

diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h
index 550d7ce..c19c1cb 100644
--- a/include/qemu/hbitmap.h
+++ b/include/qemu/hbitmap.h
@@ -65,6 +65,17 @@ struct HBitmapIter {
 HBitmap *hbitmap_alloc(uint64_t size, int granularity);
 
 /**
+ * hbitmap_merge:
+ * @a: The bitmap to store the result in.
+ * @b: The bitmap to merge into @a.
+ *
+ * Merge two bitmaps together.
+ * A := A (BITOR) B.
+ * B is left unmodified.
+ */
+bool hbitmap_merge(HBitmap *a, const HBitmap *b);
+
+/**
  * hbitmap_empty:
  * @hb: HBitmap to operate on.
  *
diff --git a/util/hbitmap.c b/util/hbitmap.c
index ab13971..962ff29 100644
--- a/util/hbitmap.c
+++ b/util/hbitmap.c
@@ -395,3 +395,35 @@ HBitmap *hbitmap_alloc(uint64_t size, int granularity)
 hb->levels[0][0] |= 1UL << (BITS_PER_LONG - 1);
 return hb;
 }
+
+/**
+ * Given HBitmaps A and B, let A := A (BITOR) B.
+ * Bitmap B will not be modified.
+ */
+bool hbitmap_merge(HBitmap *a, const HBitmap *b)
+{
+int i, j;
+uint64_t size;
+
+if ((a->size != b->size) || (a->granularity != b->granularity)) {
+return false;
+}
+
+if (hbitmap_count(b) == 0) {
+return true;
+}
+
+/* This merge is O(size), as BITS_PER_LONG and HBITMAP_LEVELS are constant.
+ * It may be possible to improve running times for sparsely populated maps
+ * by using hbitmap_iter_next, but this is suboptimal for dense maps.
+ */
+size = a->size;
+for (i = HBITMAP_LEVELS - 1; i >= 0; i--) {
+size = MAX((size + BITS_PER_LONG - 1) >> BITS_PER_LEVEL, 1);
+for (j = 0; j < size; j++) {
+a->levels[i][j] |= b->levels[i][j];
+}
+}
+
+return true;
+}
-- 
1.9.3

Re: [Qemu-devel] [PATCH 0/3] Support streaming to an intermediate layer

[Eric Blake]

On 02/20/2015 12:05 PM, Alberto Garcia wrote:
> On Fri, Feb 20, 2015 at 10:34:58AM -0700, Eric Blake wrote:
> 
>>> I followed the proposed API from the wiki, which simply adds an
>>> additional 'top' parameter to block-stream specifying the image
>>> that data is written to:
>>
>> How does one learn whether qemu is new enough to support this
>> mode? Until we add QMP introspection, learning whether an optional
>> parameter exists requires attempting the command and seeing a
>> different error depending on whether the argument is recognized.
> 
> Isn't it possible to just check the QEMU version number?

Absolutely not.  Vendors are very likely to backport this to earlier
version numbers.  Feature probes are ALWAYS a better idea than version
probes.

> 
> If not, what's the recommended way to do it?

Several design possibilities, but not all of them feasible at the moment.

1. implement QAPI introspection (we've been dreaming about this since
qemu 1.5 days), then the caller just queries to see if the version of
QMP has the optional 'top' parameter.

2. implement the new feature as a new command (then the existing
'query-commands' becomes an easy probe; but we have code duplication
with existing commands) (on the other hand, if we are going to frame
'stream' in terms of node operations instead of file name operations,
this may be the best idea after all)

3. guarantee that we have sane error messages for bogus commands used as
the probe. If {"execute":"block-stream",
"data":{"device":"no-such","top":"bogus"}} gives a reliable
DeviceNotFound error for qemu that supports optional 'top', and a
reliable GenericError about an unknown argument 'top' in older qemu,
then libvirt could use that as a poor-man's probe for the functionality
existing. (ugly, and only works if you can guarantee a difference in
error type between old and new qemu for a specific bogus command usage)
 Precedent: we do this sort of bogus command call on 'block-commit' to
learn if qemu is new enough to support active commit, and the comments
in the qemu code are explicit that a particular error message must not
be changed because libvirt depends on it.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH 2/3] block: Add QMP support for streaming to an intermediate layer

[Eric Blake]

On 02/20/2015 06:53 AM, Alberto Garcia wrote:
> This adds the 'top' parameter to the 'block-stream' QMP command and
> checks that its value is valid before passing it to stream_start().
> 
> Signed-off-by: Alberto Garcia 
> ---
>  blockdev.c| 19 +++
>  hmp.c |  2 +-
>  include/qapi/qmp/qerror.h |  3 +++
>  qapi/block-core.json  | 18 ++
>  qmp-commands.hx   |  2 +-
>  5 files changed, 34 insertions(+), 10 deletions(-)

> @@ -2123,12 +2125,21 @@ void qmp_block_stream(const char *device,
>  aio_context = bdrv_get_aio_context(bs);
>  aio_context_acquire(aio_context);
>  
> -if (bdrv_op_is_blocked(bs, BLOCK_OP_TYPE_STREAM, errp)) {
> +if (has_top) {
> +top_bs = bdrv_find_backing_image(bs, top);
> +if (top_bs == NULL) {
> +error_set(errp, QERR_TOP_NOT_FOUND, top);
> +goto out;
> +}

If I understand correctly, bdrv_find_backing_image has problems for
backing nodes that don't have a file name.  Given our shift towards node
names, I think we really want to target node names rather than file
names when specifying which node we will use as the top bound receiving
the stream operations.


> +++ b/include/qapi/qmp/qerror.h
> @@ -127,6 +127,9 @@ void qerror_report_err(Error *err);
>  #define QERR_SET_PASSWD_FAILED \
>  ERROR_CLASS_GENERIC_ERROR, "Could not set password"
>  
> +#define QERR_TOP_NOT_FOUND \
> +ERROR_CLASS_GENERIC_ERROR, "Top '%s' not found"
> +

Please don't.  Just use error_setg() at the right place with the direct
message (existing QERR_ macros are a legacy holdover, and we shouldn't
be creating more of them).

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH] Makefile.target: set icon for binary file on Mac OS X

[Programmingkid]

On Feb 20, 2015, at 12:56 PM, Paolo Bonzini wrote:

> 
> 
> On 20/02/2015 18:32, Programmingkid wrote:
>>> Ok, so I'll apply v3 to my tree as soon as I get a Tested-by.
>>> Please take a look into providing a .rsrc file with larger-sized
>>> icons (I think you can add more than one to a single .rsrc file?)
>>> and, when you do that, document in pc-bios/README how the
>>> resources were built.
>> 
>> When I was inspecting the icon using the genie feature of Mac OS X's
>> dock, the icon looked very sharp at full size, so I didn't see a
>> problem with it. What size do you have in mind?
> 
> I was thinking of 16x16, 32x32, 48x48 and 128x128.

What advantages do you think this would give us? Doesn't Mac OS X already 
resize the icon for you?

Re: [Qemu-devel] [RFC PATCH v2 04/15] cpu-model/s390: Introduce S390 CPU models

[Alexander Graf]

On 20.02.15 20:43, Michael Mueller wrote:
> On Fri, 20 Feb 2015 18:50:20 +0100
> Alexander Graf  wrote:
> 
>>
>>
>>
>>> Am 20.02.2015 um 18:37 schrieb Michael Mueller :
>>>
>>> On Fri, 20 Feb 2015 17:57:52 +0100
>>> Alexander Graf  wrote:
>>>
 Because all CPUs we have in our list only expose 128 bits?
>>>
>>> Here a STFLE result on a EC12 GA2, already more than 128 bits... Is that 
>>> model on the list?
>>
>> If that model has 3 elements, yes, the array should span 3.
>>
>> I hope it's in the list. Every model wecare about should be, no?
>>
> 
> On my list? Yes!
> 
>>>
>>> [mimu@p57lp59 s390xfac]$ ./s390xfac -b
>>> fac[0] = 0xfbfbfcfff840
>>> fac[1] = 0xffde
>>> fac[2] = 0x1800

> I want to have this independent from a future machine of the z/Arch. The 
> kernel stores the
> full facility set, KVM does and there is no good reason for QEMU not to 
> do. If other
> accelerators decide to just implement 64 or 128 bits of facilities that's 
> ok...  

 So you want to support CPUs that are not part of the list?
>>>
>>> The architecture at least defines more than 2 or 3. Do you want me to limit 
>>> it to an arbitrary
>>> size?. Only in QEMU or also in the KVM interface?
>>
>> Only internally in QEMU. The kvm interface should definitely be as big as 
>> the spec allows!
> 
> Right, now we're on the same page again. That can be taken in consideration. 
> ... Although it's
> just and optimization. :-)

Yeah. You could also consider using the QEMU built-in bitmap type and
functions and just convert from there. That would give you native
support for bit values > 64.


Alex

Re: [Qemu-devel] [PATCH 3/3] target-ppc: use separate indices for various translation modes

[Richard Henderson]

On 02/20/2015 09:57 AM, Paolo Bonzini wrote:
> PowerPC TCG flushes the TLB on every IR/DR change, which basically
> means on every user<->kernel context switch.  Encode IR/DR in the
> MMU index.
> 
> This brings the number of TLB flushes down from ~90 to ~5
> for starting up the Debian installer, which is in line with x86
> and gives a ~10% performance improvement.
> 
> Signed-off-by: Paolo Bonzini 
> ---
>  target-ppc/cpu.h | 12 +++-
>  target-ppc/excp_helper.c |  3 ---
>  target-ppc/helper_regs.h | 15 +--
>  3 files changed, 16 insertions(+), 14 deletions(-)

I'm not an expert on ppc memory management, but it certainly LGTM.


r~

Re: [Qemu-devel] [PATCH 2/3] softmmu: support up to 12 MMU modes

[Richard Henderson]

On 02/20/2015 09:57 AM, Paolo Bonzini wrote:
> + * TCG will have to generate an operand as large as the distance between
> + * tlb_table[0][0] and the tlb_table[NB_MMU_MODES - 1][0].addend.  For

Nit: the distance is from the start of env, i.e.

  offsetof(CPUArchState, tlb_table[mem_index][0].addend)

not from the start of the table.  Not that this particularly invalidates the
rest of your reasoning, but the start of the tlb_table is usually at a
non-trivial (though not huge) offset from the start of CPUArchState.

Otherwise, it looks good.


r~

Re: [Qemu-devel] [RFC PATCH v2 10/15] cpu-model/s390: Add cpu class initialization routines

[Michael Mueller]

On Fri, 20 Feb 2015 20:21:45 +0100
Alexander Graf  wrote:

> 
> 
> 
> > Am 20.02.2015 um 19:59 schrieb Michael Mueller :
> > 
> > On Fri, 20 Feb 2015 10:11:55 -0800
> > Richard Henderson  wrote:
> > 
> >>> +static inline uint64_t big_endian_bit(unsigned long nr)
> >>> +{
> >>> +return 1ul << (BITS_PER_LONG - (nr % BITS_PER_LONG));
> >>> +};  
> >> 
> >> This is buggy.  NR=0 should map to 63, not 64.
> > 
> > I'm sure I was asked to replace my constant 64 and 63 with that defines and 
> > at the end I
> > messed it up... :-(
> > 
> >> 
> >>> +return !!(*ptr & big_endian_bit(nr));  
> >> 
> >> Personally I dislike !! as an idiom.  Given that big_endian_bit isn't used
> >> anywhere else, can we integrate it and change this to
> >> 
> >> static inline int test_facility(unsigned long nr, uint64_t *fac_list)
> >> {
> >>  unsigned long word = nr / BITS_PER_LONG;
> >>  unsigned long be_bit = 63 - (nr % BITS_PER_LONG);
> >>  return (fac_list[word] >> be_bit) & 1;
> >> }
> > 
> > Yes, I just use it in this context. I will integrate your version.
> > 
> > BTW I changed the whole facility defining code to be generated by an 
> > external helper at
> > compile time. That is more simple and safe to change. I will send it with 
> > v3. See attachment
> > for an example of the generated header file.
> 
> Please make sure to use ULL with constants and uint64_t on variables. Long is 
> almost always
> wrong in QEMU.

yep

> 
> Alex
> 
> > 
> > Thanks,
> > Michael
> > 
> > 
> 

Re: [Qemu-devel] [RFC PATCH v2 04/15] cpu-model/s390: Introduce S390 CPU models

[Michael Mueller]

On Fri, 20 Feb 2015 18:50:20 +0100
Alexander Graf  wrote:

> 
> 
> 
> > Am 20.02.2015 um 18:37 schrieb Michael Mueller :
> > 
> > On Fri, 20 Feb 2015 17:57:52 +0100
> > Alexander Graf  wrote:
> > 
> >> Because all CPUs we have in our list only expose 128 bits?
> > 
> > Here a STFLE result on a EC12 GA2, already more than 128 bits... Is that 
> > model on the list?
> 
> If that model has 3 elements, yes, the array should span 3.
> 
> I hope it's in the list. Every model wecare about should be, no?
> 

On my list? Yes!

> > 
> > [mimu@p57lp59 s390xfac]$ ./s390xfac -b
> > fac[0] = 0xfbfbfcfff840
> > fac[1] = 0xffde
> > fac[2] = 0x1800
> >> 
> >>> I want to have this independent from a future machine of the z/Arch. The 
> >>> kernel stores the
> >>> full facility set, KVM does and there is no good reason for QEMU not to 
> >>> do. If other
> >>> accelerators decide to just implement 64 or 128 bits of facilities that's 
> >>> ok...  
> >> 
> >> So you want to support CPUs that are not part of the list?
> > 
> > The architecture at least defines more than 2 or 3. Do you want me to limit 
> > it to an arbitrary
> > size?. Only in QEMU or also in the KVM interface?
> 
> Only internally in QEMU. The kvm interface should definitely be as big as the 
> spec allows!

Right, now we're on the same page again. That can be taken in consideration. 
... Although it's
just and optimization. :-)

Michael

> 
> Alex
> 
> > 
> > Thanks
> > Michael
> > 
> 

Re: [Qemu-devel] [PATCH 1/3] tcg: add TCG_TARGET_TLB_DISPLACEMENT_BITS

[Richard Henderson]

On 02/20/2015 09:57 AM, Paolo Bonzini wrote:
> diff --git a/tcg/i386/tcg-target.h b/tcg/i386/tcg-target.h
> index 7a9980e..8ba977a 100644
> --- a/tcg/i386/tcg-target.h
> +++ b/tcg/i386/tcg-target.h
> @@ -25,6 +25,7 @@
>  #define TCG_TARGET_I386 1
>  
>  #define TCG_TARGET_INSN_UNIT_SIZE  1
> +#define TCG_TARGET_TLB_DISPLACEMENT_BITS 32

31.

Positive displacements only in 64-bit mode.  I don't think it's worth
conditionalizing this for 32-bit, since we can't actually allocate 2G of TLBs
and then actually accomplish anything.  ;-)

> diff --git a/tcg/mips/tcg-target.h b/tcg/mips/tcg-target.h
> index c88a1c9..f5ba52c 100644
> --- a/tcg/mips/tcg-target.h
> +++ b/tcg/mips/tcg-target.h
> @@ -27,6 +27,7 @@
>  #define TCG_TARGET_MIPS 1
>  
>  #define TCG_TARGET_INSN_UNIT_SIZE 4
> +#define TCG_TARGET_TLB_DISPLACEMENT_BITS 16
>  #define TCG_TARGET_NB_REGS 32
>  
>  typedef enum {
> diff --git a/tcg/ppc/tcg-target.h b/tcg/ppc/tcg-target.h
> index 32ac442..7ce7048 100644
> --- a/tcg/ppc/tcg-target.h
> +++ b/tcg/ppc/tcg-target.h
> @@ -32,6 +32,7 @@
>  
>  #define TCG_TARGET_NB_REGS 32
>  #define TCG_TARGET_INSN_UNIT_SIZE 4
> +#define TCG_TARGET_TLB_DISPLACEMENT_BITS 16

Close enough, since the BUILD_BUG_ON should still catch this out if somehow the
size is within that last 16 bytes of 64k.


r~

Re: [Qemu-devel] [RFC PATCH v2 13/15] cpu-model/s390: Add processor property routines

[Michael Mueller]

On Fri, 20 Feb 2015 18:00:19 +0100
Alexander Graf  wrote:

> > So above s390_set/get_processor_props() the code is accelerator 
> > independent.  
> 
> Any particular reason you can't do it like PPC?

That seems to be a short question... and when I started one year ago, I 
oriented myself on
the PPC version and I'm also willing to revisit it but I can't give you a quick 
answer different
from no currently to that.

There are no PVRs for s390x CPUs and thus I came up with "pseudo PVRs":

/*
 * bits 0-7   : CMOS generation
 * bits 8-9   : reserved
 * bits 10-11 : machine class 0=unknown 1=EC 2=BC
 * bits 12-15 : GA
 * bits 16-31 : machine type
 *
 * note: bits are named according to s390
 *   architecture specific endienness
 */
enum {
CPU_S390_2064_GA1 = 0x07112064,
CPU_S390_2064_GA2 = 0x07122064,
CPU_S390_2064_GA3 = 0x07132064,
CPU_S390_2066_GA1 = 0x07212066,
CPU_S390_2084_GA1 = 0x08112084,
CPU_S390_2084_GA2 = 0x08122084,
CPU_S390_2084_GA3 = 0x08132084,
CPU_S390_2084_GA4 = 0x08142084,
CPU_S390_2084_GA5 = 0x08152084,
CPU_S390_2086_GA1 = 0x08212086,
CPU_S390_2086_GA2 = 0x08222086,
CPU_S390_2086_GA3 = 0x08232086,
CPU_S390_2094_GA1 = 0x09112094,
CPU_S390_2094_GA2 = 0x09122094,
CPU_S390_2094_GA3 = 0x09132094,
CPU_S390_2096_GA1 = 0x09212096,
CPU_S390_2096_GA2 = 0x09222096,
CPU_S390_2097_GA1 = 0x0a112097,
CPU_S390_2097_GA2 = 0x0a122097,
CPU_S390_2097_GA3 = 0x0a132097,
CPU_S390_2098_GA1 = 0x0a212098,
CPU_S390_2098_GA2 = 0x0a222098,
CPU_S390_2817_GA1 = 0x0b112817,
CPU_S390_2817_GA2 = 0x0b122817,
CPU_S390_2818_GA1 = 0x0b212818,
CPU_S390_2827_GA1 = 0x0c112827,
CPU_S390_2827_GA2 = 0x0c122827,
CPU_S390_2828_GA1 = 0x0c212828,
CPU_S390_2964_GA1 = 0x0d112964,
};

And initially I had a version that was limiting the accelerator to be able to 
implement just them
with all their properties encapsulated in the a accelerator as well. After 
identifying the real
processor related attributes defining the model, I changed the interface such 
that KVM or
other accelerators give hints what it is able to support in dependency of the 
current code
version and the hosting machine and let QEMU decide how to set these attributes
(cpuid,ibc,fac_list). Thus I think the implementation is now quite open and 
easily adoptable also
for TCG and possibly others as well. Eventually the integration and also some 
trigger points of
my code are to adjust. So coming back to your question, the answer is still no 
for the whole item
but eventually yes if you have limited it to the s390_set/get_processor_props() 
triggers. But I
have to look into it first again. I will do that when I'm back on Tuesday 
morning.

Thanks and have a nice WE
Michael

[Qemu-devel] [PATCH 4/4] scsi: Convert remaining PCI HBAs to realize()

[Markus Armbruster]

These are "am53c974", "dc390", "lsi53c895a", "lsi53c810", "megasas",
"megasas-gen2".

Signed-off-by: Markus Armbruster 
---
 hw/scsi/esp-pci.c| 30 +++---
 hw/scsi/lsi53c895a.c | 14 +++---
 hw/scsi/megasas.c| 14 +++---
 3 files changed, 17 insertions(+), 41 deletions(-)

diff --git a/hw/scsi/esp-pci.c b/hw/scsi/esp-pci.c
index a75fcfa..8d2242d 100644
--- a/hw/scsi/esp-pci.c
+++ b/hw/scsi/esp-pci.c
@@ -28,7 +28,6 @@
 #include "hw/scsi/esp.h"
 #include "trace.h"
 #include "qemu/log.h"
-#include "qapi/qmp/qerror.h"
 
 #define TYPE_AM53C974_DEVICE "am53c974"
 
@@ -343,13 +342,12 @@ static const struct SCSIBusInfo esp_pci_scsi_info = {
 .cancel = esp_request_cancelled,
 };
 
-static int esp_pci_scsi_init(PCIDevice *dev)
+static void esp_pci_scsi_realize(PCIDevice *dev, Error **errp)
 {
 PCIESPState *pci = PCI_ESP(dev);
 DeviceState *d = DEVICE(dev);
 ESPState *s = &pci->esp;
 uint8_t *pci_conf;
-Error *err = NULL;
 
 pci_conf = dev->config;
 
@@ -368,14 +366,8 @@ static int esp_pci_scsi_init(PCIDevice *dev)
 
 scsi_bus_new(&s->bus, sizeof(s->bus), d, &esp_pci_scsi_info, NULL);
 if (!d->hotplugged) {
-scsi_bus_legacy_handle_cmdline(&s->bus, &err);
-if (err != NULL) {
-qerror_report_err(err);
-error_free(err);
-return -1;
-}
+scsi_bus_legacy_handle_cmdline(&s->bus, errp);
 }
-return 0;
 }
 
 static void esp_pci_scsi_uninit(PCIDevice *d)
@@ -390,7 +382,7 @@ static void esp_pci_class_init(ObjectClass *klass, void 
*data)
 DeviceClass *dc = DEVICE_CLASS(klass);
 PCIDeviceClass *k = PCI_DEVICE_CLASS(klass);
 
-k->init = esp_pci_scsi_init;
+k->realize = esp_pci_scsi_realize;
 k->exit = esp_pci_scsi_uninit;
 k->vendor_id = PCI_VENDOR_ID_AMD;
 k->device_id = PCI_DEVICE_ID_AMD_SCSI;
@@ -468,17 +460,19 @@ static void dc390_write_config(PCIDevice *dev,
 }
 }
 
-static int dc390_scsi_init(PCIDevice *dev)
+static void dc390_scsi_realize(PCIDevice *dev, Error **errp)
 {
 DC390State *pci = DC390(dev);
+Error *err = NULL;
 uint8_t *contents;
 uint16_t chksum = 0;
-int i, ret;
+int i;
 
 /* init base class */
-ret = esp_pci_scsi_init(dev);
-if (ret < 0) {
-return ret;
+esp_pci_scsi_realize(dev, &err);
+if (err) {
+error_propagate(errp, err);
+return;
 }
 
 /* EEPROM */
@@ -505,8 +499,6 @@ static int dc390_scsi_init(PCIDevice *dev)
 chksum = 0x1234 - chksum;
 contents[EE_CHKSUM1] = chksum & 0xff;
 contents[EE_CHKSUM2] = chksum >> 8;
-
-return 0;
 }
 
 static void dc390_class_init(ObjectClass *klass, void *data)
@@ -514,7 +506,7 @@ static void dc390_class_init(ObjectClass *klass, void *data)
 DeviceClass *dc = DEVICE_CLASS(klass);
 PCIDeviceClass *k = PCI_DEVICE_CLASS(klass);
 
-k->init = dc390_scsi_init;
+k->realize = dc390_scsi_realize;
 k->config_read = dc390_read_config;
 k->config_write = dc390_write_config;
 set_bit(DEVICE_CATEGORY_STORAGE, dc->categories);
diff --git a/hw/scsi/lsi53c895a.c b/hw/scsi/lsi53c895a.c
index 4ffab03..c5b0cc5 100644
--- a/hw/scsi/lsi53c895a.c
+++ b/hw/scsi/lsi53c895a.c
@@ -19,7 +19,6 @@
 #include "hw/pci/pci.h"
 #include "hw/scsi/scsi.h"
 #include "sysemu/dma.h"
-#include "qapi/qmp/qerror.h"
 
 //#define DEBUG_LSI
 //#define DEBUG_LSI_REG
@@ -2089,12 +2088,11 @@ static const struct SCSIBusInfo lsi_scsi_info = {
 .cancel = lsi_request_cancelled
 };
 
-static int lsi_scsi_init(PCIDevice *dev)
+static void lsi_scsi_realize(PCIDevice *dev, Error **errp)
 {
 LSIState *s = LSI53C895A(dev);
 DeviceState *d = DEVICE(dev);
 uint8_t *pci_conf;
-Error *err = NULL;
 
 pci_conf = dev->config;
 
@@ -2117,14 +2115,8 @@ static int lsi_scsi_init(PCIDevice *dev)
 
 scsi_bus_new(&s->bus, sizeof(s->bus), d, &lsi_scsi_info, NULL);
 if (!d->hotplugged) {
-scsi_bus_legacy_handle_cmdline(&s->bus, &err);
-if (err != NULL) {
-qerror_report_err(err);
-error_free(err);
-return -1;
-}
+scsi_bus_legacy_handle_cmdline(&s->bus, errp);
 }
-return 0;
 }
 
 static void lsi_class_init(ObjectClass *klass, void *data)
@@ -2132,7 +2124,7 @@ static void lsi_class_init(ObjectClass *klass, void *data)
 DeviceClass *dc = DEVICE_CLASS(klass);
 PCIDeviceClass *k = PCI_DEVICE_CLASS(klass);
 
-k->init = lsi_scsi_init;
+k->realize = lsi_scsi_realize;
 k->vendor_id = PCI_VENDOR_ID_LSI_LOGIC;
 k->device_id = PCI_DEVICE_ID_LSI_53C895A;
 k->class_id = PCI_CLASS_STORAGE_SCSI;
diff --git a/hw/scsi/megasas.c b/hw/scsi/megasas.c
index 69ffdbd..bf83b65 100644
--- a/hw/scsi/megasas.c
+++ b/hw/scsi/megasas.c
@@ -28,7 +28,6 @@
 #include "hw/scsi/scsi.h"
 #include "block/scsi.h"
 #include "trace.h"
-#include "qapi/qmp/qerror.h"
 
 #include "mfi.h"
 
@@ -2321,14 +2320,13 @@ static const struct SCSIBusInfo megasas_scsi_in

[Qemu-devel] [PATCH 2/4] hw: Propagate errors through qdev_prop_set_drive()

[Markus Armbruster]

Three kinds of callers:

1. On failure, report the error and abort

   Passing &error_abort does the job.  No functional change.

2. On failure, report the error and exit()

   This is qdev_prop_set_drive_nofail().  Error reporting moves from
   qdev_prop_set_drive() to its caller.  Because hiding away the error
   in the monitor right before exit() isn't helpful, replace
   qerror_report_err() by error_report_err().  Shouldn't make a
   difference, because qdev_prop_set_drive_nofail() should never be
   used in QMP context.

3. On failure, report the error and recover

   This is usb_msd_init() and scsi_bus_legacy_add_drive().  Error
   reporting and freeing the error object moves from
   qdev_prop_set_drive() to its callers.

   Because usb_msd_init() can't run in QMP context, replace
   qerror_report_err() by error_report_err() there.

   No functional change.

   scsi_bus_legacy_add_drive() calling qerror_report_err() is of
   course inappropriate, but this commit merely makes it more obvious.
   The next one will clean it up.

Signed-off-by: Markus Armbruster 
---
 hw/arm/vexpress.c|  6 +++---
 hw/arm/virt.c|  6 +++---
 hw/block/pflash_cfi01.c  |  4 ++--
 hw/block/pflash_cfi02.c  |  4 ++--
 hw/core/qdev-properties-system.c | 22 ++
 hw/scsi/scsi-bus.c   |  6 +-
 hw/usb/dev-storage.c |  7 +--
 include/hw/qdev-properties.h |  4 ++--
 8 files changed, 32 insertions(+), 27 deletions(-)

diff --git a/hw/arm/vexpress.c b/hw/arm/vexpress.c
index 5933454..8496c16 100644
--- a/hw/arm/vexpress.c
+++ b/hw/arm/vexpress.c
@@ -515,9 +515,9 @@ static pflash_t *ve_pflash_cfi01_register(hwaddr base, 
const char *name,
 {
 DeviceState *dev = qdev_create(NULL, "cfi.pflash01");
 
-if (di && qdev_prop_set_drive(dev, "drive",
-  blk_by_legacy_dinfo(di))) {
-abort();
+if (di) {
+qdev_prop_set_drive(dev, "drive", blk_by_legacy_dinfo(di),
+&error_abort);
 }
 
 qdev_prop_set_uint32(dev, "num-blocks",
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 69f51ac..93b7605 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -522,9 +522,9 @@ static void create_one_flash(const char *name, hwaddr 
flashbase,
 DeviceState *dev = qdev_create(NULL, "cfi.pflash01");
 const uint64_t sectorlength = 256 * 1024;
 
-if (dinfo && qdev_prop_set_drive(dev, "drive",
- blk_by_legacy_dinfo(dinfo))) {
-abort();
+if (dinfo) {
+qdev_prop_set_drive(dev, "drive", blk_by_legacy_dinfo(dinfo),
+&error_abort);
 }
 
 qdev_prop_set_uint32(dev, "num-blocks", flashsize / sectorlength);
diff --git a/hw/block/pflash_cfi01.c b/hw/block/pflash_cfi01.c
index 89d380e..d282695 100644
--- a/hw/block/pflash_cfi01.c
+++ b/hw/block/pflash_cfi01.c
@@ -969,8 +969,8 @@ pflash_t *pflash_cfi01_register(hwaddr base,
 {
 DeviceState *dev = qdev_create(NULL, TYPE_CFI_PFLASH01);
 
-if (blk && qdev_prop_set_drive(dev, "drive", blk)) {
-abort();
+if (blk) {
+qdev_prop_set_drive(dev, "drive", blk, &error_abort);
 }
 qdev_prop_set_uint32(dev, "num-blocks", nb_blocs);
 qdev_prop_set_uint64(dev, "sector-length", sector_len);
diff --git a/hw/block/pflash_cfi02.c b/hw/block/pflash_cfi02.c
index 389b4aa..074a005 100644
--- a/hw/block/pflash_cfi02.c
+++ b/hw/block/pflash_cfi02.c
@@ -773,8 +773,8 @@ pflash_t *pflash_cfi02_register(hwaddr base,
 {
 DeviceState *dev = qdev_create(NULL, TYPE_CFI_PFLASH02);
 
-if (blk && qdev_prop_set_drive(dev, "drive", blk)) {
-abort();
+if (blk) {
+qdev_prop_set_drive(dev, "drive", blk, &error_abort);
 }
 qdev_prop_set_uint32(dev, "num-blocks", nb_blocs);
 qdev_prop_set_uint32(dev, "sector-length", sector_len);
diff --git a/hw/core/qdev-properties-system.c b/hw/core/qdev-properties-system.c
index a2e44bd..c413226 100644
--- a/hw/core/qdev-properties-system.c
+++ b/hw/core/qdev-properties-system.c
@@ -341,27 +341,25 @@ PropertyInfo qdev_prop_vlan = {
 .set   = set_vlan,
 };
 
-int qdev_prop_set_drive(DeviceState *dev, const char *name,
-BlockBackend *value)
+void qdev_prop_set_drive(DeviceState *dev, const char *name,
+ BlockBackend *value, Error **errp)
 {
-Error *err = NULL;
-object_property_set_str(OBJECT(dev),
-value ? blk_name(value) : "", name, &err);
-if (err) {
-qerror_report_err(err);
-error_free(err);
-return -1;
-}
-return 0;
+object_property_set_str(OBJECT(dev), value ? blk_name(value) : "",
+name, errp);
 }
 
 void qdev_prop_set_drive_nofail(DeviceState *dev, const char *name,
 BlockBackend *value)
 {
-if (qdev_prop_set_drive(dev, name, value) < 0) {
+Error *err = NULL;
+
+qdev

Re: [Qemu-devel] [RFC PATCH v2 10/15] cpu-model/s390: Add cpu class initialization routines

[Alexander Graf]

> Am 20.02.2015 um 19:59 schrieb Michael Mueller :
> 
> On Fri, 20 Feb 2015 10:11:55 -0800
> Richard Henderson  wrote:
> 
>>> +static inline uint64_t big_endian_bit(unsigned long nr)
>>> +{
>>> +return 1ul << (BITS_PER_LONG - (nr % BITS_PER_LONG));
>>> +};  
>> 
>> This is buggy.  NR=0 should map to 63, not 64.
> 
> I'm sure I was asked to replace my constant 64 and 63 with that defines and 
> at the end I messed
> it up... :-(
> 
>> 
>>> +return !!(*ptr & big_endian_bit(nr));  
>> 
>> Personally I dislike !! as an idiom.  Given that big_endian_bit isn't used
>> anywhere else, can we integrate it and change this to
>> 
>> static inline int test_facility(unsigned long nr, uint64_t *fac_list)
>> {
>>  unsigned long word = nr / BITS_PER_LONG;
>>  unsigned long be_bit = 63 - (nr % BITS_PER_LONG);
>>  return (fac_list[word] >> be_bit) & 1;
>> }
> 
> Yes, I just use it in this context. I will integrate your version.
> 
> BTW I changed the whole facility defining code to be generated by an external 
> helper at compile
> time. That is more simple and safe to change. I will send it with v3. See 
> attachment for an
> example of the generated header file.

Please make sure to use ULL with constants and uint64_t on variables. Long is 
almost always wrong in QEMU.

Alex

> 
> Thanks,
> Michael
> 
> 

[Qemu-devel] [PATCH 1/4] scsi: Clean up duplicated error in legacy if=scsi code

[Markus Armbruster]

Commit a818a4b changed scsi_bus_legacy_handle_cmdline() to report
errors from scsi_bus_legacy_add_drive() with error_report() in
addition to returning them.  That's inappropriate.

Two kinds of callers:

1. realize methods (devices "esp" and "virtio-scsi-device")

   The error object gets passed up the call chain until it gets
   reported again and freed.

   Example:

   $ qemu-system-arm -M virt -S -display none \
   > -drive if=scsi,id=foo,bus=1,file=tmp.qcow2 \
   > -device nec-usb-xhci -device usb-storage,drive=foo \
   > -device virtio-scsi-pci
   qemu-system-arm: -drive if=scsi,id=foo,bus=1,file=tmp.qcow2: Property 
'scsi-disk.drive' can't take value 'foo', it's in use
   qemu-system-arm: -drive if=scsi,id=foo,bus=1,file=tmp.qcow2: Setting drive 
property failed
   qemu-system-arm: -device virtio-scsi-pci: Setting drive property failed
   qemu-system-arm: -device virtio-scsi-pci: Device initialization failed
   qemu-system-arm: -device virtio-scsi-pci: Device 'virtio-scsi-pci' could not 
be initialized

   The second message in this error cascade comes from
   scsi_bus_legacy_handle_cmdline().  The error object then gets
   passed up to the qdev_init() called from
   virtio_scsi_pci_init_pci(), which reports it again.

2. init methods (devices "am53c974", "dc390", "lsi53c895a",
   "lsi53c810", "megasas", "megasas-gen2", "spapr-vscsi")

   init methods need to report their errors with qerror_report().
   These don't.  The inappropriate error_report() papers over the bug.

   error_report() isn't the same as qerror_report() in QMP context,
   but this can't actually happen: QMP can still only hot-plug, and
   callers call scsi_bus_legacy_handle_cmdline() only on cold-plug.
   Except for sysbus_esp_realize(), but that can't be hot-plugged at
   all, as far as I can tell.

Fix the init methods and drop the inappropriate error_report() in
scsi_bus_legacy_handle_cmdline().

Signed-off-by: Markus Armbruster 
---
 hw/scsi/esp-pci.c | 2 ++
 hw/scsi/lsi53c895a.c  | 3 ++-
 hw/scsi/megasas.c | 2 ++
 hw/scsi/scsi-bus.c| 1 -
 hw/scsi/spapr_vscsi.c | 2 ++
 5 files changed, 8 insertions(+), 2 deletions(-)

diff --git a/hw/scsi/esp-pci.c b/hw/scsi/esp-pci.c
index 00b7297..a75fcfa 100644
--- a/hw/scsi/esp-pci.c
+++ b/hw/scsi/esp-pci.c
@@ -28,6 +28,7 @@
 #include "hw/scsi/esp.h"
 #include "trace.h"
 #include "qemu/log.h"
+#include "qapi/qmp/qerror.h"
 
 #define TYPE_AM53C974_DEVICE "am53c974"
 
@@ -369,6 +370,7 @@ static int esp_pci_scsi_init(PCIDevice *dev)
 if (!d->hotplugged) {
 scsi_bus_legacy_handle_cmdline(&s->bus, &err);
 if (err != NULL) {
+qerror_report_err(err);
 error_free(err);
 return -1;
 }
diff --git a/hw/scsi/lsi53c895a.c b/hw/scsi/lsi53c895a.c
index db7d4b8..4ffab03 100644
--- a/hw/scsi/lsi53c895a.c
+++ b/hw/scsi/lsi53c895a.c
@@ -19,7 +19,7 @@
 #include "hw/pci/pci.h"
 #include "hw/scsi/scsi.h"
 #include "sysemu/dma.h"
-#include "qemu/error-report.h"
+#include "qapi/qmp/qerror.h"
 
 //#define DEBUG_LSI
 //#define DEBUG_LSI_REG
@@ -2119,6 +2119,7 @@ static int lsi_scsi_init(PCIDevice *dev)
 if (!d->hotplugged) {
 scsi_bus_legacy_handle_cmdline(&s->bus, &err);
 if (err != NULL) {
+qerror_report_err(err);
 error_free(err);
 return -1;
 }
diff --git a/hw/scsi/megasas.c b/hw/scsi/megasas.c
index 4852237..69ffdbd 100644
--- a/hw/scsi/megasas.c
+++ b/hw/scsi/megasas.c
@@ -28,6 +28,7 @@
 #include "hw/scsi/scsi.h"
 #include "block/scsi.h"
 #include "trace.h"
+#include "qapi/qmp/qerror.h"
 
 #include "mfi.h"
 
@@ -2409,6 +2410,7 @@ static int megasas_scsi_init(PCIDevice *dev)
 if (!d->hotplugged) {
 scsi_bus_legacy_handle_cmdline(&s->bus, &err);
 if (err != NULL) {
+qerror_report_err(err);
 error_free(err);
 return -1;
 }
diff --git a/hw/scsi/scsi-bus.c b/hw/scsi/scsi-bus.c
index dca9576..f8de569 100644
--- a/hw/scsi/scsi-bus.c
+++ b/hw/scsi/scsi-bus.c
@@ -273,7 +273,6 @@ void scsi_bus_legacy_handle_cmdline(SCSIBus *bus, Error 
**errp)
 scsi_bus_legacy_add_drive(bus, blk_by_legacy_dinfo(dinfo),
   unit, false, -1, NULL, &err);
 if (err != NULL) {
-error_report("%s", error_get_pretty(err));
 error_propagate(errp, err);
 break;
 }
diff --git a/hw/scsi/spapr_vscsi.c b/hw/scsi/spapr_vscsi.c
index 3639235..f76d334 100644
--- a/hw/scsi/spapr_vscsi.c
+++ b/hw/scsi/spapr_vscsi.c
@@ -39,6 +39,7 @@
 #include "hw/ppc/spapr.h"
 #include "hw/ppc/spapr_vio.h"
 #include "viosrp.h"
+#include "qapi/qmp/qerror.h"
 
 #include 
 
@@ -1224,6 +1225,7 @@ static int spapr_vscsi_init(VIOsPAPRDevice *dev)
 if (!dev->qdev.hotplugged) {
 scsi_bus_legacy_handle_cmdline(&s->bus, &err);
 if (err != NULL) {
+qerror_report_err(err);
 error_free(err);
 return -1;
 }
-- 
1.9.3

[Qemu-devel] [PATCH] tcg: Complete handling of ALWAYS and NEVER

[Richard Henderson]

Missing from movcond, and brcondi_i32 (but not brcondi_i64).

Signed-off-by: Richard Henderson 
---
 tcg/tcg-op.c | 22 +-
 1 file changed, 17 insertions(+), 5 deletions(-)
---

On 02/20/2015 05:05 AM, Laurent Desnogues wrote:> Hi Richard,
> 
> this patch results in movcond with always as a condition to be
> generated for csel al.  The issue is that optimize.c did not get
> patched to accept that which results in some tcg_abort to fire (in
> do_constant_folding_cond_64 in this case).

Consider this patch to precede 08/11.


r~
---

diff --git a/tcg/tcg-op.c b/tcg/tcg-op.c
index 6674bb4..f7a2767 100644
--- a/tcg/tcg-op.c
+++ b/tcg/tcg-op.c
@@ -286,9 +286,13 @@ void tcg_gen_brcond_i32(TCGCond cond, TCGv_i32 arg1, 
TCGv_i32 arg2, TCGLabel *l)
 
 void tcg_gen_brcondi_i32(TCGCond cond, TCGv_i32 arg1, int32_t arg2, TCGLabel 
*l)
 {
-TCGv_i32 t0 = tcg_const_i32(arg2);
-tcg_gen_brcond_i32(cond, arg1, t0, l);
-tcg_temp_free_i32(t0);
+if (cond == TCG_COND_ALWAYS) {
+tcg_gen_br(l);
+} else if (cond != TCG_COND_NEVER) {
+TCGv_i32 t0 = tcg_const_i32(arg2);
+tcg_gen_brcond_i32(cond, arg1, t0, l);
+tcg_temp_free_i32(t0);
+}
 }
 
 void tcg_gen_setcond_i32(TCGCond cond, TCGv_i32 ret,
@@ -546,7 +550,11 @@ void tcg_gen_deposit_i32(TCGv_i32 ret, TCGv_i32 arg1, 
TCGv_i32 arg2,
 void tcg_gen_movcond_i32(TCGCond cond, TCGv_i32 ret, TCGv_i32 c1,
  TCGv_i32 c2, TCGv_i32 v1, TCGv_i32 v2)
 {
-if (TCG_TARGET_HAS_movcond_i32) {
+if (cond == TCG_COND_ALWAYS) {
+tcg_gen_mov_i32(ret, v1);
+} else if (cond == TCG_COND_NEVER) {
+tcg_gen_mov_i32(ret, v2);
+} else if (TCG_TARGET_HAS_movcond_i32) {
 tcg_gen_op6i_i32(INDEX_op_movcond_i32, ret, c1, c2, v1, v2, cond);
 } else {
 TCGv_i32 t0 = tcg_temp_new_i32();
@@ -1590,7 +1598,11 @@ void tcg_gen_deposit_i64(TCGv_i64 ret, TCGv_i64 arg1, 
TCGv_i64 arg2,
 void tcg_gen_movcond_i64(TCGCond cond, TCGv_i64 ret, TCGv_i64 c1,
  TCGv_i64 c2, TCGv_i64 v1, TCGv_i64 v2)
 {
-if (TCG_TARGET_REG_BITS == 32) {
+if (cond == TCG_COND_ALWAYS) {
+tcg_gen_mov_i64(ret, v1);
+} else if (cond == TCG_COND_NEVER) {
+tcg_gen_mov_i64(ret, v2);
+} else if (TCG_TARGET_REG_BITS == 32) {
 TCGv_i32 t0 = tcg_temp_new_i32();
 TCGv_i32 t1 = tcg_temp_new_i32();
 tcg_gen_op6i_i32(INDEX_op_setcond2_i32, t0,
-- 
2.1.0

[Qemu-devel] [PATCH 0/4] scsi: Error reporting cleanups, realize()

[Markus Armbruster]

This series is based on
* [PULL 00/96] pci, pc, virtio fixes and cleanups
* [PULL 0/8] usb: error handling fixes from Markus, make sysbus ehci
  arm-only.
* [PULL v2 00/11] Clean up around error_get_pretty(),
  qerror_report_err()
* [PATCH 0/2] pci: Bury dead legacy commands pci_add, pci_del
* [PATCH] scsi: give device a parent before setting properties

Series passes make check with my "[PATCH RFC 1/1] qtest: Add generic
PCI device test" applied.

Markus Armbruster (4):
  scsi: Clean up duplicated error in legacy if=scsi code
  hw: Propagate errors through qdev_prop_set_drive()
  scsi: Improve error reporting for invalid drive property
  scsi: Convert remaining PCI HBAs to realize()

 hw/arm/vexpress.c|  6 +++---
 hw/arm/virt.c|  6 +++---
 hw/block/pflash_cfi01.c  |  4 ++--
 hw/block/pflash_cfi02.c  |  4 ++--
 hw/core/qdev-properties-system.c | 22 ++
 hw/scsi/esp-pci.c| 28 +++-
 hw/scsi/lsi53c895a.c | 13 +++--
 hw/scsi/megasas.c| 12 +++-
 hw/scsi/scsi-bus.c   |  6 +++---
 hw/scsi/spapr_vscsi.c|  2 ++
 hw/usb/dev-storage.c |  7 +--
 include/hw/qdev-properties.h |  4 ++--
 12 files changed, 49 insertions(+), 65 deletions(-)

-- 
1.9.3

[Qemu-devel] [PATCH 3/4] scsi: Improve error reporting for invalid drive property

[Markus Armbruster]

When setting "realized" fails, scsi_bus_legacy_add_drive() passes the
error to qerror_report_err(), then returns an unspecific "Setting
drive property failed" error, which is reported further up the call
chain.

Example:

$ qemu-system-x86_64 -nodefaults -S -display none \
> -drive if=scsi,id=foo,file=tmp.qcow2 -global isa-fdc.driveA=foo
qemu-system-x86_64: -drive if=scsi,id=foo,file=tmp.qcow2: Property 
'scsi-disk.drive' can't take value 'foo', it's in use
qemu-system-x86_64: Setting drive property failed
qemu-system-x86_64: Device initialization failed
qemu-system-x86_64: Initialization of device lsi53c895a failed

Clean up the obvious way: simply return the original error to the
caller.  Gets rid of the second message in the above error cascade.

Signed-off-by: Markus Armbruster 
---
 hw/scsi/scsi-bus.c | 5 +
 1 file changed, 1 insertion(+), 4 deletions(-)

diff --git a/hw/scsi/scsi-bus.c b/hw/scsi/scsi-bus.c
index 61c595f..bd2c0e4 100644
--- a/hw/scsi/scsi-bus.c
+++ b/hw/scsi/scsi-bus.c
@@ -7,7 +7,6 @@
 #include "sysemu/blockdev.h"
 #include "trace.h"
 #include "sysemu/dma.h"
-#include "qapi/qmp/qerror.h"
 
 static char *scsibus_get_dev_path(DeviceState *dev);
 static char *scsibus_get_fw_dev_path(DeviceState *dev);
@@ -245,9 +244,7 @@ SCSIDevice *scsi_bus_legacy_add_drive(SCSIBus *bus, 
BlockBackend *blk,
 }
 qdev_prop_set_drive(dev, "drive", blk, &err);
 if (err) {
-qerror_report_err(err);
-error_free(err);
-error_setg(errp, "Setting drive property failed");
+error_propagate(errp, err);
 object_unparent(OBJECT(dev));
 return NULL;
 }
-- 
1.9.3

Re: [Qemu-devel] [PATCH 0/3] Support streaming to an intermediate layer

[Alberto Garcia]

On Fri, Feb 20, 2015 at 10:34:58AM -0700, Eric Blake wrote:

> > I followed the proposed API from the wiki, which simply adds an
> > additional 'top' parameter to block-stream specifying the image
> > that data is written to:
> 
> How does one learn whether qemu is new enough to support this
> mode? Until we add QMP introspection, learning whether an optional
> parameter exists requires attempting the command and seeing a
> different error depending on whether the argument is recognized.

Isn't it possible to just check the QEMU version number?

If not, what's the recommended way to do it?

Thanks,

Berto

Re: [Qemu-devel] [RFC PATCH v2 10/15] cpu-model/s390: Add cpu class initialization routines

[Michael Mueller]

On Fri, 20 Feb 2015 10:11:55 -0800
Richard Henderson  wrote:

> > +static inline uint64_t big_endian_bit(unsigned long nr)
> > +{
> > +return 1ul << (BITS_PER_LONG - (nr % BITS_PER_LONG));
> > +};  
> 
> This is buggy.  NR=0 should map to 63, not 64.

I'm sure I was asked to replace my constant 64 and 63 with that defines and at 
the end I messed
it up... :-(

> 
> > +return !!(*ptr & big_endian_bit(nr));  
> 
> Personally I dislike !! as an idiom.  Given that big_endian_bit isn't used
> anywhere else, can we integrate it and change this to
> 
> static inline int test_facility(unsigned long nr, uint64_t *fac_list)
> {
>   unsigned long word = nr / BITS_PER_LONG;
>   unsigned long be_bit = 63 - (nr % BITS_PER_LONG);
>   return (fac_list[word] >> be_bit) & 1;
> }

Yes, I just use it in this context. I will integrate your version.

BTW I changed the whole facility defining code to be generated by an external 
helper at compile
time. That is more simple and safe to change. I will send it with v3. See 
attachment for an
example of the generated header file.

Thanks,
Michael

/*
 * AUTOMATICALLY GENERATED, DO NOT MODIFY HERE, EDIT
 * SOURCE FILE "target-s390x/tools/gen-facilities.c" INSTEAD.
 *
 * Copyright 2014, 2015 IBM Corp.
 *
 * This work is licensed under the terms of the GNU GPL, version 2 or (at
 * your option) any later version. See the COPYING file in the top-level
 * directory.
 */

#ifndef TARGET_S390X_GEN_FACILITIES_H
#define TARGET_S390X_GEN_FACILITIES_H

/* S390 CPU facility defines per CPU model */
#define FAC_LIST_CPU_S390_2064_GA1 0xe000
#define FAC_LIST_CPU_S390_2064_GA2 0xe0008000
#define FAC_LIST_CPU_S390_2064_GA3 0xe0008000
#define FAC_LIST_CPU_S390_2066_GA1 0xe0008000
#define FAC_LIST_CPU_S390_2084_GA1 0xf000f800
#define FAC_LIST_CPU_S390_2084_GA2 0xf800f800
#define FAC_LIST_CPU_S390_2084_GA3 0xfa00fa00
#define FAC_LIST_CPU_S390_2086_GA1 0xfa00fa00
#define FAC_LIST_CPU_S390_2084_GA4 0xfa00fa00
#define FAC_LIST_CPU_S390_2086_GA2 0xfa00fa00
#define FAC_LIST_CPU_S390_2084_GA5 0xfa00fa08
#define FAC_LIST_CPU_S390_2086_GA3 0xfa00fa08
#define FAC_LIST_CPU_S390_2094_GA1 0xfb00ffcb
#define FAC_LIST_CPU_S390_2094_GA2 0xfb40ffdb8060
#define FAC_LIST_CPU_S390_2094_GA3 0xfb40ffdb8068
#define FAC_LIST_CPU_S390_2096_GA1 0xfb40ffdb8068
#define FAC_LIST_CPU_S390_2096_GA2 0xfb40ffdb8068
#define FAC_LIST_CPU_S390_2097_GA1 0xfbf0fffbf078
#define FAC_LIST_CPU_S390_2097_GA2 0xfbf0fffbf078,0x5800
#define FAC_LIST_CPU_S390_2098_GA1 0xfbf0fffbf078,0x5800
#define FAC_LIST_CPU_S390_2097_GA3 0xfbf0fffbf0f8,0x5800
#define FAC_LIST_CPU_S390_2098_GA2 0xfbf0fffbf0f8,0x5800
#define FAC_LIST_CPU_S390_2817_GA1 0xfbf0fffbfcfe,0x1810
#define FAC_LIST_CPU_S390_2817_GA2 0xfbf6fffbfcff,0x381c
#define FAC_LIST_CPU_S390_2818_GA1 0xfbf6fffbfcff,0x381c
#define FAC_LIST_CPU_S390_2827_GA1 0xfbf6fffbfcfff800,0x3dde
#define FAC_LIST_CPU_S390_2827_GA2 0xfbf6fffbfcfff800,0x3dde
#define FAC_LIST_CPU_S390_2828_GA1 0xfbf6fffbfcfff800,0x3dde
#define FAC_LIST_CPU_S390_2964_GA1 0xfbf6fffbfcfff800,0x3dde,0x4000

/* QEMU facility mask defines */
#define FAC_LIST_CPU_S390_MASK_QEMU 0x,0x0580

/* Maximum size of generated facility list defines */
#define FAC_LIST_CPU_S390_SIZE_UINT64 3

#endif

[Qemu-devel] [PATCH] qerror.h: Swap definitions that were not in alphabetical order

[Alberto Garcia]

Signed-off-by: Alberto Garcia 
---
 include/qapi/qmp/qerror.h | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/include/qapi/qmp/qerror.h b/include/qapi/qmp/qerror.h
index 986260f..57a62d4 100644
--- a/include/qapi/qmp/qerror.h
+++ b/include/qapi/qmp/qerror.h
@@ -37,12 +37,12 @@ void qerror_report_err(Error *err);
 #define QERR_BASE_NOT_FOUND \
 ERROR_CLASS_GENERIC_ERROR, "Base '%s' not found"
 
-#define QERR_BLOCK_JOB_NOT_READY \
-ERROR_CLASS_GENERIC_ERROR, "The active block job for device '%s' cannot be 
completed"
-
 #define QERR_BLOCK_FORMAT_FEATURE_NOT_SUPPORTED \
 ERROR_CLASS_GENERIC_ERROR, "Block format '%s' used by device '%s' does not 
support feature '%s'"
 
+#define QERR_BLOCK_JOB_NOT_READY \
+ERROR_CLASS_GENERIC_ERROR, "The active block job for device '%s' cannot be 
completed"
+
 #define QERR_BUS_NO_HOTPLUG \
 ERROR_CLASS_GENERIC_ERROR, "Bus '%s' does not support hotplugging"
 
-- 
2.1.4

Re: [Qemu-devel] [RFC PATCH v2 09/15] cpu-model/s390: Add KVM VM attribute interface routines

[Michael Mueller]

On Fri, 20 Feb 2015 17:59:14 +0100
Alexander Graf  wrote:

> But please give a nutshell explanation on what exactly you're patching
> at all here.

Please don't ask in riddles... :-)

S390ProcessorProps are attributes that represent cpu model related properties 
of the processor. 

typedef struct S390ProcessorProps {
uint64_t cpuid;
uint16_t ibc;
uint8_t  pad[6];
uint64_t fac_list[S390_ARCH_FAC_LIST_SIZE_UINT64];
} S390ProcessorProps;

S390MachineProps are attributes that represent cpu model related properties 
that a specific host
offers.

fac_list_mask are the facilities that are supported by the accelerator code and 
the hosting
machine in case of KVM it is kvm_s390_fac_list_mask & STFLE, fac_list is STFLE 
only

typedef struct S390MachineProps {
uint64_t cpuid;
uint32_t ibc_range;
uint8_t  pad[4];
uint64_t fac_list_mask[S390_ARCH_FAC_LIST_SIZE_UINT64];
uint64_t fac_list[S390_ARCH_FAC_LIST_SIZE_UINT64];
} S390MachineProps;

The various S390CPUModel classes represent all well defined cpu models (those 
which have been
implemented as real machines). Not all of these models are executable on a 
given host. Only the
most current machine implementation is able to run all models. The list of 
"runnable" cpu models
is calculated by "matching" the S390MachineProps with the S390CPUModel classes 
and the
qemu_s390_fac_list_mask[] (indicates the facilities that are implemented by 
QEMU itself.)

The qemu cpu_model is translated to the respective S390CPUModel class and its 
processor
properties (S390ProcessorProps) are used with the s390_set_proceccor_props() 
call to communicate
them to the accelerator what specific cpuid,ibc,fac_list shall be used.

Michael

[Qemu-devel] [PATCH v5 14/16] pc: acpi-build: drop template patching and create PCI bus tree dynamically

[Igor Mammedov]

Replace AML template patching with direct composing
of PCI device entries in C. It allows to simplify
PCI tree generation further and saves us about 400LOC
scattered through different files, confining tree
generation to one C function which is much easier
to deal with.

Signed-off-by: Igor Mammedov 
---
v2:
  * add some comments to the process of device creation
---
 hw/i386/Makefile.objs  |   1 -
 hw/i386/acpi-build.c   | 235 +
 hw/i386/ssdt-pcihp.dsl | 100 -
 3 files changed, 80 insertions(+), 256 deletions(-)
 delete mode 100644 hw/i386/ssdt-pcihp.dsl

diff --git a/hw/i386/Makefile.objs b/hw/i386/Makefile.objs
index 9b00568..e058a39 100644
--- a/hw/i386/Makefile.objs
+++ b/hw/i386/Makefile.objs
@@ -9,7 +9,6 @@ obj-y += kvmvapic.o
 obj-y += acpi-build.o
 hw/i386/acpi-build.o: hw/i386/acpi-build.c \
hw/i386/acpi-dsdt.hex hw/i386/q35-acpi-dsdt.hex \
-   hw/i386/ssdt-pcihp.hex \
hw/i386/ssdt-tpm.hex
 
 iasl-option=$(shell if test -z "`$(1) $(2) 2>&1 > /dev/null`" \
diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index ba056f0..b94e47e 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -294,26 +294,6 @@ build_header(GArray *linker, GArray *table_data,
 table_data->data, h, len, &h->checksum);
 }
 
-static GArray *build_alloc_method(const char *name, uint8_t arg_count)
-{
-GArray *method = build_alloc_array();
-
-build_append_namestring(method, "%s", name);
-build_append_byte(method, arg_count); /* MethodFlags: ArgCount */
-
-return method;
-}
-
-static void build_append_and_cleanup_method(GArray *device, GArray *method)
-{
-uint8_t op = 0x14; /* MethodOp */
-
-build_package(method, op);
-
-build_append_array(device, method);
-build_free_array(method);
-}
-
 /* End here */
 #define ACPI_PORT_SMI_CMD   0x00b2 /* TODO: this is APM_CNT_IOPORT */
 
@@ -494,71 +474,12 @@ static inline char acpi_get_hex(uint32_t val)
 return (val <= 9) ? ('0' + val) : ('A' + val - 10);
 }
 
-/* 0x5B 0x82 DeviceOp PkgLength NameString */
-#define ACPI_PCIHP_OFFSET_HEX (*ssdt_pcihp_name - *ssdt_pcihp_start + 1)
-#define ACPI_PCIHP_OFFSET_ID (*ssdt_pcihp_id - *ssdt_pcihp_start)
-#define ACPI_PCIHP_OFFSET_ADR (*ssdt_pcihp_adr - *ssdt_pcihp_start)
-#define ACPI_PCIHP_OFFSET_EJ0 (*ssdt_pcihp_ej0 - *ssdt_pcihp_start)
-#define ACPI_PCIHP_SIZEOF (*ssdt_pcihp_end - *ssdt_pcihp_start)
-#define ACPI_PCIHP_AML (ssdp_pcihp_aml + *ssdt_pcihp_start)
-
-#define ACPI_PCINOHP_OFFSET_HEX (*ssdt_pcinohp_name - *ssdt_pcinohp_start + 1)
-#define ACPI_PCINOHP_OFFSET_ADR (*ssdt_pcinohp_adr - *ssdt_pcinohp_start)
-#define ACPI_PCINOHP_SIZEOF (*ssdt_pcinohp_end - *ssdt_pcinohp_start)
-#define ACPI_PCINOHP_AML (ssdp_pcihp_aml + *ssdt_pcinohp_start)
-
-#define ACPI_PCIVGA_OFFSET_HEX (*ssdt_pcivga_name - *ssdt_pcivga_start + 1)
-#define ACPI_PCIVGA_OFFSET_ADR (*ssdt_pcivga_adr - *ssdt_pcivga_start)
-#define ACPI_PCIVGA_SIZEOF (*ssdt_pcivga_end - *ssdt_pcivga_start)
-#define ACPI_PCIVGA_AML (ssdp_pcihp_aml + *ssdt_pcivga_start)
-
-#define ACPI_PCIQXL_OFFSET_HEX (*ssdt_pciqxl_name - *ssdt_pciqxl_start + 1)
-#define ACPI_PCIQXL_OFFSET_ADR (*ssdt_pciqxl_adr - *ssdt_pciqxl_start)
-#define ACPI_PCIQXL_SIZEOF (*ssdt_pciqxl_end - *ssdt_pciqxl_start)
-#define ACPI_PCIQXL_AML (ssdp_pcihp_aml + *ssdt_pciqxl_start)
 
 #define ACPI_SSDT_SIGNATURE 0x54445353 /* SSDT */
 #define ACPI_SSDT_HEADER_LENGTH 36
 
-#include "hw/i386/ssdt-pcihp.hex"
 #include "hw/i386/ssdt-tpm.hex"
 
-static void patch_pcihp(int slot, uint8_t *ssdt_ptr)
-{
-unsigned devfn = PCI_DEVFN(slot, 0);
-
-ssdt_ptr[ACPI_PCIHP_OFFSET_HEX] = acpi_get_hex(devfn >> 4);
-ssdt_ptr[ACPI_PCIHP_OFFSET_HEX + 1] = acpi_get_hex(devfn);
-ssdt_ptr[ACPI_PCIHP_OFFSET_ID] = slot;
-ssdt_ptr[ACPI_PCIHP_OFFSET_ADR + 2] = slot;
-}
-
-static void patch_pcinohp(int slot, uint8_t *ssdt_ptr)
-{
-unsigned devfn = PCI_DEVFN(slot, 0);
-
-ssdt_ptr[ACPI_PCINOHP_OFFSET_HEX] = acpi_get_hex(devfn >> 4);
-ssdt_ptr[ACPI_PCINOHP_OFFSET_HEX + 1] = acpi_get_hex(devfn);
-ssdt_ptr[ACPI_PCINOHP_OFFSET_ADR + 2] = slot;
-}
-
-static void patch_pcivga(int slot, uint8_t *ssdt_ptr)
-{
-unsigned devfn = PCI_DEVFN(slot, 0);
-
-ssdt_ptr[ACPI_PCIVGA_OFFSET_HEX] = acpi_get_hex(devfn >> 4);
-ssdt_ptr[ACPI_PCIVGA_OFFSET_HEX + 1] = acpi_get_hex(devfn);
-ssdt_ptr[ACPI_PCIVGA_OFFSET_ADR + 2] = slot;
-}
-
-static void patch_pciqxl(int slot, uint8_t *ssdt_ptr)
-{
-unsigned devfn = PCI_DEVFN(slot, 0);
-
-ssdt_ptr[ACPI_PCIQXL_OFFSET_HEX] = acpi_get_hex(devfn >> 4);
-ssdt_ptr[ACPI_PCIQXL_OFFSET_HEX + 1] = acpi_get_hex(devfn);
-ssdt_ptr[ACPI_PCIQXL_OFFSET_ADR + 2] = slot;
-}
 
 /* Assign BSEL property to all buses.  In the future, this can be changed
  * to only assign to buses that support hotplug.
@@ -590,46 +511,30 @@ static void acpi_set_pci_info(void)
 }
 }
 
-static void build_append_pcihp_notify_entry(GArra

[Qemu-devel] [PATCH v5 12/16] pc: acpi-build: simplify PCI bus tree generation

[Igor Mammedov]

it basicaly does the same as original approach,
* just without bus/notify tables tracking (less obscure)
  which is easier to follow.
* drops unnecessary loops and bitmaps,
  creating devices and notification method in the same loop.
* saves us ~100LOC

Signed-off-by: Igor Mammedov 
---
v2:
 * fixed 'make check' failure on Q35, which was caused by
   adding slots for non present devices when ACPI
   hotplug is disabled
 * fixed 'make check' failure on Q35, which was caused by
   marking devices as hotpluggable when ACPI hotplug is disabled
---
 hw/i386/acpi-build.c | 274 ---
 1 file changed, 85 insertions(+), 189 deletions(-)

diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index 05eb80a..ba056f0 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -112,7 +112,6 @@ typedef struct AcpiPmInfo {
 typedef struct AcpiMiscInfo {
 bool has_hpet;
 bool has_tpm;
-DECLARE_BITMAP(slot_hotplug_enable, PCI_SLOT_MAX);
 const unsigned char *dsdt_code;
 unsigned dsdt_size;
 uint16_t pvpanic_port;
@@ -591,74 +590,37 @@ static void acpi_set_pci_info(void)
 }
 }
 
-static void build_pci_bus_state_init(AcpiBuildPciBusHotplugState *state,
- AcpiBuildPciBusHotplugState *parent,
- bool pcihp_bridge_en)
+static void build_append_pcihp_notify_entry(GArray *method, int slot)
 {
-state->parent = parent;
-state->device_table = build_alloc_array();
-state->notify_table = build_alloc_array();
-state->pcihp_bridge_en = pcihp_bridge_en;
+GArray *ifctx;
+
+ifctx = build_alloc_array();
+build_append_byte(ifctx, 0x7B); /* AndOp */
+build_append_byte(ifctx, 0x68); /* Arg0Op */
+build_append_int(ifctx, 0x1U << slot);
+build_append_byte(ifctx, 0x00); /* NullName */
+build_append_byte(ifctx, 0x86); /* NotifyOp */
+build_append_namestring(ifctx, "S%.02X", PCI_DEVFN(slot, 0));
+build_append_byte(ifctx, 0x69); /* Arg1Op */
+
+/* Pack it up */
+build_package(ifctx, 0xA0 /* IfOp */);
+build_append_array(method, ifctx);
+build_free_array(ifctx);
 }
 
-static void build_pci_bus_state_cleanup(AcpiBuildPciBusHotplugState *state)
+static void build_append_pci_bus_devices(GArray *parent_scope, PCIBus *bus,
+ bool pcihp_bridge_en)
 {
-build_free_array(state->device_table);
-build_free_array(state->notify_table);
-}
-
-static void *build_pci_bus_begin(PCIBus *bus, void *parent_state)
-{
-AcpiBuildPciBusHotplugState *parent = parent_state;
-AcpiBuildPciBusHotplugState *child = g_malloc(sizeof *child);
-
-build_pci_bus_state_init(child, parent, parent->pcihp_bridge_en);
-
-return child;
-}
-
-static void build_pci_bus_end(PCIBus *bus, void *bus_state)
-{
-AcpiBuildPciBusHotplugState *child = bus_state;
-AcpiBuildPciBusHotplugState *parent = child->parent;
 GArray *bus_table = build_alloc_array();
-DECLARE_BITMAP(slot_hotplug_enable, PCI_SLOT_MAX);
-DECLARE_BITMAP(slot_device_present, PCI_SLOT_MAX);
-DECLARE_BITMAP(slot_device_system, PCI_SLOT_MAX);
-DECLARE_BITMAP(slot_device_vga, PCI_SLOT_MAX);
-DECLARE_BITMAP(slot_device_qxl, PCI_SLOT_MAX);
-uint8_t op;
-int i;
+GArray *method = NULL;
 QObject *bsel;
-GArray *method;
-bool bus_hotplug_support = false;
-
-/*
- * Skip bridge subtree creation if bridge hotplug is disabled
- * to make acpi tables compatible with legacy machine types.
- * Skip creation for hotplugged bridges as well.
- */
-if (bus->parent_dev && (!child->pcihp_bridge_en ||
-DEVICE(bus->parent_dev)->hotplugged)) {
-build_free_array(bus_table);
-build_pci_bus_state_cleanup(child);
-g_free(child);
-return;
-}
+PCIBus *sec;
+int i;
 
 if (bus->parent_dev) {
-op = 0x82; /* DeviceOp */
-build_append_namestring(bus_table, "S%.02X",
- bus->parent_dev->devfn);
-build_append_byte(bus_table, 0x08); /* NameOp */
-build_append_namestring(bus_table, "_SUN");
-build_append_int(bus_table, PCI_SLOT(bus->parent_dev->devfn));
-build_append_byte(bus_table, 0x08); /* NameOp */
-build_append_namestring(bus_table, "_ADR");
-build_append_int(bus_table, (PCI_SLOT(bus->parent_dev->devfn) << 16) |
-   PCI_FUNC(bus->parent_dev->devfn));
+build_append_namestring(bus_table, "S%.02X_", bus->parent_dev->devfn);
 } else {
-op = 0x10; /* ScopeOp */;
 build_append_namestring(bus_table, "PCI0");
 }
 
@@ -667,29 +629,28 @@ static void build_pci_bus_end(PCIBus *bus, void 
*bus_state)
 build_append_byte(bus_table, 0x08); /* NameOp */
 build_append_namestring(bus_table, "BSEL");
 build_append_int(bus_table, qint_get_int(qobject_to_qint(bsel)));
-memset(slot_hot

[Qemu-devel] [PATCH v5 09/16] tests: ACPI test blobs update due to PCI0._CRS changes

[Igor Mammedov]

PCI0._CRS was moved into SSDT and became the same for
PIIX4/Q35 machines.

Signed-off-by: Igor Mammedov 
---
 tests/acpi-test-data/pc/DSDT  | Bin 3415 -> 2970 bytes
 tests/acpi-test-data/pc/SSDT  | Bin 2374 -> 2480 bytes
 tests/acpi-test-data/q35/DSDT | Bin 8005 -> 7608 bytes
 tests/acpi-test-data/q35/SSDT | Bin 578 -> 685 bytes
 4 files changed, 0 insertions(+), 0 deletions(-)

diff --git a/tests/acpi-test-data/pc/DSDT b/tests/acpi-test-data/pc/DSDT
index 
ad12cb35d9dafc4d50002873f5c4ca04552b36df..1693c3783b34cfb5473e7ef65ba6f8deee390898
 100644
GIT binary patch
delta 45
zcmcaEHA|e!CDR}W5h(Rj?Icp+>8P~()!Ht!A|i3&YlLI!H)5pvzXs<0ss&x
B3^@P*

delta 471
zcmX|;PfG$p7{=dmA=YWu6e24^q(i4D|FOqtT^BPf21n}9Vf+XZ5qMG}P$DSt5*?zz
zt4j0%@>_JQL)Q+S^3*i1+xi}6e$Vp^54_BM{B4yMM4@_Y0g&@9?KBUUx1`8R?v_%f
zTk%9JPw7L)f)E27g$F5;37artYM(Fztsfmv*i2h$pv|X1FAUWX;2Q#+9KDYK#xwmn
zQo4AJx8pW|UkE@!sd?s;9E~_!
z*g2nDf*&XeN1j}4iUU`ga<5ci
gb5S6s6jg_UiQB}Y)G87%z#z%>UA^VYfP97j0NTcIKL7v#

diff --git a/tests/acpi-test-data/pc/SSDT b/tests/acpi-test-data/pc/SSDT
index 
d0103368a0b9e3d5410372f1f589df4baf03..dc5be2497b6c2015d7cbabb1ff0cd13e02b3e0e4
 100644
GIT binary patch
delta 200
zcmX>mv_Y6FIM^j*11AFmq_(Y5Qb-R5RY{u3HX5oqF7jo=Ho~s1TU8(Mxrd)V52Z8Q%Mk;R1pa%HWv8<
z7OCtcjh#0B1}*&$HmO{9?mVa1edc|Kmz{mtf3Jy#Zgj&90DO8Dgrk#{UD9<@L96Hn
zolxiM3*zD~11Skyh7WAwT+SuBR!8Zu^kMB!`QK*@4}RaS0mMFl0Am(;)K&^22jdRD
zjCuWuIsH}`f*VXIzdpLQlHvLrOD@)juczaO!UVTfyXp;8i6bN6$OziR)|eAG
zpf~N!`Y5q>RM>NpFKua`&Te+}aX>B|wi+^>1$0SweE}}7lgo-s6TtxoH@6P#nzM1r
G%l`r~K4Uik

diff --git a/tests/acpi-test-data/q35/SSDT b/tests/acpi-test-data/q35/SSDT
index 
f86643da45ad89a97a652233cb090516afde7b31..749f368e3034bfb491dcd8ced37936e0dcbba02a
 100644
GIT binary patch
delta 200
zcmX@avX+%AIM^j*EfWI+NY;4}o2modBDslh-

delta 117
zcmZ3>dWeN9IM^k`iHU)MaoI#J={P6W7zU02gWz}`1_p)$K$dGf&;S4X|8WEuhQ)IM
or3{1P1-ZF6fU;l!6mUfs@Ies};EvIY4|a+VaP~CV_-;QV0J~ckEdT%j

-- 
1.8.3.1

[Qemu-devel] [PATCH v5 13/16] tests: ACPI: update pc/SSDT.bridge due to new alg of PCI tree creation

[Igor Mammedov]

Signed-off-by: Igor Mammedov 
---
 tests/acpi-test-data/pc/SSDT.bridge | Bin 4352 -> 4351 bytes
 1 file changed, 0 insertions(+), 0 deletions(-)

diff --git a/tests/acpi-test-data/pc/SSDT.bridge 
b/tests/acpi-test-data/pc/SSDT.bridge
index 
b807ac92dde72719fe3861c710b555fe3dd62583..ca7f63cb4e48bd95418f68daf1b7e254d5bcf8c2
 100644
GIT binary patch
delta 79
zcmZor`me|p9PASEUx0yu@ybT7Xhue_&1sCDjEoMGH!yjL@dq1P#B;vA+>;M#S6CwZr

-- 
1.8.3.1

[Qemu-devel] [PATCH v5 15/16] pc: acpi: remove not used anymore ssdt-[misc|pcihp].hex.generated blobs

[Igor Mammedov]

Signed-off-by: Igor Mammedov 
---
 hw/i386/ssdt-misc.hex.generated  | 139 --
 hw/i386/ssdt-pcihp.hex.generated | 251 ---
 2 files changed, 390 deletions(-)
 delete mode 100644 hw/i386/ssdt-misc.hex.generated
 delete mode 100644 hw/i386/ssdt-pcihp.hex.generated

diff --git a/hw/i386/ssdt-misc.hex.generated b/hw/i386/ssdt-misc.hex.generated
deleted file mode 100644
index 0b77ed4..000
--- a/hw/i386/ssdt-misc.hex.generated
+++ /dev/null
@@ -1,139 +0,0 @@
-static unsigned char acpi_pci64_length[] = {
-0x6f
-};
-static unsigned char acpi_pci32_start[] = {
-0x2f
-};
-static unsigned char acpi_pci64_valid[] = {
-0x43
-};
-static unsigned char ssdp_misc_aml[] = {
-0x53,
-0x53,
-0x44,
-0x54,
-0x77,
-0x0,
-0x0,
-0x0,
-0x1,
-0x40,
-0x42,
-0x58,
-0x50,
-0x43,
-0x0,
-0x0,
-0x42,
-0x58,
-0x53,
-0x53,
-0x44,
-0x54,
-0x53,
-0x55,
-0x1,
-0x0,
-0x0,
-0x0,
-0x49,
-0x4e,
-0x54,
-0x4c,
-0x7,
-0x11,
-0x14,
-0x20,
-0x10,
-0x42,
-0x5,
-0x5c,
-0x0,
-0x8,
-0x50,
-0x30,
-0x53,
-0x5f,
-0xc,
-0x78,
-0x56,
-0x34,
-0x12,
-0x8,
-0x50,
-0x30,
-0x45,
-0x5f,
-0xc,
-0x78,
-0x56,
-0x34,
-0x12,
-0x8,
-0x50,
-0x31,
-0x56,
-0x5f,
-0xa,
-0x12,
-0x8,
-0x50,
-0x31,
-0x53,
-0x5f,
-0x11,
-0xb,
-0xa,
-0x8,
-0x0,
-0x0,
-0x0,
-0x0,
-0x0,
-0x0,
-0x0,
-0x0,
-0x8,
-0x50,
-0x31,
-0x45,
-0x5f,
-0x11,
-0xb,
-0xa,
-0x8,
-0x0,
-0x0,
-0x0,
-0x0,
-0x0,
-0x0,
-0x0,
-0x0,
-0x8,
-0x50,
-0x31,
-0x4c,
-0x5f,
-0x11,
-0xb,
-0xa,
-0x8,
-0x0,
-0x0,
-0x0,
-0x0,
-0x0,
-0x0,
-0x0,
-0x0
-};
-static unsigned char acpi_pci64_start[] = {
-0x4d
-};
-static unsigned char acpi_pci64_end[] = {
-0x5e
-};
-static unsigned char acpi_pci32_end[] = {
-0x39
-};
diff --git a/hw/i386/ssdt-pcihp.hex.generated b/hw/i386/ssdt-pcihp.hex.generated
deleted file mode 100644
index 72ffa84..000
--- a/hw/i386/ssdt-pcihp.hex.generated
+++ /dev/null
@@ -1,251 +0,0 @@
-static unsigned char ssdt_pcihp_name[] = {
-0x34
-};
-static unsigned char ssdt_pcivga_end[] = {
-0x99
-};
-static unsigned char ssdt_pcivga_name[] = {
-0x70
-};
-static unsigned char ssdt_pcihp_adr[] = {
-0x45
-};
-static unsigned char ssdt_pcinohp_end[] = {
-0x6d
-};
-static unsigned char ssdt_pcihp_end[] = {
-0x5c
-};
-static unsigned char ssdt_pciqxl_start[] = {
-0x99
-};
-static unsigned char ssdt_pcinohp_name[] = {
-0x5f
-};
-static unsigned char ssdp_pcihp_aml[] = {
-0x53,
-0x53,
-0x44,
-0x54,
-0xc6,
-0x0,
-0x0,
-0x0,
-0x1,
-0x70,
-0x42,
-0x58,
-0x50,
-0x43,
-0x0,
-0x0,
-0x42,
-0x58,
-0x53,
-0x53,
-0x44,
-0x54,
-0x50,
-0x43,
-0x1,
-0x0,
-0x0,
-0x0,
-0x49,
-0x4e,
-0x54,
-0x4c,
-0x15,
-0x11,
-0x13,
-0x20,
-0x10,
-0x41,
-0xa,
-0x5c,
-0x2e,
-0x5f,
-0x53,
-0x42,
-0x5f,
-0x50,
-0x43,
-0x49,
-0x30,
-0x5b,
-0x82,
-0x29,
-0x53,
-0x41,
-0x41,
-0x5f,
-0x8,
-0x5f,
-0x53,
-0x55,
-0x4e,
-0xa,
-0xaa,
-0x8,
-0x5f,
-0x41,
-0x44,
-0x52,
-0xc,
-0x0,
-0x0,
-0xaa,
-0x0,
-0x14,
-0x12,
-0x5f,
-0x45,
-0x4a,
-0x30,
-0x1,
-0x50,
-0x43,
-0x45,
-0x4a,
-0x42,
-0x53,
-0x45,
-0x4c,
-0x5f,
-0x53,
-0x55,
-0x4e,
-0x5b,
-0x82,
-0xf,
-0x53,
-0x42,
-0x42,
-0x5f,
-0x8,
-0x5f,
-0x41,
-0x44,
-0x52,
-0xc,
-0x0,
-0x0,
-0xaa,
-0x0,
-0x5b,
-0x82,
-0x2a,
-0x53,
-0x43,
-0x43,
-0x5f,
-0x8,
-0x5f,
-0x41,
-0x44,
-0x52,
-0xc,
-0x0,
-0x0,
-0xaa,
-0x0,
-0x14,
-0x8,
-0x5f,
-0x53,
-0x31,
-0x44,
-0x0,
-0xa4,
-0x0,
-0x14,
-0x8,
-0x5f,
-0x53,
-0x32,
-0x44,
-0x0,
-0xa4,
-0x0,
-0x14,
-0x8,
-0x5f,
-0x53,
-0x33,
-0x44,
-0x0,
-0xa4,
-0x0,
-0x5b,
-0x82,
-0x2b,
-0x53,
-0x44,
-0x44,
-0x5f,
-0x8,
-0x5f,
-0x41,
-0x44,
-0x52,
-0xc,
-0x0,
-0x0,
-0xaa,
-0x0,
-0x14,
-0x8,
-0x5f,
-0x53,
-0x31,
-0x44,
-0x0,
-0xa4,
-0x0,
-0x14,
-0x8,
-0x5f,
-0x53,
-0x32,
-0x44,
-0x0,
-0xa4,
-0x0,
-0x14,
-0x9,
-0x5f,
-0x53,
-0x33,
-0x44,
-0x0,
-0xa4,
-0xa,
-0x3
-};
-static unsigned char ssdt_pciqxl_adr[] = {
-0xa6
-};
-static unsigned char ssdt_pcinohp_adr[] = {
-0x69
-};
-static unsigned char ssdt_pcivga_adr[] = {
-0x7a
-};
-static unsigned char ssdt_pciqxl_name[] = {
-0x9c
-};
-static unsigned char ssdt_pcivga_start[] = {
-0x6d
-};
-static unsigned char ssdt_pciqxl_end[] = {
-0xc6
-};
-static unsigned char ssdt_pcihp_start[] = {
-0x31
-};
-static unsigned char ssdt_pcihp_id[] = {
-0x3e
-};
-static unsigned char ssdt_pcinohp_start[] = {
-0x5c
-};
-- 
1.8.3.1

[Qemu-devel] [PATCH v5 10/16] tests: bios-tables-test: add support for testing bridges

[Igor Mammedov]

Adds alternative ACPI table blob selection for testing
non default QEMU configurations. If blob file for test
variant is not present, fallback to default blob.

With this change implement testing with a coldplugged
bridge.

Signed-off-by: Igor Mammedov 
---
 tests/bios-tables-test.c | 45 -
 1 file changed, 40 insertions(+), 5 deletions(-)

diff --git a/tests/bios-tables-test.c b/tests/bios-tables-test.c
index 4d0fa84..735ac61 100644
--- a/tests/bios-tables-test.c
+++ b/tests/bios-tables-test.c
@@ -40,6 +40,7 @@ typedef struct {
 
 typedef struct {
 const char *machine;
+const char *variant;
 uint32_t rsdp_addr;
 AcpiRsdpDescriptor rsdp_table;
 AcpiRsdtDescriptorRev1 rsdt_table;
@@ -396,13 +397,14 @@ static void dump_aml_files(test_data *data, bool rebuild)
 int i;
 
 for (i = 0; i < data->tables->len; ++i) {
+const char *ext = data->variant ? data->variant : "";
 sdt = &g_array_index(data->tables, AcpiSdtTable, i);
 g_assert(sdt->aml);
 
 if (rebuild) {
 uint32_t signature = cpu_to_le32(sdt->header.signature);
-aml_file = g_strdup_printf("%s/%s/%.4s", data_dir, data->machine,
-   (gchar *)&signature);
+aml_file = g_strdup_printf("%s/%s/%.4s%s", data_dir, data->machine,
+   (gchar *)&signature, ext);
 fd = g_open(aml_file, O_WRONLY|O_TRUNC|O_CREAT,
 S_IRUSR|S_IWUSR|S_IRGRP|S_IWGRP|S_IROTH);
 } else {
@@ -509,7 +511,7 @@ static GArray *load_expected_aml(test_data *data)
 {
 int i;
 AcpiSdtTable *sdt;
-gchar *aml_file;
+gchar *aml_file = NULL;
 GError *error = NULL;
 gboolean ret;
 
@@ -517,6 +519,7 @@ static GArray *load_expected_aml(test_data *data)
 for (i = 0; i < data->tables->len; ++i) {
 AcpiSdtTable exp_sdt;
 uint32_t signature;
+const char *ext = data->variant ? data->variant : "";
 
 sdt = &g_array_index(data->tables, AcpiSdtTable, i);
 
@@ -524,8 +527,15 @@ static GArray *load_expected_aml(test_data *data)
 exp_sdt.header.signature = sdt->header.signature;
 
 signature = cpu_to_le32(sdt->header.signature);
-aml_file = g_strdup_printf("%s/%s/%.4s", data_dir, data->machine,
-   (gchar *)&signature);
+
+try_again:
+aml_file = g_strdup_printf("%s/%s/%.4s%s", data_dir, data->machine,
+   (gchar *)&signature, ext);
+if (data->variant && !g_file_test(aml_file, G_FILE_TEST_EXISTS)) {
+g_free(aml_file);
+ext = "";
+goto try_again;
+}
 exp_sdt.aml_file = aml_file;
 g_assert(g_file_test(aml_file, G_FILE_TEST_EXISTS));
 ret = g_file_get_contents(aml_file, &exp_sdt.aml,
@@ -778,6 +788,17 @@ static void test_acpi_piix4_tcg(void)
 free_test_data(&data);
 }
 
+static void test_acpi_piix4_tcg_bridge(void)
+{
+test_data data;
+
+memset(&data, 0, sizeof(data));
+data.machine = MACHINE_PC;
+data.variant = ".bridge";
+test_acpi_one("-machine accel=tcg -device pci-bridge,chassis_nr=1", &data);
+free_test_data(&data);
+}
+
 static void test_acpi_q35_tcg(void)
 {
 test_data data;
@@ -788,6 +809,18 @@ static void test_acpi_q35_tcg(void)
 free_test_data(&data);
 }
 
+static void test_acpi_q35_tcg_bridge(void)
+{
+test_data data;
+
+memset(&data, 0, sizeof(data));
+data.machine = MACHINE_Q35;
+data.variant = ".bridge";
+test_acpi_one("-machine q35,accel=tcg -device pci-bridge,chassis_nr=1",
+  &data);
+free_test_data(&data);
+}
+
 int main(int argc, char *argv[])
 {
 const char *arch = qtest_get_arch();
@@ -805,7 +838,9 @@ int main(int argc, char *argv[])
 
 if (strcmp(arch, "i386") == 0 || strcmp(arch, "x86_64") == 0) {
 qtest_add_func("acpi/piix4/tcg", test_acpi_piix4_tcg);
+qtest_add_func("acpi/piix4/tcg/bridge", test_acpi_piix4_tcg_bridge);
 qtest_add_func("acpi/q35/tcg", test_acpi_q35_tcg);
+qtest_add_func("acpi/q35/tcg/bridge", test_acpi_q35_tcg_bridge);
 }
 ret = g_test_run();
 unlink(disk);
-- 
1.8.3.1

[Qemu-devel] [PATCH v5 11/16] tests: add ACPI blobs for qemu with bridge cases

[Igor Mammedov]

Signed-off-by: Igor Mammedov 
---
 tests/acpi-test-data/pc/SSDT.bridge  | Bin 0 -> 4352 bytes
 tests/acpi-test-data/q35/SSDT.bridge | Bin 0 -> 702 bytes
 2 files changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 tests/acpi-test-data/pc/SSDT.bridge
 create mode 100644 tests/acpi-test-data/q35/SSDT.bridge

diff --git a/tests/acpi-test-data/pc/SSDT.bridge 
b/tests/acpi-test-data/pc/SSDT.bridge
new file mode 100644
index 
..b807ac92dde72719fe3861c710b555fe3dd62583
GIT binary patch
literal 4352
zcmeH}&2HO96ov0ljLlF~Ov!ZYIC0FayGZCbaon9GlCk_ln$S=|3K`*|2&w=@6C@3a
zz#H2I)b6`4vdB7mg!U2o2-)-r3bgK~>b<0{9afL;WU`qh{+?%Kep^v1D1pfBjGC)xWFkv_>6N(=V@(T4xA7eQD>93;s^Y3q=d+zJINr?To
z_JT?hCtkzb1|#dBqWkWvXRlwpdG-0T*LYK=4YOQ~v|^gO;^W}sK5Koy#Reefqez3%ye
zeuo}m(hg-l0;}=GYb7CU-#ETo
z;`NVONnk`l+{OvhPRO4xuA>PnG^$n-wQ*FYxzasbCsB&p%|jP2Exa63>kwPXlMIKw
znA1=k|DB3;bXi=-*x698lT(d$Z-U6&bMR-rz-t}wcyMA`KQqc>D&z^y~*fJ6&kYZ
zEklO6m-x1D-N^N%Zj;7+&viG~lX{&r?qIH;=6X`kXN-H1>-Z<5
zFa1k(nlKJT&mN|aes{KF6m44I%-_$as8Au)}4*Sk5hOF&A1cMs$p(*#HbON
zrKk8$AWr4x4*o4T{D!`ZC@S#|D1ku%Bq)9Il%t@+%;m6C1eM1|f>AMTE0Jp+mE@{1
ziitTr{-zC2fzCp87OJyQorUWECRF3KB2>#KJ&H#>Ym4Nhq`=d(5EK)5UKWBf0#C$3
T%yO7={=b{U>qQO=MGpTBc|!Xx

literal 0
HcmV?d1

diff --git a/tests/acpi-test-data/q35/SSDT.bridge 
b/tests/acpi-test-data/q35/SSDT.bridge
new file mode 100644
index 
..c552782f5bcc57094a7d1375b67979b2efa48f1c
GIT binary patch
literal 702
zcmaJ<&2G~`5T12{vDO=5f)N!Le1$`C2}PVx?X`*1#?H#?q9RAy1F9%eMNvx+y^$aR
zcV2))p96RVj(HCb7a943`XL`SL@n(6Z
z5+_fJ5=W%hYj=XX=deayWhyV33!JajvpYmOc^QubG#Q#Pp)n%0XRz%XZ8mK2Jk^Hf
zqe8QKc5_vwQ_P*D&(LTY_;ybHc{^8Vf%W{F4nyOZC@I@tQ_t_|@bRUCfsgy&NZ`B$
U4=%uf02OciE4IM5l`Jaw0}NP~0RR91

literal 0
HcmV?d1

-- 
1.8.3.1

[Qemu-devel] [PATCH v5 16/16] acpi: make build_*() routines static to aml-build.c

[Igor Mammedov]

build_*() routines were used for composing AML
structures manually in acpi-build.c but after
conversion to AML API they are not used outside
of aml-build.c anymore, so hide them from external
users.

Signed-off-by: Igor Mammedov 
---
 hw/acpi/aml-build.c | 20 ++--
 include/hw/acpi/aml-build.h | 16 
 2 files changed, 10 insertions(+), 26 deletions(-)

diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c
index 60245e7..3e5949b 100644
--- a/hw/acpi/aml-build.c
+++ b/hw/acpi/aml-build.c
@@ -27,27 +27,27 @@
 #include "hw/acpi/aml-build.h"
 #include "qemu/bswap.h"
 
-GArray *build_alloc_array(void)
+static GArray *build_alloc_array(void)
 {
 return g_array_new(false, true /* clear */, 1);
 }
 
-void build_free_array(GArray *array)
+static void build_free_array(GArray *array)
 {
 g_array_free(array, true);
 }
 
-void build_prepend_byte(GArray *array, uint8_t val)
+static void build_prepend_byte(GArray *array, uint8_t val)
 {
 g_array_prepend_val(array, val);
 }
 
-void build_append_byte(GArray *array, uint8_t val)
+static void build_append_byte(GArray *array, uint8_t val)
 {
 g_array_append_val(array, val);
 }
 
-void build_append_array(GArray *array, GArray *val)
+static void build_append_array(GArray *array, GArray *val)
 {
 g_array_append_vals(array, val->data, val->len);
 }
@@ -141,7 +141,7 @@ build_append_namestringv(GArray *array, const char *format, 
va_list ap)
 g_strfreev(segs);
 }
 
-void build_append_namestring(GArray *array, const char *format, ...)
+static void build_append_namestring(GArray *array, const char *format, ...)
 {
 va_list ap;
 
@@ -158,7 +158,7 @@ enum {
 PACKAGE_LENGTH_4BYTE_SHIFT = 20,
 };
 
-void
+static void
 build_prepend_package_length(GArray *package, unsigned length, bool incl_self)
 {
 uint8_t byte;
@@ -226,13 +226,13 @@ build_append_pkg_length(GArray *array, unsigned length, 
bool incl_self)
 build_free_array(tmp);
 }
 
-void build_package(GArray *package, uint8_t op)
+static void build_package(GArray *package, uint8_t op)
 {
 build_prepend_package_length(package, package->len, true);
 build_prepend_byte(package, op);
 }
 
-void build_extop_package(GArray *package, uint8_t op)
+static void build_extop_package(GArray *package, uint8_t op)
 {
 build_package(package, op);
 build_prepend_byte(package, 0x5B); /* ExtOpPrefix */
@@ -248,7 +248,7 @@ static void build_append_int_noprefix(GArray *table, 
uint64_t value, int size)
 }
 }
 
-void build_append_int(GArray *table, uint64_t value)
+static void build_append_int(GArray *table, uint64_t value)
 {
 if (value == 0x00) {
 build_append_byte(table, 0x00); /* ZeroOp */
diff --git a/include/hw/acpi/aml-build.h b/include/hw/acpi/aml-build.h
index 1187197..f6735ea 100644
--- a/include/hw/acpi/aml-build.h
+++ b/include/hw/acpi/aml-build.h
@@ -188,20 +188,4 @@ Aml *aml_resource_template(void);
 Aml *aml_field(const char *name, AmlFieldFlags flags);
 Aml *aml_varpackage(uint32_t num_elements);
 
-/* other helpers */
-GArray *build_alloc_array(void);
-void build_free_array(GArray *array);
-void build_prepend_byte(GArray *array, uint8_t val);
-void build_append_byte(GArray *array, uint8_t val);
-void build_append_array(GArray *array, GArray *val);
-
-void GCC_FMT_ATTR(2, 3)
-build_append_namestring(GArray *array, const char *format, ...);
-
-void
-build_prepend_package_length(GArray *package, unsigned length, bool incl_self);
-void build_package(GArray *package, uint8_t op);
-void build_append_int(GArray *table, uint64_t value);
-void build_extop_package(GArray *package, uint8_t op);
-
 #endif
-- 
1.8.3.1

[Qemu-devel] [PATCH v5 03/16] pc: acpi: drop manual hole punching for CPU hotplug resources

[Igor Mammedov]

Drops manual hole punching in PCI0._CRS on PIIX4 machine type
for CPU hotplug resources.
Resources will be consumed by Device(PRES) that is attached
to PCI bus. The same way how it currently works for mem hotlpug.

Signed-off-by: Igor Mammedov 
---
 hw/i386/acpi-build.c | 6 +-
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index 02e2597..0de261a 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -885,11 +885,7 @@ build_ssdt(GArray *table_data, GArray *linker,
 aml_append(crs,
 aml_word_io(aml_min_fixed, aml_max_fixed,
 aml_pos_decode, aml_entire_range,
-0x, 0x0D00, 0xAEFF, 0x, 0xA200));
-aml_append(crs,
-aml_word_io(aml_min_fixed, aml_max_fixed,
-aml_pos_decode, aml_entire_range,
-0x, 0xAF20, 0xAFDF, 0x, 0x00C0));
+0x, 0x0D00, 0xAFDF, 0x, 0xA2E0));
 aml_append(crs,
 aml_word_io(aml_min_fixed, aml_max_fixed,
 aml_pos_decode, aml_entire_range,
-- 
1.8.3.1

[Qemu-devel] [PATCH v5 08/16] pc: acpi-build: drop template patching and create Device(SMC) dynamically

[Igor Mammedov]

patch moves SMC device into SSDT and creates it only
when device is present, which makes ACPI tables smaller
in default case when device is not present.

Also it fixes wrong IO range in CRS if "iobase"
property is set to a non default value.

PS:
Testing with XP shows that current default "iobase"
used SMC device conflicts with floppy controller IO,
but it's topic for another patch and I'd leave it
to SMC device author for resolving conflict.

Signed-off-by: Igor Mammedov 
CC: ag...@suse.de
---
 hw/i386/acpi-build.c  | 29 ++---
 hw/i386/acpi-dsdt-isa.dsl | 11 ---
 hw/i386/acpi-dsdt.dsl |  1 -
 hw/i386/q35-acpi-dsdt.dsl |  1 -
 4 files changed, 22 insertions(+), 20 deletions(-)

diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index badfa73..05eb80a 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -116,6 +116,7 @@ typedef struct AcpiMiscInfo {
 const unsigned char *dsdt_code;
 unsigned dsdt_size;
 uint16_t pvpanic_port;
+uint16_t applesmc_io_base;
 } AcpiMiscInfo;
 
 typedef struct AcpiBuildPciBusHotplugState {
@@ -127,7 +128,6 @@ typedef struct AcpiBuildPciBusHotplugState {
 
 static void acpi_get_dsdt(AcpiMiscInfo *info)
 {
-uint16_t *applesmc_sta;
 Object *piix = piix4_pm_find();
 Object *lpc = ich9_lpc_find();
 assert(!!piix != !!lpc);
@@ -135,17 +135,11 @@ static void acpi_get_dsdt(AcpiMiscInfo *info)
 if (piix) {
 info->dsdt_code = AcpiDsdtAmlCode;
 info->dsdt_size = sizeof AcpiDsdtAmlCode;
-applesmc_sta = piix_dsdt_applesmc_sta;
 }
 if (lpc) {
 info->dsdt_code = Q35AcpiDsdtAmlCode;
 info->dsdt_size = sizeof Q35AcpiDsdtAmlCode;
-applesmc_sta = q35_dsdt_applesmc_sta;
 }
-
-/* Patch in appropriate value for AppleSMC _STA */
-*(uint8_t *)(info->dsdt_code + *applesmc_sta) =
-applesmc_port() ? 0x0b : 0x00;
 }
 
 static
@@ -248,6 +242,7 @@ static void acpi_get_misc_info(AcpiMiscInfo *info)
 info->has_hpet = hpet_find();
 info->has_tpm = tpm_find();
 info->pvpanic_port = pvpanic_port();
+info->applesmc_io_base = applesmc_port();
 }
 
 static void acpi_get_pci_info(PcPciInfo *info)
@@ -955,6 +950,26 @@ build_ssdt(GArray *table_data, GArray *linker,
 aml_append(scope, aml_name_decl("_S5", pkg));
 aml_append(ssdt, scope);
 
+if (misc->applesmc_io_base) {
+scope = aml_scope("\\_SB.PCI0.ISA");
+dev = aml_device("SMC");
+
+aml_append(dev, aml_name_decl("_HID", aml_eisaid("APP0001")));
+/* device present, functioning, decoding, not shown in UI */
+aml_append(dev, aml_name_decl("_STA", aml_int(0xB)));
+
+crs = aml_resource_template();
+aml_append(crs,
+aml_io(aml_decode16, misc->applesmc_io_base, 
misc->applesmc_io_base,
+   0x01, APPLESMC_MAX_DATA_LENGTH)
+);
+aml_append(crs, aml_irq_no_flags(6));
+aml_append(dev, aml_name_decl("_CRS", crs));
+
+aml_append(scope, dev);
+aml_append(ssdt, scope);
+}
+
 if (misc->pvpanic_port) {
 scope = aml_scope("\\_SB.PCI0.ISA");
 
diff --git a/hw/i386/acpi-dsdt-isa.dsl b/hw/i386/acpi-dsdt-isa.dsl
index deb37de..89caa16 100644
--- a/hw/i386/acpi-dsdt-isa.dsl
+++ b/hw/i386/acpi-dsdt-isa.dsl
@@ -16,17 +16,6 @@
 /* Common legacy ISA style devices. */
 Scope(\_SB.PCI0.ISA) {
 
-Device (SMC) {
-Name(_HID, EisaId("APP0001"))
-/* _STA will be patched to 0x0B if AppleSMC is present */
-ACPI_EXTRACT_NAME_BYTE_CONST DSDT_APPLESMC_STA
-Name(_STA, 0xF0)
-Name(_CRS, ResourceTemplate () {
-IO (Decode16, 0x0300, 0x0300, 0x01, 0x20)
-IRQNoFlags() { 6 }
-})
-}
-
 Device(RTC) {
 Name(_HID, EisaId("PNP0B00"))
 Name(_CRS, ResourceTemplate() {
diff --git a/hw/i386/acpi-dsdt.dsl b/hw/i386/acpi-dsdt.dsl
index 09b68f0..a2d84ec 100644
--- a/hw/i386/acpi-dsdt.dsl
+++ b/hw/i386/acpi-dsdt.dsl
@@ -85,7 +85,6 @@ DefinitionBlock (
 }
 }
 
-#define DSDT_APPLESMC_STA piix_dsdt_applesmc_sta
 #include "acpi-dsdt-isa.dsl"
 
 
diff --git a/hw/i386/q35-acpi-dsdt.dsl b/hw/i386/q35-acpi-dsdt.dsl
index 3fb4b2f..16eaca3 100644
--- a/hw/i386/q35-acpi-dsdt.dsl
+++ b/hw/i386/q35-acpi-dsdt.dsl
@@ -150,7 +150,6 @@ DefinitionBlock (
 }
 }
 
-#define DSDT_APPLESMC_STA q35_dsdt_applesmc_sta
 #include "acpi-dsdt-isa.dsl"
 
 
-- 
1.8.3.1

[Qemu-devel] [PATCH v5 05/16] pc: acpi-build: drop remaining ssdt_misc template

[Igor Mammedov]

It drops empty ssdt_misc templete. It also hides
from user almost all pointer arithmetic when building
SSDT which makes resulting code a bit cleaner
and concentrating only on composing ASL construct
/i.e. a task build_ssdt() should be doing/.

Also it makes one binary blob less stored in QEMU
source tree by removing need to keep and update
hw/i386/ssdt-misc.hex.generated file here in total
saving us ~430LOC.

Signed-off-by: Igor Mammedov 
---
 hw/i386/Makefile.objs |  1 -
 hw/i386/acpi-build.c  |  7 ++-
 hw/i386/ssdt-misc.dsl | 21 -
 3 files changed, 2 insertions(+), 27 deletions(-)
 delete mode 100644 hw/i386/ssdt-misc.dsl

diff --git a/hw/i386/Makefile.objs b/hw/i386/Makefile.objs
index 45b90a8..9b00568 100644
--- a/hw/i386/Makefile.objs
+++ b/hw/i386/Makefile.objs
@@ -10,7 +10,6 @@ obj-y += acpi-build.o
 hw/i386/acpi-build.o: hw/i386/acpi-build.c \
hw/i386/acpi-dsdt.hex hw/i386/q35-acpi-dsdt.hex \
hw/i386/ssdt-pcihp.hex \
-   hw/i386/ssdt-misc.hex \
hw/i386/ssdt-tpm.hex
 
 iasl-option=$(shell if test -z "`$(1) $(2) 2>&1 > /dev/null`" \
diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index 2700154..01d988c 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -526,7 +526,6 @@ static inline char acpi_get_hex(uint32_t val)
 #define ACPI_SSDT_SIGNATURE 0x54445353 /* SSDT */
 #define ACPI_SSDT_HEADER_LENGTH 36
 
-#include "hw/i386/ssdt-misc.hex"
 #include "hw/i386/ssdt-pcihp.hex"
 #include "hw/i386/ssdt-tpm.hex"
 
@@ -850,7 +849,6 @@ build_ssdt(GArray *table_data, GArray *linker,
 MachineState *machine = MACHINE(qdev_get_machine());
 uint32_t nr_mem = machine->ram_slots;
 unsigned acpi_cpus = guest_info->apic_id_limit;
-uint8_t *ssdt_ptr;
 Aml *ssdt, *sb_scope, *scope, *pkg, *dev, *method, *crs, *field, *ifctx;
 int i;
 
@@ -860,9 +858,8 @@ build_ssdt(GArray *table_data, GArray *linker,
 QEMU_BUILD_BUG_ON(ACPI_CPU_HOTPLUG_ID_LIMIT > 256);
 g_assert(acpi_cpus <= ACPI_CPU_HOTPLUG_ID_LIMIT);
 
-/* Copy header and patch values in the S3_ / S4_ / S5_ packages */
-ssdt_ptr = acpi_data_push(ssdt->buf, sizeof(ssdp_misc_aml));
-memcpy(ssdt_ptr, ssdp_misc_aml, sizeof(ssdp_misc_aml));
+/* Reserve space for header */
+acpi_data_push(ssdt->buf, sizeof(AcpiTableHeader));
 
 scope = aml_scope("\\_SB.PCI0");
 /* build PCI0._CRS */
diff --git a/hw/i386/ssdt-misc.dsl b/hw/i386/ssdt-misc.dsl
deleted file mode 100644
index 8d61f21..000
--- a/hw/i386/ssdt-misc.dsl
+++ /dev/null
@@ -1,21 +0,0 @@
-/*
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
-
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU General Public License for more details.
-
- * You should have received a copy of the GNU General Public License along
- * with this program; if not, see .
- */
-#include "hw/acpi/pc-hotplug.h"
-
-ACPI_EXTRACT_ALL_CODE ssdp_misc_aml
-
-DefinitionBlock ("ssdt-misc.aml", "SSDT", 0x01, "BXPC", "BXSSDTSUSP", 0x1)
-{
-}
-- 
1.8.3.1

[Qemu-devel] [PATCH v5 02/16] pc: acpi: drop manual hole punching for PCI hotplug resources

[Igor Mammedov]

Drops manual hole punching in PCI0._CRS for PIIX4 machine type.
Resources will be consumed by Device(PHPR) that cwis attached
to PCI bus. The same way how it currently works for mem hotlpug.

Manual hole in PIIX4 _CRS wasn't correct anyway since it was
legacy size 0xF while current PCIHP MMIO region is of size 0x14.

Signed-off-by: Igor Mammedov 
---
 hw/i386/acpi-build.c | 6 +-
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index 355f9b7..02e2597 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -885,11 +885,7 @@ build_ssdt(GArray *table_data, GArray *linker,
 aml_append(crs,
 aml_word_io(aml_min_fixed, aml_max_fixed,
 aml_pos_decode, aml_entire_range,
-0x, 0x0D00, 0xADFF, 0x, 0xA100));
-aml_append(crs,
-aml_word_io(aml_min_fixed, aml_max_fixed,
-aml_pos_decode, aml_entire_range,
-0x, 0xAE0F, 0xAEFF, 0x, 0x00F1));
+0x, 0x0D00, 0xAEFF, 0x, 0xA200));
 aml_append(crs,
 aml_word_io(aml_min_fixed, aml_max_fixed,
 aml_pos_decode, aml_entire_range,
-- 
1.8.3.1

[Qemu-devel] [PATCH v5 04/16] pc: acpi: drop manual hole punching for GPE0 resources

[Igor Mammedov]

Drops manual hole punching in PCI0._CRS on PIIX4 machine type
for GPE0 resources. Resources will be consumed by Device(GPE0)
that is attached to PCI namespace.
There is GPE device with HID ACPI0006 since ACPI2.0
that should be used for this purpose but none of Windows
versions support it and show it as "unknown device",
so reserve resource in old fashioned way with PNP0A06
device to make windows happy and actually reserve resources.

Along with last hole _CRS layout of PIIX4 machine becomes
the same as Q35 one, so merge them together and use the same
_CRS for both machine types.

Signed-off-by: Igor Mammedov 
---
 hw/i386/acpi-build.c | 32 +---
 1 file changed, 17 insertions(+), 15 deletions(-)

diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index 0de261a..2700154 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -876,21 +876,10 @@ build_ssdt(GArray *table_data, GArray *linker,
 aml_word_io(aml_min_fixed, aml_max_fixed,
 aml_pos_decode, aml_entire_range,
 0x, 0x, 0x0CF7, 0x, 0x0CF8));
-if (ich9_lpc_find()) { /* Q35 */
-aml_append(crs,
-aml_word_io(aml_min_fixed, aml_max_fixed,
-aml_pos_decode, aml_entire_range,
-0x, 0x0D00, 0x, 0x, 0xF300));
-} else { /* piix4 */
-aml_append(crs,
-aml_word_io(aml_min_fixed, aml_max_fixed,
-aml_pos_decode, aml_entire_range,
-0x, 0x0D00, 0xAFDF, 0x, 0xA2E0));
-aml_append(crs,
-aml_word_io(aml_min_fixed, aml_max_fixed,
-aml_pos_decode, aml_entire_range,
-0x, 0xAFE4, 0x, 0x, 0x501C));
-}
+aml_append(crs,
+aml_word_io(aml_min_fixed, aml_max_fixed,
+aml_pos_decode, aml_entire_range,
+0x, 0x0D00, 0x, 0x, 0xF300));
 aml_append(crs,
 aml_dword_memory(aml_pos_decode, aml_min_fixed, aml_max_fixed,
  aml_cacheable, aml_ReadWrite,
@@ -909,6 +898,19 @@ build_ssdt(GArray *table_data, GArray *linker,
 }
 aml_append(scope, aml_name_decl("_CRS", crs));
 
+/* reserve GPE0 block resources */
+dev = aml_device("GPE0");
+aml_append(dev, aml_name_decl("_HID", aml_string("PNP0A06")));
+aml_append(dev, aml_name_decl("_UID", aml_string("GPE0 resources")));
+/* device present, functioning, decoding, not shown in UI */
+aml_append(dev, aml_name_decl("_STA", aml_int(0xB)));
+crs = aml_resource_template();
+aml_append(crs,
+aml_io(aml_decode16, pm->gpe0_blk, pm->gpe0_blk, 1, pm->gpe0_blk_len)
+);
+aml_append(dev, aml_name_decl("_CRS", crs));
+aml_append(scope, dev);
+
 /* reserve PCIHP resources */
 if (pm->pcihp_io_len) {
 dev = aml_device("PHPR");
-- 
1.8.3.1

[Qemu-devel] [PATCH v5 07/16] pc: export applesmc IO port/len

[Igor Mammedov]

IO port and length will be used in following patch
to correctly generate SMC ACPI device in SSDT.

Signed-off-by: Igor Mammedov 
---
 hw/i386/acpi-build.c |  2 +-
 hw/misc/applesmc.c   |  5 ++---
 include/hw/isa/isa.h | 11 +--
 3 files changed, 12 insertions(+), 6 deletions(-)

diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index 01d988c..badfa73 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -145,7 +145,7 @@ static void acpi_get_dsdt(AcpiMiscInfo *info)
 
 /* Patch in appropriate value for AppleSMC _STA */
 *(uint8_t *)(info->dsdt_code + *applesmc_sta) =
-applesmc_find() ? 0x0b : 0x00;
+applesmc_port() ? 0x0b : 0x00;
 }
 
 static
diff --git a/hw/misc/applesmc.c b/hw/misc/applesmc.c
index 6a56b07..6bd61e7 100644
--- a/hw/misc/applesmc.c
+++ b/hw/misc/applesmc.c
@@ -43,7 +43,6 @@
 /* command/status port used by Apple SMC */
 #define APPLESMC_CMD_PORT  0x4
 #define APPLESMC_NR_PORTS  32
-#define APPLESMC_MAX_DATA_LENGTH   32
 
 #define APPLESMC_READ_CMD  0x10
 #define APPLESMC_WRITE_CMD 0x11
@@ -249,8 +248,8 @@ static void applesmc_isa_realize(DeviceState *dev, Error 
**errp)
 }
 
 static Property applesmc_isa_properties[] = {
-DEFINE_PROP_UINT32("iobase", AppleSMCState, iobase,
-  APPLESMC_DEFAULT_IOBASE),
+DEFINE_PROP_UINT32(APPLESMC_PROP_IO_BASE, AppleSMCState, iobase,
+   APPLESMC_DEFAULT_IOBASE),
 DEFINE_PROP_STRING("osk", AppleSMCState, osk),
 DEFINE_PROP_END_OF_LIST(),
 };
diff --git a/include/hw/isa/isa.h b/include/hw/isa/isa.h
index e0c749f..1ee9fa0 100644
--- a/include/hw/isa/isa.h
+++ b/include/hw/isa/isa.h
@@ -21,10 +21,17 @@
 #define ISA_BUS(obj) OBJECT_CHECK(ISABus, (obj), TYPE_ISA_BUS)
 
 #define TYPE_APPLE_SMC "isa-applesmc"
+#define APPLESMC_MAX_DATA_LENGTH   32
+#define APPLESMC_PROP_IO_BASE "iobase"
 
-static inline bool applesmc_find(void)
+static inline uint16_t applesmc_port(void)
 {
-return object_resolve_path_type("", TYPE_APPLE_SMC, NULL);
+Object *obj = object_resolve_path_type("", TYPE_APPLE_SMC, NULL);
+
+if (obj) {
+return object_property_get_int(obj, APPLESMC_PROP_IO_BASE, NULL);
+}
+return 0;
 }
 
 typedef struct ISADeviceClass {
-- 
1.8.3.1

[Qemu-devel] [PATCH v5 00/16] ACPI refactoring: replace template patching with C AML API

[Igor Mammedov]

NOTE to maintaner:
Needs updating DSDT hex.generated and ACPI tests blobs

Changes since v4:
  * split PCI0._CRS patch onto simple ones
 * move _CRS into SSDT
 * a series to drop manual hole punching in PIIX4 _CRS
  * add test case for ACPI table with coldplugged bridge
  * fixed up current algorithm rewrite fallout on Q35 machine,
see for detailed changelog in
 "pc: acpi-build: simplify PCI bus tree generation"
  * fixed minor mst's notes on v4 in PCI patches

Remaining pieces of refactoring PC/Q35 machines to AML API.
There is no more template patching left.

There are TPM and DSDT table blobs left to convert but
that's left for future.

Boot tested with XP3/WS2012R2, correctnes of resource reservation
was checked with WS2012R2 help.

reference to previous iteration:
https://www.mail-archive.com/qemu-devel@nongnu.org/msg280791.html

tree for testing:
https://github.com/imammedo/qemu.git ASL_API_v5
for browsing:
https://github.com/imammedo/qemu/commits/ASL_API_v5


Igor Mammedov (16):
  pc: acpi-build: create PCI0._CRS dynamically
  pc: acpi: drop manual hole punching for PCI hotplug resources
  pc: acpi: drop manual hole punching for CPU hotplug resources
  pc: acpi: drop manual hole punching for GPE0 resources
  pc: acpi-build: drop remaining ssdt_misc template
  acpi: add acpi_irq_no_flags() term
  pc: export applesmc IO port/len
  pc: acpi-build: drop template patching and create Device(SMC)
dynamically
  tests: ACPI test blobs update due to PCI0._CRS changes
  tests: bios-tables-test: add support for testing bridges
  tests: add ACPI blobs for qemu with bridge cases
  pc: acpi-build: simplify PCI bus tree generation
  tests: ACPI: update pc/SSDT.bridge due to new alg of PCI tree creation
  pc: acpi-build: drop template patching and create PCI bus tree
dynamically
  pc: acpi: remove not used anymore ssdt-[misc|pcihp].hex.generated
blobs
  acpi: make build_*() routines static to aml-build.c

 hw/acpi/aml-build.c  |  41 ++-
 hw/i386/Makefile.objs|   2 -
 hw/i386/acpi-build.c | 491 ---
 hw/i386/acpi-dsdt-isa.dsl|  11 -
 hw/i386/acpi-dsdt-pci-crs.dsl|  92 ---
 hw/i386/acpi-dsdt.dsl|  46 
 hw/i386/q35-acpi-dsdt.dsl|  19 --
 hw/i386/ssdt-misc.dsl|  40 ---
 hw/i386/ssdt-misc.hex.generated  | 139 --
 hw/i386/ssdt-pcihp.dsl   | 100 ---
 hw/i386/ssdt-pcihp.hex.generated | 251 --
 hw/misc/applesmc.c   |   5 +-
 include/hw/acpi/aml-build.h  |  17 +-
 include/hw/isa/isa.h |  11 +-
 tests/acpi-test-data/pc/DSDT | Bin 3415 -> 2970 bytes
 tests/acpi-test-data/pc/SSDT | Bin 2374 -> 2480 bytes
 tests/acpi-test-data/pc/SSDT.bridge  | Bin 0 -> 4351 bytes
 tests/acpi-test-data/q35/DSDT| Bin 8005 -> 7608 bytes
 tests/acpi-test-data/q35/SSDT| Bin 578 -> 685 bytes
 tests/acpi-test-data/q35/SSDT.bridge | Bin 0 -> 702 bytes
 tests/bios-tables-test.c |  45 +++-
 21 files changed, 250 insertions(+), 1060 deletions(-)
 delete mode 100644 hw/i386/acpi-dsdt-pci-crs.dsl
 delete mode 100644 hw/i386/ssdt-misc.dsl
 delete mode 100644 hw/i386/ssdt-misc.hex.generated
 delete mode 100644 hw/i386/ssdt-pcihp.dsl
 delete mode 100644 hw/i386/ssdt-pcihp.hex.generated
 create mode 100644 tests/acpi-test-data/pc/SSDT.bridge
 create mode 100644 tests/acpi-test-data/q35/SSDT.bridge

-- 
1.8.3.1

[Qemu-devel] [PATCH v5 06/16] acpi: add acpi_irq_no_flags() term

[Igor Mammedov]

Signed-off-by: Igor Mammedov 
---
 hw/acpi/aml-build.c | 21 +
 include/hw/acpi/aml-build.h |  1 +
 2 files changed, 22 insertions(+)

diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c
index d793775..60245e7 100644
--- a/hw/acpi/aml-build.c
+++ b/hw/acpi/aml-build.c
@@ -514,6 +514,27 @@ Aml *aml_io(AmlIODecode dec, uint16_t min_base, uint16_t 
max_base,
 return var;
 }
 
+/*
+ * ACPI 1.0b: 6.4.2.1.1 ASL Macro for IRQ Descriptor
+ *
+ * More verbose description at:
+ * ACPI 5.0: 19.5.64 IRQNoFlags (Interrupt Resource Descriptor Macro)
+ *   6.4.2.1 IRQ Descriptor
+ */
+Aml *aml_irq_no_flags(uint8_t irq)
+{
+uint16_t irq_mask;
+Aml *var = aml_alloc();
+
+assert(irq < 16);
+build_append_byte(var->buf, 0x22); /* IRQ descriptor 2 byte form */
+
+irq_mask = 1U << irq;
+build_append_byte(var->buf, irq_mask & 0xFF); /* IRQ mask bits[7:0] */
+build_append_byte(var->buf, irq_mask >> 8); /* IRQ mask bits[15:8] */
+return var;
+}
+
 /* ACPI 1.0b: 16.2.5.4 Type 2 Opcodes Encoding: DefLEqual */
 Aml *aml_equal(Aml *arg1, Aml *arg2)
 {
diff --git a/include/hw/acpi/aml-build.h b/include/hw/acpi/aml-build.h
index d2b2c35..1187197 100644
--- a/include/hw/acpi/aml-build.h
+++ b/include/hw/acpi/aml-build.h
@@ -146,6 +146,7 @@ Aml *aml_io(AmlIODecode dec, uint16_t min_base, uint16_t 
max_base,
 uint8_t aln, uint8_t len);
 Aml *aml_operation_region(const char *name, AmlRegionSpace rs,
   uint32_t offset, uint32_t len);
+Aml *aml_irq_no_flags(uint8_t irq);
 Aml *aml_named_field(const char *name, unsigned length);
 Aml *aml_reserved_field(unsigned length);
 Aml *aml_local(int num);
-- 
1.8.3.1

[Qemu-devel] [PATCH v5 01/16] pc: acpi-build: create PCI0._CRS dynamically

[Igor Mammedov]

Replace template patching and runtime calculation
in _CRS() method with static _CRS defined in SSDT.
No functional change except of as mentined above
and _CRS being moved from DSDT to SSDT.

Signed-off-by: Igor Mammedov 
---
 hw/i386/acpi-build.c  | 88 -
 hw/i386/acpi-dsdt-pci-crs.dsl | 92 ---
 hw/i386/acpi-dsdt.dsl | 45 -
 hw/i386/q35-acpi-dsdt.dsl | 18 -
 hw/i386/ssdt-misc.dsl | 19 -
 5 files changed, 52 insertions(+), 210 deletions(-)
 delete mode 100644 hw/i386/acpi-dsdt-pci-crs.dsl

diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index 4d5d7e3..355f9b7 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -346,24 +346,6 @@ static void acpi_align_size(GArray *blob, unsigned align)
 g_array_set_size(blob, ROUND_UP(acpi_data_len(blob), align));
 }
 
-/* Set a value within table in a safe manner */
-#define ACPI_BUILD_SET_LE(table, size, off, bits, val) \
-do { \
-uint64_t ACPI_BUILD_SET_LE_val = cpu_to_le64(val); \
-memcpy(acpi_data_get_ptr(table, size, off, \
- (bits) / BITS_PER_BYTE), \
-   &ACPI_BUILD_SET_LE_val, \
-   (bits) / BITS_PER_BYTE); \
-} while (0)
-
-static inline void *acpi_data_get_ptr(uint8_t *table_data, unsigned table_size,
-  unsigned off, unsigned size)
-{
-assert(off + size > off);
-assert(off + size <= table_size);
-return table_data + off;
-}
-
 static inline void acpi_add_table(GArray *table_offsets, GArray *table_data)
 {
 uint32_t offset = cpu_to_le32(table_data->len);
@@ -860,22 +842,6 @@ static void build_pci_bus_end(PCIBus *bus, void *bus_state)
 g_free(child);
 }
 
-static void patch_pci_windows(PcPciInfo *pci, uint8_t *start, unsigned size)
-{
-ACPI_BUILD_SET_LE(start, size, acpi_pci32_start[0], 32, pci->w32.begin);
-
-ACPI_BUILD_SET_LE(start, size, acpi_pci32_end[0], 32, pci->w32.end - 1);
-
-if (pci->w64.end || pci->w64.begin) {
-ACPI_BUILD_SET_LE(start, size, acpi_pci64_valid[0], 8, 1);
-ACPI_BUILD_SET_LE(start, size, acpi_pci64_start[0], 64, 
pci->w64.begin);
-ACPI_BUILD_SET_LE(start, size, acpi_pci64_end[0], 64, pci->w64.end - 
1);
-ACPI_BUILD_SET_LE(start, size, acpi_pci64_length[0], 64, pci->w64.end 
- pci->w64.begin);
-} else {
-ACPI_BUILD_SET_LE(start, size, acpi_pci64_valid[0], 8, 0);
-}
-}
-
 static void
 build_ssdt(GArray *table_data, GArray *linker,
AcpiCpuInfo *cpu, AcpiPmInfo *pm, AcpiMiscInfo *misc,
@@ -898,9 +864,59 @@ build_ssdt(GArray *table_data, GArray *linker,
 ssdt_ptr = acpi_data_push(ssdt->buf, sizeof(ssdp_misc_aml));
 memcpy(ssdt_ptr, ssdp_misc_aml, sizeof(ssdp_misc_aml));
 
-patch_pci_windows(pci, ssdt_ptr, sizeof(ssdp_misc_aml));
-
 scope = aml_scope("\\_SB.PCI0");
+/* build PCI0._CRS */
+crs = aml_resource_template();
+aml_append(crs,
+aml_word_bus_number(aml_min_fixed, aml_max_fixed, aml_pos_decode,
+0x, 0x, 0x00FF, 0x, 0x0100));
+aml_append(crs, aml_io(aml_decode16, 0x0CF8, 0x0CF8, 0x01, 0x08));
+
+aml_append(crs,
+aml_word_io(aml_min_fixed, aml_max_fixed,
+aml_pos_decode, aml_entire_range,
+0x, 0x, 0x0CF7, 0x, 0x0CF8));
+if (ich9_lpc_find()) { /* Q35 */
+aml_append(crs,
+aml_word_io(aml_min_fixed, aml_max_fixed,
+aml_pos_decode, aml_entire_range,
+0x, 0x0D00, 0x, 0x, 0xF300));
+} else { /* piix4 */
+aml_append(crs,
+aml_word_io(aml_min_fixed, aml_max_fixed,
+aml_pos_decode, aml_entire_range,
+0x, 0x0D00, 0xADFF, 0x, 0xA100));
+aml_append(crs,
+aml_word_io(aml_min_fixed, aml_max_fixed,
+aml_pos_decode, aml_entire_range,
+0x, 0xAE0F, 0xAEFF, 0x, 0x00F1));
+aml_append(crs,
+aml_word_io(aml_min_fixed, aml_max_fixed,
+aml_pos_decode, aml_entire_range,
+0x, 0xAF20, 0xAFDF, 0x, 0x00C0));
+aml_append(crs,
+aml_word_io(aml_min_fixed, aml_max_fixed,
+aml_pos_decode, aml_entire_range,
+0x, 0xAFE4, 0x, 0x, 0x501C));
+}
+aml_append(crs,
+aml_dword_memory(aml_pos_decode, aml_min_fixed, aml_max_fixed,
+ aml_cacheable, aml_ReadWrite,
+ 0, 0x000A, 0x000B, 0, 0x0002));
+aml_append(crs,
+aml_dword_memory(aml_pos_decode, aml_min_fixed, aml_max_fixed,
+ aml_non_cacheable, aml_ReadWrite,
+ 0, pci->w32.begin, pci->w32.end - 1, 0,
+  

Re: [Qemu-devel] [RFC PATCH v2 10/15] cpu-model/s390: Add cpu class initialization routines

[Richard Henderson]

On 02/17/2015 06:24 AM, Michael Mueller wrote:
> +static inline uint64_t big_endian_bit(unsigned long nr)
> +{
> +return 1ul << (BITS_PER_LONG - (nr % BITS_PER_LONG));
> +};

This is buggy.  NR=0 should map to 63, not 64.

> +return !!(*ptr & big_endian_bit(nr));

Personally I dislike !! as an idiom.  Given that big_endian_bit isn't used
anywhere else, can we integrate it and change this to

static inline int test_facility(unsigned long nr, uint64_t *fac_list)
{
  unsigned long word = nr / BITS_PER_LONG;
  unsigned long be_bit = 63 - (nr % BITS_PER_LONG);
  return (fac_list[word] >> be_bit) & 1;
}


r~

[Qemu-devel] [PATCH v2 for-2.3 0/3] Support more than 8 MMU modes, speedup PPC by 10%

[Paolo Bonzini]

Patches 1 and 2 enable support from more than 8 MMU modes in TCG (patch
1 is in the targets, patch 2 is in cpu-defs.h).  The TLB size is reduced
proportionally on targets where that is necessary.

Patch 3 uses the new support in the PPC target.

Paolo

Paolo Bonzini (3):
  tcg: add TCG_TARGET_TLB_DISPLACEMENT_BITS
  softmmu: support up to 12 MMU modes
  target-ppc: use separate indices for various translation modes

 include/exec/cpu-defs.h  |  34 +++-
 include/exec/cpu_ldst.h  | 104 ---
 target-ppc/cpu.h |  12 +++---
 target-ppc/excp_helper.c |   3 --
 target-ppc/helper_regs.h |  15 ---
 tcg/aarch64/tcg-target.h |   1 +
 tcg/arm/tcg-target.h |   1 +
 tcg/i386/tcg-target.h|   1 +
 tcg/ia64/tcg-target.h|   2 +
 tcg/mips/tcg-target.h|   1 +
 tcg/ppc/tcg-target.h |   1 +
 tcg/s390/tcg-target.h|   1 +
 tcg/sparc/tcg-target.h   |   1 +
 tcg/tci/tcg-target.h |   1 +
 14 files changed, 156 insertions(+), 22 deletions(-)

-- 
2.3.0

Re: [Qemu-devel] [PATCH 3/3] target-ppc: use separate indices for various translation modes

[Paolo Bonzini]

On 20/02/2015 14:00, Alexander Graf wrote:
> Also please double-check that 440 still works. That was the target that
> gave me the most headaches on DR/IR switching so far.

The ppc-virtexml507-linux-2_6_34.tgz image works for me.

Paolo

> Otherwise looks simple and clean to me :).
> 
> 
> Alex
> 
>>  }
>>  
>>  static inline void hreg_compute_hflags(CPUPPCState *env)
>> @@ -56,7 +59,7 @@ static inline void hreg_compute_hflags(CPUPPCState *env)
>>  /* We 'forget' FE0 & FE1: we'll never generate imprecise exceptions */
>>  hflags_mask = (1 << MSR_VR) | (1 << MSR_AP) | (1 << MSR_SA) |
>>  (1 << MSR_PR) | (1 << MSR_FP) | (1 << MSR_SE) | (1 << MSR_BE) |
>> -(1 << MSR_LE) | (1 << MSR_VSX);
>> +(1 << MSR_LE) | (1 << MSR_VSX) | (1 << MSR_IR) | (1 << MSR_DR);
>>  hflags_mask |= (1ULL << MSR_CM) | (1ULL << MSR_SF) | MSR_HVB;
>>  hreg_compute_mem_idx(env);
>>  env->hflags = env->msr & hflags_mask;
>> @@ -82,8 +85,6 @@ static inline int hreg_store_msr(CPUPPCState *env, 
>> target_ulong value,
>>  }
>>  if (((value >> MSR_IR) & 1) != msr_ir ||
>>  ((value >> MSR_DR) & 1) != msr_dr) {
>> -/* Flush all tlb when changing translation mode */
>> -tlb_flush(cs, 1);
>>  excp = POWERPC_EXCP_NONE;
>>  cs->interrupt_request |= CPU_INTERRUPT_EXITTB;
>>  }
>>
> 
> 

[Qemu-devel] [PATCH 2/3] softmmu: support up to 12 MMU modes

[Paolo Bonzini]

At 8k per TLB (for 64-bit host or target), 8 or more modes
make the TLBs bigger than 64k, and some RISC TCG backends do
not like that.  On the affected hosts, cut the TLB size in
half---there is still a measurable speedup on PPC with the
next patch.

Signed-off-by: Paolo Bonzini 
---
 include/exec/cpu-defs.h |  34 +++-
 include/exec/cpu_ldst.h | 104 +---
 2 files changed, 130 insertions(+), 8 deletions(-)

diff --git a/include/exec/cpu-defs.h b/include/exec/cpu-defs.h
index 0ca6f0b..2662de5 100644
--- a/include/exec/cpu-defs.h
+++ b/include/exec/cpu-defs.h
@@ -27,6 +27,7 @@
 #include 
 #include "qemu/osdep.h"
 #include "qemu/queue.h"
+#include "tcg-target.h"
 #ifndef CONFIG_USER_ONLY
 #include "exec/hwaddr.h"
 #endif
@@ -69,8 +70,6 @@ typedef uint64_t target_ulong;
 #define TB_JMP_PAGE_MASK (TB_JMP_CACHE_SIZE - TB_JMP_PAGE_SIZE)
 
 #if !defined(CONFIG_USER_ONLY)
-#define CPU_TLB_BITS 8
-#define CPU_TLB_SIZE (1 << CPU_TLB_BITS)
 /* use a fully associative victim tlb of 8 entries */
 #define CPU_VTLB_SIZE 8
 
@@ -80,6 +79,37 @@ typedef uint64_t target_ulong;
 #define CPU_TLB_ENTRY_BITS 5
 #endif
 
+/* TCG_TARGET_TLB_DISPLACEMENT_BITS is used in CPU_TLB_BITS to ensure that
+ * the TLB is not unnecessarily small, but still small enough for the
+ * TLB lookup instruction sequence used by the TCG target.
+ *
+ * TCG will have to generate an operand as large as the distance between
+ * tlb_table[0][0] and the tlb_table[NB_MMU_MODES - 1][0].addend.  For
+ * simplicity, the TCG targets just round everything up to the next
+ * power of two, and count bits.  This works because: 1) the size of each
+ * TLB is a largish power of two, 2) and because the limit of the
+ * displacement is really close to a power of two.
+ *
+ * For example, the maximum displacement 0xFFF0 on PPC and MIPS, but TCG
+ * just says "the displacement is 16 bits".  TCG_TARGET_TLB_DISPLACEMENT_BITS
+ * then ensures that tlb_table at least 0x8000 bytes large ("not unnecessarily
+ * small": 2^15).  The operand then will come up smaller than 0xFFF0 without
+ * any particular care, because the TLB for a single MMU mode is larger than
+ * 0x1-0xFFF0=16 bytes.  In the end, the maximum value of the operand
+ * will be something like 0xC018; here, 0xC000 is the offset of the last TLB
+ * table (tlb_table[NB_MMU_MODES - 1][0]), and 0x18 is the offset of the
+ * addend field in each TLB entry.
+ */
+#define CPU_TLB_BITS \
+MIN(8,   \
+TCG_TARGET_TLB_DISPLACEMENT_BITS - CPU_TLB_ENTRY_BITS -  \
+(NB_MMU_MODES <= 1 ? 0 : \
+ NB_MMU_MODES <= 2 ? 1 : \
+ NB_MMU_MODES <= 4 ? 2 : \
+ NB_MMU_MODES <= 8 ? 3 : 4))
+
+#define CPU_TLB_SIZE (1 << CPU_TLB_BITS)
+
 typedef struct CPUTLBEntry {
 /* bit TARGET_LONG_BITS to TARGET_PAGE_BITS : virtual address
bit TARGET_PAGE_BITS-1..4  : Nonzero for accesses that should not
diff --git a/include/exec/cpu_ldst.h b/include/exec/cpu_ldst.h
index 1673287..0ec398c 100644
--- a/include/exec/cpu_ldst.h
+++ b/include/exec/cpu_ldst.h
@@ -263,12 +263,104 @@ uint64_t helper_ldq_cmmu(CPUArchState *env, target_ulong 
addr, int mmu_idx);
 #undef MEMSUFFIX
 #endif /* (NB_MMU_MODES >= 7) */
 
-#if (NB_MMU_MODES > 7)
-/* Note that supporting NB_MMU_MODES == 9 would require
- * changes to at least the ARM TCG backend.
- */
-#error "NB_MMU_MODES > 7 is not supported for now"
-#endif /* (NB_MMU_MODES > 7) */
+#if (NB_MMU_MODES >= 8) && defined(MMU_MODE7_SUFFIX)
+
+#define CPU_MMU_INDEX 7
+#define MEMSUFFIX MMU_MODE7_SUFFIX
+#define DATA_SIZE 1
+#include "exec/cpu_ldst_template.h"
+
+#define DATA_SIZE 2
+#include "exec/cpu_ldst_template.h"
+
+#define DATA_SIZE 4
+#include "exec/cpu_ldst_template.h"
+
+#define DATA_SIZE 8
+#include "exec/cpu_ldst_template.h"
+#undef CPU_MMU_INDEX
+#undef MEMSUFFIX
+#endif /* (NB_MMU_MODES >= 8) */
+
+#if (NB_MMU_MODES >= 9) && defined(MMU_MODE8_SUFFIX)
+
+#define CPU_MMU_INDEX 8
+#define MEMSUFFIX MMU_MODE8_SUFFIX
+#define DATA_SIZE 1
+#include "exec/cpu_ldst_template.h"
+
+#define DATA_SIZE 2
+#include "exec/cpu_ldst_template.h"
+
+#define DATA_SIZE 4
+#include "exec/cpu_ldst_template.h"
+
+#define DATA_SIZE 8
+#include "exec/cpu_ldst_template.h"
+#undef CPU_MMU_INDEX
+#undef MEMSUFFIX
+#endif /* (NB_MMU_MODES >= 9) */
+
+#if (NB_MMU_MODES >= 10) && defined(MMU_MODE9_SUFFIX)
+
+#define CPU_MMU_INDEX 9
+#define MEMSUFFIX MMU_MODE9_SUFFIX
+#define DATA_SIZE 1
+#include "exec/cpu_ldst_template.h"
+
+#define DATA_SIZE 2
+#include "exec/cpu_ldst_template.h"
+
+#define DATA_SIZE 4
+#include "exec/cpu_ldst_template.h"
+
+#define DATA_SIZE 8
+#include "exec/cpu_ldst_template.h"
+#undef CPU_MMU_INDEX
+#undef MEMSUFFIX
+#endif /* (NB_MMU_MODES >= 10) */
+
+#if (NB_MMU_MODES >= 11) && defined(MMU_MODE10_SUFFIX)
+
+#define 

[Qemu-devel] [PATCH 3/3] target-ppc: use separate indices for various translation modes

[Paolo Bonzini]

PowerPC TCG flushes the TLB on every IR/DR change, which basically
means on every user<->kernel context switch.  Encode IR/DR in the
MMU index.

This brings the number of TLB flushes down from ~90 to ~5
for starting up the Debian installer, which is in line with x86
and gives a ~10% performance improvement.

Signed-off-by: Paolo Bonzini 
---
 target-ppc/cpu.h | 12 +++-
 target-ppc/excp_helper.c |  3 ---
 target-ppc/helper_regs.h | 15 +--
 3 files changed, 16 insertions(+), 14 deletions(-)

diff --git a/target-ppc/cpu.h b/target-ppc/cpu.h
index aae33a9..80755a4 100644
--- a/target-ppc/cpu.h
+++ b/target-ppc/cpu.h
@@ -943,7 +943,13 @@ struct ppc_segment_page_sizes {
 
 /*/
 /* The whole PowerPC CPU context */
-#define NB_MMU_MODES 3
+#define NB_MMU_MODES 12
+#define MMU_IDX_IR   1
+#define MMU_IDX_DR   2
+#define MMU_IDX_PR   0
+#define MMU_IDX_SUP  4
+#define MMU_IDX_HV   8
+#define MMU_USER_IDX (MMU_IDX_PR|MMU_IDX_IR|MMU_IDX_DR)
 
 #define PPC_CPU_OPCODES_LEN  0x40
 #define PPC_CPU_INDIRECT_OPCODES_LEN 0x20
@@ -1252,10 +1258,6 @@ static inline CPUPPCState *cpu_init(const char 
*cpu_model)
 #define cpu_list ppc_cpu_list
 
 /* MMU modes definitions */
-#define MMU_MODE0_SUFFIX _user
-#define MMU_MODE1_SUFFIX _kernel
-#define MMU_MODE2_SUFFIX _hypv
-#define MMU_USER_IDX 0
 static inline int cpu_mmu_index (CPUPPCState *env)
 {
 return env->mmu_idx;
diff --git a/target-ppc/excp_helper.c b/target-ppc/excp_helper.c
index b803475..f608701 100644
--- a/target-ppc/excp_helper.c
+++ b/target-ppc/excp_helper.c
@@ -623,9 +623,6 @@ static inline void powerpc_excp(PowerPCCPU *cpu, int 
excp_model, int excp)
 
 if (env->spr[SPR_LPCR] & LPCR_AIL) {
 new_msr |= (1 << MSR_IR) | (1 << MSR_DR);
-} else if (msr & ((1 << MSR_IR) | (1 << MSR_DR))) {
-/* If we disactivated any translation, flush TLBs */
-tlb_flush(cs, 1);
 }
 
 #ifdef TARGET_PPC64
diff --git a/target-ppc/helper_regs.h b/target-ppc/helper_regs.h
index 271fddf..97d28e0 100644
--- a/target-ppc/helper_regs.h
+++ b/target-ppc/helper_regs.h
@@ -41,12 +41,17 @@ static inline void hreg_swap_gpr_tgpr(CPUPPCState *env)
 
 static inline void hreg_compute_mem_idx(CPUPPCState *env)
 {
+int mmu_idx;
+
 /* Precompute MMU index */
-if (msr_pr == 0 && msr_hv != 0) {
-env->mmu_idx = 2;
+if (msr_pr == 1) {
+mmu_idx = MMU_IDX_PR;
 } else {
-env->mmu_idx = 1 - msr_pr;
+mmu_idx = msr_hv ? MMU_IDX_HV : MMU_IDX_SUP;
 }
+mmu_idx |= msr_ir ? MMU_IDX_IR : 0;
+mmu_idx |= msr_dr ? MMU_IDX_DR : 0;
+env->mmu_idx = mmu_idx;
 }
 
 static inline void hreg_compute_hflags(CPUPPCState *env)
@@ -56,7 +61,7 @@ static inline void hreg_compute_hflags(CPUPPCState *env)
 /* We 'forget' FE0 & FE1: we'll never generate imprecise exceptions */
 hflags_mask = (1 << MSR_VR) | (1 << MSR_AP) | (1 << MSR_SA) |
 (1 << MSR_PR) | (1 << MSR_FP) | (1 << MSR_SE) | (1 << MSR_BE) |
-(1 << MSR_LE) | (1 << MSR_VSX);
+(1 << MSR_LE) | (1 << MSR_VSX) | (1 << MSR_IR) | (1 << MSR_DR);
 hflags_mask |= (1ULL << MSR_CM) | (1ULL << MSR_SF) | MSR_HVB;
 hreg_compute_mem_idx(env);
 env->hflags = env->msr & hflags_mask;
@@ -82,8 +87,6 @@ static inline int hreg_store_msr(CPUPPCState *env, 
target_ulong value,
 }
 if (((value >> MSR_IR) & 1) != msr_ir ||
 ((value >> MSR_DR) & 1) != msr_dr) {
-/* Flush all tlb when changing translation mode */
-tlb_flush(cs, 1);
 excp = POWERPC_EXCP_NONE;
 cs->interrupt_request |= CPU_INTERRUPT_EXITTB;
 }
-- 
2.3.0

[Qemu-devel] [PATCH 1/3] tcg: add TCG_TARGET_TLB_DISPLACEMENT_BITS

[Paolo Bonzini]

This will be used to size the TLB when more than 8 MMU modes are
used by the target.  Limitations come from the limited size of
the immediate fields (which sometimes, as in the case of Aarch64,
extend to instructions that shift the immediate).

Signed-off-by: Paolo Bonzini 
---
 tcg/aarch64/tcg-target.h | 1 +
 tcg/arm/tcg-target.h | 1 +
 tcg/i386/tcg-target.h| 1 +
 tcg/ia64/tcg-target.h| 2 ++
 tcg/mips/tcg-target.h| 1 +
 tcg/ppc/tcg-target.h | 1 +
 tcg/s390/tcg-target.h| 1 +
 tcg/sparc/tcg-target.h   | 1 +
 tcg/tci/tcg-target.h | 1 +
 9 files changed, 10 insertions(+)

diff --git a/tcg/aarch64/tcg-target.h b/tcg/aarch64/tcg-target.h
index 60c7493..8aec04d 100644
--- a/tcg/aarch64/tcg-target.h
+++ b/tcg/aarch64/tcg-target.h
@@ -14,6 +14,7 @@
 #define TCG_TARGET_AARCH64 1
 
 #define TCG_TARGET_INSN_UNIT_SIZE  4
+#define TCG_TARGET_TLB_DISPLACEMENT_BITS 24
 #undef TCG_TARGET_STACK_GROWSUP
 
 typedef enum {
diff --git a/tcg/arm/tcg-target.h b/tcg/arm/tcg-target.h
index 1c719e2..6559f80 100644
--- a/tcg/arm/tcg-target.h
+++ b/tcg/arm/tcg-target.h
@@ -27,6 +27,7 @@
 
 #undef TCG_TARGET_STACK_GROWSUP
 #define TCG_TARGET_INSN_UNIT_SIZE 4
+#define TCG_TARGET_TLB_DISPLACEMENT_BITS 16
 
 typedef enum {
 TCG_REG_R0 = 0,
diff --git a/tcg/i386/tcg-target.h b/tcg/i386/tcg-target.h
index 7a9980e..8ba977a 100644
--- a/tcg/i386/tcg-target.h
+++ b/tcg/i386/tcg-target.h
@@ -25,6 +25,7 @@
 #define TCG_TARGET_I386 1
 
 #define TCG_TARGET_INSN_UNIT_SIZE  1
+#define TCG_TARGET_TLB_DISPLACEMENT_BITS 32
 
 #ifdef __x86_64__
 # define TCG_TARGET_REG_BITS  64
diff --git a/tcg/ia64/tcg-target.h b/tcg/ia64/tcg-target.h
index d675589..a04ed81 100644
--- a/tcg/ia64/tcg-target.h
+++ b/tcg/ia64/tcg-target.h
@@ -26,6 +26,8 @@
 #define TCG_TARGET_IA64 1
 
 #define TCG_TARGET_INSN_UNIT_SIZE 16
+#define TCG_TARGET_TLB_DISPLACEMENT_BITS 21
+
 typedef struct {
 uint64_t lo __attribute__((aligned(16)));
 uint64_t hi;
diff --git a/tcg/mips/tcg-target.h b/tcg/mips/tcg-target.h
index c88a1c9..f5ba52c 100644
--- a/tcg/mips/tcg-target.h
+++ b/tcg/mips/tcg-target.h
@@ -27,6 +27,7 @@
 #define TCG_TARGET_MIPS 1
 
 #define TCG_TARGET_INSN_UNIT_SIZE 4
+#define TCG_TARGET_TLB_DISPLACEMENT_BITS 16
 #define TCG_TARGET_NB_REGS 32
 
 typedef enum {
diff --git a/tcg/ppc/tcg-target.h b/tcg/ppc/tcg-target.h
index 32ac442..7ce7048 100644
--- a/tcg/ppc/tcg-target.h
+++ b/tcg/ppc/tcg-target.h
@@ -32,6 +32,7 @@
 
 #define TCG_TARGET_NB_REGS 32
 #define TCG_TARGET_INSN_UNIT_SIZE 4
+#define TCG_TARGET_TLB_DISPLACEMENT_BITS 16
 
 typedef enum {
 TCG_REG_R0,  TCG_REG_R1,  TCG_REG_R2,  TCG_REG_R3,
diff --git a/tcg/s390/tcg-target.h b/tcg/s390/tcg-target.h
index 5acc28c..91576d5 100644
--- a/tcg/s390/tcg-target.h
+++ b/tcg/s390/tcg-target.h
@@ -25,6 +25,7 @@
 #define TCG_TARGET_S390 1
 
 #define TCG_TARGET_INSN_UNIT_SIZE 2
+#define TCG_TARGET_TLB_DISPLACEMENT_BITS 19
 
 typedef enum TCGReg {
 TCG_REG_R0 = 0,
diff --git a/tcg/sparc/tcg-target.h b/tcg/sparc/tcg-target.h
index 0c4c8af..f584de4 100644
--- a/tcg/sparc/tcg-target.h
+++ b/tcg/sparc/tcg-target.h
@@ -27,6 +27,7 @@
 #define TCG_TARGET_REG_BITS 64
 
 #define TCG_TARGET_INSN_UNIT_SIZE 4
+#define TCG_TARGET_TLB_DISPLACEMENT_BITS 32
 #define TCG_TARGET_NB_REGS 32
 
 typedef enum {
diff --git a/tcg/tci/tcg-target.h b/tcg/tci/tcg-target.h
index bd1e974..4c41305 100644
--- a/tcg/tci/tcg-target.h
+++ b/tcg/tci/tcg-target.h
@@ -44,6 +44,7 @@
 
 #define TCG_TARGET_INTERPRETER 1
 #define TCG_TARGET_INSN_UNIT_SIZE 1
+#define TCG_TARGET_TLB_DISPLACEMENT_BITS 32
 
 #if UINTPTR_MAX == UINT32_MAX
 # define TCG_TARGET_REG_BITS 32
-- 
2.3.0

Re: [Qemu-devel] [PATCH v3 2/3] Add migrate_incoming

[Markus Armbruster]

Eric Blake  writes:

> On 02/20/2015 01:18 AM, Markus Armbruster wrote:
>> I'd like Eric's opinion on on encoding configuration tuples as URIs
>> rather than JSON in QMP.
>> 
>
>>> +{ 'command': 'migrate-incoming', 'data': {'uri': 'str' } }
>>> +
>>>  # @xen-save-devices-state:
>>>  #
>>>  # Save the state of all devices to file. The RAM and the block devices
>> 
>> Eric, what's your take on this?
>> 
>> The general rule in QMP is "no ad hoc encoding of tuples in strings, use
>> JSON objects instead".
>
> Yes, it would be nice to be type-safe, and have a QAPI union with a
> discriminator of different URI types along with appropriate accompanying
> data.  But it is also new code (both in qemu to parse it, and in libvirt
> to generate it), and we already have existing code that knows how to
> generate the string-encoded tuple for the -incoming command line
> argument that can be reused as-is with the new QMP command as proposed.
>  So even though it is less safe, I'm okay with this particular command
> using an overloaded string argument.
>
> Down the road, if we WANT to add type-safety, we can make this command
> take an anonymous union, where a 'str' value is the compact old style,
> and where a dictionary value is the new discriminated union style.  So
> we aren't completely locked into non-type-safe string forever, but we
> probably won't add a type-safe variant until (if) we ever add a new
> migration format that just looks too ugly as a URI based on the
> parameters it requires.

I can accept this, but I want the rationale in a comment, so this
exception to the QMP rule serves less well as bad example.

Re: [Qemu-devel] [PATCH] Makefile.target: set icon for binary file on Mac OS X

[Paolo Bonzini]

On 20/02/2015 18:32, Programmingkid wrote:
>> Ok, so I'll apply v3 to my tree as soon as I get a Tested-by.
>> Please take a look into providing a .rsrc file with larger-sized
>> icons (I think you can add more than one to a single .rsrc file?)
>> and, when you do that, document in pc-bios/README how the
>> resources were built.
> 
> When I was inspecting the icon using the genie feature of Mac OS X's
> dock, the icon looked very sharp at full size, so I didn't see a
> problem with it. What size do you have in mind?

I was thinking of 16x16, 32x32, 48x48 and 128x128.

Paolo

Re: [Qemu-devel] [PATCH 00/11] target-aarch64 fix and improvments

[Alex Bennée]

https://validation.linaro.org/dashboard/streams/anonymous/alex.bennee/bundles/52315b57f77238f924b5528ad16cc549d93a9d31/2a7f8984-0d1c-4268-8249-ba86c56dcbb7/?search=&length=100#table

All passing on the current test set. I've got more to add when I get a chance,

On 20 February 2015 at 16:50, Alex Bennée  wrote:
> It's running through LAVA now:
>
> https://validation.linaro.org/scheduler/job/268253
>
>
> On 19 February 2015 at 23:52, Peter Maydell  wrote:
>> On 20 February 2015 at 06:14, Richard Henderson  wrote:
>>> While doing the mechanics of a previous patch set converting
>>> translators to use to TCGLabel pointers, I was reminded of
>>> several outstanding OPTME comments in the aarch64 translator.
>>>
>>> I had started with the csel change, which at first failed and
>>> took quite some time to debug.  See the comment for patch 1.
>>>
>>> Since this depends on the outstanding TCGLabel patch set, the
>>> full tree is available at
>>>
>>>   git://github.com/rth7680/qemu.git arm-movcond
>>
>> Alex, could you run this lot through the A64 risu testsuite,
>> please?
>>
>> thanks
>> -- PMM
>
>
>
> --
> Alex Bennée
> KVM/QEMU Hacker for Linaro



-- 
Alex Bennée
KVM/QEMU Hacker for Linaro

Re: [Qemu-devel] [RFC PATCH v2 04/15] cpu-model/s390: Introduce S390 CPU models

[Alexander Graf]

> Am 20.02.2015 um 18:37 schrieb Michael Mueller :
> 
> On Fri, 20 Feb 2015 17:57:52 +0100
> Alexander Graf  wrote:
> 
>> Because all CPUs we have in our list only expose 128 bits?
> 
> Here a STFLE result on a EC12 GA2, already more than 128 bits... Is that 
> model on the list?

If that model has 3 elements, yes, the array should span 3.

I hope it's in the list. Every model wecare about should be, no?

> 
> [mimu@p57lp59 s390xfac]$ ./s390xfac -b
> fac[0] = 0xfbfbfcfff840
> fac[1] = 0xffde
> fac[2] = 0x1800
>> 
>>> I want to have this independent from a future machine of the z/Arch. The 
>>> kernel stores the
>>> full facility set, KVM does and there is no good reason for QEMU not to do. 
>>> If other
>>> accelerators decide to just implement 64 or 128 bits of facilities that's 
>>> ok...  
>> 
>> So you want to support CPUs that are not part of the list?
> 
> The architecture at least defines more than 2 or 3. Do you want me to limit 
> it to an arbitrary
> size?. Only in QEMU or also in the KVM interface?

Only internally in QEMU. The kvm interface should definitely be as big as the 
spec allows!

Alex

> 
> Thanks
> Michael
> 

Re: [Qemu-devel] [RFC PATCH v2 04/15] cpu-model/s390: Introduce S390 CPU models

[Michael Mueller]

On Fri, 20 Feb 2015 17:57:52 +0100
Alexander Graf  wrote:

> Because all CPUs we have in our list only expose 128 bits?

Here a STFLE result on a EC12 GA2, already more than 128 bits... Is that model 
on the list?

[mimu@p57lp59 s390xfac]$ ./s390xfac -b
fac[0] = 0xfbfbfcfff840
fac[1] = 0xffde
fac[2] = 0x1800
> 
> > I want to have this independent from a future machine of the z/Arch. The 
> > kernel stores the
> > full facility set, KVM does and there is no good reason for QEMU not to do. 
> > If other
> > accelerators decide to just implement 64 or 128 bits of facilities that's 
> > ok...  
> 
> So you want to support CPUs that are not part of the list?

The architecture at least defines more than 2 or 3. Do you want me to limit it 
to an arbitrary
size?. Only in QEMU or also in the KVM interface?

Thanks
Michael

Re: [Qemu-devel] [PATCH 0/3] Support streaming to an intermediate layer

[Eric Blake]

On 02/20/2015 06:53 AM, Alberto Garcia wrote:
> Hello,
> 
> I added support to the Block Stream API for streaming to an
> intermediate layer.
> 
> I followed the proposed API from the wiki, which simply adds an
> additional 'top' parameter to block-stream specifying the image that
> data is written to:

How does one learn whether qemu is new enough to support this mode?
Until we add QMP introspection, learning whether an optional parameter
exists requires attempting the command and seeing a different error
depending on whether the argument is recognized.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH] Makefile.target: set icon for binary file on Mac OS X

[Programmingkid]

On Feb 20, 2015, at 12:05 PM, Paolo Bonzini wrote:

> 
> 
> On 20/02/2015 17:54, Programmingkid wrote:
>>> I suspect the Windows icon is not a great match for Mac OS X which likes
>>> to have big sizes (48x48 or 128x128).
>> 
>> Definitely true.
>> 
>>> If you want to generate the .rsrc
>>> file automatically, the right source probably would be the .svg file,
>>> and doing the conversion in the Makefile probably isn't entirely
>>> feasible.  It would need extra build dependency and rounding errors
>>> would make it harder to achieve reproducible builds.
>> 
>> Plus the sips command doesn't work with svg files. 
> 
> Ok, so I'll apply v3 to my tree as soon as I get a Tested-by.  Please
> take a look into providing a .rsrc file with larger-sized icons (I think
> you can add more than one to a single .rsrc file?) and, when you do
> that, document in pc-bios/README how the resources were built.

When I was inspecting the icon using the genie feature of Mac OS X's dock, the 
icon looked very sharp at full size, so I didn't see a problem with it. What 
size do you have in mind?

Re: [Qemu-devel] [RFC PATCH v2 13/15] cpu-model/s390: Add processor property routines

[Alexander Graf]

On 20.02.15 16:32, Michael Mueller wrote:
> On Fri, 20 Feb 2015 15:03:30 +0100
> Alexander Graf  wrote:
> 
>>>
>>> - s390_get_proceccor_props()
>>> - s390_set_proceccor_props()
>>>
>>> They can be used to request or retrieve processor related information from 
>>> an accelerator.
>>> That information comprises the cpu identifier, the ICB value and the 
>>> facility lists.
>>>
>>> Signed-off-by: Michael Mueller   
>>
>> Hrm, I still seem to miss the point of this interface. What do you need
>> it for?
> 
> These functions make the internal s390 cpu model API independent from a 
> specific accelerator:  
> 
> int s390_set_processor_props(S390ProcessorProps *prop)
> {
> if (kvm_enabled()) {
> return kvm_s390_set_processor_props(prop);
> }
> return -ENOSYS;
> }
> 
> It's called by:
> 
> s390_select_cpu_model(const char *model)
> 
> which is itself called by:
> 
> S390CPU *cpu_s390x_init(const char *cpu_model)
> {
> S390CPU *cpu;
> 
> cpu = S390_CPU(object_new(s390_select_cpu_model(cpu_model)));
> 
> object_property_set_bool(OBJECT(cpu), true, "realized", NULL);
> 
> return cpu;
> }
> 
> So above s390_set/get_processor_props() the code is accelerator independent.

Any particular reason you can't do it like PPC?


Alex

Re: [Qemu-devel] [PATCH v3] Makefile.target: set icon for binary file on Mac OS X

[Paolo Bonzini]

On 20/02/2015 18:06, Programmingkid wrote:
> I did some looking myself and found out that the Rez and SetFile command
> are missing on Mac OS 10.4 (and probably below). They are available on
> Mac OS 10.5 and Mac OS 10.6. Since this feature isn't important, I can
> add conditional code that would check to see if the Rez and SetFile
> commands are available on the computer before using them. The thing is
> Mac OS 10.4 and under appear to be no longer supported by QEMU, so this
> might not be a problem. 

I honestly don't know which versions are supported.  10.5 was released
in 2007 and it is enough for Darwin/PPC, so it's probably okay.

Paolo

Re: [Qemu-devel] [PATCH v3] Makefile.target: set icon for binary file on Mac OS X

[Programmingkid]

On Feb 20, 2015, at 6:41 AM, Paolo Bonzini wrote:

> 
> 
> On 19/02/2015 21:51, Programmingkid wrote:
>> This patch sets the icon for the QEMU binary on Mac OS X.
>> 
>> Signed-off-by: John Arbuckle 
>> 
>> ---
>> Added conditional code to make the icon setting happen only on Mac OS X. 
> 
> Applied, but I'd like a "Tested-by" from Peter Maydell or another OS X
> user.  Also, before sending a pull request I'll also check on my
> in-laws' Mac (which I found has gcc installed) whether it also has Rez
> and SetFile.
> 
> Paolo

I did some looking myself and found out that the Rez and SetFile command are 
missing on Mac OS 10.4 (and probably below). They are available on Mac OS 10.5 
and Mac OS 10.6. Since this feature isn't important, I can add conditional code 
that would check to see if the Rez and SetFile commands are available on the 
computer before using them. The thing is Mac OS 10.4 and under appear to be no 
longer supported by QEMU, so this might not be a problem. 

Re: [Qemu-devel] [PATCH v13 00/17] block: incremental backup series

[John Snow]

On 02/20/2015 06:09 AM, Stefan Hajnoczi wrote:

On Fri, Feb 13, 2015 at 05:08:41PM -0500, John Snow wrote:

This series requires: [PATCH v3] blkdebug: fix "once" rule

Welcome to the "incremental backup" newsletter, where we discuss
exciting developments in non-redundant backup technology.
This issue is called the "Max Reitz Fixup" issue.

This patchset enables the in-memory part of the incremental backup
feature. There are two series on the mailing list now by Vladimir
Sementsov-Ogievskiy that enable the migration and persistence of
dirty bitmaps.

This series and most patches were originally by Fam Zheng.


Please add docs/incremental-backup.txt explaining how the commands are
intended to be used to perform backups.  The QAPI documentation is
sparse and QMP clients would probably need to read the code to
understand how these commands work.



OK. I wrote a markdown formatted file that explains it all pretty well; 
should I check it in as-is, or should I convert it to some other format?


https://github.com/jnsnow/qemu/blob/bitmap-demo/docs/bitmaps.md


I'm not sure I understand the need for all the commands: add, remove,
enable, disable, clear.  Why are these commands necessary?


add/remove are self explanatory.

clear allows you to re-sync a bitmap to a full backup after you have 
already been running for some time. See the readme for the usage case. 
Yes, you *COULD* delete and re-create the bitmap with add/remove, but why?

Clear is nice, I stand by it.

enable/disable: Not necessarily useful for the simple case at this exact 
moment; they can be used to track writes during a period of time if 
desired. You could use them with transactions to record activity through 
different periods of time. They were originally used for what the 
"frozen" case covers now, but I left them in.


I could stand to part with them if you think they detract more than 
help. They seemed like useful primitives. My default action will be to 
leave them in, still.




I think just add and remove should be enough:

Initial full backup
---
Use transaction to add bitmaps to all drives atomically (i.e.
point-in-time snapshot across all drives) and launch drive-backup (full)
on all drives.

In your patch the bitmap starts disabled.  In my example bitmaps are
always enabled unless they have a successor.


No they don't:

bitmap->disabled = false;



Incremental backup
--
Use transaction to launch drive-backup (bitmap mode) on all drives.

If backup completes successfully we'll be able to run the same command
again for the next incremental backup.

If backup fails I see a problem when taking consistent incremental
backups across multiple drives:

Imagine 2 drives (A and B).  Transaction is used to launch drive-backup
on both with their respective dirty bitmaps.  drive-backup A fails and
merges successor dirty bits so nothing is lost, but what do we do with
drive-backup B?

drive-backup B could be in-progress or it could have completed before
drive-backup A failed.  Now we're in trouble because we need to take
consistent snapshots across all drives but we've thrown away the dirty
bitmap for drive B!


Just re-run the transaction. If one failed but one succeeded, just run 
it again. The one that succeeded prior will now have a trivial 
incremental backup, and the one that failed will have a new valid 
incremental backup. The two new incrementals will be point in time 
consistent.


E.G.:

[full_a] <-- [incremental_a_0: FAILED]
[full_b] <-- [incremental_b_0: SUCCESS]

then re-run:

[full_a] < [incremental_a_1]
[full_b] <-- [incremental_b_0] <-- [incremental_b_1]

If the extra image in the chain for the one that succeeded is 
problematic, you can use other tools to consolidate them later.


Either way, how do you propose getting a consistent snapshot across 
multiple drives after a failure? The only recovery option is to just 
create a new snapshot *later*, you can never go back to what was, just 
like you can't for single drives.


libvirt can tell which portions of the transaction ultimately succeeded 
by the BLOCK_JOB_COMPLETED events that it receives back, one per each drive.


For drives that complete successfully, it can make a mental note that it 
has a new proper incremental. For drives that do not, it can make a note 
that it needs to try for a new consistent drives-wide incremental, 
erasing the half-baked attempt that got created last time.


I am not convinced there is a problem. Since we don't want filename 
management (&c) as part of this solution inside of QEMU, there is 
nothing left to do except within libvirt.




Stopping incremental backup
---
Use transaction to remove bitmaps on all drives.  This case is easy.

Finally, what about bdrv_truncate()?  When the disk size changes the
dirty bitmaps keep the old size.  Seems likely to cause problems.  Have
you thought about this case?



Only just recently when reviewing Vladimir's bitm

Re: [Qemu-devel] [PATCH] Makefile.target: set icon for binary file on Mac OS X

[Paolo Bonzini]

On 20/02/2015 17:54, Programmingkid wrote:
>> I suspect the Windows icon is not a great match for Mac OS X which likes
>> to have big sizes (48x48 or 128x128).
> 
> Definitely true.
> 
>>  If you want to generate the .rsrc
>> file automatically, the right source probably would be the .svg file,
>> and doing the conversion in the Makefile probably isn't entirely
>> feasible.  It would need extra build dependency and rounding errors
>> would make it harder to achieve reproducible builds.
> 
> Plus the sips command doesn't work with svg files. 

Ok, so I'll apply v3 to my tree as soon as I get a Tested-by.  Please
take a look into providing a .rsrc file with larger-sized icons (I think
you can add more than one to a single .rsrc file?) and, when you do
that, document in pc-bios/README how the resources were built.

Thanks,

Paolo

Re: [Qemu-devel] [PATCH 0/2] Fix GCC 5.0.0 build errors

[Radim Krčmář]

Cc: qemu-triv...@nongnu.org

Re: [Qemu-devel] [RFC PATCH v2 09/15] cpu-model/s390: Add KVM VM attribute interface routines

[Alexander Graf]

On 20.02.15 16:18, Michael Mueller wrote:
> On Fri, 20 Feb 2015 14:59:20 +0100
> Alexander Graf  wrote:
> 
>>> +typedef struct S390ProcessorProps {
>>> +uint64_t cpuid;
>>> +uint16_t ibc;
>>> +uint8_t  pad[6];
>>> +uint64_t fac_list[S390_ARCH_FAC_LIST_SIZE_UINT64];
>>> +} S390ProcessorProps;
>>> +
>>> +typedef struct S390MachineProps {
>>> +uint64_t cpuid;
>>> +uint32_t ibc_range;
>>> +uint8_t  pad[4];
>>> +uint64_t fac_list_mask[S390_ARCH_FAC_LIST_SIZE_UINT64];
>>> +uint64_t fac_list[S390_ARCH_FAC_LIST_SIZE_UINT64];
>>> +} S390MachineProps;  
>>
>> What are those structs there for? To convert between a kvm facing
>> interface to an internal interface?
> 
> Yes, that's their current use, but if the interface structs: 
> 
> +struct kvm_s390_vm_cpu_processor {
> +   __u64 cpuid;
> +   __u16 ibc;
> +   __u8  pad[6];
> +   __u64 fac_list[256];
> +};
> +
> +/* kvm S390 machine related attributes are r/o */
> +#define KVM_S390_VM_CPU_MACHINE1
> +struct kvm_s390_vm_cpu_machine {
> +   __u64 cpuid;
> +   __u32 ibc_range;
> +   __u8  pad[4];
> +   __u64 fac_mask[256];
> +   __u64 fac_list[256];
> +};
> 
> are visible here, I'll reuse them... But stop, that will not work in the 
> --disable-kvm case... I need them!

I meant it the other way around - do KVM specific patching of the cpu
types from kvm.c.

But please give a nutshell explanation on what exactly you're patching
at all here.


Alex

Re: [Qemu-devel] [RFC PATCH v2 04/15] cpu-model/s390: Introduce S390 CPU models

[Alexander Graf]

On 20.02.15 16:49, Michael Mueller wrote:
> On Fri, 20 Feb 2015 16:22:20 +0100
> Alexander Graf  wrote:
> 

 Just make this uint64_t fac_list[2]. That way we don't have to track any
 messy allocations.  
>>>
>>> It will be something like "uint64_t 
>>> fac_list[S390_CPU_FAC_LIST_SIZE_UINT64]" and in total 2KB
>>> not just 16 bytes but I will change it.   
>>
>> Why? Do we actually need that many? This is a qemu internal struct.
> 
> How do you know that 2 is a good size?

Because all CPUs we have in our list only expose 128 bits?

> I want to have this independent from a future machine of the z/Arch. The 
> kernel stores the full
> facility set, KVM does and there is no good reason for QEMU not to do. If 
> other accelerators
> decide to just implement 64 or 128 bits of facilities that's ok...

So you want to support CPUs that are not part of the list?


Alex

Re: [Qemu-devel] [RFC PATCH v2 13/15] cpu-model/s390: Add processor property routines

[Michael Mueller]

On Fri, 20 Feb 2015 17:28:14 +0100
Andreas Färber  wrote:

Andreas,

> Sorry for my ignorance, but what is proc actually needed for? For
> initializing the class, there's .class_init (and cc->fac_list apparently
> is initialized here). If you need to pass info to KVM, you can do so in

yes, it is communication to the accelerator to prepare its local cpu model 
related data
structures which are used to initialize a vcpu (e.g. the facility list beside 
others) 

> DeviceClass::realize when the vCPU actually goes "live". A

I will look what "goes live" in detail means here, it should at least be before
kvm_arch_vcpu_setup() gets triggered on accelerator side.

> string-to-string (or string-to-ObjectClass) translation function seems
> like a weird point in time to take action with global effect.
> 
> Anyway, please implement the generic callback, then you can still call
> it from your own helper functions if needed.

Thanks a lot!
Michael

Re: [Qemu-devel] [RFC PATCH v2 10/15] cpu-model/s390: Add cpu class initialization routines

[Michael Mueller]

On Fri, 20 Feb 2015 17:34:28 +0100
Andreas Färber  wrote:

> Please note that QEMU uses gtk-doc style, where the description goes
> between arguments and Returns:, and the function name gets a ':'.
> There's also fancy syntax like #CPUClass, %true, etc.

On my TODOs...

Thanks,
Michael

Re: [Qemu-devel] [PATCH] Makefile.target: set icon for binary file on Mac OS X

[Programmingkid]

On Feb 20, 2015, at 7:36 AM, Paolo Bonzini wrote:

> 
> 
> On 20/02/2015 13:18, Peter Maydell wrote:
>> Why not just use the sips --out option to specify a different
>> output file? That way we automatically put the current icon
>> into the executable, and don't have to update a hand-created
>> qemu.rsrc file in git if we change the icon in future (and I
>> bet if we don't have the rules for doing this in the makefile
>> then nobody will remember how to do it).
> 
> I suspect the Windows icon is not a great match for Mac OS X which likes
> to have big sizes (48x48 or 128x128).

Definitely true.

>  If you want to generate the .rsrc
> file automatically, the right source probably would be the .svg file,
> and doing the conversion in the Makefile probably isn't entirely
> feasible.  It would need extra build dependency and rounding errors
> would make it harder to achieve reproducible builds.

Plus the sips command doesn't work with svg files. 

Re: [Qemu-devel] [PATCH v13 00/17] block: incremental backup series

[Kashyap Chamarthy]

On Fri, Feb 20, 2015 at 11:09:20AM +, Stefan Hajnoczi wrote:
> On Fri, Feb 13, 2015 at 05:08:41PM -0500, John Snow wrote:
> > This series requires: [PATCH v3] blkdebug: fix "once" rule
> > 
> > Welcome to the "incremental backup" newsletter, where we discuss
> > exciting developments in non-redundant backup technology.
> > This issue is called the "Max Reitz Fixup" issue.
> > 
> > This patchset enables the in-memory part of the incremental backup
> > feature. There are two series on the mailing list now by Vladimir
> > Sementsov-Ogievskiy that enable the migration and persistence of
> > dirty bitmaps.
> > 
> > This series and most patches were originally by Fam Zheng.
> 
> Please add docs/incremental-backup.txt explaining how the commands are
> intended to be used to perform backups.  The QAPI documentation is
> sparse and QMP clients would probably need to read the code to
> understand how these commands work.

Thanks for noting this, I have this series applied locally to test and
was intending to going through the iotests for initial documentation
when I actually make time to test this sometime next week :-)

-- 
/kashyap

Re: [Qemu-devel] [PATCH 00/11] target-aarch64 fix and improvments

[Alex Bennée]

It's running through LAVA now:

https://validation.linaro.org/scheduler/job/268253


On 19 February 2015 at 23:52, Peter Maydell  wrote:
> On 20 February 2015 at 06:14, Richard Henderson  wrote:
>> While doing the mechanics of a previous patch set converting
>> translators to use to TCGLabel pointers, I was reminded of
>> several outstanding OPTME comments in the aarch64 translator.
>>
>> I had started with the csel change, which at first failed and
>> took quite some time to debug.  See the comment for patch 1.
>>
>> Since this depends on the outstanding TCGLabel patch set, the
>> full tree is available at
>>
>>   git://github.com/rth7680/qemu.git arm-movcond
>
> Alex, could you run this lot through the A64 risu testsuite,
> please?
>
> thanks
> -- PMM



-- 
Alex Bennée
KVM/QEMU Hacker for Linaro

Re: [Qemu-devel] [RFC PATCH v2 10/15] cpu-model/s390: Add cpu class initialization routines

[Andreas Färber]

Am 20.02.2015 um 17:12 schrieb Michael Mueller:
> On Fri, 20 Feb 2015 08:02:42 -0800
> Richard Henderson  wrote:
> 
>>> +/**
>>> + * s390_test_facility - test if given facility bit is set facility list
>>> + *  of given cpu class
>>> + * @class: address of cpu class to test
>>> + * @nr: bit number to test
>>> + *
>>> + * Returns: true in case it is set
>>> + *  false in case it is not set
>>> + */

Please note that QEMU uses gtk-doc style, where the description goes
between arguments and Returns:, and the function name gets a ':'.
There's also fancy syntax like #CPUClass, %true, etc.

Regards,
Andreas

-- 
SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Jane Smithard, Jennifer Guild, Dilip Upmanyu,
Graham Norton; HRB 21284 (AG Nürnberg)

Re: [Qemu-devel] [RFC PATCH v2 10/15] cpu-model/s390: Add cpu class initialization routines

[Michael Mueller]

On Fri, 20 Feb 2015 17:12:49 +0100
Michael Mueller  wrote:

> Good spot, it's not being used yet. It's planned to be used with a patch that 
> implements zPCI
> related instructions on QEMU side. Maybe you have seen the discussion from 
> Frank Blaschka in
> this e-mail list in regard to that.

I will factor it out.

[Qemu-devel] [PATCH RFC 0/3] iothread: release iothread around aio_poll

[Paolo Bonzini]

Right now, iothreads are relying on a "contention callback" to release
the AioContext (e.g. for block device operations or to do bdrv_drain_all).
This is necessary because aio_poll needs to be called within an
aio_context_acquire.

This series drops this requirement for aio_poll, with two effects:

1) it makes it possible to remove the "contention callback" in RFifoLock
(and possibly to convert it to a normal GRecMutex, which is why I am not
including a patch to remove callbacks from RFifoLock).

2) it makes it possible to start work around making critical sections
for the block layer fine-grained.

In order to do this, some data is moved from AioContext to local storage.
Stack allocation has size limitations, so thread-local storage is used
instead.  There are no reentrancy problems because the data is only live
throughout a small part of aio_poll, and in particular not during the
invocation of callbacks.

Comments?

Paolo

Paolo Bonzini (3):
  aio-posix: move pollfds to thread-local storage
  AioContext: acquire/release AioContext during aio_poll
  iothread: release iothread around aio_poll

 aio-posix.c | 86 -
 aio-win32.c |  8 +
 async.c | 10 +--
 include/block/aio.h | 18 +--
 iothread.c  | 11 ++-
 tests/test-aio.c| 21 +++--
 6 files changed, 96 insertions(+), 58 deletions(-)

-- 
2.3.0

Re: [Qemu-devel] [RFC PATCH v2 13/15] cpu-model/s390: Add processor property routines

[Andreas Färber]

Am 20.02.2015 um 17:04 schrieb Michael Mueller:
> On Fri, 20 Feb 2015 16:41:49 +0100
> Andreas Färber  wrote:
> 
>> Can't you just implement the class-level name-to-ObjectClass callback
>> that other CPUs have grown for the above use case?
> 
> If it fulfills the requirements sure. Please point me to an example,

Take a look at include/qom/cpu.h CPUClass::class_by_name and git-grep
the existing targets - most implement it already. It's a generic hook to
be used from everywhere rather than a local function specific to the
legacy init function. Apart from the error handling it should be
straight-forward.

> sounds that
> s390_select_cpu_model() is doing something similar to that, just that it 
> hooks in
> the s390_set_processor_props() call.
> 
> const char *s390_select_cpu_model(const char *model)
> {
> S390ProcessorProps proc;
> const char *typename;
> S390CPUClass *cc;
> 
> /* return already selected cpu typename */
> typename = s390_cpu_typename();
> if (typename) {
> goto out;
> }
> 
> /* return standard cpu typename when cpu models are unavailable */
> typename = TYPE_S390_CPU;
> if (!s390_cpu_classes_initialized() || !model) {
> goto out;
> }
> cc = S390_CPU_CLASS(s390_cpu_class_by_name(model));
> if (!cc) {
> goto out;
> }
> proc.cpuid = cpuid(cc->proc);
> proc.ibc = cc->proc->ibc;
> memcpy(proc.fac_list, cc->fac_list, S390_ARCH_FAC_LIST_SIZE_BYTE);
> if (s390_set_processor_props(&proc)) {
> goto out;
> }

Sorry for my ignorance, but what is proc actually needed for? For
initializing the class, there's .class_init (and cc->fac_list apparently
is initialized here). If you need to pass info to KVM, you can do so in
DeviceClass::realize when the vCPU actually goes "live". A
string-to-string (or string-to-ObjectClass) translation function seems
like a weird point in time to take action with global effect.

Anyway, please implement the generic callback, then you can still call
it from your own helper functions if needed.

Regards,
Andreas

> 
> /* return requested cpu typename in success case */
> typename = object_class_get_name((ObjectClass *) cc);
> out:
> selected_cpu_typename = typename;
> trace_select_cpu_model(model, typename);
> return typename;
> }

-- 
SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Jane Smithard, Jennifer Guild, Dilip Upmanyu,
Graham Norton; HRB 21284 (AG Nürnberg)

[Qemu-devel] [PATCH 1/3] aio-posix: move pollfds to thread-local storage

[Paolo Bonzini]

By using thread-local storage, aio_poll can stop using global data during
g_poll_ns.  This will make it possible to drop callbacks from rfifolock.

Signed-off-by: Paolo Bonzini 
---
 aio-posix.c | 77 ++---
 async.c |  2 --
 include/block/aio.h |  3 ---
 3 files changed, 56 insertions(+), 26 deletions(-)

diff --git a/aio-posix.c b/aio-posix.c
index cbd4c34..4a30b77 100644
--- a/aio-posix.c
+++ b/aio-posix.c
@@ -24,7 +24,6 @@ struct AioHandler
 IOHandler *io_read;
 IOHandler *io_write;
 int deleted;
-int pollfds_idx;
 void *opaque;
 QLIST_ENTRY(AioHandler) node;
 };
@@ -83,7 +82,6 @@ void aio_set_fd_handler(AioContext *ctx,
 node->io_read = io_read;
 node->io_write = io_write;
 node->opaque = opaque;
-node->pollfds_idx = -1;
 
 node->pfd.events = (io_read ? G_IO_IN | G_IO_HUP | G_IO_ERR : 0);
 node->pfd.events |= (io_write ? G_IO_OUT | G_IO_ERR : 0);
@@ -186,12 +184,59 @@ bool aio_dispatch(AioContext *ctx)
 return progress;
 }
 
+/* These thread-local variables are used only in a small part of aio_poll
+ * around the call to the poll() system call.  In particular they are not
+ * used while aio_poll is performing callbacks, which makes it much easier
+ * to think about reentrancy!
+ *
+ * Stack-allocated arrays would be perfect but they have size limitations;
+ * heap allocation is expensive enough that we want to reuse arrays across
+ * calls to aio_poll().  And because poll() has to be called without holding
+ * any lock, the arrays cannot be stored in AioContext.  Thread-local data
+ * has none of the disadvantages of these three options.
+ */
+static __thread GPollFD *pollfds;
+static __thread AioHandler **nodes;
+static __thread unsigned npfd, nalloc;
+static __thread Notifier pollfds_cleanup_notifier;
+
+static void pollfds_cleanup(Notifier *n, void *unused)
+{
+g_assert(npfd == 0);
+g_free(pollfds);
+g_free(nodes);
+nalloc = 0;
+}
+
+static void add_pollfd(AioHandler *node)
+{
+if (npfd == nalloc) {
+if (nalloc == 0) {
+pollfds_cleanup_notifier.notify = pollfds_cleanup;
+qemu_thread_atexit_add(&pollfds_cleanup_notifier);
+nalloc = 8;
+} else {
+g_assert(nalloc <= INT_MAX);
+nalloc *= 2;
+}
+pollfds = g_renew(GPollFD, pollfds, nalloc);
+nodes = g_renew(AioHandler *, nodes, nalloc);
+}
+nodes[npfd] = node;
+pollfds[npfd] = (GPollFD) {
+.fd = node->pfd.fd,
+.events = node->pfd.events,
+};
+npfd++;
+}
+
 bool aio_poll(AioContext *ctx, bool blocking)
 {
 AioHandler *node;
 bool was_dispatching;
-int ret;
+int i, ret;
 bool progress;
+int64_t timeout;
 
 was_dispatching = ctx->dispatching;
 progress = false;
@@ -210,39 +255,29 @@ bool aio_poll(AioContext *ctx, bool blocking)
 
 ctx->walking_handlers++;
 
-g_array_set_size(ctx->pollfds, 0);
+npfd = 0;
 
 /* fill pollfds */
 QLIST_FOREACH(node, &ctx->aio_handlers, node) {
-node->pollfds_idx = -1;
 if (!node->deleted && node->pfd.events) {
-GPollFD pfd = {
-.fd = node->pfd.fd,
-.events = node->pfd.events,
-};
-node->pollfds_idx = ctx->pollfds->len;
-g_array_append_val(ctx->pollfds, pfd);
+add_pollfd(node);
 }
 }
 
-ctx->walking_handlers--;
+timeout = blocking ? aio_compute_timeout(ctx) : 0;
 
 /* wait until next event */
-ret = qemu_poll_ns((GPollFD *)ctx->pollfds->data,
- ctx->pollfds->len,
- blocking ? aio_compute_timeout(ctx) : 0);
+ret = qemu_poll_ns((GPollFD *)pollfds, npfd, timeout);
 
 /* if we have any readable fds, dispatch event */
 if (ret > 0) {
-QLIST_FOREACH(node, &ctx->aio_handlers, node) {
-if (node->pollfds_idx != -1) {
-GPollFD *pfd = &g_array_index(ctx->pollfds, GPollFD,
-  node->pollfds_idx);
-node->pfd.revents = pfd->revents;
-}
+for (i = 0; i < npfd; i++) {
+nodes[i]->pfd.revents = pollfds[i].revents;
 }
 }
 
+ctx->walking_handlers--;
+
 /* Run dispatch even if there were no readable fds to run timers */
 aio_set_dispatching(ctx, true);
 if (aio_dispatch(ctx)) {
diff --git a/async.c b/async.c
index cf5bd33..0463dc4 100644
--- a/async.c
+++ b/async.c
@@ -239,7 +239,6 @@ aio_ctx_finalize(GSource *source)
 event_notifier_cleanup(&ctx->notifier);
 rfifolock_destroy(&ctx->lock);
 qemu_mutex_destroy(&ctx->bh_lock);
-g_array_free(ctx->pollfds, TRUE);
 timerlistgroup_deinit(&ctx->tlg);
 }
 
@@ -311,7 +310,6 @@ AioContext *aio_context_new(Error **errp)
 aio_set_event_notifier(ctx, &ctx->notifier,
(Eve

[Qemu-devel] [PATCH 2/3] AioContext: acquire/release AioContext during aio_poll

[Paolo Bonzini]

This is the first step in pushing down acquire/release, and will let
rfifolock drop the contention callback feature.

Signed-off-by: Paolo Bonzini 
---
 aio-posix.c |  9 +
 aio-win32.c |  8 
 include/block/aio.h | 15 ---
 3 files changed, 25 insertions(+), 7 deletions(-)

diff --git a/aio-posix.c b/aio-posix.c
index 4a30b77..292ae84 100644
--- a/aio-posix.c
+++ b/aio-posix.c
@@ -238,6 +238,7 @@ bool aio_poll(AioContext *ctx, bool blocking)
 bool progress;
 int64_t timeout;
 
+aio_context_acquire(ctx);
 was_dispatching = ctx->dispatching;
 progress = false;
 
@@ -267,7 +268,13 @@ bool aio_poll(AioContext *ctx, bool blocking)
 timeout = blocking ? aio_compute_timeout(ctx) : 0;
 
 /* wait until next event */
+if (timeout) {
+aio_context_release(ctx);
+}
 ret = qemu_poll_ns((GPollFD *)pollfds, npfd, timeout);
+if (timeout) {
+aio_context_acquire(ctx);
+}
 
 /* if we have any readable fds, dispatch event */
 if (ret > 0) {
@@ -285,5 +292,7 @@ bool aio_poll(AioContext *ctx, bool blocking)
 }
 
 aio_set_dispatching(ctx, was_dispatching);
+aio_context_release(ctx);
+
 return progress;
 }
diff --git a/aio-win32.c b/aio-win32.c
index e6f4ced..233d8f5 100644
--- a/aio-win32.c
+++ b/aio-win32.c
@@ -283,6 +283,7 @@ bool aio_poll(AioContext *ctx, bool blocking)
 int count;
 int timeout;
 
+aio_context_acquire(ctx);
 have_select_revents = aio_prepare(ctx);
 if (have_select_revents) {
 blocking = false;
@@ -323,7 +324,13 @@ bool aio_poll(AioContext *ctx, bool blocking)
 
 timeout = blocking
 ? qemu_timeout_ns_to_ms(aio_compute_timeout(ctx)) : 0;
+if (timeout) {
+aio_context_release(ctx);
+}
 ret = WaitForMultipleObjects(count, events, FALSE, timeout);
+if (timeout) {
+aio_context_acquire(ctx);
+}
 aio_set_dispatching(ctx, true);
 
 if (first && aio_bh_poll(ctx)) {
@@ -349,5 +356,6 @@ bool aio_poll(AioContext *ctx, bool blocking)
 progress |= timerlistgroup_run_timers(&ctx->tlg);
 
 aio_set_dispatching(ctx, was_dispatching);
+aio_context_release(ctx);
 return progress;
 }
diff --git a/include/block/aio.h b/include/block/aio.h
index 499efd0..e77409d 100644
--- a/include/block/aio.h
+++ b/include/block/aio.h
@@ -118,13 +118,14 @@ void aio_context_ref(AioContext *ctx);
 void aio_context_unref(AioContext *ctx);
 
 /* Take ownership of the AioContext.  If the AioContext will be shared between
- * threads, a thread must have ownership when calling aio_poll().
- *
- * Note that multiple threads calling aio_poll() means timers, BHs, and
- * callbacks may be invoked from a different thread than they were registered
- * from.  Therefore, code must use AioContext acquire/release or use
- * fine-grained synchronization to protect shared state if other threads will
- * be accessing it simultaneously.
+ * threads, and a thread does not want to be interrupted, it will have to
+ * take ownership around calls to aio_poll().  Otherwise, aio_poll()
+ * automatically takes care of calling aio_context_acquire and
+ * aio_context_release.
+ *
+ * Access to timers and BHs from a thread that has not acquired AioContext
+ * is possible.  Access to callbacks for now must be done while the AioContext
+ * is owned by the thread (FIXME).
  */
 void aio_context_acquire(AioContext *ctx);
 
-- 
2.3.0

[Qemu-devel] [PATCH 3/3] iothread: release iothread around aio_poll

[Paolo Bonzini]

This is the first step towards having fine-grained critical sections in
dataplane threads, which resolves lock ordering problems between
address_space_* functions (which need the BQL when doing MMIO, even
after we complete RCU-based dispatch) and the AioContext.

Because AioContext does not use contention callbacks anymore, the
unit test has to be changed.

Signed-off-by: Paolo Bonzini 
---
 async.c  |  8 +---
 iothread.c   | 11 ++-
 tests/test-aio.c | 21 -
 3 files changed, 15 insertions(+), 25 deletions(-)

diff --git a/async.c b/async.c
index 0463dc4..e8a4c8b 100644
--- a/async.c
+++ b/async.c
@@ -289,12 +289,6 @@ static void aio_timerlist_notify(void *opaque)
 aio_notify(opaque);
 }
 
-static void aio_rfifolock_cb(void *opaque)
-{
-/* Kick owner thread in case they are blocked in aio_poll() */
-aio_notify(opaque);
-}
-
 AioContext *aio_context_new(Error **errp)
 {
 int ret;
@@ -312,7 +306,7 @@ AioContext *aio_context_new(Error **errp)
event_notifier_test_and_clear);
 ctx->thread_pool = NULL;
 qemu_mutex_init(&ctx->bh_lock);
-rfifolock_init(&ctx->lock, aio_rfifolock_cb, ctx);
+rfifolock_init(&ctx->lock, NULL, NULL);
 timerlistgroup_init(&ctx->tlg, aio_timerlist_notify, ctx);
 
 return ctx;
diff --git a/iothread.c b/iothread.c
index 342a23f..a1f9109 100644
--- a/iothread.c
+++ b/iothread.c
@@ -31,21 +31,14 @@ typedef ObjectClass IOThreadClass;
 static void *iothread_run(void *opaque)
 {
 IOThread *iothread = opaque;
-bool blocking;
 
 qemu_mutex_lock(&iothread->init_done_lock);
 iothread->thread_id = qemu_get_thread_id();
 qemu_cond_signal(&iothread->init_done_cond);
 qemu_mutex_unlock(&iothread->init_done_lock);
 
-while (!iothread->stopping) {
-aio_context_acquire(iothread->ctx);
-blocking = true;
-while (!iothread->stopping && aio_poll(iothread->ctx, blocking)) {
-/* Progress was made, keep going */
-blocking = false;
-}
-aio_context_release(iothread->ctx);
+while (!atomic_read(&iothread->stopping)) {
+aio_poll(iothread->ctx, true);
 }
 return NULL;
 }
diff --git a/tests/test-aio.c b/tests/test-aio.c
index a7cb5c9..f88a042 100644
--- a/tests/test-aio.c
+++ b/tests/test-aio.c
@@ -107,6 +107,7 @@ static void test_notify(void)
 
 typedef struct {
 QemuMutex start_lock;
+EventNotifier notifier;
 bool thread_acquired;
 } AcquireTestData;
 
@@ -118,6 +119,8 @@ static void *test_acquire_thread(void *opaque)
 qemu_mutex_lock(&data->start_lock);
 qemu_mutex_unlock(&data->start_lock);
 
+g_usleep(50);
+event_notifier_set(&data->notifier);
 aio_context_acquire(ctx);
 aio_context_release(ctx);
 
@@ -126,20 +129,19 @@ static void *test_acquire_thread(void *opaque)
 return NULL;
 }
 
-static void dummy_notifier_read(EventNotifier *unused)
+static void dummy_notifier_read(EventNotifier *n)
 {
-g_assert(false); /* should never be invoked */
+event_notifier_test_and_clear(n);
 }
 
 static void test_acquire(void)
 {
 QemuThread thread;
-EventNotifier notifier;
 AcquireTestData data;
 
 /* Dummy event notifier ensures aio_poll() will block */
-event_notifier_init(¬ifier, false);
-aio_set_event_notifier(ctx, ¬ifier, dummy_notifier_read);
+event_notifier_init(&data.notifier, false);
+aio_set_event_notifier(ctx, &data.notifier, dummy_notifier_read);
 g_assert(!aio_poll(ctx, false)); /* consume aio_notify() */
 
 qemu_mutex_init(&data.start_lock);
@@ -150,12 +152,13 @@ static void test_acquire(void)
 /* Block in aio_poll(), let other thread kick us and acquire context */
 aio_context_acquire(ctx);
 qemu_mutex_unlock(&data.start_lock); /* let the thread run */
-g_assert(!aio_poll(ctx, true));
+g_assert(aio_poll(ctx, true));
+g_assert(!data.thread_acquired);
 aio_context_release(ctx);
 
 qemu_thread_join(&thread);
-aio_set_event_notifier(ctx, ¬ifier, NULL);
-event_notifier_cleanup(¬ifier);
+aio_set_event_notifier(ctx, &data.notifier, NULL);
+event_notifier_cleanup(&data.notifier);
 
 g_assert(data.thread_acquired);
 }
-- 
2.3.0