Re: [Qemu-devel] [PATCH v2 1/6] arm: Uniquely name imx25 I2C buses.

[Cédric Le Goater]

On 12/03/2016 08:16 AM, Alexey Kardashevskiy wrote:
> On 02/12/16 18:55, Cédric Le Goater wrote:
>> On 12/02/2016 12:34 AM, Alexey Kardashevskiy wrote:
>>> On 01/12/16 23:31, Cédric Le Goater wrote:
 On 12/01/2016 01:42 AM, Alastair D'Silva wrote:
> On Wed, 2016-11-30 at 09:18 +0100, Cédric Le Goater wrote:
>> On 11/30/2016 06:36 AM, Alastair D'Silva wrote:
>>> From: Alastair D'Silva 
>>>
>>> The imx25 chip provides 3 i2c buses, but they have all been named
>>> "i2c", which makes it difficult to predict which bus a device will
>>> be connected to when specified on the command line.
>>>
>>> This patch addresses the issue by naming the buses uniquely:
>>>   i2c.0 i2c.1 i2c.2
>>>
>>> Signed-off-by: Alastair D'Silva 
>>> ---
>>>  hw/arm/imx25_pdk.c | 4 +---
>>>  hw/i2c/imx_i2c.c   | 6 +-
>>>  2 files changed, 6 insertions(+), 4 deletions(-)
>>>
>>> diff --git a/hw/arm/imx25_pdk.c b/hw/arm/imx25_pdk.c
>>> index 025b608..c6f04d3 100644
>>> --- a/hw/arm/imx25_pdk.c
>>> +++ b/hw/arm/imx25_pdk.c
>>> @@ -138,9 +138,7 @@ static void imx25_pdk_init(MachineState
>>> *machine)
>>>   * We add it here (only on qtest usage) to be able to do a
>>> bit
>>>   * of simple qtest. See "make check" for details.
>>>   */
>>> -i2c_create_slave((I2CBus *)qdev_get_child_bus(DEVICE(&s-
 soc.i2c[0]),
>>> -  "i2c"),
>>> - "ds1338", 0x68);
>>> +i2c_create_slave(s->soc.i2c[0].bus, "ds1338", 0x68);
>>>  }
>>>  }
>>>  
>>> diff --git a/hw/i2c/imx_i2c.c b/hw/i2c/imx_i2c.c
>>> index 37e5a62..7be10fb 100644
>>> --- a/hw/i2c/imx_i2c.c
>>> +++ b/hw/i2c/imx_i2c.c
>>> @@ -305,12 +305,16 @@ static const VMStateDescription
>>> imx_i2c_vmstate = {
>>>  static void imx_i2c_realize(DeviceState *dev, Error **errp)
>>>  {
>>>  IMXI2CState *s = IMX_I2C(dev);
>>> +static int bus_count;
>>
>> hmm, the static is ugly :/ 
>>
>> Isn't there other ways to achieve this naming ? 
>>
>> Thanks,
>>
>> C.  
>>
>
> I'm not seeing an obvious way around it. The busses are realized
> independently (so I can't implement what we do with the aspeed i2c
> busses), and it is named before fsl-imx25:fsl_imx25_realize() can apply
> specific properties to the bus.
>
> If you have any suggestions, I'm all ears.

 What about that ? 

@@ -310,7 +310,7 @@ static void imx_i2c_realize(DeviceState
   IMX_I2C_MEM_SIZE);
 sysbus_init_mmio(SYS_BUS_DEVICE(dev), &s->iomem);
 sysbus_init_irq(SYS_BUS_DEVICE(dev), &s->irq);
-s->bus = i2c_init_bus(DEVICE(dev), "i2c");
+s->bus = i2c_init_bus(DEVICE(dev), NULL);
 }
  
  static void imx_i2c_class_init(ObjectClass *klass, void *data)

 Which should name automatically the I2C objects :
>>>
>>>
>>> If you ever do migration, you'll have to specify "id" in the command line
>>> anyway. Even in the example below the buses are numbered in messed order,
>>> is that desired effect (may be it is, just asking :) )?
>>
>> That's how it comes out with qom-tree. I haven't dug deeper to see 
>> how this was implemented.
> 
> That's fine, your approach will give unique names, it is just hard to
> predict what i2c device will end up connected to which i2c bus, and I
> usually want to control this instead of guessing (which will work till some
> nonobvious change in QOM :) ).

yes so we could also set an id property before doing realize 
I agree. but for the purpose of this test, I don't think this 
is really needed.

Thanks,

C. 

Re: [Qemu-devel] [PATCH v2 1/6] arm: Uniquely name imx25 I2C buses.

[Alexey Kardashevskiy]

On 02/12/16 18:55, Cédric Le Goater wrote:
> On 12/02/2016 12:34 AM, Alexey Kardashevskiy wrote:
>> On 01/12/16 23:31, Cédric Le Goater wrote:
>>> On 12/01/2016 01:42 AM, Alastair D'Silva wrote:
 On Wed, 2016-11-30 at 09:18 +0100, Cédric Le Goater wrote:
> On 11/30/2016 06:36 AM, Alastair D'Silva wrote:
>> From: Alastair D'Silva 
>>
>> The imx25 chip provides 3 i2c buses, but they have all been named
>> "i2c", which makes it difficult to predict which bus a device will
>> be connected to when specified on the command line.
>>
>> This patch addresses the issue by naming the buses uniquely:
>>   i2c.0 i2c.1 i2c.2
>>
>> Signed-off-by: Alastair D'Silva 
>> ---
>>  hw/arm/imx25_pdk.c | 4 +---
>>  hw/i2c/imx_i2c.c   | 6 +-
>>  2 files changed, 6 insertions(+), 4 deletions(-)
>>
>> diff --git a/hw/arm/imx25_pdk.c b/hw/arm/imx25_pdk.c
>> index 025b608..c6f04d3 100644
>> --- a/hw/arm/imx25_pdk.c
>> +++ b/hw/arm/imx25_pdk.c
>> @@ -138,9 +138,7 @@ static void imx25_pdk_init(MachineState
>> *machine)
>>   * We add it here (only on qtest usage) to be able to do a
>> bit
>>   * of simple qtest. See "make check" for details.
>>   */
>> -i2c_create_slave((I2CBus *)qdev_get_child_bus(DEVICE(&s-
>>> soc.i2c[0]),
>> -  "i2c"),
>> - "ds1338", 0x68);
>> +i2c_create_slave(s->soc.i2c[0].bus, "ds1338", 0x68);
>>  }
>>  }
>>  
>> diff --git a/hw/i2c/imx_i2c.c b/hw/i2c/imx_i2c.c
>> index 37e5a62..7be10fb 100644
>> --- a/hw/i2c/imx_i2c.c
>> +++ b/hw/i2c/imx_i2c.c
>> @@ -305,12 +305,16 @@ static const VMStateDescription
>> imx_i2c_vmstate = {
>>  static void imx_i2c_realize(DeviceState *dev, Error **errp)
>>  {
>>  IMXI2CState *s = IMX_I2C(dev);
>> +static int bus_count;
>
> hmm, the static is ugly :/ 
>
> Isn't there other ways to achieve this naming ? 
>
> Thanks,
>
> C.  
>

 I'm not seeing an obvious way around it. The busses are realized
 independently (so I can't implement what we do with the aspeed i2c
 busses), and it is named before fsl-imx25:fsl_imx25_realize() can apply
 specific properties to the bus.

 If you have any suggestions, I'm all ears.
>>>
>>> What about that ? 
>>>
>>> @@ -310,7 +310,7 @@ static void imx_i2c_realize(DeviceState
>>>IMX_I2C_MEM_SIZE);
>>>  sysbus_init_mmio(SYS_BUS_DEVICE(dev), &s->iomem);
>>>  sysbus_init_irq(SYS_BUS_DEVICE(dev), &s->irq);
>>> -s->bus = i2c_init_bus(DEVICE(dev), "i2c");
>>> +s->bus = i2c_init_bus(DEVICE(dev), NULL);
>>>  }
>>>  
>>>  static void imx_i2c_class_init(ObjectClass *klass, void *data)
>>>
>>> Which should name automatically the I2C objects :
>>
>>
>> If you ever do migration, you'll have to specify "id" in the command line
>> anyway. Even in the example below the buses are numbered in messed order,
>> is that desired effect (may be it is, just asking :) )?
> 
> That's how it comes out with qom-tree. I haven't dug deeper to see 
> how this was implemented.

That's fine, your approach will give unique names, it is just hard to
predict what i2c device will end up connected to which i2c bus, and I
usually want to control this instead of guessing (which will work till some
nonobvious change in QOM :) ).


> 
> C.
> 
>>>
>>> (qemu) info qom-tree 
>>> /machine (imx25-pdk-machine)
>>>   /peripheral (container)
>>>   /soc (fsl,imx25)
>>>   /peripheral-anon (container)
>>>   /unattached (container)
>>> /device[0] (arm926-arm-cpu)
>>>   /unnamed-gpio-in[1] (irq)
>>>   /unnamed-gpio-in[3] (irq)
>>>   /unnamed-gpio-in[2] (irq)
>>>   /unnamed-gpio-in[0] (irq)
>>>
>>> /device[15] (imx.i2c)
>>>   /imx.i2c[0] (qemu:memory-region)
>>>   /i2c-bus.0 (i2c-bus)
>>> /device[17] (imx.i2c)
>>>   /imx.i2c[0] (qemu:memory-region)
>>>   /i2c-bus.2 (i2c-bus)
>>> /device[16] (imx.i2c)
>>>   /imx.i2c[0] (qemu:memory-region)
>>>   /i2c-bus.1 (i2c-bus)
>>>
>>>
>>>
>>> Cheers,
>>>
>>> C. 
>>>
>>
>>
> 


-- 
Alexey

[Qemu-devel] [RFC] target-ppc/fpu_helper.c: Use C99 code to speed up floating point unit

[Programmingkid]

The floating point code used in fpu_helper.c can be sped up by using the IEEE 
754 support added to the C99 standard. To test this code out simply set and 
unset the I_NEED_SPEED macro. The program to test out each version of the 
helper_fmadd() function is below the patch. It needs to be ran in the guest. 
The emulator to use is qemu-system-ppc. I used a Mac OS X guest, but the test 
program would compile on a Linux guest.

This patch does make the fused multiply-add instruction fmadd work faster and 
still give a correct result.

This documentation might be of help to those who want to learn more about C99's 
IEEE 754 support:
http://grouper.ieee.org/groups/754/meeting-materials/2001-07-18-c99.pdf

---
 target-ppc/fpu_helper.c | 105 ++--
 1 file changed, 102 insertions(+), 3 deletions(-)

diff --git a/target-ppc/fpu_helper.c b/target-ppc/fpu_helper.c
index b0760f0..05eb26e 100644
--- a/target-ppc/fpu_helper.c
+++ b/target-ppc/fpu_helper.c
@@ -20,10 +20,22 @@
 #include "cpu.h"
 #include "exec/helper-proto.h"
 #include "exec/exec-all.h"
+#include 
+#include 
+#include 
 
 #define float64_snan_to_qnan(x) ((x) | 0x0008ULL)
 #define float32_snan_to_qnan(x) ((x) | 0x0040)
 
+#define DEBUG_FPU 0
+
+#define DPRINTF(fmt, ...) do {   \
+if (DEBUG_FPU) { \
+printf("FPU: " fmt , ## __VA_ARGS__);\
+}\
+} while (0);
+
+
 /*/
 /* Floating point operations helpers */
 uint64_t helper_float32_to_float64(CPUPPCState *env, uint32_t arg)
@@ -281,29 +293,36 @@ static inline void float_inexact_excp(CPUPPCState *env)
 
 static inline void fpscr_set_rounding_mode(CPUPPCState *env)
 {
-int rnd_type;
+int rnd_type, result = 0;
 
 /* Set rounding mode */
 switch (fpscr_rn) {
 case 0:
 /* Best approximation (round to nearest) */
 rnd_type = float_round_nearest_even;
+result = fesetround(FE_TONEAREST);
 break;
 case 1:
 /* Smaller magnitude (round toward zero) */
 rnd_type = float_round_to_zero;
+result = fesetround(FE_TOWARDZERO);
 break;
 case 2:
 /* Round toward +infinite */
 rnd_type = float_round_up;
+result = fesetround(FE_UPWARD);
 break;
 default:
 case 3:
 /* Round toward -infinite */
 rnd_type = float_round_down;
+result = fesetround(FE_DOWNWARD);
 break;
 }
 set_float_rounding_mode(rnd_type, &env->fp_status);
+if (result != 0) {
+printf("Error: rounding mode was not set\n");
+}
 }
 
 void helper_fpscr_clrbit(CPUPPCState *env, uint32_t bit)
@@ -534,6 +553,7 @@ void helper_float_check_status(CPUPPCState *env)
 void helper_reset_fpstatus(CPUPPCState *env)
 {
 set_float_exception_flags(0, &env->fp_status);
+feclearexcept(FE_ALL_EXCEPT);
 }
 
 /* fadd - fadd. */
@@ -737,16 +757,94 @@ uint64_t helper_frim(CPUPPCState *env, uint64_t arg)
 return do_fri(env, arg, float_round_down);
 }
 
-/* fmadd - fmadd. */
+#define I_NEED_SPEED 1
+#ifdef I_NEED_SPEED
+
+union Converter {
+uint64_t i;
+double d;
+};
+
+typedef union Converter Converter;
+
+/* fmadd - fmadd. - fast */
 uint64_t helper_fmadd(CPUPPCState *env, uint64_t arg1, uint64_t arg2,
   uint64_t arg3)
 {
+DPRINTF("Fast helper_fmadd() called\n");
+Converter farg1, farg2, farg3, result;
+
+farg1.i = arg1;
+farg2.i = arg2;
+farg3.i = arg3;
+
+DPRINTF("farg1.d = %f\n", farg1.d);
+DPRINTF("farg2.d = %f\n", farg2.d);
+DPRINTF("farg3.d = %f\n", farg3.d);
+
+/* if signalling NaN operation */
+if (unlikely(float64_is_signaling_nan(farg1.d, &env->fp_status) ||
+ float64_is_signaling_nan(farg2.d, &env->fp_status) ||
+ float64_is_signaling_nan(farg3.d, &env->fp_status))) {
+float_invalid_op_excp(env, POWERPC_EXCP_FP_VXSNAN, 1);
+}
+
+result.d = fma(farg1.d, farg2.d, farg3.d); /* fused multiply-add function 
*/
+if (fetestexcept(FE_INEXACT)) {
+DPRINTF("FE_INEXACT\n");
+float_inexact_excp(env);
+}
+if (fetestexcept(FE_INVALID)) {
+DPRINTF("FE_INVALID\n");
+
+/* 0 * infinity */
+if ((fpclassify(farg1.d) == FP_ZERO) && isinf(farg2.d)) {
+result.i = float_invalid_op_excp(env, POWERPC_EXCP_FP_VXIMZ, 1);
+}
+
+/* infinity * 0 */
+else if (isinf(farg1.d) && (fpclassify(farg2.d) == FP_ZERO)) {
+result.i = float_invalid_op_excp(env, POWERPC_EXCP_FP_VXIMZ, 1);
+}
+
+/* infinity - infinity */
+else if (isinf(farg1.d * farg2.d) && isinf(farg3.d) && 
(signbit(farg3.d) != 0)) {
+result.i = float_invalid_op_excp(env, POWERPC_EXCP_FP_VXISI, 1);
+}
+}
+if (fetestexcept(FE_OVERFLOW)) {
+

[Qemu-devel] [PATCH 6/7] target-ppc: Implement bcdtrunc. instruction

[Jose Ricardo Ziviani]

bcdtrunc.: Decimal integer truncate. Given a BCD number in vrb and the
number of bytes to truncate in vra, the return register will have vrb
with such bits truncated.

Signed-off-by: Jose Ricardo Ziviani 
---
 target-ppc/helper.h |  1 +
 target-ppc/int_helper.c | 43 +
 target-ppc/translate/vmx-impl.inc.c |  5 +
 target-ppc/translate/vmx-ops.inc.c  |  4 ++--
 4 files changed, 51 insertions(+), 2 deletions(-)

diff --git a/target-ppc/helper.h b/target-ppc/helper.h
index d9528eb..49965b0 100644
--- a/target-ppc/helper.h
+++ b/target-ppc/helper.h
@@ -395,6 +395,7 @@ DEF_HELPER_3(bcdsetsgn, i32, avr, avr, i32)
 DEF_HELPER_4(bcds, i32, avr, avr, avr, i32)
 DEF_HELPER_4(bcdus, i32, avr, avr, avr, i32)
 DEF_HELPER_4(bcdsr, i32, avr, avr, avr, i32)
+DEF_HELPER_4(bcdtrunc, i32, avr, avr, avr, i32)
 
 DEF_HELPER_2(xsadddp, void, env, i32)
 DEF_HELPER_2(xssubdp, void, env, i32)
diff --git a/target-ppc/int_helper.c b/target-ppc/int_helper.c
index c9fcb1a..a8fc718 100644
--- a/target-ppc/int_helper.c
+++ b/target-ppc/int_helper.c
@@ -3169,6 +3169,49 @@ uint32_t helper_bcdsr(ppc_avr_t *r, ppc_avr_t *a, 
ppc_avr_t *b, uint32_t ps)
 return cr;
 }
 
+uint32_t helper_bcdtrunc(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b, uint32_t ps)
+{
+int i;
+int cr;
+int ox_flag;
+uint8_t digit;
+uint8_t trunc;
+int invalid = 0;
+int sgnb = bcd_get_sgn(b);
+ppc_avr_t ret = *b;
+
+#if defined(HOST_WORDS_BIGENDIAN)
+int upper = ARRAY_SIZE(a->u16) - 1;
+#else
+int upper = 0;
+#endif
+
+trunc = 32 - (a->u16[upper] & 31);
+for (i = 1; i < 32; i++) {
+digit = bcd_get_digit(b, i, &invalid);
+
+if (unlikely(invalid)) {
+return CRF_SO;
+}
+
+if (i >= trunc) {
+if (!ox_flag && digit > 0x0) {
+ox_flag = 1;
+}
+bcd_put_digit(&ret, 0, i);
+}
+}
+bcd_put_digit(&ret, bcd_preferred_sgn(sgnb, ps), 0);
+
+cr = bcd_cmp_zero(&ret);
+if (unlikely(ox_flag)) {
+cr |= CRF_SO;
+}
+*r = ret;
+
+return cr;
+}
+
 void helper_vsbox(ppc_avr_t *r, ppc_avr_t *a)
 {
 int i;
diff --git a/target-ppc/translate/vmx-impl.inc.c 
b/target-ppc/translate/vmx-impl.inc.c
index 451abb5..1683f42 100644
--- a/target-ppc/translate/vmx-impl.inc.c
+++ b/target-ppc/translate/vmx-impl.inc.c
@@ -1019,6 +1019,7 @@ GEN_BCD(bcdcpsgn);
 GEN_BCD(bcds);
 GEN_BCD(bcdus);
 GEN_BCD(bcdsr);
+GEN_BCD(bcdtrunc);
 
 static void gen_xpnd04_1(DisasContext *ctx)
 {
@@ -1097,6 +1098,10 @@ GEN_VXFORM_DUAL(vsubudm, PPC2_ALTIVEC_207, PPC_NONE, \
 bcds, PPC_NONE, PPC2_ISA300)
 GEN_VXFORM_DUAL(vsubuwm, PPC_ALTIVEC, PPC_NONE, \
 bcdus, PPC_NONE, PPC2_ISA300)
+GEN_VXFORM_DUAL(vsubsbs, PPC_ALTIVEC, PPC_NONE, \
+bcdtrunc, PPC_NONE, PPC2_ISA300)
+GEN_VXFORM_DUAL(vsubuqm, PPC2_ALTIVEC_207, PPC_NONE, \
+bcdtrunc, PPC_NONE, PPC2_ISA300)
 
 static void gen_vsbox(DisasContext *ctx)
 {
diff --git a/target-ppc/translate/vmx-ops.inc.c 
b/target-ppc/translate/vmx-ops.inc.c
index fa9c996..e6167a4 100644
--- a/target-ppc/translate/vmx-ops.inc.c
+++ b/target-ppc/translate/vmx-ops.inc.c
@@ -143,14 +143,14 @@ GEN_VXFORM(vaddsws, 0, 14),
 GEN_VXFORM_DUAL(vsububs, bcdadd, 0, 24, PPC_ALTIVEC, PPC_NONE),
 GEN_VXFORM_DUAL(vsubuhs, bcdsub, 0, 25, PPC_ALTIVEC, PPC_NONE),
 GEN_VXFORM(vsubuws, 0, 26),
-GEN_VXFORM(vsubsbs, 0, 28),
+GEN_VXFORM_DUAL(vsubsbs, bcdtrunc, 0, 28, PPC_NONE, PPC2_ISA300),
 GEN_VXFORM(vsubshs, 0, 29),
 GEN_VXFORM_DUAL(vsubsws, xpnd04_2, 0, 30, PPC_ALTIVEC, PPC_NONE),
 GEN_VXFORM_207(vadduqm, 0, 4),
 GEN_VXFORM_207(vaddcuq, 0, 5),
 GEN_VXFORM_DUAL(vaddeuqm, vaddecuq, 30, 0xFF, PPC_NONE, PPC2_ALTIVEC_207),
-GEN_VXFORM_207(vsubuqm, 0, 20),
 GEN_VXFORM_207(vsubcuq, 0, 21),
+GEN_VXFORM_DUAL(vsubuqm, bcdtrunc, 0, 20, PPC2_ALTIVEC_207, PPC2_ISA300),
 GEN_VXFORM_DUAL(vsubeuqm, vsubecuq, 31, 0xFF, PPC_NONE, PPC2_ALTIVEC_207),
 GEN_VXFORM(vrlb, 2, 0),
 GEN_VXFORM(vrlh, 2, 1),
-- 
2.7.4

[Qemu-devel] [PATCH 7/7] target-ppc: Implement bcdtrunc. instruction

[Jose Ricardo Ziviani]

bcdutrunc. Decimal unsigned truncate. Works like bcdtrunc. with
unsigned BCD numbers.

Signed-off-by: Jose Ricardo Ziviani 
---
 target-ppc/helper.h |  1 +
 target-ppc/int_helper.c | 39 +
 target-ppc/translate/vmx-impl.inc.c |  4 
 target-ppc/translate/vmx-ops.inc.c  |  2 +-
 4 files changed, 45 insertions(+), 1 deletion(-)

diff --git a/target-ppc/helper.h b/target-ppc/helper.h
index 49965b0..52a2707 100644
--- a/target-ppc/helper.h
+++ b/target-ppc/helper.h
@@ -396,6 +396,7 @@ DEF_HELPER_4(bcds, i32, avr, avr, avr, i32)
 DEF_HELPER_4(bcdus, i32, avr, avr, avr, i32)
 DEF_HELPER_4(bcdsr, i32, avr, avr, avr, i32)
 DEF_HELPER_4(bcdtrunc, i32, avr, avr, avr, i32)
+DEF_HELPER_4(bcdutrunc, i32, avr, avr, avr, i32)
 
 DEF_HELPER_2(xsadddp, void, env, i32)
 DEF_HELPER_2(xssubdp, void, env, i32)
diff --git a/target-ppc/int_helper.c b/target-ppc/int_helper.c
index a8fc718..ab531e8 100644
--- a/target-ppc/int_helper.c
+++ b/target-ppc/int_helper.c
@@ -3212,6 +3212,45 @@ uint32_t helper_bcdtrunc(ppc_avr_t *r, ppc_avr_t *a, 
ppc_avr_t *b, uint32_t ps)
 return cr;
 }
 
+uint32_t helper_bcdutrunc(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b, uint32_t 
ps)
+{
+int i;
+uint8_t digit;
+uint8_t trunc;
+int ox_flag = 0;
+int invalid = 0;
+ppc_avr_t ret = *b;
+
+#if defined(HOST_WORDS_BIGENDIAN)
+int upper = ARRAY_SIZE(a->u16) - 1;
+#else
+int upper = 0;
+#endif
+
+trunc = 32 - (a->u16[upper] % 33);
+for (i = 0; i < 32; i++) {
+digit = bcd_get_digit(b, i, &invalid);
+
+if (unlikely(invalid)) {
+return CRF_SO;
+}
+
+if (i >= trunc) {
+if (unlikely(!ox_flag && digit > 0x0)) {
+ox_flag = 1;
+}
+bcd_put_digit(&ret, 0, i);
+}
+}
+
+*r = ret;
+if (r->u64[HI_IDX] == 0 && r->u64[LO_IDX] == 0) {
+return (ox_flag) ? CRF_SO | CRF_EQ : CRF_EQ;
+} else {
+return (ox_flag) ? CRF_SO | CRF_GT : CRF_GT;
+}
+}
+
 void helper_vsbox(ppc_avr_t *r, ppc_avr_t *a)
 {
 int i;
diff --git a/target-ppc/translate/vmx-impl.inc.c 
b/target-ppc/translate/vmx-impl.inc.c
index 1683f42..3cb6fc2 100644
--- a/target-ppc/translate/vmx-impl.inc.c
+++ b/target-ppc/translate/vmx-impl.inc.c
@@ -1020,6 +1020,7 @@ GEN_BCD(bcds);
 GEN_BCD(bcdus);
 GEN_BCD(bcdsr);
 GEN_BCD(bcdtrunc);
+GEN_BCD(bcdutrunc);
 
 static void gen_xpnd04_1(DisasContext *ctx)
 {
@@ -1102,6 +1103,9 @@ GEN_VXFORM_DUAL(vsubsbs, PPC_ALTIVEC, PPC_NONE, \
 bcdtrunc, PPC_NONE, PPC2_ISA300)
 GEN_VXFORM_DUAL(vsubuqm, PPC2_ALTIVEC_207, PPC_NONE, \
 bcdtrunc, PPC_NONE, PPC2_ISA300)
+GEN_VXFORM_DUAL(vsubcuq, PPC2_ALTIVEC_207, PPC_NONE, \
+bcdutrunc, PPC_NONE, PPC2_ISA300)
+
 
 static void gen_vsbox(DisasContext *ctx)
 {
diff --git a/target-ppc/translate/vmx-ops.inc.c 
b/target-ppc/translate/vmx-ops.inc.c
index e6167a4..139f80c 100644
--- a/target-ppc/translate/vmx-ops.inc.c
+++ b/target-ppc/translate/vmx-ops.inc.c
@@ -149,8 +149,8 @@ GEN_VXFORM_DUAL(vsubsws, xpnd04_2, 0, 30, PPC_ALTIVEC, 
PPC_NONE),
 GEN_VXFORM_207(vadduqm, 0, 4),
 GEN_VXFORM_207(vaddcuq, 0, 5),
 GEN_VXFORM_DUAL(vaddeuqm, vaddecuq, 30, 0xFF, PPC_NONE, PPC2_ALTIVEC_207),
-GEN_VXFORM_207(vsubcuq, 0, 21),
 GEN_VXFORM_DUAL(vsubuqm, bcdtrunc, 0, 20, PPC2_ALTIVEC_207, PPC2_ISA300),
+GEN_VXFORM_DUAL(vsubcuq, bcdutrunc, 0, 21, PPC2_ALTIVEC_207, PPC2_ISA300),
 GEN_VXFORM_DUAL(vsubeuqm, vsubecuq, 31, 0xFF, PPC_NONE, PPC2_ALTIVEC_207),
 GEN_VXFORM(vrlb, 2, 0),
 GEN_VXFORM(vrlh, 2, 1),
-- 
2.7.4

[Qemu-devel] [PATCH 2/7] target-ppc: Implement unsigned quadword left/right shift and unit tests

[Jose Ricardo Ziviani]

This commit implements functions to right and left shifts and the
unittest for them. Such functions is needed due to instructions
that requires them.

Today, there is already a right shift implementation in int128.h
but it's for signed numbers.

Signed-off-by: Jose Ricardo Ziviani 
---
 include/qemu/host-utils.h | 29 ++
 tests/Makefile.include|  5 ++-
 tests/test-shift128.c | 97 +++
 util/host-utils.c | 38 +++
 4 files changed, 168 insertions(+), 1 deletion(-)
 create mode 100644 tests/test-shift128.c

diff --git a/include/qemu/host-utils.h b/include/qemu/host-utils.h
index 46187bb..b3e5f72 100644
--- a/include/qemu/host-utils.h
+++ b/include/qemu/host-utils.h
@@ -29,6 +29,33 @@
 #include "qemu/bswap.h"
 
 #ifdef CONFIG_INT128
+static inline void urshift(uint64_t *plow, uint64_t *phigh, uint32_t shift)
+{
+__uint128_t val = ((__uint128_t)*phigh << 64) | *plow;
+val >>= (shift & 127);
+*phigh = val >> 64;
+*plow = val & 0x;
+}
+
+static inline void ulshift(uint64_t *plow, uint64_t *phigh,
+   uint32_t shift, bool *overflow)
+{
+__uint128_t val = ((__uint128_t)*phigh << 64) | *plow;
+
+if (shift == 0) {
+return;
+}
+
+if (shift > 127 || (val >> (128 - (shift & 127))) != 0) {
+*overflow = true;
+}
+
+val <<= (shift & 127);
+
+*phigh = val >> 64;
+*plow = val & 0x;
+}
+
 static inline void mulu64(uint64_t *plow, uint64_t *phigh,
   uint64_t a, uint64_t b)
 {
@@ -81,6 +108,8 @@ void muls64(uint64_t *phigh, uint64_t *plow, int64_t a, 
int64_t b);
 void mulu64(uint64_t *phigh, uint64_t *plow, uint64_t a, uint64_t b);
 int divu128(uint64_t *plow, uint64_t *phigh, uint64_t divisor);
 int divs128(int64_t *plow, int64_t *phigh, int64_t divisor);
+void urshift(uint64_t *plow, uint64_t *phigh, uint32_t shift);
+void ulshift(uint64_t *plow, uint64_t *phigh, uint32_t shift, bool *overflow);
 
 static inline uint64_t muldiv64(uint64_t a, uint32_t b, uint32_t c)
 {
diff --git a/tests/Makefile.include b/tests/Makefile.include
index e98d3b6..89e5e85 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -65,6 +65,8 @@ check-unit-$(CONFIG_POSIX) += tests/test-vmstate$(EXESUF)
 endif
 check-unit-y += tests/test-cutils$(EXESUF)
 gcov-files-test-cutils-y += util/cutils.c
+check-unit-y += tests/test-shift128$(EXESUF)
+gcov-files-test-shift128-y = util/host-utils.c
 check-unit-y += tests/test-mul64$(EXESUF)
 gcov-files-test-mul64-y = util/host-utils.c
 check-unit-y += tests/test-int128$(EXESUF)
@@ -460,7 +462,7 @@ test-obj-y = tests/check-qint.o tests/check-qstring.o 
tests/check-qdict.o \
tests/test-x86-cpuid.o tests/test-mul64.o tests/test-int128.o \
tests/test-opts-visitor.o tests/test-qmp-event.o \
tests/rcutorture.o tests/test-rcu-list.o \
-   tests/test-qdist.o \
+   tests/test-qdist.o tests/test-shift128.o \
tests/test-qht.o tests/qht-bench.o tests/test-qht-par.o \
tests/atomic_add-bench.o
 
@@ -568,6 +570,7 @@ tests/test-qmp-commands$(EXESUF): tests/test-qmp-commands.o 
tests/test-qmp-marsh
 tests/test-visitor-serialization$(EXESUF): tests/test-visitor-serialization.o 
$(test-qapi-obj-y)
 tests/test-opts-visitor$(EXESUF): tests/test-opts-visitor.o $(test-qapi-obj-y)
 
+tests/test-shift128$(EXESUF): tests/test-shift128.o $(test-util-obj-y)
 tests/test-mul64$(EXESUF): tests/test-mul64.o $(test-util-obj-y)
 tests/test-bitops$(EXESUF): tests/test-bitops.o $(test-util-obj-y)
 tests/test-crypto-hash$(EXESUF): tests/test-crypto-hash.o $(test-crypto-obj-y)
diff --git a/tests/test-shift128.c b/tests/test-shift128.c
new file mode 100644
index 000..67615c9
--- /dev/null
+++ b/tests/test-shift128.c
@@ -0,0 +1,97 @@
+/*
+ * Test unsigned left and right shift
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/host-utils.h"
+
+typedef struct {
+uint64_t low;
+uint64_t high;
+uint64_t rlow;
+uint64_t rhigh;
+int32_t shift;
+bool overflow;
+} test_data;
+
+static const test_data test_ltable[] = {
+{ 1223ULL, 0, 1223ULL,   0, 0, false },
+{ 1ULL,0, 2ULL,   0, 1, false },
+{ 1ULL,0, 4ULL,   0, 2, false },
+{ 1ULL,0, 16ULL,  0, 4, false },
+{ 1ULL,0, 256ULL, 0, 8, false },
+{ 1ULL,0, 65536ULL, 0, 16, false },
+{ 1ULL,0, 2147483648ULL, 0, 31, false },
+{ 1ULL,0, 35184372088832ULL, 0, 45, false },
+{ 1ULL,0, 1152921504606846976ULL, 0, 60, false },
+{ 1ULL,0, 0, 1ULL, 64, false },
+{ 1ULL,0, 0, 65536ULL, 80, false },
+{ 1ULL,0, 0, 9223372036854775808ULL, 127, false },
+{ 0ULL,1, 0, 0, 64, true },
+{ 0xULL, 0xULL,
+0x8000ULL, 0x9888

[Qemu-devel] [PATCH 5/7] target-ppc: Implement bcdsr. instruction

[Jose Ricardo Ziviani]

bcdsr.: Decimal shift and round. This instruction works like bcds.
however, when performing right shift, 1 will be added to the
result if the last digit was >= 5.

Signed-off-by: Jose Ricardo Ziviani 
---
 target-ppc/helper.h |  1 +
 target-ppc/int_helper.c | 45 +
 target-ppc/translate/vmx-impl.inc.c |  1 +
 target-ppc/translate/vmx-ops.inc.c  |  2 ++
 4 files changed, 49 insertions(+)

diff --git a/target-ppc/helper.h b/target-ppc/helper.h
index 386ea67..d9528eb 100644
--- a/target-ppc/helper.h
+++ b/target-ppc/helper.h
@@ -394,6 +394,7 @@ DEF_HELPER_4(bcdcpsgn, i32, avr, avr, avr, i32)
 DEF_HELPER_3(bcdsetsgn, i32, avr, avr, i32)
 DEF_HELPER_4(bcds, i32, avr, avr, avr, i32)
 DEF_HELPER_4(bcdus, i32, avr, avr, avr, i32)
+DEF_HELPER_4(bcdsr, i32, avr, avr, avr, i32)
 
 DEF_HELPER_2(xsadddp, void, env, i32)
 DEF_HELPER_2(xssubdp, void, env, i32)
diff --git a/target-ppc/int_helper.c b/target-ppc/int_helper.c
index 4b5eea1..c9fcb1a 100644
--- a/target-ppc/int_helper.c
+++ b/target-ppc/int_helper.c
@@ -3124,6 +3124,51 @@ uint32_t helper_bcdus(ppc_avr_t *r, ppc_avr_t *a, 
ppc_avr_t *b, uint32_t ps)
 return cr;
 }
 
+uint32_t helper_bcdsr(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b, uint32_t ps)
+{
+int cr;
+int i;
+int unused = 0;
+int invalid = 0;
+bool ox_flag = false;
+int sgnb = bcd_get_sgn(b);
+ppc_avr_t ret = *b;
+ret.u64[LO_IDX] &= ~0xf;
+
+#if defined(HOST_WORDS_BIGENDIAN)
+ppc_avr_t bcd_one = { .u64 = { 0, 0x10 } };
+int upper = ARRAY_SIZE(a->s32) - 1;
+#else
+ppc_avr_t bcd_one = { .u64 = { 0x10, 0 } };
+int upper = 0;
+#endif
+
+if (bcd_is_valid(b) == false) {
+return CRF_SO;
+}
+
+if (a->s32[upper] > 0) {
+i = (a->s32[upper] > 31) ? 31 : a->s32[upper];
+ulshift(&ret.u64[LO_IDX], &ret.u64[HI_IDX], i * 4, &ox_flag);
+} else {
+i = (a->s32[upper] < -31) ? 31 : -a->s32[upper];
+urshift(&ret.u64[LO_IDX], &ret.u64[HI_IDX], i * 4);
+
+if (bcd_get_digit(&ret, 0, &invalid) >= 5) {
+bcd_add_mag(&ret, &ret, &bcd_one, &invalid, &unused);
+}
+}
+bcd_put_digit(&ret, bcd_preferred_sgn(sgnb, ps), 0);
+
+cr = bcd_cmp_zero(&ret);
+if (unlikely(ox_flag)) {
+cr |= CRF_SO;
+}
+*r = ret;
+
+return cr;
+}
+
 void helper_vsbox(ppc_avr_t *r, ppc_avr_t *a)
 {
 int i;
diff --git a/target-ppc/translate/vmx-impl.inc.c 
b/target-ppc/translate/vmx-impl.inc.c
index fc54881..451abb5 100644
--- a/target-ppc/translate/vmx-impl.inc.c
+++ b/target-ppc/translate/vmx-impl.inc.c
@@ -1018,6 +1018,7 @@ GEN_BCD2(bcdsetsgn)
 GEN_BCD(bcdcpsgn);
 GEN_BCD(bcds);
 GEN_BCD(bcdus);
+GEN_BCD(bcdsr);
 
 static void gen_xpnd04_1(DisasContext *ctx)
 {
diff --git a/target-ppc/translate/vmx-ops.inc.c 
b/target-ppc/translate/vmx-ops.inc.c
index cdd3abe..fa9c996 100644
--- a/target-ppc/translate/vmx-ops.inc.c
+++ b/target-ppc/translate/vmx-ops.inc.c
@@ -132,6 +132,8 @@ GEN_HANDLER_E_2(vprtybd, 0x4, 0x1, 0x18, 9, 0, PPC_NONE, 
PPC2_ISA300),
 GEN_HANDLER_E_2(vprtybq, 0x4, 0x1, 0x18, 10, 0, PPC_NONE, PPC2_ISA300),
 
 GEN_VXFORM_DUAL(vsubcuw, xpnd04_1, 0, 22, PPC_ALTIVEC, PPC_NONE),
+GEN_VXFORM_300(bcdsr, 0, 23),
+GEN_VXFORM_300(bcdsr, 0, 31),
 GEN_VXFORM_DUAL(vaddubs, vmul10uq, 0, 8, PPC_ALTIVEC, PPC_NONE),
 GEN_VXFORM_DUAL(vadduhs, vmul10euq, 0, 9, PPC_ALTIVEC, PPC_NONE),
 GEN_VXFORM(vadduws, 0, 10),
-- 
2.7.4

[Qemu-devel] [PATCH 0/7] POWER9 TCG enablements - BCD functions - final part

[Jose Ricardo Ziviani]

This serie contains 5 new instructions for POWER9 ISA3.0, left/right shifts for 
unsigned quadwords and a small improvement to check whether a bcd value is 
valid or not.

bcds.: Decimal signed shift
bcdus.: Decimal unsigned shift
bcdsr.: Decimal shift and round
bcdtrunc.: Decimal signed trucate
bcdtrunc.: Decimal unsigned truncate

Jose Ricardo Ziviani (7):
  target-ppc: Implement bcd_is_valid function
  target-ppc: Implement unsigned quadword left/right shift and unit
tests
  target-ppc: Implement bcds. instruction
  target-ppc: Implement bcdus. instruction
  target-ppc: Implement bcdsr. instruction
  target-ppc: Implement bcdtrunc. instruction
  target-ppc: Implement bcdtrunc. instruction

 include/qemu/host-utils.h   |  29 +
 target-ppc/helper.h |   5 +
 target-ppc/int_helper.c | 229 +++-
 target-ppc/translate/vmx-impl.inc.c |  16 +++
 target-ppc/translate/vmx-ops.inc.c  |  13 +-
 tests/Makefile.include  |   5 +-
 tests/test-shift128.c   |  97 +++
 util/host-utils.c   |  38 ++
 8 files changed, 422 insertions(+), 10 deletions(-)
 create mode 100644 tests/test-shift128.c

-- 
2.7.4

[Qemu-devel] [PATCH 1/7] target-ppc: Implement bcd_is_valid function

[Jose Ricardo Ziviani]

A function to check if all digits of a given BCD number is valid is
here presented because more instructions will need to reuse the
same code.

Signed-off-by: Jose Ricardo Ziviani 
---
 target-ppc/int_helper.c | 27 ---
 1 file changed, 20 insertions(+), 7 deletions(-)

diff --git a/target-ppc/int_helper.c b/target-ppc/int_helper.c
index 7030f61..7989b1f 100644
--- a/target-ppc/int_helper.c
+++ b/target-ppc/int_helper.c
@@ -2596,6 +2596,24 @@ static void bcd_put_digit(ppc_avr_t *bcd, uint8_t digit, 
int n)
 }
 }
 
+static bool bcd_is_valid(ppc_avr_t *bcd)
+{
+int i;
+int invalid = 0;
+
+if (bcd_get_sgn(bcd) == 0) {
+return false;
+}
+
+for (i = 1; i < 32; i++) {
+bcd_get_digit(bcd, i, &invalid);
+if (unlikely(invalid)) {
+return false;
+}
+}
+return true;
+}
+
 static int bcd_cmp_zero(ppc_avr_t *bcd)
 {
 if (bcd->u64[HI_IDX] == 0 && (bcd->u64[LO_IDX] >> 4) == 0) {
@@ -3013,18 +3031,13 @@ uint32_t helper_bcdcpsgn(ppc_avr_t *r, ppc_avr_t *a, 
ppc_avr_t *b, uint32_t ps)
 
 uint32_t helper_bcdsetsgn(ppc_avr_t *r, ppc_avr_t *b, uint32_t ps)
 {
-int i;
-int invalid = 0;
 int sgnb = bcd_get_sgn(b);
 
 *r = *b;
 bcd_put_digit(r, bcd_preferred_sgn(sgnb, ps), 0);
 
-for (i = 1; i < 32; i++) {
-bcd_get_digit(b, i, &invalid);
-if (unlikely(invalid)) {
-return CRF_SO;
-}
+if (bcd_is_valid(b) == false) {
+return CRF_SO;
 }
 
 return bcd_cmp_zero(r);
-- 
2.7.4

[Qemu-devel] [PATCH 3/7] target-ppc: Implement bcds. instruction

[Jose Ricardo Ziviani]

bcds.: Decimal shift. Given two registers vra and vrb, this instruction
shift the vrb value by vra bits into the result register.

Signed-off-by: Jose Ricardo Ziviani 
---
 target-ppc/helper.h |  1 +
 target-ppc/int_helper.c | 38 +
 target-ppc/translate/vmx-impl.inc.c |  3 +++
 target-ppc/translate/vmx-ops.inc.c  |  3 ++-
 4 files changed, 44 insertions(+), 1 deletion(-)

diff --git a/target-ppc/helper.h b/target-ppc/helper.h
index bc39efb..471a1da 100644
--- a/target-ppc/helper.h
+++ b/target-ppc/helper.h
@@ -392,6 +392,7 @@ DEF_HELPER_3(bcdcfsq, i32, avr, avr, i32)
 DEF_HELPER_3(bcdctsq, i32, avr, avr, i32)
 DEF_HELPER_4(bcdcpsgn, i32, avr, avr, avr, i32)
 DEF_HELPER_3(bcdsetsgn, i32, avr, avr, i32)
+DEF_HELPER_4(bcds, i32, avr, avr, avr, i32)
 
 DEF_HELPER_2(xsadddp, void, env, i32)
 DEF_HELPER_2(xssubdp, void, env, i32)
diff --git a/target-ppc/int_helper.c b/target-ppc/int_helper.c
index 7989b1f..b25c020 100644
--- a/target-ppc/int_helper.c
+++ b/target-ppc/int_helper.c
@@ -3043,6 +3043,44 @@ uint32_t helper_bcdsetsgn(ppc_avr_t *r, ppc_avr_t *b, 
uint32_t ps)
 return bcd_cmp_zero(r);
 }
 
+uint32_t helper_bcds(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b, uint32_t ps)
+{
+int cr;
+int i = 0;
+bool ox_flag = false;
+int sgnb = bcd_get_sgn(b);
+ppc_avr_t ret = *b;
+ret.u64[LO_IDX] &= ~0xf;
+
+#if defined(HOST_WORDS_BIGENDIAN)
+int upper = ARRAY_SIZE(a->s32) - 1;
+#else
+int upper = 0;
+#endif
+
+if (bcd_is_valid(b) == false) {
+return CRF_SO;
+}
+
+if (a->s32[upper] > 0) {
+i = (a->s32[upper] > 31) ? 31 : a->s32[upper];
+ulshift(&ret.u64[LO_IDX], &ret.u64[HI_IDX], i * 4, &ox_flag);
+} else {
+i = (a->s32[upper] < -31) ? 31 : -a->s32[upper];
+urshift(&ret.u64[LO_IDX], &ret.u64[HI_IDX], i * 4);
+}
+bcd_put_digit(&ret, bcd_preferred_sgn(sgnb, ps), 0);
+
+*r = ret;
+
+cr = bcd_cmp_zero(r);
+if (unlikely(ox_flag)) {
+cr |= CRF_SO;
+}
+
+return cr;
+}
+
 void helper_vsbox(ppc_avr_t *r, ppc_avr_t *a)
 {
 int i;
diff --git a/target-ppc/translate/vmx-impl.inc.c 
b/target-ppc/translate/vmx-impl.inc.c
index e8e527f..84ebb7e 100644
--- a/target-ppc/translate/vmx-impl.inc.c
+++ b/target-ppc/translate/vmx-impl.inc.c
@@ -1016,6 +1016,7 @@ GEN_BCD2(bcdcfsq)
 GEN_BCD2(bcdctsq)
 GEN_BCD2(bcdsetsgn)
 GEN_BCD(bcdcpsgn);
+GEN_BCD(bcds);
 
 static void gen_xpnd04_1(DisasContext *ctx)
 {
@@ -1090,6 +1091,8 @@ GEN_VXFORM_DUAL(vsubuhs, PPC_ALTIVEC, PPC_NONE, \
 bcdsub, PPC_NONE, PPC2_ALTIVEC_207)
 GEN_VXFORM_DUAL(vaddshs, PPC_ALTIVEC, PPC_NONE, \
 bcdcpsgn, PPC_NONE, PPC2_ISA300)
+GEN_VXFORM_DUAL(vsubudm, PPC2_ALTIVEC_207, PPC_NONE, \
+bcds, PPC_NONE, PPC2_ISA300)
 
 static void gen_vsbox(DisasContext *ctx)
 {
diff --git a/target-ppc/translate/vmx-ops.inc.c 
b/target-ppc/translate/vmx-ops.inc.c
index 57dce6e..7b4b009 100644
--- a/target-ppc/translate/vmx-ops.inc.c
+++ b/target-ppc/translate/vmx-ops.inc.c
@@ -62,7 +62,8 @@ GEN_VXFORM_207(vaddudm, 0, 3),
 GEN_VXFORM_DUAL(vsububm, bcdadd, 0, 16, PPC_ALTIVEC, PPC_NONE),
 GEN_VXFORM_DUAL(vsubuhm, bcdsub, 0, 17, PPC_ALTIVEC, PPC_NONE),
 GEN_VXFORM(vsubuwm, 0, 18),
-GEN_VXFORM_207(vsubudm, 0, 19),
+GEN_VXFORM_DUAL(vsubudm, bcds, 0, 19, PPC2_ALTIVEC_207, PPC2_ISA300),
+GEN_VXFORM_300(bcds, 0, 27),
 GEN_VXFORM(vmaxub, 1, 0),
 GEN_VXFORM(vmaxuh, 1, 1),
 GEN_VXFORM(vmaxuw, 1, 2),
-- 
2.7.4

[Qemu-devel] [PATCH 4/7] target-ppc: Implement bcdus. instruction

[Jose Ricardo Ziviani]

bcdus.: Decimal unsigned shift. This instruction works like bcds. but
considers only unsigned BCDs (no sign in least meaning 4 bits).

Signed-off-by: Jose Ricardo Ziviani 
---
 target-ppc/helper.h |  1 +
 target-ppc/int_helper.c | 43 +
 target-ppc/translate/vmx-impl.inc.c |  3 +++
 target-ppc/translate/vmx-ops.inc.c  |  2 +-
 4 files changed, 48 insertions(+), 1 deletion(-)

diff --git a/target-ppc/helper.h b/target-ppc/helper.h
index 471a1da..386ea67 100644
--- a/target-ppc/helper.h
+++ b/target-ppc/helper.h
@@ -393,6 +393,7 @@ DEF_HELPER_3(bcdctsq, i32, avr, avr, i32)
 DEF_HELPER_4(bcdcpsgn, i32, avr, avr, avr, i32)
 DEF_HELPER_3(bcdsetsgn, i32, avr, avr, i32)
 DEF_HELPER_4(bcds, i32, avr, avr, avr, i32)
+DEF_HELPER_4(bcdus, i32, avr, avr, avr, i32)
 
 DEF_HELPER_2(xsadddp, void, env, i32)
 DEF_HELPER_2(xssubdp, void, env, i32)
diff --git a/target-ppc/int_helper.c b/target-ppc/int_helper.c
index b25c020..4b5eea1 100644
--- a/target-ppc/int_helper.c
+++ b/target-ppc/int_helper.c
@@ -3081,6 +3081,49 @@ uint32_t helper_bcds(ppc_avr_t *r, ppc_avr_t *a, 
ppc_avr_t *b, uint32_t ps)
 return cr;
 }
 
+uint32_t helper_bcdus(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b, uint32_t ps)
+{
+int cr;
+int i;
+int invalid = 0;
+bool ox_flag = false;
+ppc_avr_t ret = *b;
+
+#if defined(HOST_WORDS_BIGENDIAN)
+int upper = ARRAY_SIZE(a->s32) - 1;
+#else
+int upper = 0;
+#endif
+
+for (i = 0; i < 32; i++) {
+bcd_get_digit(b, i, &invalid);
+
+if (unlikely(invalid)) {
+return CRF_SO;
+}
+}
+
+if (a->s32[upper] >= 32) {
+ox_flag = true;
+ret.u64[LO_IDX] = ret.u64[HI_IDX] = 0;
+} else if (a->s32[upper] <= -32) {
+ret.u64[LO_IDX] = ret.u64[HI_IDX] = 0;
+} else if (a->s32[upper] > 0) {
+i = a->s32[upper] & 31;
+ulshift(&ret.u64[LO_IDX], &ret.u64[HI_IDX], i * 4, &ox_flag);
+} else {
+i = (-a->s32[upper]) & 31;
+urshift(&ret.u64[LO_IDX], &ret.u64[HI_IDX], i * 4);
+}
+
+cr = bcd_cmp_zero(r);
+if (unlikely(ox_flag)) {
+cr |= CRF_SO;
+}
+
+return cr;
+}
+
 void helper_vsbox(ppc_avr_t *r, ppc_avr_t *a)
 {
 int i;
diff --git a/target-ppc/translate/vmx-impl.inc.c 
b/target-ppc/translate/vmx-impl.inc.c
index 84ebb7e..fc54881 100644
--- a/target-ppc/translate/vmx-impl.inc.c
+++ b/target-ppc/translate/vmx-impl.inc.c
@@ -1017,6 +1017,7 @@ GEN_BCD2(bcdctsq)
 GEN_BCD2(bcdsetsgn)
 GEN_BCD(bcdcpsgn);
 GEN_BCD(bcds);
+GEN_BCD(bcdus);
 
 static void gen_xpnd04_1(DisasContext *ctx)
 {
@@ -1093,6 +1094,8 @@ GEN_VXFORM_DUAL(vaddshs, PPC_ALTIVEC, PPC_NONE, \
 bcdcpsgn, PPC_NONE, PPC2_ISA300)
 GEN_VXFORM_DUAL(vsubudm, PPC2_ALTIVEC_207, PPC_NONE, \
 bcds, PPC_NONE, PPC2_ISA300)
+GEN_VXFORM_DUAL(vsubuwm, PPC_ALTIVEC, PPC_NONE, \
+bcdus, PPC_NONE, PPC2_ISA300)
 
 static void gen_vsbox(DisasContext *ctx)
 {
diff --git a/target-ppc/translate/vmx-ops.inc.c 
b/target-ppc/translate/vmx-ops.inc.c
index 7b4b009..cdd3abe 100644
--- a/target-ppc/translate/vmx-ops.inc.c
+++ b/target-ppc/translate/vmx-ops.inc.c
@@ -61,7 +61,7 @@ GEN_VXFORM(vadduwm, 0, 2),
 GEN_VXFORM_207(vaddudm, 0, 3),
 GEN_VXFORM_DUAL(vsububm, bcdadd, 0, 16, PPC_ALTIVEC, PPC_NONE),
 GEN_VXFORM_DUAL(vsubuhm, bcdsub, 0, 17, PPC_ALTIVEC, PPC_NONE),
-GEN_VXFORM(vsubuwm, 0, 18),
+GEN_VXFORM_DUAL(vsubuwm, bcdus, 0, 18, PPC_ALTIVEC, PPC2_ISA300),
 GEN_VXFORM_DUAL(vsubudm, bcds, 0, 19, PPC2_ALTIVEC_207, PPC2_ISA300),
 GEN_VXFORM_300(bcds, 0, 27),
 GEN_VXFORM(vmaxub, 1, 0),
-- 
2.7.4

Re: [Qemu-devel] [Qemu-ppc] [RFC PATCH qemu] spapr_pci: Create PCI-express root bus by default

[Alexey Kardashevskiy]

On 03/12/16 08:41, Benjamin Herrenschmidt wrote:
> On Fri, 2016-12-02 at 16:50 +1100, David Gibson wrote:
>>
>> Uh.. I don't entirely follow you.  From the host point of view there
>> are multiple iommu groups (PEs), but from the guest point of view
>> there's only one.  On the guest side iommu granularity is always
>> per-vPHB.
> 
> Ok so the H_PUT_TCE calls affect all the underlying tables ?

Yes, it updates all hardware tables attached to the same guest's LIOBN
(which in reality can only be seen on power7 as power8 shares tables
between PEs).


-- 
Alexey

[Qemu-devel] New wiki page: http://wiki.qemu.org/Hosts

[Programmingkid]

I thought we could use a Hosts page to sort all the host documentation we have. 
It is located here: http://wiki.qemu.org/Hosts

Here is what I have so far:

AIX
Darwin
FreeBSD, NetBSD, OpenBSD
Linux
Mac OS X
Solaris
Windows

Re: [Qemu-devel] ARM documentation

[Programmingkid]

On Nov 30, 2016, at 9:55 PM, G 3 wrote:

> 
> On Nov 30, 2016, at 6:58 PM, Alistair Francis wrote:
> 
>> On Fri, Nov 18, 2016 at 4:53 PM, Programmingkid
>>  wrote:
>>> 
>>> On Nov 18, 2016, at 7:31 PM, Programmingkid wrote:
>>> 
 
 On Nov 18, 2016, at 4:10 PM, Alistair Francis wrote:
 
> On Fri, Nov 11, 2016 at 8:53 AM, G 3  wrote:
>> 
>> On Nov 11, 2016, at 11:31 AM, Alex Bennée wrote:
>> 
>>> 
>>> G 3  writes:
>>> 
 On Nov 11, 2016, at 8:27 AM, G 3 wrote:
 
> 
> On Nov 11, 2016, at 8:08 AM, Stefan Hajnoczi wrote:
> 
>> On Fri, Nov 11, 2016 at 12:09:31AM -0500, G 3 wrote:
>>> 
>>> I was wondering if there is a list somewhere of all the ARM
>>> boards QEMU
>>> supports. I want to add a section to the ARM wiki page that lists
>>> at least a
>>> few of them.
>> 
>> 
>> Are you looking for something more detailed than the following?
>> 
>> $ arm-softmmu/qemu-system-arm -M \?
>> Supported machines are:
>> akitaSharp SL-C1000 (Akita) PDA (PXA270)
>> ast2500-evb  Aspeed AST2500 EVB (ARM1176)
>> borzoi   Sharp SL-C3100 (Borzoi) PDA (PXA270)
>> canon-a1100  Canon PowerShot A1100 IS
>> cheetah  Palm Tungsten|E aka. Cheetah PDA (OMAP310)
>> collie   Sharp SL-5500 (Collie) PDA (SA-1110)
>> connex   Gumstix Connex (PXA255)
>> cubieboard   cubietech cubieboard
>> highbank Calxeda Highbank (ECX-1000)
>> imx25-pdkARM i.MX25 PDK board (ARM926)
>> integratorcp ARM Integrator/CP (ARM926EJ-S)
>> kzm  ARM KZM Emulation Baseboard (ARM1136)
>> lm3s6965evb  Stellaris LM3S6965EVB
>> lm3s811evb   Stellaris LM3S811EVB
>> mainstoneMainstone II (PXA27x)
>> midway   Calxeda Midway (ECX-2000)
>> musicpal Marvell 88w8618 / MusicPal (ARM926EJ-S)
>> n800 Nokia N800 tablet aka. RX-34 (OMAP2420)
>> n810 Nokia N810 tablet aka. RX-44 (OMAP2420)
>> netduino2Netduino 2 Machine
>> none empty machine
>> nuri Samsung NURI board (Exynos4210)
>> palmetto-bmc OpenPOWER Palmetto BMC (ARM926EJ-S)
>> raspi2   Raspberry Pi 2
>> realview-eb  ARM RealView Emulation Baseboard (ARM926EJ-S)
>> realview-eb-mpcore   ARM RealView Emulation Baseboard (ARM11MPCore)
>> realview-pb-a8   ARM RealView Platform Baseboard for Cortex-A8
>> realview-pbx-a9  ARM RealView Platform Baseboard Explore for
>> Cortex-A9
>> sabreliteFreescale i.MX6 Quad SABRE Lite Board (Cortex
>> A9)
>> smdkc210 Samsung SMDKC210 board (Exynos4210)
>> spitzSharp SL-C3000 (Spitz) PDA (PXA270)
>> sx1  Siemens SX1 (OMAP310) V2
>> sx1-v1   Siemens SX1 (OMAP310) V1
>> terrier  Sharp SL-C3200 (Terrier) PDA (PXA270)
>> tosa Sharp SL-6000 (Tosa) PDA (PXA255)
>> verdex   Gumstix Verdex (PXA270)
>> versatileab  ARM Versatile/AB (ARM926EJ-S)
>> versatilepb  ARM Versatile/PB (ARM926EJ-S)
>> vexpress-a15 ARM Versatile Express for Cortex-A15
>> vexpress-a9  ARM Versatile Express for Cortex-A9
>> virt-2.6 QEMU 2.6 ARM Virtual Machine
>> virt-2.7 QEMU 2.7 ARM Virtual Machine
>> virt QEMU 2.8 ARM Virtual Machine (alias of virt-2.8)
>> virt-2.8 QEMU 2.8 ARM Virtual Machine
>> xilinx-zynq-a9   Xilinx Zynq Platform Baseboard for Cortex-A9
>> z2   Zipit Z2 (PXA27x)
>> 
>> $ aarch64-softmmu/qemu-system-aarch64 -M \?
>> Supported machines are:
>> akitaSharp SL-C1000 (Akita) PDA (PXA270)
>> ast2500-evb  Aspeed AST2500 EVB (ARM1176)
>> borzoi   Sharp SL-C3100 (Borzoi) PDA (PXA270)
>> canon-a1100  Canon PowerShot A1100 IS
>> cheetah  Palm Tungsten|E aka. Cheetah PDA (OMAP310)
>> collie   Sharp SL-5500 (Collie) PDA (SA-1110)
>> connex   Gumstix Connex (PXA255)
>> cubieboard   cubietech cubieboard
>> highbank Calxeda Highbank (ECX-1000)
>> imx25-pdkARM i.MX25 PDK board (ARM926)
>> integratorcp ARM Integrator/CP (ARM926EJ-S)
>> kzm  ARM KZM Emulation Baseboard (ARM1136)
>> lm3s6965evb  Stellaris LM

Re: [Qemu-devel] [PATCH v2] scripts: add "git.orderfile" for ordering diff hunks by pathname patterns

[John Snow]

On 12/02/2016 04:54 PM, Laszlo Ersek wrote:
> On 12/02/16 22:16, Eduardo Habkost wrote:
>> On Fri, Dec 02, 2016 at 10:01:52PM +0100, Laszlo Ersek wrote:
>> [...]
>>> +docs/*
>>> +*.txt
>>> +configure
>>> +GNUmakefile
>>> +makefile
>>> +Makefile*
>>> +*.mak
>>> +qapi-schema*.json
>>> +qapi/*.json
>>> +include/qapi/visitor.h
>>> +include/qapi/visitor-impl.h
>>> +scripts/qapi.py
>>> +scripts/*.py
>>> +*.h
>>> +qapi/qapi-visit-core.c
>>
>> Maybe we could include test/* here, so test code appears before
>> the implementation?
> 
> Hmmm, I'm not so sure about that. I certainly don't subscribe to TDD,
> when you first write up the test case, and then add implementation to
> satisfy the tests.
> 
> I don't have much experience / history with the tests subdir. Briefly
> reviewing what I've done there (git log --author=lersek), I think I
> would either keep the test patches entirely separate from the QEMU code
> patches (thus there wouldn't be a patch modifying *.c files both under
> and outside of tests/). Or else, I would actually prefer tests/*.c to
> come later in the same patch. The tests are frequently written to
> provide good coverage for the actual QEMU code, so seeing the QEMU code
> first seems preferable. (The corner cases to cover could be completely
> arbitrary, originating from quirks of POSIX functions, and so on.)
> 
> Thanks
> Laszlo
> 
>>> +*.c
>>> -- 
>>> 2.9.2
>>>
>>>
>>
> 

We also don't formally embrace TDD, as any tests that are checked in
_MUST_ pass. The patch order by definition then requires the feature
first, and then the tests.

Personally I like TDD, but I don't think it's appropriate to put tests
first in this instance, as we review with an eye for "Will this break
the tree right now?" and not "What are we about to add to QEMU?"

--js

Re: [Qemu-devel] [qemu patch V3 2/2] kvmclock: reduce kvmclock difference on migration

[Marcelo Tosatti]

On Fri, Dec 02, 2016 at 11:56:09AM -0200, Eduardo Habkost wrote:
> On Thu, Dec 01, 2016 at 12:39:03PM -0200, Marcelo Tosatti wrote:
> > On Wed, Nov 30, 2016 at 11:09:28AM -0200, Eduardo Habkost wrote:
> > > On Tue, Nov 29, 2016 at 09:54:29PM -0200, Marcelo Tosatti wrote:
> > > > On Tue, Nov 29, 2016 at 10:34:05AM -0200, Eduardo Habkost wrote:
> > > > > On Tue, Nov 29, 2016 at 08:56:00AM -0200, Marcelo Tosatti wrote:
> > > > > > On Mon, Nov 28, 2016 at 03:12:01PM -0200, Eduardo Habkost wrote:
> > > > > > > On Mon, Nov 28, 2016 at 02:45:24PM -0200, Marcelo Tosatti wrote:
> > > > > > > > On Mon, Nov 28, 2016 at 12:13:22PM -0200, Eduardo Habkost wrote:
> > > > > > > > > Sorry for not noticing the following issues on the previous
> > > > > > > > > reviews. I was only paying attention to the vmstate and 
> > > > > > > > > machine
> > > > > > > > > code:
> > > > > > > > > 
> > > > > > > > > On Mon, Nov 21, 2016 at 08:50:04AM -0200, Marcelo Tosatti 
> > > > > > > > > wrote:
> > > > > > > > > > Check for KVM_CAP_ADJUST_CLOCK capability 
> > > > > > > > > > KVM_CLOCK_TSC_STABLE, which
> > > > > > > > > > indicates that KVM_GET_CLOCK returns a value as seen by the 
> > > > > > > > > > guest at
> > > > > > > > > > that moment.
> > > > > > > > > > 
> > > > > > > > > > For new machine types, use this value rather than reading 
> > > > > > > > > > from guest memory.
> > > > > > > > > > 
> > > > > > > > > > This reduces kvmclock difference on migration from 5s to 
> > > > > > > > > > 0.1s
> > > > > > > > > > (when max_downtime == 5s).
> > > > > > > > > > 
> > > > > > > > > > Signed-off-by: Marcelo Tosatti 
> > > > > > > > > > 
> > > > > > > > > > ---
> > > > > > > > > >  hw/i386/kvm/clock.c|  107 
> > > > > > > > > > ++---
> > > > > > > > > >  include/hw/i386/pc.h   |5 ++
> > > > > > > > > >  target-i386/kvm.c  |7 +++
> > > > > > > > > >  target-i386/kvm_i386.h |1 
> > > > > > > > > >  4 files changed, 106 insertions(+), 14 deletions(-)
> > > > > > > > > > 
> > > > > > > > > > v2: 
> > > > > > > > > > - improve variable names (Juan)
> > > > > > > > > > - consolidate code on kvm_get_clock function (Paolo)
> > > > > > > > > > - return mach_use_reliable_get_clock from needed function 
> > > > > > > > > > (Paolo)
> > > > > > > > > > v3: 
> > > > > > > > > > - simplify check for src_use_reliable_get_clock (Eduardo)
> > > > > > > > > > - move src_use_reliable_get_clock initialization to realize 
> > > > > > > > > > (Eduardo)
> > > > > > > > > > 
> > > > > > > > > > 
> > > > > > > > > > Index: qemu-mig-advance-clock/hw/i386/kvm/clock.c
> > > > > > > > > > ===
> > > > > > > > > > --- qemu-mig-advance-clock.orig/hw/i386/kvm/clock.c 
> > > > > > > > > > 2016-11-17 15:07:11.220632761 -0200
> > > > > > > > > > +++ qemu-mig-advance-clock/hw/i386/kvm/clock.c  
> > > > > > > > > > 2016-11-17 15:11:51.372048640 -0200
> > > > > > > > > > @@ -36,6 +36,12 @@
> > > > > > > > > >  
> > > > > > > > > >  uint64_t clock;
> > > > > > > > > >  bool clock_valid;
> > > > > > > > > > +
> > > > > > > > > > +/* whether machine supports reliable KVM_GET_CLOCK */
> > > > > > > > > > +bool mach_use_reliable_get_clock;
> > > > > > > > > > +
> > > > > > > > > > +/* whether source host supported reliable 
> > > > > > > > > > KVM_GET_CLOCK */
> > > > > > > > > > +bool src_use_reliable_get_clock;
> > > > > > > > > >  } KVMClockState;
> > > > > > > > > >  
> > > > > > > > > >  struct pvclock_vcpu_time_info {
> > > > > > > > > > @@ -81,6 +87,19 @@
> > > > > > > > > >  return nsec + time.system_time;
> > > > > > > > > >  }
> > > > > > > > > >  
> > > > > > > > > > +static uint64_t kvm_get_clock(void)
> > > > > > > > > > +{
> > > > > > > > > > +struct kvm_clock_data data;
> > > > > > > > > > +int ret;
> > > > > > > > > > +
> > > > > > > > > > +ret = kvm_vm_ioctl(kvm_state, KVM_GET_CLOCK, &data);
> > > > > > > > > > +if (ret < 0) {
> > > > > > > > > > +fprintf(stderr, "KVM_GET_CLOCK failed: %s\n", 
> > > > > > > > > > strerror(ret));
> > > > > > > > > > +abort();
> > > > > > > > > > +}
> > > > > > > > > > +return data.clock;
> > > > > > > > > > +}
> > > > > > > > > > +
> > > > > > > > > >  static void kvmclock_vm_state_change(void *opaque, int 
> > > > > > > > > > running,
> > > > > > > > > >   RunState state)
> > > > > > > > > >  {
> > > > > > > > > > @@ -91,15 +110,36 @@
> > > > > > > > > 
> > > > > > > > > Can you please use "diff -p" on your patches?
> > > > > > > > > 
> > > > > > > > > >  
> > > > > > > > > >  if (running) {
> > > > > > > > > >  struct kvm_clock_data data = {};
> > > > > > > > > > -uint64_t time_at_migration = 
> > > > > > > > > > kvmclock_current_nsec(s);
> > > > > > > > > > +uint64_t pvclock_via_mem = 0;
> > > > > > > > > >  
> > > > > > > > > > -s->clock_valid = fal

Re: [Qemu-devel] [PATCH v2] scripts: add "git.orderfile" for ordering diff hunks by pathname patterns

[Laszlo Ersek]

On 12/02/16 22:16, Eduardo Habkost wrote:
> On Fri, Dec 02, 2016 at 10:01:52PM +0100, Laszlo Ersek wrote:
> [...]
>> +docs/*
>> +*.txt
>> +configure
>> +GNUmakefile
>> +makefile
>> +Makefile*
>> +*.mak
>> +qapi-schema*.json
>> +qapi/*.json
>> +include/qapi/visitor.h
>> +include/qapi/visitor-impl.h
>> +scripts/qapi.py
>> +scripts/*.py
>> +*.h
>> +qapi/qapi-visit-core.c
> 
> Maybe we could include test/* here, so test code appears before
> the implementation?

Hmmm, I'm not so sure about that. I certainly don't subscribe to TDD,
when you first write up the test case, and then add implementation to
satisfy the tests.

I don't have much experience / history with the tests subdir. Briefly
reviewing what I've done there (git log --author=lersek), I think I
would either keep the test patches entirely separate from the QEMU code
patches (thus there wouldn't be a patch modifying *.c files both under
and outside of tests/). Or else, I would actually prefer tests/*.c to
come later in the same patch. The tests are frequently written to
provide good coverage for the actual QEMU code, so seeing the QEMU code
first seems preferable. (The corner cases to cover could be completely
arbitrary, originating from quirks of POSIX functions, and so on.)

Thanks
Laszlo

>> +*.c
>> -- 
>> 2.9.2
>>
>>
> 

Re: [Qemu-devel] [Qemu-ppc] [RFC PATCH qemu] spapr_pci: Create PCI-express root bus by default

[Benjamin Herrenschmidt]

On Fri, 2016-12-02 at 16:50 +1100, David Gibson wrote:
> 
> Uh.. I don't entirely follow you.  From the host point of view there
> are multiple iommu groups (PEs), but from the guest point of view
> there's only one.  On the guest side iommu granularity is always
> per-vPHB.

Ok so the H_PUT_TCE calls affect all the underlying tables ?

Cheers,
Ben.

[Qemu-devel] [PATCH for-2.9 16/17] tests: query-cpu-model-test.py test code

[Eduardo Habkost]

This python script will run query-cpu-model-expansion and related
commands, and sanity-check the results. It seems to work with
qemu-system-s390x, already, but I don't have a host where I can
make sure the test case work for all CPU models.

Signed-off-by: Eduardo Habkost 
---
 tests/query-cpu-model-test.py | 421 ++
 1 file changed, 421 insertions(+)
 create mode 100755 tests/query-cpu-model-test.py

diff --git a/tests/query-cpu-model-test.py b/tests/query-cpu-model-test.py
new file mode 100755
index 000..0d99242
--- /dev/null
+++ b/tests/query-cpu-model-test.py
@@ -0,0 +1,421 @@
+#!/usr/bin/env python
+#
+# query-cpu-model-* sanity checks
+#
+#  Copyright (c) 2016 Red Hat Inc
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library; if not, see .
+#
+
+import sys, os
+sys.path.append(os.path.join(os.path.dirname(__file__), '..', 'scripts'))
+from qtest import QEMUQtestMachine
+import unittest
+import logging
+
+UNSAFE_FEATURES = {
+'x86_64': ['pmu', 'host-cache-info'],
+'i386': ['pmu', 'host-cache-info'],
+}
+
+# Validation of the expanded CPU models will be based on the QOM
+# properties of CPU objects.
+# QOM properties that don't affect guest ABI can be safely ignored
+# when validating the results.
+IGNORE_QOM_PROPS = {
+# the 'type' property just identifies the QOM class being used
+# to build the CPU, and shouldn't affect the guest ABI
+'x86_64': ['type'],
+'i386': ['type'],
+# 'static', 'migration-safe', and 'description' are just
+# information for the user, and don't affect guest ABI
+'s390x': ['type', 'static', 'migration-safe', 'description'],
+}
+
+def toQemuOpts(*args):
+"""Convert arguments to a QemuOpts string, with appropriate escaping
+
+Each argument can be a single string, or a dictionary.
+"""
+logging.debug('toQemuOpts(%r)', args)
+r = []
+for a in args:
+if type(a) is dict:
+for k,v in a.items():
+if type(v) is bool:
+if v:
+v = 'on'
+else:
+v = 'off'
+v = str(v)
+a = '%s=%s' % (k, v)
+r.append(a)
+else:
+a = str(a)
+r.append(a)
+return ','.join([o.replace(',', ',,') for o in r])
+
+def cpuPath(vm, cpu_index):
+"""Return qom_path for a given CPU, using query-cpus"""
+cpus = vm.command('query-cpus')
+return cpus[cpu_index]['qom_path']
+
+def allProps(vm, path):
+"""Return a dictionary containing all properties for a QOM object"""
+props = vm.command('qom-list', path=path)
+r = {}
+for prop in props:
+pname = prop['name']
+v = vm.command('qom-get', path=path, property=pname)
+r[pname] = v
+return r
+
+def allCpuProps(vm, cpu_index):
+"""Return all properties for a given CPU"""
+return allProps(vm, cpuPath(vm, cpu_index))
+
+
+class CPUModelTest(unittest.TestCase):
+longMessage = True
+maxDiff = None
+
+def runAndGetProps(self, model):
+# Helper to run QEMU using a CpuModelInfo struct and get
+# all CPU properties
+cpu_opt = toQemuOpts(model['name'], model.get('props', {}))
+logging.debug('cpu option: %s', cpu_opt)
+
+vm = QEMUQtestMachine(args=['-machine', 'accel=%s' % (self.accel), 
'-S',
+'-cpu', cpu_opt], name='qom-fetch',
+  logging=False)
+try:
+vm.launch()
+props = allCpuProps(vm, 0)
+finally:
+vm.shutdown()
+
+# remove the properties we can ignore
+for p in IGNORE_QOM_PROPS.get(self.target['arch'], []):
+del props[p]
+
+return props
+
+def tryGetProps(self, model, msg):
+"""Try to get QOM props for model, if runnable"""
+if model.get('runnable') != False:
+logging.info("%s: maybe runnable, fetching QOM properties", msg)
+try:
+model['qom-props'] = self.runAndGetProps(model['model'])
+except:
+if model.get('runnable'):
+# explicitly marked as runnable, raise exception
+raise
+logging.info("%s: failed to run VM, ignoring", msg)
+
+def checkOneExpansion(self, model, type, msg):
+   

[Qemu-devel] [PATCH for-2.9 11/17] target-i386: Remove AMD feature flag aliases from Opteron models

[Eduardo Habkost]

When CPU vendor is set to AMD, the AMD feature alias bits on
CPUID[0x8001].EDX are already automatically copied from CPUID[1].EDX
on x86_cpu_realizefn(). When CPU vendor is Intel, those bits are
reserved and should be zero. On either case, those bits shouldn't be set
in the CPU model table.

Commit 726a8ff68677d8d5fba17eb0ffb85076bfb598dc removed those
bits from most CPU models, but the Opteron_* entries still have
them. Remove the alias bits from Opteron_* too.

Add an assert() to x86_register_cpudef_type() to ensure we don't
make the same mistake again.

Signed-off-by: Eduardo Habkost 
---
 target-i386/cpu.c | 46 --
 1 file changed, 12 insertions(+), 34 deletions(-)

diff --git a/target-i386/cpu.c b/target-i386/cpu.c
index a584c3e..76a6a30 100644
--- a/target-i386/cpu.c
+++ b/target-i386/cpu.c
@@ -1338,12 +1338,7 @@ static X86CPUDefinition builtin_x86_defs[] = {
 .features[FEAT_1_ECX] =
 CPUID_EXT_SSE3,
 .features[FEAT_8000_0001_EDX] =
-CPUID_EXT2_LM | CPUID_EXT2_FXSR | CPUID_EXT2_MMX |
-CPUID_EXT2_NX | CPUID_EXT2_PSE36 | CPUID_EXT2_PAT |
-CPUID_EXT2_CMOV | CPUID_EXT2_MCA | CPUID_EXT2_PGE |
-CPUID_EXT2_MTRR | CPUID_EXT2_SYSCALL | CPUID_EXT2_APIC |
-CPUID_EXT2_CX8 | CPUID_EXT2_MCE | CPUID_EXT2_PAE | CPUID_EXT2_MSR |
-CPUID_EXT2_TSC | CPUID_EXT2_PSE | CPUID_EXT2_DE | CPUID_EXT2_FPU,
+CPUID_EXT2_LM | CPUID_EXT2_NX | CPUID_EXT2_SYSCALL,
 .xlevel = 0x8008,
 .model_id = "AMD Opteron 240 (Gen 1 Class Opteron)",
 },
@@ -1364,13 +1359,7 @@ static X86CPUDefinition builtin_x86_defs[] = {
 CPUID_EXT_CX16 | CPUID_EXT_SSE3,
 /* Missing: CPUID_EXT2_RDTSCP */
 .features[FEAT_8000_0001_EDX] =
-CPUID_EXT2_LM | CPUID_EXT2_FXSR |
-CPUID_EXT2_MMX | CPUID_EXT2_NX | CPUID_EXT2_PSE36 |
-CPUID_EXT2_PAT | CPUID_EXT2_CMOV | CPUID_EXT2_MCA |
-CPUID_EXT2_PGE | CPUID_EXT2_MTRR | CPUID_EXT2_SYSCALL |
-CPUID_EXT2_APIC | CPUID_EXT2_CX8 | CPUID_EXT2_MCE |
-CPUID_EXT2_PAE | CPUID_EXT2_MSR | CPUID_EXT2_TSC | CPUID_EXT2_PSE |
-CPUID_EXT2_DE | CPUID_EXT2_FPU,
+CPUID_EXT2_LM | CPUID_EXT2_NX | CPUID_EXT2_SYSCALL,
 .features[FEAT_8000_0001_ECX] =
 CPUID_EXT3_SVM | CPUID_EXT3_LAHF_LM,
 .xlevel = 0x8008,
@@ -1394,13 +1383,7 @@ static X86CPUDefinition builtin_x86_defs[] = {
 CPUID_EXT_SSE3,
 /* Missing: CPUID_EXT2_RDTSCP */
 .features[FEAT_8000_0001_EDX] =
-CPUID_EXT2_LM | CPUID_EXT2_FXSR |
-CPUID_EXT2_MMX | CPUID_EXT2_NX | CPUID_EXT2_PSE36 |
-CPUID_EXT2_PAT | CPUID_EXT2_CMOV | CPUID_EXT2_MCA |
-CPUID_EXT2_PGE | CPUID_EXT2_MTRR | CPUID_EXT2_SYSCALL |
-CPUID_EXT2_APIC | CPUID_EXT2_CX8 | CPUID_EXT2_MCE |
-CPUID_EXT2_PAE | CPUID_EXT2_MSR | CPUID_EXT2_TSC | CPUID_EXT2_PSE |
-CPUID_EXT2_DE | CPUID_EXT2_FPU,
+CPUID_EXT2_LM | CPUID_EXT2_NX | CPUID_EXT2_SYSCALL,
 .features[FEAT_8000_0001_ECX] =
 CPUID_EXT3_MISALIGNSSE | CPUID_EXT3_SSE4A |
 CPUID_EXT3_ABM | CPUID_EXT3_SVM | CPUID_EXT3_LAHF_LM,
@@ -1427,13 +1410,8 @@ static X86CPUDefinition builtin_x86_defs[] = {
 CPUID_EXT_SSE3,
 /* Missing: CPUID_EXT2_RDTSCP */
 .features[FEAT_8000_0001_EDX] =
-CPUID_EXT2_LM |
-CPUID_EXT2_PDPE1GB | CPUID_EXT2_FXSR | CPUID_EXT2_MMX |
-CPUID_EXT2_NX | CPUID_EXT2_PSE36 | CPUID_EXT2_PAT |
-CPUID_EXT2_CMOV | CPUID_EXT2_MCA | CPUID_EXT2_PGE |
-CPUID_EXT2_MTRR | CPUID_EXT2_SYSCALL | CPUID_EXT2_APIC |
-CPUID_EXT2_CX8 | CPUID_EXT2_MCE | CPUID_EXT2_PAE | CPUID_EXT2_MSR |
-CPUID_EXT2_TSC | CPUID_EXT2_PSE | CPUID_EXT2_DE | CPUID_EXT2_FPU,
+CPUID_EXT2_LM | CPUID_EXT2_PDPE1GB | CPUID_EXT2_NX |
+CPUID_EXT2_SYSCALL,
 .features[FEAT_8000_0001_ECX] =
 CPUID_EXT3_FMA4 | CPUID_EXT3_XOP |
 CPUID_EXT3_3DNOWPREFETCH | CPUID_EXT3_MISALIGNSSE |
@@ -1463,13 +1441,8 @@ static X86CPUDefinition builtin_x86_defs[] = {
 CPUID_EXT_SSSE3 | CPUID_EXT_PCLMULQDQ | CPUID_EXT_SSE3,
 /* Missing: CPUID_EXT2_RDTSCP */
 .features[FEAT_8000_0001_EDX] =
-CPUID_EXT2_LM |
-CPUID_EXT2_PDPE1GB | CPUID_EXT2_FXSR | CPUID_EXT2_MMX |
-CPUID_EXT2_NX | CPUID_EXT2_PSE36 | CPUID_EXT2_PAT |
-CPUID_EXT2_CMOV | CPUID_EXT2_MCA | CPUID_EXT2_PGE |
-CPUID_EXT2_MTRR | CPUID_EXT2_SYSCALL | CPUID_EXT2_APIC |
-CPUID_EXT2_CX8 | CPUID_EXT2_MCE | CPUID_EXT2_PAE | CPUID_EXT2_MSR |
-CPUID_EXT2_TSC | CPUID_EXT2_PSE | CPUID_EXT2_DE | CPUID_EXT2_FPU,
+CPUID_EXT2_LM | CPUID_EXT2_PDPE1GB | CPUID_EXT2_NX |
+CPUID_EXT2_SYSCALL,
 .features[FEAT_8000_0001_ECX

[Qemu-devel] [PATCH for-2.9 15/17] target-i386: Define static "base" CPU model

[Eduardo Habkost]

The query-cpu-model-expand QMP command needs at least one static
model, to allow the "static" expansion mode to be implemented.
Instead of defining static versions of every CPU model, define a
"base" CPU model that has absolutely no feature flag enabled.

Despite having no CPUID data set at all, "-cpu base" is even a
functional CPU:

* It can boot a Slackware Linux 1.01 image with a Linux 0.99.12
  kernel[1].
* It is even possible to boot[2] a modern Fedora x86_64 guest by
  manually enabling the following CPU features:
  -cpu base,+lm,+msr,+pae,+fpu,+cx8,+cmov,+sse,+sse2,+fxsr

[1] http://www.qemu-advent-calendar.org/2014/#day-1
[2] This is what can be seen in the guest:
[root@localhost ~]# cat /proc/cpuinfo
processor   : 0
vendor_id   : unknown
cpu family  : 0
model   : 0
model name  : 00/00
stepping: 0
physical id : 0
siblings: 1
core id : 0
cpu cores   : 1
apicid  : 0
initial apicid  : 0
fpu : yes
fpu_exception   : yes
cpuid level : 1
wp  : yes
flags   : fpu msr pae cx8 cmov fxsr sse sse2 lm nopl
bugs:
bogomips: 5832.70
clflush size: 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

[root@localhost ~]# x86info -v -a
x86info v1.30.  Dave Jones 2001-2011
Feedback to .

No TSC, MHz calculation cannot be performed.
Unknown vendor (0)
MP Table:

Family: 0 Model: 0 Stepping: 0
CPU Model (x86info's best guess):

eax in: 0x, eax = 0001 ebx =  ecx =  edx = 

eax in: 0x0001, eax =  ebx = 0800 ecx =  edx = 
07008161

eax in: 0x8000, eax = 8001 ebx =  ecx =  edx = 

eax in: 0x8001, eax =  ebx =  ecx =  edx = 
2000

Feature flags:
 fpuOnboard FPU
 msrModel-Specific Registers
 paePhysical Address Extensions
 cx8CMPXCHG8 instruction
 cmov   CMOV instruction
 fxsr   FXSAVE and FXRSTOR instructions
 sseSSE support
 sse2   SSE2 support

Long NOPs supported: yes

Address sizes : 0 bits physical, 0 bits virtual
0MHz processor (estimate).

 running at an estimated 0MHz
[root@localhost ~]#

Signed-off-by: Eduardo Habkost 
---
 target-i386/cpu-qom.h |  2 ++
 target-i386/cpu.c | 24 +++-
 2 files changed, 25 insertions(+), 1 deletion(-)

diff --git a/target-i386/cpu-qom.h b/target-i386/cpu-qom.h
index 7561891..279f327 100644
--- a/target-i386/cpu-qom.h
+++ b/target-i386/cpu-qom.h
@@ -49,6 +49,7 @@ typedef struct X86CPUDefinition X86CPUDefinition;
  * @cpu_def: CPU model definition
  * @ordering: Ordering on the "-cpu help" CPU model list.
  * @migration_safe: See CpuDefinitionInfo::migration_safe
+ * @static_model: See CpuDefinitionInfo::static
  * @parent_realize: The parent class' realize handler.
  * @parent_reset: The parent class' reset handler.
  *
@@ -64,6 +65,7 @@ typedef struct X86CPUClass {
 
 int ordering;
 bool migration_safe;
+bool static_model;
 
 /* Optional description of CPU model.
  * If unavailable, cpu_def->model_id is used */
diff --git a/target-i386/cpu.c b/target-i386/cpu.c
index 3539419..bf4ac09 100644
--- a/target-i386/cpu.c
+++ b/target-i386/cpu.c
@@ -2200,6 +2200,7 @@ static void x86_cpu_definition_entry(gpointer data, 
gpointer user_data)
 info->q_typename = g_strdup(object_class_get_name(oc));
 info->migration_safe = cc->migration_safe;
 info->has_migration_safe = true;
+info->q_static = cc->static_model;
 
 entry = g_malloc0(sizeof(*entry));
 entry->value = info;
@@ -3592,7 +3593,9 @@ static void x86_cpu_initfn(Object *obj)
 object_property_add_alias(obj, "sse4_1", obj, "sse4.1", &error_abort);
 object_property_add_alias(obj, "sse4_2", obj, "sse4.2", &error_abort);
 
-x86_cpu_load_def(cpu, xcc->cpu_def, &error_abort);
+if (xcc->cpu_def) {
+x86_cpu_load_def(cpu, xcc->cpu_def, &error_abort);
+}
 }
 
 static int64_t x86_cpu_get_arch_id(CPUState *cs)
@@ -3747,6 +3750,24 @@ static const TypeInfo x86_cpu_type_info = {
 .class_init = x86_cpu_common_class_init,
 };
 
+
+/* "base" CPU model, used by query-cpu-model-expansion */
+static void x86_cpu_base_class_init(ObjectClass *oc, void *data)
+{
+X86CPUClass *xcc = X86_CPU_CLASS(oc);
+
+xcc->static_model = true;
+xcc->migration_safe = true;
+xcc->model_description = "base CPU model type with no feature enabled";
+xcc->ordering = 8;
+}
+
+static const TypeInfo x86_base_cpu_type_info = {
+.name = X86_CPU_TYPE_NAME("base"),
+.parent = TYPE_X86_CPU,
+.class_init = x86_cpu_base_class_init,
+};
+
 static void x86_cpu_register_types(void)
 {
 int i;
@@

[Qemu-devel] [PATCH for-2.9 09/17] target-i386: Move "host" properties to base class

[Eduardo Habkost]

Make the "pmu" and "host-cache-info" properties configurable on
all CPU model classes. This way, query-cpu-model-expansion will
be able to return the value of those properties when returning
expansion data using the "base" CPU model as base.

Signed-off-by: Eduardo Habkost 
---
 target-i386/cpu.c | 9 ++---
 1 file changed, 2 insertions(+), 7 deletions(-)

diff --git a/target-i386/cpu.c b/target-i386/cpu.c
index 691ec5e..98e1063 100644
--- a/target-i386/cpu.c
+++ b/target-i386/cpu.c
@@ -1559,12 +1559,6 @@ static int cpu_x86_fill_model_id(char *str)
 
 static X86CPUDefinition host_cpudef;
 
-static Property host_x86_cpu_properties[] = {
-DEFINE_PROP_BOOL("migratable", X86CPU, migratable, true),
-DEFINE_PROP_BOOL("host-cache-info", X86CPU, cache_info_passthrough, false),
-DEFINE_PROP_END_OF_LIST()
-};
-
 /* class_init for the "host" CPU model
  *
  * This function may be called before KVM is initialized.
@@ -1595,7 +1589,6 @@ static void host_x86_cpu_class_init(ObjectClass *oc, void 
*data)
  * instance_init, because they require KVM to be initialized.
  */
 
-dc->props = host_x86_cpu_properties;
 /* Reason: host_x86_cpu_initfn() dies when !kvm_enabled() */
 dc->cannot_destroy_with_object_finalize_yet = true;
 }
@@ -3700,6 +3693,8 @@ static Property x86_cpu_properties[] = {
 DEFINE_PROP_BOOL("cpuid-0xb", X86CPU, enable_cpuid_0xb, true),
 DEFINE_PROP_BOOL("lmce", X86CPU, enable_lmce, false),
 DEFINE_PROP_BOOL("l3-cache", X86CPU, enable_l3_cache, true),
+DEFINE_PROP_BOOL("host-cache-info", X86CPU, cache_info_passthrough, false),
+DEFINE_PROP_BOOL("migratable", X86CPU, migratable, true),
 DEFINE_PROP_END_OF_LIST()
 };
 
-- 
2.7.4

[Qemu-devel] [PATCH for-2.9 17/17] target-i386: Implement query-cpu-model-expansion QMP command

[Eduardo Habkost]

Implement query-cpu-model-expansion for target-i386.

The code needs to be careful to handle non-migration-safe
features ("pmu" and "host-cache-info") according to the expansion
type.

Cc: libvir-l...@redhat.com
Cc: Jiri Denemark 
Signed-off-by: Eduardo Habkost 
---
 tests/Makefile.include |   3 +
 monitor.c  |   4 +-
 target-i386/cpu.c  | 195 -
 3 files changed, 200 insertions(+), 2 deletions(-)

diff --git a/tests/Makefile.include b/tests/Makefile.include
index 63c4347..c7bbfca 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -251,6 +251,9 @@ check-qtest-x86_64-y += $(check-qtest-i386-y)
 gcov-files-i386-y += i386-softmmu/hw/timer/mc146818rtc.c
 gcov-files-x86_64-y = $(subst 
i386-softmmu/,x86_64-softmmu/,$(gcov-files-i386-y))
 
+check-simpleqtest-x86_64-y += $(SRC_PATH)/tests/query-cpu-model-test.py
+check-simpleqtest-i386-y += $(SRC_PATH)/tests/query-cpu-model-test.py
+
 check-qtest-alpha-y = tests/boot-serial-test$(EXESUF)
 
 check-qtest-mips-y = tests/endianness-test$(EXESUF)
diff --git a/monitor.c b/monitor.c
index 0841d43..90c12b3 100644
--- a/monitor.c
+++ b/monitor.c
@@ -983,8 +983,10 @@ static void qmp_unregister_commands_hack(void)
 #ifndef TARGET_ARM
 qmp_unregister_command("query-gic-capabilities");
 #endif
-#if !defined(TARGET_S390X)
+#if !defined(TARGET_S390X) && !defined(TARGET_I386)
 qmp_unregister_command("query-cpu-model-expansion");
+#endif
+#if !defined(TARGET_S390X)
 qmp_unregister_command("query-cpu-model-baseline");
 qmp_unregister_command("query-cpu-model-comparison");
 #endif
diff --git a/target-i386/cpu.c b/target-i386/cpu.c
index bf4ac09..198014a 100644
--- a/target-i386/cpu.c
+++ b/target-i386/cpu.c
@@ -29,10 +29,14 @@
 #include "qemu/option.h"
 #include "qemu/config-file.h"
 #include "qapi/qmp/qerror.h"
+#include "qapi/qmp/qstring.h"
+#include "qapi/qmp/qdict.h"
+#include "qapi/qmp/qbool.h"
 
 #include "qapi-types.h"
 #include "qapi-visit.h"
 #include "qapi/visitor.h"
+#include "qom/qom-qobject.h"
 #include "sysemu/arch_init.h"
 
 #if defined(CONFIG_KVM)
@@ -2259,7 +2263,7 @@ static void x86_cpu_apply_props(X86CPU *cpu, PropValue 
*props)
 }
 }
 
-/* Load data from X86CPUDefinition
+/* Load data from X86CPUDefinition into a X86CPU object
  */
 static void x86_cpu_load_def(X86CPU *cpu, X86CPUDefinition *def, Error **errp)
 {
@@ -2268,6 +2272,11 @@ static void x86_cpu_load_def(X86CPU *cpu, 
X86CPUDefinition *def, Error **errp)
 char host_vendor[CPUID_VENDOR_SZ + 1];
 FeatureWord w;
 
+/*NOTE: any property set by this function should be returned by
+ * x86_cpu_to_dict(), so CPU model data returned by
+ * query-cpu-model-expansion is always complete.
+ */
+
 /* CPU models only set _minimum_ values for level/xlevel: */
 object_property_set_int(OBJECT(cpu), def->level, "min-level", errp);
 object_property_set_int(OBJECT(cpu), def->xlevel, "min-xlevel", errp);
@@ -2312,6 +2321,190 @@ static void x86_cpu_load_def(X86CPU *cpu, 
X86CPUDefinition *def, Error **errp)
 
 }
 
+/* Convert CPU model data from X86CPU object to a property dictionary
+ * that can recreate exactly the same CPU model.
+ *
+ * This function does the opposite of x86_cpu_load_def(). Any
+ * property changed by x86_cpu_load_def() or instance_init
+ * methods should be returned by this function too.
+ */
+static void x86_cpu_to_dict(X86CPU *cpu, QDict *props, Error **errp)
+{
+Object *obj = OBJECT(cpu);
+FeatureWord w;
+PropValue *pv;
+
+/* This code could simply iterate over all writeable properties in the
+ * CPU object, and return all of them. But then the aliases properties
+ * would be returned as well. Returning only the known features
+ * is more reliable.
+ */
+qdict_put_obj(props, "min-level",
+  object_property_get_qobject(obj, "min-level", errp));
+qdict_put_obj(props, "min-xlevel",
+  object_property_get_qobject(obj, "min-xlevel", errp));
+
+qdict_put_obj(props, "family",
+  object_property_get_qobject(obj, "family", errp));
+qdict_put_obj(props, "model",
+  object_property_get_qobject(obj, "model", errp));
+qdict_put_obj(props, "stepping",
+  object_property_get_qobject(obj, "stepping", errp));
+qdict_put_obj(props, "model-id",
+  object_property_get_qobject(obj, "model-id", errp));
+
+for (w = 0; w < FEATURE_WORDS; w++) {
+FeatureWordInfo *fi = &feature_word_info[w];
+int bit;
+for (bit = 0; bit < 32; bit++) {
+if (!fi->feat_names[bit]) {
+continue;
+}
+qdict_put_obj(props, fi->feat_names[bit],
+  object_property_get_qobject(obj, fi->feat_names[bit],
+  errp));
+}
+}
+
+for (pv = kvm_default_props; pv->prop; pv++) {
+qdict_put_obj(props, pv->

[Qemu-devel] [PATCH for-2.9 08/17] target-i386: Support "-cpu host" on TCG too

[Eduardo Habkost]

Change the meaning of "-cpu host" to "enable all features
supported by the accelerator in the current host", so that it can
be used to enable all features supported by TCG.

To make sure "host" is still at the end of the list in "-cpu
help", add a "ordering" field that will be used when sorting the
CPU model list.

Cc: Richard Henderson 
Signed-off-by: Eduardo Habkost 
---
 target-i386/cpu-qom.h |  4 ++--
 target-i386/cpu.c | 26 +-
 2 files changed, 7 insertions(+), 23 deletions(-)

diff --git a/target-i386/cpu-qom.h b/target-i386/cpu-qom.h
index 7c9a07a..93c9679 100644
--- a/target-i386/cpu-qom.h
+++ b/target-i386/cpu-qom.h
@@ -47,7 +47,7 @@ typedef struct X86CPUDefinition X86CPUDefinition;
 /**
  * X86CPUClass:
  * @cpu_def: CPU model definition
- * @kvm_required: Whether CPU model requires KVM to be enabled.
+ * @ordering: Ordering on the "-cpu help" CPU model list.
  * @parent_realize: The parent class' realize handler.
  * @parent_reset: The parent class' reset handler.
  *
@@ -61,7 +61,7 @@ typedef struct X86CPUClass {
 /* Should be eventually replaced by subclass-specific property defaults. */
 X86CPUDefinition *cpu_def;
 
-bool kvm_required;
+int ordering;
 
 /* Optional description of CPU model.
  * If unavailable, cpu_def->model_id is used */
diff --git a/target-i386/cpu.c b/target-i386/cpu.c
index 1a276db..691ec5e 100644
--- a/target-i386/cpu.c
+++ b/target-i386/cpu.c
@@ -1575,7 +1575,7 @@ static void host_x86_cpu_class_init(ObjectClass *oc, void 
*data)
 X86CPUClass *xcc = X86_CPU_CLASS(oc);
 uint32_t eax = 0, ebx = 0, ecx = 0, edx = 0;
 
-xcc->kvm_required = true;
+xcc->ordering = 9; /* Show it last on "-cpu help" */
 
 host_cpuid(0x0, 0, &eax, &ebx, &ecx, &edx);
 x86_cpu_vendor_words2str(host_cpudef.vendor, ebx, edx, ecx);
@@ -1589,8 +1589,7 @@ static void host_x86_cpu_class_init(ObjectClass *oc, void 
*data)
 
 xcc->cpu_def = &host_cpudef;
 xcc->model_description =
-"KVM processor with all supported host features "
-"(only available in KVM mode)";
+"Enables all features supported by the accelerator in the current 
host";
 
 /* level, xlevel, xlevel2, and the feature words are initialized on
  * instance_init, because they require KVM to be initialized.
@@ -2101,13 +2100,6 @@ static void 
x86_cpu_class_check_missing_features(X86CPUClass *xcc,
 Error *err = NULL;
 strList **next = missing_feats;
 
-if (xcc->kvm_required && !kvm_enabled()) {
-strList *new = g_new0(strList, 1);
-new->value = g_strdup("kvm");;
-*missing_feats = new;
-return;
-}
-
 xc = X86_CPU(object_new(object_class_get_name(OBJECT_CLASS(xcc;
 
 x86_cpu_expand_features(xc, &err);
@@ -2155,7 +2147,7 @@ static void listflags(FILE *f, fprintf_function print, 
const char **featureset)
 }
 }
 
-/* Sort alphabetically by type name, listing kvm_required models last. */
+/* Sort alphabetically by type name, respecting X86CPUClass::ordering. */
 static gint x86_cpu_list_compare(gconstpointer a, gconstpointer b)
 {
 ObjectClass *class_a = (ObjectClass *)a;
@@ -2164,9 +2156,8 @@ static gint x86_cpu_list_compare(gconstpointer a, 
gconstpointer b)
 X86CPUClass *cc_b = X86_CPU_CLASS(class_b);
 const char *name_a, *name_b;
 
-if (cc_a->kvm_required != cc_b->kvm_required) {
-/* kvm_required items go last */
-return cc_a->kvm_required ? 1 : -1;
+if (cc_a->ordering != cc_b->ordering) {
+return cc_a->ordering - cc_b->ordering;
 } else {
 name_a = object_class_get_name(class_a);
 name_b = object_class_get_name(class_b);
@@ -3247,13 +3238,6 @@ static void x86_cpu_realizefn(DeviceState *dev, Error 
**errp)
 Error *local_err = NULL;
 static bool ht_warned;
 
-if (xcc->kvm_required && !kvm_enabled()) {
-char *name = x86_cpu_class_get_model_name(xcc);
-error_setg(&local_err, "CPU model '%s' requires KVM", name);
-g_free(name);
-goto out;
-}
-
 if (cpu->apic_id == UNASSIGNED_APIC_ID) {
 error_setg(errp, "apic-id property was not initialized properly");
 return;
-- 
2.7.4

[Qemu-devel] [PATCH for-2.9 14/17] qapi: add static/migration-safe info to query-cpu-model-expansion

[Eduardo Habkost]

On x86, "-cpu host" enables some features that can't be
represented by a static CPU model definition: cache info
passthrough ("host-cache-info") and PMU passthrough ("pmu"). This
means a type=static expansion of "host" can't include those
features.

A type=full expansion of "host", on the other hand, can include
those features, but then the returned data won't be a static CPU
model representation.

Add a note to the documentation explaining that when using CPU
models that include non-migration-safe features, users need to
choose being precision and safety: a precise expansion of the CPU
model (full) won't be safe (static), (because they would include
pmu=on and host-cache-info=on), and a safe (static) expansion of
the CPU model won't be precise.

Architectures where CPU model expansion is always migration-safe
(e.g. s390x) can simply do what they already do, and set
'migration-safe' and 'static' to true.

Cc: Cornelia Huck 
Cc: Christian Borntraeger 
Cc: David Hildenbrand 
Cc: libvir-l...@redhat.com
Cc: Jiri Denemark 
Cc: "Jason J. Herne" 
Cc: Markus Armbruster 
Cc: Eric Blake 
Signed-off-by: Eduardo Habkost 
---
 qapi-schema.json  | 25 -
 target-s390x/cpu_models.c |  4 
 2 files changed, 28 insertions(+), 1 deletion(-)

diff --git a/qapi-schema.json b/qapi-schema.json
index 8d113f8..a102534 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -3291,6 +3291,15 @@
 #migration-safe, but allows tooling to get an insight and work with
 #model details.
 #
+# Note: When a non-migration-safe CPU model is expanded in static mode, some
+# features enabled by the CPU model may be omitted, because they can't be
+# implemented by a static CPU model definition (e.g. cache info passthrough and
+# PMU passthrough in x86). If you need an accurate representation of the
+# features enabled by a non-migration-safe CPU model, use @full. If you need a
+# static representation that will keep ABI compatibility even when changing 
QEMU
+# version or machine-type, use @static (but keep in mind that some features may
+# be omitted).
+#
 # Since: 2.8.0
 ##
 { 'enum': 'CpuModelExpansionType',
@@ -3304,10 +3313,24 @@
 #
 # @model: the expanded CpuModelInfo.
 #
+# @migration-safe: the expanded CPU model in @model is a migration-safe
+#  CPU model. See @CpuDefinitionInfo.migration-safe.
+#  If expansion type was @static, this is always true.
+#  (since 2.9)
+#
+# @static: the expanded CPU model in @model is a static CPU model.
+#  See @CpuDefinitionInfo.static. If expansion type was @static,
+#  this is always true.
+#  (since 2.9)
+#
+# query-cpu-model-expansion with static expansion type should always
+# return a static and migration-safe expansion.
+#
 # Since: 2.8.0
 ##
 { 'struct': 'CpuModelExpansionInfo',
-  'data': { 'model': 'CpuModelInfo' } }
+  'data': { 'model': 'CpuModelInfo', 'static': 'bool',
+'migration-safe': 'bool' } }
 
 
 ##
diff --git a/target-s390x/cpu_models.c b/target-s390x/cpu_models.c
index 5b66d33..f934add 100644
--- a/target-s390x/cpu_models.c
+++ b/target-s390x/cpu_models.c
@@ -448,6 +448,10 @@ CpuModelExpansionInfo 
*arch_query_cpu_model_expansion(CpuModelExpansionType type
 /* convert it back to a static representation */
 expansion_info = g_malloc0(sizeof(*expansion_info));
 expansion_info->model = g_malloc0(sizeof(*expansion_info->model));
+
+/* We always expand to a static and migration-safe CpuModelInfo */
+expansion_info->q_static = true;
+expansion_info->migration_safe = true;
 cpu_info_from_model(expansion_info->model, &s390_model, delta_changes);
 return expansion_info;
 }
-- 
2.7.4

[Qemu-devel] [PATCH for-2.9 07/17] target-i386: Reorganize and document CPUID initialization steps

[Eduardo Habkost]

CPU runnability checks and CPU model expansion have slightly
different requirements. Document the steps involved in loading a
CPU model and realizing a CPU, so their requirements and purpose
are clearly defined.

This patch doesn't change any implementation. It just add
comments, rename the x86_cpu_load_features() function for clarity
(so it won't be confused with x86_cpu_load_def()), and move
x86_cpu_filter_features() closer to it.

Signed-off-by: Eduardo Habkost 
---
 target-i386/cpu.c | 103 ++
 1 file changed, 72 insertions(+), 31 deletions(-)

diff --git a/target-i386/cpu.c b/target-i386/cpu.c
index 93ead1a..1a276db 100644
--- a/target-i386/cpu.c
+++ b/target-i386/cpu.c
@@ -2087,7 +2087,7 @@ static void x86_cpu_parse_featurestr(const char 
*typename, char *features,
 }
 }
 
-static void x86_cpu_load_features(X86CPU *cpu, Error **errp);
+static void x86_cpu_expand_features(X86CPU *cpu, Error **errp);
 static int x86_cpu_filter_features(X86CPU *cpu);
 
 /* Check for missing features that may prevent the CPU class from
@@ -2110,9 +2110,9 @@ static void 
x86_cpu_class_check_missing_features(X86CPUClass *xcc,
 
 xc = X86_CPU(object_new(object_class_get_name(OBJECT_CLASS(xcc;
 
-x86_cpu_load_features(xc, &err);
+x86_cpu_expand_features(xc, &err);
 if (err) {
-/* Errors at x86_cpu_load_features should never happen,
+/* Errors at x86_cpu_expand_features should never happen,
  * but in case it does, just report the model as not
  * runnable at all using the "type" property.
  */
@@ -2272,31 +2272,6 @@ static uint32_t 
x86_cpu_get_supported_feature_word(FeatureWord w,
 return r;
 }
 
-/*
- * Filters CPU feature words based on host availability of each feature.
- *
- * Returns: 0 if all flags are supported by the host, non-zero otherwise.
- */
-static int x86_cpu_filter_features(X86CPU *cpu)
-{
-CPUX86State *env = &cpu->env;
-FeatureWord w;
-int rv = 0;
-
-for (w = 0; w < FEATURE_WORDS; w++) {
-uint32_t host_feat =
-x86_cpu_get_supported_feature_word(w, false);
-uint32_t requested_features = env->features[w];
-env->features[w] &= host_feat;
-cpu->filtered_features[w] = requested_features & ~env->features[w];
-if (cpu->filtered_features[w]) {
-rv = 1;
-}
-}
-
-return rv;
-}
-
 static void x86_cpu_report_filtered_features(X86CPU *cpu)
 {
 FeatureWord w;
@@ -3113,8 +3088,48 @@ static void x86_cpu_enable_xsave_components(X86CPU *cpu)
 env->features[FEAT_XSAVE_COMP_HI] = mask >> 32;
 }
 
-/* Load CPUID data based on configured features */
-static void x86_cpu_load_features(X86CPU *cpu, Error **errp)
+/* Steps involved on loading and filtering CPUID data
+ *
+ * When initializing and realizing a CPU object, the steps
+ * involved in setting up CPUID data are:
+ *
+ * 1) Loading CPU model definition (X86CPUDefinition). This is
+ *implemented by x86_cpu_load_def() and should be completely
+ *transparent, as it is done automatically by instance_init.
+ *No code should need to look at X86CPUDefinition structs
+ *outside instance_init.
+ *
+ * 2) CPU expansion. This is done by realize before CPUID
+ *filtering, and will make sure host/accelerator data is
+ *loaded for CPU models that depend on host capabilities
+ *(e.g. "host"). CPU expansion is not supposed to trigger any
+ *errors. Done by x86_cpu_expand_features().
+ *
+ * 3) CPUID filtering. This will initialize extra data related to
+ *CPUID, and will check if the host supports all capabilities
+ *required by the CPU. Runnability of a CPU model is
+ *determined at this step. Done by x86_cpu_filter_features().
+ *
+ * Some operations don't require all steps to be performed.
+ * More precisely:
+ *
+ * - CPU instance creation (instance_init) will run only CPU
+ *   model loading. CPU expansion can't run at instance_init-time
+ *   because host/accelerator data is not yet available.
+ * - CPU realization will perform both CPU model expansion and CPUID
+ *   filtering, and return an error in case one of them fails.
+ * - query-cpu-definitions needs to run all 3 steps. It needs
+ *   to run CPUID filtering, as the 'unavailable-features'
+ *   field is set based on the filtering results.
+ * - The query-cpu-model-expansion QMP command only needs to run
+ *   CPU model loading and CPU expansion. It should not filter
+ *   any CPUID data based on host capabilities.
+ */
+
+/* Expand CPU configuration data, based on configured features
+ * and host/accelerator capabilities when appropriate.
+ */
+static void x86_cpu_expand_features(X86CPU *cpu, Error **errp)
 {
 CPUX86State *env = &cpu->env;
 FeatureWord w;
@@ -3191,6 +3206,32 @@ out:
 }
 }
 
+/*
+ * Finishes initialization of CPUID data, filters CPU feature
+ * words based on host availability of each feature.
+ *
+ * Returns: 0 if all flags are su

[Qemu-devel] [PATCH for-2.9 13/17] cpu: Support comma escaping when parsing -cpu

[Eduardo Habkost]

Currently it's impossible to use commas inside any option value
in -cpu due to the simple way the parser splits the options.

Change both cpu_common_parse_features() and
x86_cpu_parse_featurestr() to use get_opt_*() parsing options,
that can handle handle ",," escaping of commas.

The ideal solution is to use QemuOpts to parse the -cpu option.
But this will require changing the CPUClass::parse_features()
interface, so it will be done later.

Cc: Igor Mammedov 
Signed-off-by: Eduardo Habkost 
---
 tests/test-x86-cpuid-compat.c | 19 +
 qom/cpu.c | 32 +++---
 target-i386/cpu.c | 63 ++-
 3 files changed, 74 insertions(+), 40 deletions(-)

diff --git a/tests/test-x86-cpuid-compat.c b/tests/test-x86-cpuid-compat.c
index 79a2e69..06caa5b 100644
--- a/tests/test-x86-cpuid-compat.c
+++ b/tests/test-x86-cpuid-compat.c
@@ -4,6 +4,7 @@
 #include "qapi/qmp/qdict.h"
 #include "qapi/qmp/qint.h"
 #include "qapi/qmp/qbool.h"
+#include "qapi/qmp/qstring.h"
 #include "libqtest.h"
 
 static char *get_cpu0_qom_path(void)
@@ -52,6 +53,22 @@ typedef struct CpuidTestArgs {
 int64_t expected_value;
 } CpuidTestArgs;
 
+static void test_commas(void)
+{
+char *path;
+QString *value;
+
+qtest_start("-cpu 'qemu64,fpu=on,model-id=A CPU with commas,,fpu=off'");
+path = get_cpu0_qom_path();
+value = qobject_to_qstring(qom_get(path, "model-id"));
+g_assert_true(qom_get_bool(path, "fpu"));
+g_assert_cmpstr(qstring_get_str(value), ==, "A CPU with commas,fpu=off");
+qtest_end();
+
+QDECREF(value);
+g_free(path);
+}
+
 static void test_cpuid_prop(const void *data)
 {
 const CpuidTestArgs *args = data;
@@ -132,6 +149,8 @@ int main(int argc, char **argv)
 g_test_add_func("/x86/cpuid/parsing-plus-minus", test_plus_minus);
 #endif
 
+g_test_add_func("/x86/parsing/commas", test_commas);
+
 /* Original level values for CPU models: */
 add_cpuid_test("x86/cpuid/phenom/level",
"-cpu phenom", "level", 5);
diff --git a/qom/cpu.c b/qom/cpu.c
index 03d9190..47d69f7 100644
--- a/qom/cpu.c
+++ b/qom/cpu.c
@@ -295,8 +295,7 @@ static ObjectClass *cpu_common_class_by_name(const char 
*cpu_model)
 static void cpu_common_parse_features(const char *typename, char *features,
   Error **errp)
 {
-char *featurestr; /* Single "key=value" string being parsed */
-char *val;
+const char *featurestr;
 static bool cpu_globals_initialized;
 
 /* TODO: all callers of ->parse_features() need to be changed to
@@ -310,16 +309,26 @@ static void cpu_common_parse_features(const char 
*typename, char *features,
 }
 cpu_globals_initialized = true;
 
-featurestr = features ? strtok(features, ",") : NULL;
+if (!features) {
+return;
+}
 
-while (featurestr) {
-val = strchr(featurestr, '=');
-if (val) {
+/*TODO: Use QemuOpts to parse -cpu on main(), so we don't need
+ *  to manually call get_opt_*() here.
+ */
+for (featurestr = features; *featurestr != '\0'; featurestr++) {
+const char *pe = strchr(featurestr, '=');
+const char *pc = strchr(featurestr, ',');
+if (pe && (!pc || pc > pe)) {
+char option[128], val[1024];
 GlobalProperty *prop = g_new0(typeof(*prop), 1);
-*val = 0;
-val++;
+
+featurestr = get_opt_name(option, sizeof(option), featurestr, '=');
+featurestr++;
+featurestr = get_opt_value(val, sizeof(val), featurestr);
+
 prop->driver = typename;
-prop->property = g_strdup(featurestr);
+prop->property = g_strdup(option);
 prop->value = g_strdup(val);
 prop->errp = &error_fatal;
 qdev_prop_register_global(prop);
@@ -328,7 +337,10 @@ static void cpu_common_parse_features(const char 
*typename, char *features,
featurestr);
 return;
 }
-featurestr = strtok(NULL, ",");
+
+if (*featurestr != ',') {
+break;
+}
 }
 }
 
diff --git a/target-i386/cpu.c b/target-i386/cpu.c
index 78b25af..3539419 100644
--- a/target-i386/cpu.c
+++ b/target-i386/cpu.c
@@ -1968,7 +1968,7 @@ static gint compare_string(gconstpointer a, gconstpointer 
b)
 static void x86_cpu_parse_featurestr(const char *typename, char *features,
  Error **errp)
 {
-char *featurestr; /* Single 'key=value" string being parsed */
+const char *featurestr;
 static bool cpu_globals_initialized;
 bool ambiguous = false;
 
@@ -1981,36 +1981,40 @@ static void x86_cpu_parse_featurestr(const char 
*typename, char *features,
 return;
 }
 
-for (featurestr = strtok(features, ",");
- featurestr;
- featurestr = strtok(NULL, ",")) {
-const char *name;
-const char *val = NULL;
- 

[Qemu-devel] [PATCH for-2.9 03/17] qtest.py: Support QTEST_LOG environment variable

[Eduardo Habkost]

qtest logs everything to stderr by default, but we don't want it
to be the default behavior on test cases.

Implement the same behavior of libqtest.c, and redirect the qtest
log to /dev/null by default unless the QTEST_LOG environment
variable is set.

Signed-off-by: Eduardo Habkost 
---
Patch originally submitted as part of series:
* [RFC v2 00/20] qmp: Report bus information on 'query-machines'
---
 scripts/qtest.py | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/scripts/qtest.py b/scripts/qtest.py
index d5aecb5..5ac2c69 100644
--- a/scripts/qtest.py
+++ b/scripts/qtest.py
@@ -88,8 +88,14 @@ class QEMUQtestMachine(qemu.QEMUMachine):
 self._qtest_path = os.path.join(test_dir, name + "-qtest.sock")
 
 def _base_args(self):
+if os.getenv('QTEST_LOG'):
+qtest_log = '/dev/fd/2'
+else:
+qtest_log = '/dev/null'
+
 args = super(QEMUQtestMachine, self)._base_args()
 args.extend(['-qtest', 'unix:path=' + self._qtest_path,
+ '-qtest-log', qtest_log,
  '-machine', 'accel=qtest'])
 return args
 
-- 
2.7.4

[Qemu-devel] [PATCH for-2.9 12/17] target-i386: Return migration-safe field on query-cpu-definitions

[Eduardo Habkost]

Return the migration-safe field on query-cpu-definitions. All CPU
models in x86 are migration-safe except "host".

Signed-off-by: Eduardo Habkost 
---
 target-i386/cpu-qom.h | 2 ++
 target-i386/cpu.c | 3 +++
 2 files changed, 5 insertions(+)

diff --git a/target-i386/cpu-qom.h b/target-i386/cpu-qom.h
index 93c9679..7561891 100644
--- a/target-i386/cpu-qom.h
+++ b/target-i386/cpu-qom.h
@@ -48,6 +48,7 @@ typedef struct X86CPUDefinition X86CPUDefinition;
  * X86CPUClass:
  * @cpu_def: CPU model definition
  * @ordering: Ordering on the "-cpu help" CPU model list.
+ * @migration_safe: See CpuDefinitionInfo::migration_safe
  * @parent_realize: The parent class' realize handler.
  * @parent_reset: The parent class' reset handler.
  *
@@ -62,6 +63,7 @@ typedef struct X86CPUClass {
 X86CPUDefinition *cpu_def;
 
 int ordering;
+bool migration_safe;
 
 /* Optional description of CPU model.
  * If unavailable, cpu_def->model_id is used */
diff --git a/target-i386/cpu.c b/target-i386/cpu.c
index 76a6a30..78b25af 100644
--- a/target-i386/cpu.c
+++ b/target-i386/cpu.c
@@ -2195,6 +2195,8 @@ static void x86_cpu_definition_entry(gpointer data, 
gpointer user_data)
 x86_cpu_class_check_missing_features(cc, &info->unavailable_features);
 info->has_unavailable_features = true;
 info->q_typename = g_strdup(object_class_get_name(oc));
+info->migration_safe = cc->migration_safe;
+info->has_migration_safe = true;
 
 entry = g_malloc0(sizeof(*entry));
 entry->value = info;
@@ -2317,6 +2319,7 @@ static void x86_cpu_cpudef_class_init(ObjectClass *oc, 
void *data)
 X86CPUClass *xcc = X86_CPU_CLASS(oc);
 
 xcc->cpu_def = cpudef;
+xcc->migration_safe = true;
 }
 
 static void x86_register_cpudef_type(X86CPUDefinition *def)
-- 
2.7.4

[Qemu-devel] [PATCH for-2.9 10/17] target-i386: Allow short strings to be used as vendor ID

[Eduardo Habkost]

If a short string is specified, it will be padded with zeroes.
Without this, "query-cpu-model-expansion model=base" would return
an expansion that would never work in the command-line.

Signed-off-by: Eduardo Habkost 
---
 target-i386/cpu.c | 11 +++
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/target-i386/cpu.c b/target-i386/cpu.c
index 98e1063..a584c3e 100644
--- a/target-i386/cpu.c
+++ b/target-i386/cpu.c
@@ -1789,19 +1789,22 @@ static void x86_cpuid_set_vendor(Object *obj, const 
char *value,
 X86CPU *cpu = X86_CPU(obj);
 CPUX86State *env = &cpu->env;
 int i;
+char buf[CPUID_VENDOR_SZ] = { 0 };
 
-if (strlen(value) != CPUID_VENDOR_SZ) {
+if (strlen(value) > CPUID_VENDOR_SZ) {
 error_setg(errp, QERR_PROPERTY_VALUE_BAD, "", "vendor", value);
 return;
 }
 
+strncpy(buf, value, sizeof(buf));
+
 env->cpuid_vendor1 = 0;
 env->cpuid_vendor2 = 0;
 env->cpuid_vendor3 = 0;
 for (i = 0; i < 4; i++) {
-env->cpuid_vendor1 |= ((uint8_t)value[i]) << (8 * i);
-env->cpuid_vendor2 |= ((uint8_t)value[i + 4]) << (8 * i);
-env->cpuid_vendor3 |= ((uint8_t)value[i + 8]) << (8 * i);
+env->cpuid_vendor1 |= ((uint8_t)buf[i]) << (8 * i);
+env->cpuid_vendor2 |= ((uint8_t)buf[i + 4]) << (8 * i);
+env->cpuid_vendor3 |= ((uint8_t)buf[i + 8]) << (8 * i);
 }
 }
 
-- 
2.7.4

[Qemu-devel] [PATCH for-2.9 06/17] tests: Add rules to non-gtester qtest test cases

[Eduardo Habkost]

Today, simple non-gtester binaries can be run easily by a single
Makefile rule (e.g. check-tests/qemu-iotest-quick.sh), but we
don't have anything to help us automatically run the same test
binary for multiple architectures.

This add check-simpleqtest-* rules that will help us run binaries
present in $(check-simpleqtest-*-y).

Signed-off-by: Eduardo Habkost 
---
Patch originally submitted as part of series:
* [RFC v2 00/20] qmp: Report bus information on 'query-machines'
---
 tests/Makefile.include | 37 -
 1 file changed, 36 insertions(+), 1 deletion(-)

diff --git a/tests/Makefile.include b/tests/Makefile.include
index e98d3b6..63c4347 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -791,6 +791,40 @@ check-report.html: check-report.xml
$(call quiet-command,gtester-report $< > $@,"GEN","$@")
 
 
+# rules for non-gtester qtest tests:
+
+SIMPLETEST_OPTIONS = $(if $(V),--verbose,--quiet)
+
+# rule dependencies are:
+#   check-simpleqtest: check-simpleqtest-$(ARCH) for each ARCH in QTEST_ARCH
+#   check-simpleqtest-$(ARCH): check-simpleqtest-$(TEST)-$(ARCH) for each
+#  TEST in $(check-simpleqtest-$(ARCH)-y)
+#  (generated by qtest_target macro)
+#   check-simpleqtest-$(TEST)-$(ARCH): runs test with $(SIMPLETEST_OPTIONS)
+#  (generated by qtest_target macro)
+#
+# $(check-simpleqtest-$(ARCH)-y) automatically includes
+#$(check-simpleqtest-generic-y)
+
+define qtest-target
+
+check-simpleqtest-$(1)-y += $$(check-simpleqtest-generic-y)
+
+.PHONY: $$(patsubst %, check-simpleqtest-%-$(1), $$(check-simpleqtest-$(1)-y))
+$$(patsubst %, check-simpleqtest-%-$(1), $$(check-simpleqtest-$(1)-y)): 
check-simpleqtest-%-$(1): %
+   $$(call quiet-command,QTEST_QEMU_BINARY=$(1)-softmmu/qemu-system-$(1) \
+   QTEST_QEMU_IMG=qemu-img$$(EXESUF) \
+   MALLOC_PERTURB_=$${MALLOC_PERTURB_:-$$((RANDOM % 255 + 1))} \
+   $$< $$(SIMPLETEST_OPTIONS),"QTEST-$(1)", "$$<")
+
+.PHONY: check-simpleqtest-$(1)
+check-simpleqtest-$(1): $$(patsubst %, check-simpleqtest-%-$(1), 
$$(check-simpleqtest-$(1)-y))
+
+endef
+
+$(foreach TARGET,$(QTEST_TARGETS),$(eval $(call qtest-target,$(TARGET
+
+
 # Other tests
 
 QEMU_IOTESTS_HELPERS-$(CONFIG_LINUX) = 
tests/qemu-iotests/socket_scm_helper$(EXESUF)
@@ -818,7 +852,8 @@ $(patsubst %, check-%, $(check-qapi-schema-y)): 
check-%.json: $(SRC_PATH)/%.json
 
 .PHONY: check-qapi-schema check-qtest check-unit check check-clean
 check-qapi-schema: $(patsubst %,check-%, $(check-qapi-schema-y))
-check-qtest: $(patsubst %,check-qtest-%, $(QTEST_TARGETS))
+check-simpleqtest: $(patsubst %,check-simpleqtest-%, $(QTEST_TARGETS))
+check-qtest: $(patsubst %,check-qtest-%, $(QTEST_TARGETS)) check-simpleqtest
 check-unit: $(patsubst %,check-%, $(check-unit-y))
 check-block: $(patsubst %,check-%, $(check-block-y))
 check: check-qapi-schema check-unit check-qtest
-- 
2.7.4

[Qemu-devel] [PATCH for-2.9 02/17] qemu.py: Make logging optional

[Eduardo Habkost]

If a test case doesn't make QEMU generate any output, there's no
need to redirect stdout and stderr to a file. On those cases,
logging can be disabled so any errors are included on the test
case output.

Signed-off-by: Eduardo Habkost 
---
Patch originally submitted as part of series:
* [RFC v2 00/20] qmp: Report bus information on 'query-machines'
---
 scripts/qemu.py | 25 +++--
 1 file changed, 19 insertions(+), 6 deletions(-)

diff --git a/scripts/qemu.py b/scripts/qemu.py
index 6d1b623..1468286 100644
--- a/scripts/qemu.py
+++ b/scripts/qemu.py
@@ -24,13 +24,17 @@ class QEMUMachine(object):
 '''A QEMU VM'''
 
 def __init__(self, binary, args=[], wrapper=[], name=None, 
test_dir="/var/tmp",
- monitor_address=None, socket_scm_helper=None, debug=False):
+ monitor_address=None, socket_scm_helper=None, debug=False,
+ logging=True):
 if name is None:
 name = "qemu-%d" % os.getpid()
 if monitor_address is None:
 monitor_address = os.path.join(test_dir, name + "-monitor.sock")
 self._monitor_address = monitor_address
-self._qemu_log_path = os.path.join(test_dir, name + ".log")
+if logging:
+self._qemu_log_path = os.path.join(test_dir, name + ".log")
+else:
+self._qemu_log_path = None
 self._popen = None
 self._binary = binary
 self._args = list(args) # Force copy args in case we modify them
@@ -91,6 +95,8 @@ class QEMUMachine(object):
 return self._popen.pid
 
 def _load_io_log(self):
+if self._qemu_log_path is None:
+return
 with open(self._qemu_log_path, "r") as fh:
 self._iolog = fh.read()
 
@@ -115,17 +121,24 @@ class QEMUMachine(object):
 def _post_shutdown(self):
 if not isinstance(self._monitor_address, tuple):
 self._remove_if_exists(self._monitor_address)
-self._remove_if_exists(self._qemu_log_path)
+if self._qemu_log_path is not None:
+self._remove_if_exists(self._qemu_log_path)
 
 def launch(self):
 '''Launch the VM and establish a QMP connection'''
 devnull = open('/dev/null', 'rb')
-qemulog = open(self._qemu_log_path, 'wb')
+if self._qemu_log_path is not None:
+qemulog = open(self._qemu_log_path, 'wb')
+stdout=qemulog
+stderr=subprocess.STDOUT
+else:
+stdout=None
+stderr=None
 try:
 self._pre_launch()
 args = self._wrapper + [self._binary] + self._base_args() + 
self._args
-self._popen = subprocess.Popen(args, stdin=devnull, stdout=qemulog,
-   stderr=subprocess.STDOUT, 
shell=False)
+self._popen = subprocess.Popen(args, stdin=devnull, stdout=stdout,
+   stderr=stderr, shell=False)
 self._post_launch()
 except:
 if self._popen:
-- 
2.7.4

[Qemu-devel] [PATCH for-2.9 00/17] target-i386: Implement query-cpu-model-expansion

[Eduardo Habkost]

This series implements query-cpu-model-expansion on target-i386.

QAPI / interface changes


When implementing this, I have noticed that the "host" CPU model
in i386 includes some migration-unsafe features that can't be
translated to any migration-safe representation: "pmu", and
"host-cache-info".

To be able to handle the migration-unsafe features, I have
extended the query-cpu-model-expansion definition to be clear
about what happens to those features when the CPU model is
expanded (in short: static expansion removes them, full expansion
keeps them).

I also added "static" and "migration-safe" fields to the return
value of query-cpu-model-expansion, so callers can know if the
the expanded representation is static and migration-safe.

Test code
-

I have added a Python test script for the feature, that will try
multiple combinations of the expansion operation, and see if the
returned data keeps matches some constratins.

The test script works with the s390x query-cpu-model-expansion
command, except that: 1) I couldn't test it with KVM; 2) qtest.py
error handling when QEMU refuses to run is unreliable (so the
script needs runnability information to be availble in TCG mode,
too, to skip not-runnable CPU models and avoid problems).

Future versions of the test script could run a arch-specific
CPUID-dump guest binary, and validate data seen by the guest
directly. While we don't do that, the script validates all QOM
properties on the CPU objects looking for unexpected changes. At
least in the case of x86, the QOM properties will include lots of
the CPUID data seen by the guest, giving us decent coverage.

Patches from other series
-

This series includes patches from other series, just to help on
the implementation of the test code:
* "qmp: Report QOM type name on query-cpu-definitions", that
  was already submitted for 2.9
* qemu.py, qtest.py, and tests/Makefile.include changes

---
Cc: Cornelia Huck 
Cc: Christian Borntraeger 
Cc: David Hildenbrand 
Cc: libvir-l...@redhat.com
Cc: Jiri Denemark 
Cc: "Jason J. Herne" 
Cc: Markus Armbruster 
Cc: Richard Henderson 
Cc: Igor Mammedov 
Cc: Eric Blake 

Eduardo Habkost (17):
  qmp: Report QOM type name on query-cpu-definitions
  qemu.py: Make logging optional
  qtest.py: Support QTEST_LOG environment variable
  qtest.py: make logging optional
  qtest.py: Make 'binary' parameter optional
  tests: Add rules to non-gtester qtest test cases
  target-i386: Reorganize and document CPUID initialization steps
  target-i386: Support "-cpu host" on TCG too
  target-i386: Move "host" properties to base class
  target-i386: Allow short strings to be used as vendor ID
  target-i386: Remove AMD feature flag aliases from Opteron models
  target-i386: Return migration-safe field on query-cpu-definitions
  cpu: Support comma escaping when parsing -cpu
  qapi: add static/migration-safe info to query-cpu-model-expansion
  target-i386: Define static "base" CPU model
  tests: query-cpu-model-test.py test code
  target-i386: Implement query-cpu-model-expansion QMP command

 qapi-schema.json  |  30 ++-
 scripts/qemu.py   |  25 ++-
 scripts/qtest.py  |  15 +-
 target-i386/cpu-qom.h |   8 +-
 tests/Makefile.include|  40 +++-
 tests/query-cpu-model-test.py | 421 
 tests/test-x86-cpuid-compat.c |  19 ++
 monitor.c |   4 +-
 qom/cpu.c |  32 ++-
 target-arm/helper.c   |   1 +
 target-i386/cpu.c | 481 +++---
 target-ppc/translate_init.c   |   1 +
 target-s390x/cpu_models.c |   5 +
 13 files changed, 928 insertions(+), 154 deletions(-)
 create mode 100755 tests/query-cpu-model-test.py

-- 
2.7.4

[Qemu-devel] [PATCH for-2.9 01/17] qmp: Report QOM type name on query-cpu-definitions

[Eduardo Habkost]

The new typename attribute on query-cpu-definitions will be used
to help management software use device-list-properties to check
which properties can be set using -cpu or -global for the CPU
model.

Signed-off-by: Eduardo Habkost 
---
Patch originally submitted as:
  From: Eduardo Habkost 
  Date: Wed, 16 Nov 2016 16:21:39 -0200
  Message-Id: <1479320499-29818-1-git-send-email-ehabk...@redhat.com>
  Subject: [PATCH for-2.9] qmp: Report QOM type name on query-cpu-definitions
---
 qapi-schema.json| 5 -
 target-arm/helper.c | 1 +
 target-i386/cpu.c   | 1 +
 target-ppc/translate_init.c | 1 +
 target-s390x/cpu_models.c   | 1 +
 5 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/qapi-schema.json b/qapi-schema.json
index f3e9bfc..8d113f8 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -3218,6 +3218,9 @@
 # @unavailable-features: #optional List of properties that prevent
 #the CPU model from running in the current
 #host. (since 2.8)
+# @typename: Type name that can be used as argument to @device-list-properties,
+#to introspect properties configurable using -cpu or -global.
+#(since 2.9)
 #
 # @unavailable-features is a list of QOM property names that
 # represent CPU model attributes that prevent the CPU from running.
@@ -3239,7 +3242,7 @@
 ##
 { 'struct': 'CpuDefinitionInfo',
   'data': { 'name': 'str', '*migration-safe': 'bool', 'static': 'bool',
-'*unavailable-features': [ 'str' ] } }
+'*unavailable-features': [ 'str' ], 'typename': 'str' } }
 
 ##
 # @query-cpu-definitions:
diff --git a/target-arm/helper.c b/target-arm/helper.c
index b5b65ca..3fc01b5 100644
--- a/target-arm/helper.c
+++ b/target-arm/helper.c
@@ -5207,6 +5207,7 @@ static void arm_cpu_add_definition(gpointer data, 
gpointer user_data)
 info = g_malloc0(sizeof(*info));
 info->name = g_strndup(typename,
strlen(typename) - strlen("-" TYPE_ARM_CPU));
+info->q_typename = g_strdup(typename);
 
 entry = g_malloc0(sizeof(*entry));
 entry->value = info;
diff --git a/target-i386/cpu.c b/target-i386/cpu.c
index de1f30e..93ead1a 100644
--- a/target-i386/cpu.c
+++ b/target-i386/cpu.c
@@ -2234,6 +2234,7 @@ static void x86_cpu_definition_entry(gpointer data, 
gpointer user_data)
 info->name = x86_cpu_class_get_model_name(cc);
 x86_cpu_class_check_missing_features(cc, &info->unavailable_features);
 info->has_unavailable_features = true;
+info->q_typename = g_strdup(object_class_get_name(oc));
 
 entry = g_malloc0(sizeof(*entry));
 entry->value = info;
diff --git a/target-ppc/translate_init.c b/target-ppc/translate_init.c
index 626e031..19ef250 100644
--- a/target-ppc/translate_init.c
+++ b/target-ppc/translate_init.c
@@ -10305,6 +10305,7 @@ CpuDefinitionInfoList *arch_query_cpu_definitions(Error 
**errp)
 
 info = g_malloc0(sizeof(*info));
 info->name = g_strdup(alias->alias);
+info->q_typename = g_strdup(object_class_get_name(oc));
 
 entry = g_malloc0(sizeof(*entry));
 entry->value = info;
diff --git a/target-s390x/cpu_models.c b/target-s390x/cpu_models.c
index c1e729d..5b66d33 100644
--- a/target-s390x/cpu_models.c
+++ b/target-s390x/cpu_models.c
@@ -290,6 +290,7 @@ static void create_cpu_model_list(ObjectClass *klass, void 
*opaque)
 info->has_migration_safe = true;
 info->migration_safe = scc->is_migration_safe;
 info->q_static = scc->is_static;
+info->q_typename = g_strdup(object_class_get_name(klass));
 
 
 entry = g_malloc0(sizeof(*entry));
-- 
2.7.4

[Qemu-devel] [PATCH for-2.9 05/17] qtest.py: Make 'binary' parameter optional

[Eduardo Habkost]

If the 'binary' parameter is omitted, use the $QTEST_QEMU_BINARY
environment variable.

Signed-off-by: Eduardo Habkost 
---
Patch originally submitted as part of series:
* [RFC v2 00/20] qmp: Report bus information on 'query-machines'
---
 scripts/qtest.py | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/scripts/qtest.py b/scripts/qtest.py
index 0496490..5a37b48 100644
--- a/scripts/qtest.py
+++ b/scripts/qtest.py
@@ -79,8 +79,10 @@ class QEMUQtestProtocol(object):
 class QEMUQtestMachine(qemu.QEMUMachine):
 '''A QEMU VM'''
 
-def __init__(self, binary, args=[], name=None, test_dir="/var/tmp",
+def __init__(self, binary=None, args=[], name=None, test_dir="/var/tmp",
  socket_scm_helper=None, logging=True):
+if binary is None:
+binary = os.getenv('QTEST_QEMU_BINARY')
 if name is None:
 name = "qemu-%d" % os.getpid()
 super(QEMUQtestMachine, self).__init__(binary, args, name=name, 
test_dir=test_dir,
-- 
2.7.4

[Qemu-devel] [PATCH for-2.9 04/17] qtest.py: make logging optional

[Eduardo Habkost]

Support the 'logging' parameter on QEMUQtestMachine, for test
cases that don't require logging.

Signed-off-by: Eduardo Habkost 
---
Patch originally submitted as part of series:
* [RFC v2 00/20] qmp: Report bus information on 'query-machines'
---
 scripts/qtest.py | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/scripts/qtest.py b/scripts/qtest.py
index 5ac2c69..0496490 100644
--- a/scripts/qtest.py
+++ b/scripts/qtest.py
@@ -80,11 +80,12 @@ class QEMUQtestMachine(qemu.QEMUMachine):
 '''A QEMU VM'''
 
 def __init__(self, binary, args=[], name=None, test_dir="/var/tmp",
- socket_scm_helper=None):
+ socket_scm_helper=None, logging=True):
 if name is None:
 name = "qemu-%d" % os.getpid()
 super(QEMUQtestMachine, self).__init__(binary, args, name=name, 
test_dir=test_dir,
-   
socket_scm_helper=socket_scm_helper)
+   
socket_scm_helper=socket_scm_helper,
+   logging=logging)
 self._qtest_path = os.path.join(test_dir, name + "-qtest.sock")
 
 def _base_args(self):
-- 
2.7.4

Re: [Qemu-devel] [PATCH v2] scripts: add "git.orderfile" for ordering diff hunks by pathname patterns

[Eduardo Habkost]

On Fri, Dec 02, 2016 at 10:01:52PM +0100, Laszlo Ersek wrote:
[...]
> +docs/*
> +*.txt
> +configure
> +GNUmakefile
> +makefile
> +Makefile*
> +*.mak
> +qapi-schema*.json
> +qapi/*.json
> +include/qapi/visitor.h
> +include/qapi/visitor-impl.h
> +scripts/qapi.py
> +scripts/*.py
> +*.h
> +qapi/qapi-visit-core.c

Maybe we could include test/* here, so test code appears before
the implementation?

> +*.c
> -- 
> 2.9.2
> 
> 

-- 
Eduardo

[Qemu-devel] [PATCH v2] scripts: add "git.orderfile" for ordering diff hunks by pathname patterns

[Laszlo Ersek]

When passed to git-diff (and to every other git command producing diffs
and/or diffstats) with "-O" or "diff.orderFile", this list of patterns
will place the more declarative / abstract hunks first, while changes to
imperative code / details will be near the end of the patches. This saves
on scrolling / searching and makes for easier reviewing.

We intend to advise contributors in the Wiki to run

  git config diff.orderFile scripts/git.orderfile

once, as part of their initial setup, before formatting their first (or,
for repeat contributors, next) patches.

See the "-O" option and the "diff.orderFile" configuration variable in
git-diff(1) and git-config(1).

Cc: "Michael S. Tsirkin" 
Cc: Eric Blake 
Cc: Fam Zheng 
Cc: Gerd Hoffmann 
Cc: John Snow 
Cc: Max Reitz 
Cc: Stefan Hajnoczi 
Signed-off-by: Laszlo Ersek 
---

Notes:
v2:
- "Makefile" -> "Makefile*" [Gerd]
- add leading comment [Gerd]
- add "docs/*" (note, there are *.txt files outside of docs/, so keeping
  those too) [Max, Fam, Eric]

 scripts/git.orderfile | 20 
 1 file changed, 20 insertions(+)
 create mode 100644 scripts/git.orderfile

diff --git a/scripts/git.orderfile b/scripts/git.orderfile
new file mode 100644
index ..3cab16e0505c
--- /dev/null
+++ b/scripts/git.orderfile
@@ -0,0 +1,20 @@
+# Apply this diff order to your git configuration with the command
+#
+#   git config diff.orderFile scripts/git.orderfile
+
+docs/*
+*.txt
+configure
+GNUmakefile
+makefile
+Makefile*
+*.mak
+qapi-schema*.json
+qapi/*.json
+include/qapi/visitor.h
+include/qapi/visitor-impl.h
+scripts/qapi.py
+scripts/*.py
+*.h
+qapi/qapi-visit-core.c
+*.c
-- 
2.9.2

Re: [Qemu-devel] [PATCH v3] doc: Add NBD_CMD_BLOCK_STATUS extension

[John Snow]

On 12/02/2016 01:45 PM, Alex Bligh wrote:
> John,
> 
>>> +Some storage formats and operations over such formats express a
>>> +concept of data dirtiness. Whether the operation is block device
>>> +mirroring, incremental block device backup or any other operation with
>>> +a concept of data dirtiness, they all share a need to provide a list
>>> +of ranges that this particular operation treats as dirty.
>>>
>>> How can data be 'dirty' if it is static and unchangeable? (I thought)
>>>
>>
>> In a simple case, live IO goes to e.g. hda.qcow2. These writes come from
>> the VM and cause the bitmap that QEMU manages to become dirty.
>>
>> We intend to expose the ability to fleece dirty blocks via NBD. What
>> happens in this scenario would be that a snapshot of the data at the
>> time of the request is exported over NBD in a read-only manner.
>>
>> In this way, the drive itself is R/W, but the "view" of it from NBD is
>> RO. While a hypothetical backup client is busy copying data out of this
>> temporary view, new writes are coming in to the drive, but are not being
>> exposed through the NBD export.
>>
>> (This goes into QEMU-specifics, but those new writes are dirtying a
>> version of the bitmap not intended to be exposed via the NBD channel.
>> NBD gets effectively a snapshot of both the bitmap AND the data.)
> 
> Thanks. That makes sense - or enough sense for me to carry on commenting!
> 

Whew! I'm glad.

>>> I now think what you are talking about backing up a *snapshot* of a disk
>>> that's running, where the disk itself was not connected using NBD? IE it's
>>> not being 'made dirty' by NBD_CMD_WRITE etc. Rather 'dirtiness' is 
>>> effectively
>>> an opaque state represented in a bitmap, which is binary metadata
>>> at some particular level of granularity. It might as well be 'happiness'
>>> or 'is coloured blue'. The NBD server would (normally) have no way of
>>> manipulating this bitmap.
>>>
>>> In previous comments, I said 'how come we can set the dirty bit through
>>> writes but can't clear it?'. This (my statement) is now I think wrong,
>>> as NBD_CMD_WRITE etc. is not defined to set the dirty bit. The
>>> state of the bitmap comes from whatever sets the bitmap which is outside
>>> the scope of this protocol to transmit it.
>>>
>>
>> You know, this is a fair point. We have not (to my knowledge) yet
>> carefully considered the exact bitmap management scenario when NBD is
>> involved in retrieving dirty blocks.
>>
>> Humor me for a moment while I talk about a (completely hypothetical, not
>> yet fully discussed) workflow for how I envision this feature.
>>
>> (1) User sets up a drive in QEMU, a bitmap is initialized, an initial
>> backup is made, etc.
>>
>> (2) As writes come in, QEMU's bitmap is dirtied.
>>
>> (3) The user decides they want to root around to see what data has
>> changed and would like to use NBD to do so, in contrast to QEMU's own
>> facilities for dumping dirty blocks.
>>
>> (4) A command is issued that creates a temporary, lightweight snapshot
>> ('fleecing') and exports this snapshot over NBD. The bitmap is
>> associated with the NBD export at this point at NBD server startup. (For
>> the sake of QEMU discussion, maybe this command is "blockdev-fleece")
>>
>> (5) At this moment, the snapshot is static and represents the data at
>> the time the NBD server was started. The bitmap is also forked and
>> represents only this snapshot. The live data and bitmap continue to change.
>>
>> (6) Dirty blocks are queried and copied out via NBD.
>>
>> (7) The user closes the NBD instance upon completion of their task,
>> whatever it was. (Making a new incremental backup? Just taking a peek at
>> some changed data? who knows.)
>>
>> The point that's interesting here is what do we do with the two bitmaps
>> at this point? The data delta can be discarded (this was after all just
>> a lightweight read-only point-in-time snapshot) but the bitmap data
>> needs to be dealt with.
>>
>> (A) In the case of "User made a new incremental backup," the bitmap that
>> got forked off to serve the NBD read should be discarded.
>>
>> (B) In the case of "User just wanted to look around," the bitmap should
>> be merged back into the bitmap it was forked from.
>>
>> I don't advise a hybrid where "User copied some data, but not all" where
>> we need to partially clear *and* merge, but conceivably this could
>> happen, because the things we don't want to happen always will.
>>
>> At this point maybe it's becoming obvious that actually it would be very
>> prudent to allow the NBD client itself to inform QEMU via the NBD
>> protocol which extents/blocks/(etc) that it is "done" with.
>>
>> Maybe it *would* actually be useful if, in NBD allowing us to add a
>> "dirty" bit to the specification, we allow users to clear those bits.
>>
>> Then, whether the user was trying to do (A) or (B) or the unspeakable
>> amalgamation of both things, it's up to the user to clear the bits
>> desired and QEMU can do the simple task of simply 

[Qemu-devel] [PATCH v3 1/2] block: Rename raw_bsd to raw-format.c

[Eric Blake]

Given that we have raw-win32.c and raw-posix.c, my initial guess at
raw_bsd.c was that it was for dealing with raw files using code
specific to the BSD operating system (beyond what raw-posix could
do).  Not so - this name was chosen back in commit e1c66c6 to
distinguish that it was a BSD licensed file, in contrast to the
then-existing raw.c with an unclear and potentially unusable
license.  But since it has been more than three years since the
rewrite, it's time to pick a more useful name for this file to
avoid this type of confusion to future contributors that don't know
the backstory, as none of our other files are named solely by the
license they use.

In reality, this file deals with the raw format, which is useful
with any number of protocols, while raw-{win32,posix} deal with
the file protocol (and in turn, that protocol is not limited to
use with the raw format).  So rename raw_bsd to raw-format.c.  We
could have also used the shorter name raw.c, except that collides
with the earlier use of that filename for a different license,
and it's better to be safe than risk license pollution.

The next patch will also rename raw-win32.c and raw-posix.c to
further distinguish the difference in roles.

It doesn't hurt that this gets rid of an underscore in the filename,
thereby making tab-completion on 'ra' easier (now I don't have
to type the shift key, which slows things down :)

Suggested-by: Daniel P. Berrange 
Signed-off-by: Eric Blake 
Reviewed-by: Laszlo Ersek 

---
v2: rename to raw-format.c, rather than plain raw.c.
---
 block/{raw_bsd.c => raw-format.c} | 2 +-
 MAINTAINERS   | 2 +-
 block/Makefile.objs   | 2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)
 rename block/{raw_bsd.c => raw-format.c} (99%)

diff --git a/block/raw_bsd.c b/block/raw-format.c
similarity index 99%
rename from block/raw_bsd.c
rename to block/raw-format.c
index 8a5b9b0..8404a82 100644
--- a/block/raw_bsd.c
+++ b/block/raw-format.c
@@ -1,4 +1,4 @@
-/* BlockDriver implementation for "raw"
+/* BlockDriver implementation for "raw" format driver
  *
  * Copyright (C) 2010-2016 Red Hat, Inc.
  * Copyright (C) 2010, Blue Swirl 
diff --git a/MAINTAINERS b/MAINTAINERS
index 4a60579..233cb81 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1722,7 +1722,7 @@ F: block/linux-aio.c
 F: include/block/raw-aio.h
 F: block/raw-posix.c
 F: block/raw-win32.c
-F: block/raw_bsd.c
+F: block/raw-format.c
 F: block/win32-aio.c

 qcow2
diff --git a/block/Makefile.objs b/block/Makefile.objs
index 67a036a..bde742f 100644
--- a/block/Makefile.objs
+++ b/block/Makefile.objs
@@ -1,4 +1,4 @@
-block-obj-y += raw_bsd.o qcow.o vdi.o vmdk.o cloop.o bochs.o vpc.o vvfat.o 
dmg.o
+block-obj-y += raw-format.o qcow.o vdi.o vmdk.o cloop.o bochs.o vpc.o vvfat.o 
dmg.o
 block-obj-y += qcow2.o qcow2-refcount.o qcow2-cluster.o qcow2-snapshot.o 
qcow2-cache.o
 block-obj-y += qed.o qed-gencb.o qed-l2-cache.o qed-table.o qed-cluster.o
 block-obj-y += qed-check.o
-- 
2.9.3

[Qemu-devel] [PATCH v3 2/2] block: Rename raw-{posix, win32} to file-*.c

[Eric Blake]

These files deal with the file protocol, not the raw format (the
file protocol is often used with other formats, and the raw
format is not forced to use the file protocol).  Rename things
to make it a bit easier to follow.

Suggested-by: Daniel P. Berrange 
Signed-off-by: Eric Blake 
Reviewed-by: John Snow 
Reviewed-by: Laszlo Ersek 

---
v2: rebase, fix commit message
---
 include/block/block_int.h   | 2 +-
 block/{raw-posix.c => file-posix.c} | 0
 block/{raw-win32.c => file-win32.c} | 0
 block/gluster.c | 4 ++--
 MAINTAINERS | 4 ++--
 block/Makefile.objs | 4 ++--
 block/trace-events  | 4 ++--
 configure   | 2 +-
 8 files changed, 10 insertions(+), 10 deletions(-)
 rename block/{raw-posix.c => file-posix.c} (100%)
 rename block/{raw-win32.c => file-win32.c} (100%)

diff --git a/include/block/block_int.h b/include/block/block_int.h
index 83a423c..4e4562d 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -184,7 +184,7 @@ struct BlockDriver {

 /*
  * Flushes all data that was already written to the OS all the way down to
- * the disk (for example raw-posix calls fsync()).
+ * the disk (for example file-posix.c calls fsync()).
  */
 int coroutine_fn (*bdrv_co_flush_to_disk)(BlockDriverState *bs);

diff --git a/block/raw-posix.c b/block/file-posix.c
similarity index 100%
rename from block/raw-posix.c
rename to block/file-posix.c
diff --git a/block/raw-win32.c b/block/file-win32.c
similarity index 100%
rename from block/raw-win32.c
rename to block/file-win32.c
diff --git a/block/gluster.c b/block/gluster.c
index 891c13b..2eaf4fc 100644
--- a/block/gluster.c
+++ b/block/gluster.c
@@ -1253,7 +1253,7 @@ static int qemu_gluster_has_zero_init(BlockDriverState 
*bs)
  * If @start is in a trailing hole or beyond EOF, return -ENXIO.
  * If we can't find out, return a negative errno other than -ENXIO.
  *
- * (Shamefully copied from raw-posix.c, only miniscule adaptions.)
+ * (Shamefully copied from file-posix.c, only miniscule adaptions.)
  */
 static int find_allocation(BlockDriverState *bs, off_t start,
off_t *data, off_t *hole)
@@ -1349,7 +1349,7 @@ exit:
  * 'nb_sectors' is the max value 'pnum' should be set to.  If nb_sectors goes
  * beyond the end of the disk image it will be clamped.
  *
- * (Based on raw_co_get_block_status() from raw-posix.c.)
+ * (Based on raw_co_get_block_status() from file-posix.c.)
  */
 static int64_t coroutine_fn qemu_gluster_co_get_block_status(
 BlockDriverState *bs, int64_t sector_num, int nb_sectors, int *pnum,
diff --git a/MAINTAINERS b/MAINTAINERS
index 233cb81..e11ebda 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1720,9 +1720,9 @@ L: qemu-bl...@nongnu.org
 S: Supported
 F: block/linux-aio.c
 F: include/block/raw-aio.h
-F: block/raw-posix.c
-F: block/raw-win32.c
 F: block/raw-format.c
+F: block/file-posix.c
+F: block/file-win32.c
 F: block/win32-aio.c

 qcow2
diff --git a/block/Makefile.objs b/block/Makefile.objs
index bde742f..0b8fd06 100644
--- a/block/Makefile.objs
+++ b/block/Makefile.objs
@@ -6,8 +6,8 @@ block-obj-y += vhdx.o vhdx-endian.o vhdx-log.o
 block-obj-y += quorum.o
 block-obj-y += parallels.o blkdebug.o blkverify.o blkreplay.o
 block-obj-y += block-backend.o snapshot.o qapi.o
-block-obj-$(CONFIG_WIN32) += raw-win32.o win32-aio.o
-block-obj-$(CONFIG_POSIX) += raw-posix.o
+block-obj-$(CONFIG_WIN32) += file-win32.o win32-aio.o
+block-obj-$(CONFIG_POSIX) += file-posix.o
 block-obj-$(CONFIG_LINUX_AIO) += linux-aio.o
 block-obj-y += null.o mirror.o commit.o io.o
 block-obj-y += throttle-groups.o
diff --git a/block/trace-events b/block/trace-events
index cfc05f2..671a6a8 100644
--- a/block/trace-events
+++ b/block/trace-events
@@ -53,8 +53,8 @@ qmp_block_job_resume(void *job) "job %p"
 qmp_block_job_complete(void *job) "job %p"
 qmp_block_stream(void *bs, void *job) "bs %p job %p"

-# block/raw-win32.c
-# block/raw-posix.c
+# block/file-win32.c
+# block/file-posix.c
 paio_submit_co(int64_t offset, int count, int type) "offset %"PRId64" count %d 
type %d"
 paio_submit(void *acb, void *opaque, int64_t offset, int count, int type) "acb 
%p opaque %p offset %"PRId64" count %d type %d"

diff --git a/configure b/configure
index 3770d7c..397927d 100755
--- a/configure
+++ b/configure
@@ -2738,7 +2738,7 @@ if compile_prog "" "" ; then
 fi

 ##
-# xfsctl() probe, used for raw-posix
+# xfsctl() probe, used for file-posix.c
 if test "$xfs" != "no" ; then
   cat > $TMPC << EOF
 #include   /* NULL */
-- 
2.9.3

[Qemu-devel] [PATCH v3 0/2] less confusing block file names

[Eric Blake]

no real change from v2 other than trivial rebasing and R-by:
https://lists.gnu.org/archive/html/qemu-devel/2016-10/msg07220.html

but it's been more than a month since the last activity, and the
start of the release cycle is as good a time as any to avoid any
potential churn on conflicts due to renaming files

applies on Kevin's block-next branch for 2.9

Eric Blake (2):
  block: Rename raw_bsd to raw-format.c
  block: Rename raw-{posix,win32} to file-*.c

 include/block/block_int.h   | 2 +-
 block/{raw-posix.c => file-posix.c} | 0
 block/{raw-win32.c => file-win32.c} | 0
 block/gluster.c | 4 ++--
 block/{raw_bsd.c => raw-format.c}   | 2 +-
 MAINTAINERS | 6 +++---
 block/Makefile.objs | 6 +++---
 block/trace-events  | 4 ++--
 configure   | 2 +-
 9 files changed, 13 insertions(+), 13 deletions(-)
 rename block/{raw-posix.c => file-posix.c} (100%)
 rename block/{raw-win32.c => file-win32.c} (100%)
 rename block/{raw_bsd.c => raw-format.c} (99%)

-- 
2.9.3

Re: [Qemu-devel] [PATCH v4 7/7] hw/i386/pc_q35: advertise broadcast SMI if VCPU hotplug is turned off

[Laszlo Ersek]

On 12/02/16 13:18, Igor Mammedov wrote:
> On Thu, 1 Dec 2016 21:42:58 +0100
> Laszlo Ersek  wrote:
> 
>> On 12/01/16 20:13, Eduardo Habkost wrote:
>>> On Thu, Dec 01, 2016 at 06:06:24PM +0100, Laszlo Ersek wrote:  
 For the time being, we cannot handle SMIs in OVMF if VCPUs can show up
 after boot. Otherwise, advertise ICH9_LPC_SMI_F_BROADCAST.

 Implement this generally, by introducing a new PCMachineClass method,
 namely get_smi_host_features(), and implement the above logic for
 pc-q35-2.9 and later. The idea is that future machine types might want to
 calculate (the same or different) SMI host features from different
 information, and that shouldn't affect earlier machine types.

 In turn, validating guest feature requests (inter-feature dependencies)
 should be possible purely based on the available host feature set. For
 example, in the future we might enforce that the guest select
 ICH9_LPC_SMI_F_VCPU_PARKING as a prerequisite for
 ICH9_LPC_SMI_F_BROADCAST, but only if the machine type itself advertises
 ICH9_LPC_SMI_F_VCPU_PARKING.

 Cc: "Michael S. Tsirkin" 
 Cc: Eduardo Habkost 
 Cc: Gerd Hoffmann 
 Cc: Igor Mammedov 
 Cc: Paolo Bonzini 
 Signed-off-by: Laszlo Ersek 
 ---
  include/hw/i386/pc.h |  1 +
  hw/i386/pc_q35.c | 24 +++-
  2 files changed, 24 insertions(+), 1 deletion(-)

 diff --git a/include/hw/i386/pc.h b/include/hw/i386/pc.h
 index 430735e501dd..e164947116b6 100644
 --- a/include/hw/i386/pc.h
 +++ b/include/hw/i386/pc.h
 @@ -116,10 +116,11 @@ struct PCMachineClass {
  /*< public >*/
  
  /* Methods: */
  HotplugHandler *(*get_hotplug_handler)(MachineState *machine,
 DeviceState *dev);
 +uint64_t (*get_smi_host_features)(void);  
>>>
>>> I'd prefer to encode the differences between machine-types as
>>> data, instead of code, to make introspection easier in the
>>> future. Is it possible to encode this in a simple PCMachineClass
>>> struct data field or (even bettter) a QOM property?
>>>   
>>
>> I don't know how else to capture the (smp_cpus == max_cpus) question,
>> for saying that "this machine type supports SMI broadcast, as long as
>> VCPU hotplug is disabled in the configuration".
> (smp_cpus == max_cpus) doesn't mean that CPU hotplug is disabled
> (it actually can't be disabled at all).
> 
> In addition, it's possible to start machine with
>  -smp 1,sockets=2,max_cpus=2 -device mycputype,socket=2
> 
> where all cpus will be coldpugged
> It would be better to use PCMachineState.boot_cpus which contains
> present cpus count.
> 
> I'd drop hotplug check (as usecase is broken in many places anyway)
> and even won't touch PCMachineState, instead add a property like
> ICH9_LPC.enable_smi_broadcast(on by default) and disable it for
> old machine types using compat props.

I'll look into that, thanks!
Laszlo

> 
> We can work on making CPU hotplug and OVMF work in follow up patches.
> 
>> Technically we could give PCMC two new (data) fields, a host features
>> bitmap for when VCPU hotplug is enabled, and another for when VCPU
>> hotplug is disabled. Then board code would take the right field and pass
>> it on to ich9_lpc_pm_init().
>>
>> Thanks
>> Laszlo
> 

[Qemu-devel] [PATCH v3 4/5] blkdebug: Add ability to override unmap geometries

[Eric Blake]

Make it easier to simulate various unusual hardware setups (for
example, recent commits 3482b9b and b8d0a98 affect the Dell
Equallogic iSCSI with its 15M preferred and maximum unmap and
write zero sizing, or b2f95fe deals with the Linux loopback
block device having a max_transfer of 64k), by allowing blkdebug
to wrap any other device with further restrictions on various
alignments.

Signed-off-by: Eric Blake 

---
v3: improve legibility of bounds checking, improve docs
v2: new patch
---
 qapi/block-core.json | 27 ++-
 block/blkdebug.c | 96 ++--
 2 files changed, 120 insertions(+), 3 deletions(-)

diff --git a/qapi/block-core.json b/qapi/block-core.json
index c29bef7..26f3e9f 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -2068,6 +2068,29 @@
 # @align:   #optional required alignment for requests in bytes,
 #   must be power of 2, or 0 for default
 #
+# @max-transfer:#optional maximum size for I/O transfers in bytes,
+#   must be multiple of the larger of @align and and 512
+#   (but need not be a power of 2), or 0 for default
+#   (since 2.9)
+#
+# @opt-write-zero:  #optional preferred alignment for write zero requests
+#   in bytes, must be multiple of the larger of @align
+#   and 512 (but need not be a power of 2), or 0 for
+#   default (since 2.9)
+#
+# @max-write-zero:  #optional maximum size for write zero requests in bytes,
+#   must be multiple of the larger of @align and 512 (but
+#   need not be a power of 2), or 0 for default (since 2.9)
+#
+# @opt-discard: #optional preferred alignment for discard requests
+#   in bytes, must be multiple of the larger of @align
+#   and 512 (but need not be a power of 2), or 0 for
+#   default (since 2.9)
+#
+# @max-discard: #optional maximum size for discard requests in bytes,
+#   must be multiple of the larger of @align and 512 (but
+#   need not be a power of 2), or 0 for default (since 2.9)
+#
 # @inject-error:#optional array of error injection descriptions
 #
 # @set-state:   #optional array of state-change descriptions
@@ -2077,7 +2100,9 @@
 { 'struct': 'BlockdevOptionsBlkdebug',
   'data': { 'image': 'BlockdevRef',
 '*config': 'str',
-'*align': 'int',
+'*align': 'int', '*max-transfer': 'int32',
+'*opt-write-zero': 'int32', '*max-write-zero': 'int32',
+'*opt-discard': 'int32', '*max-discard': 'int32',
 '*inject-error': ['BlkdebugInjectErrorOptions'],
 '*set-state': ['BlkdebugSetStateOptions'] } }

diff --git a/block/blkdebug.c b/block/blkdebug.c
index edc2b05..45f565d 100644
--- a/block/blkdebug.c
+++ b/block/blkdebug.c
@@ -39,6 +39,11 @@ typedef struct BDRVBlkdebugState {
 int state;
 int new_state;
 uint64_t align;
+uint64_t max_transfer;
+uint64_t opt_write_zero;
+uint64_t max_write_zero;
+uint64_t opt_discard;
+uint64_t max_discard;

 /* For blkdebug_refresh_filename() */
 char *config_file;
@@ -343,6 +348,31 @@ static QemuOptsList runtime_opts = {
 .type = QEMU_OPT_SIZE,
 .help = "Required alignment in bytes",
 },
+{
+.name = "max-transfer",
+.type = QEMU_OPT_SIZE,
+.help = "Maximum transfer size in bytes",
+},
+{
+.name = "opt-write-zero",
+.type = QEMU_OPT_SIZE,
+.help = "Optimum write zero alignment in bytes",
+},
+{
+.name = "max-write-zero",
+.type = QEMU_OPT_SIZE,
+.help = "Maximum write zero size in bytes",
+},
+{
+.name = "opt-discard",
+.type = QEMU_OPT_SIZE,
+.help = "Optimum discard alignment in bytes",
+},
+{
+.name = "max-discard",
+.type = QEMU_OPT_SIZE,
+.help = "Maximum discard size in bytes",
+},
 { /* end of list */ }
 },
 };
@@ -387,11 +417,57 @@ static int blkdebug_open(BlockDriverState *bs, QDict 
*options, int flags,
 bs->supported_zero_flags = (BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP) &
 bs->file->bs->supported_zero_flags;

-/* Set request alignment */
+/* Set alignment overrides */
 s->align = qemu_opt_get_size(opts, "align", 0);
 if (s->align && (s->align >= INT_MAX || !is_power_of_2(s->align))) {
 error_setg(errp, "Invalid alignment, align must be integer power of 
2");
-ret = -EINVAL;
+goto fail_unref;
+}
+
+s->max_transfer = qemu_opt_get_size(opts, "max-transfer", 0);
+if (s->max_transfer &&
+(s->max_transfer >= INT_MAX ||
+ !QEMU_IS_ALIGNED(s->max_transfer,
+  MAX(s->align, BDRV_

[Qemu-devel] [PATCH v3 5/5] tests: Add coverage for recent block geometry fixes

[Eric Blake]

Use blkdebug's new geometry constraints to emulate setups that
have caused recent regression fixes: write zeroes asserting
when running through a loopback block device with max-transfer
smaller than cluster size, and discard rounding away portions
of requests not aligned to preferred boundaries.  Also, add
coverage that the block layer is honoring max transfer limits.

For now, a single iotest performs all actions, with the idea
that we can add future blkdebug constraint test cases in the
same file; but it can be split into multiple iotests if we find
reason to run one portion of the test in more setups than what
are possible in the other.

Signed-off-by: Eric Blake 

---
v3: make comments tied more to test at hand, rather than the
particular hardware that led to the earlier patches being tested
v2: new patch
---
 tests/qemu-iotests/173 | 115 +
 tests/qemu-iotests/173.out |  49 +++
 tests/qemu-iotests/group   |   1 +
 3 files changed, 165 insertions(+)
 create mode 100755 tests/qemu-iotests/173
 create mode 100644 tests/qemu-iotests/173.out

diff --git a/tests/qemu-iotests/173 b/tests/qemu-iotests/173
new file mode 100755
index 000..fd421fc
--- /dev/null
+++ b/tests/qemu-iotests/173
@@ -0,0 +1,115 @@
+#!/bin/bash
+#
+# Test corner cases with unusual block geometries
+#
+# Copyright (C) 2016 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see .
+#
+
+# creator
+owner=ebl...@redhat.com
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+status=1   # failure is the default!
+
+_cleanup()
+{
+   _cleanup_test_img
+}
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_supported_fmt qcow2
+_supported_proto file
+
+CLUSTER_SIZE=1M
+size=128M
+options=driver=blkdebug,image.driver=qcow2
+
+echo
+echo "== setting up files =="
+
+_make_test_img $size
+$QEMU_IO -c "write -P 11 0 $size" "$TEST_IMG" | _filter_qemu_io
+mv "$TEST_IMG" "$TEST_IMG.base"
+_make_test_img -b "$TEST_IMG.base"
+$QEMU_IO -c "write -P 22 0 $size" "$TEST_IMG" | _filter_qemu_io
+
+# Limited to 64k max-transfer
+echo
+echo "== constrained alignment and max-transfer =="
+limits=align=4k,max-transfer=64k
+$QEMU_IO -c "open -o $options,$limits blkdebug::$TEST_IMG" \
+ -c "write -P 33 1000 128k" -c "read -P 33 1000 128k" | _filter_qemu_io
+
+echo
+echo "== write zero with constrained max-transfer =="
+limits=align=512,max-transfer=64k,opt-write-zero=$CLUSTER_SIZE
+$QEMU_IO -c "open -o $options,$limits blkdebug::$TEST_IMG" \
+ -c "write -z 8003584 2093056" | _filter_qemu_io
+
+# non-power-of-2 write-zero/discard alignments
+echo
+echo "== non-power-of-2 write zeroes =="
+
+limits=align=512,opt-write-zero=15M,max-write-zero=15M,opt-discard=15M,max-discard=15M
+$QEMU_IO -c "open -o $options,$limits blkdebug::$TEST_IMG" \
+ -c "write -z 32M 32M" | _filter_qemu_io
+
+echo
+echo "== non-power-of-2 discard =="
+
+limits=align=512,opt-write-zero=15M,max-write-zero=15M,opt-discard=15M,max-discard=15M
+$QEMU_IO -c "open -o $options,$limits blkdebug::$TEST_IMG" \
+ -c "discard 8001 30M" | _filter_qemu_io
+
+echo
+echo "== verify image content =="
+
+function verify_io()
+{
+if ($QEMU_IMG info -f "$IMGFMT" "$TEST_IMG" |
+   grep "compat: 0.10" > /dev/null); then
+# For v2 images, discarded clusters are read from the backing file
+discarded=11
+else
+# Discarded clusters are zeroed for v3 or later
+discarded=0
+fi
+
+echo read -P 22 0 1000
+echo read -P 33 1000 128k
+echo read -P 22 132072 7871512
+echo read -P 0 8003584 2093056
+echo read -P 22 10096640 23457792
+echo read -P 0 32M 32M
+echo read -P 22 64M 13M
+echo read -P $discarded 77M 29M
+echo read -P 22 106M 22M
+}
+
+verify_io | $QEMU_IO "$TEST_IMG" | _filter_qemu_io
+
+_check_test_img
+
+# success, all done
+echo "*** done"
+status=0
diff --git a/tests/qemu-iotests/173.out b/tests/qemu-iotests/173.out
new file mode 100644
index 000..3665c3d
--- /dev/null
+++ b/tests/qemu-iotests/173.out
@@ -0,0 +1,49 @@
+QA output created by 173
+
+== setting up files ==
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728
+wrote 134217728/134217728 bytes at offset 0
+128 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+F

[Qemu-devel] [PATCH v3 2/5] blkdebug: Add pass-through write_zero and discard support

[Eric Blake]

In order to test the effects of artificial geometry constraints
on operations like write zero or discard, we first need blkdebug
to manage these actions.  It also allows us to inject errors on
those operations, just like we can for read/write/flush.

We can also test the contract promised by the block layer; namely,
if a device has specified limits on alignment or maximum size,
then those limits must be obeyed (for now, the blkdebug driver
merely inherits limits from whatever it is wrapping, but the next
patch will further enhance it to allow specific limit overrides).

This patch intentionally refuses to service requests smaller than
the requested alignments; this is because an upcoming patch adds
a qemu-iotest to prove that the block layer is correctly handling
fragmentation, but the test only works if there is a way to tell
the difference at artificial alignment boundaries when blkdebug is
using a larger-than-default alignment.  If we let the blkdebug
layer always defer to the underlying layer, which potentially has
a smaller granularity, the iotest will be thwarted.

Tested by setting up an NBD server with export 'foo', then invoking:
$ ./qemu-io
qemu-io> open -o driver=blkdebug blkdebug::nbd://localhost:10809/foo
qemu-io> d 0 15M
qemu-io> w -z 0 15M

Pre-patch, the server never sees the discard (it was silently
eaten by the block layer); post-patch it is passed across the
wire.  Likewise, pre-patch the write is always passed with
NBD_WRITE (with 15M of zeroes on the wire), while post-patch
it can utilize NBD_WRITE_ZEROES (for less traffic).

Signed-off-by: Eric Blake 

---
v3: rebase to byte-based read/write, improve docs on why no
partial write zero passthrough
v2: new patch
---
 block/blkdebug.c | 81 
 1 file changed, 81 insertions(+)

diff --git a/block/blkdebug.c b/block/blkdebug.c
index 37094a2..aac8184 100644
--- a/block/blkdebug.c
+++ b/block/blkdebug.c
@@ -1,6 +1,7 @@
 /*
  * Block protocol for I/O error injection
  *
+ * Copyright (C) 2016 Red Hat, Inc.
  * Copyright (c) 2010 Kevin Wolf 
  *
  * Permission is hereby granted, free of charge, to any person obtaining a copy
@@ -382,6 +383,11 @@ static int blkdebug_open(BlockDriverState *bs, QDict 
*options, int flags,
 goto out;
 }

+bs->supported_write_flags = BDRV_REQ_FUA &
+bs->file->bs->supported_write_flags;
+bs->supported_zero_flags = (BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP) &
+bs->file->bs->supported_zero_flags;
+
 /* Set request alignment */
 align = qemu_opt_get_size(opts, "align", 0);
 if (align < INT_MAX && is_power_of_2(align)) {
@@ -512,6 +518,79 @@ static int blkdebug_co_flush(BlockDriverState *bs)
 }


+static int coroutine_fn blkdebug_co_pwrite_zeroes(BlockDriverState *bs,
+  int64_t offset, int count,
+  BdrvRequestFlags flags)
+{
+BDRVBlkdebugState *s = bs->opaque;
+BlkdebugRule *rule = NULL;
+uint32_t align = MAX(bs->bl.request_alignment,
+ bs->bl.pwrite_zeroes_alignment);
+
+/* Only pass through requests that are larger than requested
+ * preferred alignment (so that we test the fallback to writes on
+ * unaligned portions), and check that the block layer never hands
+ * us anything crossing an alignment boundary.  */
+if (count < align) {
+return -ENOTSUP;
+}
+assert(QEMU_IS_ALIGNED(offset, align));
+assert(QEMU_IS_ALIGNED(count, align));
+if (bs->bl.max_pwrite_zeroes) {
+assert(count <= bs->bl.max_pwrite_zeroes);
+}
+
+QSIMPLEQ_FOREACH(rule, &s->active_rules, active_next) {
+if (rule->options.inject.offset == -1) {
+break;
+}
+}
+
+if (rule && rule->options.inject.error) {
+return inject_error(bs, rule);
+}
+
+return bdrv_co_pwrite_zeroes(bs->file, offset, count, flags);
+}
+
+
+static int coroutine_fn blkdebug_co_pdiscard(BlockDriverState *bs,
+ int64_t offset, int count)
+{
+BDRVBlkdebugState *s = bs->opaque;
+BlkdebugRule *rule = NULL;
+uint32_t align = bs->bl.pdiscard_alignment;
+
+/* Only pass through requests that are larger than requested
+ * minimum alignment, and ensure that unaligned requests do not
+ * cross optimum discard boundaries. */
+if (count < bs->bl.request_alignment) {
+return -ENOTSUP;
+}
+assert(QEMU_IS_ALIGNED(offset, bs->bl.request_alignment));
+assert(QEMU_IS_ALIGNED(count, bs->bl.request_alignment));
+if (align && count >= align) {
+assert(QEMU_IS_ALIGNED(offset, align));
+assert(QEMU_IS_ALIGNED(count, align));
+}
+if (bs->bl.max_pdiscard) {
+assert(count <= bs->bl.max_pdiscard);
+}
+
+QSIMPLEQ_FOREACH(rule, &s->active_rules, active_next) {
+if (rule->options.inject.offset == -1) {
+break;
+}
+}

[Qemu-devel] [PATCH v3 1/5] blkdebug: Sanity check block layer guarantees

[Eric Blake]

Commits 04ed95f4 and 1a62d0ac updated the block layer to auto-fragment
any I/O to fit within device boundaries. Additionally, when using a
minimum alignment of 4k, we want to ensure the block layer does proper
read-modify-write rather than requesting I/O on a slice of a sector.
Let's enforce that the contract is obeyed when using blkdebug.  For
now, blkdebug only allows alignment overrides, and just inherits other
limits from whatever device it is wrapping, but a future patch will
further enhance things.

Signed-off-by: Eric Blake 

---
v3: rebase to byte-based interfaces
v2: new patch
---
 block/blkdebug.c | 14 ++
 1 file changed, 14 insertions(+)

diff --git a/block/blkdebug.c b/block/blkdebug.c
index acccf85..37094a2 100644
--- a/block/blkdebug.c
+++ b/block/blkdebug.c
@@ -438,6 +438,13 @@ blkdebug_co_preadv(BlockDriverState *bs, uint64_t offset, 
uint64_t bytes,
 BDRVBlkdebugState *s = bs->opaque;
 BlkdebugRule *rule = NULL;

+/* Sanity check block layer guarantees */
+assert(QEMU_IS_ALIGNED(offset, bs->bl.request_alignment));
+assert(QEMU_IS_ALIGNED(bytes, bs->bl.request_alignment));
+if (bs->bl.max_transfer) {
+assert(bytes <= bs->bl.max_transfer);
+}
+
 QSIMPLEQ_FOREACH(rule, &s->active_rules, active_next) {
 uint64_t inject_offset = rule->options.inject.offset;

@@ -462,6 +469,13 @@ blkdebug_co_pwritev(BlockDriverState *bs, uint64_t offset, 
uint64_t bytes,
 BDRVBlkdebugState *s = bs->opaque;
 BlkdebugRule *rule = NULL;

+/* Sanity check block layer guarantees */
+assert(QEMU_IS_ALIGNED(offset, bs->bl.request_alignment));
+assert(QEMU_IS_ALIGNED(bytes, bs->bl.request_alignment));
+if (bs->bl.max_transfer) {
+assert(bytes <= bs->bl.max_transfer);
+}
+
 QSIMPLEQ_FOREACH(rule, &s->active_rules, active_next) {
 uint64_t inject_offset = rule->options.inject.offset;

-- 
2.9.3

[Qemu-devel] [PATCH v3 3/5] blkdebug: Simplify override logic

[Eric Blake]

Rather than store into a local variable, the copy to the struct
if the value is valid, then reporting errors otherwise, it is
simpler to just store into the struct and report errors if the
value is invalid.  This however requires that the struct store
a 64-bit number, rather than a narrower type.

Signed-off-by: Eric Blake 

---
v3: new patch
---
 block/blkdebug.c | 11 ---
 1 file changed, 4 insertions(+), 7 deletions(-)

diff --git a/block/blkdebug.c b/block/blkdebug.c
index aac8184..edc2b05 100644
--- a/block/blkdebug.c
+++ b/block/blkdebug.c
@@ -38,7 +38,7 @@
 typedef struct BDRVBlkdebugState {
 int state;
 int new_state;
-int align;
+uint64_t align;

 /* For blkdebug_refresh_filename() */
 char *config_file;
@@ -353,7 +353,6 @@ static int blkdebug_open(BlockDriverState *bs, QDict 
*options, int flags,
 BDRVBlkdebugState *s = bs->opaque;
 QemuOpts *opts;
 Error *local_err = NULL;
-uint64_t align;
 int ret;

 opts = qemu_opts_create(&runtime_opts, NULL, 0, &error_abort);
@@ -389,11 +388,9 @@ static int blkdebug_open(BlockDriverState *bs, QDict 
*options, int flags,
 bs->file->bs->supported_zero_flags;

 /* Set request alignment */
-align = qemu_opt_get_size(opts, "align", 0);
-if (align < INT_MAX && is_power_of_2(align)) {
-s->align = align;
-} else if (align) {
-error_setg(errp, "Invalid alignment");
+s->align = qemu_opt_get_size(opts, "align", 0);
+if (s->align && (s->align >= INT_MAX || !is_power_of_2(s->align))) {
+error_setg(errp, "Invalid alignment, align must be integer power of 
2");
 ret = -EINVAL;
 goto fail_unref;
 }
-- 
2.9.3

[Qemu-devel] [PATCH v3 0/5] add blkdebug tests

[Eric Blake]

Based-on: Kevin Wolf's block-next branch:
http://repo.or.cz/qemu/kevin.git/shortlog/refs/heads/block-next

v2 of the series was here:
https://lists.gnu.org/archive/html/qemu-devel/2016-11/msg03495.html

since then, the first half was applied to 2.8; leaving just
this second half to enhance blkdebug for 2.9.  Rebase the patch
on top of Kevin's work to make blkdebug byte-based, and address
lots of good findings from Max.

Available as a tag at:
git fetch git://repo.or.cz/qemu/ericb.git nbd-blkdebug-v3

001/5:[0016] [FC] 'blkdebug: Sanity check block layer guarantees'
002/5:[0032] [FC] 'blkdebug: Add pass-through write_zero and discard support'
003/5:[down] 'blkdebug: Simplify override logic'
004/5:[0093] [FC] 'blkdebug: Add ability to override unmap geometries'
005/5:[0014] [FC] 'tests: Add coverage for recent block geometry fixes'

Eric Blake (5):
  blkdebug: Sanity check block layer guarantees
  blkdebug: Add pass-through write_zero and discard support
  blkdebug: Simplify override logic
  blkdebug: Add ability to override unmap geometries
  tests: Add coverage for recent block geometry fixes

 qapi/block-core.json   |  27 +-
 block/blkdebug.c   | 202 +++--
 tests/qemu-iotests/173 | 115 ++
 tests/qemu-iotests/173.out |  49 +++
 tests/qemu-iotests/group   |   1 +
 5 files changed, 384 insertions(+), 10 deletions(-)
 create mode 100755 tests/qemu-iotests/173
 create mode 100644 tests/qemu-iotests/173.out

-- 
2.9.3

Re: [Qemu-devel] [RFC PATCH] glusterfs: allow partial reads

[Eric Blake]

On 12/01/2016 04:59 AM, Wolfgang Bumiller wrote:
> Fixes #1644754.
> 
> Signed-off-by: Wolfgang Bumiller 
> ---
> I'm not sure what the original rationale was to treat both partial
> reads as well as well as writes as I/O error. (Seems to have happened
> from original glusterfs v1 to v2 series with a note but no reasoning
> for the read side as far as I could see.)
> The general direction lately seems to be to move away from sector
> based block APIs. Also eg. the NFS code allows partial reads. (It
> does, however, have an old patch (c2eb918e3) dedicated to aligning
> sizes to 512 byte boundaries for file creation for compatibility to
> other parts of qemu like qcow2. This already happens in glusterfs,
> though, but if you move a file from a different storage over to
> glusterfs you may end up with a qcow2 file with eg. the L1 table in
> the last 80 bytes of the file aligned to _begin_ at a 512 boundary,
> but not _end_ at one.)
> 
>  block/gluster.c | 10 +-
>  1 file changed, 9 insertions(+), 1 deletion(-)
> 
> diff --git a/block/gluster.c b/block/gluster.c
> index 891c13b..3db0bf8 100644
> --- a/block/gluster.c
> +++ b/block/gluster.c
> @@ -41,6 +41,7 @@ typedef struct GlusterAIOCB {
>  int ret;
>  Coroutine *coroutine;
>  AioContext *aio_context;
> +bool is_write;
>  } GlusterAIOCB;
>  
>  typedef struct BDRVGlusterState {
> @@ -716,8 +717,10 @@ static void gluster_finish_aiocb(struct glfs_fd *fd, 
> ssize_t ret, void *arg)
>  acb->ret = 0; /* Success */
>  } else if (ret < 0) {
>  acb->ret = -errno; /* Read/Write failed */
> +} else if (acb->is_write) {
> +acb->ret = -EIO; /* Partial write - fail it */
>  } else {
> -acb->ret = -EIO; /* Partial read/write - fail it */
> +acb->ret = 0; /* Success */

Does this properly guarantee that the portion beyond EOF reads as zero?

Would it be better to switch to byte-based interfaces rather than
continue to force gluster interaction in 512-byte sector chunks, since
gluster can obviously store files that are not 512-aligned?

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [Xen-devel] [PATCH v2 4/4] 9pfs: introduce init_out/in_iov_from_pdu

[Stefano Stabellini]

On Fri, 2 Dec 2016, Greg Kurz wrote:
> On Mon, 28 Nov 2016 13:27:24 -0800
> Stefano Stabellini  wrote:
> 
> > Not all 9pfs transports share memory between request and response. For
> > those who don't, it is necessary to know how much memory is required in
> > the response.
> > 
> > Split the existing init_iov_from_pdu function in two:
> > init_out_iov_from_pdu (for writes) and init_in_iov_from_pdu (for reads).
> > init_in_iov_from_pdu takes an additional size parameter to specify the
> > memory required for the response message.
> > 
> > Signed-off-by: Stefano Stabellini 
> > ---
> 
> Reviewed-by: Greg Kurz 

Thanks Greg! I assume you are going to apply them to your tree, right?

Thanks again,

Stefano


> >  hw/9pfs/9p.c   |  6 +-
> >  hw/9pfs/9p.h   |  6 --
> >  hw/9pfs/virtio-9p-device.c | 28 ++--
> >  3 files changed, 27 insertions(+), 13 deletions(-)
> > 
> > diff --git a/hw/9pfs/9p.c b/hw/9pfs/9p.c
> > index 79d7201..ce1f9d9 100644
> > --- a/hw/9pfs/9p.c
> > +++ b/hw/9pfs/9p.c
> > @@ -1652,7 +1652,11 @@ static void v9fs_init_qiov_from_pdu(QEMUIOVector 
> > *qiov, V9fsPDU *pdu,
> >  struct iovec *iov;
> >  unsigned int niov;
> >  
> > -pdu->s->transport->init_iov_from_pdu(pdu, &iov, &niov, is_write);
> > +if (is_write) {
> > +pdu->s->transport->init_out_iov_from_pdu(pdu, &iov, &niov);
> > +} else {
> > +pdu->s->transport->init_in_iov_from_pdu(pdu, &iov, &niov, size);
> > +}
> >  
> >  qemu_iovec_init_external(&elem, iov, niov);
> >  qemu_iovec_init(qiov, niov);
> > diff --git a/hw/9pfs/9p.h b/hw/9pfs/9p.h
> > index c8c9aa8..4c4feaf 100644
> > --- a/hw/9pfs/9p.h
> > +++ b/hw/9pfs/9p.h
> > @@ -349,8 +349,10 @@ struct V9fsTransport {
> >  va_list ap);
> >  ssize_t (*pdu_vunmarshal)(V9fsPDU *pdu, size_t offset, const char 
> > *fmt,
> >va_list ap);
> > -void(*init_iov_from_pdu)(V9fsPDU *pdu, struct iovec **piov,
> > - unsigned int *pniov, bool is_write);
> > +void(*init_in_iov_from_pdu)(V9fsPDU *pdu, struct iovec **piov,
> > +unsigned int *pniov, size_t size);
> > +void(*init_out_iov_from_pdu)(V9fsPDU *pdu, struct iovec **piov,
> > + unsigned int *pniov);
> >  void(*push_and_notify)(V9fsPDU *pdu);
> >  };
> >  
> > diff --git a/hw/9pfs/virtio-9p-device.c b/hw/9pfs/virtio-9p-device.c
> > index 273425b..27a4a32 100644
> > --- a/hw/9pfs/virtio-9p-device.c
> > +++ b/hw/9pfs/virtio-9p-device.c
> > @@ -171,26 +171,34 @@ static ssize_t virtio_pdu_vunmarshal(V9fsPDU *pdu, 
> > size_t offset,
> >  return v9fs_iov_vunmarshal(elem->out_sg, elem->out_num, offset, 1, 
> > fmt, ap);
> >  }
> >  
> > -static void virtio_init_iov_from_pdu(V9fsPDU *pdu, struct iovec **piov,
> > - unsigned int *pniov, bool is_write)
> > +/* The size parameter is used by other transports. Do not drop it. */
> > +static void virtio_init_in_iov_from_pdu(V9fsPDU *pdu, struct iovec **piov,
> > +unsigned int *pniov, size_t size)
> >  {
> >  V9fsState *s = pdu->s;
> >  V9fsVirtioState *v = container_of(s, V9fsVirtioState, state);
> >  VirtQueueElement *elem = v->elems[pdu->idx];
> >  
> > -if (is_write) {
> > -*piov = elem->out_sg;
> > -*pniov = elem->out_num;
> > -} else {
> > -*piov = elem->in_sg;
> > -*pniov = elem->in_num;
> > -}
> > +*piov = elem->in_sg;
> > +*pniov = elem->in_num;
> > +}
> > +
> > +static void virtio_init_out_iov_from_pdu(V9fsPDU *pdu, struct iovec **piov,
> > + unsigned int *pniov)
> > +{
> > +V9fsState *s = pdu->s;
> > +V9fsVirtioState *v = container_of(s, V9fsVirtioState, state);
> > +VirtQueueElement *elem = v->elems[pdu->idx];
> > +
> > +*piov = elem->out_sg;
> > +*pniov = elem->out_num;
> >  }
> >  
> >  static const struct V9fsTransport virtio_9p_transport = {
> >  .pdu_vmarshal = virtio_pdu_vmarshal,
> >  .pdu_vunmarshal = virtio_pdu_vunmarshal,
> > -.init_iov_from_pdu = virtio_init_iov_from_pdu,
> > +.init_in_iov_from_pdu = virtio_init_in_iov_from_pdu,
> > +.init_out_iov_from_pdu = virtio_init_out_iov_from_pdu,
> >  .push_and_notify = virtio_9p_push_and_notify,
> >  };
> >  
> 
> 
> ___
> Xen-devel mailing list
> xen-de...@lists.xen.org
> https://lists.xen.org/xen-devel
> 

Re: [Qemu-devel] [PATCH v3] doc: Add NBD_CMD_BLOCK_STATUS extension

[Alex Bligh]

John,

>> +Some storage formats and operations over such formats express a
>> +concept of data dirtiness. Whether the operation is block device
>> +mirroring, incremental block device backup or any other operation with
>> +a concept of data dirtiness, they all share a need to provide a list
>> +of ranges that this particular operation treats as dirty.
>> 
>> How can data be 'dirty' if it is static and unchangeable? (I thought)
>> 
> 
> In a simple case, live IO goes to e.g. hda.qcow2. These writes come from
> the VM and cause the bitmap that QEMU manages to become dirty.
> 
> We intend to expose the ability to fleece dirty blocks via NBD. What
> happens in this scenario would be that a snapshot of the data at the
> time of the request is exported over NBD in a read-only manner.
> 
> In this way, the drive itself is R/W, but the "view" of it from NBD is
> RO. While a hypothetical backup client is busy copying data out of this
> temporary view, new writes are coming in to the drive, but are not being
> exposed through the NBD export.
> 
> (This goes into QEMU-specifics, but those new writes are dirtying a
> version of the bitmap not intended to be exposed via the NBD channel.
> NBD gets effectively a snapshot of both the bitmap AND the data.)

Thanks. That makes sense - or enough sense for me to carry on commenting!

>> I now think what you are talking about backing up a *snapshot* of a disk
>> that's running, where the disk itself was not connected using NBD? IE it's
>> not being 'made dirty' by NBD_CMD_WRITE etc. Rather 'dirtiness' is 
>> effectively
>> an opaque state represented in a bitmap, which is binary metadata
>> at some particular level of granularity. It might as well be 'happiness'
>> or 'is coloured blue'. The NBD server would (normally) have no way of
>> manipulating this bitmap.
>> 
>> In previous comments, I said 'how come we can set the dirty bit through
>> writes but can't clear it?'. This (my statement) is now I think wrong,
>> as NBD_CMD_WRITE etc. is not defined to set the dirty bit. The
>> state of the bitmap comes from whatever sets the bitmap which is outside
>> the scope of this protocol to transmit it.
>> 
> 
> You know, this is a fair point. We have not (to my knowledge) yet
> carefully considered the exact bitmap management scenario when NBD is
> involved in retrieving dirty blocks.
> 
> Humor me for a moment while I talk about a (completely hypothetical, not
> yet fully discussed) workflow for how I envision this feature.
> 
> (1) User sets up a drive in QEMU, a bitmap is initialized, an initial
> backup is made, etc.
> 
> (2) As writes come in, QEMU's bitmap is dirtied.
> 
> (3) The user decides they want to root around to see what data has
> changed and would like to use NBD to do so, in contrast to QEMU's own
> facilities for dumping dirty blocks.
> 
> (4) A command is issued that creates a temporary, lightweight snapshot
> ('fleecing') and exports this snapshot over NBD. The bitmap is
> associated with the NBD export at this point at NBD server startup. (For
> the sake of QEMU discussion, maybe this command is "blockdev-fleece")
> 
> (5) At this moment, the snapshot is static and represents the data at
> the time the NBD server was started. The bitmap is also forked and
> represents only this snapshot. The live data and bitmap continue to change.
> 
> (6) Dirty blocks are queried and copied out via NBD.
> 
> (7) The user closes the NBD instance upon completion of their task,
> whatever it was. (Making a new incremental backup? Just taking a peek at
> some changed data? who knows.)
> 
> The point that's interesting here is what do we do with the two bitmaps
> at this point? The data delta can be discarded (this was after all just
> a lightweight read-only point-in-time snapshot) but the bitmap data
> needs to be dealt with.
> 
> (A) In the case of "User made a new incremental backup," the bitmap that
> got forked off to serve the NBD read should be discarded.
> 
> (B) In the case of "User just wanted to look around," the bitmap should
> be merged back into the bitmap it was forked from.
> 
> I don't advise a hybrid where "User copied some data, but not all" where
> we need to partially clear *and* merge, but conceivably this could
> happen, because the things we don't want to happen always will.
> 
> At this point maybe it's becoming obvious that actually it would be very
> prudent to allow the NBD client itself to inform QEMU via the NBD
> protocol which extents/blocks/(etc) that it is "done" with.
> 
> Maybe it *would* actually be useful if, in NBD allowing us to add a
> "dirty" bit to the specification, we allow users to clear those bits.
> 
> Then, whether the user was trying to do (A) or (B) or the unspeakable
> amalgamation of both things, it's up to the user to clear the bits
> desired and QEMU can do the simple task of simply always merging the
> bitmap fork upon the conclusion of the NBD fleecing exercise.
> 
> Maybe this would allow the dirty bit to have a bit more 

[Qemu-devel] 答复： [PATCH v4 5/5] object: Add 'help' option for all available backends and properties

[Nutaro, James J.]

Hi Lin,

My comments are inline with the patch.

ping...

>>> Lin Ma  2016/10/20 星期四 下午 7:28 >>>
'-object help' prints available user creatable backends.
'-object $typename,help' prints relevant properties.

Signed-off-by: Lin Ma 
---
include/qom/object_interfaces.h |  2 ++
qemu-options.hx  |  7 +-
qom/object_interfaces.c  | 55 +
vl.c|  5 
4 files changed, 68 insertions(+), 1 deletion(-)

diff --git a/include/qom/object_interfaces.h b/include/qom/object_interfaces.h
index 8b17f4d..197cd5f 100644
--- a/include/qom/object_interfaces.h
+++ b/include/qom/object_interfaces.h
@@ -165,4 +165,6 @@ int user_creatable_add_opts_foreach(void *opaque,
  */
void user_creatable_del(const char *id, Error **errp);

+int user_creatable_help_func(void *opaque, QemuOpts *opts, Error **errp);
+
#endif
diff --git a/qemu-options.hx b/qemu-options.hx
index b1fbdb0..b9573ce 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -3761,7 +3761,9 @@ DEF("object", HAS_ARG, QEMU_OPTION_object,
 "  create a new object of type TYPENAME 
setting properties\n"
 "  in the order they are specified.  Note that 
the 'id'\n"
 "  property must be set.  These objects are 
placed in the\n"
-" '/objects' path.\n",
+" '/objects' path.\n"
+" Use '-object help' to print available backend types 
and\n"
+" '-object typename,help' to print relevant 
properties.\n",
 QEMU_ARCH_ALL)
STEXI
@item -object @var{typename}[,@address@hidden,...]
@@ -3771,6 +3773,9 @@ in the order they are specified.  Note that the 'id'
property must be set.  These objects are placed in the
'/objects' path.

+Use @code{-object help} to print available backend types and
address@hidden @var{typename},help} to print relevant properties.
+
@table @option

@item -object 
memory-backend-file,address@hidden,address@hidden,address@hidden,address@hidden|off}
diff --git a/qom/object_interfaces.c b/qom/object_interfaces.c
index bf59846..da8be39 100644
--- a/qom/object_interfaces.c
+++ b/qom/object_interfaces.c
@@ -5,6 +5,7 @@
#include "qapi-visit.h"
#include "qapi/qmp-output-visitor.h"
#include "qapi/opts-visitor.h"
+#include "qemu/help_option.h"

void user_creatable_complete(Object *obj, Error **errp)
{
@@ -212,6 +213,60 @@ void user_creatable_del(const char *id, Error **errp)
 object_unparent(obj);
}

+int user_creatable_help_func(void *opaque, QemuOpts *opts, Error **errp)
+{
+const char *type = NULL;
+Object *obj = NULL;
+ObjectClass *klass;
+ObjectProperty *prop;
+ObjectPropertyIterator iter;
+
+type = qemu_opt_get(opts, "qom-type");
+if (type && is_help_option(type)) {
+   GSList *list;
+   printf("Available object backend types:\n");
+   for (list = object_class_get_list(TYPE_USER_CREATABLE, false);  \
+   list;   
  \
+   list = list->next) {
+   const char *name;
+   name = object_class_get_name(OBJECT_CLASS(list->data));
+   if (strcmp(name, TYPE_USER_CREATABLE)) {
+   printf("%s\n", name);
+   }
+   }
+   g_slist_free(list);
+   goto out;
+}
+

Is obj == NULL at this call to goto out? If so, you could simply return 1.

+if (!type || !qemu_opt_has_help_opt(opts)) {
+   return 0;
+}
+
+klass = object_class_by_name(type);
+if (!klass) {
+   printf("invalid object type: %s\n", type);
+   goto out;
+}

If obj == NULL here too?

+if (object_class_is_abstract(klass)) {
+   printf("object type '%s' is abstract\n", type);
+   goto out;
+}

And here?

+obj = object_new(type);

If this is the only place that obj is set to a non-NULL value, then the goto 
calls above can be replaced with returns.

+object_property_iter_init(&iter, obj);
+
+while ((prop = object_property_iter_next(&iter))) {
+   if (prop->description) {
+   printf("%s (%s, %s)\n", prop->name, prop->type, 
prop->description);
+   } else {
+   printf("%s (%s)\n", prop->name, prop->type);
+   }
+}
+
+out:
+object_unref(obj);
+return 1;
+}
+

static void register_types(void)
{
 static const TypeInfo uc_interface_info = {
diff --git a/vl.c b/vl.c
index ebd47af..145
6eca 100644
--- a/vl.c
+++ b/vl.c
@@ -4100,6 +4100,11 @@ int main(int argc, char **argv, char **envp)
 exit(0);
 }

+if (qemu_opts_foreach(qemu_find_opts("object"), user_creatable_help_func,
+ NULL, NULL)) {
+   

Re: [Qemu-devel] Unable to add ram object with same ID after addition, migration and removal at the target

[Dr. David Alan Gilbert]

* Bharata B Rao (bharata@gmail.com) wrote:
> Hi,
> 
> - Add ram object and dimm device at the source
> 
> (qemu) object_add memory-backend-ram,id=ram0,size=128M
> (qemu) device_add pc-dimm,id=dimm0,memdev=ram0
> 
> - Migrate the VM and remove the dimm device and ram object at the target
> 
> (qemu) device_del dimm0
> (qemu) object_del ram0
> 
> - Adding the ram object with same id now at the target fails like this:
> 
> (qemu) info memdev
> 
> (qemu) object_add memory-backend-ram,id=ram0,size=128M
> Duplicate ID 'ram0' for object
> 
> Same behaviour is seen on x86 and Power. The problem isn't seen if
> migration isn't involved (add, remove, add at the source itself).

Is this the same as the one Michael Roth posted a fix for a few days
back; 'monitor: fix object_del for command-line-created objects'

https://lists.gnu.org/archive/html/qemu-devel/2016-11/msg05479.html

Dave

> Regards,
> Bharata.
> -- 
> http://raobharata.wordpress.com/
--
Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

[Qemu-devel] Postcopy+spice crash

[Dr. David Alan Gilbert]

Hi Gerd,
  I've got a moderately repeatable crash with spice playing
a video + postcopy.  Some of the time I just get a warning
(that I also get in precopy) but sometimes it turns into
a backtrace;

This is:
  f24 guest playing youtube fullscreen.
  migration between 2.7.0<->current head (had crash both ways)

The warning I get with precopy most of the time is:
  ./x86_64-softmmu/qemu-system-x86_64:26921): Spice-Warning **: 
red_memslots.c:94:validate_virt: virtual address out of range
virt=0x7f5397ed002a+0x2925ff31 slot_id=1 group_id=1
slot=0x7f5397c0-0x7f539bbfe000 delta=0x7f5397c0

The crash I've had with postcopy is:
red_dispatcher_loadvm_commands:
id 0, group 0, virt start 0, virt end , generation 0, delta 0
id 1, group 1, virt start 7fbe83c0, virt end 7fbe87bfe000, generation 0, 
delta 7fbe83c0
id 2, group 1, virt start 7fbe7fa0, virt end 7fbe83a0, generation 0, 
delta 7fbe7fa0
(./x86_64-softmmu/qemu-system-x86_64:22376): Spice-CRITICAL **: 
red_memslots.c:123:get_virt: slot_id 128 too big, addr=8000

#0  0x7fc0aa42f49d in read () from /lib64/libpthread.so.0
#1  0x7fc0a8c36c01 in spice_backtrace_gstack () from 
/lib64/libspice-server.so.1
#2  0x7fc0a8c3e4f7 in spice_logv () from /lib64/libspice-server.so.1
#3  0x7fc0a8c3e655 in spice_log () from /lib64/libspice-server.so.1
#4  0x7fc0a8bfc6de in get_virt () from /lib64/libspice-server.so.1
#5  0x7fc0a8bfcb73 in red_get_data_chunks_ptr () from 
/lib64/libspice-server.so.1
#6  0x7fc0a8bff3fa in red_get_cursor_cmd () from /lib64/libspice-server.so.1
#7  0x7fc0a8c0fd79 in handle_dev_loadvm_commands () from 
/lib64/libspice-server.so.1
#8  0x7fc0a8bf9523 in dispatcher_handle_recv_read () from 
/lib64/libspice-server.so.1
#9  0x7fc0a8c1d5a5 in red_worker_main () from /lib64/libspice-server.so.1
#10 0x7fc0aa428dc5 in start_thread () from /lib64/libpthread.so.0
#11 0x7fc0a61786ed in clone () from /lib64/libc.so.6

and:

red_dispatcher_loadvm_commands:
id 0, group 0, virt start 0, virt end , generation 0, delta 0
id 1, group 1, virt start 7f3b9380, virt end 7f3b977fe000, generation 0, 
delta 7f3b9380
id 2, group 1, virt start 7f3b8f40, virt end 7f3b9340, generation 0, 
delta 7f3b8f40
(/opt/qemu/v2.7.0/bin/qemu-system-x86_64:41053): Spice-CRITICAL **: 
red_memslots.c:123:get_virt: slot_id 80 too big, addr=5000

I'm using:
spice-server-devel-0.12.4-19.el7.x86_64

Dave

--
Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

[Qemu-devel] [PATCH for-2.8] target-arm/translate-a64: fix gen_load_exclusive

[Alex Bennée]

While testing rth's latest TCG patches with risu I found ldaxp was
broken. Investigating further I found it was broken by 1dd089d0 when
the cmpxchg atomic work was merged. As part of that change the code
attempted to be clever by doing a single 64 bit load and then shuffle
the data around to set the two 32 bit registers.

As I couldn't quite follow the endian magic I've simply partially
reverted the change to the original code gen_load_exclusive code. This
doesn't affect the cmpxchg functionality as that is all done on in
gen_store_exclusive part which is untouched.

I've also restored the comment that was removed (with a slight tweak
to mention cmpxchg).

Signed-off-by: Alex Bennée 
---
 target-arm/translate-a64.c | 42 +++---
 1 file changed, 19 insertions(+), 23 deletions(-)

diff --git a/target-arm/translate-a64.c b/target-arm/translate-a64.c
index de48747..6dc27a6 100644
--- a/target-arm/translate-a64.c
+++ b/target-arm/translate-a64.c
@@ -1839,41 +1839,37 @@ static void disas_b_exc_sys(DisasContext *s, uint32_t 
insn)
 }
 }
 
+/*
+ * Load/Store exclusive instructions are implemented by remembering
+ * the value/address loaded, and seeing if these are the same
+ * when the store is performed. This is not actually the architecturally
+ * mandated semantics, but it works for typical guest code sequences
+ * and avoids having to monitor regular stores.
+ *
+ * The store exclusive uses the atomic cmpxchg primitives to avoid
+ * races in multi-threaded linux-user and when MTTCG softmmu is
+ * enabled.
+ */
 static void gen_load_exclusive(DisasContext *s, int rt, int rt2,
TCGv_i64 addr, int size, bool is_pair)
 {
 TCGv_i64 tmp = tcg_temp_new_i64();
-TCGMemOp be = s->be_data;
+TCGMemOp memop = s->be_data + size;
 
 g_assert(size <= 3);
+tcg_gen_qemu_ld_i64(tmp, addr, get_mem_index(s), memop);
+
 if (is_pair) {
+TCGv_i64 addr2 = tcg_temp_new_i64();
 TCGv_i64 hitmp = tcg_temp_new_i64();
 
-if (size == 3) {
-TCGv_i64 addr2 = tcg_temp_new_i64();
-
-tcg_gen_qemu_ld_i64(tmp, addr, get_mem_index(s),
-MO_64 | MO_ALIGN_16 | be);
-tcg_gen_addi_i64(addr2, addr, 8);
-tcg_gen_qemu_ld_i64(hitmp, addr2, get_mem_index(s),
-MO_64 | MO_ALIGN | be);
-tcg_temp_free_i64(addr2);
-} else {
-g_assert(size == 2);
-tcg_gen_qemu_ld_i64(tmp, addr, get_mem_index(s),
-MO_64 | MO_ALIGN | be);
-if (be == MO_LE) {
-tcg_gen_extr32_i64(tmp, hitmp, tmp);
-} else {
-tcg_gen_extr32_i64(hitmp, tmp, tmp);
-}
-}
-
+g_assert(size >= 2);
+tcg_gen_addi_i64(addr2, addr, 1 << size);
+tcg_gen_qemu_ld_i64(hitmp, addr2, get_mem_index(s), memop);
+tcg_temp_free_i64(addr2);
 tcg_gen_mov_i64(cpu_exclusive_high, hitmp);
 tcg_gen_mov_i64(cpu_reg(s, rt2), hitmp);
 tcg_temp_free_i64(hitmp);
-} else {
-tcg_gen_qemu_ld_i64(tmp, addr, get_mem_index(s), size | MO_ALIGN | be);
 }
 
 tcg_gen_mov_i64(cpu_exclusive_val, tmp);
-- 
2.10.2

Re: [Qemu-devel] [PATCH v7 0/5] IOMMU: intel_iommu support map and unmap notifications

[Alex Williamson]

On Fri, 2 Dec 2016 14:08:59 +0800
Peter Xu  wrote:

> On Thu, Dec 01, 2016 at 04:27:52PM +0800, Lan Tianyu wrote:
> > On 2016年11月30日 17:23, Peter Xu wrote:  
> > > On Mon, Nov 28, 2016 at 05:51:50PM +0200, Aviv B.D wrote:  
> > >> * intel_iommu's replay op is not implemented yet (May come in different 
> > >> patch 
> > >>   set).
> > >>   The replay function is required for hotplug vfio device and to move 
> > >> devices 
> > >>   between existing domains.  
> > > 
> > > I am thinking about this replay thing recently and now I start to
> > > doubt whether the whole vt-d vIOMMU framework suites this...
> > > 
> > > Generally speaking, current work is throwing away the IOMMU "domain"
> > > layer here. We maintain the mapping only per device, and we don't care
> > > too much about which domain it belongs. This seems problematic.
> > > 
> > > A simplest wrong case for this is (let's assume cache-mode is
> > > enabled): if we have two assigned devices A and B, both belong to the
> > > same domain 1. Meanwhile, in domain 1 assume we have one mapping which
> > > is the first page (iova range 0-0xfff). Then, if guest wants to
> > > invalidate the page, it'll notify VT-d vIOMMU with an invalidation
> > > message. If we do this invalidation per-device, we'll need to UNMAP
> > > the region twice - once for A, once for B (if we have more devices, we
> > > will unmap more times), and we can never know we have done duplicated
> > > work since we don't keep domain info, so we don't know they are using
> > > the same address space. The first unmap will work, and then we'll
> > > possibly get some errors on the rest of dma unmap failures.  
> > 
> > 
> > Hi Peter:  
> 
> Hi, Tianyu,
> 
> > According VTD spec 6.2.2.1, "Software must ensure that, if multiple
> > context-entries (or extended-context-entries) are programmed
> > with the same Domain-id (DID), such entries must be programmed with same
> > value for the secondlevel page-table pointer (SLPTPTR) field, and same
> > value for the PASID Table Pointer (PASIDTPTR) field.".
> > 
> > So if two assigned device may have different IO page table, they should
> > be put into different domains.  
> 
> 
> By default, devices will be put into different domains. However it
> should be legal that we put two assigned devices into the same IOMMU
> domain (in the guest), right? And we should handle both cases well
> IMHO.
> 
> Actually I just wrote a tool to do it based on vfio-pci:
> 
>   
> https://github.com/xzpeter/clibs/blob/master/gpl/userspace/vfio-bind-group/vfio-bind-group.c
> 
> If we run this tool in the guest with parameter like:
> 
>   ./vfio-bind-groups 00:03.0 00:04.0
> 
> It'll create one single domain, and put PCI device 00:03.0, 00:04.0
> into the same IOMMU domain.

On the host though, I'd expect we still have separate IOMMU domains,
one for each device and we do DMA_{UN}MAP ioctls separately per
container.  Thanks,

Alex

Re: [Qemu-devel] [PATCH v7 0/5] IOMMU: intel_iommu support map and unmap notifications

[Alex Williamson]

On Fri, 2 Dec 2016 13:59:25 +0800
Peter Xu  wrote:

> On Thu, Dec 01, 2016 at 04:21:38AM +, Tian, Kevin wrote:
> > > From: Peter Xu
> > > Sent: Wednesday, November 30, 2016 5:24 PM
> > > 
> > > On Mon, Nov 28, 2016 at 05:51:50PM +0200, Aviv B.D wrote:  
> > > > * intel_iommu's replay op is not implemented yet (May come in different 
> > > > patch
> > > >   set).
> > > >   The replay function is required for hotplug vfio device and to move 
> > > > devices
> > > >   between existing domains.  
> > > 
> > > I am thinking about this replay thing recently and now I start to
> > > doubt whether the whole vt-d vIOMMU framework suites this...
> > > 
> > > Generally speaking, current work is throwing away the IOMMU "domain"
> > > layer here. We maintain the mapping only per device, and we don't care
> > > too much about which domain it belongs. This seems problematic.
> > > 
> > > A simplest wrong case for this is (let's assume cache-mode is
> > > enabled): if we have two assigned devices A and B, both belong to the
> > > same domain 1. Meanwhile, in domain 1 assume we have one mapping which
> > > is the first page (iova range 0-0xfff). Then, if guest wants to
> > > invalidate the page, it'll notify VT-d vIOMMU with an invalidation
> > > message. If we do this invalidation per-device, we'll need to UNMAP
> > > the region twice - once for A, once for B (if we have more devices, we
> > > will unmap more times), and we can never know we have done duplicated
> > > work since we don't keep domain info, so we don't know they are using
> > > the same address space. The first unmap will work, and then we'll
> > > possibly get some errors on the rest of dma unmap failures.  
> > 
> > Tianyu and I discussed there is a bigger problem: today VFIO assumes 
> > only one address space per container, which is fine w/o vIOMMU (all devices 
> > in 
> > same container share same GPA->HPA translation). However it's not the case
> > when vIOMMU is enabled, because guest Linux implements per-device 
> > IOVA space. If a VFIO container includes multiple devices, it means 
> > multiple address spaces required per container...  
> 
> IIUC the vfio container is created in:
> 
>   vfio_realize
>   vfio_get_group
>   vfio_connect_container
> 
> Along the way (for vfio_get_group()), we have:
> 
>   group = vfio_get_group(groupid, pci_device_iommu_address_space(pdev), errp);
> 
> Here the address space is per device. If without vIOMMU, they will be
> pointed to the same system address space. However if with vIOMMU,
> that address space will be per-device, no?

Correct, with VT-d present, there will be a separate AddressSpace per
device, so each device will be placed into separate containers.  This
is currently the only way to provide the flexibility that those
separate devices can be attached to different domains in the guest.  It
also automatically faults on when devices share an iommu group on the
host but the guest attempts to use separate AddressSpaces.

Trouble comes when the guest is booted with iommu=pt as each container
will need to map the full guest memory, yet each container is accounted
separately for locked memory.  libvirt doesn't account for
$NUM_HOSTDEVS x $VM_MEM_SIZE for locked memory.

Ideally we could be more flexible with dynamic containers, but it's not
currently an option to move a group from one container to another w/o
first closing all the devices within the group.  Thanks,

Alex

Re: [Qemu-devel] [PATCH] scripts: add "git.orderfile" for ordering diff hunks by pathname patterns

[Eric Blake]

On 12/02/2016 08:41 AM, Fam Zheng wrote:

>>> +++ b/scripts/git.orderfile
>>> @@ -0,0 +1,15 @@
>>> +*.txt
>>
>> We also have *.md. (OK, OK, we have a single docs/bitmaps.md, but maybe
>> it's going to be more in the future. :-))
> 
> Maybe just insert 'docs/*' here? I'm not sure if it works for the subdirs,
> though.

Yes, you can specify subdirs. I think the way git works is that for each
line in the orderfile, it finds all matches in the set of files that
have a diff, then progresses to the next line on any remaining files.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH 14/14] tests: Add test-image-lock

[Max Reitz]

On 31.10.2016 16:38, Fam Zheng wrote:
> Signed-off-by: Fam Zheng 
> ---
>  tests/Makefile.include  |   2 +
>  tests/test-image-lock.c | 179 
> 
>  2 files changed, 181 insertions(+)
>  create mode 100644 tests/test-image-lock.c
> 
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index cd058ef..65f8500 100644
> --- a/tests/Makefile.include
> +++ b/tests/Makefile.include
> @@ -56,6 +56,7 @@ check-unit-y += tests/test-hbitmap$(EXESUF)
>  gcov-files-test-hbitmap-y = blockjob.c
>  check-unit-y += tests/test-blockjob$(EXESUF)
>  check-unit-y += tests/test-blockjob-txn$(EXESUF)
> +check-unit-y += tests/test-image-lock$(EXESUF)
>  check-unit-y += tests/test-x86-cpuid$(EXESUF)
>  # all code tested by test-x86-cpuid is inside topology.h
>  gcov-files-test-x86-cpuid-y =
> @@ -493,6 +494,7 @@ tests/test-rfifolock$(EXESUF): tests/test-rfifolock.o 
> $(test-util-obj-y)
>  tests/test-throttle$(EXESUF): tests/test-throttle.o $(test-block-obj-y)
>  tests/test-blockjob$(EXESUF): tests/test-blockjob.o $(test-block-obj-y) 
> $(test-util-obj-y)
>  tests/test-blockjob-txn$(EXESUF): tests/test-blockjob-txn.o 
> $(test-block-obj-y) $(test-util-obj-y)
> +tests/test-image-lock$(EXESUF): tests/test-image-lock.o $(test-block-obj-y) 
> $(libqos-obj-y)
>  tests/test-thread-pool$(EXESUF): tests/test-thread-pool.o $(test-block-obj-y)
>  tests/test-iov$(EXESUF): tests/test-iov.o $(test-util-obj-y)
>  tests/test-hbitmap$(EXESUF): tests/test-hbitmap.o $(test-util-obj-y)
> diff --git a/tests/test-image-lock.c b/tests/test-image-lock.c
> new file mode 100644
> index 000..db5a723
> --- /dev/null
> +++ b/tests/test-image-lock.c
> @@ -0,0 +1,179 @@
> +/*
> + * Image lock tests
> + *
> + * Copyright 2016 Red Hat, Inc.
> + *
> + * Authors:
> + *  Fam Zheng 
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2 or later.
> + * See the COPYING.LIB file in the top-level directory.
> + */
> +
> +#include "qemu/osdep.h"
> +#include "qapi/error.h"
> +#include "qemu/error-report.h"
> +#include "qapi/qmp/qbool.h"
> +#include "sysemu/block-backend.h"
> +
> +#define TEST_IMAGE_SIZE 4096
> +static char test_image[] = "/tmp/qtest.XX";
> +static int test_image_fd;
> +
> +static BlockBackend *open_test_image(int flags, bool disable_lock)
> +{
> +QDict *opts = qdict_new();
> +
> +qdict_set_default_str(opts, "filename", test_image);
> +qdict_set_default_str(opts, "driver", "file");
> +if (disable_lock) {
> +qdict_put(opts, "disable-lock", qbool_from_bool(true));
> +}
> +
> +return blk_new_open(NULL, NULL, opts, flags | BDRV_O_ALLOW_RDWR, NULL);
> +}
> +
> +#define RW true
> +#define RO false
> +#define SHARE true
> +#define EXCLU false
> +
> +static struct CompatData {
> +bool write_1;
> +bool share_1;
> +bool write_2;
> +bool share_2;
> +bool compatible;
> +} compat_data[] = {
> +/* Write 1, Share 1, Write 2, Share 2, Compatible. */
> +{ RO,   SHARE,   RO,  SHARE,   true,  },
> +{ RO,   SHARE,   RO,  EXCLU,   true,  },
> +{ RO,   SHARE,   RW,  SHARE,   true,  },
> +{ RO,   SHARE,   RW,  EXCLU,   true,  },
> +{ RO,   EXCLU,   RO,  SHARE,   true,  },
> +{ RO,   EXCLU,   RO,  EXCLU,   true,  },
> +{ RO,   EXCLU,   RW,  SHARE,   false, },
> +{ RO,   EXCLU,   RW,  EXCLU,   false, },
> +};

Without having looked closer at the test, what about RW/RW compatibility?

Max



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] QEMU Advent Calendar 2016

[Kashyap Chamarthy]

On Fri, Dec 02, 2016 at 03:24:27PM +0100, Ingve Vormestrand wrote:
> Hi! First of all, QEMU rocks!

Heya!

> I'm from the IncludeOS project, and QEMU is, without a doubt, one of our
> favourite tools! (If you haven't heard about IncludeOS, it is an open
> source unikernel operating system written in modern C++ (you can find out
> more about the project at http://includeos.org/ and
> https://github.com/hioa-cs/IncludeOS or see a very well received
> presentation at https://www.youtube.com/watch?v=t4etEwG2_LY from this
> year's CppCon).
> 
> One of the demo applications for IncludeOS is Acorn, a self-contained web
> server that includes its own OS, can serve static content, handle REST
> requests and provide an interactive real-time dashboard. The source is
> available at https://github.com/includeos/acorn and there is a live preview
> running at http://acorn2.unofficial.includeos.io/
>
> We frequently distribute Acorn demos as QEMU images, and we thought that
> maybe this could be a candidate for inclusion in your advent calendar?

Sounds good!  I've added it as an idea for one of the available free
slots.

If you'd have time to prepare the disk image, please go right ahead.
And if you have any questions for logistics, feel free to write to us
(perhaps off-list -- just to avoid traffic here :-)).

A quick reminder on requirements (search for "Disk image requirements"
here):

https://lists.gnu.org/archive/html/qemu-devel/2016-09/msg03990.html

Thanks for contacting.

[...]

-- 
/kashyap

Re: [Qemu-devel] [PATCH 13/14] raw-posix: Implement image locking

[Max Reitz]

On 31.10.2016 16:38, Fam Zheng wrote:
> This implements open flag sensible image locking for local file
> and host device protocol.
> 
> virtlockd in libvirt locks the first byte, so we start looking at the
> file bytes from 1.
> 
> Quoting what was proposed by Kevin Wolf , there are
> four locking modes by combining two bits (BDRV_O_RDWR and
> BDRV_O_SHARE_RW), and implemented by taking two locks:
> 
> Lock bytes:
> 
> * byte 1: I can't allow other processes to write to the image
> * byte 2: I am writing to the image
> 
> Lock modes:
> 
> * shared writer (BDRV_O_RDWR | BDRV_O_SHARE_RW): Take shared lock on
>   byte 2. Test whether byte 1 is locked using an exclusive lock, and
>   fail if so.
> 
> * exclusive writer (BDRV_O_RDWR only): Take shared lock on byte 2. Test
>   whether byte 1 is locked using an exclusive lock, and fail if so. Then
>   take shared lock on byte 1. I suppose this is racy, but we can
>   probably tolerate that.
> 
> * reader that can tolerate writers (BDRV_O_SHARE_RW only): Don't do anything
> 
> * reader that can't tolerate writers (neither bit is set): Take shared
>   lock on byte 1. Test whether byte 2 is locked, and fail if so.
> 
> The complication is in the transactional reopen.  To make the reopen
> logic managable, and allow better reuse, the code is internally
> organized with a table from old mode to the new one.
> 
> Signed-off-by: Fam Zheng 
> ---
>  block/raw-posix.c | 710 
> ++
>  1 file changed, 660 insertions(+), 50 deletions(-)
> 
> diff --git a/block/raw-posix.c b/block/raw-posix.c
> index 7c62fc3..07ab117 100644
> --- a/block/raw-posix.c
> +++ b/block/raw-posix.c

Resuming review!

> @@ -538,6 +763,398 @@ static int raw_open(BlockDriverState *bs, QDict 
> *options, int flags,
>  return raw_open_common(bs, options, flags, 0, errp);
>  }
>  
> +typedef enum {
> +RAW_LT_PREPARE,
> +RAW_LT_COMMIT,
> +RAW_LT_ABORT
> +} RawLockTransOp;
> +
> +typedef int (*RawReopenFunc)(RawLockTransOp op,
> + int old_lock_fd, int new_lock_fd,
> + BDRVRawLockMode old_lock,
> + BDRVRawLockMode new_lock,
> + Error **errp);
> +
> +static int raw_lt_nop(RawLockTransOp op,
> +int old_lock_fd, int new_lock_fd,
> +BDRVRawLockMode old_lock,
> +BDRVRawLockMode new_lock,
> +Error **errp)

Indentation is off.

> +{
> +assert(old_lock == new_lock || new_lock == RAW_L_READ_SHARE_RW);
> +return 0;
> +}
> +

Note and question before everything (after I had read basically
everything...): A small test shows me that locks are apparently shared
between dup'ed file descriptors, i.e.:

fd = open("foo");

lock(fd);
dup(fd);
unlock(fd);
close(fd);

new_fd = open("foo");
lock(new_fd); // fails

It works the other way around, too:

fd = open("foo");

lock(fd);
duped_fd = dup(fd);
unlock(duped_fd);
close(duped_fd);

new_fd = open("foo");
lock(new_fd); // fails


(The first lock() call has to use a shared lock and the second an
exclusive lock so that it works within a single process.)

This is pretty horrible because we don't know how old_lock_fd and
new_lock_fd are related. old_lock_fd was created somewhere through
raw_dup_flags() from s->fd; we have no idea whether that was actually
dup'ed or reopened. Then, rs->fd is created from s->fd, again through
raw_dup_flags(). Finally, new_lock_fd is created from rs->fd, once more
through said raw_dup_flags().

In the end, we have no idea whether new_lock_fd() is the end of a long
dup() chain from old_lock_fd or whether it was reopened somewhere in
between (well, we know that it has to be reopened when changing from R/W
to RO or the other way around, but we have no idea when keeping R/W or
RO (because dup() or fcntl() may have simply failed)).

In my opinion, this makes reviewing the code pretty difficult because
it's hard to keep in mind what cases may happen everywhere.

Would it be better to always fully re-open() the lock_fd instead of
duplicating it? Then we would at least know that old_lock_fd and
new_lock_fd never share the same lock configuration.

If you don't want to do that, there should be at least comments in the
RawReopenFuncs explaining whether new_lock_fd may share locks with
old_lock_fd or whether that's impossible.

> +static int raw_lt_from_unlock(RawLockTransOp op,
> +  int old_lock_fd, int new_lock_fd,
> +  BDRVRawLockMode old_lock,
> +  BDRVRawLockMode new_lock,
> +  Error **errp)
> +{
> +assert(old_lock != new_lock);
> +assert(old_lock == RAW_L_READ_SHARE_RW);
> +switch (op) {
> +case RAW_LT_PREPARE:
> +return raw_lock_fd(new_lock_fd, new_lock, errp);
> +break;

That break is a tad superfluous.

> +case RAW_

[Qemu-devel] [RISU PATCH 2/9] aarch64: add hand-coded risu skeleton for directed testing

[Alex Bennée]

Sometimes you want absolute control over your test set-up to feed
explicit values into the test. This started as an experiment but might
be useful for further developing tests.

Signed-off-by: Alex Bennée 
---
 Makefile  |   7 ++
 aarch64_simd_handcoded.risu.S | 208 ++
 2 files changed, 215 insertions(+)
 create mode 100644 aarch64_simd_handcoded.risu.S

diff --git a/Makefile b/Makefile
index bfa8cac..4202c35 100644
--- a/Makefile
+++ b/Makefile
@@ -36,6 +36,13 @@ $(PROG): $(OBJS)
 %.risu.asm: %.risu.bin
${OBJDUMP} -b binary -m $(ARCH) -D $^ > $@
 
+# hand-coded tests
+%.risu.bin: %.risu.elf
+   $(OBJCOPY) -O binary $< $@
+
+%.risu.elf: %.risu.S
+   ${AS} -o $@ $^
+
 %.o: %.c $(HDRS)
$(CC) $(CPPFLAGS) $(CFLAGS) -o $@ -c $<
 
diff --git a/aarch64_simd_handcoded.risu.S b/aarch64_simd_handcoded.risu.S
new file mode 100644
index 000..61bd11a
--- /dev/null
+++ b/aarch64_simd_handcoded.risu.S
@@ -0,0 +1,208 @@
+/*
+Hand coded RISU tests for aarch64
+
+Sometimes you want slightly more than random instructions and you
+want a specifically crafted test but within RISU's framework.
+
+This file offers such a thing.
+# So the last nibble indicates the desired operation:
+my $OP_COMPARE = 0;# compare registers
+my $OP_TESTEND = 1;# end of test, stop
+my $OP_SETMEMBLOCK = 2;# r0 is address of memory block (8192 bytes)
+my $OP_GETMEMBLOCK = 3;# add the address of memory block to r0
+my $OP_COMPAREMEM = 4; # compare memory block
+
+*/
+
+.macro  risuop_comp
+.word   0x5af0
+.endm
+.macro  risuop_testend
+.word   0x5af1
+.endm
+
+.org0x0
+
+//.globl   .data
+   mov x0, #0x0// #0
+   msr fpsr, x0
+   mov x0, #0x0// #0
+   msr fpcr, x0
+   mrs x0, nzcv
+   eor w0, w0, #0xf000
+   msr nzcv, x0
+   adr x0, _q0
+   eor x0, x0, #0xf
+   b   reg_setup
+
+/*
+
+This is the of block of data used for ld/st and setting up vector regs
+Each .word is 32bits of data
+
+*/
+.align  16
+
+_q0:.word   0x7000, 0x, 0x8000, 0x
+_q1:.word   0x9000, 0x, 0xa000, 0x
+_q2:.word   0x, 0x, 0x, 0x
+_q3:.word   0x, 0x, 0x, 0x
+_q4:.word   0x8000, 0x, 0xf000, 0x
+_q5:.word   0x, 0x, 0x, 0x
+_q6:.word   0x, 0x, 0x, 0x
+_q7:.word   0x, 0x, 0x, 0x
+
+_q8:.word   0x, 0x, 0x, 0x
+_q9:.word   0x, 0x, 0x, 0x
+_q10:   .word   0x, 0x, 0x, 0x
+_q11:   .word   0x, 0x, 0x, 0x
+_q12:   .word   0x, 0x, 0x, 0x
+_q13:   .word   0x, 0x, 0x, 0x
+_q14:   .word   0x, 0x, 0x, 0x
+_q15:   .word   0x, 0x, 0x, 0x
+
+_q16:   .word   0x, 0x, 0x, 0x
+_q17:   .word   0x, 0x, 0x, 0x
+_q18:   .word   0x, 0x, 0x, 0x
+_q19:   .word   0x, 0x, 0x, 0x
+_q20:   .word   0x, 0x, 0x, 0x
+_q21:   .word   0x, 0x, 0x, 0x
+_q22:   .word   0x, 0x, 0x, 0x
+_q23:   .word   0x, 0x, 0x, 0x
+
+_q24:   .word   0x, 0x, 0x, 0x
+_q25:   .word   0x, 0x, 0x, 0x
+_q26:   .word   0x, 0x, 0x, 0x
+_q27:   .word   0x, 0x, 0x, 0x
+_q28:   .word   0x, 0x, 0x, 0x
+_q29:   .word   0x, 0x, 0x, 0x
+_q30:   .word   0x, 0x, 0x, 0x
+_q31:   .word   0x, 0x, 0x, 0x
+
+.align  16
+
+/* Setup the register state */
+reg_setup:
+   ldp q0, q1, [x0],#32
+   ldp q2, q3, [x0],#16
+   ldp q4, q5, [x0],#16
+   ldp q6, q7, [x0],#16
+   ldp q8, q9, [x0],#16
+   ldp q10, q11, [x0],#16
+   ldp q12, q13, [x0],#16
+   ldp q14, q15, [x0],#16
+   ldp q16, q17, [x0],#16
+   ldp q18, q19, [x0],#16
+   ldp q20, q21, [x0],#16
+   ldp q22, q23, [x0],#16
+   ldp q24, q25, [x0],#16
+   ldp q26, q27, [x0],#16
+   ldp q28, q29, [x0],#16
+   ldp q30, q31, [x0],#16
+
+/* Set-up integer registers */
+   

[Qemu-devel] [RISU PATCH 3/9] risu: add simple trace and replay support

[Alex Bennée]

This adds a very dumb and easily breakable trace and replay support to
the aarch64 build of risu. In --master mode the various risu ops trigger
a write of register/memory state into a binary file which can be played
back to an apprentice. Currently there is no validation of the image
source so feeding the wrong image will fail straight away.

The trace files will get very big for any appreciable sized test file
and this will be addressed in later patches.

There is an inevitable amount of copy&paste between the trace file and
socket based code. Ideally it would be nice to have as much of this as
common as possible and reduce the re-implementation between the various
backends.

Signed-off-by: Alex Bennée 

---

v2
  - moved read/write functions into main risu.c
  - cleaned up formatting
  - report more in apprentice --trace mode
v3
  - fix options parsing
---
 risu.c | 104 +
 risu.h |  12 ++
 risu_aarch64.c |  90 --
 risu_reginfo_aarch64.h |   7 
 4 files changed, 185 insertions(+), 28 deletions(-)

diff --git a/risu.c b/risu.c
index 756ca46..f36b4c6 100644
--- a/risu.c
+++ b/risu.c
@@ -31,6 +31,7 @@
 void *memblock = 0;
 
 int apprentice_socket, master_socket;
+int trace_file = 0;
 
 sigjmp_buf jmpbuf;
 
@@ -40,31 +41,57 @@ int test_fp_exc = 0;
 long executed_tests = 0;
 void report_test_status(void *pc)
 {
-   executed_tests += 1;
-   if (executed_tests % 100 == 0) {
-  fprintf(stderr,"Executed %ld test instructions (pc=%p)\r",
-  executed_tests, pc);
+   if (ismaster || trace_file) {
+  executed_tests += 1;
+  if (executed_tests % 100 == 0) {
+ fprintf(stderr,"Executed %ld test instructions (pc=%p)\r",
+ executed_tests, pc);
+  }
}
 }
 
+int write_trace(void *ptr, size_t bytes)
+{
+   size_t res = write(trace_file, ptr, bytes);
+   return res == bytes;
+}
+
+int read_trace(void *ptr, size_t bytes)
+{
+   size_t res = read(trace_file, ptr, bytes);
+   return res == bytes;
+}
+
 void master_sigill(int sig, siginfo_t *si, void *uc)
 {
-   switch (recv_and_compare_register_info(master_socket, uc))
-   {
-  case 0:
- /* match OK */
- advance_pc(uc);
- return;
-  default:
- /* mismatch, or end of test */
- siglongjmp(jmpbuf, 1);
+   if (trace_file) {
+ if (write_to_tracefile(write_trace, uc)!=0) {
+   /* end of test */
+   siglongjmp(jmpbuf, 1);
+   /* never */
+   return;
+ }
+   } else {
+ if (recv_and_compare_register_info(master_socket, uc)!=0) {
+   /* mismatch, or end of test */
+   siglongjmp(jmpbuf, 1);
+   /* never */
+   return;
+ }
}
+   advance_pc(uc);
 }
 
 void apprentice_sigill(int sig, siginfo_t *si, void *uc)
 {
-   switch (send_register_info(apprentice_socket, uc))
-   {
+  int resp;
+  if (trace_file) {
+resp = read_tracefile_and_check(read_trace, uc);
+  } else {
+resp = send_register_info(apprentice_socket, uc);
+  }
+  switch (resp)
+  {
   case 0:
  /* match OK */
  advance_pc(uc);
@@ -75,6 +102,9 @@ void apprentice_sigill(int sig, siginfo_t *si, void *uc)
  exit(0);
   default:
  /* mismatch */
+ if (trace_file) {
+report_match_status();
+ }
  exit(1);
}
 }
@@ -136,7 +166,13 @@ int master(int sock)
 {
if (sigsetjmp(jmpbuf, 1))
{
-  return report_match_status();
+  if (trace_file) {
+ close(trace_file);
+ fprintf(stderr,"Done...\n");
+ return 0;
+  } else {
+ return report_match_status();
+  }
}
master_socket = sock;
set_sigill_handler(&master_sigill);
@@ -165,6 +201,7 @@ void usage (void)
fprintf(stderr, "between master and apprentice risu processes.\n\n");
fprintf(stderr, "Options:\n");
fprintf(stderr, "  --master  Be the master (server)\n");
+   fprintf(stderr, "  -t, --trace=FILE  Record/playback trace file\n");
fprintf(stderr, "  -h, --host=HOST   Specify master host machine 
(apprentice only)\n");
fprintf(stderr, "  -p, --port=PORT   Specify the port to connect to/listen 
on (default 9191)\n");
 }
@@ -175,6 +212,7 @@ int main(int argc, char **argv)
uint16_t port = 9191;
char *hostname = "localhost";
char *imgfile;
+   char *trace_fn = NULL;
int sock;
 
// TODO clean this up later
@@ -185,13 +223,14 @@ int main(int argc, char **argv)
  {
 { "help", no_argument, 0, '?'},
 { "master", no_argument, &ismaster, 1 },
+{ "trace", required_argument, 0, 't' },
 { "host", required_argument, 0, 'h' },
 { "port", required_argument, 0, 'p' },
 { "test-fp-exc", no_argument, &test_fp_exc, 1 },
 { 0,0,0,0 }
  };
   int optidx = 0;
-  int c = getopt_long(argc, argv, "h:p:", longopts, &optidx);
+  int c 

[Qemu-devel] [RISU PATCH 5/9] risu_aarch64: it's -> it is

[Alex Bennée]

Signed-off-by: Alex Bennée 
---
 risu_aarch64.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/risu_aarch64.c b/risu_aarch64.c
index 9d538cb..77f9288 100644
--- a/risu_aarch64.c
+++ b/risu_aarch64.c
@@ -82,7 +82,7 @@ int send_register_info(int sock, void *uc)
  * NB: called from a signal handler.
  *
  * We don't have any kind of identifying info in the incoming data
- * that says whether it's register or memory data, so if the two
+ * that says whether it is register or memory data, so if the two
  * sides get out of sync then we will fail obscurely.
  */
 int recv_and_compare_register_info(int sock, void *uc)
-- 
2.10.2

[Qemu-devel] [RISU PATCH 4/9] risu: add support compressed tracefiles

[Alex Bennée]

This uses the magic of zlib's gzread/write interface to wrap the
tracefile in compression. The code changes are tiny. I spent more time
messing about with the configure/linker stuff to auto-detect bits.

Signed-off-by: Alex Bennée 
---
 Makefile  |  3 ++-
 configure | 55 +++
 risu.c| 13 +
 3 files changed, 66 insertions(+), 5 deletions(-)

diff --git a/Makefile b/Makefile
index 4202c35..4a2ef02 100644
--- a/Makefile
+++ b/Makefile
@@ -14,6 +14,7 @@
 include Makefile.in
 
 CFLAGS ?= -g -Wall
+LDFLAGS += -lz
 
 PROG=risu
 SRCS=risu.c comms.c risu_$(ARCH).c risu_reginfo_$(ARCH).c
@@ -31,7 +32,7 @@ all: $(PROG) $(BINS)
 dump: $(RISU_ASMS)
 
 $(PROG): $(OBJS)
-   $(CC) $(STATIC) $(CFLAGS) -o $@ $^
+   $(CC) $(STATIC) $(CFLAGS) -o $@ $^ $(LDFLAGS) 
 
 %.risu.asm: %.risu.bin
${OBJDUMP} -b binary -m $(ARCH) -D $^ > $@
diff --git a/configure b/configure
index f81bdb5..d11680f 100755
--- a/configure
+++ b/configure
@@ -6,6 +6,10 @@ compile() {
 $CC $CFLAGS -c -o ${1}.o ${1}.c 2>/dev/null
 }
 
+link() {
+$LD $LDFLAGS -l${2} -o ${1} ${1}.o 2>/dev/null
+}
+
 check_define() {
 c=${tmp_dir}/check_define_${1}
 cat > ${c}.c < ${c}.c <
+#include 
+#include 
+#include 
+
+int main(void) { $1 thisone; return 0; }
+EOF
+compile $c
+}
+
+check_lib() {
+c=${tmp_dir}/check_lib${1}
+cat > ${c}.c <
+#include <$2.h>
+
+int main(void) { $3; return 0; }
+EOF
+compile $c && link $c $1
+}
+
+generate_config() {
+cfg=config.h
+echo "generating config.h..."
+
+echo "/* config.h - generated by the 'configure' script */" > $cfg
+echo "#ifndef CONFIG_H" >> $cfg
+echo "#define CONFIG_H 1" >> $cfg
+
+if check_type uintptr_t ; then
+echo "#define HAVE_UINTPTR_T 1" >> $cfg
+fi
+if check_type socklen_t ; then
+echo "#define HAVE_SOCKLEN_T 1" >> $cfg
+fi
+
+echo "#endif /* CONFIG_H */" >> $cfg
+
+echo "...done"
+}
+
 generate_makefilein() {
 m=Makefile.in
 echo "generating Makefile.in..."
@@ -93,6 +141,7 @@ done
 
 CC="${CC-${CROSS_PREFIX}gcc}"
 AS="${AS-${CROSS_PREFIX}as}"
+LD="${LD-${CROSS_PREFIX}ld}"
 OBJCOPY="${OBJCOPY-${CROSS_PREFIX}objcopy}"
 OBJDUMP="${OBJDUMP-${CROSS_PREFIX}objdump}"
 
@@ -100,6 +149,12 @@ if test "x${ARCH}" = "x"; then
 guess_arch
 fi
 
+if ! check_lib z zlib "zlibVersion()"; then
+echo "Cannot find libz compression library"
+exit 1
+fi
+
+generate_config
 generate_makefilein
 
 echo "type 'make' to start the build"
diff --git a/risu.c b/risu.c
index f36b4c6..1d58d4f 100644
--- a/risu.c
+++ b/risu.c
@@ -25,6 +25,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #include "risu.h"
 
@@ -32,6 +33,7 @@ void *memblock = 0;
 
 int apprentice_socket, master_socket;
 int trace_file = 0;
+gzFile gz_trace_file;
 
 sigjmp_buf jmpbuf;
 
@@ -52,13 +54,13 @@ void report_test_status(void *pc)
 
 int write_trace(void *ptr, size_t bytes)
 {
-   size_t res = write(trace_file, ptr, bytes);
+   size_t res = gzwrite(gz_trace_file, ptr, bytes);
return res == bytes;
 }
 
 int read_trace(void *ptr, size_t bytes)
 {
-   size_t res = read(trace_file, ptr, bytes);
+   size_t res = gzread(gz_trace_file, ptr, bytes);
return res == bytes;
 }
 
@@ -166,8 +168,9 @@ int master(int sock)
 {
if (sigsetjmp(jmpbuf, 1))
{
-  if (trace_file) {
- close(trace_file);
+  if (trace_file)
+  {
+ gzclose(gz_trace_file);
  fprintf(stderr,"Done...\n");
  return 0;
   } else {
@@ -284,6 +287,7 @@ int main(int argc, char **argv)
  if (trace_fn)
{
  trace_file = open(trace_fn, O_WRONLY|O_CREAT, S_IRWXU);
+ gz_trace_file = gzdopen(trace_file, "wb9");
} else {
  fprintf(stderr, "master port %d\n", port);
  sock = master_connect(port);
@@ -295,6 +299,7 @@ int main(int argc, char **argv)
  if (trace_fn)
{
  trace_file = open(trace_fn, O_RDONLY);
+ gz_trace_file = gzdopen(trace_file, "rb");
} else {
  fprintf(stderr, "apprentice host %s port %d\n", hostname, port);
  sock = apprentice_connect(hostname, port);
-- 
2.10.2

[Qemu-devel] [RISU PATCH 7/9] new: generate_all.sh script

[Alex Bennée]

A simple script to cut up a full .risu file and generate binaries for
the whole set. Call with risu file and instruction count and directory:

  ./generate_all.sh aarch64.risu 4 2 testcases.aarch64

Signed-off-by: Alex Bennée 
---
 generate_all.sh | 55 +++
 1 file changed, 55 insertions(+)
 create mode 100755 generate_all.sh

diff --git a/generate_all.sh b/generate_all.sh
new file mode 100755
index 000..4c490be
--- /dev/null
+++ b/generate_all.sh
@@ -0,0 +1,55 @@
+#!/bin/sh
+#
+# Generate All Patterns
+#
+# Work through a given .risu file and generate full coverage
+#
+
+BASE_SHELL_PARAMS="./risugen"
+
+risufile=$1
+divisor=$2
+insns=$3
+dir=$4
+
+if test -z "$risufile"; then
+echo "Need to specify a risu defintiion file"
+exit 1
+fi
+
+if test -z "$divisor"; then
+divisor=4
+fi
+
+if test -n "$insns"; then
+BASE_SHELL_PARAMS="${BASE_SHELL_PARAMS} --numinsns $insns"
+fi
+
+if test -n "dir"; then
+mkdir -p $dir
+else
+dir="./"
+fi
+
+ALL_INSNS=$(cat $risufile | ag "^\w" | cut -f 1 -d " " | sort)
+COUNT=$(cat $risufile | ag "^\w" | cut -f 1 -d " " | wc -l)
+set -- $ALL_INSNS
+
+GROUP=$((COUNT / $divisor))
+
+while test $# -gt 0 ; do
+INSN_PATTERNS=""
+I_FILE="$dir/insn_"
+for i in `seq 1 $divisor`; do
+I=$1
+if test -n "${I}"; then
+shift
+INSN_PATTERNS="${INSN_PATTERNS} --pattern ${I}"
+I_FILE="${I_FILE}${I}_"
+fi
+done
+I_FILE="${I_FILE}_INC.risu.bin"
+CMD="$BASE_SHELL_PARAMS ${INSN_PATTERNS} $risufile ${I_FILE}"
+echo "Running: $CMD"
+$CMD
+done
-- 
2.10.2

[Qemu-devel] [RISU PATCH 9/9] new: run_risu.sh script

[Alex Bennée]

A simple script to work through running all of a bunch of files with
record/playback traces. Dumps a summary and the number of failed tests
at the end.

Signed-off-by: Alex Bennée 
---
 run_risu.sh | 51 +++
 1 file changed, 51 insertions(+)
 create mode 100755 run_risu.sh

diff --git a/run_risu.sh b/run_risu.sh
new file mode 100755
index 000..8a91e1a
--- /dev/null
+++ b/run_risu.sh
@@ -0,0 +1,51 @@
+#!/bin/bash
+#
+# Run risu against a set of binaries + trace files
+#
+#
+#set -e
+
+passed=()
+failed=()
+missing=()
+
+script_dir=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd -P)
+risu_path=${script_dir}/risu
+
+for f in $@; do
+t="$f.trace"
+echo "Running $f against $t"
+if [ -e $t ]; then
+${QEMU} ${risu_path} $f -t $t
+if [ $? == 0 ]; then
+passed=( "${passed[@]}" $f )
+else
+failed=( "${failed[@]}" $f )
+fi
+else
+missing=( "${missing[@]}" $f )
+fi
+done
+
+if test ${#missing[@]} -gt 0; then
+echo "Tests missing ${#missing[@]} trace files:"
+for m in "${missing[@]}"; do
+echo "$m"
+done
+fi
+
+if test ${#passed[@]} -gt 0; then
+echo "Passed ${#passed[@]} tests:"
+for p in "${passed[@]}"; do
+echo "$p"
+done
+fi
+
+if test ${#failed[@]} -gt 0; then
+echo "Failed ${#failed[@]} tests:"
+for f in "${failed[@]}"; do
+echo "$f"
+done
+fi
+
+exit ${#failed[@]}
-- 
2.10.2

[Qemu-devel] [RISU PATCH 0/9] Record/playback patches

[Alex Bennée]

Hi Peter,

I've been cleaning things up so I thought I should re-post my current
state. These all apply to the current master.

I had to regenerate all the risu binaries as I'd used --no-fp for a
bunch of them originally which was causing failures. I'm not sure if
this is due to the FP registers not being cleared by the kernel if FP
isn't used - but we certainly don't do anything to them except when
set by the memory/context blocks (without --no-fp). This led me to
write the 3 noddy scripts included here.

The record/playback is still aarch64 only. I'm open to ideas to do
this more cleanly otherwise if anyone has access to some PPC hardware
I can port the record/playback code to the other architectures.

Feel free to cherry-pick any of the minor patches as you wish ;-)

Alex Bennée (9):
  risu: a bit more verbosity when running
  aarch64: add hand-coded risu skeleton for directed testing
  risu: add simple trace and replay support
  risu: add support compressed tracefiles
  risu_aarch64: it's -> it is
  risugen: remove grocer's apostrophe from REs
  new: generate_all.sh script
  new: record_traces.sh helper script
  new: run_risu.sh script

 Makefile  |  10 +-
 aarch64_simd_handcoded.risu.S | 208 ++
 configure |  55 +++
 generate_all.sh   |  55 +++
 record_traces.sh  |  16 
 risu.c| 116 ++-
 risu.h|  15 +++
 risu_aarch64.c|  89 +-
 risu_reginfo_aarch64.h|   7 ++
 risugen   |   2 +-
 run_risu.sh   |  51 +++
 11 files changed, 598 insertions(+), 26 deletions(-)
 create mode 100644 aarch64_simd_handcoded.risu.S
 create mode 100755 generate_all.sh
 create mode 100755 record_traces.sh
 create mode 100755 run_risu.sh

-- 
2.10.2

[Qemu-devel] [RISU PATCH 1/9] risu: a bit more verbosity when running

[Alex Bennée]

Before this is could seem a little quite when running as you had no
indication stuff was happening (or how fast). I only dump on the master
side as I want to minimise the amount of qemu logs to sift through.

Signed-off-by: Alex Bennée 
---
 risu.c | 15 +--
 risu.h |  3 +++
 risu_aarch64.c |  3 +++
 3 files changed, 19 insertions(+), 2 deletions(-)

diff --git a/risu.c b/risu.c
index 7e42160..756ca46 100644
--- a/risu.c
+++ b/risu.c
@@ -37,6 +37,16 @@ sigjmp_buf jmpbuf;
 /* Should we test for FP exception status bits? */
 int test_fp_exc = 0;
 
+long executed_tests = 0;
+void report_test_status(void *pc)
+{
+   executed_tests += 1;
+   if (executed_tests % 100 == 0) {
+  fprintf(stderr,"Executed %ld test instructions (pc=%p)\r",
+  executed_tests, pc);
+   }
+}
+
 void master_sigill(int sig, siginfo_t *si, void *uc)
 {
switch (recv_and_compare_register_info(master_socket, uc))
@@ -61,6 +71,7 @@ void apprentice_sigill(int sig, siginfo_t *si, void *uc)
  return;
   case 1:
  /* end of test */
+ fprintf(stderr, "\nend of test\n");
  exit(0);
   default:
  /* mismatch */
@@ -129,7 +140,7 @@ int master(int sock)
}
master_socket = sock;
set_sigill_handler(&master_sigill);
-   fprintf(stderr, "starting image\n");
+   fprintf(stderr, "starting master image at 0x%lx\n", image_start_address);
image_start();
fprintf(stderr, "image returned unexpectedly\n");
exit(1);
@@ -139,7 +150,7 @@ int apprentice(int sock)
 {
apprentice_socket = sock;
set_sigill_handler(&apprentice_sigill);
-   fprintf(stderr, "starting image\n");
+   fprintf(stderr, "starting apprentice image at 0x%lx\n", 
image_start_address);
image_start();
fprintf(stderr, "image returned unexpectedly\n");
exit(1);
diff --git a/risu.h b/risu.h
index 26ed834..e4bb323 100644
--- a/risu.h
+++ b/risu.h
@@ -26,6 +26,7 @@ extern uintptr_t image_start_address;
 extern void *memblock;
 
 extern int test_fp_exc;
+extern int ismaster;
 
 /* Ops code under test can request from risu: */
 #define OP_COMPARE 0
@@ -59,6 +60,8 @@ int recv_and_compare_register_info(int sock, void *uc);
  */
 int report_match_status(void);
 
+void report_test_status(void *pc);
+
 /* Move the PC past this faulting insn by adjusting ucontext
  */
 void advance_pc(void *uc);
diff --git a/risu_aarch64.c b/risu_aarch64.c
index 547f987..1595604 100644
--- a/risu_aarch64.c
+++ b/risu_aarch64.c
@@ -28,6 +28,9 @@ void advance_pc(void *vuc)
 {
 ucontext_t *uc = vuc;
 uc->uc_mcontext.pc += 4;
+if (ismaster) {
+  report_test_status((void *) uc->uc_mcontext.pc);
+}
 }
 
 static void set_x0(void *vuc, uint64_t x0)
-- 
2.10.2

[Qemu-devel] [RISU PATCH 8/9] new: record_traces.sh helper script

[Alex Bennée]

A simple script to run through a bunch of binaries and generate their
trace files.

Signed-off-by: Alex Bennée 
---
 record_traces.sh | 16 
 1 file changed, 16 insertions(+)
 create mode 100755 record_traces.sh

diff --git a/record_traces.sh b/record_traces.sh
new file mode 100755
index 000..89c62d5
--- /dev/null
+++ b/record_traces.sh
@@ -0,0 +1,16 @@
+#!/bin/sh
+#
+# Record traces
+#
+# A risu helper script to batch process a bunch of binaries and record their 
outputs
+#
+set -e
+
+
+for f in $@; do
+echo "Running risu against $f"
+t="$f.trace"
+./risu --master $f -t $t
+echo "Checking trace file OK"
+./risu $f -t $t
+done
-- 
2.10.2

[Qemu-devel] [RISU PATCH 6/9] risugen: remove grocer's apostrophe from REs

[Alex Bennée]

This also fixes perl-modes confusion about escaped strings.

Signed-off-by: Alex Bennée 
---
 risugen | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/risugen b/risugen
index a604fe5..77a550b 100755
--- a/risugen
+++ b/risugen
@@ -288,7 +288,7 @@ Valid options:
'VMULL' will match 'VMULL A1' and 'VMULL A2' but not
'VMULL_scalar A1'. This is generally what you wanted.
 --not-pattern re[,re...] : exclude patterns matching regular expression.
-   These RE's are applied after the matching pattern which
+   These REs are applied after the matching pattern which
is useful if you want to exclude a specific instruction from
a general set you have excluded.
  --no-fp  : disable floating point: no fp init, randomization etc.
-- 
2.10.2

Re: [Qemu-devel] Livelock with qemu 2.5.0

[Brian Candler]

On 02/12/2016 14:19, Brian Candler wrote:
I am running a VM under qemu 2.5.0/Ubuntu 16.04. The guest VM is also 
Ubuntu 16.04, with ZFS and LXD


Correction: it's btrfs (not zfs).

I re-ran the test with qemu 2.7.0, and this time got a btrfs kernel 
panic in the guest [below].


The guest is running kernel 4.4.0-45, which I can update to 4.4.0-51 and 
retry. But at the moment this looks like a kernel btrfs bug, not a qemu 
bug. Sorry for the noise.


Regards,

Brian.

[  808.782266] BUG: unable to handle kernel
[  808.786126] [ cut here ]
[  808.786152] WARNING: CPU: 2 PID: 25962 at 
/build/linux-kOWph6/linux-4.4.0/fs/btrfs/inode.c:9374 
btrfs_rename2+0x386/0x7b0 [btrfs]()

[  808.786160] BTRFS: Transaction aborted (error -2)
[  808.786160] Modules linked in:
[  808.786161]  binfmt_misc
[  808.786162]  veth
[  808.786163]  ufs
[  808.786163]  qnx4
[  808.786163]  hfsplus
[  808.786164]  hfs
[  808.786164]  minix
[  808.786165]  ntfs
[  808.786165]  msdos
[  808.786166]  jfs
[  808.786166]  xfs
[  808.786167]  xt_CHECKSUM
[  808.786168]  iptable_mangle
[  808.786168]  xt_comment
[  808.786169]  xt_tcpudp
[  808.786169]  bridge
[  808.786170]  stp
[  808.786170]  llc
[  808.786170]  ip6table_filter
[  808.786171]  ip6t_MASQUERADE
[  808.786171]  nf_nat_masquerade_ipv6
[  808.786172]  ip6table_nat
[  808.786173]  nf_conntrack_ipv6
[  808.786173]  nf_defrag_ipv6
[  808.786173]  nf_nat_ipv6
[  808.786174]  ip6_tables
[  808.786174]  iptable_filter
[  808.786175]  ipt_MASQUERADE
[  808.786175]  nf_nat_masquerade_ipv4
[  808.786176]  iptable_nat
[  808.786176]  nf_conntrack_ipv4
[  808.786177]  nf_defrag_ipv4
[  808.786177]  nf_nat_ipv4
[  808.786178]  nf_nat
[  808.786178]  nf_conntrack
[  808.786179]  ip_tables
[  808.786179]  x_tables
[  808.786180]  ppdev
[  808.786180]  kvm_intel
[  808.786181]  input_leds
[  808.786181]  kvm
[  808.786182]  irqbypass
[  808.786182]  joydev
[  808.786182]  parport_pc
[  808.786183]  serio_raw
[  808.786184]  8250_fintek
[  808.786184]  parport
[  808.786185]  mac_hid
[  808.786185]  i2c_piix4
[  808.786185]  ib_iser
[  808.786186]  rdma_cm
[  808.786186]  iw_cm
[  808.786187]  ib_cm
[  808.786187]  ib_sa
[  808.786188]  ib_mad
[  808.786188]  ib_core
[  808.786189]  ib_addr
[  808.786189]  iscsi_tcp
[  808.786190]  libiscsi_tcp
[  808.786190]  libiscsi
[  808.786191]  scsi_transport_iscsi
[  808.786191]  autofs4
[  808.786191]  btrfs
[  808.786192]  raid10
[  808.786192]  raid456
[  808.786193]  async_raid6_recov
[  808.786193]  async_memcpy
[  808.786194]  async_pq
[  808.786194]  async_xor
[  808.786195]  async_tx
[  808.786195]  xor
[  808.786196]  raid6_pq
[  808.786196]  libcrc32c
[  808.786197]  raid1
[  808.786197]  raid0
[  808.786198]  multipath
[  808.786198]  linear
[  808.786199]  cirrus
[  808.786199]  crct10dif_pclmul
[  808.786200]  crc32_pclmul
[  808.786200]  ttm
[  808.786200]  drm_kms_helper
[  808.786201]  aesni_intel
[  808.786201]  syscopyarea
[  808.786202]  sysfillrect
[  808.786202]  aes_x86_64
[  808.786203]  sysimgblt
[  808.786203]  lrw
[  808.786204]  fb_sys_fops
[  808.786204]  gf128mul
[  808.786205]  glue_helper
[  808.786205]  ablk_helper
[  808.786206]  cryptd
[  808.786206]  psmouse
[  808.786206]  drm
[  808.786207]  virtio_scsi
[  808.786207]  pata_acpi
[  808.786208]  floppy

[  808.786211] CPU: 2 PID: 25962 Comm: dpkg Not tainted 4.4.0-45-generic 
#66-Ubuntu
[  808.786212] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), 
BIOS rel-1.9.3-0-ge2fc41e-prebuilt.qemu-project.org 04/01/2014

[  808.786213]  0286
[  808.786213]  39ded762
[  808.786214]  88010184bce0
[  808.786214]  813f1fe3

[  808.786215]  88010184bd28
[  808.786215]  c03e8220
[  808.786216]  88010184bd18
[  808.786216]  81081212

[  808.786217]  880138389538
[  808.786217]  000145e8
[  808.786218]  880115ef1538
[  808.786218]  88016315c418

[  808.786219] Call Trace:
[  808.786224]  [] dump_stack+0x63/0x90
[  808.786227]  [] warn_slowpath_common+0x82/0xc0
[  808.786229]  [] warn_slowpath_fmt+0x5c/0x80
[  808.786246]  [] btrfs_rename2+0x386/0x7b0 [btrfs]
[  808.786249]  [] ? __inode_permission+0x1/0xb0
[  808.786251]  [] vfs_rename+0x58c/0x8e0
[  808.786254]  [] ? 
security_kernel_fw_from_file+0x40/0x70

[  808.786255]  [] SyS_rename+0x39f/0x3c0
[  808.786259]  [] entry_SYSCALL_64_fastpath+0x16/0x71
[  808.786260] ---[ end trace 9067db09b6daeaab ]---
[  808.786262] BTRFS: error (device sda1) in btrfs_rename:9374: errno=-2 
No such entry

[  808.786263] BTRFS info (device sda1): forced readonly
[  808.961571] paging request
[  808.963405]  at fe58
[  808.965154] IP: [] __btrfs_unlink_inode+0x1c/0x470 
[btrfs]

[  808.967516] PGD 1e0d067 PUD 1e0f067 PMD 0
[  808.969640] Oops:  [#1] SMP
[  808.971654] Modules linked in: binfmt_misc veth ufs qnx4 hfsplus hfs 
minix ntfs msdos jfs xfs xt_CHECKSUM iptable_mangle xt_comment xt_tcpudp 
bridge stp llc ip6table_filt

Re: [Qemu-devel] [PATCH v5 00/10] tcg mips64 and mips r6 improvements

[Aurelien Jarno]

On 2016-12-01 21:51, Jin Guojie wrote:
> Changes in v5:
>   * Update against master(v2.8.0-rc2)
>   * Fix a bug: 64-bit big-endian guests hang on mips64 little-endian
>   hosts, and vice versa. This bug was first introduced in v2 patch,
>   due to obvious misuse of ret/arg registers in tcg_out_bswap64().
> 
> tcg_out_opc_reg(s, OPC_DSBH, ret, 0, arg);
>   - tcg_out_opc_reg(s, OPC_DSHD, ret, 0, arg);
>   + tcg_out_opc_reg(s, OPC_DSHD, ret, 0, ret);
> 
>   * Fix a style problem: checkpatch.pl forbids 'extern' to be used in .c.
> 
>   ERROR: externs should be avoided in .c files
>   #28: FILE: tcg/mips/tcg-target.inc.c:39:
>   +extern int link_error(void);
> 
>   Simply comment the type identifier to pass the check.
> 
>   * Tested successfully on following machines:
>   
> | HOST| qemu-system | Debian ISO  |
> |-|
> | mips 32 le  |i386 |i386 |
> | mips 32 le  |x86_64   |i386 |
> | mips 32 le  |x86_64   |amd64|
> | mips 64 le  |i386 |i386 |
> | mips 64 le  |x86_64   |i386 |
> | mips 64 le  |x86_64   |amd64|
> | mips 64 le  |  mips 64 be |  mips 64 be |
> |-|
> | mips 32 be  |i386 | i386|
> | mips 32 be  |x86_64   | i386|
> | mips 32 be  |x86_64   | amd64   |
> | mips 64 be  |i386 | i386|
> | mips 64 be  |x86_64   | i386|
> | mips 64 be  |x86_64   | amd64   |
> | mips n32 be |386  | i386|
> | mips n32 be |x86_64   | i386|
> | mips n32 be |x86_64   | amd64   |
> 
> (No plan to test MIPS R6 in this patch.)
> 
>   Summary of changes from v4:
> 
>   | tcg-mips: Support 64-bit opcodes   | Fix tcg_out_bswap64() |
>   | tcg-mips: Adjust qemu_ld/st for mips64 | Fix a style problem   |

Thanks for the new version. I just gave it a try on a mips64 le guest,
and I confirm this fixes booting mips64 be and ppc64 guests. I'll try to
do more tests on mips be / mips le / mips64 le over the week-end.

Aurelien

-- 
Aurelien Jarno  GPG: 4096R/1DDD8C9B
aurel...@aurel32.net http://www.aurel32.net

[Qemu-devel] QEMU Advent Calendar 2016

[Ingve Vormestrand]

Hi! First of all, QEMU rocks!

I'm from the IncludeOS project, and QEMU is, without a doubt, one of our
favourite tools! (If you haven't heard about IncludeOS, it is an open
source unikernel operating system written in modern C++ (you can find out
more about the project at http://includeos.org/ and
https://github.com/hioa-cs/IncludeOS or see a very well received
presentation at https://www.youtube.com/watch?v=t4etEwG2_LY from this
year's CppCon).

One of the demo applications for IncludeOS is Acorn, a self-contained web
server that includes its own OS, can serve static content, handle REST
requests and provide an interactive real-time dashboard. The source is
available at https://github.com/includeos/acorn and there is a live preview
running at http://acorn2.unofficial.includeos.io/

We frequently distribute Acorn demos as QEMU images, and we thought that
maybe this could be a candidate for inclusion in your advent calendar?

Let me know if this is of interest to you or if you need further
information!

Ingve

Re: [Qemu-devel] [PATCH v5 00/10] tcg mips64 and mips r6 improvements

[James Hogan]

On Fri, Dec 02, 2016 at 11:16:41AM +, James Hogan wrote:
> Hi,
> 
> On Thu, Dec 01, 2016 at 09:51:59PM +0800, Jin Guojie wrote:
> >   * Tested successfully on following machines:
> >   
> > | HOST| qemu-system | Debian ISO  |
> > |-|
> > | mips 32 le  |i386 |i386 |
> > | mips 32 le  |x86_64   |i386 |
> > | mips 32 le  |x86_64   |amd64|
> > | mips 64 le  |i386 |i386 |
> > | mips 64 le  |x86_64   |i386 |
> > | mips 64 le  |x86_64   |amd64|
> > | mips 64 le  |  mips 64 be |  mips 64 be |
> > |-|
> > | mips 32 be  |i386 | i386|
> > | mips 32 be  |x86_64   | i386|
> > | mips 32 be  |x86_64   | amd64   |
> > | mips 64 be  |i386 | i386|
> > | mips 64 be  |x86_64   | i386|
> > | mips 64 be  |x86_64   | amd64   |
> > | mips n32 be |386  | i386|
> > | mips n32 be |x86_64   | i386|
> > | mips n32 be |x86_64   | amd64   |
> > 
> > (No plan to test MIPS R6 in this patch.)
> 
> FYI I tried booting one of Aurelien's x86_64 debian QEMU images on a
> mipsel64r6 (P6600) using this, but it seems to get stuck somewhere in
> GRUB before booting guest kernel. I did see it saying loading kernel and
> ramdisk, but it seemed to return to the bios & grub and just get stuck.
> 
> I'm now attempting to reproduce in QEMU I6400 host, which is faster than
> FPGA, but having the usual faff trying to get nested video output via
> SDL/fbcon working.

Inside QEMU mipsel64r6 (emulating I6400) it eventually reaches guest
linux, but udev initiated modprobes time out. So its clearly got quite
far without any breakage becoming apparent (all the way to userland),
and I suspect on P6600 (FPGA) its just way too slow.

Cheers
James


signature.asc
Description: Digital signature

Re: [Qemu-devel] [Qemu-ppc] [RFCv2 12/12] ppc: Rework CPU compatibility testing across migration

[Greg Kurz]

On Wed, 16 Nov 2016 09:17:55 +1100
David Gibson  wrote:

> Migrating between different CPU versions is quite complicated for ppc.
> A long time ago, we ensure identical CPU versions at either end by checking
> the PVR had the same value.  However, this breaks under KVM HV, because
> we always have to use the host's PVR - it's not virtualized.  That would
> mean we couldn't migrate between hosts with different PVRs, even if the
> CPUs are close enough to compatible in practice (sometimes identical cores
> with different surrounding logic have different PVRs, so this happens in
> practice quite often).
> 
> So, we removed the PVR check, but instead checked that several flags
> indicating supported instructions matched.  This turns out to be a bad
> idea, because those instruction masks are not architected information, but
> essentially a TCG implementation detail.  So changes to qemu internal CPU
> modelling can break migration - this happened between qemu-2.6 and
> qemu-2.7.
> 
> Modern server-class CPUs can be placed into compatibility modes.  Now that
> we're handling those properly, we finally have the information to sanely
> deal with CPU compatibility across migration.
> 
> This patch bumps the migration version number for the ppc CPU removing the
> instruction mask field (and some other unwise VMSTATE_EQUAL checks), and
> adding the compatibility PVR to the migration stream.
> 

Things have changed since you posted this RFC:

commit 16a2497bd44cac1856e259654fd304079bd1dcdc
Author: David Gibson 
Date:   Mon Nov 21 16:28:12 2016 +1100

target-ppc: Fix CPU migration from qemu-2.6 <-> later versions

and

commit 146c11f16f12dbfea62cbd7f865614bb6fcbc6b5
Author: David Gibson 
Date:   Mon Nov 21 16:29:30 2016 +1100

target-ppc: Allow eventual removal of old migration mistakes

I guess that the version bumping isn't necessary anymore if we keep these.

I'll assume yes and rebase this patch against current master, simply dropping
the version bumping and related lines.

> We consider the CPUs compatible for migration if:
> * The source was running in a compatibility mode which the destination
>   supports
> OR  * The source has a PVR matching the same qemu CPU class as the
>   destination, either an exact match or an approximate match determined
>   by the cpu class's pvr_match hook.
> 
> Signed-off-by: David Gibson 
> ---
>  target-ppc/machine.c | 87 
> +++-
>  1 file changed, 79 insertions(+), 8 deletions(-)
> 
> diff --git a/target-ppc/machine.c b/target-ppc/machine.c
> index e43cb6c..25a30d5 100644
> --- a/target-ppc/machine.c
> +++ b/target-ppc/machine.c
> @@ -8,6 +8,7 @@
>  #include "helper_regs.h"
>  #include "mmu-hash64.h"
>  #include "migration/cpu.h"
> +#include "qapi/error.h"
>  
>  static int cpu_load_old(QEMUFile *f, void *opaque, int version_id)
>  {
> @@ -163,6 +164,30 @@ static void cpu_pre_save(void *opaque)
>  }
>  }
>  
> +/*
> + * Determine if a given PVR is a "close enough" match to the CPU
> + * object.  For TCG and KVM PR it would probably be sufficient to
> + * require an exact PVR match.  However for KVM HV the user is
> + * restricted to a PVR exactly matching the host CPU.  The correct way
> + * to handle this is to put the guest into an architected
> + * compatibility mode.  However, to allow a more forgiving transition
> + * and migration from before this was widely done, we allow migration
> + * between sufficiently similar PVRs, as determined by the CPU class's
> + * pvr_match() hook.
> + */
> +static bool pvr_match(PowerPCCPU *cpu, uint32_t pvr)
> +{
> +PowerPCCPUClass *pcc = POWERPC_CPU_GET_CLASS(cpu);
> +
> +if (pvr == pcc->pvr) {
> +return true;
> +}
> +if (pcc->pvr_match) {
> +return pcc->pvr_match(pcc, pvr);
> +}
> +return false;
> +}
> +
>  static int cpu_post_load(void *opaque, int version_id)
>  {
>  PowerPCCPU *cpu = opaque;
> @@ -171,10 +196,31 @@ static int cpu_post_load(void *opaque, int version_id)
>  target_ulong msr;
>  
>  /*
> - * We always ignore the source PVR. The user or management
> - * software has to take care of running QEMU in a compatible mode.
> + * If we're operating in compat mode, we should be ok as long as
> + * the destination supports the same compatiblity mode.
> + *
> + * Otherwise, however, we require that the destination has exactly
> + * the same CPU model as the source.
>   */
> -env->spr[SPR_PVR] = env->spr_cb[SPR_PVR].default_value;
> +
> +#if defined(TARGET_PPC64)
> +if (cpu->compat_pvr) {
> +Error *local_err = NULL;
> +
> +ppc_set_compat(cpu, cpu->compat_pvr, &local_err);

This calls cpu_synchronize_state(CPU(cpu)) and trashes the registers. This
is the root cause behind the program interrupts I mentioned in another mail.

Adding a sync_needed boolean argument to ppc_set_compat() seems to be enough
to get this working. So I'll just do that and rerun the te

Re: [Qemu-devel] [PATCH] scripts: add "git.orderfile" for ordering diff hunks by pathname patterns

[Fam Zheng]

On Fri, 12/02 15:33, Max Reitz wrote:
> On 02.12.2016 11:40, Laszlo Ersek wrote:
> > When passed to git-diff (and to every other git command producing diffs
> > and/or diffstats) with "-O" or "diff.orderFile", this list of patterns
> > will place the more declarative / abstract hunks first, while changes to
> > imperative code / details will be near the end of the patches. This saves
> > on scrolling / searching and makes for easier reviewing.
> > 
> > We intend to advise contributors in the Wiki to run
> > 
> >   git config diff.orderFile scripts/git.orderfile
> > 
> > once, as part of their initial setup, before formatting their first (or,
> > for repeat contributors, next) patches.
> > 
> > See the "-O" option and the "diff.orderFile" configuration variable in
> > git-diff(1) and git-config(1).
> > 
> > Cc: "Michael S. Tsirkin" 
> > Cc: Eric Blake 
> > Cc: Fam Zheng 
> > Cc: Gerd Hoffmann 
> > Cc: John Snow 
> > Cc: Stefan Hajnoczi 
> > Signed-off-by: Laszlo Ersek 
> > ---
> > 
> > Notes:
> > I think I managed to incorporate everyone's feedback!
> > 
> >  scripts/git.orderfile | 15 +++
> >  1 file changed, 15 insertions(+)
> >  create mode 100644 scripts/git.orderfile
> > 
> > diff --git a/scripts/git.orderfile b/scripts/git.orderfile
> > new file mode 100644
> > index ..600a2e4fc540
> > --- /dev/null
> > +++ b/scripts/git.orderfile
> > @@ -0,0 +1,15 @@
> > +*.txt
> 
> We also have *.md. (OK, OK, we have a single docs/bitmaps.md, but maybe
> it's going to be more in the future. :-))

Maybe just insert 'docs/*' here? I'm not sure if it works for the subdirs,
though.

Fam

> 
> Max
> 
> > +configure
> > +GNUmakefile
> > +makefile
> > +Makefile
> > +*.mak
> > +qapi-schema*.json
> > +qapi/*.json
> > +include/qapi/visitor.h
> > +include/qapi/visitor-impl.h
> > +scripts/qapi.py
> > +scripts/*.py
> > +*.h
> > +qapi/qapi-visit-core.c
> > +*.c
> > 
> 
> 

Re: [Qemu-devel] [PATCH] scripts: add "git.orderfile" for ordering diff hunks by pathname patterns

[Max Reitz]

On 02.12.2016 11:40, Laszlo Ersek wrote:
> When passed to git-diff (and to every other git command producing diffs
> and/or diffstats) with "-O" or "diff.orderFile", this list of patterns
> will place the more declarative / abstract hunks first, while changes to
> imperative code / details will be near the end of the patches. This saves
> on scrolling / searching and makes for easier reviewing.
> 
> We intend to advise contributors in the Wiki to run
> 
>   git config diff.orderFile scripts/git.orderfile
> 
> once, as part of their initial setup, before formatting their first (or,
> for repeat contributors, next) patches.
> 
> See the "-O" option and the "diff.orderFile" configuration variable in
> git-diff(1) and git-config(1).
> 
> Cc: "Michael S. Tsirkin" 
> Cc: Eric Blake 
> Cc: Fam Zheng 
> Cc: Gerd Hoffmann 
> Cc: John Snow 
> Cc: Stefan Hajnoczi 
> Signed-off-by: Laszlo Ersek 
> ---
> 
> Notes:
> I think I managed to incorporate everyone's feedback!
> 
>  scripts/git.orderfile | 15 +++
>  1 file changed, 15 insertions(+)
>  create mode 100644 scripts/git.orderfile
> 
> diff --git a/scripts/git.orderfile b/scripts/git.orderfile
> new file mode 100644
> index ..600a2e4fc540
> --- /dev/null
> +++ b/scripts/git.orderfile
> @@ -0,0 +1,15 @@
> +*.txt

We also have *.md. (OK, OK, we have a single docs/bitmaps.md, but maybe
it's going to be more in the future. :-))

Max

> +configure
> +GNUmakefile
> +makefile
> +Makefile
> +*.mak
> +qapi-schema*.json
> +qapi/*.json
> +include/qapi/visitor.h
> +include/qapi/visitor-impl.h
> +scripts/qapi.py
> +scripts/*.py
> +*.h
> +qapi/qapi-visit-core.c
> +*.c
> 




signature.asc
Description: OpenPGP digital signature

[Qemu-devel] Livelock with qemu 2.5.0

[Brian Candler]

I am running a VM under qemu 2.5.0/Ubuntu 16.04. The guest VM is also 
Ubuntu 16.04, with ZFS and LXD, and I am hitting it hard with I/O, 
concurrently building multiple LXD containers.  Previously it was 
serialized to build one container at a time, and it was fine.


With the concurrent builds I got to a point where the VM livelocked:

- CPU utilisation on the host is 100%:

  PID USER  PR  NIVIRTRESSHR S  %CPU %MEM TIME+ COMMAND
20321 libvirt+  20   0 18.864g 0.014t  23032 S 100.3 45.7 603:33.30 
qemu-system-x86


(host has 32GB of RAM, plenty of headroom there)

- VM does not respond to ping

- I can connect to VM's VNC console, and it shows the login prompt, but 
keystrokes are ignored


I attached gdb and took backtraces (attached)

I imagine I could have hit a problem either with qemu, or with the guest 
kernel itself. So next I'm going to try to replicate this with qemu 
2.7.0.  But I thought I'd post the backtraces here first to see if 
anyone has any comments/thoughts about what's going on.


Thanks,

Brian.

P.S. I'm not subscribed to qemu-devel so please CC me directly.

# gdb -p 20321 /usr/bin/qemu-system-x86_64
GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.04) 7.11.1
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
.
Find the GDB manual and other documentation resources online at:
.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/bin/qemu-system-x86_64...(no debugging symbols 
found)...done.
Attaching to program: /usr/bin/qemu-system-x86_64, process 20321
[New LWP 20322]
[New LWP 20325]
[New LWP 20326]
[New LWP 20327]
[New LWP 20328]
[New LWP 20329]
[New LWP 20330]
[New LWP 20331]
[New LWP 20332]
[New LWP 20334]

warning: .dynamic section for "/usr/lib/x86_64-linux-gnu/librbd.so.1" is not at 
the expected address (wrong library or version mismatch?)
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
0x7f9b6957fc21 in __GI_ppoll (fds=0x559d5388e080, nfds=10, 
timeout=, sigmask=0x0)
at ../sysdeps/unix/sysv/linux/ppoll.c:50
50  ../sysdeps/unix/sysv/linux/ppoll.c: No such file or directory.
(gdb) bt
#0  0x7f9b6957fc21 in __GI_ppoll (fds=0x559d5388e080, nfds=10, 
timeout=,
sigmask=0x0) at ../sysdeps/unix/sysv/linux/ppoll.c:50
#1  0x559d516ec8d9 in qemu_poll_ns ()
#2  0x559d516ec36a in main_loop_wait ()
#3  0x559d5146f2d4 in main ()
(gdb) info threads
  Id   Target Id Frame
* 1Thread 0x7f9b6e86bc00 (LWP 20321) "qemu-system-x86" 0x7f9b6957fc21 
in __GI_ppoll (
fds=0x559d5388e080, nfds=10, timeout=, sigmask=0x0)
at ../sysdeps/unix/sysv/linux/ppoll.c:50
  2Thread 0x7f9b63727700 (LWP 20322) "qemu-system-x86" syscall ()
at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
  3Thread 0x7f9b4933d700 (LWP 20325) "qemu-system-x86" 0x7f9b69581357 
in ioctl ()
at ../sysdeps/unix/syscall-template.S:84
  4Thread 0x7f9b48b3c700 (LWP 20326) "qemu-system-x86" 0x7f9b69581357 
in ioctl ()
at ../sysdeps/unix/syscall-template.S:84
  5Thread 0x7f9b3bfff700 (LWP 20327) "qemu-system-x86" 0x7f9b69581357 
in ioctl ()
at ../sysdeps/unix/syscall-template.S:84
  6Thread 0x7f9b3b7fe700 (LWP 20328) "qemu-system-x86" 0x7f9b69581357 
in ioctl ()
at ../sysdeps/unix/syscall-template.S:84
  7Thread 0x7f9b3affd700 (LWP 20329) "qemu-system-x86" 0x7f9b69581357 
in ioctl ()
at ../sysdeps/unix/syscall-template.S:84
  8Thread 0x7f9b3a7fc700 (LWP 20330) "qemu-system-x86" 0x7f9b69581357 
in ioctl ()
at ../sysdeps/unix/syscall-template.S:84
  9Thread 0x7f9b39ffb700 (LWP 20331) "qemu-system-x86" 0x7f9b69581357 
in ioctl ()
at ../sysdeps/unix/syscall-template.S:84
  10   Thread 0x7f9b397fa700 (LWP 20332) "qemu-system-x86" 0x7f9b69581357 
in ioctl ()
at ../sysdeps/unix/syscall-template.S:84
  11   Thread 0x7f979bbff700 (LWP 20334) "qemu-system-x86" 
pthread_cond_wait@@GLIBC_2.3.2 ()
at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
(gdb) thread 2
[Switching to thread 2 (Thread 0x7f9b63727700 (LWP 20322))]
#0  syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
38  ../sysdeps/unix/sysv/linux/x86_64/syscall.S: No such file or directory.
(gdb) bt
#0  syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
#1  0x559d5175d101 in qemu_event_wait ()
#2  0x559d5176b5be in ?? ()
#3  0x7f9b6985570a in start_thread (arg=0x7f9b63727700) at 
pthread_c

Re: [Qemu-devel] [PATCH 02/20] blockjob: introduce .drain callback for jobs

[Vladimir Sementsov-Ogievskiy]

27.10.2016 13:48, Paolo Bonzini wrote:

This is required to decouple block jobs from running in an
AioContext.  With multiqueue block devices, a BlockDriverState
does not really belong to a single AioContext.

The solution is to first wait until all I/O operations are
complete; then loop in the main thread for the block job to
complete entirely.

Signed-off-by: Paolo Bonzini 
---


[...]


@@ -413,14 +409,21 @@ static int block_job_finish_sync(BlockJob *job,
  assert(blk_bs(job->blk)->job == job);
  
  block_job_ref(job);

+
  finish(job, &local_err);
  if (local_err) {
  error_propagate(errp, local_err);
  block_job_unref(job);
  return -EBUSY;
  }
+/* block_job_drain calls block_job_enter, and it should be enough to
+ * induce progress until the job completes or moves to the main thread.
+*/


Hi Paolo!

Looks like I have a problem with this. block_job_drain enters the job 
only if job.busy = false. But what if job yielded with busy = true?


My case is the following: in the job I call co_aio_sleep_ns() for some 
time without setting job.busy to false, and it looks like timer doesn't 
work while we are in "while() { block_job_drain() }" loop. If I just set 
"job.busy = false" and "job.busy = true" around co_aio_sleep_ns() all 
start to work.


I don't want set job.busy to false, because actually job is working - 
several additional coroutines do their work, only the main one (job.co) 
do nothing. I can remove timer, and make other coroutines wake up the 
main one when it needed, and, anyway it looks like better way..


But the question is: is it ok, that we can't use sleep timer in the job, 
without setting busy = true? Is it right that only io can wake up block 
job coroutine, if it yielded without setting busy=false?



+while (!job->deferred_to_main_loop && !job->completed) {
+block_job_drain(job);
+}
  while (!job->completed) {
-aio_poll(block_job_get_aio_context(job), true);
+aio_poll(qemu_get_aio_context(), true);
  }
  ret = (job->cancelled && job->ret == 0) ? -ECANCELED : job->ret;
  block_job_unref(job);




--
Best regards,
Vladimir

Re: [Qemu-devel] [qemu patch V3 2/2] kvmclock: reduce kvmclock difference on migration

[Eduardo Habkost]

On Thu, Dec 01, 2016 at 12:39:03PM -0200, Marcelo Tosatti wrote:
> On Wed, Nov 30, 2016 at 11:09:28AM -0200, Eduardo Habkost wrote:
> > On Tue, Nov 29, 2016 at 09:54:29PM -0200, Marcelo Tosatti wrote:
> > > On Tue, Nov 29, 2016 at 10:34:05AM -0200, Eduardo Habkost wrote:
> > > > On Tue, Nov 29, 2016 at 08:56:00AM -0200, Marcelo Tosatti wrote:
> > > > > On Mon, Nov 28, 2016 at 03:12:01PM -0200, Eduardo Habkost wrote:
> > > > > > On Mon, Nov 28, 2016 at 02:45:24PM -0200, Marcelo Tosatti wrote:
> > > > > > > On Mon, Nov 28, 2016 at 12:13:22PM -0200, Eduardo Habkost wrote:
> > > > > > > > Sorry for not noticing the following issues on the previous
> > > > > > > > reviews. I was only paying attention to the vmstate and machine
> > > > > > > > code:
> > > > > > > > 
> > > > > > > > On Mon, Nov 21, 2016 at 08:50:04AM -0200, Marcelo Tosatti wrote:
> > > > > > > > > Check for KVM_CAP_ADJUST_CLOCK capability 
> > > > > > > > > KVM_CLOCK_TSC_STABLE, which
> > > > > > > > > indicates that KVM_GET_CLOCK returns a value as seen by the 
> > > > > > > > > guest at
> > > > > > > > > that moment.
> > > > > > > > > 
> > > > > > > > > For new machine types, use this value rather than reading 
> > > > > > > > > from guest memory.
> > > > > > > > > 
> > > > > > > > > This reduces kvmclock difference on migration from 5s to 0.1s
> > > > > > > > > (when max_downtime == 5s).
> > > > > > > > > 
> > > > > > > > > Signed-off-by: Marcelo Tosatti 
> > > > > > > > > 
> > > > > > > > > ---
> > > > > > > > >  hw/i386/kvm/clock.c|  107 
> > > > > > > > > ++---
> > > > > > > > >  include/hw/i386/pc.h   |5 ++
> > > > > > > > >  target-i386/kvm.c  |7 +++
> > > > > > > > >  target-i386/kvm_i386.h |1 
> > > > > > > > >  4 files changed, 106 insertions(+), 14 deletions(-)
> > > > > > > > > 
> > > > > > > > > v2: 
> > > > > > > > > - improve variable names (Juan)
> > > > > > > > > - consolidate code on kvm_get_clock function (Paolo)
> > > > > > > > > - return mach_use_reliable_get_clock from needed function 
> > > > > > > > > (Paolo)
> > > > > > > > > v3: 
> > > > > > > > > - simplify check for src_use_reliable_get_clock (Eduardo)
> > > > > > > > > - move src_use_reliable_get_clock initialization to realize 
> > > > > > > > > (Eduardo)
> > > > > > > > > 
> > > > > > > > > 
> > > > > > > > > Index: qemu-mig-advance-clock/hw/i386/kvm/clock.c
> > > > > > > > > ===
> > > > > > > > > --- qemu-mig-advance-clock.orig/hw/i386/kvm/clock.c   
> > > > > > > > > 2016-11-17 15:07:11.220632761 -0200
> > > > > > > > > +++ qemu-mig-advance-clock/hw/i386/kvm/clock.c
> > > > > > > > > 2016-11-17 15:11:51.372048640 -0200
> > > > > > > > > @@ -36,6 +36,12 @@
> > > > > > > > >  
> > > > > > > > >  uint64_t clock;
> > > > > > > > >  bool clock_valid;
> > > > > > > > > +
> > > > > > > > > +/* whether machine supports reliable KVM_GET_CLOCK */
> > > > > > > > > +bool mach_use_reliable_get_clock;
> > > > > > > > > +
> > > > > > > > > +/* whether source host supported reliable KVM_GET_CLOCK 
> > > > > > > > > */
> > > > > > > > > +bool src_use_reliable_get_clock;
> > > > > > > > >  } KVMClockState;
> > > > > > > > >  
> > > > > > > > >  struct pvclock_vcpu_time_info {
> > > > > > > > > @@ -81,6 +87,19 @@
> > > > > > > > >  return nsec + time.system_time;
> > > > > > > > >  }
> > > > > > > > >  
> > > > > > > > > +static uint64_t kvm_get_clock(void)
> > > > > > > > > +{
> > > > > > > > > +struct kvm_clock_data data;
> > > > > > > > > +int ret;
> > > > > > > > > +
> > > > > > > > > +ret = kvm_vm_ioctl(kvm_state, KVM_GET_CLOCK, &data);
> > > > > > > > > +if (ret < 0) {
> > > > > > > > > +fprintf(stderr, "KVM_GET_CLOCK failed: %s\n", 
> > > > > > > > > strerror(ret));
> > > > > > > > > +abort();
> > > > > > > > > +}
> > > > > > > > > +return data.clock;
> > > > > > > > > +}
> > > > > > > > > +
> > > > > > > > >  static void kvmclock_vm_state_change(void *opaque, int 
> > > > > > > > > running,
> > > > > > > > >   RunState state)
> > > > > > > > >  {
> > > > > > > > > @@ -91,15 +110,36 @@
> > > > > > > > 
> > > > > > > > Can you please use "diff -p" on your patches?
> > > > > > > > 
> > > > > > > > >  
> > > > > > > > >  if (running) {
> > > > > > > > >  struct kvm_clock_data data = {};
> > > > > > > > > -uint64_t time_at_migration = 
> > > > > > > > > kvmclock_current_nsec(s);
> > > > > > > > > +uint64_t pvclock_via_mem = 0;
> > > > > > > > >  
> > > > > > > > > -s->clock_valid = false;
> > > > > > > > > +/* local (running VM) restore */
> > > > > > > > > +if (s->clock_valid) {
> > > > > > > > > +/*
> > > > > > > > > + * if host does not support reliable 
> > > > > > > > > KVM_GET_CLOCK,
> > > > > > > > > + * read kvmclock value from memory
> > >

Re: [Qemu-devel] [PATCH v4 2/7] fw-cfg: turn FW_CFG_FILE_SLOTS into a device property

[Gerd Hoffmann]

  Hi,

> Please check one of the links in the blurb, under which Paolo noted that
> we're already above the limit in the worst (theoretical) case.

Oh, ok.  That changes the picture.

> In practice they don't hit the limit, indeed.

But creating such a case being possible (even if unlikely) is reason
enough to care, from a security point of view.

cheers,
  Gerd

Re: [Qemu-devel] [PATCH] scripts: add "git.orderfile" for ordering diff hunks by pathname patterns

[Gerd Hoffmann]

> # Apply this diff order to your git configuration with the command
> #
> #   git config diff.orderFile scripts/git.orderfile

Looks good to me.

cheers,
  Gerd

Re: [Qemu-devel] [PATCH v4 7/7] hw/i386/pc_q35: advertise broadcast SMI if VCPU hotplug is turned off

[Igor Mammedov]

On Thu, 1 Dec 2016 21:42:58 +0100
Laszlo Ersek  wrote:

> On 12/01/16 20:13, Eduardo Habkost wrote:
> > On Thu, Dec 01, 2016 at 06:06:24PM +0100, Laszlo Ersek wrote:  
> >> For the time being, we cannot handle SMIs in OVMF if VCPUs can show up
> >> after boot. Otherwise, advertise ICH9_LPC_SMI_F_BROADCAST.
> >>
> >> Implement this generally, by introducing a new PCMachineClass method,
> >> namely get_smi_host_features(), and implement the above logic for
> >> pc-q35-2.9 and later. The idea is that future machine types might want to
> >> calculate (the same or different) SMI host features from different
> >> information, and that shouldn't affect earlier machine types.
> >>
> >> In turn, validating guest feature requests (inter-feature dependencies)
> >> should be possible purely based on the available host feature set. For
> >> example, in the future we might enforce that the guest select
> >> ICH9_LPC_SMI_F_VCPU_PARKING as a prerequisite for
> >> ICH9_LPC_SMI_F_BROADCAST, but only if the machine type itself advertises
> >> ICH9_LPC_SMI_F_VCPU_PARKING.
> >>
> >> Cc: "Michael S. Tsirkin" 
> >> Cc: Eduardo Habkost 
> >> Cc: Gerd Hoffmann 
> >> Cc: Igor Mammedov 
> >> Cc: Paolo Bonzini 
> >> Signed-off-by: Laszlo Ersek 
> >> ---
> >>  include/hw/i386/pc.h |  1 +
> >>  hw/i386/pc_q35.c | 24 +++-
> >>  2 files changed, 24 insertions(+), 1 deletion(-)
> >>
> >> diff --git a/include/hw/i386/pc.h b/include/hw/i386/pc.h
> >> index 430735e501dd..e164947116b6 100644
> >> --- a/include/hw/i386/pc.h
> >> +++ b/include/hw/i386/pc.h
> >> @@ -116,10 +116,11 @@ struct PCMachineClass {
> >>  /*< public >*/
> >>  
> >>  /* Methods: */
> >>  HotplugHandler *(*get_hotplug_handler)(MachineState *machine,
> >> DeviceState *dev);
> >> +uint64_t (*get_smi_host_features)(void);  
> > 
> > I'd prefer to encode the differences between machine-types as
> > data, instead of code, to make introspection easier in the
> > future. Is it possible to encode this in a simple PCMachineClass
> > struct data field or (even bettter) a QOM property?
> >   
> 
> I don't know how else to capture the (smp_cpus == max_cpus) question,
> for saying that "this machine type supports SMI broadcast, as long as
> VCPU hotplug is disabled in the configuration".
(smp_cpus == max_cpus) doesn't mean that CPU hotplug is disabled
(it actually can't be disabled at all).

In addition, it's possible to start machine with
 -smp 1,sockets=2,max_cpus=2 -device mycputype,socket=2

where all cpus will be coldpugged
It would be better to use PCMachineState.boot_cpus which contains
present cpus count.

I'd drop hotplug check (as usecase is broken in many places anyway)
and even won't touch PCMachineState, instead add a property like
ICH9_LPC.enable_smi_broadcast(on by default) and disable it for
old machine types using compat props.

We can work on making CPU hotplug and OVMF work in follow up patches.

> Technically we could give PCMC two new (data) fields, a host features
> bitmap for when VCPU hotplug is enabled, and another for when VCPU
> hotplug is disabled. Then board code would take the right field and pass
> it on to ich9_lpc_pm_init().
> 
> Thanks
> Laszlo

Re: [Qemu-devel] [PATCH] scripts: add "git.orderfile" for ordering diff hunks by pathname patterns

[Laszlo Ersek]

On 12/02/16 11:59, Gerd Hoffmann wrote:
>>   git config diff.orderFile scripts/git.orderfile
> 
>> diff --git a/scripts/git.orderfile b/scripts/git.orderfile
>> new file mode 100644
>> index ..600a2e4fc540
>> --- /dev/null
>> +++ b/scripts/git.orderfile
>> @@ -0,0 +1,15 @@
> 
> Is it possible to have comments in here?  So we can place the git config
> command above into into this file too?

The documentation is silent on this, and I didn't want to experiment, so
I checked the git source code. Yes, comments seem to be supported, see
the prepare_order() function in "diffcore-order.c":

/* cp to ep has one line */
if (*cp == '\n' || *cp == '#')
; /* comment */

How about the following comment:

# Apply this diff order to your git configuration with the command
#
#   git config diff.orderFile scripts/git.orderfile

?

Thanks
Laszlo

>> +*.txt
>> +configure
>> +GNUmakefile
>> +makefile
>> +Makefile
> 
> We also have Makefile.objs and Makefile.target, so "Makefile*" (like the
> first version had IIRC) should work better.
> 
>> +*.mak
>> +qapi-schema*.json
>> +qapi/*.json
>> +include/qapi/visitor.h
>> +include/qapi/visitor-impl.h
>> +scripts/qapi.py
>> +scripts/*.py
>> +*.h
>> +qapi/qapi-visit-core.c
> 
> I guess there is more which could be placed here (i.e. files holding
> infrastructure which makes sense to see first).  But we can easily
> refine that incrementally.  That is the point to have this file in the
> repo in the first place ;)
> 
> cheers,
>   Gerd
> 
> 

Re: [Qemu-devel] [RFC v2 06/20] qdev: Add 'accepted-device-types' property to BusClass

[Cornelia Huck]

On Tue, 29 Nov 2016 12:18:19 -0200
Eduardo Habkost  wrote:

> On Tue, Nov 29, 2016 at 02:57:07PM +0100, Cornelia Huck wrote:
> > On Fri, 25 Nov 2016 20:05:42 -0200
> > Eduardo Habkost  wrote:
> > 
> > > Each bus class will now be aware of the specific device types
> > > that can be plugged on it. That will be useful for:
> > > 
> > > * Runtime check for which devices types can be plugged to the
> > >   machine;
> > > * Validation of query-machines output by automated tests.
> > > 
> > > By default, a single type name is used on all bus instances (set
> > > at BusClass::device_type), but bus instances can override it and
> > > return a different type list.
> > > 
> > > Signed-off-by: Eduardo Habkost 
> > 
> > I've read (well, more skimmed through) the whole patchset and I think
> > we have different cases of "multiple device types for one bus":
> > 
> > - the q35 root bus case later in this patchset, where just a certain
> > instance of the bus can accept multiple types
> > - the case where every instance of a bus may accept multiple types
> > (none currently; but this will be the case for e.g. virtual-css once we
> > have other derivates of CCW_DEVICE than virtio-ccw)
> 
> Are all going to be subclasses of TYPE_CCW_DEVICE, and are all
> TYPE_CCW_DEVICE subclasses going to be accepted by virtual-css?

That's what we plan to do, yes.

> In this case, you could just set BusClass::device_type to
> TYPE_CCW_DEVICE on virtual-css-bus.

Sounds reasonable.

> 
> > 
> > For the second case, a static initializer for multiple types in the
> > class makes sense;
> 
> If necessary, we can change BusClass::device_type to a
> BusClass::device_types list. I didn't do that because it didn't
> seem necessary (and I would like to encourage buses to encode
> compatiblity using a single type/interface name whenever
> possible).

Most busses will probably allow a single (parent) device type anyway,
so that's fine, I guess.

> 
> 
> >but the first case is a bit hackish. Should there be
> > a generic way to pass a list of types to the individual bus instance's
> > initializers (that could fall back to a static list in the class)?
> 
> Sound good, but I don't know how that generic mechanism could
> look like. Would a:
>   void bus_add_accepted_device_type(BusState *bus, const char *devtype)
> function be enough?

Would we need a remove helper as well (i.e. "this bus normally supports
types a and b, but this instance accepts only a")? It is probably
enough to add it should the need arise some day.

Re: [Qemu-devel] [PATCH v4 5/7] hw/isa/lpc_ich9: add SMI feature negotiation via fw_cfg

[Laszlo Ersek]

On 12/02/16 12:54, Igor Mammedov wrote:
> On Thu,  1 Dec 2016 18:06:22 +0100
> Laszlo Ersek  wrote:
> 
>> Introduce the following fw_cfg files:
>>
>> - "etc/smi/host-features": a little endian uint64_t feature bitmap,
>>   presenting the features known by the host to the guest. Read-only for
>>   the guest.
> 'host' here is a little bit confusing, I'd suggest 'supported-features'
> instead.
> 
>  
>>   The content of this file is calculated by QEMU at startup (the
>>   calculation will be added later). The same machine type is supposed to
>>   expose the same SMI features regardless of QEMU version, hence this file
>>   is not migrated.
>>
>> - "etc/smi/guest-features": a little endian uint64_t feature bitmap,
>>   representing the features the guest would like to request. Read-write
>>   for the guest.
> and to match above 'requested-features'

The names were supposed to follow virtio 1.0, which calls these things
"device features" and "driver features".

Not a big deal anyway, I can update the file names as you suggest.

>>   The guest can freely (re)write this file, it has no direct consequence.
>>   Initial value is zero. A nonzero value causes the SMI-related fw_cfg
>>   files and fields that are under guest influence to be migrated.
>>
>> - "etc/smi/features-ok": contains a uint8_t value, and it is read-only for
>>   the guest. When the guest selects the associated fw_cfg key, the guest
>>   features are validated against the host features. In case of error, the
>>   negotiation doesn't proceed, and the features-ok file remains zero. In
>>   case of success, the features-ok file becomes (uint8_t)1, and the
>>   negotiated features are locked down internally (to which no further
>>   changes are possible until reset).
>>
>>   The initial value is zero.  A nonzero value causes the SMI-related
>>   fw_cfg files and fields that are under guest influence to be migrated.
>>
>> The C-language fields backing the host-features and guest-features files
>> are uint8_t arrays. This is because they carry guest-side representation
>> (our choice is little endian), while VMSTATE_UINT64() assumes / implies
>> host-side endianness for any uint64_t fields. If we migrate a guest
>> between hosts with different endiannesses (which is possible with TCG),
>> then the host-side value is preserved, and the host-side representation is
>> translated. This would be visible to the guest through fw_cfg, unless we
>> used plain byte arrays. So we do.
>>
>> Cc: "Michael S. Tsirkin" 
>> Cc: Gerd Hoffmann 
>> Cc: Igor Mammedov 
>> Cc: Paolo Bonzini 
>> Signed-off-by: Laszlo Ersek 
>> ---
>>  include/hw/i386/ich9.h | 12 +++-
>>  hw/i386/pc_q35.c   |  2 +-
>>  hw/isa/lpc_ich9.c  | 83 
>> +-
>>  3 files changed, 94 insertions(+), 3 deletions(-)
>>
>> diff --git a/include/hw/i386/ich9.h b/include/hw/i386/ich9.h
>> index 5fd7e97d2347..33142eb37252 100644
>> --- a/include/hw/i386/ich9.h
>> +++ b/include/hw/i386/ich9.h
>> @@ -15,11 +15,12 @@
>>  #include "hw/pci/pci_bus.h"
>>  
>>  void ich9_lpc_set_irq(void *opaque, int irq_num, int level);
>>  int ich9_lpc_map_irq(PCIDevice *pci_dev, int intx);
>>  PCIINTxRoute ich9_route_intx_pin_to_irq(void *opaque, int pirq_pin);
>> -void ich9_lpc_pm_init(PCIDevice *pci_lpc, bool smm_enabled);
>> +void ich9_lpc_pm_init(PCIDevice *pci_lpc, bool smm_enabled,
>> +  uint64_t smi_host_features);
>>  I2CBus *ich9_smb_init(PCIBus *bus, int devfn, uint32_t smb_io_base);
>>  
>>  void ich9_generate_smi(void);
>>  void ich9_generate_nmi(void);
>>  
>> @@ -62,10 +63,19 @@ typedef struct ICH9LPCState {
>>   * register contents and IO memory region
>>   */
>>  uint8_t rst_cnt;
>>  MemoryRegion rst_cnt_mem;
>>  
>> +/* SMI feature negotiation via fw_cfg */
>> +uint8_t smi_host_features[8]; /* guest-visible, read-only, little
>> +   * endian uint64_t */
>> +uint8_t smi_guest_features[8];/* guest-visible, read-write, little
>> +   * endian uint64_t */
> how about adding _le suffix to 2 above fields?
> that way it would be easier to read usage places without chance to 
> misinterpret content

Good idea, I will do that.

Thanks!
Laszlo

> 
>> +uint8_t smi_features_ok;  /* guest-visible, read-only; 
>> selecting it
>> +   * triggers feature lockdown */
>> +uint64_t smi_negotiated_features; /* guest-invisible, host endian */
>> +
>>  /* isa bus */
>>  ISABus *isa_bus;
>>  MemoryRegion rcrb_mem; /* root complex register block */
>>  Notifier machine_ready;
>>  
>> diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c
>> index 7fa40e7cbe0e..eb0953bb6b16 100644
>> --- a/hw/i386/pc_q35.c
>> +++ b/hw/i386/pc_q35.c
>> @@ -228,11 +228,11 @@ static void pc_q35_init(MachineState *machine)
>>  /* init basic PC hardware */
>>  pc_basic_device_init(isa_bus, pcms->gsi, &rtc_state

Re: [Qemu-devel] [PATCH v4 5/7] hw/isa/lpc_ich9: add SMI feature negotiation via fw_cfg

[Igor Mammedov]

On Thu,  1 Dec 2016 18:06:22 +0100
Laszlo Ersek  wrote:

> Introduce the following fw_cfg files:
> 
> - "etc/smi/host-features": a little endian uint64_t feature bitmap,
>   presenting the features known by the host to the guest. Read-only for
>   the guest.
'host' here is a little bit confusing, I'd suggest 'supported-features'
instead.

 
>   The content of this file is calculated by QEMU at startup (the
>   calculation will be added later). The same machine type is supposed to
>   expose the same SMI features regardless of QEMU version, hence this file
>   is not migrated.
> 
> - "etc/smi/guest-features": a little endian uint64_t feature bitmap,
>   representing the features the guest would like to request. Read-write
>   for the guest.
and to match above 'requested-features'

 
>   The guest can freely (re)write this file, it has no direct consequence.
>   Initial value is zero. A nonzero value causes the SMI-related fw_cfg
>   files and fields that are under guest influence to be migrated.
> 
> - "etc/smi/features-ok": contains a uint8_t value, and it is read-only for
>   the guest. When the guest selects the associated fw_cfg key, the guest
>   features are validated against the host features. In case of error, the
>   negotiation doesn't proceed, and the features-ok file remains zero. In
>   case of success, the features-ok file becomes (uint8_t)1, and the
>   negotiated features are locked down internally (to which no further
>   changes are possible until reset).
> 
>   The initial value is zero.  A nonzero value causes the SMI-related
>   fw_cfg files and fields that are under guest influence to be migrated.
> 
> The C-language fields backing the host-features and guest-features files
> are uint8_t arrays. This is because they carry guest-side representation
> (our choice is little endian), while VMSTATE_UINT64() assumes / implies
> host-side endianness for any uint64_t fields. If we migrate a guest
> between hosts with different endiannesses (which is possible with TCG),
> then the host-side value is preserved, and the host-side representation is
> translated. This would be visible to the guest through fw_cfg, unless we
> used plain byte arrays. So we do.
> 
> Cc: "Michael S. Tsirkin" 
> Cc: Gerd Hoffmann 
> Cc: Igor Mammedov 
> Cc: Paolo Bonzini 
> Signed-off-by: Laszlo Ersek 
> ---
>  include/hw/i386/ich9.h | 12 +++-
>  hw/i386/pc_q35.c   |  2 +-
>  hw/isa/lpc_ich9.c  | 83 
> +-
>  3 files changed, 94 insertions(+), 3 deletions(-)
> 
> diff --git a/include/hw/i386/ich9.h b/include/hw/i386/ich9.h
> index 5fd7e97d2347..33142eb37252 100644
> --- a/include/hw/i386/ich9.h
> +++ b/include/hw/i386/ich9.h
> @@ -15,11 +15,12 @@
>  #include "hw/pci/pci_bus.h"
>  
>  void ich9_lpc_set_irq(void *opaque, int irq_num, int level);
>  int ich9_lpc_map_irq(PCIDevice *pci_dev, int intx);
>  PCIINTxRoute ich9_route_intx_pin_to_irq(void *opaque, int pirq_pin);
> -void ich9_lpc_pm_init(PCIDevice *pci_lpc, bool smm_enabled);
> +void ich9_lpc_pm_init(PCIDevice *pci_lpc, bool smm_enabled,
> +  uint64_t smi_host_features);
>  I2CBus *ich9_smb_init(PCIBus *bus, int devfn, uint32_t smb_io_base);
>  
>  void ich9_generate_smi(void);
>  void ich9_generate_nmi(void);
>  
> @@ -62,10 +63,19 @@ typedef struct ICH9LPCState {
>   * register contents and IO memory region
>   */
>  uint8_t rst_cnt;
>  MemoryRegion rst_cnt_mem;
>  
> +/* SMI feature negotiation via fw_cfg */
> +uint8_t smi_host_features[8]; /* guest-visible, read-only, little
> +   * endian uint64_t */
> +uint8_t smi_guest_features[8];/* guest-visible, read-write, little
> +   * endian uint64_t */
how about adding _le suffix to 2 above fields?
that way it would be easier to read usage places without chance to misinterpret 
content

> +uint8_t smi_features_ok;  /* guest-visible, read-only; selecting 
> it
> +   * triggers feature lockdown */
> +uint64_t smi_negotiated_features; /* guest-invisible, host endian */
> +
>  /* isa bus */
>  ISABus *isa_bus;
>  MemoryRegion rcrb_mem; /* root complex register block */
>  Notifier machine_ready;
>  
> diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c
> index 7fa40e7cbe0e..eb0953bb6b16 100644
> --- a/hw/i386/pc_q35.c
> +++ b/hw/i386/pc_q35.c
> @@ -228,11 +228,11 @@ static void pc_q35_init(MachineState *machine)
>  /* init basic PC hardware */
>  pc_basic_device_init(isa_bus, pcms->gsi, &rtc_state, !mc->no_floppy,
>   (pcms->vmport != ON_OFF_AUTO_ON), 0xff0104);
>  
>  /* connect pm stuff to lpc */
> -ich9_lpc_pm_init(lpc, pc_machine_is_smm_enabled(pcms));
> +ich9_lpc_pm_init(lpc, pc_machine_is_smm_enabled(pcms), 0);
>  
>  /* ahci and SATA device, for q35 1 ahci controller is built-in */
>  ahci = pci_create_simple_multif

Re: [Qemu-devel] [PATCH v4 2/7] fw-cfg: turn FW_CFG_FILE_SLOTS into a device property

[Laszlo Ersek]

On 12/02/16 12:10, Gerd Hoffmann wrote:
> On Do, 2016-12-01 at 18:06 +0100, Laszlo Ersek wrote:
>> We'd like to raise the value of FW_CFG_FILE_SLOTS. Doing it naively could
>> lead to problems with backward migration: a more recent QEMU (running an
>> older machine type) would allow the guest, in fw_cfg_select(), to select a
>> high key value that is unavailable in the same machine type implemented by
>> the older (target) QEMU.
> 
> I don't think we need this.  fw_cfg changes are guest-visible so they
> must not happen for a given machine type.

Agreed.

> So if current machine types
> don't hit the limit

Please check one of the links in the blurb, under which Paolo noted that
we're already above the limit in the worst (theoretical) case.

In practice they don't hit the limit, indeed.

> that should continue to be the case even if we
> simply raise FW_CFG_FILE_SLOTS.

I'm not trying to make it hard for myself :), so in theory I don't
disagree with the idea. However, do consider backwards migration. (As
noted under the patch, I'm aware that upstream doesn't care, but that
shouldn't be reason enough to reject a patch that does care.)

Let's say we start a guest in the pc-q35-2.8 machtype on a new QEMU
release, as source QEMU host, which has the raised FW_CFG_FILE_SLOTS
value. The guest writes a high key value to the selector register, which
is valid under the raised limit, so the key value is stored (i.e.,
fw_cfg_select() won't store FW_CFG_INVALID in cur_entry).

Then the guest is migrated to the older release QEMU, where the value of
cur_entry is larger than or equal to FW_CFG_MAX_ENTRY. The guest then
reads the data register, and fw_cfg_data_read() does this:

FWCfgEntry *e = (s->cur_entry == FW_CFG_INVALID) ? NULL :
&s->entries[arch][s->cur_entry & FW_CFG_ENTRY_MASK];
uint64_t value = 0;

assert(size > 0 && size <= sizeof(value));
if (s->cur_entry != FW_CFG_INVALID && e->data && s->cur_offset <
e->len) {

Just computing the pointer "e" is undefined behavior, let alone
evaluating "e->data".

Once again, I know upstream doesn't care about backward migration, but
we (at Red Hat) do, for specific machine types at least. I would rather
carry this patch in upstream than in downstream only.

As far as I understand, this is the first time in QEMU history that
we're looking into raising FW_CFG_FILE_SLOTS, so I'd rather be careful.
(I definitely don't want to win the privilege to implement the patch in
downstream!)

> But we have to take care that new files show up on new machine types
> only.

The series covers that, yes -- if the SMI host features bitmap that is
returned by the new machtype-specific callback function at board
initialization is zero (or the callback doesn't exist), then the fw_cfg
files are not created. See how ich9_lpc_pm_init() is called, and how it
handles the new "smi_host_features" parameter.

Thanks
Laszlo

[Qemu-devel] [PATCH] net: virtio-net: initialise local 'netcfg' variable

[P J P]

From: Prasad J Pandit 

Local 'netcfg' variable in 'virtio_net_get_config' routine was
not initialised. It could leak uninitialised 'netcfg.mtu' field
memory. Initialise 'netcfg' to avoid it.

Reported-by: Azureyang 
Signed-off-by: Prasad J Pandit 
---
 hw/net/virtio-net.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
index 01f1351..cb5b3dc 100644
--- a/hw/net/virtio-net.c
+++ b/hw/net/virtio-net.c
@@ -72,7 +72,7 @@ static int vq2q(int queue_index)
 static void virtio_net_get_config(VirtIODevice *vdev, uint8_t *config)
 {
 VirtIONet *n = VIRTIO_NET(vdev);
-struct virtio_net_config netcfg;
+struct virtio_net_config netcfg = {};
 
 virtio_stw_p(vdev, &netcfg.status, n->status);
 virtio_stw_p(vdev, &netcfg.max_virtqueue_pairs, n->max_queues);
-- 
2.7.4

Re: [Qemu-devel] [PATCH v5 00/10] tcg mips64 and mips r6 improvements

[James Hogan]

Hi,

On Thu, Dec 01, 2016 at 09:51:59PM +0800, Jin Guojie wrote:
>   * Tested successfully on following machines:
>   
> | HOST| qemu-system | Debian ISO  |
> |-|
> | mips 32 le  |i386 |i386 |
> | mips 32 le  |x86_64   |i386 |
> | mips 32 le  |x86_64   |amd64|
> | mips 64 le  |i386 |i386 |
> | mips 64 le  |x86_64   |i386 |
> | mips 64 le  |x86_64   |amd64|
> | mips 64 le  |  mips 64 be |  mips 64 be |
> |-|
> | mips 32 be  |i386 | i386|
> | mips 32 be  |x86_64   | i386|
> | mips 32 be  |x86_64   | amd64   |
> | mips 64 be  |i386 | i386|
> | mips 64 be  |x86_64   | i386|
> | mips 64 be  |x86_64   | amd64   |
> | mips n32 be |386  | i386|
> | mips n32 be |x86_64   | i386|
> | mips n32 be |x86_64   | amd64   |
> 
> (No plan to test MIPS R6 in this patch.)

FYI I tried booting one of Aurelien's x86_64 debian QEMU images on a
mipsel64r6 (P6600) using this, but it seems to get stuck somewhere in
GRUB before booting guest kernel. I did see it saying loading kernel and
ramdisk, but it seemed to return to the bios & grub and just get stuck.

I'm now attempting to reproduce in QEMU I6400 host, which is faster than
FPGA, but having the usual faff trying to get nested video output via
SDL/fbcon working.

Cheers
James


signature.asc
Description: Digital signature

[Qemu-devel] [PATCH] i386: amd_iommu: fix MMIO register count and access

[P J P]

From: Prasad J Pandit 

IOMMU MMIO registers are divided in two groups by their offsets.
Low offsets(<0x2000) registers are grouped into 'amdvi_mmio_low'
table and higher offsets(>=0x2000) registers are grouped into
'amdvi_mmio_high' table. No of registers in each table is given
by macro 'AMDVI_MMIO_REGS_LOW' and 'AMDVI_MMIO_REGS_HIGH' resp.
Values of these two macros were swapped, resulting in an OOB
access when reading 'amdvi_mmio_high' table. Correct these two
macros. Also read from 'amdvi_mmio_low' table for lower address.

Reported-by: Azureyang 
Signed-off-by: Prasad J Pandit 
---
 hw/i386/amd_iommu.c | 2 +-
 hw/i386/amd_iommu.h | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/hw/i386/amd_iommu.c b/hw/i386/amd_iommu.c
index 47b79d9..e0732cc 100644
--- a/hw/i386/amd_iommu.c
+++ b/hw/i386/amd_iommu.c
@@ -562,7 +562,7 @@ static void amdvi_mmio_trace(hwaddr addr, unsigned size)
 trace_amdvi_mmio_read(amdvi_mmio_high[index], addr, size, addr & 
~0x07);
 } else {
 index = index >= AMDVI_MMIO_REGS_LOW ? AMDVI_MMIO_REGS_LOW : index;
-trace_amdvi_mmio_read(amdvi_mmio_high[index], addr, size, addr & 
~0x07);
+trace_amdvi_mmio_read(amdvi_mmio_low[index], addr, size, addr & ~0x07);
 }
 }
 
diff --git a/hw/i386/amd_iommu.h b/hw/i386/amd_iommu.h
index 884926e..0d3dc6a 100644
--- a/hw/i386/amd_iommu.h
+++ b/hw/i386/amd_iommu.h
@@ -49,8 +49,8 @@
 #define AMDVI_CAPAB_INIT_TYPE (3 << 16)
 
 /* No. of used MMIO registers */
-#define AMDVI_MMIO_REGS_HIGH  8
-#define AMDVI_MMIO_REGS_LOW   7
+#define AMDVI_MMIO_REGS_HIGH  7
+#define AMDVI_MMIO_REGS_LOW   8
 
 /* MMIO registers */
 #define AMDVI_MMIO_DEVICE_TABLE   0x
-- 
2.7.4

Re: [Qemu-devel] [PATCH v4 00/13] aio: experimental virtio-blk polling mode

[Stefan Hajnoczi]

On Thu, Dec 01, 2016 at 07:26:39PM +, Stefan Hajnoczi wrote:
> v4:
>  * Added poll time self-tuning algorithm [Christian and Paolo]
>  * Try a single iteration of polling to avoid non-blocking 
> ppoll(2)/epoll_wait(2) [Paolo]
>  * Reordered patches to make performance analysis easier - see below
> 
> v3:
>  * Avoid ppoll(2)/epoll_wait(2) if polling succeeded [Paolo]
>  * Disable guest->host virtqueue notification during polling [Christian]
>  * Rebased on top of my virtio-blk/scsi virtqueue notification disable patches

Karl Rister found a performance regression between v2 -> v3.

There was a bug in the poll_disable_cnt code in aio_set_fd_handler()
which meant that polling was disabled unnecessarily.  This has been
fixed in v4 and I suspect it was the cause for the regression.

Stefan


signature.asc
Description: PGP signature

Re: [Qemu-devel] [PATCH v4 2/7] fw-cfg: turn FW_CFG_FILE_SLOTS into a device property

[Gerd Hoffmann]

On Do, 2016-12-01 at 18:06 +0100, Laszlo Ersek wrote:
> We'd like to raise the value of FW_CFG_FILE_SLOTS. Doing it naively could
> lead to problems with backward migration: a more recent QEMU (running an
> older machine type) would allow the guest, in fw_cfg_select(), to select a
> high key value that is unavailable in the same machine type implemented by
> the older (target) QEMU.

I don't think we need this.  fw_cfg changes are guest-visible so they
must not happen for a given machine type.  So if current machine types
don't hit the limit that should continue to be the case even if we
simply raise FW_CFG_FILE_SLOTS.

But we have to take care that new files show up on new machine types
only.

cheers,
  Gerd

Re: [Qemu-devel] [PATCH v2 4/4] 9pfs: introduce init_out/in_iov_from_pdu

[Greg Kurz]

On Mon, 28 Nov 2016 13:27:24 -0800
Stefano Stabellini  wrote:

> Not all 9pfs transports share memory between request and response. For
> those who don't, it is necessary to know how much memory is required in
> the response.
> 
> Split the existing init_iov_from_pdu function in two:
> init_out_iov_from_pdu (for writes) and init_in_iov_from_pdu (for reads).
> init_in_iov_from_pdu takes an additional size parameter to specify the
> memory required for the response message.
> 
> Signed-off-by: Stefano Stabellini 
> ---

Reviewed-by: Greg Kurz 

>  hw/9pfs/9p.c   |  6 +-
>  hw/9pfs/9p.h   |  6 --
>  hw/9pfs/virtio-9p-device.c | 28 ++--
>  3 files changed, 27 insertions(+), 13 deletions(-)
> 
> diff --git a/hw/9pfs/9p.c b/hw/9pfs/9p.c
> index 79d7201..ce1f9d9 100644
> --- a/hw/9pfs/9p.c
> +++ b/hw/9pfs/9p.c
> @@ -1652,7 +1652,11 @@ static void v9fs_init_qiov_from_pdu(QEMUIOVector 
> *qiov, V9fsPDU *pdu,
>  struct iovec *iov;
>  unsigned int niov;
>  
> -pdu->s->transport->init_iov_from_pdu(pdu, &iov, &niov, is_write);
> +if (is_write) {
> +pdu->s->transport->init_out_iov_from_pdu(pdu, &iov, &niov);
> +} else {
> +pdu->s->transport->init_in_iov_from_pdu(pdu, &iov, &niov, size);
> +}
>  
>  qemu_iovec_init_external(&elem, iov, niov);
>  qemu_iovec_init(qiov, niov);
> diff --git a/hw/9pfs/9p.h b/hw/9pfs/9p.h
> index c8c9aa8..4c4feaf 100644
> --- a/hw/9pfs/9p.h
> +++ b/hw/9pfs/9p.h
> @@ -349,8 +349,10 @@ struct V9fsTransport {
>  va_list ap);
>  ssize_t (*pdu_vunmarshal)(V9fsPDU *pdu, size_t offset, const char 
> *fmt,
>va_list ap);
> -void(*init_iov_from_pdu)(V9fsPDU *pdu, struct iovec **piov,
> - unsigned int *pniov, bool is_write);
> +void(*init_in_iov_from_pdu)(V9fsPDU *pdu, struct iovec **piov,
> +unsigned int *pniov, size_t size);
> +void(*init_out_iov_from_pdu)(V9fsPDU *pdu, struct iovec **piov,
> + unsigned int *pniov);
>  void(*push_and_notify)(V9fsPDU *pdu);
>  };
>  
> diff --git a/hw/9pfs/virtio-9p-device.c b/hw/9pfs/virtio-9p-device.c
> index 273425b..27a4a32 100644
> --- a/hw/9pfs/virtio-9p-device.c
> +++ b/hw/9pfs/virtio-9p-device.c
> @@ -171,26 +171,34 @@ static ssize_t virtio_pdu_vunmarshal(V9fsPDU *pdu, 
> size_t offset,
>  return v9fs_iov_vunmarshal(elem->out_sg, elem->out_num, offset, 1, fmt, 
> ap);
>  }
>  
> -static void virtio_init_iov_from_pdu(V9fsPDU *pdu, struct iovec **piov,
> - unsigned int *pniov, bool is_write)
> +/* The size parameter is used by other transports. Do not drop it. */
> +static void virtio_init_in_iov_from_pdu(V9fsPDU *pdu, struct iovec **piov,
> +unsigned int *pniov, size_t size)
>  {
>  V9fsState *s = pdu->s;
>  V9fsVirtioState *v = container_of(s, V9fsVirtioState, state);
>  VirtQueueElement *elem = v->elems[pdu->idx];
>  
> -if (is_write) {
> -*piov = elem->out_sg;
> -*pniov = elem->out_num;
> -} else {
> -*piov = elem->in_sg;
> -*pniov = elem->in_num;
> -}
> +*piov = elem->in_sg;
> +*pniov = elem->in_num;
> +}
> +
> +static void virtio_init_out_iov_from_pdu(V9fsPDU *pdu, struct iovec **piov,
> + unsigned int *pniov)
> +{
> +V9fsState *s = pdu->s;
> +V9fsVirtioState *v = container_of(s, V9fsVirtioState, state);
> +VirtQueueElement *elem = v->elems[pdu->idx];
> +
> +*piov = elem->out_sg;
> +*pniov = elem->out_num;
>  }
>  
>  static const struct V9fsTransport virtio_9p_transport = {
>  .pdu_vmarshal = virtio_pdu_vmarshal,
>  .pdu_vunmarshal = virtio_pdu_vunmarshal,
> -.init_iov_from_pdu = virtio_init_iov_from_pdu,
> +.init_in_iov_from_pdu = virtio_init_in_iov_from_pdu,
> +.init_out_iov_from_pdu = virtio_init_out_iov_from_pdu,
>  .push_and_notify = virtio_9p_push_and_notify,
>  };
>  

Re: [Qemu-devel] [PATCH v2 3/4] 9pfs: call v9fs_init_qiov_from_pdu before v9fs_pack

[Greg Kurz]

On Mon, 28 Nov 2016 13:27:23 -0800
Stefano Stabellini  wrote:

> v9fs_xattr_read should not access VirtQueueElement elems directly.
> Move v9fs_init_qiov_from_pdu up in the file and call
> v9fs_init_qiov_from_pdu before v9fs_pack. Use v9fs_pack on the new
> iovec.
> 
> Signed-off-by: Stefano Stabellini 
> 
> ---

Reviewed-by: Greg Kurz 

> Changes in v2:
> - add a call to qemu_iovec_destroy
> - fix commit description
> ---
>  hw/9pfs/9p.c | 59 ++-
>  1 file changed, 30 insertions(+), 29 deletions(-)
> 
> diff --git a/hw/9pfs/9p.c b/hw/9pfs/9p.c
> index 5a20a13..79d7201 100644
> --- a/hw/9pfs/9p.c
> +++ b/hw/9pfs/9p.c
> @@ -1633,14 +1633,39 @@ out_nofid:
>  pdu_complete(pdu, err);
>  }
>  
> +/*
> + * Create a QEMUIOVector for a sub-region of PDU iovecs
> + *
> + * @qiov:   uninitialized QEMUIOVector
> + * @skip:   number of bytes to skip from beginning of PDU
> + * @size:   number of bytes to include
> + * @is_write:   true - write, false - read
> + *
> + * The resulting QEMUIOVector has heap-allocated iovecs and must be cleaned 
> up
> + * with qemu_iovec_destroy().
> + */
> +static void v9fs_init_qiov_from_pdu(QEMUIOVector *qiov, V9fsPDU *pdu,
> +size_t skip, size_t size,
> +bool is_write)
> +{
> +QEMUIOVector elem;
> +struct iovec *iov;
> +unsigned int niov;
> +
> +pdu->s->transport->init_iov_from_pdu(pdu, &iov, &niov, is_write);
> +
> +qemu_iovec_init_external(&elem, iov, niov);
> +qemu_iovec_init(qiov, niov);
> +qemu_iovec_concat(qiov, &elem, skip, size);
> +}
> +
>  static int v9fs_xattr_read(V9fsState *s, V9fsPDU *pdu, V9fsFidState *fidp,
> uint64_t off, uint32_t max_count)
>  {
>  ssize_t err;
>  size_t offset = 7;
>  uint64_t read_count;
> -V9fsVirtioState *v = container_of(s, V9fsVirtioState, state);
> -VirtQueueElement *elem = v->elems[pdu->idx];
> +QEMUIOVector qiov_full;
>  
>  if (fidp->fs.xattr.len < off) {
>  read_count = 0;
> @@ -1656,9 +1681,11 @@ static int v9fs_xattr_read(V9fsState *s, V9fsPDU *pdu, 
> V9fsFidState *fidp,
>  }
>  offset += err;
>  
> -err = v9fs_pack(elem->in_sg, elem->in_num, offset,
> +v9fs_init_qiov_from_pdu(&qiov_full, pdu, 0, read_count, false);
> +err = v9fs_pack(qiov_full.iov, qiov_full.niov, offset,
>  ((char *)fidp->fs.xattr.value) + off,
>  read_count);
> +qemu_iovec_destroy(&qiov_full);
>  if (err < 0) {
>  return err;
>  }
> @@ -1732,32 +1759,6 @@ static int coroutine_fn 
> v9fs_do_readdir_with_stat(V9fsPDU *pdu,
>  return count;
>  }
>  
> -/*
> - * Create a QEMUIOVector for a sub-region of PDU iovecs
> - *
> - * @qiov:   uninitialized QEMUIOVector
> - * @skip:   number of bytes to skip from beginning of PDU
> - * @size:   number of bytes to include
> - * @is_write:   true - write, false - read
> - *
> - * The resulting QEMUIOVector has heap-allocated iovecs and must be cleaned 
> up
> - * with qemu_iovec_destroy().
> - */
> -static void v9fs_init_qiov_from_pdu(QEMUIOVector *qiov, V9fsPDU *pdu,
> -size_t skip, size_t size,
> -bool is_write)
> -{
> -QEMUIOVector elem;
> -struct iovec *iov;
> -unsigned int niov;
> -
> -pdu->s->transport->init_iov_from_pdu(pdu, &iov, &niov, is_write);
> -
> -qemu_iovec_init_external(&elem, iov, niov);
> -qemu_iovec_init(qiov, niov);
> -qemu_iovec_concat(qiov, &elem, skip, size);
> -}
> -
>  static void coroutine_fn v9fs_read(void *opaque)
>  {
>  int32_t fid;

Re: [Qemu-devel] [PATCH v2 2/4] 9pfs: introduce transport specific callbacks

[Greg Kurz]

On Mon, 28 Nov 2016 13:27:22 -0800
Stefano Stabellini  wrote:

> Don't call virtio functions from 9pfs generic code, use generic function
> callbacks instead.
> 
> Signed-off-by: Stefano Stabellini 
> 
> ---

Reviewed-by: Greg Kurz 

> Changes in v2:
> - constify virtio_9p_transport and V9fsTransport
> - assert !s->transport.
> - code style
> ---
>  hw/9pfs/9p.c   |  8 
>  hw/9pfs/9p.h   | 19 +++
>  hw/9pfs/virtio-9p-device.c | 24 +---
>  hw/9pfs/virtio-9p.h|  9 -
>  4 files changed, 40 insertions(+), 20 deletions(-)
> 
> diff --git a/hw/9pfs/9p.c b/hw/9pfs/9p.c
> index 05e950f..5a20a13 100644
> --- a/hw/9pfs/9p.c
> +++ b/hw/9pfs/9p.c
> @@ -47,7 +47,7 @@ ssize_t pdu_marshal(V9fsPDU *pdu, size_t offset, const char 
> *fmt, ...)
>  va_list ap;
>  
>  va_start(ap, fmt);
> -ret = virtio_pdu_vmarshal(pdu, offset, fmt, ap);
> +ret = pdu->s->transport->pdu_vmarshal(pdu, offset, fmt, ap);
>  va_end(ap);
>  
>  return ret;
> @@ -59,7 +59,7 @@ ssize_t pdu_unmarshal(V9fsPDU *pdu, size_t offset, const 
> char *fmt, ...)
>  va_list ap;
>  
>  va_start(ap, fmt);
> -ret = virtio_pdu_vunmarshal(pdu, offset, fmt, ap);
> +ret = pdu->s->transport->pdu_vunmarshal(pdu, offset, fmt, ap);
>  va_end(ap);
>  
>  return ret;
> @@ -67,7 +67,7 @@ ssize_t pdu_unmarshal(V9fsPDU *pdu, size_t offset, const 
> char *fmt, ...)
>  
>  static void pdu_push_and_notify(V9fsPDU *pdu)
>  {
> -virtio_9p_push_and_notify(pdu);
> +pdu->s->transport->push_and_notify(pdu);
>  }
>  
>  static int omode_to_uflags(int8_t mode)
> @@ -1751,7 +1751,7 @@ static void v9fs_init_qiov_from_pdu(QEMUIOVector *qiov, 
> V9fsPDU *pdu,
>  struct iovec *iov;
>  unsigned int niov;
>  
> -virtio_init_iov_from_pdu(pdu, &iov, &niov, is_write);
> +pdu->s->transport->init_iov_from_pdu(pdu, &iov, &niov, is_write);
>  
>  qemu_iovec_init_external(&elem, iov, niov);
>  qemu_iovec_init(qiov, niov);
> diff --git a/hw/9pfs/9p.h b/hw/9pfs/9p.h
> index 07cee01..c8c9aa8 100644
> --- a/hw/9pfs/9p.h
> +++ b/hw/9pfs/9p.h
> @@ -230,6 +230,7 @@ typedef struct V9fsState
>  enum p9_proto_version proto_version;
>  int32_t msize;
>  V9fsPDU pdus[MAX_REQ];
> +const struct V9fsTransport *transport;
>  /*
>   * lock ensuring atomic path update
>   * on rename.
> @@ -343,4 +344,22 @@ void pdu_free(V9fsPDU *pdu);
>  void pdu_submit(V9fsPDU *pdu);
>  void v9fs_reset(V9fsState *s);
>  
> +struct V9fsTransport {
> +ssize_t (*pdu_vmarshal)(V9fsPDU *pdu, size_t offset, const char *fmt,
> +va_list ap);
> +ssize_t (*pdu_vunmarshal)(V9fsPDU *pdu, size_t offset, const char 
> *fmt,
> +  va_list ap);
> +void(*init_iov_from_pdu)(V9fsPDU *pdu, struct iovec **piov,
> + unsigned int *pniov, bool is_write);
> +void(*push_and_notify)(V9fsPDU *pdu);
> +};
> +
> +static inline int v9fs_register_transport(V9fsState *s,
> +const struct V9fsTransport *t)
> +{
> +assert(!s->transport);
> +s->transport = t;
> +return 0;
> +}
> +
>  #endif
> diff --git a/hw/9pfs/virtio-9p-device.c b/hw/9pfs/virtio-9p-device.c
> index 1782e4a..273425b 100644
> --- a/hw/9pfs/virtio-9p-device.c
> +++ b/hw/9pfs/virtio-9p-device.c
> @@ -20,7 +20,9 @@
>  #include "hw/virtio/virtio-access.h"
>  #include "qemu/iov.h"
>  
> -void virtio_9p_push_and_notify(V9fsPDU *pdu)
> +static const struct V9fsTransport virtio_9p_transport;
> +
> +static void virtio_9p_push_and_notify(V9fsPDU *pdu)
>  {
>  V9fsState *s = pdu->s;
>  V9fsVirtioState *v = container_of(s, V9fsVirtioState, state);
> @@ -126,6 +128,7 @@ static void virtio_9p_device_realize(DeviceState *dev, 
> Error **errp)
>  v->config_size = sizeof(struct virtio_9p_config) + strlen(s->fsconf.tag);
>  virtio_init(vdev, "virtio-9p", VIRTIO_ID_9P, v->config_size);
>  v->vq = virtio_add_queue(vdev, MAX_REQ, handle_9p_output);
> +v9fs_register_transport(s, &virtio_9p_transport);
>  
>  out:
>  return;
> @@ -148,8 +151,8 @@ static void virtio_9p_reset(VirtIODevice *vdev)
>  v9fs_reset(&v->state);
>  }
>  
> -ssize_t virtio_pdu_vmarshal(V9fsPDU *pdu, size_t offset,
> -const char *fmt, va_list ap)
> +static ssize_t virtio_pdu_vmarshal(V9fsPDU *pdu, size_t offset,
> +   const char *fmt, va_list ap)
>  {
>  V9fsState *s = pdu->s;
>  V9fsVirtioState *v = container_of(s, V9fsVirtioState, state);
> @@ -158,8 +161,8 @@ ssize_t virtio_pdu_vmarshal(V9fsPDU *pdu, size_t offset,
>  return v9fs_iov_vmarshal(elem->in_sg, elem->in_num, offset, 1, fmt, ap);
>  }
>  
> -ssize_t virtio_pdu_vunmarshal(V9fsPDU *pdu, size_t offset,
> -  const char *fmt, va_list ap)
> +static ssize_t virtio_pdu_vunmarshal(V9fsPDU *pdu, size_t offset,
> +  

Re: [Qemu-devel] [PATCH] scripts: add "git.orderfile" for ordering diff hunks by pathname patterns

[Gerd Hoffmann]

>   git config diff.orderFile scripts/git.orderfile

> diff --git a/scripts/git.orderfile b/scripts/git.orderfile
> new file mode 100644
> index ..600a2e4fc540
> --- /dev/null
> +++ b/scripts/git.orderfile
> @@ -0,0 +1,15 @@

Is it possible to have comments in here?  So we can place the git config
command above into into this file too?

> +*.txt
> +configure
> +GNUmakefile
> +makefile
> +Makefile

We also have Makefile.objs and Makefile.target, so "Makefile*" (like the
first version had IIRC) should work better.

> +*.mak
> +qapi-schema*.json
> +qapi/*.json
> +include/qapi/visitor.h
> +include/qapi/visitor-impl.h
> +scripts/qapi.py
> +scripts/*.py
> +*.h
> +qapi/qapi-visit-core.c

I guess there is more which could be placed here (i.e. files holding
infrastructure which makes sense to see first).  But we can easily
refine that incrementally.  That is the point to have this file in the
repo in the first place ;)

cheers,
  Gerd

Re: [Qemu-devel] Support for using TCG frontend as a library

[Liviu Ionescu]

> On 2 Dec 2016, at 12:24, Peter Maydell  wrote:
> 
> Right, but if you have a bug which requires your application to
> sit there processing for half an hour (or even five minutes)
> before it appears, it's nice not to spend that time.

fully agree. for physical targets, the GNU ARM Eclipse debugging plug-ins have 
a feature called "connect to running target", when the debugger attaches to a 
live target without any reset or flash reprogramming.

> You could surface the basic functionality with a 'snapshot'
> button that invoked the monitor "savevm" command, and a
> "start from snapshot" that just added '-loadvm snapshotname' to
> the QEMU command line. Everything else (including connecting to
> gdbstub) should just work as usual.

well, yes, something like this can be done, but it is not that simple, normally 
the debugging plug-ins will reset the board and reprogram the flash, but a 
similar approach to "connect to running target" can be used.

but besides the gui aspects, the deeper conclusion from this discussion is 
that, regardless the implementation (with the current QEMU very complicated 
objects, or with the C++ natural objects that I'd like to use), the entire tree 
of peripherals should be fully serialisable, otherwise the peripheral status 
cannot be restored correctly.

I'll consider this and possibly adjust my current implementation, which does 
not use links consistently.

do you have any other suggestions for making Cortex-M devices 'snaphotable'? 
(since I understand the ones used in the Stellaris implementation are not).

> 
>> on the other side, what would be really useful for Cortex-M, are
>> the instruction and data tracing features available for some
>> devices, usually available only with very expensive multi-pin
>> JTAG probes on physical devices; were these ARM features considered?
> 
> These are painful to add to TCG (which generally doesn't bother
> to keep track of information it doesn't need for execution),
> and they'd make things very slow

any estimate on how much slower? anyway I think this should be a runtime 
option, enabled only when really needed; getting the trace output is also 
expensive on physical targets, so, I think a reasonable speed penalty can be 
accepted, at least for most Cortex-M devices, which normally run at some tens 
of MHz.

> (plus they're very invasive
> changes to the TCG frontend code). So I don't think they're
> a bad idea but they're not at the top of my priority list at the
> moment.

I see. in case you'll reconsider, please let me know, maybe I can help.


regards,

Liviu