date:20170714

On Fri, Jul 14, 2017 at 10:50:06AM -0300, Daniel Henrique Barboza wrote:
> 
> 
> On 07/14/2017 03:53 AM, David Gibson wrote:
> > On Thu, Jul 13, 2017 at 07:13:23AM -0300, Daniel Henrique Barboza wrote:
> > > 
> > > On 07/12/2017 09:57 PM, David Gibson wrote:
> > > > On Wed, Jul 12, 2017 at 10:48:38AM -0300, Daniel Henrique Barboza wrote:
> > > > > The dreaded Libvirt hotplug-migrate-hotunplug scenario is working 
> > > > > nicely.
> > > > Good to hear.
> > > > 
> > > > > device_add when the machine is in RUN_STATE_PRELAUNCH (-S) still 
> > > > > doesn't
> > > > > work but it is expected - as discussed in "[RFC drcVI PATCH] spapr: 
> > > > > reset
> > > > > DRCs
> > > > > on migration pre_load", this scenario can't be fixed solely by this 
> > > > > DRC
> > > > > cleanup.
> > > > Hmm.. what's the exact test case you're using here?  The prelaunch
> > > > case I tried _did_ work (queueing the event during prelaunch, then
> > > > completing the hotplug sequence once the guest had booted).
> > > This is the test case:
> > > 
> > > sudo ./qemu-system-ppc64 -name migrate_qemu -boot strict=on --enable-kvm
> > > -device nec-usb-xhci,id=usb,bus=pci.0,addr=0xf -device
> > > spapr-vscsi,id=scsi0,reg=0x2000 -smp 
> > > 1,maxcpus=4,sockets=4,cores=1,threads=1
> > > --machine pseries,accel=kvm,usb=off,dump-guest-core=off -m
> > > 4G,slots=32,maxmem=32G -drive 
> > > file=/home/danielhb/vm_imgs/ubuntu1704.qcow2,format=qcow2,if=none,id=drive-virtio-disk0,cache=none
> > > -device 
> > > virtio-blk-pci,scsi=off,bus=pci.0,addr=0x2,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1
> > > -nographic -S
> > > QEMU 2.9.50 monitor - type 'help' for more information
> > > (qemu)
> > > (qemu) device_add host-spapr-cpu-core,id=core1,core-id=1
> > > (qemu) cont
> > > 
> > > (...)
> > > 
> > > After OS boots:
> > > 
> > > danielhb@ubuntu1704:~$ lscpu
> > > Architecture:  ppc64le
> > > Byte Order:Little Endian
> > > CPU(s):1
> > > On-line CPU(s) list:   0
> > > Thread(s) per core:1
> > > Core(s) per socket:1
> > > Socket(s): 1
> > > NUMA node(s):  1
> > > Model: 2.1 (pvr 004b 0201)
> > > Model name:POWER8E (raw), altivec supported
> > > Hypervisor vendor: horizontal
> > > Virtualization type:   full
> > > L1d cache: 64K
> > > L1i cache: 32K
> > > NUMA node0 CPU(s): 0
> > > danielhb@ubuntu1704:~$ (qemu)
> > > (qemu) info cpus
> > > * CPU #0: nip=0xc00a3e0c thread_id=6134
> > >CPU #1: nip=0x (halted) thread_id=6163
> > > (qemu) info hotpluggable-cpus
> > > Hotpluggable CPUs:
> > >type: "host-spapr-cpu-core"
> > >vcpus_count: "1"
> > >CPUInstance Properties:
> > >  core-id: "3"
> > >type: "host-spapr-cpu-core"
> > >vcpus_count: "1"
> > >CPUInstance Properties:
> > >  core-id: "2"
> > >type: "host-spapr-cpu-core"
> > >vcpus_count: "1"
> > >qom_path: "/machine/peripheral/core1"
> > >CPUInstance Properties:
> > >  core-id: "1"
> > >type: "host-spapr-cpu-core"
> > >vcpus_count: "1"
> > >qom_path: "/machine/unattached/device[0]"
> > >CPUInstance Properties:
> > >  core-id: "0"
> > > (qemu)
> > Huh.  I tried basically the same thing, and I get the second cpu once
> > the OS is booted.  My first guess would be that the difference is in
> > the guest (mine is RHEL 7.3).  Have you double checked that rtas_errd
> > and drmgr are present in your guest?
> 
> Yeah the guest has drmgr and rtas_errd running. As you said, there might be
> something different in the guests that explains why yours work and mine
> doesn't.
> Coupling that with the fact that this is not a common usage, I believe we
> can leave it as a FYI/reminder if we need to revisit this issue in the
> future.

I concur.

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature

Re: [Qemu-devel] [Qemu-ppc] [PATCH v2] spapr: ignore decr interrupts when MSR_EE is disabled

2017-07-14 Thread Nikunj Dadhania

On 15-Jul-2017 8:10 AM, "Benjamin Herrenschmidt"  wrote:

On Fri, 2017-07-14 at 09:34 +0530, Nikunj A Dadhania wrote:
> Rebooting a SMP TCG guest is broken for both single/multi threaded TCG.
>
> When reset happens, all the CPUs are in halted state. First CPU is
brought out
> of reset and secondary CPUs would be initialized by the guest kernel
using a
> rtas call start-cpu.
>
> However, in case of TCG, decrementer interrupts keep on coming and waking
the
> secondary CPUs up.
>
> These secondary CPUs would see the decrementer interrupt pending, which
makes
> cpu::has_work() to bring them out of wait loop and start executing
> tcg_exec_cpu().
>
> The problem with this is all the CPUs wake up and start booting SLOF
image,
> causing the following exception(4 CPUs TCG VM):

This patch isn't right

> --- a/target/ppc/translate_init.c
> +++ b/target/ppc/translate_init.c
> @@ -8536,7 +8536,8 @@ static bool cpu_has_work_POWER7(CPUState *cs)
>  }
>  if ((env->pending_interrupts & (1u << PPC_INTERRUPT_DECR)) &&
>  (env->spr[SPR_LPCR] & LPCR_P7_PECE1)) {
> -return true;
> +/* Return true only when MSR_EE is enabled */
> +return msr_ee;
>  }

No, the HW will not check MSR_EE, in fact, MSR_EE can purposefully be
left off and we might expect a wakeup still.


Here we had a decr timer callback which is started during machine init and
remains on even during reset.

In case of HW, does the decr stop on reset?


I think the right fix is for reset to clear the LPCR PECE bits.


Ok, let me try that out.

Regards
Nikunj

Re: [Qemu-devel] [PATCH] test-i386: Make x86 test program compile on clang

2017-07-14 Thread Theodore Dubois

ping http://patchwork.ozlabs.org/patch/786393/

~Theodore

> On Jul 10, 2017, at 1:57 PM, Theodore Dubois  wrote:
> 
> Clang's assembler is slightly incompatible with GCC's assembler, which
> caused the program to not compile on Clang for these reasons:
> 
>  - The "q" constraint was specified for an argument to the set
>instruction, which didn't work because Clang chose the esi register,
>which has no 8-bit form on i386.
>  - Clang doesn't support size suffixes on the loop instructions.
>  - Clang requires a size suffix on the fist instruction.
>  - Clang doesn't support specifying segment prefixes before the
>instruction, and requires specifying them on the address.
>  - The arguments to the bound instruction are in the wrong order on
>Clang. https://bugs.llvm.org/show_bug.cgi?id=27653
> 
> Signed-off-by: Theodore Dubois 
> ---
> tests/tcg/test-i386.c | 20 
> 1 file changed, 16 insertions(+), 4 deletions(-)
> 
> diff --git a/tests/tcg/test-i386.c b/tests/tcg/test-i386.c
> index 0f7b943b0c..dcb8e4876e 100644
> --- a/tests/tcg/test-i386.c
> +++ b/tests/tcg/test-i386.c
> @@ -369,7 +369,7 @@ void test_lea(void)
> asm("movl $0, %0\n\t"\
> "cmpl %2, %1\n\t"\
> "set" JCC " %b0\n\t"\
> -: "=r" (res)\
> +: "=q" (res)\
> : "r" (v1), "r" (v2));\
> printf("%-10s %d\n", "set" JCC, res);\
>  if (TEST_CMOV) {\
> @@ -491,15 +491,23 @@ void test_loop(void)
> 
> #if !defined(__x86_64__)
> TEST_LOOP("jcxz");
> +#if !defined(__clang__)
> TEST_LOOP("loopw");
> TEST_LOOP("loopzw");
> TEST_LOOP("loopnzw");
> #endif
> +#endif
> 
> TEST_LOOP("jecxz");
> +#if !defined(__clang__)
> TEST_LOOP("loopl");
> TEST_LOOP("loopzl");
> TEST_LOOP("loopnzl");
> +#else
> +TEST_LOOP("loop");
> +TEST_LOOP("loopz");
> +TEST_LOOP("loopnz");
> +#endif
> }
> 
> #undef CC_MASK
> @@ -867,7 +875,7 @@ void test_fcvt(double a)
> uint16_t val16;
> val16 = (fpuc & ~0x0c00) | (i << 10);
> asm volatile ("fldcw %0" : : "m" (val16));
> -asm volatile ("fist %0" : "=m" (wa) : "t" (a));
> +asm volatile ("fists %0" : "=m" (wa) : "t" (a));
> asm volatile ("fistl %0" : "=m" (ia) : "t" (a));
> asm volatile ("fistpll %0" : "=m" (lla) : "t" (a) : "st");
> asm volatile ("frndint ; fstl %0" : "=m" (ra) : "t" (a));
> @@ -1318,12 +1326,12 @@ void test_segs(void)
> seg_data1[1] = 0xaa;
> seg_data2[1] = 0x55;
> 
> -asm volatile ("fs movzbl 0x1, %0" : "=r" (res));
> +asm volatile ("movzbl %%fs:0x1, %0" : "=r" (res));
> printf("FS[1] = %02x\n", res);
> 
> asm volatile ("pushl %%gs\n"
>   "movl %1, %%gs\n"
> -  "gs movzbl 0x1, %0\n"
> +  "movzbl %%gs:0x1, %0\n"
>   "popl %%gs\n"
>   : "=r" (res)
>   : "r" (MK_SEL(2)));
> @@ -1764,7 +1772,11 @@ void test_exceptions(void)
> /* bound exception */
> tab[0] = 1;
> tab[1] = 10;
> +#if defined(__clang__)
> +asm volatile ("bound %1, %0" : : "r" (11), "m" (tab[0]));
> +#else
> asm volatile ("bound %0, %1" : : "r" (11), "m" (tab[0]));
> +#endif
> }
> #endif
> 
> -- 
> 2.13.2
> 
> 
> 
> ~Theodore
> 
>

Re: [Qemu-devel] [PULL 00/18] ppc-for-2.10 queue 20170714

On Fri, Jul 14, 2017 at 04:13:05PM +0100, Peter Maydell wrote:
> On 14 July 2017 at 07:15, David Gibson  wrote:
> > The following changes since commit 49bcce4b9c11759678fd223aefb48691c4959d4f:
> >
> >   Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2017-07-12' 
> > into staging (2017-07-13 16:56:06 +0100)
> >
> > are available in the git repository at:
> >
> >   git://github.com/dgibson/qemu.git tags/ppc-for-2.10-20170714
> >
> > for you to fetch changes up to e11134d2ac7744569d7c387102368c507d340c30:
> >
> >   pseries: Allow HPT resizing with KVM (2017-07-14 09:28:30 +1000)
> >
> > 
> > ppc patch queue 2017-07-14
> >
> > Two major batches in this set, rather than the usual collection of
> > assorted fixes.
> >
> > * More DRC cleanup.  This gets the state management into a state
> >   which should fix many of the hotplug+migration problems we've
> >   had.  Plus it gets the migration stream format into something
> >   well defined and pretty minimal which we can reasonably support
> >   into the future.
> >
> > * Hashed Page Table resizing.  It's been a while since this was
> >   posted, but it's been through several previous rounds of review.
> >   The kernel parts (both guest and host) are merged in 4.11, so
> >   this is the only remaining piece left to allow resizing of the
> >   HPT in a running guest.
> >
> > There are also a handful of unrelated fixes.
> 
> Hi. I'm afraid this fails to build on the w32 compiler:
>   CC  ppc64-softmmu/hw/ppc/spapr.o
> /home/petmay01/linaro/qemu-for-merges/hw/ppc/spapr.c: In function
> ‘spapr_get_resize_hpt’:
> /home/petmay01/linaro/qemu-for-merges/hw/ppc/spapr.c:2642:1: error:
> control reaches end of non-void functio
> n [-Werror=return-type]
>  }

Drat.  Is there an easy way to test compile for w32?

>  ^
> 
> The compiler is confused (it doesn't realise that "assert(0)"
> cannot return), but you can make it happier like this:
> 
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index 09b6ff0..68f936c 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -2638,7 +2638,7 @@ static char *spapr_get_resize_hpt(Object *obj,
> Error **errp)
>  case SPAPR_RESIZE_HPT_REQUIRED:
>  return g_strdup("required");
>  }
> -assert(0);
> +g_assert_not_reached();
>  }
> 
>  static void spapr_set_resize_hpt(Object *obj, const char *value, Error 
> **errp)
> 
> 
> I think g_assert_not_reached() is easier to read anyway; I
> always have to think for a moment about whether assert(0)
> or assert(1) is the "always assert" condition...

Yes, seems good.  Should I resend, or will you adjust in place?

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature

Re: [Qemu-devel] [PATCH 28/28] ppc: replace cpu_ppc_init() with cpu_generic_init()

On Fri, Jul 14, 2017 at 03:52:19PM +0200, Igor Mammedov wrote:
> it's just a wrapper, drop it and use cpu_generic_init() directly
> 
> Signed-off-by: Igor Mammedov 

Acked-by: David Gibson 

> ---
> CC: Alexander Graf 
> CC: David Gibson 
> CC: "Hervé Poussineau" 
> CC: "Edgar E. Iglesias" 
> CC: qemu-...@nongnu.org
> ---
>  target/ppc/cpu.h| 3 +--
>  hw/ppc/e500.c   | 3 ++-
>  hw/ppc/mac_newworld.c   | 3 ++-
>  hw/ppc/mac_oldworld.c   | 3 ++-
>  hw/ppc/ppc440_bamboo.c  | 2 +-
>  hw/ppc/ppc4xx_devs.c| 2 +-
>  hw/ppc/prep.c   | 5 +++--
>  hw/ppc/virtex_ml507.c   | 2 +-
>  target/ppc/translate_init.c | 5 -
>  9 files changed, 13 insertions(+), 15 deletions(-)
> 
> diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
> index 6ee2a26..696cbeb 100644
> --- a/target/ppc/cpu.h
> +++ b/target/ppc/cpu.h
> @@ -1276,7 +1276,6 @@ extern const struct VMStateDescription vmstate_ppc_cpu;
>  #endif
>  
>  
> /*/
> -PowerPCCPU *cpu_ppc_init(const char *cpu_model);
>  void ppc_translate_init(void);
>  const char *ppc_cpu_lookup_alias(const char *alias);
>  /* you can call this signal handler from your SIGBUS and SIGSEGV
> @@ -1353,7 +1352,7 @@ static inline uint64_t ppc_dump_gpr(CPUPPCState *env, 
> int gprn)
>  int ppc_dcr_read (ppc_dcr_t *dcr_env, int dcrn, uint32_t *valp);
>  int ppc_dcr_write (ppc_dcr_t *dcr_env, int dcrn, uint32_t val);
>  
> -#define cpu_init(cpu_model) CPU(cpu_ppc_init(cpu_model))
> +#define cpu_init(cpu_model) cpu_generic_init(TYPE_POWERPC_CPU, cpu_model)
>  
>  #define cpu_signal_handler cpu_ppc_signal_handler
>  #define cpu_list ppc_cpu_list
> diff --git a/hw/ppc/e500.c b/hw/ppc/e500.c
> index 62f1857..f0596f3 100644
> --- a/hw/ppc/e500.c
> +++ b/hw/ppc/e500.c
> @@ -819,7 +819,8 @@ void ppce500_init(MachineState *machine, PPCE500Params 
> *params)
>  CPUState *cs;
>  qemu_irq *input;
>  
> -cpu = cpu_ppc_init(machine->cpu_model);
> +cpu = POWERPC_CPU(cpu_generic_init(TYPE_POWERPC_CPU,
> +   machine->cpu_model));
>  if (cpu == NULL) {
>  fprintf(stderr, "Unable to initialize CPU!\n");
>  exit(1);
> diff --git a/hw/ppc/mac_newworld.c b/hw/ppc/mac_newworld.c
> index bae1c0a..72c98e9 100644
> --- a/hw/ppc/mac_newworld.c
> +++ b/hw/ppc/mac_newworld.c
> @@ -187,7 +187,8 @@ static void ppc_core99_init(MachineState *machine)
>  #endif
>  }
>  for (i = 0; i < smp_cpus; i++) {
> -cpu = cpu_ppc_init(machine->cpu_model);
> +cpu = POWERPC_CPU(cpu_generic_init(TYPE_POWERPC_CPU,
> +   machine->cpu_model));
>  if (cpu == NULL) {
>  fprintf(stderr, "Unable to find PowerPC CPU definition\n");
>  exit(1);
> diff --git a/hw/ppc/mac_oldworld.c b/hw/ppc/mac_oldworld.c
> index 97bb854..6fc2a19 100644
> --- a/hw/ppc/mac_oldworld.c
> +++ b/hw/ppc/mac_oldworld.c
> @@ -116,7 +116,8 @@ static void ppc_heathrow_init(MachineState *machine)
>  if (machine->cpu_model == NULL)
>  machine->cpu_model = "G3";
>  for (i = 0; i < smp_cpus; i++) {
> -cpu = cpu_ppc_init(machine->cpu_model);
> +cpu = POWERPC_CPU(cpu_generic_init(TYPE_POWERPC_CPU,
> +   machine->cpu_model));
>  if (cpu == NULL) {
>  fprintf(stderr, "Unable to find PowerPC CPU definition\n");
>  exit(1);
> diff --git a/hw/ppc/ppc440_bamboo.c b/hw/ppc/ppc440_bamboo.c
> index 9d997bf..ca26398 100644
> --- a/hw/ppc/ppc440_bamboo.c
> +++ b/hw/ppc/ppc440_bamboo.c
> @@ -186,7 +186,7 @@ static void bamboo_init(MachineState *machine)
>  if (machine->cpu_model == NULL) {
>  machine->cpu_model = "440EP";
>  }
> -cpu = cpu_ppc_init(machine->cpu_model);
> +cpu = POWERPC_CPU(cpu_generic_init(TYPE_POWERPC_CPU, 
> machine->cpu_model));
>  if (cpu == NULL) {
>  fprintf(stderr, "Unable to initialize CPU!\n");
>  exit(1);
> diff --git a/hw/ppc/ppc4xx_devs.c b/hw/ppc/ppc4xx_devs.c
> index e7f413e..6b38ed7 100644
> --- a/hw/ppc/ppc4xx_devs.c
> +++ b/hw/ppc/ppc4xx_devs.c
> @@ -56,7 +56,7 @@ PowerPCCPU *ppc4xx_init(const char *cpu_model,
>  CPUPPCState *env;
>  
>  /* init CPUs */
> -cpu = cpu_ppc_init(cpu_model);
> +cpu = POWERPC_CPU(cpu_generic_init(TYPE_POWERPC_CPU, cpu_model));
>  if (cpu == NULL) {
>  fprintf(stderr, "Unable to find PowerPC %s CPU definition\n",
>  cpu_model);
> diff --git a/hw/ppc/prep.c b/hw/ppc/prep.c
> index 36d3dcd..00f3321 100644
> --- a/hw/ppc/prep.c
> +++ b/hw/ppc/prep.c
> @@ -520,7 +520,8 @@ static void ppc_prep_init(MachineState *machine)
>  if (machine->cpu_model == NULL)
>  machine->cpu_model = "602";
>  for (i = 0; i < smp_cpus; i++) {
> -cpu = cpu_ppc_init(machine->cpu_model);
> +cpu = POWERPC_CPU(cpu_generic_

Re: [Qemu-devel] [PULL 00/18] ppc-for-2.10 queue 20170714

On Fri, Jul 14, 2017 at 11:25:01AM +0100, Peter Maydell wrote:
> On 14 July 2017 at 07:15, David Gibson  wrote:
> > The following changes since commit 49bcce4b9c11759678fd223aefb48691c4959d4f:
> >
> >   Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2017-07-12' 
> > into staging (2017-07-13 16:56:06 +0100)
> >
> > are available in the git repository at:
> >
> >   git://github.com/dgibson/qemu.git tags/ppc-for-2.10-20170714
> >
> > for you to fetch changes up to e11134d2ac7744569d7c387102368c507d340c30:
> >
> >   pseries: Allow HPT resizing with KVM (2017-07-14 09:28:30 +1000)
> >
> > 
> > ppc patch queue 2017-07-14
> >
> > Two major batches in this set, rather than the usual collection of
> > assorted fixes.
> >
> > * More DRC cleanup.  This gets the state management into a state
> >   which should fix many of the hotplug+migration problems we've
> >   had.  Plus it gets the migration stream format into something
> >   well defined and pretty minimal which we can reasonably support
> >   into the future.
> >
> > * Hashed Page Table resizing.  It's been a while since this was
> >   posted, but it's been through several previous rounds of review.
> >   The kernel parts (both guest and host) are merged in 4.11, so
> >   this is the only remaining piece left to allow resizing of the
> >   HPT in a running guest.
> >
> > There are also a handful of unrelated fixes.
> 
> I see Alex Graf's update to the e500 u-boot image didn't make
> it in to this -- are you planning to send that separately
> or should I pick it up directly? (I definitely would like to
> see that in the 2.10 release, because it fixes a problem where
> our u-boot sources are (a) very old and (b) not even the version
> corresponding to the binary blob we ship...)

Oh, I didn't spot that one.  Can you take it directly?

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature

Re: [Qemu-devel] [Qemu-ppc] [PATCH v2] spapr: ignore decr interrupts when MSR_EE is disabled

2017-07-14 Thread Benjamin Herrenschmidt

On Fri, 2017-07-14 at 09:34 +0530, Nikunj A Dadhania wrote:
> Rebooting a SMP TCG guest is broken for both single/multi threaded TCG.
> 
> When reset happens, all the CPUs are in halted state. First CPU is brought out
> of reset and secondary CPUs would be initialized by the guest kernel using a
> rtas call start-cpu.
> 
> However, in case of TCG, decrementer interrupts keep on coming and waking the
> secondary CPUs up.
> 
> These secondary CPUs would see the decrementer interrupt pending, which makes
> cpu::has_work() to bring them out of wait loop and start executing
> tcg_exec_cpu().
> 
> The problem with this is all the CPUs wake up and start booting SLOF image,
> causing the following exception(4 CPUs TCG VM):

This patch isn't right

> --- a/target/ppc/translate_init.c
> +++ b/target/ppc/translate_init.c
> @@ -8536,7 +8536,8 @@ static bool cpu_has_work_POWER7(CPUState *cs)
>  }
>  if ((env->pending_interrupts & (1u << PPC_INTERRUPT_DECR)) &&
>  (env->spr[SPR_LPCR] & LPCR_P7_PECE1)) {
> -return true;
> +/* Return true only when MSR_EE is enabled */
> +return msr_ee;
>  }

No, the HW will not check MSR_EE, in fact, MSR_EE can purposefully be
left off and we might expect a wakeup still.

I think the right fix is for reset to clear the LPCR PECE bits.

 Cheers
Ben.
 
> >  if ((env->pending_interrupts & (1u << PPC_INTERRUPT_MCK)) &&
>  (env->spr[SPR_LPCR] & LPCR_P7_PECE2)) {
> @@ -8693,7 +8694,8 @@ static bool cpu_has_work_POWER8(CPUState *cs)
>  }
>  if ((env->pending_interrupts & (1u << PPC_INTERRUPT_DECR)) &&
>  (env->spr[SPR_LPCR] & LPCR_P8_PECE3)) {
> -return true;
> +/* Return true only when MSR_EE is enabled */
> +return msr_ee;
>  }
>  if ((env->pending_interrupts & (1u << PPC_INTERRUPT_MCK)) &&
>  (env->spr[SPR_LPCR] & LPCR_P8_PECE4)) {
> @@ -8876,7 +8878,8 @@ static bool cpu_has_work_POWER9(CPUState *cs)
>  /* Decrementer Exception */
>  if ((env->pending_interrupts & (1u << PPC_INTERRUPT_DECR)) &&
>  (env->spr[SPR_LPCR] & LPCR_DEE)) {
> -return true;
> +/* Return true only when MSR_EE is enabled */
> +return msr_ee;
>  }
>  /* Machine Check or Hypervisor Maintenance Exception */
>  if ((env->pending_interrupts & (1u << PPC_INTERRUPT_MCK |

Re: [Qemu-devel] [PATCH] qapi: Fix error handling code on alternate conflict

2017-07-14 Thread no-reply

Hi,

This series failed automatic build test. Please find the testing commands and
their output below. If you have docker installed, you can probably reproduce it
locally.

Message-id: 20170714203313.31041-1-ehabk...@redhat.com
Subject: [Qemu-devel] [PATCH] qapi: Fix error handling code on alternate 
conflict
Type: series

=== TEST SCRIPT BEGIN ===
#!/bin/bash
set -e
git submodule update --init dtc
# Let docker tests dump environment info
export SHOW_ENV=1
export J=8
time make docker-test-quick@centos6
time make docker-test-build@min-glib
time make docker-test-mingw@fedora
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
Switched to a new branch 'test'
41eba1c qapi: Fix error handling code on alternate conflict

=== OUTPUT BEGIN ===
Submodule 'dtc' (git://git.qemu-project.org/dtc.git) registered for path 'dtc'
Cloning into '/var/tmp/patchew-tester-tmp-8tc5gxgh/src/dtc'...
Submodule path 'dtc': checked out '558cd81bdd432769b59bff01240c44f82cfb1a9d'
  BUILD   centos6
make[1]: Entering directory '/var/tmp/patchew-tester-tmp-8tc5gxgh/src'
  ARCHIVE qemu.tgz
  ARCHIVE dtc.tgz
  COPYRUNNER
RUN test-quick in qemu:centos6 
Packages installed:
SDL-devel-1.2.14-7.el6_7.1.x86_64
bison-2.4.1-5.el6.x86_64
ccache-3.1.6-2.el6.x86_64
epel-release-6-8.noarch
flex-2.5.35-9.el6.x86_64
gcc-4.4.7-18.el6.x86_64
git-1.7.1-8.el6.x86_64
glib2-devel-2.28.8-9.el6.x86_64
libfdt-devel-1.4.0-1.el6.x86_64
make-3.81-23.el6.x86_64
package g++ is not installed
pixman-devel-0.32.8-1.el6.x86_64
tar-1.23-15.el6_8.x86_64
zlib-devel-1.2.3-29.el6.x86_64

Environment variables:
PACKAGES=libfdt-devel ccache tar git make gcc g++ flex bison zlib-devel 
glib2-devel SDL-devel pixman-devel epel-release
HOSTNAME=8d368d04784d
TERM=xterm
MAKEFLAGS= -j8
HISTSIZE=1000
J=8
USER=root
CCACHE_DIR=/var/tmp/ccache
EXTRA_CONFIGURE_OPTS=
V=
SHOW_ENV=1
MAIL=/var/spool/mail/root
PATH=/usr/lib/ccache:/usr/lib64/ccache:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PWD=/
LANG=en_US.UTF-8
TARGET_LIST=
HISTCONTROL=ignoredups
SHLVL=1
HOME=/root
TEST_DIR=/tmp/qemu-test
LOGNAME=root
LESSOPEN=||/usr/bin/lesspipe.sh %s
FEATURES= dtc
DEBUG=
G_BROKEN_FILENAMES=1
CCACHE_HASHDIR=
_=/usr/bin/env

Configure options:
--enable-werror --target-list=x86_64-softmmu,aarch64-softmmu 
--prefix=/var/tmp/qemu-build/install
No C++ compiler available; disabling C++ specific optional code
Install prefix/var/tmp/qemu-build/install
BIOS directory/var/tmp/qemu-build/install/share/qemu
binary directory  /var/tmp/qemu-build/install/bin
library directory /var/tmp/qemu-build/install/lib
module directory  /var/tmp/qemu-build/install/lib/qemu
libexec directory /var/tmp/qemu-build/install/libexec
include directory /var/tmp/qemu-build/install/include
config directory  /var/tmp/qemu-build/install/etc
local state directory   /var/tmp/qemu-build/install/var
Manual directory  /var/tmp/qemu-build/install/share/man
ELF interp prefix /usr/gnemul/qemu-%M
Source path   /tmp/qemu-test/src
C compilercc
Host C compiler   cc
C++ compiler  
Objective-C compiler cc
ARFLAGS   rv
CFLAGS-O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -g 
QEMU_CFLAGS   -I/usr/include/pixman-1   -I$(SRC_PATH)/dtc/libfdt -pthread 
-I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include   -fPIE -DPIE -m64 -mcx16 
-D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes 
-Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes 
-fno-strict-aliasing -fno-common -fwrapv  -Wendif-labels 
-Wno-missing-include-dirs -Wempty-body -Wnested-externs -Wformat-security 
-Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration 
-Wold-style-definition -Wtype-limits -fstack-protector-all
LDFLAGS   -Wl,--warn-common -Wl,-z,relro -Wl,-z,now -pie -m64 -g 
make  make
install   install
pythonpython -B
smbd  /usr/sbin/smbd
module supportno
host CPU  x86_64
host big endian   no
target list   x86_64-softmmu aarch64-softmmu
gprof enabled no
sparse enabledno
strip binariesyes
profiler  no
static build  no
pixmansystem
SDL support   yes (1.2.14)
GTK support   no 
GTK GL supportno
VTE support   no 
TLS priority  NORMAL
GNUTLS supportno
GNUTLS rndno
libgcrypt no
libgcrypt kdf no
nettleno 
nettle kdfno
libtasn1  no
curses supportno
virgl support no
curl support  no
mingw32 support   no
Audio drivers oss
Block whitelist (rw) 
Block whitelist (ro) 
VirtFS supportno
VNC support   yes
VNC SASL support  no
VNC JPEG support  no
VNC PNG support   no
xen support   no
brlapi supportno
bluez  supportno
Documentation no
PIE   yes
vde support   no
netmap supportno
Linux AIO support no
ATTR/XATTR support yes
Install blobs yes
KVM support   yes
HAX support   no
TCG support   yes
TCG debug enabled

Re: [Qemu-devel] [PATCH v3 1/8] target/s390x: Implement CSST


On 07/14/2017 11:01 AM, Aurelien Jarno wrote:

+if (parallel_cpus) {
+int mask = 0;
+#if !defined(CONFIG_ATOMIC64)
+mask = -8;
+#elif !defined(CONFIG_ATOMIC128)
+mask = -16;
+#endif
+if (((4 << fc) | (1 << sc)) & mask) {
+cpu_loop_exit_atomic(ENV_GET_CPU(env), ra);
+}
+}


This doesn't look correct. For a 16-byte store, ie sc = 4, and with
ATOMIC128 support, ie mask = -16, the condition is true.


That's WITHOUT atomic128 support that mask = -16.
If we have atomic128, then mask = 0.


+if (parallel_cpus) {
+#ifdef CONFIG_USER_ONLY
+uint32_t *haddr = g2h(a1);
+ov = atomic_cmpxchg__nocheck(haddr, cv, nv);
+#else
+TCGMemOpIdx oi = make_memop_idx(MO_TEUL | MO_ALIGN, mem_idx);
+ov = helper_atomic_cmpxchgl_be_mmu(env, a1, cv, nv, oi, ra);
+#endif


Not a problem with the patch itself, but how complicated would it be to
make helper_atomic_cmpxchgl_be_mmu (just like the o version) available
also in user mode? That would make less #ifdef in the backends.


Dunno.  Probably not too bad.  I have wondered about doing that, as these sorts 
of functions are quite ugly to write.



r~

Re: [Qemu-devel] [PATCH v3 2/8] target/s390x: Implement CONVERT UNICODE insns


On 07/14/2017 11:08 AM, Aurelien Jarno wrote:

On 2017-07-11 17:18, Thomas Huth wrote:

On 10.07.2017 22:45, Richard Henderson wrote:

Signed-off-by: Richard Henderson 
---
  target/s390x/helper.h  |   6 +
  target/s390x/mem_helper.c  | 310 +
  target/s390x/translate.c   |  43 +++
  target/s390x/insn-data.def |  13 ++
  4 files changed, 372 insertions(+)

diff --git a/target/s390x/helper.h b/target/s390x/helper.h
index 23e8d1d..2793cf3 100644
--- a/target/s390x/helper.h
+++ b/target/s390x/helper.h
@@ -107,6 +107,12 @@ DEF_HELPER_2(stfle, i32, env, i64)
  DEF_HELPER_FLAGS_2(lpq, TCG_CALL_NO_WG, i64, env, i64)
  DEF_HELPER_FLAGS_4(stpq, TCG_CALL_NO_WG, void, env, i64, i64, i64)
  DEF_HELPER_4(mvcos, i32, env, i64, i64, i64)
+DEF_HELPER_4(cu12, i32, env, i32, i32, i32)
+DEF_HELPER_4(cu14, i32, env, i32, i32, i32)
+DEF_HELPER_4(cu21, i32, env, i32, i32, i32)
+DEF_HELPER_4(cu24, i32, env, i32, i32, i32)
+DEF_HELPER_4(cu41, i32, env, i32, i32, i32)
+DEF_HELPER_4(cu42, i32, env, i32, i32, i32)
  
  #ifndef CONFIG_USER_ONLY

  DEF_HELPER_3(servc, i32, env, i64, i64)
diff --git a/target/s390x/mem_helper.c b/target/s390x/mem_helper.c
index 513b402..0b18560 100644
--- a/target/s390x/mem_helper.c
+++ b/target/s390x/mem_helper.c

[...]

+static inline uint32_t convert_unicode(CPUS390XState *env, uint32_t r1,
+   uint32_t r2, uint32_t m3, uintptr_t ra,
+   decode_unicode_fn decode,
+   encode_unicode_fn encode)
+{
+uint64_t dst = get_address(env, r1);
+uint64_t dlen = get_length(env, r1 + 1);
+uint64_t src = get_address(env, r2);
+uint64_t slen = get_length(env, r2 + 1);
+bool enh_check = m3 & 1;
+int cc, i;


According to the PoP:

"The R1 and R2 fields [...] must designate an even-
  numbered register; otherwise, a specification excep-
  tion is recognized."

I think you should add such a check for even-numbered registers here.


Actually it should not be done here, but at translation time in
translate.c.

There are a few places where the register number is checked to be even
and later loaded into a temp. I guess that can be replaced by generators
instead?


Yes, that's done in a v3.5 patch that's a part of this thread.


r~

Re: [Qemu-devel] [PATCH v5 1/3] ACPI: Add new ACPI structures and macros

2017-07-14 Thread gengdongjiu

Hi Michael
  Thanks for your comments.

> -邮件原件-
> 发件人: Michael S. Tsirkin [mailto:m...@redhat.com]
> 主题: Re: [Qemu-devel] [PATCH v5 1/3] ACPI: Add new ACPI structures and
> macros
> 
> On Thu, Jul 13, 2017 at 08:00:26PM +0800, gengdongjiu wrote:
> >Laszlo, I pasted Michael's comments here, as shown below. Michael
> > said the definition should use build_append_int_noprefix to add data.
> > but I think it may not good, becuase the section
> > "UEFI_CPER_SEC_PLATFORM_MEM" is runtime recorded as CPER,
> 
> One thing I wanted to understand is how are races avoided. E.g. what if you 
> are
> in the process of writing out CPER and guest reads it.
> I tried to find where is this function writing CPER called and couldn't.
> Can you clarify pls?

Michael, I understand your idea. You want to see the read-write race condition.
this function writing CPER is called in another series of patches. Because they 
are needed
to be review together with KVM kernel, so I put it in another series.
I clarify the method to avoid the read-write race condition

ACPI 6.1, "Generic Hardware Error Source version 2 (GHESv2 - Type 10)" has 
mentioned the avoid method.
For GHESv2 error sources, the OS must acknowledge the consumption of the Error 
Status Block by writing to the “Read Ack Register” listed in the GHESv2 
structure

These are the steps the OS must take once detecting an error from a particular 
GHESv2 error source:
• OS detects error (via interrupt/exception or polling the block status)
• OS copies the error status block
• OS clears the block status field of the error status block
• OS acknowledges the error via Read Ack register. For example:
—OS reads the Read Ack register  X
—OS writes  (( X & ReadAckPreserve) | ReadAckWrite)

So in the Qemu, before Qemu writes the next CPER, it needs to check the "Read 
Ack register". If the "Read Ack Register" does not been written with the 
response, Qemu will wait until 
OS responds the "Read Ack Register", so can avoid the read-write race condition.

In my patch, it misses the check to "Read Ack Register" to avoid the read-write 
race condition. I will add the code in the next code version. 
> 
> > not a ACPI/HEST
> > table member, so it is not generated when system boot up. On the other
> > hand,UEFI_CPER_SEC_PLATFORM_MEM definition is from UEFI spec 2.6,
> N.2.2 Section Descriptor: {0xA5BC1114, 0x6F64, 0x4EDE, {0xB8, 0x63, 0x3E,
> 0x83, 0xED, 0x7C, 0x83, 0xB1}}.
> > if use  build_append_int_noprefix to add, may confuse others.
> 
> Right but ACPI/HEST generation could use cleanup, and
> build_append_int_noprefix would be a nice way to do it.

Ok, I will follow your method to do it. Thanks for the suggestion.

> 
> --
> MST

Re: [Qemu-devel] [PATCH qemu v3] vfio-pci, ppc64/spapr: Reorder group-to-container attaching

2017-07-14 Thread Alexey Kardashevskiy

On 15/07/17 07:17, Alex Williamson wrote:
> On Tue, 11 Jul 2017 15:32:18 +1000
> Alexey Kardashevskiy  wrote:
> 
>> At the moment VFIO PCI device initialization works as follows:
>> vfio_realize
>>  vfio_get_group
>>  vfio_connect_container
>>  register memory listeners (1)
>>  update QEMU groups lists
>>  vfio_kvm_device_add_group
>>
>> Then (example for pseries) the machine reset hook triggers region_add()
>> for all regions where listeners from (1) are listening:
>>
>> ppc_spapr_reset
>>  spapr_phb_reset
>>  spapr_tce_table_enable
>>  memory_region_add_subregion
>>  vfio_listener_region_add
>>  vfio_spapr_create_window
>>
>> This scheme works fine until we need to handle VFIO PCI device hotplug
>> and we want to enable PPC64/sPAPR in-kernel TCE acceleration on,
>> i.e. after PCI hotplug we need a place to call
>> ioctl(vfio_kvm_device_fd, KVM_DEV_VFIO_GROUP_SET_SPAPR_TCE).
>> Since the ioctl needs a LIOBN fd (from sPAPRTCETable) and a IOMMU group fd
>> (from VFIOGroup), vfio_listener_region_add() seems to be the only place
>> for this ioctl().
>>
>> However this only works during boot time because the machine reset
>> happens strictly after all devices are finalized. When hotplug happens,
>> vfio_listener_region_add() is called when a memory listener is registered
>> but when this happens:
>> 1. new group is not added to the container->group_list yet;
>> 2. VFIO KVM device is unaware of the new IOMMU group.
>>
>> This moves bits around to have all necessary VFIO infrastructure
>> in place for both initial startup and hotplug cases.
> 
> 
> Wow, for a fairly straight forward patch this changelog is brutal.  Can
> this be summarized as "Register vfio groups with kvm prior to memory
> listener registration such that kvm-vfio pseudo device ioctls are
> available during the region_add callback"?  If so I'll queue this up
> for a pull request prior to soft freeze.  Thanks,


It just could have looked like an unnecessary change, hence this changelog.
Since it is straight forward, I am ok with your shorter version as well.
Thanks.



> 
> Alex
> 
>  
>> Signed-off-by: Alexey Kardashevskiy 
>> Reviewed-by: David Gibson 
>> ---
>>
>>
>> This is an independend part of a bigger patchset to enable in-kernel
>> acceleration of TCE operations. While I stuck with IOMMU MR QOM-fication,
>> let's try to parallel patchset pieces' review/acceptance.
>>
>> ---
>> Changes:
>> v3:
>> * rebased on current upstream
>>
>> v2:
>> * moved container->initialized back to its correct location
>> * added missing QLIST_REMOVE()
>> ---
>>  hw/vfio/common.c | 19 +++
>>  1 file changed, 11 insertions(+), 8 deletions(-)
>>
>> diff --git a/hw/vfio/common.c b/hw/vfio/common.c
>> index 2965b68e5d..4e75dc8c56 100644
>> --- a/hw/vfio/common.c
>> +++ b/hw/vfio/common.c
>> @@ -1107,6 +1107,14 @@ static int vfio_connect_container(VFIOGroup *group, 
>> AddressSpace *as,
>>  goto free_container_exit;
>>  }
>>  
>> +vfio_kvm_device_add_group(group);
>> +
>> +QLIST_INIT(&container->group_list);
>> +QLIST_INSERT_HEAD(&space->containers, container, next);
>> +
>> +group->container = container;
>> +QLIST_INSERT_HEAD(&container->group_list, group, container_next);
>> +
>>  container->listener = vfio_memory_listener;
>>  
>>  memory_listener_register(&container->listener, container->space->as);
>> @@ -1120,14 +1128,11 @@ static int vfio_connect_container(VFIOGroup *group, 
>> AddressSpace *as,
>>  
>>  container->initialized = true;
>>  
>> -QLIST_INIT(&container->group_list);
>> -QLIST_INSERT_HEAD(&space->containers, container, next);
>> -
>> -group->container = container;
>> -QLIST_INSERT_HEAD(&container->group_list, group, container_next);
>> -
>>  return 0;
>>  listener_release_exit:
>> +QLIST_REMOVE(group, container_next);
>> +QLIST_REMOVE(container, next);
>> +vfio_kvm_device_del_group(group);
>>  vfio_listener_release(container);
>>  
>>  free_container_exit:
>> @@ -1232,8 +1237,6 @@ VFIOGroup *vfio_get_group(int groupid, AddressSpace 
>> *as, Error **errp)
>>  
>>  QLIST_INSERT_HEAD(&vfio_group_list, group, next);
>>  
>> -vfio_kvm_device_add_group(group);
>> -
>>  return group;
>>  
>>  close_fd_exit:
> 


-- 
Alexey

Re: [Qemu-devel] [PATCH v4 2/8] acpi: add vmcoreinfo device

On Sat, Jul 15, 2017 at 1:40 AM, Michael S. Tsirkin  wrote:
> On Sat, Jul 15, 2017 at 01:30:56AM +0200, Marc-André Lureau wrote:
>> Hi
>>
>> On Sat, Jul 15, 2017 at 1:09 AM Michael S. Tsirkin  wrote:
>> >
>> > On Sat, Jul 15, 2017 at 12:12:06AM +0200, Marc-André Lureau wrote:
>> > > Hi
>> > >
>> > > On Fri, Jul 14, 2017 at 10:17 PM, Michael S. Tsirkin  
>> > > wrote:
>> > > > On Fri, Jul 14, 2017 at 10:04:31PM +0200, Laszlo Ersek wrote:
>> > > >> > It worries me that the format of this seems completely undefined
>> > > >> > except in the patchset cover letter.
>> > > >> > I don't think we should merge this before it is.
>> > > >>
>> > > >> I'm not sure what you mean, this patch adds 
>> > > >> "docs/specs/vmcoreinfo.txt".
>> > > >> That file is the first level contract between the guest firmware
>> > > >> (generally, via the linker/loader), the guest kernel driver
>> > > >> (specifically), and QEMU (also specifically).
>> > > >>
>> > > >> The second level contract is the guest kernel's vmcoreinfo ELF note
>> > > >> (which is pointed-to by the first level contract). The layout of that 
>> > > >> is
>> > > >> specified elsewhere indeed (I don't think it belongs here).
>> > > >>
>> > > >> We've taken care not to trust anything coming from the guest kernel.
>> > > >>
>> > > >> Can you clarify please?
>> > > >>
>> > > >> Thanks
>> > > >> Laszlo
>> > > >
>> > > > All there is is this:
>> > > >
>> > > > +Version 0 content:
>> > > > +
>> > > > + uint64 paddr:
>> > > > +  Physical address of the Linux vmcoreinfo ELF note.
>> > > > + uint32 size:
>> > > > +  Size of the vmcoreinfo ELF note.
>> > > >
>> > > > It isn't defined here what is the Linux vmcoreinfo ELF note.
>> > > > You want a bit more info so people trying to use it know where to start
>> > > > and what they can get out of it.
>> > >
>> > > QEMU is not responsible for the content of the ELF note. All there is 
>> > > afaik is:
>> > >
>> > > https://www.kernel.org/doc/Documentation/ABI/testing/sysfs-kernel-vmcoreinfo
>> > >
>> > > The rest you need to dig in the kernel code and git history I think.
>> >
>> > We need to do a good enough job to at least make it possible for people
>> > to make use of it without reading your python code.
>> >
>> > Documentation/kdump/kdump.txt has a bit more information.
>> >
>> > All of the necessary information about the system kernel's core 
>> > image is
>> > encoded in the ELF format, and stored in a reserved area of memory
>> > before a crash. The physical address of the start of the ELF 
>> > header is
>> > passed to the dump-capture kernel through the elfcorehdr= boot
>> > parameter. Optionally the size of the ELF header can also be passed
>> > when using the elfcorehdr=[size[KMG]@]offset[KMG] syntax.
>> >
>> >
>> > With the dump-capture kernel, you can access the memory image 
>> > through
>> > /proc/vmcore. This exports the dump as an ELF-format file that you 
>> > can
>> > write out using file copy commands such as cp or scp. Further, you 
>> > can
>> > use analysis tools such as the GNU Debugger (GDB) and the Crash 
>> > tool to
>> > debug the dump file. This method ensures that the dump pages are 
>> > correctly
>> > ordered.
>> >
>> > I know where to find it but most people likely won't be able to.
>> >
>>
>> What do you learn from that regarding vmcoreinfo? That is it being
>> used by kdump? and that you can use gdb/crash, that we already use to
>> analyse our dumps.
>
> Some things I learn:
>
> That its written by a dump-capture kernel (so you need to set one up
> if you want it to work!).

I don't follow you, you don't need a dump-capture kernel / kdump to
have the vmcoreinfo note.

>
> That it's an ELF-format file, that one can use analysis tools such as
> the GNU Debugger (GDB) and the Crash tool to debug the dump file.

Well, that's what kdump does, but qemu is the one making the dump in
our case. Quite irrelevant for us.

>
> There's more info scattered in other places.
>
> Why do you get to document it? Because you are the one exposing it
> across the hypervisor/vm boundary where it will need to be
> understood by people/tools not running within guest.
>
> So "just read the script in qemu source" is not how an interface
> should be documented.

I don't understand the issue, it's a kernel ELF note that qemu passes
for dump/crash tools in the dump headers/sections.

>
>> >
>> > BTW why does it pass ELF header size? Any idea?
>>
>>
>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d3bf37955d46718ee1a7f1fc69f953d2328ba7c2
>
> Right but I mean, ELF header has two options: 32 and 64 bit, that's all.
>
> --
> MST



-- 
Marc-André Lureau

Re: [Qemu-devel] [PATCH v4 2/8] acpi: add vmcoreinfo device

On Sat, Jul 15, 2017 at 01:30:56AM +0200, Marc-André Lureau wrote:
> Hi
> 
> On Sat, Jul 15, 2017 at 1:09 AM Michael S. Tsirkin  wrote:
> >
> > On Sat, Jul 15, 2017 at 12:12:06AM +0200, Marc-André Lureau wrote:
> > > Hi
> > >
> > > On Fri, Jul 14, 2017 at 10:17 PM, Michael S. Tsirkin  
> > > wrote:
> > > > On Fri, Jul 14, 2017 at 10:04:31PM +0200, Laszlo Ersek wrote:
> > > >> > It worries me that the format of this seems completely undefined
> > > >> > except in the patchset cover letter.
> > > >> > I don't think we should merge this before it is.
> > > >>
> > > >> I'm not sure what you mean, this patch adds 
> > > >> "docs/specs/vmcoreinfo.txt".
> > > >> That file is the first level contract between the guest firmware
> > > >> (generally, via the linker/loader), the guest kernel driver
> > > >> (specifically), and QEMU (also specifically).
> > > >>
> > > >> The second level contract is the guest kernel's vmcoreinfo ELF note
> > > >> (which is pointed-to by the first level contract). The layout of that 
> > > >> is
> > > >> specified elsewhere indeed (I don't think it belongs here).
> > > >>
> > > >> We've taken care not to trust anything coming from the guest kernel.
> > > >>
> > > >> Can you clarify please?
> > > >>
> > > >> Thanks
> > > >> Laszlo
> > > >
> > > > All there is is this:
> > > >
> > > > +Version 0 content:
> > > > +
> > > > + uint64 paddr:
> > > > +  Physical address of the Linux vmcoreinfo ELF note.
> > > > + uint32 size:
> > > > +  Size of the vmcoreinfo ELF note.
> > > >
> > > > It isn't defined here what is the Linux vmcoreinfo ELF note.
> > > > You want a bit more info so people trying to use it know where to start
> > > > and what they can get out of it.
> > >
> > > QEMU is not responsible for the content of the ELF note. All there is 
> > > afaik is:
> > >
> > > https://www.kernel.org/doc/Documentation/ABI/testing/sysfs-kernel-vmcoreinfo
> > >
> > > The rest you need to dig in the kernel code and git history I think.
> >
> > We need to do a good enough job to at least make it possible for people
> > to make use of it without reading your python code.
> >
> > Documentation/kdump/kdump.txt has a bit more information.
> >
> > All of the necessary information about the system kernel's core 
> > image is
> > encoded in the ELF format, and stored in a reserved area of memory
> > before a crash. The physical address of the start of the ELF header 
> > is
> > passed to the dump-capture kernel through the elfcorehdr= boot
> > parameter. Optionally the size of the ELF header can also be passed
> > when using the elfcorehdr=[size[KMG]@]offset[KMG] syntax.
> >
> >
> > With the dump-capture kernel, you can access the memory image 
> > through
> > /proc/vmcore. This exports the dump as an ELF-format file that you 
> > can
> > write out using file copy commands such as cp or scp. Further, you 
> > can
> > use analysis tools such as the GNU Debugger (GDB) and the Crash 
> > tool to
> > debug the dump file. This method ensures that the dump pages are 
> > correctly
> > ordered.
> >
> > I know where to find it but most people likely won't be able to.
> >
> 
> What do you learn from that regarding vmcoreinfo? That is it being
> used by kdump? and that you can use gdb/crash, that we already use to
> analyse our dumps.

Some things I learn:

That its written by a dump-capture kernel (so you need to set one up
if you want it to work!).

That it's an ELF-format file, that one can use analysis tools such as
the GNU Debugger (GDB) and the Crash tool to debug the dump file.

There's more info scattered in other places.

Why do you get to document it? Because you are the one exposing it
across the hypervisor/vm boundary where it will need to be
understood by people/tools not running within guest.

So "just read the script in qemu source" is not how an interface
should be documented.

> >
> > BTW why does it pass ELF header size? Any idea?
> 
> 
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d3bf37955d46718ee1a7f1fc69f953d2328ba7c2

Right but I mean, ELF header has two options: 32 and 64 bit, that's all.

-- 
MST

Re: [Qemu-devel] [PATCH v4 0/8] KASLR kernel dump support

2017-07-14 Thread no-reply

Hi,

This series seems to have some coding style problems. See output below for
more information:

Message-id: 20170714182012.4595-1-marcandre.lur...@redhat.com
Subject: [Qemu-devel] [PATCH v4 0/8] KASLR kernel dump support
Type: series

=== TEST SCRIPT BEGIN ===
#!/bin/bash

BASE=base
n=1
total=$(git log --oneline $BASE.. | wc -l)
failed=0

git config --local diff.renamelimit 0
git config --local diff.renames True

commits="$(git log --format=%H --reverse $BASE..)"
for c in $commits; do
echo "Checking PATCH $n/$total: $(git log -n 1 --format=%s $c)..."
if ! git show $c --format=email | ./scripts/checkpatch.pl --mailback -; then
failed=1
echo
fi
n=$((n+1))
done

exit $failed
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
Switched to a new branch 'test'
04e62b8 MAINTAINERS: add Dump maintainers
947599d scripts/dump-guest-memory.py: add vmcoreinfo
7f07c2f kdump: add vmcoreinfo ELF note
dffc775 dump: add vmcoreinfo ELF note
5f6bf26 tests: add simple vmcoreinfo test
6dd3154 stubs: add vmcoreinfo_get() stub
232c73c acpi: add vmcoreinfo device
e13fbf6 vmgenid: replace x-write-pointer-available hack

=== OUTPUT BEGIN ===
Checking PATCH 1/8: vmgenid: replace x-write-pointer-available hack...
Checking PATCH 2/8: acpi: add vmcoreinfo device...
WARNING: line over 80 characters
#388: FILE: hw/acpi/vmcoreinfo.c:153:
+VMSTATE_UINT8_ARRAY(vmcoreinfo_addr_le, VMCoreInfoState, 
sizeof(uint64_t)),

total: 0 errors, 1 warnings, 470 lines checked

Your patch has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
Checking PATCH 3/8: stubs: add vmcoreinfo_get() stub...
Checking PATCH 4/8: tests: add simple vmcoreinfo test...
Checking PATCH 5/8: dump: add vmcoreinfo ELF note...
Checking PATCH 6/8: kdump: add vmcoreinfo ELF note...
WARNING: line over 80 characters
#30: FILE: dump.c:847:
+get_note_sizes(s, s->vmcoreinfo, &hsize, &name_size, 
&size_vmcoreinfo_desc);

WARNING: line over 80 characters
#47: FILE: dump.c:957:
+get_note_sizes(s, s->vmcoreinfo, &hsize, &name_size, 
&size_vmcoreinfo_desc);

total: 0 errors, 2 warnings, 32 lines checked

Your patch has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
Checking PATCH 7/8: scripts/dump-guest-memory.py: add vmcoreinfo...
ERROR: Use of volatile is usually wrong: see 
Documentation/volatile-considered-harmful.txt
#26: FILE: hw/acpi/vmcoreinfo.c:168:
+static volatile VMCoreInfoState *vmcoreinfo_gdb_helper G_GNUC_UNUSED;

total: 1 errors, 0 warnings, 86 lines checked

Your patch has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

Checking PATCH 8/8: MAINTAINERS: add Dump maintainers...
=== OUTPUT END ===

Test command exited with code: 1


---
Email generated automatically by Patchew [http://patchew.org/].
Please send your feedback to patchew-de...@freelists.org

Re: [Qemu-devel] [PATCH v4 2/8] acpi: add vmcoreinfo device

Hi

On Sat, Jul 15, 2017 at 1:09 AM Michael S. Tsirkin  wrote:
>
> On Sat, Jul 15, 2017 at 12:12:06AM +0200, Marc-André Lureau wrote:
> > Hi
> >
> > On Fri, Jul 14, 2017 at 10:17 PM, Michael S. Tsirkin  
> > wrote:
> > > On Fri, Jul 14, 2017 at 10:04:31PM +0200, Laszlo Ersek wrote:
> > >> > It worries me that the format of this seems completely undefined
> > >> > except in the patchset cover letter.
> > >> > I don't think we should merge this before it is.
> > >>
> > >> I'm not sure what you mean, this patch adds "docs/specs/vmcoreinfo.txt".
> > >> That file is the first level contract between the guest firmware
> > >> (generally, via the linker/loader), the guest kernel driver
> > >> (specifically), and QEMU (also specifically).
> > >>
> > >> The second level contract is the guest kernel's vmcoreinfo ELF note
> > >> (which is pointed-to by the first level contract). The layout of that is
> > >> specified elsewhere indeed (I don't think it belongs here).
> > >>
> > >> We've taken care not to trust anything coming from the guest kernel.
> > >>
> > >> Can you clarify please?
> > >>
> > >> Thanks
> > >> Laszlo
> > >
> > > All there is is this:
> > >
> > > +Version 0 content:
> > > +
> > > + uint64 paddr:
> > > +  Physical address of the Linux vmcoreinfo ELF note.
> > > + uint32 size:
> > > +  Size of the vmcoreinfo ELF note.
> > >
> > > It isn't defined here what is the Linux vmcoreinfo ELF note.
> > > You want a bit more info so people trying to use it know where to start
> > > and what they can get out of it.
> >
> > QEMU is not responsible for the content of the ELF note. All there is afaik 
> > is:
> >
> > https://www.kernel.org/doc/Documentation/ABI/testing/sysfs-kernel-vmcoreinfo
> >
> > The rest you need to dig in the kernel code and git history I think.
>
> We need to do a good enough job to at least make it possible for people
> to make use of it without reading your python code.
>
> Documentation/kdump/kdump.txt has a bit more information.
>
> All of the necessary information about the system kernel's core image 
> is
> encoded in the ELF format, and stored in a reserved area of memory
> before a crash. The physical address of the start of the ELF header is
> passed to the dump-capture kernel through the elfcorehdr= boot
> parameter. Optionally the size of the ELF header can also be passed
> when using the elfcorehdr=[size[KMG]@]offset[KMG] syntax.
>
>
> With the dump-capture kernel, you can access the memory image through
> /proc/vmcore. This exports the dump as an ELF-format file that you can
> write out using file copy commands such as cp or scp. Further, you can
> use analysis tools such as the GNU Debugger (GDB) and the Crash tool 
> to
> debug the dump file. This method ensures that the dump pages are 
> correctly
> ordered.
>
> I know where to find it but most people likely won't be able to.
>

What do you learn from that regarding vmcoreinfo? That is it being
used by kdump? and that you can use gdb/crash, that we already use to
analyse our dumps.

>
> BTW why does it pass ELF header size? Any idea?


https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d3bf37955d46718ee1a7f1fc69f953d2328ba7c2

Re: [Qemu-devel] [PATCH v4 0/8] KASLR kernel dump support

On Sat, Jul 15, 2017 at 12:31:36AM +0200, Marc-André Lureau wrote:
> Hi
> 
> On Sat, Jul 15, 2017 at 12:23 AM, Michael S. Tsirkin  wrote:
> > On Fri, Jul 14, 2017 at 10:21:43PM +0200, Laszlo Ersek wrote:
> >> On 07/14/17 21:59, Michael S. Tsirkin wrote:
> >> > On Fri, Jul 14, 2017 at 08:20:03PM +0200, Marc-André Lureau wrote:
> >> >> Recent linux kernels enable KASLR to randomize phys/virt memory
> >> >> addresses. This series aims to provide enough information in qemu
> >> >> dumps so that crash utility can work with randomized kernel too (it
> >> >> hasn't been tested on other archs than x86 though, help welcome).
> >> >>
> >> >> The vmcoreinfo device is an emulated ACPI device that exposes a 4k
> >> >> memory range to the guest to store various informations useful to
> >> >> debug the guest OS. (it is greatly inspired by the VMGENID device
> >> >> implementation). The version field with value 0 is meant to give
> >> >> paddr/size of the VMCOREINFO ELF PT_NOTE, other values can be used for
> >> >> different purposes or OSes. (note: some wanted to see pvpanic somehow
> >> >> merged with this device, I have no clear idea how to do that, nor do I
> >> >> think this is a good idea since the devices are quite different, used
> >> >> at different time for different purposes. And it can be done as a
> >> >> future iteration if it is appropriate, feel free to send patches)
> >> >
> >> > First, I think you underestimate the difficulty of maintaining
> >> > compatibility.
> >> >
> >> > Second, this seems racy - how do you know when is guest done writing out
> >> > the data?
> >>
> >> What data exactly?
> >>
> >> The guest kernel module points the fields in the "vmcoreinfo page" to
> >> the then-existent vmcoreinfo ELF note. At that point, the ELF note is
> >> complete.
> >
> > When does this happen?
> 
> Very early boot afaik. But the exact details on when to expose it is
> left to the kernel side. For now, it's a test module I load manually.
> 
> >
> >> If we catch the guest with a dump request while the kernel module is
> >> setting up the fields (i.e., the fields are not consistent), then we'll
> >> catch that in our sanity checks, and the note won't be extracted.
> >
> > Are there assumptions about e.g. in which order pa and size
> > are written out then? Atomicity of these writes?
> 
> I expect it to be atomic, but as Laszlo said, the code should be quite
> careful when trying to read the data.

Same when writing it out.  Worth documenting too.

> >
> >> This
> >> is no different from the case when you simply dump the guest RAM before
> >> the module got invoked.
> >>
> >> > Given you have very little data to export (PA, size - do
> >> > you even need size?)
> >>
> >> Yes, it tells us in advance how much memory to allocate before we copy
> >> out the vmcoreinfo ELF note (and we enforce a sanity limit on the size).
> >>
> >> > - how about just using an ACPI method do it,
> >>
> >> Do what exactly?
> >
> > Pass address + size to host - that's what the interface is doing,
> > isn't it?
> >
> 
> 
> The memory region is meant to be usable for other OS, or to export
> more details in the future.

That's the issue. You will never be able to increment version
just to add more data because that will break old hypervisors.

> I think if we add a method, it would be to
> tell qemu that the memory has been written, but it may still be
> corrupted at the time we read it. So I am not sure it will really help

So don't. Just pass PA and size to method as arguments and let it figure
out how to pass it to QEMU. To extend, you will simply add another
method - which one is present tells you what does hypervisor
support, which one is called tells you what does guest support.

What to do there internally? I don't mind it if it sticks this data
in reserved memory like you did here. And then you won't need to
reserve a full 4K, just a couple of bytes as it will be safe to extend.

> 
> >> > instead of exporting a physical addess and storing address there.  This
> >> > way you can add more methods as you add functionality.
> >>
> >> I'm not saying this is a bad idea (especially because I don't fully
> >> understand your point), but I will say that I'm quite sad that you are
> >> sending Marc-André back to the drawing board after he posted v4 -- also
> >> invalidating my review efforts. :/
> >>
> >> Laszlo
> >
> > You are right, I should have looked at this sooner. Early RFC
> > suggested writing into fw cfg directly. I couldn't find any
> > discussion around this - why was this abandoned?
> 
> Violation (or rather abuse) of layers iirc

Hmm.  I tried to find discussion about that and failed.  It is available
through QEMU0002 in ACPI - would it be OK if guests went through that?
I do not mind the extra indirection so much, but I don't think
host/guest interface compatibility issues are fully thought through.

> 
> 
> -- 
> Marc-André Lureau

Re: [Qemu-devel] [PATCH v4 2/8] acpi: add vmcoreinfo device

On Sat, Jul 15, 2017 at 12:12:06AM +0200, Marc-André Lureau wrote:
> Hi
> 
> On Fri, Jul 14, 2017 at 10:17 PM, Michael S. Tsirkin  wrote:
> > On Fri, Jul 14, 2017 at 10:04:31PM +0200, Laszlo Ersek wrote:
> >> > It worries me that the format of this seems completely undefined
> >> > except in the patchset cover letter.
> >> > I don't think we should merge this before it is.
> >>
> >> I'm not sure what you mean, this patch adds "docs/specs/vmcoreinfo.txt".
> >> That file is the first level contract between the guest firmware
> >> (generally, via the linker/loader), the guest kernel driver
> >> (specifically), and QEMU (also specifically).
> >>
> >> The second level contract is the guest kernel's vmcoreinfo ELF note
> >> (which is pointed-to by the first level contract). The layout of that is
> >> specified elsewhere indeed (I don't think it belongs here).
> >>
> >> We've taken care not to trust anything coming from the guest kernel.
> >>
> >> Can you clarify please?
> >>
> >> Thanks
> >> Laszlo
> >
> > All there is is this:
> >
> > +Version 0 content:
> > +
> > + uint64 paddr:
> > +  Physical address of the Linux vmcoreinfo ELF note.
> > + uint32 size:
> > +  Size of the vmcoreinfo ELF note.
> >
> > It isn't defined here what is the Linux vmcoreinfo ELF note.
> > You want a bit more info so people trying to use it know where to start
> > and what they can get out of it.
> 
> QEMU is not responsible for the content of the ELF note. All there is afaik 
> is:
> 
> https://www.kernel.org/doc/Documentation/ABI/testing/sysfs-kernel-vmcoreinfo
> 
> The rest you need to dig in the kernel code and git history I think.

We need to do a good enough job to at least make it possible for people
to make use of it without reading your python code.

Documentation/kdump/kdump.txt has a bit more information.

All of the necessary information about the system kernel's core image is
encoded in the ELF format, and stored in a reserved area of memory
before a crash. The physical address of the start of the ELF header is
passed to the dump-capture kernel through the elfcorehdr= boot
parameter. Optionally the size of the ELF header can also be passed
when using the elfcorehdr=[size[KMG]@]offset[KMG] syntax.

With the dump-capture kernel, you can access the memory image through
/proc/vmcore. This exports the dump as an ELF-format file that you can
write out using file copy commands such as cp or scp. Further, you can
use analysis tools such as the GNU Debugger (GDB) and the Crash tool to
debug the dump file. This method ensures that the dump pages are 
correctly
ordered.

I know where to find it but most people likely won't be able to.

BTW why does it pass ELF header size? Any idea?

> -- 
> Marc-André Lureau

Re: [Qemu-devel] [PATCH v4 0/8] KASLR kernel dump support

Hi

On Sat, Jul 15, 2017 at 12:23 AM, Michael S. Tsirkin  wrote:
> On Fri, Jul 14, 2017 at 10:21:43PM +0200, Laszlo Ersek wrote:
>> On 07/14/17 21:59, Michael S. Tsirkin wrote:
>> > On Fri, Jul 14, 2017 at 08:20:03PM +0200, Marc-André Lureau wrote:
>> >> Recent linux kernels enable KASLR to randomize phys/virt memory
>> >> addresses. This series aims to provide enough information in qemu
>> >> dumps so that crash utility can work with randomized kernel too (it
>> >> hasn't been tested on other archs than x86 though, help welcome).
>> >>
>> >> The vmcoreinfo device is an emulated ACPI device that exposes a 4k
>> >> memory range to the guest to store various informations useful to
>> >> debug the guest OS. (it is greatly inspired by the VMGENID device
>> >> implementation). The version field with value 0 is meant to give
>> >> paddr/size of the VMCOREINFO ELF PT_NOTE, other values can be used for
>> >> different purposes or OSes. (note: some wanted to see pvpanic somehow
>> >> merged with this device, I have no clear idea how to do that, nor do I
>> >> think this is a good idea since the devices are quite different, used
>> >> at different time for different purposes. And it can be done as a
>> >> future iteration if it is appropriate, feel free to send patches)
>> >
>> > First, I think you underestimate the difficulty of maintaining
>> > compatibility.
>> >
>> > Second, this seems racy - how do you know when is guest done writing out
>> > the data?
>>
>> What data exactly?
>>
>> The guest kernel module points the fields in the "vmcoreinfo page" to
>> the then-existent vmcoreinfo ELF note. At that point, the ELF note is
>> complete.
>
> When does this happen?

Very early boot afaik. But the exact details on when to expose it is
left to the kernel side. For now, it's a test module I load manually.

>
>> If we catch the guest with a dump request while the kernel module is
>> setting up the fields (i.e., the fields are not consistent), then we'll
>> catch that in our sanity checks, and the note won't be extracted.
>
> Are there assumptions about e.g. in which order pa and size
> are written out then? Atomicity of these writes?

I expect it to be atomic, but as Laszlo said, the code should be quite
careful when trying to read the data.

>
>> This
>> is no different from the case when you simply dump the guest RAM before
>> the module got invoked.
>>
>> > Given you have very little data to export (PA, size - do
>> > you even need size?)
>>
>> Yes, it tells us in advance how much memory to allocate before we copy
>> out the vmcoreinfo ELF note (and we enforce a sanity limit on the size).
>>
>> > - how about just using an ACPI method do it,
>>
>> Do what exactly?
>
> Pass address + size to host - that's what the interface is doing,
> isn't it?
>


The memory region is meant to be usable for other OS, or to export
more details in the future. I think if we add a method, it would be to
tell qemu that the memory has been written, but it may still be
corrupted at the time we read it. So I am not sure it will really help


>> > instead of exporting a physical addess and storing address there.  This
>> > way you can add more methods as you add functionality.
>>
>> I'm not saying this is a bad idea (especially because I don't fully
>> understand your point), but I will say that I'm quite sad that you are
>> sending Marc-André back to the drawing board after he posted v4 -- also
>> invalidating my review efforts. :/
>>
>> Laszlo
>
> You are right, I should have looked at this sooner. Early RFC
> suggested writing into fw cfg directly. I couldn't find any
> discussion around this - why was this abandoned?

Violation (or rather abuse) of layers iirc



-- 
Marc-André Lureau

Re: [Qemu-devel] [PATCH v4 0/8] KASLR kernel dump support

On Fri, Jul 14, 2017 at 10:21:43PM +0200, Laszlo Ersek wrote:
> On 07/14/17 21:59, Michael S. Tsirkin wrote:
> > On Fri, Jul 14, 2017 at 08:20:03PM +0200, Marc-André Lureau wrote:
> >> Recent linux kernels enable KASLR to randomize phys/virt memory
> >> addresses. This series aims to provide enough information in qemu
> >> dumps so that crash utility can work with randomized kernel too (it
> >> hasn't been tested on other archs than x86 though, help welcome).
> >>
> >> The vmcoreinfo device is an emulated ACPI device that exposes a 4k
> >> memory range to the guest to store various informations useful to
> >> debug the guest OS. (it is greatly inspired by the VMGENID device
> >> implementation). The version field with value 0 is meant to give
> >> paddr/size of the VMCOREINFO ELF PT_NOTE, other values can be used for
> >> different purposes or OSes. (note: some wanted to see pvpanic somehow
> >> merged with this device, I have no clear idea how to do that, nor do I
> >> think this is a good idea since the devices are quite different, used
> >> at different time for different purposes. And it can be done as a
> >> future iteration if it is appropriate, feel free to send patches)
> > 
> > First, I think you underestimate the difficulty of maintaining
> > compatibility.
> > 
> > Second, this seems racy - how do you know when is guest done writing out
> > the data?
> 
> What data exactly?
> 
> The guest kernel module points the fields in the "vmcoreinfo page" to
> the then-existent vmcoreinfo ELF note. At that point, the ELF note is
> complete.

When does this happen?

> If we catch the guest with a dump request while the kernel module is
> setting up the fields (i.e., the fields are not consistent), then we'll
> catch that in our sanity checks, and the note won't be extracted.

Are there assumptions about e.g. in which order pa and size
are written out then? Atomicity of these writes?

> This
> is no different from the case when you simply dump the guest RAM before
> the module got invoked.
> 
> > Given you have very little data to export (PA, size - do
> > you even need size?)
> 
> Yes, it tells us in advance how much memory to allocate before we copy
> out the vmcoreinfo ELF note (and we enforce a sanity limit on the size).
> 
> > - how about just using an ACPI method do it,
> 
> Do what exactly?

Pass address + size to host - that's what the interface is doing,
isn't it?

> > instead of exporting a physical addess and storing address there.  This
> > way you can add more methods as you add functionality.
> 
> I'm not saying this is a bad idea (especially because I don't fully
> understand your point), but I will say that I'm quite sad that you are
> sending Marc-André back to the drawing board after he posted v4 -- also
> invalidating my review efforts. :/
> 
> Laszlo

You are right, I should have looked at this sooner. Early RFC
suggested writing into fw cfg directly. I couldn't find any
discussion around this - why was this abandoned?

-- 
MST

Re: [Qemu-devel] [PATCH v4 2/8] acpi: add vmcoreinfo device

Hi

On Fri, Jul 14, 2017 at 10:17 PM, Michael S. Tsirkin  wrote:
> On Fri, Jul 14, 2017 at 10:04:31PM +0200, Laszlo Ersek wrote:
>> > It worries me that the format of this seems completely undefined
>> > except in the patchset cover letter.
>> > I don't think we should merge this before it is.
>>
>> I'm not sure what you mean, this patch adds "docs/specs/vmcoreinfo.txt".
>> That file is the first level contract between the guest firmware
>> (generally, via the linker/loader), the guest kernel driver
>> (specifically), and QEMU (also specifically).
>>
>> The second level contract is the guest kernel's vmcoreinfo ELF note
>> (which is pointed-to by the first level contract). The layout of that is
>> specified elsewhere indeed (I don't think it belongs here).
>>
>> We've taken care not to trust anything coming from the guest kernel.
>>
>> Can you clarify please?
>>
>> Thanks
>> Laszlo
>
> All there is is this:
>
> +Version 0 content:
> +
> + uint64 paddr:
> +  Physical address of the Linux vmcoreinfo ELF note.
> + uint32 size:
> +  Size of the vmcoreinfo ELF note.
>
> It isn't defined here what is the Linux vmcoreinfo ELF note.
> You want a bit more info so people trying to use it know where to start
> and what they can get out of it.

QEMU is not responsible for the content of the ELF note. All there is afaik is:

https://www.kernel.org/doc/Documentation/ABI/testing/sysfs-kernel-vmcoreinfo

The rest you need to dig in the kernel code and git history I think.

-- 
Marc-André Lureau

Re: [Qemu-devel] [PATCH V3 1/9] specs/qcow2: add compress format extension

2017-07-14 Thread Peter Lieven

Am 14.07.2017 um 16:52 schrieb Eric Blake:
> On 07/14/2017 04:56 AM, Peter Lieven wrote:
>> Signed-off-by: Peter Lieven 
>> ---
>>  docs/interop/qcow2.txt | 35 ++-
>>  1 file changed, 34 insertions(+), 1 deletion(-)
>>
>> diff --git a/docs/interop/qcow2.txt b/docs/interop/qcow2.txt
>> index d7fdb1f..c2d3dab 100644
>> --- a/docs/interop/qcow2.txt
>> +++ b/docs/interop/qcow2.txt
>> @@ -86,7 +86,12 @@ in the description of a field.
>>  be written to (unless for regaining
>>  consistency).
>>  
>> -Bits 2-63:  Reserved (set to 0)
>> +Bit 2:  Compress format bit.  If and only if this 
>> bit
>> +is set then the compress format extension
>> +MUST be present and MUST be parsed and 
>> checked
>> +for compatibility.
>> +
>> +Bits 3-63:  Reserved (set to 0)
>>  
>>   80 -  87:  compatible_features
>>  Bitmask of compatible features. An implementation can
>> @@ -137,6 +142,7 @@ be stored. Each extension has a structure like the 
>> following:
>>  0x6803f857 - Feature name table
>>  0x23852875 - Bitmaps extension
>>  0x0537be77 - Full disk encryption header pointer
>> +0xC03183A3 - Compress format extension
>>  other  - Unknown header extension, can be safely
>>   ignored
>>  
>> @@ -311,6 +317,33 @@ The algorithms used for encryption vary depending on 
>> the method
>> in the LUKS header, with the physical disk sector as the
>> input tweak.
>>  
>> +
>> +== Compress format extension ==
>> +
>> +The compress format extension is an optional header extension. It provides
>> +the ability to specify the compress algorithm and compress parameters
>> +that are used for compressed clusters. This new header MUST be present if
>> +the incompatible-feature bit "compress format bit" is set and MUST be absent
>> +otherwise.
> Probably worth mentioning that omitting the incompatible "Compress
> format bit" is the same as setting it and populating the Compress format
> extension with "zlib"/0.  Don't know if that fits better here, or up
> above at the incompatible feature bit section.

Its not exactly the same. Its zlib with default compression, but the old
default was to use 12 windowBits whereas windowBits are 15 when
the header is present. The reason behind this is that the inflate call
needs the maximum alllowed windowBits as parameter. In theory
it should fail if it has a 12 as parameter and receives an input stream
with 15. My zlib doesn't, but the docs say something else.
The change of the windowBits to 15 yields in about 15% better compression
time and about 5% better compression ratio.

>
>> +
>> +The fields of the compress format extension are:
>> +
>> +Byte  0 - 15:  compress_format_name (padded with zeros, but not
>> +   necessarily null terminated if it has full length).
>> +   Valid compression format names currently are:
>> +
>> +   zlib: Standard zlib deflate compression
> 16 bytes,
>
>> +
>> +  16:  compress_level (uint8_t)
> + 1, means you now have to pad the struct. I'd make the
> compress_format_name be only 15 bits (bytes 0-14, compress_level at 15),
> so that you have nice alignment without wasting padding.

Good point, I will change that.

>
>> +
>> +   0 = default compress level (valid for all formats, 
>> default)
>> +
>> +   Additional valid compression levels for zlib compression:
>> +
>> +   All values between 1 and 9. 1 gives best speed, 9 gives 
>> best
>> +   compression.
> Is there a particular value between 1 and 9 that matches the output you
> get when using 0?
>
Thats depending on what the macro Z_DEFAULT_COMPRESSION points to.

Its provided by libz. The zlib docs say it currently is set to level 6.


Peter

Re: [Qemu-devel] [edk2] investigating TPM for OVMF-on-QEMU

On 07/14/17 22:30, Peter Jones wrote:
> On Fri, Jul 14, 2017 at 08:04:14PM +0200, Laszlo Ersek wrote:

>> (2e) Modules that we should use. Again, in increasing order of
>>  dependence. And here I'll comment as well on what these do:
>>
>>Tcg2Config/Tcg2ConfigPei.inf -- Informs the firmware globally
>>about the TPM device type. This
>>module can perform device
>>detection or read a cached value
>>from a non-volatile UEFI
>>variable.
>>
>>Tcg2Pei/Tcg2Pei.inf  -- Initializes the TPM device and
>>measures the firmware volumes in
>>the PEI phase into the TPM's
>>platform config registers.
>>
>>Tcg2Dxe/Tcg2Dxe.inf  -- Measures DXE phase (and later)
>>modules into the TPM's PCRs, and
>>also lets the OS boot loader
>>measure things, by exposing the
>>EFI_TCG2_PROTOCOL.
>>
>>Tcg2Config/Tcg2ConfigDxe.inf -- Provides a Setup TUI interface to
>>configure the TPM. IIUC, it can
>>also save the configured TPM type
>>for subsequent boots (see
>>Tcg2ConfigPei.inf above).
>>
>>  This driver stack supports the TIS (MMIO) hardware interface, which
>>  is advertized to the OS in the TPM2 ACPI Table's "start method"
>>  field with value 6. (The according macro is TPM2_START_METHOD_MMIO
>>  in the QEMU source code, and EFI_TPM2_ACPI_TABLE_START_METHOD_TIS
>>  in the edk2 source code.)
>>
>>  Including these drivers should result in a functional
>>  EFI_TCG2_PROTOCOL, which is what OS boot loaders primarily care
>>  about, as I understand.
>>
>>  Importantly, the driver stack above requires PEI-phase variable
>>  access, therefore
>>   must be solved
>>  first.
>>
>>  (I have had patches for said BZ ready for a while. I've failed to
>>  upstream them thus far because a pflash-based varstore is a hard
>>  requirement for them. I think that's a natural requirement, but
>>  thus far my arguments haven't proved compelling enough.)
>
> It does seem pretty natural... what's the counter argument?

Jordan holds the opinion that "we should continue to support the memory
vars, even if they have some obvious drawbacks":

http://mid.mail-archive.com/149065122031.28789.9113394760317457361@jljusten-skl

I'm of a different opinion: while I agree that we should not break the
existing memory emulation, I'm also convinced that we should not develop
new features for it, especially when a new feature (such as PEI-phase
R/O variable access) simply cannot be *sensibly* extended to said
emulation.

Please see the first discussion under my original patch set:

http://mid.mail-archive.com/148969026858.27582.5519307275216644796@jljusten-skl.jf.intel.com
http://mid.mail-archive.com/319ff8f1-4e99-f977-5c2e-75509a222706@redhat.com
http://mid.mail-archive.com/c20c6604-2153-aa57-cee5-24089a79b1e9@redhat.com
http://mid.mail-archive.com/149034108785.2439.14733776608486243050@jljusten-skl
http://mid.mail-archive.com/4b06b195-d8fb-a33e-3492-74fe396ebf6e@redhat.com

And the second discussion under Jordan's counter-proposal:

http://mid.mail-archive.com/a12bbf61-3f55-2fda-b855-58aeb99a4f42@redhat.com
http://mid.mail-archive.com/149065122031.28789.9113394760317457361@jljusten-skl
http://mid.mail-archive.com/121bb81b-08d1-d854-cf6f-ebc8268eb360@redhat.com
http://mid.mail-archive.com/149076505135.12962.12298731900768295093@jljusten-skl

Thanks
Laszlo

Re: [Qemu-devel] [PATCH qemu v3] vfio-pci, ppc64/spapr: Reorder group-to-container attaching

2017-07-14 Thread Alex Williamson

On Tue, 11 Jul 2017 15:32:18 +1000
Alexey Kardashevskiy  wrote:

> At the moment VFIO PCI device initialization works as follows:
> vfio_realize
>   vfio_get_group
>   vfio_connect_container
>   register memory listeners (1)
>   update QEMU groups lists
>   vfio_kvm_device_add_group
> 
> Then (example for pseries) the machine reset hook triggers region_add()
> for all regions where listeners from (1) are listening:
> 
> ppc_spapr_reset
>   spapr_phb_reset
>   spapr_tce_table_enable
>   memory_region_add_subregion
>   vfio_listener_region_add
>   vfio_spapr_create_window
> 
> This scheme works fine until we need to handle VFIO PCI device hotplug
> and we want to enable PPC64/sPAPR in-kernel TCE acceleration on,
> i.e. after PCI hotplug we need a place to call
> ioctl(vfio_kvm_device_fd, KVM_DEV_VFIO_GROUP_SET_SPAPR_TCE).
> Since the ioctl needs a LIOBN fd (from sPAPRTCETable) and a IOMMU group fd
> (from VFIOGroup), vfio_listener_region_add() seems to be the only place
> for this ioctl().
> 
> However this only works during boot time because the machine reset
> happens strictly after all devices are finalized. When hotplug happens,
> vfio_listener_region_add() is called when a memory listener is registered
> but when this happens:
> 1. new group is not added to the container->group_list yet;
> 2. VFIO KVM device is unaware of the new IOMMU group.
> 
> This moves bits around to have all necessary VFIO infrastructure
> in place for both initial startup and hotplug cases.


Wow, for a fairly straight forward patch this changelog is brutal.  Can
this be summarized as "Register vfio groups with kvm prior to memory
listener registration such that kvm-vfio pseudo device ioctls are
available during the region_add callback"?  If so I'll queue this up
for a pull request prior to soft freeze.  Thanks,

Alex

 
> Signed-off-by: Alexey Kardashevskiy 
> Reviewed-by: David Gibson 
> ---
> 
> 
> This is an independend part of a bigger patchset to enable in-kernel
> acceleration of TCE operations. While I stuck with IOMMU MR QOM-fication,
> let's try to parallel patchset pieces' review/acceptance.
> 
> ---
> Changes:
> v3:
> * rebased on current upstream
> 
> v2:
> * moved container->initialized back to its correct location
> * added missing QLIST_REMOVE()
> ---
>  hw/vfio/common.c | 19 +++
>  1 file changed, 11 insertions(+), 8 deletions(-)
> 
> diff --git a/hw/vfio/common.c b/hw/vfio/common.c
> index 2965b68e5d..4e75dc8c56 100644
> --- a/hw/vfio/common.c
> +++ b/hw/vfio/common.c
> @@ -1107,6 +1107,14 @@ static int vfio_connect_container(VFIOGroup *group, 
> AddressSpace *as,
>  goto free_container_exit;
>  }
>  
> +vfio_kvm_device_add_group(group);
> +
> +QLIST_INIT(&container->group_list);
> +QLIST_INSERT_HEAD(&space->containers, container, next);
> +
> +group->container = container;
> +QLIST_INSERT_HEAD(&container->group_list, group, container_next);
> +
>  container->listener = vfio_memory_listener;
>  
>  memory_listener_register(&container->listener, container->space->as);
> @@ -1120,14 +1128,11 @@ static int vfio_connect_container(VFIOGroup *group, 
> AddressSpace *as,
>  
>  container->initialized = true;
>  
> -QLIST_INIT(&container->group_list);
> -QLIST_INSERT_HEAD(&space->containers, container, next);
> -
> -group->container = container;
> -QLIST_INSERT_HEAD(&container->group_list, group, container_next);
> -
>  return 0;
>  listener_release_exit:
> +QLIST_REMOVE(group, container_next);
> +QLIST_REMOVE(container, next);
> +vfio_kvm_device_del_group(group);
>  vfio_listener_release(container);
>  
>  free_container_exit:
> @@ -1232,8 +1237,6 @@ VFIOGroup *vfio_get_group(int groupid, AddressSpace 
> *as, Error **errp)
>  
>  QLIST_INSERT_HEAD(&vfio_group_list, group, next);
>  
> -vfio_kvm_device_add_group(group);
> -
>  return group;
>  
>  close_fd_exit:

Re: [Qemu-devel] [PATCH v3 2/8] target/s390x: Implement CONVERT UNICODE insns

2017-07-14 Thread Aurelien Jarno

On 2017-07-11 17:18, Thomas Huth wrote:
> On 10.07.2017 22:45, Richard Henderson wrote:
> > Signed-off-by: Richard Henderson 
> > ---
> >  target/s390x/helper.h  |   6 +
> >  target/s390x/mem_helper.c  | 310 
> > +
> >  target/s390x/translate.c   |  43 +++
> >  target/s390x/insn-data.def |  13 ++
> >  4 files changed, 372 insertions(+)
> > 
> > diff --git a/target/s390x/helper.h b/target/s390x/helper.h
> > index 23e8d1d..2793cf3 100644
> > --- a/target/s390x/helper.h
> > +++ b/target/s390x/helper.h
> > @@ -107,6 +107,12 @@ DEF_HELPER_2(stfle, i32, env, i64)
> >  DEF_HELPER_FLAGS_2(lpq, TCG_CALL_NO_WG, i64, env, i64)
> >  DEF_HELPER_FLAGS_4(stpq, TCG_CALL_NO_WG, void, env, i64, i64, i64)
> >  DEF_HELPER_4(mvcos, i32, env, i64, i64, i64)
> > +DEF_HELPER_4(cu12, i32, env, i32, i32, i32)
> > +DEF_HELPER_4(cu14, i32, env, i32, i32, i32)
> > +DEF_HELPER_4(cu21, i32, env, i32, i32, i32)
> > +DEF_HELPER_4(cu24, i32, env, i32, i32, i32)
> > +DEF_HELPER_4(cu41, i32, env, i32, i32, i32)
> > +DEF_HELPER_4(cu42, i32, env, i32, i32, i32)
> >  
> >  #ifndef CONFIG_USER_ONLY
> >  DEF_HELPER_3(servc, i32, env, i64, i64)
> > diff --git a/target/s390x/mem_helper.c b/target/s390x/mem_helper.c
> > index 513b402..0b18560 100644
> > --- a/target/s390x/mem_helper.c
> > +++ b/target/s390x/mem_helper.c
> [...]
> > +static inline uint32_t convert_unicode(CPUS390XState *env, uint32_t r1,
> > +   uint32_t r2, uint32_t m3, uintptr_t 
> > ra,
> > +   decode_unicode_fn decode,
> > +   encode_unicode_fn encode)
> > +{
> > +uint64_t dst = get_address(env, r1);
> > +uint64_t dlen = get_length(env, r1 + 1);
> > +uint64_t src = get_address(env, r2);
> > +uint64_t slen = get_length(env, r2 + 1);
> > +bool enh_check = m3 & 1;
> > +int cc, i;
> 
> According to the PoP:
> 
> "The R1 and R2 fields [...] must designate an even-
>  numbered register; otherwise, a specification excep-
>  tion is recognized."
> 
> I think you should add such a check for even-numbered registers here.

Actually it should not be done here, but at translation time in
translate.c.

There are a few places where the register number is checked to be even
and later loaded into a temp. I guess that can be replaced by generators
instead?

-- 
Aurelien Jarno  GPG: 4096R/1DDD8C9B
aurel...@aurel32.net http://www.aurel32.net

Re: [Qemu-devel] [PATCH v3 2/8] target/s390x: Implement CONVERT UNICODE insns

2017-07-14 Thread Aurelien Jarno

On 2017-07-10 10:45, Richard Henderson wrote:
> Signed-off-by: Richard Henderson 
> ---
>  target/s390x/helper.h  |   6 +
>  target/s390x/mem_helper.c  | 310 
> +
>  target/s390x/translate.c   |  43 +++
>  target/s390x/insn-data.def |  13 ++
>  4 files changed, 372 insertions(+)
> 

Besides the check for even r1 and r3, this now looks good.

Reviewed-by: Aurelien Jarno 

-- 
Aurelien Jarno  GPG: 4096R/1DDD8C9B
aurel...@aurel32.net http://www.aurel32.net

Re: [Qemu-devel] [PATCH v3 1/8] target/s390x: Implement CSST

2017-07-14 Thread Aurelien Jarno

On 2017-07-10 10:45, Richard Henderson wrote:
> Signed-off-by: Richard Henderson 
> ---
>  target/s390x/helper.h  |   1 +
>  target/s390x/cpu_models.c  |   2 +
>  target/s390x/mem_helper.c  | 189 
> +
>  target/s390x/translate.c   |  13 +++-
>  target/s390x/insn-data.def |   2 +
>  5 files changed, 206 insertions(+), 1 deletion(-)
> 
> diff --git a/target/s390x/mem_helper.c b/target/s390x/mem_helper.c
> index ede8471..513b402 100644
> --- a/target/s390x/mem_helper.c
> +++ b/target/s390x/mem_helper.c
> @@ -1353,6 +1353,195 @@ void HELPER(cdsg)(CPUS390XState *env, uint64_t addr,
>  env->regs[r1 + 1] = int128_getlo(oldv);
>  }
>  
> +uint32_t HELPER(csst)(CPUS390XState *env, uint32_t r3, uint64_t a1, uint64_t 
> a2)
> +{
> +#if !defined(CONFIG_USER_ONLY) || defined(CONFIG_ATOMIC128)
> +uint32_t mem_idx = cpu_mmu_index(env, false);
> +#endif
> +uintptr_t ra = GETPC();
> +uint32_t fc = extract32(env->regs[0], 0, 8);
> +uint32_t sc = extract32(env->regs[0], 8, 8);
> +uint64_t pl = get_address(env, 1) & -16;
> +uint64_t svh, svl;
> +uint32_t cc;
> +
> +/* Sanity check the function code and storage characteristic.  */
> +if (fc > 1 || sc > 3) {
> +if (!s390_has_feat(S390_FEAT_COMPARE_AND_SWAP_AND_STORE_2)) {
> +goto spec_exception;
> +}
> +if (fc > 2 || sc > 4 || (fc == 2 && (r3 & 1))) {
> +goto spec_exception;
> +}
> +}
> +
> +/* Sanity check the alignments.  */
> +if (extract32(a1, 0, 4 << fc) || extract32(a2, 0, 1 << sc)) {
> +goto spec_exception;
> +}
> +
> +/* Sanity check writability of the store address.  */
> +#ifndef CONFIG_USER_ONLY
> +probe_write(env, a2, mem_idx, ra);
> +#endif
> +
> +/* Note that the compare-and-swap is atomic, and the store is atomic, but
> +   the complete operation is not.  Therefore we do not need to assert 
> serial
> +   context in order to implement this.  That said, restart early if we 
> can't
> +   support either operation that is supposed to be atomic.  */
> +if (parallel_cpus) {
> +int mask = 0;
> +#if !defined(CONFIG_ATOMIC64)
> +mask = -8;
> +#elif !defined(CONFIG_ATOMIC128)
> +mask = -16;
> +#endif
> +if (((4 << fc) | (1 << sc)) & mask) {
> +cpu_loop_exit_atomic(ENV_GET_CPU(env), ra);
> +}
> +}

This doesn't look correct. For a 16-byte store, ie sc = 4, and with
ATOMIC128 support, ie mask = -16, the condition is true.

> +/* All loads happen before all stores.  For simplicity, load the entire
> +   store value area from the parameter list.  */
> +svh = cpu_ldq_data_ra(env, pl + 16, ra);
> +svl = cpu_ldq_data_ra(env, pl + 24, ra);
> +
> +switch (fc) {
> +case 0:
> +{
> +uint32_t nv = cpu_ldl_data_ra(env, pl, ra);
> +uint32_t cv = env->regs[r3];
> +uint32_t ov;
> +
> +if (parallel_cpus) {
> +#ifdef CONFIG_USER_ONLY
> +uint32_t *haddr = g2h(a1);
> +ov = atomic_cmpxchg__nocheck(haddr, cv, nv);
> +#else
> +TCGMemOpIdx oi = make_memop_idx(MO_TEUL | MO_ALIGN, mem_idx);
> +ov = helper_atomic_cmpxchgl_be_mmu(env, a1, cv, nv, oi, ra);
> +#endif

Not a problem with the patch itself, but how complicated would it be to
make helper_atomic_cmpxchgl_be_mmu (just like the o version) available
also in user mode? That would make less #ifdef in the backends.


The remaining of the patch looks all good to me.

-- 
Aurelien Jarno  GPG: 4096R/1DDD8C9B
aurel...@aurel32.net http://www.aurel32.net

[Qemu-devel] [PATCH] qapi: Fix error handling code on alternate conflict

The conflict check added by commit c0644771 ("qapi: Reject
alternates that can't work with keyval_parse()") doesn't work
with the following declaration:

  { 'alternate': 'Alt',
'data': { 'one': 'bool',
  'two': 'str' } }

It crashes with:

  Traceback (most recent call last):
File "./scripts/qapi-types.py", line 295, in 
  schema = QAPISchema(input_file)
File "/home/ehabkost/rh/proj/virt/qemu/scripts/qapi.py", line 1468, in 
__init__
  self.exprs = check_exprs(parser.exprs)
File "/home/ehabkost/rh/proj/virt/qemu/scripts/qapi.py", line 958, in 
check_exprs
  check_alternate(expr, info)
File "/home/ehabkost/rh/proj/virt/qemu/scripts/qapi.py", line 830, in 
check_alternate
  % (name, key, types_seen[qtype]))
  KeyError: 'QTYPE_QSTRING'

This happens because the previously-seen conflicting member
('one') can't be found at types_seen[qtype], but at
types_seen['QTYPE_BOOL'].

Fix the bug by moving the error check to the same loop that adds
new items to types_seen, raising an exception if types_seen[qt]
is already set.

Add two additional test cases that can detect the bug.

Signed-off-by: Eduardo Habkost 
---
 scripts/qapi.py   | 8 
 tests/Makefile.include| 2 ++
 tests/qapi-schema/alternate-conflict-bool-string.err  | 1 +
 tests/qapi-schema/alternate-conflict-bool-string.exit | 1 +
 tests/qapi-schema/alternate-conflict-bool-string.json | 4 
 tests/qapi-schema/alternate-conflict-bool-string.out  | 0
 tests/qapi-schema/alternate-conflict-num-string.err   | 1 +
 tests/qapi-schema/alternate-conflict-num-string.exit  | 1 +
 tests/qapi-schema/alternate-conflict-num-string.json  | 4 
 tests/qapi-schema/alternate-conflict-num-string.out   | 0
 tests/qapi-schema/alternate-multi-conflict.json   | 0
 11 files changed, 18 insertions(+), 4 deletions(-)
 create mode 100644 tests/qapi-schema/alternate-conflict-bool-string.err
 create mode 100644 tests/qapi-schema/alternate-conflict-bool-string.exit
 create mode 100644 tests/qapi-schema/alternate-conflict-bool-string.json
 create mode 100644 tests/qapi-schema/alternate-conflict-bool-string.out
 create mode 100644 tests/qapi-schema/alternate-conflict-num-string.err
 create mode 100644 tests/qapi-schema/alternate-conflict-num-string.exit
 create mode 100644 tests/qapi-schema/alternate-conflict-num-string.json
 create mode 100644 tests/qapi-schema/alternate-conflict-num-string.out
 create mode 100644 tests/qapi-schema/alternate-multi-conflict.json

diff --git a/scripts/qapi.py b/scripts/qapi.py
index 84e2eb4..a9c12e5 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -824,11 +824,11 @@ def check_alternate(expr, info):
 else:
 conflicting.add('QTYPE_QNUM')
 conflicting.add('QTYPE_QBOOL')
-if conflicting & set(types_seen):
-raise QAPISemError(info, "Alternate '%s' member '%s' can't "
-   "be distinguished from member '%s'"
-   % (name, key, types_seen[qtype]))
 for qt in conflicting:
+if qt in types_seen:
+raise QAPISemError(info, "Alternate '%s' member '%s' can't "
+   "be distinguished from member '%s'"
+   % (name, key, types_seen[qt]))
 types_seen[qt] = key
 
 
diff --git a/tests/Makefile.include b/tests/Makefile.include
index 42e17e2..40cf1dd 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -346,6 +346,8 @@ qapi-schema += alternate-conflict-dict.json
 qapi-schema += alternate-conflict-enum-bool.json
 qapi-schema += alternate-conflict-enum-int.json
 qapi-schema += alternate-conflict-string.json
+qapi-schema += alternate-conflict-bool-string.json
+qapi-schema += alternate-conflict-num-string.json
 qapi-schema += alternate-empty.json
 qapi-schema += alternate-nested.json
 qapi-schema += alternate-unknown.json
diff --git a/tests/qapi-schema/alternate-conflict-bool-string.err 
b/tests/qapi-schema/alternate-conflict-bool-string.err
new file mode 100644
index 000..5c9af91
--- /dev/null
+++ b/tests/qapi-schema/alternate-conflict-bool-string.err
@@ -0,0 +1 @@
+tests/qapi-schema/alternate-conflict-bool-string.json:3: Alternate 'Alt' 
member 'two' can't be distinguished from member 'one'
diff --git a/tests/qapi-schema/alternate-conflict-bool-string.exit 
b/tests/qapi-schema/alternate-conflict-bool-string.exit
new file mode 100644
index 000..d00491f
--- /dev/null
+++ b/tests/qapi-schema/alternate-conflict-bool-string.exit
@@ -0,0 +1 @@
+1
diff --git a/tests/qapi-schema/alternate-conflict-bool-string.json 
b/tests/qapi-schema/alternate-conflict-bool-string.json
new file mode 100644
index 000..0544de1
--- /dev/null
+++ b/tests/qapi-schema/alternate-conflict-bool-string.json
@@ -0,0 +1,4 @@
+# alternate branches of 'str' type conflict with all scalar types
+{ 'alternate': 'Alt',
+  'data': { 'one'

Re: [Qemu-devel] investigating TPM for OVMF-on-QEMU

2017-07-14 Thread Peter Jones

On Fri, Jul 14, 2017 at 08:04:14PM +0200, Laszlo Ersek wrote:

> - TPM2 is basically the standardized version of TrEE, the most
>   recent set of specs, and what we should focus on.

100% agreed.

> (2) Drivers (and features) in edk2/SecurityPkg/Tcg.
> 
> There are 19 modules under SecurityPkg/Tcg/. Let me categorize them.
> In each category, I'll try to list modules in loosely increasing
> dependency (or PI/UEFI phase) order.
> 
> (2a) Modules that are obsolete because they are tied to TPM1:
> 
>TcgPei/TcgPei.inf
>PhysicalPresencePei/PhysicalPresencePei.inf
>TcgDxe/TcgDxe.inf
>TcgConfigDxe/TcgConfigDxe.inf
>TcgSmm/TcgSmm.inf
> 
>  The TPM enablement instructions at
> 
>
> https://github.com/tianocore/tianocore.github.io/wiki/How-to-Enable-Security#Enabling_Trusted_Compute_Module_TPM
> 
>  are written up in terms of these modules, but that doesn't obsolete
>  the article too much.
> 
> (2b) Modules that are obsolete due to being tied to TrEE:
> 
>TrEEConfig/TrEEConfigPei.inf
>TrEEPei/TrEEPei.inf
>TrEEDxe/TrEEDxe.inf
>TrEEConfig/TrEEConfigDxe.inf
>TrEESmm/TrEESmm.inf
> 
>  There is a great Intel whitepaper called
> 
>A Tour Beyond BIOS with the UEFI TPM2 Support in EDKII
> 
>  which is written in terms of TrEE and these modules, explaining how
>  they supersede TPM1 and the modules under (2a).
> 
>  This whitepaper seems to apply to the most recent TPM2 stack as
>  well (mostly just replace TrEE references with TPM2 references), so
>  it is very useful.
> 
> (2c) Modules that call themselves obsolete or deprecated (without being
>  obviously tied to TPM1 or TrEE):
> 
>MemoryOverwriteRequestControlLock/TcgMorLockSmm.inf

Definitely do not care.

> (2d) Modules that we won't need due to no hardware support in QEMU:
> 
>Opal/OpalPasswordDxe/OpalPasswordDxe.inf
>Opal/OpalPasswordSmm/OpalPasswordSmm.inf
> 
>  TCG's "Opal" seems to be about self-encrypting drives, see
> 
>https://en.wikipedia.org/wiki/Opal_Storage_Specification
> 
>  so these modules are not relevant for us.

Well, at some point somebody's going to want us to implement this,
but... they can do that when they do that.  Personally I'm never going
to trust on-drive encryption not to have an escrow key I don't know
about and can't remove, so it's a bit of a non-starter, but that's
neither here nor there.

> (2e) Modules that we should use. Again, in increasing order of
>  dependence. And here I'll comment as well on what these do:
> 
>Tcg2Config/Tcg2ConfigPei.inf -- Informs the firmware globally
>about the TPM device type. This
>module can perform device
>detection or read a cached value
>from a non-volatile UEFI
>variable.
> 
>Tcg2Pei/Tcg2Pei.inf  -- Initializes the TPM device and
>measures the firmware volumes in
>the PEI phase into the TPM's
>platform config registers.
> 
>Tcg2Dxe/Tcg2Dxe.inf  -- Measures DXE phase (and later)
>modules into the TPM's PCRs, and
>also lets the OS boot loader
>measure things, by exposing the
>EFI_TCG2_PROTOCOL.
> 
>Tcg2Config/Tcg2ConfigDxe.inf -- Provides a Setup TUI interface to
>configure the TPM. IIUC, it can
>also save the configured TPM type
>for subsequent boots (see
>Tcg2ConfigPei.inf above).
> 
>  This driver stack supports the TIS (MMIO) hardware interface, which
>  is advertized to the OS in the TPM2 ACPI Table's "start method"
>  field with value 6. (The according macro is TPM2_START_METHOD_MMIO
>  in the QEMU source code, and EFI_TPM2_ACPI_TABLE_START_METHOD_TIS
>  in the edk2 source code.)
> 
>  Including these drivers should result in a functional
>  EFI_TCG2_PROTOCOL, which is what OS boot loaders primarily care
>  about, as I understand.
> 
>  Importantly, the driver stack above requires PEI-phase variable
>  access, therefore
>   must be solved
>  first.
> 
>  (I have had patches for said BZ ready for a while. I've failed to
>  upstream them thus far because a pflash-based varstore is a hard
>  requirement for them. I think that's a natural requirement, but
>  thus far my arguments haven't proved compelling enough.)

Re: [Qemu-devel] [PATCH v4 0/8] KASLR kernel dump support

On 07/14/17 21:59, Michael S. Tsirkin wrote:
> On Fri, Jul 14, 2017 at 08:20:03PM +0200, Marc-André Lureau wrote:
>> Recent linux kernels enable KASLR to randomize phys/virt memory
>> addresses. This series aims to provide enough information in qemu
>> dumps so that crash utility can work with randomized kernel too (it
>> hasn't been tested on other archs than x86 though, help welcome).
>>
>> The vmcoreinfo device is an emulated ACPI device that exposes a 4k
>> memory range to the guest to store various informations useful to
>> debug the guest OS. (it is greatly inspired by the VMGENID device
>> implementation). The version field with value 0 is meant to give
>> paddr/size of the VMCOREINFO ELF PT_NOTE, other values can be used for
>> different purposes or OSes. (note: some wanted to see pvpanic somehow
>> merged with this device, I have no clear idea how to do that, nor do I
>> think this is a good idea since the devices are quite different, used
>> at different time for different purposes. And it can be done as a
>> future iteration if it is appropriate, feel free to send patches)
> 
> First, I think you underestimate the difficulty of maintaining
> compatibility.
> 
> Second, this seems racy - how do you know when is guest done writing out
> the data?

What data exactly?

The guest kernel module points the fields in the "vmcoreinfo page" to
the then-existent vmcoreinfo ELF note. At that point, the ELF note is
complete.

If we catch the guest with a dump request while the kernel module is
setting up the fields (i.e., the fields are not consistent), then we'll
catch that in our sanity checks, and the note won't be extracted. This
is no different from the case when you simply dump the guest RAM before
the module got invoked.

> Given you have very little data to export (PA, size - do
> you even need size?)

Yes, it tells us in advance how much memory to allocate before we copy
out the vmcoreinfo ELF note (and we enforce a sanity limit on the size).

> - how about just using an ACPI method do it,

Do what exactly?

> instead of exporting a physical addess and storing address there.  This
> way you can add more methods as you add functionality.

I'm not saying this is a bad idea (especially because I don't fully
understand your point), but I will say that I'm quite sad that you are
sending Marc-André back to the drawing board after he posted v4 -- also
invalidating my review efforts. :/

Laszlo

Re: [Qemu-devel] [PATCH v4 2/8] acpi: add vmcoreinfo device

On Fri, Jul 14, 2017 at 10:04:31PM +0200, Laszlo Ersek wrote:
> > It worries me that the format of this seems completely undefined
> > except in the patchset cover letter.
> > I don't think we should merge this before it is.
> 
> I'm not sure what you mean, this patch adds "docs/specs/vmcoreinfo.txt".
> That file is the first level contract between the guest firmware
> (generally, via the linker/loader), the guest kernel driver
> (specifically), and QEMU (also specifically).
> 
> The second level contract is the guest kernel's vmcoreinfo ELF note
> (which is pointed-to by the first level contract). The layout of that is
> specified elsewhere indeed (I don't think it belongs here).
> 
> We've taken care not to trust anything coming from the guest kernel.
> 
> Can you clarify please?
> 
> Thanks
> Laszlo

All there is is this:

+Version 0 content:
+
+ uint64 paddr:
+  Physical address of the Linux vmcoreinfo ELF note.
+ uint32 size:
+  Size of the vmcoreinfo ELF note.

It isn't defined here what is the Linux vmcoreinfo ELF note.
You want a bit more info so people trying to use it know where to start
and what they can get out of it.

-- 
MST

Re: [Qemu-devel] [PATCH v4 4/8] tests: add simple vmcoreinfo test

On 07/14/17 20:20, Marc-André Lureau wrote:
> This test is based off vmgenid test from Ben Warren
> . It simply checks the vmcoreinfo ACPI device
> is present and that the memory region associated can be read.
> 
> Signed-off-by: Marc-André Lureau 
> ---
>  tests/vmcoreinfo-test.c | 141 
> 
>  tests/Makefile.include  |   2 +
>  2 files changed, 143 insertions(+)
>  create mode 100644 tests/vmcoreinfo-test.c
> 
> diff --git a/tests/vmcoreinfo-test.c b/tests/vmcoreinfo-test.c
> new file mode 100644
> index 00..a54f3b5c6a
> --- /dev/null
> +++ b/tests/vmcoreinfo-test.c
> @@ -0,0 +1,141 @@
> +/*
> + * QTest testcase for VM coreinfo device
> + *
> + * Copyright (c) 2017 Red Hat, Inc.
> + * Copyright (c) 2017 Skyport Systems
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> + * See the COPYING file in the top-level directory.
> + */
> +
> +#include 
> +#include 
> +#include 
> +#include "qemu/osdep.h"
> +#include "qemu/bitmap.h"
> +#include "qemu/uuid.h"
> +#include "hw/acpi/acpi-defs.h"
> +#include "boot-sector.h"
> +#include "acpi-utils.h"
> +#include "libqtest.h"
> +
> +#define RSDP_ADDR_INVALID 0x10 /* RSDP must be below this address */
> +
> +typedef struct {
> +AcpiTableHeader header;
> +gchar name_op;
> +gchar vcia[4];
> +gchar val_op;
> +uint32_t vcia_val;
> +} QEMU_PACKED VmciTable;
> +
> +static uint32_t acpi_find_vcia(void)
> +{
> +uint32_t rsdp_offset;
> +AcpiRsdpDescriptor rsdp_table;
> +uint32_t rsdt;
> +AcpiRsdtDescriptorRev1 rsdt_table;
> +int tables_nr;
> +uint32_t *tables;
> +AcpiTableHeader ssdt_table;
> +VmciTable vmci_table;
> +int i;
> +
> +/* Wait for guest firmware to finish and start the payload. */
> +boot_sector_test();
> +
> +/* Tables should be initialized now. */
> +rsdp_offset = acpi_find_rsdp_address();
> +
> +g_assert_cmphex(rsdp_offset, <, RSDP_ADDR_INVALID);
> +
> +acpi_parse_rsdp_table(rsdp_offset, &rsdp_table);
> +
> +rsdt = rsdp_table.rsdt_physical_address;
> +/* read the header */
> +ACPI_READ_TABLE_HEADER(&rsdt_table, rsdt);
> +ACPI_ASSERT_CMP(rsdt_table.signature, "RSDT");
> +
> +/* compute the table entries in rsdt */
> +tables_nr = (rsdt_table.length - sizeof(AcpiRsdtDescriptorRev1)) /
> +sizeof(uint32_t);
> +g_assert_cmpint(tables_nr, >, 0);
> +
> +/* get the addresses of the tables pointed by rsdt */
> +tables = g_new0(uint32_t, tables_nr);
> +ACPI_READ_ARRAY_PTR(tables, tables_nr, rsdt);
> +
> +for (i = 0; i < tables_nr; i++) {
> +ACPI_READ_TABLE_HEADER(&ssdt_table, tables[i]);
> +if (!strncmp((char *)ssdt_table.oem_table_id, "VMCOREIN", 8)) {
> +/* the first entry in the table should be VCIA
> + * That's all we need
> + */
> +ACPI_READ_FIELD(vmci_table.name_op, tables[i]);
> +g_assert(vmci_table.name_op == 0x08);  /* name */
> +ACPI_READ_ARRAY(vmci_table.vcia, tables[i]);
> +g_assert(memcmp(vmci_table.vcia, "VCIA", 4) == 0);
> +ACPI_READ_FIELD(vmci_table.val_op, tables[i]);
> +g_assert(vmci_table.val_op == 0x0C);  /* dword */
> +ACPI_READ_FIELD(vmci_table.vcia_val, tables[i]);
> +/* The GUID is written at a fixed offset into the fw_cfg file
> + * in order to implement the "OVMF SDT Header probe suppressor"
> + * see docs/specs/vmgenid.txt for more details
> + */
> +g_free(tables);
> +return vmci_table.vcia_val;
> +}
> +}
> +g_free(tables);
> +return 0;
> +}
> +
> +static char disk[] = "tests/vmci-test-disk-XX";
> +
> +static char *cmd_strdup(void)
> +{
> +return g_strdup_printf("-machine accel=tcg "
> +   "-device vmcoreinfo,id=testvmvci "
> +   "-drive id=hd0,if=none,file=%s,format=raw "
> +   "-device ide-hd,drive=hd0 ",
> +   disk);
> +}
> +
> +static void vmcoreinfo_test(void)
> +{
> +gchar *cmd;
> +uint32_t vmci_addr;
> +int i;
> +
> +cmd = cmd_strdup();
> +qtest_start(cmd);
> +
> +vmci_addr = acpi_find_vcia();
> +g_assert(vmci_addr);
> +
> +for (i = 0; i < 4096; i++) {
> +/* check the memory region can be read */
> +readb(vmci_addr + i);
> +}
> +
> +qtest_quit(global_qtest);
> +g_free(cmd);
> +}
> +
> +int main(int argc, char **argv)
> +{
> +int ret;
> +
> +ret = boot_sector_init(disk);
> +if (ret) {
> +return ret;
> +}
> +
> +g_test_init(&argc, &argv, NULL);
> +
> +qtest_add_func("/vmcoreinfo/test", vmcoreinfo_test);
> +ret = g_test_run();
> +boot_sector_cleanup(disk);
> +
> +return ret;
> +}
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index cfbb689e0e..dd515f2159 100644
> ---

Re: [Qemu-devel] [PATCH v4 3/8] stubs: add vmcoreinfo_get() stub

On 07/14/17 20:20, Marc-André Lureau wrote:
> Common dump code may call vmcoreinfo_get() which is not available on
> all targets.
> 
> Signed-off-by: Marc-André Lureau 
> ---
>  stubs/vmcoreinfo.c  | 9 +
>  stubs/Makefile.objs | 1 +
>  2 files changed, 10 insertions(+)
>  create mode 100644 stubs/vmcoreinfo.c
> 
> diff --git a/stubs/vmcoreinfo.c b/stubs/vmcoreinfo.c
> new file mode 100644
> index 00..a994153832
> --- /dev/null
> +++ b/stubs/vmcoreinfo.c
> @@ -0,0 +1,9 @@
> +#include "qemu/osdep.h"
> +
> +#include "hw/acpi/vmcoreinfo.h"
> +
> +bool vmcoreinfo_get(VMCoreInfoState *vis, uint64_t *paddr, uint32_t *size,
> +Error **errp)
> +{
> +return false;
> +}
> diff --git a/stubs/Makefile.objs b/stubs/Makefile.objs
> index f5b47bfd74..3bedc73252 100644
> --- a/stubs/Makefile.objs
> +++ b/stubs/Makefile.objs
> @@ -36,6 +36,7 @@ stub-obj-y += qmp_pc_dimm_device_list.o
>  stub-obj-y += target-monitor-defs.o
>  stub-obj-y += target-get-monitor-def.o
>  stub-obj-y += pc_madt_cpu_entry.o
> +stub-obj-y += vmcoreinfo.o
>  stub-obj-y += vmgenid.o
>  stub-obj-y += xen-common.o
>  stub-obj-y += xen-hvm.o
> 

Looks like a good idea -- but now that you are adding this patch, I
think we should actually report an error like "unsupported" via errp:

  error_setg(errp, QERR_UNSUPPORTED);

In patch #5 you report the error (as a warning) with warn_report_err(),
and I think that's going to crash there -- err is pre-inited to NULL,
the above impl. doesn't change it, and warn_report_err() doesn't expect
NULL.

Thanks
Laszlo

Re: [Qemu-devel] [PATCH v4 2/8] acpi: add vmcoreinfo device

On 07/14/17 21:26, Michael S. Tsirkin wrote:
> On Fri, Jul 14, 2017 at 08:20:05PM +0200, Marc-André Lureau wrote:
>> The VM coreinfo (vmcoreinfo) device is an emulated device which
>> exposes a 4k memory range to the guest to store various informations
>> useful to debug the guest OS. (it is greatly inspired by the VMGENID
>> device implementation)
>>
>> This is an early-boot alternative to the qemu-ga VMDUMP_INFO event
>> proposed in "[PATCH 00/21] WIP: dump: add kaslr support".
>>
>> A proof-of-concept kernel module:
>> https://github.com/elmarco/vmgenid-test/blob/master/qemuvmci-test.c
>>
>> Signed-off-by: Marc-André Lureau 
>> Reviewed-by: Laszlo Ersek 
> 
> It worries me that the format of this seems completely undefined
> except in the patchset cover letter.
> I don't think we should merge this before it is.

I'm not sure what you mean, this patch adds "docs/specs/vmcoreinfo.txt".
That file is the first level contract between the guest firmware
(generally, via the linker/loader), the guest kernel driver
(specifically), and QEMU (also specifically).

The second level contract is the guest kernel's vmcoreinfo ELF note
(which is pointed-to by the first level contract). The layout of that is
specified elsewhere indeed (I don't think it belongs here).

We've taken care not to trust anything coming from the guest kernel.

Can you clarify please?

Thanks
Laszlo

> 
>> ---
>>  include/hw/acpi/aml-build.h|   1 +
>>  include/hw/acpi/vmcoreinfo.h   |  37 +++
>>  hw/acpi/aml-build.c|   2 +
>>  hw/acpi/vmcoreinfo.c   | 208 
>> +
>>  hw/i386/acpi-build.c   |  14 +++
>>  default-configs/arm-softmmu.mak|   1 +
>>  default-configs/i386-softmmu.mak   |   1 +
>>  default-configs/x86_64-softmmu.mak |   1 +
>>  docs/specs/vmcoreinfo.txt  | 138 
>>  hw/acpi/Makefile.objs  |   1 +
>>  10 files changed, 404 insertions(+)
>>  create mode 100644 include/hw/acpi/vmcoreinfo.h
>>  create mode 100644 hw/acpi/vmcoreinfo.c
>>  create mode 100644 docs/specs/vmcoreinfo.txt
>>
>> diff --git a/include/hw/acpi/aml-build.h b/include/hw/acpi/aml-build.h
>> index 88d0738d76..cf781bcd34 100644
>> --- a/include/hw/acpi/aml-build.h
>> +++ b/include/hw/acpi/aml-build.h
>> @@ -211,6 +211,7 @@ struct AcpiBuildTables {
>>  GArray *rsdp;
>>  GArray *tcpalog;
>>  GArray *vmgenid;
>> +GArray *vmcoreinfo;
>>  BIOSLinker *linker;
>>  } AcpiBuildTables;
>>  
>> diff --git a/include/hw/acpi/vmcoreinfo.h b/include/hw/acpi/vmcoreinfo.h
>> new file mode 100644
>> index 00..6a73bcd1b2
>> --- /dev/null
>> +++ b/include/hw/acpi/vmcoreinfo.h
>> @@ -0,0 +1,37 @@
>> +#ifndef ACPI_VMCOREINFO_H
>> +#define ACPI_VMCOREINFO_H
>> +
>> +#include "hw/acpi/bios-linker-loader.h"
>> +#include "hw/qdev.h"
>> +
>> +#define VMCOREINFO_DEVICE   "vmcoreinfo"
>> +#define VMCOREINFO_FW_CFG_FILE  "etc/vmcoreinfo"
>> +#define VMCOREINFO_ADDR_FW_CFG_FILE "etc/vmcoreinfo-addr"
>> +
>> +/* Occupy a page of memory */
>> +#define VMCOREINFO_FW_CFG_SIZE  4096
>> +
>> +/* allow space for OVMF SDT Header Probe Supressor */
>> +#define VMCOREINFO_OFFSET   sizeof(AcpiTableHeader)
>> +
>> +#define VMCOREINFO(obj) OBJECT_CHECK(VMCoreInfoState, (obj), 
>> VMCOREINFO_DEVICE)
>> +
>> +typedef struct VMCoreInfoState {
>> +DeviceClass parent_obj;
>> +uint8_t vmcoreinfo_addr_le[8];   /* Address of memory region */
>> +bool write_pointer_available;
>> +} VMCoreInfoState;
>> +
>> +/* returns NULL unless there is exactly one device */
>> +static inline Object *find_vmcoreinfo_dev(void)
>> +{
>> +return object_resolve_path_type("", VMCOREINFO_DEVICE, NULL);
>> +}
>> +
>> +void vmcoreinfo_build_acpi(VMCoreInfoState *vis, GArray *table_data,
>> +   GArray *vmci, BIOSLinker *linker);
>> +void vmcoreinfo_add_fw_cfg(VMCoreInfoState *vis, FWCfgState *s, GArray 
>> *vmci);
>> +bool vmcoreinfo_get(VMCoreInfoState *vis, uint64_t *paddr, uint32_t *size,
>> +Error **errp);
>> +
>> +#endif
>> diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c
>> index 36a6cc450e..47043ade4a 100644
>> --- a/hw/acpi/aml-build.c
>> +++ b/hw/acpi/aml-build.c
>> @@ -1561,6 +1561,7 @@ void acpi_build_tables_init(AcpiBuildTables *tables)
>>  tables->table_data = g_array_new(false, true /* clear */, 1);
>>  tables->tcpalog = g_array_new(false, true /* clear */, 1);
>>  tables->vmgenid = g_array_new(false, true /* clear */, 1);
>> +tables->vmcoreinfo = g_array_new(false, true /* clear */, 1);
>>  tables->linker = bios_linker_loader_init();
>>  }
>>  
>> @@ -1571,6 +1572,7 @@ void acpi_build_tables_cleanup(AcpiBuildTables 
>> *tables, bool mfre)
>>  g_array_free(tables->table_data, true);
>>  g_array_free(tables->tcpalog, mfre);
>>  g_array_free(tables->vmgenid, mfre);
>> +g_array_free(tables->vmcoreinfo, mfre);
>>  }
>>  
>>  /* Build rsdt table */
>> d

Re: [Qemu-devel] [PATCH v4 0/8] KASLR kernel dump support

On Fri, Jul 14, 2017 at 08:20:03PM +0200, Marc-André Lureau wrote:
> Recent linux kernels enable KASLR to randomize phys/virt memory
> addresses. This series aims to provide enough information in qemu
> dumps so that crash utility can work with randomized kernel too (it
> hasn't been tested on other archs than x86 though, help welcome).
> 
> The vmcoreinfo device is an emulated ACPI device that exposes a 4k
> memory range to the guest to store various informations useful to
> debug the guest OS. (it is greatly inspired by the VMGENID device
> implementation). The version field with value 0 is meant to give
> paddr/size of the VMCOREINFO ELF PT_NOTE, other values can be used for
> different purposes or OSes. (note: some wanted to see pvpanic somehow
> merged with this device, I have no clear idea how to do that, nor do I
> think this is a good idea since the devices are quite different, used
> at different time for different purposes. And it can be done as a
> future iteration if it is appropriate, feel free to send patches)

First, I think you underestimate the difficulty of maintaining
compatibility.

Second, this seems racy - how do you know when is guest done writing out
the data?

Given you have very little data to export (PA, size - do
you even need size?) - how about just using an ACPI method do it,
instead of exporting a physical addess and storing address there.  This
way you can add more methods as you add functionality.

VMGENID has very specific requirements around performance,
and does not care about consistency at all.
This does not apply here.


> Crash 7.1.9 will parse the "phys_base" value from the VMCOREINFO note,
> and thus will work with KASLR-dump produced by this series.
> 
> By priority, VMCOREINFO "phys_base" value is the most accurate. If not
> available, qemu will keep the current guessed value.
> 
> The series implements the VMCOREINFO note addition in qemu ELF/kdump,
> as well as the python scripts/dump-guest-memory.py.
> 
> To test:
> 
> Compile and run a guest kernel with CONFIG_RANDOMIZE_BASE=y.
> 
> Run qemu with -device vmcoreinfo.
> 
> Load the experimental vmcoreinfo module in guest
> https://github.com/elmarco/vmgenid-test/blob/master/qemuvmci-test.c.
> 
> Produce an ELF dump:
> { "execute": "dump-guest-memory", "arguments": { "protocol": "file:dump", 
> "paging": false } }
> 
> Produce a kdump:
> { "execute": "dump-guest-memory", "arguments": { "protocol": "file:dump", 
> "paging": false, "format": "kdump-zlib" } }
> 
> Or with (gdb) dump-guest-memory, with scripts/dump-guest-memory.py script.
> 
> Analyze with crash >= 7.1.9
> $ crash vmlinux dump
> 
> v4: from Laszlo review
> - switch to warn_report*()
> - update test to follow vmgenid and use boot-sector infrastructure
> - fix range checks in the python script
> - add vmcoreinfo_get() stub
> 
> v3: from Laszlo review
> - change vmcoreinfo offset to 36
> - reset err to null after report
> - use PRIu32
> - change name_size and desc_size against MAX_VMCOREINFO_SIZE
> - python code simplification
> - check boundaries of blocks in phys_memory_read()
> - fix some vmgi vs vmci names
> - add more comments in code
> - fix comment indentation
> - add r-b tags
> 
> v2: from Laszlo review
> - vmci: fix guest endianess handling
> - vmci: fix wrong sizeof()
> - vmci: add back reset logic from vmgenid
> - dump: have 1MB size limit for vmcoreinfo
> - dump: fix potential off-by-1 buffer manipulation
> - dump: use temporary variable for qemu_strtou64
> - dump: fixed VMCOREINFO duplication in kdump
> - update gdb script to not call into qemu process
> - update MAINTAINERS with some new files
> 
> Marc-André Lureau (8):
>   vmgenid: replace x-write-pointer-available hack
>   acpi: add vmcoreinfo device
>   stubs: add vmcoreinfo_get() stub
>   tests: add simple vmcoreinfo test
>   dump: add vmcoreinfo ELF note
>   kdump: add vmcoreinfo ELF note
>   scripts/dump-guest-memory.py: add vmcoreinfo
>   MAINTAINERS: add Dump maintainers
> 
>  scripts/dump-guest-memory.py |  47 
>  include/hw/acpi/aml-build.h  |   1 +
>  include/hw/acpi/bios-linker-loader.h |   2 +
>  include/hw/acpi/vmcoreinfo.h |  37 ++
>  include/hw/compat.h  |   4 -
>  include/sysemu/dump.h|   2 +
>  dump.c   | 154 +
>  hw/acpi/aml-build.c  |   2 +
>  hw/acpi/bios-linker-loader.c |  10 ++
>  hw/acpi/vmcoreinfo.c | 211 
> +++
>  hw/acpi/vmgenid.c|   9 +-
>  hw/i386/acpi-build.c |  14 +++
>  stubs/vmcoreinfo.c   |   9 ++
>  tests/vmcoreinfo-test.c  | 141 +++
>  MAINTAINERS  |   9 ++
>  default-configs/arm-softmmu.mak  |   1 +
>  default-configs/i386-softmmu.mak |   1 +
>  default-configs/x86_64-softmmu.mak   |   1 +
>  docs/specs/vmcoreinfo.txt| 138 ++

Re: [Qemu-devel] [PATCH v4 2/8] acpi: add vmcoreinfo device

On Fri, Jul 14, 2017 at 08:20:05PM +0200, Marc-André Lureau wrote:
> The VM coreinfo (vmcoreinfo) device is an emulated device which
> exposes a 4k memory range to the guest to store various informations
> useful to debug the guest OS. (it is greatly inspired by the VMGENID
> device implementation)
> 
> This is an early-boot alternative to the qemu-ga VMDUMP_INFO event
> proposed in "[PATCH 00/21] WIP: dump: add kaslr support".
> 
> A proof-of-concept kernel module:
> https://github.com/elmarco/vmgenid-test/blob/master/qemuvmci-test.c
> 
> Signed-off-by: Marc-André Lureau 
> Reviewed-by: Laszlo Ersek 

It worries me that the format of this seems completely undefined
except in the patchset cover letter.
I don't think we should merge this before it is.

> ---
>  include/hw/acpi/aml-build.h|   1 +
>  include/hw/acpi/vmcoreinfo.h   |  37 +++
>  hw/acpi/aml-build.c|   2 +
>  hw/acpi/vmcoreinfo.c   | 208 
> +
>  hw/i386/acpi-build.c   |  14 +++
>  default-configs/arm-softmmu.mak|   1 +
>  default-configs/i386-softmmu.mak   |   1 +
>  default-configs/x86_64-softmmu.mak |   1 +
>  docs/specs/vmcoreinfo.txt  | 138 
>  hw/acpi/Makefile.objs  |   1 +
>  10 files changed, 404 insertions(+)
>  create mode 100644 include/hw/acpi/vmcoreinfo.h
>  create mode 100644 hw/acpi/vmcoreinfo.c
>  create mode 100644 docs/specs/vmcoreinfo.txt
> 
> diff --git a/include/hw/acpi/aml-build.h b/include/hw/acpi/aml-build.h
> index 88d0738d76..cf781bcd34 100644
> --- a/include/hw/acpi/aml-build.h
> +++ b/include/hw/acpi/aml-build.h
> @@ -211,6 +211,7 @@ struct AcpiBuildTables {
>  GArray *rsdp;
>  GArray *tcpalog;
>  GArray *vmgenid;
> +GArray *vmcoreinfo;
>  BIOSLinker *linker;
>  } AcpiBuildTables;
>  
> diff --git a/include/hw/acpi/vmcoreinfo.h b/include/hw/acpi/vmcoreinfo.h
> new file mode 100644
> index 00..6a73bcd1b2
> --- /dev/null
> +++ b/include/hw/acpi/vmcoreinfo.h
> @@ -0,0 +1,37 @@
> +#ifndef ACPI_VMCOREINFO_H
> +#define ACPI_VMCOREINFO_H
> +
> +#include "hw/acpi/bios-linker-loader.h"
> +#include "hw/qdev.h"
> +
> +#define VMCOREINFO_DEVICE   "vmcoreinfo"
> +#define VMCOREINFO_FW_CFG_FILE  "etc/vmcoreinfo"
> +#define VMCOREINFO_ADDR_FW_CFG_FILE "etc/vmcoreinfo-addr"
> +
> +/* Occupy a page of memory */
> +#define VMCOREINFO_FW_CFG_SIZE  4096
> +
> +/* allow space for OVMF SDT Header Probe Supressor */
> +#define VMCOREINFO_OFFSET   sizeof(AcpiTableHeader)
> +
> +#define VMCOREINFO(obj) OBJECT_CHECK(VMCoreInfoState, (obj), 
> VMCOREINFO_DEVICE)
> +
> +typedef struct VMCoreInfoState {
> +DeviceClass parent_obj;
> +uint8_t vmcoreinfo_addr_le[8];   /* Address of memory region */
> +bool write_pointer_available;
> +} VMCoreInfoState;
> +
> +/* returns NULL unless there is exactly one device */
> +static inline Object *find_vmcoreinfo_dev(void)
> +{
> +return object_resolve_path_type("", VMCOREINFO_DEVICE, NULL);
> +}
> +
> +void vmcoreinfo_build_acpi(VMCoreInfoState *vis, GArray *table_data,
> +   GArray *vmci, BIOSLinker *linker);
> +void vmcoreinfo_add_fw_cfg(VMCoreInfoState *vis, FWCfgState *s, GArray 
> *vmci);
> +bool vmcoreinfo_get(VMCoreInfoState *vis, uint64_t *paddr, uint32_t *size,
> +Error **errp);
> +
> +#endif
> diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c
> index 36a6cc450e..47043ade4a 100644
> --- a/hw/acpi/aml-build.c
> +++ b/hw/acpi/aml-build.c
> @@ -1561,6 +1561,7 @@ void acpi_build_tables_init(AcpiBuildTables *tables)
>  tables->table_data = g_array_new(false, true /* clear */, 1);
>  tables->tcpalog = g_array_new(false, true /* clear */, 1);
>  tables->vmgenid = g_array_new(false, true /* clear */, 1);
> +tables->vmcoreinfo = g_array_new(false, true /* clear */, 1);
>  tables->linker = bios_linker_loader_init();
>  }
>  
> @@ -1571,6 +1572,7 @@ void acpi_build_tables_cleanup(AcpiBuildTables *tables, 
> bool mfre)
>  g_array_free(tables->table_data, true);
>  g_array_free(tables->tcpalog, mfre);
>  g_array_free(tables->vmgenid, mfre);
> +g_array_free(tables->vmcoreinfo, mfre);
>  }
>  
>  /* Build rsdt table */
> diff --git a/hw/acpi/vmcoreinfo.c b/hw/acpi/vmcoreinfo.c
> new file mode 100644
> index 00..0ea41de8d9
> --- /dev/null
> +++ b/hw/acpi/vmcoreinfo.c
> @@ -0,0 +1,208 @@
> +/*
> + *  Virtual Machine coreinfo device
> + *  (based on Virtual Machine Generation ID Device)
> + *
> + *  Copyright (C) 2017 Red Hat, Inc.
> + *  Copyright (C) 2017 Skyport Systems.
> + *
> + *  Authors: Marc-André Lureau 
> + *   Ben Warren 
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> + * See the COPYING file in the top-level directory.
> + *
> + */
> +#include "qemu/osdep.h"
> +#include "hw/acpi/acpi.h"
> +#include "hw/acpi/aml-build.h"
> +#include "hw/acpi/vmcoreinfo.h"
> +#incl

Re: [Qemu-devel] [PATCH v4 1/8] vmgenid: replace x-write-pointer-available hack

On Fri, Jul 14, 2017 at 08:20:04PM +0200, Marc-André Lureau wrote:
> This compat property sole function is to prevent the device from being
> instantiated. Instead of requiring an extra compat property, check if
> fw_cfg has DMA enabled.
> 
> fw_cfg is a built-in device that is initialized very early by the
> machine init code.  We have at least one other device that also
> assumes fw_cfg_find() can be safely used on realize: pvpanic.
> 
> This has the additional benefit of handling other cases properly, like:
> 
>   $ qemu-system-x86_64 -device vmgenid -machine none
>   qemu-system-x86_64: -device vmgenid: vmgenid requires DMA write support in 
> fw_cfg, which this machine type does not provide
>   $ qemu-system-x86_64 -device vmgenid -machine pc-i440fx-2.9 -global 
> fw_cfg.dma_enabled=off
>   qemu-system-x86_64: -device vmgenid: vmgenid requires DMA write support in 
> fw_cfg, which this machine type does not provide
>   $ qemu-system-x86_64 -device vmgenid -machine pc-i440fx-2.6 -global 
> fw_cfg.dma_enabled=on
>   [boots normally]
> 
> Suggested-by: Eduardo Habkost 
> Signed-off-by: Marc-André Lureau 
> Reviewed-by: Michael S. Tsirkin 
> Reviewed-by: Eduardo Habkost 
> Reviewed-by: Ben Warren 
> Reviewed-by: Laszlo Ersek 

Why tree would you like to merge this through?
If it's mine pls copy me on all patches in the set.

> ---
>  include/hw/acpi/bios-linker-loader.h |  2 ++
>  include/hw/compat.h  |  4 
>  hw/acpi/bios-linker-loader.c | 10 ++
>  hw/acpi/vmgenid.c|  9 +
>  4 files changed, 13 insertions(+), 12 deletions(-)
> 
> diff --git a/include/hw/acpi/bios-linker-loader.h 
> b/include/hw/acpi/bios-linker-loader.h
> index efe17b0b9c..a711dbced8 100644
> --- a/include/hw/acpi/bios-linker-loader.h
> +++ b/include/hw/acpi/bios-linker-loader.h
> @@ -7,6 +7,8 @@ typedef struct BIOSLinker {
>  GArray *file_list;
>  } BIOSLinker;
>  
> +bool bios_linker_loader_can_write_pointer(void);
> +
>  BIOSLinker *bios_linker_loader_init(void);
>  
>  void bios_linker_loader_alloc(BIOSLinker *linker,
> diff --git a/include/hw/compat.h b/include/hw/compat.h
> index 08f36004da..f414786604 100644
> --- a/include/hw/compat.h
> +++ b/include/hw/compat.h
> @@ -150,10 +150,6 @@
>  .driver   = "fw_cfg_io",\
>  .property = "dma_enabled",\
>  .value= "off",\
> -},{\
> -.driver   = "vmgenid",\
> -.property = "x-write-pointer-available",\
> -.value= "off",\
>  },
>  
>  #define HW_COMPAT_2_3 \
> diff --git a/hw/acpi/bios-linker-loader.c b/hw/acpi/bios-linker-loader.c
> index 046183a0f1..d16b8bbcb1 100644
> --- a/hw/acpi/bios-linker-loader.c
> +++ b/hw/acpi/bios-linker-loader.c
> @@ -168,6 +168,16 @@ bios_linker_find_file(const BIOSLinker *linker, const 
> char *name)
>  return NULL;
>  }
>  
> +/*
> + * board code must realize fw_cfg first, as a fixed device, before
> + * another device realize function call 
> bios_linker_loader_can_write_pointer()
> + */
> +bool bios_linker_loader_can_write_pointer(void)
> +{
> +FWCfgState *fw_cfg = fw_cfg_find();
> +return fw_cfg && fw_cfg_dma_enabled(fw_cfg);
> +}
> +
>  /*
>   * bios_linker_loader_alloc: ask guest to load file into guest memory.
>   *
> diff --git a/hw/acpi/vmgenid.c b/hw/acpi/vmgenid.c
> index a32b847fe0..ab5da293fd 100644
> --- a/hw/acpi/vmgenid.c
> +++ b/hw/acpi/vmgenid.c
> @@ -205,17 +205,11 @@ static void vmgenid_handle_reset(void *opaque)
>  memset(vms->vmgenid_addr_le, 0, ARRAY_SIZE(vms->vmgenid_addr_le));
>  }
>  
> -static Property vmgenid_properties[] = {
> -DEFINE_PROP_BOOL("x-write-pointer-available", VmGenIdState,
> - write_pointer_available, true),
> -DEFINE_PROP_END_OF_LIST(),
> -};
> -
>  static void vmgenid_realize(DeviceState *dev, Error **errp)
>  {
>  VmGenIdState *vms = VMGENID(dev);
>  
> -if (!vms->write_pointer_available) {
> +if (!bios_linker_loader_can_write_pointer()) {
>  error_setg(errp, "%s requires DMA write support in fw_cfg, "
> "which this machine type does not provide", 
> VMGENID_DEVICE);
>  return;
> @@ -239,7 +233,6 @@ static void vmgenid_device_class_init(ObjectClass *klass, 
> void *data)
>  dc->vmsd = &vmstate_vmgenid;
>  dc->realize = vmgenid_realize;
>  dc->hotpluggable = false;
> -dc->props = vmgenid_properties;
>  
>  object_class_property_add_str(klass, VMGENID_GUID, NULL,
>vmgenid_set_guid, NULL);
> -- 
> 2.13.1.395.gf7b71de06

Re: [Qemu-devel] [PATCH v12 6/8] mm: support reporting free page blocks

On Fri, Jul 14, 2017 at 02:30:23PM +0200, Michal Hocko wrote:
> On Wed 12-07-17 20:40:19, Wei Wang wrote:
> > This patch adds support for reporting blocks of pages on the free list
> > specified by the caller.
> > 
> > As pages can leave the free list during this call or immediately
> > afterwards, they are not guaranteed to be free after the function
> > returns. The only guarantee this makes is that the page was on the free
> > list at some point in time after the function has been invoked.
> > 
> > Therefore, it is not safe for caller to use any pages on the returned
> > block or to discard data that is put there after the function returns.
> > However, it is safe for caller to discard data that was in one of these
> > pages before the function was invoked.
> 
> I do not understand what is the point of such a function and how it is
> used because the patch doesn't give us any user (I haven't checked other
> patches yet).
> 
> But just from the semantic point of view this sounds like a horrible
> idea. The only way to get a free block of pages is to call the page
> allocator. I am tempted to give it Nack right on those grounds but I
> would like to hear more about what you actually want to achieve.

Basically it's a performance hint to the hypervisor.
For example, these pages would be good candidates to
move around as they are not mapped into any running
applications.

As such, it's important not to slow down other parts of the system too
much - otherwise we are speeding up one part of the system while we slow
down other parts of it, which is why it's trying to drop the lock as
soon a possible.

As long as hypervisor does not assume it can drop these pages, and as
long it's correct in most cases.  we are OK even if the hint is slightly
wrong because hypervisor notifications are racing with allocations.

There are patches to do more tricks - if hypervisor tracks all
memory writes we might actually use this hint to discard data -
but that is just implementation detail.


> > Signed-off-by: Wei Wang 
> > Signed-off-by: Liang Li 
> > ---
> >  include/linux/mm.h |  5 +++
> >  mm/page_alloc.c| 96 
> > ++
> >  2 files changed, 101 insertions(+)
> > 
> > diff --git a/include/linux/mm.h b/include/linux/mm.h
> > index 46b9ac5..76cb433 100644
> > --- a/include/linux/mm.h
> > +++ b/include/linux/mm.h
> > @@ -1835,6 +1835,11 @@ extern void free_area_init_node(int nid, unsigned 
> > long * zones_size,
> > unsigned long zone_start_pfn, unsigned long *zholes_size);
> >  extern void free_initmem(void);
> >  
> > +#if IS_ENABLED(CONFIG_VIRTIO_BALLOON)
> > +extern int report_unused_page_block(struct zone *zone, unsigned int order,
> > +   unsigned int migratetype,
> > +   struct page **page);
> > +#endif
> >  /*
> >   * Free reserved pages within range [PAGE_ALIGN(start), end & PAGE_MASK)
> >   * into the buddy system. The freed pages will be poisoned with pattern
> > diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> > index 64b7d82..8b3c9dd 100644
> > --- a/mm/page_alloc.c
> > +++ b/mm/page_alloc.c
> > @@ -4753,6 +4753,102 @@ void show_free_areas(unsigned int filter, 
> > nodemask_t *nodemask)
> > show_swap_cache_info();
> >  }
> >  
> > +#if IS_ENABLED(CONFIG_VIRTIO_BALLOON)
> > +
> > +/*
> > + * Heuristically get a page block in the system that is unused.
> > + * It is possible that pages from the page block are used immediately after
> > + * report_unused_page_block() returns. It is the caller's responsibility
> > + * to either detect or prevent the use of such pages.
> > + *
> > + * The free list to check: zone->free_area[order].free_list[migratetype].
> > + *
> > + * If the caller supplied page block (i.e. **page) is on the free list, 
> > offer
> > + * the next page block on the list to the caller. Otherwise, offer the 
> > first
> > + * page block on the list.
> > + *
> > + * Note: it is not safe for caller to use any pages on the returned
> > + * block or to discard data that is put there after the function returns.
> > + * However, it is safe for caller to discard data that was in one of these
> > + * pages before the function was invoked.
> > + *
> > + * Return 0 when a page block is found on the caller specified free list.
> > + */
> > +int report_unused_page_block(struct zone *zone, unsigned int order,
> > +unsigned int migratetype, struct page **page)
> > +{
> > +   struct zone *this_zone;
> > +   struct list_head *this_list;
> > +   int ret = 0;
> > +   unsigned long flags;
> > +
> > +   /* Sanity check */
> > +   if (zone == NULL || page == NULL || order >= MAX_ORDER ||
> > +   migratetype >= MIGRATE_TYPES)
> > +   return -EINVAL;
> > +
> > +   /* Zone validity check */
> > +   for_each_populated_zone(this_zone) {
> > +   if (zone == this_zone)
> > +   break;
> > +   }
> > +
> > +   /* Got a non-existent zone from the calle

Re: [Qemu-devel] Supporting unsafe create when backing file is not accessible

2017-07-14 Thread Nir Soffer

On Wed, Jul 12, 2017 at 5:40 PM Eric Blake  wrote:

> On 07/12/2017 03:56 AM, Ala Hino wrote:
> > Hi,
> >
> > We encountered a performance issue when creating a volume for a running
> VM
> > and we'd like to share the info with you. The root cause of the issue is
> in
> > our code but we found a workaround that relies on qemu-img create
> > undocumented behavior.
> >
> > During our tests, we found that in order to create a volume with a
> backing
> > file, the baking file has to be valid and accessible.
>
> In general, that's a good thing. But you're also right that it is nice
> to have a mode where the backing file is not probed.
>
> > This requires us to
> > activate the entire chain before creating the volume, and deactivate the
> > chain after the operation completes. Activating and deactivating the
> chain
> > are expensive operations that we prefer to avoid when possible. Below is
> > the command we use to create volumes:
> >
> > qemu-img create -f qcow2 -o compat=1.1 -b
> > 8a28cda2-548d-47da-bbba-faa81284f6ba -F raw
> >
> /rhev/data-center/e6b272af-84cb-43fc-ae5b-7fe18bc469a2/f47c980a-fd76-44a9-8b78-00d3ab2ffd2f/images/2ff0a3c0-f145-4f83-b668-fc0c39ba191f/d3b69657-892f-499c-9ac3-9c443ead7d9b
> > 1073741824
> >
> > We also found that when providing the size and the backing format for
> qemu,
> > qemu doesn't open the backing chain, and in this case we don't have to
> > activate/deactivate the entire chain - exactly the behavior we wish to
> have.
>
> Yes, that is currently the case. You are correct that patches have been
> proposed to tighten things so that we would probe the existence of the
> backing file in more cases (even when the size is provided); and that if
> we do so, we'd also have to provide a backdoor for creating an image
> without probing the backing file.  But it is also the case that you can
> create an image with NO backing file, and then use 'qemu-img rebase -u'
> to add the backing file after the fact, without waiting for any proposed
> patches to land.
>
> > We we'd like to get your confirmation of the above behavior as it isn't
> > documented, and whether it can be documented.
>
> The fact that you are asking does mean that we should revive John's
> proposed patches, in some form or another.
>

Eric, we are more concerned about using the current qemu version.

We can use the fact that providing both size and backing format,
qemu does not open the backing file, but this is not documented, and
we don't want to base oVirt code on undocumented behavior.

What we would like to have is:
- qemu blessing for using this undocumented behaviour
- and documenting this behavior in qemu-img(1)

With this we can fix https://bugzilla.redhat.com/1395941

now, instead of waiting for qemu 2.10.

For future version, having a explicit way to allow unsafe create
is of course better.

Nir

>
> > In addition, we are aware of https://bugzilla.redhat.com/1213786, where
> a
> > -u (unsafe) option is planned to be added (see comment #4 in the BZ). Can
> > you please confirm that once that support is released, it won't break
> > existing, i.e. code that provides size and backing format assuming that
> > "unsafe" create is supported?
>
> If we tighten things to require the backing file to exist unless -u is
> provided, then providing the size alone will no longer be sufficient to
> prevent the probe - you'd have to use -u to prevent the probe, or change
> your workflow to create the image without a backing file then add in the
> backing information via 'qemu-img rebase -u'.
>
> --
> Eric Blake, Principal Software Engineer
> Red Hat, Inc.   +1-919-301-3266 <(919)%20301-3266>
> Virtualization:  qemu.org | libvirt.org
>
>

[Qemu-devel] [PATCH 4/5] qtest: Avoid passing raw strings through hmp()

The next patch will add __attribute__((__format__)) to hmp(), which
in turn causes gcc to warn about non-literal format strings.  Rather
than risk an arbitrary string containing % being mis-handled, always
pass variable strings along with a %s format.

Signed-off-by: Eric Blake 
---
 tests/test-hmp.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tests/test-hmp.c b/tests/test-hmp.c
index d77b3c8..0af0664 100644
--- a/tests/test-hmp.c
+++ b/tests/test-hmp.c
@@ -80,7 +80,7 @@ static void test_commands(void)
 if (verbose) {
 fprintf(stderr, "\t%s\n", hmp_cmds[i]);
 }
-response = hmp(hmp_cmds[i]);
+response = hmp("%s", hmp_cmds[i]);
 g_free(response);
 }

@@ -103,7 +103,7 @@ static void test_info_commands(void)
 if (verbose) {
 fprintf(stderr, "\t%s\n", info);
 }
-resp = hmp(info);
+resp = hmp("%s", info);
 g_free(resp);
 /* And move forward to the next line */
 info = strchr(endp + 1, '\n');
-- 
2.9.4

[Qemu-devel] [PATCH 1/5] qapi: Further enhance visitor virtual walk doc example

Markus pointed out that the example given for virtual walks did
not discuss how to do a virtual walk of an alternate type.  It
turns out that for output, we don't need to visit an alternate
(just directly visit the type that we want); and for input,
visit_start_alternate() is not currently wired up for alternates
(it requires a QAPI type, rather than permitting NULL for a
virtual walk).  Also, the example was never updated in commit
3b098d5 about where visit_complete() would fit in.  Improve the
description and example to give more details along these lines.

Signed-off-by: Eric Blake 
---
 include/qapi/visitor.h | 41 -
 1 file changed, 28 insertions(+), 13 deletions(-)

diff --git a/include/qapi/visitor.h b/include/qapi/visitor.h
index 74768aa..b0a048f 100644
--- a/include/qapi/visitor.h
+++ b/include/qapi/visitor.h
@@ -183,19 +183,25 @@
  *
  * It is also possible to use the visitors to do a virtual walk, where
  * no actual QAPI struct is present.  In this situation, decisions
- * about what needs to be walked are made by the calling code, and
- * structured visits are split between pairs of start and end methods
- * (where the end method must be called if the start function
- * succeeded, even if an intermediate visit encounters an error).
- * Thus, a virtual walk corresponding to '{ "list": [1, 2] }' looks
- * like:
+ * about what needs to be walked are made by the calling code (that
+ * is, there is no use for QAPI alternate types in a virtual walk,
+ * because the code can just directly visit the appropriate type
+ * within the alternate), and structured visits are split between
+ * pairs of start and end methods (where the end method must be called
+ * if the start function succeeded, even if an intermediate visit
+ * encounters an error).  Thus, a virtual output walk of an object
+ * containing a list of alternates between an integer or nested
+ * object, corresponding to '{ "list": [1, { "value": "2" } ] }',
+ * would look like:
  *
  * 
  *  Visitor *v;
  *  Error *err = NULL;
- *  int value;
+ *  int i = 1;
+ *  const char *s = "2";
+ *  FOO output;
  *
- *  v = FOO_visitor_new(...);
+ *  v = FOO_visitor_new(..., &output);
  *  visit_start_struct(v, NULL, NULL, 0, &err);
  *  if (err) {
  *  goto out;
@@ -204,16 +210,21 @@
  *  if (err) {
  *  goto outobj;
  *  }
- *  value = 1;
- *  visit_type_int(v, NULL, &value, &err);
+ *  visit_type_int(v, NULL, &i, &err);
  *  if (err) {
  *  goto outlist;
  *  }
- *  value = 2;
- *  visit_type_int(v, NULL, &value, &err);
+ *  visit_type_start(v, NULL, NULL, 0, &err);
  *  if (err) {
- *  goto outlist;
+ *  goto outnest;
+ *  }
+ *  visit_type_str(v, "value", (char **)&s, &err);
+ *  if (err) {
+ *  goto outnest;
  *  }
+ *  visit_check_struct(v, &err);
+ * outnest:
+ *  visit_end_struct(v, NULL);
  * outlist:
  *  visit_end_list(v, NULL);
  *  if (!err) {
@@ -221,6 +232,10 @@
  *  }
  * outobj:
  *  visit_end_struct(v, NULL);
+ *  if (!err) {
+ *  visit_complete(v, &output);
+ *  ...use output...
+ *  }
  * out:
  *  error_propagate(errp, err);
  *  visit_free(v);
-- 
2.9.4

[Qemu-devel] [PATCH 3/5] qapi: Visitor documentation tweak

Make it clear that the name parameter to visit_start_struct()
has the same semantics as for visit_start_int().

Signed-off-by: Eric Blake 
---
 include/qapi/visitor.h | 14 +++---
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/include/qapi/visitor.h b/include/qapi/visitor.h
index b0a048f..dc09cc7 100644
--- a/include/qapi/visitor.h
+++ b/include/qapi/visitor.h
@@ -60,13 +60,13 @@
  * visitors are declared here; the remaining visitors are generated in
  * qapi-visit.h.
  *
- * The @name parameter of visit_type_FOO() describes the relation
- * between this QAPI value and its parent container.  When visiting
- * the root of a tree, @name is ignored; when visiting a member of an
- * object, @name is the key associated with the value; when visiting a
- * member of a list, @name is NULL; and when visiting the member of an
- * alternate, @name should equal the name used for visiting the
- * alternate.
+ * The @name parameter of visit_type_FOO() and visit_start_OBJECT()
+ * describes the relation between this QAPI value and its parent
+ * container.  When visiting the root of a tree, @name is ignored;
+ * when visiting a member of an object, @name is the key associated
+ * with the value; when visiting a member of a list, @name is NULL;
+ * and when visiting the member of an alternate, @name should equal
+ * the name used for visiting the alternate.
  *
  * The visit_type_FOO() functions expect a non-null @obj argument;
  * they allocate *@obj during input visits, leave it unchanged on
-- 
2.9.4

[Qemu-devel] [PATCH 5/5] qtest: Document calling conventions

We have two flavors of vararg usage in qtest; make it clear that
qmp() has different semantics than hmp(), and let the compiler
enforce that hmp() is used correctly. Since qmp() only accepts
a subset of printf flags (namely, those that our JSON parser
understands), I figured that it is probably not worth adding a
format attribute to qmp() at this time.

Signed-off-by: Eric Blake 
---
 tests/libqtest.h | 23 ++-
 1 file changed, 14 insertions(+), 9 deletions(-)

diff --git a/tests/libqtest.h b/tests/libqtest.h
index 38bc1e9..762ed13 100644
--- a/tests/libqtest.h
+++ b/tests/libqtest.h
@@ -50,7 +50,8 @@ void qtest_quit(QTestState *s);
 /**
  * qtest_qmp_discard_response:
  * @s: #QTestState instance to operate on.
- * @fmt...: QMP message to send to qemu
+ * @fmt...: QMP message to send to qemu; only recognizes formats
+ * understood by json-lexer.c
  *
  * Sends a QMP message to QEMU and consumes the response.
  */
@@ -59,7 +60,8 @@ void qtest_qmp_discard_response(QTestState *s, const char 
*fmt, ...);
 /**
  * qtest_qmp:
  * @s: #QTestState instance to operate on.
- * @fmt...: QMP message to send to qemu
+ * @fmt...: QMP message to send to qemu; only recognizes formats
+ * understood by json-lexer.c
  *
  * Sends a QMP message to QEMU and returns the response.
  */
@@ -134,14 +136,14 @@ QDict *qtest_qmp_eventwait_ref(QTestState *s, const char 
*event);
 /**
  * qtest_hmp:
  * @s: #QTestState instance to operate on.
- * @fmt...: HMP command to send to QEMU
+ * @fmt...: HMP command to send to QEMU, passed through sprintf()
  *
  * Send HMP command to QEMU via QMP's human-monitor-command.
  * QMP events are discarded.
  *
  * Returns: the command's output.  The caller should g_free() it.
  */
-char *qtest_hmp(QTestState *s, const char *fmt, ...);
+char *qtest_hmp(QTestState *s, const char *fmt, ...) GCC_FMT_ATTR(2, 3);

 /**
  * qtest_hmpv:
@@ -535,7 +537,8 @@ static inline void qtest_end(void)

 /**
  * qmp:
- * @fmt...: QMP message to send to qemu
+ * @fmt...: QMP message to send to qemu; only recognizes formats
+ * understood by json-lexer.c
  *
  * Sends a QMP message to QEMU and returns the response.
  */
@@ -543,7 +546,8 @@ QDict *qmp(const char *fmt, ...);

 /**
  * qmp_async:
- * @fmt...: QMP message to send to qemu
+ * @fmt...: QMP message to send to qemu; only recognizes formats
+ * understood by json-lexer.c
  *
  * Sends a QMP message to QEMU and leaves the response in the stream.
  */
@@ -551,7 +555,8 @@ void qmp_async(const char *fmt, ...);

 /**
  * qmp_discard_response:
- * @fmt...: QMP message to send to qemu
+ * @fmt...: QMP message to send to qemu; only recognizes formats
+ * understood by json-lexer.c
  *
  * Sends a QMP message to QEMU and consumes the response.
  */
@@ -592,13 +597,13 @@ static inline QDict *qmp_eventwait_ref(const char *event)

 /**
  * hmp:
- * @fmt...: HMP command to send to QEMU
+ * @fmt...: HMP command to send to QEMU, passed through printf()
  *
  * Send HMP command to QEMU via QMP's human-monitor-command.
  *
  * Returns: the command's output.  The caller should g_free() it.
  */
-char *hmp(const char *fmt, ...);
+char *hmp(const char *fmt, ...) GCC_FMT_ATTR(1, 2);

 /**
  * get_irq:
-- 
2.9.4

[Qemu-devel] [PATCH 2/5] tests: Enhance qobject output to cover partial visit

Add a test that proves (at least when run under valgrind) that
we are correctly handling allocated memory even when a visit
is aborted in the middle for whatever other reason.

See commit f24582d "qapi: fix double free in
qmp_output_visitor_cleanup()" for a fix that was lacking
testsuite exposure prior to this patch.

Signed-off-by: Eric Blake 
---
 tests/test-qobject-output-visitor.c | 46 -
 1 file changed, 45 insertions(+), 1 deletion(-)

diff --git a/tests/test-qobject-output-visitor.c 
b/tests/test-qobject-output-visitor.c
index 749c540..1e9a5d1 100644
--- a/tests/test-qobject-output-visitor.c
+++ b/tests/test-qobject-output-visitor.c
@@ -1,7 +1,7 @@
 /*
  * QObject Output Visitor unit-tests.
  *
- * Copyright (C) 2011-2016 Red Hat Inc.
+ * Copyright (C) 2011-2017 Red Hat Inc.
  *
  * Authors:
  *  Luiz Capitulino 
@@ -251,6 +251,48 @@ static void 
test_visitor_out_struct_errors(TestOutputVisitorData *data,
 }


+static void test_visitor_out_partial_visit(TestOutputVisitorData *data,
+   const void *unused)
+{
+/* Various checks that a mid-visit abort doesn't leak or double-free. */
+const char *str = "hi";
+Error *err = NULL;
+UserDefAlternate uda = {
+.type = QTYPE_QDICT,
+.u.udfu = { .integer = 1,
+.string = (char *) "bye",
+.enum1 = -1 } /* intentionally bad */
+};
+UserDefAlternate *obj = &uda;
+
+/* Abort within a nested object with no data members */
+visit_start_struct(data->ov, NULL, NULL, 0, &error_abort);
+visit_start_struct(data->ov, "nested", NULL, 0, &error_abort);
+visitor_reset(data);
+
+/* Abort in the middle of a list of strings */
+visit_start_list(data->ov, "list", NULL, 0, &error_abort);
+visit_type_str(data->ov, NULL, (char **)&str, &error_abort);
+visit_type_str(data->ov, NULL, (char **)&str, &error_abort);
+visitor_reset(data);
+
+/*
+ * Abort in the middle of an alternate. Alternates can't be
+ * virtually visited, so we get to inline the first half of
+ * visit_type_UserDefAlternate().
+ */
+visit_start_alternate(data->ov, NULL, (GenericAlternate **)&obj,
+  sizeof(uda), &error_abort);
+visit_start_struct(data->ov, NULL, NULL, 0, &error_abort);
+visit_type_UserDefUnionBase_members(data->ov,
+(UserDefUnionBase *)&uda.u.udfu,
+&err);
+/* error expected because of bad "enum1" discriminator value */
+error_free_or_abort(&err);
+visitor_reset(data);
+}
+
+
 static void test_visitor_out_list(TestOutputVisitorData *data,
   const void *unused)
 {
@@ -815,6 +857,8 @@ int main(int argc, char **argv)
 &out_visitor_data, test_visitor_out_struct_nested);
 output_visitor_test_add("/visitor/output/struct-errors",
 &out_visitor_data, test_visitor_out_struct_errors);
+output_visitor_test_add("/visitor/output/partial-visit",
+&out_visitor_data, test_visitor_out_partial_visit);
 output_visitor_test_add("/visitor/output/list",
 &out_visitor_data, test_visitor_out_list);
 output_visitor_test_add("/visitor/output/any",
-- 
2.9.4

[Qemu-devel] [PATCH 0/5] random qapi cleanups

Trying to flush some of the random patches I have lying around
locally.  Some of these may have been posted to the list before,
perhaps as part of larger series, but I didn't bother to research
which parts came from where.

I'm not sure if this is in time for softfreeze; and if not, I'm
not sure how much is appropriate during freeze vs. being deferred
to 2.11.

Eric Blake (5):
  qapi: Further enhance visitor virtual walk doc example
  tests: Enhance qobject output to cover partial visit
  qapi: Visitor documentation tweak
  qtest: Avoid passing raw strings through hmp()
  qtest: Document calling conventions

 include/qapi/visitor.h  | 55 +++--
 tests/libqtest.h| 23 ++--
 tests/test-hmp.c|  4 +--
 tests/test-qobject-output-visitor.c | 46 ++-
 4 files changed, 96 insertions(+), 32 deletions(-)

-- 
2.9.4

Re: [Qemu-devel] [PATCHv9 3/3] fw_cfg: move QOM type defines and fw_cfg types into fw_cfg.h

On Fri, Jul 14, 2017 at 08:28:37PM +0200, Laszlo Ersek wrote:
> On 07/14/17 20:09, Eduardo Habkost wrote:
> > On Fri, Jul 14, 2017 at 10:40:08AM +0100, Mark Cave-Ayland wrote:
> >> By exposing FWCfgIoState and FWCfgMemState internals we allow the 
> >> possibility
> >> for the internal MemoryRegion fields to be mapped by name for boards that 
> >> wish
> >> to wire up the fw_cfg device themselves.
> >>
> >> Signed-off-by: Mark Cave-Ayland 
> >> Reviewed-by: Laszlo Ersek 
> > [...]
> >> diff --git a/include/hw/nvram/fw_cfg.h b/include/hw/nvram/fw_cfg.h
> >> index b980cba..b77ea48 100644
> >> --- a/include/hw/nvram/fw_cfg.h
> >> +++ b/include/hw/nvram/fw_cfg.h
> >> @@ -1,8 +1,19 @@
> >>  #ifndef FW_CFG_H
> >>  #define FW_CFG_H
> >>  
> >> +#include "qemu/typedefs.h"
> >>  #include "exec/hwaddr.h"
> >>  #include "hw/nvram/fw_cfg_keys.h"
> >> +#include "hw/sysbus.h"
> >> +#include "sysemu/dma.h"
> >> +
> >> +#define TYPE_FW_CFG "fw_cfg"
> >> +#define TYPE_FW_CFG_IO  "fw_cfg_io"
> >> +#define TYPE_FW_CFG_MEM "fw_cfg_mem"
> >> +
> >> +#define FW_CFG(obj) OBJECT_CHECK(FWCfgState,(obj), TYPE_FW_CFG)
> >> +#define FW_CFG_IO(obj)  OBJECT_CHECK(FWCfgIoState,  (obj), TYPE_FW_CFG_IO)
> >> +#define FW_CFG_MEM(obj) OBJECT_CHECK(FWCfgMemState, (obj), 
> >> TYPE_FW_CFG_MEM)
> >>  
> >>  typedef struct FWCfgFile {
> >>  uint32_t  size;/* file size */
> >> @@ -35,6 +46,45 @@ typedef struct FWCfgDmaAccess {
> >>  
> >>  typedef void (*FWCfgReadCallback)(void *opaque);
> >>  
> >> +struct FWCfgState {
> >> +/*< private >*/
> >> +SysBusDevice parent_obj;
> >> +/*< public >*/
> >> +
> >> +uint16_t file_slots;
> >> +FWCfgEntry *entries[2];
> >> +int *entry_order;
> >> +FWCfgFiles *files;
> >> +uint16_t cur_entry;
> >> +uint32_t cur_offset;
> >> +Notifier machine_ready;
> >> +
> >> +int fw_cfg_order_override;
> >> +
> >> +bool dma_enabled;
> >> +dma_addr_t dma_addr;
> >> +AddressSpace *dma_as;
> >> +MemoryRegion dma_iomem;
> >> +};
> >> +
> >> +struct FWCfgIoState {
> >> +/*< private >*/
> >> +FWCfgState parent_obj;
> >> +/*< public >*/
> >> +
> >> +MemoryRegion comb_iomem;
> >> +};
> >> +
> >> +struct FWCfgMemState {
> >> +/*< private >*/
> >> +FWCfgState parent_obj;
> >> +/*< public >*/
> >> +
> >> +MemoryRegion ctl_iomem, data_iomem;
> >> +uint32_t data_width;
> >> +MemoryRegionOps wide_data_ops;
> >> +};
> > 
> > Why do you need the full struct declaration to be exposed in the
> > header?
> 
> Different board code wants to hook up "comb_iomem" manually to different
> address spaces, so they need to access the field directly. This is the
> ultimate goal of the entire exercise, IIRC.
> 
> > The memory regions are supposed to be visible as QOM
> > children to the fw_cfg device, already.
> 
> I don't understand this. How else can board code work with "comb_iomem"
> than described above? If there is a way, I agree it would be preferable.

object_resolve_path_component(fw_cfg, "fwcfg[0]") and
object_resolve_path_component(fw_cfg, "fwcfg.dma[0]") should
return fw_cfg->comb_iomem and fw_cfg->dma_iomem, respectively.

I don't know why those names were chosen, though.  Probably it's
a good idea to call object_property_add_child() manually with
more appropriate names inside the fw_cfg code instead of letting
memory_region_init() pick the child name.

-- 
Eduardo

[Qemu-devel] [PATCH 1/2] nbd: Trace client command being sent

Make the client trace slightly more legible by including the name
of the command being sent.

Signed-off-by: Eric Blake 
---
 nbd/client.c | 3 ++-
 nbd/trace-events | 2 +-
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/nbd/client.c b/nbd/client.c
index c3ee9f3..4c12fff 100644
--- a/nbd/client.c
+++ b/nbd/client.c
@@ -902,7 +902,8 @@ ssize_t nbd_send_request(QIOChannel *ioc, NBDRequest 
*request)
 uint8_t buf[NBD_REQUEST_SIZE];

 trace_nbd_send_request(request->from, request->len, request->handle,
-   request->flags, request->type);
+   request->flags, request->type,
+   nbd_cmd_lookup(request->type));

 stl_be_p(buf, NBD_REQUEST_MAGIC);
 stw_be_p(buf + 4, request->flags);
diff --git a/nbd/trace-events b/nbd/trace-events
index f5024d8..d7c7746 100644
--- a/nbd/trace-events
+++ b/nbd/trace-events
@@ -28,7 +28,7 @@ nbd_client_loop(void) "Doing NBD loop"
 nbd_client_loop_ret(int ret, const char *error) "NBD loop returned %d: %s"
 nbd_client_clear_queue(void) "Clearing NBD queue"
 nbd_client_clear_socket(void) "Clearing NBD socket"
-nbd_send_request(uint64_t from, uint32_t len, uint64_t handle, uint16_t flags, 
uint16_t type) "Sending request to server: { .from = %" PRIu64", .len = %" 
PRIu32 ", .handle = %" PRIu64 ", .flags = %" PRIx16 ", .type = %" PRIu16 " }"
+nbd_send_request(uint64_t from, uint32_t len, uint64_t handle, uint16_t flags, 
uint16_t type, const char *name) "Sending request to server: { .from = %" 
PRIu64", .len = %" PRIu32 ", .handle = %" PRIu64 ", .flags = %" PRIx16 ", .type 
= %" PRIu16 " (%s) }"
 nbd_receive_reply(uint32_t magic, int32_t error, uint64_t handle) "Got reply: 
{ magic = 0x%" PRIx32 ", .error = % " PRId32 ", handle = %" PRIu64" }"

 # nbd/server.c
-- 
2.9.4

[Qemu-devel] [PATCH 2/2] nbd: Fix server reply to NBD_OPT_EXPORT_NAME of older clients

A typo in commit 23e099c set the size of buf[] used in response
to NBD_OPT_EXPORT_NAME according to the length needed for old-style
negotiation (4 bytes of flag information) instead of the intended
2 bytes used in new style.  If the client doesn't enable
NBD_FLAG_C_NO_ZEROES, then the server sends two bytes too many,
and is then out of sync in response to the client's next command
(the bug is masked when modern qemu is the client, since we enable
the no zeroes flag).

Signed-off-by: Eric Blake 
---
 nbd/server.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/nbd/server.c b/nbd/server.c
index 49ed574..bcb241c 100644
--- a/nbd/server.c
+++ b/nbd/server.c
@@ -283,7 +283,7 @@ static int nbd_negotiate_handle_export_name(NBDClient 
*client, uint32_t length,
 Error **errp)
 {
 char name[NBD_MAX_NAME_SIZE + 1];
-char buf[8 + 4 + 124] = "";
+char buf[8 + 2 + 124] = "";
 size_t len;
 int ret;

@@ -800,7 +800,7 @@ static int nbd_negotiate_options(NBDClient *client, 
uint16_t myflags,
  */
 static coroutine_fn int nbd_negotiate(NBDClient *client, Error **errp)
 {
-char buf[8 + 8 + 8 + 128];
+char buf[8 + 8 + 8 + 2 + 2 + 124];
 int ret;
 const uint16_t myflags = (NBD_FLAG_HAS_FLAGS | NBD_FLAG_SEND_TRIM |
   NBD_FLAG_SEND_FLUSH | NBD_FLAG_SEND_FUA |
-- 
2.9.4

[Qemu-devel] [PATCH 0/2] NBD fixes before softfreeze

Paolo's inclusion of my NBD series in his miscellaneous tree
ended up with a couple of bugs that were pointed out in the
thread after the pull request was first created; these are the
fixes.  They are small enough that I don't mind using maintainer
status to bundle them up as a PULL request for NBD sometime on
Monday (in order to make soft freeze), whether or not they've
been reviewed by then; but of course, review is appreciated.

https://lists.gnu.org/archive/html/qemu-devel/2017-07/msg01983.html
https://lists.gnu.org/archive/html/qemu-devel/2017-07/msg01981.html

Eric Blake (2):
  nbd: Trace client command being sent
  nbd: Fix server reply to NBD_OPT_EXPORT_NAME of older clients

 nbd/client.c | 3 ++-
 nbd/server.c | 4 ++--
 nbd/trace-events | 2 +-
 3 files changed, 5 insertions(+), 4 deletions(-)

-- 
2.9.4

Re: [Qemu-devel] [PATCHv9 3/3] fw_cfg: move QOM type defines and fw_cfg types into fw_cfg.h

On 07/14/17 20:09, Eduardo Habkost wrote:
> On Fri, Jul 14, 2017 at 10:40:08AM +0100, Mark Cave-Ayland wrote:
>> By exposing FWCfgIoState and FWCfgMemState internals we allow the possibility
>> for the internal MemoryRegion fields to be mapped by name for boards that 
>> wish
>> to wire up the fw_cfg device themselves.
>>
>> Signed-off-by: Mark Cave-Ayland 
>> Reviewed-by: Laszlo Ersek 
> [...]
>> diff --git a/include/hw/nvram/fw_cfg.h b/include/hw/nvram/fw_cfg.h
>> index b980cba..b77ea48 100644
>> --- a/include/hw/nvram/fw_cfg.h
>> +++ b/include/hw/nvram/fw_cfg.h
>> @@ -1,8 +1,19 @@
>>  #ifndef FW_CFG_H
>>  #define FW_CFG_H
>>  
>> +#include "qemu/typedefs.h"
>>  #include "exec/hwaddr.h"
>>  #include "hw/nvram/fw_cfg_keys.h"
>> +#include "hw/sysbus.h"
>> +#include "sysemu/dma.h"
>> +
>> +#define TYPE_FW_CFG "fw_cfg"
>> +#define TYPE_FW_CFG_IO  "fw_cfg_io"
>> +#define TYPE_FW_CFG_MEM "fw_cfg_mem"
>> +
>> +#define FW_CFG(obj) OBJECT_CHECK(FWCfgState,(obj), TYPE_FW_CFG)
>> +#define FW_CFG_IO(obj)  OBJECT_CHECK(FWCfgIoState,  (obj), TYPE_FW_CFG_IO)
>> +#define FW_CFG_MEM(obj) OBJECT_CHECK(FWCfgMemState, (obj), TYPE_FW_CFG_MEM)
>>  
>>  typedef struct FWCfgFile {
>>  uint32_t  size;/* file size */
>> @@ -35,6 +46,45 @@ typedef struct FWCfgDmaAccess {
>>  
>>  typedef void (*FWCfgReadCallback)(void *opaque);
>>  
>> +struct FWCfgState {
>> +/*< private >*/
>> +SysBusDevice parent_obj;
>> +/*< public >*/
>> +
>> +uint16_t file_slots;
>> +FWCfgEntry *entries[2];
>> +int *entry_order;
>> +FWCfgFiles *files;
>> +uint16_t cur_entry;
>> +uint32_t cur_offset;
>> +Notifier machine_ready;
>> +
>> +int fw_cfg_order_override;
>> +
>> +bool dma_enabled;
>> +dma_addr_t dma_addr;
>> +AddressSpace *dma_as;
>> +MemoryRegion dma_iomem;
>> +};
>> +
>> +struct FWCfgIoState {
>> +/*< private >*/
>> +FWCfgState parent_obj;
>> +/*< public >*/
>> +
>> +MemoryRegion comb_iomem;
>> +};
>> +
>> +struct FWCfgMemState {
>> +/*< private >*/
>> +FWCfgState parent_obj;
>> +/*< public >*/
>> +
>> +MemoryRegion ctl_iomem, data_iomem;
>> +uint32_t data_width;
>> +MemoryRegionOps wide_data_ops;
>> +};
> 
> Why do you need the full struct declaration to be exposed in the
> header?

Different board code wants to hook up "comb_iomem" manually to different
address spaces, so they need to access the field directly. This is the
ultimate goal of the entire exercise, IIRC.

> The memory regions are supposed to be visible as QOM
> children to the fw_cfg device, already.

I don't understand this. How else can board code work with "comb_iomem"
than described above? If there is a way, I agree it would be preferable.

Thanks
Laszlo

[Qemu-devel] [PATCH v4 2/8] acpi: add vmcoreinfo device

The VM coreinfo (vmcoreinfo) device is an emulated device which
exposes a 4k memory range to the guest to store various informations
useful to debug the guest OS. (it is greatly inspired by the VMGENID
device implementation)

This is an early-boot alternative to the qemu-ga VMDUMP_INFO event
proposed in "[PATCH 00/21] WIP: dump: add kaslr support".

A proof-of-concept kernel module:
https://github.com/elmarco/vmgenid-test/blob/master/qemuvmci-test.c

Signed-off-by: Marc-André Lureau 
Reviewed-by: Laszlo Ersek 
---
 include/hw/acpi/aml-build.h|   1 +
 include/hw/acpi/vmcoreinfo.h   |  37 +++
 hw/acpi/aml-build.c|   2 +
 hw/acpi/vmcoreinfo.c   | 208 +
 hw/i386/acpi-build.c   |  14 +++
 default-configs/arm-softmmu.mak|   1 +
 default-configs/i386-softmmu.mak   |   1 +
 default-configs/x86_64-softmmu.mak |   1 +
 docs/specs/vmcoreinfo.txt  | 138 
 hw/acpi/Makefile.objs  |   1 +
 10 files changed, 404 insertions(+)
 create mode 100644 include/hw/acpi/vmcoreinfo.h
 create mode 100644 hw/acpi/vmcoreinfo.c
 create mode 100644 docs/specs/vmcoreinfo.txt

diff --git a/include/hw/acpi/aml-build.h b/include/hw/acpi/aml-build.h
index 88d0738d76..cf781bcd34 100644
--- a/include/hw/acpi/aml-build.h
+++ b/include/hw/acpi/aml-build.h
@@ -211,6 +211,7 @@ struct AcpiBuildTables {
 GArray *rsdp;
 GArray *tcpalog;
 GArray *vmgenid;
+GArray *vmcoreinfo;
 BIOSLinker *linker;
 } AcpiBuildTables;
 
diff --git a/include/hw/acpi/vmcoreinfo.h b/include/hw/acpi/vmcoreinfo.h
new file mode 100644
index 00..6a73bcd1b2
--- /dev/null
+++ b/include/hw/acpi/vmcoreinfo.h
@@ -0,0 +1,37 @@
+#ifndef ACPI_VMCOREINFO_H
+#define ACPI_VMCOREINFO_H
+
+#include "hw/acpi/bios-linker-loader.h"
+#include "hw/qdev.h"
+
+#define VMCOREINFO_DEVICE   "vmcoreinfo"
+#define VMCOREINFO_FW_CFG_FILE  "etc/vmcoreinfo"
+#define VMCOREINFO_ADDR_FW_CFG_FILE "etc/vmcoreinfo-addr"
+
+/* Occupy a page of memory */
+#define VMCOREINFO_FW_CFG_SIZE  4096
+
+/* allow space for OVMF SDT Header Probe Supressor */
+#define VMCOREINFO_OFFSET   sizeof(AcpiTableHeader)
+
+#define VMCOREINFO(obj) OBJECT_CHECK(VMCoreInfoState, (obj), VMCOREINFO_DEVICE)
+
+typedef struct VMCoreInfoState {
+DeviceClass parent_obj;
+uint8_t vmcoreinfo_addr_le[8];   /* Address of memory region */
+bool write_pointer_available;
+} VMCoreInfoState;
+
+/* returns NULL unless there is exactly one device */
+static inline Object *find_vmcoreinfo_dev(void)
+{
+return object_resolve_path_type("", VMCOREINFO_DEVICE, NULL);
+}
+
+void vmcoreinfo_build_acpi(VMCoreInfoState *vis, GArray *table_data,
+   GArray *vmci, BIOSLinker *linker);
+void vmcoreinfo_add_fw_cfg(VMCoreInfoState *vis, FWCfgState *s, GArray *vmci);
+bool vmcoreinfo_get(VMCoreInfoState *vis, uint64_t *paddr, uint32_t *size,
+Error **errp);
+
+#endif
diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c
index 36a6cc450e..47043ade4a 100644
--- a/hw/acpi/aml-build.c
+++ b/hw/acpi/aml-build.c
@@ -1561,6 +1561,7 @@ void acpi_build_tables_init(AcpiBuildTables *tables)
 tables->table_data = g_array_new(false, true /* clear */, 1);
 tables->tcpalog = g_array_new(false, true /* clear */, 1);
 tables->vmgenid = g_array_new(false, true /* clear */, 1);
+tables->vmcoreinfo = g_array_new(false, true /* clear */, 1);
 tables->linker = bios_linker_loader_init();
 }
 
@@ -1571,6 +1572,7 @@ void acpi_build_tables_cleanup(AcpiBuildTables *tables, 
bool mfre)
 g_array_free(tables->table_data, true);
 g_array_free(tables->tcpalog, mfre);
 g_array_free(tables->vmgenid, mfre);
+g_array_free(tables->vmcoreinfo, mfre);
 }
 
 /* Build rsdt table */
diff --git a/hw/acpi/vmcoreinfo.c b/hw/acpi/vmcoreinfo.c
new file mode 100644
index 00..0ea41de8d9
--- /dev/null
+++ b/hw/acpi/vmcoreinfo.c
@@ -0,0 +1,208 @@
+/*
+ *  Virtual Machine coreinfo device
+ *  (based on Virtual Machine Generation ID Device)
+ *
+ *  Copyright (C) 2017 Red Hat, Inc.
+ *  Copyright (C) 2017 Skyport Systems.
+ *
+ *  Authors: Marc-André Lureau 
+ *   Ben Warren 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+#include "qemu/osdep.h"
+#include "hw/acpi/acpi.h"
+#include "hw/acpi/aml-build.h"
+#include "hw/acpi/vmcoreinfo.h"
+#include "hw/nvram/fw_cfg.h"
+#include "sysemu/sysemu.h"
+#include "qapi/error.h"
+
+void vmcoreinfo_build_acpi(VMCoreInfoState *vis, GArray *table_data,
+   GArray *vmci, BIOSLinker *linker)
+{
+Aml *ssdt, *dev, *scope, *method, *addr, *if_ctx;
+uint32_t vcia_offset;
+
+g_array_set_size(vmci, VMCOREINFO_FW_CFG_SIZE);
+
+/* Put this in a separate SSDT table */
+ssdt = init_aml_allocator();
+
+/* Reserve space for header */
+acpi_data_push

[Qemu-devel] [PATCH v4 3/8] stubs: add vmcoreinfo_get() stub

Common dump code may call vmcoreinfo_get() which is not available on
all targets.

Signed-off-by: Marc-André Lureau 
---
 stubs/vmcoreinfo.c  | 9 +
 stubs/Makefile.objs | 1 +
 2 files changed, 10 insertions(+)
 create mode 100644 stubs/vmcoreinfo.c

diff --git a/stubs/vmcoreinfo.c b/stubs/vmcoreinfo.c
new file mode 100644
index 00..a994153832
--- /dev/null
+++ b/stubs/vmcoreinfo.c
@@ -0,0 +1,9 @@
+#include "qemu/osdep.h"
+
+#include "hw/acpi/vmcoreinfo.h"
+
+bool vmcoreinfo_get(VMCoreInfoState *vis, uint64_t *paddr, uint32_t *size,
+Error **errp)
+{
+return false;
+}
diff --git a/stubs/Makefile.objs b/stubs/Makefile.objs
index f5b47bfd74..3bedc73252 100644
--- a/stubs/Makefile.objs
+++ b/stubs/Makefile.objs
@@ -36,6 +36,7 @@ stub-obj-y += qmp_pc_dimm_device_list.o
 stub-obj-y += target-monitor-defs.o
 stub-obj-y += target-get-monitor-def.o
 stub-obj-y += pc_madt_cpu_entry.o
+stub-obj-y += vmcoreinfo.o
 stub-obj-y += vmgenid.o
 stub-obj-y += xen-common.o
 stub-obj-y += xen-hvm.o
-- 
2.13.1.395.gf7b71de06

[Qemu-devel] [PATCH v4 6/8] kdump: add vmcoreinfo ELF note

kdump header provides offset and size of the vmcoreinfo ELF note,
append it if available.

crash-7.1.9 was the first version that started looking in the
vmcoreinfo data for phys_base instead of in the kdump_sub_header.

Signed-off-by: Marc-André Lureau 
Reviewed-by: Laszlo Ersek 
---
 dump.c | 20 
 1 file changed, 20 insertions(+)

diff --git a/dump.c b/dump.c
index d86b601248..b78d58ab0a 100644
--- a/dump.c
+++ b/dump.c
@@ -841,6 +841,16 @@ static void create_header32(DumpState *s, Error **errp)
 kh->dump_level = cpu_to_dump32(s, DUMP_LEVEL);
 
 offset_note = DISKDUMP_HEADER_BLOCKS * block_size + size;
+if (s->vmcoreinfo) {
+uint64_t hsize, name_size, size_vmcoreinfo_desc, offset_vmcoreinfo;
+
+get_note_sizes(s, s->vmcoreinfo, &hsize, &name_size, 
&size_vmcoreinfo_desc);
+offset_vmcoreinfo = offset_note + s->note_size - s->vmcoreinfo_size +
+(DIV_ROUND_UP(hsize, 4) + DIV_ROUND_UP(name_size, 4)) * 4;
+kh->offset_vmcoreinfo = cpu_to_dump64(s, offset_vmcoreinfo);
+kh->size_vmcoreinfo = cpu_to_dump32(s, size_vmcoreinfo_desc);
+}
+
 kh->offset_note = cpu_to_dump64(s, offset_note);
 kh->note_size = cpu_to_dump32(s, s->note_size);
 
@@ -941,6 +951,16 @@ static void create_header64(DumpState *s, Error **errp)
 kh->dump_level = cpu_to_dump32(s, DUMP_LEVEL);
 
 offset_note = DISKDUMP_HEADER_BLOCKS * block_size + size;
+if (s->vmcoreinfo) {
+uint64_t hsize, name_size, size_vmcoreinfo_desc, offset_vmcoreinfo;
+
+get_note_sizes(s, s->vmcoreinfo, &hsize, &name_size, 
&size_vmcoreinfo_desc);
+offset_vmcoreinfo = offset_note + s->note_size - s->vmcoreinfo_size +
+(DIV_ROUND_UP(hsize, 4) + DIV_ROUND_UP(name_size, 4)) * 4;
+kh->offset_vmcoreinfo = cpu_to_dump64(s, offset_vmcoreinfo);
+kh->size_vmcoreinfo = cpu_to_dump64(s, size_vmcoreinfo_desc);
+}
+
 kh->offset_note = cpu_to_dump64(s, offset_note);
 kh->note_size = cpu_to_dump64(s, s->note_size);
 
-- 
2.13.1.395.gf7b71de06

[Qemu-devel] [PATCH v4 8/8] MAINTAINERS: add Dump maintainers

Proposing myself, since I have some familiarity with the code now.

Signed-off-by: Marc-André Lureau 
Acked-by: Laszlo Ersek 
---
 MAINTAINERS | 9 +
 1 file changed, 9 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index b9aa878b84..391a28315c 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1274,6 +1274,15 @@ S: Maintained
 F: device_tree.c
 F: include/sysemu/device_tree.h
 
+Dump
+S: Supported
+M: Marc-André Lureau 
+F: dump.c
+F: stubs/dump.c
+F: include/sysemu/dump.h
+F: include/sysemu/dump-arch.h
+F: scripts/dump-guest-memory.py
+
 Error reporting
 M: Markus Armbruster 
 S: Supported
-- 
2.13.1.395.gf7b71de06

[Qemu-devel] [PATCH v4 1/8] vmgenid: replace x-write-pointer-available hack

This compat property sole function is to prevent the device from being
instantiated. Instead of requiring an extra compat property, check if
fw_cfg has DMA enabled.

fw_cfg is a built-in device that is initialized very early by the
machine init code.  We have at least one other device that also
assumes fw_cfg_find() can be safely used on realize: pvpanic.

This has the additional benefit of handling other cases properly, like:

  $ qemu-system-x86_64 -device vmgenid -machine none
  qemu-system-x86_64: -device vmgenid: vmgenid requires DMA write support in 
fw_cfg, which this machine type does not provide
  $ qemu-system-x86_64 -device vmgenid -machine pc-i440fx-2.9 -global 
fw_cfg.dma_enabled=off
  qemu-system-x86_64: -device vmgenid: vmgenid requires DMA write support in 
fw_cfg, which this machine type does not provide
  $ qemu-system-x86_64 -device vmgenid -machine pc-i440fx-2.6 -global 
fw_cfg.dma_enabled=on
  [boots normally]

Suggested-by: Eduardo Habkost 
Signed-off-by: Marc-André Lureau 
Reviewed-by: Michael S. Tsirkin 
Reviewed-by: Eduardo Habkost 
Reviewed-by: Ben Warren 
Reviewed-by: Laszlo Ersek 
---
 include/hw/acpi/bios-linker-loader.h |  2 ++
 include/hw/compat.h  |  4 
 hw/acpi/bios-linker-loader.c | 10 ++
 hw/acpi/vmgenid.c|  9 +
 4 files changed, 13 insertions(+), 12 deletions(-)

diff --git a/include/hw/acpi/bios-linker-loader.h 
b/include/hw/acpi/bios-linker-loader.h
index efe17b0b9c..a711dbced8 100644
--- a/include/hw/acpi/bios-linker-loader.h
+++ b/include/hw/acpi/bios-linker-loader.h
@@ -7,6 +7,8 @@ typedef struct BIOSLinker {
 GArray *file_list;
 } BIOSLinker;
 
+bool bios_linker_loader_can_write_pointer(void);
+
 BIOSLinker *bios_linker_loader_init(void);
 
 void bios_linker_loader_alloc(BIOSLinker *linker,
diff --git a/include/hw/compat.h b/include/hw/compat.h
index 08f36004da..f414786604 100644
--- a/include/hw/compat.h
+++ b/include/hw/compat.h
@@ -150,10 +150,6 @@
 .driver   = "fw_cfg_io",\
 .property = "dma_enabled",\
 .value= "off",\
-},{\
-.driver   = "vmgenid",\
-.property = "x-write-pointer-available",\
-.value= "off",\
 },
 
 #define HW_COMPAT_2_3 \
diff --git a/hw/acpi/bios-linker-loader.c b/hw/acpi/bios-linker-loader.c
index 046183a0f1..d16b8bbcb1 100644
--- a/hw/acpi/bios-linker-loader.c
+++ b/hw/acpi/bios-linker-loader.c
@@ -168,6 +168,16 @@ bios_linker_find_file(const BIOSLinker *linker, const char 
*name)
 return NULL;
 }
 
+/*
+ * board code must realize fw_cfg first, as a fixed device, before
+ * another device realize function call bios_linker_loader_can_write_pointer()
+ */
+bool bios_linker_loader_can_write_pointer(void)
+{
+FWCfgState *fw_cfg = fw_cfg_find();
+return fw_cfg && fw_cfg_dma_enabled(fw_cfg);
+}
+
 /*
  * bios_linker_loader_alloc: ask guest to load file into guest memory.
  *
diff --git a/hw/acpi/vmgenid.c b/hw/acpi/vmgenid.c
index a32b847fe0..ab5da293fd 100644
--- a/hw/acpi/vmgenid.c
+++ b/hw/acpi/vmgenid.c
@@ -205,17 +205,11 @@ static void vmgenid_handle_reset(void *opaque)
 memset(vms->vmgenid_addr_le, 0, ARRAY_SIZE(vms->vmgenid_addr_le));
 }
 
-static Property vmgenid_properties[] = {
-DEFINE_PROP_BOOL("x-write-pointer-available", VmGenIdState,
- write_pointer_available, true),
-DEFINE_PROP_END_OF_LIST(),
-};
-
 static void vmgenid_realize(DeviceState *dev, Error **errp)
 {
 VmGenIdState *vms = VMGENID(dev);
 
-if (!vms->write_pointer_available) {
+if (!bios_linker_loader_can_write_pointer()) {
 error_setg(errp, "%s requires DMA write support in fw_cfg, "
"which this machine type does not provide", VMGENID_DEVICE);
 return;
@@ -239,7 +233,6 @@ static void vmgenid_device_class_init(ObjectClass *klass, 
void *data)
 dc->vmsd = &vmstate_vmgenid;
 dc->realize = vmgenid_realize;
 dc->hotpluggable = false;
-dc->props = vmgenid_properties;
 
 object_class_property_add_str(klass, VMGENID_GUID, NULL,
   vmgenid_set_guid, NULL);
-- 
2.13.1.395.gf7b71de06

[Qemu-devel] [PATCH v4 7/8] scripts/dump-guest-memory.py: add vmcoreinfo

Add vmcoreinfo ELF note if vmcoreinfo device is ready.

To help the python script, add a static vmcoreinfo_gdb_helper value to
be looked up to get vmcoreinfo device singleton.

Signed-off-by: Marc-André Lureau 
Acked-by: Laszlo Ersek 
---
 scripts/dump-guest-memory.py | 47 
 hw/acpi/vmcoreinfo.c |  3 +++
 2 files changed, 50 insertions(+)

diff --git a/scripts/dump-guest-memory.py b/scripts/dump-guest-memory.py
index f7c6635f15..e0589e5b7c 100644
--- a/scripts/dump-guest-memory.py
+++ b/scripts/dump-guest-memory.py
@@ -14,6 +14,7 @@ the COPYING file in the top-level directory.
 """
 
 import ctypes
+import struct
 
 UINTPTR_T = gdb.lookup_type("uintptr_t")
 
@@ -120,6 +121,22 @@ class ELF(object):
 self.segments[0].p_filesz += ctypes.sizeof(note)
 self.segments[0].p_memsz += ctypes.sizeof(note)
 
+
+def add_vmcoreinfo_note(self, vmcoreinfo):
+"""Adds a vmcoreinfo note to the ELF dump."""
+# compute the header size, and copy that many bytes from the note
+header = get_arch_note(self.endianness, 0, 0)
+ctypes.memmove(ctypes.pointer(header),
+   vmcoreinfo, ctypes.sizeof(header))
+# now get the full note
+note = get_arch_note(self.endianness,
+ header.n_namesz - 1, header.n_descsz)
+ctypes.memmove(ctypes.pointer(note), vmcoreinfo, ctypes.sizeof(note))
+
+self.notes.append(note)
+self.segments[0].p_filesz += ctypes.sizeof(note)
+self.segments[0].p_memsz += ctypes.sizeof(note)
+
 def add_segment(self, p_type, p_paddr, p_size):
 """Adds a segment to the elf."""
 
@@ -505,6 +522,35 @@ shape and this command should mostly work."""
 cur += chunk_size
 left -= chunk_size
 
+def phys_memory_read(self, addr, size):
+qemu_core = gdb.inferiors()[0]
+for block in self.guest_phys_blocks:
+if block["target_start"] <= addr \
+   and addr + size <= block["target_end"]:
+haddr = block["host_addr"] + (addr - block["target_start"])
+return qemu_core.read_memory(haddr, size)
+return None
+
+def add_vmcoreinfo(self):
+if not gdb.parse_and_eval("vmcoreinfo_realize::vmcoreinfo_gdb_helper"):
+return
+
+addr = gdb.parse_and_eval("vmcoreinfo_realize::vmcoreinfo_gdb_helper->"
+  "vmcoreinfo_addr_le")
+addr = bytes([addr[i] for i in range(4)])
+addr = struct.unpack("

[Qemu-devel] [PATCH v4 5/8] dump: add vmcoreinfo ELF note

Read the vmcoreinfo ELF PT_NOTE from guest memory when vmcoreinfo
device provides the location, and write it as an ELF note in the dump.

There are now 2 possible sources of phys_base information.

(1) arch guessed value from cpu_get_dump_info()
(2) vmcoreinfo ELF note NUMBER(phys_base)= field

NUMBER(phys_base) in vmcoreinfo has only been recently introduced
in Linux 4.10 (401721ecd1dc "kexec: export the value of phys_base
instead of symbol address").

Since (2) has better chances to be accurate, the guessed value is
replaced by the value from the vmcoreinfo ELF note.

The phys_base value is stored in the same dump field locations as
before, and may duplicate the information available in the vmcoreinfo
ELF PT_NOTE. Crash tools should be prepared to handle this case.

Signed-off-by: Marc-André Lureau 
Reviewed-by: Laszlo Ersek 
---
 include/sysemu/dump.h |   2 +
 dump.c| 134 ++
 2 files changed, 136 insertions(+)

diff --git a/include/sysemu/dump.h b/include/sysemu/dump.h
index 2672a15f8b..111a7dcaa4 100644
--- a/include/sysemu/dump.h
+++ b/include/sysemu/dump.h
@@ -192,6 +192,8 @@ typedef struct DumpState {
   * this could be used to calculate
   * how much work we have
   * finished. */
+uint8_t *vmcoreinfo; /* ELF note content */
+size_t vmcoreinfo_size;
 } DumpState;
 
 uint16_t cpu_to_dump16(DumpState *s, uint16_t val);
diff --git a/dump.c b/dump.c
index d9090a24cc..d86b601248 100644
--- a/dump.c
+++ b/dump.c
@@ -26,6 +26,8 @@
 #include "qapi/qmp/qerror.h"
 #include "qmp-commands.h"
 #include "qapi-event.h"
+#include "qemu/error-report.h"
+#include "hw/acpi/vmcoreinfo.h"
 
 #include 
 #ifdef CONFIG_LZO
@@ -38,6 +40,13 @@
 #define ELF_MACHINE_UNAME "Unknown"
 #endif
 
+#define MAX_VMCOREINFO_SIZE (1 << 20) /* 1MB should be enough */
+
+#define ELF_NOTE_SIZE(hdr_size, name_size, desc_size)   \
+((DIV_ROUND_UP((hdr_size), 4) + \
+  DIV_ROUND_UP((name_size), 4) +\
+  DIV_ROUND_UP((desc_size), 4)) * 4)
+
 uint16_t cpu_to_dump16(DumpState *s, uint16_t val)
 {
 if (s->dump_info.d_endian == ELFDATA2LSB) {
@@ -76,6 +85,8 @@ static int dump_cleanup(DumpState *s)
 guest_phys_blocks_free(&s->guest_phys_blocks);
 memory_mapping_list_free(&s->list);
 close(s->fd);
+g_free(s->vmcoreinfo);
+s->vmcoreinfo = NULL;
 if (s->resume) {
 if (s->detached) {
 qemu_mutex_lock_iothread();
@@ -235,6 +246,19 @@ static inline int cpu_index(CPUState *cpu)
 return cpu->cpu_index + 1;
 }
 
+static void write_vmcoreinfo_note(WriteCoreDumpFunction f, DumpState *s,
+  Error **errp)
+{
+int ret;
+
+if (s->vmcoreinfo) {
+ret = f(s->vmcoreinfo, s->vmcoreinfo_size, s);
+if (ret < 0) {
+error_setg(errp, "dump: failed to write vmcoreinfo");
+}
+}
+}
+
 static void write_elf64_notes(WriteCoreDumpFunction f, DumpState *s,
   Error **errp)
 {
@@ -258,6 +282,8 @@ static void write_elf64_notes(WriteCoreDumpFunction f, 
DumpState *s,
 return;
 }
 }
+
+write_vmcoreinfo_note(f, s, errp);
 }
 
 static void write_elf32_note(DumpState *s, Error **errp)
@@ -303,6 +329,8 @@ static void write_elf32_notes(WriteCoreDumpFunction f, 
DumpState *s,
 return;
 }
 }
+
+write_vmcoreinfo_note(f, s, errp);
 }
 
 static void write_elf_section(DumpState *s, int type, Error **errp)
@@ -714,6 +742,44 @@ static int buf_write_note(const void *buf, size_t size, 
void *opaque)
 return 0;
 }
 
+/*
+ * This function retrieves various sizes from an elf header.
+ *
+ * @note has to be a valid ELF note. The return sizes are unmodified
+ * (not padded or rounded up to be multiple of 4).
+ */
+static void get_note_sizes(DumpState *s, const void *note,
+   uint64_t *note_head_size,
+   uint64_t *name_size,
+   uint64_t *desc_size)
+{
+uint64_t note_head_sz;
+uint64_t name_sz;
+uint64_t desc_sz;
+
+if (s->dump_info.d_class == ELFCLASS64) {
+const Elf64_Nhdr *hdr = note;
+note_head_sz = sizeof(Elf64_Nhdr);
+name_sz = tswap64(hdr->n_namesz);
+desc_sz = tswap64(hdr->n_descsz);
+} else {
+const Elf32_Nhdr *hdr = note;
+note_head_sz = sizeof(Elf32_Nhdr);
+name_sz = tswap32(hdr->n_namesz);
+desc_sz = tswap32(hdr->n_descsz);
+}
+
+if (note_head_size) {
+*note_head_size = note_head_sz;
+}
+if (name_size) {
+*name_size = name_sz;
+}
+if (desc_size) {
+*desc_size = desc_sz;
+}
+}
+
 /* write common header, sub header and elf note to vmcore */
 static void create_header32(DumpState *s, Error **errp)
 {
@@ -1488,10 +1554,40 @@ static int64_

[Qemu-devel] [PATCH v6 17/18] tests: crypto: add hash speed benchmark support

From: "Longpeng(Mike)" 

This patch add a hash speed benchmark, it helps us to
measure the performance by using "make check-speed" or
using "./tests/benchmark-crypto-hash" directly.

Signed-off-by: Longpeng(Mike) 
---
 tests/Makefile.include|  2 ++
 tests/benchmark-crypto-hash.c | 67 +++
 2 files changed, 69 insertions(+)
 create mode 100644 tests/benchmark-crypto-hash.c

diff --git a/tests/Makefile.include b/tests/Makefile.include
index 2f34c60..c5a67a3 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -97,6 +97,7 @@ gcov-files-test-keyval-y = util/keyval.c
 check-unit-y += tests/test-write-threshold$(EXESUF)
 gcov-files-test-write-threshold-y = block/write-threshold.c
 check-unit-y += tests/test-crypto-hash$(EXESUF)
+check-speed-y += tests/benchmark-crypto-hash$(EXESUF)
 check-unit-y += tests/test-crypto-hmac$(EXESUF)
 check-unit-y += tests/test-crypto-cipher$(EXESUF)
 check-speed-y += tests/benchmark-crypto-cipher$(EXESUF)
@@ -635,6 +636,7 @@ tests/test-mul64$(EXESUF): tests/test-mul64.o 
$(test-util-obj-y)
 tests/test-bitops$(EXESUF): tests/test-bitops.o $(test-util-obj-y)
 tests/test-bitcnt$(EXESUF): tests/test-bitcnt.o $(test-util-obj-y)
 tests/test-crypto-hash$(EXESUF): tests/test-crypto-hash.o $(test-crypto-obj-y)
+tests/benchmark-crypto-hash$(EXESUF): tests/benchmark-crypto-hash.o 
$(test-crypto-obj-y)
 tests/test-crypto-hmac$(EXESUF): tests/test-crypto-hmac.o $(test-crypto-obj-y)
 tests/test-crypto-cipher$(EXESUF): tests/test-crypto-cipher.o 
$(test-crypto-obj-y)
 tests/benchmark-crypto-cipher$(EXESUF): tests/benchmark-crypto-cipher.o 
$(test-crypto-obj-y)
diff --git a/tests/benchmark-crypto-hash.c b/tests/benchmark-crypto-hash.c
new file mode 100644
index 000..6769d2a
--- /dev/null
+++ b/tests/benchmark-crypto-hash.c
@@ -0,0 +1,67 @@
+/*
+ * QEMU Crypto hash speed benchmark
+ *
+ * Copyright (c) 2017 HUAWEI TECHNOLOGIES CO., LTD.
+ *
+ * Authors:
+ *Longpeng(Mike) 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or
+ * (at your option) any later version.  See the COPYING file in the
+ * top-level directory.
+ */
+#include "qemu/osdep.h"
+#include "crypto/init.h"
+#include "crypto/hash.h"
+
+static void test_hash_speed(const void *opaque)
+{
+size_t chunk_size = (size_t)opaque;
+uint8_t *in = NULL, *out = NULL;
+size_t out_len = 0;
+double total = 0.0;
+struct iovec iov;
+int ret;
+
+in = g_new0(uint8_t, chunk_size);
+memset(in, g_test_rand_int(), chunk_size);
+
+iov.iov_base = (char *)in;
+iov.iov_len = chunk_size;
+
+g_test_timer_start();
+do {
+ret = qcrypto_hash_bytesv(QCRYPTO_HASH_ALG_SHA256,
+  &iov, 1, &out, &out_len,
+  NULL);
+g_assert(ret == 0);
+
+total += chunk_size;
+} while (g_test_timer_elapsed() < 5.0);
+
+total /= 1024 * 1024; /* to MB */
+g_print("sha256: ");
+g_print("Testing chunk_size %ld bytes ", chunk_size);
+g_print("done: %.2f MB in %.2f secs: ", total, g_test_timer_last());
+g_print("%.2f MB/sec\n", total / g_test_timer_last());
+
+g_free(out);
+g_free(in);
+}
+
+int main(int argc, char **argv)
+{
+size_t i;
+char name[64];
+
+g_test_init(&argc, &argv, NULL);
+g_assert(qcrypto_init(NULL) == 0);
+
+for (i = 512; i <= (64 * 1204); i *= 2) {
+memset(name, 0 , sizeof(name));
+snprintf(name, sizeof(name), "/crypto/hash/speed-%lu", i);
+g_test_add_data_func(name, (void *)i, test_hash_speed);
+}
+
+return g_test_run();
+}
-- 
1.8.3.1

[Qemu-devel] [PATCH v4 4/8] tests: add simple vmcoreinfo test

This test is based off vmgenid test from Ben Warren
. It simply checks the vmcoreinfo ACPI device
is present and that the memory region associated can be read.

Signed-off-by: Marc-André Lureau 
---
 tests/vmcoreinfo-test.c | 141 
 tests/Makefile.include  |   2 +
 2 files changed, 143 insertions(+)
 create mode 100644 tests/vmcoreinfo-test.c

diff --git a/tests/vmcoreinfo-test.c b/tests/vmcoreinfo-test.c
new file mode 100644
index 00..a54f3b5c6a
--- /dev/null
+++ b/tests/vmcoreinfo-test.c
@@ -0,0 +1,141 @@
+/*
+ * QTest testcase for VM coreinfo device
+ *
+ * Copyright (c) 2017 Red Hat, Inc.
+ * Copyright (c) 2017 Skyport Systems
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include 
+#include 
+#include 
+#include "qemu/osdep.h"
+#include "qemu/bitmap.h"
+#include "qemu/uuid.h"
+#include "hw/acpi/acpi-defs.h"
+#include "boot-sector.h"
+#include "acpi-utils.h"
+#include "libqtest.h"
+
+#define RSDP_ADDR_INVALID 0x10 /* RSDP must be below this address */
+
+typedef struct {
+AcpiTableHeader header;
+gchar name_op;
+gchar vcia[4];
+gchar val_op;
+uint32_t vcia_val;
+} QEMU_PACKED VmciTable;
+
+static uint32_t acpi_find_vcia(void)
+{
+uint32_t rsdp_offset;
+AcpiRsdpDescriptor rsdp_table;
+uint32_t rsdt;
+AcpiRsdtDescriptorRev1 rsdt_table;
+int tables_nr;
+uint32_t *tables;
+AcpiTableHeader ssdt_table;
+VmciTable vmci_table;
+int i;
+
+/* Wait for guest firmware to finish and start the payload. */
+boot_sector_test();
+
+/* Tables should be initialized now. */
+rsdp_offset = acpi_find_rsdp_address();
+
+g_assert_cmphex(rsdp_offset, <, RSDP_ADDR_INVALID);
+
+acpi_parse_rsdp_table(rsdp_offset, &rsdp_table);
+
+rsdt = rsdp_table.rsdt_physical_address;
+/* read the header */
+ACPI_READ_TABLE_HEADER(&rsdt_table, rsdt);
+ACPI_ASSERT_CMP(rsdt_table.signature, "RSDT");
+
+/* compute the table entries in rsdt */
+tables_nr = (rsdt_table.length - sizeof(AcpiRsdtDescriptorRev1)) /
+sizeof(uint32_t);
+g_assert_cmpint(tables_nr, >, 0);
+
+/* get the addresses of the tables pointed by rsdt */
+tables = g_new0(uint32_t, tables_nr);
+ACPI_READ_ARRAY_PTR(tables, tables_nr, rsdt);
+
+for (i = 0; i < tables_nr; i++) {
+ACPI_READ_TABLE_HEADER(&ssdt_table, tables[i]);
+if (!strncmp((char *)ssdt_table.oem_table_id, "VMCOREIN", 8)) {
+/* the first entry in the table should be VCIA
+ * That's all we need
+ */
+ACPI_READ_FIELD(vmci_table.name_op, tables[i]);
+g_assert(vmci_table.name_op == 0x08);  /* name */
+ACPI_READ_ARRAY(vmci_table.vcia, tables[i]);
+g_assert(memcmp(vmci_table.vcia, "VCIA", 4) == 0);
+ACPI_READ_FIELD(vmci_table.val_op, tables[i]);
+g_assert(vmci_table.val_op == 0x0C);  /* dword */
+ACPI_READ_FIELD(vmci_table.vcia_val, tables[i]);
+/* The GUID is written at a fixed offset into the fw_cfg file
+ * in order to implement the "OVMF SDT Header probe suppressor"
+ * see docs/specs/vmgenid.txt for more details
+ */
+g_free(tables);
+return vmci_table.vcia_val;
+}
+}
+g_free(tables);
+return 0;
+}
+
+static char disk[] = "tests/vmci-test-disk-XX";
+
+static char *cmd_strdup(void)
+{
+return g_strdup_printf("-machine accel=tcg "
+   "-device vmcoreinfo,id=testvmvci "
+   "-drive id=hd0,if=none,file=%s,format=raw "
+   "-device ide-hd,drive=hd0 ",
+   disk);
+}
+
+static void vmcoreinfo_test(void)
+{
+gchar *cmd;
+uint32_t vmci_addr;
+int i;
+
+cmd = cmd_strdup();
+qtest_start(cmd);
+
+vmci_addr = acpi_find_vcia();
+g_assert(vmci_addr);
+
+for (i = 0; i < 4096; i++) {
+/* check the memory region can be read */
+readb(vmci_addr + i);
+}
+
+qtest_quit(global_qtest);
+g_free(cmd);
+}
+
+int main(int argc, char **argv)
+{
+int ret;
+
+ret = boot_sector_init(disk);
+if (ret) {
+return ret;
+}
+
+g_test_init(&argc, &argv, NULL);
+
+qtest_add_func("/vmcoreinfo/test", vmcoreinfo_test);
+ret = g_test_run();
+boot_sector_cleanup(disk);
+
+return ret;
+}
diff --git a/tests/Makefile.include b/tests/Makefile.include
index cfbb689e0e..dd515f2159 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -251,6 +251,7 @@ gcov-files-i386-y += hw/usb/hcd-xhci.c
 check-qtest-i386-y += tests/pc-cpu-test$(EXESUF)
 check-qtest-i386-y += tests/q35-test$(EXESUF)
 check-qtest-i386-y += tests/vmgenid-test$(EXESUF)
+check-qtest-i386-y += tests/vmcoreinfo-test$(EXESUF)
 gcov-files-i386-y += hw/pci-host/q35.c
 check-qtest-i

[Qemu-devel] [PATCH v4 0/8] KASLR kernel dump support

Recent linux kernels enable KASLR to randomize phys/virt memory
addresses. This series aims to provide enough information in qemu
dumps so that crash utility can work with randomized kernel too (it
hasn't been tested on other archs than x86 though, help welcome).

The vmcoreinfo device is an emulated ACPI device that exposes a 4k
memory range to the guest to store various informations useful to
debug the guest OS. (it is greatly inspired by the VMGENID device
implementation). The version field with value 0 is meant to give
paddr/size of the VMCOREINFO ELF PT_NOTE, other values can be used for
different purposes or OSes. (note: some wanted to see pvpanic somehow
merged with this device, I have no clear idea how to do that, nor do I
think this is a good idea since the devices are quite different, used
at different time for different purposes. And it can be done as a
future iteration if it is appropriate, feel free to send patches)

Crash 7.1.9 will parse the "phys_base" value from the VMCOREINFO note,
and thus will work with KASLR-dump produced by this series.

By priority, VMCOREINFO "phys_base" value is the most accurate. If not
available, qemu will keep the current guessed value.

The series implements the VMCOREINFO note addition in qemu ELF/kdump,
as well as the python scripts/dump-guest-memory.py.

To test:

Compile and run a guest kernel with CONFIG_RANDOMIZE_BASE=y.

Run qemu with -device vmcoreinfo.

Load the experimental vmcoreinfo module in guest
https://github.com/elmarco/vmgenid-test/blob/master/qemuvmci-test.c.

Produce an ELF dump:
{ "execute": "dump-guest-memory", "arguments": { "protocol": "file:dump", 
"paging": false } }

Produce a kdump:
{ "execute": "dump-guest-memory", "arguments": { "protocol": "file:dump", 
"paging": false, "format": "kdump-zlib" } }

Or with (gdb) dump-guest-memory, with scripts/dump-guest-memory.py script.

Analyze with crash >= 7.1.9
$ crash vmlinux dump

v4: from Laszlo review
- switch to warn_report*()
- update test to follow vmgenid and use boot-sector infrastructure
- fix range checks in the python script
- add vmcoreinfo_get() stub

v3: from Laszlo review
- change vmcoreinfo offset to 36
- reset err to null after report
- use PRIu32
- change name_size and desc_size against MAX_VMCOREINFO_SIZE
- python code simplification
- check boundaries of blocks in phys_memory_read()
- fix some vmgi vs vmci names
- add more comments in code
- fix comment indentation
- add r-b tags

v2: from Laszlo review
- vmci: fix guest endianess handling
- vmci: fix wrong sizeof()
- vmci: add back reset logic from vmgenid
- dump: have 1MB size limit for vmcoreinfo
- dump: fix potential off-by-1 buffer manipulation
- dump: use temporary variable for qemu_strtou64
- dump: fixed VMCOREINFO duplication in kdump
- update gdb script to not call into qemu process
- update MAINTAINERS with some new files

Marc-André Lureau (8):
  vmgenid: replace x-write-pointer-available hack
  acpi: add vmcoreinfo device
  stubs: add vmcoreinfo_get() stub
  tests: add simple vmcoreinfo test
  dump: add vmcoreinfo ELF note
  kdump: add vmcoreinfo ELF note
  scripts/dump-guest-memory.py: add vmcoreinfo
  MAINTAINERS: add Dump maintainers

 scripts/dump-guest-memory.py |  47 
 include/hw/acpi/aml-build.h  |   1 +
 include/hw/acpi/bios-linker-loader.h |   2 +
 include/hw/acpi/vmcoreinfo.h |  37 ++
 include/hw/compat.h  |   4 -
 include/sysemu/dump.h|   2 +
 dump.c   | 154 +
 hw/acpi/aml-build.c  |   2 +
 hw/acpi/bios-linker-loader.c |  10 ++
 hw/acpi/vmcoreinfo.c | 211 +++
 hw/acpi/vmgenid.c|   9 +-
 hw/i386/acpi-build.c |  14 +++
 stubs/vmcoreinfo.c   |   9 ++
 tests/vmcoreinfo-test.c  | 141 +++
 MAINTAINERS  |   9 ++
 default-configs/arm-softmmu.mak  |   1 +
 default-configs/i386-softmmu.mak |   1 +
 default-configs/x86_64-softmmu.mak   |   1 +
 docs/specs/vmcoreinfo.txt| 138 +++
 hw/acpi/Makefile.objs|   1 +
 stubs/Makefile.objs  |   1 +
 tests/Makefile.include   |   2 +
 22 files changed, 785 insertions(+), 12 deletions(-)
 create mode 100644 include/hw/acpi/vmcoreinfo.h
 create mode 100644 hw/acpi/vmcoreinfo.c
 create mode 100644 stubs/vmcoreinfo.c
 create mode 100644 tests/vmcoreinfo-test.c
 create mode 100644 docs/specs/vmcoreinfo.txt

-- 
2.13.1.395.gf7b71de06

[Qemu-devel] [PATCH v6 16/18] tests: crypto: add cipher speed benchmark support

From: "Longpeng(Mike)" 

Now we have two qcrypto backends, libiary-backend and afalg-backend,
but which one is faster? This patch add a cipher speed benchmark, it
helps us to measure the performance by using "make check-speed" or
using "./tests/benchmark-crypto-cipher" directly.

Signed-off-by: Longpeng(Mike) 
---
 tests/Makefile.include  |  9 -
 tests/benchmark-crypto-cipher.c | 88 +
 2 files changed, 95 insertions(+), 2 deletions(-)
 create mode 100644 tests/benchmark-crypto-cipher.c

diff --git a/tests/Makefile.include b/tests/Makefile.include
index 42e17e2..2f34c60 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -99,6 +99,7 @@ gcov-files-test-write-threshold-y = block/write-threshold.c
 check-unit-y += tests/test-crypto-hash$(EXESUF)
 check-unit-y += tests/test-crypto-hmac$(EXESUF)
 check-unit-y += tests/test-crypto-cipher$(EXESUF)
+check-speed-y += tests/benchmark-crypto-cipher$(EXESUF)
 check-unit-y += tests/test-crypto-secret$(EXESUF)
 check-unit-$(CONFIG_GNUTLS) += tests/test-crypto-tlscredsx509$(EXESUF)
 check-unit-$(CONFIG_GNUTLS) += tests/test-crypto-tlssession$(EXESUF)
@@ -532,6 +533,7 @@ test-qom-obj-y = $(qom-obj-y) $(test-util-obj-y)
 test-qapi-obj-y = tests/test-qapi-visit.o tests/test-qapi-types.o \
tests/test-qapi-event.o tests/test-qmp-introspect.o \
$(test-qom-obj-y)
+benchmark-crypto-obj-y = $(crypto-obj-y) $(test-qom-obj-y)
 test-crypto-obj-y = $(crypto-obj-y) $(test-qom-obj-y)
 test-io-obj-y = $(io-obj-y) $(test-crypto-obj-y)
 test-block-obj-y = $(block-obj-y) $(test-io-obj-y) tests/iothread.o
@@ -635,6 +637,7 @@ tests/test-bitcnt$(EXESUF): tests/test-bitcnt.o 
$(test-util-obj-y)
 tests/test-crypto-hash$(EXESUF): tests/test-crypto-hash.o $(test-crypto-obj-y)
 tests/test-crypto-hmac$(EXESUF): tests/test-crypto-hmac.o $(test-crypto-obj-y)
 tests/test-crypto-cipher$(EXESUF): tests/test-crypto-cipher.o 
$(test-crypto-obj-y)
+tests/benchmark-crypto-cipher$(EXESUF): tests/benchmark-crypto-cipher.o 
$(test-crypto-obj-y)
 tests/test-crypto-secret$(EXESUF): tests/test-crypto-secret.o 
$(test-crypto-obj-y)
 tests/test-crypto-xts$(EXESUF): tests/test-crypto-xts.o $(test-crypto-obj-y)
 
@@ -803,6 +806,7 @@ check-help:
@echo " make check-qtest-TARGET   Run qtest tests for given target"
@echo " make check-qtest  Run qtest tests"
@echo " make check-unit   Run qobject tests"
+   @echo " make check-speed  Run qobject speed tests"
@echo " make check-qapi-schemaRun QAPI schema tests"
@echo " make check-block  Run block tests"
@echo " make check-report.htmlGenerates an HTML test report"
@@ -833,8 +837,8 @@ $(patsubst %, check-qtest-%, $(QTEST_TARGETS)): 
check-qtest-%: $(check-qtest-y)
  $(GCOV) $(GCOV_OPTIONS) $$f -o `dirname $$f`; \
done,)
 
-.PHONY: $(patsubst %, check-%, $(check-unit-y))
-$(patsubst %, check-%, $(check-unit-y)): check-%: %
+.PHONY: $(patsubst %, check-%, $(check-unit-y) $(check-speed-y))
+$(patsubst %, check-%, $(check-unit-y) $(check-speed-y)): check-%: %
$(if $(CONFIG_GCOV),@rm -f *.gcda */*.gcda */*/*.gcda */*/*/*.gcda,)
$(call quiet-command, \
MALLOC_PERTURB_=$${MALLOC_PERTURB_:-$$((RANDOM % 255 + 1))} \
@@ -893,6 +897,7 @@ check-tests/qapi-schema/doc-good.texi: 
tests/qapi-schema/doc-good.test.texi
 check-qapi-schema: $(patsubst %,check-%, $(check-qapi-schema-y)) 
check-tests/qapi-schema/doc-good.texi
 check-qtest: $(patsubst %,check-qtest-%, $(QTEST_TARGETS))
 check-unit: $(patsubst %,check-%, $(check-unit-y))
+check-speed: $(patsubst %,check-%, $(check-speed-y))
 check-block: $(patsubst %,check-%, $(check-block-y))
 check: check-qapi-schema check-unit check-qtest
 check-clean:
diff --git a/tests/benchmark-crypto-cipher.c b/tests/benchmark-crypto-cipher.c
new file mode 100644
index 000..c6a4092
--- /dev/null
+++ b/tests/benchmark-crypto-cipher.c
@@ -0,0 +1,88 @@
+/*
+ * QEMU Crypto cipher speed benchmark
+ *
+ * Copyright (c) 2017 HUAWEI TECHNOLOGIES CO., LTD.
+ *
+ * Authors:
+ *Longpeng(Mike) 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or
+ * (at your option) any later version.  See the COPYING file in the
+ * top-level directory.
+ */
+#include "qemu/osdep.h"
+#include "crypto/init.h"
+#include "crypto/cipher.h"
+
+static void test_cipher_speed(const void *opaque)
+{
+QCryptoCipher *cipher;
+Error *err = NULL;
+double total = 0.0;
+size_t chunk_size = (size_t)opaque;
+uint8_t *key = NULL, *iv = NULL;
+uint8_t *plaintext = NULL, *ciphertext = NULL;
+size_t nkey = qcrypto_cipher_get_key_len(QCRYPTO_CIPHER_ALG_AES_128);
+size_t niv = qcrypto_cipher_get_iv_len(QCRYPTO_CIPHER_ALG_AES_128,
+   QCRYPTO_CIPHER_MODE_CBC);
+
+key = g_new0(uint8_t, nkey);
+memset(key, g_test_rand_int(), nkey);
+
+iv = g_new0(uint8_t, niv);
+memset(iv, g_t

Re: [Qemu-devel] [RFC PATCH 7/8] VFIO: Add new IOCTL for IOMMU TLB invalidate propagation

2017-07-14 Thread Alex Williamson

On Fri, 14 Jul 2017 08:58:02 +
"Liu, Yi L"  wrote:

> Hi Alex,
> 
> Against to the opaque open, I'd like to propose the following definition
> based on the existing comments. Pls note that I've merged the pasid
> table binding and iommu tlb invalidation into a single IOCTL and make
> different flags to indicate the iommu operations. Per Kevin's comments,
> there may be iommu invalidation for guest IOVA tlb, so I renamed the
> IOCTL and data structure to be non-svm specific. Pls kindly have a review,
> so that we can make the opaque open closed and move forward. Surely,
> comments and ideas are welcomed. And for the scope and flags definition
> in struct iommu_tlb_invalidate, it's also welcomed to give your ideas on it.
> 
> 1. Add a VFIO IOCTL for iommu operations from user-space
> 
> #define VFIO_IOMMU_OP_IOCTL _IO(VFIO_TYPE, VFIO_BASE + 24)
> 
> Corresponding data structure:
> struct vfio_iommu_operation_info {
>   __u32   argsz;
> #define VFIO_IOMMU_BIND_PASIDTBL  (1 << 0) /* Bind PASID Table */
> #define VFIO_IOMMU_BIND_PASID (1 << 1) /* Bind PASID from userspace driver*/
> #define VFIO_IOMMU_BIND_PGTABLE   (1 << 2) /* Bind guest mmu page table */
> #define VFIO_IOMMU_INVAL_IOTLB(1 << 3) /* Invalidate iommu tlb */
>   __u32   flag;
>   __u32   length; // length of the data[] part in byte
>   __u8data[]; // stores the data for iommu op indicated by flag field
> };

If we're doing a generic "Ops" ioctl, then we should have an "op" field
which is defined by an enum.  It doesn't make sense to use flags for
this, for example can we set multiple flag bits?  If not then it's not
a good use for a bit field.  I'm also not sure I understand the value
of the "length" field, can't it always be calculated from argsz?

> For iommu tlb invalidation from userspace, the "__u8 data[]" stores
> data which would be parsed by the "struct iommu_tlb_invalidate" defined
> below.
> 
> 2. Definitions in include/uapi/linux/iommu.h(newly added header file)
> 
> /* IOMMU model definition for iommu operations from userspace */
> enum iommu_model {
>   INTLE_IOMMU,
>   ARM_SMMU,
>   AMD_IOMMU,
>   SPAPR_IOMMU,
>   S390_IOMMU,
> };
> 
> struct iommu_tlb_invalidate {
>   __u32   scope;
> /* pasid-selective invalidation described by @pasid */
> #define IOMMU_INVALIDATE_PASID(1 << 0)
> /* address-selevtive invalidation described by (@vaddr, @size) */
> #define IOMMU_INVALIDATE_VADDR(1 << 1)

Again, is a bit field appropriate here, can a user set both bits?

>   __u32   flags;
> /*  targets non-pasid mappings, @pasid is not valid */
> #define IOMMU_INVALIDATE_NO_PASID (1 << 0)
> /* indicating that the pIOMMU doesn't need to invalidate
>   all intermediate tables cached as part of the PTE for
>   vaddr, only the last-level entry (pte). This is a hint. */
> #define IOMMU_INVALIDATE_VADDR_LEAF   (1 << 1)

Are we venturing into vendor specific attributes here?

>   __u32   pasid;
>   __u64   vaddr;
>   __u64   size;
>   enum iommu_model model;

How does a user learn which model(s) are supported by the interface?
How do they learn which ops are supported?  Perhaps a good use for one
of those flag bits in the outer data structure is "probe".

>   /*
>Vendor may have different HW version and thus the
>data part of this structure differs, use sub_version
>to indicate such difference.
>*/
>   __u322 sub_version;

Not sure I see the value of this vs creating an INTEL_IOMMUv2 entry in
the model enum.

>   __u64 length; // length of the data[] part in byte

Questionably useful vs calculating from argsz again , but it certainly
doesn't need to be a qword :-o

>   __u8data[];
> };
> 
> For Intel, the data structue is:
> struct intel_iommu_invalidate_data {
>   __u64 low;
>   __u64 high;
> }

high/low what?  This is a pretty weak uapi definition.  Thanks,

Alex

[Qemu-devel] [PATCH v6 15/18] crypto: hmac: add af_alg-backend hmac support

From: "Longpeng(Mike)" 

Adds afalg-backend hmac support: introduces some private APIs
firstly, and then intergrates them into qcrypto_hmac_afalg_driver.

Signed-off-by: Longpeng(Mike) 
---
 crypto/hash-afalg.c | 104 +---
 crypto/hmac.c   |  22 +--
 crypto/hmacpriv.h   |  12 ++
 3 files changed, 121 insertions(+), 17 deletions(-)

diff --git a/crypto/hash-afalg.c b/crypto/hash-afalg.c
index 70ab414..cf34c69 100644
--- a/crypto/hash-afalg.c
+++ b/crypto/hash-afalg.c
@@ -1,5 +1,5 @@
 /*
- * QEMU Crypto af_alg-backend hash support
+ * QEMU Crypto af_alg-backend hash/hmac support
  *
  * Copyright (c) 2017 HUAWEI TECHNOLOGIES CO., LTD.
  *
@@ -16,10 +16,13 @@
 #include "qemu-common.h"
 #include "qapi/error.h"
 #include "crypto/hash.h"
+#include "crypto/hmac.h"
 #include "hashpriv.h"
+#include "hmacpriv.h"
 
 static char *
 qcrypto_afalg_hash_format_name(QCryptoHashAlgorithm alg,
+   bool is_hmac,
Error **errp)
 {
 char *name;
@@ -53,18 +56,24 @@ qcrypto_afalg_hash_format_name(QCryptoHashAlgorithm alg,
 return NULL;
 }
 
-name = g_strdup_printf("%s", alg_name);
+if (is_hmac) {
+name = g_strdup_printf("hmac(%s)", alg_name);
+} else {
+name = g_strdup_printf("%s", alg_name);
+}
 
 return name;
 }
 
 static QCryptoAFAlg *
-qcrypto_afalg_hash_ctx_new(QCryptoHashAlgorithm alg, Error **errp)
+qcrypto_afalg_hash_hmac_ctx_new(QCryptoHashAlgorithm alg,
+const uint8_t *key, size_t nkey,
+bool is_hmac, Error **errp)
 {
 QCryptoAFAlg *afalg;
 char *name;
 
-name = qcrypto_afalg_hash_format_name(alg, errp);
+name = qcrypto_afalg_hash_format_name(alg, is_hmac, errp);
 if (!name) {
 return NULL;
 }
@@ -77,19 +86,46 @@ qcrypto_afalg_hash_ctx_new(QCryptoHashAlgorithm alg, Error 
**errp)
 
 g_free(name);
 
+/* HMAC needs setkey */
+if (is_hmac) {
+if (qemu_setsockopt(afalg->tfmfd, SOL_ALG, ALG_SET_KEY,
+key, nkey) != 0) {
+error_setg_errno(errp, errno, "Set hmac key failed");
+qcrypto_afalg_comm_free(afalg);
+return NULL;
+}
+}
+
 return afalg;
 }
 
+static QCryptoAFAlg *
+qcrypto_afalg_hash_ctx_new(QCryptoHashAlgorithm alg,
+   Error **errp)
+{
+return qcrypto_afalg_hash_hmac_ctx_new(alg, NULL, 0, false, errp);
+}
+
+QCryptoAFAlg *
+qcrypto_afalg_hmac_ctx_new(QCryptoHashAlgorithm alg,
+   const uint8_t *key, size_t nkey,
+   Error **errp)
+{
+return qcrypto_afalg_hash_hmac_ctx_new(alg, key, nkey, true, errp);
+}
+
 static int
-qcrypto_afalg_hash_bytesv(QCryptoHashAlgorithm alg,
-  const struct iovec *iov,
-  size_t niov, uint8_t **result,
-  size_t *resultlen,
-  Error **errp)
+qcrypto_afalg_hash_hmac_bytesv(QCryptoAFAlg *hmac,
+   QCryptoHashAlgorithm alg,
+   const struct iovec *iov,
+   size_t niov, uint8_t **result,
+   size_t *resultlen,
+   Error **errp)
 {
 QCryptoAFAlg *afalg;
 struct iovec outv;
 int ret = 0;
+bool is_hmac = (hmac != NULL) ? true : false;
 const int expect_len = qcrypto_hash_digest_len(alg);
 
 if (*resultlen == 0) {
@@ -102,9 +138,13 @@ qcrypto_afalg_hash_bytesv(QCryptoHashAlgorithm alg,
 return -1;
 }
 
-afalg = qcrypto_afalg_hash_ctx_new(alg, errp);
-if (!afalg) {
-return -1;
+if (is_hmac) {
+afalg = hmac;
+} else {
+afalg = qcrypto_afalg_hash_ctx_new(alg, errp);
+if (!afalg) {
+return -1;
+}
 }
 
 /* send data to kernel's crypto core */
@@ -127,10 +167,48 @@ qcrypto_afalg_hash_bytesv(QCryptoHashAlgorithm alg,
 }
 
 out:
-qcrypto_afalg_comm_free(afalg);
+if (!is_hmac) {
+qcrypto_afalg_comm_free(afalg);
+}
 return ret;
 }
 
+static int
+qcrypto_afalg_hash_bytesv(QCryptoHashAlgorithm alg,
+  const struct iovec *iov,
+  size_t niov, uint8_t **result,
+  size_t *resultlen,
+  Error **errp)
+{
+return qcrypto_afalg_hash_hmac_bytesv(NULL, alg, iov, niov, result,
+  resultlen, errp);
+}
+
+static int
+qcrypto_afalg_hmac_bytesv(QCryptoHmac *hmac,
+  const struct iovec *iov,
+  size_t niov, uint8_t **result,
+  size_t *resultlen,
+  Error **errp)
+{
+return qcrypto_afalg_hash_hmac_bytesv(hmac->opaque, hmac->alg,
+  iov, n

[Qemu-devel] [PATCH v6 14/18] crypto: hash: add afalg-backend hash support

From: "Longpeng(Mike)" 

Adds afalg-backend hash support: introduces some private APIs
firstly, and then intergrates them into qcrypto_hash_afalg_driver.

Signed-off-by: Longpeng(Mike) 
---
 crypto/Makefile.objs |   1 +
 crypto/afalgpriv.h   |   1 +
 crypto/hash-afalg.c  | 136 +++
 crypto/hash.c|  17 +++
 crypto/hashpriv.h|   8 +++
 5 files changed, 163 insertions(+)
 create mode 100644 crypto/hash-afalg.c

diff --git a/crypto/Makefile.objs b/crypto/Makefile.objs
index d2e8fa8..2b99e08 100644
--- a/crypto/Makefile.objs
+++ b/crypto/Makefile.objs
@@ -12,6 +12,7 @@ crypto-obj-y += desrfb.o
 crypto-obj-y += cipher.o
 crypto-obj-$(CONFIG_AF_ALG) += afalg.o
 crypto-obj-$(CONFIG_AF_ALG) += cipher-afalg.o
+crypto-obj-$(CONFIG_AF_ALG) += hash-afalg.o
 crypto-obj-y += tlscreds.o
 crypto-obj-y += tlscredsanon.o
 crypto-obj-y += tlscredsx509.o
diff --git a/crypto/afalgpriv.h b/crypto/afalgpriv.h
index d0941d4..f6550b5 100644
--- a/crypto/afalgpriv.h
+++ b/crypto/afalgpriv.h
@@ -24,6 +24,7 @@
 #endif
 
 #define AFALG_TYPE_CIPHER "skcipher"
+#define AFALG_TYPE_HASH "hash"
 
 #define ALG_OPTYPE_LEN 4
 #define ALG_MSGIV_LEN(len) (sizeof(struct af_alg_iv) + (len))
diff --git a/crypto/hash-afalg.c b/crypto/hash-afalg.c
new file mode 100644
index 000..70ab414
--- /dev/null
+++ b/crypto/hash-afalg.c
@@ -0,0 +1,136 @@
+/*
+ * QEMU Crypto af_alg-backend hash support
+ *
+ * Copyright (c) 2017 HUAWEI TECHNOLOGIES CO., LTD.
+ *
+ * Authors:
+ *Longpeng(Mike) 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or
+ * (at your option) any later version.  See the COPYING file in the
+ * top-level directory.
+ */
+#include "qemu/osdep.h"
+#include "qemu/iov.h"
+#include "qemu/sockets.h"
+#include "qemu-common.h"
+#include "qapi/error.h"
+#include "crypto/hash.h"
+#include "hashpriv.h"
+
+static char *
+qcrypto_afalg_hash_format_name(QCryptoHashAlgorithm alg,
+   Error **errp)
+{
+char *name;
+const char *alg_name;
+
+switch (alg) {
+case QCRYPTO_HASH_ALG_MD5:
+alg_name = "md5";
+break;
+case QCRYPTO_HASH_ALG_SHA1:
+alg_name = "sha1";
+break;
+case QCRYPTO_HASH_ALG_SHA224:
+alg_name = "sha224";
+break;
+case QCRYPTO_HASH_ALG_SHA256:
+alg_name = "sha256";
+break;
+case QCRYPTO_HASH_ALG_SHA384:
+alg_name = "sha384";
+break;
+case QCRYPTO_HASH_ALG_SHA512:
+alg_name = "sha512";
+break;
+case QCRYPTO_HASH_ALG_RIPEMD160:
+alg_name = "rmd160";
+break;
+
+default:
+error_setg(errp, "Unsupported hash algorithm %d", alg);
+return NULL;
+}
+
+name = g_strdup_printf("%s", alg_name);
+
+return name;
+}
+
+static QCryptoAFAlg *
+qcrypto_afalg_hash_ctx_new(QCryptoHashAlgorithm alg, Error **errp)
+{
+QCryptoAFAlg *afalg;
+char *name;
+
+name = qcrypto_afalg_hash_format_name(alg, errp);
+if (!name) {
+return NULL;
+}
+
+afalg = qcrypto_afalg_comm_alloc(AFALG_TYPE_HASH, name, errp);
+if (!afalg) {
+g_free(name);
+return NULL;
+}
+
+g_free(name);
+
+return afalg;
+}
+
+static int
+qcrypto_afalg_hash_bytesv(QCryptoHashAlgorithm alg,
+  const struct iovec *iov,
+  size_t niov, uint8_t **result,
+  size_t *resultlen,
+  Error **errp)
+{
+QCryptoAFAlg *afalg;
+struct iovec outv;
+int ret = 0;
+const int expect_len = qcrypto_hash_digest_len(alg);
+
+if (*resultlen == 0) {
+*resultlen = expect_len;
+*result = g_new0(uint8_t, *resultlen);
+} else if (*resultlen != expect_len) {
+error_setg(errp,
+   "Result buffer size %zu is not match hash %d",
+   *resultlen, expect_len);
+return -1;
+}
+
+afalg = qcrypto_afalg_hash_ctx_new(alg, errp);
+if (!afalg) {
+return -1;
+}
+
+/* send data to kernel's crypto core */
+ret = iov_send_recv(afalg->opfd, iov, niov,
+0, iov_size(iov, niov), true);
+if (ret < 0) {
+error_setg_errno(errp, errno, "Send data to afalg-core failed");
+goto out;
+}
+
+/* hash && get result */
+outv.iov_base = *result;
+outv.iov_len = *resultlen;
+ret = iov_send_recv(afalg->opfd, &outv, 1,
+0, iov_size(&outv, 1), false);
+if (ret < 0) {
+error_setg_errno(errp, errno, "Recv result from afalg-core failed");
+} else {
+ret = 0;
+}
+
+out:
+qcrypto_afalg_comm_free(afalg);
+return ret;
+}
+
+QCryptoHashDriver qcrypto_hash_afalg_driver = {
+.hash_bytesv = qcrypto_afalg_hash_bytesv,
+};
diff --git a/crypto/hash.c b/crypto/hash.c
index c43fd87..ac59c63 100644
--- a/crypto/hash.c
+++ b/crypto/hash.c
@@ -46,6 +46,23 @@ int qcrypto_hash_bytesv(QCryptoHashAlg

Re: [Qemu-devel] [PATCHv9 3/3] fw_cfg: move QOM type defines and fw_cfg types into fw_cfg.h

On Fri, Jul 14, 2017 at 10:40:08AM +0100, Mark Cave-Ayland wrote:
> By exposing FWCfgIoState and FWCfgMemState internals we allow the possibility
> for the internal MemoryRegion fields to be mapped by name for boards that wish
> to wire up the fw_cfg device themselves.
> 
> Signed-off-by: Mark Cave-Ayland 
> Reviewed-by: Laszlo Ersek 
[...]
> diff --git a/include/hw/nvram/fw_cfg.h b/include/hw/nvram/fw_cfg.h
> index b980cba..b77ea48 100644
> --- a/include/hw/nvram/fw_cfg.h
> +++ b/include/hw/nvram/fw_cfg.h
> @@ -1,8 +1,19 @@
>  #ifndef FW_CFG_H
>  #define FW_CFG_H
>  
> +#include "qemu/typedefs.h"
>  #include "exec/hwaddr.h"
>  #include "hw/nvram/fw_cfg_keys.h"
> +#include "hw/sysbus.h"
> +#include "sysemu/dma.h"
> +
> +#define TYPE_FW_CFG "fw_cfg"
> +#define TYPE_FW_CFG_IO  "fw_cfg_io"
> +#define TYPE_FW_CFG_MEM "fw_cfg_mem"
> +
> +#define FW_CFG(obj) OBJECT_CHECK(FWCfgState,(obj), TYPE_FW_CFG)
> +#define FW_CFG_IO(obj)  OBJECT_CHECK(FWCfgIoState,  (obj), TYPE_FW_CFG_IO)
> +#define FW_CFG_MEM(obj) OBJECT_CHECK(FWCfgMemState, (obj), TYPE_FW_CFG_MEM)
>  
>  typedef struct FWCfgFile {
>  uint32_t  size;/* file size */
> @@ -35,6 +46,45 @@ typedef struct FWCfgDmaAccess {
>  
>  typedef void (*FWCfgReadCallback)(void *opaque);
>  
> +struct FWCfgState {
> +/*< private >*/
> +SysBusDevice parent_obj;
> +/*< public >*/
> +
> +uint16_t file_slots;
> +FWCfgEntry *entries[2];
> +int *entry_order;
> +FWCfgFiles *files;
> +uint16_t cur_entry;
> +uint32_t cur_offset;
> +Notifier machine_ready;
> +
> +int fw_cfg_order_override;
> +
> +bool dma_enabled;
> +dma_addr_t dma_addr;
> +AddressSpace *dma_as;
> +MemoryRegion dma_iomem;
> +};
> +
> +struct FWCfgIoState {
> +/*< private >*/
> +FWCfgState parent_obj;
> +/*< public >*/
> +
> +MemoryRegion comb_iomem;
> +};
> +
> +struct FWCfgMemState {
> +/*< private >*/
> +FWCfgState parent_obj;
> +/*< public >*/
> +
> +MemoryRegion ctl_iomem, data_iomem;
> +uint32_t data_width;
> +MemoryRegionOps wide_data_ops;
> +};

Why do you need the full struct declaration to be exposed in the
header?  The memory regions are supposed to be visible as QOM
children to the fw_cfg device, already.

-- 
Eduardo

[Qemu-devel] [PATCH v6 12/18] crypto: introduce some common functions for af_alg backend

From: "Longpeng(Mike)" 

The AF_ALG socket family is the userspace interface for linux
crypto API, this patch adds af_alg family support and some common
functions for af_alg backend. It'll be used by afalg-backend crypto
latter.

Signed-off-by: Longpeng(Mike) 
---
 configure|  30 +
 crypto/Makefile.objs |   1 +
 crypto/afalg.c   | 116 +++
 crypto/afalgpriv.h   |  54 
 4 files changed, 201 insertions(+)
 create mode 100644 crypto/afalg.c
 create mode 100644 crypto/afalgpriv.h

diff --git a/configure b/configure
index 902653a..4beab2a 100755
--- a/configure
+++ b/configure
@@ -375,6 +375,7 @@ libnfs=""
 coroutine=""
 coroutine_pool=""
 debug_stack_usage="no"
+crypto_afalg="no"
 seccomp=""
 glusterfs=""
 glusterfs_xlator_opt="no"
@@ -1124,6 +1125,8 @@ for opt do
   ;;
   --enable-debug-stack-usage) debug_stack_usage="yes"
   ;;
+  --enable-crypto-afalg) crypto_afalg="yes"
+  ;;
   --disable-docs) docs="no"
   ;;
   --enable-docs) docs="yes"
@@ -1443,6 +1446,8 @@ Advanced options (experts only):
xen pv domain builder
   --enable-debug-stack-usage
track the maximum stack usage of stacks created by 
qemu_alloc_stack
+  --enable-crypto-afalg
+   enable afalg-backend crypto and try to use it first.
 
 Optional features, enabled with --enable-FEATURE and
 disabled with --disable-FEATURE, default is enabled if available:
@@ -4834,6 +4839,24 @@ if compile_prog "" "" ; then
 have_af_vsock=yes
 fi
 
+##
+# check for usable AF_ALG environment
+hava_af_alg=no
+cat > $TMPC << EOF
+#include 
+#include 
+#include 
+#include 
+int main(void) {
+int sock;
+sock = socket(AF_ALG, SOCK_SEQPACKET, 0);
+return sock;
+}
+EOF
+if compile_prog "" "" ; then
+have_afalg=yes
+fi
+
 #
 # Sparc implicitly links with --relax, which is
 # incompatible with -r, so --no-relax should be
@@ -5300,6 +5323,7 @@ echo "seccomp support   $seccomp"
 echo "coroutine backend $coroutine"
 echo "coroutine pool$coroutine_pool"
 echo "debug stack usage $debug_stack_usage"
+echo "crypto afalg  $crypto_afalg"
 echo "GlusterFS support $glusterfs"
 echo "gcov  $gcov_tool"
 echo "gcov enabled  $gcov"
@@ -5811,6 +5835,12 @@ if test "$debug_stack_usage" = "yes" ; then
   echo "CONFIG_DEBUG_STACK_USAGE=y" >> $config_host_mak
 fi
 
+if test "$crypto_afalg" = "yes" ; then
+  if test "$have_afalg" = "yes" ; then
+echo "CONFIG_AF_ALG=y" >> $config_host_mak
+  fi
+fi
+
 if test "$open_by_handle_at" = "yes" ; then
   echo "CONFIG_OPEN_BY_HANDLE=y" >> $config_host_mak
 fi
diff --git a/crypto/Makefile.objs b/crypto/Makefile.objs
index 1f749f2..2be5a3a 100644
--- a/crypto/Makefile.objs
+++ b/crypto/Makefile.objs
@@ -10,6 +10,7 @@ crypto-obj-$(if $(CONFIG_NETTLE),n,$(if 
$(CONFIG_GCRYPT_HMAC),n,y)) += hmac-glib
 crypto-obj-y += aes.o
 crypto-obj-y += desrfb.o
 crypto-obj-y += cipher.o
+crypto-obj-$(CONFIG_AF_ALG) += afalg.o
 crypto-obj-y += tlscreds.o
 crypto-obj-y += tlscredsanon.o
 crypto-obj-y += tlscredsx509.o
diff --git a/crypto/afalg.c b/crypto/afalg.c
new file mode 100644
index 000..10046bb
--- /dev/null
+++ b/crypto/afalg.c
@@ -0,0 +1,116 @@
+/*
+ * QEMU Crypto af_alg support
+ *
+ * Copyright (c) 2017 HUAWEI TECHNOLOGIES CO., LTD.
+ *
+ * Authors:
+ *Longpeng(Mike) 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or
+ * (at your option) any later version.  See the COPYING file in the
+ * top-level directory.
+ */
+#include "qemu/osdep.h"
+#include "qemu/cutils.h"
+#include "qemu/sockets.h"
+#include "qapi/error.h"
+#include "afalgpriv.h"
+
+static bool
+qcrypto_afalg_build_saddr(const char *type, const char *name,
+  struct sockaddr_alg *salg, Error **errp)
+{
+salg->salg_family = AF_ALG;
+
+if (strnlen(type, SALG_TYPE_LEN_MAX) >= SALG_TYPE_LEN_MAX) {
+error_setg(errp, "Afalg type(%s) is larger than %d bytes",
+   type, SALG_TYPE_LEN_MAX);
+return false;
+}
+
+if (strnlen(name, SALG_NAME_LEN_MAX) >= SALG_NAME_LEN_MAX) {
+error_setg(errp, "Afalg name(%s) is larger than %d bytes",
+   name, SALG_NAME_LEN_MAX);
+return false;
+}
+
+pstrcpy((char *)salg->salg_type, SALG_TYPE_LEN_MAX, type);
+pstrcpy((char *)salg->salg_name, SALG_NAME_LEN_MAX, name);
+
+return true;
+}
+
+static int
+qcrypto_afalg_socket_bind(const char *type, const char *name,
+  Error **errp)
+{
+int sbind;
+struct sockaddr_alg salg = {0};
+
+if (!qcrypto_afalg_build_saddr(type, name, &salg, errp)) {
+return -1;
+}
+
+sbind = qemu_socket(AF_ALG, SOCK_SEQPACKET, 0);
+if (sbind < 0) {
+error_setg_errno(errp, errno, "Failed to create socket");
+return -1;
+}
+
+if (b

[Qemu-devel] [PATCH v6 13/18] crypto: cipher: add afalg-backend cipher support

From: "Longpeng(Mike)" 

Adds afalg-backend cipher support: introduces some private APIs
firstly, and then intergrates them into qcrypto_cipher_afalg_driver.

Signed-off-by: Longpeng(Mike) 
---
 crypto/Makefile.objs  |   1 +
 crypto/afalgpriv.h|   9 ++
 crypto/cipher-afalg.c | 226 ++
 crypto/cipher.c   |  23 -
 crypto/cipherpriv.h   |  16 
 5 files changed, 271 insertions(+), 4 deletions(-)
 create mode 100644 crypto/cipher-afalg.c

diff --git a/crypto/Makefile.objs b/crypto/Makefile.objs
index 2be5a3a..d2e8fa8 100644
--- a/crypto/Makefile.objs
+++ b/crypto/Makefile.objs
@@ -11,6 +11,7 @@ crypto-obj-y += aes.o
 crypto-obj-y += desrfb.o
 crypto-obj-y += cipher.o
 crypto-obj-$(CONFIG_AF_ALG) += afalg.o
+crypto-obj-$(CONFIG_AF_ALG) += cipher-afalg.o
 crypto-obj-y += tlscreds.o
 crypto-obj-y += tlscredsanon.o
 crypto-obj-y += tlscredsx509.o
diff --git a/crypto/afalgpriv.h b/crypto/afalgpriv.h
index 76118cf..d0941d4 100644
--- a/crypto/afalgpriv.h
+++ b/crypto/afalgpriv.h
@@ -19,6 +19,15 @@
 #define SALG_TYPE_LEN_MAX 14
 #define SALG_NAME_LEN_MAX 64
 
+#ifndef SOL_ALG
+#define SOL_ALG 279
+#endif
+
+#define AFALG_TYPE_CIPHER "skcipher"
+
+#define ALG_OPTYPE_LEN 4
+#define ALG_MSGIV_LEN(len) (sizeof(struct af_alg_iv) + (len))
+
 typedef struct QCryptoAFAlg QCryptoAFAlg;
 
 struct QCryptoAFAlg {
diff --git a/crypto/cipher-afalg.c b/crypto/cipher-afalg.c
new file mode 100644
index 000..01343b2
--- /dev/null
+++ b/crypto/cipher-afalg.c
@@ -0,0 +1,226 @@
+/*
+ * QEMU Crypto af_alg-backend cipher support
+ *
+ * Copyright (c) 2017 HUAWEI TECHNOLOGIES CO., LTD.
+ *
+ * Authors:
+ *Longpeng(Mike) 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or
+ * (at your option) any later version.  See the COPYING file in the
+ * top-level directory.
+ */
+#include "qemu/osdep.h"
+#include "qemu/sockets.h"
+#include "qemu-common.h"
+#include "qapi/error.h"
+#include "crypto/cipher.h"
+#include "cipherpriv.h"
+
+
+static char *
+qcrypto_afalg_cipher_format_name(QCryptoCipherAlgorithm alg,
+ QCryptoCipherMode mode,
+ Error **errp)
+{
+char *name;
+const char *alg_name;
+const char *mode_name;
+
+switch (alg) {
+case QCRYPTO_CIPHER_ALG_AES_128:
+case QCRYPTO_CIPHER_ALG_AES_192:
+case QCRYPTO_CIPHER_ALG_AES_256:
+alg_name = "aes";
+break;
+case QCRYPTO_CIPHER_ALG_CAST5_128:
+alg_name = "cast5";
+break;
+case QCRYPTO_CIPHER_ALG_SERPENT_128:
+case QCRYPTO_CIPHER_ALG_SERPENT_192:
+case QCRYPTO_CIPHER_ALG_SERPENT_256:
+alg_name = "serpent";
+break;
+case QCRYPTO_CIPHER_ALG_TWOFISH_128:
+case QCRYPTO_CIPHER_ALG_TWOFISH_192:
+case QCRYPTO_CIPHER_ALG_TWOFISH_256:
+alg_name = "twofish";
+break;
+
+default:
+error_setg(errp, "Unsupported cipher algorithm %d", alg);
+return NULL;
+}
+
+mode_name = QCryptoCipherMode_lookup[mode];
+name = g_strdup_printf("%s(%s)", mode_name, alg_name);
+
+return name;
+}
+
+QCryptoAFAlg *
+qcrypto_afalg_cipher_ctx_new(QCryptoCipherAlgorithm alg,
+ QCryptoCipherMode mode,
+ const uint8_t *key,
+ size_t nkey, Error **errp)
+{
+QCryptoAFAlg *afalg;
+size_t expect_niv;
+char *name;
+
+name = qcrypto_afalg_cipher_format_name(alg, mode, errp);
+if (!name) {
+return NULL;
+}
+
+afalg = qcrypto_afalg_comm_alloc(AFALG_TYPE_CIPHER, name, errp);
+if (!afalg) {
+g_free(name);
+return NULL;
+}
+
+g_free(name);
+
+/* setkey */
+if (qemu_setsockopt(afalg->tfmfd, SOL_ALG, ALG_SET_KEY, key,
+nkey) != 0) {
+error_setg_errno(errp, errno, "Set key failed");
+qcrypto_afalg_comm_free(afalg);
+return NULL;
+}
+
+/* prepare msg header */
+afalg->msg = g_new0(struct msghdr, 1);
+afalg->msg->msg_controllen += CMSG_SPACE(ALG_OPTYPE_LEN);
+expect_niv = qcrypto_cipher_get_iv_len(alg, mode);
+if (expect_niv) {
+afalg->msg->msg_controllen += CMSG_SPACE(ALG_MSGIV_LEN(expect_niv));
+}
+afalg->msg->msg_control = g_new0(uint8_t, afalg->msg->msg_controllen);
+
+/* We use 1st msghdr for crypto-info and 2nd msghdr for IV-info */
+afalg->cmsg = CMSG_FIRSTHDR(afalg->msg);
+afalg->cmsg->cmsg_type = ALG_SET_OP;
+afalg->cmsg->cmsg_len = CMSG_SPACE(ALG_OPTYPE_LEN);
+if (expect_niv) {
+afalg->cmsg = CMSG_NXTHDR(afalg->msg, afalg->cmsg);
+afalg->cmsg->cmsg_type = ALG_SET_IV;
+afalg->cmsg->cmsg_len = CMSG_SPACE(ALG_MSGIV_LEN(expect_niv));
+}
+afalg->cmsg = CMSG_FIRSTHDR(afalg->msg);
+
+return afalg;
+}
+
+static int
+qcrypto_afalg_cipher_setiv(QCryptoCipher *cipher,
+   const uint8_t *iv,
+   size_t niv, Er

[Qemu-devel] [PATCH v6 18/18] tests: crypto: add hmac speed benchmark support

From: "Longpeng(Mike)" 

This patch add a hmac speed benchmark, it helps us to
measure the performance by using "make check-speed" or
using "./tests/benchmark-crypto-hmac" directly.

Signed-off-by: Longpeng(Mike) 
---
 tests/Makefile.include|  2 ++
 tests/benchmark-crypto-hmac.c | 82 +++
 2 files changed, 84 insertions(+)
 create mode 100644 tests/benchmark-crypto-hmac.c

diff --git a/tests/Makefile.include b/tests/Makefile.include
index c5a67a3..18f9359 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -99,6 +99,7 @@ gcov-files-test-write-threshold-y = block/write-threshold.c
 check-unit-y += tests/test-crypto-hash$(EXESUF)
 check-speed-y += tests/benchmark-crypto-hash$(EXESUF)
 check-unit-y += tests/test-crypto-hmac$(EXESUF)
+check-speed-y += tests/benchmark-crypto-hmac$(EXESUF)
 check-unit-y += tests/test-crypto-cipher$(EXESUF)
 check-speed-y += tests/benchmark-crypto-cipher$(EXESUF)
 check-unit-y += tests/test-crypto-secret$(EXESUF)
@@ -638,6 +639,7 @@ tests/test-bitcnt$(EXESUF): tests/test-bitcnt.o 
$(test-util-obj-y)
 tests/test-crypto-hash$(EXESUF): tests/test-crypto-hash.o $(test-crypto-obj-y)
 tests/benchmark-crypto-hash$(EXESUF): tests/benchmark-crypto-hash.o 
$(test-crypto-obj-y)
 tests/test-crypto-hmac$(EXESUF): tests/test-crypto-hmac.o $(test-crypto-obj-y)
+tests/benchmark-crypto-hmac$(EXESUF): tests/benchmark-crypto-hmac.o 
$(test-crypto-obj-y)
 tests/test-crypto-cipher$(EXESUF): tests/test-crypto-cipher.o 
$(test-crypto-obj-y)
 tests/benchmark-crypto-cipher$(EXESUF): tests/benchmark-crypto-cipher.o 
$(test-crypto-obj-y)
 tests/test-crypto-secret$(EXESUF): tests/test-crypto-secret.o 
$(test-crypto-obj-y)
diff --git a/tests/benchmark-crypto-hmac.c b/tests/benchmark-crypto-hmac.c
new file mode 100644
index 000..72408be
--- /dev/null
+++ b/tests/benchmark-crypto-hmac.c
@@ -0,0 +1,82 @@
+/*
+ * QEMU Crypto hmac speed benchmark
+ *
+ * Copyright (c) 2017 HUAWEI TECHNOLOGIES CO., LTD.
+ *
+ * Authors:
+ *Longpeng(Mike) 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or
+ * (at your option) any later version.  See the COPYING file in the
+ * top-level directory.
+ */
+#include "qemu/osdep.h"
+#include "crypto/init.h"
+#include "crypto/hmac.h"
+
+#define KEY "monkey monkey monkey monkey"
+
+static void test_hmac_speed(const void *opaque)
+{
+size_t chunk_size = (size_t)opaque;
+QCryptoHmac *hmac = NULL;
+uint8_t *in = NULL, *out = NULL;
+size_t out_len = 0;
+double total = 0.0;
+struct iovec iov;
+Error *err = NULL;
+int ret;
+
+if (!qcrypto_hmac_supports(QCRYPTO_HASH_ALG_SHA256)) {
+return;
+}
+
+in = g_new0(uint8_t, chunk_size);
+memset(in, g_test_rand_int(), chunk_size);
+
+iov.iov_base = (char *)in;
+iov.iov_len = chunk_size;
+
+g_test_timer_start();
+do {
+hmac = qcrypto_hmac_new(QCRYPTO_HASH_ALG_SHA256,
+(const uint8_t *)KEY, strlen(KEY), &err);
+g_assert(err == NULL);
+g_assert(hmac != NULL);
+
+ret = qcrypto_hmac_bytesv(hmac, &iov, 1, &out, &out_len, &err);
+g_assert(ret == 0);
+g_assert(err == NULL);
+
+qcrypto_hmac_free(hmac);
+
+total += chunk_size;
+} while (g_test_timer_elapsed() < 5.0);
+
+total /= 1024 * 1024; /* to MB */
+
+g_print("hmac(sha256): ");
+g_print("Testing chunk_size %ld bytes ", chunk_size);
+g_print("done: %.2f MB in %.2f secs: ", total, g_test_timer_last());
+g_print("%.2f MB/sec\n", total / g_test_timer_last());
+
+g_free(out);
+g_free(in);
+}
+
+int main(int argc, char **argv)
+{
+size_t i;
+char name[64];
+
+g_test_init(&argc, &argv, NULL);
+g_assert(qcrypto_init(NULL) == 0);
+
+for (i = 512; i <= (64 * 1204); i *= 2) {
+memset(name, 0 , sizeof(name));
+snprintf(name, sizeof(name), "/crypto/hmac/speed-%lu", i);
+g_test_add_data_func(name, (void *)i, test_hmac_speed);
+}
+
+return g_test_run();
+}
-- 
1.8.3.1

[Qemu-devel] investigating TPM for OVMF-on-QEMU

Hi,

I spent most of today reading TPM related material (specs, guides and
source code), and below I'd like to describe what I think should be our
initial attack on enabling TPM on QEMU/OVMF.

(1) Versions of UEFI abstractions regarding TPM (very roughly speaking):

There have been three sets of specs for this, and accordingly,
edk2's SecurityPkg/Tcg/ directory supports each set. We can call
these TPM1, TrEE, and TPM2.

- TPM1 was the initial TPM thing, supporting only SHA-1, and is now
obsolete.

- TrEE was Microsoft's own update to the TPM1 software interfaces,
supporting TPM2 hardware with more hash algorithms, and is now
obsolete. Because,

- TPM2 is basically the standardized version of TrEE, the most
recent set of specs, and what we should focus on.

(2) Drivers (and features) in edk2/SecurityPkg/Tcg.

There are 19 modules under SecurityPkg/Tcg/. Let me categorize them.
In each category, I'll try to list modules in loosely increasing
dependency (or PI/UEFI phase) order.

(2a) Modules that are obsolete because they are tied to TPM1:

TcgPei/TcgPei.inf
PhysicalPresencePei/PhysicalPresencePei.inf
TcgDxe/TcgDxe.inf
TcgConfigDxe/TcgConfigDxe.inf
TcgSmm/TcgSmm.inf

The TPM enablement instructions at

https://github.com/tianocore/tianocore.github.io/wiki/How-to-Enable-Security#Enabling_Trusted_Compute_Module_TPM

are written up in terms of these modules, but that doesn't obsolete
the article too much.

(2b) Modules that are obsolete due to being tied to TrEE:

TrEEConfig/TrEEConfigPei.inf
TrEEPei/TrEEPei.inf
TrEEDxe/TrEEDxe.inf
TrEEConfig/TrEEConfigDxe.inf
TrEESmm/TrEESmm.inf

There is a great Intel whitepaper called

A Tour Beyond BIOS with the UEFI TPM2 Support in EDKII

which is written in terms of TrEE and these modules, explaining how
they supersede TPM1 and the modules under (2a).

This whitepaper seems to apply to the most recent TPM2 stack as
well (mostly just replace TrEE references with TPM2 references), so
it is very useful.

(2c) Modules that call themselves obsolete or deprecated (without being
obviously tied to TPM1 or TrEE):

MemoryOverwriteRequestControlLock/TcgMorLockSmm.inf

(2d) Modules that we won't need due to no hardware support in QEMU:

Opal/OpalPasswordDxe/OpalPasswordDxe.inf
Opal/OpalPasswordSmm/OpalPasswordSmm.inf

TCG's "Opal" seems to be about self-encrypting drives, see

https://en.wikipedia.org/wiki/Opal_Storage_Specification

so these modules are not relevant for us.

(2e) Modules that we should use. Again, in increasing order of
dependence. And here I'll comment as well on what these do:

Tcg2Config/Tcg2ConfigPei.inf -- Informs the firmware globally
about the TPM device type. This
module can perform device
detection or read a cached value
from a non-volatile UEFI
variable.

Tcg2Pei/Tcg2Pei.inf -- Initializes the TPM device and
measures the firmware volumes in
the PEI phase into the TPM's
platform config registers.

Tcg2Dxe/Tcg2Dxe.inf -- Measures DXE phase (and later)
modules into the TPM's PCRs, and
also lets the OS boot loader
measure things, by exposing the
EFI_TCG2_PROTOCOL.

Tcg2Config/Tcg2ConfigDxe.inf -- Provides a Setup TUI interface to
configure the TPM. IIUC, it can
also save the configured TPM type
for subsequent boots (see
Tcg2ConfigPei.inf above).

This driver stack supports the TIS (MMIO) hardware interface, which
is advertized to the OS in the TPM2 ACPI Table's "start method"
field with value 6. (The according macro is TPM2_START_METHOD_MMIO
in the QEMU source code, and EFI_TPM2_ACPI_TABLE_START_METHOD_TIS
in the edk2 source code.)

Including these drivers should result in a functional
EFI_TCG2_PROTOCOL, which is what OS boot loaders primarily care
about, as I understand.

Importantly, the driver stack above requires PEI-phase variable
access, therefore
must be solved
first.

(I have had patches for said BZ ready for a while. I've failed to
upstream them thus far because a pflash-based varstore is a hard
requirement for them. I think that's a n

[Qemu-devel] [PATCH v6 10/18] crypto: hmac: introduce qcrypto_hmac_ctx_new for glib-backend

From: "Longpeng(Mike)" 

Extracts qcrypto_hmac_ctx_new() from qcrypto_hmac_new() for
glib-backend impls.

Reviewed-by: Daniel P. Berrange 
Reviewed-by: Gonglei 
Signed-off-by: Longpeng(Mike) 
---
 crypto/hmac-glib.c | 34 --
 1 file changed, 24 insertions(+), 10 deletions(-)

diff --git a/crypto/hmac-glib.c b/crypto/hmac-glib.c
index 08a1fdd..f0ccfd6 100644
--- a/crypto/hmac-glib.c
+++ b/crypto/hmac-glib.c
@@ -49,11 +49,11 @@ bool qcrypto_hmac_supports(QCryptoHashAlgorithm alg)
 return false;
 }
 
-QCryptoHmac *qcrypto_hmac_new(QCryptoHashAlgorithm alg,
-  const uint8_t *key, size_t nkey,
-  Error **errp)
+static QCryptoHmacGlib *
+qcrypto_hmac_ctx_new(QCryptoHashAlgorithm alg,
+ const uint8_t *key, size_t nkey,
+ Error **errp)
 {
-QCryptoHmac *hmac;
 QCryptoHmacGlib *ctx;
 
 if (!qcrypto_hmac_supports(alg)) {
@@ -62,9 +62,6 @@ QCryptoHmac *qcrypto_hmac_new(QCryptoHashAlgorithm alg,
 return NULL;
 }
 
-hmac = g_new0(QCryptoHmac, 1);
-hmac->alg = alg;
-
 ctx = g_new0(QCryptoHmacGlib, 1);
 
 ctx->ghmac = g_hmac_new(qcrypto_hmac_alg_map[alg],
@@ -74,12 +71,10 @@ QCryptoHmac *qcrypto_hmac_new(QCryptoHashAlgorithm alg,
 goto error;
 }
 
-hmac->opaque = ctx;
-return hmac;
+return ctx;
 
 error:
 g_free(ctx);
-g_free(hmac);
 return NULL;
 }
 
@@ -134,6 +129,25 @@ int qcrypto_hmac_bytesv(QCryptoHmac *hmac,
 return 0;
 }
 
+QCryptoHmac *qcrypto_hmac_new(QCryptoHashAlgorithm alg,
+  const uint8_t *key, size_t nkey,
+  Error **errp)
+{
+QCryptoHmac *hmac;
+QCryptoHmacGlib *ctx;
+
+ctx = qcrypto_hmac_ctx_new(alg, key, nkey, errp);
+if (!ctx) {
+return NULL;
+}
+
+hmac = g_new0(QCryptoHmac, 1);
+hmac->alg = alg;
+hmac->opaque = ctx;
+
+return hmac;
+}
+
 #else
 
 bool qcrypto_hmac_supports(QCryptoHashAlgorithm alg)
-- 
1.8.3.1

[Qemu-devel] [PATCH v6 11/18] crypto: hmac: add hmac driver framework

From: "Longpeng(Mike)" 

1) makes the public APIs in hmac-nettle/gcrypt/glib static,
   and rename them with "nettle/gcrypt/glib" prefix.

2) introduces hmac framework, including QCryptoHmacDriver
   and new public APIs.

Reviewed-by: Daniel P. Berrange 
Signed-off-by: Longpeng(Mike) 
---
 crypto/hmac-gcrypt.c  | 51 ---
 crypto/hmac-glib.c| 75 +--
 crypto/hmac-nettle.c  | 52 ---
 crypto/hmac.c | 44 ++
 crypto/hmacpriv.h | 36 +
 include/crypto/hmac.h |  1 +
 6 files changed, 145 insertions(+), 114 deletions(-)
 create mode 100644 crypto/hmacpriv.h

diff --git a/crypto/hmac-gcrypt.c b/crypto/hmac-gcrypt.c
index 372ad7f..76ca61b 100644
--- a/crypto/hmac-gcrypt.c
+++ b/crypto/hmac-gcrypt.c
@@ -15,6 +15,7 @@
 #include "qemu/osdep.h"
 #include "qapi/error.h"
 #include "crypto/hmac.h"
+#include "hmacpriv.h"
 #include 
 
 static int qcrypto_hmac_alg_map[QCRYPTO_HASH_ALG__MAX] = {
@@ -42,10 +43,9 @@ bool qcrypto_hmac_supports(QCryptoHashAlgorithm alg)
 return false;
 }
 
-static QCryptoHmacGcrypt *
-qcrypto_hmac_ctx_new(QCryptoHashAlgorithm alg,
- const uint8_t *key, size_t nkey,
- Error **errp)
+void *qcrypto_hmac_ctx_new(QCryptoHashAlgorithm alg,
+   const uint8_t *key, size_t nkey,
+   Error **errp)
 {
 QCryptoHmacGcrypt *ctx;
 gcry_error_t err;
@@ -81,27 +81,24 @@ error:
 return NULL;
 }
 
-void qcrypto_hmac_free(QCryptoHmac *hmac)
+static void
+qcrypto_gcrypt_hmac_ctx_free(QCryptoHmac *hmac)
 {
 QCryptoHmacGcrypt *ctx;
 
-if (!hmac) {
-return;
-}
-
 ctx = hmac->opaque;
 gcry_mac_close(ctx->handle);
 
 g_free(ctx);
-g_free(hmac);
 }
 
-int qcrypto_hmac_bytesv(QCryptoHmac *hmac,
-const struct iovec *iov,
-size_t niov,
-uint8_t **result,
-size_t *resultlen,
-Error **errp)
+static int
+qcrypto_gcrypt_hmac_bytesv(QCryptoHmac *hmac,
+   const struct iovec *iov,
+   size_t niov,
+   uint8_t **result,
+   size_t *resultlen,
+   Error **errp)
 {
 QCryptoHmacGcrypt *ctx;
 gcry_error_t err;
@@ -147,21 +144,7 @@ int qcrypto_hmac_bytesv(QCryptoHmac *hmac,
 return 0;
 }
 
-QCryptoHmac *qcrypto_hmac_new(QCryptoHashAlgorithm alg,
-  const uint8_t *key, size_t nkey,
-  Error **errp)
-{
-QCryptoHmac *hmac;
-QCryptoHmacGcrypt *ctx;
-
-ctx = qcrypto_hmac_ctx_new(alg, key, nkey, errp);
-if (!ctx) {
-return NULL;
-}
-
-hmac = g_new0(QCryptoHmac, 1);
-hmac->alg = alg;
-hmac->opaque = ctx;
-
-return hmac;
-}
+QCryptoHmacDriver qcrypto_hmac_lib_driver = {
+.hmac_bytesv = qcrypto_gcrypt_hmac_bytesv,
+.hmac_free = qcrypto_gcrypt_hmac_ctx_free,
+};
diff --git a/crypto/hmac-glib.c b/crypto/hmac-glib.c
index f0ccfd6..8cf6b22 100644
--- a/crypto/hmac-glib.c
+++ b/crypto/hmac-glib.c
@@ -15,6 +15,7 @@
 #include "qemu/osdep.h"
 #include "qapi/error.h"
 #include "crypto/hmac.h"
+#include "hmacpriv.h"
 
 /* Support for HMAC Algos has been added in GLib 2.30 */
 #if GLIB_CHECK_VERSION(2, 30, 0)
@@ -49,10 +50,9 @@ bool qcrypto_hmac_supports(QCryptoHashAlgorithm alg)
 return false;
 }
 
-static QCryptoHmacGlib *
-qcrypto_hmac_ctx_new(QCryptoHashAlgorithm alg,
- const uint8_t *key, size_t nkey,
- Error **errp)
+void *qcrypto_hmac_ctx_new(QCryptoHashAlgorithm alg,
+   const uint8_t *key, size_t nkey,
+   Error **errp)
 {
 QCryptoHmacGlib *ctx;
 
@@ -78,27 +78,24 @@ error:
 return NULL;
 }
 
-void qcrypto_hmac_free(QCryptoHmac *hmac)
+static void
+qcrypto_glib_hmac_ctx_free(QCryptoHmac *hmac)
 {
 QCryptoHmacGlib *ctx;
 
-if (!hmac) {
-return;
-}
-
 ctx = hmac->opaque;
 g_hmac_unref(ctx->ghmac);
 
 g_free(ctx);
-g_free(hmac);
 }
 
-int qcrypto_hmac_bytesv(QCryptoHmac *hmac,
-const struct iovec *iov,
-size_t niov,
-uint8_t **result,
-size_t *resultlen,
-Error **errp)
+static int
+qcrypto_glib_hmac_bytesv(QCryptoHmac *hmac,
+ const struct iovec *iov,
+ size_t niov,
+ uint8_t **result,
+ size_t *resultlen,
+ Error **errp)
 {
 QCryptoHmacGlib *ctx;
 int i, ret;
@@ -129,25 +126,6 @@ int qcrypto_hmac_bytesv(QCryptoHmac *hmac,
 return 0;
 }
 
-QCryptoHmac *qcrypto_hmac_new(QCryptoHashAlgorithm alg,
-

[Qemu-devel] [PATCH v6 05/18] crypto: cipher: add cipher driver framework

From: "Longpeng(Mike)" 

1) makes the public APIs in cipher-nettle/gcrypt/builtin static,
   and rename them with "nettle/gcrypt/builtin" prefix.

2) introduces cipher framework, including QCryptoCipherDriver
   and new public APIs.

Reviewed-by: Daniel P. Berrange 
Reviewed-by: Gonglei 
Signed-off-by: Longpeng(Mike) 
---
 crypto/cipher-builtin.c | 64 +--
 crypto/cipher-gcrypt.c  | 72 +
 crypto/cipher-nettle.c  | 71 
 crypto/cipher.c | 65 
 crypto/cipherpriv.h | 40 +++
 include/crypto/cipher.h |  1 +
 6 files changed, 190 insertions(+), 123 deletions(-)
 create mode 100644 crypto/cipherpriv.h

diff --git a/crypto/cipher-builtin.c b/crypto/cipher-builtin.c
index 4ecd15e..16a36d4 100644
--- a/crypto/cipher-builtin.c
+++ b/crypto/cipher-builtin.c
@@ -22,6 +22,7 @@
 #include "crypto/aes.h"
 #include "crypto/desrfb.h"
 #include "crypto/xts.h"
+#include "cipherpriv.h"
 
 typedef struct QCryptoCipherBuiltinAESContext QCryptoCipherBuiltinAESContext;
 struct QCryptoCipherBuiltinAESContext {
@@ -466,25 +467,22 @@ static QCryptoCipherBuiltin 
*qcrypto_cipher_ctx_new(QCryptoCipherAlgorithm alg,
 return ctxt;
 }
 
-void qcrypto_cipher_free(QCryptoCipher *cipher)
+static void
+qcrypto_builtin_cipher_ctx_free(QCryptoCipher *cipher)
 {
 QCryptoCipherBuiltin *ctxt;
 
-if (!cipher) {
-return;
-}
-
 ctxt = cipher->opaque;
 ctxt->free(cipher);
-g_free(cipher);
 }
 
 
-int qcrypto_cipher_encrypt(QCryptoCipher *cipher,
-   const void *in,
-   void *out,
-   size_t len,
-   Error **errp)
+static int
+qcrypto_builtin_cipher_encrypt(QCryptoCipher *cipher,
+   const void *in,
+   void *out,
+   size_t len,
+   Error **errp)
 {
 QCryptoCipherBuiltin *ctxt = cipher->opaque;
 
@@ -498,11 +496,12 @@ int qcrypto_cipher_encrypt(QCryptoCipher *cipher,
 }
 
 
-int qcrypto_cipher_decrypt(QCryptoCipher *cipher,
-   const void *in,
-   void *out,
-   size_t len,
-   Error **errp)
+static int
+qcrypto_builtin_cipher_decrypt(QCryptoCipher *cipher,
+   const void *in,
+   void *out,
+   size_t len,
+   Error **errp)
 {
 QCryptoCipherBuiltin *ctxt = cipher->opaque;
 
@@ -516,9 +515,10 @@ int qcrypto_cipher_decrypt(QCryptoCipher *cipher,
 }
 
 
-int qcrypto_cipher_setiv(QCryptoCipher *cipher,
- const uint8_t *iv, size_t niv,
- Error **errp)
+static int
+qcrypto_builtin_cipher_setiv(QCryptoCipher *cipher,
+ const uint8_t *iv, size_t niv,
+ Error **errp)
 {
 QCryptoCipherBuiltin *ctxt = cipher->opaque;
 
@@ -526,23 +526,9 @@ int qcrypto_cipher_setiv(QCryptoCipher *cipher,
 }
 
 
-QCryptoCipher *qcrypto_cipher_new(QCryptoCipherAlgorithm alg,
-  QCryptoCipherMode mode,
-  const uint8_t *key, size_t nkey,
-  Error **errp)
-{
-QCryptoCipher *cipher;
-QCryptoCipherBuiltin *ctxt;
-
-ctxt = qcrypto_cipher_ctx_new(alg, mode, key, nkey, errp);
-if (!ctxt) {
-return NULL;
-}
-
-cipher = g_new0(QCryptoCipher, 1);
-cipher->alg = alg;
-cipher->mode = mode;
-cipher->opaque = ctxt;
-
-return cipher;
-}
+static struct QCryptoCipherDriver qcrypto_cipher_lib_driver = {
+.cipher_encrypt = qcrypto_builtin_cipher_encrypt,
+.cipher_decrypt = qcrypto_builtin_cipher_decrypt,
+.cipher_setiv = qcrypto_builtin_cipher_setiv,
+.cipher_free = qcrypto_builtin_cipher_ctx_free,
+};
diff --git a/crypto/cipher-gcrypt.c b/crypto/cipher-gcrypt.c
index 4939386..0489147 100644
--- a/crypto/cipher-gcrypt.c
+++ b/crypto/cipher-gcrypt.c
@@ -20,6 +20,7 @@
 
 #include "qemu/osdep.h"
 #include "crypto/xts.h"
+#include "cipherpriv.h"
 
 #include 
 
@@ -64,8 +65,9 @@ struct QCryptoCipherGcrypt {
 uint8_t *iv;
 };
 
-static void gcrypt_cipher_free_ctx(QCryptoCipherGcrypt *ctx,
-   QCryptoCipherMode mode)
+static void
+qcrypto_gcrypt_cipher_free_ctx(QCryptoCipherGcrypt *ctx,
+   QCryptoCipherMode mode)
 {
 if (!ctx) {
 return;
@@ -239,18 +241,15 @@ static QCryptoCipherGcrypt 
*qcrypto_cipher_ctx_new(QCryptoCipherAlgorithm alg,
 return ctx;
 
  error:
-gcrypt_cipher_free_ctx(ctx, mode);
+qcrypto_gcrypt_cipher_free_ctx(ctx, mode);
 return NULL;
 }
 
 
-void qcrypto_cipher_free(QCrypt

[Qemu-devel] [PATCH v6 08/18] crypto: hmac: introduce qcrypto_hmac_ctx_new for gcrypt-backend

From: "Longpeng(Mike)" 

1) Fix a handle-leak problem in qcrypto_hmac_new(), didn't free
   ctx->handle if gcry_mac_setkey fails.

2) Extracts qcrypto_hmac_ctx_new() from qcrypto_hmac_new() for
   gcrypt-backend impls.

Reviewed-by: Daniel P. Berrange 
Reviewed-by: Gonglei 
Signed-off-by: Longpeng(Mike) 
---
 crypto/hmac-gcrypt.c | 35 +--
 1 file changed, 25 insertions(+), 10 deletions(-)

diff --git a/crypto/hmac-gcrypt.c b/crypto/hmac-gcrypt.c
index 21189e6..372ad7f 100644
--- a/crypto/hmac-gcrypt.c
+++ b/crypto/hmac-gcrypt.c
@@ -42,11 +42,11 @@ bool qcrypto_hmac_supports(QCryptoHashAlgorithm alg)
 return false;
 }
 
-QCryptoHmac *qcrypto_hmac_new(QCryptoHashAlgorithm alg,
-  const uint8_t *key, size_t nkey,
-  Error **errp)
+static QCryptoHmacGcrypt *
+qcrypto_hmac_ctx_new(QCryptoHashAlgorithm alg,
+ const uint8_t *key, size_t nkey,
+ Error **errp)
 {
-QCryptoHmac *hmac;
 QCryptoHmacGcrypt *ctx;
 gcry_error_t err;
 
@@ -56,9 +56,6 @@ QCryptoHmac *qcrypto_hmac_new(QCryptoHashAlgorithm alg,
 return NULL;
 }
 
-hmac = g_new0(QCryptoHmac, 1);
-hmac->alg = alg;
-
 ctx = g_new0(QCryptoHmacGcrypt, 1);
 
 err = gcry_mac_open(&ctx->handle, qcrypto_hmac_alg_map[alg],
@@ -73,15 +70,14 @@ QCryptoHmac *qcrypto_hmac_new(QCryptoHashAlgorithm alg,
 if (err != 0) {
 error_setg(errp, "Cannot set key: %s",
gcry_strerror(err));
+gcry_mac_close(ctx->handle);
 goto error;
 }
 
-hmac->opaque = ctx;
-return hmac;
+return ctx;
 
 error:
 g_free(ctx);
-g_free(hmac);
 return NULL;
 }
 
@@ -150,3 +146,22 @@ int qcrypto_hmac_bytesv(QCryptoHmac *hmac,
 
 return 0;
 }
+
+QCryptoHmac *qcrypto_hmac_new(QCryptoHashAlgorithm alg,
+  const uint8_t *key, size_t nkey,
+  Error **errp)
+{
+QCryptoHmac *hmac;
+QCryptoHmacGcrypt *ctx;
+
+ctx = qcrypto_hmac_ctx_new(alg, key, nkey, errp);
+if (!ctx) {
+return NULL;
+}
+
+hmac = g_new0(QCryptoHmac, 1);
+hmac->alg = alg;
+hmac->opaque = ctx;
+
+return hmac;
+}
-- 
1.8.3.1

[Qemu-devel] [PATCH v6 09/18] crypto: hmac: introduce qcrypto_hmac_ctx_new for nettle-backend

From: "Longpeng(Mike)" 

Extracts qcrypto_hmac_ctx_new() from qcrypto_hmac_new() for
nettle-backend impls.

Reviewed-by: Daniel P. Berrange 
Reviewed-by: Gonglei 
Signed-off-by: Longpeng(Mike) 
---
 crypto/hmac-nettle.c | 34 --
 1 file changed, 24 insertions(+), 10 deletions(-)

diff --git a/crypto/hmac-nettle.c b/crypto/hmac-nettle.c
index 4a9e6b2..000dfd9 100644
--- a/crypto/hmac-nettle.c
+++ b/crypto/hmac-nettle.c
@@ -97,11 +97,11 @@ bool qcrypto_hmac_supports(QCryptoHashAlgorithm alg)
 return false;
 }
 
-QCryptoHmac *qcrypto_hmac_new(QCryptoHashAlgorithm alg,
-  const uint8_t *key, size_t nkey,
-  Error **errp)
+static QCryptoHmacNettle *
+qcrypto_hmac_ctx_new(QCryptoHashAlgorithm alg,
+ const uint8_t *key, size_t nkey,
+ Error **errp)
 {
-QCryptoHmac *hmac;
 QCryptoHmacNettle *ctx;
 
 if (!qcrypto_hmac_supports(alg)) {
@@ -110,16 +110,11 @@ QCryptoHmac *qcrypto_hmac_new(QCryptoHashAlgorithm alg,
 return NULL;
 }
 
-hmac = g_new0(QCryptoHmac, 1);
-hmac->alg = alg;
-
 ctx = g_new0(QCryptoHmacNettle, 1);
 
 qcrypto_hmac_alg_map[alg].setkey(&ctx->u, nkey, key);
 
-hmac->opaque = ctx;
-
-return hmac;
+return ctx;
 }
 
 void qcrypto_hmac_free(QCryptoHmac *hmac)
@@ -173,3 +168,22 @@ int qcrypto_hmac_bytesv(QCryptoHmac *hmac,
 
 return 0;
 }
+
+QCryptoHmac *qcrypto_hmac_new(QCryptoHashAlgorithm alg,
+  const uint8_t *key, size_t nkey,
+  Error **errp)
+{
+QCryptoHmac *hmac;
+QCryptoHmacNettle *ctx;
+
+ctx = qcrypto_hmac_ctx_new(alg, key, nkey, errp);
+if (!ctx) {
+return NULL;
+}
+
+hmac = g_new0(QCryptoHmac, 1);
+hmac->alg = alg;
+hmac->opaque = ctx;
+
+return hmac;
+}
-- 
1.8.3.1

[Qemu-devel] [PATCH v6 07/18] crypto: hmac: move crypto/hmac.h into include/crypto/

From: "Longpeng(Mike)" 

Moves crypto/hmac.h into include/crypto/, likes cipher.h and hash.h

Reviewed-by: Daniel P. Berrange 
Reviewed-by: Philippe Mathieu-Daudé 
Reviewed-by: Gonglei 
Signed-off-by: Longpeng(Mike) 
---
 crypto/hmac.h | 166 --
 include/crypto/hmac.h | 166 ++
 2 files changed, 166 insertions(+), 166 deletions(-)
 delete mode 100644 crypto/hmac.h
 create mode 100644 include/crypto/hmac.h

diff --git a/crypto/hmac.h b/crypto/hmac.h
deleted file mode 100644
index 0d3acd7..000
--- a/crypto/hmac.h
+++ /dev/null
@@ -1,166 +0,0 @@
-/*
- * QEMU Crypto hmac algorithms
- *
- * Copyright (c) 2016 HUAWEI TECHNOLOGIES CO., LTD.
- *
- * This work is licensed under the terms of the GNU GPL, version 2 or
- * (at your option) any later version.  See the COPYING file in the
- * top-level directory.
- *
- */
-
-#ifndef QCRYPTO_HMAC_H
-#define QCRYPTO_HMAC_H
-
-#include "qapi-types.h"
-
-typedef struct QCryptoHmac QCryptoHmac;
-struct QCryptoHmac {
-QCryptoHashAlgorithm alg;
-void *opaque;
-};
-
-/**
- * qcrypto_hmac_supports:
- * @alg: the hmac algorithm
- *
- * Determine if @alg hmac algorithm is supported by
- * the current configured build
- *
- * Returns:
- *  true if the algorithm is supported, false otherwise
- */
-bool qcrypto_hmac_supports(QCryptoHashAlgorithm alg);
-
-/**
- * qcrypto_hmac_new:
- * @alg: the hmac algorithm
- * @key: the key bytes
- * @nkey: the length of @key
- * @errp: pointer to a NULL-initialized error object
- *
- * Creates a new hmac object with the algorithm @alg
- *
- * The @key parameter provides the bytes representing
- * the secret key to use. The @nkey parameter specifies
- * the length of @key in bytes
- *
- * Note: must use qcrypto_hmac_free() to release the
- * returned hmac object when no longer required
- *
- * Returns:
- *  a new hmac object, or NULL on error
- */
-QCryptoHmac *qcrypto_hmac_new(QCryptoHashAlgorithm alg,
-  const uint8_t *key, size_t nkey,
-  Error **errp);
-
-/**
- * qcrypto_hmac_free:
- * @hmac: the hmac object
- *
- * Release the memory associated with @hmac that was
- * previously allocated by qcrypto_hmac_new()
- */
-void qcrypto_hmac_free(QCryptoHmac *hmac);
-
-/**
- * qcrypto_hmac_bytesv:
- * @hmac: the hmac object
- * @iov: the array of memory regions to hmac
- * @niov: the length of @iov
- * @result: pointer to hold output hmac
- * @resultlen: pointer to hold length of @result
- * @errp: pointer to a NULL-initialized error object
- *
- * Computes the hmac across all the memory regions
- * present in @iov. The @result pointer will be
- * filled with raw bytes representing the computed
- * hmac, which will have length @resultlen. The
- * memory pointer in @result must be released
- * with a call to g_free() when no longer required.
- *
- * Returns:
- *  0 on success, -1 on error
- */
-int qcrypto_hmac_bytesv(QCryptoHmac *hmac,
-const struct iovec *iov,
-size_t niov,
-uint8_t **result,
-size_t *resultlen,
-Error **errp);
-
-/**
- * qcrypto_hmac_bytes:
- * @hmac: the hmac object
- * @buf: the memory region to hmac
- * @len: the length of @buf
- * @result: pointer to hold output hmac
- * @resultlen: pointer to hold length of @result
- * @errp: pointer to a NULL-initialized error object
- *
- * Computes the hmac across all the memory region
- * @buf of length @len. The @result pointer will be
- * filled with raw bytes representing the computed
- * hmac, which will have length @resultlen. The
- * memory pointer in @result must be released
- * with a call to g_free() when no longer required.
- *
- * Returns:
- *  0 on success, -1 on error
- */
-int qcrypto_hmac_bytes(QCryptoHmac *hmac,
-   const char *buf,
-   size_t len,
-   uint8_t **result,
-   size_t *resultlen,
-   Error **errp);
-
-/**
- * qcrypto_hmac_digestv:
- * @hmac: the hmac object
- * @iov: the array of memory regions to hmac
- * @niov: the length of @iov
- * @digest: pointer to hold output hmac
- * @errp: pointer to a NULL-initialized error object
- *
- * Computes the hmac across all the memory regions
- * present in @iov. The @digest pointer will be
- * filled with the printable hex digest of the computed
- * hmac, which will be terminated by '\0'. The
- * memory pointer in @digest must be released
- * with a call to g_free() when no longer required.
- *
- * Returns:
- *  0 on success, -1 on error
- */
-int qcrypto_hmac_digestv(QCryptoHmac *hmac,
- const struct iovec *iov,
- size_t niov,
- char **digest,
- Error **errp);
-
-/**
- * qcrypto_hmac_digest:
- * @hmac: the hmac object
- * @buf: the memory region

[Qemu-devel] [PATCH v6 00/18] crypto: add afalg-backend support

From: "Longpeng(Mike)" 

The AF_ALG socket family is the userspace interface for linux
crypto API, users can use it to access hardware accelerators.

This patchset adds a afalg-backend for qemu crypto subsystem. QEMU
would try to use afalg-backend first if configures '--enable-crypto
-afalg' and hostos support AF_ALG.

I measured the performance about the afalg-backend impls, I tested
how many data could be encrypted in 5 seconds.

NOTE: In some scenarios library-backend using CPU insns for crypto
  would be faster.

test-environment: Intel(R) Core(TM) i5-6500 CPU @ 3.20GHz

*sha256*
chunk_size(bytes)   MB/sec(afalg:sha256-ssse3)  MB/sec(nettle)
512 93.03   185.87
1024146.32  201.78
2048213.32  210.93
4096275.48  215.26
8192321.77  217.49
16384   349.60  219.26
32768   363.59  219.73
65536   375.79  219.99

*hmac(sha256)*
chunk_size(bytes)   MB/sec(afalg:sha256-ssse3)  MB/sec(nettle)
512 71.26   165.55
1024117.43  189.15
2048180.96  203.24
4096247.60  211.38
8192301.99  215.65
16384   340.79  218.22
32768   365.51  219.49
65536   377.92  220.24

*cbc(aes128)*
chunk_size(bytes)   MB/sec(afalg:cbc-aes-aesni)  MB/sec(nettle)
512 371.76   188.41
1024559.86   189.64
2048768.66   192.11
4096939.15   192.40
81921029.48  192.49
16384   1072.79  190.52
32768   1109.38  190.41
65536   1102.38  190.40

---
Changes since v5:
- use afalg-backend first only when configured with
  '--enable-crypto-afalg' and AF_ALG is supportted. 
  [Daniel]

Changes since v4:
- remove 'name' field in 'struct CryptoAFAlg'. [Daniel]
- add error handling for read() returning less than requested. [Daniel]
- use iov_send_recv to recv msg in hash-afalg.c. [Daniel]
- refactor hmac benchmark as suggestion. [Daniel]

Changes since v3:
- add "Reviewed-by: Daniel P. Berrange " in
  commit messages of PATCH 1/2/3/4/5/7/8/9/10/11.
- PATCH 12: use strlen() instead of qemu_strnlen() in 
qcrypto_afalg_build_saddr(). [Daniel]
- PATCH 12: rather than indenting the entire method, just return immediately
if afalg=NULL. [Daniel]
- PATCH 13: use g_strdup_printf() instead of g_new0+snprintf() and remove
redundant bounds check in qcrypto_afalg_cipher_format_name(). 
[Daniel]
- PATCH 13: s/except_niv/expect_niv  s/origin_contorllen/origin_controllen. 
[Daniel]
- PATCH 13: use '%zu' to print 'size_t' in qcrypto_afalg_cipher_setiv(). 
[Daniel]
- PATCH 13: remove qcrypto_cipher_using_afalg_drv(). [Daniel]
- PATCH 13: refactor the qcrypto_cipher_new() as Daniel's suggestion. 
[Daniel]
- PATCH 13: correct the ->cmsg initialization int 
qcrypto_afalg_cipher_ctx_new() to
avoid different behaviour in test_cipher_null_iv(). [Daniel]
- PATCH 14: use g_strdup_printf() instead of g_new0+snprintf() and remove
edundant bounds check in qcrypto_afalg_hash_format_name(). 
[Daniel]
- PATCH 14: s/except_len/expect_len. [Daniel]
- PATCH 14: free 'errp' if afalg_driver.hash_bytesv() failed. [Daniel]
- PATCH 14: maybe some afalg errors should be treated as fatal, but we
have no idea yet, so add a "TODO" comment.
- PATCH 15: refactor the qcrypto_hmac_new() as Daniel's suggestion. [Daniel]

Changes since v2:
- init sockaddr_alg object when it's defined. [Gonglei]
- fix some superfluous initialization. [Gonglei]
- s/opeartion/operation/g in crypto/afalgpriv.h. [Gonglei]
- check 'niv' in qcrypto_afalg_cipher_setiv. [Gonglei]

Changes since v1:
- use "make check-speed" to testing the performance. [Daniel]
- put private definations into crypto/***priv.h. [Daniel]
- remove afalg socket from qapi-schema, put them into crypto/. [Daniel]
- some Error report change. [Daniel]
- s/QCryptoAfalg/QCryptoAFAlg. [Daniel]
- use snprintf with bounds checking instead of sprintf. [Daniel]
- use "qcrypto_afalg_" prefix and "qcrypto_nettle(gcrypt,glib,builtin)_" 
prefix. [Daniel]
- add testing results in cover-letter. [Gonglei]

---
Longpeng(Mike) (18):
  crypto: cipher: introduce context free function
  crypto: cipher: introduce qcrypto_cipher_ctx_new for gcrypt-backend
  crypt

[Qemu-devel] [PATCH v6 04/18] crypto: cipher: introduce qcrypto_cipher_ctx_new for builtin-backend

From: "Longpeng(Mike)" 

Extracts qcrypto_cipher_ctx_new() from qcrypto_cipher_new() for
builtin-backend impls.

Reviewed-by: Daniel P. Berrange 
Reviewed-by: Gonglei 
Signed-off-by: Longpeng(Mike) 
---
 crypto/cipher-builtin.c | 101 ++--
 1 file changed, 55 insertions(+), 46 deletions(-)

diff --git a/crypto/cipher-builtin.c b/crypto/cipher-builtin.c
index b4bc2b9..4ecd15e 100644
--- a/crypto/cipher-builtin.c
+++ b/crypto/cipher-builtin.c
@@ -235,23 +235,24 @@ static int qcrypto_cipher_setiv_aes(QCryptoCipher *cipher,
 
 
 
-static int qcrypto_cipher_init_aes(QCryptoCipher *cipher,
-   const uint8_t *key, size_t nkey,
-   Error **errp)
+static QCryptoCipherBuiltin *
+qcrypto_cipher_init_aes(QCryptoCipherMode mode,
+const uint8_t *key, size_t nkey,
+Error **errp)
 {
 QCryptoCipherBuiltin *ctxt;
 
-if (cipher->mode != QCRYPTO_CIPHER_MODE_CBC &&
-cipher->mode != QCRYPTO_CIPHER_MODE_ECB &&
-cipher->mode != QCRYPTO_CIPHER_MODE_XTS) {
+if (mode != QCRYPTO_CIPHER_MODE_CBC &&
+mode != QCRYPTO_CIPHER_MODE_ECB &&
+mode != QCRYPTO_CIPHER_MODE_XTS) {
 error_setg(errp, "Unsupported cipher mode %s",
-   QCryptoCipherMode_lookup[cipher->mode]);
-return -1;
+   QCryptoCipherMode_lookup[mode]);
+return NULL;
 }
 
 ctxt = g_new0(QCryptoCipherBuiltin, 1);
 
-if (cipher->mode == QCRYPTO_CIPHER_MODE_XTS) {
+if (mode == QCRYPTO_CIPHER_MODE_XTS) {
 if (AES_set_encrypt_key(key, nkey * 4, &ctxt->state.aes.key.enc) != 0) 
{
 error_setg(errp, "Failed to set encryption key");
 goto error;
@@ -291,13 +292,11 @@ static int qcrypto_cipher_init_aes(QCryptoCipher *cipher,
 ctxt->encrypt = qcrypto_cipher_encrypt_aes;
 ctxt->decrypt = qcrypto_cipher_decrypt_aes;
 
-cipher->opaque = ctxt;
-
-return 0;
+return ctxt;
 
  error:
 g_free(ctxt);
-return -1;
+return NULL;
 }
 
 
@@ -370,16 +369,17 @@ static int qcrypto_cipher_setiv_des_rfb(QCryptoCipher 
*cipher,
 }
 
 
-static int qcrypto_cipher_init_des_rfb(QCryptoCipher *cipher,
-   const uint8_t *key, size_t nkey,
-   Error **errp)
+static QCryptoCipherBuiltin *
+qcrypto_cipher_init_des_rfb(QCryptoCipherMode mode,
+const uint8_t *key, size_t nkey,
+Error **errp)
 {
 QCryptoCipherBuiltin *ctxt;
 
-if (cipher->mode != QCRYPTO_CIPHER_MODE_ECB) {
+if (mode != QCRYPTO_CIPHER_MODE_ECB) {
 error_setg(errp, "Unsupported cipher mode %s",
-   QCryptoCipherMode_lookup[cipher->mode]);
-return -1;
+   QCryptoCipherMode_lookup[mode]);
+return NULL;
 }
 
 ctxt = g_new0(QCryptoCipherBuiltin, 1);
@@ -394,9 +394,7 @@ static int qcrypto_cipher_init_des_rfb(QCryptoCipher 
*cipher,
 ctxt->encrypt = qcrypto_cipher_encrypt_des_rfb;
 ctxt->decrypt = qcrypto_cipher_decrypt_des_rfb;
 
-cipher->opaque = ctxt;
-
-return 0;
+return ctxt;
 }
 
 
@@ -426,12 +424,13 @@ bool qcrypto_cipher_supports(QCryptoCipherAlgorithm alg,
 }
 
 
-QCryptoCipher *qcrypto_cipher_new(QCryptoCipherAlgorithm alg,
-  QCryptoCipherMode mode,
-  const uint8_t *key, size_t nkey,
-  Error **errp)
+static QCryptoCipherBuiltin *qcrypto_cipher_ctx_new(QCryptoCipherAlgorithm alg,
+QCryptoCipherMode mode,
+const uint8_t *key,
+size_t nkey,
+Error **errp)
 {
-QCryptoCipher *cipher;
+QCryptoCipherBuiltin *ctxt;
 
 switch (mode) {
 case QCRYPTO_CIPHER_MODE_ECB:
@@ -444,39 +443,27 @@ QCryptoCipher *qcrypto_cipher_new(QCryptoCipherAlgorithm 
alg,
 return NULL;
 }
 
-cipher = g_new0(QCryptoCipher, 1);
-cipher->alg = alg;
-cipher->mode = mode;
-
 if (!qcrypto_cipher_validate_key_length(alg, mode, nkey, errp)) {
-goto error;
+return NULL;
 }
 
-switch (cipher->alg) {
+switch (alg) {
 case QCRYPTO_CIPHER_ALG_DES_RFB:
-if (qcrypto_cipher_init_des_rfb(cipher, key, nkey, errp) < 0) {
-goto error;
-}
+ctxt = qcrypto_cipher_init_des_rfb(mode, key, nkey, errp);
 break;
 case QCRYPTO_CIPHER_ALG_AES_128:
 case QCRYPTO_CIPHER_ALG_AES_192:
 case QCRYPTO_CIPHER_ALG_AES_256:
-if (qcrypto_cipher_init_aes(cipher, key, nkey, errp) < 0) {
-goto error;
-}
+ctxt = qcrypto_cipher_init_aes(mode, key, nkey, errp);
 break;
 default:
 error_setg(errp,

[Qemu-devel] [PATCH v6 03/18] crypto: cipher: introduce qcrypto_cipher_ctx_new for nettle-backend

From: "Longpeng(Mike)" 

Extracts qcrypto_cipher_ctx_new() from qcrypto_cipher_new() for
nettle-backend impls.

Reviewed-by: Daniel P. Berrange 
Reviewed-by: Gonglei 
Signed-off-by: Longpeng(Mike) 
---
 crypto/cipher-nettle.c | 41 +
 1 file changed, 29 insertions(+), 12 deletions(-)

diff --git a/crypto/cipher-nettle.c b/crypto/cipher-nettle.c
index e04e3a1..e6d6e6c 100644
--- a/crypto/cipher-nettle.c
+++ b/crypto/cipher-nettle.c
@@ -262,12 +262,12 @@ static void nettle_cipher_free_ctx(QCryptoCipherNettle 
*ctx)
 }
 
 
-QCryptoCipher *qcrypto_cipher_new(QCryptoCipherAlgorithm alg,
-  QCryptoCipherMode mode,
-  const uint8_t *key, size_t nkey,
-  Error **errp)
+static QCryptoCipherNettle *qcrypto_cipher_ctx_new(QCryptoCipherAlgorithm alg,
+   QCryptoCipherMode mode,
+   const uint8_t *key,
+   size_t nkey,
+   Error **errp)
 {
-QCryptoCipher *cipher;
 QCryptoCipherNettle *ctx;
 uint8_t *rfbkey;
 
@@ -287,12 +287,7 @@ QCryptoCipher *qcrypto_cipher_new(QCryptoCipherAlgorithm 
alg,
 return NULL;
 }
 
-cipher = g_new0(QCryptoCipher, 1);
-cipher->alg = alg;
-cipher->mode = mode;
-
 ctx = g_new0(QCryptoCipherNettle, 1);
-cipher->opaque = ctx;
 
 switch (alg) {
 case QCRYPTO_CIPHER_ALG_DES_RFB:
@@ -436,10 +431,10 @@ QCryptoCipher *qcrypto_cipher_new(QCryptoCipherAlgorithm 
alg,
 
 ctx->iv = g_new0(uint8_t, ctx->blocksize);
 
-return cipher;
+return ctx;
 
  error:
-qcrypto_cipher_free(cipher);
+nettle_cipher_free_ctx(ctx);
 return NULL;
 }
 
@@ -561,3 +556,25 @@ int qcrypto_cipher_setiv(QCryptoCipher *cipher,
 memcpy(ctx->iv, iv, niv);
 return 0;
 }
+
+
+QCryptoCipher *qcrypto_cipher_new(QCryptoCipherAlgorithm alg,
+  QCryptoCipherMode mode,
+  const uint8_t *key, size_t nkey,
+  Error **errp)
+{
+QCryptoCipher *cipher;
+QCryptoCipherNettle *ctx;
+
+ctx = qcrypto_cipher_ctx_new(alg, mode, key, nkey, errp);
+if (!ctx) {
+return NULL;
+}
+
+cipher = g_new0(QCryptoCipher, 1);
+cipher->alg = alg;
+cipher->mode = mode;
+cipher->opaque = ctx;
+
+return cipher;
+}
-- 
1.8.3.1

[Qemu-devel] [PATCH v6 06/18] crypto: hash: add hash driver framework

From: "Longpeng(Mike)" 

1) makes the public APIs in hash-nettle/gcrypt/glib static,
   and rename them with "nettle/gcrypt/glib" prefix.

2) introduces hash framework, including QCryptoHashDriver
   and new public APIs.

Reviewed-by: Daniel P. Berrange 
Reviewed-by: Gonglei 
Signed-off-by: Longpeng(Mike) 
---
 crypto/hash-gcrypt.c | 19 +--
 crypto/hash-glib.c   | 19 +--
 crypto/hash-nettle.c | 19 +--
 crypto/hash.c| 13 +
 crypto/hashpriv.h| 31 +++
 5 files changed, 83 insertions(+), 18 deletions(-)
 create mode 100644 crypto/hashpriv.h

diff --git a/crypto/hash-gcrypt.c b/crypto/hash-gcrypt.c
index 7690690..972beaa 100644
--- a/crypto/hash-gcrypt.c
+++ b/crypto/hash-gcrypt.c
@@ -22,6 +22,7 @@
 #include 
 #include "qapi/error.h"
 #include "crypto/hash.h"
+#include "hashpriv.h"
 
 
 static int qcrypto_hash_alg_map[QCRYPTO_HASH_ALG__MAX] = {
@@ -44,12 +45,13 @@ gboolean qcrypto_hash_supports(QCryptoHashAlgorithm alg)
 }
 
 
-int qcrypto_hash_bytesv(QCryptoHashAlgorithm alg,
-const struct iovec *iov,
-size_t niov,
-uint8_t **result,
-size_t *resultlen,
-Error **errp)
+static int
+qcrypto_gcrypt_hash_bytesv(QCryptoHashAlgorithm alg,
+   const struct iovec *iov,
+   size_t niov,
+   uint8_t **result,
+   size_t *resultlen,
+   Error **errp)
 {
 int i, ret;
 gcry_md_hd_t md;
@@ -107,3 +109,8 @@ int qcrypto_hash_bytesv(QCryptoHashAlgorithm alg,
 gcry_md_close(md);
 return -1;
 }
+
+
+QCryptoHashDriver qcrypto_hash_lib_driver = {
+.hash_bytesv = qcrypto_gcrypt_hash_bytesv,
+};
diff --git a/crypto/hash-glib.c b/crypto/hash-glib.c
index ec99ac9..a5871cc 100644
--- a/crypto/hash-glib.c
+++ b/crypto/hash-glib.c
@@ -21,6 +21,7 @@
 #include "qemu/osdep.h"
 #include "qapi/error.h"
 #include "crypto/hash.h"
+#include "hashpriv.h"
 
 
 static int qcrypto_hash_alg_map[QCRYPTO_HASH_ALG__MAX] = {
@@ -47,12 +48,13 @@ gboolean qcrypto_hash_supports(QCryptoHashAlgorithm alg)
 }
 
 
-int qcrypto_hash_bytesv(QCryptoHashAlgorithm alg,
-const struct iovec *iov,
-size_t niov,
-uint8_t **result,
-size_t *resultlen,
-Error **errp)
+static int
+qcrypto_glib_hash_bytesv(QCryptoHashAlgorithm alg,
+ const struct iovec *iov,
+ size_t niov,
+ uint8_t **result,
+ size_t *resultlen,
+ Error **errp)
 {
 int i, ret;
 GChecksum *cs;
@@ -95,3 +97,8 @@ int qcrypto_hash_bytesv(QCryptoHashAlgorithm alg,
 g_checksum_free(cs);
 return -1;
 }
+
+
+QCryptoHashDriver qcrypto_hash_lib_driver = {
+.hash_bytesv = qcrypto_glib_hash_bytesv,
+};
diff --git a/crypto/hash-nettle.c b/crypto/hash-nettle.c
index 6a206dc..96f186f 100644
--- a/crypto/hash-nettle.c
+++ b/crypto/hash-nettle.c
@@ -21,6 +21,7 @@
 #include "qemu/osdep.h"
 #include "qapi/error.h"
 #include "crypto/hash.h"
+#include "hashpriv.h"
 #include 
 #include 
 #include 
@@ -103,12 +104,13 @@ gboolean qcrypto_hash_supports(QCryptoHashAlgorithm alg)
 }
 
 
-int qcrypto_hash_bytesv(QCryptoHashAlgorithm alg,
-const struct iovec *iov,
-size_t niov,
-uint8_t **result,
-size_t *resultlen,
-Error **errp)
+static int
+qcrypto_nettle_hash_bytesv(QCryptoHashAlgorithm alg,
+   const struct iovec *iov,
+   size_t niov,
+   uint8_t **result,
+   size_t *resultlen,
+   Error **errp)
 {
 int i;
 union qcrypto_hash_ctx ctx;
@@ -152,3 +154,8 @@ int qcrypto_hash_bytesv(QCryptoHashAlgorithm alg,
 
 return 0;
 }
+
+
+QCryptoHashDriver qcrypto_hash_lib_driver = {
+.hash_bytesv = qcrypto_nettle_hash_bytesv,
+};
diff --git a/crypto/hash.c b/crypto/hash.c
index 0f1ceac..c43fd87 100644
--- a/crypto/hash.c
+++ b/crypto/hash.c
@@ -21,6 +21,7 @@
 #include "qemu/osdep.h"
 #include "qapi/error.h"
 #include "crypto/hash.h"
+#include "hashpriv.h"
 
 static size_t qcrypto_hash_alg_size[QCRYPTO_HASH_ALG__MAX] = {
 [QCRYPTO_HASH_ALG_MD5] = 16,
@@ -38,6 +39,18 @@ size_t qcrypto_hash_digest_len(QCryptoHashAlgorithm alg)
 return qcrypto_hash_alg_size[alg];
 }
 
+int qcrypto_hash_bytesv(QCryptoHashAlgorithm alg,
+const struct iovec *iov,
+size_t niov,
+uint8_t **result,
+size_t *resultlen,
+Error **errp)
+{
+return qcrypto_hash_lib_driver.hash_bytesv(

[Qemu-devel] [PATCH v6 01/18] crypto: cipher: introduce context free function

From: "Longpeng(Mike)" 

Refactors the qcrypto_cipher_free(), splits it into two parts. One
is gcrypt/nettle__cipher_free_ctx() to free the special context.

This makes code more clear, what's more, it would be used by the
later patch.

Reviewed-by: Daniel P. Berrange 
Reviewed-by: Gonglei 
Signed-off-by: Longpeng(Mike) 
---
 crypto/cipher-gcrypt.c | 31 ++-
 crypto/cipher-nettle.c | 18 ++
 2 files changed, 32 insertions(+), 17 deletions(-)

diff --git a/crypto/cipher-gcrypt.c b/crypto/cipher-gcrypt.c
index 6487eca..0ecffa2 100644
--- a/crypto/cipher-gcrypt.c
+++ b/crypto/cipher-gcrypt.c
@@ -64,6 +64,22 @@ struct QCryptoCipherGcrypt {
 uint8_t *iv;
 };
 
+static void gcrypt_cipher_free_ctx(QCryptoCipherGcrypt *ctx,
+   QCryptoCipherMode mode)
+{
+if (!ctx) {
+return;
+}
+
+gcry_cipher_close(ctx->handle);
+if (mode == QCRYPTO_CIPHER_MODE_XTS) {
+gcry_cipher_close(ctx->tweakhandle);
+}
+g_free(ctx->iv);
+g_free(ctx);
+}
+
+
 QCryptoCipher *qcrypto_cipher_new(QCryptoCipherAlgorithm alg,
   QCryptoCipherMode mode,
   const uint8_t *key, size_t nkey,
@@ -228,11 +244,7 @@ QCryptoCipher *qcrypto_cipher_new(QCryptoCipherAlgorithm 
alg,
 return cipher;
 
  error:
-gcry_cipher_close(ctx->handle);
-if (cipher->mode == QCRYPTO_CIPHER_MODE_XTS) {
-gcry_cipher_close(ctx->tweakhandle);
-}
-g_free(ctx);
+gcrypt_cipher_free_ctx(ctx, mode);
 g_free(cipher);
 return NULL;
 }
@@ -240,17 +252,10 @@ QCryptoCipher *qcrypto_cipher_new(QCryptoCipherAlgorithm 
alg,
 
 void qcrypto_cipher_free(QCryptoCipher *cipher)
 {
-QCryptoCipherGcrypt *ctx;
 if (!cipher) {
 return;
 }
-ctx = cipher->opaque;
-gcry_cipher_close(ctx->handle);
-if (cipher->mode == QCRYPTO_CIPHER_MODE_XTS) {
-gcry_cipher_close(ctx->tweakhandle);
-}
-g_free(ctx->iv);
-g_free(ctx);
+gcrypt_cipher_free_ctx(cipher->opaque, cipher->mode);
 g_free(cipher);
 }
 
diff --git a/crypto/cipher-nettle.c b/crypto/cipher-nettle.c
index dfc9030..e04e3a1 100644
--- a/crypto/cipher-nettle.c
+++ b/crypto/cipher-nettle.c
@@ -249,6 +249,19 @@ bool qcrypto_cipher_supports(QCryptoCipherAlgorithm alg,
 }
 
 
+static void nettle_cipher_free_ctx(QCryptoCipherNettle *ctx)
+{
+if (!ctx) {
+return;
+}
+
+g_free(ctx->iv);
+g_free(ctx->ctx);
+g_free(ctx->ctx_tweak);
+g_free(ctx);
+}
+
+
 QCryptoCipher *qcrypto_cipher_new(QCryptoCipherAlgorithm alg,
   QCryptoCipherMode mode,
   const uint8_t *key, size_t nkey,
@@ -440,10 +453,7 @@ void qcrypto_cipher_free(QCryptoCipher *cipher)
 }
 
 ctx = cipher->opaque;
-g_free(ctx->iv);
-g_free(ctx->ctx);
-g_free(ctx->ctx_tweak);
-g_free(ctx);
+nettle_cipher_free_ctx(ctx);
 g_free(cipher);
 }
 
-- 
1.8.3.1

[Qemu-devel] [PATCH v6 02/18] crypto: cipher: introduce qcrypto_cipher_ctx_new for gcrypt-backend

From: "Longpeng(Mike)" 

Extracts qcrypto_cipher_ctx_new() from qcrypto_cipher_new() for
gcrypt-backend impls.

Reviewed-by: Daniel P. Berrange 
Reviewed-by: Gonglei 
Signed-off-by: Longpeng(Mike) 
---
 crypto/cipher-gcrypt.c | 50 +-
 1 file changed, 33 insertions(+), 17 deletions(-)

diff --git a/crypto/cipher-gcrypt.c b/crypto/cipher-gcrypt.c
index 0ecffa2..4939386 100644
--- a/crypto/cipher-gcrypt.c
+++ b/crypto/cipher-gcrypt.c
@@ -80,12 +80,12 @@ static void gcrypt_cipher_free_ctx(QCryptoCipherGcrypt *ctx,
 }
 
 
-QCryptoCipher *qcrypto_cipher_new(QCryptoCipherAlgorithm alg,
-  QCryptoCipherMode mode,
-  const uint8_t *key, size_t nkey,
-  Error **errp)
+static QCryptoCipherGcrypt *qcrypto_cipher_ctx_new(QCryptoCipherAlgorithm alg,
+   QCryptoCipherMode mode,
+   const uint8_t *key,
+   size_t nkey,
+   Error **errp)
 {
-QCryptoCipher *cipher;
 QCryptoCipherGcrypt *ctx;
 gcry_error_t err;
 int gcryalg, gcrymode;
@@ -162,10 +162,6 @@ QCryptoCipher *qcrypto_cipher_new(QCryptoCipherAlgorithm 
alg,
 return NULL;
 }
 
-cipher = g_new0(QCryptoCipher, 1);
-cipher->alg = alg;
-cipher->mode = mode;
-
 ctx = g_new0(QCryptoCipherGcrypt, 1);
 
 err = gcry_cipher_open(&ctx->handle, gcryalg, gcrymode, 0);
@@ -174,7 +170,7 @@ QCryptoCipher *qcrypto_cipher_new(QCryptoCipherAlgorithm 
alg,
gcry_strerror(err));
 goto error;
 }
-if (cipher->mode == QCRYPTO_CIPHER_MODE_XTS) {
+if (mode == QCRYPTO_CIPHER_MODE_XTS) {
 err = gcry_cipher_open(&ctx->tweakhandle, gcryalg, gcrymode, 0);
 if (err != 0) {
 error_setg(errp, "Cannot initialize cipher: %s",
@@ -183,7 +179,7 @@ QCryptoCipher *qcrypto_cipher_new(QCryptoCipherAlgorithm 
alg,
 }
 }
 
-if (cipher->alg == QCRYPTO_CIPHER_ALG_DES_RFB) {
+if (alg == QCRYPTO_CIPHER_ALG_DES_RFB) {
 /* We're using standard DES cipher from gcrypt, so we need
  * to munge the key so that the results are the same as the
  * bizarre RFB variant of DES :-)
@@ -193,7 +189,7 @@ QCryptoCipher *qcrypto_cipher_new(QCryptoCipherAlgorithm 
alg,
 g_free(rfbkey);
 ctx->blocksize = 8;
 } else {
-if (cipher->mode == QCRYPTO_CIPHER_MODE_XTS) {
+if (mode == QCRYPTO_CIPHER_MODE_XTS) {
 nkey /= 2;
 err = gcry_cipher_setkey(ctx->handle, key, nkey);
 if (err != 0) {
@@ -210,7 +206,7 @@ QCryptoCipher *qcrypto_cipher_new(QCryptoCipherAlgorithm 
alg,
gcry_strerror(err));
 goto error;
 }
-switch (cipher->alg) {
+switch (alg) {
 case QCRYPTO_CIPHER_ALG_AES_128:
 case QCRYPTO_CIPHER_ALG_AES_192:
 case QCRYPTO_CIPHER_ALG_AES_256:
@@ -230,7 +226,7 @@ QCryptoCipher *qcrypto_cipher_new(QCryptoCipherAlgorithm 
alg,
 }
 }
 
-if (cipher->mode == QCRYPTO_CIPHER_MODE_XTS) {
+if (mode == QCRYPTO_CIPHER_MODE_XTS) {
 if (ctx->blocksize != XTS_BLOCK_SIZE) {
 error_setg(errp,
"Cipher block size %zu must equal XTS block size %d",
@@ -240,12 +236,10 @@ QCryptoCipher *qcrypto_cipher_new(QCryptoCipherAlgorithm 
alg,
 ctx->iv = g_new0(uint8_t, ctx->blocksize);
 }
 
-cipher->opaque = ctx;
-return cipher;
+return ctx;
 
  error:
 gcrypt_cipher_free_ctx(ctx, mode);
-g_free(cipher);
 return NULL;
 }
 
@@ -385,3 +379,25 @@ int qcrypto_cipher_setiv(QCryptoCipher *cipher,
 
 return 0;
 }
+
+
+QCryptoCipher *qcrypto_cipher_new(QCryptoCipherAlgorithm alg,
+  QCryptoCipherMode mode,
+  const uint8_t *key, size_t nkey,
+  Error **errp)
+{
+QCryptoCipher *cipher;
+QCryptoCipherGcrypt *ctx;
+
+ctx = qcrypto_cipher_ctx_new(alg, mode, key, nkey, errp);
+if (!ctx) {
+return NULL;
+}
+
+cipher = g_new0(QCryptoCipher, 1);
+cipher->alg = alg;
+cipher->mode = mode;
+cipher->opaque = ctx;
+
+return cipher;
+}
-- 
1.8.3.1

Re: [Qemu-devel] [PATCH v13 18/26] target/arm: [tcg] Port to breakpoint_check


On 07/14/2017 07:26 AM, Richard Henderson wrote:

On 07/13/2017 11:26 PM, Lluís Vilanova wrote:

Incrementally paves the way towards using the generic instruction translation
loop.

Signed-off-by: Lluís Vilanova 
Reviewed-by: Richard Henderson 
---
  target/arm/translate.c |   53 +++-
  1 file changed, 34 insertions(+), 19 deletions(-)

diff --git a/target/arm/translate.c b/target/arm/translate.c
index b9183fc511..55bef09739 100644
--- a/target/arm/translate.c
+++ b/target/arm/translate.c
@@ -11917,6 +11917,33 @@ static void arm_tr_insn_start(DisasContextBase 
*dcbase, CPUState *cpu)

  #endif
  }
+static bool arm_tr_breakpoint_check(DisasContextBase *dcbase, CPUState *cpu,
+const CPUBreakpoint *bp)
+{
+DisasContext *dc = container_of(dcbase, DisasContext, base);
+
+if (bp->flags & BP_CPU) {
+gen_set_condexec(dc);
+gen_set_pc_im(dc, dc->pc);
+gen_helper_check_breakpoints(cpu_env);
+/* End the TB early; it's likely not going to be executed */
+dc->base.is_jmp = DISAS_UPDATE;
+} else {
+gen_exception_internal_insn(dc, 0, EXCP_DEBUG);
+/* The address covered by the breakpoint must be
+   included in [tb->pc, tb->pc + tb->size) in order
+   to for it to be properly cleared -- thus we
+   increment the PC here so that the logic setting
+   tb->size below does the right thing.  */
+/* TODO: Advance PC by correct instruction length to
+ * avoid disassembler error messages */
+dc->pc += 2;
+dc->base.is_jmp = DISAS_NORETURN;
+}
+
+return true;
+}
+
  /* generate intermediate code for basic block 'tb'.  */
  void gen_intermediate_code(CPUState *cs, TranslationBlock *tb)
  {
@@ -11965,28 +11992,16 @@ void gen_intermediate_code(CPUState *cs, 
TranslationBlock *tb)

  if (unlikely(!QTAILQ_EMPTY(&cs->breakpoints))) {
  CPUBreakpoint *bp;
  QTAILQ_FOREACH(bp, &cs->breakpoints, entry) {
-if (bp->pc == dc->pc) {
-if (bp->flags & BP_CPU) {
-gen_set_condexec(dc);
-gen_set_pc_im(dc, dc->pc);
-gen_helper_check_breakpoints(cpu_env);
-/* End the TB early; it's likely not going to be 
executed */

-dc->base.is_jmp = DISAS_UPDATE;


Oh I see what you're doing there in the main loop.
And I see that you're copying existing behaviour.

That said, I do wonder if there's a better way.

Looking back at the original patch (5d98bf8f), there do not
seem to have been any other side effects intended; simply
"single step" any insn for which this bp condition is met.

Another way to handle this would be if we could adjust max_insns = num_insns.
That would cause the loop to exit after the current insn, with DISAS_TOO_MANY 
if nothing else.


Another possibility is is_jmp = DISAS_TOO_MANY, and exit the translation loop 
after the breakpoint check only for is_jmp > DISAS_TOO_MANY.  That allows all 
of the DISAS_TARGET_N values to exit as well.



r~

Re: [Qemu-devel] qapi: Stop abusing "special" values for something entirely different

On 07/14/2017 12:12 PM, Markus Armbruster wrote:
> 
> Instead of the last part, I prefer either
> 
> * so we add a *new* value, such as JSON null.

I like that idea.

> 
> 1. Stop abusing values the schema accepts, but are invalid to mean "do
> something else entirely".
> 
> 2. Add a first class null type to QAPI.
> 
> 3. Turn MigrationParameters members tls-creds and tls-hostname into
> alternate of str and null.  Deprecate "".
> 
> 4. Add a null member to alternate BlockdefRef.  Deprecate "".

Back-compat concerns: would we still accept "" in place of null for a
release or two?  Is it time to figure out how to add deprecation
notices/events to QMP?  Or would this be a clean break-over point (since
introspection exists), where if introspection shows there is an
alternate between string and null, then libvirt MUST use null instead of
"" to get the desired semantics?

> 
> I got patches for 2., and I intend to work on 3. and 4.
> 
> Since this is "only" about "less than general and ugly", we may decide
> to leave things as they are if my patches turn out even uglier.
> 
> Meanwhile, opinions?

Not much time left for soft freeze (which kind of echoes the dilemma we
had at 2.9).  Is this something you are aiming for in 2.10, or will it
be all the harder to worry about back-compat (because we'll have two
releases rather than one before we introduce the alternate-with-null
semantics)?

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v13 22/26] target/arm: [tcg] Port to tb_stop


On 07/13/2017 11:42 PM, Lluís Vilanova wrote:

-dc->base.is_jmp = DISAS_SKIP;
+dc->base.is_jmp = DISAS_NORETURN;


Really should fold this back into the patch that introduced SKIP temporarily.


r~

Re: [Qemu-devel] [PATCH v13 18/26] target/arm: [tcg] Port to breakpoint_check


On 07/13/2017 11:26 PM, Lluís Vilanova wrote:

Incrementally paves the way towards using the generic instruction translation
loop.

Signed-off-by: Lluís Vilanova 
Reviewed-by: Richard Henderson 
---
  target/arm/translate.c |   53 +++-
  1 file changed, 34 insertions(+), 19 deletions(-)

diff --git a/target/arm/translate.c b/target/arm/translate.c
index b9183fc511..55bef09739 100644
--- a/target/arm/translate.c
+++ b/target/arm/translate.c
@@ -11917,6 +11917,33 @@ static void arm_tr_insn_start(DisasContextBase 
*dcbase, CPUState *cpu)
  #endif
  }
  
+static bool arm_tr_breakpoint_check(DisasContextBase *dcbase, CPUState *cpu,

+const CPUBreakpoint *bp)
+{
+DisasContext *dc = container_of(dcbase, DisasContext, base);
+
+if (bp->flags & BP_CPU) {
+gen_set_condexec(dc);
+gen_set_pc_im(dc, dc->pc);
+gen_helper_check_breakpoints(cpu_env);
+/* End the TB early; it's likely not going to be executed */
+dc->base.is_jmp = DISAS_UPDATE;
+} else {
+gen_exception_internal_insn(dc, 0, EXCP_DEBUG);
+/* The address covered by the breakpoint must be
+   included in [tb->pc, tb->pc + tb->size) in order
+   to for it to be properly cleared -- thus we
+   increment the PC here so that the logic setting
+   tb->size below does the right thing.  */
+/* TODO: Advance PC by correct instruction length to
+ * avoid disassembler error messages */
+dc->pc += 2;
+dc->base.is_jmp = DISAS_NORETURN;
+}
+
+return true;
+}
+
  /* generate intermediate code for basic block 'tb'.  */
  void gen_intermediate_code(CPUState *cs, TranslationBlock *tb)
  {
@@ -11965,28 +11992,16 @@ void gen_intermediate_code(CPUState *cs, 
TranslationBlock *tb)
  if (unlikely(!QTAILQ_EMPTY(&cs->breakpoints))) {
  CPUBreakpoint *bp;
  QTAILQ_FOREACH(bp, &cs->breakpoints, entry) {
-if (bp->pc == dc->pc) {
-if (bp->flags & BP_CPU) {
-gen_set_condexec(dc);
-gen_set_pc_im(dc, dc->pc);
-gen_helper_check_breakpoints(cpu_env);
-/* End the TB early; it's likely not going to be 
executed */
-dc->base.is_jmp = DISAS_UPDATE;


Oh I see what you're doing there in the main loop.
And I see that you're copying existing behaviour.

That said, I do wonder if there's a better way.

Looking back at the original patch (5d98bf8f), there do not
seem to have been any other side effects intended; simply
"single step" any insn for which this bp condition is met.

Another way to handle this would be if we could adjust max_insns = num_insns.
That would cause the loop to exit after the current insn, with DISAS_TOO_MANY 
if nothing else.



r~

Re: [Qemu-devel] [PATCHv9 1/3] fw_cfg: switch fw_cfg_find() to locate the fw_cfg device by type rather than path

On Fri, Jul 14, 2017 at 10:40:06AM +0100, Mark Cave-Ayland wrote:
> This will enable the fw_cfg device to be placed anywhere within the QOM tree
> regardless of its machine location.
> 
> Note that we also add a comment to document the behaviour that we return NULL 
> to
> indicate failure where either no fw_cfg device or multiple fw_cfg devices are
> found.
> 
> Signed-off-by: Mark Cave-Ayland 

Reviewed-by: Eduardo Habkost 

-- 
Eduardo

Re: [Qemu-devel] [RFC 16/29] vhost+postcopy: Stash RAMBlock and offset

2017-07-14 Thread Dr. David Alan Gilbert

* Peter Xu (pet...@redhat.com) wrote:
> On Wed, Jun 28, 2017 at 08:00:34PM +0100, Dr. David Alan Gilbert (git) wrote:
> > From: "Dr. David Alan Gilbert" 
> > 
> > Stash the RAMBlock and offset for later use looking up
> > addresses.
> > 
> > Signed-off-by: Dr. David Alan Gilbert 
> > ---
> >  hw/virtio/trace-events |  1 +
> >  hw/virtio/vhost-user.c | 11 +++
> >  2 files changed, 12 insertions(+)
> > 
> > diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events
> > index f7be340a45..1fd194363a 100644
> > --- a/hw/virtio/trace-events
> > +++ b/hw/virtio/trace-events
> > @@ -3,6 +3,7 @@
> >  # hw/virtio/vhost-user.c
> >  vhost_user_postcopy_listen(void) ""
> >  vhost_user_set_mem_table_postcopy(uint64_t client_addr, uint64_t qhva, int 
> > reply_i, int region_i) "client:%"PRIx64" for hva: %"PRIx64" reply %d region 
> > %d"
> > +vhost_user_set_mem_table_withfd(int index, const char *name, uint64_t 
> > memory_size, uint64_t guest_phys_addr, uint64_t userspace_addr, uint64_t 
> > offset) "%d:%s: size:%"PRIx64" GPA:%"PRIx64" QVA/userspace:%"PRIx64" RB 
> > offset:%"PRIx64
> >  
> >  # hw/virtio/virtio.c
> >  virtqueue_alloc_element(void *elem, size_t sz, unsigned in_num, unsigned 
> > out_num) "elem %p size %zd in_num %u out_num %u"
> > diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
> > index 6be3e7ff2d..3185af7a45 100644
> > --- a/hw/virtio/vhost-user.c
> > +++ b/hw/virtio/vhost-user.c
> > @@ -133,6 +133,11 @@ struct vhost_user {
> >  NotifierWithReturn postcopy_notifier;
> >  struct PostCopyFD  postcopy_fd;
> >  uint64_t   postcopy_client_bases[VHOST_MEMORY_MAX_NREGIONS];
> > +RAMBlock  *region_rb[VHOST_MEMORY_MAX_NREGIONS];
> > +/* The offset from the start of the RAMBlock to the start of the
> > + * vhost region.
> > + */
> > +ram_addr_t region_rb_offset[VHOST_MEMORY_MAX_NREGIONS];
> 
> Here the array size is VHOST_MEMORY_MAX_NREGIONS, while...
> 
> >  };
> >  
> >  static bool ioeventfd_enabled(void)
> > @@ -324,8 +329,14 @@ static int vhost_user_set_mem_table(struct vhost_dev 
> > *dev,
> >  assert((uintptr_t)reg->userspace_addr == reg->userspace_addr);
> >  mr = memory_region_from_host((void 
> > *)(uintptr_t)reg->userspace_addr,
> >   &offset);
> > +u->region_rb_offset[i] = offset;
> > +u->region_rb[i] = mr->ram_block;
> 
> ... can i>=VHOST_MEMORY_MAX_NREGIONS here? Or do we only need to note
> this down if fd > 0 below?  Thanks,

I don't *think* so - I mean:
for (i = 0; i < dev->mem->nregions; ++i) {

so if that's the maximum number of regions and that's the number of
regions we should be safe???

Dave

> >  fd = memory_region_get_fd(mr);
> >  if (fd > 0) {
> > +trace_vhost_user_set_mem_table_withfd(fd_num, mr->name,
> > +  reg->memory_size,
> > +  reg->guest_phys_addr,
> > +  reg->userspace_addr, 
> > offset);
> >  msg.payload.memory.regions[fd_num].userspace_addr = 
> > reg->userspace_addr;
> >  msg.payload.memory.regions[fd_num].memory_size  = 
> > reg->memory_size;
> >  msg.payload.memory.regions[fd_num].guest_phys_addr = 
> > reg->guest_phys_addr;
> > -- 
> > 2.13.0
> > 
> 
> -- 
> Peter Xu
--
Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-devel] [PATCHv9 2/3] fw_cfg: move qdev_init_nofail() from fw_cfg_init1() to callers

On Fri, Jul 14, 2017 at 10:40:07AM +0100, Mark Cave-Ayland wrote:
> When looking to instantiate a TYPE_FW_CFG_MEM or TYPE_FW_CFG_IO device to be
> able to wire it up differently, it is much more convenient for the caller to
> instantiate the device and have the fw_cfg default files already preloaded
> during realize.
> 
> Move fw_cfg_init1() to the end of both the fw_cfg_mem_realize() and
> fw_cfg_io_realize() functions so it no longer needs to be called manually
> when instantiating the device, and also rename it to fw_cfg_common_realize()
> which better describes its new purpose.
> 
> Since it is now the responsibility of the machine to wire up the fw_cfg device
> it is necessary to introduce a object_property_add_child() call into
> fw_cfg_init_io() and fw_cfg_init_mem() to link the fw_cfg device to the root
> machine object as before.
> 
> Finally with the previous change to fw_cfg_find() we can now remove the
> assert() preventing multiple fw_cfg devices being instantiated and replace
> them with a simple call to fw_cfg_find() at realize time instead. This allows
> us to remove FW_CFG_NAME and FW_CFG_PATH since they are no longer required.
> 
> Signed-off-by: Mark Cave-Ayland 
> Reviewed-by: Igor Mammedov 

Reviewed-by: Eduardo Habkost 

-- 
Eduardo

[Qemu-devel] qapi: Stop abusing "special" values for something entirely different

2017-07-14 Thread Markus Armbruster

Back in March, we discussed Dan's "[PATCH] migration: allow clearing
migration string parameters".  The patch extends command
migrate-set-parameters to interpret empty string arguments to parameters
tls-creds and tls-hostname specially:

# @tls-creds: ID of the 'tls-creds' object that provides credentials
# for establishing a TLS connection over the migration data
# channel. On the outgoing side of the migration, the 
credentials
# must be for a 'client' endpoint, while for the incoming side 
the
# credentials must be for a 'server' endpoint. Setting this
# will enable TLS for all migrations. The default is unset,
# resulting in unsecured migration at the QEMU level. (Since 
2.7)
--> # An empty string means that QEMU will use plain text mode for
--> # migration, rather than TLS (Since 2.9)
#
# @tls-hostname: hostname of the target host for the migration. This
#is required when using x509 based TLS credentials and the
#migration URI does not already include a hostname. For
#example if using fd: or exec: based migration, the
#hostname must be provided so that the server's x509
#certificate identity can be validated. (Since 2.7)
--> #An empty string means that QEMU will use the hostname
--> #associated with the migration URI, if any. (Since 2.9)

Works, since "" is not a valid TLS credentials ID, and not a valid host
name.

I objected:

The command means "set parameter P to value V".  *Except* when V is
"", it means something else, namely "reset parameter P to its
default, whatever that may be".

This is (a) not general, because it won't do for cases where "" may
occur as value, and (b) ugly.

Ugliness is the eye of the beholder.  Lack of generality isn't.

https://lists.gnu.org/archive/html/qemu-devel/2017-03/msg02841.html

I proposed to instead add JSON null to the set of accepted values, using
an alternate type.

However, Dan needed this for 2.9, adding a first class null type to QAPI
takes a bit of work (~50 lines, plus tests), and the freeze was
literally tomorrow, so I relented, and let this pass in the hope of
deprecating it in favor of the cleaner solution in 2.10.  It became
commit 4af245d.

There's a minor complication with my deprecation plan: the
migrate-set-parameters parameters in question are *also* results of
query-migrate-parameters, because both are defined by reference to type
MigrationParameters.  So, unless we decouple the two by duplicating the
type, changing the parameters of migrate-set-parameters also changes the
results of query-migrate-parameters.  However:

1. *Actual* results don't change (query-migrate-parameters doesn't
   suddenly return null values), only the introspection value changes.

2. This isn't actually a new issue.  The members of MigrationParameters
   are all optional, because query-migrate-parameters needs them to be
   optional.  But they're not actually optional in
   query-migrate-parameters.  Sharing types can make introspection less
   tight than it could be.


Another, similar case has crept in without me noticing: BlockdevRef
(commit d26c9a1, Sep 2013, but declared stable only in 2.9).

# @reference:   references the ID of an existing block device. An
#   empty string means that no block device should be
#   referenced.

Works, since "" is not a valid block ID.

BlockdevRef occurs only within arguments of blockdev-add.  blockdev-add
accepts "" in just one place: optional member @backing of COW formats.
Semantics:

* Present means "use this block device" as backing storage

* Absent means "default to the one stored in the image"

* Except "" means "don't use backing storage at all"

The first two are perfectly normal: when the parameter is absent, it
defaults to an implied value, but the value's meaning is the same.

The third one overloads the parameter with a second meaning.  The
overloading *implicit*, i.e. it's not visible in the types.

This is really the same problem as migrate-set-parameters:

* we need a way to say "do something else entirely",

* but we can't use "absent", since it's already taken,

* so we pick one the QAPI schema already allows, but that is not
  actually valid: "".

Instead of the last part, I prefer either

* so we add a *new* value, such as JSON null.

or perhaps

* so we add a second optional parameter to ask for "do something else
  entirely" (since the command can do only one of the two things, you
  may give only one of the two optional parameters).


Here's what I propose to do:

1. Stop abusing values the schema accepts, but are invalid to mean "do
something else entirely".

2. Add a first class null type to QAPI.

3. Turn MigrationParameters members tls-creds and tls-hostname into
alternate of str and nu

Re: [Qemu-devel] [PATCH v4 1/2] docs: document support lifetime for features

On Fri, Jul 14, 2017 at 11:15:48AM +0100, Daniel P. Berrange wrote:
> There is currently no explicit guidance on the duration of support
> for features such as versioned machine types, which have a finite
> useful lifespan. Thus apps / users cannot predict how much time
> they might be able to use a feature for, before it is removed (if
> ever).
> 
> This adds a new appendix that lists items which have finite lifecycles,
> such as machine types. For items which are generally expected to be
> supported indefinitely, it sets out the policy around deprecation
> and removal, should it be needed.
> 
> Signed-off-by: Daniel P. Berrange 
> ---
>  qemu-doc.texi | 37 +
>  1 file changed, 37 insertions(+)
> 
> diff --git a/qemu-doc.texi b/qemu-doc.texi
> index 48af5155c7..f067015d4b 100644
> --- a/qemu-doc.texi
> +++ b/qemu-doc.texi
> @@ -38,6 +38,7 @@
>  * QEMU Guest Agent::
>  * QEMU User space emulator::
>  * Implementation notes::
> +* Support lifetime::
>  * License::
>  * Index::
>  @end menu
> @@ -3128,6 +3129,42 @@ Run the emulation in single step mode.
>  
>  @include qemu-tech.texi
>  
> +@node Support lifetime
> +@appendix Support lifetime
> +
> +In general features are intended to be supported indefinitely once
> +introduced into QEMU.
> +
> +In the event that a feature needs to be removed, it will be listed
> +in the ``Deprecated features'' appendix of this document. The feature
> +will remain functional for 2 major releases prior to actual removal.

Is a "major release" a x.0 release?  So if we decide to to
deprecate a feature in QEMU 2.10, does this mean we will be able
to remove it only on QEMU 4.0?

This doesn't sound like a predictable feature lifetime, if we
don't have a predictable policy for major release numbering.

> +
> +Deprecated features may also generate warnings on the console when
> +QEMU starts up, or if activated via a monitor command, however,
> +this is not a mandatory requirement.
> +
> +Certain features have an inherently finite lifetime, and thus
> +will be removed on a fixed schedule, without following the normal
> +deprecation process. Such features are listed in the sections
> +that follow.
> +
> +@node Machine types
> +@section Machine types
> +
> +For architectures which aim to support live migration compatibility
> +across releases, each release will introduce a new versioned machine
> +type. For example, the 2.8.0 release introduced machine types
> +``pc-i440fx-2.8'' and ``pc-q35-2.8'' for the x86_64/i686 architectures.
> +
> +To allow live migration of a guest running on a 2.8.0 release to a
> +2.9.0, the QEMU 2.9.0 version must support the ``pc-i440fx-2.8'' and
> +``pc-q35-2.8''. To allow users live migrating VMs to skip multiple
> +intermediate releases when upgrading, new releases of QEMU will
> +support machine types from many previous versions.
> +
> +The supported lifetime for versioned machine types is 12 releases,
> +which is equivalent to 4 years worth of previous QEMU releases.
> +
>  @node License
>  @appendix License
>  
> -- 
> 2.13.0
> 

-- 
Eduardo

Re: [Qemu-devel] [PATCH 00/11] Make memory_region_init_ram() and friends handle migration

2017-07-14 Thread Peter Maydell

On 10 July 2017 at 11:05, Paolo Bonzini  wrote:
> On 07/07/2017 16:42, Peter Maydell wrote:
>> This patchset changes the memory region functions
>>  - memory_region_init_ram()
>>  - memory_region_init_rom()
>>  - memory_region_init_rom_device()
>> to all automatically register the backing memory they allocate
>> for migration using vmstate_register_ram(). Renamed functions
>>  - memory_region_init_ram_nomigrate()
>>  - memory_region_init_rom_nomigrate()
>>  - memory_region_init_rom_device_nomigrate()
>> are provided which only do the MR init, for the oddball
>> cases which want to manage migration of the backing memory
>> themselves (and to avoid behavioural changes for callers
>> which weren't managing correctly migration themselves...)
>>
>> The idea is based on discussion from a previous patchset:
>>  https://lists.gnu.org/archive/html/qemu-devel/2017-07/msg00764.html
>>
>> The series includes a largish coccinelle-scripted patch and a
>> few by-hand conversions which change callsites which previously
>> manually created the region and registered its backing ram in
>> two separate steps to use the new functions.
>>
>> This series does not include any patches to fix bugs like:
>>  * caller forgot to call vmstate_register_ram() so region
>>is not actually migrated (eg hw/arm/highbank.c)
>>  * caller used vmstate_register_ram_global() even though it is
>>a device, so you can't create 2 copies of the device (eg sm501)
>> because I wanted to stick to strictly no-behaviour-change
>> for this patch -- we can fix the bugs separately (fixes will
>> tend to imply migration compat breaks so can only be done
>> for some boards.) Most of the remaining callers of the
>> _nomigrate functions are buggy, I think (and a demonstration
>> that our current API does not score well on the "hard to
>> get wrong by accident" scale).
>
> Reviewed-by: Paolo Bonzini 

Thanks. I'm going to apply this directly to master (so then
I can check that nothing has got applied to it since which
uses the old semantics for these functions; nothing in
fact has.)

thanks
-- PMM

Re: [Qemu-devel] [PULL 22/42] configure: factor out list of supported Xen/KVM/HAX targets

2017-07-14 Thread Paolo Bonzini



- Original Message -
> From: "Philippe Mathieu-Daudé" 
> To: "Paolo Bonzini" , "Peter Maydell" 
> 
> Cc: "Anthony PERARD" , "Stefano Stabellini" 
> , "Alex Bennée"
> , "QEMU Developers" 
> Sent: Friday, July 14, 2017 5:26:36 PM
> Subject: Re: [Qemu-devel] [PULL 22/42] configure: factor out list of 
> supported Xen/KVM/HAX targets
> 
> what about this RFC series?
> http://lists.nongnu.org/archive/html/qemu-devel/2017-07/msg02513.html

Too late for 2.10, but the way to go is to define a new "xenpv" target.
In fact, --disable-tcg was the prerequisite for that if I remember correctly
my old discussions with Citrix folks (could it have been Wei Liu?)

Paolo

> On 07/10/2017 01:28 PM, Paolo Bonzini wrote:
> > On 10/07/2017 18:24, Peter Maydell wrote:
> >> On 10 July 2017 at 17:14, Paolo Bonzini  wrote:
> >>> On 10/07/2017 17:49, Peter Maydell wrote:
>  On 5 July 2017 at 08:14, Paolo Bonzini  wrote:
> > This will be useful when the functions are called, early in the
> > configure
> > process, to filter out targets that do not support hardware
> > acceleration.
> >
> > Signed-off-by: Paolo Bonzini 
> 
> > +supported_xen_target() {
> > +test "$xen" = "yes" || return 1
> > +glob "$1" "*-softmmu" || return 1
> > +case "${1%-softmmu}:$cpu" in
> > +arm:arm | aarch64:aarch64 | \
> > +i386:i386 | i386:x86_64 | x86_64:i386 | x86_64:x86_64)
> > +return 0
> > +;;
> 
>  This says that arm-on-arm and aarch64-on-aarch64 are supported
>  Xen targets...
> >>>
> >>> Hmm, this comes from my old patches.  IIRC the reason for the change,
> >>> when it wasn't a change (many conflicts ago) was that Xen folks were
> >>> using --disable-tcg because their device model for Xen PV on ARM was
> >>> actually an x86_64 QEMU.
> >>>
> >>> Stefano and Anthony, is this still true?  If so, would it make sense to
> >>> add the Xen PV machine type to qemu-system-arm---that is, is it
> >>> something you can whip up easily, or should I just remove that line?
> >>
> >> I think you should just fix configure for the moment, because
> >> this patch wasn't supposed to change anything about what we
> >> build (AIUI). We can think about changing the Xen PV on ARM
> >> build setup as a separate thing if we want to, I suspect it
> >> is more invasive than a couple of lines changing in configure.
> > 
> > Yes, definitely more invasive.
> > 
> > I'll prepare a fix.
> > 
> > Paolo
> > 
>

Re: [Qemu-devel] [PATCH v5 14/18] qcow2: Switch qcow2_measure() to byte-based iteration

On 07/13/2017 01:49 PM, John Snow wrote:
> 
> 
> On 07/12/2017 09:05 PM, Eric Blake wrote:
>> This is new code, but it is easier to read if it makes passes over
>> the image using bytes rather than sectors (and will get easier in
>> the future when bdrv_get_block_status is converted to byte-based).
>>
>> While at it, fix a bug in the computation of nb_sectors using MAX
>> rather than MIN and thus always passing INT_MAX (thankfully, the
>> bug was harmless, as bdrv_get_block_status_above() clamps its
>> answers according to image size).
>>
> 
> Well, unless ssize was >= 2^40, and then it wouldn't have chosen
> INT_MAX, right!?

But then it would have wrapped the value around while assigning it to
'int nb_sectors'.  I suppose for some very-interesting size images, you
could get nb_sectors to turn into a negative value?  At any rate, it's
good the bug was short-lived.

> 
>> Signed-off-by: Eric Blake 
>>
> 
> Reviewed-by: John Snow 
> 

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v13 03/26] target: [tcg] Add generic translation framework