Re: [PATCH] target/arm: allow setting SCR_EL3.EnTP2 when FEAT_SME is implemented

[Andre Przywara]

On Tue,  4 Oct 2022 09:23:54 +0200
Jerome Forissier  wrote:

Hi,

> Updates write_scr() to allow setting SCR_EL3.EnTP2 when FEAT_SME is
> implemented. SCR_EL3 being a 64-bit register, valid_mask is changed
> to uint64_t and the SCR_* constants in target/arm/cpu.h are extended
> to 64-bit so that masking and bitwise not (~) behave as expected.
> 
> This enables booting Linux with Trusted Firmware-A at EL3 with
> "-M virt,secure=on -cpu max".
> 
> Cc: qemu-sta...@nongnu.org
> Fixes: 78cb9776662a ("target/arm: Enable SME for -cpu max")
> Signed-off-by: Jerome Forissier 

Good catch!
So I can confirm that this fixes the issue, given a TF-A patch to actually
enable SME (and SVE).
Checked against the ARM ARM, also verified that the
defines don't accidentally changed their values.

Reviewed-by: Andre Przywara 

> ---
>  target/arm/cpu.h| 54 ++---
>  target/arm/helper.c |  5 -
>  2 files changed, 31 insertions(+), 28 deletions(-)
> 
> diff --git a/target/arm/cpu.h b/target/arm/cpu.h
> index 5168e3d837..d5e9949eb6 100644
> --- a/target/arm/cpu.h
> +++ b/target/arm/cpu.h
> @@ -1653,33 +1653,33 @@ static inline void xpsr_write(CPUARMState *env, 
> uint32_t val, uint32_t mask)
>  
>  #define HPFAR_NS  (1ULL << 63)
>  
> -#define SCR_NS(1U << 0)
> -#define SCR_IRQ   (1U << 1)
> -#define SCR_FIQ   (1U << 2)
> -#define SCR_EA(1U << 3)
> -#define SCR_FW(1U << 4)
> -#define SCR_AW(1U << 5)
> -#define SCR_NET   (1U << 6)
> -#define SCR_SMD   (1U << 7)
> -#define SCR_HCE   (1U << 8)
> -#define SCR_SIF   (1U << 9)
> -#define SCR_RW(1U << 10)
> -#define SCR_ST(1U << 11)
> -#define SCR_TWI   (1U << 12)
> -#define SCR_TWE   (1U << 13)
> -#define SCR_TLOR  (1U << 14)
> -#define SCR_TERR  (1U << 15)
> -#define SCR_APK   (1U << 16)
> -#define SCR_API   (1U << 17)
> -#define SCR_EEL2  (1U << 18)
> -#define SCR_EASE  (1U << 19)
> -#define SCR_NMEA  (1U << 20)
> -#define SCR_FIEN  (1U << 21)
> -#define SCR_ENSCXT(1U << 25)
> -#define SCR_ATA   (1U << 26)
> -#define SCR_FGTEN (1U << 27)
> -#define SCR_ECVEN (1U << 28)
> -#define SCR_TWEDEN(1U << 29)
> +#define SCR_NS(1ULL << 0)
> +#define SCR_IRQ   (1ULL << 1)
> +#define SCR_FIQ   (1ULL << 2)
> +#define SCR_EA(1ULL << 3)
> +#define SCR_FW(1ULL << 4)
> +#define SCR_AW(1ULL << 5)
> +#define SCR_NET   (1ULL << 6)
> +#define SCR_SMD   (1ULL << 7)
> +#define SCR_HCE   (1ULL << 8)
> +#define SCR_SIF   (1ULL << 9)
> +#define SCR_RW(1ULL << 10)
> +#define SCR_ST(1ULL << 11)
> +#define SCR_TWI   (1ULL << 12)
> +#define SCR_TWE   (1ULL << 13)
> +#define SCR_TLOR  (1ULL << 14)
> +#define SCR_TERR  (1ULL << 15)
> +#define SCR_APK   (1ULL << 16)
> +#define SCR_API   (1ULL << 17)
> +#define SCR_EEL2  (1ULL << 18)
> +#define SCR_EASE  (1ULL << 19)
> +#define SCR_NMEA  (1ULL << 20)
> +#define SCR_FIEN  (1ULL << 21)
> +#define SCR_ENSCXT(1ULL << 25)
> +#define SCR_ATA   (1ULL << 26)
> +#define SCR_FGTEN (1ULL << 27)
> +#define SCR_ECVEN (1ULL << 28)
> +#define SCR_TWEDEN(1ULL << 29)
>  #define SCR_TWEDELMAKE_64BIT_MASK(30, 4)
>  #define SCR_TME   (1ULL << 34)
>  #define SCR_AMVOFFEN  (1ULL << 35)
> diff --git a/target/arm/helper.c b/target/arm/helper.c
> index d7bc467a2a..5cde8a0425 100644
> --- a/target/arm/helper.c
> +++ b/target/arm/helper.c
> @@ -1669,7 +1669,7 @@ static void vbar_write(CPUARMState *env, const 
> ARMCPRegInfo *ri,
>  static void scr_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t 
> value)
>  {
>  /* Begin with base v8.0 state.  */
> -uint32_t valid_mask = 0x3fff;
> +uint64_t valid_mask = 0x3fff;
>  ARMCPU *cpu = env_archcpu(env);
>  
>  /*
> @@ -1706,6 +1706,9 @@ static void scr_write(CPUARMState *env, const 
> ARMCPRegInfo *ri, uint64_t value)
>  if (cpu_isar_feature(aa64_doublefault, cpu)) {
>  valid_mask |= SCR_EASE | SCR_NMEA;
>  }
> +if (cpu_isar_feature(aa64_sme, cpu)) {
> +valid_mask |= SCR_ENTP2;
> +}
>  } else {
>  valid_mask &= ~(SCR_RW | SCR_ST);
>  if (cpu_isar_feature(aa32_ras, cpu)) {

Re: [PATCH 05/16] hw/arm: allwinner: Don't enable PSCI conduit when booting guest in EL3

[Andre Przywara]

On Sun, 30 Jan 2022 23:35:37 +0100
Niek Linnenbank  wrote:

Hi,

(CC:ing Samuel for his intimate Allwinner BootROM knowledge)

> Hi Peter,
> 
> 
> 
> On Thu, Jan 27, 2022 at 4:46 PM Peter Maydell 
> wrote:
> 
> > Change the allwinner-h3 based board to use the new boot.c
> > functionality to allow us to enable psci-conduit only if the guest is
> > being booted in EL1 or EL2, so that if the user runs guest EL3
> > firmware code our PSCI emulation doesn't get in its way.
> >
> > To do this we stop setting the psci-conduit property on the CPU
> > objects in the SoC code, and instead set the psci_conduit field in
> > the arm_boot_info struct to tell the common boot loader code that
> > we'd like PSCI if the guest is starting at an EL that it makes sense
> > with.
> >
> > This affects the orangepi-pc board.
> >
> > This commit leaves the secondary CPUs in the powered-down state if
> > the guest is booting at EL3, which is the same behaviour as before
> > this commit.  The secondaries can no longer be started by that EL3
> > code making a PSCI call but can still be started via the CPU
> > Configuration Module registers (which we model in
> > hw/misc/allwinner-cpucfg.c).
> >
> > Signed-off-by: Peter Maydell 
> >  
> 
> While testing your patches on the orangepi-pc machine, I've found that two
> acceptance tests BootLinuxConsole.test_arm_orangepi_bionic_20_08 and
> BootLinuxConsole.test_arm_orangepi_uboot_netbsd9 of the orangepi-pc board
> are no longer passing on current master:
> 
>   ARMBIAN_ARTIFACTS_CACHED=yes AVOCADO_ALLOW_LARGE_STORAGE=yes avocado
> --show=app,console run -t machine:orangepi-pc
> tests/avocado/boot_linux_console.py
>   ...
>  (4/5)
> tests/avocado/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_bionic_20_08:
> -console: U-Boot SPL 2020.04-armbian (Sep 02 2020 - 10:16:13 +0200)
> \console: DRAM:
> INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred: Timeout
> reached\nOriginal status: ERROR\n{'name':
> '4-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_bionic_20_08',
> 'logdi>  
>  (5/5)
> tests/avocado/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_uboot_netbsd9:
> /console: U-Boot SPL 2020.01+dfsg-1 (Jan 08 2020 - 08:19:44 +)
> console: DRAM:
> INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred: Timeout
> reached\nOriginal status: ERROR\n{'name':
> '5-tests/avocado/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_uboot_netbsd9',
> 'logd>  
> RESULTS: PASS 3 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 2 |
> CANCEL 0
> JOB TIME   : 221.25 s
> 
> Bisecting the error I was able to trace it back to commit 5ead62185d
> ("memory: Make memory_region_is_mapped() succeed when mapped via an alias").
> I'll try to find the original thread and respond to that with this
> information.
> 
> However, with commit 5ead62185d reverted, all tested passed fine:
> 
> ARMBIAN_ARTIFACTS_CACHED=yes AVOCADO_ALLOW_LARGE_STORAGE=yes avocado
> --show=app,console run -t machine:orangepi-pc
> tests/avocado/boot_linux_console.py
> ...
> PASS (16.48 s)
> RESULTS: PASS 5 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 |
> CANCEL 0
> JOB TIME   : 116.63 s
> 
> So for the orangepi-pc and allwinner-h3:
> 
> Reviewed-by: Niek Linnenbank 
> Tested-by: Niek Linnenbank 
> 
> 
> > ---
> > If anybody knows for definite that the secondaries should be
> > powered-down at startup for guest firmware, we can delete the TODO.
> >  
> 
> Looking at the Allwinner H3 datasheet in 4.4.3.7 page 146, it says that
> for the CPU1 Status Register the default value indicates at least that its
> not in a
> wait for interrupt standby mode. And if I look in U-Boot's
> arm/arm/cpu/armv7/sunxi/psci.c code
> in the psci_cpu_on implementation, there is an explicit 'power on' part
> there, suggesting the secondary CPUs
> are by default off. So while I don't have any hard proof, these findings
> suggest we are modeling the correct behavior
> with secondary CPUs by default off.

So when it comes to firmware, indeed the secondaries seem to be off when
the first user provided code (boot0/SPL) is entered. However there is an
MPIDR read in the BROM, with the corresponding "branch if not primary
core". I think this is because the BROM is mapped at the reset vector, so
upon SMP firmware releasing the reset line it always starts in ROM, then
gets diverted to the actual entry point.
Maybe Samuel can confirm that the secondary cores are power gated when the
SoCs comes out of reset?

Cheers,
Andre

> > The allwinner-cpucfg.c code makes the reset value for the
> > REG_CPU*_RST_CTRL registers "CPUX_RESET_RELEASED", which might
> > suggest otherwise, but that could easily just be a QEMU error.
> > ---
> >  hw/arm/allwinner-h3.c | 9 -
> >  hw/arm/orangepi.c | 1 +
> >  2 files changed, 5 insertions(+), 5 deletions(-)
> >
> > diff --git a/hw/arm/allwinner-h3.c b/hw/arm/allwinner-h3.c
> > index f9b7ed18711..f54aff6d2d2 100644
> > --- a/hw/arm/allwinner-h3.c
> > +++ 

Re: [PATCH v2 00/13] arm gicv3 ITS: Various bug fixes and refactorings

[Andre Przywara]

On Wed, 19 Jan 2022 10:15:52 +
Peter Maydell  wrote:

Hi Peter,

> On Tue, 18 Jan 2022 at 23:30, Andre Przywara  wrote:
> > Looking at k-u-t's arm/gic.c and QEMU's hw/intc/arm_gic.c I see two
> > problems here: QEMU implements word accesses as four successive calls to
> > gic_dist_readb() - which is probably fine if that helps code design,
> > but it won't allow it to actually spot access size issues. I just
> > remember that we spent some brain cells and CPP macros on getting the
> > access size right in KVM - hence those tests in kvm-unit-tests.  
> 
> Thanks for looking at this. I should have read the code rather
> than dashing off a reply last thing in the evening based just
> on the test case output! I think I was confusing how our GICv3
> emulation handles register accesses (with separate functions for
> byte/halfword/word/quad accesses) with the GICv2 emulation
> (which as you say calls down into the byte emulation code
> wherever it can).

No worries!

> > But more importantly it looks like GICD_IIDR is actually not
> > implemented: There is a dubious "if (offset < 0x08) return 0;" line,
> > but IIDR (offset 0x8) would actually fall through, and hit the bad_reg
> > label, which would return 0 (and cause the message, if enabled).  
> 
> Mmm. I actually have a patch in target-arm.next from Petr Pavlu
> which implements GICC_IIDR, but we do indeed not implement the
> distributor equivalent.

Well, returning 0 is actually not the worst idea. Using proper ID
values might not even be feasible for QEMU, or would create some hassle
with versioning. With 0 all a user can assume is spec compliance.

> > If that helps: from a GIC MMIO perspective 8-bit accesses are actually
> > the exception rather than the norm (ARM IHI 0048B.b 4.1.4 GIC register
> > access).  
> 
> Yes. We got this right in the GICv3 emulation design, where almost
> all the logic is in the 32-bit accessor functions and the 8/16-bit
> functions deal only with the very few registers that have to
> permit non-word accesses.

Indeed. I dusted off my old GICv3 MMIO patches for kvm-unit-tests, and
QEMU passed with flying colours. With the debug switch I see it
reporting exactly the violating accesses we except to see.
Will send those patches ASAP.

> The GICv2 code is a lot older (and to
> be fair to it, started out as 11MPcore interrupt controller
> emulation, and I bet the docs of that were not very specific about
> what registers could or could not be accessed byte at a time).
> Unless we want to rewrite all that logic in the GICv2 emulation
> (which I at least do not :-))

... and I can't ...

> I think we'll have to live with
> the warnings about bad-offsets reporting for each byte rather
> than just once for the word access.

Yeah, if those warnings appear only with that debug switch, there is
probably little reason to change that code just because of this. At
least it seemed to work quite well over the years.

Cheers,
Andre

P.S. I changed k-u-t to special case the UP case, so that TCG passes.
But now KVM fails, of course. So I will have to make a patch for the
kernel ...

Re: [PATCH v2 00/13] arm gicv3 ITS: Various bug fixes and refactorings

[Andre Przywara]

On Tue, 18 Jan 2022 19:41:56 +
Peter Maydell  wrote:

Hi Peter, Alex,

thanks for the heads up!

> On Tue, 18 Jan 2022 at 17:42, Alex Bennée  wrote:
> >
> >
> > Peter Maydell  writes:
> >  
> > > I've been working on the ITS to add support for the GICv4 functionality.
> > > In the course of that I found a handful of bugs in it and also some
> > > places where the code benefited from refactoring to make it a better
> > > base to put in the GICv4 parts. This patchset is just the bugfixes
> > > and cleanups, because there are enough patches here that I figured it
> > > made sense to send them out now rather than holding on to them.
> > >
> > > Most of these patches were in v1 and have been reviewed already.  
> >
> > I've reviewed the patches and they look good to me. kvm-unit-tests is
> > still failing some tests but the ones it fails hasn't changed from
> > before this patch:
> >
> >   ✗  env QEMU=$HOME/lsrc/qemu.git/builds/arm.all/qemu-system-aarch64 
> > ./run_tests.sh -g gic
> >   PASS gicv2-ipi (3 tests)
> >   PASS gicv2-mmio (17 tests, 1 skipped)
> >   FAIL gicv2-mmio-up (17 tests, 2 unexpected failures)
> >   FAIL gicv2-mmio-3p (17 tests, 3 unexpected failures)
> >   PASS gicv3-ipi (3 tests)
> >   PASS gicv2-active (1 tests)
> >   PASS gicv3-active (1 tests)
> >
> > That said running with -d unimp,guest_errors there are some things that
> > should probably be double checked, e.g.:  
> 
> Almost all of the logging seems to be where the test code is
> doing stuff that the GIC spec says isn't valid.

That sounds like a plausible explanation for a unit test suite, but
does not seem to be actually true in this case, see below.

> Also, this
> test is gicv2, which is unrelated to either the gicv3 code
> or to the ITS...

This is true.

> 
> >   /home/alex/lsrc/qemu.git/builds/arm.all/qemu-system-aarch64 -nodefaults 
> > -machine virt -accel tcg -cpu cortex-a57 -device virtio-serial-device 
> > -device virtconsole,chardev=
> >   ctd -chardev testdev,id=ctd -device pci-testdev -display none -serial 
> > stdio -kernel arm/gic.flat -machine gic-version=2 -smp 1 -append "mmio" -d 
> > unimp,guest_errors
> >   PASS: gicv2: mmio: all CPUs have interrupts
> >   gic_dist_readb: Bad offset 8
> >   gic_dist_readb: Bad offset 9
> >   gic_dist_readb: Bad offset a
> >   gic_dist_readb: Bad offset b  
> 
> This is GICD_IIDR, which is a 32-bit register. The test looks like it's
> trying to read it as 4 separate bytes, which is out of spec, and
> is why our implementation is warning about it.

Looking at k-u-t's arm/gic.c and QEMU's hw/intc/arm_gic.c I see two
problems here: QEMU implements word accesses as four successive calls to
gic_dist_readb() - which is probably fine if that helps code design,
but it won't allow it to actually spot access size issues. I just
remember that we spent some brain cells and CPP macros on getting the
access size right in KVM - hence those tests in kvm-unit-tests.
 
But more importantly it looks like GICD_IIDR is actually not
implemented: There is a dubious "if (offset < 0x08) return 0;" line,
but IIDR (offset 0x8) would actually fall through, and hit the bad_reg
label, which would return 0 (and cause the message, if enabled).
Also the name and how it's called suggests that this deals with bytes
only, but returns uint32_t, and for instance deals with bit 10 in
TYPER. I see how this eventually falls into place magically (the upper
three bytes return 0, and get ORed into the >8 bit result of offset 8),
but those messages are definitely false alarm then.

If that helps: from a GIC MMIO perspective 8-bit accesses are actually
the exception rather than the norm (ARM IHI 0048B.b 4.1.4 GIC register
access).

> >   INFO: gicv2: mmio: IIDR: 0x
> >   gic_dist_writeb: Bad offset 4
> >   gic_dist_writeb: Bad offset 5
> >   gic_dist_writeb: Bad offset 6
> >   gic_dist_writeb: Bad offset 7
> >   gic_dist_writeb: Bad offset 4
> >   gic_dist_writeb: Bad offset 5
> >   gic_dist_writeb: Bad offset 6
> >   gic_dist_writeb: Bad offset 7
> >   gic_dist_writeb: Bad offset 4
> >   gic_dist_writeb: Bad offset 5
> >   gic_dist_writeb: Bad offset 6
> >   gic_dist_writeb: Bad offset 7  
> 
> These complaints are because the test is trying to write
> to GICD_TYPER, which is not permitted.

Writes are not permitted, yes, but k-u-t emits a proper str, so there
should be only three lines, not twelve.

> >   PASS: gicv2: mmio: GICD_TYPER is read-only
> >   gic_dist_readb: Bad offset 8
> >   gic_dist_readb: Bad offset 9
> >   gic_dist_readb: Bad offset a
> >   gic_dist_readb: Bad offset b  
> 
> More attempts to do byte accesses to a word-only register.

The messages come actually again because IIDR is not handled at all,
and there are only four of them because of the design of gic_dist_read().
k-u-t issues a proper ldr here.
I think we refrained from actually testing illegal access sizes,
because that could trigger external aborts/SErrors on real hardware.

> >   gic_dist_writeb: Bad offset 8
> >   

Re: [PATCH 00/16] fdt: Make OF_BOARD a boolean option

[Andre Przywara]

On Thu, 14 Oct 2021 09:17:52 -0600
Simon Glass  wrote:

> Hi Tom,
> 
> On Thu, 14 Oct 2021 at 08:56, Tom Rini  wrote:
> >
> > On Wed, Oct 13, 2021 at 12:06:02PM -0600, Simon Glass wrote:  
> > > Hi François,
> > >
> > > On Wed, 13 Oct 2021 at 11:35, François Ozog  
> > > wrote:  
> > > >
> > > > Hi Simon
> > > >
> > > > Le mer. 13 oct. 2021 à 16:49, Simon Glass  a écrit : 
> > > >  
> > > >>
> > > >> Hi Tom, Bin,François,
> > > >>
> > > >> On Tue, 12 Oct 2021 at 19:34, Tom Rini  wrote:  
> > > >> >
> > > >> > On Wed, Oct 13, 2021 at 09:29:14AM +0800, Bin Meng wrote:  
> > > >> > > Hi Simon,
> > > >> > >
> > > >> > > On Wed, Oct 13, 2021 at 9:01 AM Simon Glass  
> > > >> > > wrote:  
> > > >> > > >
> > > >> > > > With Ilias' efforts we have dropped OF_PRIOR_STAGE and 
> > > >> > > > OF_HOSTFILE so
> > > >> > > > there are only three ways to obtain a devicetree:
> > > >> > > >
> > > >> > > >- OF_SEPARATE - the normal way, where the devicetree is built 
> > > >> > > > and
> > > >> > > >   appended to U-Boot
> > > >> > > >- OF_EMBED - for development purposes, the devicetree is 
> > > >> > > > embedded in
> > > >> > > >   the ELF file (also used for EFI)
> > > >> > > >- OF_BOARD - the board figures it out on its own
> > > >> > > >
> > > >> > > > The last one is currently set up so that no devicetree is needed 
> > > >> > > > at all
> > > >> > > > in the U-Boot tree. Most boards do provide one, but some don't. 
> > > >> > > > Some
> > > >> > > > don't even provide instructions on how to boot on the board.
> > > >> > > >
> > > >> > > > The problems with this approach are documented at [1].
> > > >> > > >
> > > >> > > > In practice, OF_BOARD is not really distinct from OF_SEPARATE. 
> > > >> > > > Any board
> > > >> > > > can obtain its devicetree at runtime, even it is has a 
> > > >> > > > devicetree built
> > > >> > > > in U-Boot. This is because U-Boot may be a second-stage 
> > > >> > > > bootloader and its
> > > >> > > > caller may have a better idea about the hardware available in 
> > > >> > > > the machine.
> > > >> > > > This is the case with a few QEMU boards, for example.
> > > >> > > >
> > > >> > > > So it makes no sense to have OF_BOARD as a 'choice'. It should 
> > > >> > > > be an
> > > >> > > > option, available with either OF_SEPARATE or OF_EMBED.
> > > >> > > >
> > > >> > > > This series makes this change, adding various missing devicetree 
> > > >> > > > files
> > > >> > > > (and placeholders) to make the build work.  
> > > >> > >
> > > >> > > Adding device trees that are never used sounds like a hack to me.
> > > >> > >
> > > >> > > For QEMU, device tree is dynamically generated on the fly based on
> > > >> > > command line parameters, and the device tree you put in this series
> > > >> > > has various hardcoded  values which normally do not show 
> > > >> > > up
> > > >> > > in hand-written dts files.
> > > >> > >
> > > >> > > I am not sure I understand the whole point of this.  
> > > >> >
> > > >> > I am also confused and do not like the idea of adding device trees 
> > > >> > for
> > > >> > platforms that are capable of and can / do have a device tree to 
> > > >> > give us
> > > >> > at run time.  
> > > >>
> > > >> (I'll just reply to this one email, since the same points applies to
> > > >> all replies I think)
> > > >>
> > > >> I have been thinking about this and discussing it with people for a
> > > >> few months now. I've been signalling a change like this for over a
> > > >> month now, on U-Boot contributor calls and in discussions with Linaro
> > > >> people. I sent a patch (below) to try to explain things. I hope it is
> > > >> not a surprise!
> > > >>
> > > >> The issue here is that we need a devicetree in-tree in U-Boot, to
> > > >> avoid the mess that has been created by OF_PRIOR_STAGE, OF_BOARD,
> > > >> BINMAN_STANDALONE_FDT and to a lesser extent, OF_HOSTFILE. Between
> > > >> Ilias' series and this one we can get ourselves on a stronger footing.
> > > >> There is just OF_SEPARATE, with OF_EMBED for debugging/ELF use.
> > > >> For more context:
> > > >>
> > > >> http://patchwork.ozlabs.org/project/uboot/patch/20210919215111.3830278-3-...@chromium.org/
> > > >>
> > > >> BTW I did suggest to QEMU ARM that they support a way of adding the
> > > >> u-boot.dtsi but there was not much interest there (in fact the
> > > >> maintainer would prefer there was no special support even for booting
> > > >> Linux directly!)  
> > > >
> > > > i understand their point of view and agree with it.  
> > > >>
> > > >> But in any case it doesn't really help U-Boot. I
> > > >> think the path forward might be to run QEMU twice, once to get its
> > > >> generated tree and once to give the 'merged' tree with the U-Boot
> > > >> properties in it, if people want to use U-Boot features.
> > > >>
> > > >> I do strongly believe that OF_BOARD must be a run-time option, not a
> > > >> build-time one. It creates all sorts of problems and obscurity which
> > > >> have taken months to 

Re: [PATCH 00/16] fdt: Make OF_BOARD a boolean option

[Andre Przywara]

On Tue, 12 Oct 2021 19:01:04 -0600
Simon Glass  wrote:

Hi,

> With Ilias' efforts we have dropped OF_PRIOR_STAGE and OF_HOSTFILE so
> there are only three ways to obtain a devicetree:
> 
>- OF_SEPARATE - the normal way, where the devicetree is built and
>   appended to U-Boot
>- OF_EMBED - for development purposes, the devicetree is embedded in
>   the ELF file (also used for EFI)
>- OF_BOARD - the board figures it out on its own
> 
> The last one is currently set up so that no devicetree is needed at all
> in the U-Boot tree. Most boards do provide one, but some don't. Some
> don't even provide instructions on how to boot on the board.
> 
> The problems with this approach are documented at [1].
> 
> In practice, OF_BOARD is not really distinct from OF_SEPARATE. Any board
> can obtain its devicetree at runtime, even it is has a devicetree built
> in U-Boot. This is because U-Boot may be a second-stage bootloader and its
> caller may have a better idea about the hardware available in the machine.
> This is the case with a few QEMU boards, for example.
> 
> So it makes no sense to have OF_BOARD as a 'choice'. It should be an
> option, available with either OF_SEPARATE or OF_EMBED.

So I am possibly fine with that, but:

> This series makes this change, adding various missing devicetree files
> (and placeholders) to make the build work.

If we just need it to make the build work, we not just have pure stub DTs,
as you do for highbank, everywhere?
Adding *some* DT files for those platforms that actually do the right
thing seems like the wrong direction to me.
Providing DTs in the source repositories of the actual consumers is more
of a bad habit that dragged on since Linux started this around 10 years
ago (for practical reasons). For *some* platforms U-Boot is the firmware
component that is in the best situation to provide the DTB (because it's
more than a mere bootloader), but for other it's just not. And this is not
even looking at really dynamic platforms like QEMU, where providing some
kind of fixed DT is just not working.

I don't get the argument that people would need to see the DT in the tree
to develop code. The DT spec and binding documentation (currently living
in the Linux kernel source tree) provide the specification to code
against, and the platform specific selection of drivers in Kconfig and
_defconfig select the drivers for the devices that people are expected to
see. Why does one need actual DT files in the tree?

I totally agree on adding more documentation, possibly *pointing* to example
DTs or giving commands on how to obtain the actual copy (-dumpdtb,
/sys/firmware/devicetree), but don't think that adding some .dts files for
platforms that don't need them is the right way.

Cheers,
Andre.

> 
> It also provides a few qemu clean-ups discovered along the way.
> 
> This series is based on Ilias' two series for OF_HOSTFILE and
> OF_PRIOR_STAGE removal.
> 
> It is available at u-boot-dm/ofb-working
> 
> [1]
> https://patchwork.ozlabs.org/project/uboot/patch/20210919215111.3830278-3-...@chromium.org/
> 
> 
> Simon Glass (16):
>   arm: qemu: Mention -nographic in the docs
>   arm: qemu: Explain how to extract the generate devicetree
>   riscv: qemu: Explain how to extract the generate devicetree
>   arm: qemu: Add a devicetree file for qemu_arm
>   arm: qemu: Add a devicetree file for qemu_arm64
>   riscv: qemu: Add devicetree files for qemu_riscv32/64
>   arm: rpi: Add a devicetree file for rpi_4
>   arm: vexpress: Add a devicetree file for juno
>   arm: xenguest_arm64: Add a fake devicetree file
>   arm: octeontx: Add a fake devicetree file
>   arm: xilinx_versal_virt: Add a devicetree file
>   arm: bcm7xxx: Add a devicetree file
>   arm: qemu-ppce500: Add a devicetree file
>   arm: highbank: Add a fake devicetree file
>   fdt: Make OF_BOARD a bool option
>   Drop CONFIG_BINMAN_STANDALONE_FDT
> 
>  Makefile   |3 +-
>  arch/arm/dts/Makefile  |   20 +-
>  arch/arm/dts/bcm2711-rpi-4-b.dts   | 1958 
>  arch/arm/dts/bcm7xxx.dts   |   15 +
>  arch/arm/dts/highbank.dts  |   14 +
>  arch/arm/dts/juno-r2.dts   | 1038 +
>  arch/arm/dts/octeontx.dts  |   14 +
>  arch/arm/dts/qemu-arm.dts  |  402 +
>  arch/arm/dts/qemu-arm64.dts|  381 +
>  arch/arm/dts/xenguest-arm64.dts|   15 +
>  arch/arm/dts/xilinx-versal-virt.dts|  307 
>  arch/powerpc/dts/Makefile  |1 +
>  arch/powerpc/dts/qemu-ppce500.dts  |  264 
>  arch/riscv/dts/Makefile|2 +-
>  arch/riscv/dts/qemu-virt.dts   |8 -
>  arch/riscv/dts/qemu-virt32.dts |  217 +++
>  arch/riscv/dts/qemu-virt64.dts |  217 +++
>  configs/bcm7260_defconfig  |1 +
>  configs/bcm7445_defconfig  |1 +
>  configs/highbank_defconfig |2 +-
>  

Re: arm: Launching EFI-enabled arm32 Linux

[Andre Przywara]

On Wed, 8 Sep 2021 01:25:04 +0200
Adam Lackorzynski  wrote:

Hi Adam,

> On Mon Sep 06, 2021 at 16:34:03 +0100, Andre Przywara wrote:
> > On Sat, 4 Sep 2021 21:26:45 +0200
> > Adam Lackorzynski  wrote:
> > 
> > Hi Adam,
> >   
> > > while trying to launch an EFI-enabled arm32 Linux binary (zImage) I
> > > noticed I get an undefined instruction exception on the first
> > > instruction. Now this is a bit special because Linux uses a nop
> > > instruction there that also is a PE file signature ('MZ') such that the
> > > CPU runs over it and the file is still recognized as a PE binary. Linux
> > > uses 0x13105a4d (tstne r0, #0x4d000) as the instruction (see also
> > > arch/arm/boot/compressed/head.S and efi-header.S in Linux).
> > > However, QEMU's instruction decoder will only recognize TST with bits
> > > 12-15 being 0, which this instruction is not fullfilling, and thus the
> > > undef exception. I guess other CPU implementations will allow this
> > > encoding. So while investigating I was doing the following to make Linux
> > > proceed. I also believe this was working in a previous version of QEMU.
> > > 
> > > diff --git a/target/arm/a32.decode b/target/arm/a32.decode
> > > index fcd8cd4f7d..222553750e 100644
> > > --- a/target/arm/a32.decode
> > > +++ b/target/arm/a32.decode
> > > @@ -127,7 +127,7 @@ ADD_rri   001 0100 .   
> > >    @s_rri_rot
> > >  ADC_rri   001 0101 .      @s_rri_rot
> > >  SBC_rri   001 0110 .      @s_rri_rot
> > >  RSC_rri   001 0111 .      @s_rri_rot
> > > -TST_xri   001 1000 1      @S_xri_rot
> > > +TST_xri   001 1000 1      @S_xri_rot
> > >  TEQ_xri   001 1001 1      @S_xri_rot
> > >  CMP_xri   001 1010 1      @S_xri_rot
> > >  CMN_xri   001 1011 1      @S_xri_rot
> > > 
> > > 
> > > Any thoughts on this?  
> > 
> > thanks for the report, I was looking at this and have a kernel patch
> > to fix this properly as Peter suggested. And while I agree on the
> > problem, I was struggling to reproduce this in reality: both with
> > -kernel and when booting through U-Boot the "Z" bit is set, which lets
> > QEMU not even bother about the rest of the encoding - the condition
> > flags don't match, so it proceeds. If I change the __nop to use "tsteq",
> > I see it hanging due to the missing exception handler, but not with
> > "tstne".
> > So can you say how you spotted this issue? This would be needed as a
> > justification for patching the guts of the ARM Linux kernel port.  
> 
> Good point with the condition flags. I'm doing this with our own vmm
> where I'm loading the binary directly as the payload (as mandated by the
> header), and where psr is set to a defined value with all flags cleared.

Right, I was thinking something like this.

> If I set the Z bit than it also works (of course).
> Looking a bit around in QEMU as well as u-boot I it looks like this is
> rather by luck how flags are set.

Yes, the kernel boot protocol doesn't say anything about the condition
flags, so any combination would be valid and we were just lucky before.
I did also test on an Cortex-A7 and A53, both ignore bits [15:12] (so
execute the instruction as if they were 0), which explains why it works
on real hardware.

> Thanks for doing the Linux patch, I'll scrap mine, and also thanks to
> Peter for the idea!

Oh, didn't want to cut you off, if you want to have the commit: be my
guest!
Otherwise I will send something tomorrow, with a Reported-by: to you.

Grüße an die Elbe!

Cheers,
Andre

Re: arm: Launching EFI-enabled arm32 Linux

[Andre Przywara]

On Sat, 4 Sep 2021 21:26:45 +0200
Adam Lackorzynski  wrote:

Hi Adam,

> while trying to launch an EFI-enabled arm32 Linux binary (zImage) I
> noticed I get an undefined instruction exception on the first
> instruction. Now this is a bit special because Linux uses a nop
> instruction there that also is a PE file signature ('MZ') such that the
> CPU runs over it and the file is still recognized as a PE binary. Linux
> uses 0x13105a4d (tstne r0, #0x4d000) as the instruction (see also
> arch/arm/boot/compressed/head.S and efi-header.S in Linux).
> However, QEMU's instruction decoder will only recognize TST with bits
> 12-15 being 0, which this instruction is not fullfilling, and thus the
> undef exception. I guess other CPU implementations will allow this
> encoding. So while investigating I was doing the following to make Linux
> proceed. I also believe this was working in a previous version of QEMU.
> 
> diff --git a/target/arm/a32.decode b/target/arm/a32.decode
> index fcd8cd4f7d..222553750e 100644
> --- a/target/arm/a32.decode
> +++ b/target/arm/a32.decode
> @@ -127,7 +127,7 @@ ADD_rri   001 0100 .      
> @s_rri_rot
>  ADC_rri   001 0101 .      @s_rri_rot
>  SBC_rri   001 0110 .      @s_rri_rot
>  RSC_rri   001 0111 .      @s_rri_rot
> -TST_xri   001 1000 1      @S_xri_rot
> +TST_xri   001 1000 1      @S_xri_rot
>  TEQ_xri   001 1001 1      @S_xri_rot
>  CMP_xri   001 1010 1      @S_xri_rot
>  CMN_xri   001 1011 1      @S_xri_rot
> 
> 
> Any thoughts on this?

thanks for the report, I was looking at this and have a kernel patch
to fix this properly as Peter suggested. And while I agree on the
problem, I was struggling to reproduce this in reality: both with
-kernel and when booting through U-Boot the "Z" bit is set, which lets
QEMU not even bother about the rest of the encoding - the condition
flags don't match, so it proceeds. If I change the __nop to use "tsteq",
I see it hanging due to the missing exception handler, but not with
"tstne".
So can you say how you spotted this issue? This would be needed as a
justification for patching the guts of the ARM Linux kernel port.

Cheers,
Andre

Re: [kvm-unit-tests PATCH v2 5/9] arm: pmu: Basic event counter Tests

[Andre Przywara]

On Thu, 30 Jan 2020 12:25:06 +0100
Eric Auger  wrote:

> Adds the following tests:
> - event-counter-config: test event counter configuration
> - basic-event-count:
>   - programs counters #0 and #1 to count 2 required events
>   (resp. CPU_CYCLES and INST_RETIRED). Counter #0 is preset
>   to a value close enough to the 32b
>   overflow limit so that we check the overflow bit is set
>   after the execution of the asm loop.
> - mem-access: counts MEM_ACCESS event on counters #0 and #1
>   with and without 32-bit overflow.
> 
> Signed-off-by: Eric Auger 
> 
> ---
> 
> v1 -> v2:
> - fix PMCNTENSET_EL0 and PMCNTENCLR_EL0 op0
> - print PMEVTYPER SH
> - properly clobber used regs and add "cc"
> - simplify mem_access_loop
> ---
>  arm/pmu.c | 269 ++
>  arm/unittests.cfg |  18 
>  2 files changed, 287 insertions(+)
> 
> diff --git a/arm/pmu.c b/arm/pmu.c
> index 4a26a76..1b0101f 100644
> --- a/arm/pmu.c
> +++ b/arm/pmu.c
> @@ -18,9 +18,15 @@
>  #include "asm/barrier.h"
>  #include "asm/sysreg.h"
>  #include "asm/processor.h"
> +#include 
> +#include 
>  
>  #define PMU_PMCR_E (1 << 0)
> +#define PMU_PMCR_P (1 << 1)
>  #define PMU_PMCR_C (1 << 2)
> +#define PMU_PMCR_D (1 << 3)
> +#define PMU_PMCR_X (1 << 4)
> +#define PMU_PMCR_DP(1 << 5)
>  #define PMU_PMCR_LC(1 << 6)
>  #define PMU_PMCR_N_SHIFT   11
>  #define PMU_PMCR_N_MASK0x1f
> @@ -104,6 +110,9 @@ static inline void precise_instrs_loop(int loop, uint32_t 
> pmcr)
>  
>  /* event counter tests only implemented for aarch64 */
>  static void test_event_introspection(void) {}
> +static void test_event_counter_config(void) {}
> +static void test_basic_event_count(void) {}
> +static void test_mem_access(void) {}
>  
>  #elif defined(__aarch64__)
>  #define ID_AA64DFR0_PERFMON_SHIFT 8
> @@ -145,6 +154,33 @@ static inline void precise_instrs_loop(int loop, 
> uint32_t pmcr)
>  }
>  
>  #define PMCEID1_EL0 sys_reg(3, 3, 9, 12, 7)
> +#define PMCNTENSET_EL0 sys_reg(3, 3, 9, 12, 1)
> +#define PMCNTENCLR_EL0 sys_reg(3, 3, 9, 12, 2)
> +
> +#define PMEVTYPER_EXCLUDE_EL1 (1 << 31)
> +#define PMEVTYPER_EXCLUDE_EL0 (1 << 30)

Please use 1U << or BIT() ;-)

Rest looks OK now:

Reviewed-by: Andre Przywara 


> +
> +#define regn_el0(__reg, __n) __reg ## __n  ## _el0
> +#define write_regn(__reg, __n, __val) \
> + write_sysreg((__val), __reg ## __n ## _el0)
> +
> +#define read_regn(__reg, __n) \
> + read_sysreg(__reg ## __n ## _el0)
> +
> +#define print_pmevtyper(__s, __n) do { \
> + uint32_t val; \
> + val = read_regn(pmevtyper, __n);\
> + report_info("%s pmevtyper%d=0x%x, eventcount=0x%x (p=%ld, u=%ld 
> nsk=%ld, nsu=%ld, nsh=%ld m=%ld, mt=%ld, sh=%ld)", \
> + (__s), (__n), val, val & 0x,\
> + (BIT_MASK(31) & val) >> 31, \
> + (BIT_MASK(30) & val) >> 30, \
> + (BIT_MASK(29) & val) >> 29, \
> + (BIT_MASK(28) & val) >> 28, \
> + (BIT_MASK(27) & val) >> 27, \
> + (BIT_MASK(26) & val) >> 26, \
> + (BIT_MASK(25) & val) >> 25);\
> + (BIT_MASK(24) & val) >> 24);\
> + } while (0)
>  
>  static bool is_event_supported(uint32_t n, bool warn)
>  {
> @@ -198,6 +234,230 @@ static void test_event_introspection(void)
>   report(required_events, "Check required events are implemented");
>  }
>  
> +/*
> + * Extra instructions inserted by the compiler would be difficult to 
> compensate
> + * for, so hand assemble everything between, and including, the PMCR accesses
> + * to start and stop counting. isb instructions are inserted to make sure
> + * pmccntr read after this function returns the exact instructions executed
> + * in the controlled block. Loads @loop times the data at @address into x9.
> + */
> +static void mem_access_loop(void *addr, int loop, uint32_t pmcr)
> +{
> +asm volatile(
> + "   msr pmcr_el0, %[pmcr]\n"
> + "   isb\n"
> + "   mov x10, %[loop]\n"
> + "1: sub x10, x10, #1\n"
> + "   ldr x9, [%[addr]]\n"
> + "   cmp x10, #0x0\n"
> + "   b.gt1b\n"
> + "  

Re: [kvm-unit-tests PATCH v2 4/9] arm: pmu: Check Required Event Support

[Andre Przywara]

On Thu, 30 Jan 2020 12:25:05 +0100
Eric Auger  wrote:

Hi,

> If event counters are implemented check the common events
> required by the PMUv3 are implemented.
> 
> Some are unconditionally required (SW_INCR, CPU_CYCLES,
> either INST_RETIRED or INST_SPEC). Some others only are
> required if the implementation implements some other features.
> 
> Check those wich are unconditionally required.
> 
> This test currently fails on TCG as neither INST_RETIRED
> or INST_SPEC are supported.
> 
> Signed-off-by: Eric Auger 
> 
> ---
> 
> v1 -> v2:
> - fix is_event_supported()
> - fix boolean condition for PMU v4
> - fix PMCEID0 definition
> 
> RFC ->v1:
> - add a comment to explain the PMCEID0/1 splits
> ---
>  arm/pmu.c | 62 +++
>  arm/unittests.cfg |  6 +
>  2 files changed, 68 insertions(+)
> 
> diff --git a/arm/pmu.c b/arm/pmu.c
> index d24857e..4a26a76 100644
> --- a/arm/pmu.c
> +++ b/arm/pmu.c
> @@ -101,6 +101,10 @@ static inline void precise_instrs_loop(int loop, 
> uint32_t pmcr)
>   : [pmcr] "r" (pmcr), [z] "r" (0)
>   : "cc");
>  }
> +
> +/* event counter tests only implemented for aarch64 */
> +static void test_event_introspection(void) {}
> +
>  #elif defined(__aarch64__)
>  #define ID_AA64DFR0_PERFMON_SHIFT 8
>  #define ID_AA64DFR0_PERFMON_MASK  0xf
> @@ -139,6 +143,61 @@ static inline void precise_instrs_loop(int loop, 
> uint32_t pmcr)
>   : [pmcr] "r" (pmcr)
>   : "cc");
>  }
> +
> +#define PMCEID1_EL0 sys_reg(3, 3, 9, 12, 7)
> +
> +static bool is_event_supported(uint32_t n, bool warn)
> +{
> + uint64_t pmceid0 = read_sysreg(pmceid0_el0);
> + uint64_t pmceid1 = read_sysreg_s(PMCEID1_EL0);
> + bool supported;
> + uint64_t reg;
> +
> + /*
> +  * The low 32-bits of PMCEID0/1 respectly describe
> +  * event support for events 0-31/32-63. Their High
> +  * 32-bits describe support for extended events
> +  * starting at 0x4000, using the same split.
> +  */
> + if (n >= 0x0  && n <= 0x3F)
> + reg = (pmceid0 & 0x) | ((pmceid1 & 0x) << 32);
> + else if  (n >= 0x4000 && n <= 0x403F)
> + reg = (pmceid0 >> 32) | ((pmceid1 >> 32) << 32);
> + else
> + abort();
> +
> + supported =  reg & (1UL << (n & 0x3F));
> +
> + if (!supported && warn)
> + report_info("event %d is not supported", n);
> + return supported;
> +}
> +
> +static void test_event_introspection(void)
> +{
> + bool required_events;
> +
> + if (!pmu.nb_implemented_counters) {
> + report_skip("No event counter, skip ...");
> + return;
> + }
> +
> + /* PMUv3 requires an implementation includes some common events */
> + required_events = is_event_supported(0x0, true) /* SW_INCR */ &&
> +   is_event_supported(0x11, true) /* CPU_CYCLES */ &&
> +   (is_event_supported(0x8, true) /* INST_RETIRED */ ||
> +is_event_supported(0x1B, true) /* INST_PREC */);
> +
> + if (pmu.version == 0x4) {

I think this should read >= 0x4, since those requirements are stacked on top of 
the prevision revision's requirements. Even better with some symbolic name.

With that fixed:

Reviewed-by: Andre Przywara 

Cheers,
Andre

> + /* ARMv8.1 PMU: STALL_FRONTEND and STALL_BACKEND are required */
> + required_events = required_events &&
> +   is_event_supported(0x23, true) &&
> +   is_event_supported(0x24, true);
> + }
> +
> + report(required_events, "Check required events are implemented");
> +}
> +
>  #endif
>  
>  /*
> @@ -326,6 +385,9 @@ int main(int argc, char *argv[])
>  "Monotonically increasing cycle count");
>   report(check_cpi(cpi), "Cycle/instruction ratio");
>   pmccntr64_test();
> + } else if (strcmp(argv[1], "event-introspection") == 0) {
> + report_prefix_push(argv[1]);
> + test_event_introspection();
>   } else {
>   report_abort("Unknown sub-test '%s'", argv[1]);
>   }
> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> index 79f0d7a..4433ef3 100644
> --- a/arm/unittests.cfg
> +++ b/arm/unittests.cfg
> @@ -66,6 +66,12 @@ file = pmu.flat
>  groups = pmu
>  extra_params = -append 'cycle-counter 0'
>  
> +[pmu-event-introspection]
> +file = pmu.flat
> +groups = pmu
> +arch = arm64
> +extra_params = -append 'event-introspection'
> +
>  # Test PMU support (TCG) with -icount IPC=1
>  #[pmu-tcg-icount-1]
>  #file = pmu.flat

Re: [kvm-unit-tests PATCH v2 2/9] arm: pmu: Let pmu tests take a sub-test parameter

[Andre Przywara]

On Thu, 30 Jan 2020 12:25:03 +0100
Eric Auger  wrote:

> As we intend to introduce more PMU tests, let's add
> a sub-test parameter that will allow to categorize
> them. Existing tests are in the cycle-counter category.
> 
> Signed-off-by: Eric Auger 

Did you change anything? Or just forgot to add my previous R-b?

Anyway,

Reviewed-by: Andre Przywara 

Cheers,
Andre

> ---
>  arm/pmu.c | 24 +++-
>  arm/unittests.cfg |  7 ---
>  2 files changed, 19 insertions(+), 12 deletions(-)
> 
> diff --git a/arm/pmu.c b/arm/pmu.c
> index d5a03a6..e5e012d 100644
> --- a/arm/pmu.c
> +++ b/arm/pmu.c
> @@ -287,22 +287,28 @@ int main(int argc, char *argv[])
>  {
>   int cpi = 0;
>  
> - if (argc > 1)
> - cpi = atol(argv[1]);
> -
>   if (!pmu_probe()) {
>   printf("No PMU found, test skipped...\n");
>   return report_summary();
>   }
>  
> + if (argc < 2)
> + report_abort("no test specified");
> +
>   report_prefix_push("pmu");
>  
> - report(check_pmcr(), "Control register");
> - report(check_cycles_increase(),
> -"Monotonically increasing cycle count");
> - report(check_cpi(cpi), "Cycle/instruction ratio");
> -
> - pmccntr64_test();
> + if (strcmp(argv[1], "cycle-counter") == 0) {
> + report_prefix_push(argv[1]);
> + if (argc > 2)
> + cpi = atol(argv[2]);
> + report(check_pmcr(), "Control register");
> + report(check_cycles_increase(),
> +"Monotonically increasing cycle count");
> + report(check_cpi(cpi), "Cycle/instruction ratio");
> + pmccntr64_test();
> + } else {
> + report_abort("Unknown sub-test '%s'", argv[1]);
> + }
>  
>   return report_summary();
>  }
> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> index daeb5a0..79f0d7a 100644
> --- a/arm/unittests.cfg
> +++ b/arm/unittests.cfg
> @@ -61,21 +61,22 @@ file = pci-test.flat
>  groups = pci
>  
>  # Test PMU support
> -[pmu]
> +[pmu-cycle-counter]
>  file = pmu.flat
>  groups = pmu
> +extra_params = -append 'cycle-counter 0'
>  
>  # Test PMU support (TCG) with -icount IPC=1
>  #[pmu-tcg-icount-1]
>  #file = pmu.flat
> -#extra_params = -icount 0 -append '1'
> +#extra_params = -icount 0 -append 'cycle-counter 1'
>  #groups = pmu
>  #accel = tcg
>  
>  # Test PMU support (TCG) with -icount IPC=256
>  #[pmu-tcg-icount-256]
>  #file = pmu.flat
> -#extra_params = -icount 8 -append '256'
> +#extra_params = -icount 8 -append 'cycle-counter 256'
>  #groups = pmu
>  #accel = tcg
>  

Re: [kvm-unit-tests PATCH v2 3/9] arm: pmu: Add a pmu struct

[Andre Przywara]

On Thu, 30 Jan 2020 12:25:04 +0100
Eric Auger  wrote:

> This struct aims at storing information potentially used by
> all tests such as the pmu version, the read-only part of the
> PMCR, the number of implemented event counters, ...
> 
> Signed-off-by: Eric Auger 

As I stated already in v1:

Reviewed-by: Andre Przywara 

Cheers,
Andre

> ---
>  arm/pmu.c | 30 +-
>  1 file changed, 25 insertions(+), 5 deletions(-)
> 
> diff --git a/arm/pmu.c b/arm/pmu.c
> index e5e012d..d24857e 100644
> --- a/arm/pmu.c
> +++ b/arm/pmu.c
> @@ -33,7 +33,14 @@
>  
>  #define NR_SAMPLES 10
>  
> -static unsigned int pmu_version;
> +struct pmu {
> + unsigned int version;
> + unsigned int nb_implemented_counters;
> + uint32_t pmcr_ro;
> +};
> +
> +static struct pmu pmu;
> +
>  #if defined(__arm__)
>  #define ID_DFR0_PERFMON_SHIFT 24
>  #define ID_DFR0_PERFMON_MASK  0xf
> @@ -265,7 +272,7 @@ static bool check_cpi(int cpi)
>  static void pmccntr64_test(void)
>  {
>  #ifdef __arm__
> - if (pmu_version == 0x3) {
> + if (pmu.version == 0x3) {
>   if (ERRATA(9e3f7a296940)) {
>   write_sysreg(0xdead, PMCCNTR64);
>   report(read_sysreg(PMCCNTR64) == 0xdead, "pmccntr64");
> @@ -278,9 +285,22 @@ static void pmccntr64_test(void)
>  /* Return FALSE if no PMU found, otherwise return TRUE */
>  static bool pmu_probe(void)
>  {
> - pmu_version = get_pmu_version();
> - report_info("PMU version: %d", pmu_version);
> - return pmu_version != 0 && pmu_version != 0xf;
> + uint32_t pmcr;
> +
> + pmu.version = get_pmu_version();
> + report_info("PMU version: %d", pmu.version);
> +
> + if (pmu.version == 0 || pmu.version == 0xF)
> + return false;
> +
> + pmcr = get_pmcr();
> + pmu.pmcr_ro = pmcr & 0xFF80;
> + pmu.nb_implemented_counters =
> + (pmcr >> PMU_PMCR_N_SHIFT) & PMU_PMCR_N_MASK;
> + report_info("Implements %d event counters",
> + pmu.nb_implemented_counters);
> +
> + return true;
>  }
>  
>  int main(int argc, char *argv[])

Re: [kvm-unit-tests PATCH 05/10] arm: pmu: Basic event counter Tests

[Andre Przywara]

On Mon, 16 Dec 2019 21:47:52 +0100
Eric Auger  wrote:

Hi Eric,

thanks a lot for your work on these elaborate tests! I have some PMU test 
extensions as well, but they are nowhere as sophisticated as yours!

Just ran this on my ThunderX2 desktop (4.15.0-65-generic #74-Ubuntu kernel, 
QEMU emulator version 2.11.1(Debian 1:2.11+dfsg-1ubuntu7.21)), and it reported 
the following fails:
INFO: pmu: basic-event-count: counter #0 is 0x207e (CPU_CYCLES)
INFO: pmu: basic-event-count: counter #1 is 0xc89 (INST_RETIRED)
INFO: pmu: basic-event-count: overflow reg = 0x0
FAIL: pmu: basic-event-count: check overflow happened on #0 only

INFO: PMU version: 4
INFO: Implements 6 event counters
INFO: pmu: mem-access: counter #0 is 1297 (MEM_ACCESS)
INFO: pmu: mem-access: counter #1 is 1287 (MEM_ACCESS)
FAIL: pmu: mem-access: Ran 20 mem accesses
FAIL: pmu: mem-access: Ran 20 mem accesses with expected overflows on both 
counters
INFO: pmu: mem-access: cnt#0 = 1353 cnt#1=1259 overflow=0x0

Do you know about this? Is this due to kernel bugs? Because Ubuntu cleverly 
chose an EOL kernel for their stable distro :-P
Will try to have a look and repeat on a Juno.

Comments inline 

> Adds the following tests:
> - event-counter-config: test event counter configuration
> - basic-event-count:
>   - programs counters #0 and #1 to count 2 required events
>   (resp. CPU_CYCLES and INST_RETIRED). Counter #0 is preset
>   to a value close enough to the 32b
>   overflow limit so that we check the overflow bit is set
>   after the execution of the asm loop.
> - mem-access: counts MEM_ACCESS event on counters #0 and #1
>   with and without 32-bit overflow.
> 
> Signed-off-by: Eric Auger 
> ---
>  arm/pmu.c | 261 ++
>  arm/unittests.cfg |  18 
>  2 files changed, 279 insertions(+)
> 
> diff --git a/arm/pmu.c b/arm/pmu.c
> index d88ef22..139dae3 100644
> --- a/arm/pmu.c
> +++ b/arm/pmu.c
> @@ -18,9 +18,15 @@
>  #include "asm/barrier.h"
>  #include "asm/sysreg.h"
>  #include "asm/processor.h"
> +#include 
> +#include 
>  
>  #define PMU_PMCR_E (1 << 0)
> +#define PMU_PMCR_P (1 << 1)
>  #define PMU_PMCR_C (1 << 2)
> +#define PMU_PMCR_D (1 << 3)
> +#define PMU_PMCR_X (1 << 4)
> +#define PMU_PMCR_DP(1 << 5)
>  #define PMU_PMCR_LC(1 << 6)
>  #define PMU_PMCR_N_SHIFT   11
>  #define PMU_PMCR_N_MASK0x1f
> @@ -104,6 +110,9 @@ static inline void precise_instrs_loop(int loop, uint32_t 
> pmcr)
>  
>  /* event counter tests only implemented for aarch64 */
>  static void test_event_introspection(void) {}
> +static void test_event_counter_config(void) {}
> +static void test_basic_event_count(void) {}
> +static void test_mem_access(void) {}
>  
>  #elif defined(__aarch64__)
>  #define ID_AA64DFR0_PERFMON_SHIFT 8
> @@ -145,6 +154,32 @@ static inline void precise_instrs_loop(int loop, 
> uint32_t pmcr)
>  }
>  
>  #define PMCEID1_EL0 sys_reg(11, 3, 9, 12, 7)
> +#define PMCNTENSET_EL0 sys_reg(11, 3, 9, 12, 1)
> +#define PMCNTENCLR_EL0 sys_reg(11, 3, 9, 12, 2)

op0 (the first argument) is only two bits, so it should read "3" instead of 
"11" here. That's already a bug in the existing PMCEID1_EL0 definition. We get 
away with it because the macro masks with 3, but it should be still written 
correctly here. Not sure where the "11" actually comes from.

> +
> +#define PMEVTYPER_EXCLUDE_EL1 (1 << 31)
> +#define PMEVTYPER_EXCLUDE_EL0 (1 << 30)
> +
> +#define regn_el0(__reg, __n) __reg ## __n  ## _el0
> +#define write_regn(__reg, __n, __val) \
> + write_sysreg((__val), __reg ## __n ## _el0)
> +
> +#define read_regn(__reg, __n) \
> + read_sysreg(__reg ## __n ## _el0)
> +
> +#define print_pmevtyper(__s, __n) do { \
> + uint32_t val; \
> + val = read_regn(pmevtyper, __n);\
> + report_info("%s pmevtyper%d=0x%x, eventcount=0x%x (p=%ld, u=%ld 
> nsk=%ld, nsu=%ld, nsh=%ld m=%ld, mt=%ld)", \
> + (__s), (__n), val, val & 0x,  \
> + (BIT_MASK(31) & val) >> 31, \
> + (BIT_MASK(30) & val) >> 30, \
> + (BIT_MASK(29) & val) >> 29, \
> + (BIT_MASK(28) & val) >> 28, \
> + (BIT_MASK(27) & val) >> 27, \
> + (BIT_MASK(26) & val) >> 26, \
> + (BIT_MASK(25) & val) >> 25); \

Just a nit, but later versions of the ARMv8 ARM list bit 24 as "SH", for 
filtering Secure EL2 events. For the sake of completeness you could add this as 
well, since we list the EL3 filter bit as well.

> + } while (0)
>  
>  static bool is_event_supported(uint32_t n, bool warn)
>  {
> @@ -207,6 +242,223 @@ static void test_event_introspection(void)
>   report(required_events, "Check required events are implemented");
>  }
>  
> +static inline void mem_access_loop(void *addr, int loop, uint32_t pmcr)

Do we really need the "inline" here? If you rely on this being inlined, we need 
something stronger, I 

Re: [kvm-unit-tests PATCH 04/10] arm: pmu: Check Required Event Support

[Andre Przywara]

On Mon, 16 Dec 2019 21:47:51 +0100
Eric Auger  wrote:

Hi Eric,

> If event counters are implemented check the common events
> required by the PMUv3 are implemented.
> 
> Some are unconditionally required (SW_INCR, CPU_CYCLES,
> either INST_RETIRED or INST_SPEC). Some others only are
> required if the implementation implements some other features.
> 
> Check those wich are unconditionally required.
> 
> This test currently fails on TCG as neither INST_RETIRED
> or INST_SPEC are supported.
> 
> Signed-off-by: Eric Auger 
> 
> ---
> 
> v1 ->v2:
> - add a comment to explain the PMCEID0/1 splits
> ---
>  arm/pmu.c | 71 +++
>  arm/unittests.cfg |  6 
>  2 files changed, 77 insertions(+)
> 
> diff --git a/arm/pmu.c b/arm/pmu.c
> index d24857e..d88ef22 100644
> --- a/arm/pmu.c
> +++ b/arm/pmu.c
> @@ -101,6 +101,10 @@ static inline void precise_instrs_loop(int loop, 
> uint32_t pmcr)
>   : [pmcr] "r" (pmcr), [z] "r" (0)
>   : "cc");
>  }
> +
> +/* event counter tests only implemented for aarch64 */
> +static void test_event_introspection(void) {}
> +
>  #elif defined(__aarch64__)
>  #define ID_AA64DFR0_PERFMON_SHIFT 8
>  #define ID_AA64DFR0_PERFMON_MASK  0xf
> @@ -139,6 +143,70 @@ static inline void precise_instrs_loop(int loop, 
> uint32_t pmcr)
>   : [pmcr] "r" (pmcr)
>   : "cc");
>  }
> +
> +#define PMCEID1_EL0 sys_reg(11, 3, 9, 12, 7)
> +
> +static bool is_event_supported(uint32_t n, bool warn)
> +{
> + uint64_t pmceid0 = read_sysreg(pmceid0_el0);
> + uint64_t pmceid1 = read_sysreg_s(PMCEID1_EL0);
> + bool supported;
> + uint32_t reg;
> +
> + /*
> +  * The low 32-bits of PMCEID0/1 respectly describe
> +  * event support for events 0-31/32-63. Their High
> +  * 32-bits describe support for extended events
> +  * starting at 0x4000, using the same split.
> +  */
> + if (n >= 0x0  && n <= 0x1F)
> + reg = pmceid0 & 0x;
> + else if  (n >= 0x4000 && n <= 0x401F)
> + reg = pmceid0 >> 32;
> + else if (n >= 0x20  && n <= 0x3F)
> + reg = pmceid1 & 0x;
> + else if (n >= 0x4020 && n <= 0x403F)
> + reg = pmceid1 >> 32;
> + else
> + abort();
> +
> + supported =  reg & (1 << n);

Don't we need to mask off everything but the lowest 5 bits of "n"? Probably 
also using "1U" is better.

> + if (!supported && warn)
> + report_info("event %d is not supported", n);
> + return supported;
> +}
> +
> +static void test_event_introspection(void)

"introspection" sounds quite sophisticated. Are you planning to extend this? If 
not, could we maybe rename it to "test_available_events"?

> +{
> + bool required_events;
> +
> + if (!pmu.nb_implemented_counters) {
> + report_skip("No event counter, skip ...");
> + return;
> + }
> +
> + /* PMUv3 requires an implementation includes some common events */
> + required_events = is_event_supported(0x0, true) /* SW_INCR */ &&
> +   is_event_supported(0x11, true) /* CPU_CYCLES */ &&
> +   (is_event_supported(0x8, true) /* INST_RETIRED */ ||
> +is_event_supported(0x1B, true) /* INST_PREC */);
> +
> + if (pmu.version == 0x4) {
> + /* ARMv8.1 PMU: STALL_FRONTEND and STALL_BACKEND are required */
> + required_events = required_events ||
> +   is_event_supported(0x23, true) ||

Shouldn't those two operators be '&&' instead?

> +   is_event_supported(0x24, true);
> + }
> +
> + /*
> +  * L1D_CACHE_REFILL(0x3) and L1D_CACHE(0x4) are only required if
> +  * L1 data / unified cache. BR_MIS_PRED(0x10), BR_PRED(0x12) are only
> +  * required if program-flow prediction is implemented.
> +  */

Is this a TODO?

Cheers,
Andre


> +
> + report(required_events, "Check required events are implemented");
> +}
> +
>  #endif
>  
>  /*
> @@ -326,6 +394,9 @@ int main(int argc, char *argv[])
>  "Monotonically increasing cycle count");
>   report(check_cpi(cpi), "Cycle/instruction ratio");
>   pmccntr64_test();
> + } else if (strcmp(argv[1], "event-introspection") == 0) {
> + report_prefix_push(argv[1]);
> + test_event_introspection();
>   } else {
>   report_abort("Unknown sub-test '%s'", argv[1]);
>   }
> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> index 79f0d7a..4433ef3 100644
> --- a/arm/unittests.cfg
> +++ b/arm/unittests.cfg
> @@ -66,6 +66,12 @@ file = pmu.flat
>  groups = pmu
>  extra_params = -append 'cycle-counter 0'
>  
> +[pmu-event-introspection]
> +file = pmu.flat
> +groups = pmu
> +arch = arm64
> +extra_params = -append 'event-introspection'
> +
>  # Test PMU support (TCG) with -icount IPC=1
>  #[pmu-tcg-icount-1]
>  #file = pmu.flat

Re: [kvm-unit-tests PATCH 03/10] arm: pmu: Add a pmu struct

[Andre Przywara]

On Mon, 16 Dec 2019 21:47:50 +0100
Eric Auger  wrote:

> This struct aims at storing information potentially used by
> all tests such as the pmu version, the read-only part of the
> PMCR, the number of implemented event counters, ...
> 
> Signed-off-by: Eric Auger 

Reviewed-by: Andre Przywara 

Cheers,
Andre

> ---
>  arm/pmu.c | 30 +-
>  1 file changed, 25 insertions(+), 5 deletions(-)
> 
> diff --git a/arm/pmu.c b/arm/pmu.c
> index e5e012d..d24857e 100644
> --- a/arm/pmu.c
> +++ b/arm/pmu.c
> @@ -33,7 +33,14 @@
>  
>  #define NR_SAMPLES 10
>  
> -static unsigned int pmu_version;
> +struct pmu {
> + unsigned int version;
> + unsigned int nb_implemented_counters;
> + uint32_t pmcr_ro;
> +};
> +
> +static struct pmu pmu;
> +
>  #if defined(__arm__)
>  #define ID_DFR0_PERFMON_SHIFT 24
>  #define ID_DFR0_PERFMON_MASK  0xf
> @@ -265,7 +272,7 @@ static bool check_cpi(int cpi)
>  static void pmccntr64_test(void)
>  {
>  #ifdef __arm__
> - if (pmu_version == 0x3) {
> + if (pmu.version == 0x3) {
>   if (ERRATA(9e3f7a296940)) {
>   write_sysreg(0xdead, PMCCNTR64);
>   report(read_sysreg(PMCCNTR64) == 0xdead, "pmccntr64");
> @@ -278,9 +285,22 @@ static void pmccntr64_test(void)
>  /* Return FALSE if no PMU found, otherwise return TRUE */
>  static bool pmu_probe(void)
>  {
> - pmu_version = get_pmu_version();
> - report_info("PMU version: %d", pmu_version);
> - return pmu_version != 0 && pmu_version != 0xf;
> + uint32_t pmcr;
> +
> + pmu.version = get_pmu_version();
> + report_info("PMU version: %d", pmu.version);
> +
> + if (pmu.version == 0 || pmu.version == 0xF)
> + return false;
> +
> + pmcr = get_pmcr();
> + pmu.pmcr_ro = pmcr & 0xFF80;
> + pmu.nb_implemented_counters =
> + (pmcr >> PMU_PMCR_N_SHIFT) & PMU_PMCR_N_MASK;
> + report_info("Implements %d event counters",
> + pmu.nb_implemented_counters);
> +
> + return true;
>  }
>  
>  int main(int argc, char *argv[])

Re: [kvm-unit-tests PATCH 02/10] arm: pmu: Let pmu tests take a sub-test parameter

[Andre Przywara]

On Mon, 16 Dec 2019 21:47:49 +0100
Eric Auger  wrote:

> As we intend to introduce more PMU tests, let's add
> a sub-test parameter that will allow to categorize
> them. Existing tests are in the cycle-counter category.
> 
> Signed-off-by: Eric Auger 

Reviewed-by: Andre Przywara 

Cheers,
Andre

> ---
>  arm/pmu.c | 24 +++-
>  arm/unittests.cfg |  7 ---
>  2 files changed, 19 insertions(+), 12 deletions(-)
> 
> diff --git a/arm/pmu.c b/arm/pmu.c
> index d5a03a6..e5e012d 100644
> --- a/arm/pmu.c
> +++ b/arm/pmu.c
> @@ -287,22 +287,28 @@ int main(int argc, char *argv[])
>  {
>   int cpi = 0;
>  
> - if (argc > 1)
> - cpi = atol(argv[1]);
> -
>   if (!pmu_probe()) {
>   printf("No PMU found, test skipped...\n");
>   return report_summary();
>   }
>  
> + if (argc < 2)
> + report_abort("no test specified");
> +
>   report_prefix_push("pmu");
>  
> - report(check_pmcr(), "Control register");
> - report(check_cycles_increase(),
> -"Monotonically increasing cycle count");
> - report(check_cpi(cpi), "Cycle/instruction ratio");
> -
> - pmccntr64_test();
> + if (strcmp(argv[1], "cycle-counter") == 0) {
> + report_prefix_push(argv[1]);
> + if (argc > 2)
> + cpi = atol(argv[2]);
> + report(check_pmcr(), "Control register");
> + report(check_cycles_increase(),
> +"Monotonically increasing cycle count");
> + report(check_cpi(cpi), "Cycle/instruction ratio");
> + pmccntr64_test();
> + } else {
> + report_abort("Unknown sub-test '%s'", argv[1]);
> + }
>  
>   return report_summary();
>  }
> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> index daeb5a0..79f0d7a 100644
> --- a/arm/unittests.cfg
> +++ b/arm/unittests.cfg
> @@ -61,21 +61,22 @@ file = pci-test.flat
>  groups = pci
>  
>  # Test PMU support
> -[pmu]
> +[pmu-cycle-counter]
>  file = pmu.flat
>  groups = pmu
> +extra_params = -append 'cycle-counter 0'
>  
>  # Test PMU support (TCG) with -icount IPC=1
>  #[pmu-tcg-icount-1]
>  #file = pmu.flat
> -#extra_params = -icount 0 -append '1'
> +#extra_params = -icount 0 -append 'cycle-counter 1'
>  #groups = pmu
>  #accel = tcg
>  
>  # Test PMU support (TCG) with -icount IPC=256
>  #[pmu-tcg-icount-256]
>  #file = pmu.flat
> -#extra_params = -icount 8 -append '256'
> +#extra_params = -icount 8 -append 'cycle-counter 256'
>  #groups = pmu
>  #accel = tcg
>  

Re: [Qemu-devel] [PATCH v2] hw/arm/boot: Increase compliance with kernel arm64 boot protocol

[Andre Przywara]

On Tue, 16 Oct 2018 15:06:29 +
Stewart Hildebrand  wrote:

Hi, 

> "The Image must be placed text_offset bytes from a 2MB aligned base
> address anywhere in usable system RAM and called there."
> 
> For the virt board, we write our startup bootloader at the very
> bottom of RAM, so that bit can't be used for the image. To avoid
> overlap in case the image requests to be loaded at an offset
> smaller than our bootloader, we increment the load offset to the
> next 2MB.
> 
> This fixes a boot failure for Xen AArch64.

Thanks for that, works for me.

> Signed-off-by: Stewart Hildebrand 

Tested-by: Andre Przywara 

Cheers,
Andre.

Re: [Qemu-devel] [PATCH] hw/arm/boot: Increase compliance with kernel arm64 boot protocol.

[Andre Przywara]

On Tue, 16 Oct 2018 01:19:35 +
Stewart Hildebrand  wrote:

Hi,

Stewart, thanks a lot for picking this up!

> On Monday, October 15, 2018 6:05 PM, Philippe Mathieu-Daudé wrote:
> > Hi Stewart,
> > 
> > On 15/10/2018 23:26, Stewart Hildebrand wrote:  
> > > +/* For the virt board, we write our startup
> > > "bootloader" at the very
> > > + * bottom of RAM, so that bit can't be used for the
> > > image. To avoid
> > > + * overlap in case the image requests to be loaded
> > > at an offset
> > > + * smaller than our bootloader, we increment the
> > > load offset to the
> > > + * next 2MB.
> > > + */
> > > +if (kernel_load_offset < FIXUP_MAX) {  
> > 
> > I don't understand how this is related to FIXUP_MAX...  
> 
> You're right, my apologies, it's not directly related.
> write_bootloader() calculates the size of the bootloader like so:
> len = 0;
> while (insns[len].fixup != FIXUP_TERMINATOR) {
> len++;
> }
> 
> The size of the bootloader then would be len * sizeof(uint32_t)
> 
> It would be nice not to have to repeat that logic in
> load_aarch64_image(). I'll send out a v2 after I take some time to
> wrap my head around it... 

I wonder if this could be done much easier, since TEXT_OFFSET must
actually be page aligned. So all we would need is an *upper bound* for
the bootloader size, as long as that is below 4K we wouldn't even loose
anything.

Cheers,
Andre.

> >   
> > > +kernel_load_offset += 2 << 20;  
> > 
> > You can use += 2 * MiB; which is easier to review.  
> 
> OK, I will include this in v2.

[Qemu-devel] [kvm-unit-tests PATCH v2 3/4] arm/arm64: GICv2: add GICD_IPRIORITYR testing

[Andre Przywara]

Some tests for the IPRIORITY registers. The significant number of bits
is IMPLEMENTATION DEFINED, but should be the same for every IRQ.
Also these registers must be byte-accessible.
Check that accesses beyond the implemented IRQ limit are actually
read-as-zero/write-ignore.

Signed-off-by: Andre Przywara 
---
 arm/gic.c | 79 +++
 1 file changed, 79 insertions(+)

diff --git a/arm/gic.c b/arm/gic.c
index 23cb9a4..57a2995 100644
--- a/arm/gic.c
+++ b/arm/gic.c
@@ -354,6 +354,83 @@ static void test_typer_v2(uint32_t reg)
   nr_gic_cpus);
 }
 
+#define BYTE(reg32, byte) (((reg32) >> ((byte) * 8)) & 0xff)
+#define REPLACE_BYTE(reg32, byte, new) (((reg32) & ~(0xff << ((byte) * 8))) |\
+   ((new) << ((byte) * 8)))
+
+/*
+ * Some registers are byte accessible, do a byte-wide read and write of known
+ * content to check for this.
+ * Apply a @mask to cater for special register properties.
+ * @pattern contains the value already in the register.
+ */
+static void test_byte_access(void *base_addr, u32 pattern, u32 mask)
+{
+   u32 reg = readb(base_addr + 1);
+
+   report("byte reads successful (0x%08x => 0x%02x)",
+  reg == (BYTE(pattern, 1) & (mask >> 8)),
+  pattern & mask, reg);
+
+   pattern = REPLACE_BYTE(pattern, 2, 0x1f);
+   writeb(BYTE(pattern, 2), base_addr + 2);
+   reg = readl(base_addr);
+   report("byte writes successful (0x%02x => 0x%08x)",
+  reg == (pattern & mask), BYTE(pattern, 2), reg);
+}
+
+static void test_priorities(int nr_irqs, void *priptr)
+{
+   u32 orig_prio, reg, pri_bits;
+   u32 pri_mask, pattern;
+   void *first_spi = priptr + GIC_FIRST_SPI;
+
+   orig_prio = readl(first_spi);
+   report_prefix_push("IPRIORITYR");
+
+   /*
+* Determine implemented number of priority bits by writing all 1's
+* and checking the number of cleared bits in the value read back.
+*/
+   writel(0x, first_spi);
+   pri_mask = readl(first_spi);
+
+   reg = ~pri_mask;
+   report("consistent priority masking (0x%08x)",
+  (((reg >> 16) == (reg & 0x)) &&
+   ((reg & 0xff) == ((reg >> 8) & 0xff))), pri_mask);
+
+   reg = reg & 0xff;
+   for (pri_bits = 8; reg & 1; reg >>= 1, pri_bits--)
+   ;
+   report("implements at least 4 priority bits (%d)",
+  pri_bits >= 4, pri_bits);
+
+   pattern = 0;
+   writel(pattern, first_spi);
+   report("clearing priorities", readl(first_spi) == pattern);
+
+   /* setting all priorities to their max valus was tested above */
+
+   report("accesses beyond limit RAZ/WI",
+  test_readonly_32(priptr + nr_irqs, true));
+
+   writel(pattern, priptr + nr_irqs - 4);
+   report("accessing last SPIs",
+  readl(priptr + nr_irqs - 4) == (pattern & pri_mask));
+
+   pattern = 0xff7fbf3f;
+   writel(pattern, first_spi);
+   report("priorities are preserved",
+  readl(first_spi) == (pattern & pri_mask));
+
+   /* The PRIORITY registers are byte accessible. */
+   test_byte_access(first_spi, pattern, pri_mask);
+
+   report_prefix_pop();
+   writel(orig_prio, first_spi);
+}
+
 static void gic_test_mmio(void)
 {
u32 reg;
@@ -388,6 +465,8 @@ static void gic_test_mmio(void)
report("ICPIDR2 is read-only (0x%08x)",
   test_readonly_32(idreg, false),
   reg);
+
+   test_priorities(nr_irqs, gic_dist_base + GICD_IPRIORITYR);
 }
 
 int main(int argc, char **argv)
-- 
2.14.4

[Qemu-devel] [kvm-unit-tests PATCH v2 4/4] arm/arm64: GICv2: add GICD_ITARGETSR testing

[Andre Przywara]

Some tests for the ITARGETS registers.
Bits corresponding to non-existent CPUs must be RAZ/WI.
These registers must be byte-accessible, also check that accesses beyond
the implemented IRQ limit are actually read-as-zero/write-ignore.

Signed-off-by: Andre Przywara 
---
 arm/gic.c | 43 +++
 lib/arm/asm/gic.h |  1 +
 2 files changed, 44 insertions(+)

diff --git a/arm/gic.c b/arm/gic.c
index 57a2995..ed5642e 100644
--- a/arm/gic.c
+++ b/arm/gic.c
@@ -431,6 +431,46 @@ static void test_priorities(int nr_irqs, void *priptr)
writel(orig_prio, first_spi);
 }
 
+/* GICD_ITARGETSR is only used by GICv2. */
+static void test_targets(int nr_irqs)
+{
+   void *targetsptr = gicv2_dist_base() + GICD_ITARGETSR;
+   u32 orig_targets;
+   u32 cpu_mask;
+   u32 pattern, reg;
+
+   orig_targets = readl(targetsptr + GIC_FIRST_SPI);
+   report_prefix_push("ITARGETSR");
+
+   cpu_mask = (1 << nr_cpus) - 1;
+   cpu_mask |= cpu_mask << 8;
+   cpu_mask |= cpu_mask << 16;
+
+   /* Check that bits for non implemented CPUs are RAZ/WI. */
+   if (nr_cpus < 8) {
+   writel(0x, targetsptr + GIC_FIRST_SPI);
+   report("bits for %d non-existent CPUs masked",
+  !(readl(targetsptr + GIC_FIRST_SPI) & ~cpu_mask),
+  8 - nr_cpus);
+   } else {
+   report_skip("CPU masking (all CPUs implemented)");
+   }
+
+   report("accesses beyond limit RAZ/WI",
+  test_readonly_32(targetsptr + nr_irqs, true));
+
+   pattern = 0x0103020f;
+   writel(pattern, targetsptr + GIC_FIRST_SPI);
+   reg = readl(targetsptr + GIC_FIRST_SPI);
+   report("register content preserved (%08x => %08x)",
+  reg == (pattern & cpu_mask), pattern & cpu_mask, reg);
+
+   /* The TARGETS registers are byte accessible. */
+   test_byte_access(targetsptr + GIC_FIRST_SPI, pattern, cpu_mask);
+
+   writel(orig_targets, targetsptr + GIC_FIRST_SPI);
+}
+
 static void gic_test_mmio(void)
 {
u32 reg;
@@ -467,6 +507,9 @@ static void gic_test_mmio(void)
   reg);
 
test_priorities(nr_irqs, gic_dist_base + GICD_IPRIORITYR);
+
+   if (gic_version() == 2)
+   test_targets(nr_irqs);
 }
 
 int main(int argc, char **argv)
diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
index a469645..f6dfb90 100644
--- a/lib/arm/asm/gic.h
+++ b/lib/arm/asm/gic.h
@@ -20,6 +20,7 @@
 #define GICD_ISACTIVER 0x0300
 #define GICD_ICACTIVER 0x0380
 #define GICD_IPRIORITYR0x0400
+#define GICD_ITARGETSR 0x0800
 #define GICD_SGIR  0x0f00
 #define GICD_ICPIDR2   0x0fe8
 
-- 
2.14.4

[Qemu-devel] [kvm-unit-tests PATCH v2 1/4] mark exit() and abort() as non-returning functions

[Andre Przywara]

exit() and abort() are functions that never return, and (at least)
GCC has an attribute to flag those functions accordingly. This allows
the compiler to do further optimizations and to omit various warnings
about uninitialized variables, for instance.
Since the actual "play-dead" function is in (inline) assembly, the
compiler does not recognize its fatal nature, so help it with the
__builtin_unreachable() hint.
Flag the prototypes of our fatal functions accordingly.

Signed-off-by: Andre Przywara 
---
 lib/arm/io.c | 1 +
 lib/libcflat.h   | 7 ---
 lib/powerpc/io.c | 1 +
 lib/x86/io.c | 1 +
 4 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/lib/arm/io.c b/lib/arm/io.c
index 603456f..d2c1a07 100644
--- a/lib/arm/io.c
+++ b/lib/arm/io.c
@@ -83,4 +83,5 @@ void exit(int code)
 {
chr_testdev_exit(code);
halt(code);
+   __builtin_unreachable();
 }
diff --git a/lib/libcflat.h b/lib/libcflat.h
index cc56553..7529958 100644
--- a/lib/libcflat.h
+++ b/lib/libcflat.h
@@ -84,8 +84,8 @@ typedef u64   phys_addr_t;
 extern void puts(const char *s);
 extern int __getchar(void);
 extern int getchar(void);
-extern void exit(int code);
-extern void abort(void);
+extern void exit(int code) __attribute__((noreturn));
+extern void abort(void) __attribute__((noreturn));
 extern long atol(const char *ptr);
 extern char *getenv(const char *name);
 
@@ -107,7 +107,8 @@ extern void report(const char *msg_fmt, bool pass, ...)
 extern void report_xfail(const char *msg_fmt, bool xfail, bool pass, ...)
__attribute__((format(printf, 1, 4)));
 extern void report_abort(const char *msg_fmt, ...)
-   __attribute__((format(printf, 1, 2)));
+   __attribute__((format(printf, 1, 2)))
+   __attribute__((noreturn));
 extern void report_skip(const char *msg_fmt, ...)
__attribute__((format(printf, 1, 2)));
 extern void report_info(const char *msg_fmt, ...)
diff --git a/lib/powerpc/io.c b/lib/powerpc/io.c
index 915e12e..217eb07 100644
--- a/lib/powerpc/io.c
+++ b/lib/powerpc/io.c
@@ -35,4 +35,5 @@ void exit(int code)
printf("\nEXIT: STATUS=%d\n", ((code) << 1) | 1);
rtas_power_off();
halt(code);
+   __builtin_unreachable();
 }
diff --git a/lib/x86/io.c b/lib/x86/io.c
index 7e1c16d..057f579 100644
--- a/lib/x86/io.c
+++ b/lib/x86/io.c
@@ -82,6 +82,7 @@ void exit(int code)
 #else
 asm volatile("out %0, %1" : : "a"(code), "d"((short)0xf4));
 #endif
+   __builtin_unreachable();
 }
 
 void __iomem *ioremap(phys_addr_t phys_addr, size_t size)
-- 
2.14.4

[Qemu-devel] [kvm-unit-tests PATCH v2 0/4] arm: add GICv2 MMIO tests

[Andre Przywara]

I found this in one my branches: this is an updated version of what I sent
end of 2016 [1]. I tried to address all comments that Drew and Eric had at
the time.
Please have a look whether this makes sense.

Changelog v1..v2:
- made many functions void
- use symbolic name for first SPI being number 32
- add test runs with one and three vCPUs
- use gic_version() directly
- factor out test_byte_access()
- drop redundant "filling priorities" test
- dropped GICv3 test

[1] https://lists.cs.columbia.edu/pipermail/kvmarm/2016-November/022352.html

Original cover letter:
==
The GIC spec mandates certain constraints on how to acccess the MMIO
mapped registers, both in terms of which registers are available and also
in terms of which bits within a register should be masked, for instance.
Since we went through some lengths in the KVM emulation to implement this,
it's about time to give this actually a test beyond what the kernel as a
GIC user actually implements - for instance we ignore priorities in Linux.

This series tries to attack some constraints, on a low-hanging-fruit base.
It focusses on some generic registers and the PRIORITY and TARGETS registers
of GICv2. GICv3 is not covered yet.

This actually revealed genuine bugs in the KVM emulation in the past. KVM
passes these tests now, but QEMU fails some UP and 3-way-SMP tests.

Cheers,
Andre.

Andre Przywara (4):
  mark exit() and abort() as non-returning functions
  arm/arm64: GIC: basic GICv2 MMIO tests
  arm/arm64: GICv2: add GICD_IPRIORITYR testing
  arm/arm64: GICv2: add GICD_ITARGETSR testing

 arm/gic.c | 213 ++
 arm/unittests.cfg |  18 +
 lib/arm/asm/gic.h |   5 ++
 lib/arm/io.c  |   1 +
 lib/libcflat.h|   7 +-
 lib/powerpc/io.c  |   1 +
 lib/x86/io.c  |   1 +
 7 files changed, 243 insertions(+), 3 deletions(-)

-- 
2.14.4

[Qemu-devel] [kvm-unit-tests PATCH v2 2/4] arm/arm64: GIC: basic GICv2 MMIO tests

[Andre Przywara]

This adds an MMIO subtest to the GIC test.
It accesses some generic GICv2 registers and does some sanity tests,
like checking for some of them being read-only.

Signed-off-by: Andre Przywara 
---
 arm/gic.c | 91 +++
 arm/unittests.cfg | 18 +++
 lib/arm/asm/gic.h |  4 +++
 3 files changed, 113 insertions(+)

diff --git a/arm/gic.c b/arm/gic.c
index 5dd958e..23cb9a4 100644
--- a/arm/gic.c
+++ b/arm/gic.c
@@ -3,6 +3,7 @@
  *
  * GICv2
  *   + test sending/receiving IPIs
+ *   + MMIO access tests
  * GICv3
  *   + test sending/receiving IPIs
  *
@@ -303,6 +304,92 @@ static void run_active_clear_test(void)
report_prefix_pop();
 }
 
+static bool test_ro_pattern_32(void *address, u32 pattern, u32 orig)
+{
+   u32 reg;
+
+   writel(pattern, address);
+   reg = readl(address);
+
+   if (reg != orig)
+   writel(orig, address);
+
+   return reg == orig;
+}
+
+static bool test_readonly_32(void *address, bool razwi)
+{
+   u32 orig, pattern;
+
+   orig = readl(address);
+   if (razwi && orig)
+   return false;
+
+   pattern = 0x;
+   if (orig != pattern) {
+   if (!test_ro_pattern_32(address, pattern, orig))
+   return false;
+   }
+
+   pattern = 0xa5a55a5a;
+   if (orig != pattern) {
+   if (!test_ro_pattern_32(address, pattern, orig))
+   return false;
+   }
+
+   pattern = 0;
+   if (orig != pattern) {
+   if (!test_ro_pattern_32(address, pattern, orig))
+   return false;
+   }
+
+   return true;
+}
+
+static void test_typer_v2(uint32_t reg)
+{
+   int nr_gic_cpus = ((reg >> 5) & 0x7) + 1;
+
+   report("all %d CPUs have interrupts", nr_cpus == nr_gic_cpus,
+  nr_gic_cpus);
+}
+
+static void gic_test_mmio(void)
+{
+   u32 reg;
+   int nr_irqs;
+   void *gic_dist_base, *idreg;
+
+   switch(gic_version()) {
+   case 0x2:
+   gic_dist_base = gicv2_dist_base();
+   idreg = gic_dist_base + GICD_ICPIDR2;
+   break;
+   case 0x3:
+   report_abort("GICv3 MMIO tests NYI");
+   default:
+   report_abort("GIC version %d not supported", gic_version());
+   }
+
+   reg = readl(gic_dist_base + GICD_TYPER);
+   nr_irqs = GICD_TYPER_IRQS(reg);
+   report_info("number of implemented SPIs: %d", nr_irqs - GIC_FIRST_SPI);
+
+   test_typer_v2(reg);
+
+   report_info("IIDR: 0x%08x", readl(gic_dist_base + GICD_IIDR));
+
+   report("GICD_TYPER is read-only",
+  test_readonly_32(gic_dist_base + GICD_TYPER, false));
+   report("GICD_IIDR is read-only",
+  test_readonly_32(gic_dist_base + GICD_IIDR, false));
+
+   reg = readl(idreg);
+   report("ICPIDR2 is read-only (0x%08x)",
+  test_readonly_32(idreg, false),
+  reg);
+}
+
 int main(int argc, char **argv)
 {
if (!gic_init()) {
@@ -330,6 +417,10 @@ int main(int argc, char **argv)
on_cpus(ipi_test, NULL);
} else if (strcmp(argv[1], "active") == 0) {
run_active_clear_test();
+   } else if (strcmp(argv[1], "mmio") == 0) {
+   report_prefix_push(argv[1]);
+   gic_test_mmio();
+   report_prefix_pop();
} else {
report_abort("Unknown subtest '%s'", argv[1]);
}
diff --git a/arm/unittests.cfg b/arm/unittests.cfg
index 44b98cf..7f3a321 100644
--- a/arm/unittests.cfg
+++ b/arm/unittests.cfg
@@ -86,6 +86,24 @@ smp = $((($MAX_SMP < 8)?$MAX_SMP:8))
 extra_params = -machine gic-version=2 -append 'ipi'
 groups = gic
 
+[gicv2-mmio]
+file = gic.flat
+smp = $((($MAX_SMP < 8)?$MAX_SMP:8))
+extra_params = -machine gic-version=2 -append 'mmio'
+groups = gic
+
+[gicv2-mmio-up]
+file = gic.flat
+smp = 1
+extra_params = -machine gic-version=2 -append 'mmio'
+groups = gic
+
+[gicv2-mmio-3p]
+file = gic.flat
+smp = $((($MAX_SMP < 3)?$MAX_SMP:3))
+extra_params = -machine gic-version=2 -append 'mmio'
+groups = gic
+
 [gicv3-ipi]
 file = gic.flat
 smp = $MAX_SMP
diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
index 2eb4af8..a469645 100644
--- a/lib/arm/asm/gic.h
+++ b/lib/arm/asm/gic.h
@@ -6,10 +6,13 @@
 #ifndef _ASMARM_GIC_H_
 #define _ASMARM_GIC_H_
 
+#define GIC_NR_PRIVATE_IRQS32
+#define GIC_FIRST_SPI  GIC_NR_PRIVATE_IRQS
 
 /* Distributor registers */
 #define GICD_CTLR  0x
 #define GICD_TYPER 0x0004
+#define GICD_IIDR  0x0008
 #define GICD_IGROUPR   0x0080
 #define GICD_ISENABLER 0x0100
 #define GICD_ISPENDR   0x0200
@@ -18,6 +21,7 @@
 #define GICD_ICACTI

[Qemu-devel] [kvm-unit-tests PATCH 0/2] arm64: extend arch timer tests

[Andre Przywara]

While debugging some timer emulation IRQ problem, I realised that the
failure pattern I was looking at is not covered by kvm-unit-tests.

Add a test for the TVAL register, also check that KVM correctly injects
the timer interrupt for the emulated physical timer when the guest is in
WFI (hint: it does not, starting with 4.15-rc, I am on it).

Cheers,
Andre.

Andre Przywara (2):
  arm64: timer: add TVAL accessors
  arm64: timer: Add TVAL timeout test and test IRQ trigger

 arm/timer.c | 36 
 1 file changed, 36 insertions(+)

-- 
2.14.4

[Qemu-devel] [kvm-unit-tests PATCH 1/2] arm64: timer: add TVAL accessors

[Andre Przywara]

The ARM arch timer features the CVAL register, which holds an absolute
value that is compared against the counter value.
But there is also the TVAL register, which is defined as (CVAL - counter),
and can be used to program or read relative timeouts.

Add accessors for the TVAL register, to be able to read and write them
easily later.

Signed-off-by: Andre Przywara 
---
 arm/timer.c | 26 ++
 1 file changed, 26 insertions(+)

diff --git a/arm/timer.c b/arm/timer.c
index 5f3135f..1c9ef44 100644
--- a/arm/timer.c
+++ b/arm/timer.c
@@ -40,6 +40,16 @@ static void write_vtimer_cval(u64 val)
write_sysreg(val, cntv_cval_el0);
 }
 
+static s32 read_vtimer_tval(void)
+{
+   return read_sysreg(cntv_tval_el0);
+}
+
+static void write_vtimer_tval(s32 val)
+{
+   write_sysreg(val, cntv_tval_el0);
+}
+
 static u64 read_vtimer_ctl(void)
 {
return read_sysreg(cntv_ctl_el0);
@@ -65,6 +75,16 @@ static void write_ptimer_cval(u64 val)
write_sysreg(val, cntp_cval_el0);
 }
 
+static s32 read_ptimer_tval(void)
+{
+   return read_sysreg(cntp_tval_el0);
+}
+
+static void write_ptimer_tval(s32 val)
+{
+   write_sysreg(val, cntp_tval_el0);
+}
+
 static u64 read_ptimer_ctl(void)
 {
return read_sysreg(cntp_ctl_el0);
@@ -82,6 +102,8 @@ struct timer_info {
u64 (*read_counter)(void);
u64 (*read_cval)(void);
void (*write_cval)(u64);
+   s32 (*read_tval)(void);
+   void (*write_tval)(s32);
u64 (*read_ctl)(void);
void (*write_ctl)(u64);
 };
@@ -91,6 +113,8 @@ static struct timer_info vtimer_info = {
.read_counter = read_vtimer_counter,
.read_cval = read_vtimer_cval,
.write_cval = write_vtimer_cval,
+   .read_tval = read_vtimer_tval,
+   .write_tval = write_vtimer_tval,
.read_ctl = read_vtimer_ctl,
.write_ctl = write_vtimer_ctl,
 };
@@ -100,6 +124,8 @@ static struct timer_info ptimer_info = {
.read_counter = read_ptimer_counter,
.read_cval = read_ptimer_cval,
.write_cval = write_ptimer_cval,
+   .read_tval = read_ptimer_tval,
+   .write_tval = write_ptimer_tval,
.read_ctl = read_ptimer_ctl,
.write_ctl = write_ptimer_ctl,
 };
-- 
2.14.4

[Qemu-devel] [kvm-unit-tests PATCH 2/2] arm64: timer: Add TVAL timeout IRQ trigger test

[Andre Przywara]

So far we were only testing the CVAL register. Add a test which programs
a (relative) TVAL value to check this functionality as well.
Also we go into WFI and wait for the interrupt to release us from it.
The timer tests are run with a 2 second timeout on the host side, so
that any failure in coming back would be covered.

Signed-off-by: Andre Przywara 
---
 arm/timer.c | 10 ++
 1 file changed, 10 insertions(+)

diff --git a/arm/timer.c b/arm/timer.c
index 1c9ef44..275d049 100644
--- a/arm/timer.c
+++ b/arm/timer.c
@@ -211,6 +211,7 @@ static void test_timer(struct timer_info *info)
u64 now = info->read_counter();
u64 time_10s = read_sysreg(cntfrq_el0) * 10;
u64 later = now + time_10s;
+   s32 left;
 
/* We don't want the irq handler to fire because that will change the
 * timer state and we want to test the timer output signal.  We can
@@ -236,6 +237,15 @@ static void test_timer(struct timer_info *info)
 
/* Disable the timer again */
info->write_ctl(0);
+
+   /* Test TVAL and IRQ trigger */
+   info->irq_received = false;
+   info->write_tval(read_sysreg(cntfrq_el0) / 100);/* 10 ms */
+   info->write_ctl(ARCH_TIMER_CTL_ENABLE);
+   wfi();
+   left = info->read_tval();
+   report("interrupt received after TVAL/WFI", info->irq_received);
+   report("timer has expired (%d)", left < 0, left);
 }
 
 static void test_vtimer(void)
-- 
2.14.4

Re: [Qemu-devel] [PATCH kvm-unit-tests v8 09/10] arm/arm64: gicv3: add an IPI test

[Andre Przywara]

pu >= nr_cpus)
> + break;
> +
> + mpidr = cpus[cpu];
> +
> + if (cluster_id != (mpidr & ~0xffUL)) {
> + /*
> +  * The next cpu isn't in our cluster. Roll
> +  * back the cpu index allowing the outer
> +  * for_each_cpu to find it again with
> +  * cpumask_next
> +  */
> + --cpu;
> + break;
> + }
> + }
> +
> + /* Send the IPIs for the target list of this cluster */
> + sgi1r = (MPIDR_TO_SGI_AFFINITY(cluster_id, 3)   |
> +  MPIDR_TO_SGI_AFFINITY(cluster_id, 2)   |
> +  irq << 24  |
> +  MPIDR_TO_SGI_AFFINITY(cluster_id, 1)   |
> +  tlist);
> +
> + gicv3_write_sgi1r(sgi1r);
> + }
> +
> + /* Force the above writes to ICC_SGI1R_EL1 to be executed */
> + isb();
> +}

Wow, this is really heavy stuff, especially for a Friday afternoon ;-)
But I convinced myself that it's correct. The only issue is that it's
sub-optimal if the MPIDRs of the VCPUs are not in order, say: 0x000,
0x100, 0x001.
In this case we do three register writes instead of the minimal two.
But it's still correct, so it's actually a minor nit just to prove that
I checked the algorithm ;-)

So apart from the minor comment above:

Reviewed-by: Andre Przywara <andre.przyw...@arm.com>

Cheers,
Andre.

> +
> +void gicv3_ipi_send_single(int irq, int cpu)
> +{
> + cpumask_t dest;
> +
> + cpumask_clear();
> + cpumask_set_cpu(cpu, );
> + gicv3_ipi_send_mask(irq, );
> +}
> diff --git a/lib/arm/gic.c b/lib/arm/gic.c
> index 4d3ddd9722b1..3ed539727f8c 100644
> --- a/lib/arm/gic.c
> +++ b/lib/arm/gic.c
> @@ -10,6 +10,38 @@
>  struct gicv2_data gicv2_data;
>  struct gicv3_data gicv3_data;
>  
> +struct gic_common_ops {
> + int gic_version;
> + void (*enable_defaults)(void);
> + u32 (*read_iar)(void);
> + u32 (*iar_irqnr)(u32 iar);
> + void (*write_eoir)(u32 irqstat);
> + void (*ipi_send_single)(int irq, int cpu);
> + void (*ipi_send_mask)(int irq, const cpumask_t *dest);
> +};
> +
> +static const struct gic_common_ops *gic_common_ops;
> +
> +static const struct gic_common_ops gicv2_common_ops = {
> + .gic_version = 2,
> + .enable_defaults = gicv2_enable_defaults,
> + .read_iar = gicv2_read_iar,
> + .iar_irqnr = gicv2_iar_irqnr,
> + .write_eoir = gicv2_write_eoir,
> + .ipi_send_single = gicv2_ipi_send_single,
> + .ipi_send_mask = gicv2_ipi_send_mask,
> +};
> +
> +static const struct gic_common_ops gicv3_common_ops = {
> + .gic_version = 3,
> + .enable_defaults = gicv3_enable_defaults,
> + .read_iar = gicv3_read_iar,
> + .iar_irqnr = gicv3_iar_irqnr,
> + .write_eoir = gicv3_write_eoir,
> + .ipi_send_single = gicv3_ipi_send_single,
> + .ipi_send_mask = gicv3_ipi_send_mask,
> +};
> +
>  /*
>   * Documentation/devicetree/bindings/interrupt-controller/arm,gic.txt
>   * Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.txt
> @@ -58,9 +90,58 @@ int gicv3_init(void)
>  
>  int gic_init(void)
>  {
> - if (gicv2_init())
> + if (gicv2_init()) {
> + gic_common_ops = _common_ops;
>   return 2;
> - else if (gicv3_init())
> + } else if (gicv3_init()) {
> + gic_common_ops = _common_ops;
>   return 3;
> + }
>   return 0;
>  }
> +
> +void gic_enable_defaults(void)
> +{
> + if (!gic_common_ops) {
> + int ret = gic_init();
> + assert(ret != 0);
> + } else
> + assert(gic_common_ops->enable_defaults);
> + gic_common_ops->enable_defaults();
> +}
> +
> +int gic_version(void)
> +{
> + assert(gic_common_ops);
> + return gic_common_ops->gic_version;
> +}
> +
> +u32 gic_read_iar(void)
> +{
> + assert(gic_common_ops && gic_common_ops->read_iar);
> + return gic_common_ops->read_iar();
> +}
> +
> +u32 gic_iar_irqnr(u32 iar)
> +{
> + assert(gic_common_ops && gic_common_ops->iar_irqnr);
> + return gic_common_ops->iar_irqnr(iar);
> +}
> +
> +void gic_write_eoir(u32 irqstat)
> +{
> + assert(gic_common_ops && gic_common_ops->write_eoir);
> + gic_common_ops->write_eoir(irqstat);
> +}
> +
> +void gic_ipi_send_single(int irq, int cpu)
> +{
> + assert(gic_common_ops && gic_common_ops->ipi_send_single);
> + gic_common_ops->ipi_send_single(irq, cpu);
> +}
> +
> +void gic_ipi_send_mask(int irq, const cpumask_t *dest)
> +{
> + assert(gic_common_ops && gic_common_ops->ipi_send_mask);
> + gic_common_ops->ipi_send_mask(irq, dest);
> +}
> 

Re: [Qemu-devel] [PATCH kvm-unit-tests v8 03/10] arm/arm64: add some delay routines

[Andre Przywara]

Hi,

On 08/12/16 17:50, Andrew Jones wrote:
> Allow a thread to wait some specified amount of time. Can
> specify in cycles, usecs, and msecs.
> 
> Signed-off-by: Andrew Jones <drjo...@redhat.com>
> 
> ---
> v8: rewrote basing on new sysreg framework. Also decided delay
> functions warrant their own files (delay.[ch])
> ---
>  arm/Makefile.common   |  1 +
>  lib/arm/asm/delay.h   | 14 ++
>  lib/arm/asm/processor.h   | 15 +++
>  lib/arm64/asm/delay.h |  1 +
>  lib/arm64/asm/processor.h | 12 
>  lib/arm/delay.c   | 29 +
>  6 files changed, 72 insertions(+)
>  create mode 100644 lib/arm/asm/delay.h
>  create mode 100644 lib/arm64/asm/delay.h
>  create mode 100644 lib/arm/delay.c
> 
> diff --git a/arm/Makefile.common b/arm/Makefile.common
> index b2c0fc8a2fdc..89fe3f69eb44 100644
> --- a/arm/Makefile.common
> +++ b/arm/Makefile.common
> @@ -48,6 +48,7 @@ cflatobjs += lib/arm/mmu.o
>  cflatobjs += lib/arm/bitops.o
>  cflatobjs += lib/arm/psci.o
>  cflatobjs += lib/arm/smp.o
> +cflatobjs += lib/arm/delay.o
>  
>  libeabi = lib/arm/libeabi.a
>  eabiobjs = lib/arm/eabi_compat.o
> diff --git a/lib/arm/asm/delay.h b/lib/arm/asm/delay.h
> new file mode 100644
> index ..2436b28c77ae
> --- /dev/null
> +++ b/lib/arm/asm/delay.h
> @@ -0,0 +1,14 @@
> +#ifndef _ASMARM_DELAY_H_
> +#define _ASMARM_DELAY_H_
> +/*
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjo...@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#include 
> +
> +extern void delay(u64 cycles);

Nit: Shouldn't this parameter be called "ticks"? Cycles might be a bit
misleading, especially since this prototype is the only documentation on
this. You might just want to fix this when applying the patches.

That notwithstanding:
Reviewed-by: Andre Przywara <andre.przyw...@arm.com>

Cheers,
Andre.

> +extern void udelay(unsigned long usecs);
> +extern void mdelay(unsigned long msecs);
> +
> +#endif /* _ASMARM_DELAY_H_ */
> diff --git a/lib/arm/asm/processor.h b/lib/arm/asm/processor.h
> index 6b0d36b87817..857bdd96a3cc 100644
> --- a/lib/arm/asm/processor.h
> +++ b/lib/arm/asm/processor.h
> @@ -7,6 +7,7 @@
>   */
>  #include 
>  #include 
> +#include 
>  
>  enum vector {
>   EXCPTN_RST,
> @@ -51,4 +52,18 @@ extern int mpidr_to_cpu(uint64_t mpidr);
>  extern void start_usr(void (*func)(void *arg), void *arg, unsigned long 
> sp_usr);
>  extern bool is_user(void);
>  
> +#define CNTVCT   __ACCESS_CP15_64(1, c14)
> +#define CNTFRQ   __ACCESS_CP15(c14, 0, c0, 0)
> +
> +static inline u64 get_cntvct(void)
> +{
> + isb();
> + return read_sysreg(CNTVCT);
> +}
> +
> +static inline u32 get_cntfrq(void)
> +{
> + return read_sysreg(CNTFRQ);
> +}
> +
>  #endif /* _ASMARM_PROCESSOR_H_ */
> diff --git a/lib/arm64/asm/delay.h b/lib/arm64/asm/delay.h
> new file mode 100644
> index ..288e4b3fe610
> --- /dev/null
> +++ b/lib/arm64/asm/delay.h
> @@ -0,0 +1 @@
> +#include "../../arm/asm/delay.h"
> diff --git a/lib/arm64/asm/processor.h b/lib/arm64/asm/processor.h
> index 48abf2c9e358..0898d89f9761 100644
> --- a/lib/arm64/asm/processor.h
> +++ b/lib/arm64/asm/processor.h
> @@ -20,6 +20,7 @@
>  #include 
>  #include 
>  #include 
> +#include 
>  
>  enum vector {
>   EL1T_SYNC,
> @@ -83,5 +84,16 @@ extern int mpidr_to_cpu(uint64_t mpidr);
>  extern void start_usr(void (*func)(void *arg), void *arg, unsigned long 
> sp_usr);
>  extern bool is_user(void);
>  
> +static inline u64 get_cntvct(void)
> +{
> + isb();
> + return read_sysreg(cntvct_el0);
> +}
> +
> +static inline u32 get_cntfrq(void)
> +{
> + return read_sysreg(cntfrq_el0);
> +}
> +
>  #endif /* !__ASSEMBLY__ */
>  #endif /* _ASMARM64_PROCESSOR_H_ */
> diff --git a/lib/arm/delay.c b/lib/arm/delay.c
> new file mode 100644
> index ..fa65e2dc9e35
> --- /dev/null
> +++ b/lib/arm/delay.c
> @@ -0,0 +1,29 @@
> +/*
> + * Delay loops
> + *
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjo...@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#include 
> +#include 
> +#include 
> +
> +void delay(u64 cycles)
> +{
> + u64 start = get_cntvct();
> +
> + while ((get_cntvct() - start) < cycles)
> + cpu_relax();
> +}
> +
> +void udelay(unsigned long usec)
> +{
> + delay((u64)usec * get_cntfrq() / 100);
> +}
> +
> +void mdelay(unsigned long msecs)
> +{
> + while (msecs--)
> + udelay(1000);
> +}
> 

Re: [Qemu-devel] [kvm-unit-tests RFC 00/15] arm/arm64: add ITS framework

[Andre Przywara]

Hi,

On 06/12/16 09:48, Andrew Jones wrote:
> On Mon, Dec 05, 2016 at 10:46:31PM +0100, Eric Auger wrote:
>> This series proposes a framework to test the virtual ITS.
>> This is based on Drew's v7 series [1]. The last patch tests
>> several ITS commands (collection/device mapping, interrupt
>> translation service entry creation and LPI trigger through INT
>> command). At this point we don't use any external PCIe device
>> to write into the GITS_TRANSLATER register.
>>
>> The bulk of the code derives from the ITS driver code so all
>> the credit is due to Marc.
>>
>> Many other ITS commands could be tested. Also existing MMIO
>> accesses could be enhanced into standalone tests. Current focus
>> was to make it functional.
>>
>> The code deserves more cleanup with respect to cacheability
>> attributes in general.
>>
>> Tested on Cavium ThunderX [2].
>>
>> Best Regards
>>
>> Eric
>>
>> [1] [kvm-unit-tests PATCH v7 00/11] arm/arm64: add gic framework
>>
>> [2] sample command line:
>>
>> $QEMU -machine virt,accel=kvm -cpu host \
>>  -device virtio-serial-device \
>>  -device virtconsole,chardev=ctd -chardev testdev,id=ctd \
>>  -display none -serial stdio \
>>  -kernel arm/gic.flat \
>>  -smp 8 -machine gic-version=3 -append 'its'
>>
>> Eric Auger (15):
>>   libcflat: Add other size defines
>>   arm/arm64: gicv3: Add some re-distributor defines
>>   arm/arm64: ITS skeleton
>>   arm/arm64: ITS: BASER parsing and setup
>>   arm/arm64: GICv3: add cpu count
>>   arm/arm64: ITS: Set the LPI config and pending tables
>>   arm/arm64: ITS: Init the command queue
>>   arm/arm64: ITS: enable LPIs at re-distributor level
>>   arm/arm64: ITS: Parse the typer register
>>   arm/arm64: ITS: its_enable_defaults
>>   arm/arm64: ITS: create device
>>   arm/arm64: ITS: create collection
>>   arm/arm64: ITS: commands
>>   arm/arm64: gic: Generalize ipi_enable()
>>   arm/arm64: ITS test
>>
>>  arm/Makefile.common|   1 +
>>  arm/gic.c  | 101 +++-
>>  lib/arm/asm/gic-v3-its.h   | 238 +++
>>  lib/arm/asm/gic-v3.h   |  84 ++
>>  lib/arm/asm/gic.h  |   1 +
>>  lib/arm/gic-v3-its-cmd.c   | 399 
>> +
>>  lib/arm/gic-v3-its.c   | 305 ++
>>  lib/arm/gic-v3.c   |   2 +
>>  lib/arm/gic.c  |  30 +++-
>>  lib/arm64/asm/gic-v3-its.h |   1 +
>>  lib/libcflat.h |   3 +
>>  11 files changed, 1154 insertions(+), 11 deletions(-)
>>  create mode 100644 lib/arm/asm/gic-v3-its.h
>>  create mode 100644 lib/arm/gic-v3-its-cmd.c
>>  create mode 100644 lib/arm/gic-v3-its.c
>>  create mode 100644 lib/arm64/asm/gic-v3-its.h
>>
>> -- 
>> 2.5.5
>>
>>
> 
> Thanks for this Eric! I'm glad to see we're getting more GIC test
> coverage written, even before v8 of the gic series is posted :-)
> v8 will be rebased on some sysreg stuff Wei is doing for the PMU
> series,

Are you planning on a v8 post any time soon?

> that's why it's held up. I'll need to set plenty of time
> aside to learn enough in order to review all the 'ITS:' patches
> in this series.

Are you sure you want to really taint yourself with this stuff? You
wouldn't be the first who risks his mental health by understanding the
ITS ;-)

That being said, I will take a look, I am in ITS land anyway for Xen ...

Cheers,
Andre.


> Apologies if I can't get to it right away.
> 
> Thanks again,
> drew
> 

Re: [Qemu-devel] [kvm-unit-tests RFC 05/15] arm/arm64: GICv3: add cpu count

[Andre Przywara]

Hi,

On 06/12/16 09:29, Andrew Jones wrote:
> On Mon, Dec 05, 2016 at 10:46:36PM +0100, Eric Auger wrote:
>> Add a new cpu_count field in gicv3_data indicating the
>> number of redistributors. This will be useful for enumeration
>> of their resources such as LPI pending tables.
> 
> I'm fine with the additional state, but just curious, will it
> ever be possible for gicv3.cpu_count != nr_cpus?

If not you are in trouble, so that should in fact be one test.

Which brings me to my comment ...

>>
>> Signed-off-by: Eric Auger 
>> ---
>>  lib/arm/asm/gic-v3.h | 1 +
>>  lib/arm/gic-v3.c | 2 ++
>>  2 files changed, 3 insertions(+)
>>
>> diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
>> index ed330af..039b7c2 100644
>> --- a/lib/arm/asm/gic-v3.h
>> +++ b/lib/arm/asm/gic-v3.h
>> @@ -58,6 +58,7 @@ struct gicv3_data {
>>  void *dist_base;
>>  void *redist_base[NR_CPUS];
>>  unsigned int irq_nr;
>> +unsigned int cpu_count;

Should that be called "nr_redists" or the like?
Since this is what it counts in the code below.
Later we can then compare this with nr_cpus to check for a match.

Cheers,
Andre.

>>  };
>>  extern struct gicv3_data gicv3_data;
>>  
>> diff --git a/lib/arm/gic-v3.c b/lib/arm/gic-v3.c
>> index 6246221..9921f4d 100644
>> --- a/lib/arm/gic-v3.c
>> +++ b/lib/arm/gic-v3.c
>> @@ -12,12 +12,14 @@ void gicv3_set_redist_base(size_t stride)
>>  void *ptr = gicv3_data.redist_base[0];
>>  u64 typer;
>>  
>> +gicv3_data.cpu_count = 0;
>>  do {
>>  typer = gicv3_read_typer(ptr + GICR_TYPER);
>>  if ((typer >> 32) == aff) {
>>  gicv3_redist_base() = ptr;
>>  return;
>>  }
>> +gicv3_data.cpu_count++;
>>  ptr += stride; /* skip RD_base, SGI_base, etc. */
>>  } while (!(typer & GICR_TYPER_LAST));
>>  
>> -- 
>> 2.5.5
>>
>>

Re: [Qemu-devel] [kvm-unit-tests PATCH v13 4/4] arm: pmu: Add CPI checking

[Andre Przywara]

Hi,

On 01/12/16 05:16, Wei Huang wrote:
> From: Christopher Covington 
> 
> Calculate the numbers of cycles per instruction (CPI) implied by ARM
> PMU cycle counter values. The code includes a strict checking facility
> intended for the -icount option in TCG mode in the configuration file.
> 
> Signed-off-by: Christopher Covington 
> Signed-off-by: Wei Huang 
> Reviewed-by: Andrew Jones 
> ---
>  arm/pmu.c | 123 
> +-
>  arm/unittests.cfg |  14 +++
>  2 files changed, 136 insertions(+), 1 deletion(-)
> 
> diff --git a/arm/pmu.c b/arm/pmu.c
> index 3566a27..29d7c2c 100644
> --- a/arm/pmu.c
> +++ b/arm/pmu.c
> @@ -69,6 +69,27 @@ static inline void set_pmccfiltr(uint32_t value)
>   set_pmxevtyper(value);
>   isb();
>  }
> +
> +/*
> + * Extra instructions inserted by the compiler would be difficult to 
> compensate
> + * for, so hand assemble everything between, and including, the PMCR accesses
> + * to start and stop counting. isb instructions were inserted to make sure
> + * pmccntr read after this function returns the exact instructions executed 
> in
> + * the controlled block. Total instrs = isb + mcr + 2*loop = 2 + 2*loop.
> + */
> +static inline void precise_instrs_loop(int loop, uint32_t pmcr)
> +{
> + asm volatile(
> + "   mcr p15, 0, %[pmcr], c9, c12, 0\n"
> + "   isb\n"
> + "1: subs%[loop], %[loop], #1\n"
> + "   bgt 1b\n"
> + "   mcr p15, 0, %[z], c9, c12, 0\n"
> + "   isb\n"
> + : [loop] "+r" (loop)
> + : [pmcr] "r" (pmcr), [z] "r" (0)
> + : "cc");
> +}
>  #elif defined(__aarch64__)
>  DEFINE_GET_SYSREG32(pmcr, el0)
>  DEFINE_SET_SYSREG32(pmcr, el0)
> @@ -77,6 +98,27 @@ DEFINE_GET_SYSREG64(pmccntr, el0);
>  DEFINE_SET_SYSREG64(pmccntr, el0);
>  DEFINE_SET_SYSREG32(pmcntenset, el0);
>  DEFINE_SET_SYSREG32(pmccfiltr, el0);
> +
> +/*
> + * Extra instructions inserted by the compiler would be difficult to 
> compensate
> + * for, so hand assemble everything between, and including, the PMCR accesses
> + * to start and stop counting. isb instructions are inserted to make sure
> + * pmccntr read after this function returns the exact instructions executed
> + * in the controlled block. Total instrs = isb + msr + 2*loop = 2 + 2*loop.
> + */
> +static inline void precise_instrs_loop(int loop, uint32_t pmcr)
> +{
> + asm volatile(
> + "   msr pmcr_el0, %[pmcr]\n"
> + "   isb\n"
> + "1: subs%[loop], %[loop], #1\n"
> + "   b.gt1b\n"
> + "   msr pmcr_el0, xzr\n"
> + "   isb\n"
> + : [loop] "+r" (loop)
> + : [pmcr] "r" (pmcr)
> + : "cc");
> +}
>  #endif
>  
>  /*
> @@ -134,6 +176,79 @@ static bool check_cycles_increase(void)
>   return success;
>  }
>  
> +/*
> + * Execute a known number of guest instructions. Only even instruction counts
> + * greater than or equal to 4 are supported by the in-line assembly code. The
> + * control register (PMCR_EL0) is initialized with the provided value 
> (allowing
> + * for example for the cycle counter or event counters to be reset). At the 
> end
> + * of the exact instruction loop, zero is written to PMCR_EL0 to disable
> + * counting, allowing the cycle counter or event counters to be read at the
> + * leisure of the calling code.
> + */
> +static void measure_instrs(int num, uint32_t pmcr)
> +{
> + int loop = (num - 2) / 2;
> +
> + assert(num >= 4 && ((num - 2) % 2 == 0));
> + precise_instrs_loop(loop, pmcr);
> +}
> +
> +/*
> + * Measure cycle counts for various known instruction counts. Ensure that the
> + * cycle counter progresses (similar to check_cycles_increase() but with more
> + * instructions and using reset and stop controls). If supplied a positive,
> + * nonzero CPI parameter, also strictly check that every measurement matches
> + * it. Strict CPI checking is used to test -icount mode.
> + */
> +static bool check_cpi(int cpi)
> +{
> + uint32_t pmcr = get_pmcr() | PMU_PMCR_LC | PMU_PMCR_C | PMU_PMCR_E;
> +
> + /* init before event access, this test only cares about cycle count */
> + set_pmcntenset(1 << PMU_CYCLE_IDX);
> + set_pmccfiltr(0); /* count cycles in EL0, EL1, but not EL2 */
> +
> + if (cpi > 0)
> + printf("Checking for CPI=%d.\n", cpi);
> + printf("instrs : cycles0 cycles1 ...\n");

Do we really need this line?

In general I find the output quite confusing, actually distracting from
the other, actual tests. To make it more readable, I tweaked it a bit to
look like:
  4: 9996  173  222  122  118  119  120  212  240  233 avg=1155: 288 cpi
 36:  773  282  291  314  291  335  315  264  162  308 avg= 333:   9 cpi
 68:  229  356  400  339  203  201  335  233  201  372 avg= 286:   4 cpi

with some padding hints and limiting the line to at most 80 characters, by:

> +
> + for (unsigned int 

Re: [Qemu-devel] [kvm-unit-tests PATCH v13 2/4] arm: Add PMU test

[Andre Przywara]

Hi,

On 01/12/16 12:02, Peter Maydell wrote:
> On 1 December 2016 at 11:28, Andre Przywara <andre.przyw...@arm.com> wrote:
>> I don't think so. At least here as a _variable_ type uint32_t is
>> probably the right one, as the ARMv8 ARM explicitly says that PMCR is a
>> 32-bit register, for both bitnesses.
> 
> For 64-bit ARM this is strictly speaking just shorthand for "64-bit
> register with the top 32-bit being RES0". It is in theory possible that
> a future architecture extension might define uses for those RES0
> bits.

I trade: "in theory possible that a future architecture extension might"
(that's four speculative terms, right?) against:

ARMv8 ARM, D7.4.7  PMCR_EL0, Performance Monitors Control Register:
Attributes
PMCR_EL0 is a 32-bit register.

If this ever gets extended, we would need extra code to deal with the
new bits, so we would need to touch the code anyway. And again, it's
just a local variable, not an interface.

Cheers,
Andre.

P.S. We really should save this discussion for a Friday afternoon ;-)

Re: [Qemu-devel] [kvm-unit-tests PATCH v13 2/4] arm: Add PMU test

[Andre Przywara]

Hi,

On 01/12/16 09:03, Andrew Jones wrote:
> On Wed, Nov 30, 2016 at 11:16:40PM -0600, Wei Huang wrote:
>> From: Christopher Covington 
>>
>> Beginning with a simple sanity check of the control register, add
>> a unit test for the ARM Performance Monitors Unit (PMU). PMU register
>> was read using the newly defined macros.
>>
>> Signed-off-by: Christopher Covington 
>> Signed-off-by: Wei Huang 
>> Reviewed-by: Andrew Jones 
>> ---
>>  arm/Makefile.common |  3 ++-
>>  arm/pmu.c   | 62 
>> +
>>  arm/unittests.cfg   |  5 +
>>  3 files changed, 69 insertions(+), 1 deletion(-)
>>  create mode 100644 arm/pmu.c
>>
>> diff --git a/arm/Makefile.common b/arm/Makefile.common
>> index f37b5c2..5da2fdd 100644
>> --- a/arm/Makefile.common
>> +++ b/arm/Makefile.common
>> @@ -12,7 +12,8 @@ endif
>>  tests-common = \
>>  $(TEST_DIR)/selftest.flat \
>>  $(TEST_DIR)/spinlock-test.flat \
>> -$(TEST_DIR)/pci-test.flat
>> +$(TEST_DIR)/pci-test.flat \
>> +$(TEST_DIR)/pmu.flat
>>  
>>  all: test_cases
>>  
>> diff --git a/arm/pmu.c b/arm/pmu.c
>> new file mode 100644
>> index 000..1fe2b1a
>> --- /dev/null
>> +++ b/arm/pmu.c
>> @@ -0,0 +1,62 @@
>> +/*
>> + * Test the ARM Performance Monitors Unit (PMU).
>> + *
>> + * Copyright (c) 2015-2016, The Linux Foundation. All rights reserved.
>> + *
>> + * This program is free software; you can redistribute it and/or modify it
>> + * under the terms of the GNU Lesser General Public License version 2.1 and
>> + * only version 2.1 as published by the Free Software Foundation.
>> + *
>> + * This program is distributed in the hope that it will be useful, but 
>> WITHOUT
>> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
>> + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public 
>> License
>> + * for more details.
>> + */
>> +#include "libcflat.h"
>> +#include "asm/barrier.h"
>> +#include "asm/processor.h"
>> +
>> +#define PMU_PMCR_N_SHIFT   11
>> +#define PMU_PMCR_N_MASK0x1f
>> +#define PMU_PMCR_ID_SHIFT  16
>> +#define PMU_PMCR_ID_MASK   0xff
>> +#define PMU_PMCR_IMP_SHIFT 24
>> +#define PMU_PMCR_IMP_MASK  0xff
>> +
>> +#if defined(__arm__)
>> +DEFINE_GET_SYSREG32(pmcr, 0, c9, c12, 0)
>> +#elif defined(__aarch64__)
>> +DEFINE_GET_SYSREG32(pmcr, el0)
>> +#endif
>> +
>> +/*
>> + * As a simple sanity check on the PMCR_EL0, ensure the implementer field 
>> isn't
>> + * null. Also print out a couple other interesting fields for diagnostic
>> + * purposes. For example, as of fall 2016, QEMU TCG mode doesn't implement
>> + * event counters and therefore reports zero event counters, but hopefully
>> + * support for at least the instructions event will be added in the future 
>> and
>> + * the reported number of event counters will become nonzero.
>> + */
>> +static bool check_pmcr(void)
>> +{
>> +uint32_t pmcr;
> 
> So based on my comments from the previous patch, pmcr should be
> 'unsigned int'

I don't think so. At least here as a _variable_ type uint32_t is
probably the right one, as the ARMv8 ARM explicitly says that PMCR is a
32-bit register, for both bitnesses. I find it only natural to express
it here accordingly.
I believe this is a different (though related) discussion from the
return type of the accessor functions.

Cheers,
Andre.

>> +
>> +pmcr = get_pmcr();
>> +
>> +report_info("PMU implementer/ID code/counters: 0x%x(\"%c\")/0x%x/%d",
>> +(pmcr >> PMU_PMCR_IMP_SHIFT) & PMU_PMCR_IMP_MASK,
>> +((pmcr >> PMU_PMCR_IMP_SHIFT) & PMU_PMCR_IMP_MASK) ? : ' ',
>> +(pmcr >> PMU_PMCR_ID_SHIFT) & PMU_PMCR_ID_MASK,
>> +(pmcr >> PMU_PMCR_N_SHIFT) & PMU_PMCR_N_MASK);
>> +
>> +return ((pmcr >> PMU_PMCR_IMP_SHIFT) & PMU_PMCR_IMP_MASK) != 0;
>> +}
>> +
>> +int main(void)
>> +{
>> +report_prefix_push("pmu");
>> +
>> +report("Control register", check_pmcr());
>> +
>> +return report_summary();
>> +}
>> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
>> index ae32a42..816f494 100644
>> --- a/arm/unittests.cfg
>> +++ b/arm/unittests.cfg
>> @@ -58,3 +58,8 @@ groups = selftest
>>  [pci-test]
>>  file = pci-test.flat
>>  groups = pci
>> +
>> +# Test PMU support
>> +[pmu]
>> +file = pmu.flat
>> +groups = pmu
>> -- 
>> 1.8.3.1
>>
>>
> 
> drew 
> 

Re: [Qemu-devel] [kvm-unit-tests PATCH v13 3/4] arm: pmu: Check cycle count increases

[Andre Przywara]

Hi,

On 01/12/16 05:16, Wei Huang wrote:
> From: Christopher Covington 
> 
> Ensure that reads of the PMCCNTR_EL0 are monotonically increasing,
> even for the smallest delta of two subsequent reads.
> 
> Signed-off-by: Christopher Covington 
> Signed-off-by: Wei Huang 
> Reviewed-by: Andrew Jones 
> ---
>  arm/pmu.c | 94 
> +++
>  1 file changed, 94 insertions(+)
> 
> diff --git a/arm/pmu.c b/arm/pmu.c
> index 1fe2b1a..3566a27 100644
> --- a/arm/pmu.c
> +++ b/arm/pmu.c
> @@ -16,6 +16,9 @@
>  #include "asm/barrier.h"
>  #include "asm/processor.h"
>  
> +#define PMU_PMCR_E (1 << 0)
> +#define PMU_PMCR_C (1 << 2)
> +#define PMU_PMCR_LC(1 << 6)
>  #define PMU_PMCR_N_SHIFT   11
>  #define PMU_PMCR_N_MASK0x1f
>  #define PMU_PMCR_ID_SHIFT  16
> @@ -23,10 +26,57 @@
>  #define PMU_PMCR_IMP_SHIFT 24
>  #define PMU_PMCR_IMP_MASK  0xff
>  
> +#define ID_DFR0_PERFMON_SHIFT 24
> +#define ID_DFR0_PERFMON_MASK  0xf
> +
> +#define PMU_CYCLE_IDX 31
> +
> +#define NR_SAMPLES 10
> +
> +static unsigned int pmu_version;
>  #if defined(__arm__)
>  DEFINE_GET_SYSREG32(pmcr, 0, c9, c12, 0)
> +DEFINE_SET_SYSREG32(pmcr, 0, c9, c12, 0)
> +DEFINE_GET_SYSREG32(id_dfr0, 0, c0, c1, 2)
> +DEFINE_SET_SYSREG32(pmselr, 0, c9, c12, 5)
> +DEFINE_SET_SYSREG32(pmxevtyper, 0, c9, c13, 1)
> +DEFINE_GET_SYSREG32(pmccntr32, 0, c9, c13, 0)
> +DEFINE_SET_SYSREG32(pmccntr32, 0, c9, c13, 0)
> +DEFINE_GET_SYSREG64(pmccntr64, 0, c9)
> +DEFINE_SET_SYSREG64(pmccntr64, 0, c9)
> +DEFINE_SET_SYSREG32(pmcntenset, 0, c9, c12, 1)
> +
> +static inline uint64_t get_pmccntr(void)
> +{
> + if (pmu_version == 0x3)
> + return get_pmccntr64();
> + else
> + return get_pmccntr32();
> +}
> +
> +static inline void set_pmccntr(uint64_t value)
> +{
> + if (pmu_version == 0x3)
> + set_pmccntr64(value);
> + else
> + set_pmccntr32(value & 0x);
> +}
> +
> +/* PMCCFILTR is an obsolete name for PMXEVTYPER31 in ARMv7 */
> +static inline void set_pmccfiltr(uint32_t value)
> +{
> + set_pmselr(PMU_CYCLE_IDX);
> + set_pmxevtyper(value);
> + isb();
> +}
>  #elif defined(__aarch64__)
>  DEFINE_GET_SYSREG32(pmcr, el0)
> +DEFINE_SET_SYSREG32(pmcr, el0)
> +DEFINE_GET_SYSREG32(id_dfr0, el1)
> +DEFINE_GET_SYSREG64(pmccntr, el0);
> +DEFINE_SET_SYSREG64(pmccntr, el0);
> +DEFINE_SET_SYSREG32(pmcntenset, el0);
> +DEFINE_SET_SYSREG32(pmccfiltr, el0);
>  #endif
>  
>  /*
> @@ -52,11 +102,55 @@ static bool check_pmcr(void)
>   return ((pmcr >> PMU_PMCR_IMP_SHIFT) & PMU_PMCR_IMP_MASK) != 0;
>  }
>  
> +/*
> + * Ensure that the cycle counter progresses between back-to-back reads.
> + */
> +static bool check_cycles_increase(void)
> +{
> + bool success = true;
> +
> + /* init before event access, this test only cares about cycle count */
> + set_pmcntenset(1 << PMU_CYCLE_IDX);
> + set_pmccfiltr(0); /* count cycles in EL0, EL1, but not EL2 */
> + set_pmccntr(0);

Why do we need this? Shouldn't PMU_PMCR_C below take care of that?

> +
> + set_pmcr(get_pmcr() | PMU_PMCR_LC | PMU_PMCR_C | PMU_PMCR_E);
> +
> + for (int i = 0; i < NR_SAMPLES; i++) {
> + uint64_t a, b;
> +
> + a = get_pmccntr();
> + b = get_pmccntr();
> +
> + if (a >= b) {
> + printf("Read %"PRId64" then %"PRId64".\n", a, b);
> + success = false;
> + break;
> + }
> + }
> +
> + set_pmcr(get_pmcr() & ~PMU_PMCR_E);
> +
> + return success;
> +}
> +
> +void pmu_init(void)

Mmh, this function doesn't really initialize anything, does it?
Should it be named pmu_available() or pmu_version() or the like?

And should we bail out early here (or rather at the caller) if this
register reports that no PMU is available? For instance by making it
return a boolean?

> +{
> + uint32_t dfr0;
> +
> + /* probe pmu version */
> + dfr0 = get_id_dfr0();
> + pmu_version = (dfr0 >> ID_DFR0_PERFMON_SHIFT) & ID_DFR0_PERFMON_MASK;
> + report_info("PMU version: %d", pmu_version);
> +}
> +
>  int main(void)
>  {
>   report_prefix_push("pmu");
>  
> + pmu_init();
>   report("Control register", check_pmcr());
> + report("Monotonically increasing cycle count", check_cycles_increase());

I wonder if we should skip this test if check_pmcr() has returned false
before? We let it return a boolean, so it seems quite natural to use
this information here.
This would avoid a lot of false FAILs due to the PMU not being available
(because QEMU is too old, for instance).

Cheers,
Andre.

Re: [Qemu-devel] [kvm-unit-tests PATCH v13 1/4] arm: Define macros for accessing system registers

[Andre Przywara]

Hi,

On 01/12/16 08:59, Andrew Jones wrote:
> 
> Should this be From: Andre?

No need from my side, this way all the bug reports are send to Wei ;-)

> On Wed, Nov 30, 2016 at 11:16:39PM -0600, Wei Huang wrote:
>> This patch defines four macros to assist creating system register
>> accessors under both ARMv7 and AArch64:
>>* DEFINE_GET_SYSREG32(name, ...)
>>* DEFINE_SET_SYSREG32(name, ...)
>>* DEFINE_GET_SYSREG64(name, ...)
>>* DEFINE_SET_SYSREG64(name, ...)
>> These macros are translated to inline functions with consistent naming,
>> get_##name() and set_##name(), which can be used by C code directly.
>>
>> Signed-off-by: Andre Przywara <andre.przyw...@arm.com>
>> Signed-off-by: Wei Huang <w...@redhat.com>
>> ---
>>  lib/arm/asm/processor.h   | 37 -
>>  lib/arm64/asm/processor.h | 35 ---
>>  2 files changed, 60 insertions(+), 12 deletions(-)
>>
>> diff --git a/lib/arm/asm/processor.h b/lib/arm/asm/processor.h
>> index f25e7ee..3ca6b42 100644
>> --- a/lib/arm/asm/processor.h
>> +++ b/lib/arm/asm/processor.h
>> @@ -33,13 +33,40 @@ static inline unsigned long current_cpsr(void)
>>  
>>  #define current_mode() (current_cpsr() & MODE_MASK)
>>  
>> -static inline unsigned int get_mpidr(void)
>> -{
>> -unsigned int mpidr;
>> -asm volatile("mrc p15, 0, %0, c0, c0, 5" : "=r" (mpidr));
>> -return mpidr;
>> +#define DEFINE_GET_SYSREG32(name, opc1, crn, crm, opc2) 
>> \
>> +static inline uint32_t get_##name(void) 
>> \
>> +{   \
>> +uint32_t reg;   \
>> +asm volatile("mrc p15, " #opc1 ", %0, " #crn ", " #crm ", " \
>> + #opc2 : "=r" (reg));   \
>> +return reg; \
>> +}
>> +
>> +#define DEFINE_SET_SYSREG32(name, opc1, crn, crm, opc2) 
>> \
>> +static inline void set_##name(uint32_t value)   
>> \
>> +{   \
>> +asm volatile("mcr p15, " #opc1 ", %0, " #crn ", " #crm ", " \
>> + #opc2 :: "r" (value)); \
>^ nit: no space here, checkpatch would complain
>> +}
>> +
>> +#define DEFINE_GET_SYSREG64(name, opc, crm) \
>> +static inline uint64_t get_##name(void) 
>> \
>> +{   \
>> +uint32_t lo, hi;\
>> +asm volatile("mrrc p15, " #opc ", %0, %1, " #crm\
>> + : "=r" (lo), "=r" (hi));   \
>> +return (uint64_t)hi << 32 | lo; \
>> +}
>> +
>> +#define DEFINE_SET_SYSREG64(name, opc, crm) \
>> +static inline void set_##name(uint64_t value)   
>> \
>> +{   \
>> +asm volatile("mcrr p15, " #opc ", %0, %1, " #crm\
>> + :: "r" (value & 0x), "r" (value >> 32));   \
>>  }
>>  
>> +DEFINE_GET_SYSREG32(mpidr, 0, c0, c0, 5)
>> +
>>  /* Only support Aff0 for now, up to 4 cpus */
>>  #define mpidr_to_cpu(mpidr) ((int)((mpidr) & 0xff))
>>  
>> diff --git a/lib/arm64/asm/processor.h b/lib/arm64/asm/processor.h
>> index 84d5c7c..dfa75eb 100644
>> --- a/lib/arm64/asm/processor.h
>> +++ b/lib/arm64/asm/processor.h
>> @@ -66,14 +66,35 @@ static inline unsigned long current_level(void)
>>  return el & 0xc;
>>  }
>>  
>> -#define DEFINE_GET_SYSREG32(reg)\
>> -static inline unsigned int get_##reg(void)  \
>> -{   \
>> -unsigned int reg;   \
>> -asm volatile("mrs %0, " #reg "_el1" : "=r" (reg));  \
>> -return reg; 

Re: [Qemu-devel] [kvm-unit-tests PATCH v13 4/4] arm: pmu: Add CPI checking

[Andre Przywara]

Hi Drew,

actually unrelated to this actual patch, but since you mentioned it:

> As we work out how best to handle tcg-only tests in order to get Alex
> Bennee's MTTCG tests merged, we'll probably revisit this file,

So when I was experimenting with kvmtool, I realized that some tests
would need to be QEMU only. Also I was tempted to try some tests either
on bare metal machines or in a Fast Model.
So I wonder if we should have some constraints or tags on the tests, so
that a certain backend can filter on this and skip if it's not capable?

Just wanted to mention this so that we can use this refactoring
opportunity to come up with something more generic than just a boolean
TGC vs. KVM.
Maybe we should introduce the notion of a "test backend"? That could be
QEMU/KVM and TCG for now, but later extended to cover kvmtool and
probably other hypervisors like Xen as well.

Cheers,
Andre.

Re: [Qemu-devel] [kvm-unit-tests PATCH v11 1/3] arm: Add PMU test

[Andre Przywara]

Hi,

On 25/11/16 14:26, Andrew Jones wrote:
> On Fri, Nov 25, 2016 at 12:32:24PM +0000, Andre Przywara wrote:
>> Hi Drew,
>>
>> 
>>
>> On 23/11/16 17:15, Andrew Jones wrote:
>>>>> +
>>>>> +#if defined(__arm__)
>>>>
>>>> I guess you should use the arch specific header files we have in place
>>>> for that (lib/arm{.64}/asm/processor.h). Also there are sysreg read
>>>> wrappers (at least for arm64) in there already, can't we base this
>>>> function on them: DEFINE_GET_SYSREG32(pmcr, el0)?
>>>> (Requires a small change to get rid of the forced "_el1" suffix)
>>>>
>>>> We should wait for the GIC series to be merged, as this contains some
>>>> changes in this area.
>>>
>>> As this unit test is the only consumer of PMC registers so far, then
>>> I'd prefer the defines and accessors stay here for now. Once we see
>>> a use in other unit tests then we can move some of it out.
>>
>> Well, I was more thinking of something like below.
>> I am fine with keeping the PMU sysregs private to pmu.c, but we can still
>> use the sysreg wrappers, can't we?
>> This is on top of Wei's series, so doesn't have your SYSREG32/64
>> unification, but I leave this as an exercise to the reader.
>> There is some churn in pmu.c below due to the change of _write to
>> set_, but the rest looks like simplification to me.
>>
>> Does that make sense?
> 
> Ah, now I see what you mean, and I think I like that. The question is
> whether or not I like my SYSREG macros :-) I see value in having the
> asm's easy to read (open-coded),

Mmh, where is this easy to read? Especially the v7 syntax is really
error prone and hard to grasp with all these c9s, c12s and friends, in
addition to the weird ordering and mixing in of register names in the
(inline) assembly syntax. Blame the company that came up with this
architecture ;-)

Compare:
mrc p15, 0, %0, c9, c12, 0
with:
DEFINE_GET_SYSREG32(pmcr, 0, c9, c12, 0)

The "0, %0" in there always kills me when reading this.

On top of this a list of one-line-per-sysreg declarations is much easier
to compare with the manual than the open coded versions.

Another benefit is that we don't need to encode versions for both
bitnesses in each test. For shared sysregs we could put the definitions
in processor.h, the code then just uses one architecture agnostic version.

But I leave it up to you guys to decide on this, you don't break my
heart if you stick with inline assembly ;-)

Cheers,
Andre.

 as well as value in making sure we
> only have to review sysreg functions once. Let's ask for Wei's and
> Cov's votes. If they like the SYSREG direction, then they can vote
> with another version of this series :-)
> 
> Thanks,
> drew
> 
>>
>> Cheers,
>> Andre.
>>
>> ---
>>  arm/pmu.c | 159 
>> +-
>>  lib/arm/asm/processor.h   |  34 --
>>  lib/arm64/asm/processor.h |  23 ++-
>>  3 files changed, 92 insertions(+), 124 deletions(-)
>>
>> diff --git a/arm/pmu.c b/arm/pmu.c
>> index f667676..f0ad02a 100644
>> --- a/arm/pmu.c
>> +++ b/arm/pmu.c
>> @@ -14,6 +14,7 @@
>>   */
>>  #include "libcflat.h"
>>  #include "asm/barrier.h"
>> +#include "asm/processor.h"
>>  
>>  #define PMU_PMCR_E (1 << 0)
>>  #define PMU_PMCR_C (1 << 2)
>> @@ -33,78 +34,42 @@
>>  #define NR_SAMPLES 10
>>  
>>  static unsigned int pmu_version;
>> -#if defined(__arm__)
>> -static inline uint32_t pmcr_read(void)
>> -{
>> -uint32_t ret;
>> -
>> -asm volatile("mrc p15, 0, %0, c9, c12, 0" : "=r" (ret));
>> -return ret;
>> -}
>> -
>> -static inline void pmcr_write(uint32_t value)
>> -{
>> -asm volatile("mcr p15, 0, %0, c9, c12, 0" : : "r" (value));
>> -isb();
>> -}
>>  
>> -static inline void pmselr_write(uint32_t value)
>> -{
>> -asm volatile("mcr p15, 0, %0, c9, c12, 5" : : "r" (value));
>> -isb();
>> -}
>> -
>> -static inline void pmxevtyper_write(uint32_t value)
>> -{
>> -asm volatile("mcr p15, 0, %0, c9, c13, 1" : : "r" (value));
>> -}
>> -
>> -static inline uint64_t pmccntr_read(void)
>> +#if defined(__arm__)
>> +DEFINE_GET_SYSREG32(pmcr, 0, c9, c12, 0)
>> +DEFINE_SET_SYSREG32(pmcr, 0, c9, c12, 0)
>> +DEFINE_GET_SYSREG32(id_dfr0, 0, c0, c1, 2)
>> +DEFINE_SE

Re: [Qemu-devel] [kvm-unit-tests PATCH v11 1/3] arm: Add PMU test

[Andre Przywara]

Hi Drew,



On 23/11/16 17:15, Andrew Jones wrote:
>>> +
>>> +#if defined(__arm__)
>>
>> I guess you should use the arch specific header files we have in place
>> for that (lib/arm{.64}/asm/processor.h). Also there are sysreg read
>> wrappers (at least for arm64) in there already, can't we base this
>> function on them: DEFINE_GET_SYSREG32(pmcr, el0)?
>> (Requires a small change to get rid of the forced "_el1" suffix)
>>
>> We should wait for the GIC series to be merged, as this contains some
>> changes in this area.
>
> As this unit test is the only consumer of PMC registers so far, then
> I'd prefer the defines and accessors stay here for now. Once we see
> a use in other unit tests then we can move some of it out.

Well, I was more thinking of something like below.
I am fine with keeping the PMU sysregs private to pmu.c, but we can still
use the sysreg wrappers, can't we?
This is on top of Wei's series, so doesn't have your SYSREG32/64
unification, but I leave this as an exercise to the reader.
There is some churn in pmu.c below due to the change of _write to
set_, but the rest looks like simplification to me.

Does that make sense?

Cheers,
Andre.

---
 arm/pmu.c | 159 +-
 lib/arm/asm/processor.h   |  34 --
 lib/arm64/asm/processor.h |  23 ++-
 3 files changed, 92 insertions(+), 124 deletions(-)

diff --git a/arm/pmu.c b/arm/pmu.c
index f667676..f0ad02a 100644
--- a/arm/pmu.c
+++ b/arm/pmu.c
@@ -14,6 +14,7 @@
  */
 #include "libcflat.h"
 #include "asm/barrier.h"
+#include "asm/processor.h"
 
 #define PMU_PMCR_E (1 << 0)
 #define PMU_PMCR_C (1 << 2)
@@ -33,78 +34,42 @@
 #define NR_SAMPLES 10
 
 static unsigned int pmu_version;
-#if defined(__arm__)
-static inline uint32_t pmcr_read(void)
-{
-   uint32_t ret;
-
-   asm volatile("mrc p15, 0, %0, c9, c12, 0" : "=r" (ret));
-   return ret;
-}
-
-static inline void pmcr_write(uint32_t value)
-{
-   asm volatile("mcr p15, 0, %0, c9, c12, 0" : : "r" (value));
-   isb();
-}
 
-static inline void pmselr_write(uint32_t value)
-{
-   asm volatile("mcr p15, 0, %0, c9, c12, 5" : : "r" (value));
-   isb();
-}
-
-static inline void pmxevtyper_write(uint32_t value)
-{
-   asm volatile("mcr p15, 0, %0, c9, c13, 1" : : "r" (value));
-}
-
-static inline uint64_t pmccntr_read(void)
+#if defined(__arm__)
+DEFINE_GET_SYSREG32(pmcr, 0, c9, c12, 0)
+DEFINE_SET_SYSREG32(pmcr, 0, c9, c12, 0)
+DEFINE_GET_SYSREG32(id_dfr0, 0, c0, c1, 2)
+DEFINE_SET_SYSREG32(pmselr, 0, c9, c12, 5)
+DEFINE_SET_SYSREG32(pmxevtyper, 0, c9, c13, 1)
+DEFINE_GET_SYSREG32(pmccntr32, 0, c9, c13, 0)
+DEFINE_SET_SYSREG32(pmccntr32, 0, c9, c13, 0)
+DEFINE_GET_SYSREG64(pmccntr64, 0, c9)
+DEFINE_SET_SYSREG64(pmccntr64, 0, c9)
+DEFINE_SET_SYSREG32(pmcntenset, 0, c9, c12, 1)
+
+static inline uint64_t get_pmccntr(void)
 {
-   uint32_t lo, hi = 0;
-
if (pmu_version == 0x3)
-   asm volatile("mrrc p15, 0, %0, %1, c9" : "=r" (lo), "=r" (hi));
+   return get_pmccntr32();
else
-   asm volatile("mrc p15, 0, %0, c9, c13, 0" : "=r" (lo));
-
-   return ((uint64_t)hi << 32) | lo;
+   return get_pmccntr64();
 }
 
-static inline void pmccntr_write(uint64_t value)
+static inline void set_pmccntr(uint64_t value)
 {
-   uint32_t lo, hi;
-
-   lo = value & 0x;
-   hi = (value >> 32) & 0x;
-
if (pmu_version == 0x3)
-   asm volatile("mcrr p15, 0, %0, %1, c9" : : "r" (lo), "r" (hi));
+   set_pmccntr64(value);
else
-   asm volatile("mcr p15, 0, %0, c9, c13, 0" : : "r" (lo));
+   set_pmccntr64(value & 0x);
 }
-
-static inline void pmcntenset_write(uint32_t value)
-{
-   asm volatile("mcr p15, 0, %0, c9, c12, 1" : : "r" (value));
-}
-
 /* PMCCFILTR is an obsolete name for PMXEVTYPER31 in ARMv7 */
-static inline void pmccfiltr_write(uint32_t value)
+static inline void set_pmccfiltr(uint32_t value)
 {
-   pmselr_write(PMU_CYCLE_IDX);
-   pmxevtyper_write(value);
+   set_pmselr(PMU_CYCLE_IDX);
+   set_pmxevtyper(value);
isb();
 }
 
-static inline uint32_t id_dfr0_read(void)
-{
-   uint32_t val;
-
-   asm volatile("mrc p15, 0, %0, c0, c1, 2" : "=r" (val));
-   return val;
-}
-
 /*
  * Extra instructions inserted by the compiler would be difficult to compensate
  * for, so hand assemble everything between, and including, the PMCR accesses
@@ -126,51 +91,13 @@ static inline void precise_instrs_loop(int loop, uint32_t 
pmcr)
: "cc");
 }
 #elif defined(__aarch64__)
-static inline uint32_t pmcr_read(void)
-{
-   uint32_t ret;
-
-   asm volatile("mrs %0, pmcr_el0" : "=r" (ret));
-   return ret;
-}
-
-static inline void pmcr_write(uint32_t value)
-{
-   asm volatile("msr pmcr_el0, %0" : : "r" (value));
-   isb();
-}
-
-static inline uint64_t pmccntr_read(void)
-{
-   uint64_t cycles;
-

Re: [Qemu-devel] [kvm-unit-tests PATCH v11 1/3] arm: Add PMU test

[Andre Przywara]

Hi,

On 22/11/16 18:29, Wei Huang wrote:
> From: Christopher Covington 
> 
> Beginning with a simple sanity check of the control register, add
> a unit test for the ARM Performance Monitors Unit (PMU).

Mmh, the output of this is a bit confusing. How about to join some
information? I changed it to give me:
INFO: pmu: PMU implementer/ID code: "A"(0x41)/0x0
INFO: pmu: Event counters:  0
PASS: pmu: Control register

... by using the newly introduced report_info() to make it look nicer.

> 
> Signed-off-by: Christopher Covington 
> Signed-off-by: Wei Huang 
> Reviewed-by: Andrew Jones 
> ---
>  arm/Makefile.common |  3 ++-
>  arm/pmu.c   | 74 
> +
>  arm/unittests.cfg   |  5 
>  3 files changed, 81 insertions(+), 1 deletion(-)
>  create mode 100644 arm/pmu.c
> 
> diff --git a/arm/Makefile.common b/arm/Makefile.common
> index f37b5c2..5da2fdd 100644
> --- a/arm/Makefile.common
> +++ b/arm/Makefile.common
> @@ -12,7 +12,8 @@ endif
>  tests-common = \
>   $(TEST_DIR)/selftest.flat \
>   $(TEST_DIR)/spinlock-test.flat \
> - $(TEST_DIR)/pci-test.flat
> + $(TEST_DIR)/pci-test.flat \
> + $(TEST_DIR)/pmu.flat
>  
>  all: test_cases
>  
> diff --git a/arm/pmu.c b/arm/pmu.c
> new file mode 100644
> index 000..9d9c53b
> --- /dev/null
> +++ b/arm/pmu.c
> @@ -0,0 +1,74 @@
> +/*
> + * Test the ARM Performance Monitors Unit (PMU).
> + *
> + * Copyright (c) 2015-2016, The Linux Foundation. All rights reserved.
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms of the GNU Lesser General Public License version 2.1 and
> + * only version 2.1 as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful, but 
> WITHOUT
> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
> + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public 
> License
> + * for more details.
> + */
> +#include "libcflat.h"
> +#include "asm/barrier.h"
> +
> +#define PMU_PMCR_N_SHIFT   11
> +#define PMU_PMCR_N_MASK0x1f
> +#define PMU_PMCR_ID_SHIFT  16
> +#define PMU_PMCR_ID_MASK   0xff
> +#define PMU_PMCR_IMP_SHIFT 24
> +#define PMU_PMCR_IMP_MASK  0xff
> +
> +#if defined(__arm__)

I guess you should use the arch specific header files we have in place
for that (lib/arm{.64}/asm/processor.h). Also there are sysreg read
wrappers (at least for arm64) in there already, can't we base this
function on them: DEFINE_GET_SYSREG32(pmcr, el0)?
(Requires a small change to get rid of the forced "_el1" suffix)

We should wait for the GIC series to be merged, as this contains some
changes in this area.

> +static inline uint32_t pmcr_read(void)
> +{
> + uint32_t ret;
> +
> + asm volatile("mrc p15, 0, %0, c9, c12, 0" : "=r" (ret));
> + return ret;
> +}
> +#elif defined(__aarch64__)
> +static inline uint32_t pmcr_read(void)
> +{
> + uint32_t ret;
> +
> + asm volatile("mrs %0, pmcr_el0" : "=r" (ret));
> + return ret;
> +}
> +#endif
> +
> +/*
> + * As a simple sanity check on the PMCR_EL0, ensure the implementer field 
> isn't
> + * null. Also print out a couple other interesting fields for diagnostic
> + * purposes. For example, as of fall 2016, QEMU TCG mode doesn't implement
> + * event counters and therefore reports zero event counters, but hopefully
> + * support for at least the instructions event will be added in the future 
> and
> + * the reported number of event counters will become nonzero.
> + */
> +static bool check_pmcr(void)
> +{
> + uint32_t pmcr;
> +
> + pmcr = pmcr_read();
> +
> + printf("PMU implementer: %c\n",
> +(pmcr >> PMU_PMCR_IMP_SHIFT) & PMU_PMCR_IMP_MASK);

If this register reads as zero, the output is mangled (since it cuts off
the string before the newline):
=
PMU implementer: Identification code: 0x0
=

I guess you need something like:
(pmcr >> PMU_PMCR_IMP_SHIFT) & PMU_PMCR_IMP_MASK ?: ' '

> + printf("Identification code: 0x%x\n",
> +(pmcr >> PMU_PMCR_ID_SHIFT) & PMU_PMCR_ID_MASK);

As mentioned above this should use report_info() now, also it would be
nice to merge this with the message above into one line of output.

Cheers,
Andre

> + printf("Event counters:  %d\n",
> +(pmcr >> PMU_PMCR_N_SHIFT) & PMU_PMCR_N_MASK);
> +
> + return ((pmcr >> PMU_PMCR_IMP_SHIFT) & PMU_PMCR_IMP_MASK) != 0;
> +}
> +
> +int main(void)
> +{
> + report_prefix_push("pmu");
> +
> + report("Control register", check_pmcr());
> +
> + return report_summary();
> +}
> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> index ae32a42..816f494 100644
> --- a/arm/unittests.cfg
> +++ b/arm/unittests.cfg
> @@ -58,3 +58,8 @@ groups = selftest
>  [pci-test]
>  file = pci-test.flat
>  groups = pci
> +
> +# Test PMU support
> +[pmu]
> +file = 

Re: [Qemu-devel] [kvm-unit-tests PATCH 3/4] arm/arm64: GICv2: add GICD_ITARGETSR testing

[Andre Przywara]

Hi Eric,

thanks for having such a close look (as always!).

On 23/11/16 13:51, Auger Eric wrote:
> Hi Andre,
> 
> On 23/11/2016 14:24, Auger Eric wrote:
>> Hi,
>>
>> On 18/11/2016 15:20, Andrew Jones wrote:
>>> On Thu, Nov 17, 2016 at 05:57:51PM +, Andre Przywara wrote:
>>>> Some tests for the ITARGETS registers.
>>>> Bits corresponding to non-existent CPUs must be RAZ/WI.
>>>> These registers must be byte-accessible, also check that accesses beyond
>>>> the implemented IRQ limit are actually read-as-zero/write-ignore.
>>>>
>>>> Signed-off-by: Andre Przywara <andre.przyw...@arm.com>
>>>> ---
>>>>  arm/gic.c | 54 
>>>> ++
>>>>  lib/arm/asm/gic.h |  1 +
>>>>  2 files changed, 55 insertions(+)
>>>>
>>>> diff --git a/arm/gic.c b/arm/gic.c
>>>> index a27da2c..02b1be1 100644
>>>> --- a/arm/gic.c
>>>> +++ b/arm/gic.c
>>>> @@ -397,6 +397,57 @@ static bool test_priorities(int nr_irqs, void *priptr)
>>>>return true;
>>>>  }
>>>>  
>>>> +static bool test_targets(int nr_irqs)
>>>> +{
>>>> +  void *targetsptr = gicv2_dist_base() + GICD_ITARGETSR;
>>>> +  u32 orig_targets;
>>>> +  u32 cpu_mask;
>>>> +  u32 pattern, reg;
>>>> +
>>>> +  orig_targets = readl(targetsptr + 32);
>>>> +  report_prefix_push("ITARGETSR");
>>>> +
>>>> +  cpu_mask = (1 << nr_cpus) - 1;
>>>
>>> Shouldn't this be 1 << (nr_cpus - 1) ?
>> original looks correct to me.
>>>
>>> Is this test always going to be gicv2-only? We should probably comment it,
>>> if so. We don't want to risk this being run when nr_cpus can be larger
>>> than 8.
>>>
>>>> +  cpu_mask |= cpu_mask << 8;

So this instruction is supposed to copy bits[7:0] into bits[15:8] (not
caring about the other bits, which are zero anyway).

>>>> +  cpu_mask |= cpu_mask << 16;

And this one copies bits[15:0] into bits[31:16].

>> Don't we miss the 4th byte mask?

I don't think so, the idea is just to copy the lowest byte into all the
other three bytes of that word and thus to propagate the byte mask for
one IRQ into a word covering four interrupts. Does that make sense?
I take it this deserves a comment then ...

>>>> +
>>>> +  /* Check that bits for non implemented CPUs are RAZ/WI. */
>>>> +  if (nr_cpus < 8) {
>>>> +  writel(0x, targetsptr + 32);
>>>> +  report("bits for %d non-existent CPUs masked",
>>>> + !(readl(targetsptr + 32) & ~cpu_mask), 8 - nr_cpus);
> 
> yep on AMD overdrive with smp=4 I get:
> 
> FAIL: gicv2: mmio: ITARGETSR: bits for 4 non-existent CPUs masked

I guess because you don't have
http://lists.infradead.org/pipermail/linux-arm-kernel/2016-November/468296.html
in your host kernel?
This was one of the two genuine bugs I spotted with this.

Cheers,
Andre.

>>>> +  } else {
>>>> +  report_skip("CPU masking (all CPUs implemented)");
>>>> +  }
>>>> +
>>>> +  report("accesses beyond limit RAZ/WI",
>>>> + test_readonly_32(targetsptr + nr_irqs, true));
>>>> +
>>>> +  pattern = 0x0103020f;
>>>> +  writel(pattern, targetsptr + 32);
>>>> +  reg = readl(targetsptr + 32);
>>>> +  report("register content preserved (%08x => %08x)",
>>>> + reg == (pattern & cpu_mask), pattern & cpu_mask, reg);
>>>> +
>>>> +  /*
>>>> +   * The TARGETS registers are byte accessible, do a byte-wide
>>>> +   * read and write of known content to check for this.
>>>> +   */
>>>> +  reg = readb(targetsptr + 33);
>>>> +  report("byte reads successful (0x%08x => 0x%02x)",
>>>> + reg == (BYTE(pattern, 1) & cpu_mask),
>>>> + pattern & cpu_mask, reg);
>>>> +
>>>> +  pattern = REPLACE_BYTE(pattern, 2, 0x04);
>>>> +  writeb(BYTE(pattern, 2), targetsptr + 34);
>>>> +  reg = readl(targetsptr + 32);
>>>> +  report("byte writes successful (0x%02x => 0x%08x)",
>>>> + reg == (pattern & cpu_mask), BYTE(pattern, 2), reg);
>>>
>>> Last patch also had a byte addressability test. Maybe we should make
>>> a helper function?
>>>
>>>> +
>>>> +  writel(orig_targets, targetsptr + 32);
>>>> +  return true;
>>>
>>> Function can/should be void.
>>>
>>>> +}
>>>> +
>>>>  static int gic_test_mmio(int gic_version)
>>>>  {
>>>>u32 reg;
>>>> @@ -436,6 +487,9 @@ static int gic_test_mmio(int gic_version)
>>>>  
>>>>test_priorities(nr_irqs, gic_dist_base + GICD_IPRIORITYR);
>>>>  
>>>> +  if (gic_version == 2)
>>>> +  test_targets(nr_irqs);
>>>> +
>>>>return 0;
>>>>  }
>>>>  
>>>> diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
>>>> index cef748d..6f170cb 100644
>>>> --- a/lib/arm/asm/gic.h
>>>> +++ b/lib/arm/asm/gic.h
>>>> @@ -14,6 +14,7 @@
>>>>  #define GICD_IGROUPR  0x0080
>>>>  #define GICD_ISENABLER0x0100
>>>>  #define GICD_IPRIORITYR   0x0400
>>>> +#define GICD_ITARGETSR0x0800
>>>>  #define GICD_SGIR 0x0f00
>>>>  #define GICD_ICPIDR2  0x0fe8
>>>>  
>>>> -- 
>>>> 2.9.0
>>>>
>>>>
>>>
>>> Thanks,
>>> drew
>>>

Re: [Qemu-devel] [kvm-unit-tests PATCH v6 00/11] arm/arm64: add gic framework

[Andre Przywara]

Hi,

On 22/11/16 18:45, Andrew Jones wrote:
> 
> Andre, Alex, Eric, anybody,
> 
> Any more comments on this? If not, I'll send a pull request
> to Radim and Paolo to finally get this merged.

I didn't manage to look in detail at the IPI test, but I am OK with the
rest of the GIC framework.
So I'd say: go ahead.

Cheers,
Andre.

> On Mon, Nov 14, 2016 at 10:08:28PM +0100, Andrew Jones wrote:
>> v6:
>>  - rebased to latest master
>>  - several other changes thanks to Andre and Alex, changes in
>>individual patch change logs
>>  - some code cleanups
>>
>> v5:
>>  - fix arm32/gicv3 compile [drew]
>>  - use modern register names [Andre]
>>  - one Andre r-b
>>
>> v4:
>>  - Eric's r-b's
>>  - Andre's suggestion to only take defines we need
>>  - several other changes listed in individual patches
>>
>> v3:
>>  - Rebased on latest master
>>  - Added Alex's r-b's
>>
>> v2:
>>  Rebased on latest master + my "populate argv[0]" series (will
>>  send a REPOST for that shortly. Additionally a few patches got
>>  fixes/features;
>>  07/10 got same fix as kernel 7c9b973061 "irqchip/gic-v3: Configure
>>all interrupts as non-secure Group-1" in order to continue
>>working over TCG, as the gicv3 code for TCG removed a hack
>>it had there to make Linux happy.
>>  08/10 added more output for when things fail (if they fail)
>>  09/10 switched gicv3 broadcast implementation to using IRM. This
>>found a bug in a recent (but not tip) kernel, which I was
>>about to fix, but then I saw MarcZ beat me to it.
>>  10/10 actually check that the input irq is the received irq
>>
>>
>> Import defines, and steal enough helper functions, from Linux to
>> enable programming of the gic (v2 and v3). Then use the framework
>> to add an initial test (an ipi test; self, target-list, broadcast).
>>
>> It's my hope that this framework will be a suitable base on which
>> more tests may be easily added, particularly because we have
>> vgic-new and tcg gicv3 emulation getting close to merge. (v3 UPDATE:
>> vgic-new and tcg gicv3 are merged now)
>>
>> To run it, along with other tests, just do
>>
>>  ./configure [ --arch=[arm|arm64] --cross-prefix=$PREFIX ]
>>  make
>>  export QEMU=$PATH_TO_QEMU
>>  ./run_tests.sh
>>
>> To run it separately do, e.g.
>>
>> $QEMU -machine virt,accel=tcg -cpu cortex-a57 \
>>  -device virtio-serial-device \
>>  -device virtconsole,chardev=ctd -chardev testdev,id=ctd \
>>  -display none -serial stdio \
>>  -kernel arm/gic.flat \
>>  -smp 123 -machine gic-version=3 -append ipi
>>   ^^ note, we can go nuts with nr-cpus on TCG :-)
>>
>> Or, a KVM example using a different "sender" cpu and irq (other than zero)
>>
>> $QEMU -machine virt,accel=kvm -cpu host \
>>  -device virtio-serial-device \
>>  -device virtconsole,chardev=ctd -chardev testdev,id=ctd \
>>  -display none -serial stdio \
>>  -kernel arm/gic.flat \
>>  -smp 48 -machine gic-version=3 -append 'ipi sender=42 irq=1'
>>
>>
>> Patches:
>> 01-05: fixes and functionality needed by the later gic patches
>> 06-07: enable gicv2 and gicv2 IPI test
>> 08-10: enable gicv3 and gicv3 IPI test
>>11: extend the IPI tests to take variable sender and irq
>>
>> Available here: 
>> https://github.com/rhdrjones/kvm-unit-tests/commits/arm/gic-v6
>>
>>
>> Andrew Jones (10):
>>   lib: xstr: allow multiple args
>>   arm64: fix get_"sysreg32" and make MPIDR 64bit
>>   arm/arm64: smp: support more than 8 cpus
>>   arm/arm64: add some delay routines
>>   arm/arm64: irq enable/disable
>>   arm/arm64: add initial gicv2 support
>>   arm/arm64: gicv2: add an IPI test
>>   arm/arm64: add initial gicv3 support
>>   arm/arm64: gicv3: add an IPI test
>>   arm/arm64: gic: don't just use zero
>>
>> Peter Xu (1):
>>   libcflat: add IS_ALIGNED() macro, and page sizes
>>
>>  arm/Makefile.common|   9 +-
>>  arm/gic.c  | 340 
>> +
>>  arm/run|  19 ++-
>>  arm/selftest.c |   5 +-
>>  arm/unittests.cfg  |  14 ++
>>  lib/arm/asm/arch_gicv3.h   |  70 ++
>>  lib/arm/asm/gic-v2.h   |  36 +
>>  lib/arm/asm/gic-v3.h   | 112 +++
>>  lib/arm/asm/gic.h  | 106 ++
>>  lib/arm/asm/processor.h|  42 +-
>>  lib/arm/asm/setup.h|   4 +-
>>  lib/arm/gic.c  | 267 +++
>>  lib/arm/processor.c|  15 ++
>>  lib/arm/setup.c|  10 ++
>>  lib/arm64/asm/arch_gicv3.h |  66 +
>>  lib/arm64/asm/gic-v2.h |   1 +
>>  lib/arm64/asm/gic-v3.h |   1 +
>>  lib/arm64/asm/gic.h|   1 +
>>  lib/arm64/asm/processor.h  |  53 +--
>>  lib/arm64/asm/sysreg.h |  44 ++
>>  lib/arm64/processor.c  |  15 ++
>>  lib/libcflat.h |  10 +-
>>  22 files changed, 1212 insertions(+), 28 deletions(-)
>>  create mode 100644 arm/gic.c
>>  create mode 100644 lib/arm/asm/arch_gicv3.h
>>  create mode 100644 lib/arm/asm/gic-v2.h
>>  create 

[Qemu-devel] [kvm-unit-tests PATCH] configure: honour $ARCH and $CROSS_COMPILE

[Andre Przywara]

Both environment variables seem to be standard in cross-compilation
environments, especially with Linux.
Let the configure script take those into account when setting the default
values for --arch and --cross-prefix. Explicitly specifying the latter
on the configure command line still works as expected.

Signed-off-by: Andre Przywara <andre.przyw...@arm.com>
---
Hi,

this maybe a personal itch to scratch here, since I set these two
variables in my environment via a (sourced) script here and never have
to care about the particular cross-compiler prefix, for instance.
It looks rather generic, though, so I was wondering if this is useful
upstream as well.

Cheers,
Andre.

 configure | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/configure b/configure
index 995c8fa..9bb38ba 100755
--- a/configure
+++ b/configure
@@ -8,8 +8,9 @@ objdump=objdump
 ar=ar
 addr2line=addr2line
 arch=`uname -m | sed -e 's/i.86/i386/;s/arm.*/arm/;s/ppc64.*/ppc64/'`
+[ -n "$ARCH" ] && arch="$ARCH"
 host=$arch
-cross_prefix=
+cross_prefix=${CROSS_COMPILE}
 endian=""
 pretty_print_stacks=yes
 
-- 
2.9.0

Re: [Qemu-devel] [kvm-unit-tests PATCH 1/4] arm/arm64: GIC: basic GICv2 MMIO tests

[Andre Przywara]

Hi Drew,

On 18/11/16 13:06, Andrew Jones wrote:
> Hi Andre,
> 
> I'm so pleased to see this series. Thank you!
> 
> On Thu, Nov 17, 2016 at 05:57:49PM +, Andre Przywara wrote:
>> This adds an MMIO subtest to the GIC test.
>> It accesses some generic GICv2 registers and does some sanity tests,
>> like checking for some of them being read-only.
>>
>> Signed-off-by: Andre Przywara <andre.przyw...@arm.com>
>> ---
>>  arm/gic.c | 99 
>> +++
>>  arm/unittests.cfg |  6 
>>  lib/arm/asm/gic.h |  2 ++
>>  3 files changed, 107 insertions(+)
>>
>> diff --git a/arm/gic.c b/arm/gic.c
>> index 638b8b1..ba2585b 100644
>> --- a/arm/gic.c
>> +++ b/arm/gic.c
>> @@ -3,6 +3,7 @@
>>   *
>>   * GICv2
>>   *   + test sending/receiving IPIs
>> + *   + MMIO access tests
>>   * GICv3
>>   *   + test sending/receiving IPIs
>>   *
>> @@ -274,6 +275,98 @@ static struct gic gicv3 = {
>>  },
>>  };
>>  
>> +static bool test_ro_pattern_32(void *address, u32 pattern, u32 orig)
>> +{
>> +u32 reg;
>> +
>> +writel(pattern, address);
>> +reg = readl(address);
>> +
>> +if (reg != orig)
>> +writel(orig, address);
>> +
>> +return reg == orig;
>> +}
>> +
>> +static bool test_readonly_32(void *address, bool razwi)
>> +{
>> +u32 orig, pattern;
>> +
>> +orig = readl(address);
>> +if (razwi && orig)
>> +return false;
>> +
>> +pattern = 0x;
>> +if (orig != pattern) {
>> +if (!test_ro_pattern_32(address, pattern, orig))
>> +return false;
>> +}
>> +
>> +pattern = 0xa5a55a5a;
>> +if (orig != pattern) {
>> +if (!test_ro_pattern_32(address, pattern, orig))
>> +return false;
>> +}
>> +
>> +pattern = 0;
>> +if (orig != pattern) {
>> +if (!test_ro_pattern_32(address, pattern, orig))
>> +return false;
>> +}
>> +
>> +return true;
>> +}
>> +
>> +static bool test_typer_v2(uint32_t reg)
>> +{
>> +int nr_gic_cpus = ((reg >> 5) & 0x7) + 1;
>> +
>> +report("all %d CPUs have interrupts", nr_cpus == nr_gic_cpus,
>> +   nr_gic_cpus);
>> +
>> +return true;
> 
> This test function can be a void.
> 
>> +}
>> +
>> +static int gic_test_mmio(int gic_version)
>> +{
>> +u32 reg;
>> +int nr_irqs;
>> +void *gic_dist_base, *idreg;
>> +
>> +switch(gic_version) {
>> +case 0x2:
>> +gic_dist_base = gicv2_dist_base();
>> +idreg = gic_dist_base + 0xfe8;
> 
> I see below you introduce GICD_ICPIDR2, so I guess you can use it here.
> 
>> +break;
>> +case 0x3:
>> +report_abort("GICv3 MMIO tests NYI");
>> +return -1;
> 
> can't reach this return

But we need to tell GCC about this, because otherwise we get all kind of
warnings (including bogus "maybe unused" warnings).
__attribute__ ((noreturn)) seems the way to go here, but this is
currently giving me a hard time ...

>> +default:
>> +report_abort("GIC version %d not supported", gic_version);
>> +return 0;
> 
> can't reach this return
> 
>> +}
>> +
>> +reg = readl(gic_dist_base + GICD_TYPER);
>> +nr_irqs = 32 * ((reg & 0x1f) + 1);
> 
> Any reason to avoid using GICD_TYPER_IRQS() here?

On the first write I wasn't aware of it, on a second thought then I
wanted to avoid using the macro copied from Linux.

But you are right, I will use it here.

> 
>> +report("number of implemented SPIs: %d", 1, nr_irqs - 32);
> 
> We usually just use printf for informational output (but we should
> probably add a 'report_info' in order to keep the prefixes. I can
> do that now.) Anyway, please s/1/true

I saw your patch, will use that.

>> +
>> +test_typer_v2(reg);
>> +
>> +report("IIDR: 0x%x", 1, readl(gic_dist_base + GICD_IIDR));
>> +
>> +report("GICD_TYPER is read-only",
>> +   test_readonly_32(gic_dist_base + GICD_TYPER, false));
>> +report("GICD_IIDR is read-only",
>> +   test_readonly_32(gic_dist_base + GICD_IIDR, false));
>> +
>> +reg = readl(idreg);
>> +   

[Qemu-devel] [kvm-unit-tests PATCH 0/4] kvm-unit-tests: add first GIC MMIO tests

[Andre Przywara]

The GIC spec mandates certain constraints on how to acccess the MMIO
mapped registers, both in terms of which registers are available and also
in terms of which bits within a register should be masked, for instance.
Since we went through some lengths in the KVM emulation to implement this,
it's about time to give this actually a test beyond what the kernel as a
GIC user actually implements - for instance we ignore priorities in Linux.

This series tries to attack some constraints, on a low-hanging-fruit base.
It focusses on some generic registers and the PRIORITY and TARGETS registers
of GICv2. GICv3 is only covered to the point where the GICv2 tests overlap
with GICv3.

This actually revealed genuine bugs in the KVM emulation, for which I sent
fixes in the last days. With those fixes the kernel passes all tests.

Please have a look whether this makes sense and should eventually be
extended on that base.

Applies on top of master + Drew's v6 series.

Cheers,
Andre.

Andre Przywara (4):
  arm/arm64: GIC: basic GICv2 MMIO tests
  arm/arm64: GICv2: add GICD_IPRIORITYR testing
  arm/arm64: GICv2: add GICD_ITARGETSR testing
  arm/arm64: GICv3: add TYPER test

 arm/gic.c | 253 ++
 arm/unittests.cfg |  12 +++
 lib/arm/asm/gic.h |   3 +
 3 files changed, 268 insertions(+)

-- 
2.9.0

[Qemu-devel] [kvm-unit-tests PATCH 3/4] arm/arm64: GICv2: add GICD_ITARGETSR testing

[Andre Przywara]

Some tests for the ITARGETS registers.
Bits corresponding to non-existent CPUs must be RAZ/WI.
These registers must be byte-accessible, also check that accesses beyond
the implemented IRQ limit are actually read-as-zero/write-ignore.

Signed-off-by: Andre Przywara <andre.przyw...@arm.com>
---
 arm/gic.c | 54 ++
 lib/arm/asm/gic.h |  1 +
 2 files changed, 55 insertions(+)

diff --git a/arm/gic.c b/arm/gic.c
index a27da2c..02b1be1 100644
--- a/arm/gic.c
+++ b/arm/gic.c
@@ -397,6 +397,57 @@ static bool test_priorities(int nr_irqs, void *priptr)
return true;
 }
 
+static bool test_targets(int nr_irqs)
+{
+   void *targetsptr = gicv2_dist_base() + GICD_ITARGETSR;
+   u32 orig_targets;
+   u32 cpu_mask;
+   u32 pattern, reg;
+
+   orig_targets = readl(targetsptr + 32);
+   report_prefix_push("ITARGETSR");
+
+   cpu_mask = (1 << nr_cpus) - 1;
+   cpu_mask |= cpu_mask << 8;
+   cpu_mask |= cpu_mask << 16;
+
+   /* Check that bits for non implemented CPUs are RAZ/WI. */
+   if (nr_cpus < 8) {
+   writel(0x, targetsptr + 32);
+   report("bits for %d non-existent CPUs masked",
+  !(readl(targetsptr + 32) & ~cpu_mask), 8 - nr_cpus);
+   } else {
+   report_skip("CPU masking (all CPUs implemented)");
+   }
+
+   report("accesses beyond limit RAZ/WI",
+  test_readonly_32(targetsptr + nr_irqs, true));
+
+   pattern = 0x0103020f;
+   writel(pattern, targetsptr + 32);
+   reg = readl(targetsptr + 32);
+   report("register content preserved (%08x => %08x)",
+  reg == (pattern & cpu_mask), pattern & cpu_mask, reg);
+
+   /*
+* The TARGETS registers are byte accessible, do a byte-wide
+* read and write of known content to check for this.
+*/
+   reg = readb(targetsptr + 33);
+   report("byte reads successful (0x%08x => 0x%02x)",
+  reg == (BYTE(pattern, 1) & cpu_mask),
+  pattern & cpu_mask, reg);
+
+   pattern = REPLACE_BYTE(pattern, 2, 0x04);
+   writeb(BYTE(pattern, 2), targetsptr + 34);
+   reg = readl(targetsptr + 32);
+   report("byte writes successful (0x%02x => 0x%08x)",
+  reg == (pattern & cpu_mask), BYTE(pattern, 2), reg);
+
+   writel(orig_targets, targetsptr + 32);
+   return true;
+}
+
 static int gic_test_mmio(int gic_version)
 {
u32 reg;
@@ -436,6 +487,9 @@ static int gic_test_mmio(int gic_version)
 
test_priorities(nr_irqs, gic_dist_base + GICD_IPRIORITYR);
 
+   if (gic_version == 2)
+   test_targets(nr_irqs);
+
return 0;
 }
 
diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
index cef748d..6f170cb 100644
--- a/lib/arm/asm/gic.h
+++ b/lib/arm/asm/gic.h
@@ -14,6 +14,7 @@
 #define GICD_IGROUPR   0x0080
 #define GICD_ISENABLER 0x0100
 #define GICD_IPRIORITYR0x0400
+#define GICD_ITARGETSR 0x0800
 #define GICD_SGIR  0x0f00
 #define GICD_ICPIDR2   0x0fe8
 
-- 
2.9.0

[Qemu-devel] [kvm-unit-tests PATCH 2/4] arm/arm64: GICv2: add GICD_IPRIORITYR testing

[Andre Przywara]

Some tests for the IPRIORITY registers. The significant number of bits
is IMPLEMENTATION DEFINED, but should be the same for every IRQ.
Also these registers must be byte-accessible.
Check that accesses beyond the implemented IRQ limit are actually
read-as-zero/write-ignore.

Signed-off-by: Andre Przywara <andre.przyw...@arm.com>
---
 arm/gic.c | 72 +++
 1 file changed, 72 insertions(+)

diff --git a/arm/gic.c b/arm/gic.c
index ba2585b..a27da2c 100644
--- a/arm/gic.c
+++ b/arm/gic.c
@@ -327,6 +327,76 @@ static bool test_typer_v2(uint32_t reg)
return true;
 }
 
+#define BYTE(reg32, byte) (((reg32) >> ((byte) * 8)) & 0xff)
+#define REPLACE_BYTE(reg32, byte, new) (((reg32) & ~(0xff << ((byte) * 8))) |\
+   ((new) << ((byte) * 8)))
+
+static bool test_priorities(int nr_irqs, void *priptr)
+{
+   u32 orig_prio, reg, pri_bits;
+   u32 pri_mask, pattern;
+
+   orig_prio = readl(priptr + 32);
+   report_prefix_push("IPRIORITYR");
+
+   /*
+* Determine implemented number of priority bits by writing all 1's
+* and checking the number of cleared bits in the value read back.
+*/
+   writel(0x, priptr + 32);
+   pri_mask = readl(priptr + 32);
+
+   reg = ~pri_mask;
+   report("consistent priority masking (0x%08x)",
+  (((reg >> 16) == (reg & 0x)) &&
+   ((reg & 0xff) == ((reg >> 8) & 0xff))), pri_mask);
+
+   reg = reg & 0xff;
+   for (pri_bits = 8; reg & 1; reg >>= 1, pri_bits--)
+   ;
+   report("implements at least 4 priority bits (%d)",
+  pri_bits >= 4, pri_bits);
+
+   pattern = 0;
+   writel(pattern, priptr + 32);
+   report("clearing priorities", readl(priptr + 32) == pattern);
+
+   pattern = 0x;
+   writel(pattern, priptr + 32);
+   report("filling priorities",
+  readl(priptr + 32) == (pattern & pri_mask));
+
+   report("accesses beyond limit RAZ/WI",
+  test_readonly_32(priptr + nr_irqs, true));
+
+   writel(pattern, priptr + nr_irqs - 4);
+   report("accessing last SPIs",
+  readl(priptr + nr_irqs - 4) == (pattern & pri_mask));
+
+   pattern = 0xff7fbf3f;
+   writel(pattern, priptr + 32);
+   report("priorities are preserved",
+  readl(priptr + 32) == (pattern & pri_mask));
+
+   /*
+* The PRIORITY registers are byte accessible, do a byte-wide
+* read and write of known content to check for this.
+*/
+   reg = readb(priptr + 33);
+   report("byte reads successful (0x%08x => 0x%02x)",
+  reg == (BYTE(pattern, 1) & pri_mask), pattern & pri_mask, reg);
+
+   pattern = REPLACE_BYTE(pattern, 2, 0x1f);
+   writeb(BYTE(pattern, 2), priptr + 34);
+   reg = readl(priptr + 32);
+   report("byte writes successful (0x%02x => 0x%08x)",
+  reg == (pattern & pri_mask), BYTE(pattern, 2) & pri_mask, reg);
+
+   report_prefix_pop();
+   writel(orig_prio, priptr + 32);
+   return true;
+}
+
 static int gic_test_mmio(int gic_version)
 {
u32 reg;
@@ -364,6 +434,8 @@ static int gic_test_mmio(int gic_version)
   test_readonly_32(idreg, false),
   reg);
 
+   test_priorities(nr_irqs, gic_dist_base + GICD_IPRIORITYR);
+
return 0;
 }
 
-- 
2.9.0

[Qemu-devel] [kvm-unit-tests PATCH 4/4] arm/arm64: GICv3: add TYPER test

[Andre Przywara]

Add a simple test for the GICv3 TYPER test, which does only one basic
check to ensure we have actually enough interrupt IDs if we support
LPIs.
Allow a GICv3 guest to do the common MMIO checks as well, where the
register semantics are shared with a GICv2.

Signed-off-by: Andre Przywara <andre.przyw...@arm.com>
---
 arm/gic.c | 34 +++---
 arm/unittests.cfg |  6 ++
 2 files changed, 37 insertions(+), 3 deletions(-)

diff --git a/arm/gic.c b/arm/gic.c
index 02b1be1..7de0e47 100644
--- a/arm/gic.c
+++ b/arm/gic.c
@@ -327,6 +327,30 @@ static bool test_typer_v2(uint32_t reg)
return true;
 }
 
+static bool test_typer_v3(uint32_t reg)
+{
+   int nr_intids;
+
+   report("GIC emulation %ssupport%s MBIs", 1,
+  reg & BIT(16) ? "" : "does not ",
+  reg & BIT(16) ? "s" : "");
+   report("GIC emulation %ssupport%s LPIs", 1,
+  reg & BIT(17) ? "" : "does not ",
+  reg & BIT(17) ? "s" : "");
+   report("GIC emulation %ssupport%s Aff3", 1,
+  reg & BIT(24) ? "" : "does not ",
+  reg & BIT(24) ? "s" : "");
+
+   nr_intids = BIT(((reg >> 19) & 0x1f) + 1);
+   report("%d interrupt IDs implemented", 1, nr_intids);
+
+   if (reg & BIT(17))
+   report("%d LPIs supported", nr_intids > 8192,
+  nr_intids > 8192 ? nr_intids - 8192 : 0);
+
+   return true;
+}
+
 #define BYTE(reg32, byte) (((reg32) >> ((byte) * 8)) & 0xff)
 #define REPLACE_BYTE(reg32, byte, new) (((reg32) & ~(0xff << ((byte) * 8))) |\
((new) << ((byte) * 8)))
@@ -460,8 +484,9 @@ static int gic_test_mmio(int gic_version)
idreg = gic_dist_base + 0xfe8;
break;
case 0x3:
-   report_abort("GICv3 MMIO tests NYI");
-   return -1;
+   gic_dist_base = gicv3_dist_base();
+   idreg = gic_dist_base + 0xffe8;
+   break;
default:
report_abort("GIC version %d not supported", gic_version);
return 0;
@@ -471,7 +496,10 @@ static int gic_test_mmio(int gic_version)
nr_irqs = 32 * ((reg & 0x1f) + 1);
report("number of implemented SPIs: %d", 1, nr_irqs - 32);
 
-   test_typer_v2(reg);
+   if (gic_version == 2)
+   test_typer_v2(reg);
+   else
+   test_typer_v3(reg);
 
report("IIDR: 0x%x", 1, readl(gic_dist_base + GICD_IIDR));
 
diff --git a/arm/unittests.cfg b/arm/unittests.cfg
index 0162e5a..b432346 100644
--- a/arm/unittests.cfg
+++ b/arm/unittests.cfg
@@ -78,3 +78,9 @@ file = gic.flat
 smp = $MAX_SMP
 extra_params = -machine gic-version=3 -append 'ipi'
 groups = gic
+
+[gicv3-mmio]
+file = gic.flat
+smp = $MAX_SMP
+extra_params = -machine gic-version=3 -append 'mmio'
+groups = gic
-- 
2.9.0

[Qemu-devel] [kvm-unit-tests PATCH 1/4] arm/arm64: GIC: basic GICv2 MMIO tests

[Andre Przywara]

This adds an MMIO subtest to the GIC test.
It accesses some generic GICv2 registers and does some sanity tests,
like checking for some of them being read-only.

Signed-off-by: Andre Przywara <andre.przyw...@arm.com>
---
 arm/gic.c | 99 +++
 arm/unittests.cfg |  6 
 lib/arm/asm/gic.h |  2 ++
 3 files changed, 107 insertions(+)

diff --git a/arm/gic.c b/arm/gic.c
index 638b8b1..ba2585b 100644
--- a/arm/gic.c
+++ b/arm/gic.c
@@ -3,6 +3,7 @@
  *
  * GICv2
  *   + test sending/receiving IPIs
+ *   + MMIO access tests
  * GICv3
  *   + test sending/receiving IPIs
  *
@@ -274,6 +275,98 @@ static struct gic gicv3 = {
},
 };
 
+static bool test_ro_pattern_32(void *address, u32 pattern, u32 orig)
+{
+   u32 reg;
+
+   writel(pattern, address);
+   reg = readl(address);
+
+   if (reg != orig)
+   writel(orig, address);
+
+   return reg == orig;
+}
+
+static bool test_readonly_32(void *address, bool razwi)
+{
+   u32 orig, pattern;
+
+   orig = readl(address);
+   if (razwi && orig)
+   return false;
+
+   pattern = 0x;
+   if (orig != pattern) {
+   if (!test_ro_pattern_32(address, pattern, orig))
+   return false;
+   }
+
+   pattern = 0xa5a55a5a;
+   if (orig != pattern) {
+   if (!test_ro_pattern_32(address, pattern, orig))
+   return false;
+   }
+
+   pattern = 0;
+   if (orig != pattern) {
+   if (!test_ro_pattern_32(address, pattern, orig))
+   return false;
+   }
+
+   return true;
+}
+
+static bool test_typer_v2(uint32_t reg)
+{
+   int nr_gic_cpus = ((reg >> 5) & 0x7) + 1;
+
+   report("all %d CPUs have interrupts", nr_cpus == nr_gic_cpus,
+  nr_gic_cpus);
+
+   return true;
+}
+
+static int gic_test_mmio(int gic_version)
+{
+   u32 reg;
+   int nr_irqs;
+   void *gic_dist_base, *idreg;
+
+   switch(gic_version) {
+   case 0x2:
+   gic_dist_base = gicv2_dist_base();
+   idreg = gic_dist_base + 0xfe8;
+   break;
+   case 0x3:
+   report_abort("GICv3 MMIO tests NYI");
+   return -1;
+   default:
+   report_abort("GIC version %d not supported", gic_version);
+   return 0;
+   }
+
+   reg = readl(gic_dist_base + GICD_TYPER);
+   nr_irqs = 32 * ((reg & 0x1f) + 1);
+   report("number of implemented SPIs: %d", 1, nr_irqs - 32);
+
+   test_typer_v2(reg);
+
+   report("IIDR: 0x%x", 1, readl(gic_dist_base + GICD_IIDR));
+
+   report("GICD_TYPER is read-only",
+  test_readonly_32(gic_dist_base + GICD_TYPER, false));
+   report("GICD_IIDR is read-only",
+  test_readonly_32(gic_dist_base + GICD_IIDR, false));
+
+   reg = readl(idreg);
+   report("ICPIDR2 is read-only (0x%x)",
+  test_readonly_32(idreg, false),
+  reg);
+
+   return 0;
+}
+
 int main(int argc, char **argv)
 {
char pfx[8];
@@ -332,6 +425,12 @@ int main(int argc, char **argv)
}
ipi_test();
 
+   } else if (!strcmp(argv[1], "mmio")) {
+   report_prefix_push(argv[1]);
+
+   gic_test_mmio(gic_version());
+
+   report_prefix_pop();
} else {
report_abort("Unknown subtest '%s'", argv[1]);
}
diff --git a/arm/unittests.cfg b/arm/unittests.cfg
index c7392c7..0162e5a 100644
--- a/arm/unittests.cfg
+++ b/arm/unittests.cfg
@@ -67,6 +67,12 @@ smp = $((($MAX_SMP < 8)?$MAX_SMP:8))
 extra_params = -machine gic-version=2 -append 'ipi'
 groups = gic
 
+[gicv2-mmio]
+file = gic.flat
+smp = $((($MAX_SMP < 8)?$MAX_SMP:8))
+extra_params = -machine gic-version=2 -append 'mmio'
+groups = gic
+
 [gicv3-ipi]
 file = gic.flat
 smp = $MAX_SMP
diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
index c2267b6..cef748d 100644
--- a/lib/arm/asm/gic.h
+++ b/lib/arm/asm/gic.h
@@ -10,10 +10,12 @@
 /* Distributor registers */
 #define GICD_CTLR  0x
 #define GICD_TYPER 0x0004
+#define GICD_IIDR  0x0008
 #define GICD_IGROUPR   0x0080
 #define GICD_ISENABLER 0x0100
 #define GICD_IPRIORITYR0x0400
 #define GICD_SGIR  0x0f00
+#define GICD_ICPIDR2   0x0fe8
 
 #define GICD_TYPER_IRQS(typer) typer) & 0x1f) + 1) * 32)
 #define GICD_INT_EN_SET_SGI0x
-- 
2.9.0

Re: [Qemu-devel] [kvm-unit-tests PATCH v5 09/11] arm/arm64: add initial gicv3 support

[Andre Przywara]

Hi,

On 10/11/16 17:21, Andrew Jones wrote:
> Signed-off-by: Andrew Jones 
> 
> ---
> v5: use modern register names [Andre]
> v4:
>  - only take defines from kernel we need now [Andre]
>  - simplify enable by not caring if we reinit the distributor [drew]
> v2:
>  - configure irqs as NS GRP1
> ---
>  lib/arm/asm/arch_gicv3.h   | 42 +
>  lib/arm/asm/gic-v3.h   | 94 
> ++
>  lib/arm/asm/gic.h  |  6 ++-
>  lib/arm/gic.c  | 65 
>  lib/arm64/asm/arch_gicv3.h | 44 ++
>  lib/arm64/asm/gic-v3.h |  1 +
>  lib/arm64/asm/sysreg.h | 44 ++
>  7 files changed, 294 insertions(+), 2 deletions(-)
>  create mode 100644 lib/arm/asm/arch_gicv3.h
>  create mode 100644 lib/arm/asm/gic-v3.h
>  create mode 100644 lib/arm64/asm/arch_gicv3.h
>  create mode 100644 lib/arm64/asm/gic-v3.h
>  create mode 100644 lib/arm64/asm/sysreg.h
> 
> diff --git a/lib/arm/asm/arch_gicv3.h b/lib/arm/asm/arch_gicv3.h
> new file mode 100644
> index ..81a1e5f6c29c
> --- /dev/null
> +++ b/lib/arm/asm/arch_gicv3.h
> @@ -0,0 +1,42 @@
> +/*
> + * All ripped off from arch/arm/include/asm/arch_gicv3.h
> + *
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones 
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM_ARCH_GICV3_H_
> +#define _ASMARM_ARCH_GICV3_H_
> +
> +#ifndef __ASSEMBLY__
> +#include 
> +#include 
> +#include 
> +
> +#define __stringify xstr
> +
> +#define __ACCESS_CP15(CRn, Op1, CRm, Op2)p15, Op1, %0, CRn, CRm, Op2
> +
> +#define ICC_PMR  __ACCESS_CP15(c4, 0, c6, 0)
> +#define ICC_IGRPEN1  __ACCESS_CP15(c12, 0, c12, 7)
> +
> +static inline void gicv3_write_pmr(u32 val)
> +{
> + asm volatile("mcr " __stringify(ICC_PMR) : : "r" (val));
> +}
> +
> +static inline void gicv3_write_grpen1(u32 val)
> +{
> + asm volatile("mcr " __stringify(ICC_IGRPEN1) : : "r" (val));
> + isb();
> +}
> +
> +static inline u64 gicv3_read_typer(const volatile void __iomem *addr)

It may be worth to add that this is for GICR_TYPER (or GITS_TYPER),
because GICD_TYPER is 32-bit only.
Or to make the naming generic (because the code actually is), along the
lines of read_64bit_reg or the like?

> +{
> + u64 val = readl(addr);
> + val |= (u64)readl(addr + 4) << 32;
> + return val;
> +}
> +
> +#endif /* !__ASSEMBLY__ */
> +#endif /* _ASMARM_ARCH_GICV3_H_ */
> diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
> new file mode 100644
> index ..e0f303d82508
> --- /dev/null
> +++ b/lib/arm/asm/gic-v3.h
> @@ -0,0 +1,94 @@
> +/*
> + * All GIC* defines are lifted from include/linux/irqchip/arm-gic-v3.h
> + *
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones 
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM_GIC_V3_H_
> +#define _ASMARM_GIC_V3_H_
> +
> +#ifndef _ASMARM_GIC_H_
> +#error Do not directly include . Include 
> +#endif
> +
> +#define GICD_CTLR_RWP(1U << 31)
> +#define GICD_CTLR_ARE_NS (1U << 4)
> +#define GICD_CTLR_ENABLE_G1A (1U << 1)
> +#define GICD_CTLR_ENABLE_G1  (1U << 0)

+1 to Alex for adding a comment noting the non-secure view here.

> +
> +/* Re-Distributor registers, offsets from RD_base */
> +#define GICR_TYPER   0x0008
> +
> +#define GICR_TYPER_LAST  (1U << 4)
> +
> +/* Re-Distributor registers, offsets from SGI_base */
> +#define GICR_IGROUPR0GICD_IGROUPR
> +#define GICR_ISENABLER0  GICD_ISENABLER
> +#define GICR_IPRIORITYR0 GICD_IPRIORITYR
> +
> +#include 
> +
> +#ifndef __ASSEMBLY__
> +#include 
> +#include 
> +#include 
> +#include 
> +
> +struct gicv3_data {
> + void *dist_base;
> + void *redist_base[NR_CPUS];
> + unsigned int irq_nr;
> +};
> +extern struct gicv3_data gicv3_data;
> +
> +#define gicv3_dist_base()(gicv3_data.dist_base)
> +#define gicv3_redist_base()  
> (gicv3_data.redist_base[smp_processor_id()])
> +#define gicv3_sgi_base() 
> (gicv3_data.redist_base[smp_processor_id()] + SZ_64K)
> +
> +extern int gicv3_init(void);
> +extern void gicv3_enable_defaults(void);
> +extern void gicv3_set_redist_base(void);
> +
> +static inline void gicv3_do_wait_for_rwp(void *base)
> +{
> + int count = 10; /* 1s */
> +
> + while (readl(base + GICD_CTLR) & GICD_CTLR_RWP) {
> + if (!--count) {
> + printf("GICv3: RWP timeout!\n");
> + abort();
> + }
> + cpu_relax();
> + udelay(10);
> + };
> +}
> +
> +static inline void gicv3_dist_wait_for_rwp(void)
> +{
> + gicv3_do_wait_for_rwp(gicv3_dist_base());
> +}
> +
> +static inline void 

Re: [Qemu-devel] [kvm-unit-tests PATCH v5 06/11] arm/arm64: add initial gicv2 support

[Andre Przywara]

Hi,

On 10/11/16 17:21, Andrew Jones wrote:
> Add some gicv2 support. This just adds init and enable
> functions, allowing unit tests to start messing with it.
> 
> Signed-off-by: Andrew Jones <drjo...@redhat.com>
> 
> ---
> v5: share/use only the modern register names [Andre]

Thanks! That looks much better now.

Reviewed-by: Andre Przywara <andre.przyw...@arm.com>

> v4:
>  - only take defines from kernel we need now [Andre]
>  - moved defines to asm/gic.h so they'll be shared with v3 [drew]
>  - simplify enable by not caring if we reinit the distributor [drew]
>  - init all GICD_INT_DEF_PRI_X4 registers [Eric]
> ---
>  arm/Makefile.common|  1 +
>  lib/arm/asm/gic-v2.h   | 34 ++
>  lib/arm/asm/gic.h  | 37 
>  lib/arm/gic.c  | 76 
> ++
>  lib/arm64/asm/gic-v2.h |  1 +
>  lib/arm64/asm/gic.h|  1 +
>  6 files changed, 150 insertions(+)
>  create mode 100644 lib/arm/asm/gic-v2.h
>  create mode 100644 lib/arm/asm/gic.h
>  create mode 100644 lib/arm/gic.c
>  create mode 100644 lib/arm64/asm/gic-v2.h
>  create mode 100644 lib/arm64/asm/gic.h
> 
> diff --git a/arm/Makefile.common b/arm/Makefile.common
> index ccb554d9251a..41239c37e092 100644
> --- a/arm/Makefile.common
> +++ b/arm/Makefile.common
> @@ -42,6 +42,7 @@ cflatobjs += lib/arm/mmu.o
>  cflatobjs += lib/arm/bitops.o
>  cflatobjs += lib/arm/psci.o
>  cflatobjs += lib/arm/smp.o
> +cflatobjs += lib/arm/gic.o
>  
>  libeabi = lib/arm/libeabi.a
>  eabiobjs = lib/arm/eabi_compat.o
> diff --git a/lib/arm/asm/gic-v2.h b/lib/arm/asm/gic-v2.h
> new file mode 100644
> index ..c2d5fecd4886
> --- /dev/null
> +++ b/lib/arm/asm/gic-v2.h
> @@ -0,0 +1,34 @@
> +/*
> + * All GIC* defines are lifted from include/linux/irqchip/arm-gic.h
> + *
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjo...@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM_GIC_V2_H_
> +#define _ASMARM_GIC_V2_H_
> +
> +#ifndef _ASMARM_GIC_H_
> +#error Do not directly include . Include 
> +#endif
> +
> +#define GICD_ENABLE  0x1
> +#define GICC_ENABLE  0x1
> +
> +#ifndef __ASSEMBLY__
> +
> +struct gicv2_data {
> + void *dist_base;
> + void *cpu_base;
> + unsigned int irq_nr;
> +};
> +extern struct gicv2_data gicv2_data;
> +
> +#define gicv2_dist_base()(gicv2_data.dist_base)
> +#define gicv2_cpu_base() (gicv2_data.cpu_base)
> +
> +extern int gicv2_init(void);
> +extern void gicv2_enable_defaults(void);
> +
> +#endif /* !__ASSEMBLY__ */
> +#endif /* _ASMARM_GIC_V2_H_ */
> diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
> new file mode 100644
> index ..d44e47bcf404
> --- /dev/null
> +++ b/lib/arm/asm/gic.h
> @@ -0,0 +1,37 @@
> +/*
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjo...@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM_GIC_H_
> +#define _ASMARM_GIC_H_
> +
> +#include 
> +
> +#define GICD_CTLR0x
> +#define GICD_TYPER   0x0004
> +#define GICD_ISENABLER   0x0100
> +#define GICD_IPRIORITYR  0x0400
> +
> +#define GICD_TYPER_IRQS(typer)   typer) & 0x1f) + 1) * 32)
> +#define GICD_INT_EN_SET_SGI  0x
> +#define GICD_INT_DEF_PRI_X4  0xa0a0a0a0
> +
> +#define GICC_CTLR0x
> +#define GICC_PMR 0x0004
> +
> +#define GICC_INT_PRI_THRESHOLD   0xf0
> +
> +#ifndef __ASSEMBLY__
> +
> +/*
> + * gic_init will try to find all known gics, and then
> + * initialize the gic data for the one found.
> + * returns
> + *  0   : no gic was found
> + *  > 0 : the gic version of the gic found
> + */
> +extern int gic_init(void);
> +
> +#endif /* !__ASSEMBLY__ */
> +#endif /* _ASMARM_GIC_H_ */
> diff --git a/lib/arm/gic.c b/lib/arm/gic.c
> new file mode 100644
> index ..d655105e058b
> --- /dev/null
> +++ b/lib/arm/gic.c
> @@ -0,0 +1,76 @@
> +/*
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjo...@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#include 
> +#include 
> +#include 
> +
> +struct gicv2_data gicv2_data;
> +
> +/*
> + * Documentation/devicetree/bindings/interrupt-controller/arm,gic.txt
> + */
> +static bool
> +gic_get_dt_bases(const char *compatible, void **base

Re: [Qemu-devel] [kvm-unit-tests PATCH v5 03/11] arm/arm64: smp: support more than 8 cpus

[Andre Przywara]

Hi,

On 10/11/16 17:21, Andrew Jones wrote:
> By adding support for launching with gicv3 we can break the 8 vcpu
> limit. This patch adds support to smp code and also selects the
> vgic model corresponding to the host. The vgic model may also be
> manually selected by adding e.g. -machine gic-version=3 to
> extra_params.
> 
> Reviewed-by: Alex Bennée <alex.ben...@linaro.org>
> Signed-off-by: Andrew Jones <drjo...@redhat.com>
> 
> ---
> v5: left cpus a u32 for now. Changing to u64 requires a change to
> devicetree. Will do it later. [Andre]

Given that we address this in the future:

Reviewed-by: Andre Przywara <andre.przyw...@arm.com>

Re: [Qemu-devel] [kvm-unit-tests PATCH v5 10/11] arm/arm64: gicv3: add an IPI test

[Andre Przywara]

Hi,

On 11/11/16 14:53, Alex Bennée wrote:
> 
> Andrew Jones  writes:
> 
>> On Fri, Nov 11, 2016 at 10:02:59AM +, Alex Bennée wrote:
>>>
>>> Andrew Jones  writes:
>>>
 On Thu, Nov 10, 2016 at 07:53:58PM +, Alex Bennée wrote:
 [...]
>> +struct gic gicv2 = {
>> +.ipi = {
>> +.enable = gicv2_enable_defaults,
>> +.send_self = gicv2_ipi_send_self,
>> +.send_tlist = gicv2_ipi_send_tlist,
>> +.send_broadcast = gicv2_ipi_send_broadcast,
>> +},
>> +.read_iar = gicv2_read_iar,
>> +.irqnr = gicv2_irqnr,
>> +.write_eoi = gicv2_write_eoi,
>> +};
>> +
>> +struct gic gicv3 = {
>> +.ipi = {
>> +.enable = gicv3_enable_defaults,
>> +.send_self = gicv3_ipi_send_self,
>> +.send_tlist = gicv3_ipi_send_tlist,
>> +.send_broadcast = gicv3_ipi_send_broadcast,
>> +},
>> +.read_iar = gicv3_read_iar,
>> +.irqnr = gicv3_irqnr,
>> +.write_eoi = gicv3_write_eoir,
>> +};
>> +
>
> So I was re-basing my kvm-unit-tests against your GIC rework and found
> myself copy and pasting a bunch of this into my tests that fire IRQs.
> That makes me think the abstraction should be in the library code so
> other tests can fiddle with sending IRQs.
>
> What do you think?
>

 I guess you mean moving the above two structs and their corresponding
 functions (all which aren't already common) to lib/arm/ ? Or do you
 just mean the one non-trivial function gicv3_ipi_send_tlist? I think
 agree with gicv3_ipi_send_tlist getting shared, but the others are
 mostly one-liners, so I'm not sure. I guess I'd have to see how you're
 using them first.
>>>
>>> So it looked like there were some functions in the common code for one
>>> GIC which had local test defined functions for the other. They should at
>>> least be consistent.
>>
>> gicv3_read_iar and gicv3_write_eoir being common already is a product of
>> being sysreg wrappers, allowing for both arm32 and arm64 to use functions
>> of the same names, not because I wanted gicv3 to be inconsistent with
>> gicv2 (which uses MMIO and thus doesn't need wrappers)
>>
>>>
>>> For my use case I could do with a common:
>>>
>>>   gic_enable
>>
>> OK, I can extend gic_init() to initialize a 'struct gic_common_ops' that
>> includes an enable -> *_enable_defaults(void), ipi_send(int cpu),
>> read_iar(void), iar_irqnr(u32 iar), and write_eoi(u32 irqstat). And also
>> provide the wrappers gic_enable, gic_ipi_send(cpu), ...
>>
>>>   gic_send_spi(cpu, irq)
>>
>> I'll let you add this one to the new common ops struct :-)
>>
>>>   gic_irq_ack() which returns the iar.
>>
>> This one will be called read_iar.
>>
>> Would that work for you, Alex?
> 
> Sounds good to me :-)
> 
>>
>> Andre,
>>
>> Would this also satisfy your needs for more common code?

TBH I haven't look deeply enough to give an educated answer. I just
wanted to +1 Alex for the general direction. So I guess it's OK. ;-)

I guess we may need some refactoring later anyway, so any missing pieces
can be added/refactored later once we exactly know what we need.

Cheers,
Andre.

Re: [Qemu-devel] [kvm-unit-tests PATCH v5 06/11] arm/arm64: add initial gicv2 support

[Andre Przywara]

Hi,

On 11/11/16 14:52, Alex Bennée wrote:
> 
> Andrew Jones  writes:
> 
>> Add some gicv2 support. This just adds init and enable
>> functions, allowing unit tests to start messing with it.
>>
>> Signed-off-by: Andrew Jones 
>>
>> ---
>> v5: share/use only the modern register names [Andre]
>> v4:
>>  - only take defines from kernel we need now [Andre]
>>  - moved defines to asm/gic.h so they'll be shared with v3 [drew]
>>  - simplify enable by not caring if we reinit the distributor [drew]
>>  - init all GICD_INT_DEF_PRI_X4 registers [Eric]
>> ---
>>  arm/Makefile.common|  1 +
>>  lib/arm/asm/gic-v2.h   | 34 ++
>>  lib/arm/asm/gic.h  | 37 
>>  lib/arm/gic.c  | 76 
>> ++
>>  lib/arm64/asm/gic-v2.h |  1 +
>>  lib/arm64/asm/gic.h|  1 +
>>  6 files changed, 150 insertions(+)
>>  create mode 100644 lib/arm/asm/gic-v2.h
>>  create mode 100644 lib/arm/asm/gic.h
>>  create mode 100644 lib/arm/gic.c
>>  create mode 100644 lib/arm64/asm/gic-v2.h
>>  create mode 100644 lib/arm64/asm/gic.h
>>
>> diff --git a/arm/Makefile.common b/arm/Makefile.common
>> index ccb554d9251a..41239c37e092 100644
>> --- a/arm/Makefile.common
>> +++ b/arm/Makefile.common
>> @@ -42,6 +42,7 @@ cflatobjs += lib/arm/mmu.o
>>  cflatobjs += lib/arm/bitops.o
>>  cflatobjs += lib/arm/psci.o
>>  cflatobjs += lib/arm/smp.o
>> +cflatobjs += lib/arm/gic.o
>>
>>  libeabi = lib/arm/libeabi.a
>>  eabiobjs = lib/arm/eabi_compat.o
>> diff --git a/lib/arm/asm/gic-v2.h b/lib/arm/asm/gic-v2.h
>> new file mode 100644
>> index ..c2d5fecd4886
>> --- /dev/null
>> +++ b/lib/arm/asm/gic-v2.h
>> @@ -0,0 +1,34 @@
>> +/*
>> + * All GIC* defines are lifted from include/linux/irqchip/arm-gic.h
>> + *
>> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones 
>> + *
>> + * This work is licensed under the terms of the GNU LGPL, version 2.
>> + */
>> +#ifndef _ASMARM_GIC_V2_H_
>> +#define _ASMARM_GIC_V2_H_
>> +
>> +#ifndef _ASMARM_GIC_H_
>> +#error Do not directly include . Include 
>> +#endif
>> +
>> +#define GICD_ENABLE 0x1
>> +#define GICC_ENABLE 0x1
>> +
>> +#ifndef __ASSEMBLY__
>> +
>> +struct gicv2_data {
>> +void *dist_base;
>> +void *cpu_base;
>> +unsigned int irq_nr;
>> +};
>> +extern struct gicv2_data gicv2_data;
>> +
>> +#define gicv2_dist_base()   (gicv2_data.dist_base)
>> +#define gicv2_cpu_base()(gicv2_data.cpu_base)
>> +
>> +extern int gicv2_init(void);
>> +extern void gicv2_enable_defaults(void);
>> +
>> +#endif /* !__ASSEMBLY__ */
>> +#endif /* _ASMARM_GIC_V2_H_ */
>> diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
>> new file mode 100644
>> index ..d44e47bcf404
>> --- /dev/null
>> +++ b/lib/arm/asm/gic.h
>> @@ -0,0 +1,37 @@
>> +/*
>> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones 
>> + *
>> + * This work is licensed under the terms of the GNU LGPL, version 2.
>> + */
>> +#ifndef _ASMARM_GIC_H_
>> +#define _ASMARM_GIC_H_
>> +
>> +#include 
>> +
>> +#define GICD_CTLR   0x
>> +#define GICD_TYPER  0x0004
>> +#define GICD_ISENABLER  0x0100
>> +#define GICD_IPRIORITYR 0x0400
> 
> Maybe GICD_ISENABLER_BASE and GICD_IPRIORITYR_BASE as they are the start
> of a series of registers?

We should keep the naming consistent to both the spec and the Linux headers.

> Also what happened to the formatting?

Isn't that the usual diff artifact caused by prepending a single
character? Which moves the tab stops, also the mailer adding quotation
characters?

>> +
>> +#define GICD_TYPER_IRQS(typer)  typer) & 0x1f) + 1)
>> * 32)
>> +#define GICD_INT_EN_SET_SGI 0x
>> +#define GICD_INT_DEF_PRI_X4 0xa0a0a0a0
> 
> This doesn't seem to be used and I'm not sure what GICD_TYPER_IRQS it is
> trying to achieve.

The idea is to calculate the number of implemented SPIs. But I am not a
big fan of copying a macro from the emulation code base to the test code.

Cheers,
Andre.

> A comment above and here to make it clear we are talking about offsets
> in the distributor and cpu register maps would aid confusion.
> 
>> +
>> +#define GICC_CTLR   0x
>> +#define GICC_PMR0x0004
>> +
>> +#define GICC_INT_PRI_THRESHOLD  0xf0
>> +
>> +#ifndef __ASSEMBLY__
>> +
>> +/*
>> + * gic_init will try to find all known gics, and then
>> + * initialize the gic data for the one found.
>> + * returns
>> + *  0   : no gic was found
>> + *  > 0 : the gic version of the gic found
>> + */
>> +extern int gic_init(void);
> 
> If we are going to make the library API agnostic I guess returning NULL
> or an ops structure would be best here?
> 
>> +
>> +#endif /* !__ASSEMBLY__ */
>> +#endif /* _ASMARM_GIC_H_ */
>> diff --git a/lib/arm/gic.c b/lib/arm/gic.c
>> new file mode 100644
>> 

Re: [Qemu-devel] [kvm-unit-tests PATCH v5 07/11] arm/arm64: gicv2: add an IPI test

[Andre Przywara]

Hi,

more a comment loosely related to this patch ...

> diff --git a/arm/unittests.cfg b/arm/unittests.cfg
> index 3f6fa45c587e..68bf5cd6008f 100644
> --- a/arm/unittests.cfg
> +++ b/arm/unittests.cfg
> @@ -54,3 +54,10 @@ file = selftest.flat
>  smp = $MAX_SMP
>  extra_params = -append 'smp'
>  groups = selftest
> +
> +# Test GIC emulation
> +[gicv2-ipi]
> +file = gic.flat
> +smp = $((($MAX_SMP < 8)?$MAX_SMP:8))

So here we always go with the maximum number of VCPUs in the guest.
However (as you also noted in your cover-letter) running with a
different number of CPUs might be interesting, for instance with less
than 8 CPUs on a GICv2 (the ITARGETSR register must be masked) or in
general with an odd number (both literally and in the broader sense). I
have a test case with passes with 8 VCPUs but fails with less.

Is there any good way to run some tests multiple times with different
numbers of VCPUS?
Shall we add some "set" functionality to the smp parameter, so that we
can specify a list of desired test points?

Cheers,
Andre.

Re: [Qemu-devel] [kvm-unit-tests PATCH v5 10/11] arm/arm64: gicv3: add an IPI test

[Andre Przywara]

Hi,

On 10/11/16 19:53, Alex Bennée wrote:


> So I was re-basing my kvm-unit-tests against your GIC rework and found
> myself copy and pasting a bunch of this into my tests that fire IRQs.

So I take it you are working on (or already have) code to test SPIs,
probably via GICD_ISPENDR?
Just asking because I was thinking about going there and thus could save
my time if you are on it already...

> That makes me think the abstraction should be in the library code so
> other tests can fiddle with sending IRQs.

...because I was wondering the same.

Cheers,
Andre.

Re: [Qemu-devel] [kvm-unit-tests PATCH v4 00/11] arm/arm64: add gic framework

[Andre Przywara]

Hi,

so is this actually v4 just resent?
Or is this is a new version with s/5/4/?
I can't spot any of the key changes quickly ...

Cheers,
Andre.

On 10/11/16 16:07, Andrew Jones wrote:
> v4:
>  - Eric's r-b's
>  - Andre's suggestion to only take defines we need
>  - several other changes listed in individual patches
> 
> v3:
>  - Rebased on latest master
>  - Added Alex's r-b's
> 
> v2:
>  Rebased on latest master + my "populate argv[0]" series (will
>  send a REPOST for that shortly. Additionally a few patches got
>  fixes/features;
>  07/10 got same fix as kernel 7c9b973061 "irqchip/gic-v3: Configure
>all interrupts as non-secure Group-1" in order to continue
>working over TCG, as the gicv3 code for TCG removed a hack
>it had there to make Linux happy.
>  08/10 added more output for when things fail (if they fail)
>  09/10 switched gicv3 broadcast implementation to using IRM. This
>found a bug in a recent (but not tip) kernel, which I was
>about to fix, but then I saw MarcZ beat me to it.
>  10/10 actually check that the input irq is the received irq
> 
> 
> Import defines, and steal enough helper functions, from Linux to
> enable programming of the gic (v2 and v3). Then use the framework
> to add an initial test (an ipi test; self, target-list, broadcast).
> 
> It's my hope that this framework will be a suitable base on which
> more tests may be easily added, particularly because we have
> vgic-new and tcg gicv3 emulation getting close to merge. (v3 UPDATE:
> vgic-new and tcg gicv3 are merged now)
> 
> To run it, along with other tests, just do
> 
>  ./configure [ --arch=[arm|arm64] --cross-prefix=$PREFIX ]
>  make
>  export QEMU=$PATH_TO_QEMU
>  ./run_tests.sh
> 
> To run it separately do, e.g.
> 
> $QEMU -machine virt,accel=tcg -cpu cortex-a57 \
>  -device virtio-serial-device \
>  -device virtconsole,chardev=ctd -chardev testdev,id=ctd \
>  -display none -serial stdio \
>  -kernel arm/gic.flat \
>  -smp 123 -machine gic-version=3 -append ipi
>   ^^ note, we can go nuts with nr-cpus on TCG :-)
> 
> Or, a KVM example using a different "sender" cpu and irq (other than zero)
> 
> $QEMU -machine virt,accel=kvm -cpu host \
>  -device virtio-serial-device \
>  -device virtconsole,chardev=ctd -chardev testdev,id=ctd \
>  -display none -serial stdio \
>  -kernel arm/gic.flat \
>  -smp 48 -machine gic-version=3 -append 'ipi sender=42 irq=1'
> 
> 
> Patches:
> 01-05: fixes and functionality needed by the later gic patches
> 06-07: enable gicv2 and gicv2 IPI test
> 08-10: enable gicv3 and gicv3 IPI test
>11: extend the IPI tests to take variable sender and irq
> 
> Available here: https://github.com/rhdrjones/kvm-unit-tests/commits/arm/gic-v4
> 
> 
> Andrew Jones (10):
>   lib: xstr: allow multiple args
>   arm64: fix get_"sysreg32" and make MPIDR 64bit
>   arm/arm64: smp: support more than 8 cpus
>   arm/arm64: add some delay routines
>   arm/arm64: irq enable/disable
>   arm/arm64: add initial gicv2 support
>   arm/arm64: gicv2: add an IPI test
>   arm/arm64: add initial gicv3 support
>   arm/arm64: gicv3: add an IPI test
>   arm/arm64: gic: don't just use zero
> 
> Peter Xu (1):
>   libcflat: add IS_ALIGNED() macro, and page sizes
> 
>  arm/Makefile.common|   7 +-
>  arm/gic.c  | 417 
> +
>  arm/run|  19 ++-
>  arm/selftest.c |   5 +-
>  arm/unittests.cfg  |  13 ++
>  lib/arm/asm/arch_gicv3.h   |  65 +++
>  lib/arm/asm/gic-v2.h   |  28 +++
>  lib/arm/asm/gic-v3.h   |  92 ++
>  lib/arm/asm/gic.h  |  51 ++
>  lib/arm/asm/processor.h|  38 -
>  lib/arm/asm/setup.h|   4 +-
>  lib/arm/gic.c  | 131 ++
>  lib/arm/processor.c|  15 ++
>  lib/arm/setup.c|  12 +-
>  lib/arm64/asm/arch_gicv3.h |  66 +++
>  lib/arm64/asm/gic-v2.h |   1 +
>  lib/arm64/asm/gic-v3.h |   1 +
>  lib/arm64/asm/gic.h|   1 +
>  lib/arm64/asm/processor.h  |  53 +-
>  lib/arm64/asm/sysreg.h |  44 +
>  lib/arm64/processor.c  |  15 ++
>  lib/libcflat.h |  10 +-
>  22 files changed, 1062 insertions(+), 26 deletions(-)
>  create mode 100644 arm/gic.c
>  create mode 100644 lib/arm/asm/arch_gicv3.h
>  create mode 100644 lib/arm/asm/gic-v2.h
>  create mode 100644 lib/arm/asm/gic-v3.h
>  create mode 100644 lib/arm/asm/gic.h
>  create mode 100644 lib/arm/gic.c
>  create mode 100644 lib/arm64/asm/arch_gicv3.h
>  create mode 100644 lib/arm64/asm/gic-v2.h
>  create mode 100644 lib/arm64/asm/gic-v3.h
>  create mode 100644 lib/arm64/asm/gic.h
>  create mode 100644 lib/arm64/asm/sysreg.h
> 

Re: [Qemu-devel] [kvm-unit-tests PATCH v4 08/11] libcflat: add IS_ALIGNED() macro, and page sizes

[Andre Przywara]

Hi,

On 08/11/16 20:21, Andrew Jones wrote:
> From: Peter Xu <pet...@redhat.com>
> 
> These macros will be useful to do page alignment checks.

Reviewed-by: Andre Przywara <andre.przyw...@arm.com>

Cheers,
Andre.

> Signed-off-by: Peter Xu <pet...@redhat.com>
> [drew: also added SZ_64K]
> Signed-off-by: Andrew Jones <drjo...@redhat.com>
> ---
>  lib/libcflat.h | 6 ++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/lib/libcflat.h b/lib/libcflat.h
> index 82005f5d014f..143fc53061fe 100644
> --- a/lib/libcflat.h
> +++ b/lib/libcflat.h
> @@ -33,6 +33,12 @@
>  #define __ALIGN_MASK(x, mask)(((x) + (mask)) & ~(mask))
>  #define __ALIGN(x, a)__ALIGN_MASK(x, (typeof(x))(a) - 1)
>  #define ALIGN(x, a)  __ALIGN((x), (a))
> +#define IS_ALIGNED(x, a) (((x) & ((typeof(x))(a) - 1)) == 0)
> +
> +#define SZ_4K(0x1000)
> +#define SZ_64K   (0x1)
> +#define SZ_2M(0x20)
> +#define SZ_1G(0x4000)
>  
>  typedef uint8_t  u8;
>  typedef int8_t   s8;
> 

Re: [Qemu-devel] [kvm-unit-tests PATCH v4 09/11] arm/arm64: add initial gicv3 support

[Andre Przywara]

Hi,

On 09/11/16 13:08, Andrew Jones wrote:
> On Wed, Nov 09, 2016 at 12:35:48PM +0000, Andre Przywara wrote:
> [...]
>>> diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
>>> new file mode 100644
>>> index ..03321f8c860f
>>> --- /dev/null
>>> +++ b/lib/arm/asm/gic-v3.h
>>> @@ -0,0 +1,92 @@
>>> +/*
>>> + * All GIC* defines are lifted from include/linux/irqchip/arm-gic-v3.h
>>> + *
>>> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones <drjo...@redhat.com>
>>> + *
>>> + * This work is licensed under the terms of the GNU LGPL, version 2.
>>> + */
>>> +#ifndef _ASMARM_GIC_V3_H_
>>> +#define _ASMARM_GIC_V3_H_
>>> +
>>> +#ifndef _ASMARM_GIC_H_
>>> +#error Do not directly include . Include 
>>> +#endif
>>> +
>>> +#define GICD_CTLR  0x
>>> +#define GICD_TYPER 0x0004
>>
>> So if we share the distributor register definition with GICv2, these
>> shouldn't be here, but in gic.h.
>> But this is the right naming scheme we should use (instead of GIC_DIST_xxx).
>>
>> Now this gets interesting with your wish to both share the definitions
>> for the GICv2 and GICv3 distributors, but also stick to the names the
>> kernel uses (because they differ between the two) ;-)
>> So now you loose the greppability for either GIC_DIST_CTR or GICD_TYPER,
>> for instance.
> 
> Well, we just have the same offset with two names (giving us two
> symbols to grep). I put them here, vs. asm/gic.h, because the kernel
> only uses theses symbols for gicv3. Now, nothing stops a unit test
> from using them with gicv2 tests, though, because unit tests include
> gic.h, which includes both gic-v2.h and gic-v3.h, and thus it gets
> both. I know, it's sounding messy... Shouldn't we post some churn to
> the kernel? :-)

Well, on top of that the distributor registers are slightly different
(check CTLR and TYPER, for instance). So it's churn plus a stretch, I
guess Marc won't like that.

So if greppability is important, should we revert to separate
definitions in separate header files then, like in v3?
I don't think we actually share _code_ between the two GIC revisions, do we?

> Note, I tried to only add defines to asm/gic.h that are actually
> shared in the kernel between v2 and v3, e.g. GIC_DIST_ENABLE_SET.

Huh? GICv3 uses GICD_ISENABLER for that register.

> Actually, GIC_DIST_CTRL and GIC_DIST_CTR may be the only exceptions
> we have so far.

Note that it's GIC_DIST_CTLR (L and R swapped), one reason more to dump
_CTR ;-)

>>
>>> +#define GICD_IGROUPR   0x0080
>>> +
>>> +#define GICD_CTLR_RWP  (1U << 31)
>>> +#define GICD_CTLR_ARE_NS   (1U << 4)
>>> +#define GICD_CTLR_ENABLE_G1A   (1U << 1)
>>> +#define GICD_CTLR_ENABLE_G1(1U << 0)
>>> +
>>> +#define GICR_TYPER 0x0008
>>> +#define GICR_IGROUPR0  GICD_IGROUPR
>>> +#define GICR_TYPER_LAST(1U << 4)
>>> +
>>> +
>>> +#include 
>>> +
>>> +#ifndef __ASSEMBLY__
>>> +#include 
>>> +#include 
>>> +#include 
>>> +#include 
>>> +
>>> +struct gicv3_data {
>>> +   void *dist_base;
>>> +   void *redist_base[NR_CPUS];
>>> +   unsigned int irq_nr;
>>> +};
>>> +extern struct gicv3_data gicv3_data;
>>> +
>>> +#define gicv3_dist_base()  (gicv3_data.dist_base)
>>> +#define gicv3_redist_base()
>>> (gicv3_data.redist_base[smp_processor_id()])
>>> +#define gicv3_sgi_base()   
>>> (gicv3_data.redist_base[smp_processor_id()] + SZ_64K)
>>> +
>>> +extern int gicv3_init(void);
>>> +extern void gicv3_enable_defaults(void);
>>> +
>>> +static inline void gicv3_do_wait_for_rwp(void *base)
>>> +{
>>> +   int count = 10; /* 1s */
>>> +
>>> +   while (readl(base + GICD_CTLR) & GICD_CTLR_RWP) {
>>> +   if (!--count) {
>>> +   printf("GICv3: RWP timeout!\n");
>>> +   abort();
>>> +   }
>>> +   cpu_relax();
>>> +   udelay(10);
>>> +   };
>>> +}
>>> +
>>> +static inline void gicv3_dist_wait_for_rwp(void)
>>> +{
>>> +   gicv3_do_wait_for_rwp(gicv3_dist_base());
>>> +}
>>> +
>>> +static inline void gicv3_redist_

Re: [Qemu-devel] [kvm-unit-tests PATCH v4 09/11] arm/arm64: add initial gicv3 support

[Andre Przywara]

Hi,

On 08/11/16 20:21, Andrew Jones wrote:
> Signed-off-by: Andrew Jones 
> 
> ---
> v4:
>  - only take defines from kernel we need now [Andre]
>  - simplify enable by not caring if we reinit the distributor [drew]
> v2:
>  - configure irqs as NS GRP1
> ---
>  lib/arm/asm/arch_gicv3.h   | 42 +
>  lib/arm/asm/gic-v3.h   | 92 
> ++
>  lib/arm/asm/gic.h  |  1 +
>  lib/arm/gic.c  | 56 
>  lib/arm64/asm/arch_gicv3.h | 44 ++
>  lib/arm64/asm/gic-v3.h |  1 +
>  lib/arm64/asm/sysreg.h | 44 ++
>  7 files changed, 280 insertions(+)
>  create mode 100644 lib/arm/asm/arch_gicv3.h
>  create mode 100644 lib/arm/asm/gic-v3.h
>  create mode 100644 lib/arm64/asm/arch_gicv3.h
>  create mode 100644 lib/arm64/asm/gic-v3.h
>  create mode 100644 lib/arm64/asm/sysreg.h
> 
> diff --git a/lib/arm/asm/arch_gicv3.h b/lib/arm/asm/arch_gicv3.h
> new file mode 100644
> index ..81a1e5f6c29c
> --- /dev/null
> +++ b/lib/arm/asm/arch_gicv3.h
> @@ -0,0 +1,42 @@
> +/*
> + * All ripped off from arch/arm/include/asm/arch_gicv3.h
> + *
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones 
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM_ARCH_GICV3_H_
> +#define _ASMARM_ARCH_GICV3_H_
> +
> +#ifndef __ASSEMBLY__
> +#include 
> +#include 
> +#include 
> +
> +#define __stringify xstr
> +
> +#define __ACCESS_CP15(CRn, Op1, CRm, Op2)p15, Op1, %0, CRn, CRm, Op2
> +
> +#define ICC_PMR  __ACCESS_CP15(c4, 0, c6, 0)
> +#define ICC_IGRPEN1  __ACCESS_CP15(c12, 0, c12, 7)
> +
> +static inline void gicv3_write_pmr(u32 val)
> +{
> + asm volatile("mcr " __stringify(ICC_PMR) : : "r" (val));
> +}
> +
> +static inline void gicv3_write_grpen1(u32 val)
> +{
> + asm volatile("mcr " __stringify(ICC_IGRPEN1) : : "r" (val));
> + isb();
> +}
> +
> +static inline u64 gicv3_read_typer(const volatile void __iomem *addr)
> +{
> + u64 val = readl(addr);
> + val |= (u64)readl(addr + 4) << 32;
> + return val;
> +}
> +
> +#endif /* !__ASSEMBLY__ */
> +#endif /* _ASMARM_ARCH_GICV3_H_ */
> diff --git a/lib/arm/asm/gic-v3.h b/lib/arm/asm/gic-v3.h
> new file mode 100644
> index ..03321f8c860f
> --- /dev/null
> +++ b/lib/arm/asm/gic-v3.h
> @@ -0,0 +1,92 @@
> +/*
> + * All GIC* defines are lifted from include/linux/irqchip/arm-gic-v3.h
> + *
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones 
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM_GIC_V3_H_
> +#define _ASMARM_GIC_V3_H_
> +
> +#ifndef _ASMARM_GIC_H_
> +#error Do not directly include . Include 
> +#endif
> +
> +#define GICD_CTLR0x
> +#define GICD_TYPER   0x0004

So if we share the distributor register definition with GICv2, these
shouldn't be here, but in gic.h.
But this is the right naming scheme we should use (instead of GIC_DIST_xxx).

Now this gets interesting with your wish to both share the definitions
for the GICv2 and GICv3 distributors, but also stick to the names the
kernel uses (because they differ between the two) ;-)
So now you loose the greppability for either GIC_DIST_CTR or GICD_TYPER,
for instance.

> +#define GICD_IGROUPR 0x0080
> +
> +#define GICD_CTLR_RWP(1U << 31)
> +#define GICD_CTLR_ARE_NS (1U << 4)
> +#define GICD_CTLR_ENABLE_G1A (1U << 1)
> +#define GICD_CTLR_ENABLE_G1  (1U << 0)
> +
> +#define GICR_TYPER   0x0008
> +#define GICR_IGROUPR0GICD_IGROUPR
> +#define GICR_TYPER_LAST  (1U << 4)
> +
> +
> +#include 
> +
> +#ifndef __ASSEMBLY__
> +#include 
> +#include 
> +#include 
> +#include 
> +
> +struct gicv3_data {
> + void *dist_base;
> + void *redist_base[NR_CPUS];
> + unsigned int irq_nr;
> +};
> +extern struct gicv3_data gicv3_data;
> +
> +#define gicv3_dist_base()(gicv3_data.dist_base)
> +#define gicv3_redist_base()  
> (gicv3_data.redist_base[smp_processor_id()])
> +#define gicv3_sgi_base() 
> (gicv3_data.redist_base[smp_processor_id()] + SZ_64K)
> +
> +extern int gicv3_init(void);
> +extern void gicv3_enable_defaults(void);
> +
> +static inline void gicv3_do_wait_for_rwp(void *base)
> +{
> + int count = 10; /* 1s */
> +
> + while (readl(base + GICD_CTLR) & GICD_CTLR_RWP) {
> + if (!--count) {
> + printf("GICv3: RWP timeout!\n");
> + abort();
> + }
> + cpu_relax();
> + udelay(10);
> + };
> +}
> +
> +static inline void gicv3_dist_wait_for_rwp(void)
> +{
> + gicv3_do_wait_for_rwp(gicv3_dist_base());
> +}
> +
> +static inline void 

Re: [Qemu-devel] [kvm-unit-tests PATCH v4 03/11] arm/arm64: smp: support more than 8 cpus

[Andre Przywara]

Hi,

On 09/11/16 11:57, Andrew Jones wrote:
> On Wed, Nov 09, 2016 at 11:12:03AM +0000, Andre Przywara wrote:
> [...]
>>> diff --git a/lib/arm/setup.c b/lib/arm/setup.c
>>> index 7e7b39f11dde..b6e2d5815e72 100644
>>> --- a/lib/arm/setup.c
>>> +++ b/lib/arm/setup.c
>>> @@ -24,12 +24,22 @@ extern unsigned long stacktop;
>>>  extern void io_init(void);
>>>  extern void setup_args_progname(const char *args);
>>>  
>>> -u32 cpus[NR_CPUS] = { [0 ... NR_CPUS-1] = (~0U) };
>>> +u64 cpus[NR_CPUS] = { [0 ... NR_CPUS-1] = (~0U) };
>>
>> This should be ~0UL.
> 
> Indeed. Thanks.
> 
>> Also I think the type should be unsigned long to match the types used
>> everywhere else.
> 
> I'll change mpidr_to_cpu to return u64 instead of unsigned long.

I am not sure this is the right direction. unsigned long is really the
natural type for MPIDR, since this is a system register with exactly the
native register size.
I think we use it this way in the kernel.

Cheers,
Andre

> Actually I think Alex suggested that. I'm not sure why I haven't
> done it...
> 
>>
>>>  int nr_cpus;
>>>  
>>>  struct mem_region mem_regions[NR_MEM_REGIONS];
>>>  phys_addr_t __phys_offset, __phys_end;
>>>  
>>> +int mpidr_to_cpu(unsigned long mpidr)
>>> +{
>>> +   int i;
>>> +
>>> +   for (i = 0; i < nr_cpus; ++i)
>>> +   if (cpus[i] == (mpidr & MPIDR_HWID_BITMASK))
>>> +   return i;
>>> +   return -1;
>>> +}
>>> +
>>>  static void cpu_set(int fdtnode __unused, u32 regval, void *info __unused)
>>
>> I guess this needs to be extended as well, including
>> dt_for_each_cpu_node() to cope with 64-bit reg properties in the CPU
>> node (where the upper word is not 0).
>> But this is not really crucial atm, so can be fixed in a follow-up patch.
> 
> Yeah, I'll do it as a followup series, because it'll affect powerpc too.
> 
> drew
> 

Re: [Qemu-devel] [kvm-unit-tests PATCH v4 06/11] arm/arm64: add initial gicv2 support

[Andre Przywara]

Hi,

On 08/11/16 20:21, Andrew Jones wrote:
> Add some gicv2 support. This just adds init and enable
> functions, allowing unit tests to start messing with it.
> 
> Signed-off-by: Andrew Jones 
> 
> ---
> v4:
>  - only take defines from kernel we need now [Andre]
>  - moved defines to asm/gic.h so they'll be shared with v3 [drew]
>  - simplify enable by not caring if we reinit the distributor [drew]
>  - init all GICD_INT_DEF_PRI_X4 registers [Eric]
> ---
>  arm/Makefile.common|  1 +
>  lib/arm/asm/gic-v2.h   | 28 +++
>  lib/arm/asm/gic.h  | 44 +
>  lib/arm/gic.c  | 75 
> ++
>  lib/arm64/asm/gic-v2.h |  1 +
>  lib/arm64/asm/gic.h|  1 +
>  6 files changed, 150 insertions(+)
>  create mode 100644 lib/arm/asm/gic-v2.h
>  create mode 100644 lib/arm/asm/gic.h
>  create mode 100644 lib/arm/gic.c
>  create mode 100644 lib/arm64/asm/gic-v2.h
>  create mode 100644 lib/arm64/asm/gic.h
> 
> diff --git a/arm/Makefile.common b/arm/Makefile.common
> index ccb554d9251a..41239c37e092 100644
> --- a/arm/Makefile.common
> +++ b/arm/Makefile.common
> @@ -42,6 +42,7 @@ cflatobjs += lib/arm/mmu.o
>  cflatobjs += lib/arm/bitops.o
>  cflatobjs += lib/arm/psci.o
>  cflatobjs += lib/arm/smp.o
> +cflatobjs += lib/arm/gic.o
>  
>  libeabi = lib/arm/libeabi.a
>  eabiobjs = lib/arm/eabi_compat.o
> diff --git a/lib/arm/asm/gic-v2.h b/lib/arm/asm/gic-v2.h
> new file mode 100644
> index ..f91530f88355
> --- /dev/null
> +++ b/lib/arm/asm/gic-v2.h
> @@ -0,0 +1,28 @@
> +/*
> + * All GIC* defines are lifted from include/linux/irqchip/arm-gic.h
> + *
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones 
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM_GIC_V2_H_
> +#define _ASMARM_GIC_V2_H_
> +
> +#ifndef _ASMARM_GIC_H_
> +#error Do not directly include . Include 
> +#endif
> +
> +struct gicv2_data {
> + void *dist_base;
> + void *cpu_base;
> + unsigned int irq_nr;
> +};
> +extern struct gicv2_data gicv2_data;
> +
> +#define gicv2_dist_base()(gicv2_data.dist_base)
> +#define gicv2_cpu_base() (gicv2_data.cpu_base)
> +
> +extern int gicv2_init(void);
> +extern void gicv2_enable_defaults(void);
> +
> +#endif /* _ASMARM_GIC_V2_H_ */
> diff --git a/lib/arm/asm/gic.h b/lib/arm/asm/gic.h
> new file mode 100644
> index ..ec92f1064dc0
> --- /dev/null
> +++ b/lib/arm/asm/gic.h
> @@ -0,0 +1,44 @@
> +/*
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones 
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM_GIC_H_
> +#define _ASMARM_GIC_H_
> +
> +#include 
> +
> +#define GIC_CPU_CTRL 0x00
> +#define GIC_CPU_PRIMASK  0x04
> +
> +#define GICC_ENABLE  0x1
> +#define GICC_INT_PRI_THRESHOLD   0xf0
> +
> +#define GIC_DIST_CTRL0x000
> +#define GIC_DIST_CTR 0x004

I think we shouldn't copy this old name here, which stems from pre-GICv2
times (PL390?), IIUC. Both GIC specs talk of TYPER here.

Also if we now use the same defines for both the GICv2 and GICv3
distributor, we should really stick with the (modern) spec naming, which
is GICD_CTRL, GICD_TYPER and so on. Same for the CPU interface (GICC_CTRL).
In the kernel these names are used for GICv3, but we didn't bother to
adjust the existing GICv2 names to avoid pointless churn.
Also that would line up with the bit field defines below.

Cheers,
Andre.

> +#define GIC_DIST_ENABLE_SET  0x100
> +#define GIC_DIST_PRI 0x400
> +
> +#define GICD_ENABLE  0x1
> +#define GICD_INT_EN_SET_SGI  0x
> +#define GICD_INT_DEF_PRI 0xa0
> +#define GICD_INT_DEF_PRI_X4  ((GICD_INT_DEF_PRI << 24) |\
> + (GICD_INT_DEF_PRI << 16) |\
> + (GICD_INT_DEF_PRI << 8) |\
> + GICD_INT_DEF_PRI)
> +
> +#define GICD_TYPER_IRQS(typer)   typer) & 0x1f) + 1) * 32)
> +
> +#ifndef __ASSEMBLY__
> +
> +/*
> + * gic_init will try to find all known gics, and then
> + * initialize the gic data for the one found.
> + * returns
> + *  0   : no gic was found
> + *  > 0 : the gic version of the gic found
> + */
> +extern int gic_init(void);
> +
> +#endif /* !__ASSEMBLY__ */
> +#endif /* _ASMARM_GIC_H_ */
> diff --git a/lib/arm/gic.c b/lib/arm/gic.c
> new file mode 100644
> index ..91d78c9a0cc2
> --- /dev/null
> +++ b/lib/arm/gic.c
> @@ -0,0 +1,75 @@
> +/*
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones 
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#include 
> +#include 
> +#include 
> +
> +struct gicv2_data gicv2_data;
> +
> +/*
> + * 

Re: [Qemu-devel] [kvm-unit-tests PATCH v4 03/11] arm/arm64: smp: support more than 8 cpus

[Andre Przywara]

Hi,

On 08/11/16 20:21, Andrew Jones wrote:
> By adding support for launching with gicv3 we can break the 8 vcpu
> limit. This patch adds support to smp code and also selects the
> vgic model corresponding to the host. The vgic model may also be
> manually selected by adding e.g. -machine gic-version=3 to
> extra_params.
> 
> Reviewed-by: Alex Bennée 
> Signed-off-by: Andrew Jones 
> 
> ---
> v4: improved commit message
> ---
>  arm/run   | 19 ---
>  arm/selftest.c|  5 -
>  lib/arm/asm/processor.h   |  9 +++--
>  lib/arm/asm/setup.h   |  4 ++--
>  lib/arm/setup.c   | 12 +++-
>  lib/arm64/asm/processor.h |  9 +++--
>  6 files changed, 43 insertions(+), 15 deletions(-)
> 
> diff --git a/arm/run b/arm/run
> index a2f35ef6a7e6..2d0698619606 100755
> --- a/arm/run
> +++ b/arm/run
> @@ -31,13 +31,6 @@ if [ -z "$ACCEL" ]; then
>   fi
>  fi
>  
> -if [ "$HOST" = "aarch64" ] && [ "$ACCEL" = "kvm" ]; then
> - processor="host"
> - if [ "$ARCH" = "arm" ]; then
> - processor+=",aarch64=off"
> - fi
> -fi
> -
>  qemu="${QEMU:-qemu-system-$ARCH_NAME}"
>  qpath=$(which $qemu 2>/dev/null)
>  
> @@ -53,6 +46,18 @@ fi
>  
>  M='-machine virt'
>  
> +if [ "$ACCEL" = "kvm" ]; then
> + if $qemu $M,\? 2>&1 | grep gic-version > /dev/null; then
> + M+=',gic-version=host'
> + fi
> + if [ "$HOST" = "aarch64" ]; then
> + processor="host"
> + if [ "$ARCH" = "arm" ]; then
> + processor+=",aarch64=off"
> + fi
> + fi
> +fi
> +
>  if ! $qemu $M -device '?' 2>&1 | grep virtconsole > /dev/null; then
>   echo "$qpath doesn't support virtio-console for chr-testdev. Exiting."
>   exit 2
> diff --git a/arm/selftest.c b/arm/selftest.c
> index 196164f5313d..2f117f795d2d 100644
> --- a/arm/selftest.c
> +++ b/arm/selftest.c
> @@ -312,9 +312,10 @@ static bool psci_check(void)
>  static cpumask_t smp_reported;
>  static void cpu_report(void)
>  {
> + unsigned long mpidr = get_mpidr();
>   int cpu = smp_processor_id();
>  
> - report("CPU%d online", true, cpu);
> + report("CPU(%3d) mpidr=%lx", mpidr_to_cpu(mpidr) == cpu, cpu, mpidr);
>   cpumask_set_cpu(cpu, _reported);
>   halt();
>  }
> @@ -343,6 +344,7 @@ int main(int argc, char **argv)
>  
>   } else if (strcmp(argv[1], "smp") == 0) {
>  
> + unsigned long mpidr = get_mpidr();
>   int cpu;
>  
>   report("PSCI version", psci_check());
> @@ -353,6 +355,7 @@ int main(int argc, char **argv)
>   smp_boot_secondary(cpu, cpu_report);
>   }
>  
> + report("CPU(%3d) mpidr=%lx", mpidr_to_cpu(mpidr) == 0, 0, 
> mpidr);
>   cpumask_set_cpu(0, _reported);
>   while (!cpumask_full(_reported))
>   cpu_relax();
> diff --git a/lib/arm/asm/processor.h b/lib/arm/asm/processor.h
> index f25e7eee3666..d2048f5f5f7e 100644
> --- a/lib/arm/asm/processor.h
> +++ b/lib/arm/asm/processor.h
> @@ -40,8 +40,13 @@ static inline unsigned int get_mpidr(void)
>   return mpidr;
>  }
>  
> -/* Only support Aff0 for now, up to 4 cpus */
> -#define mpidr_to_cpu(mpidr) ((int)((mpidr) & 0xff))
> +#define MPIDR_HWID_BITMASK 0xff
> +extern int mpidr_to_cpu(unsigned long mpidr);
> +
> +#define MPIDR_LEVEL_SHIFT(level) \
> + (((1 << level) >> 1) << 3)
> +#define MPIDR_AFFINITY_LEVEL(mpidr, level) \
> + ((mpidr >> MPIDR_LEVEL_SHIFT(level)) & 0xff)
>  
>  extern void start_usr(void (*func)(void *arg), void *arg, unsigned long 
> sp_usr);
>  extern bool is_user(void);
> diff --git a/lib/arm/asm/setup.h b/lib/arm/asm/setup.h
> index cb8fdbd38dd5..b0d51f5f0721 100644
> --- a/lib/arm/asm/setup.h
> +++ b/lib/arm/asm/setup.h
> @@ -10,8 +10,8 @@
>  #include 
>  #include 
>  
> -#define NR_CPUS  8
> -extern u32 cpus[NR_CPUS];
> +#define NR_CPUS  255
> +extern u64 cpus[NR_CPUS];/* per-cpu IDs (MPIDRs) */
>  extern int nr_cpus;
>  
>  #define NR_MEM_REGIONS   8
> diff --git a/lib/arm/setup.c b/lib/arm/setup.c
> index 7e7b39f11dde..b6e2d5815e72 100644
> --- a/lib/arm/setup.c
> +++ b/lib/arm/setup.c
> @@ -24,12 +24,22 @@ extern unsigned long stacktop;
>  extern void io_init(void);
>  extern void setup_args_progname(const char *args);
>  
> -u32 cpus[NR_CPUS] = { [0 ... NR_CPUS-1] = (~0U) };
> +u64 cpus[NR_CPUS] = { [0 ... NR_CPUS-1] = (~0U) };

This should be ~0UL.
Also I think the type should be unsigned long to match the types used
everywhere else.

>  int nr_cpus;
>  
>  struct mem_region mem_regions[NR_MEM_REGIONS];
>  phys_addr_t __phys_offset, __phys_end;
>  
> +int mpidr_to_cpu(unsigned long mpidr)
> +{
> + int i;
> +
> + for (i = 0; i < nr_cpus; ++i)
> + if (cpus[i] == (mpidr & MPIDR_HWID_BITMASK))
> + return i;
> + return -1;
> +}
> +
>  static 

[Qemu-devel] [RFC PATCH] kvm-unit-tests: arm/arm64: strip GIC headers

[Andre Przywara]

Hi,

this is an illustrative patch which shows what can be removed from
the kvm-unit-tests GIC headers. If this patch finds mercy, it should be
squashed into the in-flight GIC patches, eventually.
The rationale for this patch is that while it seems like a good idea to
copy the header files with the GIC definitions from Linux, we should
avoid the danger of copying bugs on the way. This is especially true
for those definitions that are used by the very emulation code that we
are about to test. Beside this the headers contain much more definitions
than we actually need.
So strip those header files, by first removing all definitions that are
actually only useful for hypervisors. Also we remove definitions for
registers that are not needed yet. The idea is to add them one by one as
soon as we start to test functionality, so that review can be much easier
and safer.
Applies on top of Drew's v3 GIC series (as in his github branch).

Signed-off-by: Andre Przywara <andre.przyw...@arm.com>
---
 lib/arm/asm/arch_gicv3.h   |  62 -
 lib/arm/asm/gic-v2.h   |  22 -
 lib/arm/asm/gic-v3.h   | 227 -
 lib/arm64/asm/arch_gicv3.h |  43 -
 4 files changed, 354 deletions(-)

diff --git a/lib/arm/asm/arch_gicv3.h b/lib/arm/asm/arch_gicv3.h
index d529a7e..333bf49 100644
--- a/lib/arm/asm/arch_gicv3.h
+++ b/lib/arm/asm/arch_gicv3.h
@@ -16,7 +16,6 @@
 
 #define __stringify xstr
 
-
 #define __ACCESS_CP15(CRn, Op1, CRm, Op2)  p15, Op1, %0, CRn, CRm, Op2
 #define __ACCESS_CP15_64(Op1, CRm) p15, Op1, %Q0, %R0, CRm
 
@@ -31,67 +30,6 @@
 
 #define ICC_HSRE   __ACCESS_CP15(c12, 4, c9, 5)
 
-#define ICH_VSEIR  __ACCESS_CP15(c12, 4, c9, 4)
-#define ICH_HCR__ACCESS_CP15(c12, 4, c11, 0)
-#define ICH_VTR__ACCESS_CP15(c12, 4, c11, 1)
-#define ICH_MISR   __ACCESS_CP15(c12, 4, c11, 2)
-#define ICH_EISR   __ACCESS_CP15(c12, 4, c11, 3)
-#define ICH_ELSR   __ACCESS_CP15(c12, 4, c11, 5)
-#define ICH_VMCR   __ACCESS_CP15(c12, 4, c11, 7)
-
-#define __LR0(x)   __ACCESS_CP15(c12, 4, c12, x)
-#define __LR8(x)   __ACCESS_CP15(c12, 4, c13, x)
-
-#define ICH_LR0__LR0(0)
-#define ICH_LR1__LR0(1)
-#define ICH_LR2__LR0(2)
-#define ICH_LR3__LR0(3)
-#define ICH_LR4__LR0(4)
-#define ICH_LR5__LR0(5)
-#define ICH_LR6__LR0(6)
-#define ICH_LR7__LR0(7)
-#define ICH_LR8__LR8(0)
-#define ICH_LR9__LR8(1)
-#define ICH_LR10   __LR8(2)
-#define ICH_LR11   __LR8(3)
-#define ICH_LR12   __LR8(4)
-#define ICH_LR13   __LR8(5)
-#define ICH_LR14   __LR8(6)
-#define ICH_LR15   __LR8(7)
-
-/* LR top half */
-#define __LRC0(x)  __ACCESS_CP15(c12, 4, c14, x)
-#define __LRC8(x)  __ACCESS_CP15(c12, 4, c15, x)
-
-#define ICH_LRC0   __LRC0(0)
-#define ICH_LRC1   __LRC0(1)
-#define ICH_LRC2   __LRC0(2)
-#define ICH_LRC3   __LRC0(3)
-#define ICH_LRC4   __LRC0(4)
-#define ICH_LRC5   __LRC0(5)
-#define ICH_LRC6   __LRC0(6)
-#define ICH_LRC7   __LRC0(7)
-#define ICH_LRC8   __LRC8(0)
-#define ICH_LRC9   __LRC8(1)
-#define ICH_LRC10  __LRC8(2)
-#define ICH_LRC11  __LRC8(3)
-#define ICH_LRC12  __LRC8(4)
-#define ICH_LRC13  __LRC8(5)
-#define ICH_LRC14  __LRC8(6)
-#define ICH_LRC15  __LRC8(7)
-
-#define __AP0Rx(x) __ACCESS_CP15(c12, 4, c8, x)
-#define ICH_AP0R0  __AP0Rx(0)
-#define ICH_AP0R1  __AP0Rx(1)
-#define ICH_AP0R2  __AP0Rx(2)
-#define ICH_AP0R3  __AP0Rx(3)
-
-#define __AP1Rx(x) __ACCESS_CP15(c12, 4, c9, x)
-#define ICH_AP1R0  __AP1Rx(0)
-#define ICH_AP1R1  __AP1Rx(1)
-#define ICH_AP1R2  __AP1Rx(2)
-#define ICH_AP1R3  __AP1Rx(3)
-
 /* Low-level accessors */
 
 static inline void gicv3_write_eoir(u32 irq)
diff --git a/lib/arm/asm/gic-v2.h b/lib/arm/asm/gic-v2.h
index 973c2bf..019d553 100644
--- a/lib/arm/asm/gic-v2.h
+++ b/lib/arm/asm/gic-v2.h
@@ -10,44 +10,22 @@
 
 #define GIC_CP

Re: [Qemu-devel] [kvm-unit-tests PATCH v3 07/10] arm/arm64: add initial gicv3 support

[Andre Przywara]

Hi Drew,

On 15/07/16 14:00, Andrew Jones wrote:
> Signed-off-by: Andrew Jones 
> 
> ---
> v2: configure irqs as NS GRP1
> ---
>  lib/arm/asm/arch_gicv3.h   | 184 ++
>  lib/arm/asm/gic-v3.h   | 321 
> +
>  lib/arm/asm/gic.h  |   1 +
>  lib/arm/gic.c  |  73 +++
>  lib/arm64/asm/arch_gicv3.h | 169 
>  lib/arm64/asm/gic-v3.h |   1 +
>  lib/arm64/asm/sysreg.h |  44 +++
>  7 files changed, 793 insertions(+)
>  create mode 100644 lib/arm/asm/arch_gicv3.h
>  create mode 100644 lib/arm/asm/gic-v3.h
>  create mode 100644 lib/arm64/asm/arch_gicv3.h
>  create mode 100644 lib/arm64/asm/gic-v3.h
>  create mode 100644 lib/arm64/asm/sysreg.h
> 
> diff --git a/lib/arm/asm/arch_gicv3.h b/lib/arm/asm/arch_gicv3.h
> new file mode 100644
> index 0..d529a7eb62807
> --- /dev/null
> +++ b/lib/arm/asm/arch_gicv3.h
> @@ -0,0 +1,184 @@
> +/*
> + * All ripped off from arch/arm/include/asm/arch_gicv3.h

So I was wondering if - from a test suite's perspective - it's really
clever to pull in copies of Linux headers here.
First it's really a lot of text we pull in while not using most of it
(at least now). Also they keep changing (4.9-rc1 saw so me changes, for
instance). So do we update them?

But more importantly those headers are also used in the emulation code,
so we would just copy any bugs or typos and would probably not detect
them here. IIRC there was a fix for a bitmask lately.
It's probably fine to copy the register offsets, but anything that
defines Linux specific things like default priorities or more complex
macros should be avoided, I think. This just makes kvm-unit-test copying
Linux behaviour.

Maybe we stick to the Linux naming, but pull in only the fields as we
need them? This would both limit the amount of lines being merged, as
would simplify the review effort (and quality), as people would just
need to look at a very limited number of defines, allowing them to
actually check it against the specification?

I gave this a try and could reduce the header files significantly.
Please let me know if you need any bits from this effort to be shared.

> + *
> + * Copyright (C) 2016, Red Hat Inc, Andrew Jones 
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2.
> + */
> +#ifndef _ASMARM_ARCH_GICV3_H_
> +#define _ASMARM_ARCH_GICV3_H_
> +
> +#ifndef __ASSEMBLY__
> +
> +#include 
> +#include 
> +#include 
> +
> +#define __stringify xstr
> +
> +
> +#define __ACCESS_CP15(CRn, Op1, CRm, Op2)p15, Op1, %0, CRn, CRm, Op2
> +#define __ACCESS_CP15_64(Op1, CRm)   p15, Op1, %Q0, %R0, CRm
> +
> +#define ICC_EOIR1__ACCESS_CP15(c12, 0, c12, 1)
> +#define ICC_DIR  __ACCESS_CP15(c12, 0, c11, 1)
> +#define ICC_IAR1 __ACCESS_CP15(c12, 0, c12, 0)
> +#define ICC_SGI1R__ACCESS_CP15_64(0, c12)
> +#define ICC_PMR  __ACCESS_CP15(c4, 0, c6, 0)
> +#define ICC_CTLR __ACCESS_CP15(c12, 0, c12, 4)
> +#define ICC_SRE  __ACCESS_CP15(c12, 0, c12, 5)
> +#define ICC_IGRPEN1  __ACCESS_CP15(c12, 0, c12, 7)
> +
> +#define ICC_HSRE __ACCESS_CP15(c12, 4, c9, 5)
> +
> +#define ICH_VSEIR__ACCESS_CP15(c12, 4, c9, 4)
> +#define ICH_HCR  __ACCESS_CP15(c12, 4, c11, 0)
> +#define ICH_VTR  __ACCESS_CP15(c12, 4, c11, 1)
> +#define ICH_MISR __ACCESS_CP15(c12, 4, c11, 2)
> +#define ICH_EISR __ACCESS_CP15(c12, 4, c11, 3)
> +#define ICH_ELSR __ACCESS_CP15(c12, 4, c11, 5)
> +#define ICH_VMCR __ACCESS_CP15(c12, 4, c11, 7)
> +
> +#define __LR0(x) __ACCESS_CP15(c12, 4, c12, x)
> +#define __LR8(x) __ACCESS_CP15(c12, 4, c13, x)

So for instance we clearly don't need those defines (the list registers
being hypervisor only).

> +
> +#define ICH_LR0  __LR0(0)
> +#define ICH_LR1  __LR0(1)
> +#define ICH_LR2  __LR0(2)
> +#define ICH_LR3  __LR0(3)
> +#define ICH_LR4  __LR0(4)
> +#define ICH_LR5  __LR0(5)
> +#define ICH_LR6  __LR0(6)
> +#define ICH_LR7  __LR0(7)
> +#define ICH_LR8  __LR8(0)
> +#define ICH_LR9  __LR8(1)
> +#define ICH_LR10 __LR8(2)
> +#define ICH_LR11 __LR8(3)
> +#define ICH_LR12 __LR8(4)
> +#define ICH_LR13 __LR8(5)
> +#define ICH_LR14 __LR8(6)
> +#define ICH_LR15 __LR8(7)

Re: [Qemu-devel] [PATCH arm-devs v2 8/8] arm/highbank.c: Fix MPCore periphbase name

[Andre Przywara]

On 11/28/2013 08:41 PM, Peter Maydell wrote:

(CCing Rob)


On 28 November 2013 03:31, Peter Crosthwaite
peter.crosthwa...@xilinx.com wrote:

GIC_BASE_ADDR is not the base address of the GIC. Its clear from the
code that this is the base address of the MPCore. Rename to
MPCORE_PERIPHBASE accordingly.


MPCore is one of those terms I dislike because it
doesn't actually match up with the documentation's
terminology...


Signed-off-by: Peter Crosthwaite peter.crosthwa...@xilinx.com
---

  hw/arm/highbank.c | 15 ---
  1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/hw/arm/highbank.c b/hw/arm/highbank.c
index cb32325..be2cc20 100644
--- a/hw/arm/highbank.c
+++ b/hw/arm/highbank.c
@@ -28,11 +28,11 @@
  #include exec/address-spaces.h
  #include qemu/error-report.h

-#define SMP_BOOT_ADDR 0x100
-#define SMP_BOOT_REG  0x40
-#define GIC_BASE_ADDR 0xfff1
+#define SMP_BOOT_ADDR   0x100
+#define SMP_BOOT_REG0x40
+#define MPCORE_PERIPHBASE   0xfff1

-#define NIRQ_GIC  160
+#define NIRQ_GIC160

  /* Board init.  */

@@ -55,7 +55,7 @@ static void hb_write_secondary(ARMCPU *cpu, const struct 
arm_boot_info *info)
  0xe1110001, /* tst r1, r1 */
  0x0afb, /* beq wfi */
  0xe12fff11, /* bx  r1 */
-GIC_BASE_ADDR  /* privbase: gic address.  */
+MPCORE_PERIPHBASE   /* privbase: gic address.  */


Comment no longer matches code.

More seriously, this secondary-boot code has a bunch
of hardcoded offsets for GIC registers which are
correct for the A9 but wrong for A15. Andre, did you
test Midway with an SMP boot config?


Obviously not ;-)
In the moment I cannot care about this, but it looks like we need 
different SMP stubs for Highbank and Midway.

Thanks for the hint,

Andre.



The generic hw/arm/boot code dealt with this problem
by feeding the boot code the GIC CPU interface base
address rather than the generic peripheral base address;
the offsets within the CPU i/f are the same for both
cores.


  };
  for (n = 0; n  ARRAY_SIZE(smpboot); n++) {
  smpboot[n] = tswap32(smpboot[n]);
@@ -236,7 +236,8 @@ static void calxeda_init(QEMUMachineInitArgs *args, enum 
cxmachines machine)

  cpu = ARM_CPU(object_new(object_class_get_name(oc)));

-object_property_set_int(OBJECT(cpu), GIC_BASE_ADDR, reset-cbar, 
err);
+object_property_set_int(OBJECT(cpu), MPCORE_PERIPHBASE, reset-cbar,
+err);
  if (err) {
  error_report(%s, error_get_pretty(err));
  exit(1);
@@ -287,7 +288,7 @@ static void calxeda_init(QEMUMachineInitArgs *args, enum 
cxmachines machine)
  qdev_prop_set_uint32(dev, num-irq, NIRQ_GIC);
  qdev_init_nofail(dev);
  busdev = SYS_BUS_DEVICE(dev);
-sysbus_mmio_map(busdev, 0, GIC_BASE_ADDR);
+sysbus_mmio_map(busdev, 0, MPCORE_PERIPHBASE);
  for (n = 0; n  smp_cpus; n++) {
  sysbus_connect_irq(busdev, n, cpu_irq[n]);
  }
--
1.8.4.4



thanks
-- PMM

[Qemu-devel] [Bug 1243287] Re: [KVM/QEMU][ARM][SAUCY] fails to boot cloud-image due to host kvm fail

[Andre Przywara]

Peter, the config option is called: CONFIG_STRICT_DEVMEM
And /dev/mem behaves differently between doing read() and doing mmap().

As Peter already hinted, the memory layout is different on native Midway (which 
has DRAM starting at 0) and mach-virt/vexpress (which start at 128MB / 2GB 
respectively). So accessing 0xf reads memory on native, but faults in stage 
2 mapping in the host kernel.
But the actual problem is that the host kernel passes this abort on to 
userspace, where QEMU just does abort(), thus crashing the whole virtual 
machine. Actually I think we should change the host kernel to inject the data 
abort into the guest. I tested this with a simple patch, it now makes dmidecode 
crash (due to a signal), but keeps the guest alive.

This approach needs some discussion on the ML, I guess.

But near term we should not call dmidecode on ARM, since AFAIK it does
not give any information. There once was a discussion on patching
dmidecode to output hardcoded or elsewhere gathered information to make
scripts happy, but I don't know where this ended.

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1243287

Title:
  [KVM/QEMU][ARM][SAUCY] fails to boot cloud-image due to host kvm fail

Status in QEMU:
  Confirmed

Bug description:
  On booting the cloud image using qemu/kvm fails with the following
  error:

  Cloud-init v. 0.7.3 running 'init' at Thu, 03 Oct 2013 16:45:21 +. Up 
360.78 seconds.
  ci-info: +Net device info+
  ci-info: ++--+---+---+---+
  ci-info: | Device | Up | Address | Mask | Hw-Address |
  ci-info: ++--+---+---+---+
  ci-info: | lo | True | 127.0.0.1 | 255.0.0.0 | . |
  ci-info: | eth0 | True | 10.0.2.15 | 255.255.255.0 | 52:54:00:12:34:56 |
  ci-info: ++--+---+---+---+
  ci-info: ++Route 
info++
  ci-info: 
+---+-+--+---+---+---+
  ci-info: | Route | Destination | Gateway | Genmask | Interface | Flags |
  ci-info: 
+---+-+--+---+---+---+
  ci-info: | 0 | 0.0.0.0 | 10.0.2.2 | 0.0.0.0 | eth0 | UG |
  ci-info: | 1 | 10.0.2.0 | 0.0.0.0 | 255.255.255.0 | eth0 | U |
  ci-info: 
+---+-+--+---+---+---+
  error: kvm run failed Function not implemented

  /usr/lib/python2.7/dist-
  packages/cloudinit/sources/DataSourceAltCloud.py assumes that
  dmidecode command is availabe (ie it assumes that system is x86) on
  arm systems there is no dmidecode command so host kvm fails with the
  message error: kvm run failed Function not implemented

  The patch makes get_cloud_type() function return with UNKNOWN for ARM
  systems.

  I was able to boot the cloud-image on ARM after applying this change.

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1243287/+subscriptions

[Qemu-devel] [PATCH v2 0/2] add initial Calxeda Midway A15 support

[Andre Przywara]

While the Calxeda Midway part is actually a bit more than a Highbank
with A15s, for QEMU's purposes this view is sufficient.
So refactor the Highbank initialization to allow a very similar
machine and add the necessary changes to support the new machine
called midway.

This is based on work by Rob Herring rob.herr...@calxeda.com.

Signed-off-by: Andre Przywara andre.przyw...@calxeda.com

Andre Przywara (2):
  ARM/highbank: prepare for adding similar machines
  ARM/highbank: add support for Calxeda ECX-2000 / Midway

 hw/arm/highbank.c | 61 +--
 1 file changed, 50 insertions(+), 11 deletions(-)

-- 
1.7.12.1

[Qemu-devel] [PATCH v2 1/2] ARM/highbank: prepare for adding similar machines

[Andre Przywara]

To allow the modelling of machines similar to Calxeda Highbank,
introduce a parameter to the init function and call it from a
wrapper. This allows to tweak the definition for individual machines
later on.

Signed-off-by: Andre Przywara andre.przyw...@calxeda.com
---
 hw/arm/highbank.c | 29 +++--
 1 file changed, 23 insertions(+), 6 deletions(-)

diff --git a/hw/arm/highbank.c b/hw/arm/highbank.c
index 4405dbd..2c32a2b 100644
--- a/hw/arm/highbank.c
+++ b/hw/arm/highbank.c
@@ -183,20 +183,24 @@ type_init(highbank_regs_register_types)
 
 static struct arm_boot_info highbank_binfo;
 
+enum cxmachines {
+CALXEDA_HIGHBANK,
+};
+
 /* ram_size must be set to match the upper bound of memory in the
  * device tree (linux/arch/arm/boot/dts/highbank.dts), which is
  * normally 0xff90 or -m 4089. When running this board on a
  * 32-bit host, set the reg value of memory to 0xf7ff0 in the
  * device tree and pass -m 2047 to QEMU.
  */
-static void highbank_init(QEMUMachineInitArgs *args)
+static void calxeda_init(QEMUMachineInitArgs *args, enum cxmachines machine)
 {
 ram_addr_t ram_size = args-ram_size;
 const char *cpu_model = args-cpu_model;
 const char *kernel_filename = args-kernel_filename;
 const char *kernel_cmdline = args-kernel_cmdline;
 const char *initrd_filename = args-initrd_filename;
-DeviceState *dev;
+DeviceState *dev = NULL;
 SysBusDevice *busdev;
 qemu_irq *irqp;
 qemu_irq pic[128];
@@ -208,7 +212,11 @@ static void highbank_init(QEMUMachineInitArgs *args)
 char *sysboot_filename;
 
 if (!cpu_model) {
-cpu_model = cortex-a9;
+switch (machine) {
+case CALXEDA_HIGHBANK:
+cpu_model = cortex-a9;
+break;
+}
 }
 
 for (n = 0; n  smp_cpus; n++) {
@@ -246,7 +254,11 @@ static void highbank_init(QEMUMachineInitArgs *args)
 }
 }
 
-dev = qdev_create(NULL, a9mpcore_priv);
+switch (machine) {
+case CALXEDA_HIGHBANK:
+dev = qdev_create(NULL, a9mpcore_priv);
+break;
+}
 qdev_prop_set_uint32(dev, num-cpu, smp_cpus);
 qdev_prop_set_uint32(dev, num-irq, NIRQ_GIC);
 qdev_init_nofail(dev);
@@ -324,6 +336,11 @@ static void highbank_init(QEMUMachineInitArgs *args)
 arm_load_kernel(arm_env_get_cpu(first_cpu), highbank_binfo);
 }
 
+static void highbank_init(QEMUMachineInitArgs *args)
+{
+calxeda_init(args, CALXEDA_HIGHBANK);
+}
+
 static QEMUMachine highbank_machine = {
 .name = highbank,
 .desc = Calxeda Highbank (ECX-1000),
@@ -333,9 +350,9 @@ static QEMUMachine highbank_machine = {
 DEFAULT_MACHINE_OPTIONS,
 };
 
-static void highbank_machine_init(void)
+static void calxeda_machines_init(void)
 {
 qemu_register_machine(highbank_machine);
 }
 
-machine_init(highbank_machine_init);
+machine_init(calxeda_machines_init);
-- 
1.7.12.1

[Qemu-devel] [PATCH v2 2/2] ARM/highbank: add support for Calxeda ECX-2000 / Midway

[Andre Przywara]

The Calxeda ECX-2000 chip (aka. Midway) is model-wise quite similar
to the Highbank. The most prominent difference is the Cortex-A15 CPU
core in it, together with the associated core peripherals.

Add a new ARM machine type called midway.
Move the L2 cache controller device into the Highbank specific part,
since Midway does not have (and need) it.

Signed-off-by: Andre Przywara andre.przyw...@calxeda.com
---
 hw/arm/highbank.c | 32 +++-
 1 file changed, 27 insertions(+), 5 deletions(-)

diff --git a/hw/arm/highbank.c b/hw/arm/highbank.c
index 2c32a2b..3c99a81 100644
--- a/hw/arm/highbank.c
+++ b/hw/arm/highbank.c
@@ -185,6 +185,7 @@ static struct arm_boot_info highbank_binfo;
 
 enum cxmachines {
 CALXEDA_HIGHBANK,
+CALXEDA_MIDWAY,
 };
 
 /* ram_size must be set to match the upper bound of memory in the
@@ -216,6 +217,9 @@ static void calxeda_init(QEMUMachineInitArgs *args, enum 
cxmachines machine)
 case CALXEDA_HIGHBANK:
 cpu_model = cortex-a9;
 break;
+case CALXEDA_MIDWAY:
+cpu_model = cortex-a15;
+break;
 }
 }
 
@@ -256,8 +260,16 @@ static void calxeda_init(QEMUMachineInitArgs *args, enum 
cxmachines machine)
 
 switch (machine) {
 case CALXEDA_HIGHBANK:
+dev = qdev_create(NULL, l2x0);
+qdev_init_nofail(dev);
+busdev = SYS_BUS_DEVICE(dev);
+sysbus_mmio_map(busdev, 0, 0xfff12000);
+
 dev = qdev_create(NULL, a9mpcore_priv);
 break;
+case CALXEDA_MIDWAY:
+dev = qdev_create(NULL, a15mpcore_priv);
+break;
 }
 qdev_prop_set_uint32(dev, num-cpu, smp_cpus);
 qdev_prop_set_uint32(dev, num-irq, NIRQ_GIC);
@@ -272,11 +284,6 @@ static void calxeda_init(QEMUMachineInitArgs *args, enum 
cxmachines machine)
 pic[n] = qdev_get_gpio_in(dev, n);
 }
 
-dev = qdev_create(NULL, l2x0);
-qdev_init_nofail(dev);
-busdev = SYS_BUS_DEVICE(dev);
-sysbus_mmio_map(busdev, 0, 0xfff12000);
-
 dev = qdev_create(NULL, sp804);
 qdev_prop_set_uint32(dev, freq0, 15000);
 qdev_prop_set_uint32(dev, freq1, 15000);
@@ -341,6 +348,11 @@ static void highbank_init(QEMUMachineInitArgs *args)
 calxeda_init(args, CALXEDA_HIGHBANK);
 }
 
+static void midway_init(QEMUMachineInitArgs *args)
+{
+calxeda_init(args, CALXEDA_MIDWAY);
+}
+
 static QEMUMachine highbank_machine = {
 .name = highbank,
 .desc = Calxeda Highbank (ECX-1000),
@@ -350,9 +362,19 @@ static QEMUMachine highbank_machine = {
 DEFAULT_MACHINE_OPTIONS,
 };
 
+static QEMUMachine midway_machine = {
+.name = midway,
+.desc = Calxeda Midway (ECX-2000),
+.init = midway_init,
+.block_default_type = IF_SCSI,
+.max_cpus = 4,
+DEFAULT_MACHINE_OPTIONS,
+};
+
 static void calxeda_machines_init(void)
 {
 qemu_register_machine(highbank_machine);
+qemu_register_machine(midway_machine);
 }
 
 machine_init(calxeda_machines_init);
-- 
1.7.12.1

[Qemu-devel] [PATCH] highbank: add initial Calxeda Midway A15 support

[Andre Przywara]

From: Rob Herring rob.herr...@calxeda.com

While the Calxeda Midway part is actually a bit more than a Highbank
with A15s, for QEMU's purposes this view is sufficient. So to allow
both emulation with that chip as well as KVM guests using that model
add an A15 CPU and it's peripherals as an option. The use of:
-M highbank -cpu cortex-a15 simply gives the new chip without the
need for a new model.

Signed-off-by: Rob Herring rob.herr...@calxeda.com
Signed-off-by: Andre Przywara andre.przyw...@calxeda.com
---
 hw/arm/highbank.c | 19 +--
 1 file changed, 13 insertions(+), 6 deletions(-)

diff --git a/hw/arm/highbank.c b/hw/arm/highbank.c
index 4405dbd..ed864c6 100644
--- a/hw/arm/highbank.c
+++ b/hw/arm/highbank.c
@@ -196,6 +196,7 @@ static void highbank_init(QEMUMachineInitArgs *args)
 const char *kernel_filename = args-kernel_filename;
 const char *kernel_cmdline = args-kernel_cmdline;
 const char *initrd_filename = args-initrd_filename;
+CPUARMState *env = NULL;
 DeviceState *dev;
 SysBusDevice *busdev;
 qemu_irq *irqp;
@@ -223,6 +224,8 @@ static void highbank_init(QEMUMachineInitArgs *args)
 cpu-reset_cbar = GIC_BASE_ADDR;
 irqp = arm_pic_init_cpu(cpu);
 cpu_irq[n] = irqp[ARM_PIC_CPU_IRQ];
+
+env = cpu-env;
 }
 
 sysmem = get_system_memory();
@@ -246,7 +249,16 @@ static void highbank_init(QEMUMachineInitArgs *args)
 }
 }
 
-dev = qdev_create(NULL, a9mpcore_priv);
+if (arm_feature(env, ARM_FEATURE_LPAE)) {
+dev = qdev_create(NULL, a15mpcore_priv);
+} else {
+dev = qdev_create(NULL, l2x0);
+qdev_init_nofail(dev);
+busdev = SYS_BUS_DEVICE(dev);
+sysbus_mmio_map(busdev, 0, 0xfff12000);
+
+dev = qdev_create(NULL, a9mpcore_priv);
+}
 qdev_prop_set_uint32(dev, num-cpu, smp_cpus);
 qdev_prop_set_uint32(dev, num-irq, NIRQ_GIC);
 qdev_init_nofail(dev);
@@ -260,11 +272,6 @@ static void highbank_init(QEMUMachineInitArgs *args)
 pic[n] = qdev_get_gpio_in(dev, n);
 }
 
-dev = qdev_create(NULL, l2x0);
-qdev_init_nofail(dev);
-busdev = SYS_BUS_DEVICE(dev);
-sysbus_mmio_map(busdev, 0, 0xfff12000);
-
 dev = qdev_create(NULL, sp804);
 qdev_prop_set_uint32(dev, freq0, 15000);
 qdev_prop_set_uint32(dev, freq1, 15000);
-- 
1.7.12.1

[Qemu-devel] [PATCH] fdt: update embedded header file from upstream to fix compilation

[Andre Przywara]

Upstream dtc.git introduced a change in libfdt_env.h, which breaks
compilation with QEMU's version of it:

  CC arm-softmmu/device_tree.o
In file included from /usr/include/libfdt.h:55:0,
 from /src/qemu.git/device_tree.c:28:
/usr/include/fdt.h:7:2: error: unknown type name 'fdt32_t'
   ...

The culprit is:
commit feafcd972cb744750a65728440c99526e6199a6d
Author: Kim Phillips kim.phill...@freescale.com
Date:   Wed Nov 28 17:33:01 2012 -0600

dtc/libfdt: introduce fdt types for annotation by endian checkers
...

Pull the new definitions into QEMU's version of the file. This change
also works with older installed versions of dtc.
The upstream version got a GPL or BSD dual license header meanwhile.
I retained the original GPL license header from QEMU, only added
the original copyrights.

Signed-off-by: Andre Przywara andre.przyw...@linaro.org
---
 include/libfdt_env.h | 25 -
 1 file changed, 20 insertions(+), 5 deletions(-)

diff --git a/include/libfdt_env.h b/include/libfdt_env.h
index 3667d4c..aad54bb 100644
--- a/include/libfdt_env.h
+++ b/include/libfdt_env.h
@@ -1,4 +1,12 @@
+#ifndef _LIBFDT_ENV_H
+#define _LIBFDT_ENV_H
 /*
+ * libfdt - Flat Device Tree manipulation
+ * Copyright (C) 2006 David Gibson, IBM Corporation.
+ * Copyright 2012 Kim Phillips, Freescale Semiconductor.
+ * Adaptation to QEMU: Copyright IBM Corp. 2008
+ * by Hollis Blanchard holl...@us.ibm.com
+ *
  * This program is free software; you can redistribute it and/or modify
  * it under the terms of the GNU General Public License, version 2, as
  * published by the Free Software Foundation.
@@ -11,16 +19,23 @@
  * You should have received a copy of the GNU General Public License
  * along with this program; if not, see http://www.gnu.org/licenses/.
  *
- * Copyright IBM Corp. 2008
- * Authors: Hollis Blanchard holl...@us.ibm.com
  *
  */
 
-#ifndef _LIBFDT_ENV_H
-#define _LIBFDT_ENV_H
-
 #include qemu/bswap.h
 
+#ifdef __CHECKER__
+#define __force __attribute__((force))
+#define __bitwise __attribute__((bitwise))
+#else
+#define __force
+#define __bitwise
+#endif
+
+typedef uint16_t __bitwise fdt16_t;
+typedef uint32_t __bitwise fdt32_t;
+typedef uint64_t __bitwise fdt64_t;
+
 #ifdef HOST_WORDS_BIGENDIAN
 #define fdt32_to_cpu(x)  (x)
 #define cpu_to_fdt32(x)  (x)
-- 
1.7.12.1

Re: [Qemu-devel] [PATCH] fdt: update embedded header file from upstream to fix compilation

[Andre Przywara]

On 05/07/2013 02:44 PM, Peter Maydell wrote:

On 7 May 2013 13:36, Andre Przywara andre.przyw...@linaro.org wrote:

Upstream dtc.git introduced a change in libfdt_env.h, which breaks
compilation with QEMU's version of it:

   CC arm-softmmu/device_tree.o
In file included from /usr/include/libfdt.h:55:0,
  from /src/qemu.git/device_tree.c:28:
/usr/include/fdt.h:7:2: error: unknown type name 'fdt32_t'
...


I'm not entirely sure I understand why we need change.
Have upstream really introduced a breaking change for
everybody who uses libfdt, or are we using it wrongly?


Everybody who copies and changes a header from the original distribution 
and uses this now instead of the installed version ;-)


libfdt (or better: dtc) seems to be quite out of date in the official 
distribution packages. I just happened to have git HEAD installed manually.


Regards,
Andre.

Re: [Qemu-devel] [PATCH] fdt: update embedded header file from upstream to fix compilation

[Andre Przywara]

On 05/07/2013 03:24 PM, Peter Maydell wrote:

On 7 May 2013 13:52, Andre Przywara andre.przyw...@linaro.org wrote:

On 05/07/2013 02:44 PM, Peter Maydell wrote:

I'm not entirely sure I understand why we need change.
Have upstream really introduced a breaking change for
everybody who uses libfdt, or are we using it wrongly?



Everybody who copies and changes a header from the original distribution and
uses this now instead of the installed version ;-)


Right, but the distro libfdt-dev package doesn't ship with
libfdt_env.h, so presumably the expectation is that users
of libfdt copy and adjust it, and so the set of things it
has to provide is part of the public interface?


Seems to be true for the CentOS package for instance, but not for Wheezy:
http://packages.debian.org/wheezy/amd64/libfdt-dev/filelist

Also make install on the source distribution copies the file to 
/usr/include. Even if I delete it, fdt.h still references fdt32_t, so I 
get the same error again.


Regards,
Andre.

[Qemu-devel] [PATCH] vnc-tls: Fix compilation with newer versions of GNU-TLS

[Andre Przywara]

In my installation of GNU-TLS (v3.0.23) the type
gnutls_anon_server_credentials is marked deprecated, so -Werror
breaks compilation.
Simply replacing it with the newer ..._t version fixed the compilation
on my machine (Slackware 14.0). I cannot tell how far back this new
type goes, at least the header file in RHEL 5.0 (v1.4.1) seems to have
it already. If someone finds a broken distribution, tell me and I
insert some compat code.

Signed-off-by: Andre Przywara andre.przyw...@amd.com
---
 ui/vnc-tls.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/ui/vnc-tls.c b/ui/vnc-tls.c
index a7f7d07..ba3827b 100644
--- a/ui/vnc-tls.c
+++ b/ui/vnc-tls.c
@@ -99,9 +99,9 @@ static ssize_t vnc_tls_pull(gnutls_transport_ptr_t transport,
 }
 
 
-static gnutls_anon_server_credentials vnc_tls_initialize_anon_cred(void)
+static gnutls_anon_server_credentials_t vnc_tls_initialize_anon_cred(void)
 {
-gnutls_anon_server_credentials anon_cred;
+gnutls_anon_server_credentials_t anon_cred;
 int ret;
 
 if ((ret = gnutls_anon_allocate_server_credentials(anon_cred))  0) {
@@ -382,7 +382,7 @@ int vnc_tls_client_setup(struct VncState *vs,
 }
 
 } else {
-gnutls_anon_server_credentials anon_cred = 
vnc_tls_initialize_anon_cred();
+gnutls_anon_server_credentials_t anon_cred = 
vnc_tls_initialize_anon_cred();
 if (!anon_cred) {
 gnutls_deinit(vs-tls.session);
 vs-tls.session = NULL;
-- 
1.7.12.1

Re: [Qemu-devel] [PATCH] i386/cpu: name new CPUID bits

[Andre Przywara]

On 10/18/12 18:33, Eduardo Habkost wrote:

On Wed, Oct 17, 2012 at 11:17:26PM +0200, Andre Przywara wrote:

Update QEMU's knowledge of CPUID bit names. This allows to
enable/disable those new features on QEMU's command line when
using KVM and prepares future feature enablement in QEMU.

...

@@ -79,10 +79,10 @@ static const char *ext3_feature_name[] = {
  lahf_lm /* AMD LahfSahf */, cmp_legacy, svm, extapic /* AMD 
ExtApicSpace */,
  cr8legacy /* AMD AltMovCr8 */, abm, sse4a, misalignsse,
  3dnowprefetch, osvw, ibs, xop,
-skinit, wdt, NULL, NULL,
-fma4, NULL, cvt16, nodeid_msr,
-NULL, NULL, NULL, NULL,
-NULL, NULL, NULL, NULL,
+skinit, wdt, NULL, lwp,
+fma4, tce, NULL, nodeid_msr,


You removed cvt16, here.


On purpose, cvt16 is dead. It seems to have been advertised in the 
documentation for some time, but later disappeared. The respective 
instructions are now reported via the F16C flag in the Intel leaf.


If you want to know more, I can dig deeper and ask some people. But the 
bit is now reserved and was never '1' in any silicon. (And was 
introduced by me into QEMU :-(




All the rest of the flags look OK to me.


Thanks.

Regards,
Andre.

[Qemu-devel] [PATCH] i386/cpu: name new CPUID bits

[Andre Przywara]

Update QEMU's knowledge of CPUID bit names. This allows to
enable/disable those new features on QEMU's command line when
using KVM and prepares future feature enablement in QEMU.

This adds F16C, RDRAND, LWP, TBM, TopoExt, PerfCtr_Core, PerfCtr_NB,
FSGSBASE, BMI1, AVX2, BMI2, ERMS, InvPCID, RTM, RDSeed and ADX.

Sources where the AMD BKDG for Family 15h/Model 10h and the Linux kernel
for the leaf 7 bits.

Signed-off-by: Andre Przywara andre.przyw...@amd.com
---
 target-i386/cpu.c | 16 
 target-i386/cpu.h | 21 +
 2 files changed, 29 insertions(+), 8 deletions(-)

diff --git a/target-i386/cpu.c b/target-i386/cpu.c
index f3708e6..49f9561 100644
--- a/target-i386/cpu.c
+++ b/target-i386/cpu.c
@@ -59,7 +59,7 @@ static const char *ext_feature_name[] = {
 NULL, pcid, dca, sse4.1|sse4_1,
 sse4.2|sse4_2, x2apic, movbe, popcnt,
 tsc-deadline, aes, xsave, osxsave,
-avx, NULL, NULL, hypervisor,
+avx, f16c, rdrand, hypervisor,
 };
 /* Feature names that are already defined on feature_name[] but are set on
  * CPUID[8000_0001].EDX on AMD CPUs don't have their names on
@@ -79,10 +79,10 @@ static const char *ext3_feature_name[] = {
 lahf_lm /* AMD LahfSahf */, cmp_legacy, svm, extapic /* AMD 
ExtApicSpace */,
 cr8legacy /* AMD AltMovCr8 */, abm, sse4a, misalignsse,
 3dnowprefetch, osvw, ibs, xop,
-skinit, wdt, NULL, NULL,
-fma4, NULL, cvt16, nodeid_msr,
-NULL, NULL, NULL, NULL,
-NULL, NULL, NULL, NULL,
+skinit, wdt, NULL, lwp,
+fma4, tce, NULL, nodeid_msr,
+NULL, tbm, topoext, perfctr_core,
+perfctr_nb, NULL, NULL, NULL,
 NULL, NULL, NULL, NULL,
 };
 
@@ -105,9 +105,9 @@ static const char *svm_feature_name[] = {
 };
 
 static const char *cpuid_7_0_ebx_feature_name[] = {
-NULL, NULL, NULL, NULL, NULL, NULL, NULL, smep,
-NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL,
-NULL, NULL, NULL, NULL, smap, NULL, NULL, NULL,
+fsgsbase, NULL, NULL, bmi1, hle, avx2, NULL, smep,
+bmi2, erms, invpcid, rtm, NULL, NULL, NULL, NULL,
+NULL, NULL, rdseed, adx, smap, NULL, NULL, NULL,
 NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL,
 };
 
diff --git a/target-i386/cpu.h b/target-i386/cpu.h
index 871c270..e496b5b 100644
--- a/target-i386/cpu.h
+++ b/target-i386/cpu.h
@@ -403,6 +403,7 @@
 #define CPUID_EXT_TM2  (1  8)
 #define CPUID_EXT_SSSE3(1  9)
 #define CPUID_EXT_CID  (1  10)
+#define CPUID_EXT_FMA  (1  12)
 #define CPUID_EXT_CX16 (1  13)
 #define CPUID_EXT_XTPR (1  14)
 #define CPUID_EXT_PDCM (1  15)
@@ -417,6 +418,8 @@
 #define CPUID_EXT_XSAVE(1  26)
 #define CPUID_EXT_OSXSAVE  (1  27)
 #define CPUID_EXT_AVX  (1  28)
+#define CPUID_EXT_F16C (1  29)
+#define CPUID_EXT_RDRAND   (1  30)
 #define CPUID_EXT_HYPERVISOR  (1  31)
 
 #define CPUID_EXT2_FPU (1  0)
@@ -472,7 +475,15 @@
 #define CPUID_EXT3_IBS (1  10)
 #define CPUID_EXT3_XOP (1  11)
 #define CPUID_EXT3_SKINIT  (1  12)
+#define CPUID_EXT3_WDT (1  13)
+#define CPUID_EXT3_LWP (1  15)
 #define CPUID_EXT3_FMA4(1  16)
+#define CPUID_EXT3_TCE (1  17)
+#define CPUID_EXT3_NODEID  (1  19)
+#define CPUID_EXT3_TBM (1  21)
+#define CPUID_EXT3_TOPOEXT (1  22)
+#define CPUID_EXT3_PERFCORE (1  23)
+#define CPUID_EXT3_PERFNB  (1  24)
 
 #define CPUID_SVM_NPT  (1  0)
 #define CPUID_SVM_LBRV (1  1)
@@ -485,7 +496,17 @@
 #define CPUID_SVM_PAUSEFILTER  (1  10)
 #define CPUID_SVM_PFTHRESHOLD  (1  12)
 
+#define CPUID_7_0_EBX_FSGSBASE (1  0)
+#define CPUID_7_0_EBX_BMI1 (1  3)
+#define CPUID_7_0_EBX_HLE  (1  4)
+#define CPUID_7_0_EBX_AVX2 (1  5)
 #define CPUID_7_0_EBX_SMEP (1  7)
+#define CPUID_7_0_EBX_BMI2 (1  8)
+#define CPUID_7_0_EBX_ERMS (1  9)
+#define CPUID_7_0_EBX_INVPCID  (1  10)
+#define CPUID_7_0_EBX_RTM  (1  11)
+#define CPUID_7_0_EBX_RDSEED   (1  18)
+#define CPUID_7_0_EBX_ADX  (1  19)
 #define CPUID_7_0_EBX_SMAP (1  20)
 
 #define CPUID_VENDOR_INTEL_1 0x756e6547 /* Genu */
-- 
1.7.12.1

Re: [Qemu-devel] qemu -numa option and non-contiguous CPU ranges

[Andre Przywara]

On 06/21/2012 07:51 PM, Eduardo Habkost wrote:

Hi,

I just noticed libvirt tries to use the -numa option in a way that qemu
never understood: if a node is configured to have a non-contiguous set
of CPUs, it tries to generate a command-line option that looks like:

-numa node,nodeid=...,cpus=0,2,4,mem=...
 ^

But this format was never supported by qemu. This format is even a bit
weird, as , is an option separator, and it is being used as a
separator _inside_ an option.


Exactly this was the reason back then to not support non-contiguous set 
of CPUs. Inside qemu there is no reason why this shouldn't work, it was 
just hard to write on the command line. So after a short discussion we 
decided to drop this for the time being. If you have a great idea how to 
specify this (I think a comma will not work, because it will be catched 
earlier), I am all ears.


Regards,
Andre.


My question is: should we support this option format in qemu, or should
we change libvirt to use another format (that has yet to be implemented,
because currently there's no way to specify a non-contiguous set of CPUs
for a NUMA node).

Any suggestions?




--
Andre Przywara
AMD-Operating System Research Center (OSRC), Dresden, Germany

Re: [Qemu-devel] Semantics of -cpu host (was Re: [PATCH 2/2] Expose tsc deadline timer cpuid to guest)

[Andre Przywara]

On 05/07/2012 08:21 PM, Eduardo Habkost wrote:


Andre? Are you able to help to answer the question below?


Sorry for the delay, the easy answers first:


I would like to clarify what's the expected behavior of -cpu host to
be able to continue working on it.


The purpose of -cpu host is to let guests use ISA features that the host 
CPU provides. Those would not need any KVM/QEMU intervention, because 
they work out of the box. Things like AES or SSE4.2, which are used by 
Linux and glibc, for instance. Users would expect those to be usable, 
and actually we only need to set the bits and are done.


A second goal is to get rid of the awkward and artificial 
family/model/stepping settings and just let the guest see the actual CPU 
model. This has some strings attached, though, but other virtualization 
solution do it the same way and they can cope with it.


A third thing currently not really addressed are the more informational 
bits like cache size and organization and TLB layout. Those are 
properties of the (core) CPU (except shared L3, cache maybe) and apply 
to guests as well. I don't see any reason why this should not be exposed 
to the guest. From the top of my head I don't know any prominent user 
(just glibc reading the cache size, but not really using it), but I 
guess there are applications which benefit from it.


What clearly is not the intention of -cpu host is to provide access to 
every feature a certain CPU model provides. So things which require 
emulation or hypervisor intervention are not in the primary use case. 
That does not mean that they don't need to implemented, but that was not 
the purpose of -cpu host and they should be handled independently.


Regards,
Andre.

I believe the code will need to be

fixed on either case, but first we need to figure out what are the
expectations/requirements, to know _which_ changes will be needed.


On Tue, Apr 24, 2012 at 02:19:25PM -0300, Eduardo Habkost wrote:

(CCing Andre Przywara, in case he can help to clarify what's the
expected meaning of -cpu host)


[...]

I am not sure I understand what you are proposing. Let me explain the
use case I am thinking about:

- Feature FOO is of type (A) (e.g. just a new instruction set that
   doesn't require additional userspace support)
- User has a Qemu vesion that doesn't know anything about feature FOO
- User gets a new CPU that supports feature FOO
- User gets a new kernel that supports feature FOO (i.e. has FOO in
   GET_SUPPORTED_CPUID)
- User does _not_ upgrade Qemu.
- User expects to get feature FOO enabled if using -cpu host, without
   upgrading Qemu.

The problem here is: to support the above use-case, userspace need a
probing mechanism that can differentiate _new_ (previously unknown)
features that are in group (A) (safe to blindly enable) from features
that are in group (B) (that can't be enabled without an userspace
upgrade).

In short, it becomes a problem if we consider the following case:

- Feature BAR is of type (B) (it can't be enabled without extra
   userspace support)
- User has a Qemu version that doesn't know anything about feature BAR
- User gets a new CPU that supports feature BAR
- User gets a new kernel that supports feature BAR (i.e. has BAR in
   GET_SUPPORTED_CPUID)
- User does _not_ upgrade Qemu.
- User simply shouldn't get feature BAR enabled, even if using -cpu
   host, otherwise Qemu would break.

If userspace always limited itself to features it knows about, it would
be really easy to implement the feature without any new probing
mechanism from the kernel. But that's not how I think users expect -cpu
host to work. Maybe I am wrong, I don't know. I am CCing Andre, who
introduced the -cpu host feature, in case he can explain what's the
expected semantics on the cases above.






--
Andre Przywara
AMD-Operating System Research Center (OSRC), Dresden, Germany

Re: [Qemu-devel] [PATCH resend] vl.c: Don't limit node count by smp count

[Andre Przywara]

On 06/30/2011 05:29 AM, Sasha Levin wrote:

[I've sent this patch couple of months ago and noticed it
  didn't make it's way in - so I'm sending it again]

It is possible to create CPU-less NUMA nodes, node amount shouldn't be
limited by amount of CPUs.

Tested-by: Michael Rothmdr...@linux.vnet.ibm.com
Signed-off-by: Sasha Levinlevinsasha...@gmail.com
---
  vl.c |4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/vl.c b/vl.c
index e0191e1..b95ae8d 100644
--- a/vl.c
+++ b/vl.c
@@ -3147,8 +3147,8 @@ int main(int argc, char **argv, char **envp)
  if (nb_numa_nodes  0) {
  int i;

-if (nb_numa_nodes  smp_cpus) {
-nb_numa_nodes = smp_cpus;
+if (nb_numa_nodes  MAX_NODES) {
+nb_numa_nodes = MAX_NODES;
  }

  /* If no memory size if given for any node, assume the default case


Acked-by: Andre Przywara andre.przyw...@amd.com

Regards,
Andre.


--
Andre Przywara
AMD-Operating System Research Center (OSRC), Dresden, Germany

[Qemu-devel] Re: CPU type qemu64 breaks guest code

[Andre Przywara]

On 03/14/2011 03:13 PM, Alexander Graf wrote:

Hi guys,

While I was off on vacation a pretty nasty bug emerged. It's our old
friend the non-existent -cpu qemu64 CPU type. To refresh your memories,
this is the definition of the default 64-bit CPU type in Qemu:

 {
 .name = qemu64,
 .level = 4,
 .vendor1 = CPUID_VENDOR_AMD_1,
 .vendor2 = CPUID_VENDOR_AMD_2,
 .vendor3 = CPUID_VENDOR_AMD_3,
 .family = 6,
 .model = 2,
 .stepping = 3,

 ...

Well, yes, this is strange, and a different CPU model mimicking some 
really existing CPU would be better. But in my experience this opens up 
a can of worms, since a different family will trigger a lot of other 
nasty code that was silent before. Although I think that eventually we 
will not get around it doing so, but we should use a lot of testing 
before triggering tons of regressions.
What about the kvm64 CPU model? Back then I used quite some testing to 
find a model which runs pretty well, so I came up with 15/6/1, which 
seemed to be relatively sane. We could try copying this F/M/S over to 
qemu64, I suppose with emulation the issues are easier to fix than in KVM.


Another idea I think I already posted some time ago was to make kvm64 
the new default if KVM is enabled. This wouldn't solve the above issue 
for TCG of course, but would make further development easier, since we 
would have a better separation between KVM and TCG and could tweak each 
independently.



Before I left, we already had weird build breakages where gcc simply

 abort()ed when running inside of KVM. This breakage has been tracked

down to libgmp, which has this code
(http://gmplib.org:8000/gmp-5.0/file/1ebe39104437/mpn/x86_64/fat/fat.c):

 



   if (strcmp (vendor_string, GenuineIntel) == 0)
 {
   
 }
   else if (strcmp (vendor_string, AuthenticAMD) == 0)
 {
   switch (family)
 {
case 5:
case 6:
abort ();
break;
case 15:
   /* CPUVEC_SETUP_athlon */
   break;
 }

The reasoning behind it is that for AMD, all 64-bit CPUs were family
15.


I guess that should be fixed there, as it is obviously nonsense. I 
haven't check what they actually need that family 6 does not provide, if 
it is long mode, then this bit should be checked.



This breakage is obviously pretty bad for guests running qemu and
shows once again that deriving from real hardware is a bad idea. I guess
the best way to move forward is to change the CPU type to something that
actually exists in the real world - even if we have to strip off some
features.


I found that there is no valid combination for both Intel and AMD. We 
had to go to family 15, and it seems that there is no F/M/S combination 
that were both a valid K8 and a Pentium4 processor. The 15/6/1 mentioned 
before was the closest match I could find.


Summarizing I would suggest:
1) Make kvm64 the new default model if KVM is used. Maybe we could tie
   this to the qemu-0.15 machine type.
2) Test as many OSes as possible whether they show strange behavior.
   In my experience we could catch most of the stuff with just booting
   the system, with Linux -kernel vmlinuz suffices (without a rootfs).
3) If we are happy with that, tweak the qemu64 model to those values,
   too.
4) Write some note somewhere to let users know what they could do to
   fix problems that rise due to the new model.

I can easily send patches and will try to contribute to testing, if that 
plan sounds OK.


--
Andre Przywara
AMD-Operating System Research Center (OSRC), Dresden, Germany

[Qemu-devel] Re: [PATCH] kvm/x86: enlarge number of possible CPUID leaves

[Andre Przywara]

Avi, Marcello,

can you please commit this simple fix? (turning 40 to 80?)
Without it QEMU crashes reliably on our new CPUs (they return 46 leaves) 
and causes pain in our testing, because we have to manually apply this 
patch on each tree.


Thanks!
Andre.


Currently the number of CPUID leaves KVM handles is limited to 40.
My desktop machine (AthlonII) already has 35 and future CPUs will
expand this well beyond the limit. Extend the limit to 80 to make
room for future processors.

Signed-off-by: Andre Przywara andre.przyw...@amd.com
---
 arch/x86/include/asm/kvm_host.h |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

Hi,
I found that either KVM or QEMU (possibly both) are broken in respect
to handling more CPUID entries than the limit dictates. KVM will
return -E2BIG, which is the same error as if the user hasn't provided
enough space to hold all entries. Now QEMU will continue to enlarge
the allocated memory until it gets into an out-of-memory condition.
I have tried to fix this with teaching KVM how to deal with a capped
number of entries (there are some bugs in the current code), but this
will limit the number of CPUID entries KVM handles, which will surely
cut of the lastly appended PV leaves.
A proper fix would be to make this allocation dynamic. Is this a
feasible way or will this lead to issues or side-effects?

Regards,
Andre.

diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 54e42c8..3cc80c4 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -79,7 +79,7 @@
 #define KVM_NUM_MMU_PAGES (1  KVM_MMU_HASH_SHIFT)
 #define KVM_MIN_FREE_MMU_PAGES 5
 #define KVM_REFILL_PAGES 25
-#define KVM_MAX_CPUID_ENTRIES 40
+#define KVM_MAX_CPUID_ENTRIES 80
 #define KVM_NR_FIXED_MTRR_REGION 88
 #define KVM_NR_VAR_MTRR 8
 

[Qemu-devel] [PATCH] kvm/x86: enlarge number of possible CPUID leaves

[Andre Przywara]

Currently the number of CPUID leaves KVM handles is limited to 40.
My desktop machine (AthlonII) already has 35 and future CPUs will
expand this well beyond the limit. Extend the limit to 80 to make
room for future processors.

Signed-off-by: Andre Przywara andre.przyw...@amd.com
---
 arch/x86/include/asm/kvm_host.h |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

Hi,
I found that either KVM or QEMU (possibly both) are broken in respect
to handling more CPUID entries than the limit dictates. KVM will
return -E2BIG, which is the same error as if the user hasn't provided
enough space to hold all entries. Now QEMU will continue to enlarge
the allocated memory until it gets into an out-of-memory condition.
I have tried to fix this with teaching KVM how to deal with a capped
number of entries (there are some bugs in the current code), but this
will limit the number of CPUID entries KVM handles, which will surely
cut of the lastly appended PV leaves.
A proper fix would be to make this allocation dynamic. Is this a
feasible way or will this lead to issues or side-effects?

Regards,
Andre.

diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 54e42c8..3cc80c4 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -79,7 +79,7 @@
 #define KVM_NUM_MMU_PAGES (1  KVM_MMU_HASH_SHIFT)
 #define KVM_MIN_FREE_MMU_PAGES 5
 #define KVM_REFILL_PAGES 25
-#define KVM_MAX_CPUID_ENTRIES 40
+#define KVM_MAX_CPUID_ENTRIES 80
 #define KVM_NR_FIXED_MTRR_REGION 88
 #define KVM_NR_VAR_MTRR 8
 
-- 
1.6.4

[Qemu-devel] Re: [PATCH] kvm/x86: enlarge number of possible CPUID leaves

[Andre Przywara]

Avi Kivity wrote:

On 12/01/2010 01:17 PM, Andre Przywara wrote:

Currently the number of CPUID leaves KVM handles is limited to 40.
My desktop machine (AthlonII) already has 35 and future CPUs will
expand this well beyond the limit. Extend the limit to 80 to make
room for future processors.

Signed-off-by: Andre Przywaraandre.przyw...@amd.com
---
  arch/x86/include/asm/kvm_host.h |2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)

Hi,
I found that either KVM or QEMU (possibly both) are broken in respect
to handling more CPUID entries than the limit dictates. KVM will
return -E2BIG, which is the same error as if the user hasn't provided
enough space to hold all entries. Now QEMU will continue to enlarge
the allocated memory until it gets into an out-of-memory condition.
I have tried to fix this with teaching KVM how to deal with a capped
number of entries (there are some bugs in the current code), but this
will limit the number of CPUID entries KVM handles, which will surely
cut of the lastly appended PV leaves.
A proper fix would be to make this allocation dynamic. Is this a
feasible way or will this lead to issues or side-effects?



Well, there has to be some limit, or userspace can allocate unbounded 
kernel memory.
But this limit should not be a compile-time constant, but a runtime one. 
The needed size depends on the host CPU (plus the KVM PV leaves) and 
thus could be determined once for all VMs and vCPUs at module load-time. 
But then we cannot use the static array allocation we currently have in 
struct kvm_vcpu_arch:

struct kvm_cpuid_entry2 cpuid_entries[KVM_MAX_CPUID_ENTRIES];
So we would use a kind-of dynamic allocation bounded by the host CPU's 
need. But for the code is does not make much difference to a real 
dynamic allocation.


Also we could implement kvm_dev_ioctl_get_supported_cpuid without the 
vmalloc, if we don't care about some dozens of copy_to_user() calls in 
this function. Then we would not need this limit in GET_SUPPORTED_CPUID 
at all, but it will strike us again at KVM_SET_CPUID[2], where we may 
not fulfill the promises we gave earlier.

Having said this, what about that:
kvm_dev_ioctl_get_supported_cpuid is invariant to the VM or vCPU (as it 
is used by a system ioctl), so it could be run once at initialization, 
which would limit the ioctl implementation to a plain bounded copy.

Would you want such a patch (removing the vmalloc and maybe even the limit)?

Regards,
Andre.

--
Andre Przywara
AMD-OSRC (Dresden)
Tel: x29712

Re: [Qemu-devel] New Bitmap module ?

[Andre Przywara]

Corentin Chary wrote:

Hi,
I was working on merging VNC updates into bigger ones to see if it
lower the overhead (big updates sometime use less network/cpu than a
lot of small updates).
For that, I needed some new bitmap functions, and no we got, in vnc.c:
- set_bit
- clear_bit
- set_bits
- clear_bits
- find_next_bit
- find_next_zero_bit
- find_next_zero_area

Should we move that into bitmap.c/bitmap.h ?

Definitely! For my NUMA work I have also coded some bitmap functions.
Since mine are not performance critical, I reverted from implementing 
set_bits and clear_bits and replaced them with a loop in the calling 
code. So I could just could come around with a macro only 
implementation, for which a header file suffices (attached for reference).



Which part of QEMU could
use that (block.c maybe ?)
As mentioned, the yet to be submitted NUMA code would benefit from it. 
Linux has also bitmap code, maybe you could leverage this (if not 
already done).


Regards,
Andre.

--
Andre Przywara
AMD-Operating System Research Center (OSRC), Dresden, Germany
Tel: +49 351 448-3567-12
#ifndef __BITMAP_H__
#define __BITMAP_H__

#ifndef HOST_LONG_BITS
#define HOST_LONG_BITS (sizeof(long) * 8)
#endif

#define bitmap_isset(bm,bit) \
(!!bm[(bit) / HOST_LONG_BITS]  (1ULL  ((bit) % HOST_LONG_BITS)))
#define bitmap_set(bm,bit) \
(bm[(bit) / HOST_LONG_BITS] |= (1ULL  ((bit) % HOST_LONG_BITS)))
#define bitmap_unset(bm,bit) \
(bm[(bit) / HOST_LONG_BITS] = ~(1ULL  ((bit) % HOST_LONG_BITS)))
#define DECLARE_BITMAP(bm,len) \
unsigned long bm[((len) + HOST_LONG_BITS - 1) / HOST_LONG_BITS]
#define bitmap_clear(bm,len) \
memset(bm, 0, (len + 7) / 8)
#define bitmap_fill(bm,len) \
memset(bm, 0xFF, (len + 7) / 8)

#endif

[Qemu-devel] Using Linux's CPUSET for KVM VCPUs

[Andre Przywara]

Hi all,

while working on NUMA host pinning, I experimented with vCPU affinity 
within QEMU, but left it alone as it would complicate the code and would 
not achieve better experience than using taskset with the monitor 
provided thread ids like it is done currently. During that I looked at 
Linux' CPUSET implementation 
(/src/linux-2.6/Documentation/cgroups/cpusets.txt).
In brief, this is a pseudo file system based, truly hierarchical 
implementation of restricting a set of processes (or threads, it uses 
PIDs) to a certain subset of the machine.
Sadly we cannot leverage this for true guest NUMA memory assignment, but 
it would work nice for pinning (or not) guest vCPUs. I had the following 
structure in mind:
For each guest there is a new CPUSET (mkdir $CPUSET_MNT/`cat 
/proc/$$/cpuset`/kvm_$guestname). One could then assign the guest global 
resources to this CPUSET.
For each vCPU there is a separate CPUSET located under this guest global 
one. This would allow for easy manipulation of the pinning of vCPUs, 
even from the console without any mgt app (although this could be easily 
implemented in libvirt).


/
|
+--/ kvm_guest_01
|  |
|  +-- VCPU0
|  |
|  +-- VCPU1
|
+--/ kvm_guest_02
...

What do you think about it? It is worth implementing this?

Regards,
Andre.


--
Andre Przywara
AMD-Operating System Research Center (OSRC), Dresden, Germany
Tel: +49 351 448-3567-12

Re: [Qemu-devel] [PATCH] resent: x86/cpuid: propagate further CPUID leafs when -cpu host

[Andre Przywara]

Anthony Liguori wrote:

On 05/25/2010 08:21 AM, Andre Przywara wrote:

What's the benefit of exposing this information to the guest?
That is mostly to propagate the cache size and organization parameters 
to the guest:

+/* safe CPUID leafs to propagate to guest if -cpu host is specified
+ * Intel defined leafs:
+ * Cache descriptors (0x02)
+ * Deterministic cache parameters (0x04)
+ * Monitor/MWAIT parameters (0x05)
+ *
+ * AMD defined leafs:
+ * L1 Cache and TLB (0x05)
+ * L2+L3 TLB (0x06)
+ * LongMode address size (0x08)
+ * 1GB page TLB (0x19)
+ * Performance optimization (0x1A)
+ */
Since at least L1 and L2 caches are mostly private to vCPUs, I see no 
reason to disguise them.


But in practice, what is it useful for?  Just because we can expose it 
doesn't mean we should.
Beside the obvious high performance libraries (like the AMD ACML) also 
JVMs (and probably other managed code runtimes) use the cache 
information for optimization. Given the fact that we have a fairly large 
range of actual L2 cache sizes (from 256KB to 6MB on Intel) the fixed 
cache size that QEMU reports (1MB) is quite a bit off most of the times.
-cpu host is targeted to give the best performance to the user, with the 
drawback of missing or very limited migration experience. So we should 
expose as many features as possible.


Regards,
Andre.

--
Andre Przywara
AMD-Operating System Research Center (OSRC), Dresden, Germany
Tel: +49 351 448-3567-12

Re: [Qemu-devel] [PATCH] resent: x86/cpuid: propagate further CPUID leafs when -cpu host

[Andre Przywara]

Anthony Liguori wrote:

On 05/21/2010 02:50 AM, Andre Przywara wrote:

-cpu host currently only propagates the CPU's family/model/stepping,
the brand name and the feature bits.
Add a whitelist of safe CPUID leafs to let the guest see the actual
CPU's cache details and other things.

Signed-off-by: Andre Przywaraandre.przyw...@amd.com
   


The problem I can see is that this greatly increases the chances of 
problems with live migration since we don't migrate the cpuid state.
I think that should be fixed. Although -cpu host is not a wise choice 
for migration, even without these additional leaves the feature bits 
probably don't match between source and target.



What's the benefit of exposing this information to the guest?


That is mostly to propagate the cache size and organization parameters 
to the guest:

 +/* safe CPUID leafs to propagate to guest if -cpu host is specified
 + * Intel defined leafs:
 + * Cache descriptors (0x02)
 + * Deterministic cache parameters (0x04)
 + * Monitor/MWAIT parameters (0x05)
 + *
 + * AMD defined leafs:
 + * L1 Cache and TLB (0x05)
 + * L2+L3 TLB (0x06)
 + * LongMode address size (0x08)
 + * 1GB page TLB (0x19)
 + * Performance optimization (0x1A)
 + */
Since at least L1 and L2 caches are mostly private to vCPUs, I see no 
reason to disguise them.


Regards,
Andre.

--
Andre Przywara
AMD-Operating System Research Center (OSRC), Dresden, Germany
Tel: +49 351 448-3567-12

[Qemu-devel] [PATCH] resent: fix CPUID vendor override

[Andre Przywara]

the meaning of vendor_override is actually the opposite of how it
is currently used :-(
Fix it to allow KVM to export the non-native CPUID vendor if
explicitly requested by the user.
The semantic is now as intended:
- With TCG, the guest always sees the configured vendor.
- With KVM, the default is to propagate the host's vendor
  - when explicitly requested via -cpu base,vendor=xxx obey this
and use the specified vendor

Signed-off-by: Andre Przywara andre.przyw...@amd.com
---
 target-i386/cpuid.c |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

Hi,

this hasn't been picked up the last time I sent it out, are there any
objections?

Regards,
Andre.

diff --git a/target-i386/cpuid.c b/target-i386/cpuid.c
index 56938e2..99d1f44 100644
--- a/target-i386/cpuid.c
+++ b/target-i386/cpuid.c
@@ -962,7 +962,7 @@ static void get_cpuid_vendor(CPUX86State *env, uint32_t 
*ebx,
  * this if you want to use KVM's sysenter/syscall emulation
  * in compatibility mode and when doing cross vendor migration
  */
-if (kvm_enabled()  env-cpuid_vendor_override) {
+if (kvm_enabled()  ! env-cpuid_vendor_override) {
 host_cpuid(0, 0, NULL, ebx, ecx, edx);
 }
 }
-- 
1.6.4

[Qemu-devel] [PATCH] resent: x86/cpuid: Add kvm32 CPU model

[Andre Przywara]

Create a kvm32 CPU model that describes a least common denominator
for KVM capable guest CPUs. Useful for migration purposes.

Signed-off-by: Andre Przywara andre.przyw...@amd.com
---
 target-i386/cpuid.c |   14 ++
 1 files changed, 14 insertions(+), 0 deletions(-)

diff --git a/target-i386/cpuid.c b/target-i386/cpuid.c
index a80baa4..76b897d 100644
--- a/target-i386/cpuid.c
+++ b/target-i386/cpuid.c
@@ -363,6 +363,20 @@ static x86_def_t builtin_x86_defs[] = {
 .model_id = QEMU Virtual CPU version  QEMU_VERSION,
 },
 {
+.name = kvm32,
+.level = 5,
+.family = 15,
+.model = 6,
+.stepping = 1,
+.features = PPRO_FEATURES |
+CPUID_MTRR | CPUID_CLFLUSH | CPUID_MCA | CPUID_PSE36,
+.ext_features = CPUID_EXT_SSE3,
+.ext2_features = PPRO_FEATURES  EXT2_FEATURE_MASK,
+.ext3_features = 0,
+.xlevel = 0x8008,
+.model_id = Common 32-bit KVM processor
+},
+{
 .name = coreduo,
 .level = 10,
 .family = 6,
-- 
1.6.4

[Qemu-devel] [PATCH] resent: x86/cpuid: propagate further CPUID leafs when -cpu host

[Andre Przywara]

-cpu host currently only propagates the CPU's family/model/stepping,
the brand name and the feature bits.
Add a whitelist of safe CPUID leafs to let the guest see the actual
CPU's cache details and other things.

Signed-off-by: Andre Przywara andre.przyw...@amd.com
---
 target-i386/cpu.h   |6 +-
 target-i386/cpuid.c |   45 +
 2 files changed, 42 insertions(+), 9 deletions(-)

Again this patch was not yet applied, without any discussion I can remember.
Please apply.

Thanks,
Andre.

diff --git a/target-i386/cpu.h b/target-i386/cpu.h
index 548ab80..77cbbdb 100644
--- a/target-i386/cpu.h
+++ b/target-i386/cpu.h
@@ -581,6 +581,10 @@ typedef struct {
 
 #define NB_MMU_MODES 2
 
+#define CPUID_FLAGS_BUILTIN  (1  0)
+#define CPUID_FLAGS_VENDOR_OVERRIDE  (1  1)
+#define CPUID_FLAGS_HOST (1  2) 
+
 typedef struct CPUX86State {
 /* standard registers */
 target_ulong regs[CPU_NB_REGS];
@@ -685,7 +689,7 @@ typedef struct CPUX86State {
 uint32_t cpuid_ext2_features;
 uint32_t cpuid_ext3_features;
 uint32_t cpuid_apic_id;
-int cpuid_vendor_override;
+uint32_t cpuid_flags;
 
 /* MTRRs */
 uint64_t mtrr_fixed[11];
diff --git a/target-i386/cpuid.c b/target-i386/cpuid.c
index 99d1f44..a80baa4 100644
--- a/target-i386/cpuid.c
+++ b/target-i386/cpuid.c
@@ -213,7 +213,6 @@ typedef struct x86_def_t {
 uint32_t features, ext_features, ext2_features, ext3_features, 
kvm_features;
 uint32_t xlevel;
 char model_id[48];
-int vendor_override;
 uint32_t flags;
 } x86_def_t;
 
@@ -489,7 +488,7 @@ static int cpu_x86_fill_host(x86_def_t *x86_cpu_def)
 x86_cpu_def-ext2_features = edx;
 x86_cpu_def-ext3_features = ecx;
 cpu_x86_fill_model_id(x86_cpu_def-model_id);
-x86_cpu_def-vendor_override = 0;
+x86_cpu_def-flags = CPUID_FLAGS_HOST;
 
 return 0;
 }
@@ -633,7 +632,7 @@ static int cpu_x86_find_by_name(x86_def_t *x86_cpu_def, 
const char *cpu_model)
 x86_cpu_def-vendor2 |= ((uint8_t)val[i + 4])  (8 * i);
 x86_cpu_def-vendor3 |= ((uint8_t)val[i + 8])  (8 * i);
 }
-x86_cpu_def-vendor_override = 1;
+x86_cpu_def-flags |= CPUID_FLAGS_VENDOR_OVERRIDE;
 } else if (!strcmp(featurestr, model_id)) {
 pstrcpy(x86_cpu_def-model_id, sizeof(x86_cpu_def-model_id),
 val);
@@ -731,7 +730,8 @@ void x86_cpu_list (FILE *f, int (*cpu_fprintf)(FILE *f, 
const char *fmt, ...),
 return;
 }
 for (def = x86_defs; def; def = def-next) {
-snprintf(buf, sizeof (buf), def-flags ? [%s]: %s, def-name);
+snprintf(buf, sizeof (buf),
+ def-flags  CPUID_FLAGS_BUILTIN ? [%s]: %s, def-name);
 if (model || dump) {
 (*cpu_fprintf)(f, x86 %16s  %-48s\n, buf, def-model_id);
 } else {
@@ -785,7 +785,7 @@ int cpu_x86_register (CPUX86State *env, const char 
*cpu_model)
 env-cpuid_vendor2 = CPUID_VENDOR_INTEL_2;
 env-cpuid_vendor3 = CPUID_VENDOR_INTEL_3;
 }
-env-cpuid_vendor_override = def-vendor_override;
+env-cpuid_flags = def-flags;
 env-cpuid_level = def-level;
 if (def-family  0x0f)
 env-cpuid_version = 0xf00 | ((def-family - 0x0f)  20);
@@ -941,7 +941,7 @@ void x86_cpudef_setup(void)
 
 for (i = 0; i  ARRAY_SIZE(builtin_x86_defs); ++i) {
 builtin_x86_defs[i].next = x86_defs;
-builtin_x86_defs[i].flags = 1;
+builtin_x86_defs[i].flags |= CPUID_FLAGS_BUILTIN;
 x86_defs = builtin_x86_defs[i];
 }
 #if !defined(CONFIG_USER_ONLY)
@@ -962,22 +962,51 @@ static void get_cpuid_vendor(CPUX86State *env, uint32_t 
*ebx,
  * this if you want to use KVM's sysenter/syscall emulation
  * in compatibility mode and when doing cross vendor migration
  */
-if (kvm_enabled()  ! env-cpuid_vendor_override) {
+if (kvm_enabled() 
+(env-cpuid_flags  CPUID_FLAGS_VENDOR_OVERRIDE) == 0) {
 host_cpuid(0, 0, NULL, ebx, ecx, edx);
 }
 }
 
+/* safe CPUID leafs to propagate to guest if -cpu host is specified
+ * Intel defined leafs:
+ * Cache descriptors (0x02)
+ * Deterministic cache parameters (0x04)
+ * Monitor/MWAIT parameters (0x05)
+ */
+#define CPUID_LEAF_PROPAGATE ((1  0x02) | (1  0x04) | (1  0x05))
+
+/* AMD defined leafs:
+ * L1 Cache and TLB (0x05)
+ * L2+L3 TLB (0x06)
+ * LongMode address size (0x08)
+ * 1GB page TLB (0x19)
+ * Performance optimization (0x1A)
+ */
+#define CPUID_LEAF_PROPAGATE_EXTENDED ((1  0x05) | (1  0x06) |\
+   (1  0x08) | (1  0x19) | (1  0x1A))
+
 void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
uint32_t *eax, uint32_t *ebx,
uint32_t *ecx, uint32_t *edx)
 {
-/* test if maximum index reached */
 if (index  0x8000) {
+/* test if maximum index reached */
 if (index  env

[Qemu-devel] [Bug 267542] Re: MINIX 3 won't boot in qemu 0.9.1

[Andre Przywara]

Is that still a problem? What was the exact error?
I quickly tried the 3.1.2a on qemu 0.12.4 (with and without KVM) and I could 
easily login.


** Changed in: qemu
   Status: New = Incomplete

-- 
MINIX 3 won't boot in qemu 0.9.1
https://bugs.launchpad.net/bugs/267542
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.

Status in QEMU: Incomplete

Bug description:
CD Image 3.1.2a was downloaded from http://www.minix3.org/download/

It booted with previous version of qemu but hangs at startup with 0.9.1.

Hardware acceleration is disabled.

Please ask if there is other information I can give you.

Re: [Qemu-devel] [RFC] Bug Day - June 1st, 2010

[Andre Przywara]

Michael Tokarev wrote:

20.05.2010 02:30, Anthony Liguori wrote:

On 05/19/2010 05:29 PM, Andre Przywara wrote:

Michael Tokarev wrote:

...

Also, thanks to Andre Przywara, whole winNT thing works but it requires
-cpu qemu64,level=1 (or level=2 or =3), -- _not_ with default CPU. This

[]

It'd be nice if we had more flexibility in defining custom machine types
so you could just do qemu -M win98.


This is wrong IMHO.  win98 and winNT can run on various different
machines, including all modern ones (yes I tried the same winNT
on my Athlon X2-64, just had to switch SATA from AHCI to IDE;
win95 works too)...  just not in kvm :)
Well, not really. You were lucky with your Athlon X2-64, actually it is 
the last machine not triggering the bug. I tried it on a AthlonII-X4 
(which has maxleaf=5 as any newer AMD machines) and it showed the same 
bug. On Intel boxes this bug should trigger on every CPU starting with 
some Pentium4 models, including all Core chips.

Have you tried versions with a newer service pack (SP6)?


BTW: Does anyone knows what the problem with Windows95/98 on KVM is? I
tried some tracing today, but couldn't find a hint.


Um.  The bugreport(s) come as a surprize for me: I tried to
install win98 in kvm several times in the past but setup
always failed - different messages in different versions
of kvm, either unable to emulate or real mode trap or
something else, or just lockup, usually on first reboot.
So - the bugreports talks about mouse non-working, but
this means win98 itself works somehow...  I dunno :)
I think these bug reports are about plain QEMU. I tried it yesterday, in 
fact the mouse is non-functional. In KVM Windows95 gives me a black 
screen after the welcome screen with the moving bottom row. There are 
just two lines at the top: (translated from the german version)

While initializing device NTKERN:
Windows protection fault. Restart the computer.

KVM catched some #UDs due to ARPL from VM86 mode, but TCG got them too 
and it survived. So if anyone has some more hints, I'd be grateful.


Regards,
Andre.

--
Andre Przywara
AMD-Operating System Research Center (OSRC), Dresden, Germany
Tel: +49 351 448-3567-12

Re: [Qemu-devel] [RFC] Bug Day - June 1st, 2010

[Andre Przywara]

Michael Tokarev wrote:

...

Also, thanks to Andre Przywara, whole winNT thing works but it requires
-cpu qemu64,level=1 (or level=2 or =3), -- _not_ with default CPU.  This
is also testing, but it's not obvious what to do witht the result...
Can't we use the file based CPU models for that? Actually it looks like 
a template config file for certain guest operation systems (like -vga 
std -net nic,model=ne2k_pci for older Windows version) make sense. These 
could include all quirks that we find on the way.


BTW: Does anyone knows what the problem with Windows95/98 on KVM is? I 
tried some tracing today, but couldn't find a hint.


Regards,
Andre.

--
Andre Przywara
AMD-Operating System Research Center (OSRC), Dresden, Germany
Tel: +49 351 488-3567-12

Re: [Qemu-devel] Re: [PATCH][STABLE] fix CPUID vendor override

[Andre Przywara]

Aurelien Jarno wrote:

On Sun, Apr 11, 2010 at 09:49:40PM +0200, Andre Przywara wrote:

Avi Kivity wrote:

On 04/11/2010 10:21 PM, Andre Przywara wrote:

the meaning of vendor_override is actually the opposite of how it
is currently used :-(
Fix it to allow KVM to export the non-native CPUID vendor if
explicitly requested by the user.

Signed-off-by: Andre Przywaraandre.przyw...@amd.com
---
  target-i386/helper.c |2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)

I will send a refactoring patch including this fix for git HEAD later.

Regards,
Andre.

diff --git a/target-i386/helper.c b/target-i386/helper.c
index 9d7fec3..c17adc1 100644
--- a/target-i386/helper.c
+++ b/target-i386/helper.c
@@ -1655,7 +1655,7 @@ static void get_cpuid_vendor(CPUX86State *env,  
uint32_t *ebx,

   * this if you want to use KVM's sysenter/syscall emulation
   * in compatibility mode and when doing cross vendor migration
   */
-if (kvm_enabled()  env-cpuid_vendor_override) {
+if (kvm_enabled()  ! env-cpuid_vendor_override) {
  host_cpuid(0, 0, NULL, ebx, ecx, edx);
  }
  }
   
Why is the original code wrong?  I would say vendor_override means  
overriding the qemu-picked vendor ID in favour of the host cpuid.
I meant it to mean: override the automatically chosen vendor ID (which  
is the host ID in case of KVM). I think the reason for KVM to use the  
host ID is valid, so I wanted to have an explicit override only if the  
user says so.
If you look at the code, you will see that it is initialized to 0 and  
only set to 1 if one specifies an explicit vendor ID on the command line.
Honestly I cannot say how this bug slipped through, I can only guess  
that I tricked myself while making the final version of the patch  
(lost-in-branches(TM))




While it clearly fixes the -cpu vendor option when kvm is enabled, it
also forces the vendor id to the one of the host. Try for example -cpu
qemu64. Before your patch the vendor id is Authentic AMD and after
your patch it is Genuine Intel

You are using the wrong CPU ;-)
This is actually correct for KVM. (or are you seeing this with TCG?)
To make sure we are actually talk about the same problem, the intended 
behavior is:

With TCG:
- always inject the configured vendor (either hard-coded, in config 
files or via ,vendor= commandline)

With KVM:
- by default inject the host's vendor
- if the user specifies ,vendor= on the commandline, use this 
instead of the host's vendor

- all pre-configured vendors (hard-coded, config file) are ignored

The reason KVM is injecting the host's vendor is the following:
There is a nasty difference between AMD and Intel chips in supporting 
syscall and sysenter in _compat_ mode (that is: running a 32bit binary 
while the OS is running 64bit). This mode is 99% compatible with the 
legacy protected mode, but AMD does not support sysenter here and Intel 
does not support syscall. So for instance the Linux kernel checks the 
vendor id and tells the userspace to use the supported instruction. On 
TCG we don't care as we properly emulate both(?), but on KVM the 
instruction is called natively. This leads to a crash (#UD exception) if 
the wrong vendor is used. So a long time ago the workaround of always 
propagating the host vendor was introduced.
Later last year I introduced emulation of the missing instructions, so 
to overcome the former limitation (and allow for more hacking with KVM) 
I introduced the vendor override. Sadly my patch was broken (I used a 
slightly different version for development), that's why this fix. 
qemu-kvm had an additional fix to kind of work-around this bug.


Hope that clarifies the issue, if you meant something else, tell me.

Regards,
Andre.

--
Andre Przywara
AMD-Operating System Research Center (OSRC), Dresden, Germany
Tel: +49 351 488-3567-12

[Qemu-devel] Re: [PATCH][STABLE] fix CPUID vendor override

[Andre Przywara]

Avi Kivity wrote:

On 04/11/2010 10:21 PM, Andre Przywara wrote:

the meaning of vendor_override is actually the opposite of how it
is currently used :-(
Fix it to allow KVM to export the non-native CPUID vendor if
explicitly requested by the user.

Signed-off-by: Andre Przywaraandre.przyw...@amd.com
---
  target-i386/helper.c |2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)

I will send a refactoring patch including this fix for git HEAD later.
   


The protocol is to first fix HEAD, then backport, so we don't introduce 
regressions inadvertently.
Well, I just followed the call for 0.12.4. Actually this is the 
backport, one cannot easily cherry-pick the upstream patch, because it 
is in a different file now. I will send the (separated) upstream patch 
in a few minutes. Feel free to apply them in whatever order you prefer ;-)



Regards,
Andre.

P.S. I also have a qemu-kvm patch in the queue (because there was a 
separate workaround for this bug).


--
Andre Przywara
AMD-Operating System Research Center (OSRC), Dresden, Germany
Tel: +49 351 488-3567-12

[Qemu-devel] [PATCH][STABLE] fix CPUID vendor override

[Andre Przywara]

the meaning of vendor_override is actually the opposite of how it
is currently used :-(
Fix it to allow KVM to export the non-native CPUID vendor if
explicitly requested by the user.

Signed-off-by: Andre Przywara andre.przyw...@amd.com
---
 target-i386/helper.c |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

I will send a refactoring patch including this fix for git HEAD later.

Regards,
Andre.

diff --git a/target-i386/helper.c b/target-i386/helper.c
index 9d7fec3..c17adc1 100644
--- a/target-i386/helper.c
+++ b/target-i386/helper.c
@@ -1655,7 +1655,7 @@ static void get_cpuid_vendor(CPUX86State *env, uint32_t 
*ebx,
  * this if you want to use KVM's sysenter/syscall emulation
  * in compatibility mode and when doing cross vendor migration
  */
-if (kvm_enabled()  env-cpuid_vendor_override) {
+if (kvm_enabled()  ! env-cpuid_vendor_override) {
 host_cpuid(0, 0, NULL, ebx, ecx, edx);
 }
 }
-- 
1.6.4

[Qemu-devel] Re: [PATCH][STABLE] fix CPUID vendor override

[Andre Przywara]

Avi Kivity wrote:

On 04/11/2010 10:21 PM, Andre Przywara wrote:

the meaning of vendor_override is actually the opposite of how it
is currently used :-(
Fix it to allow KVM to export the non-native CPUID vendor if
explicitly requested by the user.

Signed-off-by: Andre Przywaraandre.przyw...@amd.com
---
  target-i386/helper.c |2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)

I will send a refactoring patch including this fix for git HEAD later.

Regards,
Andre.

diff --git a/target-i386/helper.c b/target-i386/helper.c
index 9d7fec3..c17adc1 100644
--- a/target-i386/helper.c
+++ b/target-i386/helper.c
@@ -1655,7 +1655,7 @@ static void get_cpuid_vendor(CPUX86State *env, 
uint32_t *ebx,

   * this if you want to use KVM's sysenter/syscall emulation
   * in compatibility mode and when doing cross vendor migration
   */
-if (kvm_enabled()  env-cpuid_vendor_override) {
+if (kvm_enabled()  ! env-cpuid_vendor_override) {
  host_cpuid(0, 0, NULL, ebx, ecx, edx);
  }
  }
   


Why is the original code wrong?  I would say vendor_override means 
overriding the qemu-picked vendor ID in favour of the host cpuid.
I meant it to mean: override the automatically chosen vendor ID (which 
is the host ID in case of KVM). I think the reason for KVM to use the 
host ID is valid, so I wanted to have an explicit override only if the 
user says so.
If you look at the code, you will see that it is initialized to 0 and 
only set to 1 if one specifies an explicit vendor ID on the command line.
Honestly I cannot say how this bug slipped through, I can only guess 
that I tricked myself while making the final version of the patch 
(lost-in-branches(TM))


Regards,
Andre.

--
Andre Przywara
AMD-Operating System Research Center (OSRC), Dresden, Germany
Tel: +49 351 488-3567-12

[Qemu-devel] [PATCH] x86/cpuid: Add kvm32 CPU model

[Andre Przywara]

Create a kvm32 CPU model that describes a least common denominator
for KVM capable guest CPUs. Useful for migration purposes.

Signed-off-by: Andre Przywara andre.przyw...@amd.com
---
 target-i386/cpuid.c |   14 ++
 1 files changed, 14 insertions(+), 0 deletions(-)

diff --git a/target-i386/cpuid.c b/target-i386/cpuid.c
index a80baa4..76b897d 100644
--- a/target-i386/cpuid.c
+++ b/target-i386/cpuid.c
@@ -363,6 +363,20 @@ static x86_def_t builtin_x86_defs[] = {
 .model_id = QEMU Virtual CPU version  QEMU_VERSION,
 },
 {
+.name = kvm32,
+.level = 5,
+.family = 15,
+.model = 6,
+.stepping = 1,
+.features = PPRO_FEATURES |
+CPUID_MTRR | CPUID_CLFLUSH | CPUID_MCA | CPUID_PSE36,
+.ext_features = CPUID_EXT_SSE3,
+.ext2_features = PPRO_FEATURES  EXT2_FEATURE_MASK,
+.ext3_features = 0,
+.xlevel = 0x8008,
+.model_id = Common 32-bit KVM processor
+},
+{
 .name = coreduo,
 .level = 10,
 .family = 6,
-- 
1.6.4

[Qemu-devel] tainted Linux kernel in default SMP QEMU/KVM guests

[Andre Przywara]

Hi,

since the default CPU model for QEMU (qemu64) is an AMD K7, the Linux 
kernel complains and taints the kernel when it detects multiple 
processors. The reason for this is a check for SMP safe CPUs in 
arch/x86/kernel/cpu/amd.c:amd_k7_smp_check(). In recent kernels (since 
about 2.6.29) this in only executed in 32bit kernels, since the check 
has been #ifdef'ed CONFIG_X86_32.
The failing check triggers a nasty dump along with a call trace and 
taints the kernel, this confuses users, so I want to get rid of it.
One can work around it (using -cpu kvm64 or -cpu host), but I want to 
avoid it in the default case, too.


I see these possible solutions:

1) Change the default CPUID bits from 6/2/3 to 6/6/1, this passes the 
Linux kernel check. But I am not sure if that would introduce 
regressions, since some OSes apply quirks if they detect certain models 
(like we had with the sysenter issue in the past)


2) Only change the CPUID bits to 6/6/1 if we use SMP. Still has the 
above drawback, but would be limited to SMP guests only.


3) Set kvm64/kvm32 as the default CPU model if KVM is enabled. This 
would limit the report and taint to TCG, where SMP is rarely used. 
Additionally less people (if any) use it for production systems.


4) Make the Linux' kernel quirk dependent on the missing hypervisor bit. 
I don't think this will be accepted easily upstream (and I don't want to 
support Ingo's recent ideas ;-), also this would not fix older kernels.


I can easily provide patches for all solutions, but I'd like to get 
advice from people on which one to pursue.


Regards,
Andre.

 --
Andre Przywara
AMD-Operating System Research Center (OSRC), Dresden, Germany
Tel: +49 351 448-3567-12

[Qemu-devel] [PATCH 03/13] x86/cpuid: fix missing feature set bits

[Andre Przywara]

This one was accidently removed with commit
bb0300dc57c10b3721451b0ff566a03f9276cc77

Signed-off-by: Andre Przywara andre.przyw...@amd.com

---
 target-i386/cpuid.c |1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/target-i386/cpuid.c b/target-i386/cpuid.c
index 5e4f057..1bceb4b 100644
--- a/target-i386/cpuid.c
+++ b/target-i386/cpuid.c
@@ -754,6 +754,7 @@ int cpu_x86_register (CPUX86State *env, const char 
*cpu_model)
 env-pat = 0x0007040600070406ULL;
 env-cpuid_ext_features = def-ext_features;
 env-cpuid_ext2_features = def-ext2_features;
+env-cpuid_ext3_features = def-ext3_features;
 env-cpuid_xlevel = def-xlevel;
 env-cpuid_kvm_features = def-kvm_features;
 {
-- 
1.6.4