Re: [Xen-devel] [PATCH v9 00/10] error: auto propagated local_err part I

2020-03-12 Thread Vladimir Sementsov-Ogievskiy
12.03.2020 17:24, Markus Armbruster wrote: Vladimir Sementsov-Ogievskiy writes: v9 01: A lot of rewordings [thanks to Eric] Still, keep all r-b marks, assuming that they are mostly about macro definition 02: significant changes are: 1. Do not match double propagation pattern in ERRP

Re: [Xen-devel] [PATCH v9 02/10] scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()

2020-03-12 Thread Vladimir Sementsov-Ogievskiy
12.03.2020 19:36, Markus Armbruster wrote: I may have a second look tomorrow with fresher eyes, but let's get this out now as is. Vladimir Sementsov-Ogievskiy writes: Script adds ERRP_AUTO_PROPAGATE macro invocation where appropriate and does corresponding changes in code (look for details in

Re: [Xen-devel] [PATCH 0/2] xen: credit2: fix vcpu starvation due to too few credits

2020-03-12 Thread Dario Faggioli
On Thu, 2020-03-12 at 18:59 +0100, Roger Pau Monné wrote: > On Thu, Mar 12, 2020 at 06:02:03PM +0100, Dario Faggioli wrote: > > What do you mean with "Which timer does this hardware use" ? > > Xen uses a hardware timer (HPET, PMTIMER or PIT IIRC) in order to get > interrupts at specified times, on

Re: [Xen-devel] [PATCH] x86/vvmx: Fix deadlock with MSR bitmap merging

2020-03-12 Thread Tian, Kevin
> From: Andrew Cooper > Sent: Thursday, March 12, 2020 2:35 AM > > c/s c47984aabead "nvmx: implement support for MSR bitmaps" introduced a > use of > map_domain_page() which may get used in the middle of context switch. > > This is not safe, and causes Xen to deadlock on the mapcache lock: > >

[Xen-devel] [linux-4.19 test] 148453: regressions - FAIL

2020-03-12 Thread osstest service owner
flight 148453 linux-4.19 real [real] http://logs.test-lab.xenproject.org/osstest/logs/148453/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-amd64-amd64-qemuu-nested-intel 17 debian-hvm-install/l1/l2 fail REGR. vs. 142932 Tests which

[Xen-devel] [libvirt test] 148459: regressions - FAIL

2020-03-12 Thread osstest service owner
flight 148459 libvirt real [real] http://logs.test-lab.xenproject.org/osstest/logs/148459/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: build-amd64-libvirt 6 libvirt-buildfail REGR. vs. 146182 build-i386-libvirt

Re: [Xen-devel] [PATCH v3] IOMMU: make DMA containment of quarantined devices optional

2020-03-12 Thread Tian, Kevin
> From: Paul Durrant > Sent: Wednesday, March 11, 2020 12:05 AM > [...] > > > > > > However, is a really saying that things will break if any of the > > > PTEs has their present bit clear? > > > > Well, you said that read faults are fatal (to the host). Reads will, > > for any address with an un

Re: [Xen-devel] [PATCH v3] IOMMU: make DMA containment of quarantined devices optional

2020-03-12 Thread Tian, Kevin
> From: Jan Beulich > Sent: Tuesday, March 10, 2020 6:31 PM > > On 10.03.2020 06:30, Tian, Kevin wrote: > >> From: Jan Beulich > >> Sent: Monday, March 9, 2020 7:09 PM > >> > >> Containing still in flight DMA was introduced to work around certain > >> devices / systems hanging hard upon hitting

Re: [Xen-devel] [PATCH v3] IOMMU: make DMA containment of quarantined devices optional

2020-03-12 Thread Tian, Kevin
> From: Jan Beulich > Sent: Tuesday, March 10, 2020 6:27 PM > > On 10.03.2020 04:43, Tian, Kevin wrote: > >> From: Jan Beulich > >> Sent: Monday, March 9, 2020 7:09 PM > >> > >> I'm happy to take better suggestions to replace the "full" command line > >> option and Kconfig prompt tokens. I don't

Re: [Xen-devel] [PATCH v2 8/9] x86/HVM: reduce hvm.h include dependencies

2020-03-12 Thread Tian, Kevin
> From: Jan Beulich > Sent: Tuesday, March 10, 2020 11:52 PM > > Drop #include-s not needed by the header itself, and add smaller scope > ones instead. Put the ones needed into whichever other files actually > need them. > > Signed-off-by: Jan Beulich Reviewed-by: Kevin Tian _

Re: [Xen-devel] [PATCH v2 7/9] x86/HVM: reduce io.h include dependencies

2020-03-12 Thread Tian, Kevin
> From: Jan Beulich > Sent: Tuesday, March 10, 2020 11:51 PM > > Drop #include-s not needed by the header itself as well as one include > of the header which isn't needed. Put the one needed into the file > actually requiring it. > > Signed-off-by: Jan Beulich Reviewed-by: Kevin Tian

[Xen-devel] [linux-5.4 test] 148437: regressions - FAIL

2020-03-12 Thread osstest service owner
flight 148437 linux-5.4 real [real] http://logs.test-lab.xenproject.org/osstest/logs/148437/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-amd64-i386-xl-shadowbroken in 148282 test-amd64-i386-xl-qemuu-debianhv

[Xen-devel] [linux-linus bisection] complete test-amd64-amd64-qemuu-nested-intel

2020-03-12 Thread osstest service owner
branch xen-unstable xenbranch xen-unstable job test-amd64-amd64-qemuu-nested-intel testid debian-hvm-install/l1/l2 Tree: linux git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git Tree: ovmf git://xenbits.xen.org/

[Xen-devel] [linux-4.4 test] 148426: regressions - FAIL

2020-03-12 Thread osstest service owner
flight 148426 linux-4.4 real [real] http://logs.test-lab.xenproject.org/osstest/logs/148426/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-amd64-i386-libvirt-xsm 12 guest-start fail REGR. vs. 139698 test-amd64-i386-libvi

[Xen-devel] [PATCH] tools/helpers: xen-init-dom0: Mark clear_domid_history() static

2020-03-12 Thread julien
From: Julien Grall xen-init-dom0 is a standalone binary, so all the functions but the main() should be static. Signed-off-by: Julien Grall Cc: p...@xen.org --- tools/helpers/xen-init-dom0.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/helpers/xen-init-dom0.c b/tool

[Xen-devel] [linux-next test] 148424: regressions - FAIL

2020-03-12 Thread osstest service owner
flight 148424 linux-next real [real] http://logs.test-lab.xenproject.org/osstest/logs/148424/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-amd64-i386-xl-qemut-debianhvm-i386-xsm 7 xen-boot fail REGR. vs. 148333 test-amd64-i386-libv

Re: [Xen-devel] [XEN PATCH v3 1/2] xen/arm: Rename all early printk macro

2020-03-12 Thread Julien Grall
Hi, On 11/03/2020 14:46, Anthony PERARD wrote: On Wed, Mar 11, 2020 at 01:57:37PM +, Julien Grall wrote: Hi Anthony, On 09/03/2020 17:45, Anthony PERARD wrote: diff --git a/xen/arch/arm/arm32/head.S b/xen/arch/arm/arm32/head.S index e9d356f05c2b..2b593c5ef99a 100644 --- a/xen/arch/arm/arm

Re: [Xen-devel] [XEN PATCH v3 2/2] xen/arm: Configure early printk via Kconfig

2020-03-12 Thread Julien Grall
Hi, On 11/03/2020 17:38, Anthony PERARD wrote: On Wed, Mar 11, 2020 at 05:21:24PM +, Julien Grall wrote: On 11/03/2020 15:26, Anthony PERARD wrote: On Wed, Mar 11, 2020 at 02:18:20PM +, Julien Grall wrote: +config EARLY_UART_BASE_ADDRESS + depends on EARLY_PRINTK + hex "Ea

Re: [Xen-devel] [PATCH 0/2] xen: credit2: fix vcpu starvation due to too few credits

2020-03-12 Thread Roger Pau Monné
On Thu, Mar 12, 2020 at 06:02:03PM +0100, Dario Faggioli wrote: > On Thu, 2020-03-12 at 16:08 +0100, Roger Pau Monné wrote: > > Thanks for looking into this, seems like a specially tricky issue to > > tackle! > > > It was tricky indeed! :-) > > > On Thu, Mar 12, 2020 at 02:44:07PM +0100, Dario Fa

Re: [Xen-devel] Stopping much Linux testing in Xen Project CI

2020-03-12 Thread Roger Pau Monné
On Thu, Mar 12, 2020 at 04:49:51PM +, Ian Jackson wrote: > Linux stable branches, and Linux upstream tip, are badly broken and > have been for months. Apparently no-one is able to (or has time to) > to investigate and fix. > > linux-4.4 218 days to be suspended > linux-4.

Re: [Xen-devel] Stopping much Linux testing in Xen Project CI

2020-03-12 Thread Jürgen Groß
On 12.03.20 17:49, Ian Jackson wrote: Linux stable branches, and Linux upstream tip, are badly broken and have been for months. Apparently no-one is able to (or has time to) to investigate and fix. linux-4.4 218 days to be suspended linux-4.9 134 days to

Re: [Xen-devel] [PATCH 1/2] xen: credit2: avoid vCPUs to ever reach lower credits than idle

2020-03-12 Thread Dario Faggioli
On Thu, 2020-03-12 at 14:45 +, George Dunlap wrote: > > On Mar 12, 2020, at 1:44 PM, Dario Faggioli > > wrote: > > > > This patch makes Credit2 more robust to events like this, whatever > > the cause is, and should hence be backported (as far as possible). > > > > Signed-off-by: Dario Faggio

Re: [Xen-devel] [PATCH 0/2] xen: credit2: fix vcpu starvation due to too few credits

2020-03-12 Thread Dario Faggioli
On Thu, 2020-03-12 at 16:08 +0100, Roger Pau Monné wrote: > Thanks for looking into this, seems like a specially tricky issue to > tackle! > It was tricky indeed! :-) > On Thu, Mar 12, 2020 at 02:44:07PM +0100, Dario Faggioli wrote: > [...] > > For example, I have a trace showing that csched2_sch

Re: [Xen-devel] [PATCH 1/2] xen: credit2: avoid vCPUs to ever reach lower credits than idle

2020-03-12 Thread Dario Faggioli
On Thu, 2020-03-12 at 17:00 +0100, Jürgen Groß wrote: > On 12.03.20 16:26, Jan Beulich wrote: > > On 12.03.2020 14:44, Dario Faggioli wrote: > > > --- a/xen/common/sched/credit2.c > > > +++ b/xen/common/sched/credit2.c > > > @@ -234,7 +234,7 @@ > > >* units does not consume credits, and it must

Re: [Xen-devel] Stopping much Linux testing in Xen Project CI

2020-03-12 Thread Ian Jackson
Additionally, linux-next 2162 days This is obviously useless. I am stopping it. Ian. ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] Stopping much Linux testing in Xen Project CI

2020-03-12 Thread Ian Jackson
Linux stable branches, and Linux upstream tip, are badly broken and have been for months. Apparently no-one is able to (or has time to) to investigate and fix. linux-4.4 218 days to be suspended linux-4.9 134 days to be suspended linux-4.14 134 days

Re: [Xen-devel] [PATCH v9 02/10] scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()

2020-03-12 Thread Markus Armbruster
I may have a second look tomorrow with fresher eyes, but let's get this out now as is. Vladimir Sementsov-Ogievskiy writes: > Script adds ERRP_AUTO_PROPAGATE macro invocation where appropriate and > does corresponding changes in code (look for details in > include/qapi/error.h) > > Usage example

Re: [Xen-devel] [PATCH 1/2] xen: credit2: avoid vCPUs to ever reach lower credits than idle

2020-03-12 Thread Jan Beulich
On 12.03.2020 17:11, Dario Faggioli wrote: > On Thu, 2020-03-12 at 16:26 +0100, Jan Beulich wrote: >> Looking at t2c_update() I'm also getting the impression that >> there's UB when the subtraction underflows. After all, if >> -1 << 30 wasn't small enough a value, I don't see why -1 << 31 >> would

Re: [Xen-devel] [PATCH 0/2] xen: credit2: fix vcpu starvation due to too few credits

2020-03-12 Thread Andrew Cooper
On 12/03/2020 15:51, Jürgen Groß wrote: > On 12.03.20 14:44, Dario Faggioli wrote: >> Hello everyone, >> >> There have been reports of a Credit2 issue due to which vCPUs where >> being starved, to the point that guest kernel would complain or even >> crash. >> >> See the following xen-users and x

Re: [Xen-devel] [PATCH 1/2] xen: credit2: avoid vCPUs to ever reach lower credits than idle

2020-03-12 Thread Dario Faggioli
On Thu, 2020-03-12 at 16:26 +0100, Jan Beulich wrote: > On 12.03.2020 14:44, Dario Faggioli wrote: > > --- a/xen/common/sched/credit2.c > > +++ b/xen/common/sched/credit2.c > > @@ -234,7 +234,7 @@ > > * units does not consume credits, and it must be lower than > > whatever > > * amount of credi

[Xen-devel] [qemu-mainline test] 148421: regressions - FAIL

2020-03-12 Thread osstest service owner
flight 148421 qemu-mainline real [real] http://logs.test-lab.xenproject.org/osstest/logs/148421/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 10 debian-hvm-install fail REGR. vs. 14486

Re: [Xen-devel] [PATCH 1/2] xen: credit2: avoid vCPUs to ever reach lower credits than idle

2020-03-12 Thread Jürgen Groß
On 12.03.20 16:26, Jan Beulich wrote: On 12.03.2020 14:44, Dario Faggioli wrote: --- a/xen/common/sched/credit2.c +++ b/xen/common/sched/credit2.c @@ -234,7 +234,7 @@ * units does not consume credits, and it must be lower than whatever * amount of credit 'regular' unit would end up with.

Re: [Xen-devel] [PATCH 0/2] xen: credit2: fix vcpu starvation due to too few credits

2020-03-12 Thread Jürgen Groß
On 12.03.20 14:44, Dario Faggioli wrote: Hello everyone, There have been reports of a Credit2 issue due to which vCPUs where being starved, to the point that guest kernel would complain or even crash. See the following xen-users and xen-devel threads: https://lists.xenproject.org/archives/html/

Re: [Xen-devel] [PATCH 1/2] xen: credit2: avoid vCPUs to ever reach lower credits than idle

2020-03-12 Thread Jan Beulich
On 12.03.2020 14:44, Dario Faggioli wrote: > --- a/xen/common/sched/credit2.c > +++ b/xen/common/sched/credit2.c > @@ -234,7 +234,7 @@ > * units does not consume credits, and it must be lower than whatever > * amount of credit 'regular' unit would end up with. > */ > -#define CSCHED2_IDLE_CRE

Re: [Xen-devel] nvmx deadlock with MSR bitmaps

2020-03-12 Thread Jürgen Groß
On 12.03.20 16:11, Jan Beulich wrote: On 12.03.2020 14:44, Roger Pau Monné wrote: On Thu, Mar 12, 2020 at 12:12:12PM +0100, Jürgen Groß wrote: On 12.03.20 11:56, Roger Pau Monné wrote: On Thu, Mar 12, 2020 at 09:59:48AM +0100, Jan Beulich wrote: On 11.03.2020 19:04, Andrew Cooper wrote: Spec

Re: [Xen-devel] [PATCH 1/2] x86/xen: Make the boot CPU idle task reliable

2020-03-12 Thread Miroslav Benes
On Thu, 12 Mar 2020, Andrew Cooper wrote: > On 12/03/2020 14:20, Miroslav Benes wrote: > > The unwinder reports the boot CPU idle task's stack on XEN PV as > > unreliable, which affects at least live patching. There are two reasons > > for this. First, the task does not follow the x86 convention t

Re: [Xen-devel] [PATCH 1/2] xen: credit2: avoid vCPUs to ever reach lower credits than idle

2020-03-12 Thread Dario Faggioli
On Thu, 2020-03-12 at 14:40 +, George Dunlap wrote: > > On Mar 12, 2020, at 1:55 PM, Andrew Cooper < > > andrew.coop...@citrix.com> wrote: > > > > Certainly as far as XenServer is concerned, we haven't seen > > symptoms > > like this in a decade of running credit1. > > One reason could be bec

Re: [Xen-devel] nvmx deadlock with MSR bitmaps

2020-03-12 Thread Jan Beulich
On 12.03.2020 14:44, Roger Pau Monné wrote: > On Thu, Mar 12, 2020 at 12:12:12PM +0100, Jürgen Groß wrote: >> On 12.03.20 11:56, Roger Pau Monné wrote: >>> On Thu, Mar 12, 2020 at 09:59:48AM +0100, Jan Beulich wrote: On 11.03.2020 19:04, Andrew Cooper wrote: > Specifically, this is a switc

Re: [Xen-devel] [PATCH 0/2] xen: credit2: fix vcpu starvation due to too few credits

2020-03-12 Thread Roger Pau Monné
Thanks for looking into this, seems like a specially tricky issue to tackle! On Thu, Mar 12, 2020 at 02:44:07PM +0100, Dario Faggioli wrote: [...] > For example, I have a trace showing that csched2_schedule() is invoked at > t=57970746155ns. At t=57970747658ns (+1503ns) the s_timer is set to > fir

Re: [Xen-devel] [PATCH 1/2] x86/xen: Make the boot CPU idle task reliable

2020-03-12 Thread Andrew Cooper
On 12/03/2020 14:20, Miroslav Benes wrote: > The unwinder reports the boot CPU idle task's stack on XEN PV as > unreliable, which affects at least live patching. There are two reasons > for this. First, the task does not follow the x86 convention that its > stack starts at the offset right below sa

Re: [Xen-devel] [PATCH 1/2] xen: credit2: avoid vCPUs to ever reach lower credits than idle

2020-03-12 Thread Dario Faggioli
On Thu, 2020-03-12 at 13:55 +, Andrew Cooper wrote: > On 12/03/2020 13:44, Dario Faggioli wrote: > > > > NOTE: investigations have been done about _how_ it is possible for > > a > > vCPU to execute for so long that its credits becomes so low. While > > still > > not completely clear, there are

[Xen-devel] [PATCH v2 0/2] scripts: Use stat to check lock claim

2020-03-12 Thread Jason Andryuk
This patch series replaces perl with stat to check lock ownership in the Linux hotplug scripts. This removes Xen's runtime dependency on perl. An additional patch is a tabs to space whitespace cleanup. Jason Andryuk (2): scripts: Replace tabs in locking.sh scripts: Use stat to check lock cla

[Xen-devel] [PATCH 2/2] scripts: Use stat to check lock claim

2020-03-12 Thread Jason Andryuk
Replace the perl locking check with stat(1). Stat is able to fstat stdin (file descriptor 0) when passed '-' as an argument. This is now used to check $_lockfd. stat(1) support for '-' was introduced to coreutils in 2009. After A releases its lock, script B will return from flock and execute st

[Xen-devel] [PATCH 1/2] scripts: Replace tabs in locking.sh

2020-03-12 Thread Jason Andryuk
Replace two stray tabs with spaces to make the file whitespace consistent. Signed-off-by: Jason Andryuk --- tools/hotplug/Linux/locking.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tools/hotplug/Linux/locking.sh b/tools/hotplug/Linux/locking.sh index c6a7e96ff9..baa

Re: [Xen-devel] [PATCH OSSTEST v6 4/5] examine: detect IOMMU availability and add it as a hostflag

2020-03-12 Thread Ian Jackson
Roger Pau Monne writes ("[PATCH OSSTEST v6 4/5] examine: detect IOMMU availability and add it as a hostflag"): > Introduce a new test to check for iommu availability and add it as a > hostflag if found. Thanks, Reviewed-by: Ian Jackson ___ Xen-devel

[Xen-devel] [RFC PATCH 2/2] x86/xen: Make the secondary CPU idle tasks reliable

2020-03-12 Thread Miroslav Benes
The unwinder reports the secondary CPU idle tasks' stack on XEN PV as unreliable, which affects at least live patching. cpu_initialize_context() sets up the context of the CPU through VCPUOP_initialise hypercall. After it is woken up, the idle task starts in cpu_bringup_and_idle() function and its

[Xen-devel] [PATCH 1/2] x86/xen: Make the boot CPU idle task reliable

2020-03-12 Thread Miroslav Benes
The unwinder reports the boot CPU idle task's stack on XEN PV as unreliable, which affects at least live patching. There are two reasons for this. First, the task does not follow the x86 convention that its stack starts at the offset right below saved pt_regs. It allows the unwinder to easily detec

[Xen-devel] [PATCH 0/2] x86/xen: Make idle tasks reliable

2020-03-12 Thread Miroslav Benes
The unwinder reports idle tasks' stack on XEN PV as unreliable which complicates things for at least live patching. The two patches in the series try to amend that by using similar approach as non-XEN x86 does. However, I did not come up with a nice solution for secondary CPUs idle tasks. The patc

Re: [Xen-devel] [PATCH 1/2] xen: credit2: avoid vCPUs to ever reach lower credits than idle

2020-03-12 Thread George Dunlap
> On Mar 12, 2020, at 1:44 PM, Dario Faggioli wrote: > > There have been report of stalls of guest vCPUs, when Credit2 was used. > It seemed like these vCPUs were not getting scheduled for very long > time, even under light load conditions (e.g., during dom0 boot). > > Investigations led to th

[Xen-devel] [PATCH OSSTEST v6 4/5] examine: detect IOMMU availability and add it as a hostflag

2020-03-12 Thread Roger Pau Monne
Introduce a new test to check for iommu availability and add it as a hostflag if found. Signed-off-by: Roger Pau Monné --- Changes since v5: - Use a more restrictive regex. - Place iommu test before log collection. Changes since v4: - Split out code into separate patches. Changes since v3:

Re: [Xen-devel] [PATCH 1/2] xen: credit2: avoid vCPUs to ever reach lower credits than idle

2020-03-12 Thread George Dunlap
> On Mar 12, 2020, at 1:55 PM, Andrew Cooper wrote: > > On 12/03/2020 13:44, Dario Faggioli wrote: >> There have been report of stalls of guest vCPUs, when Credit2 was used. >> It seemed like these vCPUs were not getting scheduled for very long >> time, even under light load conditions (e.g., d

Re: [Xen-devel] [PATCH v9 00/10] error: auto propagated local_err part I

2020-03-12 Thread Markus Armbruster
Vladimir Sementsov-Ogievskiy writes: > v9 > 01: A lot of rewordings [thanks to Eric] > Still, keep all r-b marks, assuming that they are mostly about macro > definition > 02: significant changes are: > 1. Do not match double propagation pattern in ERRP_AUTO_PROPAGATE-adding > rule >

[Xen-devel] [xen-unstable test] 148419: regressions - FAIL

2020-03-12 Thread osstest service owner
flight 148419 xen-unstable real [real] http://logs.test-lab.xenproject.org/osstest/logs/148419/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-amd64-i386-qemut-rhel6hvm-amd 12 guest-start/redhat.repeat fail REGR. vs. 148098 Tests whic

Re: [Xen-devel] [PATCH 1/2] xen: credit2: avoid vCPUs to ever reach lower credits than idle

2020-03-12 Thread Andrew Cooper
On 12/03/2020 13:44, Dario Faggioli wrote: > There have been report of stalls of guest vCPUs, when Credit2 was used. > It seemed like these vCPUs were not getting scheduled for very long > time, even under light load conditions (e.g., during dom0 boot). > > Investigations led to the discovery that

Re: [Xen-devel] nvmx deadlock with MSR bitmaps

2020-03-12 Thread Roger Pau Monné
On Thu, Mar 12, 2020 at 12:12:12PM +0100, Jürgen Groß wrote: > On 12.03.20 11:56, Roger Pau Monné wrote: > > On Thu, Mar 12, 2020 at 09:59:48AM +0100, Jan Beulich wrote: > > > On 11.03.2020 19:04, Andrew Cooper wrote: > > > > Specifically, this is a switch from an HVM vcpu, to a PV vcpu, where the

[Xen-devel] [PATCH 1/2] xen: credit2: avoid vCPUs to ever reach lower credits than idle

2020-03-12 Thread Dario Faggioli
There have been report of stalls of guest vCPUs, when Credit2 was used. It seemed like these vCPUs were not getting scheduled for very long time, even under light load conditions (e.g., during dom0 boot). Investigations led to the discovery that --although rarely-- it can happen that a vCPU manage

[Xen-devel] [PATCH 2/2] xen: credit2: fix credit reset happening too few times

2020-03-12 Thread Dario Faggioli
There is a bug in commit 5e4b4199667b9 ("xen: credit2: only reset credit on reset condition"). In fact, the aim of that commit was to make sure that we do not perform too many credit reset operations (which are not super cheap, and in an hot-path). But the check used to determine whether a reset is

[Xen-devel] [PATCH 0/2] xen: credit2: fix vcpu starvation due to too few credits

2020-03-12 Thread Dario Faggioli
Hello everyone, There have been reports of a Credit2 issue due to which vCPUs where being starved, to the point that guest kernel would complain or even crash. See the following xen-users and xen-devel threads: https://lists.xenproject.org/archives/html/xen-users/2020-02/msg00018.html https://lis

Re: [Xen-devel] [PATCH] x86/vvmx: Fix deadlock with MSR bitmap merging

2020-03-12 Thread Andrew Cooper
On 12/03/2020 13:25, Jan Beulich wrote: > On 12.03.2020 13:06, Andrew Cooper wrote: >> On 12/03/2020 10:58, Roger Pau Monné wrote: >>> On Thu, Mar 12, 2020 at 10:30:35AM +0100, Roger Pau Monné wrote: On Wed, Mar 11, 2020 at 06:34:55PM +, Andrew Cooper wrote: > c/s c47984aabead "nvmx: i

Re: [Xen-devel] [PATCH] x86/vvmx: Fix deadlock with MSR bitmap merging

2020-03-12 Thread Roger Pau Monné
On Thu, Mar 12, 2020 at 12:21:29PM +, Andrew Cooper wrote: > On 12/03/2020 09:21, Jan Beulich wrote: > > On 11.03.2020 19:34, Andrew Cooper wrote: > >> c/s c47984aabead "nvmx: implement support for MSR bitmaps" introduced a > >> use of > >> map_domain_page() which may get used in the middle of

Re: [Xen-devel] [PATCH] x86/vvmx: Fix deadlock with MSR bitmap merging

2020-03-12 Thread Jan Beulich
On 12.03.2020 13:06, Andrew Cooper wrote: > On 12/03/2020 10:58, Roger Pau Monné wrote: >> On Thu, Mar 12, 2020 at 10:30:35AM +0100, Roger Pau Monné wrote: >>> On Wed, Mar 11, 2020 at 06:34:55PM +, Andrew Cooper wrote: c/s c47984aabead "nvmx: implement support for MSR bitmaps" introduced a

Re: [Xen-devel] [PATCH] x86/vvmx: Fix deadlock with MSR bitmap merging

2020-03-12 Thread Andrew Cooper
On 12/03/2020 09:21, Jan Beulich wrote: > On 11.03.2020 19:34, Andrew Cooper wrote: >> c/s c47984aabead "nvmx: implement support for MSR bitmaps" introduced a use >> of >> map_domain_page() which may get used in the middle of context switch. >> >> This is not safe, and causes Xen to deadlock on th

Re: [Xen-devel] [PATCH] x86/vvmx: Fix deadlock with MSR bitmap merging

2020-03-12 Thread Andrew Cooper
On 12/03/2020 10:58, Roger Pau Monné wrote: > On Thu, Mar 12, 2020 at 10:30:35AM +0100, Roger Pau Monné wrote: >> On Wed, Mar 11, 2020 at 06:34:55PM +, Andrew Cooper wrote: >>> c/s c47984aabead "nvmx: implement support for MSR bitmaps" introduced a use >>> of >>> map_domain_page() which may ge

Re: [Xen-devel] [PATCH OSSTEST v5 4/5] examine: detect IOMMU availability and add it as a hostflag

2020-03-12 Thread Ian Jackson
Roger Pau Monne writes ("[PATCH OSSTEST v5 4/5] examine: detect IOMMU availability and add it as a hostflag"): > Introduce a new test to check for iommu availability and add it as a > hostflag if found. > > Signed-off-by: Roger Pau Monné > --- > Changes since v4: > - Split out code into separat

Re: [Xen-devel] nvmx deadlock with MSR bitmaps

2020-03-12 Thread Jürgen Groß
On 12.03.20 11:56, Roger Pau Monné wrote: On Thu, Mar 12, 2020 at 09:59:48AM +0100, Jan Beulich wrote: On 11.03.2020 19:04, Andrew Cooper wrote: Specifically, this is a switch from an HVM vcpu, to a PV vcpu, where the mapcache code tries to access the per-domain mappings on the HVM monitor tabl

Re: [Xen-devel] nvmx deadlock with MSR bitmaps

2020-03-12 Thread Jan Beulich
On 12.03.2020 11:56, Roger Pau Monné wrote: > On Thu, Mar 12, 2020 at 09:59:48AM +0100, Jan Beulich wrote: >> On 11.03.2020 19:04, Andrew Cooper wrote: >>> Specifically, this is a switch from an HVM vcpu, to a PV vcpu, where the >>> mapcache code tries to access the per-domain mappings on the HVM m

Re: [Xen-devel] [PATCH OSSTEST v5 4/5] examine: detect IOMMU availability and add it as a hostflag

2020-03-12 Thread Ian Jackson
Roger Pau Monne writes ("[PATCH OSSTEST v5 4/5] examine: detect IOMMU availability and add it as a hostflag"): > Introduce a new test to check for iommu availability and add it as a > hostflag if found. ... > +our $has_iommu = $info =~ /directio/; I think this regexp is too lax. For example, if

Re: [Xen-devel] [PATCH] x86/vvmx: Fix deadlock with MSR bitmap merging

2020-03-12 Thread Roger Pau Monné
On Thu, Mar 12, 2020 at 10:30:35AM +0100, Roger Pau Monné wrote: > On Wed, Mar 11, 2020 at 06:34:55PM +, Andrew Cooper wrote: > > c/s c47984aabead "nvmx: implement support for MSR bitmaps" introduced a use > > of > > map_domain_page() which may get used in the middle of context switch. > > >

Re: [Xen-devel] [PATCH OSSTEST v5 3/5] ts-examine-hostprops-save: record hostflags also

2020-03-12 Thread Ian Jackson
Roger Pau Monne writes ("[PATCH OSSTEST v5 3/5] ts-examine-hostprops-save: record hostflags also"): > Commit putative hotflags into the database if present on the runvars. > > Signed-off-by: Roger Pau Monné Reviewed-by: Ian Jackson ___ Xen-devel mai

Re: [Xen-devel] [PATCH OSSTEST v5 2/5] host: introduce a helper to modify hostflags

2020-03-12 Thread Ian Jackson
Roger Pau Monne writes ("[PATCH OSSTEST v5 2/5] host: introduce a helper to modify hostflags"): > Add a generic function to perform database changes related to a host > flag and add a wrapper to TestSupport. Reviewed-by: Ian Jackson ___ Xen-devel mail

Re: [Xen-devel] [PATCH OSSTEST v5 1/5] host: introduce modify_host

2020-03-12 Thread Ian Jackson
Roger Pau Monne writes ("[PATCH OSSTEST v5 1/5] host: introduce modify_host"): > Abstract the set_property checks and DB call into a helper. > > No functional change. > > Signed-off-by: Roger Pau Monné Reviewed-by: Ian Jackson ___ Xen-devel mailing

Re: [Xen-devel] nvmx deadlock with MSR bitmaps

2020-03-12 Thread Roger Pau Monné
On Thu, Mar 12, 2020 at 09:59:48AM +0100, Jan Beulich wrote: > On 11.03.2020 19:04, Andrew Cooper wrote: > > Specifically, this is a switch from an HVM vcpu, to a PV vcpu, where the > > mapcache code tries to access the per-domain mappings on the HVM monitor > > table.  It ends up trying to recursi

Re: [Xen-devel] [PATCH v2 2/2] libxl: make creation of xenstore 'suspend event channel' node optional...

2020-03-12 Thread Durrant, Paul
Ping? > -Original Message- > From: pdurr...@amzn.com > Sent: 05 March 2020 12:14 > To: xen-devel@lists.xenproject.org > Cc: Durrant, Paul ; Ian Jackson > ; Wei Liu > ; Andrew Cooper ; George Dunlap > ; Jan > Beulich ; Julien Grall ; Stefano Stabellini > ; Anthony PERARD > Subject: [PAT

Re: [Xen-devel] [PATCH v5 2/2] x86: add accessors for scratch cpu mask

2020-03-12 Thread Roger Pau Monné
On Wed, Mar 11, 2020 at 05:20:23PM +0100, Jan Beulich wrote: > On 11.03.2020 16:51, Roger Pau Monné wrote: > > On Wed, Mar 11, 2020 at 04:37:50PM +0100, Jan Beulich wrote: > >> On 11.03.2020 16:34, Roger Pau Monné wrote: > >>> On Fri, Feb 28, 2020 at 01:42:58PM +0100, Jan Beulich wrote: > On 2

Re: [Xen-devel] [PATCH] x86/vvmx: Fix deadlock with MSR bitmap merging

2020-03-12 Thread Roger Pau Monné
On Wed, Mar 11, 2020 at 06:34:55PM +, Andrew Cooper wrote: > c/s c47984aabead "nvmx: implement support for MSR bitmaps" introduced a use of > map_domain_page() which may get used in the middle of context switch. > > This is not safe, and causes Xen to deadlock on the mapcache lock: > > (XEN

Re: [Xen-devel] [EXTERNAL][PATCH v5 6/6] domain: use PGC_extra domheap page for shared_info

2020-03-12 Thread Jan Beulich
On 12.03.2020 10:09, Paul Durrant wrote: >> -Original Message- >> From: Jan Beulich >> Sent: 12 March 2020 08:34 >> To: p...@xen.org >> Cc: xen-devel@lists.xenproject.org; Durrant, Paul ; >> 'Stefano Stabellini' >> ; 'Julien Grall' ; 'Volodymyr >> Babchuk' >> ; 'Andrew Cooper' ; >> 'Geo

Re: [Xen-devel] [PATCH] x86/vvmx: Fix deadlock with MSR bitmap merging

2020-03-12 Thread Jan Beulich
On 11.03.2020 19:34, Andrew Cooper wrote: > c/s c47984aabead "nvmx: implement support for MSR bitmaps" introduced a use of > map_domain_page() which may get used in the middle of context switch. > > This is not safe, and causes Xen to deadlock on the mapcache lock: > > (XEN) Xen call trace: >

Re: [Xen-devel] [EXTERNAL][PATCH v5 6/6] domain: use PGC_extra domheap page for shared_info

2020-03-12 Thread Paul Durrant
> -Original Message- > From: Jan Beulich > Sent: 12 March 2020 08:34 > To: p...@xen.org > Cc: xen-devel@lists.xenproject.org; Durrant, Paul ; > 'Stefano Stabellini' > ; 'Julien Grall' ; 'Volodymyr Babchuk' > ; 'Andrew Cooper' ; > 'George Dunlap' > ; 'Ian Jackson' ; > 'Konrad Rzeszutek W

[Xen-devel] [PATCH v9 02/10] scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()

2020-03-12 Thread Vladimir Sementsov-Ogievskiy
Script adds ERRP_AUTO_PROPAGATE macro invocation where appropriate and does corresponding changes in code (look for details in include/qapi/error.h) Usage example: spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \ --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \

[Xen-devel] [PATCH v9 10/10] xen: introduce ERRP_AUTO_PROPAGATE

2020-03-12 Thread Vladimir Sementsov-Ogievskiy
If we want to add some info to errp (by error_prepend() or error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro. Otherwise, this info will not be added when errp == &error_fatal (the program will exit prior to the error_append_hint() or error_prepend() call). Fix such cases. If we want

[Xen-devel] [PATCH v9 01/10] error: auto propagated local_err

2020-03-12 Thread Vladimir Sementsov-Ogievskiy
Introduce a new ERRP_AUTO_PROPAGATE macro, to be used at start of functions with an errp OUT parameter. It has three goals: 1. Fix issue with error_fatal and error_prepend/error_append_hint: user can't see this additional information, because exit() happens in error_setg earlier than information

[Xen-devel] [PATCH v9 00/10] error: auto propagated local_err part I

2020-03-12 Thread Vladimir Sementsov-Ogievskiy
v9 01: A lot of rewordings [thanks to Eric] Still, keep all r-b marks, assuming that they are mostly about macro definition 02: significant changes are: 1. Do not match double propagation pattern in ERRP_AUTO_PROPAGATE-adding rule 2. Introduce errp->->errp scheme to match only fun

Re: [Xen-devel] nvmx deadlock with MSR bitmaps

2020-03-12 Thread Jan Beulich
On 11.03.2020 19:04, Andrew Cooper wrote: > Specifically, this is a switch from an HVM vcpu, to a PV vcpu, where the > mapcache code tries to access the per-domain mappings on the HVM monitor > table.  It ends up trying to recursively acquire the mapcache lock while > trying to walk %cr2 to identif

[Xen-devel] [linux-linus test] 148417: regressions - FAIL

2020-03-12 Thread osstest service owner
flight 148417 linux-linus real [real] http://logs.test-lab.xenproject.org/osstest/logs/148417/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-amd64-amd64-qemuu-nested-intel 17 debian-hvm-install/l1/l2 fail REGR. vs. 133580 Regressions

Re: [Xen-devel] [PATCH v5 6/6] domain: use PGC_extra domheap page for shared_info

2020-03-12 Thread Jan Beulich
On 11.03.2020 16:28, Paul Durrant wrote: >> -Original Message- >> From: Jan Beulich >> Sent: 11 March 2020 09:17 >> To: p...@xen.org >> Cc: xen-devel@lists.xenproject.org; 'Paul Durrant' ; >> 'Stefano Stabellini' >> ; 'Julien Grall' ; 'Volodymyr >> Babchuk' >> ; 'Andrew Cooper' ; >> 'Ge

[Xen-devel] [PATCH v5 3/4] xen/rcu: add assertions to debug build

2020-03-12 Thread Juergen Gross
Xen's RCU implementation relies on no softirq handling taking place while being in a RCU critical section. Add ASSERT()s in debug builds in order to catch any violations. For that purpose modify rcu_read_[un]lock() to use a dedicated percpu counter instead of preempt_[en|dis]able() as this enables

[Xen-devel] [PATCH v5 1/4] xen/rcu: don't use stop_machine_run() for rcu_barrier()

2020-03-12 Thread Juergen Gross
Today rcu_barrier() is calling stop_machine_run() to synchronize all physical cpus in order to ensure all pending rcu calls have finished when returning. As stop_machine_run() is using tasklets this requires scheduling of idle vcpus on all cpus imposing the need to call rcu_barrier() on idle cpus

[Xen-devel] [PATCH v5 0/4] xen/rcu: let rcu work better with core scheduling

2020-03-12 Thread Juergen Gross
Today the RCU handling in Xen is affecting scheduling in several ways. It is raising sched softirqs without any real need and it requires tasklets for rcu_barrier(), which interacts badly with core scheduling. This small series repairs those issues. Additionally some ASSERT()s are added for verif

[Xen-devel] [PATCH v5 2/4] xen: don't process rcu callbacks when holding a rcu_read_lock()

2020-03-12 Thread Juergen Gross
Some keyhandlers are calling process_pending_softirqs() while holding a rcu_read_lock(). This is wrong, as process_pending_softirqs() might activate rcu calls which should not happen inside a rcu_read_lock(). For that purpose modify process_pending_softirqs() to not allow rcu callback processing w

[Xen-devel] [PATCH v5 4/4] xen/rcu: add per-lock counter in debug builds

2020-03-12 Thread Juergen Gross
Add a lock specific counter to rcu read locks in debug builds. This allows to test for matching lock/unlock calls. This will help to avoid cases like the one fixed by commit 98ed1f43cc2c89 where different rcu read locks were referenced in the lock and unlock calls. Signed-off-by: Juergen Gross -

Re: [Xen-devel] [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp

2020-03-12 Thread Vladimir Sementsov-Ogievskiy
12.03.2020 10:23, Markus Armbruster wrote: Vladimir Sementsov-Ogievskiy writes: 11.03.2020 17:41, Markus Armbruster wrote: Vladimir Sementsov-Ogievskiy writes: 11.03.2020 12:38, Markus Armbruster wrote: Vladimir Sementsov-Ogievskiy writes: 09.03.2020 12:56, Markus Armbruster wrote: Su

Re: [Xen-devel] [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp

2020-03-12 Thread Markus Armbruster
Vladimir Sementsov-Ogievskiy writes: > 11.03.2020 17:41, Markus Armbruster wrote: >> Vladimir Sementsov-Ogievskiy writes: >> >>> 11.03.2020 12:38, Markus Armbruster wrote: Vladimir Sementsov-Ogievskiy writes: > 09.03.2020 12:56, Markus Armbruster wrote: >> Suggest >> >