Re: Recent upgrade of 4.13 -> 4.14 issue

2020-10-26 Thread Jürgen Groß

On 26.10.20 17:31, Dario Faggioli wrote:

On Mon, 2020-10-26 at 15:30 +0100, Jürgen Groß wrote:

On 26.10.20 14:54, Andrew Cooper wrote:

On 26/10/2020 13:37, Frédéric Pierret wrote:


If anyone would have any idea of what's going on, that would be
very
appreciated. Thank you.


Does booting Xen with `sched=credit` make a difference?


Hmm, I think I have spotted a problem in credit2 which could explain
the
hang:

csched2_unit_wake() will NOT put the sched unit on a runqueue in case
it
has CSFLAG_scheduled set. This bit will be reset only in
csched2_context_saved().


Exactly, it does not put it back there. However, if it finds a vCPU
with the CSFLAG_scheduled flag set, It should set
CSFLAG_delayed_runq_add flag.

Unless curr_on_cpu(cpu)==unit or unit_on_runq(svc)==true... which
should not be the case. Or where you saying that we actually are in one
of this situations?

In fact...


So in case a vcpu (and its unit, of course) is blocked and there has
been no other vcpu active on its physical cpu but the idle vcpu,
there
will be no call of csched2_context_saved(). This will block the vcpu
to become active in theory for eternity, in case there is no need to
run another vcpu on the physical cpu.


...I maybe am not seeing what exact situation and sequence of events
you're exactly thinking to. What I see is this: [*]

- vCPU V is running, i.e., CSFLAG_scheduled is set
- vCPU V blocks
- we enter schedule()
   - schedule calls do_schedule() --> csched2_schedule()
 - we pick idle, so CSFLAG_delayed_runq_add is set for V
   - schedule calls sched_context_switch()
 - sched_context_switch() calls context_switch()
   - context_switch() calls sched_context_switched()
 - sched_context_switched() calls:
   - vcpu_context_saved()
   - unit_context_saved()
 - unit_context_saved() calls sched_context_saved() -->
   csched2_context_saved()
   - csched2_context_saved():
 - clears CSFLAG_scheduled
 - checks (and clear) CSFLAG_delayed_runq_add

[*] this assumes granularity 1, i.e., no core-scheduling and no
 rendezvous. Or was core-scheduling actually enabled?

And if CSFLAG_delayed_runq_add is set **and** the vCPU is runnable, the
task is added back to the runqueue.

So, even if we don't do the actual context switch (i.e., we don't call
__context_switch() ) if the next vCPU that we pick when vCPU V blocks
is the idle one, it looks to me that we go get to call
csched2_context_saved().

And it also looks to me that, when we get to that, if the vCPU is
runnable, even if it has the CSFLAG_scheduled still set, we do put it
back to the runqueue.

And if the vCPU blocked, but csched2_unit_wake() run while
CSFLAG_scheduled was still set, it indeed should mean that the vCPU
itself will be runnable again when we get to csched2_context_saved().

Or did you have something completely different in mind, and I'm missing
it?


No, I think you are right. I mixed that up with __context_switch() not
being called.

Sorry for the noise,


Juergen




[linux-linus test] 156247: regressions - FAIL

2020-10-26 Thread osstest service owner
flight 156247 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/156247/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-xsm7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ws16-amd64  7 xen-install   fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-intel  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 7 xen-install fail REGR. 
vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-intel  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow 7 xen-install fail REGR. vs. 
152332
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-libvirt-xsm   7 xen-install  fail REGR. vs. 152332
 test-amd64-coresched-i386-xl  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 
152332
 test-amd64-i386-qemut-rhel6hvm-amd  7 xen-installfail REGR. vs. 152332
 test-amd64-i386-xl-qemut-ws16-amd64  7 xen-install   fail REGR. vs. 152332
 test-amd64-i386-pair 10 xen-install/src_host fail REGR. vs. 152332
 test-amd64-i386-pair 11 xen-install/dst_host fail REGR. vs. 152332
 test-amd64-i386-libvirt   7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-amd  7 xen-installfail REGR. vs. 152332
 test-amd64-i386-xl7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-examine   6 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-raw7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-pvshim 7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-freebsd10-amd64  7 xen-install   fail REGR. vs. 152332
 test-amd64-i386-xl-shadow 7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-freebsd10-i386  7 xen-installfail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-win7-amd64  7 xen-install   fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-win7-amd64  7 xen-install   fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ovmf-amd64  7 xen-install   fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 10 xen-install/src_host fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 11 xen-install/dst_host fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 7 xen-install fail REGR. 
vs. 152332
 test-arm64-arm64-xl-xsm  12 debian-install   fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  12 debian-install   fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1   8 xen-boot fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm  8 xen-boot fail REGR. vs. 152332
 test-arm64-arm64-examine  8 reboot   fail REGR. vs. 152332
 test-amd64-amd64-amd64-pvgrub 20 guest-stop  fail REGR. vs. 152332
 test-amd64-amd64-i386-pvgrub 20 guest-stop   fail REGR. vs. 152332
 test-armhf-armhf-xl-credit1   8 xen-boot fail REGR. vs. 152332
 test-armhf-armhf-libvirt  8 xen-boot fail REGR. vs. 152332
 test-armhf-armhf-examine  8 reboot   fail REGR. vs. 152332
 test-armhf-armhf-xl-cubietruck  8 xen-boot   fail REGR. vs. 152332
 test-armhf-armhf-xl-multivcpu  8 xen-bootfail REGR. vs. 152332
 test-armhf-armhf-libvirt-raw  8 xen-boot fail REGR. vs. 152332
 test-arm64-arm64-xl-seattle   8 xen-boot fail REGR. vs. 152332
 test-armhf-armhf-xl   8 xen-boot fail REGR. vs. 152332
 test-armhf-armhf-xl-credit2   8 xen-boot fail REGR. vs. 152332
 test-armhf-armhf-xl-vhd   8 xen-boot fail REGR. vs. 152332

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds  8 xen-boot fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl  11 leak-check/basis(11)fail blocked in 152332
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stopfail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stopfail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stopfail like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stopfail like 152332
 test-amd64-amd64-libvirt 15 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-checkfail   never pass
 

Re: [PATCH] xen/arm: Remove EXPERT dependancy

2020-10-26 Thread Stefano Stabellini
On Mon, 26 Oct 2020, Julien Grall wrote:
> Hi Stefano,
> 
> On 23/10/2020 17:57, Stefano Stabellini wrote:
> > On Fri, 23 Oct 2020, Julien Grall wrote:
> > > Hi Stefano,
> > > 
> > > On 22/10/2020 22:17, Stefano Stabellini wrote:
> > > > On Thu, 22 Oct 2020, Julien Grall wrote:
> > > > > On 22/10/2020 02:43, Elliott Mitchell wrote:
> > > > > > Linux requires UEFI support to be enabled on ARM64 devices.  While
> > > > > > many
> > > > > > ARM64 devices lack ACPI, the writing seems to be on the wall of
> > > > > > UEFI/ACPI
> > > > > > potentially taking over.  Some common devices may need ACPI table
> > > > > > support.
> > > > > > 
> > > > > > Presently I think it is worth removing the dependancy on
> > > > > > CONFIG_EXPERT.
> > > > > 
> > > > > The idea behind EXPERT is to gate any feature that is not considered
> > > > > to be
> > > > > stable/complete enough to be used in production.
> > > > 
> > > > Yes, and from that point of view I don't think we want to remove EXPERT
> > > > from ACPI yet. However, the idea of hiding things behind EXPERT works
> > > > very well for new esoteric features, something like memory introspection
> > > > or memory overcommit.
> > > 
> > > Memaccess is not very new ;).
> > > 
> > > > It does not work well for things that are actually
> > > > required to boot on the platform.
> > > 
> > > I am not sure where is the problem. It is easy to select EXPERT from the
> > > menuconfig. It also hints the user that the feature may not fully work.
> > > 
> > > > 
> > > > Typically ACPI systems don't come with device tree at all (RPi4 being an
> > > > exception), so users don't really have much of a choice in the matter.
> > > 
> > > And they typically have IOMMUs.
> > > 
> > > > 
> > > >   From that point of view, it would be better to remove EXPERT from
> > > > ACPI,
> > > > maybe even build ACPI by default, *but* to add a warning at boot saying
> > > > something like:
> > > > 
> > > > "ACPI support is experimental. Boot using Device Tree if you can."
> > > > 
> > > > 
> > > > That would better convey the risks of using ACPI, while at the same time
> > > > making it a bit easier for users to boot on their ACPI-only platforms.
> > > 
> > > Right, I agree that this make easier for users to boot Xen on ACPI-only
> > > platform. However, based on above, it is easy enough for a developper to
> > > rebuild Xen with ACPI and EXPERT enabled.
> > > 
> > > So what sort of users are you targeting?
> > 
> > Somebody trying Xen for the first time, they might know how to build it
> > but they might not know that ACPI is not available by default, and they
> > might not know that they need to enable EXPERT in order to get the ACPI
> > option in the menu. It is easy to do once you know it is there,
> > otherwise one might not know where to look in the menu.
> 
> Right, EXPERT can now be enabled using Kconfig. So it is not very different
> from an option Foo has been hidden because the dependency Bar has not been
> selected.
> 
> It should be easy enough (if it is not we should fix it) to figure out the
> dependency when searching the option via menuconfig.

I do `make menuconfig` and there is no ACPI option. How do I even know
that ACPI has a kconfig option to enable? I'd assume that ACPI is always
enabled in the kconfig unless told otherwise.

But let's say that you know that you need to look for ACPI. I'd use the
search function, and it tells me:

  Symbol: ACPI [=n] 
 │  
  Type  : bool  
 │  
  Prompt: ACPI (Advanced Configuration and Power Interface) Support 
 │  
Location:   
 │  
  (1) -> Architecture Features  
 │  
Defined at arch/arm/Kconfig:34  
 │  
Depends on: ARM_64 [=y]
 
I go and look "Architecture Features" as told, but it is not there. How
do I need that I need to enable "Configure standard Xen features (expert
users)" to get that option?

 
> > > I am sort of okay to remove EXPERT.
> > 
> > OK. This would help (even without building it by default) because as you
> > go and look at the menu the first time, you'll find ACPI among the
> > options right away.
> 
> To be honest, this step is probably the easiest in the full process to get Xen
> build and booted on Arm.
> 
> I briefly looked at Elliot's v2, and I can't keep thinking that we are trying
> to re-invent EXPERT for ACPI because we think the feature is *more* important
> than any other feature 

[PATCH] fix swiotlb panic on Xen

2020-10-26 Thread Stefano Stabellini
From: Stefano Stabellini 

kernel/dma/swiotlb.c:swiotlb_init gets called first and tries to
allocate a buffer for the swiotlb. It does so by calling

  memblock_alloc_low(PAGE_ALIGN(bytes), PAGE_SIZE);

If the allocation must fail, no_iotlb_memory is set.


Later during initialization swiotlb-xen comes in
(drivers/xen/swiotlb-xen.c:xen_swiotlb_init) and given that io_tlb_start
is != 0, it thinks the memory is ready to use when actually it is not.

When the swiotlb is actually needed, swiotlb_tbl_map_single gets called
and since no_iotlb_memory is set the kernel panics.

Instead, if swiotlb-xen.c:xen_swiotlb_init knew the swiotlb hadn't been
initialized, it would do the initialization itself, which might still
succeed.


Fix the panic by setting io_tlb_start to 0 on swiotlb initialization
failure, and also by setting no_iotlb_memory to false on swiotlb
initialization success.

Signed-off-by: Stefano Stabellini 


diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index c19379fabd20..9924214df60a 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -231,6 +231,7 @@ int __init swiotlb_init_with_tbl(char *tlb, unsigned long 
nslabs, int verbose)
io_tlb_orig_addr[i] = INVALID_PHYS_ADDR;
}
io_tlb_index = 0;
+   no_iotlb_memory = false;
 
if (verbose)
swiotlb_print_info();
@@ -262,9 +263,11 @@ swiotlb_init(int verbose)
if (vstart && !swiotlb_init_with_tbl(vstart, io_tlb_nslabs, verbose))
return;
 
-   if (io_tlb_start)
+   if (io_tlb_start) {
memblock_free_early(io_tlb_start,
PAGE_ALIGN(io_tlb_nslabs << IO_TLB_SHIFT));
+   io_tlb_start = 0;
+   }
pr_warn("Cannot allocate buffer");
no_iotlb_memory = true;
 }
@@ -362,6 +365,7 @@ swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs)
io_tlb_orig_addr[i] = INVALID_PHYS_ADDR;
}
io_tlb_index = 0;
+   no_iotlb_memory = false;
 
swiotlb_print_info();
 



[PATCH AUTOSEL 5.8 054/132] xen: gntdev: fix common struct sg_table related issues

2020-10-26 Thread Sasha Levin
From: Marek Szyprowski 

[ Upstream commit d1749eb1ab85e04e58c29e58900e3abebbdd6e82 ]

The Documentation/DMA-API-HOWTO.txt states that the dma_map_sg() function
returns the number of the created entries in the DMA address space.
However the subsequent calls to the dma_sync_sg_for_{device,cpu}() and
dma_unmap_sg must be called with the original number of the entries
passed to the dma_map_sg().

struct sg_table is a common structure used for describing a non-contiguous
memory buffer, used commonly in the DRM and graphics subsystems. It
consists of a scatterlist with memory pages and DMA addresses (sgl entry),
as well as the number of scatterlist entries: CPU pages (orig_nents entry)
and DMA mapped pages (nents entry).

It turned out that it was a common mistake to misuse nents and orig_nents
entries, calling DMA-mapping functions with a wrong number of entries or
ignoring the number of mapped entries returned by the dma_map_sg()
function.

To avoid such issues, lets use a common dma-mapping wrappers operating
directly on the struct sg_table objects and use scatterlist page
iterators where possible. This, almost always, hides references to the
nents and orig_nents entries, making the code robust, easier to follow
and copy/paste safe.

Signed-off-by: Marek Szyprowski 
Acked-by: Juergen Gross 
Signed-off-by: Sasha Levin 
---
 drivers/xen/gntdev-dmabuf.c | 13 ++---
 1 file changed, 6 insertions(+), 7 deletions(-)

diff --git a/drivers/xen/gntdev-dmabuf.c b/drivers/xen/gntdev-dmabuf.c
index b1b6eebafd5de..4c13cbc99896a 100644
--- a/drivers/xen/gntdev-dmabuf.c
+++ b/drivers/xen/gntdev-dmabuf.c
@@ -247,10 +247,9 @@ static void dmabuf_exp_ops_detach(struct dma_buf *dma_buf,
 
if (sgt) {
if (gntdev_dmabuf_attach->dir != DMA_NONE)
-   dma_unmap_sg_attrs(attach->dev, sgt->sgl,
-  sgt->nents,
-  gntdev_dmabuf_attach->dir,
-  DMA_ATTR_SKIP_CPU_SYNC);
+   dma_unmap_sgtable(attach->dev, sgt,
+ gntdev_dmabuf_attach->dir,
+ DMA_ATTR_SKIP_CPU_SYNC);
sg_free_table(sgt);
}
 
@@ -288,8 +287,8 @@ dmabuf_exp_ops_map_dma_buf(struct dma_buf_attachment 
*attach,
sgt = dmabuf_pages_to_sgt(gntdev_dmabuf->pages,
  gntdev_dmabuf->nr_pages);
if (!IS_ERR(sgt)) {
-   if (!dma_map_sg_attrs(attach->dev, sgt->sgl, sgt->nents, dir,
- DMA_ATTR_SKIP_CPU_SYNC)) {
+   if (dma_map_sgtable(attach->dev, sgt, dir,
+   DMA_ATTR_SKIP_CPU_SYNC)) {
sg_free_table(sgt);
kfree(sgt);
sgt = ERR_PTR(-ENOMEM);
@@ -633,7 +632,7 @@ dmabuf_imp_to_refs(struct gntdev_dmabuf_priv *priv, struct 
device *dev,
 
/* Now convert sgt to array of pages and check for page validity. */
i = 0;
-   for_each_sg_page(sgt->sgl, _iter, sgt->nents, 0) {
+   for_each_sgtable_page(sgt, _iter, 0) {
struct page *page = sg_page_iter_page(_iter);
/*
 * Check if page is valid: this can happen if we are given
-- 
2.25.1




[PATCH AUTOSEL 5.9 062/147] xen: gntdev: fix common struct sg_table related issues

2020-10-26 Thread Sasha Levin
From: Marek Szyprowski 

[ Upstream commit d1749eb1ab85e04e58c29e58900e3abebbdd6e82 ]

The Documentation/DMA-API-HOWTO.txt states that the dma_map_sg() function
returns the number of the created entries in the DMA address space.
However the subsequent calls to the dma_sync_sg_for_{device,cpu}() and
dma_unmap_sg must be called with the original number of the entries
passed to the dma_map_sg().

struct sg_table is a common structure used for describing a non-contiguous
memory buffer, used commonly in the DRM and graphics subsystems. It
consists of a scatterlist with memory pages and DMA addresses (sgl entry),
as well as the number of scatterlist entries: CPU pages (orig_nents entry)
and DMA mapped pages (nents entry).

It turned out that it was a common mistake to misuse nents and orig_nents
entries, calling DMA-mapping functions with a wrong number of entries or
ignoring the number of mapped entries returned by the dma_map_sg()
function.

To avoid such issues, lets use a common dma-mapping wrappers operating
directly on the struct sg_table objects and use scatterlist page
iterators where possible. This, almost always, hides references to the
nents and orig_nents entries, making the code robust, easier to follow
and copy/paste safe.

Signed-off-by: Marek Szyprowski 
Acked-by: Juergen Gross 
Signed-off-by: Sasha Levin 
---
 drivers/xen/gntdev-dmabuf.c | 13 ++---
 1 file changed, 6 insertions(+), 7 deletions(-)

diff --git a/drivers/xen/gntdev-dmabuf.c b/drivers/xen/gntdev-dmabuf.c
index b1b6eebafd5de..4c13cbc99896a 100644
--- a/drivers/xen/gntdev-dmabuf.c
+++ b/drivers/xen/gntdev-dmabuf.c
@@ -247,10 +247,9 @@ static void dmabuf_exp_ops_detach(struct dma_buf *dma_buf,
 
if (sgt) {
if (gntdev_dmabuf_attach->dir != DMA_NONE)
-   dma_unmap_sg_attrs(attach->dev, sgt->sgl,
-  sgt->nents,
-  gntdev_dmabuf_attach->dir,
-  DMA_ATTR_SKIP_CPU_SYNC);
+   dma_unmap_sgtable(attach->dev, sgt,
+ gntdev_dmabuf_attach->dir,
+ DMA_ATTR_SKIP_CPU_SYNC);
sg_free_table(sgt);
}
 
@@ -288,8 +287,8 @@ dmabuf_exp_ops_map_dma_buf(struct dma_buf_attachment 
*attach,
sgt = dmabuf_pages_to_sgt(gntdev_dmabuf->pages,
  gntdev_dmabuf->nr_pages);
if (!IS_ERR(sgt)) {
-   if (!dma_map_sg_attrs(attach->dev, sgt->sgl, sgt->nents, dir,
- DMA_ATTR_SKIP_CPU_SYNC)) {
+   if (dma_map_sgtable(attach->dev, sgt, dir,
+   DMA_ATTR_SKIP_CPU_SYNC)) {
sg_free_table(sgt);
kfree(sgt);
sgt = ERR_PTR(-ENOMEM);
@@ -633,7 +632,7 @@ dmabuf_imp_to_refs(struct gntdev_dmabuf_priv *priv, struct 
device *dev,
 
/* Now convert sgt to array of pages and check for page validity. */
i = 0;
-   for_each_sg_page(sgt->sgl, _iter, sgt->nents, 0) {
+   for_each_sgtable_page(sgt, _iter, 0) {
struct page *page = sg_page_iter_page(_iter);
/*
 * Check if page is valid: this can happen if we are given
-- 
2.25.1




[qemu-mainline test] 156249: regressions - FAIL

2020-10-26 Thread osstest service owner
flight 156249 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/156249/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm   6 xen-buildfail REGR. vs. 152631
 build-amd64   6 xen-buildfail REGR. vs. 152631
 build-arm64-xsm   6 xen-buildfail REGR. vs. 152631
 build-i3866 xen-buildfail REGR. vs. 152631
 build-arm64   6 xen-buildfail REGR. vs. 152631
 build-i386-xsm6 xen-buildfail REGR. vs. 152631
 build-armhf   6 xen-buildfail REGR. vs. 152631

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-thunderx  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-arndale   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-credit1   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-rtds  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-vhd   1 build-check(1)   blocked  n/a
 test-amd64-i386-pair  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-i386  1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-amd64  1 build-check(1)   blocked  n/a
 test-amd64-coresched-i386-xl  1 build-check(1)   blocked  n/a
 test-amd64-coresched-amd64-xl  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-xsm   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-shadow1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-rtds  1 build-check(1)   blocked  n/a
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-ws16-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-win7-amd64  1 build-check(1) blocked n/a
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked 
n/a
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm  1 build-check(1) blocked n/a
 build-armhf-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow  1 build-check(1) blocked n/a
 build-i386-libvirt1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-amd64-amd64-amd64-pvgrub  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qcow2 1 build-check(1)   blocked  n/a
 test-amd64-amd64-dom0pvh-xl-amd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvshim1 build-check(1)   blocked  n/a
 test-amd64-amd64-dom0pvh-xl-intel  1 build-check(1)   blocked  n/a
 test-amd64-amd64-i386-pvgrub  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvhv2-intel  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-xl-pvhv2-amd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-amd64-amd64-pair 1 build-check(1)   blocked  n/a
 test-amd64-amd64-pygrub   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-credit2   1 build-check(1)   blocked  n/a
 test-amd64-amd64-qemuu-freebsd11-amd64  1 build-check(1)   blocked n/a
 test-amd64-amd64-qemuu-freebsd12-amd64  1 build-check(1)   blocked n/a
 test-amd64-amd64-xl-credit1   1 build-check(1)   blocked  n/a
 test-amd64-amd64-qemuu-nested-amd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-qemuu-nested-intel  1 

Re: Xen on RP4

2020-10-26 Thread Elliott Mitchell
On Mon, Oct 26, 2020 at 06:44:27PM +, Julien Grall wrote:
> Hi Elliott,
> 
> On 26/10/2020 16:03, Elliott Mitchell wrote:
> > On Mon, Oct 26, 2020 at 01:31:42PM +, Julien Grall wrote:
> >> On 24/10/2020 06:35, Elliott Mitchell wrote:
> >>> ACPI has a distinct
> >>> means of specifying a limited DMA-width; the above fails, because it
> >>> assumes a *device-tree*.
> >>
> >> Do you know if it would be possible to infer from the ACPI static table
> >> the DMA-width?
> > 
> > Yes, and it is.  Due to not knowing much about ACPI tables I don't know
> > what the C code would look like though (problem is which documentation
> > should I be looking at first?).
> 
> What you provided below is an excerpt of the DSDT. AFAIK, DSDT content 
> is written in AML. So far the shortest implementation I have seen for 
> the AML parser is around 5000 lines (see [1]). It might be possible to 
> strip some the code, although I think this will still probably too big 
> for a single workaround.
> 
> What I meant by "static table" is a table that looks like a structure 
> and can be parsed in a few lines. If we can't find on contain the DMA 
> window, then the next best solution is to find a way to identity the 
> platform.
> 
> I don't know enough ACPI to know if this solution is possible. A good 
> starter would probably be the ACPI spec [2].

Be assured, you likely know more about ACPI than I do.  :-)

A crucial point though is the mentions of dealing with DMA on the
Raspberry PI 4B using ACPI pointed at that "_DMA" string.  What is there
is Good Enough(tm) to make a 5.8 Linux kernel successfully operate
using ACPI.

Looking at the 5.8 source, apparently _DMA is an ACPI method.  That
almost looks straightforward enough for me to tackle for Xen...
Good news is looks like only supportting a single DMA window...


-- 
(\___(\___(\__  --=> 8-) EHM <=--  __/)___/)___/)
 \BS (| ehem+sig...@m5p.com  PGP 87145445 |)   /
  \_CS\   |  _  -O #include  O-   _  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445





[PATCH v1] libacpi: use temporary files for generated files

2020-10-26 Thread Olaf Hering
Use a temporay file, and move it in place once done.
The same pattern exists already for other dependencies.

Signed-off-by: Olaf Hering 
---
 tools/libacpi/Makefile | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/tools/libacpi/Makefile b/tools/libacpi/Makefile
index c17f3924cc..2cc4cc585b 100644
--- a/tools/libacpi/Makefile
+++ b/tools/libacpi/Makefile
@@ -43,7 +43,8 @@ all: $(C_SRC) $(H_SRC)
 
 $(H_SRC): $(ACPI_BUILD_DIR)/%.h: %.asl iasl
iasl -vs -p $(ACPI_BUILD_DIR)/$*.$(TMP_SUFFIX) -tc $<
-   sed -e 's/AmlCode/$*/g' -e 's/_aml_code//g' $(ACPI_BUILD_DIR)/$*.hex >$@
+   sed -e 's/AmlCode/$*/g' -e 's/_aml_code//g' $(ACPI_BUILD_DIR)/$*.hex 
>$@.$(TMP_SUFFIX)
+   mv -f $@.$(TMP_SUFFIX) $@
rm -f $(addprefix $(ACPI_BUILD_DIR)/, $*.aml $*.hex)
  
 $(MK_DSDT): mk_dsdt.c



Re: Recent upgrade of 4.13 -> 4.14 issue

2020-10-26 Thread Frédéric Pierret



Le 10/26/20 à 6:54 PM, Dario Faggioli a écrit :

On Mon, 2020-10-26 at 17:11 +0100, Frédéric Pierret wrote:

Le 10/26/20 à 2:54 PM, Andrew Cooper a écrit :

If anyone would have any idea of what's going on, that would be
very
appreciated. Thank you.


Does booting Xen with `sched=credit` make a difference?

~Andrew


Thank you Andrew. Since your mail I'm currently testing this on
production and it's clearly more stable than this morning. I will not
say yet it's solved because yesterday I had some few hours of
stability too. but clearly, it's encouraging because this morning it
was just hell every 15/30 minutes.


Ok, yes, let us know if the credit scheduler seems to not suffer from
the issue.



Yes unfortunately, I had few hours of stability but it just end up to:

```
[15883.967829] rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
[15883.967868] rcu: 12-...0: (75 ticks this GP) 
idle=5c6/1/0x4000 softirq=139356/139357 fqs=14879
[15883.967884]  (detected by 0, t=60002 jiffies, g=460221, q=89)
[15883.967901] Sending NMI from CPU 0 to CPUs 12:
[15893.970590] rcu: rcu_sched kthread starved for 9994 jiffies! g460221 f0x0 
RCU_GP_DOING_FQS(6) ->state=0x0 ->cpu=9
[15893.970622] rcu: RCU grace-period kthread stack dump:
[15893.970631] rcu_sched   R  running task010  2 0x80004008
[15893.970645] Call Trace:
[15893.970658]  ? xen_hypercall_xen_version+0xa/0x20
[15893.970670]  ? xen_force_evtchn_callback+0x9/0x10
[15893.970679]  ? check_events+0x12/0x20
[15893.970687]  ? xen_restore_fl_direct+0x1f/0x20
[15893.970697]  ? _raw_spin_unlock_irqrestore+0x14/0x20
[15893.970708]  ? force_qs_rnp+0x6f/0x170
[15893.970715]  ? rcu_nocb_unlock_irqrestore+0x30/0x30
[15893.970724]  ? rcu_gp_fqs_loop+0x234/0x2a0
[15893.970732]  ? rcu_gp_kthread+0xb5/0x140
[15893.970740]  ? rcu_gp_init+0x470/0x470
[15893.970748]  ? kthread+0x115/0x140
[15893.970756]  ? __kthread_bind_mask+0x60/0x60
[15893.970764]  ? ret_from_fork+0x35/0x40
[16063.972793] rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
[16063.972825] rcu: 12-...0: (75 ticks this GP) 
idle=5c6/1/0x4000 softirq=139356/139357 fqs=57364
[16063.972840]  (detected by 5, t=240007 jiffies, g=460221, q=6439)
[16063.972855] Sending NMI from CPU 5 to CPUs 12:
[16243.977769] rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
[16243.977802] rcu: 12-...0: (75 ticks this GP) 
idle=5c6/1/0x4000 softirq=139356/139357 fqs=99504
[16243.977817]  (detected by 11, t=420012 jiffies, g=460221, q=6710)
[16243.977830] Sending NMI from CPU 11 to CPUs 12:
[16253.980496] rcu: rcu_sched kthread starved for 10001 jiffies! g460221 f0x0 
RCU_GP_DOING_FQS(6) ->state=0x0 ->cpu=9
[16253.980528] rcu: RCU grace-period kthread stack dump:
[16253.980537] rcu_sched   R  running task010  2 0x80004008
[16253.980550] Call Trace:
[16253.980563]  ? xen_hypercall_xen_version+0xa/0x20
[16253.980575]  ? xen_force_evtchn_callback+0x9/0x10
[16253.980584]  ? check_events+0x12/0x20
[16253.980592]  ? xen_restore_fl_direct+0x1f/0x20
[16253.980602]  ? _raw_spin_unlock_irqrestore+0x14/0x20
[16253.980613]  ? force_qs_rnp+0x6f/0x170
[16253.980620]  ? rcu_nocb_unlock_irqrestore+0x30/0x30
[16253.980629]  ? rcu_gp_fqs_loop+0x234/0x2a0
[16253.980637]  ? rcu_gp_kthread+0xb5/0x140
[16253.980645]  ? rcu_gp_init+0x470/0x470
[16253.980653]  ? kthread+0x115/0x140
[16253.980661]  ? __kthread_bind_mask+0x60/0x60
[16253.980669]  ? ret_from_fork+0x35/0x40
[16423.982735] rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
[16423.982789] rcu: 12-...0: (75 ticks this GP) 
idle=5c6/1/0x4000 softirq=139356/139357 fqs=139435
[16423.982820]  (detected by 10, t=600017 jiffies, g=460221, q=7354)
[16423.982842] Sending NMI from CPU 10 to CPUs 12:
[16433.984844] rcu: rcu_sched kthread starved for 10001 jiffies! g460221 f0x0 
RCU_GP_DOING_FQS(6) ->state=0x0 ->cpu=3
[16433.984875] rcu: RCU grace-period kthread stack dump:
[16433.984885] rcu_sched   R  running task010  2 0x80004000
[16433.984897] Call Trace:
[16433.984910]  ? xen_hypercall_xen_version+0xa/0x20
[16433.984922]  ? xen_force_evtchn_callback+0x9/0x10
[16433.984931]  ? check_events+0x12/0x20
[16433.984939]  ? xen_restore_fl_direct+0x1f/0x20
[16433.984949]  ? _raw_spin_unlock_irqrestore+0x14/0x20
[16433.984960]  ? force_qs_rnp+0x6f/0x170
[16433.984967]  ? rcu_nocb_unlock_irqrestore+0x30/0x30
[16433.984976]  ? rcu_gp_fqs_loop+0x234/0x2a0
[16433.984984]  ? rcu_gp_kthread+0xb5/0x140
[16433.984992]  ? rcu_gp_init+0x470/0x470
[16433.985000]  ? kthread+0x115/0x140
[16433.985007]  ? __kthread_bind_mask+0x60/0x60
[16433.985015]  ? ret_from_fork+0x35/0x40
[16603.987677] rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
[16603.987710] rcu: 12-...0: (75 ticks this GP) 
idle=5c6/1/0x4000 softirq=139356/139357 fqs=179313
[16603.987725]  (detected by 0, t=780022 jiffies, g=460221, q=7869)
[16603.987740] Sending NMI from CPU 0 to CPUs 12:
[16783.992658] rcu: INFO: 

Re: [XEN PATCH v1] xen/arm : Add support for SMMUv3 driver

2020-10-26 Thread Julien Grall

On 26/10/2020 12:10, Ash Wilding wrote:

Hi,


Hi Ash,


1. atomic_set_release
2. atomic_fetch_andnot_relaxed
3. atomic_cond_read_relaxed
4. atomic_long_cond_read_relaxed
5. atomic_long_xor
6. atomic_set_release
7. atomic_cmpxchg_relaxed might be we can use atomic_cmpxchg that is
implemented in XEN need to check.
8. atomic_dec_return_release
9. atomic_fetch_inc_relaxed



If we're going to pull in Linux's implementations of the above atomics
helpers for SMMUv3, and given the majority of SMMUv3 systems are v8.1+
with LSE, perhaps this would be a good time to drop the current
atomic.h in Xen completely and pull in both Linux's LL/SC and LSE
helpers,


When I originally answered to the thread, I thought about suggesting 
importing LSE. But I felt it was too much to ask in order to merge the 
SMMUv3 code.


However, I would love to have support for LSE in Xen as this would solve 
other not yet fully closed security issue with LL/SC (see XSA-295 [2]).


Would Arm be willing to add support for LSE before merging the SMMUv3?

As an alternative, it might also be possible to provide "dumb" 
implementation for all the helpers even if they are stricter than 
necessary for the memory ordering requirements.


then use a new Kconfig to toggle between them?

I would prefer to follow the same approach as Linux and allow Xen to 
select at boot time which implementations to use. This would enable 
distro to provide a single binary that boot on all Armv8 and still allow 
Xen to select the best set of instructions.


Xen already provides a framework to switch between two sets of 
instructions at boot. This was borrowed from Linux, so I don't expect a 
big hurdle to get this supported.




Back in 5d45ecabe3 [1] Jan mentioned we probably want to avoid relying
on gcc atomics helpers as we can't switch between LL/SC and LSE
atomics. 


I asked Jan to add this line in the commit message :). My concern was 
that even if we provided a runtime switch (or sanity check for XSA-295), 
the GCC helpers would not be able to take advantage (the code is not 
written by Xen community).


Cheers,


[1] https://xenbits.xen.org/gitweb/?p=xen.git;a=commit;h=5d45ecabe3


[2] https://xenbits.xen.org/xsa/advisory-295.html






--
Julien Grall



Re: Xen on RP4

2020-10-26 Thread Julien Grall

Hi Elliott,

On 26/10/2020 16:03, Elliott Mitchell wrote:

On Mon, Oct 26, 2020 at 01:31:42PM +, Julien Grall wrote:

On 24/10/2020 06:35, Elliott Mitchell wrote:

ACPI has a distinct
means of specifying a limited DMA-width; the above fails, because it
assumes a *device-tree*.


Do you know if it would be possible to infer from the ACPI static table
the DMA-width?


Yes, and it is.  Due to not knowing much about ACPI tables I don't know
what the C code would look like though (problem is which documentation
should I be looking at first?).


What you provided below is an excerpt of the DSDT. AFAIK, DSDT content 
is written in AML. So far the shortest implementation I have seen for 
the AML parser is around 5000 lines (see [1]). It might be possible to 
strip some the code, although I think this will still probably too big 
for a single workaround.


What I meant by "static table" is a table that looks like a structure 
and can be parsed in a few lines. If we can't find on contain the DMA 
window, then the next best solution is to find a way to identity the 
platform.


I don't know enough ACPI to know if this solution is possible. A good 
starter would probably be the ACPI spec [2].




Handy bit of information is in the RP4 Tianocore table source:
https://github.com/tianocore/edk2-platforms/blob/d492639638eee331ac3389e6cf53ea266c3c84b3/Platform/RaspberryPi/AcpiTables/Dsdt.asl

   Name (_DMA, ResourceTemplate() {
 //
 // Only the first GB is available.
 // Bus 0xC000 -> CPU 0x.
 //
 QWordMemory (ResourceConsumer,
   ,
   MinFixed,
   MaxFixed,
   NonCacheable,
   ReadWrite,
   0x0,
   0xC000, // MIN
   0x, // MAX
   0x4000, // TRA
   0x4000, // LEN
   ,
   ,
   )
   })

There should be some corresponding code in the Linux 5.9 kernels.  From
the look of that, it might even be possible to specify a memory range
which didn't start at address 0.




Cheers,

[1] https://github.com/openbsd/src/blob/master/sys/dev/acpi/dsdt.c
[2] https://www.uefi.org/sites/default/files/resources/ACPI_6.0.pdf

--
Julien Grall



[qemu-mainline test] 156246: regressions - FAIL

2020-10-26 Thread osstest service owner
flight 156246 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/156246/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm   6 xen-buildfail REGR. vs. 152631
 build-amd64   6 xen-buildfail REGR. vs. 152631
 build-arm64-xsm   6 xen-buildfail REGR. vs. 152631
 build-i3866 xen-buildfail REGR. vs. 152631
 build-arm64   6 xen-buildfail REGR. vs. 152631
 build-i386-xsm6 xen-buildfail REGR. vs. 152631
 build-armhf   6 xen-buildfail REGR. vs. 152631

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-thunderx  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-arndale   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-credit1   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-rtds  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-vhd   1 build-check(1)   blocked  n/a
 test-amd64-i386-pair  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-i386  1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-amd64  1 build-check(1)   blocked  n/a
 test-amd64-coresched-i386-xl  1 build-check(1)   blocked  n/a
 test-amd64-coresched-amd64-xl  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-xsm   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-shadow1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-rtds  1 build-check(1)   blocked  n/a
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-ws16-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-win7-amd64  1 build-check(1) blocked n/a
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked 
n/a
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm  1 build-check(1) blocked n/a
 build-armhf-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow  1 build-check(1) blocked n/a
 build-i386-libvirt1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-amd64-amd64-amd64-pvgrub  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qcow2 1 build-check(1)   blocked  n/a
 test-amd64-amd64-dom0pvh-xl-amd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvshim1 build-check(1)   blocked  n/a
 test-amd64-amd64-dom0pvh-xl-intel  1 build-check(1)   blocked  n/a
 test-amd64-amd64-i386-pvgrub  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvhv2-intel  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-xl-pvhv2-amd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-amd64-amd64-pair 1 build-check(1)   blocked  n/a
 test-amd64-amd64-pygrub   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-credit2   1 build-check(1)   blocked  n/a
 test-amd64-amd64-qemuu-freebsd11-amd64  1 build-check(1)   blocked n/a
 test-amd64-amd64-qemuu-freebsd12-amd64  1 build-check(1)   blocked n/a
 test-amd64-amd64-xl-credit1   1 build-check(1)   blocked  n/a
 test-amd64-amd64-qemuu-nested-amd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-qemuu-nested-intel  1 

[xen-unstable-smoke test] 156245: tolerable all pass - PUSHED

2020-10-26 Thread osstest service owner
flight 156245 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/156245/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt 15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  16 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  964781c6f162893677c50a779b7d562a299727ba
baseline version:
 xen  6ca70821b59849ad97c3fadc47e63c1a4af1a78c

Last test of basis   156117  2020-10-23 09:01:23 Z3 days
Failing since156120  2020-10-23 14:01:24 Z3 days   38 attempts
Testing same since   156245  2020-10-26 16:01:22 Z0 days1 attempts


People who touched revisions under test:
  Andrew Cooper 
  Bertrand Marquis 
  Christian Lindig 
  George Dunlap 
  Ian Jackson 
  Ian Jackson 
  Jan Beulich 
  Jason Andryuk 
  Juergen Gross 
  Julien Grall 
  Wei Liu 

jobs:
 build-arm64-xsm  pass
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  pass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   6ca70821b5..964781c6f1  964781c6f162893677c50a779b7d562a299727ba -> smoke



[linux-linus test] 156238: regressions - FAIL

2020-10-26 Thread osstest service owner
flight 156238 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/156238/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-xsm7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ws16-amd64  7 xen-install   fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-intel  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 7 xen-install fail REGR. 
vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-intel  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow 7 xen-install fail REGR. vs. 
152332
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-libvirt-xsm   7 xen-install  fail REGR. vs. 152332
 test-amd64-coresched-i386-xl  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 
152332
 test-amd64-i386-qemut-rhel6hvm-amd  7 xen-installfail REGR. vs. 152332
 test-amd64-i386-xl-qemut-ws16-amd64  7 xen-install   fail REGR. vs. 152332
 test-amd64-i386-pair 10 xen-install/src_host fail REGR. vs. 152332
 test-amd64-i386-pair 11 xen-install/dst_host fail REGR. vs. 152332
 test-amd64-i386-libvirt   7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-amd  7 xen-installfail REGR. vs. 152332
 test-amd64-i386-xl7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-examine   6 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-raw7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-pvshim 7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-freebsd10-amd64  7 xen-install   fail REGR. vs. 152332
 test-amd64-i386-xl-shadow 7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-freebsd10-i386  7 xen-installfail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-win7-amd64  7 xen-install   fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-win7-amd64  7 xen-install   fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ovmf-amd64  7 xen-install   fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 10 xen-install/src_host fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 11 xen-install/dst_host fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 7 xen-install fail REGR. 
vs. 152332
 test-arm64-arm64-xl  10 host-ping-check-xen  fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  12 debian-install   fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  10 host-ping-check-xen  fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm  8 xen-boot fail REGR. vs. 152332
 test-arm64-arm64-examine 13 examine-iommufail REGR. vs. 152332
 test-amd64-amd64-amd64-pvgrub 20 guest-stop  fail REGR. vs. 152332
 test-amd64-amd64-i386-pvgrub 20 guest-stop   fail REGR. vs. 152332
 test-armhf-armhf-xl-credit1   8 xen-boot fail REGR. vs. 152332
 test-armhf-armhf-libvirt  8 xen-boot fail REGR. vs. 152332
 test-armhf-armhf-examine  8 reboot   fail REGR. vs. 152332
 test-armhf-armhf-xl-cubietruck  8 xen-boot   fail REGR. vs. 152332
 test-armhf-armhf-xl-multivcpu  8 xen-bootfail REGR. vs. 152332
 test-armhf-armhf-libvirt-raw  8 xen-boot fail REGR. vs. 152332
 test-armhf-armhf-xl   8 xen-boot fail REGR. vs. 152332
 test-armhf-armhf-xl-credit2   8 xen-boot fail REGR. vs. 152332
 test-armhf-armhf-xl-vhd   8 xen-boot fail REGR. vs. 152332
 test-arm64-arm64-xl-xsm 10 host-ping-check-xen fail in 156225 REGR. vs. 152332

Tests which are failing intermittently (not blocking):
 test-arm64-arm64-xl   8 xen-boot fail in 156225 pass in 156238
 test-arm64-arm64-xl-credit1   8 xen-boot fail in 156225 pass in 156238
 test-amd64-amd64-xl-rtds   18 guest-localmigrate fail in 156225 pass in 156238
 test-arm64-arm64-examine  8 reboot   fail in 156225 pass in 156238
 test-amd64-amd64-examine4 memdisk-try-append fail in 156225 pass in 156238
 test-amd64-amd64-i386-pvgrub 19 guest-localmigrate/x10 fail in 156225 pass in 
156238
 test-arm64-arm64-xl-xsm   8 xen-boot   fail pass in 156225

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-rtds 20 guest-localmigrate/x10   fail REGR. vs. 152332
 test-armhf-armhf-xl-rtds  8 xen-boot fail REGR. vs. 

Re: Recent upgrade of 4.13 -> 4.14 issue

2020-10-26 Thread Dario Faggioli
On Mon, 2020-10-26 at 17:11 +0100, Frédéric Pierret wrote:
> Le 10/26/20 à 2:54 PM, Andrew Cooper a écrit :
> > > If anyone would have any idea of what's going on, that would be
> > > very
> > > appreciated. Thank you.
> > 
> > Does booting Xen with `sched=credit` make a difference?
> > 
> > ~Andrew
> 
> Thank you Andrew. Since your mail I'm currently testing this on
> production and it's clearly more stable than this morning. I will not
> say yet it's solved because yesterday I had some few hours of
> stability too. but clearly, it's encouraging because this morning it
> was just hell every 15/30 minutes.
> 
Ok, yes, let us know if the credit scheduler seems to not suffer from
the issue.

I'm curious about another thing, though. You mentioned, in your
previous email (and in the subject :-)) that this is a 4.13 -> 4.14
issue for you?

Does that mean that the problem was not there on 4.13?

I'm asking because Credit2 was already the default scheduler in 4.13. 

So, unless you were configuring things differently, you were already
using it there.

If this is the case, it would hint at the fact that something that
changed between .13 and .14 could be the cause.

Regards
-- 
Dario Faggioli, Ph.D
http://about.me/dario.faggioli
Virtualization Software Engineer
SUSE Labs, SUSE https://www.suse.com/
---
<> (Raistlin Majere)


signature.asc
Description: This is a digitally signed message part


Re: [Xen-devel] [PATCH] xen: credit2: document that min_rqd is valid and ok to use

2020-10-26 Thread Dario Faggioli
On Mon, 2020-10-26 at 10:43 +, George Dunlap wrote:
> On Thu, Mar 26, 2020 at 5:09 PM Dario Faggioli 
> wrote:
> > diff --git a/xen/common/sched/credit2.c
> > b/xen/common/sched/credit2.c
> > index c7241944a8..9da51e624b 100644
> > --- a/xen/common/sched/credit2.c
> > +++ b/xen/common/sched/credit2.c
> > @@ -2387,6 +2387,13 @@ csched2_res_pick(const struct scheduler
> > *ops, const struct sched_unit *unit)
> >          goto out_up;
> >      }
> > 
> > +    /*
> > +     * If we're here, min_rqd must be valid. In fact, either we
> > picked a
> > +     * runqueue in the "list_for_each" (as min_avgload is
> > initialized to
> > +     * MAX_LOAD) or we just did that (in the "else" branch) above.
> > +     */
> 
> 
> Sorry it's taken so long to get back to you on this.
> 
> The problem with this is that there are actually *three* alternate
> clauses above:
> 
> 1. (has_soft && min_s_rqd)
> 2. min_rqd
> 3. 
> 
Yes, indeed.

However, one of the three is "if (min_rqs)", and I think it is clear
that in that case (which would be 2 in the list above) min_rqd is
valid.

Therefore, this part of the comment "In fact, either we picked a
runqueue in the "list_for_each" (as min_avgload is initialized to
MAX_LOAD)", was referring to 1.

And this other part "or we just did that (in the "else" branch) above",
was referring to 3.

> It's obvious that if we hit #2 or #3, that min_rqd will be set.  But
> it's not immediately obvious why the condition in #1 guarantees that
> min_rqd will be set.
> 
That's what I tried to explain with this: "we picked a runqueue in the
"list_for_each" (as min_avgload is initialized to MAX_LOAD)"

> Is it because if we get to the point in the above loop where
> min_s_rqd is set, then min_rqd will always be set if it hasn't been
> set already?  Or to put it a different way -- the only way for
> min_rqd *not* to be set is if it always bailed before min_s_rqd was
> set?
> 
The point is really that the "list_for_each" loop scans all the
runqueues. If we do at least one step of the loop, min_rqd is ok,
because min_avgload is initialized to MAX_LOAD, and hence we have done
at least one assignment of min_rq=rqd (in the body of the very last if
of the loop itself).

min_s_rqd may or may not have been set to point to any runqueue. But if
it is valid, it means we have done at least one step of the loop, and
hence min_rqd is valid too.

Makes sense? :-)

Regards
-- 
Dario Faggioli, Ph.D
http://about.me/dario.faggioli
Virtualization Software Engineer
SUSE Labs, SUSE https://www.suse.com/
---
<> (Raistlin Majere)


signature.asc
Description: This is a digitally signed message part


[PATCH 3/3] x86/ucode: Introduce ucode=allow-same for testing purposes

2020-10-26 Thread Andrew Cooper
Many CPUs will actually reload microcode when offered the same version as
currently loaded.  This allows for easy testing of the late microcode loading
path.

Signed-off-by: Andrew Cooper 
---
CC: Jan Beulich 
CC: Roger Pau Monné 
CC: Wei Liu 
CC: Juergen Gross 
CC: Igor Druzhinin 

I was hoping to make this a runtime parameter, but I honestly can't figure out
the new HYPFS-only infrastructure is supposed to work.
---
 docs/misc/xen-command-line.pandoc| 7 ++-
 xen/arch/x86/cpu/microcode/amd.c | 3 +++
 xen/arch/x86/cpu/microcode/core.c| 4 
 xen/arch/x86/cpu/microcode/intel.c   | 3 +++
 xen/arch/x86/cpu/microcode/private.h | 2 ++
 5 files changed, 18 insertions(+), 1 deletion(-)

diff --git a/docs/misc/xen-command-line.pandoc 
b/docs/misc/xen-command-line.pandoc
index 4ae9391fcd..97b1cc67a4 100644
--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -2216,7 +2216,7 @@ logic applies:
active by default.
 
 ### ucode
-> `= List of [  | scan=, nmi= ]`
+> `= List of [  | scan=, nmi=, allow-same= ]`
 
 Applicability: x86
 Default: `nmi`
@@ -2248,6 +2248,11 @@ precedence over `scan`.
 stop_machine context. In NMI handler, even NMIs are blocked, which is
 considered safer. The default value is `true`.
 
+'allow-same' alters the default acceptance policy for new microcode to permit
+trying to reload the same version.  Many CPUs will actually reload microcode
+of the same version, and this allows for easily testing of the late microcode
+loading path.
+
 ### unrestricted_guest (Intel)
 > `= `
 
diff --git a/xen/arch/x86/cpu/microcode/amd.c b/xen/arch/x86/cpu/microcode/amd.c
index 7d2f57c4cb..5255028af7 100644
--- a/xen/arch/x86/cpu/microcode/amd.c
+++ b/xen/arch/x86/cpu/microcode/amd.c
@@ -174,6 +174,9 @@ static enum microcode_match_result compare_revisions(
 if ( new_rev > old_rev )
 return NEW_UCODE;
 
+if ( opt_ucode_allow_same && new_rev == old_rev )
+return NEW_UCODE;
+
 return OLD_UCODE;
 }
 
diff --git a/xen/arch/x86/cpu/microcode/core.c 
b/xen/arch/x86/cpu/microcode/core.c
index 18ebc07b13..ac3ceb567c 100644
--- a/xen/arch/x86/cpu/microcode/core.c
+++ b/xen/arch/x86/cpu/microcode/core.c
@@ -95,6 +95,8 @@ static bool_t __initdata ucode_scan;
 /* By default, ucode loading is done in NMI handler */
 static bool ucode_in_nmi = true;
 
+bool __read_mostly opt_ucode_allow_same;
+
 /* Protected by microcode_mutex */
 static struct microcode_patch *microcode_cache;
 
@@ -121,6 +123,8 @@ static int __init parse_ucode(const char *s)
 
 if ( (val = parse_boolean("nmi", s, ss)) >= 0 )
 ucode_in_nmi = val;
+else if ( (val = parse_boolean("allow-same", s, ss)) >= 0 )
+opt_ucode_allow_same = val;
 else if ( !ucode_mod_forced ) /* Not forced by EFI */
 {
 if ( (val = parse_boolean("scan", s, ss)) >= 0 )
diff --git a/xen/arch/x86/cpu/microcode/intel.c 
b/xen/arch/x86/cpu/microcode/intel.c
index 5fa2821cdb..f6d01490e0 100644
--- a/xen/arch/x86/cpu/microcode/intel.c
+++ b/xen/arch/x86/cpu/microcode/intel.c
@@ -232,6 +232,9 @@ static enum microcode_match_result compare_revisions(
 if ( new_rev > old_rev )
 return NEW_UCODE;
 
+if ( opt_ucode_allow_same && new_rev == old_rev )
+return NEW_UCODE;
+
 /*
  * Treat pre-production as always applicable - anyone using pre-production
  * microcode knows what they are doing, and can keep any resulting pieces.
diff --git a/xen/arch/x86/cpu/microcode/private.h 
b/xen/arch/x86/cpu/microcode/private.h
index 9a15cdc879..c085a10268 100644
--- a/xen/arch/x86/cpu/microcode/private.h
+++ b/xen/arch/x86/cpu/microcode/private.h
@@ -3,6 +3,8 @@
 
 #include 
 
+extern bool opt_ucode_allow_same;
+
 enum microcode_match_result {
 OLD_UCODE, /* signature matched, but revision id is older or equal */
 NEW_UCODE, /* signature matched, but revision id is newer */
-- 
2.11.0




[PATCH 2/3] x86/ucode/intel: Fix handling of microcode revision

2020-10-26 Thread Andrew Cooper
For Intel microcodes, the revision field is signed (as documented in the SDM)
and negative revisions are used for pre-production/test microcode (not
documented publicly anywhere I can spot).

Adjust the revision checking to match the algorithm presented here:

  
https://software.intel.com/security-software-guidance/best-practices/microcode-update-guidance

This treats pre-production microcode as always applicable, but also production
microcode having higher precident than pre-production.  It is expected that
anyone using pre-production microcode knows what they are doing.

This is necessary to load production microcode on an SDP with pre-production
microcode embedded in firmware.

Signed-off-by: Andrew Cooper 
---
CC: Jan Beulich 
CC: Roger Pau Monné 
CC: Wei Liu 
CC: Juergen Gross 
CC: Igor Druzhinin 

"signed" is somewhat of a problem.  The actual numbers only make sense as
sign-magnitude, and not as twos-compliement, which is why the rule is "any
debug microcode goes".

The actual upgrade/downgrade rules are quite complicated, but in general, any
change is permitted so long as the Security Version Number (embedded inside
the patch) doesn't go backwards.

---
 xen/arch/x86/cpu/microcode/intel.c | 15 +--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/xen/arch/x86/cpu/microcode/intel.c 
b/xen/arch/x86/cpu/microcode/intel.c
index e1ccb5d232..5fa2821cdb 100644
--- a/xen/arch/x86/cpu/microcode/intel.c
+++ b/xen/arch/x86/cpu/microcode/intel.c
@@ -33,7 +33,7 @@
 
 struct microcode_patch {
 uint32_t hdrver;
-uint32_t rev;
+int32_t rev;
 uint16_t year;
 uint8_t  day;
 uint8_t  month;
@@ -222,12 +222,23 @@ static int microcode_sanity_check(const struct 
microcode_patch *patch)
 return 0;
 }
 
+/*
+ * Production microcode has a positive revision.  Pre-production microcode has
+ * a negative revision.
+ */
 static enum microcode_match_result compare_revisions(
-uint32_t old_rev, uint32_t new_rev)
+int32_t old_rev, int32_t new_rev)
 {
 if ( new_rev > old_rev )
 return NEW_UCODE;
 
+/*
+ * Treat pre-production as always applicable - anyone using pre-production
+ * microcode knows what they are doing, and can keep any resulting pieces.
+ */
+if ( new_rev < 0 )
+return NEW_UCODE;
+
 return OLD_UCODE;
 }
 
-- 
2.11.0




[PATCH 0/3] x86/ucode: Fixes and improvements to ucode revision handling

2020-10-26 Thread Andrew Cooper
Patch 2 fixes a bug with the Intel revision handling, which is causing
problems on IceLake SDPs.

Patch 3 adds ucode=allow-same to allow for sensible testing of the late
microcode loading path.

Andrew Cooper (3):
  x86/ucode: Break out compare_revisions() from existing infrastructure
  x86/ucode/intel: Fix handling of microcode revision
  x86/ucode: Introduce ucode=allow-same for testing purposes

 docs/misc/xen-command-line.pandoc|  7 ++-
 xen/arch/x86/cpu/microcode/amd.c | 25 ++---
 xen/arch/x86/cpu/microcode/core.c|  4 
 xen/arch/x86/cpu/microcode/intel.c   | 31 +++
 xen/arch/x86/cpu/microcode/private.h |  2 ++
 5 files changed, 53 insertions(+), 16 deletions(-)

-- 
2.11.0




[PATCH 1/3] x86/ucode: Break out compare_revisions() from existing infrastructure

2020-10-26 Thread Andrew Cooper
Drop some unnecesserily verbose pr_debug()'s on the AMD side.

No functional change.

Signed-off-by: Andrew Cooper 
---
CC: Jan Beulich 
CC: Roger Pau Monné 
CC: Wei Liu 
CC: Juergen Gross 
CC: Igor Druzhinin 
---
 xen/arch/x86/cpu/microcode/amd.c   | 22 +++---
 xen/arch/x86/cpu/microcode/intel.c | 15 ---
 2 files changed, 23 insertions(+), 14 deletions(-)

diff --git a/xen/arch/x86/cpu/microcode/amd.c b/xen/arch/x86/cpu/microcode/amd.c
index e80f7cd3e4..7d2f57c4cb 100644
--- a/xen/arch/x86/cpu/microcode/amd.c
+++ b/xen/arch/x86/cpu/microcode/amd.c
@@ -168,6 +168,15 @@ static bool check_final_patch_levels(const struct 
cpu_signature *sig)
 return false;
 }
 
+static enum microcode_match_result compare_revisions(
+uint32_t old_rev, uint32_t new_rev)
+{
+if ( new_rev > old_rev )
+return NEW_UCODE;
+
+return OLD_UCODE;
+}
+
 static enum microcode_match_result microcode_fits(
 const struct microcode_patch *patch)
 {
@@ -178,16 +187,7 @@ static enum microcode_match_result microcode_fits(
  equiv.id  != patch->processor_rev_id )
 return MIS_UCODE;
 
-if ( patch->patch_id <= sig->rev )
-{
-pr_debug("microcode: patch is already at required level or 
greater.\n");
-return OLD_UCODE;
-}
-
-pr_debug("microcode: CPU%d found a matching microcode update with version 
%#x (current=%#x)\n",
- cpu, patch->patch_id, sig->rev);
-
-return NEW_UCODE;
+return compare_revisions(sig->rev, patch->patch_id);
 }
 
 static enum microcode_match_result compare_header(
@@ -196,7 +196,7 @@ static enum microcode_match_result compare_header(
 if ( new->processor_rev_id != old->processor_rev_id )
 return MIS_UCODE;
 
-return new->patch_id > old->patch_id ? NEW_UCODE : OLD_UCODE;
+return compare_revisions(old->patch_id, new->patch_id);
 }
 
 static enum microcode_match_result compare_patch(
diff --git a/xen/arch/x86/cpu/microcode/intel.c 
b/xen/arch/x86/cpu/microcode/intel.c
index 72c07fcd1d..e1ccb5d232 100644
--- a/xen/arch/x86/cpu/microcode/intel.c
+++ b/xen/arch/x86/cpu/microcode/intel.c
@@ -222,6 +222,15 @@ static int microcode_sanity_check(const struct 
microcode_patch *patch)
 return 0;
 }
 
+static enum microcode_match_result compare_revisions(
+uint32_t old_rev, uint32_t new_rev)
+{
+if ( new_rev > old_rev )
+return NEW_UCODE;
+
+return OLD_UCODE;
+}
+
 /* Check an update against the CPU signature and current update revision */
 static enum microcode_match_result microcode_update_match(
 const struct microcode_patch *mc)
@@ -245,7 +254,7 @@ static enum microcode_match_result microcode_update_match(
 return MIS_UCODE;
 
  found:
-return mc->rev > cpu_sig->rev ? NEW_UCODE : OLD_UCODE;
+return compare_revisions(cpu_sig->rev, mc->rev);
 }
 
 static enum microcode_match_result compare_patch(
@@ -258,7 +267,7 @@ static enum microcode_match_result compare_patch(
 ASSERT(microcode_update_match(old) != MIS_UCODE);
 ASSERT(microcode_update_match(new) != MIS_UCODE);
 
-return new->rev > old->rev ? NEW_UCODE : OLD_UCODE;
+return compare_revisions(old->rev, new->rev);
 }
 
 static int apply_microcode(const struct microcode_patch *patch)
@@ -332,7 +341,7 @@ static struct microcode_patch *cpu_request_microcode(const 
void *buf,
  * one with higher revision.
  */
 if ( (microcode_update_match(mc) != MIS_UCODE) &&
- (!saved || (mc->rev > saved->rev)) )
+ (!saved || compare_revisions(saved->rev, mc->rev) == NEW_UCODE) )
 saved = mc;
 
 buf  += blob_size;
-- 
2.11.0




[PATCH v1 4/4] xen/pci: solve compilation error when memory paging is not enabled.

2020-10-26 Thread Rahul Singh
d->vm_event_paging struct is defined under CONFIG_HAS_MEM_PAGING in
sched.h but referenced in passthrough/pci.c directly.

If CONFIG_HAS_MEM_PAGING is not enabled for architecture, compiler will
throws an error.

No functional change.

Signed-off-by: Rahul Singh 
---
 xen/drivers/passthrough/pci.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
index c6fbb7172c..3125c23e87 100644
--- a/xen/drivers/passthrough/pci.c
+++ b/xen/drivers/passthrough/pci.c
@@ -1419,13 +1419,15 @@ static int assign_device(struct domain *d, u16 seg, u8 
bus, u8 devfn, u32 flag)
 if ( !is_iommu_enabled(d) )
 return 0;
 
-/* Prevent device assign if mem paging or mem sharing have been 
+#if defined(CONFIG_HAS_MEM_PAGING) || defined(CONFIG_MEM_SHARING)
+/* Prevent device assign if mem paging or mem sharing have been
  * enabled for this domain */
 if ( d != dom_io &&
  unlikely(mem_sharing_enabled(d) ||
   vm_event_check_ring(d->vm_event_paging) ||
   p2m_get_hostp2m(d)->global_logdirty) )
 return -EXDEV;
+#endif
 
 /* device_assigned() should already have cleared the device for assignment 
*/
 ASSERT(pcidevs_locked());
-- 
2.17.1




[PATCH v1 3/4] xen/pci: Move x86 specific code to x86 directory.

2020-10-26 Thread Rahul Singh
passthrough/pci.c file is common for all architecture, but there is x86
sepcific code in this file.

Move x86 specific code to the x86 directory to avoid compilation error
for other architecture.

No functional change.

Signed-off-by: Rahul Singh 
---
 xen/drivers/passthrough/pci.c| 75 +
 xen/drivers/passthrough/x86/Makefile |  1 +
 xen/drivers/passthrough/x86/iommu.c  |  7 ++
 xen/drivers/passthrough/x86/pci.c| 97 
 xen/include/xen/pci.h|  2 +
 5 files changed, 108 insertions(+), 74 deletions(-)
 create mode 100644 xen/drivers/passthrough/x86/pci.c

diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c
index 2a3bce1462..c6fbb7172c 100644
--- a/xen/drivers/passthrough/pci.c
+++ b/xen/drivers/passthrough/pci.c
@@ -24,7 +24,6 @@
 #include 
 #include 
 #include 
-#include 
 #include 
 #include 
 #include 
@@ -847,71 +846,6 @@ int pci_remove_device(u16 seg, u8 bus, u8 devfn)
 return ret;
 }
 
-static int pci_clean_dpci_irq(struct domain *d,
-  struct hvm_pirq_dpci *pirq_dpci, void *arg)
-{
-struct dev_intx_gsi_link *digl, *tmp;
-
-pirq_guest_unbind(d, dpci_pirq(pirq_dpci));
-
-if ( pt_irq_need_timer(pirq_dpci->flags) )
-kill_timer(_dpci->timer);
-
-list_for_each_entry_safe ( digl, tmp, _dpci->digl_list, list )
-{
-list_del(>list);
-xfree(digl);
-}
-
-radix_tree_delete(>pirq_tree, dpci_pirq(pirq_dpci)->pirq);
-
-if ( !pt_pirq_softirq_active(pirq_dpci) )
-return 0;
-
-domain_get_irq_dpci(d)->pending_pirq_dpci = pirq_dpci;
-
-return -ERESTART;
-}
-
-static int pci_clean_dpci_irqs(struct domain *d)
-{
-struct hvm_irq_dpci *hvm_irq_dpci = NULL;
-
-if ( !is_iommu_enabled(d) )
-return 0;
-
-if ( !is_hvm_domain(d) )
-return 0;
-
-spin_lock(>event_lock);
-hvm_irq_dpci = domain_get_irq_dpci(d);
-if ( hvm_irq_dpci != NULL )
-{
-int ret = 0;
-
-if ( hvm_irq_dpci->pending_pirq_dpci )
-{
-if ( pt_pirq_softirq_active(hvm_irq_dpci->pending_pirq_dpci) )
- ret = -ERESTART;
-else
- hvm_irq_dpci->pending_pirq_dpci = NULL;
-}
-
-if ( !ret )
-ret = pt_pirq_iterate(d, pci_clean_dpci_irq, NULL);
-if ( ret )
-{
-spin_unlock(>event_lock);
-return ret;
-}
-
-hvm_domain_irq(d)->dpci = NULL;
-free_hvm_irq_dpci(hvm_irq_dpci);
-}
-spin_unlock(>event_lock);
-return 0;
-}
-
 /* Caller should hold the pcidevs_lock */
 static int deassign_device(struct domain *d, uint16_t seg, uint8_t bus,
uint8_t devfn)
@@ -971,7 +905,7 @@ int pci_release_devices(struct domain *d)
 int ret;
 
 pcidevs_lock();
-ret = pci_clean_dpci_irqs(d);
+ret = arch_pci_release_devices(d);
 if ( ret )
 {
 pcidevs_unlock();
@@ -1375,13 +1309,6 @@ static int __init setup_dump_pcidevs(void)
 }
 __initcall(setup_dump_pcidevs);
 
-int iommu_update_ire_from_msi(
-struct msi_desc *msi_desc, struct msi_msg *msg)
-{
-return iommu_intremap
-   ? iommu_call(_ops, update_ire_from_msi, msi_desc, msg) : 0;
-}
-
 static int iommu_add_device(struct pci_dev *pdev)
 {
 const struct domain_iommu *hd;
diff --git a/xen/drivers/passthrough/x86/Makefile 
b/xen/drivers/passthrough/x86/Makefile
index 05e6f51f25..642f673ed2 100644
--- a/xen/drivers/passthrough/x86/Makefile
+++ b/xen/drivers/passthrough/x86/Makefile
@@ -1,2 +1,3 @@
 obj-$(CONFIG_HAS_PCI_ATS) += ats.o
 obj-y += iommu.o
+obj-y += pci.o
diff --git a/xen/drivers/passthrough/x86/iommu.c 
b/xen/drivers/passthrough/x86/iommu.c
index f17b1820f4..875e67b53b 100644
--- a/xen/drivers/passthrough/x86/iommu.c
+++ b/xen/drivers/passthrough/x86/iommu.c
@@ -308,6 +308,13 @@ struct page_info *iommu_alloc_pgtable(struct domain *d)
 return pg;
 }
 
+int iommu_update_ire_from_msi(
+struct msi_desc *msi_desc, struct msi_msg *msg)
+{
+return iommu_intremap
+   ? iommu_call(_ops, update_ire_from_msi, msi_desc, msg) : 0;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/drivers/passthrough/x86/pci.c 
b/xen/drivers/passthrough/x86/pci.c
new file mode 100644
index 00..443712aa22
--- /dev/null
+++ b/xen/drivers/passthrough/x86/pci.c
@@ -0,0 +1,97 @@
+/*
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; If not, see 

[PATCH v1 1/4] xen/ns16550: solve compilation error on ARM with CONFIG_HAS_PCI enabled.

2020-10-26 Thread Rahul Singh
ARM platforms does not support ns16550 PCI support. When CONFIG_HAS_PCI
is enabled for ARM a compilation error is observed.

Fixed compilation error after introducing new kconfig option
CONFIG_HAS_NS16550_PCI for x86 platforms to support ns16550 PCI.

No functional change.

Signed-off-by: Rahul Singh 
---
 xen/drivers/char/Kconfig   |  7 +++
 xen/drivers/char/ns16550.c | 32 
 2 files changed, 23 insertions(+), 16 deletions(-)

diff --git a/xen/drivers/char/Kconfig b/xen/drivers/char/Kconfig
index b572305657..8887e86afe 100644
--- a/xen/drivers/char/Kconfig
+++ b/xen/drivers/char/Kconfig
@@ -4,6 +4,13 @@ config HAS_NS16550
help
  This selects the 16550-series UART support. For most systems, say Y.
 
+config HAS_NS16550_PCI
+   bool "NS16550 UART PCI support" if X86
+   default y
+   depends on X86 && HAS_NS16550 && HAS_PCI
+   help
+ This selects the 16550-series UART PCI support. For most systems, say 
Y.
+
 config HAS_CADENCE_UART
bool "Xilinx Cadence UART driver"
default y
diff --git a/xen/drivers/char/ns16550.c b/xen/drivers/char/ns16550.c
index d8b52eb813..bd1c2af956 100644
--- a/xen/drivers/char/ns16550.c
+++ b/xen/drivers/char/ns16550.c
@@ -16,7 +16,7 @@
 #include 
 #include 
 #include 
-#ifdef CONFIG_HAS_PCI
+#ifdef CONFIG_HAS_NS16550_PCI
 #include 
 #include 
 #include 
@@ -54,7 +54,7 @@ enum serial_param_type {
 reg_shift,
 reg_width,
 stop_bits,
-#ifdef CONFIG_HAS_PCI
+#ifdef CONFIG_HAS_NS16550_PCI
 bridge_bdf,
 device,
 port_bdf,
@@ -83,7 +83,7 @@ static struct ns16550 {
 unsigned int timeout_ms;
 bool_t intr_works;
 bool_t dw_usr_bsy;
-#ifdef CONFIG_HAS_PCI
+#ifdef CONFIG_HAS_NS16550_PCI
 /* PCI card parameters. */
 bool_t pb_bdf_enable;   /* if =1, pb-bdf effective, port behind bridge */
 bool_t ps_bdf_enable;   /* if =1, ps_bdf effective, port on pci card */
@@ -117,14 +117,14 @@ static const struct serial_param_var __initconst 
sp_vars[] = {
 {"reg-shift", reg_shift},
 {"reg-width", reg_width},
 {"stop-bits", stop_bits},
-#ifdef CONFIG_HAS_PCI
+#ifdef CONFIG_HAS_NS16550_PCI
 {"bridge", bridge_bdf},
 {"dev", device},
 {"port", port_bdf},
 #endif
 };
 
-#ifdef CONFIG_HAS_PCI
+#ifdef CONFIG_HAS_NS16550_PCI
 struct ns16550_config {
 u16 vendor_id;
 u16 dev_id;
@@ -620,7 +620,7 @@ static int ns16550_getc(struct serial_port *port, char *pc)
 
 static void pci_serial_early_init(struct ns16550 *uart)
 {
-#ifdef CONFIG_HAS_PCI
+#ifdef CONFIG_HAS_NS16550_PCI
 if ( !uart->ps_bdf_enable || uart->io_base >= 0x1 )
 return;
 
@@ -719,7 +719,7 @@ static void __init ns16550_init_preirq(struct serial_port 
*port)
 
 static void __init ns16550_init_irq(struct serial_port *port)
 {
-#ifdef CONFIG_HAS_PCI
+#ifdef CONFIG_HAS_NS16550_PCI
 struct ns16550 *uart = port->uart;
 
 if ( uart->msi )
@@ -761,7 +761,7 @@ static void __init ns16550_init_postirq(struct serial_port 
*port)
 uart->timeout_ms = max_t(
 unsigned int, 1, (bits * uart->fifo_size * 1000) / uart->baud);
 
-#ifdef CONFIG_HAS_PCI
+#ifdef CONFIG_HAS_NS16550_PCI
 if ( uart->bar || uart->ps_bdf_enable )
 {
 if ( uart->param && uart->param->mmio &&
@@ -841,7 +841,7 @@ static void ns16550_suspend(struct serial_port *port)
 
 stop_timer(>timer);
 
-#ifdef CONFIG_HAS_PCI
+#ifdef CONFIG_HAS_NS16550_PCI
 if ( uart->bar )
uart->cr = pci_conf_read16(PCI_SBDF(0, uart->ps_bdf[0], uart->ps_bdf[1],
   uart->ps_bdf[2]), PCI_COMMAND);
@@ -850,7 +850,7 @@ static void ns16550_suspend(struct serial_port *port)
 
 static void _ns16550_resume(struct serial_port *port)
 {
-#ifdef CONFIG_HAS_PCI
+#ifdef CONFIG_HAS_NS16550_PCI
 struct ns16550 *uart = port->uart;
 
 if ( uart->bar )
@@ -1013,7 +1013,7 @@ static int __init check_existence(struct ns16550 *uart)
 return 1; /* Everything is MMIO */
 #endif
 
-#ifdef CONFIG_HAS_PCI
+#ifdef CONFIG_HAS_NS16550_PCI
 pci_serial_early_init(uart);
 #endif
 
@@ -1044,7 +1044,7 @@ static int __init check_existence(struct ns16550 *uart)
 return (status == 0x90);
 }
 
-#ifdef CONFIG_HAS_PCI
+#ifdef CONFIG_HAS_NS16550_PCI
 static int __init
 pci_uart_config(struct ns16550 *uart, bool_t skip_amt, unsigned int idx)
 {
@@ -1305,7 +1305,7 @@ static bool __init parse_positional(struct ns16550 *uart, 
char **str)
 
 if ( *conf == ',' && *++conf != ',' )
 {
-#ifdef CONFIG_HAS_PCI
+#ifdef CONFIG_HAS_NS16550_PCI
 if ( strncmp(conf, "pci", 3) == 0 )
 {
 if ( pci_uart_config(uart, 1/* skip AMT */, uart - ns16550_com) )
@@ -1327,7 +1327,7 @@ static bool __init parse_positional(struct ns16550 *uart, 
char **str)
 
 if ( *conf == ',' && *++conf != ',' )
 {
-#ifdef CONFIG_HAS_PCI
+#ifdef CONFIG_HAS_NS16550_PCI
 if ( strncmp(conf, "msi", 3) == 0 )
 {
 conf += 3;
@@ -1339,7 +1339,7 @@ static bool 

[PATCH v1 2/4] xen/pci: Introduce new CONFIG_HAS_PCI_ATS flag for PCI ATS functionality.

2020-10-26 Thread Rahul Singh
PCI ATS functionality is not enabled and tested for ARM architecture
but it is enabled for x86 and referenced in common passthrough/pci.c
code.

Therefore introducing the new flag to enable the ATS functionality for
x86 only to avoid issues for ARM architecture.

No functional change.

Signed-off-by: Rahul Singh 
---
 xen/arch/x86/Kconfig |  1 +
 xen/drivers/passthrough/ats.h| 24 
 xen/drivers/passthrough/vtd/x86/Makefile |  2 +-
 xen/drivers/passthrough/x86/Makefile |  2 +-
 xen/drivers/pci/Kconfig  |  3 +++
 5 files changed, 30 insertions(+), 2 deletions(-)

diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig
index 24868aa6ad..31906e9c97 100644
--- a/xen/arch/x86/Kconfig
+++ b/xen/arch/x86/Kconfig
@@ -20,6 +20,7 @@ config X86
select HAS_NS16550
select HAS_PASSTHROUGH
select HAS_PCI
+   select HAS_PCI_ATS
select HAS_PDX
select HAS_SCHED_GRANULARITY
select HAS_UBSAN
diff --git a/xen/drivers/passthrough/ats.h b/xen/drivers/passthrough/ats.h
index 22ae209b37..a0af07b287 100644
--- a/xen/drivers/passthrough/ats.h
+++ b/xen/drivers/passthrough/ats.h
@@ -17,6 +17,8 @@
 
 #include 
 
+#ifdef CONFIG_HAS_PCI_ATS
+
 #define ATS_REG_CAP4
 #define ATS_REG_CTL6
 #define ATS_QUEUE_DEPTH_MASK 0x1f
@@ -48,5 +50,27 @@ static inline int pci_ats_device(int seg, int bus, int devfn)
 return pci_find_ext_capability(seg, bus, devfn, PCI_EXT_CAP_ID_ATS);
 }
 
+#else
+
+static inline int enable_ats_device(struct pci_dev *pdev,
+struct list_head *ats_list)
+{
+return -ENODEV;
+}
+
+static inline void disable_ats_device(struct pci_dev *pdev) { }
+
+static inline int pci_ats_enabled(int seg, int bus, int devfn)
+{
+return -ENODEV;
+}
+
+static inline int pci_ats_device(int seg, int bus, int devfn)
+{
+return -ENODEV;
+}
+
+#endif /* CONFIG_HAS_PCI_ATS */
+
 #endif /* _ATS_H_ */
 
diff --git a/xen/drivers/passthrough/vtd/x86/Makefile 
b/xen/drivers/passthrough/vtd/x86/Makefile
index 4ef00a4c5b..60f79fe263 100644
--- a/xen/drivers/passthrough/vtd/x86/Makefile
+++ b/xen/drivers/passthrough/vtd/x86/Makefile
@@ -1,3 +1,3 @@
-obj-y += ats.o
+obj-$(CONFIG_HAS_PCI_ATS) += ats.o
 obj-$(CONFIG_HVM) += hvm.o
 obj-y += vtd.o
diff --git a/xen/drivers/passthrough/x86/Makefile 
b/xen/drivers/passthrough/x86/Makefile
index a70cf9460d..05e6f51f25 100644
--- a/xen/drivers/passthrough/x86/Makefile
+++ b/xen/drivers/passthrough/x86/Makefile
@@ -1,2 +1,2 @@
-obj-y += ats.o
+obj-$(CONFIG_HAS_PCI_ATS) += ats.o
 obj-y += iommu.o
diff --git a/xen/drivers/pci/Kconfig b/xen/drivers/pci/Kconfig
index 7da03fa13b..1588d4a91e 100644
--- a/xen/drivers/pci/Kconfig
+++ b/xen/drivers/pci/Kconfig
@@ -1,3 +1,6 @@
 
 config HAS_PCI
bool
+
+config HAS_PCI_ATS
+   bool
-- 
2.17.1




[PATCH v1 0/4] xen/arm: Make PCI passthrough code non-x86 specific

2020-10-26 Thread Rahul Singh
This patch series is preparatory work to make PCI passthrough code non-x86
specific.

Rahul Singh (4):
  xen/ns16550: solve compilation error on ARM with CONFIG_HAS_PCI
enabled.
  xen/pci: Introduce new CONFIG_HAS_PCI_ATS flag for PCI ATS
functionality.
  xen/pci: Move x86 specific code to x86 directory.
  xen/pci: solve compilation error when memory paging is not enabled.

 xen/arch/x86/Kconfig |  1 +
 xen/drivers/char/Kconfig |  7 ++
 xen/drivers/char/ns16550.c   | 32 
 xen/drivers/passthrough/ats.h| 24 ++
 xen/drivers/passthrough/pci.c| 79 +--
 xen/drivers/passthrough/vtd/x86/Makefile |  2 +-
 xen/drivers/passthrough/x86/Makefile |  3 +-
 xen/drivers/passthrough/x86/iommu.c  |  7 ++
 xen/drivers/passthrough/x86/pci.c| 97 
 xen/drivers/pci/Kconfig  |  3 +
 xen/include/xen/pci.h|  2 +
 11 files changed, 164 insertions(+), 93 deletions(-)
 create mode 100644 xen/drivers/passthrough/x86/pci.c

-- 
2.17.1




Re: [Xen-devel] [XEN PATCH for-4.13 v7 01/11] libxl: Offer API versions 0x040700 and 0x040800

2020-10-26 Thread Jan Beulich
On 26.10.2020 17:42, Olaf Hering wrote:
> On Wed, Oct 23, Ian Jackson wrote:
> 
>> 0x040700 was introduced in 304400459ef0 (aka 4.7.0-rc1~481)
>> 0x040800 was introduced in 57f8b13c7240 (aka 4.8.0-rc1~437)
> 
>> Anyway, in the meantime, we should fix it.  Backporting this is
>> probably a good idea: it won't change the behaviour for existing
>> callers but it will avoid errors for some older correct uses.
> 
>> +LIBXL_API_VERSION != 0x040700 && LIBXL_API_VERSION != 0x040800 && \
> 
> 
> Why was this never backported to staging-4.12 and older?
> Please backport it to assist libvirt.

I'm afraid the request comes too late for 4.12 (branch now
closed for its final stable release to be cut) and older
(already in security-only mode).

Jan



Re: XSM and the idle domain

2020-10-26 Thread Andrew Cooper
On 26/10/2020 13:37, Jason Andryuk wrote:
> On Thu, Oct 22, 2020 at 1:01 PM Hongyan Xia  wrote:
>> On Thu, 2020-10-22 at 13:51 +0100, Andrew Cooper wrote:
>>> On 21/10/2020 15:34, Hongyan Xia wrote:
 The first question came up during ongoing work in LiveUpdate. After
 an
 LU, the next Xen needs to restore all domains. To do that, some
 hypercalls need to be issued from the idle domain context and
 apparently XSM does not like it.
>>> There is no such thing as issuing hypercalls from the idle domain
>>> (context or otherwise), because the idle domain does not have enough
>>> associated guest state for anything to make the requisite
>>> SYSCALL/INT80/VMCALL/VMMCALL invocation.
>>>
>>> I presume from this comment that what you mean is that you're calling
>>> the plain hypercall functions, context checks and everything, from
>>> the
>>> idle context?
>> Yep, the restore code just calls the hypercall functions from idle
>> context.
>>
>>> If so, this is buggy for more reasons than just XSM objecting to its
>>> calling context, and that XSM is merely the first thing to explode.
>>> Therefore, I don't think modifications to XSM are applicable to
>>> solving
>>> the problem.
>>>
>>> (Of course, this is all speculation because there's no concrete
>>> implementation to look at.)
>> Another explosion is the inability to create hypercall preemption,
>> which for now is disabled when the calling context is the idle domain.
>> Apart from XSM and preemption, the LU prototype works fine. We only
>> reuse a limited number of hypercall functions and are not trying to be
>> able to call all possible hypercalls from idle.
> I wonder if for domain_create, it wouldn't be better to move
> xsm_domain_create() out to the domctl (hypercall entry) and check it
> there.  That would side-step xsm in domain_create.  Flask would need
> to be modified for that.  I've an untested patch doing the
> rearranging, which I'll send as a follow up.
>
> What other hypercalls are you having issues with?  Those could also be
> refactored so the hypercall entry checks permissions, and the actual
> work is done in a directly callable function.
>
>> Having a dedicated domLU just like domB (or reusing domB) sounds like a
>> viable option. If the overhead can be made low enough then we won't
>> need to work around XSM and hypercall preemption.
>>
>> Although the question was whether XSM should interact with the idle
>> domain. With a good design LU should be able to sidestep this though.
> Circling back to the main topic, is the idle domain Xen, or is it
> distinct?

It "is" Xen, IMO.

> It runs in the context of Xen, so Xen isn't really in a
> place to enforce policy on itself.  Hongyan, as you said earlier,
> applying XSM is more of a debugging feature at that point than a
> security feature.  And as Jan pointed out, you can have problems if
> XSM prevents the hypervisor from performing an action it doesn't
> expect to fail.

We have several system DOMID's which are SELF, IO, XEN, COW, INVALID and
IDLE.

SELF is a magic constant expected to be used in most hypercalls on
oneself, to simplify callers.  INVALID is also a magic constant.

The others all have struct domain's allocated for them, and are concrete
objects as far as Xen is concerned.  IO/XEN/COW all exist for the
purpose of fitting into the memory/device ownership models, while IDLE
exists for the purpose of encapsulating the idle vcpus in the scheduling
model.

None of them have any kind of outside-Xen state associated with them. 
"scheduling" an idle vCPU runs the idle loop, but it is all code within
the hypervisor.

The problem here is that idle context is also used in certain "normal"
cases in Xen (startup, shutdown, possibly also for softirq/tasklet
context), all of which we (currently) expect not to be making hypercalls
from.

~Andrew



Re: [Xen-devel] [XEN PATCH for-4.13 v7 01/11] libxl: Offer API versions 0x040700 and 0x040800

2020-10-26 Thread Olaf Hering
On Wed, Oct 23, Ian Jackson wrote:

> 0x040700 was introduced in 304400459ef0 (aka 4.7.0-rc1~481)
> 0x040800 was introduced in 57f8b13c7240 (aka 4.8.0-rc1~437)

> Anyway, in the meantime, we should fix it.  Backporting this is
> probably a good idea: it won't change the behaviour for existing
> callers but it will avoid errors for some older correct uses.

> +LIBXL_API_VERSION != 0x040700 && LIBXL_API_VERSION != 0x040800 && \


Why was this never backported to staging-4.12 and older?
Please backport it to assist libvirt.

Olaf


signature.asc
Description: PGP signature


Re: [RFC PATCH] xsm: Re-work domain_create and domain_alloc_security

2020-10-26 Thread Jason Andryuk
On Mon, Oct 26, 2020 at 12:23 PM Daniel Smith
 wrote:
>
>  On Mon, 26 Oct 2020 09:46:51 -0400 Jason Andryuk  
> wrote 
>
>  > Untested!
>  >
>  > This only really matters for flask, but all of xsm is updated.
>  >
>  > flask_domain_create() and flask_domain_alloc_security() are a strange
>  > pair.
>  >
>  > flask_domain_create() serves double duty.  It both assigns sid and
>  > self_sid values and checks if the calling domain has permission to
>  > create the target domain.  It also has special casing for handling dom0.
>  > Meanwhile flask_domain_alloc_security() assigns some special sids, but
>  > waits for others to be assigned in flask_domain_create.  This split
>  > seems to have come about so that the structures are allocated before
>  > calling flask_domain_create().  It also means flask_domain_create is
>  > called in the middle of domain_create.
>  >
>  > Re-arrange the two calls.  Let flask_domain_create just check if current
>  > has permission to create ssidref.  Then it can be moved out to do_domctl
>  > and gate entry into domain_create.  This avoids doing partial domain
>  > creation before the permission check.
>  >
>  > Have flask_domain_alloc_security() take a ssidref argument.  The ssidref
>  > was already permission checked earlier, so it can just be assigned.
>  > Then the self_sid can be calculated here as well rather than in
>  > flask_domain_create().
>  >
>  > The dom0 special casing is moved into flask_domain_alloc_security().
>  > Maybe this should be just a fall-through for the dom0 already created
>  > case.  This code may not be needed any longer.
>  >
>  > Signed-off-by: Jason Andryuk 
>  > ---



>  > -static int flask_domain_alloc_security(struct domain *d)
>  > +static int flask_domain_alloc_security(struct domain *d, u32 ssidref)
>  >  {
>  >  struct domain_security_struct *dsec;
>  > +static int dom0_created = 0;
>  > +int rc;
>  >
>  >  dsec = xzalloc(struct domain_security_struct);
>  >  if ( !dsec )
>  > @@ -175,14 +177,24 @@ static int flask_domain_alloc_security(struct domain 
> *d)
>  >  case DOMID_IO:
>  >  dsec->sid = SECINITSID_DOMIO;
>  >  break;
>  > +case 0:
>  > +if ( !dom0_created ) {
>  > +dsec->sid = SECINITSID_DOM0;
>  > +dom0_created = 1;
>  > +} else {
>  > +dsec->sid = SECINITSID_UNLABELED;
>  > +}
>
> While the handling of this case is not wrong, I have to wonder if there is a 
> better way to handle the dom0 creation case.

dom0_cfg.ssidref could be set to SECINITSID_DOM0.  I guess that would
need some xsm_ssid_dom0 wrapper.  Then maybe this logic can go away
and the default case used.

pv-shim doesn't necessarily use domid 0, so this may be broken there.
dom0_cfg.ssidref would fix that, I think.  But I'm not familiar with
pv-shim.

>  > +break;
>  >  default:
>  > -dsec->sid = SECINITSID_UNLABELED;
>  > +dsec->sid = ssidref;
>  >  }
>  >
>  >  dsec->self_sid = dsec->sid;
>  > -d->ssid = dsec;
>
> I don't think you meant to deleted that, without it domains will have no ssid 
> assigned to them.

Yes, this should be retained.

Thanks for looking.

-Jason



Re: Recent upgrade of 4.13 -> 4.14 issue

2020-10-26 Thread Dario Faggioli
On Mon, 2020-10-26 at 15:30 +0100, Jürgen Groß wrote:
> On 26.10.20 14:54, Andrew Cooper wrote:
> > On 26/10/2020 13:37, Frédéric Pierret wrote:
> > > 
> > > If anyone would have any idea of what's going on, that would be
> > > very
> > > appreciated. Thank you.
> > 
> > Does booting Xen with `sched=credit` make a difference?
> 
> Hmm, I think I have spotted a problem in credit2 which could explain
> the
> hang:
> 
> csched2_unit_wake() will NOT put the sched unit on a runqueue in case
> it
> has CSFLAG_scheduled set. This bit will be reset only in
> csched2_context_saved().
> 
Exactly, it does not put it back there. However, if it finds a vCPU
with the CSFLAG_scheduled flag set, It should set
CSFLAG_delayed_runq_add flag.

Unless curr_on_cpu(cpu)==unit or unit_on_runq(svc)==true... which
should not be the case. Or where you saying that we actually are in one
of this situations?

In fact...

> So in case a vcpu (and its unit, of course) is blocked and there has
> been no other vcpu active on its physical cpu but the idle vcpu,
> there
> will be no call of csched2_context_saved(). This will block the vcpu
> to become active in theory for eternity, in case there is no need to
> run another vcpu on the physical cpu.
> 
...I maybe am not seeing what exact situation and sequence of events
you're exactly thinking to. What I see is this: [*]

- vCPU V is running, i.e., CSFLAG_scheduled is set
- vCPU V blocks
- we enter schedule()
  - schedule calls do_schedule() --> csched2_schedule()
- we pick idle, so CSFLAG_delayed_runq_add is set for V
  - schedule calls sched_context_switch()
- sched_context_switch() calls context_switch()
  - context_switch() calls sched_context_switched()
- sched_context_switched() calls:
  - vcpu_context_saved()
  - unit_context_saved()
- unit_context_saved() calls sched_context_saved() -->
  csched2_context_saved()
  - csched2_context_saved():
- clears CSFLAG_scheduled
- checks (and clear) CSFLAG_delayed_runq_add

[*] this assumes granularity 1, i.e., no core-scheduling and no 
rendezvous. Or was core-scheduling actually enabled?

And if CSFLAG_delayed_runq_add is set **and** the vCPU is runnable, the
task is added back to the runqueue.

So, even if we don't do the actual context switch (i.e., we don't call
__context_switch() ) if the next vCPU that we pick when vCPU V blocks
is the idle one, it looks to me that we go get to call
csched2_context_saved().

And it also looks to me that, when we get to that, if the vCPU is
runnable, even if it has the CSFLAG_scheduled still set, we do put it
back to the runqueue.

And if the vCPU blocked, but csched2_unit_wake() run while
CSFLAG_scheduled was still set, it indeed should mean that the vCPU
itself will be runnable again when we get to csched2_context_saved().

Or did you have something completely different in mind, and I'm missing
it?


Regards
-- 
Dario Faggioli, Ph.D
http://about.me/dario.faggioli
Virtualization Software Engineer
SUSE Labs, SUSE https://www.suse.com/
---
<> (Raistlin Majere)


signature.asc
Description: This is a digitally signed message part


Re: [RFC PATCH] xsm: Re-work domain_create and domain_alloc_security

2020-10-26 Thread Daniel Smith
 On Mon, 26 Oct 2020 09:46:51 -0400 Jason Andryuk  
wrote 

 > Untested! 
 >  
 > This only really matters for flask, but all of xsm is updated. 
 >  
 > flask_domain_create() and flask_domain_alloc_security() are a strange 
 > pair. 
 >  
 > flask_domain_create() serves double duty.  It both assigns sid and 
 > self_sid values and checks if the calling domain has permission to 
 > create the target domain.  It also has special casing for handling dom0. 
 > Meanwhile flask_domain_alloc_security() assigns some special sids, but 
 > waits for others to be assigned in flask_domain_create.  This split 
 > seems to have come about so that the structures are allocated before 
 > calling flask_domain_create().  It also means flask_domain_create is 
 > called in the middle of domain_create. 
 >  
 > Re-arrange the two calls.  Let flask_domain_create just check if current 
 > has permission to create ssidref.  Then it can be moved out to do_domctl 
 > and gate entry into domain_create.  This avoids doing partial domain 
 > creation before the permission check. 
 >  
 > Have flask_domain_alloc_security() take a ssidref argument.  The ssidref 
 > was already permission checked earlier, so it can just be assigned. 
 > Then the self_sid can be calculated here as well rather than in 
 > flask_domain_create(). 
 >  
 > The dom0 special casing is moved into flask_domain_alloc_security(). 
 > Maybe this should be just a fall-through for the dom0 already created 
 > case.  This code may not be needed any longer. 
 >  
 > Signed-off-by: Jason Andryuk  
 > --- 
 >  xen/common/domain.c |  6 ++ 
 >  xen/common/domctl.c |  4  
 >  xen/include/xsm/dummy.h |  6 +++--- 
 >  xen/include/xsm/xsm.h   | 12 +-- 
 >  xen/xsm/flask/hooks.c   | 48 - 
 >  5 files changed, 34 insertions(+), 42 deletions(-) 
 >  
 > diff --git a/xen/common/domain.c b/xen/common/domain.c 
 > index f748806a45..6b1f5ed59d 100644 
 > --- a/xen/common/domain.c 
 > +++ b/xen/common/domain.c 
 > @@ -407,7 +407,8 @@ struct domain *domain_create(domid_t domid, 
 >  
 >  lock_profile_register_struct(LOCKPROF_TYPE_PERDOM, d, domid); 
 >  
 > -if ( (err = xsm_alloc_security_domain(d)) != 0 ) 
 > +if ( (err = xsm_alloc_security_domain(d, config ? config->ssidref : 
 > +  0)) != 0 ) 
 >  goto fail; 
 >  
 >  atomic_set(>refcnt, 1); 
 > @@ -470,9 +471,6 @@ struct domain *domain_create(domid_t domid, 
 >  if ( !d->iomem_caps || !d->irq_caps ) 
 >  goto fail; 
 >  
 > -if ( (err = xsm_domain_create(XSM_HOOK, d, config->ssidref)) != 0 ) 
 > -goto fail; 
 > - 
 >  d->controller_pause_count = 1; 
 >  atomic_inc(>pause_count); 
 >  
 > diff --git a/xen/common/domctl.c b/xen/common/domctl.c 
 > index af044e2eda..ffdc1a41cd 100644 
 > --- a/xen/common/domctl.c 
 > +++ b/xen/common/domctl.c 
 > @@ -406,6 +406,10 @@ long do_domctl(XEN_GUEST_HANDLE_PARAM(xen_domctl_t) 
 > u_domctl) 
 >  domid_tdom; 
 >  static domid_t rover = 0; 
 >  
 > +ret = xsm_domain_create(XSM_HOOK, op->u.createdomain.ssidref); 
 > +if (ret) 
 > +break; 
 > + 
 >  dom = op->domain; 
 >  if ( (dom > 0) && (dom < DOMID_FIRST_RESERVED) ) 
 >  { 
 > diff --git a/xen/include/xsm/dummy.h b/xen/include/xsm/dummy.h 
 > index 7ae3c40eb5..29c4ca9951 100644 
 > --- a/xen/include/xsm/dummy.h 
 > +++ b/xen/include/xsm/dummy.h 
 > @@ -104,10 +104,10 @@ static XSM_INLINE void xsm_security_domaininfo(struct 
 > domain *d, 
 >  return; 
 >  } 
 >  
 > -static XSM_INLINE int xsm_domain_create(XSM_DEFAULT_ARG struct domain *d, 
 > u32 ssidref) 
 > +static XSM_INLINE int xsm_domain_create(XSM_DEFAULT_ARG u32 ssidref) 
 >  { 
 >  XSM_ASSERT_ACTION(XSM_HOOK); 
 > -return xsm_default_action(action, current->domain, d); 
 > +return xsm_default_action(action, current->domain, NULL); 
 >  } 
 >  
 >  static XSM_INLINE int xsm_getdomaininfo(XSM_DEFAULT_ARG struct domain *d) 
 > @@ -163,7 +163,7 @@ static XSM_INLINE int xsm_readconsole(XSM_DEFAULT_ARG 
 > uint32_t clear) 
 >  return xsm_default_action(action, current->domain, NULL); 
 >  } 
 >  
 > -static XSM_INLINE int xsm_alloc_security_domain(struct domain *d) 
 > +static XSM_INLINE int xsm_alloc_security_domain(struct domain *d, uint32_t 
 > ssidref) 
 >  { 
 >  return 0; 
 >  } 
 > diff --git a/xen/include/xsm/xsm.h b/xen/include/xsm/xsm.h 
 > index 358ec13ba8..c1d2ef5832 100644 
 > --- a/xen/include/xsm/xsm.h 
 > +++ b/xen/include/xsm/xsm.h 
 > @@ -46,7 +46,7 @@ typedef enum xsm_default xsm_default_t; 
 >  struct xsm_operations { 
 >  void (*security_domaininfo) (struct domain *d, 
 >  struct xen_domctl_getdomaininfo *info); 
 > -int (*domain_create) (struct domain *d, u32 ssidref); 
 > +int (*domain_create) (u32 ssidref); 
 >  int (*getdomaininfo) (struct domain *d); 
 >  int (*domctl_scheduler_op) (struct domain *d, int op); 
 >  int (*sysctl_scheduler_op) (int op); 
 > @@ -71,7 

[PATCH v2 3/3] xen/arm: Warn user on cpu errata 832075

2020-10-26 Thread Bertrand Marquis
When a Cortex A57 processor is affected by CPU errata 832075, a guest
not implementing the workaround for it could deadlock the system.
Add a warning during boot informing the user that only trusted guests
should be executed on the system.
An equivalent warning is already given to the user by KVM on cores
affected by this errata.

Also taint the hypervisor as unsecure when this errata applies and
mention Cortex A57 r0p0 - r1p2 as not security supported in SUPPORT.md

Signed-off-by: Bertrand Marquis 
---
 SUPPORT.md   |  1 +
 xen/arch/arm/cpuerrata.c | 13 +
 2 files changed, 14 insertions(+)

diff --git a/SUPPORT.md b/SUPPORT.md
index 5fbe5fc444..f7a3b046b0 100644
--- a/SUPPORT.md
+++ b/SUPPORT.md
@@ -38,6 +38,7 @@ supported in this document.
 ### ARM v8
 
 Status: Supported
+Status, Cortex A57 r0p0 - r1p2, not security supported (Errata 832075)
 
 ## Host hardware support
 
diff --git a/xen/arch/arm/cpuerrata.c b/xen/arch/arm/cpuerrata.c
index 0430069a84..b35e8cd0b9 100644
--- a/xen/arch/arm/cpuerrata.c
+++ b/xen/arch/arm/cpuerrata.c
@@ -503,6 +503,19 @@ void check_local_cpu_errata(void)
 void __init enable_errata_workarounds(void)
 {
 enable_cpu_capabilities(arm_errata);
+
+#ifdef CONFIG_ARM64_ERRATUM_832075
+if ( cpus_have_cap(ARM64_WORKAROUND_DEVICE_LOAD_ACQUIRE) )
+{
+printk_once(" This CPU is affected by the errata 832075. \n"
+" Guests without CPU erratum workarounds \n"
+" can deadlock the system!   \n"
+" Only trusted guests should be used.\n");
+
+/* Taint the machine has being insecure */
+add_taint(TAINT_MACHINE_UNSECURE);
+}
+#endif
 }
 
 static int cpu_errata_callback(struct notifier_block *nfb,
-- 
2.17.1




[PATCH v2 1/3] xen/arm: use printk_once for errata warning prints

2020-10-26 Thread Bertrand Marquis
Replace usage of warning_add by printk_once with a  prefix and
suffix for errata related warnings.

This prevents the need for the assert which is not secure enough to
protect this print against wrong usage.

Signed-off-by: Bertrand Marquis 
---
 xen/arch/arm/cpuerrata.c | 10 ++
 1 file changed, 2 insertions(+), 8 deletions(-)

diff --git a/xen/arch/arm/cpuerrata.c b/xen/arch/arm/cpuerrata.c
index 0c63dfa779..0430069a84 100644
--- a/xen/arch/arm/cpuerrata.c
+++ b/xen/arch/arm/cpuerrata.c
@@ -157,7 +157,6 @@ extern char __smccc_workaround_1_smc_start[], 
__smccc_workaround_1_smc_end[];
 static int enable_smccc_arch_workaround_1(void *data)
 {
 struct arm_smccc_res res;
-static bool warned = false;
 const struct arm_cpu_capabilities *entry = data;
 
 /*
@@ -182,13 +181,8 @@ static int enable_smccc_arch_workaround_1(void *data)
  "call ARM_SMCCC_ARCH_WORKAROUND_1");
 
 warn:
-if ( !warned )
-{
-ASSERT(system_state < SYS_STATE_active);
-warning_add("No support for ARM_SMCCC_ARCH_WORKAROUND_1.\n"
-"Please update your firmware.\n");
-warned = true;
-}
+printk_once(" No support for ARM_SMCCC_ARCH_WORKAROUND_1. \n"
+" Please update your firmware.\n");
 
 return 0;
 }
-- 
2.17.1




[PATCH v2 2/3] xen: Add an unsecure Taint type

2020-10-26 Thread Bertrand Marquis
Define a new Unsecure taint type to be used to signal a system tainted
due to an unsecure configuration or hardware feature/errata.

Signed-off-by: Bertrand Marquis 
---
 xen/common/kernel.c   | 4 +++-
 xen/include/xen/lib.h | 1 +
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/xen/common/kernel.c b/xen/common/kernel.c
index c3a943f077..7a345ae45e 100644
--- a/xen/common/kernel.c
+++ b/xen/common/kernel.c
@@ -326,6 +326,7 @@ unsigned int tainted;
  *  'E' - An error (e.g. a machine check exceptions) has been injected.
  *  'H' - HVM forced emulation prefix is permitted.
  *  'M' - Machine had a machine check experience.
+ *  'U' - Platform is unsecure (usually due to an errata on the platform).
  *
  *  The string is overwritten by the next call to print_taint().
  */
@@ -333,7 +334,8 @@ char *print_tainted(char *str)
 {
 if ( tainted )
 {
-snprintf(str, TAINT_STRING_MAX_LEN, "Tainted: %c%c%c%c",
+snprintf(str, TAINT_STRING_MAX_LEN, "Tainted: %c%c%c%c%c",
+ tainted & TAINT_MACHINE_UNSECURE ? 'U' : ' ',
  tainted & TAINT_MACHINE_CHECK ? 'M' : ' ',
  tainted & TAINT_SYNC_CONSOLE ? 'C' : ' ',
  tainted & TAINT_ERROR_INJECT ? 'E' : ' ',
diff --git a/xen/include/xen/lib.h b/xen/include/xen/lib.h
index 1983bd6b86..a9679c913d 100644
--- a/xen/include/xen/lib.h
+++ b/xen/include/xen/lib.h
@@ -193,6 +193,7 @@ uint64_t muldiv64(uint64_t a, uint32_t b, uint32_t c);
 #define TAINT_MACHINE_CHECK (1u << 1)
 #define TAINT_ERROR_INJECT  (1u << 2)
 #define TAINT_HVM_FEP   (1u << 3)
+#define TAINT_MACHINE_UNSECURE  (1u << 4)
 extern unsigned int tainted;
 #define TAINT_STRING_MAX_LEN20
 extern char *print_tainted(char *str);
-- 
2.17.1




[PATCH v2 0/3] xen/arm: Warn user on cpu errata 832075

2020-10-26 Thread Bertrand Marquis
This serie is a v2 after a discussion of [1] to introduce several
changes to handle properly the warning for Arm cpu errata 832075:
- use printk_once instead of warning add
- introduce a tainted type "Unsecure"

The last patch is adding the warning and flags A57 core affected as
unsupported.

[1] https://lists.xenproject.org/archives/html/xen-devel/2020-10/msg00896.html

Bertrand Marquis (3):
  xen/arm: use printk_once for errata warning prints
  xen: Add an unsecure Taint type
  xen/arm: Warn user on cpu errata 832075

 SUPPORT.md   |  1 +
 xen/arch/arm/cpuerrata.c | 23 +++
 xen/common/kernel.c  |  4 +++-
 xen/include/xen/lib.h|  1 +
 4 files changed, 20 insertions(+), 9 deletions(-)

-- 
2.17.1




Re: Recent upgrade of 4.13 -> 4.14 issue

2020-10-26 Thread Frédéric Pierret



Le 10/26/20 à 2:54 PM, Andrew Cooper a écrit :

On 26/10/2020 13:37, Frédéric Pierret wrote:

Hi all,

I'm experiencing problem with a HP ProLiant DL360p Gen8 and recent
upgrade of 4.13 -> 4.14. dom0 and the entire system becomes unstable
and freeze at some point.

I've managed to get three pieces of logs (last one after a reboot and
just before total freeze) by connecting to the serial console with
loglvl options and also redirecting linux kernel output to the serial
console:

```
[ 2150.954883] rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
[ 2150.954934] rcu: 7-...0: (3 GPs behind)
idle=842/1/0x4000 softirq=64670/64671 fqs=14673
[ 2150.954962] (detected by 12, t=60002 jiffies, g=236593, q=126)
[ 2150.954984] Sending NMI from CPU 12 to CPUs 7:
[ 2160.968519] rcu: rcu_sched kthread starved for 10008 jiffies!
g236593 f0x0 RCU_GP_DOING_FQS(6) ->state=0x0 ->cpu=9
[ 2160.968568] rcu: RCU grace-period kthread stack dump:
[ 2160.968586] rcu_sched   R  running task    0    10  2
0x80004000
[ 2160.968612] Call Trace:
[ 2160.968634]  ? xen_hypercall_xen_version+0xa/0x20
[ 2160.968658]  ? xen_force_evtchn_callback+0x9/0x10
[ 2160.968918]  ? check_events+0x12/0x20
[ 2160.968946]  ? schedule+0x39/0xa0
[ 2160.968964]  ? schedule_timeout+0x96/0x150
[ 2160.968981]  ? __next_timer_interrupt+0xd0/0xd0
[ 2160.969002]  ? rcu_gp_fqs_loop+0xea/0x2a0
[ 2160.969019]  ? rcu_gp_kthread+0xb5/0x140
[ 2160.969035]  ? rcu_gp_init+0x470/0x470
[ 2160.969052]  ? kthread+0x115/0x140
[ 2160.969067]  ? __kthread_bind_mask+0x60/0x60
[ 2160.969085]  ? ret_from_fork+0x35/0x40
```

and also

```
[ 2495.945931] CPU: 14 PID: 24181 Comm: Xorg Not tainted
5.4.72-2.qubes.x86_64 #1
[ 2495.945954] Hardware name: HP ProLiant DL360p Gen8, BIOS P71
05/24/2019
[ 2495.945984] RIP: e030:smp_call_function_many+0x20a/0x270
[ 2495.946004] Code: 8a 00 3b 05 4c b5 69 01 89 c7 0f 83 89 fe ff ff
48 63 c7 49 8b 17 48 03 14 c5 80 f9 3d 82 8b 42 18 a8 01 74 09 f3 90
8b 42 18  01 75 f7 eb c9 48 c7 c2 a0 6f 82 82 4c 89 f6 89 df e8 bf b2
8a
[ 2495.946051] RSP: e02b:c90003aa7cf0 EFLAGS: 0202
[ 2495.946068] RAX: 0003 RBX: 0010 RCX:
0007
[ 2495.946090] RDX: 8882413ef640 RSI: 0010 RDI:
0007
[ 2495.946113] RBP: 81082fc0 R08: 0007 R09:

[ 2495.946134] R10:  R11: 8265b6a8 R12:

[ 2495.946156] R13: 0001 R14: 00029ac0 R15:
8882415a9b00
[ 2495.946211] FS:  7a0d51a91a40() GS:88824158()
knlGS:
[ 2495.946235] CS:  e030 DS:  ES:  CR0: 80050033
[ 2495.946253] CR2: 70abab15a000 CR3: 0001e18ee000 CR4:
00040660
[ 2495.946290] Call Trace:
[ 2495.946314]  ? do_kernel_range_flush+0x50/0x50
[ 2495.946334]  on_each_cpu+0x28/0x50
[ 2495.946354]  decrease_reservation+0x22f/0x310
[ 2495.946377]  alloc_xenballooned_pages+0xeb/0x120
[ 2495.946396]  ? __kmalloc+0x183/0x260
[ 2495.946413]  gnttab_alloc_pages+0x11/0x40
[ 2495.946434]  gntdev_alloc_map+0x12f/0x250 [xen_gntdev]
[ 2495.946454]  gntdev_ioctl_map_grant_ref+0x73/0x1d0 [xen_gntdev]
[ 2495.946479]  do_vfs_ioctl+0x2fb/0x490
[ 2495.946500]  ? syscall_trace_enter+0x1d1/0x2c0
[ 2495.946551]  ksys_ioctl+0x5e/0x90
[ 2495.946567]  __x64_sys_ioctl+0x16/0x20
[ 2495.946583]  do_syscall_64+0x5b/0xf0
[ 2495.946601]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 2495.946620] RIP: 0033:0x7a0d51f763bb
[ 2495.946727] Code: 0f 1e fa 48 8b 05 dd aa 0c 00 64 c7 00 26 00 00
00 48 c7 c0 ff ff ff ff c3 66 0f 1f 44 00 00 f3 0f 1e fa b8 10 00 00
00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d ad aa 0c 00 f7 d8 64 89 01
48
[ 2495.946804] RSP: 002b:7fffc48d5058 EFLAGS: 0206 ORIG_RAX:
0010
[ 2495.946863] RAX: ffda RBX: 1000 RCX:
7a0d51f763bb
[ 2495.946885] RDX: 7fffc48d5060 RSI: 00184700 RDI:
0009
[ 2495.946939] RBP: 7fffc48d5100 R08: 7fffc48d512c R09:
7a0d51a30010
[ 2495.946998] R10:  R11: 0206 R12:
7fffc48d5060
[ 2495.947020] R13: 0001 R14: 0009 R15:
0001
[ 2510.964667] rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
[ 2510.964716] rcu: 7-...0: (3 GPs behind)
idle=842/1/0x4000 softirq=64670/64671 fqs=96182
[ 2510.964744] (detected by 12, t=420012 jiffies, g=236593, q=11404)
[ 2510.964769] Sending NMI from CPU 12 to CPUs 7:
[ 2523.945643] watchdog: BUG: soft lockup - CPU#14 stuck for 22s!
[Xorg:24181]
[ 2523.945686] Modules linked in: snd_seq_dummy snd_hrtimer snd_seq
snd_seq_device snd_timer snd soundcore br_netfilter xt_physdev
xen_netback loop bridge stp llc rfkill ebtable_filter ebtables
ip6table_filter ip
6_tables iptable_filter intel_rapl_msr iTCO_wdt ipmi_ssif
iTCO_vendor_support intel_rapl_common sb_edac rapl raid456
async_raid6_recov async_memcpy async_pq async_xor async_tx xor
raid6_pq pcspkr joydev hpilo lpc

[qemu-mainline test] 156242: regressions - FAIL

2020-10-26 Thread osstest service owner
flight 156242 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/156242/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm   6 xen-buildfail REGR. vs. 152631
 build-amd64   6 xen-buildfail REGR. vs. 152631
 build-arm64-xsm   6 xen-buildfail REGR. vs. 152631
 build-arm64   6 xen-buildfail REGR. vs. 152631
 build-i3866 xen-buildfail REGR. vs. 152631
 build-i386-xsm6 xen-buildfail REGR. vs. 152631
 build-armhf   6 xen-buildfail REGR. vs. 152631

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-thunderx  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-arndale   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-credit1   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-rtds  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-vhd   1 build-check(1)   blocked  n/a
 test-amd64-i386-pair  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-i386  1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-amd64  1 build-check(1)   blocked  n/a
 test-amd64-coresched-i386-xl  1 build-check(1)   blocked  n/a
 test-amd64-coresched-amd64-xl  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-xsm   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-shadow1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-rtds  1 build-check(1)   blocked  n/a
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-ws16-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-win7-amd64  1 build-check(1) blocked n/a
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked 
n/a
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm  1 build-check(1) blocked n/a
 build-armhf-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow  1 build-check(1) blocked n/a
 build-i386-libvirt1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-amd64-amd64-amd64-pvgrub  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qcow2 1 build-check(1)   blocked  n/a
 test-amd64-amd64-dom0pvh-xl-amd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvshim1 build-check(1)   blocked  n/a
 test-amd64-amd64-dom0pvh-xl-intel  1 build-check(1)   blocked  n/a
 test-amd64-amd64-i386-pvgrub  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvhv2-intel  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-xl-pvhv2-amd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-amd64-amd64-pair 1 build-check(1)   blocked  n/a
 test-amd64-amd64-pygrub   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-credit2   1 build-check(1)   blocked  n/a
 test-amd64-amd64-qemuu-freebsd11-amd64  1 build-check(1)   blocked n/a
 test-amd64-amd64-qemuu-freebsd12-amd64  1 build-check(1)   blocked n/a
 test-amd64-amd64-xl-credit1   1 build-check(1)   blocked  n/a
 test-amd64-amd64-qemuu-nested-amd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-qemuu-nested-intel  1 

Re: Xen on RP4

2020-10-26 Thread Elliott Mitchell
On Mon, Oct 26, 2020 at 01:31:42PM +, Julien Grall wrote:
> On 24/10/2020 06:35, Elliott Mitchell wrote:
> > ACPI has a distinct
> > means of specifying a limited DMA-width; the above fails, because it
> > assumes a *device-tree*.
> 
> Do you know if it would be possible to infer from the ACPI static table 
> the DMA-width?

Yes, and it is.  Due to not knowing much about ACPI tables I don't know
what the C code would look like though (problem is which documentation
should I be looking at first?).

Handy bit of information is in the RP4 Tianocore table source:
https://github.com/tianocore/edk2-platforms/blob/d492639638eee331ac3389e6cf53ea266c3c84b3/Platform/RaspberryPi/AcpiTables/Dsdt.asl

  Name (_DMA, ResourceTemplate() {
//
// Only the first GB is available.
// Bus 0xC000 -> CPU 0x.
//
QWordMemory (ResourceConsumer,
  ,
  MinFixed,
  MaxFixed,
  NonCacheable,
  ReadWrite,
  0x0,
  0xC000, // MIN
  0x, // MAX
  0x4000, // TRA
  0x4000, // LEN
  ,
  ,
  )
  })

There should be some corresponding code in the Linux 5.9 kernels.  From
the look of that, it might even be possible to specify a memory range
which didn't start at address 0.


-- 
(\___(\___(\__  --=> 8-) EHM <=--  __/)___/)___/)
 \BS (| ehem+sig...@m5p.com  PGP 87145445 |)   /
  \_CS\   |  _  -O #include  O-   _  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445





Re: [PATCH] x86: don't open-code le_to_mfn()

2020-10-26 Thread Wei Liu
On Mon, Oct 26, 2020 at 04:47:43PM +0100, Jan Beulich wrote:
> Signed-off-by: Jan Beulich 
> 

Reviewed-by: Wei Liu 



Re: [PATCH] x86: don't open-code vmap_to_mfn()

2020-10-26 Thread Wei Liu
On Mon, Oct 26, 2020 at 04:53:58PM +0100, Jan Beulich wrote:
> Signed-off-by: Jan Beulich 

Reviewed-by: Wei Liu 



[PATCH] x86: don't open-code vmap_to_mfn()

2020-10-26 Thread Jan Beulich
Signed-off-by: Jan Beulich 

--- a/xen/arch/x86/domain_page.c
+++ b/xen/arch/x86/domain_page.c
@@ -333,21 +333,14 @@ void unmap_domain_page_global(const void
 mfn_t domain_page_map_to_mfn(const void *ptr)
 {
 unsigned long va = (unsigned long)ptr;
-const l1_pgentry_t *pl1e;
 
 if ( va >= DIRECTMAP_VIRT_START )
 return _mfn(virt_to_mfn(ptr));
 
 if ( va >= VMAP_VIRT_START && va < VMAP_VIRT_END )
-{
-pl1e = virt_to_xen_l1e(va);
-BUG_ON(!pl1e);
-}
-else
-{
-ASSERT(va >= MAPCACHE_VIRT_START && va < MAPCACHE_VIRT_END);
-pl1e = &__linear_l1_table[l1_linear_offset(va)];
-}
+return vmap_to_mfn(va);
 
-return l1e_get_mfn(*pl1e);
+ASSERT(va >= MAPCACHE_VIRT_START && va < MAPCACHE_VIRT_END);
+
+return l1e_get_mfn(__linear_l1_table[l1_linear_offset(va)]);
 }



[PATCH] x86: don't open-code le_to_mfn()

2020-10-26 Thread Jan Beulich
Signed-off-by: Jan Beulich 

--- a/xen/arch/x86/mm/p2m-pt.c
+++ b/xen/arch/x86/mm/p2m-pt.c
@@ -779,9 +779,9 @@ pod_retry_l3:
 }
 if ( flags & _PAGE_PSE )
 {
-mfn = _mfn(l3e_get_pfn(*l3e) +
-   l2_table_offset(addr) * L1_PAGETABLE_ENTRIES +
-   l1_table_offset(addr));
+mfn = mfn_add(l3e_get_mfn(*l3e),
+  l2_table_offset(addr) * L1_PAGETABLE_ENTRIES +
+  l1_table_offset(addr));
 *t = p2m_recalc_type(recalc || _needs_recalc(flags),
  p2m_flags_to_type(flags), p2m, gfn);
 unmap_domain_page(l3e);
@@ -820,7 +820,7 @@ pod_retry_l2:
 }
 if ( flags & _PAGE_PSE )
 {
-mfn = _mfn(l2e_get_pfn(*l2e) + l1_table_offset(addr));
+mfn = mfn_add(l2e_get_mfn(*l2e), l1_table_offset(addr));
 *t = p2m_recalc_type(recalc || _needs_recalc(flags),
  p2m_flags_to_type(flags), p2m, gfn);
 unmap_domain_page(l2e);
--- a/xen/include/asm-x86/page.h
+++ b/xen/include/asm-x86/page.h
@@ -291,7 +291,7 @@ void copy_page_sse2(void *, const void *
 #define pfn_to_paddr(pfn)   __pfn_to_paddr(pfn)
 #define paddr_to_pfn(pa)__paddr_to_pfn(pa)
 #define paddr_to_pdx(pa)pfn_to_pdx(paddr_to_pfn(pa))
-#define vmap_to_mfn(va) _mfn(l1e_get_pfn(*virt_to_xen_l1e((unsigned 
long)(va
+#define vmap_to_mfn(va) l1e_get_mfn(*virt_to_xen_l1e((unsigned long)(va)))
 #define vmap_to_page(va)mfn_to_page(vmap_to_mfn(va))
 
 #endif /* !defined(__ASSEMBLY__) */



[xen-unstable-smoke test] 156243: regressions - trouble: blocked/fail

2020-10-26 Thread osstest service owner
flight 156243 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/156243/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64   6 xen-buildfail REGR. vs. 156117
 build-arm64-xsm   6 xen-buildfail REGR. vs. 156117
 build-armhf   6 xen-buildfail REGR. vs. 156117

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl   1 build-check(1)   blocked  n/a

version targeted for testing:
 xen  20cd1be5e29577ba4c6c952cc86dfd7cfbd841b3
baseline version:
 xen  6ca70821b59849ad97c3fadc47e63c1a4af1a78c

Last test of basis   156117  2020-10-23 09:01:23 Z3 days
Failing since156120  2020-10-23 14:01:24 Z3 days   37 attempts
Testing same since   156243  2020-10-26 14:00:24 Z0 days1 attempts


People who touched revisions under test:
  Andrew Cooper 
  Bertrand Marquis 
  Christian Lindig 
  George Dunlap 
  Ian Jackson 
  Ian Jackson 
  Jan Beulich 
  Jason Andryuk 
  Juergen Gross 
  Julien Grall 
  Wei Liu 

jobs:
 build-arm64-xsm  fail
 build-amd64  fail
 build-armhf  fail
 build-amd64-libvirt  blocked 
 test-armhf-armhf-xl  blocked 
 test-arm64-arm64-xl-xsm  blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-amd64blocked 
 test-amd64-amd64-libvirt blocked 



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.


commit 20cd1be5e29577ba4c6c952cc86dfd7cfbd841b3
Author: Jan Beulich 
Date:   Mon Oct 26 14:39:42 2020 +0100

PCI: drop dead pci_lock_*pdev() declarations

They have no definitions, and hence users, anywhere.

Signed-off-by: Jan Beulich 
Acked-by: Julien Grall 

commit 2a758376f9e2bd277b6067952517a301da87dc86
Author: Jan Beulich 
Date:   Mon Oct 26 14:38:35 2020 +0100

AMD/IOMMU: correct shattering of super pages

Fill the new page table _before_ installing into a live page table
hierarchy, as installing a blank page first risks I/O faults on
sub-ranges of the original super page which aren't part of the range
for which mappings are being updated.

While at it also do away with mapping and unmapping the same fresh
intermediate page table page once per entry to be written.

Signed-off-by: Jan Beulich 
Reviewed-by: Paul Durrant 

commit 92abe1481c1181b95c7f91846bd1d77f37ee5c5e
Author: Juergen Gross 
Date:   Sun Oct 25 06:45:46 2020 +0100

tools/helpers: fix Arm build by excluding init-xenstore-domain

The support for PVH xenstore-stubdom has broken the Arm build.

Xenstore stubdom isn't supported on Arm, so there is no need to build
the init-xenstore-domain helper.

Build the helper on x86 only.

Signed-off-by: Juergen Gross 
Acked-by: Wei Liu 

commit 4ddd6499d999a7d08cabfda5b0262e473dd5beed
Author: Jason Andryuk 
Date:   Sun May 24 22:55:06 2020 -0400

SUPPORT: Add linux device model stubdom to Toolstack

Add qemu-xen linux device model stubdomain to the Toolstack section as a
Tech Preview.

Signed-off-by: Jason Andryuk 
Acked-by: George Dunlap 
Acked-by: Ian Jackson 

commit 06f0598b41f23c9e4cf7d8c5a05b282de92f3a35
Author: Jan Beulich 
Date:   Fri Oct 23 18:03:18 2020 +0200

x86emul: fix PINSRW and adjust other {,V}PINSR*

The use of simd_packed_int together with no further update to op_bytes
has lead to wrong signaling of #GP(0) for PINSRW without a 16-byte
aligned memory operand. Use simd_none instead and override it after
general decoding with simd_other, like is done for the B/D/Q siblings.

While benign, 

Re: [OSSTEST PATCH] ts-xen-build-prep: Install ninja

2020-10-26 Thread Ian Jackson
Anthony PERARD writes ("[OSSTEST PATCH] ts-xen-build-prep: Install ninja"):
> QEMU upstream now requires ninja to build. (Probably since QEMU commit
> 09e93326e448 ("build: replace ninjatool with ninja"))
> 
> Signed-off-by: Anthony PERARD 

Acked-by: Ian Jackson 

and pushed, thanks.

Ian.



Re: [xen-unstable-smoke test] 156241: regressions - trouble: blocked/fail

2020-10-26 Thread Wei Liu
On Mon, Oct 26, 2020 at 02:35:00PM +0100, Jürgen Groß wrote:
> On 26.10.20 14:27, osstest service owner wrote:
> > flight 156241 xen-unstable-smoke real [real]
> > http://logs.test-lab.xenproject.org/osstest/logs/156241/
> > 
> > Regressions :-(
> > 
> > Tests which did not succeed and are blocking,
> > including tests which could not be run:
> >   build-amd64   6 xen-buildfail REGR. vs. 
> > 156117
> >   build-arm64-xsm   6 xen-buildfail REGR. vs. 
> > 156117
> >   build-armhf   6 xen-buildfail REGR. vs. 
> > 156117
> 
> I'm pretty sure these failures will be fixed by my patch
> 
> "tools/libs: let build depend on official headers"
> 

I've applied that patch. Let's see how it goes.

Wei.



Re: [PATCH] tools/libs: let build depend on official headers

2020-10-26 Thread Wei Liu
On Sun, Oct 25, 2020 at 11:11:29AM +0100, Juergen Gross wrote:
> The build target of a library should depend on the official headers
> of that library, too, as those might be required for building other
> tools.
> 
> Signed-off-by: Juergen Gross 

Acked-by: Wei Liu 



Re: Recent upgrade of 4.13 -> 4.14 issue

2020-10-26 Thread Jürgen Groß

On 26.10.20 14:54, Andrew Cooper wrote:

On 26/10/2020 13:37, Frédéric Pierret wrote:

Hi all,

I'm experiencing problem with a HP ProLiant DL360p Gen8 and recent
upgrade of 4.13 -> 4.14. dom0 and the entire system becomes unstable
and freeze at some point.

I've managed to get three pieces of logs (last one after a reboot and
just before total freeze) by connecting to the serial console with
loglvl options and also redirecting linux kernel output to the serial
console:

```
[ 2150.954883] rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
[ 2150.954934] rcu: 7-...0: (3 GPs behind)
idle=842/1/0x4000 softirq=64670/64671 fqs=14673
[ 2150.954962] (detected by 12, t=60002 jiffies, g=236593, q=126)
[ 2150.954984] Sending NMI from CPU 12 to CPUs 7:
[ 2160.968519] rcu: rcu_sched kthread starved for 10008 jiffies!
g236593 f0x0 RCU_GP_DOING_FQS(6) ->state=0x0 ->cpu=9
[ 2160.968568] rcu: RCU grace-period kthread stack dump:
[ 2160.968586] rcu_sched   R  running task    0    10  2
0x80004000
[ 2160.968612] Call Trace:
[ 2160.968634]  ? xen_hypercall_xen_version+0xa/0x20
[ 2160.968658]  ? xen_force_evtchn_callback+0x9/0x10
[ 2160.968918]  ? check_events+0x12/0x20
[ 2160.968946]  ? schedule+0x39/0xa0
[ 2160.968964]  ? schedule_timeout+0x96/0x150
[ 2160.968981]  ? __next_timer_interrupt+0xd0/0xd0
[ 2160.969002]  ? rcu_gp_fqs_loop+0xea/0x2a0
[ 2160.969019]  ? rcu_gp_kthread+0xb5/0x140
[ 2160.969035]  ? rcu_gp_init+0x470/0x470
[ 2160.969052]  ? kthread+0x115/0x140
[ 2160.969067]  ? __kthread_bind_mask+0x60/0x60
[ 2160.969085]  ? ret_from_fork+0x35/0x40
```

and also

```
[ 2495.945931] CPU: 14 PID: 24181 Comm: Xorg Not tainted
5.4.72-2.qubes.x86_64 #1
[ 2495.945954] Hardware name: HP ProLiant DL360p Gen8, BIOS P71
05/24/2019
[ 2495.945984] RIP: e030:smp_call_function_many+0x20a/0x270
[ 2495.946004] Code: 8a 00 3b 05 4c b5 69 01 89 c7 0f 83 89 fe ff ff
48 63 c7 49 8b 17 48 03 14 c5 80 f9 3d 82 8b 42 18 a8 01 74 09 f3 90
8b 42 18  01 75 f7 eb c9 48 c7 c2 a0 6f 82 82 4c 89 f6 89 df e8 bf b2
8a
[ 2495.946051] RSP: e02b:c90003aa7cf0 EFLAGS: 0202
[ 2495.946068] RAX: 0003 RBX: 0010 RCX:
0007
[ 2495.946090] RDX: 8882413ef640 RSI: 0010 RDI:
0007
[ 2495.946113] RBP: 81082fc0 R08: 0007 R09:

[ 2495.946134] R10:  R11: 8265b6a8 R12:

[ 2495.946156] R13: 0001 R14: 00029ac0 R15:
8882415a9b00
[ 2495.946211] FS:  7a0d51a91a40() GS:88824158()
knlGS:
[ 2495.946235] CS:  e030 DS:  ES:  CR0: 80050033
[ 2495.946253] CR2: 70abab15a000 CR3: 0001e18ee000 CR4:
00040660
[ 2495.946290] Call Trace:
[ 2495.946314]  ? do_kernel_range_flush+0x50/0x50
[ 2495.946334]  on_each_cpu+0x28/0x50
[ 2495.946354]  decrease_reservation+0x22f/0x310
[ 2495.946377]  alloc_xenballooned_pages+0xeb/0x120
[ 2495.946396]  ? __kmalloc+0x183/0x260
[ 2495.946413]  gnttab_alloc_pages+0x11/0x40
[ 2495.946434]  gntdev_alloc_map+0x12f/0x250 [xen_gntdev]
[ 2495.946454]  gntdev_ioctl_map_grant_ref+0x73/0x1d0 [xen_gntdev]
[ 2495.946479]  do_vfs_ioctl+0x2fb/0x490
[ 2495.946500]  ? syscall_trace_enter+0x1d1/0x2c0
[ 2495.946551]  ksys_ioctl+0x5e/0x90
[ 2495.946567]  __x64_sys_ioctl+0x16/0x20
[ 2495.946583]  do_syscall_64+0x5b/0xf0
[ 2495.946601]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 2495.946620] RIP: 0033:0x7a0d51f763bb
[ 2495.946727] Code: 0f 1e fa 48 8b 05 dd aa 0c 00 64 c7 00 26 00 00
00 48 c7 c0 ff ff ff ff c3 66 0f 1f 44 00 00 f3 0f 1e fa b8 10 00 00
00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d ad aa 0c 00 f7 d8 64 89 01
48
[ 2495.946804] RSP: 002b:7fffc48d5058 EFLAGS: 0206 ORIG_RAX:
0010
[ 2495.946863] RAX: ffda RBX: 1000 RCX:
7a0d51f763bb
[ 2495.946885] RDX: 7fffc48d5060 RSI: 00184700 RDI:
0009
[ 2495.946939] RBP: 7fffc48d5100 R08: 7fffc48d512c R09:
7a0d51a30010
[ 2495.946998] R10:  R11: 0206 R12:
7fffc48d5060
[ 2495.947020] R13: 0001 R14: 0009 R15:
0001
[ 2510.964667] rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
[ 2510.964716] rcu: 7-...0: (3 GPs behind)
idle=842/1/0x4000 softirq=64670/64671 fqs=96182
[ 2510.964744] (detected by 12, t=420012 jiffies, g=236593, q=11404)
[ 2510.964769] Sending NMI from CPU 12 to CPUs 7:
[ 2523.945643] watchdog: BUG: soft lockup - CPU#14 stuck for 22s!
[Xorg:24181]
[ 2523.945686] Modules linked in: snd_seq_dummy snd_hrtimer snd_seq
snd_seq_device snd_timer snd soundcore br_netfilter xt_physdev
xen_netback loop bridge stp llc rfkill ebtable_filter ebtables
ip6table_filter ip
6_tables iptable_filter intel_rapl_msr iTCO_wdt ipmi_ssif
iTCO_vendor_support intel_rapl_common sb_edac rapl raid456
async_raid6_recov async_memcpy async_pq async_xor async_tx xor
raid6_pq pcspkr joydev hpilo lpc
_ich hpwdt 

Re: Recent upgrade of 4.13 -> 4.14 issue

2020-10-26 Thread Andrew Cooper
On 26/10/2020 13:37, Frédéric Pierret wrote:
> Hi all,
>
> I'm experiencing problem with a HP ProLiant DL360p Gen8 and recent
> upgrade of 4.13 -> 4.14. dom0 and the entire system becomes unstable
> and freeze at some point.
>
> I've managed to get three pieces of logs (last one after a reboot and
> just before total freeze) by connecting to the serial console with
> loglvl options and also redirecting linux kernel output to the serial
> console:
>
> ```
> [ 2150.954883] rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
> [ 2150.954934] rcu: 7-...0: (3 GPs behind)
> idle=842/1/0x4000 softirq=64670/64671 fqs=14673
> [ 2150.954962] (detected by 12, t=60002 jiffies, g=236593, q=126)
> [ 2150.954984] Sending NMI from CPU 12 to CPUs 7:
> [ 2160.968519] rcu: rcu_sched kthread starved for 10008 jiffies!
> g236593 f0x0 RCU_GP_DOING_FQS(6) ->state=0x0 ->cpu=9
> [ 2160.968568] rcu: RCU grace-period kthread stack dump:
> [ 2160.968586] rcu_sched   R  running task    0    10  2
> 0x80004000
> [ 2160.968612] Call Trace:
> [ 2160.968634]  ? xen_hypercall_xen_version+0xa/0x20
> [ 2160.968658]  ? xen_force_evtchn_callback+0x9/0x10
> [ 2160.968918]  ? check_events+0x12/0x20
> [ 2160.968946]  ? schedule+0x39/0xa0
> [ 2160.968964]  ? schedule_timeout+0x96/0x150
> [ 2160.968981]  ? __next_timer_interrupt+0xd0/0xd0
> [ 2160.969002]  ? rcu_gp_fqs_loop+0xea/0x2a0
> [ 2160.969019]  ? rcu_gp_kthread+0xb5/0x140
> [ 2160.969035]  ? rcu_gp_init+0x470/0x470
> [ 2160.969052]  ? kthread+0x115/0x140
> [ 2160.969067]  ? __kthread_bind_mask+0x60/0x60
> [ 2160.969085]  ? ret_from_fork+0x35/0x40
> ```
>
> and also
>
> ```
> [ 2495.945931] CPU: 14 PID: 24181 Comm: Xorg Not tainted
> 5.4.72-2.qubes.x86_64 #1
> [ 2495.945954] Hardware name: HP ProLiant DL360p Gen8, BIOS P71
> 05/24/2019
> [ 2495.945984] RIP: e030:smp_call_function_many+0x20a/0x270
> [ 2495.946004] Code: 8a 00 3b 05 4c b5 69 01 89 c7 0f 83 89 fe ff ff
> 48 63 c7 49 8b 17 48 03 14 c5 80 f9 3d 82 8b 42 18 a8 01 74 09 f3 90
> 8b 42 18  01 75 f7 eb c9 48 c7 c2 a0 6f 82 82 4c 89 f6 89 df e8 bf b2
> 8a
> [ 2495.946051] RSP: e02b:c90003aa7cf0 EFLAGS: 0202
> [ 2495.946068] RAX: 0003 RBX: 0010 RCX:
> 0007
> [ 2495.946090] RDX: 8882413ef640 RSI: 0010 RDI:
> 0007
> [ 2495.946113] RBP: 81082fc0 R08: 0007 R09:
> 
> [ 2495.946134] R10:  R11: 8265b6a8 R12:
> 
> [ 2495.946156] R13: 0001 R14: 00029ac0 R15:
> 8882415a9b00
> [ 2495.946211] FS:  7a0d51a91a40() GS:88824158()
> knlGS:
> [ 2495.946235] CS:  e030 DS:  ES:  CR0: 80050033
> [ 2495.946253] CR2: 70abab15a000 CR3: 0001e18ee000 CR4:
> 00040660
> [ 2495.946290] Call Trace:
> [ 2495.946314]  ? do_kernel_range_flush+0x50/0x50
> [ 2495.946334]  on_each_cpu+0x28/0x50
> [ 2495.946354]  decrease_reservation+0x22f/0x310
> [ 2495.946377]  alloc_xenballooned_pages+0xeb/0x120
> [ 2495.946396]  ? __kmalloc+0x183/0x260
> [ 2495.946413]  gnttab_alloc_pages+0x11/0x40
> [ 2495.946434]  gntdev_alloc_map+0x12f/0x250 [xen_gntdev]
> [ 2495.946454]  gntdev_ioctl_map_grant_ref+0x73/0x1d0 [xen_gntdev]
> [ 2495.946479]  do_vfs_ioctl+0x2fb/0x490
> [ 2495.946500]  ? syscall_trace_enter+0x1d1/0x2c0
> [ 2495.946551]  ksys_ioctl+0x5e/0x90
> [ 2495.946567]  __x64_sys_ioctl+0x16/0x20
> [ 2495.946583]  do_syscall_64+0x5b/0xf0
> [ 2495.946601]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
> [ 2495.946620] RIP: 0033:0x7a0d51f763bb
> [ 2495.946727] Code: 0f 1e fa 48 8b 05 dd aa 0c 00 64 c7 00 26 00 00
> 00 48 c7 c0 ff ff ff ff c3 66 0f 1f 44 00 00 f3 0f 1e fa b8 10 00 00
> 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d ad aa 0c 00 f7 d8 64 89 01
> 48
> [ 2495.946804] RSP: 002b:7fffc48d5058 EFLAGS: 0206 ORIG_RAX:
> 0010
> [ 2495.946863] RAX: ffda RBX: 1000 RCX:
> 7a0d51f763bb
> [ 2495.946885] RDX: 7fffc48d5060 RSI: 00184700 RDI:
> 0009
> [ 2495.946939] RBP: 7fffc48d5100 R08: 7fffc48d512c R09:
> 7a0d51a30010
> [ 2495.946998] R10:  R11: 0206 R12:
> 7fffc48d5060
> [ 2495.947020] R13: 0001 R14: 0009 R15:
> 0001
> [ 2510.964667] rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
> [ 2510.964716] rcu: 7-...0: (3 GPs behind)
> idle=842/1/0x4000 softirq=64670/64671 fqs=96182
> [ 2510.964744] (detected by 12, t=420012 jiffies, g=236593, q=11404)
> [ 2510.964769] Sending NMI from CPU 12 to CPUs 7:
> [ 2523.945643] watchdog: BUG: soft lockup - CPU#14 stuck for 22s!
> [Xorg:24181]
> [ 2523.945686] Modules linked in: snd_seq_dummy snd_hrtimer snd_seq
> snd_seq_device snd_timer snd soundcore br_netfilter xt_physdev
> xen_netback loop bridge stp llc rfkill ebtable_filter ebtables
> ip6table_filter ip
> 6_tables iptable_filter intel_rapl_msr iTCO_wdt 

[PATCH] x86/svm: Merge hsa and host_vmcb to reduce memory overhead

2020-10-26 Thread Andrew Cooper
The format of the Host State Area is, and has always been, a VMCB.  It is
explicitly safe to put the host VMSAVE data in.

This removes 4k of memory allocation per pCPU.

Signed-off-by: Andrew Cooper 
---
CC: Jan Beulich 
CC: Roger Pau Monné 
CC: Wei Liu 
---
 xen/arch/x86/hvm/svm/svm.c | 27 ---
 1 file changed, 4 insertions(+), 23 deletions(-)

diff --git a/xen/arch/x86/hvm/svm/svm.c b/xen/arch/x86/hvm/svm/svm.c
index cfea5b5523..9ec9ad0646 100644
--- a/xen/arch/x86/hvm/svm/svm.c
+++ b/xen/arch/x86/hvm/svm/svm.c
@@ -72,11 +72,10 @@ static void svm_update_guest_efer(struct vcpu *);
 static struct hvm_function_table svm_function_table;
 
 /*
- * Physical addresses of the Host State Area (for hardware) and vmcb (for Xen)
- * which contains Xen's fs/gs/tr/ldtr and GSBASE/STAR/SYSENTER state when in
- * guest vcpu context.
+ * Host State Area.  This area is used by the processor in non-root mode, and
+ * contains Xen's fs/gs/tr/ldtr and GSBASE/STAR/SYSENTER state required to
+ * leave guest vcpu context.
  */
-static DEFINE_PER_CPU_READ_MOSTLY(paddr_t, hsa);
 static DEFINE_PER_CPU_READ_MOSTLY(paddr_t, host_vmcb);
 #ifdef CONFIG_PV
 static DEFINE_PER_CPU(struct vmcb_struct *, host_vmcb_va);
@@ -1436,15 +1435,8 @@ static bool svm_event_pending(const struct vcpu *v)
 
 static void svm_cpu_dead(unsigned int cpu)
 {
-paddr_t *this_hsa = _cpu(hsa, cpu);
 paddr_t *this_vmcb = _cpu(host_vmcb, cpu);
 
-if ( *this_hsa )
-{
-free_domheap_page(maddr_to_page(*this_hsa));
-*this_hsa = 0;
-}
-
 #ifdef CONFIG_PV
 if ( per_cpu(host_vmcb_va, cpu) )
 {
@@ -1462,7 +1454,6 @@ static void svm_cpu_dead(unsigned int cpu)
 
 static int svm_cpu_up_prepare(unsigned int cpu)
 {
-paddr_t *this_hsa = _cpu(hsa, cpu);
 paddr_t *this_vmcb = _cpu(host_vmcb, cpu);
 nodeid_t node = cpu_to_node(cpu);
 unsigned int memflags = 0;
@@ -1471,16 +1462,6 @@ static int svm_cpu_up_prepare(unsigned int cpu)
 if ( node != NUMA_NO_NODE )
 memflags = MEMF_node(node);
 
-if ( !*this_hsa )
-{
-pg = alloc_domheap_page(NULL, memflags);
-if ( !pg )
-goto err;
-
-clear_domain_page(page_to_mfn(pg));
-*this_hsa = page_to_maddr(pg);
-}
-
 if ( !*this_vmcb )
 {
 pg = alloc_domheap_page(NULL, memflags);
@@ -1597,7 +1578,7 @@ static int _svm_cpu_up(bool bsp)
 write_efer(read_efer() | EFER_SVME);
 
 /* Initialize the HSA for this core. */
-wrmsrl(MSR_K8_VM_HSAVE_PA, per_cpu(hsa, cpu));
+wrmsrl(MSR_K8_VM_HSAVE_PA, per_cpu(host_vmcb, cpu));
 
 /* check for erratum 383 */
 svm_init_erratum_383(c);
-- 
2.11.0




[RFC PATCH] xsm: Re-work domain_create and domain_alloc_security

2020-10-26 Thread Jason Andryuk
Untested!

This only really matters for flask, but all of xsm is updated.

flask_domain_create() and flask_domain_alloc_security() are a strange
pair.

flask_domain_create() serves double duty.  It both assigns sid and
self_sid values and checks if the calling domain has permission to
create the target domain.  It also has special casing for handling dom0.
Meanwhile flask_domain_alloc_security() assigns some special sids, but
waits for others to be assigned in flask_domain_create.  This split
seems to have come about so that the structures are allocated before
calling flask_domain_create().  It also means flask_domain_create is
called in the middle of domain_create.

Re-arrange the two calls.  Let flask_domain_create just check if current
has permission to create ssidref.  Then it can be moved out to do_domctl
and gate entry into domain_create.  This avoids doing partial domain
creation before the permission check.

Have flask_domain_alloc_security() take a ssidref argument.  The ssidref
was already permission checked earlier, so it can just be assigned.
Then the self_sid can be calculated here as well rather than in
flask_domain_create().

The dom0 special casing is moved into flask_domain_alloc_security().
Maybe this should be just a fall-through for the dom0 already created
case.  This code may not be needed any longer.

Signed-off-by: Jason Andryuk 
---
 xen/common/domain.c |  6 ++
 xen/common/domctl.c |  4 
 xen/include/xsm/dummy.h |  6 +++---
 xen/include/xsm/xsm.h   | 12 +--
 xen/xsm/flask/hooks.c   | 48 -
 5 files changed, 34 insertions(+), 42 deletions(-)

diff --git a/xen/common/domain.c b/xen/common/domain.c
index f748806a45..6b1f5ed59d 100644
--- a/xen/common/domain.c
+++ b/xen/common/domain.c
@@ -407,7 +407,8 @@ struct domain *domain_create(domid_t domid,
 
 lock_profile_register_struct(LOCKPROF_TYPE_PERDOM, d, domid);
 
-if ( (err = xsm_alloc_security_domain(d)) != 0 )
+if ( (err = xsm_alloc_security_domain(d, config ? config->ssidref :
+  0)) != 0 )
 goto fail;
 
 atomic_set(>refcnt, 1);
@@ -470,9 +471,6 @@ struct domain *domain_create(domid_t domid,
 if ( !d->iomem_caps || !d->irq_caps )
 goto fail;
 
-if ( (err = xsm_domain_create(XSM_HOOK, d, config->ssidref)) != 0 )
-goto fail;
-
 d->controller_pause_count = 1;
 atomic_inc(>pause_count);
 
diff --git a/xen/common/domctl.c b/xen/common/domctl.c
index af044e2eda..ffdc1a41cd 100644
--- a/xen/common/domctl.c
+++ b/xen/common/domctl.c
@@ -406,6 +406,10 @@ long do_domctl(XEN_GUEST_HANDLE_PARAM(xen_domctl_t) 
u_domctl)
 domid_tdom;
 static domid_t rover = 0;
 
+ret = xsm_domain_create(XSM_HOOK, op->u.createdomain.ssidref);
+if (ret)
+break;
+
 dom = op->domain;
 if ( (dom > 0) && (dom < DOMID_FIRST_RESERVED) )
 {
diff --git a/xen/include/xsm/dummy.h b/xen/include/xsm/dummy.h
index 7ae3c40eb5..29c4ca9951 100644
--- a/xen/include/xsm/dummy.h
+++ b/xen/include/xsm/dummy.h
@@ -104,10 +104,10 @@ static XSM_INLINE void xsm_security_domaininfo(struct 
domain *d,
 return;
 }
 
-static XSM_INLINE int xsm_domain_create(XSM_DEFAULT_ARG struct domain *d, u32 
ssidref)
+static XSM_INLINE int xsm_domain_create(XSM_DEFAULT_ARG u32 ssidref)
 {
 XSM_ASSERT_ACTION(XSM_HOOK);
-return xsm_default_action(action, current->domain, d);
+return xsm_default_action(action, current->domain, NULL);
 }
 
 static XSM_INLINE int xsm_getdomaininfo(XSM_DEFAULT_ARG struct domain *d)
@@ -163,7 +163,7 @@ static XSM_INLINE int xsm_readconsole(XSM_DEFAULT_ARG 
uint32_t clear)
 return xsm_default_action(action, current->domain, NULL);
 }
 
-static XSM_INLINE int xsm_alloc_security_domain(struct domain *d)
+static XSM_INLINE int xsm_alloc_security_domain(struct domain *d, uint32_t 
ssidref)
 {
 return 0;
 }
diff --git a/xen/include/xsm/xsm.h b/xen/include/xsm/xsm.h
index 358ec13ba8..c1d2ef5832 100644
--- a/xen/include/xsm/xsm.h
+++ b/xen/include/xsm/xsm.h
@@ -46,7 +46,7 @@ typedef enum xsm_default xsm_default_t;
 struct xsm_operations {
 void (*security_domaininfo) (struct domain *d,
 struct xen_domctl_getdomaininfo *info);
-int (*domain_create) (struct domain *d, u32 ssidref);
+int (*domain_create) (u32 ssidref);
 int (*getdomaininfo) (struct domain *d);
 int (*domctl_scheduler_op) (struct domain *d, int op);
 int (*sysctl_scheduler_op) (int op);
@@ -71,7 +71,7 @@ struct xsm_operations {
 int (*grant_copy) (struct domain *d1, struct domain *d2);
 int (*grant_query_size) (struct domain *d1, struct domain *d2);
 
-int (*alloc_security_domain) (struct domain *d);
+int (*alloc_security_domain) (struct domain *d, uint32_t ssidref);
 void (*free_security_domain) (struct domain *d);
 int (*alloc_security_evtchn) 

Recent upgrade of 4.13 -> 4.14 issue

2020-10-26 Thread Frédéric Pierret

Hi all,

I'm experiencing problem with a HP ProLiant DL360p Gen8 and recent upgrade of 4.13 
-> 4.14. dom0 and the entire system becomes unstable and freeze at some point.

I've managed to get three pieces of logs (last one after a reboot and just 
before total freeze) by connecting to the serial console with loglvl options 
and also redirecting linux kernel output to the serial console:

```
[ 2150.954883] rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
[ 2150.954934] rcu: 7-...0: (3 GPs behind) idle=842/1/0x4000 
softirq=64670/64671 fqs=14673
[ 2150.954962]  (detected by 12, t=60002 jiffies, g=236593, q=126)
[ 2150.954984] Sending NMI from CPU 12 to CPUs 7:
[ 2160.968519] rcu: rcu_sched kthread starved for 10008 jiffies! g236593 f0x0 
RCU_GP_DOING_FQS(6) ->state=0x0 ->cpu=9
[ 2160.968568] rcu: RCU grace-period kthread stack dump:
[ 2160.968586] rcu_sched   R  running task010  2 0x80004000
[ 2160.968612] Call Trace:
[ 2160.968634]  ? xen_hypercall_xen_version+0xa/0x20
[ 2160.968658]  ? xen_force_evtchn_callback+0x9/0x10
[ 2160.968918]  ? check_events+0x12/0x20
[ 2160.968946]  ? schedule+0x39/0xa0
[ 2160.968964]  ? schedule_timeout+0x96/0x150
[ 2160.968981]  ? __next_timer_interrupt+0xd0/0xd0
[ 2160.969002]  ? rcu_gp_fqs_loop+0xea/0x2a0
[ 2160.969019]  ? rcu_gp_kthread+0xb5/0x140
[ 2160.969035]  ? rcu_gp_init+0x470/0x470
[ 2160.969052]  ? kthread+0x115/0x140
[ 2160.969067]  ? __kthread_bind_mask+0x60/0x60
[ 2160.969085]  ? ret_from_fork+0x35/0x40
```

and also

```
[ 2495.945931] CPU: 14 PID: 24181 Comm: Xorg Not tainted 5.4.72-2.qubes.x86_64 
#1
[ 2495.945954] Hardware name: HP ProLiant DL360p Gen8, BIOS P71 05/24/2019
[ 2495.945984] RIP: e030:smp_call_function_many+0x20a/0x270
[ 2495.946004] Code: 8a 00 3b 05 4c b5 69 01 89 c7 0f 83 89 fe ff ff 48 63 c7 49 8b 
17 48 03 14 c5 80 f9 3d 82 8b 42 18 a8 01 74 09 f3 90 8b 42 18  01 75 f7 eb 
c9 48 c7 c2 a0 6f 82 82 4c 89 f6 89 df e8 bf b2
8a
[ 2495.946051] RSP: e02b:c90003aa7cf0 EFLAGS: 0202
[ 2495.946068] RAX: 0003 RBX: 0010 RCX: 0007
[ 2495.946090] RDX: 8882413ef640 RSI: 0010 RDI: 0007
[ 2495.946113] RBP: 81082fc0 R08: 0007 R09: 
[ 2495.946134] R10:  R11: 8265b6a8 R12: 
[ 2495.946156] R13: 0001 R14: 00029ac0 R15: 8882415a9b00
[ 2495.946211] FS:  7a0d51a91a40() GS:88824158() 
knlGS:
[ 2495.946235] CS:  e030 DS:  ES:  CR0: 80050033
[ 2495.946253] CR2: 70abab15a000 CR3: 0001e18ee000 CR4: 00040660
[ 2495.946290] Call Trace:
[ 2495.946314]  ? do_kernel_range_flush+0x50/0x50
[ 2495.946334]  on_each_cpu+0x28/0x50
[ 2495.946354]  decrease_reservation+0x22f/0x310
[ 2495.946377]  alloc_xenballooned_pages+0xeb/0x120
[ 2495.946396]  ? __kmalloc+0x183/0x260
[ 2495.946413]  gnttab_alloc_pages+0x11/0x40
[ 2495.946434]  gntdev_alloc_map+0x12f/0x250 [xen_gntdev]
[ 2495.946454]  gntdev_ioctl_map_grant_ref+0x73/0x1d0 [xen_gntdev]
[ 2495.946479]  do_vfs_ioctl+0x2fb/0x490
[ 2495.946500]  ? syscall_trace_enter+0x1d1/0x2c0
[ 2495.946551]  ksys_ioctl+0x5e/0x90
[ 2495.946567]  __x64_sys_ioctl+0x16/0x20
[ 2495.946583]  do_syscall_64+0x5b/0xf0
[ 2495.946601]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 2495.946620] RIP: 0033:0x7a0d51f763bb
[ 2495.946727] Code: 0f 1e fa 48 8b 05 dd aa 0c 00 64 c7 00 26 00 00 00 48 c7 c0 ff 
ff ff ff c3 66 0f 1f 44 00 00 f3 0f 1e fa b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff 
ff 73 01 c3 48 8b 0d ad aa 0c 00 f7 d8 64 89 01
48
[ 2495.946804] RSP: 002b:7fffc48d5058 EFLAGS: 0206 ORIG_RAX: 
0010
[ 2495.946863] RAX: ffda RBX: 1000 RCX: 7a0d51f763bb
[ 2495.946885] RDX: 7fffc48d5060 RSI: 00184700 RDI: 0009
[ 2495.946939] RBP: 7fffc48d5100 R08: 7fffc48d512c R09: 7a0d51a30010
[ 2495.946998] R10:  R11: 0206 R12: 7fffc48d5060
[ 2495.947020] R13: 0001 R14: 0009 R15: 0001
[ 2510.964667] rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
[ 2510.964716] rcu: 7-...0: (3 GPs behind) idle=842/1/0x4000 
softirq=64670/64671 fqs=96182
[ 2510.964744]  (detected by 12, t=420012 jiffies, g=236593, q=11404)
[ 2510.964769] Sending NMI from CPU 12 to CPUs 7:
[ 2523.945643] watchdog: BUG: soft lockup - CPU#14 stuck for 22s! [Xorg:24181]
[ 2523.945686] Modules linked in: snd_seq_dummy snd_hrtimer snd_seq 
snd_seq_device snd_timer snd soundcore br_netfilter xt_physdev xen_netback loop 
bridge stp llc rfkill ebtable_filter ebtables ip6table_filter ip
6_tables iptable_filter intel_rapl_msr iTCO_wdt ipmi_ssif iTCO_vendor_support 
intel_rapl_common sb_edac rapl raid456 async_raid6_recov async_memcpy async_pq 
async_xor async_tx xor raid6_pq pcspkr joydev hpilo lpc
_ich hpwdt ioatdma dca tg3 r8169 ipmi_si ipmi_devintf ipmi_msghandler 
acpi_power_meter 

Re: XSM and the idle domain

2020-10-26 Thread Jason Andryuk
On Thu, Oct 22, 2020 at 1:01 PM Hongyan Xia  wrote:
>
> On Thu, 2020-10-22 at 13:51 +0100, Andrew Cooper wrote:
> > On 21/10/2020 15:34, Hongyan Xia wrote:
> > > The first question came up during ongoing work in LiveUpdate. After
> > > an
> > > LU, the next Xen needs to restore all domains. To do that, some
> > > hypercalls need to be issued from the idle domain context and
> > > apparently XSM does not like it.
> >
> > There is no such thing as issuing hypercalls from the idle domain
> > (context or otherwise), because the idle domain does not have enough
> > associated guest state for anything to make the requisite
> > SYSCALL/INT80/VMCALL/VMMCALL invocation.
> >
> > I presume from this comment that what you mean is that you're calling
> > the plain hypercall functions, context checks and everything, from
> > the
> > idle context?
>
> Yep, the restore code just calls the hypercall functions from idle
> context.
>
> > If so, this is buggy for more reasons than just XSM objecting to its
> > calling context, and that XSM is merely the first thing to explode.
> > Therefore, I don't think modifications to XSM are applicable to
> > solving
> > the problem.
> >
> > (Of course, this is all speculation because there's no concrete
> > implementation to look at.)
>
> Another explosion is the inability to create hypercall preemption,
> which for now is disabled when the calling context is the idle domain.
> Apart from XSM and preemption, the LU prototype works fine. We only
> reuse a limited number of hypercall functions and are not trying to be
> able to call all possible hypercalls from idle.

I wonder if for domain_create, it wouldn't be better to move
xsm_domain_create() out to the domctl (hypercall entry) and check it
there.  That would side-step xsm in domain_create.  Flask would need
to be modified for that.  I've an untested patch doing the
rearranging, which I'll send as a follow up.

What other hypercalls are you having issues with?  Those could also be
refactored so the hypercall entry checks permissions, and the actual
work is done in a directly callable function.

> Having a dedicated domLU just like domB (or reusing domB) sounds like a
> viable option. If the overhead can be made low enough then we won't
> need to work around XSM and hypercall preemption.
>
> Although the question was whether XSM should interact with the idle
> domain. With a good design LU should be able to sidestep this though.

Circling back to the main topic, is the idle domain Xen, or is it
distinct?  It runs in the context of Xen, so Xen isn't really in a
place to enforce policy on itself.  Hongyan, as you said earlier,
applying XSM is more of a debugging feature at that point than a
security feature.  And as Jan pointed out, you can have problems if
XSM prevents the hypervisor from performing an action it doesn't
expect to fail.

Regards,
Jason



Re: [xen-unstable-smoke test] 156241: regressions - trouble: blocked/fail

2020-10-26 Thread Jürgen Groß

On 26.10.20 14:27, osstest service owner wrote:

flight 156241 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/156241/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
  build-amd64   6 xen-buildfail REGR. vs. 156117
  build-arm64-xsm   6 xen-buildfail REGR. vs. 156117
  build-armhf   6 xen-buildfail REGR. vs. 156117


I'm pretty sure these failures will be fixed by my patch

"tools/libs: let build depend on official headers"


Juergen



Re: Xen on RP4

2020-10-26 Thread Julien Grall

Hi Elliott,

On 24/10/2020 06:35, Elliott Mitchell wrote:

On Fri, Oct 23, 2020 at 04:59:30PM -0700, Stefano Stabellini wrote:

Note that I tried to repro the issue here at my end but it works for me
with device tree. So the swiotlb_init memory allocation failure probably
only shows on ACPI, maybe because ACPI is reserving too much low memory.


Found it.  Take a look at 437b0aa06a014ce174e24c0d3530b3e9ab19b18b

  PLATFORM_START(rpi4, "Raspberry Pi 4")
  .compatible = rpi4_dt_compat,
  .blacklist_dev  = rpi4_blacklist_dev,
+.dma_bitsize= 30,
  PLATFORM_END

Where this is used to match against a *device-tree*.


Right. When we introduced ACPI in Xen, we made the assumption there 
would be no need for per-platform workaround.



ACPI has a distinct
means of specifying a limited DMA-width; the above fails, because it
assumes a *device-tree*.


Do you know if it would be possible to infer from the ACPI static table 
the DMA-width?


If not, is there a way to uniquely identify the platform?


Cheers,

--
Julien Grall



[xen-unstable-smoke test] 156241: regressions - trouble: blocked/fail

2020-10-26 Thread osstest service owner
flight 156241 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/156241/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64   6 xen-buildfail REGR. vs. 156117
 build-arm64-xsm   6 xen-buildfail REGR. vs. 156117
 build-armhf   6 xen-buildfail REGR. vs. 156117

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl   1 build-check(1)   blocked  n/a

version targeted for testing:
 xen  92abe1481c1181b95c7f91846bd1d77f37ee5c5e
baseline version:
 xen  6ca70821b59849ad97c3fadc47e63c1a4af1a78c

Last test of basis   156117  2020-10-23 09:01:23 Z3 days
Failing since156120  2020-10-23 14:01:24 Z2 days   36 attempts
Testing same since   156241  2020-10-26 12:02:24 Z0 days1 attempts


People who touched revisions under test:
  Andrew Cooper 
  Bertrand Marquis 
  Christian Lindig 
  George Dunlap 
  Ian Jackson 
  Ian Jackson 
  Jan Beulich 
  Jason Andryuk 
  Juergen Gross 
  Wei Liu 

jobs:
 build-arm64-xsm  fail
 build-amd64  fail
 build-armhf  fail
 build-amd64-libvirt  blocked 
 test-armhf-armhf-xl  blocked 
 test-arm64-arm64-xl-xsm  blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-amd64blocked 
 test-amd64-amd64-libvirt blocked 



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.


commit 92abe1481c1181b95c7f91846bd1d77f37ee5c5e
Author: Juergen Gross 
Date:   Sun Oct 25 06:45:46 2020 +0100

tools/helpers: fix Arm build by excluding init-xenstore-domain

The support for PVH xenstore-stubdom has broken the Arm build.

Xenstore stubdom isn't supported on Arm, so there is no need to build
the init-xenstore-domain helper.

Build the helper on x86 only.

Signed-off-by: Juergen Gross 
Acked-by: Wei Liu 

commit 4ddd6499d999a7d08cabfda5b0262e473dd5beed
Author: Jason Andryuk 
Date:   Sun May 24 22:55:06 2020 -0400

SUPPORT: Add linux device model stubdom to Toolstack

Add qemu-xen linux device model stubdomain to the Toolstack section as a
Tech Preview.

Signed-off-by: Jason Andryuk 
Acked-by: George Dunlap 
Acked-by: Ian Jackson 

commit 06f0598b41f23c9e4cf7d8c5a05b282de92f3a35
Author: Jan Beulich 
Date:   Fri Oct 23 18:03:18 2020 +0200

x86emul: fix PINSRW and adjust other {,V}PINSR*

The use of simd_packed_int together with no further update to op_bytes
has lead to wrong signaling of #GP(0) for PINSRW without a 16-byte
aligned memory operand. Use simd_none instead and override it after
general decoding with simd_other, like is done for the B/D/Q siblings.

While benign, for consistency also use DstImplicit instead of DstReg
in x86_decode_twobyte().

PINSR{B,D,Q} also had a stray (redundant) get_fpu() invocation, which
gets dropped.

For further consistency also
- use src.bytes instead of op_bytes in relevant memcpy() invocations,
- avoid the pointless updating of op_bytes (all we care about later is
  that the value be less than 16).

Signed-off-by: Jan Beulich 
Acked-by: Andrew Cooper 

commit 9af5e2b31b4e6f3892b4614ecd0a619af5d64d7e
Author: Juergen Gross 
Date:   Mon Oct 19 17:27:54 2020 +0200

tools/libs/store: don't use symbolic links for external files

Instead of using symbolic links to include files from xenstored use
the vpath directive and an include path.

Signed-off-by: Juergen Gross 
Acked-by: Christian Lindig 
Tested-by: Bertrand Marquis 
Acked-by: Ian Jackson 

commit 

[qemu-mainline test] 156240: regressions - trouble: blocked/fail/pass/starved

2020-10-26 Thread osstest service owner
flight 156240 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/156240/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm   6 xen-buildfail REGR. vs. 152631
 build-amd64   6 xen-buildfail REGR. vs. 152631
 build-arm64-xsm   6 xen-buildfail REGR. vs. 152631
 build-arm64   6 xen-buildfail REGR. vs. 152631
 build-i3866 xen-buildfail REGR. vs. 152631
 build-i386-xsm6 xen-buildfail REGR. vs. 152631

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-thunderx  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 test-amd64-i386-pair  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-i386  1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-amd64  1 build-check(1)   blocked  n/a
 test-amd64-coresched-i386-xl  1 build-check(1)   blocked  n/a
 test-amd64-coresched-amd64-xl  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-xsm   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-shadow1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-rtds  1 build-check(1)   blocked  n/a
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-ws16-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-win7-amd64  1 build-check(1) blocked n/a
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked 
n/a
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow  1 build-check(1) blocked n/a
 build-i386-libvirt1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-amd64-amd64-amd64-pvgrub  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qcow2 1 build-check(1)   blocked  n/a
 test-amd64-amd64-dom0pvh-xl-amd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvshim1 build-check(1)   blocked  n/a
 test-amd64-amd64-dom0pvh-xl-intel  1 build-check(1)   blocked  n/a
 test-amd64-amd64-i386-pvgrub  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvhv2-intel  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-xl-pvhv2-amd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-amd64-amd64-pair 1 build-check(1)   blocked  n/a
 test-amd64-amd64-pygrub   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-credit2   1 build-check(1)   blocked  n/a
 test-amd64-amd64-qemuu-freebsd11-amd64  1 build-check(1)   blocked n/a
 test-amd64-amd64-qemuu-freebsd12-amd64  1 build-check(1)   blocked n/a
 test-amd64-amd64-xl-credit1   1 build-check(1)   blocked  n/a
 test-amd64-amd64-qemuu-nested-amd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-qemuu-nested-intel  1 build-check(1)  blocked n/a
 test-amd64-amd64-xl   1 build-check(1)   blocked  n/a
 test-amd64-i386-qemuu-rhel6hvm-amd  1 build-check(1)   blocked n/a
 test-amd64-i386-qemuu-rhel6hvm-intel  1 build-check(1) blocked n/a
 test-amd64-i386-xl1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-pvshim 1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64  1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked 
n/a
 

Re: [XEN PATCH v1] xen/arm : Add support for SMMUv3 driver

2020-10-26 Thread Ash Wilding
Hi,

> 1. atomic_set_release
> 2. atomic_fetch_andnot_relaxed
> 3. atomic_cond_read_relaxed
> 4. atomic_long_cond_read_relaxed
> 5. atomic_long_xor
> 6. atomic_set_release
> 7. atomic_cmpxchg_relaxed might be we can use atomic_cmpxchg that is
>implemented in XEN need to check.
> 8. atomic_dec_return_release
> 9. atomic_fetch_inc_relaxed


If we're going to pull in Linux's implementations of the above atomics
helpers for SMMUv3, and given the majority of SMMUv3 systems are v8.1+
with LSE, perhaps this would be a good time to drop the current
atomic.h in Xen completely and pull in both Linux's LL/SC and LSE
helpers, then use a new Kconfig to toggle between them?

Back in 5d45ecabe3 [1] Jan mentioned we probably want to avoid relying
on gcc atomics helpers as we can't switch between LL/SC and LSE
atomics. With the above we'd be able to drop the reference to gcc's
built-in __sync_fetch_and_add() in xen/include/asm-arm/system.h by
making arch_fetch_and_add() pull in the explicit implementation of the
helper.

Thoughts?

Thanks,
Ash.

[1] https://xenbits.xen.org/gitweb/?p=xen.git;a=commit;h=5d45ecabe3




[xen-unstable-smoke test] 156239: regressions - trouble: blocked/fail

2020-10-26 Thread osstest service owner
flight 156239 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/156239/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64   6 xen-buildfail REGR. vs. 156117
 build-arm64-xsm   6 xen-buildfail REGR. vs. 156117
 build-armhf   6 xen-buildfail REGR. vs. 156117

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl   1 build-check(1)   blocked  n/a

version targeted for testing:
 xen  4ddd6499d999a7d08cabfda5b0262e473dd5beed
baseline version:
 xen  6ca70821b59849ad97c3fadc47e63c1a4af1a78c

Last test of basis   156117  2020-10-23 09:01:23 Z3 days
Failing since156120  2020-10-23 14:01:24 Z2 days   35 attempts
Testing same since   156129  2020-10-23 18:01:24 Z2 days   34 attempts


People who touched revisions under test:
  Andrew Cooper 
  Bertrand Marquis 
  Christian Lindig 
  George Dunlap 
  Ian Jackson 
  Ian Jackson 
  Jan Beulich 
  Jason Andryuk 
  Juergen Gross 
  Wei Liu 

jobs:
 build-arm64-xsm  fail
 build-amd64  fail
 build-armhf  fail
 build-amd64-libvirt  blocked 
 test-armhf-armhf-xl  blocked 
 test-arm64-arm64-xl-xsm  blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-amd64blocked 
 test-amd64-amd64-libvirt blocked 



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.


commit 4ddd6499d999a7d08cabfda5b0262e473dd5beed
Author: Jason Andryuk 
Date:   Sun May 24 22:55:06 2020 -0400

SUPPORT: Add linux device model stubdom to Toolstack

Add qemu-xen linux device model stubdomain to the Toolstack section as a
Tech Preview.

Signed-off-by: Jason Andryuk 
Acked-by: George Dunlap 
Acked-by: Ian Jackson 

commit 06f0598b41f23c9e4cf7d8c5a05b282de92f3a35
Author: Jan Beulich 
Date:   Fri Oct 23 18:03:18 2020 +0200

x86emul: fix PINSRW and adjust other {,V}PINSR*

The use of simd_packed_int together with no further update to op_bytes
has lead to wrong signaling of #GP(0) for PINSRW without a 16-byte
aligned memory operand. Use simd_none instead and override it after
general decoding with simd_other, like is done for the B/D/Q siblings.

While benign, for consistency also use DstImplicit instead of DstReg
in x86_decode_twobyte().

PINSR{B,D,Q} also had a stray (redundant) get_fpu() invocation, which
gets dropped.

For further consistency also
- use src.bytes instead of op_bytes in relevant memcpy() invocations,
- avoid the pointless updating of op_bytes (all we care about later is
  that the value be less than 16).

Signed-off-by: Jan Beulich 
Acked-by: Andrew Cooper 

commit 9af5e2b31b4e6f3892b4614ecd0a619af5d64d7e
Author: Juergen Gross 
Date:   Mon Oct 19 17:27:54 2020 +0200

tools/libs/store: don't use symbolic links for external files

Instead of using symbolic links to include files from xenstored use
the vpath directive and an include path.

Signed-off-by: Juergen Gross 
Acked-by: Christian Lindig 
Tested-by: Bertrand Marquis 
Acked-by: Ian Jackson 

commit 588756db020e73e6f5e4407bbf78fbd53f15b731
Author: Juergen Gross 
Date:   Mon Oct 19 17:27:54 2020 +0200

tools/libs/guest: don't use symbolic links for xenctrl headers

Instead of using symbolic links for accessing the xenctrl private
headers use an include path instead.

Signed-off-by: Juergen Gross 
Acked-by: Christian Lindig 
Tested-by: Bertrand Marquis 
Acked-by: Ian Jackson 

commit 4664034cdc720a52913bc26358240bb9d3798527
Author: Juergen 

Re: [PATCH] tools/helpers: fix Arm build by excluding init-xenstore-domain

2020-10-26 Thread Wei Liu
On Sun, Oct 25, 2020 at 06:45:46AM +0100, Juergen Gross wrote:
> The support for PVH xenstore-stubdom has broken the Arm build.
> 
> Xenstore stubdom isn't supported on Arm, so there is no need to build
> the init-xenstore-domain helper.
> 
> Build the helper on x86 only.
> 
> Signed-off-by: Juergen Gross 

Acked-by: Wei Liu 

I have applied this patch to unblock osstest.



Re: [XEN PATCH v1] xen/arm : Add support for SMMUv3 driver

2020-10-26 Thread Rahul Singh
Hello Julien,

> On 23 Oct 2020, at 4:19 pm, Julien Grall  wrote:
> 
> 
> 
> On 23/10/2020 15:27, Rahul Singh wrote:
>> Hello Julien,
>>> On 23 Oct 2020, at 2:00 pm, Julien Grall  wrote:
>>> 
>>> 
>>> 
>>> On 23/10/2020 12:35, Rahul Singh wrote:
 Hello,
> On 23 Oct 2020, at 1:02 am, Stefano Stabellini  
> wrote:
> 
> On Thu, 22 Oct 2020, Julien Grall wrote:
 On 20/10/2020 16:25, Rahul Singh wrote:
> Add support for ARM architected SMMUv3 implementations. It is based on
> the Linux SMMUv3 driver.
> Major differences between the Linux driver are as follows:
> 1. Only Stage-2 translation is supported as compared to the Linux 
> driver
>that supports both Stage-1 and Stage-2 translations.
> 2. Use P2M  page table instead of creating one as SMMUv3 has the
>capability to share the page tables with the CPU.
> 3. Tasklets is used in place of threaded IRQ's in Linux for event 
> queue
>and priority queue IRQ handling.
 
 Tasklets are not a replacement for threaded IRQ. In particular, they 
 will
 have priority over anything else (IOW nothing will run on the pCPU 
 until
 they are done).
 
 Do you know why Linux is using thread. Is it because of long running
 operations?
>>> 
>>> Yes you are right because of long running operations Linux is using the
>>> threaded IRQs.
>>> 
>>> SMMUv3 reports fault/events bases on memory-based circular buffer 
>>> queues not
>>> based on the register. As per my understanding, it is time-consuming to
>>> process the memory based queues in interrupt context because of that 
>>> Linux
>>> is using threaded IRQ to process the faults/events from SMMU.
>>> 
>>> I didn’t find any other solution in XEN in place of tasklet to defer the
>>> work, that’s why I used tasklet in XEN in replacement of threaded IRQs. 
>>> If
>>> we do all work in interrupt context we will make XEN less responsive.
>> 
>> So we need to make sure that Xen continue to receives interrupts, but we 
>> also
>> need to make sure that a vCPU bound to the pCPU is also responsive.
>> 
>>> 
>>> If you know another solution in XEN that will be used to defer the work 
>>> in
>>> the interrupt please let me know I will try to use that.
>> 
>> One of my work colleague encountered a similar problem recently. He had 
>> a long
>> running tasklet and wanted to be broken down in smaller chunk.
>> 
>> We decided to use a timer to reschedule the taslket in the future. This 
>> allows
>> the scheduler to run other loads (e.g. vCPU) for some time.
>> 
>> This is pretty hackish but I couldn't find a better solution as tasklet 
>> have
>> high priority.
>> 
>> Maybe the other will have a better idea.
> 
> Julien's suggestion is a good one.
> 
> But I think tasklets can be configured to be called from the idle_loop,
> in which case they are not run in interrupt context?
> 
  Yes you are right tasklet will be scheduled from the idle_loop that is 
 not interrupt conext.
>>> 
>>> This depends on your tasklet. Some will run from the softirq context which 
>>> is usually (for Arm) on the return of an exception.
>>> 
>> Thanks for the info. I will check and will get better understanding of the 
>> tasklet how it will run in XEN.
> 
> 4. Latest version of the Linux SMMUv3 code implements the commands 
> queue
>access functions based on atomic operations implemented in Linux.
 
 Can you provide more details?
>>> 
>>> I tried to port the latest version of the SMMUv3 code than I observed 
>>> that
>>> in order to port that code I have to also port atomic operation 
>>> implemented
>>> in Linux to XEN. As latest Linux code uses atomic operation to process 
>>> the
>>> command queues 
>>> (atomic_cond_read_relaxed(),atomic_long_cond_read_relaxed() ,
>>> atomic_fetch_andnot_relaxed()) .
>> 
>> Thank you for the explanation. I think it would be best to import the 
>> atomic
>> helpers and use the latest code.
>> 
>> This will ensure that we don't re-introduce bugs and also buy us some 
>> time
>> before the Linux and Xen driver diverge again too much.
>> 
>> Stefano, what do you think?
> 
> I think you are right.
 Yes, I agree with you to have XEN code in sync with Linux code that's why 
 I started with to port the Linux atomic operations to XEN  then I realised 
 that it is not straightforward to port atomic operations and it requires 
 lots of effort and testing. Therefore I decided to port the code before 
 the atomic operation is introduced in Linux.
>>> 
>>> Hmmm... I would not have expected a lot of effort required to 

Re: [Xen-devel] [PATCH] xen: credit2: document that min_rqd is valid and ok to use

2020-10-26 Thread George Dunlap
On Thu, Mar 26, 2020 at 5:09 PM Dario Faggioli  wrote:
>
> Code is a bit involved, and it is not easy to tell that min_rqd, inside
> csched2_res_pick() is actually pointing to a runqueue, when it is
> dereferenced.
>
> Add a comment and an ASSERT() for that.
>
> Suggested-by: Jan Beulich 
> Signed-off-by: Dario Faggioli 
> ---
> Cc: Jürgen Groß 
> ---
>  xen/common/sched/credit2.c |7 +++
>  1 file changed, 7 insertions(+)
>
> diff --git a/xen/common/sched/credit2.c b/xen/common/sched/credit2.c
> index c7241944a8..9da51e624b 100644
> --- a/xen/common/sched/credit2.c
> +++ b/xen/common/sched/credit2.c
> @@ -2387,6 +2387,13 @@ csched2_res_pick(const struct scheduler *ops,
const struct sched_unit *unit)
>  goto out_up;
>  }
>
> +/*
> + * If we're here, min_rqd must be valid. In fact, either we picked a
> + * runqueue in the "list_for_each" (as min_avgload is initialized to
> + * MAX_LOAD) or we just did that (in the "else" branch) above.
> + */


Sorry it's taken so long to get back to you on this.

The problem with this is that there are actually *three* alternate clauses
above:

1. (has_soft && min_s_rqd)
2. min_rqd
3. 

It's obvious that if we hit #2 or #3, that min_rqd will be set.  But it's
not immediately obvious why the condition in #1 guarantees that min_rqd
will be set.

Is it because if we get to the point in the above loop where min_s_rqd is
set, then min_rqd will always be set if it hasn't been set already?  Or to
put it a different way -- the only way for min_rqd *not* to be set is if it
always bailed before min_s_rqd was set?

 -George


[xen-unstable test] 156228: tolerable FAIL

2020-10-26 Thread osstest service owner
flight 156228 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/156228/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-rtds 20 guest-localmigrate/x10 fail pass in 156196

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt 16 saverestore-support-checkfail  like 156196
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stopfail like 156196
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stopfail like 156196
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop fail like 156196
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stopfail like 156196
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop fail like 156196
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop fail like 156196
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stopfail like 156196
 test-armhf-armhf-libvirt-raw 15 saverestore-support-checkfail  like 156196
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 156196
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop fail like 156196
 test-amd64-i386-xl-pvshim14 guest-start  fail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt 15 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-checkfail never pass
 test-armhf-armhf-libvirt 15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-checkfail  never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-vhd  14 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  15 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-checkfail   never pass

version targeted for testing:
 xen  6ca70821b59849ad97c3fadc47e63c1a4af1a78c
baseline version:
 xen  6ca70821b59849ad97c3fadc47e63c1a4af1a78c

Last test of basis   156228  2020-10-26 01:51:27 Z0 days
Testing same since  (not found) 0 attempts

jobs:
 build-amd64-xsm  pass
 build-arm64-xsm  pass
 build-i386-xsm   pass
 build-amd64-xtf  

Re: [PATCH] tools/libs/light: fix race in Makefile

2020-10-26 Thread Jan Beulich
On 26.10.2020 10:46, Jürgen Groß wrote:
> On 26.10.20 10:34, Jan Beulich wrote:
>> What I don't understand here is why this two step moving around of
>> headers is used: Instead of the above pattern rule, can't the rule
>> to generate _libxl_type%.h, _libxl_type%_json.h,
>> _libxl_type%_private.h, and _libxl_type%.c put the relevant header
>> files right into their designated place? This would allow the
>> pattern rule to go away, albeit I'd then still be unclear about
>> the specific race you did observe.
> 
> This would require to replace the pattern rules used to generate the
> files by per-file rules instead, as e.g. _libxl_types_json.h and
> _libxl_types_internal_json.h are matching the same pattern, but they
> need to end up in different directories.

Ah, right - I didn't pay attention to the *_internal*.h needs.

Jan



[qemu-mainline test] 156234: regressions - FAIL

2020-10-26 Thread osstest service owner
flight 156234 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/156234/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-xsm   6 xen-buildfail REGR. vs. 152631
 build-amd64   6 xen-buildfail REGR. vs. 152631
 build-arm64-xsm   6 xen-buildfail REGR. vs. 152631
 build-arm64   6 xen-buildfail REGR. vs. 152631
 build-i3866 xen-buildfail REGR. vs. 152631
 build-i386-xsm6 xen-buildfail REGR. vs. 152631
 build-armhf   6 xen-buildfail REGR. vs. 152631

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-thunderx  1 build-check(1)   blocked  n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-arndale   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-credit1   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-rtds  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-vhd   1 build-check(1)   blocked  n/a
 test-amd64-i386-pair  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-i386  1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-amd64  1 build-check(1)   blocked  n/a
 test-amd64-coresched-i386-xl  1 build-check(1)   blocked  n/a
 test-amd64-coresched-amd64-xl  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-xsm   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-shadow1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-rtds  1 build-check(1)   blocked  n/a
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-ws16-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-win7-amd64  1 build-check(1) blocked n/a
 build-arm64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-dmrestrict-amd64-dmrestrict 1 build-check(1) blocked 
n/a
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm  1 build-check(1) blocked n/a
 build-armhf-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-shadow  1 build-check(1) blocked n/a
 build-i386-libvirt1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-amd64-amd64-amd64-pvgrub  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qcow2 1 build-check(1)   blocked  n/a
 test-amd64-amd64-dom0pvh-xl-amd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvshim1 build-check(1)   blocked  n/a
 test-amd64-amd64-dom0pvh-xl-intel  1 build-check(1)   blocked  n/a
 test-amd64-amd64-i386-pvgrub  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvhv2-intel  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-xl-pvhv2-amd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-amd64-amd64-pair 1 build-check(1)   blocked  n/a
 test-amd64-amd64-pygrub   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-credit2   1 build-check(1)   blocked  n/a
 test-amd64-amd64-qemuu-freebsd11-amd64  1 build-check(1)   blocked n/a
 test-amd64-amd64-qemuu-freebsd12-amd64  1 build-check(1)   blocked n/a
 test-amd64-amd64-xl-credit1   1 build-check(1)   blocked  n/a
 test-amd64-amd64-qemuu-nested-amd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-qemuu-nested-intel  1 

Re: [PATCH] tools/libs/light: fix race in Makefile

2020-10-26 Thread Jürgen Groß

On 26.10.20 10:34, Jan Beulich wrote:

On 25.10.2020 11:12, Juergen Gross wrote:

The header $(INCLUDE)/_lixl_list.h matches two different rules, which
can result in build breakage. Fix that.


While I don't doubt you having observed a race, I'm not sure this is
true, and hence I'm also not sure the change is going to address it:
Aiui the two rules you talk about are the one you change and

$(XEN_INCLUDE)/_%.h: _%.h
$(call move-if-changed,_$*.h,$(XEN_INCLUDE)/_$*.h)

But a pattern rule doesn't come into play when a specific rule for
a file exists.


Hmm, true. I didn't see the race, but spotted the suspected ambiguity
just by chance.



What I don't understand here is why this two step moving around of
headers is used: Instead of the above pattern rule, can't the rule
to generate _libxl_type%.h, _libxl_type%_json.h,
_libxl_type%_private.h, and _libxl_type%.c put the relevant header
files right into their designated place? This would allow the
pattern rule to go away, albeit I'd then still be unclear about
the specific race you did observe.


This would require to replace the pattern rules used to generate the
files by per-file rules instead, as e.g. _libxl_types_json.h and
_libxl_types_internal_json.h are matching the same pattern, but they
need to end up in different directories.

In the end I think this patch can just be dropped.

Sorry for the noise,

Juergen



[ovmf test] 156232: all pass - PUSHED

2020-10-26 Thread osstest service owner
flight 156232 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/156232/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf b70c4fdcde83689d8cd1e5e2faf598d0087934a3
baseline version:
 ovmf 264eccb5dfc345c2e004883f00e62959f818fafd

Last test of basis   156102  2020-10-22 17:10:41 Z3 days
Testing same since   156232  2020-10-26 03:10:04 Z0 days1 attempts


People who touched revisions under test:
  Bob Feng 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 pass
 test-amd64-i386-xl-qemuu-ovmf-amd64  pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   264eccb5df..b70c4fdcde  b70c4fdcde83689d8cd1e5e2faf598d0087934a3 -> 
xen-tested-master



Re: [PATCH v3] x86/pv: inject #UD for entirely missing SYSCALL callbacks

2020-10-26 Thread Jan Beulich
On 26.10.2020 10:40, Jan Beulich wrote:

And of course this should have

From: Andrew Cooper 

right here, sorry.

Jan

> In the case that no 64-bit SYSCALL callback is registered, the guest
> will be crashed when 64-bit userspace executes a SYSCALL instruction,
> which would be a userspace => kernel DoS.  Similarly for 32-bit
> userspace when no 32-bit SYSCALL callback was registered either.
> 
> This has been the case ever since the introduction of 64bit PV support,
> but behaves unlike all other SYSCALL/SYSENTER callbacks in Xen, which
> yield #GP/#UD in userspace before the callback is registered, and are
> therefore safe by default.
> 
> This change does constitute a change in the PV ABI, for the corner case
> of a PV guest kernel not registering a 64-bit callback (which has to be
> considered a defacto requirement of the unwritten PV ABI, considering
> there is no PV equivalent of EFER.SCE).
> 
> It brings the behaviour in line with PV32 SYSCALL/SYSENTER, and PV64
> SYSENTER (safe by default, until explicitly enabled).
> 
> Signed-off-by: Andrew Cooper 
> Signed-off-by: Jan Beulich 



[PATCH v3] x86/pv: inject #UD for entirely missing SYSCALL callbacks

2020-10-26 Thread Jan Beulich
In the case that no 64-bit SYSCALL callback is registered, the guest
will be crashed when 64-bit userspace executes a SYSCALL instruction,
which would be a userspace => kernel DoS.  Similarly for 32-bit
userspace when no 32-bit SYSCALL callback was registered either.

This has been the case ever since the introduction of 64bit PV support,
but behaves unlike all other SYSCALL/SYSENTER callbacks in Xen, which
yield #GP/#UD in userspace before the callback is registered, and are
therefore safe by default.

This change does constitute a change in the PV ABI, for the corner case
of a PV guest kernel not registering a 64-bit callback (which has to be
considered a defacto requirement of the unwritten PV ABI, considering
there is no PV equivalent of EFER.SCE).

It brings the behaviour in line with PV32 SYSCALL/SYSENTER, and PV64
SYSENTER (safe by default, until explicitly enabled).

Signed-off-by: Andrew Cooper 
Signed-off-by: Jan Beulich 
---
v3:
 * Split this change off of "x86/pv: Inject #UD for missing SYSCALL
   callbacks", to allow the uncontroversial part of that change to go
   in. Add conditional "rex64" for UREGS_rip adjustment. (Is branching
   over just the REX prefix too much trickery even for an unlikely to be
   taken code path?)

v2:
 * Drop unnecessary instruction suffixes
 * Don't truncate #UD entrypoint to 32 bits

--- a/xen/arch/x86/x86_64/entry.S
+++ b/xen/arch/x86/x86_64/entry.S
@@ -33,11 +33,27 @@ ENTRY(switch_to_kernel)
 cmoveq VCPU_syscall32_addr(%rbx),%rax
 testq %rax,%rax
 cmovzq VCPU_syscall_addr(%rbx),%rax
-movq  %rax,TRAPBOUNCE_eip(%rdx)
 /* TB_flags = VGCF_syscall_disables_events ? TBF_INTERRUPT : 0 */
 btl   $_VGCF_syscall_disables_events,VCPU_guest_context_flags(%rbx)
 setc  %cl
 leal  (,%rcx,TBF_INTERRUPT),%ecx
+
+test  %rax, %rax
+UNLIKELY_START(z, syscall_no_callback) /* TB_eip == 0 => #UD */
+mov   VCPU_trap_ctxt(%rbx), %rdi
+movl  $X86_EXC_UD, UREGS_entry_vector(%rsp)
+cmpw  $FLAT_USER_CS32, UREGS_cs(%rsp)
+je0f
+rex64   # subl => subq
+0:
+subl  $2, UREGS_rip(%rsp)
+mov   X86_EXC_UD * TRAPINFO_sizeof + TRAPINFO_eip(%rdi), %rax
+testb $4, X86_EXC_UD * TRAPINFO_sizeof + TRAPINFO_flags(%rdi)
+setnz %cl
+lea   TBF_EXCEPTION(, %rcx, TBF_INTERRUPT), %ecx
+UNLIKELY_END(syscall_no_callback)
+
+movq  %rax, TRAPBOUNCE_eip(%rdx)
 movb  %cl,TRAPBOUNCE_flags(%rdx)
 call  create_bounce_frame
 andl  $~X86_EFLAGS_DF,UREGS_eflags(%rsp)



[linux-linus test] 156225: regressions - FAIL

2020-10-26 Thread osstest service owner
flight 156225 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/156225/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-xsm7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ws16-amd64  7 xen-install   fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-intel  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 7 xen-install fail REGR. 
vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-intel  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow 7 xen-install fail REGR. vs. 
152332
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-libvirt-xsm   7 xen-install  fail REGR. vs. 152332
 test-amd64-coresched-i386-xl  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-amd  7 xen-installfail REGR. vs. 152332
 test-amd64-i386-xl-qemut-ws16-amd64  7 xen-install   fail REGR. vs. 152332
 test-amd64-i386-pair 10 xen-install/src_host fail REGR. vs. 152332
 test-amd64-i386-pair 11 xen-install/dst_host fail REGR. vs. 152332
 test-amd64-i386-libvirt   7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 
152332
 test-amd64-i386-qemuu-rhel6hvm-amd  7 xen-installfail REGR. vs. 152332
 test-amd64-i386-xl7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-examine   6 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-raw7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-pvshim 7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-freebsd10-amd64  7 xen-install   fail REGR. vs. 152332
 test-amd64-i386-xl-shadow 7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-freebsd10-i386  7 xen-installfail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-win7-amd64  7 xen-install   fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-win7-amd64  7 xen-install   fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ovmf-amd64  7 xen-install   fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 10 xen-install/src_host fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 11 xen-install/dst_host fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 7 xen-install fail REGR. 
vs. 152332
 test-arm64-arm64-xl   8 xen-boot fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1   8 xen-boot fail REGR. vs. 152332
 test-arm64-arm64-xl-xsm  10 host-ping-check-xen  fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm  8 xen-boot fail REGR. vs. 152332
 test-arm64-arm64-examine  8 reboot   fail REGR. vs. 152332
 test-amd64-amd64-examine  4 memdisk-try-append   fail REGR. vs. 152332
 test-amd64-amd64-amd64-pvgrub 20 guest-stop  fail REGR. vs. 152332
 test-amd64-amd64-i386-pvgrub 19 guest-localmigrate/x10   fail REGR. vs. 152332
 test-armhf-armhf-xl-credit1   8 xen-boot fail REGR. vs. 152332
 test-armhf-armhf-libvirt  8 xen-boot fail REGR. vs. 152332
 test-armhf-armhf-examine  8 reboot   fail REGR. vs. 152332
 test-armhf-armhf-xl-cubietruck  8 xen-boot   fail REGR. vs. 152332
 test-armhf-armhf-xl-multivcpu  8 xen-bootfail REGR. vs. 152332
 test-armhf-armhf-libvirt-raw  8 xen-boot fail REGR. vs. 152332
 test-armhf-armhf-xl   8 xen-boot fail REGR. vs. 152332
 test-armhf-armhf-xl-credit2   8 xen-boot fail REGR. vs. 152332
 test-armhf-armhf-xl-vhd   8 xen-boot fail REGR. vs. 152332

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-rtds 18 guest-localmigrate   fail REGR. vs. 152332
 test-armhf-armhf-xl-rtds  8 xen-boot fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-seattle  11 leak-check/basis(11)fail blocked in 152332
 test-arm64-arm64-xl-credit2  11 leak-check/basis(11)fail blocked in 152332
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stopfail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stopfail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stopfail like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stopfail like 152332
 

Re: [PATCH 05/12] docs: fix hypfs path documentation

2020-10-26 Thread Jan Beulich
On 26.10.2020 10:13, Juergen Gross wrote:
> The /params/* entry is missing a writable tag.
> 
> Signed-off-by: Juergen Gross 

Acked-by: Jan Beulich 



Re: [PATCH] tools/libs/light: fix race in Makefile

2020-10-26 Thread Jan Beulich
On 25.10.2020 11:12, Juergen Gross wrote:
> The header $(INCLUDE)/_lixl_list.h matches two different rules, which
> can result in build breakage. Fix that.

While I don't doubt you having observed a race, I'm not sure this is
true, and hence I'm also not sure the change is going to address it:
Aiui the two rules you talk about are the one you change and

$(XEN_INCLUDE)/_%.h: _%.h
$(call move-if-changed,_$*.h,$(XEN_INCLUDE)/_$*.h)

But a pattern rule doesn't come into play when a specific rule for
a file exists.

What I don't understand here is why this two step moving around of
headers is used: Instead of the above pattern rule, can't the rule
to generate _libxl_type%.h, _libxl_type%_json.h,
_libxl_type%_private.h, and _libxl_type%.c put the relevant header
files right into their designated place? This would allow the
pattern rule to go away, albeit I'd then still be unclear about
the specific race you did observe.

Jan

> Signed-off-by: Juergen Gross 
> ---
>  tools/libs/light/Makefile | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/tools/libs/light/Makefile b/tools/libs/light/Makefile
> index 3424fdb61b..370537ed38 100644
> --- a/tools/libs/light/Makefile
> +++ b/tools/libs/light/Makefile
> @@ -203,9 +203,9 @@ _libxl.api-for-check: $(XEN_INCLUDE)/libxl.h $(AUTOINCS)
>   >$@.new
>   mv -f $@.new $@
>  
> -$(XEN_INCLUDE)/_libxl_list.h: 
> $(XEN_INCLUDE)/xen-external/bsd-sys-queue-h-seddery 
> $(XEN_INCLUDE)/xen-external/bsd-sys-queue.h
> - $(PERL) $^ --prefix=libxl >$(notdir $@).new
> - $(call move-if-changed,$(notdir $@).new,$@)
> +_libxl_list.h: $(XEN_INCLUDE)/xen-external/bsd-sys-queue-h-seddery 
> $(XEN_INCLUDE)/xen-external/bsd-sys-queue.h
> + $(PERL) $^ --prefix=libxl >$@.new
> + $(call move-if-changed,$@.new,$@)
>  
>  _libxl_save_msgs_helper.c _libxl_save_msgs_callout.c \
>  _libxl_save_msgs_helper.h _libxl_save_msgs_callout.h: \
> 




[libvirt test] 156233: regressions - FAIL

2020-10-26 Thread osstest service owner
flight 156233 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/156233/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-libvirt   6 libvirt-buildfail REGR. vs. 151777
 build-i386-libvirt6 libvirt-buildfail REGR. vs. 151777
 build-arm64-libvirt   6 libvirt-buildfail REGR. vs. 151777
 build-armhf-libvirt   6 libvirt-buildfail REGR. vs. 151777

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt  1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-qcow2  1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked  n/a

version targeted for testing:
 libvirt  04b1c2d1e2e12abcca22380827edaa058399f4fa
baseline version:
 libvirt  2c846fa6bcc11929c9fb857a22430fb9945654ad

Last test of basis   151777  2020-07-10 04:19:19 Z  108 days
Failing since151818  2020-07-11 04:18:52 Z  107 days  102 attempts
Testing same since   156163  2020-10-24 04:19:13 Z2 days3 attempts


People who touched revisions under test:
  Adolfo Jayme Barrientos 
  Andika Triwidada 
  Andrea Bolognani 
  Balázs Meskó 
  Bastien Orivel 
  Bihong Yu 
  Binfeng Wu 
  Boris Fiuczynski 
  Christian Ehrhardt 
  Cole Robinson 
  Collin Walling 
  Cornelia Huck 
  Côme Borsoi 
  Daniel Henrique Barboza 
  Daniel Letai 
  Daniel P. Berrange 
  Daniel P. Berrangé 
  Erik Skultety 
  Fabian Freyer 
  Fangge Jin 
  Fedora Weblate Translation 
  Halil Pasic 
  Han Han 
  Hao Wang 
  Ian Wienand 
  Jamie Strandboge 
  Jamie Strandboge 
  Jean-Baptiste Holcroft 
  Jianan Gao 
  Jim Fehlig 
  Jin Yan 
  Jiri Denemark 
  Jonathon Jongsma 
  Ján Tomko 
  Kashyap Chamarthy 
  Kevin Locke 
  Laine Stump 
  Liao Pingfang 
  Lin Ma 
  Lin Ma 
  Marc Hartmayer 
  Marek Marczykowski-Górecki 
  Markus Schade 
  Martin Kletzander 
  Masayoshi Mizuma 
  Matt Coleman 
  Matt Coleman 
  Mauro Matteo Cascella 
  Michal Privoznik 
  Michał Smyk 
  Milo Casagrande 
  Neal Gompa 
  Nico Pache 
  Nikolay Shirokovskiy 
  Olesya Gerasimenko 
  Patrick Magauran 
  Paulo de Rezende Pinatti 
  Pavel Hrdina 
  Peter Krempa 
  Pino Toscano 
  Pino Toscano 
  Piotr Drąg 
  Prathamesh Chavan 
  Roman Bogorodskiy 
  Roman Bolshakov 
  Ryan Schmidt 
  Sam Hartman 
  Scott Shambarger 
  Sebastian Mitterle 
  Simon Gaiser 
  Stefan Bader 
  Stefan Berger 
  Szymon Scholz 
  Thomas Huth 
  Tim Wiederhake 
  Tomáš Golembiovský 
  Wang Xin 
  Weblate 
  Yang Hang 
  Yanqiu Zhang 
  Yi Li 
  Yi Wang 
  Yuri Chornoivan 
  Zheng Chuan 
  zhenwei pi 
  Zhenyu Zheng 

jobs:
 build-amd64-xsm  pass
 build-arm64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-arm64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  fail
 build-arm64-libvirt  fail
 build-armhf-libvirt  fail
 build-i386-libvirt   fail
 build-amd64-pvopspass
 build-arm64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm   blocked 
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsmblocked 
 test-amd64-amd64-libvirt-xsm blocked 
 test-arm64-arm64-libvirt-xsm blocked 
 

Re: [PATCH] xen/arm: Remove EXPERT dependancy

2020-10-26 Thread Julien Grall

Hi Stefano,

On 23/10/2020 17:57, Stefano Stabellini wrote:

On Fri, 23 Oct 2020, Julien Grall wrote:

Hi Stefano,

On 22/10/2020 22:17, Stefano Stabellini wrote:

On Thu, 22 Oct 2020, Julien Grall wrote:

On 22/10/2020 02:43, Elliott Mitchell wrote:

Linux requires UEFI support to be enabled on ARM64 devices.  While many
ARM64 devices lack ACPI, the writing seems to be on the wall of
UEFI/ACPI
potentially taking over.  Some common devices may need ACPI table
support.

Presently I think it is worth removing the dependancy on CONFIG_EXPERT.


The idea behind EXPERT is to gate any feature that is not considered to be
stable/complete enough to be used in production.


Yes, and from that point of view I don't think we want to remove EXPERT
from ACPI yet. However, the idea of hiding things behind EXPERT works
very well for new esoteric features, something like memory introspection
or memory overcommit.


Memaccess is not very new ;).


It does not work well for things that are actually
required to boot on the platform.


I am not sure where is the problem. It is easy to select EXPERT from the
menuconfig. It also hints the user that the feature may not fully work.



Typically ACPI systems don't come with device tree at all (RPi4 being an
exception), so users don't really have much of a choice in the matter.


And they typically have IOMMUs.



  From that point of view, it would be better to remove EXPERT from ACPI,
maybe even build ACPI by default, *but* to add a warning at boot saying
something like:

"ACPI support is experimental. Boot using Device Tree if you can."


That would better convey the risks of using ACPI, while at the same time
making it a bit easier for users to boot on their ACPI-only platforms.


Right, I agree that this make easier for users to boot Xen on ACPI-only
platform. However, based on above, it is easy enough for a developper to
rebuild Xen with ACPI and EXPERT enabled.

So what sort of users are you targeting?


Somebody trying Xen for the first time, they might know how to build it
but they might not know that ACPI is not available by default, and they
might not know that they need to enable EXPERT in order to get the ACPI
option in the menu. It is easy to do once you know it is there,
otherwise one might not know where to look in the menu.


Right, EXPERT can now be enabled using Kconfig. So it is not very 
different from an option Foo has been hidden because the dependency Bar 
has not been selected.


It should be easy enough (if it is not we should fix it) to figure out 
the dependency when searching the option via menuconfig.






I am sort of okay to remove EXPERT.


OK. This would help (even without building it by default) because as you
go and look at the menu the first time, you'll find ACPI among the
options right away.


To be honest, this step is probably the easiest in the full process to 
get Xen build and booted on Arm.


I briefly looked at Elliot's v2, and I can't keep thinking that we are 
trying to re-invent EXPERT for ACPI because we think the feature is 
*more* important than any other feature gated by EXPERT.


In fact, all the features behind EXPERT are important. But they have 
been gated by EXPERT because they are not mature enough.


We already moved EXPERT from a command line option to a menuconfig 
option. So it should be easy enough to enable it now. If it still not 
the case, then we should improve it.


But I don't think ACPI is mature enough to deserve a different 
treatment. It would be more useful to get to the stage where ACPI can 
work without any crash/issue first.






But I still think building ACPI by default
is still wrong because our default .config is meant to be (security)
supported. I don't think ACPI can earn this qualification today.


Certainly we don't want to imply ACPI is security supported. I was
looking at SUPPORT.md and it is only says:

"""
EXPERT and DEBUG Kconfig options are not security supported. Other
Kconfig options are supported, if the related features are marked as
supported in this document.
"""

So technically we could enable ACPI in the build by default as ACPI for
ARM is marked as experimental. However, I can see that it is not a
great idea to enable by default an unsupported option in the kconfig, so
from that point of view it might be best to leave ACPI disabled by
default. Probably the best compromise at this time.
From my understanding, the goal of EXPERT was to gate such features. 
With your suggestion, it is not clear to me what's the difference 
between "experimental" and option gated by "EXPERT".


Do you mind clarifying?

Cheers,

--
Julien Grall



[xen-unstable-smoke test] 156237: regressions - trouble: blocked/fail

2020-10-26 Thread osstest service owner
flight 156237 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/156237/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64   6 xen-buildfail REGR. vs. 156117
 build-arm64-xsm   6 xen-buildfail REGR. vs. 156117
 build-armhf   6 xen-buildfail REGR. vs. 156117

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl   1 build-check(1)   blocked  n/a

version targeted for testing:
 xen  4ddd6499d999a7d08cabfda5b0262e473dd5beed
baseline version:
 xen  6ca70821b59849ad97c3fadc47e63c1a4af1a78c

Last test of basis   156117  2020-10-23 09:01:23 Z3 days
Failing since156120  2020-10-23 14:01:24 Z2 days   34 attempts
Testing same since   156129  2020-10-23 18:01:24 Z2 days   33 attempts


People who touched revisions under test:
  Andrew Cooper 
  Bertrand Marquis 
  Christian Lindig 
  George Dunlap 
  Ian Jackson 
  Ian Jackson 
  Jan Beulich 
  Jason Andryuk 
  Juergen Gross 
  Wei Liu 

jobs:
 build-arm64-xsm  fail
 build-amd64  fail
 build-armhf  fail
 build-amd64-libvirt  blocked 
 test-armhf-armhf-xl  blocked 
 test-arm64-arm64-xl-xsm  blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-amd64blocked 
 test-amd64-amd64-libvirt blocked 



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.


commit 4ddd6499d999a7d08cabfda5b0262e473dd5beed
Author: Jason Andryuk 
Date:   Sun May 24 22:55:06 2020 -0400

SUPPORT: Add linux device model stubdom to Toolstack

Add qemu-xen linux device model stubdomain to the Toolstack section as a
Tech Preview.

Signed-off-by: Jason Andryuk 
Acked-by: George Dunlap 
Acked-by: Ian Jackson 

commit 06f0598b41f23c9e4cf7d8c5a05b282de92f3a35
Author: Jan Beulich 
Date:   Fri Oct 23 18:03:18 2020 +0200

x86emul: fix PINSRW and adjust other {,V}PINSR*

The use of simd_packed_int together with no further update to op_bytes
has lead to wrong signaling of #GP(0) for PINSRW without a 16-byte
aligned memory operand. Use simd_none instead and override it after
general decoding with simd_other, like is done for the B/D/Q siblings.

While benign, for consistency also use DstImplicit instead of DstReg
in x86_decode_twobyte().

PINSR{B,D,Q} also had a stray (redundant) get_fpu() invocation, which
gets dropped.

For further consistency also
- use src.bytes instead of op_bytes in relevant memcpy() invocations,
- avoid the pointless updating of op_bytes (all we care about later is
  that the value be less than 16).

Signed-off-by: Jan Beulich 
Acked-by: Andrew Cooper 

commit 9af5e2b31b4e6f3892b4614ecd0a619af5d64d7e
Author: Juergen Gross 
Date:   Mon Oct 19 17:27:54 2020 +0200

tools/libs/store: don't use symbolic links for external files

Instead of using symbolic links to include files from xenstored use
the vpath directive and an include path.

Signed-off-by: Juergen Gross 
Acked-by: Christian Lindig 
Tested-by: Bertrand Marquis 
Acked-by: Ian Jackson 

commit 588756db020e73e6f5e4407bbf78fbd53f15b731
Author: Juergen Gross 
Date:   Mon Oct 19 17:27:54 2020 +0200

tools/libs/guest: don't use symbolic links for xenctrl headers

Instead of using symbolic links for accessing the xenctrl private
headers use an include path instead.

Signed-off-by: Juergen Gross 
Acked-by: Christian Lindig 
Tested-by: Bertrand Marquis 
Acked-by: Ian Jackson 

commit 4664034cdc720a52913bc26358240bb9d3798527
Author: Juergen 

[PATCH 07/12] xen/hypfs: pass real failure reason up from hypfs_get_entry()

2020-10-26 Thread Juergen Gross
Instead of handling all errors from hypfs_get_entry() as ENOENT pass
up the real error value via ERR_PTR().

Signed-off-by: Juergen Gross 
---
 xen/common/hypfs.c | 12 ++--
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/xen/common/hypfs.c b/xen/common/hypfs.c
index e655e8cfc7..97260bd4a3 100644
--- a/xen/common/hypfs.c
+++ b/xen/common/hypfs.c
@@ -182,7 +182,7 @@ static struct hypfs_entry *hypfs_get_entry_rel(struct 
hypfs_entry_dir *dir,
 while ( again )
 {
 if ( dir->e.type != XEN_HYPFS_TYPE_DIR )
-return NULL;
+return ERR_PTR(-ENOENT);
 
 if ( !*path )
 return >e;
@@ -201,7 +201,7 @@ static struct hypfs_entry *hypfs_get_entry_rel(struct 
hypfs_entry_dir *dir,
  struct hypfs_entry_dir, 
e);
 
 if ( cmp < 0 )
-return NULL;
+return ERR_PTR(-ENOENT);
 if ( !cmp && strlen(entry->name) == name_len )
 {
 if ( !*end )
@@ -216,13 +216,13 @@ static struct hypfs_entry *hypfs_get_entry_rel(struct 
hypfs_entry_dir *dir,
 }
 }
 
-return NULL;
+return ERR_PTR(-ENOENT);
 }
 
 static struct hypfs_entry *hypfs_get_entry(const char *path)
 {
 if ( path[0] != '/' )
-return NULL;
+return ERR_PTR(-EINVAL);
 
 return hypfs_get_entry_rel(_root, path + 1);
 }
@@ -446,9 +446,9 @@ long do_hypfs_op(unsigned int cmd,
 goto out;
 
 entry = hypfs_get_entry(path);
-if ( !entry )
+if ( IS_ERR(entry) )
 {
-ret = -ENOENT;
+ret = PTR_ERR(entry);
 goto out;
 }
 
-- 
2.26.2




[PATCH 10/12] xen/hypfs: add cpupool directories

2020-10-26 Thread Juergen Gross
Add /cpupool/ directories to hypfs. Those are completely
dynamic, so the related hypfs access functions need to be implemented.

Signed-off-by: Juergen Gross 
---
 docs/misc/hypfs-paths.pandoc |  9 +
 xen/common/sched/cpupool.c   | 78 
 2 files changed, 87 insertions(+)

diff --git a/docs/misc/hypfs-paths.pandoc b/docs/misc/hypfs-paths.pandoc
index 6c7b2f7ee3..aaca1cdf92 100644
--- a/docs/misc/hypfs-paths.pandoc
+++ b/docs/misc/hypfs-paths.pandoc
@@ -175,6 +175,15 @@ The major version of Xen.
 
 The minor version of Xen.
 
+ /cpupool/
+
+A directory of all current cpupools.
+
+ /cpupool/*/
+
+The individual cpupools. Each entry is a directory with the name being the
+cpupool-id (e.g. /cpupool/0/).
+
  /params/
 
 A directory of runtime parameters.
diff --git a/xen/common/sched/cpupool.c b/xen/common/sched/cpupool.c
index 84f326ea63..8612ee5cf6 100644
--- a/xen/common/sched/cpupool.c
+++ b/xen/common/sched/cpupool.c
@@ -13,6 +13,8 @@
 
 #include 
 #include 
+#include 
+#include 
 #include 
 #include 
 #include 
@@ -992,6 +994,78 @@ static struct notifier_block cpu_nfb = {
 .notifier_call = cpu_callback
 };
 
+#ifdef CONFIG_HYPFS
+static HYPFS_DIR_INIT(cpupool_pooldir, "id");
+
+static int cpupool_dir_read(const struct hypfs_entry *entry,
+XEN_GUEST_HANDLE_PARAM(void) uaddr)
+{
+int ret = 0;
+struct cpupool **q;
+
+spin_lock(_lock);
+
+for_each_cpupool(q)
+{
+ret = hypfs_read_dyndir_id_entry(_pooldir, (*q)->cpupool_id,
+ !(*q)->next, );
+if ( ret )
+break;
+}
+
+spin_unlock(_lock);
+
+return ret;
+}
+
+static unsigned int cpupool_dir_getsize(const struct hypfs_entry *entry)
+{
+struct cpupool **q;
+unsigned int size = 0;
+
+spin_lock(_lock);
+
+for_each_cpupool(q)
+size += HYPFS_DIRENTRY_SIZE(snprintf(NULL, 0, "%d", (*q)->cpupool_id));
+
+spin_unlock(_lock);
+
+return size;
+}
+
+static struct hypfs_entry *cpupool_dir_findentry(struct hypfs_entry_dir *dir,
+ const char *name,
+ unsigned int name_len)
+{
+unsigned long id;
+const char *end;
+struct cpupool *cpupool;
+
+id = simple_strtoul(name, , 10);
+if ( id > INT_MAX || end != name + name_len )
+return ERR_PTR(-ENOENT);
+
+spin_lock(_lock);
+
+cpupool = __cpupool_find_by_id(id, true);
+
+spin_unlock(_lock);
+
+if ( !cpupool )
+return ERR_PTR(-ENOENT);
+
+return hypfs_gen_dyndir_entry_id(_pooldir, id);
+}
+
+static struct hypfs_funcs cpupool_dir_funcs = {
+.read = cpupool_dir_read,
+.getsize = cpupool_dir_getsize,
+.findentry = cpupool_dir_findentry,
+};
+
+static HYPFS_VARDIR_INIT(cpupool_dir, "cpupool", _dir_funcs);
+#endif
+
 static int __init cpupool_init(void)
 {
 unsigned int cpu;
@@ -999,6 +1073,10 @@ static int __init cpupool_init(void)
 
 cpupool_gran_init();
 
+#ifdef CONFIG_HYPFS
+hypfs_add_dir(_root, _dir, true);
+#endif
+
 cpupool0 = cpupool_create(0, 0, );
 BUG_ON(cpupool0 == NULL);
 cpupool_put(cpupool0);
-- 
2.26.2




[PATCH 08/12] xen/hypfs: support dynamic hypfs nodes

2020-10-26 Thread Juergen Gross
Add a getsize() function pointer to struct hypfs_funcs for being able
to have dynamically filled entries without the need to take the hypfs
lock each time the contents are being generated.

For directories add a findentry callback to the vector and modify
hypfs_get_entry_rel() to use it.

Add a HYPFS_VARDIR_INIT() macro for intializing such a directory
statically, taking a struct hypfs_funcs pointer as parameter additional
to those of HYPFS_DIR_INIT().

Modify HYPFS_VARSIZE_INIT() to take the function vector pointer as an
additional parameter as this will be needed for dynamical entries.

Move DIRENTRY_SIZE() macro to hypfs.h as it will be needed by the read
function of a directory with dynamically generated entries.

For being able to let the generic hypfs coding continue to work on
normal struct hypfs_entry entities even for dynamical nodes add some
infrastructure for allocating a working area for the current hypfs
request in order to store needed information for traversing the tree.
This area is anchored in a percpu pointer and can be retrieved by any
level of the dynamic entries. It will be freed automatically when
dropping the hypfs lock.

Signed-off-by: Juergen Gross 
---
 xen/common/hypfs.c  | 124 +---
 xen/include/xen/hypfs.h |  39 +
 2 files changed, 108 insertions(+), 55 deletions(-)

diff --git a/xen/common/hypfs.c b/xen/common/hypfs.c
index 97260bd4a3..4c226a06b4 100644
--- a/xen/common/hypfs.c
+++ b/xen/common/hypfs.c
@@ -19,28 +19,29 @@
 CHECK_hypfs_dirlistentry;
 #endif
 
-#define DIRENTRY_NAME_OFF offsetof(struct xen_hypfs_dirlistentry, name)
-#define DIRENTRY_SIZE(name_len) \
-(DIRENTRY_NAME_OFF +\
- ROUNDUP((name_len) + 1, alignof(struct xen_hypfs_direntry)))
-
 struct hypfs_funcs hypfs_dir_funcs = {
 .read = hypfs_read_dir,
+.getsize = hypfs_getsize,
+.findentry = hypfs_dir_findentry,
 };
 struct hypfs_funcs hypfs_leaf_ro_funcs = {
 .read = hypfs_read_leaf,
+.getsize = hypfs_getsize,
 };
 struct hypfs_funcs hypfs_leaf_wr_funcs = {
 .read = hypfs_read_leaf,
 .write = hypfs_write_leaf,
+.getsize = hypfs_getsize,
 };
 struct hypfs_funcs hypfs_bool_wr_funcs = {
 .read = hypfs_read_leaf,
 .write = hypfs_write_bool,
+.getsize = hypfs_getsize,
 };
 struct hypfs_funcs hypfs_custom_wr_funcs = {
 .read = hypfs_read_leaf,
 .write = hypfs_write_custom,
+.getsize = hypfs_getsize,
 };
 
 static DEFINE_RWLOCK(hypfs_lock);
@@ -50,6 +51,7 @@ enum hypfs_lock_state {
 hypfs_write_locked
 };
 static DEFINE_PER_CPU(enum hypfs_lock_state, hypfs_locked);
+static DEFINE_PER_CPU(void *, hypfs_dyndata);
 
 HYPFS_DIR_INIT(hypfs_root, "");
 
@@ -71,9 +73,12 @@ static void hypfs_write_lock(void)
 
 static void hypfs_unlock(void)
 {
-enum hypfs_lock_state locked = this_cpu(hypfs_locked);
+unsigned int cpu = smp_processor_id();
+enum hypfs_lock_state locked = per_cpu(hypfs_locked, cpu);
+
+XFREE(per_cpu(hypfs_dyndata, cpu));
 
-this_cpu(hypfs_locked) = hypfs_unlocked;
+per_cpu(hypfs_locked, cpu) = hypfs_unlocked;
 
 switch ( locked )
 {
@@ -88,6 +93,23 @@ static void hypfs_unlock(void)
 }
 }
 
+void *hypfs_alloc_dyndata(unsigned long size, unsigned long align)
+{
+unsigned int cpu = smp_processor_id();
+
+ASSERT(per_cpu(hypfs_locked, cpu) != hypfs_unlocked);
+ASSERT(per_cpu(hypfs_dyndata, cpu) == NULL);
+
+per_cpu(hypfs_dyndata, cpu) = _xzalloc(size, align);
+
+return per_cpu(hypfs_dyndata, cpu);
+}
+
+void *hypfs_get_dyndata(void)
+{
+return this_cpu(hypfs_dyndata);
+}
+
 static int add_entry(struct hypfs_entry_dir *parent, struct hypfs_entry *new)
 {
 int ret = -ENOENT;
@@ -122,7 +144,7 @@ static int add_entry(struct hypfs_entry_dir *parent, struct 
hypfs_entry *new)
 {
 unsigned int sz = strlen(new->name);
 
-parent->e.size += DIRENTRY_SIZE(sz);
+parent->e.size += HYPFS_DIRENTRY_SIZE(sz);
 }
 
 hypfs_unlock();
@@ -171,15 +193,34 @@ static int hypfs_get_path_user(char *buf,
 return 0;
 }
 
+struct hypfs_entry *hypfs_dir_findentry(struct hypfs_entry_dir *dir,
+const char *name,
+unsigned int name_len)
+{
+struct hypfs_entry *entry;
+
+list_for_each_entry ( entry, >dirlist, list )
+{
+int cmp = strncmp(name, entry->name, name_len);
+
+if ( cmp < 0 )
+return ERR_PTR(-ENOENT);
+
+if ( !cmp && strlen(entry->name) == name_len )
+return entry;
+}
+
+return ERR_PTR(-ENOENT);
+}
+
 static struct hypfs_entry *hypfs_get_entry_rel(struct hypfs_entry_dir *dir,
const char *path)
 {
 const char *end;
 struct hypfs_entry *entry;
 unsigned int name_len;
-bool again = true;
 
-while ( again )
+for ( ;; )
 {
 if ( dir->e.type != XEN_HYPFS_TYPE_DIR )
 return 

[PATCH 04/12] xen/sched: sort included headers in cpupool.c

2020-10-26 Thread Juergen Gross
Common style is to include header files in alphabetical order. Sort the
#include statements in cpupool.c accordingly.

Signed-off-by: Juergen Gross 
---
 xen/common/sched/cpupool.c | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/xen/common/sched/cpupool.c b/xen/common/sched/cpupool.c
index 6429c8f7b5..84f326ea63 100644
--- a/xen/common/sched/cpupool.c
+++ b/xen/common/sched/cpupool.c
@@ -11,15 +11,15 @@
  * (C) 2009, Juergen Gross, Fujitsu Technology Solutions
  */
 
-#include 
-#include 
+#include 
 #include 
+#include 
+#include 
+#include 
 #include 
 #include 
 #include 
 #include 
-#include 
-#include 
 
 #include "private.h"
 
-- 
2.26.2




[PATCH 02/12] xen/cpupool: add missing bits for per-cpupool scheduling granularity

2020-10-26 Thread Juergen Gross
Even with storing the scheduling granularity in struct cpupool there
are still a few bits missing for being able to have cpupools with
different granularity (apart from the missing interface for setting
the individual granularities): the number of cpus in a scheduling
unit is always taken from the global sched_granularity variable.

So store the value in struct cpupool and use that instead of
sched_granularity.

Signed-off-by: Juergen Gross 
---
 xen/common/sched/cpupool.c | 3 ++-
 xen/common/sched/private.h | 1 +
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/xen/common/sched/cpupool.c b/xen/common/sched/cpupool.c
index 7ea641ca26..6429c8f7b5 100644
--- a/xen/common/sched/cpupool.c
+++ b/xen/common/sched/cpupool.c
@@ -151,7 +151,7 @@ static void __init cpupool_gran_init(void)
 
 unsigned int cpupool_get_granularity(const struct cpupool *c)
 {
-return c ? sched_granularity : 1;
+return c ? c->sched_gran : 1;
 }
 
 static void free_cpupool_struct(struct cpupool *c)
@@ -289,6 +289,7 @@ static struct cpupool *cpupool_create(
 }
 c->sched->cpupool = c;
 c->gran = opt_sched_granularity;
+c->sched_gran = sched_granularity;
 
 *q = c;
 
diff --git a/xen/common/sched/private.h b/xen/common/sched/private.h
index df50976eb2..685992cab9 100644
--- a/xen/common/sched/private.h
+++ b/xen/common/sched/private.h
@@ -514,6 +514,7 @@ struct cpupool
 struct scheduler *sched;
 atomic_t refcnt;
 enum sched_gran  gran;
+unsigned int sched_gran; /* Number of cpus per sched-item. */
 };
 
 static inline cpumask_t *cpupool_domain_master_cpumask(const struct domain *d)
-- 
2.26.2




[PATCH 03/12] xen/sched: support moving a domain between cpupools with different granularity

2020-10-26 Thread Juergen Gross
When moving a domain between cpupools with different scheduling
granularity the sched_units of the domain need to be adjusted.

Do that by allocating new sched_units and throwing away the old ones
in sched_move_domain().

Signed-off-by: Juergen Gross 
---
 xen/common/sched/core.c | 121 ++--
 1 file changed, 90 insertions(+), 31 deletions(-)

diff --git a/xen/common/sched/core.c b/xen/common/sched/core.c
index f8c81592af..8f1db88593 100644
--- a/xen/common/sched/core.c
+++ b/xen/common/sched/core.c
@@ -613,17 +613,45 @@ static void sched_move_irqs(const struct sched_unit *unit)
 vcpu_move_irqs(v);
 }
 
+/*
+ * Move a domain from one cpupool to another.
+ *
+ * A domain with any vcpu having temporary affinity settings will be denied
+ * to move. Hard and soft affinities will be reset.
+ *
+ * In order to support cpupools with different scheduling granularities all
+ * scheduling units are replaced by new ones.
+ *
+ * The complete move is done in the following steps:
+ * - check prerequisites (no vcpu with temporary affinities)
+ * - allocate all new data structures (scheduler specific domain data, unit
+ *   memory, scheduler specific unit data)
+ * - pause domain
+ * - temporarily move all (old) units to the same scheduling resource (this
+ *   makes the final resource assignment easier in case the new cpupool has
+ *   a larger granularity than the old one, as the scheduling locks for all
+ *   vcpus must be held for that operation)
+ * - remove old units from scheduling
+ * - set new cpupool and scheduler domain data pointers in struct domain
+ * - switch all vcpus to new units, still assigned to the old scheduling
+ *   resource
+ * - migrate all new units to scheduling resources of the new cpupool
+ * - unpause the domain
+ * - free the old memory (scheduler specific domain data, unit memory,
+ *   scheduler specific unit data)
+ */
 int sched_move_domain(struct domain *d, struct cpupool *c)
 {
 struct vcpu *v;
-struct sched_unit *unit;
+struct sched_unit *unit, *old_unit;
+struct sched_unit *new_units = NULL, *old_units;
+struct sched_unit **unit_ptr = _units;
 unsigned int new_p, unit_idx;
-void **unit_priv;
 void *domdata;
-void *unitdata;
-struct scheduler *old_ops;
+struct scheduler *old_ops = dom_scheduler(d);
 void *old_domdata;
 unsigned int gran = cpupool_get_granularity(c);
+unsigned int n_units = DIV_ROUND_UP(d->max_vcpus, gran);
 int ret = 0;
 
 for_each_vcpu ( d, v )
@@ -641,53 +669,78 @@ int sched_move_domain(struct domain *d, struct cpupool *c)
 goto out;
 }
 
-unit_priv = xzalloc_array(void *, DIV_ROUND_UP(d->max_vcpus, gran));
-if ( unit_priv == NULL )
+for ( unit_idx = 0; unit_idx < n_units; unit_idx++ )
 {
-sched_free_domdata(c->sched, domdata);
-ret = -ENOMEM;
-goto out;
-}
+unit = sched_alloc_unit_mem();
+if ( unit )
+{
+/* Initialize unit for sched_alloc_udata() to work. */
+unit->domain = d;
+unit->unit_id = unit_idx * gran;
+unit->vcpu_list = d->vcpu[unit->unit_id];
+unit->priv = sched_alloc_udata(c->sched, unit, domdata);
+*unit_ptr = unit;
+}
 
-unit_idx = 0;
-for_each_sched_unit ( d, unit )
-{
-unit_priv[unit_idx] = sched_alloc_udata(c->sched, unit, domdata);
-if ( unit_priv[unit_idx] == NULL )
+if ( !unit || !unit->priv )
 {
-for ( unit_idx = 0; unit_priv[unit_idx]; unit_idx++ )
-sched_free_udata(c->sched, unit_priv[unit_idx]);
-xfree(unit_priv);
-sched_free_domdata(c->sched, domdata);
+old_units = new_units;
+old_domdata = domdata;
 ret = -ENOMEM;
-goto out;
+goto out_free;
 }
-unit_idx++;
+
+unit_ptr = >next_in_list;
 }
 
 domain_pause(d);
 
-old_ops = dom_scheduler(d);
 old_domdata = d->sched_priv;
 
+new_p = cpumask_first(d->cpupool->cpu_valid);
 for_each_sched_unit ( d, unit )
 {
+spinlock_t *lock;
+
+/*
+ * Temporarily move all units to same processor to make locking
+ * easier when moving the new units to the new processors.
+ */
+lock = unit_schedule_lock_irq(unit);
+sched_set_res(unit, get_sched_res(new_p));
+spin_unlock_irq(lock);
+
 sched_remove_unit(old_ops, unit);
 }
 
+old_units = d->sched_unit_list;
+
 d->cpupool = c;
 d->sched_priv = domdata;
 
+unit = new_units;
+for_each_vcpu ( d, v )
+{
+old_unit = v->sched_unit;
+if ( unit->unit_id + gran == v->vcpu_id )
+unit = unit->next_in_list;
+
+unit->state_entry_time = old_unit->state_entry_time;
+unit->runstate_cnt[v->runstate.state]++;
+/* Temporarily use old resource assignment */
+

[PATCH 01/12] xen/cpupool: add cpu to sched_res_mask when removing it from cpupool

2020-10-26 Thread Juergen Gross
When a cpu is removed from a cpupool and added to the free cpus it
should be added to sched_res_mask, too.

The related removal from sched_res_mask in case of core scheduling
is already done in schedule_cpu_add().

As long as all cpupools share the same scheduling granularity there
is nothing going wrong with the missing removal, but this will change
when per-cpupool granularity is fully supported.

Signed-off-by: Juergen Gross 
---
 xen/common/sched/core.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/xen/common/sched/core.c b/xen/common/sched/core.c
index ed973e90ec..f8c81592af 100644
--- a/xen/common/sched/core.c
+++ b/xen/common/sched/core.c
@@ -3189,6 +3189,7 @@ int schedule_cpu_rm(unsigned int cpu)
 /* Adjust cpu masks of resources (old and new). */
 cpumask_clear_cpu(cpu_iter, sr->cpus);
 cpumask_set_cpu(cpu_iter, sr_new[idx]->cpus);
+cpumask_set_cpu(cpu_iter, _res_mask);
 
 /* Init timer. */
 init_timer(_new[idx]->s_timer, s_timer_fn, NULL, cpu_iter);
-- 
2.26.2




[PATCH 11/12] xen/hypfs: add scheduling granularity entry to cpupool entries

2020-10-26 Thread Juergen Gross
Add a "sched-gran" entry to the per-cpupool hypfs directories.

For now make this entry read-only and let it contain one of the
strings "cpu", "core" or "socket".

Signed-off-by: Juergen Gross 
---
 docs/misc/hypfs-paths.pandoc |  4 +++
 xen/common/sched/cpupool.c   | 51 +---
 2 files changed, 52 insertions(+), 3 deletions(-)

diff --git a/docs/misc/hypfs-paths.pandoc b/docs/misc/hypfs-paths.pandoc
index aaca1cdf92..f1ce24d7fe 100644
--- a/docs/misc/hypfs-paths.pandoc
+++ b/docs/misc/hypfs-paths.pandoc
@@ -184,6 +184,10 @@ A directory of all current cpupools.
 The individual cpupools. Each entry is a directory with the name being the
 cpupool-id (e.g. /cpupool/0/).
 
+ /cpupool/*/sched-gran = ("cpu" | "core" | "socket")
+
+The scheduling granularity of a cpupool.
+
  /params/
 
 A directory of runtime parameters.
diff --git a/xen/common/sched/cpupool.c b/xen/common/sched/cpupool.c
index 8612ee5cf6..8674ac0fdd 100644
--- a/xen/common/sched/cpupool.c
+++ b/xen/common/sched/cpupool.c
@@ -42,9 +42,10 @@ static DEFINE_SPINLOCK(cpupool_lock);
 static enum sched_gran __read_mostly opt_sched_granularity = SCHED_GRAN_cpu;
 static unsigned int __read_mostly sched_granularity = 1;
 
+#define SCHED_GRAN_NAME_LEN  8
 struct sched_gran_name {
 enum sched_gran mode;
-char name[8];
+char name[SCHED_GRAN_NAME_LEN];
 };
 
 static const struct sched_gran_name sg_name[] = {
@@ -53,7 +54,7 @@ static const struct sched_gran_name sg_name[] = {
 {SCHED_GRAN_socket, "socket"},
 };
 
-static void sched_gran_print(enum sched_gran mode, unsigned int gran)
+static const char *sched_gran_get_name(enum sched_gran mode)
 {
 const char *name = "";
 unsigned int i;
@@ -67,8 +68,13 @@ static void sched_gran_print(enum sched_gran mode, unsigned 
int gran)
 }
 }
 
+return name;
+}
+
+static void sched_gran_print(enum sched_gran mode, unsigned int gran)
+{
 printk("Scheduling granularity: %s, %u CPU%s per sched-resource\n",
-   name, gran, gran == 1 ? "" : "s");
+   sched_gran_get_name(mode), gran, gran == 1 ? "" : "s");
 }
 
 #ifdef CONFIG_HAS_SCHED_GRANULARITY
@@ -1057,6 +1063,43 @@ static struct hypfs_entry *cpupool_dir_findentry(struct 
hypfs_entry_dir *dir,
 return hypfs_gen_dyndir_entry_id(_pooldir, id);
 }
 
+static int cpupool_gran_read(const struct hypfs_entry *entry,
+ XEN_GUEST_HANDLE_PARAM(void) uaddr)
+{
+const struct hypfs_dyndir_id *data;
+struct cpupool *cpupool;
+const char *name = "";
+
+data = hypfs_get_dyndata();
+if ( !data )
+return -ENOENT;
+
+spin_lock(_lock);
+
+cpupool = __cpupool_find_by_id(data->id, true);
+if ( cpupool )
+name = sched_gran_get_name(cpupool->gran);
+
+spin_unlock(_lock);
+
+if ( !cpupool )
+return -ENOENT;
+
+return copy_to_guest(uaddr, name, strlen(name) + 1) ? -EFAULT : 0;
+}
+
+static struct hypfs_funcs cpupool_gran_funcs = {
+.read = cpupool_gran_read,
+.getsize = hypfs_getsize,
+};
+
+static HYPFS_VARSIZE_INIT(cpupool_gran, XEN_HYPFS_TYPE_STRING, "sched-gran",
+  0, _gran_funcs);
+static char granstr[SCHED_GRAN_NAME_LEN] = {
+[0 ... SCHED_GRAN_NAME_LEN - 2] = '?',
+[SCHED_GRAN_NAME_LEN - 1] = 0
+};
+
 static struct hypfs_funcs cpupool_dir_funcs = {
 .read = cpupool_dir_read,
 .getsize = cpupool_dir_getsize,
@@ -1075,6 +1118,8 @@ static int __init cpupool_init(void)
 
 #ifdef CONFIG_HYPFS
 hypfs_add_dir(_root, _dir, true);
+hypfs_string_set_reference(_gran, granstr);
+hypfs_add_leaf(_pooldir, _gran, true);
 #endif
 
 cpupool0 = cpupool_create(0, 0, );
-- 
2.26.2




[PATCH 00/12] xen: support per-cpupool scheduling granularity

2020-10-26 Thread Juergen Gross
Support scheduling granularity per cpupool. Setting the granularity is
done via hypfs, which needed to gain dynamical entries for that
purpose.

Apart from the hypfs related additional functionality the main change
for cpupools was the support for moving a domain to a new granularity,
as this requires to modify the scheduling unit/vcpu relationship.

I have tried to do the hypfs modifications in a rather generic way in
order to be able to use the same infrastructure in other cases, too
(e.g. for per-domain entries).

The complete series has been tested by creating cpupools with different
granularities and moving busy and idle domains between those.

Juergen Gross (12):
  xen/cpupool: add cpu to sched_res_mask when removing it from cpupool
  xen/cpupool: add missing bits for per-cpupool scheduling granularity
  xen/sched: support moving a domain between cpupools with different
granularity
  xen/sched: sort included headers in cpupool.c
  docs: fix hypfs path documentation
  xen/hypfs: move per-node function pointers into a dedicated struct
  xen/hypfs: pass real failure reason up from hypfs_get_entry()
  xen/hypfs: support dynamic hypfs nodes
  xen/hypfs: add support for id-based dynamic directories
  xen/hypfs: add cpupool directories
  xen/hypfs: add scheduling granularity entry to cpupool entries
  xen/cpupool: make per-cpupool sched-gran hypfs node writable

 docs/misc/hypfs-paths.pandoc |  18 ++-
 xen/common/hypfs.c   | 233 +++
 xen/common/sched/core.c  | 122 +-
 xen/common/sched/cpupool.c   | 213 +---
 xen/common/sched/private.h   |   1 +
 xen/include/xen/hypfs.h  | 106 +++-
 xen/include/xen/param.h  |  15 +--
 7 files changed, 567 insertions(+), 141 deletions(-)

-- 
2.26.2




[PATCH 09/12] xen/hypfs: add support for id-based dynamic directories

2020-10-26 Thread Juergen Gross
Add some helpers to hypfs.c to support dynamic directories with a
numerical id as name.

The dynamic directory is based on a template specified by the user
allowing to use specific access functions and having a predefined
set of entries in the directory.

Signed-off-by: Juergen Gross 
---
 xen/common/hypfs.c  | 76 +
 xen/include/xen/hypfs.h | 14 
 2 files changed, 90 insertions(+)

diff --git a/xen/common/hypfs.c b/xen/common/hypfs.c
index 4c226a06b4..12be2f6d16 100644
--- a/xen/common/hypfs.c
+++ b/xen/common/hypfs.c
@@ -257,6 +257,82 @@ unsigned int hypfs_getsize(const struct hypfs_entry *entry)
 return entry->size;
 }
 
+int hypfs_read_dyndir_id_entry(struct hypfs_entry_dir *template,
+   unsigned int id, bool is_last,
+   XEN_GUEST_HANDLE_PARAM(void) *uaddr)
+{
+struct xen_hypfs_dirlistentry direntry;
+char name[12];
+unsigned int e_namelen, e_len;
+
+e_namelen = snprintf(name, sizeof(name), "%u", id);
+e_len = HYPFS_DIRENTRY_SIZE(e_namelen);
+direntry.e.pad = 0;
+direntry.e.type = template->e.type;
+direntry.e.encoding = template->e.encoding;
+direntry.e.content_len = template->e.funcs->getsize(>e);
+direntry.e.max_write_len = template->e.max_size;
+direntry.off_next = is_last ? 0 : e_len;
+if ( copy_to_guest(*uaddr, , 1) )
+return -EFAULT;
+if ( copy_to_guest_offset(*uaddr, HYPFS_DIRENTRY_NAME_OFF, name,
+  e_namelen + 1) )
+return -EFAULT;
+
+guest_handle_add_offset(*uaddr, e_len);
+
+return 0;
+}
+
+static struct hypfs_entry *hypfs_dyndir_findentry(struct hypfs_entry_dir *dir,
+  const char *name,
+  unsigned int name_len)
+{
+struct hypfs_dyndir_id *data;
+
+data = hypfs_get_dyndata();
+if ( !data )
+return ERR_PTR(-ENOENT);
+
+/* Use template with original findentry function. */
+return data->template->e.funcs->findentry(data->template, name, name_len);
+}
+
+static int hypfs_read_dyndir(const struct hypfs_entry *entry,
+ XEN_GUEST_HANDLE_PARAM(void) uaddr)
+{
+struct hypfs_dyndir_id *data;
+
+data = hypfs_get_dyndata();
+if ( !data )
+return -ENOENT;
+
+/* Use template with original read function. */
+return data->template->e.funcs->read(>template->e, uaddr);
+}
+
+struct hypfs_entry *hypfs_gen_dyndir_entry_id(struct hypfs_entry_dir *template,
+  unsigned int id)
+{
+struct hypfs_dyndir_id *data;
+
+data = hypfs_alloc_dyndata(sizeof(*data), alignof(*data));
+if ( !data )
+return ERR_PTR(-ENOMEM);
+
+data->template = template;
+data->id = id;
+snprintf(data->name, sizeof(data->name), "%u", id);
+data->dir = *template;
+data->dir.e.name = data->name;
+data->dir.e.funcs = >funcs;
+data->funcs = *template->e.funcs;
+data->funcs.findentry = hypfs_dyndir_findentry;
+data->funcs.read = hypfs_read_dyndir;
+
+return >dir.e;
+}
+
 int hypfs_read_dir(const struct hypfs_entry *entry,
XEN_GUEST_HANDLE_PARAM(void) uaddr)
 {
diff --git a/xen/include/xen/hypfs.h b/xen/include/xen/hypfs.h
index c8999b5381..adfb522496 100644
--- a/xen/include/xen/hypfs.h
+++ b/xen/include/xen/hypfs.h
@@ -50,6 +50,15 @@ struct hypfs_entry_dir {
 struct list_head dirlist;
 };
 
+struct hypfs_dyndir_id {
+struct hypfs_entry_dir dir;   /* Modified copy of template. */
+struct hypfs_funcs funcs; /* Dynamic functions. */
+struct hypfs_entry_dir *template; /* Template used. */
+char name[12];/* Name of hypfs entry. */
+
+unsigned int id;  /* Numerical id. */
+};
+
 #define HYPFS_DIRENTRY_NAME_OFF offsetof(struct xen_hypfs_dirlistentry, name)
 #define HYPFS_DIRENTRY_SIZE(name_len) \
 (HYPFS_DIRENTRY_NAME_OFF +\
@@ -150,6 +159,11 @@ struct hypfs_entry *hypfs_dir_findentry(struct 
hypfs_entry_dir *dir,
 unsigned int name_len);
 void *hypfs_alloc_dyndata(unsigned long size, unsigned long align);
 void *hypfs_get_dyndata(void);
+int hypfs_read_dyndir_id_entry(struct hypfs_entry_dir *template,
+   unsigned int id, bool is_last,
+   XEN_GUEST_HANDLE_PARAM(void) *uaddr);
+struct hypfs_entry *hypfs_gen_dyndir_entry_id(struct hypfs_entry_dir *template,
+  unsigned int id);
 #endif
 
 #endif /* __XEN_HYPFS_H__ */
-- 
2.26.2




[PATCH 06/12] xen/hypfs: move per-node function pointers into a dedicated struct

2020-10-26 Thread Juergen Gross
Move the function pointers currently stored in each hypfs node into a
dedicated structure in order to save some space for each node. This
will save even more space with additional callbacks added in future.

Provide some standard function vectors.

Signed-off-by: Juergen Gross 
---
 xen/common/hypfs.c  | 25 +++---
 xen/include/xen/hypfs.h | 57 +
 xen/include/xen/param.h | 15 ---
 3 files changed, 62 insertions(+), 35 deletions(-)

diff --git a/xen/common/hypfs.c b/xen/common/hypfs.c
index 8e932b5cf9..e655e8cfc7 100644
--- a/xen/common/hypfs.c
+++ b/xen/common/hypfs.c
@@ -24,6 +24,25 @@ CHECK_hypfs_dirlistentry;
 (DIRENTRY_NAME_OFF +\
  ROUNDUP((name_len) + 1, alignof(struct xen_hypfs_direntry)))
 
+struct hypfs_funcs hypfs_dir_funcs = {
+.read = hypfs_read_dir,
+};
+struct hypfs_funcs hypfs_leaf_ro_funcs = {
+.read = hypfs_read_leaf,
+};
+struct hypfs_funcs hypfs_leaf_wr_funcs = {
+.read = hypfs_read_leaf,
+.write = hypfs_write_leaf,
+};
+struct hypfs_funcs hypfs_bool_wr_funcs = {
+.read = hypfs_read_leaf,
+.write = hypfs_write_bool,
+};
+struct hypfs_funcs hypfs_custom_wr_funcs = {
+.read = hypfs_read_leaf,
+.write = hypfs_write_custom,
+};
+
 static DEFINE_RWLOCK(hypfs_lock);
 enum hypfs_lock_state {
 hypfs_unlocked,
@@ -284,7 +303,7 @@ static int hypfs_read(const struct hypfs_entry *entry,
 
 guest_handle_add_offset(uaddr, sizeof(e));
 
-ret = entry->read(entry, uaddr);
+ret = entry->funcs->read(entry, uaddr);
 
  out:
 return ret;
@@ -387,14 +406,14 @@ static int hypfs_write(struct hypfs_entry *entry,
 {
 struct hypfs_entry_leaf *l;
 
-if ( !entry->write )
+if ( !entry->funcs->write )
 return -EACCES;
 
 ASSERT(entry->max_size);
 
 l = container_of(entry, struct hypfs_entry_leaf, e);
 
-return entry->write(l, uaddr, ulen);
+return entry->funcs->write(l, uaddr, ulen);
 }
 
 long do_hypfs_op(unsigned int cmd,
diff --git a/xen/include/xen/hypfs.h b/xen/include/xen/hypfs.h
index 5ad99cb558..77916ebb58 100644
--- a/xen/include/xen/hypfs.h
+++ b/xen/include/xen/hypfs.h
@@ -7,6 +7,20 @@
 #include 
 
 struct hypfs_entry_leaf;
+struct hypfs_entry;
+
+struct hypfs_funcs {
+int (*read)(const struct hypfs_entry *entry,
+XEN_GUEST_HANDLE_PARAM(void) uaddr);
+int (*write)(struct hypfs_entry_leaf *leaf,
+ XEN_GUEST_HANDLE_PARAM(void) uaddr, unsigned int ulen);
+};
+
+extern struct hypfs_funcs hypfs_dir_funcs;
+extern struct hypfs_funcs hypfs_leaf_ro_funcs;
+extern struct hypfs_funcs hypfs_leaf_wr_funcs;
+extern struct hypfs_funcs hypfs_bool_wr_funcs;
+extern struct hypfs_funcs hypfs_custom_wr_funcs;
 
 struct hypfs_entry {
 unsigned short type;
@@ -15,10 +29,7 @@ struct hypfs_entry {
 unsigned int max_size;
 const char *name;
 struct list_head list;
-int (*read)(const struct hypfs_entry *entry,
-XEN_GUEST_HANDLE_PARAM(void) uaddr);
-int (*write)(struct hypfs_entry_leaf *leaf,
- XEN_GUEST_HANDLE_PARAM(void) uaddr, unsigned int ulen);
+struct hypfs_funcs *funcs;
 };
 
 struct hypfs_entry_leaf {
@@ -42,7 +53,7 @@ struct hypfs_entry_dir {
 .e.size = 0,  \
 .e.max_size = 0,  \
 .e.list = LIST_HEAD_INIT(var.e.list), \
-.e.read = hypfs_read_dir, \
+.e.funcs = _dir_funcs,  \
 .dirlist = LIST_HEAD_INIT(var.dirlist),   \
 }
 
@@ -52,7 +63,7 @@ struct hypfs_entry_dir {
 .e.encoding = XEN_HYPFS_ENC_PLAIN,\
 .e.name = (nam),  \
 .e.max_size = (msz),  \
-.e.read = hypfs_read_leaf,\
+.e.funcs = _leaf_ro_funcs,  \
 }
 
 /* Content and size need to be set via hypfs_string_set_reference(). */
@@ -72,35 +83,37 @@ static inline void hypfs_string_set_reference(struct 
hypfs_entry_leaf *leaf,
 leaf->e.size = strlen(str) + 1;
 }
 
-#define HYPFS_FIXEDSIZE_INIT(var, typ, nam, contvar, wr) \
-struct hypfs_entry_leaf __read_mostly var = {\
-.e.type = (typ), \
-.e.encoding = XEN_HYPFS_ENC_PLAIN,   \
-.e.name = (nam), \
-.e.size = sizeof(contvar),   \
-.e.max_size = (wr) ? sizeof(contvar) : 0,\
-.e.read = hypfs_read_leaf,   \
-.e.write = (wr), \
-.u.content = &(contvar), \
+#define HYPFS_FIXEDSIZE_INIT(var, typ, nam, contvar, fn, wr) \
+struct hypfs_entry_leaf __read_mostly var = {\
+.e.type = (typ), \
+.e.encoding = XEN_HYPFS_ENC_PLAIN,   \
+.e.name = (nam),   

[PATCH 12/12] xen/cpupool: make per-cpupool sched-gran hypfs node writable

2020-10-26 Thread Juergen Gross
Make /cpupool//sched-gran in hypfs writable. This will enable per
cpupool selectable scheduling granularity.

Writing this node is allowed only with no cpu assigned to the cpupool.
Allowed are values "cpu", "core" and "socket".

Signed-off-by: Juergen Gross 
---
 docs/misc/hypfs-paths.pandoc |  5 ++-
 xen/common/sched/cpupool.c   | 75 +++-
 2 files changed, 69 insertions(+), 11 deletions(-)

diff --git a/docs/misc/hypfs-paths.pandoc b/docs/misc/hypfs-paths.pandoc
index f1ce24d7fe..e86f7d0dbe 100644
--- a/docs/misc/hypfs-paths.pandoc
+++ b/docs/misc/hypfs-paths.pandoc
@@ -184,10 +184,13 @@ A directory of all current cpupools.
 The individual cpupools. Each entry is a directory with the name being the
 cpupool-id (e.g. /cpupool/0/).
 
- /cpupool/*/sched-gran = ("cpu" | "core" | "socket")
+ /cpupool/*/sched-gran = ("cpu" | "core" | "socket") [w]
 
 The scheduling granularity of a cpupool.
 
+Writing a value is allowed only for cpupools with no cpu assigned and if the
+architecture is supporting different scheduling granularities.
+
  /params/
 
 A directory of runtime parameters.
diff --git a/xen/common/sched/cpupool.c b/xen/common/sched/cpupool.c
index 8674ac0fdd..d0c61fb720 100644
--- a/xen/common/sched/cpupool.c
+++ b/xen/common/sched/cpupool.c
@@ -78,7 +78,7 @@ static void sched_gran_print(enum sched_gran mode, unsigned 
int gran)
 }
 
 #ifdef CONFIG_HAS_SCHED_GRANULARITY
-static int __init sched_select_granularity(const char *str)
+static int sched_gran_get(const char *str, enum sched_gran *mode)
 {
 unsigned int i;
 
@@ -86,36 +86,43 @@ static int __init sched_select_granularity(const char *str)
 {
 if ( strcmp(sg_name[i].name, str) == 0 )
 {
-opt_sched_granularity = sg_name[i].mode;
+*mode = sg_name[i].mode;
 return 0;
 }
 }
 
 return -EINVAL;
 }
+
+static int __init sched_select_granularity(const char *str)
+{
+return sched_gran_get(str, _sched_granularity);
+}
 custom_param("sched-gran", sched_select_granularity);
+#else
+static int sched_gran_get(const char *str, enum sched_gran *mode)
+{
+return -EINVAL;
+}
 #endif
 
-static unsigned int __init cpupool_check_granularity(void)
+static unsigned int cpupool_check_granularity(enum sched_gran mode)
 {
 unsigned int cpu;
 unsigned int siblings, gran = 0;
 
-if ( opt_sched_granularity == SCHED_GRAN_cpu )
+if ( mode == SCHED_GRAN_cpu )
 return 1;
 
 for_each_online_cpu ( cpu )
 {
-siblings = cpumask_weight(sched_get_opt_cpumask(opt_sched_granularity,
-cpu));
+siblings = cpumask_weight(sched_get_opt_cpumask(mode, cpu));
 if ( gran == 0 )
 gran = siblings;
 else if ( gran != siblings )
 return 0;
 }
 
-sched_disable_smt_switching = true;
-
 return gran;
 }
 
@@ -127,7 +134,7 @@ static void __init cpupool_gran_init(void)
 
 while ( gran == 0 )
 {
-gran = cpupool_check_granularity();
+gran = cpupool_check_granularity(opt_sched_granularity);
 
 if ( gran == 0 )
 {
@@ -153,6 +160,9 @@ static void __init cpupool_gran_init(void)
 if ( fallback )
 warning_add(fallback);
 
+if ( opt_sched_granularity != SCHED_GRAN_cpu )
+sched_disable_smt_switching = true;
+
 sched_granularity = gran;
 sched_gran_print(opt_sched_granularity, sched_granularity);
 }
@@ -1088,13 +1098,58 @@ static int cpupool_gran_read(const struct hypfs_entry 
*entry,
 return copy_to_guest(uaddr, name, strlen(name) + 1) ? -EFAULT : 0;
 }
 
+static int cpupool_gran_write(struct hypfs_entry_leaf *leaf,
+  XEN_GUEST_HANDLE_PARAM(void) uaddr,
+  unsigned int ulen)
+{
+const struct hypfs_dyndir_id *data;
+struct cpupool *cpupool;
+enum sched_gran gran;
+unsigned int sched_gran;
+char name[SCHED_GRAN_NAME_LEN];
+int ret = 0;
+
+if ( ulen > SCHED_GRAN_NAME_LEN )
+return -ENOSPC;
+
+if ( copy_from_guest(name, uaddr, ulen) )
+return -EFAULT;
+
+sched_gran = sched_gran_get(name, ) ? 0
+ : cpupool_check_granularity(gran);
+if ( memchr(name, 0, ulen) != (name + ulen - 1) || sched_gran == 0 )
+return -EINVAL;
+
+data = hypfs_get_dyndata();
+if ( !data )
+return -ENOENT;
+
+spin_lock(_lock);
+
+cpupool = __cpupool_find_by_id(data->id, true);
+if ( !cpupool )
+ret = -ENOENT;
+else if ( !cpumask_empty(cpupool->cpu_valid) )
+ret = -EBUSY;
+else
+{
+cpupool->gran = gran;
+cpupool->sched_gran = sched_gran;
+}
+
+spin_unlock(_lock);
+
+return ret;
+}
+
 static struct hypfs_funcs cpupool_gran_funcs = {
 .read = cpupool_gran_read,
+.write = cpupool_gran_write,
 .getsize = hypfs_getsize,
 };
 
 static 

[PATCH 05/12] docs: fix hypfs path documentation

2020-10-26 Thread Juergen Gross
The /params/* entry is missing a writable tag.

Signed-off-by: Juergen Gross 
---
 docs/misc/hypfs-paths.pandoc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/misc/hypfs-paths.pandoc b/docs/misc/hypfs-paths.pandoc
index dddb592bc5..6c7b2f7ee3 100644
--- a/docs/misc/hypfs-paths.pandoc
+++ b/docs/misc/hypfs-paths.pandoc
@@ -179,7 +179,7 @@ The minor version of Xen.
 
 A directory of runtime parameters.
 
- /params/*
+ /params/* [w]
 
 The individual parameters. The description of the different parameters can be
 found in `docs/misc/xen-command-line.pandoc`.
-- 
2.26.2




Re: [xen-unstable test] 156196: tolerable FAIL

2020-10-26 Thread Jan Beulich
On 25.10.2020 10:27, osstest service owner wrote:
> flight 156196 xen-unstable real [real]
> http://logs.test-lab.xenproject.org/osstest/logs/156196/
> 
> Failures :-/ but no regressions.

This and the prior two flights have shown no issue at all with
the test-amd64-amd64-xl-qemu*-debianhvm-i386-xsm tests. I begin
wondering whether the failures previously observed here as well
as for 4.14 and 4.13 were in fact "glitches" caused by
something outside of the software under test.

Jan

> Tests which did not succeed, but are not blocking:
>  test-armhf-armhf-libvirt 16 saverestore-support-checkfail  like 
> 156167
>  test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stopfail like 
> 156167
>  test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stopfail like 
> 156167
>  test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop fail like 
> 156167
>  test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stopfail like 
> 156167
>  test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop fail like 
> 156167
>  test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop fail like 
> 156167
>  test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stopfail like 
> 156167
>  test-armhf-armhf-libvirt-raw 15 saverestore-support-checkfail  like 
> 156167
>  test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 
> 156167
>  test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop fail like 
> 156167
>  test-amd64-i386-xl-pvshim14 guest-start  fail   never 
> pass
>  test-amd64-amd64-libvirt 15 migrate-support-checkfail   never 
> pass
>  test-amd64-amd64-libvirt-xsm 15 migrate-support-checkfail   never 
> pass
>  test-arm64-arm64-xl-seattle  15 migrate-support-checkfail   never 
> pass
>  test-arm64-arm64-xl-seattle  16 saverestore-support-checkfail   never 
> pass
>  test-amd64-i386-libvirt-xsm  15 migrate-support-checkfail   never 
> pass
>  test-amd64-i386-libvirt  15 migrate-support-checkfail   never 
> pass
>  test-arm64-arm64-xl  15 migrate-support-checkfail   never 
> pass
>  test-arm64-arm64-xl  16 saverestore-support-checkfail   never 
> pass
>  test-arm64-arm64-xl-xsm  15 migrate-support-checkfail   never 
> pass
>  test-arm64-arm64-xl-xsm  16 saverestore-support-checkfail   never 
> pass
>  test-arm64-arm64-xl-thunderx 15 migrate-support-checkfail   never 
> pass
>  test-arm64-arm64-xl-thunderx 16 saverestore-support-checkfail   never 
> pass
>  test-arm64-arm64-xl-credit2  15 migrate-support-checkfail   never 
> pass
>  test-arm64-arm64-xl-credit2  16 saverestore-support-checkfail   never 
> pass
>  test-arm64-arm64-libvirt-xsm 15 migrate-support-checkfail   never 
> pass
>  test-arm64-arm64-libvirt-xsm 16 saverestore-support-checkfail   never 
> pass
>  test-arm64-arm64-xl-credit1  15 migrate-support-checkfail   never 
> pass
>  test-arm64-arm64-xl-credit1  16 saverestore-support-checkfail   never 
> pass
>  test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check 
> fail never pass
>  test-armhf-armhf-xl-arndale  15 migrate-support-checkfail   never 
> pass
>  test-armhf-armhf-xl-arndale  16 saverestore-support-checkfail   never 
> pass
>  test-amd64-amd64-libvirt-vhd 14 migrate-support-checkfail   never 
> pass
>  test-armhf-armhf-xl  15 migrate-support-checkfail   never 
> pass
>  test-armhf-armhf-xl  16 saverestore-support-checkfail   never 
> pass
>  test-armhf-armhf-xl-credit1  15 migrate-support-checkfail   never 
> pass
>  test-armhf-armhf-xl-credit1  16 saverestore-support-checkfail   never 
> pass
>  test-armhf-armhf-xl-credit2  15 migrate-support-checkfail   never 
> pass
>  test-armhf-armhf-xl-credit2  16 saverestore-support-checkfail   never 
> pass
>  test-armhf-armhf-xl-cubietruck 15 migrate-support-checkfail never 
> pass
>  test-armhf-armhf-xl-cubietruck 16 saverestore-support-checkfail never 
> pass
>  test-armhf-armhf-libvirt 15 migrate-support-checkfail   never 
> pass
>  test-armhf-armhf-xl-multivcpu 15 migrate-support-checkfail  never 
> pass
>  test-armhf-armhf-xl-multivcpu 16 saverestore-support-checkfail  never 
> pass
>  test-armhf-armhf-xl-rtds 15 migrate-support-checkfail   never 
> pass
>  test-armhf-armhf-xl-rtds 16 saverestore-support-checkfail   never 
> pass
>  test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check 
> fail never pass
>  test-armhf-armhf-xl-vhd  14 migrate-support-checkfail   never 
> pass
>  test-armhf-armhf-xl-vhd  15 saverestore-support-checkfail   never 
> pass
>  test-armhf-armhf-libvirt-raw 14 migrate-support-checkfail   never 
> pass
> 
> version targeted for testing:
>  xen  

Re: [PATCH 00/25] xl / libxl: named PCI pass-through devices

2020-10-26 Thread Christian Lindig
> NOTE: The OCaml bindings are adjusted to contain the interface change. It
> should therefore not affect compatibility with OCaml-based utilities.

Acked-by: Christian Lindig 


From: Paul Durrant 
Sent: 23 October 2020 17:22
To: xen-devel@lists.xenproject.org
Cc: Paul Durrant; Anthony Perard; Christian Lindig; David Scott; George Dunlap; 
Ian Jackson; Nick Rosbrook; Wei Liu
Subject: [PATCH 00/25] xl / libxl: named PCI pass-through devices

From: Paul Durrant 

This series adds support for naming devices added to the assignable list
and then using a name (instead of a BDF) for convenience when attaching
a device to a domain.

The first 15 patches are cleanup. The remaining 10 modify documentation
and add the new functionality.

Paul Durrant (25):
  xl / libxl: s/pcidev/pci and remove DEFINE_DEVICE_TYPE_STRUCT_X
  libxl: use LIBXL_DEFINE_DEVICE_LIST for pci devices
  libxl: use LIBXL_DEFINE_DEVICE_LIST for nic devices
  libxl: s/domainid/domid/g in libxl_pci.c
  libxl: s/detatched/detached in libxl_pci.c
  libxl: remove extraneous arguments to do_pci_remove() in libxl_pci.c
  libxl: stop using aodev->device_config in libxl__device_pci_add()...
  libxl: generalise 'driver_path' xenstore access functions in
libxl_pci.c
  libxl: remove unnecessary check from libxl__device_pci_add()
  libxl: remove get_all_assigned_devices() from libxl_pci.c
  libxl: make sure callers of libxl_device_pci_list() free the list
after use
  libxl: add libxl_device_pci_assignable_list_free()...
  libxl: use COMPARE_PCI() macro is_pci_in_array()...
  libxl: add/recover 'rdm_policy' to/from PCI backend in xenstore
  libxl: Make sure devices added by pci-attach are reflected in the
config
  docs/man: extract documentation of PCI_SPEC_STRING from the xl.cfg
manpage...
  docs/man: improve documentation of PCI_SPEC_STRING...
  docs/man: fix xl(1) documentation for 'pci' operations
  libxl: introduce 'libxl_pci_bdf' in the idl...
  libxlu: introduce xlu_pci_parse_spec_string()
  libxl: modify
libxl_device_pci_assignable_add/remove/list/list_free()...
  docs/man: modify xl(1) in preparation for naming of assignable devices
  xl / libxl: support naming of assignable devices
  docs/man: modify xl-pci-configuration(5) to add 'name' field to
PCI_SPEC_STRING
  xl / libxl: support 'xl pci-attach/detach' by name

 docs/man/xl-pci-configuration.5.pod  |  218 +++
 docs/man/xl.1.pod.in |   39 +-
 docs/man/xl.cfg.5.pod.in |   68 +--
 tools/golang/xenlight/helpers.gen.go |   77 ++-
 tools/golang/xenlight/types.gen.go   |8 +-
 tools/include/libxl.h|   67 ++-
 tools/include/libxlutil.h|8 +-
 tools/libs/light/libxl_create.c  |6 +-
 tools/libs/light/libxl_dm.c  |   18 +-
 tools/libs/light/libxl_internal.h|   53 +-
 tools/libs/light/libxl_nic.c |   19 +-
 tools/libs/light/libxl_pci.c | 1072 ++
 tools/libs/light/libxl_types.idl |   19 +-
 tools/libs/util/libxlu_pci.c |  359 ++--
 tools/ocaml/libs/xl/xenlight_stubs.c |   19 +-
 tools/xl/xl_cmdtable.c   |   16 +-
 tools/xl/xl_parse.c  |   30 +-
 tools/xl/xl_pci.c|  164 +++---
 tools/xl/xl_sxp.c|   12 +-
 19 files changed, 1337 insertions(+), 935 deletions(-)
 create mode 100644 docs/man/xl-pci-configuration.5.pod
---
Cc: Anthony PERARD 
Cc: Christian Lindig 
Cc: David Scott 
Cc: George Dunlap 
Cc: Ian Jackson 
Cc: Nick Rosbrook 
Cc: Wei Liu 
--
2.11.0




Re: [PATCH v5 10/10] drm/fb_helper: Support framebuffers in I/O memory

2020-10-26 Thread Thomas Zimmermann
Hi

Am 24.10.20 um 22:38 schrieb Sam Ravnborg:
> Hi Thomas.
> 
> On Tue, Oct 20, 2020 at 02:20:46PM +0200, Thomas Zimmermann wrote:
>> At least sparc64 requires I/O-specific access to framebuffers. This
>> patch updates the fbdev console accordingly.
>>
>> For drivers with direct access to the framebuffer memory, the callback
>> functions in struct fb_ops test for the type of memory and call the rsp
>> fb_sys_ of fb_cfb_ functions. Read and write operations are implemented
>> internally by DRM's fbdev helper.
>>
>> For drivers that employ a shadow buffer, fbdev's blit function retrieves
>> the framebuffer address as struct dma_buf_map, and uses dma_buf_map
>> interfaces to access the buffer.
>>
>> The bochs driver on sparc64 uses a workaround to flag the framebuffer as
>> I/O memory and avoid a HW exception. With the introduction of struct
>> dma_buf_map, this is not required any longer. The patch removes the rsp
>> code from both, bochs and fbdev.
>>
>> v5:
>>  * implement fb_read/fb_write internally (Daniel, Sam)
>> v4:
>>  * move dma_buf_map changes into separate patch (Daniel)
>>  * TODO list: comment on fbdev updates (Daniel)
>>
>> Signed-off-by: Thomas Zimmermann 
>> Tested-by: Sam Ravnborg 
> Reviewed-by: Sam Ravnborg 
> 
> But see a few comments below on naming for you to consider.
> 
>   Sam
> 
>> ---
>>  Documentation/gpu/todo.rst|  19 ++-
>>  drivers/gpu/drm/bochs/bochs_kms.c |   1 -
>>  drivers/gpu/drm/drm_fb_helper.c   | 227 --
>>  include/drm/drm_mode_config.h |  12 --
>>  4 files changed, 230 insertions(+), 29 deletions(-)
>>
>> diff --git a/Documentation/gpu/todo.rst b/Documentation/gpu/todo.rst
>> index 7e6fc3c04add..638b7f704339 100644
>> --- a/Documentation/gpu/todo.rst
>> +++ b/Documentation/gpu/todo.rst
>> @@ -197,13 +197,28 @@ Convert drivers to use drm_fbdev_generic_setup()
>>  
>>  
>>  Most drivers can use drm_fbdev_generic_setup(). Driver have to implement
>> -atomic modesetting and GEM vmap support. Current generic fbdev emulation
>> -expects the framebuffer in system memory (or system-like memory).
>> +atomic modesetting and GEM vmap support. Historically, generic fbdev 
>> emulation
>> +expected the framebuffer in system memory or system-like memory. By 
>> employing
>> +struct dma_buf_map, drivers with frambuffers in I/O memory can be supported
>> +as well.
>>  
>>  Contact: Maintainer of the driver you plan to convert
>>  
>>  Level: Intermediate
>>  
>> +Reimplement functions in drm_fbdev_fb_ops without fbdev
>> +---
>> +
>> +A number of callback functions in drm_fbdev_fb_ops could benefit from
>> +being rewritten without dependencies on the fbdev module. Some of the
>> +helpers could further benefit from using struct dma_buf_map instead of
>> +raw pointers.
>> +
>> +Contact: Thomas Zimmermann , Daniel Vetter
>> +
>> +Level: Advanced
>> +
>> +
>>  drm_framebuffer_funcs and drm_mode_config_funcs.fb_create cleanup
>>  -
>>  
>> diff --git a/drivers/gpu/drm/bochs/bochs_kms.c 
>> b/drivers/gpu/drm/bochs/bochs_kms.c
>> index 13d0d04c4457..853081d186d5 100644
>> --- a/drivers/gpu/drm/bochs/bochs_kms.c
>> +++ b/drivers/gpu/drm/bochs/bochs_kms.c
>> @@ -151,7 +151,6 @@ int bochs_kms_init(struct bochs_device *bochs)
>>  bochs->dev->mode_config.preferred_depth = 24;
>>  bochs->dev->mode_config.prefer_shadow = 0;
>>  bochs->dev->mode_config.prefer_shadow_fbdev = 1;
>> -bochs->dev->mode_config.fbdev_use_iomem = true;
>>  bochs->dev->mode_config.quirk_addfb_prefer_host_byte_order = true;
>>  
>>  bochs->dev->mode_config.funcs = _mode_funcs;
>> diff --git a/drivers/gpu/drm/drm_fb_helper.c 
>> b/drivers/gpu/drm/drm_fb_helper.c
>> index 6212cd7cde1d..1d3180841778 100644
>> --- a/drivers/gpu/drm/drm_fb_helper.c
>> +++ b/drivers/gpu/drm/drm_fb_helper.c
>> @@ -372,24 +372,22 @@ static void drm_fb_helper_resume_worker(struct 
>> work_struct *work)
>>  }
>>  
>>  static void drm_fb_helper_dirty_blit_real(struct drm_fb_helper *fb_helper,
>> -  struct drm_clip_rect *clip)
>> +  struct drm_clip_rect *clip,
>> +  struct dma_buf_map *dst)
>>  {
>>  struct drm_framebuffer *fb = fb_helper->fb;
>>  unsigned int cpp = fb->format->cpp[0];
>>  size_t offset = clip->y1 * fb->pitches[0] + clip->x1 * cpp;
>>  void *src = fb_helper->fbdev->screen_buffer + offset;
>> -void *dst = fb_helper->buffer->map.vaddr + offset;
>>  size_t len = (clip->x2 - clip->x1) * cpp;
>>  unsigned int y;
>>  
>> -for (y = clip->y1; y < clip->y2; y++) {
>> -if (!fb_helper->dev->mode_config.fbdev_use_iomem)
>> -memcpy(dst, src, len);
>> -else
>> -memcpy_toio((void __iomem *)dst, src, len);
>> +  

[xen-unstable-smoke test] 156236: regressions - trouble: blocked/fail

2020-10-26 Thread osstest service owner
flight 156236 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/156236/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64   6 xen-buildfail REGR. vs. 156117
 build-arm64-xsm   6 xen-buildfail REGR. vs. 156117
 build-armhf   6 xen-buildfail REGR. vs. 156117

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl   1 build-check(1)   blocked  n/a

version targeted for testing:
 xen  4ddd6499d999a7d08cabfda5b0262e473dd5beed
baseline version:
 xen  6ca70821b59849ad97c3fadc47e63c1a4af1a78c

Last test of basis   156117  2020-10-23 09:01:23 Z2 days
Failing since156120  2020-10-23 14:01:24 Z2 days   33 attempts
Testing same since   156129  2020-10-23 18:01:24 Z2 days   32 attempts


People who touched revisions under test:
  Andrew Cooper 
  Bertrand Marquis 
  Christian Lindig 
  George Dunlap 
  Ian Jackson 
  Ian Jackson 
  Jan Beulich 
  Jason Andryuk 
  Juergen Gross 
  Wei Liu 

jobs:
 build-arm64-xsm  fail
 build-amd64  fail
 build-armhf  fail
 build-amd64-libvirt  blocked 
 test-armhf-armhf-xl  blocked 
 test-arm64-arm64-xl-xsm  blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-amd64blocked 
 test-amd64-amd64-libvirt blocked 



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.


commit 4ddd6499d999a7d08cabfda5b0262e473dd5beed
Author: Jason Andryuk 
Date:   Sun May 24 22:55:06 2020 -0400

SUPPORT: Add linux device model stubdom to Toolstack

Add qemu-xen linux device model stubdomain to the Toolstack section as a
Tech Preview.

Signed-off-by: Jason Andryuk 
Acked-by: George Dunlap 
Acked-by: Ian Jackson 

commit 06f0598b41f23c9e4cf7d8c5a05b282de92f3a35
Author: Jan Beulich 
Date:   Fri Oct 23 18:03:18 2020 +0200

x86emul: fix PINSRW and adjust other {,V}PINSR*

The use of simd_packed_int together with no further update to op_bytes
has lead to wrong signaling of #GP(0) for PINSRW without a 16-byte
aligned memory operand. Use simd_none instead and override it after
general decoding with simd_other, like is done for the B/D/Q siblings.

While benign, for consistency also use DstImplicit instead of DstReg
in x86_decode_twobyte().

PINSR{B,D,Q} also had a stray (redundant) get_fpu() invocation, which
gets dropped.

For further consistency also
- use src.bytes instead of op_bytes in relevant memcpy() invocations,
- avoid the pointless updating of op_bytes (all we care about later is
  that the value be less than 16).

Signed-off-by: Jan Beulich 
Acked-by: Andrew Cooper 

commit 9af5e2b31b4e6f3892b4614ecd0a619af5d64d7e
Author: Juergen Gross 
Date:   Mon Oct 19 17:27:54 2020 +0200

tools/libs/store: don't use symbolic links for external files

Instead of using symbolic links to include files from xenstored use
the vpath directive and an include path.

Signed-off-by: Juergen Gross 
Acked-by: Christian Lindig 
Tested-by: Bertrand Marquis 
Acked-by: Ian Jackson 

commit 588756db020e73e6f5e4407bbf78fbd53f15b731
Author: Juergen Gross 
Date:   Mon Oct 19 17:27:54 2020 +0200

tools/libs/guest: don't use symbolic links for xenctrl headers

Instead of using symbolic links for accessing the xenctrl private
headers use an include path instead.

Signed-off-by: Juergen Gross 
Acked-by: Christian Lindig 
Tested-by: Bertrand Marquis 
Acked-by: Ian Jackson 

commit 4664034cdc720a52913bc26358240bb9d3798527
Author: Juergen 

[xen-unstable-smoke test] 156235: regressions - trouble: blocked/fail

2020-10-26 Thread osstest service owner
flight 156235 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/156235/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64   6 xen-buildfail REGR. vs. 156117
 build-arm64-xsm   6 xen-buildfail REGR. vs. 156117
 build-armhf   6 xen-buildfail REGR. vs. 156117

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-arm64-arm64-xl-xsm   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl   1 build-check(1)   blocked  n/a

version targeted for testing:
 xen  4ddd6499d999a7d08cabfda5b0262e473dd5beed
baseline version:
 xen  6ca70821b59849ad97c3fadc47e63c1a4af1a78c

Last test of basis   156117  2020-10-23 09:01:23 Z2 days
Failing since156120  2020-10-23 14:01:24 Z2 days   32 attempts
Testing same since   156129  2020-10-23 18:01:24 Z2 days   31 attempts


People who touched revisions under test:
  Andrew Cooper 
  Bertrand Marquis 
  Christian Lindig 
  George Dunlap 
  Ian Jackson 
  Ian Jackson 
  Jan Beulich 
  Jason Andryuk 
  Juergen Gross 
  Wei Liu 

jobs:
 build-arm64-xsm  fail
 build-amd64  fail
 build-armhf  fail
 build-amd64-libvirt  blocked 
 test-armhf-armhf-xl  blocked 
 test-arm64-arm64-xl-xsm  blocked 
 test-amd64-amd64-xl-qemuu-debianhvm-amd64blocked 
 test-amd64-amd64-libvirt blocked 



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.


commit 4ddd6499d999a7d08cabfda5b0262e473dd5beed
Author: Jason Andryuk 
Date:   Sun May 24 22:55:06 2020 -0400

SUPPORT: Add linux device model stubdom to Toolstack

Add qemu-xen linux device model stubdomain to the Toolstack section as a
Tech Preview.

Signed-off-by: Jason Andryuk 
Acked-by: George Dunlap 
Acked-by: Ian Jackson 

commit 06f0598b41f23c9e4cf7d8c5a05b282de92f3a35
Author: Jan Beulich 
Date:   Fri Oct 23 18:03:18 2020 +0200

x86emul: fix PINSRW and adjust other {,V}PINSR*

The use of simd_packed_int together with no further update to op_bytes
has lead to wrong signaling of #GP(0) for PINSRW without a 16-byte
aligned memory operand. Use simd_none instead and override it after
general decoding with simd_other, like is done for the B/D/Q siblings.

While benign, for consistency also use DstImplicit instead of DstReg
in x86_decode_twobyte().

PINSR{B,D,Q} also had a stray (redundant) get_fpu() invocation, which
gets dropped.

For further consistency also
- use src.bytes instead of op_bytes in relevant memcpy() invocations,
- avoid the pointless updating of op_bytes (all we care about later is
  that the value be less than 16).

Signed-off-by: Jan Beulich 
Acked-by: Andrew Cooper 

commit 9af5e2b31b4e6f3892b4614ecd0a619af5d64d7e
Author: Juergen Gross 
Date:   Mon Oct 19 17:27:54 2020 +0200

tools/libs/store: don't use symbolic links for external files

Instead of using symbolic links to include files from xenstored use
the vpath directive and an include path.

Signed-off-by: Juergen Gross 
Acked-by: Christian Lindig 
Tested-by: Bertrand Marquis 
Acked-by: Ian Jackson 

commit 588756db020e73e6f5e4407bbf78fbd53f15b731
Author: Juergen Gross 
Date:   Mon Oct 19 17:27:54 2020 +0200

tools/libs/guest: don't use symbolic links for xenctrl headers

Instead of using symbolic links for accessing the xenctrl private
headers use an include path instead.

Signed-off-by: Juergen Gross 
Acked-by: Christian Lindig 
Tested-by: Bertrand Marquis 
Acked-by: Ian Jackson 

commit 4664034cdc720a52913bc26358240bb9d3798527
Author: Juergen