[linux-5.4 test] 156620: regressions - FAIL

2020-11-10 Thread osstest service owner
flight 156620 linux-5.4 real [real]
flight 156676 linux-5.4 real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/156620/
http://logs.test-lab.xenproject.org/osstest/logs/156676/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-pvhv2-amd 20 guest-localmigrate/x10  fail REGR. vs. 156412

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-rtds 20 guest-localmigrate/x10   fail REGR. vs. 156412

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stopfail like 156412
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop fail like 156412
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stopfail like 156412
 test-armhf-armhf-libvirt 16 saverestore-support-checkfail  like 156412
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 156412
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stopfail like 156412
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stopfail like 156412
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop fail like 156412
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop fail like 156412
 test-armhf-armhf-libvirt-raw 15 saverestore-support-checkfail  like 156412
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop fail like 156412
 test-amd64-i386-xl-pvshim14 guest-start  fail   never pass
 test-amd64-amd64-libvirt 15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  15 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check 
fail never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-checkfail never pass
 test-armhf-armhf-libvirt 15 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  14 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  15 saverestore-support-checkfail   never pass

version targeted for testing:
 linuxec9c6b417e271ee76d1430d2b197794858238d3b
baseline version:
 linux6e97ed6efa701db070da0054b055c085895aba86

Last test of basis   156412  2020-11-05 11:13:59 Z5 days
Testing same since   156620  2020-11-10 12:10:31 Z0 days1 attempts


Re: [PATCH] xen/events: fix build

2020-11-10 Thread Jürgen Groß

On 11.11.20 08:37, Jan Beulich wrote:

On 11.11.2020 08:27, Jürgen Groß wrote:

On 11.11.20 08:20, Jan Beulich wrote:

On 11.11.2020 06:49, Juergen Gross wrote:

--- a/xen/common/event_channel.c
+++ b/xen/common/event_channel.c
@@ -61,7 +61,9 @@ static inline void evtchn_write_lock(struct evtchn *evtchn)
   {
   write_lock(>lock);
   
+#ifndef NDEBUG

   evtchn->old_state = evtchn->state;
+#endif
   }
   
   static inline void evtchn_write_unlock(struct evtchn *evtchn)

diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index 7251b3ae3e..95f96e7a33 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -114,9 +114,7 @@ struct evtchn
   u16 virq;  /* state == ECS_VIRQ */
   } u;
   u8 priority;
-#ifndef NDEBUG
   u8 old_state;  /* State when taking lock in write mode. */
-#endif
   u8 last_priority;
   u16 last_vcpu_id;
   #ifdef CONFIG_XSM


Did you mean just either of the two changes (and then the former),
not both? If so
Reviewed-by: Jan Beulich 
and I'll be happy to drop the other half for committing.


The header fix is required for NDEBUG builds, while the other one is
removing a write with no accompanied read for NDEBUG builds.


Oh, that's because of our absurd ASSERT() implementation in the
NDEBUG case. I stand by my position that the field should not be
there in the first place for release builds. Even more so with
the original patch having got re-based ahead of what was patch 1
of the series, which I did not the least because I want that one
backported urgently, while I continue to be hesitant altogether
about the other one.

I guess the course of action is then to put #ifndef NDEBUG
around the ASSERT() itself, however strange this may look. Or
introduce an evtchn_old_state() wrapper, perhaps returning
ECS_RESERVED in the NDEBUG case. I guess it'll be quicker if I
take your patch and massage it before throwing in, than to have
you make a v2.


Fine with me.


Juergen


OpenPGP_0xB0DE9DD628BF132F.asc
Description: application/pgp-keys


OpenPGP_signature
Description: OpenPGP digital signature


Re: [PATCH] xen/events: fix build

2020-11-10 Thread Jan Beulich
On 11.11.2020 08:27, Jürgen Groß wrote:
> On 11.11.20 08:20, Jan Beulich wrote:
>> On 11.11.2020 06:49, Juergen Gross wrote:
>>> --- a/xen/common/event_channel.c
>>> +++ b/xen/common/event_channel.c
>>> @@ -61,7 +61,9 @@ static inline void evtchn_write_lock(struct evtchn 
>>> *evtchn)
>>>   {
>>>   write_lock(>lock);
>>>   
>>> +#ifndef NDEBUG
>>>   evtchn->old_state = evtchn->state;
>>> +#endif
>>>   }
>>>   
>>>   static inline void evtchn_write_unlock(struct evtchn *evtchn)
>>> diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
>>> index 7251b3ae3e..95f96e7a33 100644
>>> --- a/xen/include/xen/sched.h
>>> +++ b/xen/include/xen/sched.h
>>> @@ -114,9 +114,7 @@ struct evtchn
>>>   u16 virq;  /* state == ECS_VIRQ */
>>>   } u;
>>>   u8 priority;
>>> -#ifndef NDEBUG
>>>   u8 old_state;  /* State when taking lock in write mode. */
>>> -#endif
>>>   u8 last_priority;
>>>   u16 last_vcpu_id;
>>>   #ifdef CONFIG_XSM
>>
>> Did you mean just either of the two changes (and then the former),
>> not both? If so
>> Reviewed-by: Jan Beulich 
>> and I'll be happy to drop the other half for committing.
> 
> The header fix is required for NDEBUG builds, while the other one is
> removing a write with no accompanied read for NDEBUG builds.

Oh, that's because of our absurd ASSERT() implementation in the
NDEBUG case. I stand by my position that the field should not be
there in the first place for release builds. Even more so with
the original patch having got re-based ahead of what was patch 1
of the series, which I did not the least because I want that one
backported urgently, while I continue to be hesitant altogether
about the other one.

I guess the course of action is then to put #ifndef NDEBUG
around the ASSERT() itself, however strange this may look. Or
introduce an evtchn_old_state() wrapper, perhaps returning
ECS_RESERVED in the NDEBUG case. I guess it'll be quicker if I
take your patch and massage it before throwing in, than to have
you make a v2.

Janan



Re: [PATCH] xen/events: fix build

2020-11-10 Thread Jürgen Groß

On 11.11.20 08:20, Jan Beulich wrote:

On 11.11.2020 06:49, Juergen Gross wrote:

--- a/xen/common/event_channel.c
+++ b/xen/common/event_channel.c
@@ -61,7 +61,9 @@ static inline void evtchn_write_lock(struct evtchn *evtchn)
  {
  write_lock(>lock);
  
+#ifndef NDEBUG

  evtchn->old_state = evtchn->state;
+#endif
  }
  
  static inline void evtchn_write_unlock(struct evtchn *evtchn)

diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index 7251b3ae3e..95f96e7a33 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -114,9 +114,7 @@ struct evtchn
  u16 virq;  /* state == ECS_VIRQ */
  } u;
  u8 priority;
-#ifndef NDEBUG
  u8 old_state;  /* State when taking lock in write mode. */
-#endif
  u8 last_priority;
  u16 last_vcpu_id;
  #ifdef CONFIG_XSM


Did you mean just either of the two changes (and then the former),
not both? If so
Reviewed-by: Jan Beulich 
and I'll be happy to drop the other half for committing.


The header fix is required for NDEBUG builds, while the other one is
removing a write with no accompanied read for NDEBUG builds.


Juergen



OpenPGP_0xB0DE9DD628BF132F.asc
Description: application/pgp-keys


OpenPGP_signature
Description: OpenPGP digital signature


Re: [PATCH V2 04/23] xen/ioreq: Provide alias for the handle_mmio()

2020-11-10 Thread Jan Beulich
On 10.11.2020 20:44, Oleksandr wrote:
> 
> On 20.10.20 13:38, Paul Durrant wrote:
> 
> Hi Jan, Paul
> 
> Sorry for the late response.
> 
>>> -Original Message-
>>> From: Jan Beulich 
>>> Sent: 20 October 2020 11:05
>>> To: p...@xen.org
>>> Cc: 'Oleksandr Tyshchenko' ; 
>>> xen-devel@lists.xenproject.org; 'Oleksandr
>>> Tyshchenko' ; 'Andrew Cooper' 
>>> ; 'Roger Pau
>>> Monné' ; 'Wei Liu' ; 'Julien Grall' 
>>> ; 'Stefano
>>> Stabellini' ; 'Julien Grall' 
>>> Subject: Re: [PATCH V2 04/23] xen/ioreq: Provide alias for the handle_mmio()
>>>
>>> On 20.10.2020 11:14, Paul Durrant wrote:
> From: Xen-devel  On Behalf Of 
> Oleksandr Tyshchenko
> Sent: 15 October 2020 17:44
>
> --- a/xen/include/asm-x86/hvm/ioreq.h
> +++ b/xen/include/asm-x86/hvm/ioreq.h
> @@ -181,6 +181,8 @@ static inline bool arch_hvm_ioreq_destroy(struct 
> domain *d)
>   #define IOREQ_STATUS_UNHANDLED   X86EMUL_UNHANDLEABLE
>   #define IOREQ_STATUS_RETRY   X86EMUL_RETRY
>
> +#define ioreq_complete_mmio   handle_mmio
> +
 A #define? Really? Can we not have a static inline?
>>> I guess this would require further shuffling: handle_mmio() is
>>> an inline function in hvm/emulate.h, and hvm/ioreq.h has no
>>> need to include the former (and imo it also shouldn't have).
>>>
>> I see. I think we need an x86 ioreq.c anyway, to deal with the legacy use of 
>> magic pages, so it could be dealt with there instead.
> I am afraid I don't entirely understand the required changes. Could you 
> please clarify where the "inline(?)" ioreq_complete_mmio() should
> live? I included hvm/emulate.h here not for the "handle_mmio()" reason 
> only, but for "struct hvm_emulate_ctxt" also (see arch_io_completion()).

I'm sorry, but in the context of this patch there's no use of any
struct hvm_emulate_ctxt instance. I'm not going to wade through 23
patches to find what you mean.

> But, if we return x86 ioreq.c back I can move arch_io_completion() to it 
> as well as "non-online" ioreq_complete_mmio().
> This will avoid including hvm/emulate.h here. Or I missed something?

I suppose an out-of-line function as kind of a last resort solution
is what Paul had in mind. To be honest I'd prefer to avoid the
extra call layer though, if possible.

Jan



Re: [PATCH] xen/events: fix build

2020-11-10 Thread Jan Beulich
On 11.11.2020 06:49, Juergen Gross wrote:
> --- a/xen/common/event_channel.c
> +++ b/xen/common/event_channel.c
> @@ -61,7 +61,9 @@ static inline void evtchn_write_lock(struct evtchn *evtchn)
>  {
>  write_lock(>lock);
>  
> +#ifndef NDEBUG
>  evtchn->old_state = evtchn->state;
> +#endif
>  }
>  
>  static inline void evtchn_write_unlock(struct evtchn *evtchn)
> diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
> index 7251b3ae3e..95f96e7a33 100644
> --- a/xen/include/xen/sched.h
> +++ b/xen/include/xen/sched.h
> @@ -114,9 +114,7 @@ struct evtchn
>  u16 virq;  /* state == ECS_VIRQ */
>  } u;
>  u8 priority;
> -#ifndef NDEBUG
>  u8 old_state;  /* State when taking lock in write mode. */
> -#endif
>  u8 last_priority;
>  u16 last_vcpu_id;
>  #ifdef CONFIG_XSM

Did you mean just either of the two changes (and then the former),
not both? If so
Reviewed-by: Jan Beulich 
and I'll be happy to drop the other half for committing.

Jan



[xen-unstable-smoke test] 156672: regressions - FAIL

2020-11-10 Thread osstest service owner
flight 156672 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/156672/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64   6 xen-buildfail REGR. vs. 156622

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-arm64-arm64-xl-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  16 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  628e1becb6fb121475a6ce68e3f1cb4499851255
baseline version:
 xen  3059178798a23ba870ff86ff54d442a07e6651fc

Last test of basis   156622  2020-11-10 13:01:19 Z0 days
Failing since156628  2020-11-10 17:00:28 Z0 days5 attempts
Testing same since   156642  2020-11-10 20:00:30 Z0 days4 attempts


People who touched revisions under test:
  Andrew Cooper 
  Jan Beulich 
  Juergen Gross 
  Julien Grall 

jobs:
 build-arm64-xsm  pass
 build-amd64  fail
 build-armhf  pass
 build-amd64-libvirt  blocked 
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  pass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64blocked 
 test-amd64-amd64-libvirt blocked 



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.


commit 628e1becb6fb121475a6ce68e3f1cb4499851255
Author: Julien Grall 
Date:   Mon Nov 9 20:28:59 2020 +

xen/arm: Always trap AMU system registers

The Activity Monitors Unit (AMU) has been introduced by ARMv8.4. It is
considered to be unsafe to be expose to guests as they might expose
information about code executed by other guests or the host.

Arm provided a way to trap all the AMU system registers by setting
CPTR_EL2.TAM to 1.

Unfortunately, on older revision of the specification, the bit 30 (now
CPTR_EL1.TAM) was RES0. Because of that, Xen is setting it to 0 and
therefore the system registers would be exposed to the guest when it is
run on processors with AMU.

As the bit is mark as UNKNOWN at boot in Armv8.4, the only safe solution
for us is to always set CPTR_EL1.TAM to 1.

Guest trying to access the AMU system registers will now receive an
undefined instruction. Unfortunately, this means that even well-behaved
guest may fail to boot because we don't sanitize the ID registers.

This is a known issues with other Armv8.0+ features (e.g. SVE, Pointer
Auth). This will taken care separately.

This is part of XSA-351 (or XSA-93 re-born).

Signed-off-by: Julien Grall 
Reviewed-by: Andre Przywara 
Reviewed-by: Stefano Stabellini 
Reviewed-by: Bertrand Marquis 

commit e6e85b662be9eab96f4cfc58e9945580cce8b2bb
Author: Jan Beulich 
Date:   Tue Nov 10 14:40:09 2020 +0100

x86/CPUID: also check leaf 7 max subleaf to be compatible

Just like is done for basic and extended major leaves.

Signed-off-by: Jan Beulich 
Acked-by: Andrew Cooper 

commit f5cfa09856732b1d78ff6a21ca3dc33a010da951
Author: Jan Beulich 
Date:   Tue Nov 10 14:39:30 2020 +0100

x86/CPUID: suppress IOMMU related hypervisor leaf data

Now that the IOMMU for guests can't be enabled "on demand" anymore,
there's also no reason to expose the related CPUID bit "just in case".

Signed-off-by: Jan Beulich 
Acked-by: Andrew Cooper 

commit db1a9fdd554cb1d8a7099af7925318fc06c6875b
Author: Jan Beulich 
Date:   Tue Nov 10 14:39:03 2020 +0100

x86/CPUID: don't use UB shift when library is built as 32-bit

At least the insn emulator test harness will 

[PATCH] xen/events: fix build

2020-11-10 Thread Juergen Gross
Commit 5f2df45ead7c1195 ("xen/evtchn: rework per event channel lock")
introduced a build failure for NDEBUG builds.

Fixes: 5f2df45ead7c1195 ("xen/evtchn: rework per event channel lock")
Signed-off-by: Juergen Gross 
---
 xen/common/event_channel.c | 2 ++
 xen/include/xen/sched.h| 2 --
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/xen/common/event_channel.c b/xen/common/event_channel.c
index eacd96b92f..da85d536f4 100644
--- a/xen/common/event_channel.c
+++ b/xen/common/event_channel.c
@@ -61,7 +61,9 @@ static inline void evtchn_write_lock(struct evtchn *evtchn)
 {
 write_lock(>lock);
 
+#ifndef NDEBUG
 evtchn->old_state = evtchn->state;
+#endif
 }
 
 static inline void evtchn_write_unlock(struct evtchn *evtchn)
diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index 7251b3ae3e..95f96e7a33 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -114,9 +114,7 @@ struct evtchn
 u16 virq;  /* state == ECS_VIRQ */
 } u;
 u8 priority;
-#ifndef NDEBUG
 u8 old_state;  /* State when taking lock in write mode. */
-#endif
 u8 last_priority;
 u16 last_vcpu_id;
 #ifdef CONFIG_XSM
-- 
2.26.2




test

2020-11-10 Thread Ian Jackson
Please don't reply.
.



RE: [PATCH V2 23/23] [RFC] libxl: Add support for virtio-disk configuration

2020-11-10 Thread Wei Chen
Hi Oleksandr,

> -Original Message-
> From: Oleksandr 
> Sent: 2020年11月11日 8:53
> To: Wei Chen ; xen-devel@lists.xenproject.org
> Cc: Oleksandr Tyshchenko ; Ian Jackson
> ; Wei Liu ; Anthony PERARD
> ; Julien Grall ; Stefano Stabellini
> 
> Subject: Re: [PATCH V2 23/23] [RFC] libxl: Add support for virtio-disk
> configuration
> 
> 
> On 09.11.20 08:45, Wei Chen wrote:
> > Hi Oleksandr,
> 
> Hi Wei
> 
> 
> >
> >> -Original Message-
> >> From: Xen-devel  On Behalf Of
> >> Oleksandr Tyshchenko
> >> Sent: 2020年10月16日 0:45
> >> To: xen-devel@lists.xenproject.org
> >> Cc: Oleksandr Tyshchenko ; Ian Jackson
> >> ; Wei Liu ; Anthony PERARD
> >> ; Julien Grall ; Stefano
> Stabellini
> >> 
> >> Subject: [PATCH V2 23/23] [RFC] libxl: Add support for virtio-disk
> configuration
> >>
> >> From: Oleksandr Tyshchenko 
> >>
> >> This patch adds basic support for configuring and assisting virtio-disk
> >> backend (emualator) which is intended to run out of Qemu and could be run
> >> in any domain.
> >>
> >> Xenstore was chosen as a communication interface for the emulator running
> >> in non-toolstack domain to be able to get configuration either by reading
> >> Xenstore directly or by receiving command line parameters (an updated 'xl
> devd'
> >> running in the same domain would read Xenstore beforehand and call
> backend
> >> executable with the required arguments).
> >>
> >> An example of domain configuration (two disks are assigned to the guest,
> >> the latter is in readonly mode):
> >>
> >> vdisk = [ 'backend=DomD, disks=rw:/dev/mmcblk0p3;ro:/dev/mmcblk1p3' ]
> >>
> > Can we keep use the same 'disk' parameter for virtio-disk, but add an option
> like
> >   "model=virtio-disk"?
> > For example:
> > disk = [ 'backend=DomD, disks=rw:/dev/mmcblk0p3,model=virtio-disk' ]
> > Just like what Xen has done for x86 virtio-net.
> 
> I think, this needs an additional investigation. In general I agree with
> you that it would be nice to reuse existing 'disk' parameter somehow
> rather than introducing new one
> for the same purpose (to handle virtual block device(s)).
> 
> 
> One note, although both are used for the same purpose they are different
> in at least one important option.
> 
> For example:
> 1. disk = [ 'backend=DomD, phy:/dev/mmcblk0p3, xvda1' ]
> 2. vdisk = [ 'backend=DomD, disks=rw:/dev/mmcblk0p3' ]
> As you can see existing "disk" parameter contains xvda1, this means that
> a new device /dev/xvda1 will appear at the guest side, but "vdisk"
> doesn't contain anything similar. So with Xen PV driver (xen-blkfront)

Yes, I understand your concerns. But I think specifying a device name
for virtio disk is not a mandatory requirement. Even if we're using physical
disks on bare metal machine, we can't guarantee slot#1 disk is always 'sda'.
So most modern OS are prefer to use blkid to mount filesystem.

> we are able to configure a device name, but with VirtIO solution
> (virtio-blk) we aren't (at least I don't know how exactly).
> 

Just my understanding, not exactly accurate:
The virtio-blk could not get VDEV information for a bus like Xen-bus. So the 
disk
ID is allocated dynamically in bus probe progress. The first probed disk will 
get
ID 'a'. And then the ID keeps increasing. If we want to specify the disk ID for 
virtio
disk, one possible way to do this is to construct a reasonable position on bus
(fdt node position of virtio mmio device, PCI Function ID of virtio pci block) 
for
virtio disk. But it is not always successful, we can't skip 'vda' to specify a 
virtio
disk as 'vdb'.

Regards,
Wei Chen
> 
> 
> 
> 
> --
> Regards,
> 
> Oleksandr Tyshchenko



Re: [PATCH 04/24] sd: update the bdev size in sd_revalidate_disk

2020-11-10 Thread Martin K. Petersen


Christoph,

> This avoids the extra call to revalidate_disk_size in sd_rescan and
> is otherwise a no-op because the size did not change, or we are in
> the probe path.

Acked-by: Martin K. Petersen 

-- 
Martin K. Petersen  Oracle Linux Engineering



[xen-unstable-smoke test] 156667: regressions - FAIL

2020-11-10 Thread osstest service owner
flight 156667 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/156667/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64   6 xen-buildfail REGR. vs. 156622

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-arm64-arm64-xl-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  16 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  628e1becb6fb121475a6ce68e3f1cb4499851255
baseline version:
 xen  3059178798a23ba870ff86ff54d442a07e6651fc

Last test of basis   156622  2020-11-10 13:01:19 Z0 days
Failing since156628  2020-11-10 17:00:28 Z0 days4 attempts
Testing same since   156642  2020-11-10 20:00:30 Z0 days3 attempts


People who touched revisions under test:
  Andrew Cooper 
  Jan Beulich 
  Juergen Gross 
  Julien Grall 

jobs:
 build-arm64-xsm  pass
 build-amd64  fail
 build-armhf  pass
 build-amd64-libvirt  blocked 
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  pass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64blocked 
 test-amd64-amd64-libvirt blocked 



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.


commit 628e1becb6fb121475a6ce68e3f1cb4499851255
Author: Julien Grall 
Date:   Mon Nov 9 20:28:59 2020 +

xen/arm: Always trap AMU system registers

The Activity Monitors Unit (AMU) has been introduced by ARMv8.4. It is
considered to be unsafe to be expose to guests as they might expose
information about code executed by other guests or the host.

Arm provided a way to trap all the AMU system registers by setting
CPTR_EL2.TAM to 1.

Unfortunately, on older revision of the specification, the bit 30 (now
CPTR_EL1.TAM) was RES0. Because of that, Xen is setting it to 0 and
therefore the system registers would be exposed to the guest when it is
run on processors with AMU.

As the bit is mark as UNKNOWN at boot in Armv8.4, the only safe solution
for us is to always set CPTR_EL1.TAM to 1.

Guest trying to access the AMU system registers will now receive an
undefined instruction. Unfortunately, this means that even well-behaved
guest may fail to boot because we don't sanitize the ID registers.

This is a known issues with other Armv8.0+ features (e.g. SVE, Pointer
Auth). This will taken care separately.

This is part of XSA-351 (or XSA-93 re-born).

Signed-off-by: Julien Grall 
Reviewed-by: Andre Przywara 
Reviewed-by: Stefano Stabellini 
Reviewed-by: Bertrand Marquis 

commit e6e85b662be9eab96f4cfc58e9945580cce8b2bb
Author: Jan Beulich 
Date:   Tue Nov 10 14:40:09 2020 +0100

x86/CPUID: also check leaf 7 max subleaf to be compatible

Just like is done for basic and extended major leaves.

Signed-off-by: Jan Beulich 
Acked-by: Andrew Cooper 

commit f5cfa09856732b1d78ff6a21ca3dc33a010da951
Author: Jan Beulich 
Date:   Tue Nov 10 14:39:30 2020 +0100

x86/CPUID: suppress IOMMU related hypervisor leaf data

Now that the IOMMU for guests can't be enabled "on demand" anymore,
there's also no reason to expose the related CPUID bit "just in case".

Signed-off-by: Jan Beulich 
Acked-by: Andrew Cooper 

commit db1a9fdd554cb1d8a7099af7925318fc06c6875b
Author: Jan Beulich 
Date:   Tue Nov 10 14:39:03 2020 +0100

x86/CPUID: don't use UB shift when library is built as 32-bit

At least the insn emulator test harness will 

[xen-4.14-testing test] 156617: regressions - FAIL

2020-11-10 Thread osstest service owner
flight 156617 xen-4.14-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/156617/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail REGR. 
vs. 156525

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-multivcpu 13 debian-fixupfail in 156594 pass in 156617
 test-amd64-i386-xl-qemuu-debianhvm-amd64 18 guest-localmigrate/x10 fail pass 
in 156594

Tests which did not succeed, but are not blocking:
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop fail like 156525
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stopfail like 156525
 test-armhf-armhf-libvirt 16 saverestore-support-checkfail  like 156525
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 156525
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stopfail like 156525
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stopfail like 156525
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop fail like 156525
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop fail like 156525
 test-armhf-armhf-libvirt-raw 15 saverestore-support-checkfail  like 156525
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop fail like 156525
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stopfail like 156525
 test-amd64-i386-xl-pvshim14 guest-start  fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt 15 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt  15 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-checkfail  never pass
 test-armhf-armhf-libvirt 15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  14 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  15 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  a38060ece699f22cd317219bec53c0d27279155b
baseline version:
 xen  0c96e4297da07944525729ddbe438b0131ab5b7e

Last test of basis   156525  2020-11-06 16:01:25 Z4 days
Testing same since   156594  2020-11-09 18:08:08 Z1 days2 attempts


Re: Xen on RP4

2020-11-10 Thread Christopher Clark
On Thu, Oct 29, 2020 at 12:58 PM Stefano Stabellini
 wrote:
>
> On Thu, 29 Oct 2020, Jürgen Groß wrote:
> > On 29.10.20 01:37, Stefano Stabellini wrote:
> > > On Tue, 27 Oct 2020, Elliott Mitchell wrote:
> > > > On Mon, Oct 26, 2020 at 06:44:27PM +, Julien Grall wrote:
> > > > > On 26/10/2020 16:03, Elliott Mitchell wrote:
> > > > > > On Mon, Oct 26, 2020 at 01:31:42PM +, Julien Grall wrote:
> > > > > > > On 24/10/2020 06:35, Elliott Mitchell wrote:
> > > > > > > > ACPI has a distinct
> > > > > > > > means of specifying a limited DMA-width; the above fails, 
> > > > > > > > because
> > > > > > > > it
> > > > > > > > assumes a *device-tree*.
> > > > > > >
> > > > > > > Do you know if it would be possible to infer from the ACPI static
> > > > > > > table
> > > > > > > the DMA-width?
> > > > > >
> > > > > > Yes, and it is.  Due to not knowing much about ACPI tables I don't
> > > > > > know
> > > > > > what the C code would look like though (problem is which 
> > > > > > documentation
> > > > > > should I be looking at first?).
> > > > >
> > > > > What you provided below is an excerpt of the DSDT. AFAIK, DSDT content
> > > > > is written in AML. So far the shortest implementation I have seen for
> > > > > the AML parser is around 5000 lines (see [1]). It might be possible to
> > > > > strip some the code, although I think this will still probably too big
> > > > > for a single workaround.
> > > > >
> > > > > What I meant by "static table" is a table that looks like a structure
> > > > > and can be parsed in a few lines. If we can't find on contain the DMA
> > > > > window, then the next best solution is to find a way to identity the
> > > > > platform.
> > > > >
> > > > > I don't know enough ACPI to know if this solution is possible. A good
> > > > > starter would probably be the ACPI spec [2].
> > > >
> > > > Okay, that is worse than I had thought (okay, ACPI is impressively
> > > > complex for something nominally firmware-level).
> > > >
> > > > There are strings in the present Tianocore implementation for Raspberry
> > > > PI 4B which could be targeted.  Notably included in the output during
> > > > boot listing the tables, "RPIFDN", "RPIFDN RPI" and "RPIFDN RPI4" (I'm
> > > > unsure how kosher these are as this wsn't implemented nor blessed by the
> > > > Raspberry PI Foundation).
> > > >
> > > > I strongly dislike this approach as you soon turn the Xen project into a
> > > > database of hardware.  This is already occurring with
> > > > xen/arch/arm/platforms and I would love to do something about this.  On
> > > > that thought, how about utilizing Xen's command-line for this purpose?
> > >
> > > I don't think that a command line option is a good idea: basically it is
> > > punting to users the task of platform detection. Also, it means that
> > > users will be necessarily forced to edit the uboot script or grub
> > > configuration file to boot.
> > >
> > > Note that even if we introduced a new command line, we wouldn't take
> > > away the need for xen/arch/arm/platforms anyway.
> > >
> > > It would be far better for Xen to autodetect the platform if we can.
> > >
> > >
> > > > Have a procedure of during installation/updates retrieve DMA limitation
> > > > information from the running OS and the following boot Xen will follow
> > > > the appropriate setup.  By its nature, Domain 0 will have the 
> > > > information
> > > > needed, just becomes an issue of how hard that is to retrieve...
> > >
> > > Historically that is what we used to do for many things related to ACPI,
> > > but unfortunately it leads to a pretty bad architecture where Xen
> > > depends on Dom0 for booting rather than the other way around. (Dom0
> > > should be the one requiring Xen for booting, given that Xen is higher
> > > privilege and boots first.)
> > >
> > >
> > > I think the best compromise is still to use an ACPI string to detect the
> > > platform. For instance, would it be possible to use the OEMID fields in
> > > RSDT, XSDT, FADT?  Possibly even a combination of them?
> > >
> > > Another option might be to get the platform name from UEFI somehow.
> >
> > What about having a small domain parsing the ACPI booting first and use
> > that information for booting dom0?
> >
> > That dom would be part of the Xen build and the hypervisor wouldn't need
> > to gain all the ACPI AML logic.
>
> That could work, but in practice we don't have such a domain today --
> the infrastructure is missing. I wonder whether the bootloader (uboot or
> grub) would know about the platform and might be able to pass that
> information to Xen somehow.

This is one of the functions envisioned for the Boot Domain in the
proposal to add DomB mode to dom0less[1], in the work developed by
Daniel Smith and I.

With DomB as much or as little of platform detection and initial
domain configuration will be possible from the Boot Domain. ACPI was
specifically discussed in the second DomB Design Session at this
year's Xen Summit[2].

Further information on 

[xen-unstable-smoke test] 156659: regressions - FAIL

2020-11-10 Thread osstest service owner
flight 156659 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/156659/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64   6 xen-buildfail REGR. vs. 156622

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-arm64-arm64-xl-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  16 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  628e1becb6fb121475a6ce68e3f1cb4499851255
baseline version:
 xen  3059178798a23ba870ff86ff54d442a07e6651fc

Last test of basis   156622  2020-11-10 13:01:19 Z0 days
Failing since156628  2020-11-10 17:00:28 Z0 days3 attempts
Testing same since   156642  2020-11-10 20:00:30 Z0 days2 attempts


People who touched revisions under test:
  Andrew Cooper 
  Jan Beulich 
  Juergen Gross 
  Julien Grall 

jobs:
 build-arm64-xsm  pass
 build-amd64  fail
 build-armhf  pass
 build-amd64-libvirt  blocked 
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  pass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64blocked 
 test-amd64-amd64-libvirt blocked 



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.


commit 628e1becb6fb121475a6ce68e3f1cb4499851255
Author: Julien Grall 
Date:   Mon Nov 9 20:28:59 2020 +

xen/arm: Always trap AMU system registers

The Activity Monitors Unit (AMU) has been introduced by ARMv8.4. It is
considered to be unsafe to be expose to guests as they might expose
information about code executed by other guests or the host.

Arm provided a way to trap all the AMU system registers by setting
CPTR_EL2.TAM to 1.

Unfortunately, on older revision of the specification, the bit 30 (now
CPTR_EL1.TAM) was RES0. Because of that, Xen is setting it to 0 and
therefore the system registers would be exposed to the guest when it is
run on processors with AMU.

As the bit is mark as UNKNOWN at boot in Armv8.4, the only safe solution
for us is to always set CPTR_EL1.TAM to 1.

Guest trying to access the AMU system registers will now receive an
undefined instruction. Unfortunately, this means that even well-behaved
guest may fail to boot because we don't sanitize the ID registers.

This is a known issues with other Armv8.0+ features (e.g. SVE, Pointer
Auth). This will taken care separately.

This is part of XSA-351 (or XSA-93 re-born).

Signed-off-by: Julien Grall 
Reviewed-by: Andre Przywara 
Reviewed-by: Stefano Stabellini 
Reviewed-by: Bertrand Marquis 

commit e6e85b662be9eab96f4cfc58e9945580cce8b2bb
Author: Jan Beulich 
Date:   Tue Nov 10 14:40:09 2020 +0100

x86/CPUID: also check leaf 7 max subleaf to be compatible

Just like is done for basic and extended major leaves.

Signed-off-by: Jan Beulich 
Acked-by: Andrew Cooper 

commit f5cfa09856732b1d78ff6a21ca3dc33a010da951
Author: Jan Beulich 
Date:   Tue Nov 10 14:39:30 2020 +0100

x86/CPUID: suppress IOMMU related hypervisor leaf data

Now that the IOMMU for guests can't be enabled "on demand" anymore,
there's also no reason to expose the related CPUID bit "just in case".

Signed-off-by: Jan Beulich 
Acked-by: Andrew Cooper 

commit db1a9fdd554cb1d8a7099af7925318fc06c6875b
Author: Jan Beulich 
Date:   Tue Nov 10 14:39:03 2020 +0100

x86/CPUID: don't use UB shift when library is built as 32-bit

At least the insn emulator test harness will 

[xen-unstable bisection] complete test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm

2020-11-10 Thread osstest service owner
branch xen-unstable
xenbranch xen-unstable
job test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm
testid debian-hvm-install

Tree: linux git://xenbits.xen.org/linux-pvops.git
Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: xen git://xenbits.xen.org/xen.git

*** Found and reproduced problem changeset ***

  Bug is in tree:  xen git://xenbits.xen.org/xen.git
  Bug introduced:  e19bcb626f50a652fb1854a8b2f2c9c371687a11
  Bug not present: c3453a23f7905d24f2404787543e26ec7d02301c
  Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/156664/


  commit e19bcb626f50a652fb1854a8b2f2c9c371687a11
  Author: Juergen Gross 
  Date:   Fri Nov 6 10:48:07 2020 +0100
  
  xen/rwlock: add check_lock() handling to rwlocks
  
  Checking whether a lock is consistently used regarding interrupts on
  or off is beneficial for rwlocks, too.
  
  So add check_lock() calls to rwlock functions. For this purpose make
  check_lock() globally accessible.
  
  Signed-off-by: Juergen Gross 
  Reviewed-by: Julien Grall 
  Reviewed-by: Jan Beulich 


For bisection revision-tuple graph see:
   
http://logs.test-lab.xenproject.org/osstest/results/bisect/xen-unstable/test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm.debian-hvm-install.html
Revision IDs in each graph node refer, respectively, to the Trees above.


Running cs-bisection-step 
--graph-out=/home/logs/results/bisect/xen-unstable/test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm.debian-hvm-install
 --summary-out=tmp/156664.bisection-summary --basis-template=156443 
--blessings=real,real-bisect,real-retry xen-unstable 
test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm debian-hvm-install
Searching for failure / basis pass:
 156608 fail [host=godello0] / 156443 [host=albana0] 156401 [host=huxelrebe1] 
156389 [host=godello1] 156373 [host=albana1] 156354 [host=fiano1] 156339 ok.
Failure / basis pass flights: 156608 / 156339
(tree with no url: minios)
(tree with no url: ovmf)
(tree with no url: seabios)
Tree: linux git://xenbits.xen.org/linux-pvops.git
Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: xen git://xenbits.xen.org/xen.git
Latest c3038e718a19fc596f7b1baba0f83d5146dc7784 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 
7ea428895af2840d85c524f0bd11a38aac308308 
0a5e0ce0fb7e5a3b5dfdc936058d2c0e04e5e258
Basis pass c3038e718a19fc596f7b1baba0f83d5146dc7784 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 
677cbe1324c29294bb1d1b8454b3f214725e40fd 
7056f2f89f03f2f804ac7e776c7b2b000cd716cd
Generating revisions with ./adhoc-revtuple-generator  
git://xenbits.xen.org/linux-pvops.git#c3038e718a19fc596f7b1baba0f83d5146dc7784-c3038e718a19fc596f7b1baba0f83d5146dc7784
 
git://xenbits.xen.org/osstest/linux-firmware.git#c530a75c1e6a472b0eb9558310b518f0dfcd8860-c530a75c1e6a472b0eb9558310b518f0dfcd8860
 
git://xenbits.xen.org/qemu-xen-traditional.git#3d273dd05e51e5a1ffba3d98c7437ee84e8f8764-3d273dd05e51e5a1ffba3d98c7437ee84e8f8764
 git://xenbits.xen.org/qemu-xen.git#677cbe1324c29294bb1d1b8454b3f21\
 4725e40fd-7ea428895af2840d85c524f0bd11a38aac308308 
git://xenbits.xen.org/xen.git#7056f2f89f03f2f804ac7e776c7b2b000cd716cd-0a5e0ce0fb7e5a3b5dfdc936058d2c0e04e5e258
Loaded 10007 nodes in revision graph
Searching for test results:
 156331 [host=elbling0]
 156339 pass c3038e718a19fc596f7b1baba0f83d5146dc7784 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 
677cbe1324c29294bb1d1b8454b3f214725e40fd 
7056f2f89f03f2f804ac7e776c7b2b000cd716cd
 156354 [host=fiano1]
 156373 [host=albana1]
 156389 [host=godello1]
 156401 [host=huxelrebe1]
 156443 [host=albana0]
 156524 fail c3038e718a19fc596f7b1baba0f83d5146dc7784 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 
677cbe1324c29294bb1d1b8454b3f214725e40fd 
2a5f9f6a6932214fda76b9b3c03e024772882d34
 156538 fail irrelevant
 156556 fail irrelevant
 156577 fail irrelevant
 156588 fail c3038e718a19fc596f7b1baba0f83d5146dc7784 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 
7ea428895af2840d85c524f0bd11a38aac308308 
0a5e0ce0fb7e5a3b5dfdc936058d2c0e04e5e258
 156626 pass c3038e718a19fc596f7b1baba0f83d5146dc7784 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 
677cbe1324c29294bb1d1b8454b3f214725e40fd 
7056f2f89f03f2f804ac7e776c7b2b000cd716cd
 156627 fail c3038e718a19fc596f7b1baba0f83d5146dc7784 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 
7ea428895af2840d85c524f0bd11a38aac308308 
0a5e0ce0fb7e5a3b5dfdc936058d2c0e04e5e258
 156629 pass 

Re: [PATCH V2 23/23] [RFC] libxl: Add support for virtio-disk configuration

2020-11-10 Thread Oleksandr



On 09.11.20 08:45, Wei Chen wrote:

Hi Oleksandr,


Hi Wei





-Original Message-
From: Xen-devel  On Behalf Of
Oleksandr Tyshchenko
Sent: 2020年10月16日 0:45
To: xen-devel@lists.xenproject.org
Cc: Oleksandr Tyshchenko ; Ian Jackson
; Wei Liu ; Anthony PERARD
; Julien Grall ; Stefano Stabellini

Subject: [PATCH V2 23/23] [RFC] libxl: Add support for virtio-disk configuration

From: Oleksandr Tyshchenko 

This patch adds basic support for configuring and assisting virtio-disk
backend (emualator) which is intended to run out of Qemu and could be run
in any domain.

Xenstore was chosen as a communication interface for the emulator running
in non-toolstack domain to be able to get configuration either by reading
Xenstore directly or by receiving command line parameters (an updated 'xl devd'
running in the same domain would read Xenstore beforehand and call backend
executable with the required arguments).

An example of domain configuration (two disks are assigned to the guest,
the latter is in readonly mode):

vdisk = [ 'backend=DomD, disks=rw:/dev/mmcblk0p3;ro:/dev/mmcblk1p3' ]


Can we keep use the same 'disk' parameter for virtio-disk, but add an option 
like
  "model=virtio-disk"?
For example:
disk = [ 'backend=DomD, disks=rw:/dev/mmcblk0p3,model=virtio-disk' ]
Just like what Xen has done for x86 virtio-net.


I think, this needs an additional investigation. In general I agree with 
you that it would be nice to reuse existing 'disk' parameter somehow 
rather than introducing new one

for the same purpose (to handle virtual block device(s)).


One note, although both are used for the same purpose they are different 
in at least one important option.


For example:
1. disk = [ 'backend=DomD, phy:/dev/mmcblk0p3, xvda1' ]
2. vdisk = [ 'backend=DomD, disks=rw:/dev/mmcblk0p3' ]
As you can see existing "disk" parameter contains xvda1, this means that 
a new device /dev/xvda1 will appear at the guest side, but "vdisk" 
doesn't contain anything similar. So with Xen PV driver (xen-blkfront) 
we are able to configure a device name, but with VirtIO solution 
(virtio-blk) we aren't (at least I don't know how exactly).






--
Regards,

Oleksandr Tyshchenko




[xen-unstable test] 156608: regressions - FAIL

2020-11-10 Thread osstest service owner
flight 156608 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/156608/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-xsm   14 guest-start  fail REGR. vs. 156443
 test-amd64-amd64-xl-xsm  14 guest-start  fail REGR. vs. 156443
 test-amd64-amd64-libvirt-xsm 14 guest-start  fail REGR. vs. 156443
 test-amd64-i386-libvirt-xsm  14 guest-start  fail REGR. vs. 156443
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 12 debian-hvm-install 
fail REGR. vs. 156443
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 12 debian-hvm-install 
fail REGR. vs. 156443
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail REGR. 
vs. 156443
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 12 debian-hvm-install fail 
REGR. vs. 156443
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 12 debian-hvm-install fail 
REGR. vs. 156443
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm 12 debian-hvm-install fail REGR. 
vs. 156443
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail REGR. 
vs. 156443
 test-amd64-amd64-xl-qemuu-debianhvm-i386-xsm 12 debian-hvm-install fail REGR. 
vs. 156443

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-rtds 20 guest-localmigrate/x10 fail pass in 156588
 test-armhf-armhf-xl-rtds 18 guest-start/debian.repeat  fail pass in 156588

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt 16 saverestore-support-checkfail  like 156443
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stopfail like 156443
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stopfail like 156443
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop fail like 156443
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stopfail like 156443
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop fail like 156443
 test-armhf-armhf-libvirt-raw 15 saverestore-support-checkfail  like 156443
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop fail like 156443
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stopfail like 156443
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 156443
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop fail like 156443
 test-amd64-i386-xl-pvshim14 guest-start  fail   never pass
 test-amd64-amd64-libvirt 15 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-checkfail never pass
 test-armhf-armhf-libvirt 15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 16 saverestore-support-checkfail   never pass
 

Re: [PATCH V2 21/23] xen/arm: Add mapcache invalidation handling

2020-11-10 Thread Oleksandr



On 16.10.20 11:41, Julien Grall wrote:

Hi Jan, Julien

Sorry for the late response.


Hi Jan,

On 16/10/2020 07:29, Jan Beulich wrote:

On 15.10.2020 18:44, Oleksandr Tyshchenko wrote:
@@ -1067,7 +1068,14 @@ static int __p2m_set_entry(struct p2m_domain 
*p2m,

   */
  if ( p2m_is_valid(orig_pte) &&
   !mfn_eq(lpae_get_mfn(*entry), lpae_get_mfn(orig_pte)) )
+    {
+#ifdef CONFIG_IOREQ_SERVER
+    if ( domain_has_ioreq_server(p2m->domain) &&
+ (p2m->domain == current->domain) && 
p2m_is_ram(orig_pte.p2m.type) )

+    p2m->domain->qemu_mapcache_invalidate = true;
+#endif
  p2m_free_entry(p2m, orig_pte, level);
+    }


For all I have to say here, please bear in mind that I don't know
the internals of Arm memory management.

The first odd thing here the merely MFN-based condition. It may
well be that's sufficient, if there's no way to get a "not present"
entry with an MFN matching any valid MFN. (This isn't just with
your addition, but even before.
Invalid entries are always zeroed. So in theory the problem could 
arise if MFN 0 used in the guest. It should not be possible on 
staging, but I agree this should be fixed.




Given how p2m_free_entry() works (or is supposed to work in the
long run), is the new code you add guaranteed to only alter leaf
entries?


This path may also be called with tables. I think we want to move the 
check in p2m_free_entry() so we can find the correct leaf type.


Well, but inside p2m_free_entry() we don't have a new entry in order to 
check whether new MFN is actually different. An extra arg only comes to 
mind...


[Didn't update call sites yet and didn't tested]

diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c
index 5bb23df..4001f46 100644
--- a/xen/arch/arm/p2m.c
+++ b/xen/arch/arm/p2m.c
@@ -739,7 +739,7 @@ static void p2m_put_l3_page(const lpae_t pte)

 /* Free lpae sub-tree behind an entry */
 static void p2m_free_entry(struct p2m_domain *p2m,
-   lpae_t entry, unsigned int level)
+   lpae_t entry, lpae_t new_entry, unsigned int 
level)

 {
 unsigned int i;
 lpae_t *table;
@@ -750,17 +750,19 @@ static void p2m_free_entry(struct p2m_domain *p2m,
 if ( !p2m_is_valid(entry) )
 return;

-    /* Nothing to do but updating the stats if the entry is a 
super-page. */

-    if ( p2m_is_superpage(entry, level) )
+    if ( p2m_is_superpage(entry, level) || (level == 3) )
 {
-    p2m->stats.mappings[level]--;
-    return;
-    }
+#ifdef CONFIG_IOREQ_SERVER
+    if ( domain_has_ioreq_server(p2m->domain) &&
+ (p2m->domain == current->domain) &&
+ !mfn_eq(lpae_get_mfn(new_entry), lpae_get_mfn(entry)) &&
+ p2m_is_ram(entry.p2m.type) )
+    p2m->domain->qemu_mapcache_invalidate = true;
+#endif

-    if ( level == 3 )
-    {
 p2m->stats.mappings[level]--;
-    p2m_put_l3_page(entry);
+    if ( level == 3 )
+    p2m_put_l3_page(entry);
 return;
 }

(END)

--
Regards,

Oleksandr Tyshchenko




[xen-unstable-smoke bisection] complete build-amd64

2020-11-10 Thread osstest service owner
branch xen-unstable-smoke
xenbranch xen-unstable-smoke
job build-amd64
testid xen-build

Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: xen git://xenbits.xen.org/xen.git

*** Found and reproduced problem changeset ***

  Bug is in tree:  xen git://xenbits.xen.org/xen.git
  Bug introduced:  5f2df45ead7c1195142f68b7923047a1e9479d54
  Bug not present: 3059178798a23ba870ff86ff54d442a07e6651fc
  Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/156660/


  commit 5f2df45ead7c1195142f68b7923047a1e9479d54
  Author: Juergen Gross 
  Date:   Tue Nov 10 14:36:15 2020 +0100
  
  xen/evtchn: rework per event channel lock
  
  Currently the lock for a single event channel needs to be taken with
  interrupts off, which causes deadlocks in some cases.
  
  Rework the per event channel lock to be non-blocking for the case of
  sending an event and removing the need for disabling interrupts for
  taking the lock.
  
  The lock is needed for avoiding races between event channel state
  changes (creation, closing, binding) against normal operations (set
  pending, [un]masking, priority changes).
  
  Use a rwlock, but with some restrictions:
  
  - Changing the state of an event channel (creation, closing, binding)
needs to use write_lock(), with ASSERT()ing that the lock is taken as
writer only when the state of the event channel is either before or
after the locked region appropriate (either free or unbound).
  
  - Sending an event needs to use read_trylock() mostly, in case of not
obtaining the lock the operation is omitted. This is needed as
sending an event can happen with interrupts off (at least in some
cases).
  
  - Dumping the event channel state for debug purposes is using
read_trylock(), too, in order to avoid blocking in case the lock is
taken as writer for a long time.
  
  - All other cases can use read_lock().
  
  Fixes: e045199c7c9c54 ("evtchn: address races with evtchn_reset()")
  Signed-off-by: Juergen Gross 
  Reviewed-by: Jan Beulich 
  Acked-by: Julien Grall 


For bisection revision-tuple graph see:
   
http://logs.test-lab.xenproject.org/osstest/results/bisect/xen-unstable-smoke/build-amd64.xen-build.html
Revision IDs in each graph node refer, respectively, to the Trees above.


Running cs-bisection-step 
--graph-out=/home/logs/results/bisect/xen-unstable-smoke/build-amd64.xen-build 
--summary-out=tmp/156660.bisection-summary --basis-template=156622 
--blessings=real,real-bisect,real-retry xen-unstable-smoke build-amd64 xen-build
Searching for failure / basis pass:
 156642 fail [host=himrod1] / 156622 [host=himrod2] 156532 [host=himrod2] 
156523 ok.
Failure / basis pass flights: 156642 / 156523
(tree with no url: minios)
(tree with no url: ovmf)
(tree with no url: seabios)
Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: xen git://xenbits.xen.org/xen.git
Latest 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 
7ea428895af2840d85c524f0bd11a38aac308308 
628e1becb6fb121475a6ce68e3f1cb4499851255
Basis pass 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 
677cbe1324c29294bb1d1b8454b3f214725e40fd 
2a5f9f6a6932214fda76b9b3c03e024772882d34
Generating revisions with ./adhoc-revtuple-generator  
git://xenbits.xen.org/qemu-xen-traditional.git#3d273dd05e51e5a1ffba3d98c7437ee84e8f8764-3d273dd05e51e5a1ffba3d98c7437ee84e8f8764
 
git://xenbits.xen.org/qemu-xen.git#677cbe1324c29294bb1d1b8454b3f214725e40fd-7ea428895af2840d85c524f0bd11a38aac308308
 
git://xenbits.xen.org/xen.git#2a5f9f6a6932214fda76b9b3c03e024772882d34-628e1becb6fb121475a6ce68e3f1cb4499851255
Loaded 10007 nodes in revision graph
Searching for test results:
 156523 pass 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 
677cbe1324c29294bb1d1b8454b3f214725e40fd 
2a5f9f6a6932214fda76b9b3c03e024772882d34
 156532 [host=himrod2]
 156622 [host=himrod2]
 156628 fail 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 
7ea428895af2840d85c524f0bd11a38aac308308 
e6e85b662be9eab96f4cfc58e9945580cce8b2bb
 156641 pass 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 
677cbe1324c29294bb1d1b8454b3f214725e40fd 
2a5f9f6a6932214fda76b9b3c03e024772882d34
 156644 fail 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 
7ea428895af2840d85c524f0bd11a38aac308308 
e6e85b662be9eab96f4cfc58e9945580cce8b2bb
 156647 pass 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 
7ea428895af2840d85c524f0bd11a38aac308308 
0a5e0ce0fb7e5a3b5dfdc936058d2c0e04e5e258
 156650 fail 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 
7ea428895af2840d85c524f0bd11a38aac308308 
b5ad37f8e9284cc147218f7a5193d739ae7b956f
 156652 pass 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 
7ea428895af2840d85c524f0bd11a38aac308308 
3059178798a23ba870ff86ff54d442a07e6651fc
 156654 fail 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 

Re: Xen on RP4

2020-11-10 Thread Elliott Mitchell
On Thu, Oct 29, 2020 at 12:57:58PM -0700, Stefano Stabellini wrote:
> On Thu, 29 Oct 2020, J??rgen Gro?? wrote:
> > What about having a small domain parsing the ACPI booting first and use
> > that information for booting dom0?
> > 
> > That dom would be part of the Xen build and the hypervisor wouldn't need
> > to gain all the ACPI AML logic.
> 
> That could work, but in practice we don't have such a domain today --
> the infrastructure is missing. I wonder whether the bootloader (uboot or
> grub) would know about the platform and might be able to pass that
> information to Xen somehow.

How long would such likely take to implement?  This reads like a
complicated project, and likely to take a while...


Then would be the issue of efifb.



I've been pondering allocate_memory_11() and coming up with a rather
complicated potential problem.  ACPI appears to potentially allow for
non-power of 2 DMA ranges; I'm unaware of any such device, but the code
should allow for such.

I can imagine a device which has multiple DMA ranges.  The ranges could
be fully contained within each other, the ranges could partially overlap,
or the ranges could be disjoint.

Someone might wish to allocate all DMA-capable memory to domain 0,
someone might wish to allocate less.  Additionally if all DMA-capable
memory is allocated to domain 0, some non-DMA-capable memory could be
desired.

Ideally Xen would move to non-DMA memory.  This would protect Xen against
a malicious domain 0 and allow allocating more DMA-capable memory to
domain 0.

This interacts with ballooning.  If memory is removed from domain 0,
non-DMA memory should be removed first.  If domain 0 is allocated more
memory, DMA memory should be added first (if any isn't allocated to
domain 0).

Then again I may be severely overthinking things.


-- 
(\___(\___(\__  --=> 8-) EHM <=--  __/)___/)___/)
 \BS (| ehem+sig...@m5p.com  PGP 87145445 |)   /
  \_CS\   |  _  -O #include  O-   _  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445





[xen-unstable-smoke test] 156642: regressions - FAIL

2020-11-10 Thread osstest service owner
flight 156642 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/156642/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64   6 xen-buildfail REGR. vs. 156622

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-arm64-arm64-xl-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  16 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  628e1becb6fb121475a6ce68e3f1cb4499851255
baseline version:
 xen  3059178798a23ba870ff86ff54d442a07e6651fc

Last test of basis   156622  2020-11-10 13:01:19 Z0 days
Failing since156628  2020-11-10 17:00:28 Z0 days2 attempts
Testing same since   156642  2020-11-10 20:00:30 Z0 days1 attempts


People who touched revisions under test:
  Andrew Cooper 
  Jan Beulich 
  Juergen Gross 
  Julien Grall 

jobs:
 build-arm64-xsm  pass
 build-amd64  fail
 build-armhf  pass
 build-amd64-libvirt  blocked 
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  pass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64blocked 
 test-amd64-amd64-libvirt blocked 



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.


commit 628e1becb6fb121475a6ce68e3f1cb4499851255
Author: Julien Grall 
Date:   Mon Nov 9 20:28:59 2020 +

xen/arm: Always trap AMU system registers

The Activity Monitors Unit (AMU) has been introduced by ARMv8.4. It is
considered to be unsafe to be expose to guests as they might expose
information about code executed by other guests or the host.

Arm provided a way to trap all the AMU system registers by setting
CPTR_EL2.TAM to 1.

Unfortunately, on older revision of the specification, the bit 30 (now
CPTR_EL1.TAM) was RES0. Because of that, Xen is setting it to 0 and
therefore the system registers would be exposed to the guest when it is
run on processors with AMU.

As the bit is mark as UNKNOWN at boot in Armv8.4, the only safe solution
for us is to always set CPTR_EL1.TAM to 1.

Guest trying to access the AMU system registers will now receive an
undefined instruction. Unfortunately, this means that even well-behaved
guest may fail to boot because we don't sanitize the ID registers.

This is a known issues with other Armv8.0+ features (e.g. SVE, Pointer
Auth). This will taken care separately.

This is part of XSA-351 (or XSA-93 re-born).

Signed-off-by: Julien Grall 
Reviewed-by: Andre Przywara 
Reviewed-by: Stefano Stabellini 
Reviewed-by: Bertrand Marquis 

commit e6e85b662be9eab96f4cfc58e9945580cce8b2bb
Author: Jan Beulich 
Date:   Tue Nov 10 14:40:09 2020 +0100

x86/CPUID: also check leaf 7 max subleaf to be compatible

Just like is done for basic and extended major leaves.

Signed-off-by: Jan Beulich 
Acked-by: Andrew Cooper 

commit f5cfa09856732b1d78ff6a21ca3dc33a010da951
Author: Jan Beulich 
Date:   Tue Nov 10 14:39:30 2020 +0100

x86/CPUID: suppress IOMMU related hypervisor leaf data

Now that the IOMMU for guests can't be enabled "on demand" anymore,
there's also no reason to expose the related CPUID bit "just in case".

Signed-off-by: Jan Beulich 
Acked-by: Andrew Cooper 

commit db1a9fdd554cb1d8a7099af7925318fc06c6875b
Author: Jan Beulich 
Date:   Tue Nov 10 14:39:03 2020 +0100

x86/CPUID: don't use UB shift when library is built as 32-bit

At least the insn emulator test harness will 

Re: [PATCH for-5.10] swiotlb: remove the tbl_dma_addr argument to swiotlb_tbl_map_single

2020-11-10 Thread Konrad Rzeszutek Wilk
On Tue, Nov 10, 2020 at 10:14:21AM +0100, Christoph Hellwig wrote:
> On Wed, Nov 04, 2020 at 09:04:38AM -0500, Konrad Rzeszutek Wilk wrote:
> > On Tue, Nov 03, 2020 at 10:46:43AM +0100, Christoph Hellwig wrote:
> > > ping?
> > 
> > Hopefully this goes through. I am in the process of testing it but ran
> > into testing issues that I believe are unrelated.
> 
> Did you manage to make any progress?  This fixes an issue with the

YES!!
> new support for systems with DMA offsets in 5.10..

OK. Sending the git pull request in a minute or two.



Re: [PATCH V2 11/23] xen/ioreq: Move x86's io_completion/io_req fields to struct vcpu

2020-11-10 Thread Oleksandr



On 20.10.20 13:55, Paul Durrant wrote:

Hi Paul.

Sorry for the late response.


-Original Message-
From: Oleksandr Tyshchenko 
Sent: 15 October 2020 17:44
To: xen-devel@lists.xenproject.org
Cc: Oleksandr Tyshchenko ; Paul Durrant 
; Jan Beulich
; Andrew Cooper ; Roger Pau Monné
; Wei Liu ; George Dunlap 
; Ian Jackson
; Julien Grall ; Stefano Stabellini 
; Jun
Nakajima ; Kevin Tian ; Julien 
Grall

Subject: [PATCH V2 11/23] xen/ioreq: Move x86's io_completion/io_req fields to 
struct vcpu

From: Oleksandr Tyshchenko 

The IOREQ is a common feature now and these fields will be used
on Arm as is. Move them to common struct vcpu as a part of new
struct vcpu_io. Also move enum hvm_io_completion to xen/sched.h
and remove "hvm" prefixes.

This patch completely removes layering violation in the common code.

Signed-off-by: Oleksandr Tyshchenko 
CC: Julien Grall 

---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"

***
I was thinking that it may be better to place these two fields
into struct vcpu directly (without intermediate "io" struct).
I think, this way the code which operates with these fields
would become cleaner. Another possible option would be either
to rename "io" struct (I failed to think of a better name) or
to drop(replace?) duplicating "io" prefixes from these fields.

Just drop the 'io_' prefix from the field names.


Will drop. This would look like indeed better.


Thank you.


--
Regards,

Oleksandr Tyshchenko




[qemu-mainline test] 156603: regressions - FAIL

2020-11-10 Thread osstest service owner
flight 156603 qemu-mainline real [real]
flight 156645 qemu-mainline real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/156603/
http://logs.test-lab.xenproject.org/osstest/logs/156645/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-libvirt-xsm 14 guest-start  fail REGR. vs. 152631
 test-armhf-armhf-xl-vhd  12 debian-di-installfail REGR. vs. 152631
 test-amd64-amd64-libvirt-vhd 19 guest-start/debian.repeat fail REGR. vs. 152631
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 16 guest-saverestore.2 fail 
REGR. vs. 152631
 test-armhf-armhf-libvirt-raw 12 debian-di-installfail REGR. vs. 152631
 test-amd64-amd64-xl-qcow2   21 guest-start/debian.repeat fail REGR. vs. 152631
 test-armhf-armhf-libvirt 14 guest-start  fail REGR. vs. 152631

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stopfail like 152631
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop fail like 152631
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stopfail like 152631
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop fail like 152631
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152631
 test-arm64-arm64-xl-seattle  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt 15 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  15 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-checkfail   never pass
 test-amd64-i386-xl-pvshim14 guest-start  fail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  16 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check 
fail never pass
 test-armhf-armhf-xl  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 16 saverestore-support-checkfail   never pass

version targeted for testing:
 qemuu43afbbd9fea1b255cc81f5f4bfd0b6a88826c735
baseline version:
 qemuu1d806cef0e38b5db8347a8e12f214d543204a314

Last test of basis   152631  2020-08-20 09:07:46 Z   82 days
Failing since152659  2020-08-21 14:07:39 Z   81 days  177 attempts
Testing same since   156603  2020-11-09 22:07:49 Z0 days1 attempts


People who touched revisions under test:
Aaron Lindsay 
  Alberto Garcia 
  Aleksandar Markovic 
  Alex Bennée 
  Alex Chen 
  Alex Williamson 
  Alexander Bulekov 
  AlexChen 
  Alexey Kirillov 
  Alistair Francis 
  Alistair Francis 
  Amey Narkhede 
  Ana Pazos 
  Andreas Gustafsson 
  Andrew Jones 
  Andrey Konovalov 
  Andrey Shinkevich 
  Ani Sinha 
  Anthony PERARD 
  Anton Blanchard 
  Anup Patel 
  Artyom Tarasenko 
  Babu Moger 
  BALATON Zoltan 
  

Re: [PATCH V2 17/23] xen/ioreq: Introduce domain_has_ioreq_server()

2020-11-10 Thread Oleksandr



On 20.10.20 13:51, Paul Durrant wrote:

Hi Paul.

Sorry for the late response.




-Original Message-
From: Oleksandr Tyshchenko 
Sent: 15 October 2020 17:44
To: xen-devel@lists.xenproject.org
Cc: Oleksandr Tyshchenko ; Stefano Stabellini 
;
Julien Grall ; Volodymyr Babchuk ; 
Andrew Cooper
; George Dunlap ; Ian 
Jackson
; Jan Beulich ; Wei Liu ; 
Paul Durrant
; Julien Grall 
Subject: [PATCH V2 17/23] xen/ioreq: Introduce domain_has_ioreq_server()

From: Oleksandr Tyshchenko 

This patch introduces a helper the main purpose of which is to check
if a domain is using IOREQ server(s).

On Arm the current benefit is to avoid calling handle_io_completion()
(which implies iterating over all possible IOREQ servers anyway)
on every return in leave_hypervisor_to_guest() if there is no active
servers for the particular domain.
Also this helper will be used by one of the subsequent patches on Arm.

This involves adding an extra per-domain variable to store the count
of servers in use.

Signed-off-by: Oleksandr Tyshchenko 
CC: Julien Grall 

---
Please note, this is a split/cleanup/hardening of Julien's PoC:
"Add support for Guest IO forwarding to a device emulator"

Changes RFC -> V1:
- new patch

Changes V1 -> V2:
- update patch description
- guard helper with CONFIG_IOREQ_SERVER
- remove "hvm" prefix
- modify helper to just return d->arch.hvm.ioreq_server.nr_servers
- put suitable ASSERT()s
- use ASSERT(d->ioreq_server.server[id] ? !s : !!s) in set_ioreq_server()
- remove d->ioreq_server.nr_servers = 0 from hvm_ioreq_init()
---
  xen/arch/arm/traps.c| 15 +--
  xen/common/ioreq.c  |  7 ++-
  xen/include/xen/ioreq.h | 14 ++
  xen/include/xen/sched.h |  1 +
  4 files changed, 30 insertions(+), 7 deletions(-)

diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
index 507c095..a8f5fdf 100644
--- a/xen/arch/arm/traps.c
+++ b/xen/arch/arm/traps.c
@@ -2261,14 +2261,17 @@ static bool check_for_vcpu_work(void)
  struct vcpu *v = current;

  #ifdef CONFIG_IOREQ_SERVER
-bool handled;
+if ( domain_has_ioreq_server(v->domain) )
+{
+bool handled;

-local_irq_enable();
-handled = handle_io_completion(v);
-local_irq_disable();
+local_irq_enable();
+handled = handle_io_completion(v);
+local_irq_disable();

-if ( !handled )
-return true;
+if ( !handled )
+return true;
+}
  #endif

  if ( likely(!v->arch.need_flush_to_ram) )
diff --git a/xen/common/ioreq.c b/xen/common/ioreq.c
index bcd4961..a72bc0e 100644
--- a/xen/common/ioreq.c
+++ b/xen/common/ioreq.c
@@ -39,9 +39,14 @@ static void set_ioreq_server(struct domain *d, unsigned int 
id,
   struct ioreq_server *s)
  {
  ASSERT(id < MAX_NR_IOREQ_SERVERS);
-ASSERT(!s || !d->ioreq_server.server[id]);
+ASSERT(d->ioreq_server.server[id] ? !s : !!s);

That looks odd. How about ASSERT(!s ^ !d->ioreq_server.server[id])?


ok, looks like it will work.



   Paul


  d->ioreq_server.server[id] = s;
+
+if ( s )
+d->ioreq_server.nr_servers++;
+else
+d->ioreq_server.nr_servers--;
  }

  #define GET_IOREQ_SERVER(d, id) \
diff --git a/xen/include/xen/ioreq.h b/xen/include/xen/ioreq.h
index 7b03ab5..0679fef 100644
--- a/xen/include/xen/ioreq.h
+++ b/xen/include/xen/ioreq.h
@@ -55,6 +55,20 @@ struct ioreq_server {
  uint8_tbufioreq_handling;
  };

+#ifdef CONFIG_IOREQ_SERVER
+static inline bool domain_has_ioreq_server(const struct domain *d)
+{
+ASSERT((current->domain == d) || atomic_read(>pause_count));
+

This seems like an odd place to put such an assertion.


I might miss something or interpreted incorrectly but these asserts are 
the result of how I understood the review comment on previous version [1].


I will copy a comment here for the convenience:
"This is safe only when d == current->domain and it's not paused,
or when they're distinct and d is paused. Otherwise the result is
stale before the caller can inspect it. This wants documenting by
at least a comment, but perhaps better by suitable ASSERT()s."





+return d->ioreq_server.nr_servers;
+}
+#else
+static inline bool domain_has_ioreq_server(const struct domain *d)
+{
+return false;
+}
+#endif
+

Can this be any more compact? E.g.

return IS_ENABLED(CONFIG_IOREQ_SERVER) && d->ioreq_server.nr_servers;

?
I have got a compilation error this way (if CONFIG_IOREQ_SERVER is 
disabled):


...xen/4.14.0+gitAUTOINC+ee22110219-r0/git/xen/include/xen/ioreq.h:62:48: 
error: ‘const struct domain’ has no member named ‘ioreq_server’

 return IS_ENABLED(CONFIG_IOREQ_SERVER) && d->ioreq_server.nr_servers;
    ^
as domain's ioreq_server struct is guarded by CONFIG_IOREQ_SERVER as well.


[1] 
https://patchwork.kernel.org/project/xen-devel/patch/1599769330-17656-12-git-send-email-olekst...@gmail.com/#23618623


Thank you.

--

Re: [PATCH V2 08/23] xen/ioreq: Introduce ioreq_params to abstract accesses to arch.hvm.params

2020-11-10 Thread Oleksandr



On 20.10.20 13:41, Paul Durrant wrote:

Hi Paul

Sorry for the late response.



-Original Message-
From: Oleksandr Tyshchenko 
Sent: 15 October 2020 17:44
To: xen-devel@lists.xenproject.org
Cc: Oleksandr Tyshchenko ; Paul Durrant 
; Jan Beulich
; Andrew Cooper ; Roger Pau Monné
; Wei Liu ; Julien Grall ; 
Stefano Stabellini
; Julien Grall 
Subject: [PATCH V2 08/23] xen/ioreq: Introduce ioreq_params to abstract 
accesses to arch.hvm.params

From: Oleksandr Tyshchenko 

We don't want to move HVM params field out of *arch.hvm* in this particular
case as although it stores a few IOREQ params, it is not a (completely)
IOREQ stuff and is specific to the architecture. Instead, abstract
accesses by the proposed macro.

This is a follow up action to reduce layering violation in the common code.

Signed-off-by: Oleksandr Tyshchenko 
CC: Julien Grall 


Keeping the 'legacy' magic page code under an x86 ioreq.c would avoid the need 
for this patch.


In that case, yes, agree.


--
Regards,

Oleksandr Tyshchenko




[xen-unstable-smoke test] 156628: regressions - FAIL

2020-11-10 Thread osstest service owner
flight 156628 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/156628/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64   6 xen-buildfail REGR. vs. 156622

Tests which did not succeed, but are not blocking:
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-arm64-arm64-xl-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  16 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  e6e85b662be9eab96f4cfc58e9945580cce8b2bb
baseline version:
 xen  3059178798a23ba870ff86ff54d442a07e6651fc

Last test of basis   156622  2020-11-10 13:01:19 Z0 days
Testing same since   156628  2020-11-10 17:00:28 Z0 days1 attempts


People who touched revisions under test:
  Andrew Cooper 
  Jan Beulich 
  Juergen Gross 
  Julien Grall 

jobs:
 build-arm64-xsm  pass
 build-amd64  fail
 build-armhf  pass
 build-amd64-libvirt  blocked 
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  pass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64blocked 
 test-amd64-amd64-libvirt blocked 



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.


commit e6e85b662be9eab96f4cfc58e9945580cce8b2bb
Author: Jan Beulich 
Date:   Tue Nov 10 14:40:09 2020 +0100

x86/CPUID: also check leaf 7 max subleaf to be compatible

Just like is done for basic and extended major leaves.

Signed-off-by: Jan Beulich 
Acked-by: Andrew Cooper 

commit f5cfa09856732b1d78ff6a21ca3dc33a010da951
Author: Jan Beulich 
Date:   Tue Nov 10 14:39:30 2020 +0100

x86/CPUID: suppress IOMMU related hypervisor leaf data

Now that the IOMMU for guests can't be enabled "on demand" anymore,
there's also no reason to expose the related CPUID bit "just in case".

Signed-off-by: Jan Beulich 
Acked-by: Andrew Cooper 

commit db1a9fdd554cb1d8a7099af7925318fc06c6875b
Author: Jan Beulich 
Date:   Tue Nov 10 14:39:03 2020 +0100

x86/CPUID: don't use UB shift when library is built as 32-bit

At least the insn emulator test harness will continue to be buildable
(and ought to continue to be usable) also as a 32-bit binary. (Right now
the CPU policy test harness is, too, but there it may be less relevant
to keep it functional, just like e.g. we don't support fuzzing the insn
emulator in 32-bit mode.) Hence the library code needs to cope with
this.

Signed-off-by: Jan Beulich 
Acked-by: Andrew Cooper 

commit b5ad37f8e9284cc147218f7a5193d739ae7b956f
Author: Juergen Gross 
Date:   Tue Nov 10 14:37:15 2020 +0100

xen/evtchn: revert 52e1fc47abc3a0123

With the event channel lock no longer disabling interrupts commit
52e1fc47abc3a0123 ("evtchn/Flask: pre-allocate node on send path") can
be reverted again.

Signed-off-by: Juergen Gross 
Acked-by: Jan Beulich 

commit 5f2df45ead7c1195142f68b7923047a1e9479d54
Author: Juergen Gross 
Date:   Tue Nov 10 14:36:15 2020 +0100

xen/evtchn: rework per event channel lock

Currently the lock for a single event channel needs to be taken with
interrupts off, which causes deadlocks in some cases.

Rework the per event channel lock to be non-blocking for the case of
sending an event and removing the need for disabling interrupts for
taking the lock.

The lock is needed for avoiding races between event channel state
changes (creation, closing, binding) against normal operations (set
pending, [un]masking, priority changes).

Use a rwlock, but with some 

Re: [PATCH V2 04/23] xen/ioreq: Provide alias for the handle_mmio()

2020-11-10 Thread Oleksandr



On 20.10.20 13:38, Paul Durrant wrote:

Hi Jan, Paul

Sorry for the late response.


-Original Message-
From: Jan Beulich 
Sent: 20 October 2020 11:05
To: p...@xen.org
Cc: 'Oleksandr Tyshchenko' ; 
xen-devel@lists.xenproject.org; 'Oleksandr
Tyshchenko' ; 'Andrew Cooper' 
; 'Roger Pau
Monné' ; 'Wei Liu' ; 'Julien Grall' 
; 'Stefano
Stabellini' ; 'Julien Grall' 
Subject: Re: [PATCH V2 04/23] xen/ioreq: Provide alias for the handle_mmio()

On 20.10.2020 11:14, Paul Durrant wrote:

From: Xen-devel  On Behalf Of Oleksandr 
Tyshchenko
Sent: 15 October 2020 17:44

--- a/xen/include/asm-x86/hvm/ioreq.h
+++ b/xen/include/asm-x86/hvm/ioreq.h
@@ -181,6 +181,8 @@ static inline bool arch_hvm_ioreq_destroy(struct domain *d)
  #define IOREQ_STATUS_UNHANDLED   X86EMUL_UNHANDLEABLE
  #define IOREQ_STATUS_RETRY   X86EMUL_RETRY

+#define ioreq_complete_mmio   handle_mmio
+

A #define? Really? Can we not have a static inline?

I guess this would require further shuffling: handle_mmio() is
an inline function in hvm/emulate.h, and hvm/ioreq.h has no
need to include the former (and imo it also shouldn't have).


I see. I think we need an x86 ioreq.c anyway, to deal with the legacy use of 
magic pages, so it could be dealt with there instead.
I am afraid I don't entirely understand the required changes. Could you 
please clarify where the "inline(?)" ioreq_complete_mmio() should
live? I included hvm/emulate.h here not for the "handle_mmio()" reason 
only, but for "struct hvm_emulate_ctxt" also (see arch_io_completion()).



But, if we return x86 ioreq.c back I can move arch_io_completion() to it 
as well as "non-online" ioreq_complete_mmio().

This will avoid including hvm/emulate.h here. Or I missed something?

--
Regards,

Oleksandr Tyshchenko




[PATCH v1 05/10] stubs/xen-hw-stub: drop xenstore_store_pv_console_info stub

2020-11-10 Thread Alex Bennée
We should never build something that calls this without having it.

Signed-off-by: Alex Bennée 
Reviewed-by: Philippe Mathieu-Daudé 
Message-Id: <20201105175153.30489-13-alex.ben...@linaro.org>
Signed-off-by: Alex Bennée 
---
 stubs/xen-hw-stub.c | 4 
 1 file changed, 4 deletions(-)

diff --git a/stubs/xen-hw-stub.c b/stubs/xen-hw-stub.c
index 2ea8190921..15f3921a76 100644
--- a/stubs/xen-hw-stub.c
+++ b/stubs/xen-hw-stub.c
@@ -10,10 +10,6 @@
 #include "hw/xen/xen.h"
 #include "hw/xen/xen-x86.h"
 
-void xenstore_store_pv_console_info(int i, Chardev *chr)
-{
-}
-
 int xen_pci_slot_get_pirq(PCIDevice *pci_dev, int irq_num)
 {
 return -1;
-- 
2.20.1




[PATCH v1 04/10] include/hw/xen.h: drop superfluous struct

2020-11-10 Thread Alex Bennée
Chardev is already a typedef'ed struct.

Signed-off-by: Alex Bennée 
Reviewed-by: Philippe Mathieu-Daudé 
Message-Id: <20201105175153.30489-12-alex.ben...@linaro.org>
Signed-off-by: Alex Bennée 
---
 include/hw/xen/xen.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/include/hw/xen/xen.h b/include/hw/xen/xen.h
index 1406648ca5..0f9962b1c1 100644
--- a/include/hw/xen/xen.h
+++ b/include/hw/xen/xen.h
@@ -28,7 +28,7 @@ int xen_is_pirq_msi(uint32_t msi_data);
 
 qemu_irq *xen_interrupt_controller_init(void);
 
-void xenstore_store_pv_console_info(int i, struct Chardev *chr);
+void xenstore_store_pv_console_info(int i, Chardev *chr);
 
 void xen_register_framebuffer(struct MemoryRegion *mr);
 
-- 
2.20.1




[libvirt test] 156611: regressions - FAIL

2020-11-10 Thread osstest service owner
flight 156611 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/156611/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-libvirt   6 libvirt-buildfail REGR. vs. 151777
 build-i386-libvirt6 libvirt-buildfail REGR. vs. 151777
 build-arm64-libvirt   6 libvirt-buildfail REGR. vs. 151777
 build-armhf-libvirt   6 libvirt-buildfail REGR. vs. 151777

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt  1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-qcow2  1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked  n/a

version targeted for testing:
 libvirt  43ee7c6db119add9591357b643378d090769307b
baseline version:
 libvirt  2c846fa6bcc11929c9fb857a22430fb9945654ad

Last test of basis   151777  2020-07-10 04:19:19 Z  123 days
Failing since151818  2020-07-11 04:18:52 Z  122 days  117 attempts
Testing same since   156611  2020-11-10 04:18:54 Z0 days1 attempts


People who touched revisions under test:
  Adolfo Jayme Barrientos 
  Aleksandr Alekseev 
  Andika Triwidada 
  Andrea Bolognani 
  Balázs Meskó 
  Bastien Orivel 
  Bihong Yu 
  Binfeng Wu 
  Boris Fiuczynski 
  Brian Turek 
  Christian Ehrhardt 
  Christian Schoenebeck 
  Cole Robinson 
  Collin Walling 
  Cornelia Huck 
  Côme Borsoi 
  Daniel Henrique Barboza 
  Daniel Letai 
  Daniel P. Berrange 
  Daniel P. Berrangé 
  Erik Skultety 
  Fabian Freyer 
  Fangge Jin 
  Fedora Weblate Translation 
  Göran Uddeborg 
  Halil Pasic 
  Han Han 
  Hao Wang 
  Ian Wienand 
  Jamie Strandboge 
  Jamie Strandboge 
  Jean-Baptiste Holcroft 
  Jianan Gao 
  Jim Fehlig 
  Jin Yan 
  Jiri Denemark 
  Jonathon Jongsma 
  Julio Faracco 
  Ján Tomko 
  Kashyap Chamarthy 
  Kevin Locke 
  Laine Stump 
  Liao Pingfang 
  Lin Ma 
  Lin Ma 
  Marc Hartmayer 
  Marek Marczykowski-Górecki 
  Markus Schade 
  Martin Kletzander 
  Masayoshi Mizuma 
  Matt Coleman 
  Matt Coleman 
  Mauro Matteo Cascella 
  Michal Privoznik 
  Michał Smyk 
  Milo Casagrande 
  Neal Gompa 
  Nico Pache 
  Nikolay Shirokovskiy 
  Olesya Gerasimenko 
  Orion Poplawski 
  Patrick Magauran 
  Paulo de Rezende Pinatti 
  Pavel Hrdina 
  Peter Krempa 
  Pino Toscano 
  Pino Toscano 
  Piotr Drąg 
  Prathamesh Chavan 
  Roman Bogorodskiy 
  Roman Bolshakov 
  Ryan Schmidt 
  Sam Hartman 
  Scott Shambarger 
  Sebastian Mitterle 
  Simon Gaiser 
  Stefan Bader 
  Stefan Berger 
  Szymon Scholz 
  Thomas Huth 
  Tim Wiederhake 
  Tomáš Golembiovský 
  Wang Xin 
  Weblate 
  Yang Hang 
  Yanqiu Zhang 
  Yi Li 
  Yi Wang 
  Yuri Chornoivan 
  Zheng Chuan 
  zhenwei pi 
  Zhenyu Zheng 

jobs:
 build-amd64-xsm  pass
 build-arm64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-arm64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  fail
 build-arm64-libvirt  fail
 build-armhf-libvirt  fail
 build-i386-libvirt   fail
 build-amd64-pvopspass
 build-arm64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm   blocked 
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsmblocked 
 test-amd64-amd64-libvirt-xsm 

Xen Security Advisory 351 v1 - Information leak via power sidechannel

2020-11-10 Thread Xen . org security team
(Copy of advisory)

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Xen Security Advisory XSA-351

 Information leak via power sidechannel

ISSUE DESCRIPTION
=

Researchers have demonstrated using software power/energy monitoring
interfaces to create covert channels, and infer the operations/data used
by other contexts within the system.

Access to these interfaces should be restricted to privileged software,
but it was found that Xen doesn't restrict access suitably, and the
interfaces are accessible to all guests.

For more information, see:
  https://platypusattack.com
  
https://www.intel.com/content/www/us/en/security-center/advisory/intel-sa-00389.html

IMPACT
==

An unprivileged guest administrator can sample platform power/energy
data.  This may be used to infer the operations/data used by other
contexts within the system.

The research demonstrates using this sidechannel to leak the AES keys
used elsewhere in the system.

VULNERABLE SYSTEMS
==

Power/energy monitoring interfaces are platform and architecture
specific.  Consult your hardware vendor to ascertain what power feedback
interfaces are available.

For ARM systems, all versions of Xen are vulnerable.  The fix restricts
access to the AMU (Activity Monitors Unit) interface, introduced in
Armv8.4.

For x86 systems, Xen 4.14 and earlier are vulnerable - master is not
vulnerable, as these issues have been addressed in a more general
fashion.

The x86 fixes restrict access to:
 * Intel RAPL interface, introduced in SandyBridge CPUs.
 * Intel platform energy interface.
 * Intel perf_ctl interface, introduced in Pentium 4 CPUs and also
   implemented by other vendors.
 * AMD RAPL interface, introduced in Ryzen/EPYC CPUs.
 * AMD compute unit energy interface, present in Fam15/16 CPUs.

MITIGATION
==

There are no mitigations available.

RESOLUTION
==

Applying the appropriate attached patch resolves this issue.

Note that patches for released versions are generally prepared to
apply to the stable branches, and may not apply cleanly to the most
recent release tarball.  Downstreams are encouraged to update to the
tip of the stable branch before applying these patches.

xsa351-arm.patch Xen unstable - 4.10.x [ARM]
xsa351-x86-4.14-?.patch  Xen 4.14.x[x86]
xsa351-x86-4.13-?.patch  Xen 4.13.x[x86]
xsa351-x86-4.12-?.patch  Xen 4.12.x[x86]
xsa351-x86-4.11-?.patch  Xen 4.11.x - 4.10.x   [x86]

$ sha256sum xsa351*
cad287981a870f13484834fa2364ffee68178517e906f55d2889304a4a9eae06  xsa351.meta
70ebd0e93af240af2680374dcfd8ff4a5dd3eefccf670f1cb9b546d763d6a554  
xsa351-arm.patch
49b52a1366912a29e184e3014a9f1f579e8a0dd8a36f01d38d995d2c8ed81928  
xsa351-arm-4.11.patch
2e7b7c2b98625d70c8b10047a9f668372f3ccede167344dedb712312606acbca  
xsa351-x86-4.11-1.patch
ab9e2cb7d5e3e0c3a916f006c697495f4f01146e09df60ece59ce0a8f7aa5ed0  
xsa351-x86-4.11-2.patch
bb68f6e6905bc1566156cafab058cbaf02a17c197385c33a83b7f73885913c1c  
xsa351-x86-4.12-1.patch
53f464269f59498f8a9a614f10a47cfb1d81c666f0d684346e28005015de962c  
xsa351-x86-4.12-2.patch
67a29d66230faafd9a8047ac80ec18130b5659e80a38c3a412cb2be6d3288a8f  
xsa351-x86-4.13-1.patch
f7d8717dec33ee7484b36490402d113f1e7e168e7541bcf193fef620df299f08  
xsa351-x86-4.13-2.patch
7d4fbe11a766226d7f1b93c5bf34664d8855deee09d1feebc76f11e49f2aa9c9  
xsa351-x86-4.14-1.patch
41df825deafe3ef28e8594ec956033689af69f84a4a6dd92f97d1071e925203d  
xsa351-x86-4.14-2.patch
$

NOTE REGARDING LACK OF EMBARGO
==

Despite an attempt to organise predisclosure, the discoverers ultimately
did not authorise a predisclosure.
-BEGIN PGP SIGNATURE-

iQFABAEBCAAqFiEEI+MiLBRfRHX6gGCng/4UyVfoK9kFAl+q1WwMHHBncEB4ZW4u
b3JnAAoJEIP+FMlX6CvZANkH+wf8pft4t9KoC9HFxd96DfCjZ+FQnD0hMp+890cY
ztNJM4+o+SBP2ytEMZLIoN1oJeTSQqyNgQh2sXNm7/WpseklOTR6s8zw4LWATEfz
rqF8G2xIN8ka7AAqAwOzkzj6qlxuWbiXKm4ENd5ocRxVvF1A2PYyEX88uCPgmupg
dqfufhYQF7hrz8VKDRDYtLsMrRaIFCWqGdOdQfVF64pHGHLvGZkANGN8yva8mBfC
uavwvX+O3CdVMENS4AA3TNo6p2nnWp1iQJCiBwLGCRbTQaRtRucV4Q/eSLC3pHLp
NO26OxieT4tLJN7Ox4ex43KZIsyweZSaUl18rfg0J8MB3FM=
=/6Fo
-END PGP SIGNATURE-


xsa351.meta
Description: Binary data


xsa351-arm.patch
Description: Binary data


xsa351-arm-4.11.patch
Description: Binary data


xsa351-x86-4.11-1.patch
Description: Binary data


xsa351-x86-4.11-2.patch
Description: Binary data


xsa351-x86-4.12-1.patch
Description: Binary data


xsa351-x86-4.12-2.patch
Description: Binary data


xsa351-x86-4.13-1.patch
Description: Binary data


xsa351-x86-4.13-2.patch
Description: Binary data


xsa351-x86-4.14-1.patch
Description: Binary data


xsa351-x86-4.14-2.patch
Description: Binary data


[PATCH] docs: fix documentation about default scheduler

2020-11-10 Thread Roger Pau Monne
Fix the command line document to account for the default scheduler not
being credit anymore likely, and the fact that it's selectable from
Kconfig and thus different builds could end up with different default
schedulers.

Fixes: dafd936dddbd ('Make credit2 the default scheduler')
Signed-off-by: Roger Pau Monné 
---
Changes since v1:
 - Point that the default scheduler is being selected by Kconfig,
   don't mention the default Kconfig selection.
---
 docs/misc/xen-command-line.pandoc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/misc/xen-command-line.pandoc 
b/docs/misc/xen-command-line.pandoc
index 4ae9391fcd..eb1db25f92 100644
--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -1876,7 +1876,7 @@ with read and write permissions.
 ### sched
 > `= credit | credit2 | arinc653 | rtds | null`
 
-> Default: `sched=credit`
+> Default: selectable via Kconfig.  Depends on enabled schedulers.
 
 Choose the default scheduler.
 
-- 
2.29.0




Xen Security Advisory 351 v1 - Information leak via power sidechannel

2020-11-10 Thread Xen . org security team
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Xen Security Advisory XSA-351

 Information leak via power sidechannel

ISSUE DESCRIPTION
=

Researchers have demonstrated using software power/energy monitoring
interfaces to create covert channels, and infer the operations/data used
by other contexts within the system.

Access to these interfaces should be restricted to privileged software,
but it was found that Xen doesn't restrict access suitably, and the
interfaces are accessible to all guests.

For more information, see:
  https://platypusattack.com
  
https://www.intel.com/content/www/us/en/security-center/advisory/intel-sa-00389.html

IMPACT
==

An unprivileged guest administrator can sample platform power/energy
data.  This may be used to infer the operations/data used by other
contexts within the system.

The research demonstrates using this sidechannel to leak the AES keys
used elsewhere in the system.

VULNERABLE SYSTEMS
==

Power/energy monitoring interfaces are platform and architecture
specific.  Consult your hardware vendor to ascertain what power feedback
interfaces are available.

For ARM systems, all versions of Xen are vulnerable.  The fix restricts
access to the AMU (Activity Monitors Unit) interface, introduced in
Armv8.4.

For x86 systems, Xen 4.14 and earlier are vulnerable - master is not
vulnerable, as these issues have been addressed in a more general
fashion.

The x86 fixes restrict access to:
 * Intel RAPL interface, introduced in SandyBridge CPUs.
 * Intel platform energy interface.
 * Intel perf_ctl interface, introduced in Pentium 4 CPUs and also
   implemented by other vendors.
 * AMD RAPL interface, introduced in Ryzen/EPYC CPUs.
 * AMD compute unit energy interface, present in Fam15/16 CPUs.

MITIGATION
==

There are no mitigations available.

RESOLUTION
==

Applying the appropriate attached patch resolves this issue.

Note that patches for released versions are generally prepared to
apply to the stable branches, and may not apply cleanly to the most
recent release tarball.  Downstreams are encouraged to update to the
tip of the stable branch before applying these patches.

xsa351-arm.patch Xen unstable - 4.10.x [ARM]
xsa351-x86-4.14-?.patch  Xen 4.14.x[x86]
xsa351-x86-4.13-?.patch  Xen 4.13.x[x86]
xsa351-x86-4.12-?.patch  Xen 4.12.x[x86]
xsa351-x86-4.11-?.patch  Xen 4.11.x - 4.10.x   [x86]

$ sha256sum xsa351*
cad287981a870f13484834fa2364ffee68178517e906f55d2889304a4a9eae06  xsa351.meta
70ebd0e93af240af2680374dcfd8ff4a5dd3eefccf670f1cb9b546d763d6a554  
xsa351-arm.patch
49b52a1366912a29e184e3014a9f1f579e8a0dd8a36f01d38d995d2c8ed81928  
xsa351-arm-4.11.patch
2e7b7c2b98625d70c8b10047a9f668372f3ccede167344dedb712312606acbca  
xsa351-x86-4.11-1.patch
ab9e2cb7d5e3e0c3a916f006c697495f4f01146e09df60ece59ce0a8f7aa5ed0  
xsa351-x86-4.11-2.patch
bb68f6e6905bc1566156cafab058cbaf02a17c197385c33a83b7f73885913c1c  
xsa351-x86-4.12-1.patch
53f464269f59498f8a9a614f10a47cfb1d81c666f0d684346e28005015de962c  
xsa351-x86-4.12-2.patch
67a29d66230faafd9a8047ac80ec18130b5659e80a38c3a412cb2be6d3288a8f  
xsa351-x86-4.13-1.patch
f7d8717dec33ee7484b36490402d113f1e7e168e7541bcf193fef620df299f08  
xsa351-x86-4.13-2.patch
7d4fbe11a766226d7f1b93c5bf34664d8855deee09d1feebc76f11e49f2aa9c9  
xsa351-x86-4.14-1.patch
41df825deafe3ef28e8594ec956033689af69f84a4a6dd92f97d1071e925203d  
xsa351-x86-4.14-2.patch
$

NOTE REGARDING LACK OF EMBARGO
==

Despite an attempt to organise predisclosure, the discoverers ultimately
did not authorise a predisclosure.
-BEGIN PGP SIGNATURE-

iQFABAEBCAAqFiEEI+MiLBRfRHX6gGCng/4UyVfoK9kFAl+q1WwMHHBncEB4ZW4u
b3JnAAoJEIP+FMlX6CvZANkH+wf8pft4t9KoC9HFxd96DfCjZ+FQnD0hMp+890cY
ztNJM4+o+SBP2ytEMZLIoN1oJeTSQqyNgQh2sXNm7/WpseklOTR6s8zw4LWATEfz
rqF8G2xIN8ka7AAqAwOzkzj6qlxuWbiXKm4ENd5ocRxVvF1A2PYyEX88uCPgmupg
dqfufhYQF7hrz8VKDRDYtLsMrRaIFCWqGdOdQfVF64pHGHLvGZkANGN8yva8mBfC
uavwvX+O3CdVMENS4AA3TNo6p2nnWp1iQJCiBwLGCRbTQaRtRucV4Q/eSLC3pHLp
NO26OxieT4tLJN7Ox4ex43KZIsyweZSaUl18rfg0J8MB3FM=
=/6Fo
-END PGP SIGNATURE-


xsa351.meta
Description: Binary data


xsa351-arm.patch
Description: Binary data


xsa351-arm-4.11.patch
Description: Binary data


xsa351-x86-4.11-1.patch
Description: Binary data


xsa351-x86-4.11-2.patch
Description: Binary data


xsa351-x86-4.12-1.patch
Description: Binary data


xsa351-x86-4.12-2.patch
Description: Binary data


xsa351-x86-4.13-1.patch
Description: Binary data


xsa351-x86-4.13-2.patch
Description: Binary data


xsa351-x86-4.14-1.patch
Description: Binary data


xsa351-x86-4.14-2.patch
Description: Binary data


Re: Duplicated ABI entries - Was: Re: [PATCH v2 20/39] docs: ABI: testing: make the files compatible with ReST output

2020-11-10 Thread Randy Dunlap
On 11/9/20 11:26 PM, Mauro Carvalho Chehab wrote:
> Hi Jonathan,
> 
> Let's view ABI from the PoV of a system admin that doesn't know
> yet about a certain ABI symbol.
> 
> He'll try to seek for the symbol, more likely using the HTML 
> documentation. Only very senior system admins might try to take
> a look at the Kernel.

FWIW, I think that the likely search methods are $search_engine
and 'grep'.

Have a good few days off.

-- 
~Randy




[PATCH v2 11/24] libxl: add libxl_device_pci_assignable_list_free()...

2020-11-10 Thread Paul Durrant
From: Paul Durrant 

... to be used by callers of libxl_device_pci_assignable_list().

Currently there is no API for callers of libxl_device_pci_assignable_list()
to free the list. The xl function pciassignable_list() calls
libxl_device_pci_dispose() on each element of the returned list, but
libxl_pci_assignable() in libxl_pci.c does not. Neither does the implementation
of libxl_device_pci_assignable_list() call libxl_device_pci_init().

This patch adds the new API function, makes sure it is used everywhere and
also modifies libxl_device_pci_assignable_list() to initialize list
entries rather than just zeroing them.

Signed-off-by: Paul Durrant 
---
Cc: Ian Jackson 
Cc: Wei Liu 
Cc: Christian Lindig 
Cc: David Scott 
Cc: Anthony PERARD 
---
 tools/include/libxl.h|  7 +++
 tools/libs/light/libxl_pci.c | 14 --
 tools/ocaml/libs/xl/xenlight_stubs.c |  3 +--
 tools/xl/xl_pci.c|  3 +--
 4 files changed, 21 insertions(+), 6 deletions(-)

diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index ee52d3cf7e7e..8225809d94a8 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -457,6 +457,12 @@
  */
 #define LIBXL_HAVE_DEVICE_PCI_LIST_FREE 1
 
+/*
+ * LIBXL_HAVE_DEVICE_PCI_ASSIGNABLE_LIST_FREE indicates that the
+ * libxl_device_pci_assignable_list_free() function is defined.
+ */
+#define LIBXL_HAVE_DEVICE_PCI_ASSIGNABLE_LIST_FREE 1
+
 /*
  * libxl ABI compatibility
  *
@@ -2369,6 +2375,7 @@ int libxl_device_events_handler(libxl_ctx *ctx,
 int libxl_device_pci_assignable_add(libxl_ctx *ctx, libxl_device_pci *pci, int 
rebind);
 int libxl_device_pci_assignable_remove(libxl_ctx *ctx, libxl_device_pci *pci, 
int rebind);
 libxl_device_pci *libxl_device_pci_assignable_list(libxl_ctx *ctx, int *num);
+void libxl_device_pci_assignable_list_free(libxl_device_pci *list, int num);
 
 /* CPUID handling */
 int libxl_cpuid_parse_config(libxl_cpuid_policy_list *cpuid, const char* str);
diff --git a/tools/libs/light/libxl_pci.c b/tools/libs/light/libxl_pci.c
index a69cba793583..b87e121c4d5c 100644
--- a/tools/libs/light/libxl_pci.c
+++ b/tools/libs/light/libxl_pci.c
@@ -438,7 +438,7 @@ libxl_device_pci 
*libxl_device_pci_assignable_list(libxl_ctx *ctx, int *num)
 pcis = new;
 new = pcis + *num;
 
-memset(new, 0, sizeof(*new));
+libxl_device_pci_init(new);
 pci_struct_fill(new, dom, bus, dev, func, 0);
 
 if (pci_info_xs_read(gc, new, "domid")) /* already assigned */
@@ -453,6 +453,16 @@ out:
 return pcis;
 }
 
+void libxl_device_pci_assignable_list_free(libxl_device_pci *list, int num)
+{
+int i;
+
+for (i = 0; i < num; i++)
+libxl_device_pci_dispose([i]);
+
+free(list);
+}
+
 /* Unbind device from its current driver, if any.  If driver_path is non-NULL,
  * store the path to the original driver in it. */
 static int sysfs_dev_unbind(libxl__gc *gc, libxl_device_pci *pci,
@@ -1471,7 +1481,7 @@ static int libxl_pci_assignable(libxl_ctx *ctx, 
libxl_device_pci *pci)
 pcis[i].func == pci->func)
 break;
 }
-free(pcis);
+libxl_device_pci_assignable_list_free(pcis, num);
 return i != num;
 }
 
diff --git a/tools/ocaml/libs/xl/xenlight_stubs.c 
b/tools/ocaml/libs/xl/xenlight_stubs.c
index 1181971da4e7..352a00134d70 100644
--- a/tools/ocaml/libs/xl/xenlight_stubs.c
+++ b/tools/ocaml/libs/xl/xenlight_stubs.c
@@ -894,9 +894,8 @@ value stub_xl_device_pci_assignable_list(value ctx)
Field(list, 1) = temp;
temp = list;
Store_field(list, 0, Val_device_pci(_list[i]));
-   libxl_device_pci_dispose(_list[i]);
}
-   free(c_list);
+   libxl_device_pci_assignable_list_free(c_list, nb);
 
CAMLreturn(list);
 }
diff --git a/tools/xl/xl_pci.c b/tools/xl/xl_pci.c
index 7c0f102ac7b7..f71498cbb570 100644
--- a/tools/xl/xl_pci.c
+++ b/tools/xl/xl_pci.c
@@ -164,9 +164,8 @@ static void pciassignable_list(void)
 for (i = 0; i < num; i++) {
 printf("%04x:%02x:%02x.%01x\n",
pcis[i].domain, pcis[i].bus, pcis[i].dev, pcis[i].func);
-libxl_device_pci_dispose([i]);
 }
-free(pcis);
+libxl_device_pci_assignable_list_free(pcis, num);
 }
 
 int main_pciassignable_list(int argc, char **argv)
-- 
2.20.1




[PATCH v2 24/24] xl / libxl: support 'xl pci-attach/detach' by name

2020-11-10 Thread Paul Durrant
From: Paul Durrant 

This patch adds a 'name' field into the idl for 'libxl_device_pci' and
libxlu_pci_parse_spec_string() is modified to parse the new 'name'
parameter of PCI_SPEC_STRING detailed in the updated documention in
xl-pci-configuration(5).

If the 'name' field is non-NULL then both libxl_device_pci_add() and
libxl_device_pci_remove() will use it to look up the device BDF in
the list of assignable devices.

Signed-off-by: Paul Durrant 
---
Cc: Ian Jackson 
Cc: Wei Liu 
Cc: Anthony PERARD 
---
 tools/include/libxl.h|  6 +++
 tools/libs/light/libxl_pci.c | 67 +---
 tools/libs/light/libxl_types.idl |  1 +
 tools/libs/util/libxlu_pci.c |  7 +++-
 4 files changed, 75 insertions(+), 6 deletions(-)

diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index 4025d3a3d437..5b55a2015533 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -484,6 +484,12 @@
  */
 #define LIBXL_HAVE_PCI_ASSIGNABLE_NAME 1
 
+/*
+ * LIBXL_HAVE_DEVICE_PCI_NAME indicates that the 'name' field of
+ * libxl_device_pci is defined.
+ */
+#define LIBXL_HAVE_DEVICE_PCI_NAME 1
+
 /*
  * libxl ABI compatibility
  *
diff --git a/tools/libs/light/libxl_pci.c b/tools/libs/light/libxl_pci.c
index a789ade036fa..8dfbcf8d390d 100644
--- a/tools/libs/light/libxl_pci.c
+++ b/tools/libs/light/libxl_pci.c
@@ -60,6 +60,10 @@ static void libxl_create_pci_backend_device(libxl__gc *gc,
 int num,
 const libxl_device_pci *pci)
 {
+if (pci->name) {
+flexarray_append(back, GCSPRINTF("name-%d", num));
+flexarray_append(back, GCSPRINTF("%s", pci->name));
+}
 flexarray_append(back, GCSPRINTF("key-%d", num));
 flexarray_append(back, GCSPRINTF(PCI_BDF, pci->bdf.domain, pci->bdf.bus, 
pci->bdf.dev, pci->bdf.func));
 flexarray_append(back, GCSPRINTF("dev-%d", num));
@@ -252,6 +256,7 @@ retry_transaction:
 
 retry_transaction2:
 t = xs_transaction_start(ctx->xsh);
+xs_rm(ctx->xsh, t, GCSPRINTF("%s/name-%d", be_path, i));
 xs_rm(ctx->xsh, t, GCSPRINTF("%s/state-%d", be_path, i));
 xs_rm(ctx->xsh, t, GCSPRINTF("%s/key-%d", be_path, i));
 xs_rm(ctx->xsh, t, GCSPRINTF("%s/dev-%d", be_path, i));
@@ -290,6 +295,12 @@ retry_transaction2:
 xs_write(ctx->xsh, t, GCSPRINTF("%s/vdevfn-%d", be_path, j - 1), 
tmp, strlen(tmp));
 xs_rm(ctx->xsh, t, tmppath);
 }
+tmppath = GCSPRINTF("%s/name-%d", be_path, j);
+tmp = libxl__xs_read(gc, t, tmppath);
+if (tmp) {
+xs_write(ctx->xsh, t, GCSPRINTF("%s/name-%d", be_path, j - 1), 
tmp, strlen(tmp));
+xs_rm(ctx->xsh, t, tmppath);
+}
 }
 if (!xs_transaction_end(ctx->xsh, t, 0))
 if (errno == EAGAIN)
@@ -1587,6 +1598,23 @@ void libxl__device_pci_add(libxl__egc *egc, uint32_t 
domid,
 pas->starting = starting;
 pas->callback = device_pci_add_stubdom_done;
 
+if (pci->name) {
+libxl_pci_bdf *pcibdf =
+libxl_device_pci_assignable_name2bdf(CTX, pci->name);
+
+if (!pcibdf) {
+rc = ERROR_FAIL;
+goto out;
+}
+
+LOGD(DETAIL, domid, "'%s' -> %04x:%02x:%02x.%u", pci->name,
+ pcibdf->domain, pcibdf->bus, pcibdf->dev, pcibdf->func);
+
+libxl_pci_bdf_copy(CTX, >bdf, pcibdf);
+libxl_pci_bdf_dispose(pcibdf);
+free(pcibdf);
+}
+
 if (libxl__domain_type(gc, domid) == LIBXL_DOMAIN_TYPE_HVM) {
 rc = xc_test_assign_device(ctx->xch, domid,
pci_encode_bdf(>bdf));
@@ -1735,11 +1763,19 @@ static void device_pci_add_done(libxl__egc *egc,
 libxl_device_pci *pci = >pci;
 
 if (rc) {
-LOGD(ERROR, domid,
- "libxl__device_pci_add  failed for "
- "PCI device %x:%x:%x.%x (rc %d)",
- pci->bdf.domain, pci->bdf.bus, pci->bdf.dev, pci->bdf.func,
- rc);
+if (pci->name) {
+LOGD(ERROR, domid,
+ "libxl__device_pci_add failed for "
+ "PCI device '%s' (rc %d)",
+ pci->name,
+ rc);
+} else {
+LOGD(ERROR, domid,
+ "libxl__device_pci_add failed for "
+ "PCI device %x:%x:%x.%x (rc %d)",
+ pci->bdf.domain, pci->bdf.bus, pci->bdf.dev, pci->bdf.func,
+ rc);
+}
 pci_info_xs_remove(gc, >bdf, "domid");
 }
 libxl_device_pci_dispose(pci);
@@ -2256,6 +2292,23 @@ static void libxl__device_pci_remove_common(libxl__egc 
*egc,
 libxl__ev_time_init(>timeout);
 libxl__ev_time_init(>retry_timer);
 
+if (pci->name) {
+libxl_pci_bdf *pcibdf =
+libxl_device_pci_assignable_name2bdf(CTX, pci->name);
+
+if (!pcibdf) {
+rc = ERROR_FAIL;
+goto out;
+}
+
+LOGD(DETAIL, domid, "'%s' -> %04x:%02x:%02x.%u", 

[PATCH v2 18/24] libxl: introduce 'libxl_pci_bdf' in the idl...

2020-11-10 Thread Paul Durrant
From: Paul Durrant 

... and use in 'libxl_device_pci'

This patch is preparatory work for restricting the type passed to functions
that only require BDF information, rather than passing a 'libxl_device_pci'
structure which is only partially filled. In this patch only the minimal
mechanical changes necessary to deal with the structural changes are made.
Subsequent patches will adjust the code to make better use of the new type.

Signed-off-by: Paul Durrant 
---
Cc: George Dunlap 
Cc: Nick Rosbrook 
Cc: Ian Jackson 
Cc: Wei Liu 
Cc: Anthony PERARD 
---
 tools/golang/xenlight/helpers.gen.go |  77 ++
 tools/golang/xenlight/types.gen.go   |   8 +-
 tools/include/libxl.h|   6 ++
 tools/libs/light/libxl_dm.c  |   8 +-
 tools/libs/light/libxl_internal.h|   3 +-
 tools/libs/light/libxl_pci.c | 148 +--
 tools/libs/light/libxl_types.idl |  16 +--
 tools/libs/util/libxlu_pci.c |   8 +-
 tools/xl/xl_pci.c|   6 +-
 tools/xl/xl_sxp.c|   4 +-
 10 files changed, 167 insertions(+), 117 deletions(-)

diff --git a/tools/golang/xenlight/helpers.gen.go 
b/tools/golang/xenlight/helpers.gen.go
index c8605994e7fe..b7230f693c69 100644
--- a/tools/golang/xenlight/helpers.gen.go
+++ b/tools/golang/xenlight/helpers.gen.go
@@ -1999,6 +1999,41 @@ xc.colo_checkpoint_port = 
C.CString(x.ColoCheckpointPort)}
  return nil
  }
 
+// NewPciBdf returns an instance of PciBdf initialized with defaults.
+func NewPciBdf() (*PciBdf, error) {
+var (
+x PciBdf
+xc C.libxl_pci_bdf)
+
+C.libxl_pci_bdf_init()
+defer C.libxl_pci_bdf_dispose()
+
+if err := x.fromC(); err != nil {
+return nil, err }
+
+return , nil}
+
+func (x *PciBdf) fromC(xc *C.libxl_pci_bdf) error {
+ x.Func = byte(xc._func)
+x.Dev = byte(xc.dev)
+x.Bus = byte(xc.bus)
+x.Domain = int(xc.domain)
+
+ return nil}
+
+func (x *PciBdf) toC(xc *C.libxl_pci_bdf) (err error){defer func(){
+if err != nil{
+C.libxl_pci_bdf_dispose(xc)}
+}()
+
+xc._func = C.uint8_t(x.Func)
+xc.dev = C.uint8_t(x.Dev)
+xc.bus = C.uint8_t(x.Bus)
+xc.domain = C.int(x.Domain)
+
+ return nil
+ }
+
 // NewDevicePci returns an instance of DevicePci initialized with defaults.
 func NewDevicePci() (*DevicePci, error) {
 var (
@@ -2014,10 +2049,9 @@ return nil, err }
 return , nil}
 
 func (x *DevicePci) fromC(xc *C.libxl_device_pci) error {
- x.Func = byte(xc._func)
-x.Dev = byte(xc.dev)
-x.Bus = byte(xc.bus)
-x.Domain = int(xc.domain)
+ if err := x.Bdf.fromC();err != nil {
+return fmt.Errorf("converting field Bdf: %v", err)
+}
 x.Vdevfn = uint32(xc.vdevfn)
 x.VfuncMask = uint32(xc.vfunc_mask)
 x.Msitranslate = bool(xc.msitranslate)
@@ -2033,10 +2067,9 @@ if err != nil{
 C.libxl_device_pci_dispose(xc)}
 }()
 
-xc._func = C.uint8_t(x.Func)
-xc.dev = C.uint8_t(x.Dev)
-xc.bus = C.uint8_t(x.Bus)
-xc.domain = C.int(x.Domain)
+if err := x.Bdf.toC(); err != nil {
+return fmt.Errorf("converting field Bdf: %v", err)
+}
 xc.vdevfn = C.uint32_t(x.Vdevfn)
 xc.vfunc_mask = C.uint32_t(x.VfuncMask)
 xc.msitranslate = C.bool(x.Msitranslate)
@@ -2766,13 +2799,13 @@ if err := x.Nics[i].fromC(); err != nil {
 return fmt.Errorf("converting field Nics: %v", err) }
 }
 }
-x.Pcidevs = nil
-if n := int(xc.num_pcidevs); n > 0 {
-cPcidevs := (*[1<<28]C.libxl_device_pci)(unsafe.Pointer(xc.pcidevs))[:n:n]
-x.Pcidevs = make([]DevicePci, n)
-for i, v := range cPcidevs {
-if err := x.Pcidevs[i].fromC(); err != nil {
-return fmt.Errorf("converting field Pcidevs: %v", err) }
+x.Pcis = nil
+if n := int(xc.num_pcis); n > 0 {
+cPcis := (*[1<<28]C.libxl_device_pci)(unsafe.Pointer(xc.pcis))[:n:n]
+x.Pcis = make([]DevicePci, n)
+for i, v := range cPcis {
+if err := x.Pcis[i].fromC(); err != nil {
+return fmt.Errorf("converting field Pcis: %v", err) }
 }
 }
 x.Rdms = nil
@@ -2922,13 +2955,13 @@ return fmt.Errorf("converting field Nics: %v", err)
 }
 }
 }
-if numPcidevs := len(x.Pcidevs); numPcidevs > 0 {
-xc.pcidevs = 
(*C.libxl_device_pci)(C.malloc(C.ulong(numPcidevs)*C.sizeof_libxl_device_pci))
-xc.num_pcidevs = C.int(numPcidevs)
-cPcidevs := 
(*[1<<28]C.libxl_device_pci)(unsafe.Pointer(xc.pcidevs))[:numPcidevs:numPcidevs]
-for i,v := range x.Pcidevs {
-if err := v.toC([i]); err != nil {
-return fmt.Errorf("converting field Pcidevs: %v", err)
+if numPcis := len(x.Pcis); numPcis > 0 {
+xc.pcis = 
(*C.libxl_device_pci)(C.malloc(C.ulong(numPcis)*C.sizeof_libxl_device_pci))
+xc.num_pcis = C.int(numPcis)
+cPcis := 
(*[1<<28]C.libxl_device_pci)(unsafe.Pointer(xc.pcis))[:numPcis:numPcis]
+for i,v := range x.Pcis {
+if err := v.toC([i]); err != nil {
+return fmt.Errorf("converting field Pcis: %v", err)
 }
 }
 }
diff --git a/tools/golang/xenlight/types.gen.go 
b/tools/golang/xenlight/types.gen.go
index b4c5df0f2c5c..bc62ae8ce9d1 100644
--- a/tools/golang/xenlight/types.gen.go
+++ b/tools/golang/xenlight/types.gen.go
@@ -707,11 +707,15 @@ ColoCheckpointHost string
 ColoCheckpointPort string
 }
 
-type DevicePci struct {
+type PciBdf struct {
 Func 

[PATCH v2 20/24] libxl: modify libxl_device_pci_assignable_add/remove/list/list_free()...

2020-11-10 Thread Paul Durrant
From: Paul Durrant 

... to use 'libxl_pci_bdf' rather than 'libxl_device_pci'.

This patch modifies the API and callers accordingly. It also modifies
several internal functions in libxl_pci.c that support the API to also use
'libxl_pci_bdf'.

NOTE: The OCaml bindings are adjusted to contain the interface change. It
  should therefore not affect compatibility with OCaml-based utilities.

Signed-off-by: Paul Durrant 
Acked-by: Christian Lindig 
---
Cc: Ian Jackson 
Cc: Wei Liu 
Cc: David Scott 
Cc: Anthony PERARD 
---
 tools/include/libxl.h|  15 +-
 tools/libs/light/libxl_pci.c | 215 +++
 tools/ocaml/libs/xl/xenlight_stubs.c |  15 +-
 tools/xl/xl_pci.c|  32 ++--
 4 files changed, 157 insertions(+), 120 deletions(-)

diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index 5edacccbd1da..5703fdf367c5 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -469,6 +469,13 @@
  */
 #define LIBXL_HAVE_PCI_BDF 1
 
+/*
+ * LIBXL_HAVE_PCI_ASSIGNABLE_BDF indicates that the
+ * libxl_device_pci_assignable_add/remove/list/list_free() functions all
+ * use the 'libxl_pci_bdf' type rather than 'libxl_device_pci' type.
+ */
+#define LIBXL_HAVE_PCI_ASSIGNABLE_BDF 1
+
 /*
  * libxl ABI compatibility
  *
@@ -2378,10 +2385,10 @@ int libxl_device_events_handler(libxl_ctx *ctx,
  * added or is not bound, the functions will emit a warning but return
  * SUCCESS.
  */
-int libxl_device_pci_assignable_add(libxl_ctx *ctx, libxl_device_pci *pci, int 
rebind);
-int libxl_device_pci_assignable_remove(libxl_ctx *ctx, libxl_device_pci *pci, 
int rebind);
-libxl_device_pci *libxl_device_pci_assignable_list(libxl_ctx *ctx, int *num);
-void libxl_device_pci_assignable_list_free(libxl_device_pci *list, int num);
+int libxl_device_pci_assignable_add(libxl_ctx *ctx, libxl_pci_bdf *pcibdf, int 
rebind);
+int libxl_device_pci_assignable_remove(libxl_ctx *ctx, libxl_pci_bdf *pcibdf, 
int rebind);
+libxl_pci_bdf *libxl_device_pci_assignable_list(libxl_ctx *ctx, int *num);
+void libxl_device_pci_assignable_list_free(libxl_pci_bdf *list, int num);
 
 /* CPUID handling */
 int libxl_cpuid_parse_config(libxl_cpuid_policy_list *cpuid, const char* str);
diff --git a/tools/libs/light/libxl_pci.c b/tools/libs/light/libxl_pci.c
index 382c10674313..d436e0a42b0c 100644
--- a/tools/libs/light/libxl_pci.c
+++ b/tools/libs/light/libxl_pci.c
@@ -25,26 +25,33 @@
 #define PCI_BDF_XSPATH "%04x-%02x-%02x-%01x"
 #define PCI_PT_QDEV_ID "pci-pt-%02x_%02x.%01x"
 
-static unsigned int pci_encode_bdf(libxl_device_pci *pci)
+static unsigned int pci_encode_bdf(libxl_pci_bdf *pcibdf)
 {
 unsigned int value;
 
-value = pci->bdf.domain << 16;
-value |= (pci->bdf.bus & 0xff) << 8;
-value |= (pci->bdf.dev & 0x1f) << 3;
-value |= (pci->bdf.func & 0x7);
+value = pcibdf->domain << 16;
+value |= (pcibdf->bus & 0xff) << 8;
+value |= (pcibdf->dev & 0x1f) << 3;
+value |= (pcibdf->func & 0x7);
 
 return value;
 }
 
+static void pcibdf_struct_fill(libxl_pci_bdf *pcibdf, unsigned int domain,
+   unsigned int bus, unsigned int dev,
+   unsigned int func)
+{
+pcibdf->domain = domain;
+pcibdf->bus = bus;
+pcibdf->dev = dev;
+pcibdf->func = func;
+}
+
 static void pci_struct_fill(libxl_device_pci *pci, unsigned int domain,
 unsigned int bus, unsigned int dev,
 unsigned int func, unsigned int vdevfn)
 {
-pci->bdf.domain = domain;
-pci->bdf.bus = bus;
-pci->bdf.dev = dev;
-pci->bdf.func = func;
+pcibdf_struct_fill(>bdf, domain, bus, dev, func);
 pci->vdevfn = vdevfn;
 }
 
@@ -318,8 +325,8 @@ static bool is_pci_in_array(libxl_device_pci *pcis, int num,
 }
 
 /* Write the standard BDF into the sysfs path given by sysfs_path. */
-static int sysfs_write_bdf(libxl__gc *gc, const char * sysfs_path,
-   libxl_device_pci *pci)
+static int sysfs_write_bdf(libxl__gc *gc, const char *sysfs_path,
+   libxl_pci_bdf *pcibdf)
 {
 int rc, fd;
 char *buf;
@@ -330,8 +337,8 @@ static int sysfs_write_bdf(libxl__gc *gc, const char * 
sysfs_path,
 return ERROR_FAIL;
 }
 
-buf = GCSPRINTF(PCI_BDF, pci->bdf.domain, pci->bdf.bus,
-pci->bdf.dev, pci->bdf.func);
+buf = GCSPRINTF(PCI_BDF, pcibdf->domain, pcibdf->bus,
+pcibdf->dev, pcibdf->func);
 rc = write(fd, buf, strlen(buf));
 /* Annoying to have two if's, but we need the errno */
 if (rc < 0)
@@ -346,22 +353,22 @@ static int sysfs_write_bdf(libxl__gc *gc, const char * 
sysfs_path,
 
 #define PCI_INFO_PATH "/libxl/pci"
 
-static char *pci_info_xs_path(libxl__gc *gc, libxl_device_pci *pci,
+static char *pci_info_xs_path(libxl__gc *gc, libxl_pci_bdf *pcibdf,
   const char *node)
 {
 return node ?
 

[PATCH v2 14/24] libxl: Make sure devices added by pci-attach are reflected in the config

2020-11-10 Thread Paul Durrant
From: Paul Durrant 

Currently libxl__device_pci_add_xenstore() is broken in that does not
update the domain's configuration for the first device added (which causes
creation of the overall backend area in xenstore). This can be easily observed
by running 'xl list -l' after adding a single device: the device will be
missing.

This patch fixes the problem and adds a DEBUG log line to allow easy
verification that the domain configuration is being modified.

NOTE: This patch includes a whitespace in add_pcis_done().

Signed-off-by: Paul Durrant 
---
Cc: Ian Jackson 
Cc: Wei Liu 

v2:
 - Avoid having two completely different ways of adding devices into xenstore
---
 tools/libs/light/libxl_pci.c | 71 +++-
 1 file changed, 21 insertions(+), 50 deletions(-)

diff --git a/tools/libs/light/libxl_pci.c b/tools/libs/light/libxl_pci.c
index 06fdc50ce10c..29a450f7e649 100644
--- a/tools/libs/light/libxl_pci.c
+++ b/tools/libs/light/libxl_pci.c
@@ -79,39 +79,18 @@ static void libxl__device_from_pci(libxl__gc *gc, uint32_t 
domid,
 device->kind = LIBXL__DEVICE_KIND_PCI;
 }
 
-static int libxl__create_pci_backend(libxl__gc *gc, uint32_t domid,
- const libxl_device_pci *pci,
- int num)
+static void libxl__create_pci_backend(libxl__gc *gc, uint32_t domid,
+  flexarray_t *front, flexarray_t *back)
 {
-flexarray_t *front = NULL;
-flexarray_t *back = NULL;
-libxl__device device;
-int i;
-
-front = flexarray_make(gc, 16, 1);
-back = flexarray_make(gc, 16, 1);
-
 LOGD(DEBUG, domid, "Creating pci backend");
 
-/* add pci device */
-libxl__device_from_pci(gc, domid, pci, );
-
 flexarray_append_pair(back, "frontend-id", GCSPRINTF("%d", domid));
-flexarray_append_pair(back, "online", "1");
+flexarray_append_pair(back, "online", GCSPRINTF("%d", 1));
 flexarray_append_pair(back, "state", GCSPRINTF("%d", 
XenbusStateInitialising));
 flexarray_append_pair(back, "domain", libxl__domid_to_name(gc, domid));
 
-for (i = 0; i < num; i++, pci++)
-libxl_create_pci_backend_device(gc, back, i, pci);
-
-flexarray_append_pair(back, "num_devs", GCSPRINTF("%d", num));
 flexarray_append_pair(front, "backend-id", GCSPRINTF("%d", 0));
 flexarray_append_pair(front, "state", GCSPRINTF("%d", 
XenbusStateInitialising));
-
-return libxl__device_generic_add(gc, XBT_NULL, ,
- libxl__xs_kvs_of_flexarray(gc, back),
- libxl__xs_kvs_of_flexarray(gc, front),
- NULL);
 }
 
 static int libxl__device_pci_add_xenstore(libxl__gc *gc,
@@ -119,7 +98,7 @@ static int libxl__device_pci_add_xenstore(libxl__gc *gc,
   const libxl_device_pci *pci,
   bool starting)
 {
-flexarray_t *back;
+flexarray_t *front, *back;
 char *num_devs, *be_path;
 int num = 0;
 xs_transaction_t t = XBT_NULL;
@@ -127,16 +106,22 @@ static int libxl__device_pci_add_xenstore(libxl__gc *gc,
 libxl_domain_config d_config;
 libxl__flock *lock = NULL;
 bool is_stubdomain = libxl_is_stubdom(CTX, domid, NULL);
+libxl__device device;
+
+libxl__device_from_pci(gc, domid, pci, );
 
 /* Stubdomain doesn't have own config. */
 if (!is_stubdomain)
 libxl_domain_config_init(_config);
 
+front = flexarray_make(gc, 16, 1);
+back = flexarray_make(gc, 16, 1);
+
 be_path = libxl__domain_device_backend_path(gc, 0, domid, 0,
 LIBXL__DEVICE_KIND_PCI);
 num_devs = libxl__xs_read(gc, XBT_NULL, GCSPRINTF("%s/num_devs", be_path));
 if (!num_devs)
-return libxl__create_pci_backend(gc, domid, pci, 1);
+libxl__create_pci_backend(gc, domid, front, back);
 
 libxl_domain_type domtype = libxl__domain_type(gc, domid);
 if (domtype == LIBXL_DOMAIN_TYPE_INVALID)
@@ -147,20 +132,18 @@ static int libxl__device_pci_add_xenstore(libxl__gc *gc,
 return ERROR_FAIL;
 }
 
-back = flexarray_make(gc, 16, 1);
-
 LOGD(DEBUG, domid, "Adding new pci device to xenstore");
-num = atoi(num_devs);
+num = num_devs ? atoi(num_devs) : 0;
 libxl_create_pci_backend_device(gc, back, num, pci);
 flexarray_append_pair(back, "num_devs", GCSPRINTF("%d", num + 1));
-if (!starting)
+if (num && !starting)
 flexarray_append_pair(back, "state", GCSPRINTF("%d", 
XenbusStateReconfiguring));
 
 /*
  * Stubdomin config is derived from its target domain, it doesn't have
  * its own file.
  */
-if (!is_stubdomain) {
+if (!is_stubdomain && !starting) {
 lock = libxl__lock_domain_userdata(gc, domid);
 if (!lock) {
 rc = ERROR_LOCK_FAIL;
@@ -170,6 +153,7 @@ static int libxl__device_pci_add_xenstore(libxl__gc *gc,

[PATCH v2 16/24] docs/man: improve documentation of PCI_SPEC_STRING...

2020-11-10 Thread Paul Durrant
From: Paul Durrant 

... and prepare for adding support for non-positional parsing of 'bdf' and
'vslot' in a subsequent patch.

Also document 'BDF' as a first-class parameter type and fix the documentation
to state that the default value of 'rdm_policy' is actually 'strict', not
'relaxed', as can be seen in libxl__device_pci_setdefault().

Signed-off-by: Paul Durrant 
---
Cc: Ian Jackson 
Cc: Wei Liu 
---
 docs/man/xl-pci-configuration.5.pod | 187 +++-
 1 file changed, 153 insertions(+), 34 deletions(-)

diff --git a/docs/man/xl-pci-configuration.5.pod 
b/docs/man/xl-pci-configuration.5.pod
index 72a27bd95dec..4dd73bc498d6 100644
--- a/docs/man/xl-pci-configuration.5.pod
+++ b/docs/man/xl-pci-configuration.5.pod
@@ -6,32 +6,105 @@ xl-pci-configuration - XL PCI Configuration Syntax
 
 =head1 SYNTAX
 
-This document specifies the format for B which is used by
-the L pci configuration option, and related L commands.
+This document specifies the format for B and B which are
+used by the L pci configuration option, and related L
+commands.
 
-Each B has the form of
-B<[:]BB:DD.F[@VSLOT],KEY=VALUE,KEY=VALUE,...> where:
+A B has the following form:
+
+[:]BB:SS.F
+
+B is the domain number, B is the bus number, B is the device (or
+slot) number, and B is the function number. This is the same scheme as
+used in the output of L for the device in question. By default
+L will omit the domain (B) if it is zero and hence a zero
+value for domain may also be omitted when specifying a B.
+
+Each B has the one of the forms:
 
 =over 4
 
-=item B<[:]BB:DD.F>
+[[@,][=,]*
+[=,]*
 
-Identifies the PCI device from the host perspective in the domain
-(B), Bus (B), Device (B) and Function (B) syntax. This is
-the same scheme as used in the output of B for the device in
-question.
+=back
 
-Note: by default B will omit the domain (B) if it
-is zero and it is optional here also. You may specify the function
-(B) as B<*> to indicate all functions.
+For example, these strings are equivalent:
 
-=item B<@VSLOT>
+=over 4
 
-Specifies the virtual slot where the guest will see this
-device. This is equivalent to the B which the guest sees. In a
-guest B and B are C<:00>.
+36:00.0@20,seize=1
+36:00.0,vslot=20,seize=1
+bdf=36:00.0,vslot=20,seize=1
 
-=item B
+=back
+
+More formally, the string is a series of comma-separated keyword/value
+pairs, flags and positional parameters.  Parameters which are not bare
+keywords and which do not contain "=" symbols are assigned to the
+positional parameters, in the order specified below.  The positional
+parameters may also be specified by name.
+
+Each parameter may be specified at most once, either as a positional
+parameter or a named parameter.  Default values apply if the parameter
+is not specified, or if it is specified with an empty value (whether
+positionally or explicitly).
+
+B: In context of B (see L), parameters other than
+B will be ignored.
+
+=head1 Positional Parameters
+
+=over 4
+
+=item B=I
+
+=over 4
+
+=item Description
+
+This identifies the PCI device from the host perspective.
+
+In the context of a B you may specify the function (B) as
+B<*> to indicate all functions of a multi-function device.
+
+=item Default Value
+
+None. This parameter is mandatory as it identifies the device.
+
+=back
+
+=item B=I
+
+=over 4
+
+=item Description
+
+Specifies the virtual slot (device) number where the guest will see this
+device. For example, running L in a Linux guest where B
+was specified as C<8> would identify the device as C<00:08.0>. Virtual domain
+and bus numbers are always 0.
+
+B This parameter is always parsed as a hexidecimal value.
+
+=item Default Value
+
+None. This parameter is not mandatory. An available B will be selected
+if this parameter is not specified.
+
+=back
+
+=back
+
+=head1 Other Parameters and Flags
+
+=over 4
+
+=item B=I
+
+=over 4
+
+=item Description
 
 By default pciback only allows PV guests to write "known safe" values
 into PCI configuration space, likewise QEMU (both qemu-xen and
@@ -46,33 +119,79 @@ more control over the device, which may have security or 
stability
 implications.  It is recommended to only enable this option for
 trusted VMs under administrator's control.
 
-=item B
+=item Default Value
+
+0
+
+=back
+
+=item B=I
+
+=over 4
+
+=item Description
 
 Specifies that MSI-INTx translation should be turned on for the PCI
 device. When enabled, MSI-INTx translation will always enable MSI on
-the PCI device regardless of whether the guest uses INTx or MSI. Some
-device drivers, such as NVIDIA's, detect an inconsistency and do not
+the PCI device regardless of whether the guest uses INTx or MSI.
+
+=item Default Value
+
+Some device drivers, such as NVIDIA's, detect an inconsistency and do not
 function when this option is enabled. Therefore the default is false (0).
 
-=item B
+=back
 
-Tells B to automatically attempt to re-assign a device to
-pciback if it is not already 

[PATCH v2 21/24] docs/man: modify xl(1) in preparation for naming of assignable devices

2020-11-10 Thread Paul Durrant
From: Paul Durrant 

A subsequent patch will introduce code to allow a name to be specified to
'xl pci-assignable-add' such that the assignable device may be referred to
by than name in subsequent operations.

Signed-off-by: Paul Durrant 
---
Cc: Ian Jackson 
Cc: Wei Liu 
---
 docs/man/xl.1.pod.in | 19 ---
 1 file changed, 12 insertions(+), 7 deletions(-)

diff --git a/docs/man/xl.1.pod.in b/docs/man/xl.1.pod.in
index c5fbce3b5c4b..0822a5842835 100644
--- a/docs/man/xl.1.pod.in
+++ b/docs/man/xl.1.pod.in
@@ -1595,19 +1595,23 @@ List virtual network interfaces for a domain.
 
 =over 4
 
-=item B
+=item B [I<-n>]
 
 List all the B of assignable PCI devices. See
-L for more information.
+L for more information. If the -n option is
+specified then any name supplied when the device was made assignable
+will also be displayed.
 
 These are devices in the system which are configured to be
 available for passthrough and are bound to a suitable PCI
 backend driver in domain 0 rather than a real driver.
 
-=item B I
+=item B [I<-n NAME>] I
 
 Make the device at B assignable to guests. See
-L for more information.
+L for more information. If the -n option is
+supplied then the assignable device entry will the named with the
+given B.
 
 This will bind the device to the pciback driver and assign it to the
 "quarantine domain".  If it is already bound to a driver, it will
@@ -1622,10 +1626,11 @@ not to do this on a device critical to domain 0's 
operation, such as
 storage controllers, network interfaces, or GPUs that are currently
 being used.
 
-=item B [I<-r>] I
+=item B [I<-r>] I|I
 
-Make the device at B not assignable to guests. See
-L for more information.
+Make a device non-assignable to guests. The device may be identified
+either by its B or the B supplied when the device was made
+assignable. See L for more information.
 
 This will at least unbind the device from pciback, and
 re-assign it from the "quarantine domain" back to domain 0.  If the -r
-- 
2.20.1




[PATCH v2 22/24] xl / libxl: support naming of assignable devices

2020-11-10 Thread Paul Durrant
From: Paul Durrant 

This patch modifies libxl_device_pci_assignable_add() to take an optional
'name' argument, which (if supplied) is saved into xenstore and can hence be
used to refer to the now-assignable BDF in subsequent operations. To
facilitate this, a new libxl_device_pci_assignable_name2bdf() function is
added.

The xl code is modified to allow a name to be specified in the
'pci-assignable-add' operation and also allow an option to be specified to
'pci-assignable-list' requesting that names be displayed. The latter is
facilitated by a new libxl_device_pci_assignable_bdf2name() function. Finally
xl 'pci-assignable-remove' is modified to that either a name or BDF can be
supplied. The supplied 'identifier' is first assumed to be a name, but if
libxl_device_pci_assignable_name2bdf() fails to find a matching BDF the
identifier itself will be parsed as a BDF. Names my only include printable
characters and may not include whitespace.

Signed-off-by: Paul Durrant 
---
Cc: Ian Jackson 
Cc: Wei Liu 
Cc: Christian Lindig 
Cc: David Scott 
Cc: Anthony PERARD 
---
 tools/include/libxl.h| 19 +-
 tools/libs/light/libxl_pci.c | 86 ++--
 tools/ocaml/libs/xl/xenlight_stubs.c |  3 +-
 tools/xl/xl_cmdtable.c   | 12 ++--
 tools/xl/xl_pci.c| 80 ++
 5 files changed, 164 insertions(+), 36 deletions(-)

diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index 5703fdf367c5..4025d3a3d437 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -476,6 +476,14 @@
  */
 #define LIBXL_HAVE_PCI_ASSIGNABLE_BDF 1
 
+/*
+ * LIBXL_HAVE_PCI_ASSIGNABLE_NAME indicates that the
+ * libxl_device_pci_assignable_add() function takes a 'name' argument
+ * and that the libxl_device_pci_assignable_name2bdf() and
+ * libxl_device_pci_assignable_bdf2name() functions are defined.
+ */
+#define LIBXL_HAVE_PCI_ASSIGNABLE_NAME 1
+
 /*
  * libxl ABI compatibility
  *
@@ -2385,11 +2393,18 @@ int libxl_device_events_handler(libxl_ctx *ctx,
  * added or is not bound, the functions will emit a warning but return
  * SUCCESS.
  */
-int libxl_device_pci_assignable_add(libxl_ctx *ctx, libxl_pci_bdf *pcibdf, int 
rebind);
-int libxl_device_pci_assignable_remove(libxl_ctx *ctx, libxl_pci_bdf *pcibdf, 
int rebind);
+int libxl_device_pci_assignable_add(libxl_ctx *ctx, libxl_pci_bdf *pcibdf,
+const char *name, int rebind);
+int libxl_device_pci_assignable_remove(libxl_ctx *ctx, libxl_pci_bdf *pcibdf,
+   int rebind);
 libxl_pci_bdf *libxl_device_pci_assignable_list(libxl_ctx *ctx, int *num);
 void libxl_device_pci_assignable_list_free(libxl_pci_bdf *list, int num);
 
+libxl_pci_bdf *libxl_device_pci_assignable_name2bdf(libxl_ctx *ctx,
+const char *name);
+char *libxl_device_pci_assignable_bdf2name(libxl_ctx *ctx,
+   libxl_pci_bdf *pcibdf);
+
 /* CPUID handling */
 int libxl_cpuid_parse_config(libxl_cpuid_policy_list *cpuid, const char* str);
 int libxl_cpuid_parse_config_xend(libxl_cpuid_policy_list *cpuid,
diff --git a/tools/libs/light/libxl_pci.c b/tools/libs/light/libxl_pci.c
index d436e0a42b0c..a789ade036fa 100644
--- a/tools/libs/light/libxl_pci.c
+++ b/tools/libs/light/libxl_pci.c
@@ -713,6 +713,7 @@ static int pciback_dev_unassign(libxl__gc *gc, 
libxl_pci_bdf *pcibdf)
 
 static int libxl__device_pci_assignable_add(libxl__gc *gc,
 libxl_pci_bdf *pcibdf,
+const char *name,
 int rebind)
 {
 libxl_ctx *ctx = libxl__gc_owner(gc);
@@ -721,6 +722,23 @@ static int libxl__device_pci_assignable_add(libxl__gc *gc,
 int rc;
 struct stat st;
 
+/* Sanitise any name that was passed */
+if (name) {
+unsigned int i, n = strlen(name);
+
+if (n > 64) { /* Reasonable upper bound on name length */
+LOG(ERROR, "Name too long");
+return ERROR_FAIL;
+}
+
+for (i = 0; i < n; i++) {
+if (!isgraph(name[i])) {
+LOG(ERROR, "Names may only include printable characters");
+return ERROR_FAIL;
+}
+}
+}
+
 /* Local copy for convenience */
 dom = pcibdf->domain;
 bus = pcibdf->bus;
@@ -741,7 +759,7 @@ static int libxl__device_pci_assignable_add(libxl__gc *gc,
 }
 if ( rc ) {
 LOG(WARN, PCI_BDF" already assigned to pciback", dom, bus, dev, func);
-goto quarantine;
+goto name;
 }
 
 /* Check to see if there's already a driver that we need to unbind from */
@@ -772,7 +790,12 @@ static int libxl__device_pci_assignable_add(libxl__gc *gc,
 return ERROR_FAIL;
 }
 
-quarantine:
+name:
+if (name)
+pci_info_xs_write(gc, pcibdf, "name", name);
+else
+

[PATCH v2 13/24] libxl: add/recover 'rdm_policy' to/from PCI backend in xenstore

2020-11-10 Thread Paul Durrant
From: Paul Durrant 

Other parameters, such as 'msitranslate' and 'permissive' are dealt with
but 'rdm_policy' appears to be have been completely missed.

Signed-off-by: Paul Durrant 
---
Cc: Ian Jackson 
Cc: Wei Liu 
---
 tools/libs/light/libxl_pci.c | 9 ++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/tools/libs/light/libxl_pci.c b/tools/libs/light/libxl_pci.c
index 278ebd9f561b..06fdc50ce10c 100644
--- a/tools/libs/light/libxl_pci.c
+++ b/tools/libs/light/libxl_pci.c
@@ -61,9 +61,9 @@ static void libxl_create_pci_backend_device(libxl__gc *gc,
 flexarray_append_pair(back, GCSPRINTF("vdevfn-%d", num), 
GCSPRINTF("%x", pci->vdevfn));
 flexarray_append(back, GCSPRINTF("opts-%d", num));
 flexarray_append(back,
-  GCSPRINTF("msitranslate=%d,power_mgmt=%d,permissive=%d",
- pci->msitranslate, pci->power_mgmt,
- pci->permissive));
+  
GCSPRINTF("msitranslate=%d,power_mgmt=%d,permissive=%d,rdm_policy=%s",
+pci->msitranslate, pci->power_mgmt,
+pci->permissive, 
libxl_rdm_reserve_policy_to_string(pci->rdm_policy)));
 flexarray_append_pair(back, GCSPRINTF("state-%d", num), GCSPRINTF("%d", 
XenbusStateInitialising));
 }
 
@@ -2313,6 +2313,9 @@ static int libxl__device_pci_from_xs_be(libxl__gc *gc,
 } else if (!strcmp(p, "permissive")) {
 p = strtok_r(NULL, ",=", );
 pci->permissive = atoi(p);
+} else if (!strcmp(p, "rdm_policy")) {
+p = strtok_r(NULL, ",=", );
+libxl_rdm_reserve_policy_from_string(p, >rdm_policy);
 }
 } while ((p = strtok_r(NULL, ",=", )) != NULL);
 }
-- 
2.20.1




[PATCH v2 10/24] libxl: make sure callers of libxl_device_pci_list() free the list after use

2020-11-10 Thread Paul Durrant
From: Paul Durrant 

A previous patch introduced libxl_device_pci_list_free() which should be used
by callers of libxl_device_pci_list() to properly dispose of the exported
'libxl_device_pci' types and the free the memory holding them. Whilst all
current callers do ensure the memory is freed, only the code in xl's
pcilist() function actually calls libxl_device_pci_dispose(). As it stands
this laxity does not lead to any memory leaks, but the simple addition of
.e.g. a 'string' into the idl definition of 'libxl_device_pci' would lead
to leaks.

This patch makes sure all callers of libxl_device_pci_list() can call
libxl_device_pci_list_free() by keeping copies of 'libxl_device_pci'
structures inline in 'pci_add_state' and 'pci_remove_state' (and also making
sure these are properly disposed at the end of the operations) rather
than keeping pointers to the structures returned by libxl_device_pci_list().

Signed-off-by: Paul Durrant 
---
Cc: Ian Jackson 
Cc: Wei Liu 
Cc: Anthony PERARD 
---
 tools/libs/light/libxl_pci.c | 68 
 tools/xl/xl_pci.c|  3 +-
 2 files changed, 38 insertions(+), 33 deletions(-)

diff --git a/tools/libs/light/libxl_pci.c b/tools/libs/light/libxl_pci.c
index ff37dc5b5921..a69cba793583 100644
--- a/tools/libs/light/libxl_pci.c
+++ b/tools/libs/light/libxl_pci.c
@@ -1006,7 +1006,7 @@ typedef struct pci_add_state {
 libxl__xswait_state xswait;
 libxl__ev_qmp qmp;
 libxl__ev_time timeout;
-libxl_device_pci *pci;
+libxl_device_pci pci;
 libxl_domid pci_domid;
 } pci_add_state;
 
@@ -1078,7 +1078,7 @@ static void pci_add_qemu_trad_watch_state_cb(libxl__egc 
*egc,
 
 /* Convenience aliases */
 libxl_domid domid = pas->domid;
-libxl_device_pci *pci = pas->pci;
+libxl_device_pci *pci = >pci;
 
 rc = check_qemu_running(gc, domid, xswa, rc, state);
 if (rc == ERROR_NOT_READY)
@@ -1099,7 +1099,7 @@ static void pci_add_qmp_device_add(libxl__egc *egc, 
pci_add_state *pas)
 
 /* Convenience aliases */
 libxl_domid domid = pas->domid;
-libxl_device_pci *pci = pas->pci;
+libxl_device_pci *pci = >pci;
 libxl__ev_qmp *const qmp = >qmp;
 
 rc = libxl__ev_time_register_rel(ao, >timeout,
@@ -1180,7 +1180,7 @@ static void pci_add_qmp_query_pci_cb(libxl__egc *egc,
 int dev_slot, dev_func;
 
 /* Convenience aliases */
-libxl_device_pci *pci = pas->pci;
+libxl_device_pci *pci = >pci;
 
 if (rc) goto out;
 
@@ -1281,7 +1281,7 @@ static void pci_add_dm_done(libxl__egc *egc,
 
 /* Convenience aliases */
 bool starting = pas->starting;
-libxl_device_pci *pci = pas->pci;
+libxl_device_pci *pci = >pci;
 bool hvm = libxl__domain_type(gc, domid) == LIBXL_DOMAIN_TYPE_HVM;
 
 libxl__ev_qmp_dispose(gc, >qmp);
@@ -1497,7 +1497,10 @@ void libxl__device_pci_add(libxl__egc *egc, uint32_t 
domid,
 GCNEW(pas);
 pas->aodev = aodev;
 pas->domid = domid;
-pas->pci = pci;
+
+libxl_device_pci_copy(CTX, >pci, pci);
+pci = >pci;
+
 pas->starting = starting;
 pas->callback = device_pci_add_stubdom_done;
 
@@ -1536,12 +1539,6 @@ void libxl__device_pci_add(libxl__egc *egc, uint32_t 
domid,
 
 stubdomid = libxl_get_stubdom_id(ctx, domid);
 if (stubdomid != 0) {
-libxl_device_pci *pci_s;
-
-GCNEW(pci_s);
-libxl_device_pci_init(pci_s);
-libxl_device_pci_copy(CTX, pci_s, pci);
-pas->pci = pci_s;
 pas->callback = device_pci_add_stubdom_wait;
 
 do_pci_add(egc, stubdomid, pas); /* must be last */
@@ -1600,7 +1597,7 @@ static void device_pci_add_stubdom_done(libxl__egc *egc,
 
 /* Convenience aliases */
 libxl_domid domid = pas->domid;
-libxl_device_pci *pci = pas->pci;
+libxl_device_pci *pci = >pci;
 
 if (rc) goto out;
 
@@ -1651,7 +1648,7 @@ static void device_pci_add_done(libxl__egc *egc,
 EGC_GC;
 libxl__ao_device *aodev = pas->aodev;
 libxl_domid domid = pas->domid;
-libxl_device_pci *pci = pas->pci;
+libxl_device_pci *pci = >pci;
 
 if (rc) {
 LOGD(ERROR, domid,
@@ -1661,6 +1658,7 @@ static void device_pci_add_done(libxl__egc *egc,
  rc);
 pci_info_xs_remove(gc, pci, "domid");
 }
+libxl_device_pci_dispose(pci);
 aodev->rc = rc;
 aodev->callback(egc, aodev);
 }
@@ -1767,7 +1765,7 @@ static int qemu_pci_remove_xenstore(libxl__gc *gc, 
uint32_t domid,
 typedef struct pci_remove_state {
 libxl__ao_device *aodev;
 libxl_domid domid;
-libxl_device_pci *pci;
+libxl_device_pci pci;
 bool force;
 bool hvm;
 unsigned int orig_vdev;
@@ -1809,23 +1807,26 @@ static void do_pci_remove(libxl__egc *egc, 
pci_remove_state *prs)
 {
 STATE_AO_GC(prs->aodev->ao);
 libxl_ctx *ctx = libxl__gc_owner(gc);
-libxl_device_pci *assigned;
+libxl_device_pci *pcis;
+bool attached;
 uint32_t domid = prs->domid;
 libxl_domain_type type = libxl__domain_type(gc, domid);
-

[PATCH v2 19/24] libxlu: introduce xlu_pci_parse_spec_string()

2020-11-10 Thread Paul Durrant
From: Paul Durrant 

This patch largely re-writes the code to parse a PCI_SPEC_STRING and enters
it via the newly introduced function. The new parser also deals with 'bdf'
and 'vslot' as non-positional paramaters, as per the documentation in
xl-pci-configuration(5).

The existing xlu_pci_parse_bdf() function remains, but now strictly parses
BDF values. Some existing callers of xlu_pci_parse_bdf() are
modified to call xlu_pci_parse_spec_string() as per the documentation in xl(1).

NOTE: Usage text in xl_cmdtable.c and error messages are also modified
  appropriately.

Fixes: d25cc3ec93eb ("libxl: workaround gcc 10.2 maybe-uninitialized warning")
Signed-off-by: Paul Durrant 
---
Cc: Ian Jackson 
Cc: Wei Liu 
Cc: Anthony PERARD 
---
 tools/include/libxlutil.h|   8 +-
 tools/libs/util/libxlu_pci.c | 374 +++
 tools/xl/xl_cmdtable.c   |   4 +-
 tools/xl/xl_parse.c  |   4 +-
 tools/xl/xl_pci.c|  37 ++--
 5 files changed, 230 insertions(+), 197 deletions(-)

diff --git a/tools/include/libxlutil.h b/tools/include/libxlutil.h
index 92e35c546278..cdd6aab4f816 100644
--- a/tools/include/libxlutil.h
+++ b/tools/include/libxlutil.h
@@ -108,10 +108,16 @@ int xlu_disk_parse(XLU_Config *cfg, int nspecs, const 
char *const *specs,
* resulting disk struct is used with libxl.
*/
 
+/*
+ * PCI BDF
+ */
+int xlu_pci_parse_bdf(XLU_Config *cfg, libxl_pci_bdf *bdf, const char *str);
+
 /*
  * PCI specification parsing
  */
-int xlu_pci_parse_bdf(XLU_Config *cfg, libxl_device_pci *pcidev, const char 
*str);
+int xlu_pci_parse_spec_string(XLU_Config *cfg, libxl_device_pci *pci,
+  const char *str);
 
 /*
  * RDM parsing
diff --git a/tools/libs/util/libxlu_pci.c b/tools/libs/util/libxlu_pci.c
index 5c107f264260..a8b6ce542736 100644
--- a/tools/libs/util/libxlu_pci.c
+++ b/tools/libs/util/libxlu_pci.c
@@ -1,5 +1,7 @@
 #define _GNU_SOURCE
 
+#include 
+
 #include "libxlu_internal.h"
 #include "libxlu_disk_l.h"
 #include "libxlu_disk_i.h"
@@ -9,185 +11,213 @@
 #define XLU__PCI_ERR(_c, _x, _a...) \
 if((_c) && (_c)->report) fprintf((_c)->report, _x, ##_a)
 
-static int hex_convert(const char *str, unsigned int *val, unsigned int mask)
+static int parse_bdf(libxl_pci_bdf *bdfp, uint32_t *vfunc_maskp,
+ const char *str, const char **endp)
 {
-unsigned long ret;
-char *end;
+const char *ptr = str;
+unsigned int colons = 0;
+unsigned int domain, bus, dev, func;
+int n;
 
-ret = strtoul(str, , 16);
-if ( end == str || *end != '\0' )
-return -1;
-if ( ret & ~mask )
-return -1;
-*val = (unsigned int)ret & mask;
-return 0;
-}
-
-static int pci_struct_fill(libxl_device_pci *pci, unsigned int domain,
-   unsigned int bus, unsigned int dev,
-   unsigned int func, unsigned int vdevfn)
-{
-pci->bdf.domain = domain;
-pci->bdf.bus = bus;
-pci->bdf.dev = dev;
-pci->bdf.func = func;
-pci->vdevfn = vdevfn;
-return 0;
-}
-
-#define STATE_DOMAIN0
-#define STATE_BUS   1
-#define STATE_DEV   2
-#define STATE_FUNC  3
-#define STATE_VSLOT 4
-#define STATE_OPTIONS_K 6
-#define STATE_OPTIONS_V 7
-#define STATE_TERMINAL  8
-#define STATE_TYPE  9
-#define STATE_RDM_STRATEGY  10
-#define STATE_RESERVE_POLICY11
-#define INVALID 0x
-int xlu_pci_parse_bdf(XLU_Config *cfg, libxl_device_pci *pci, const char *str)
-{
-unsigned state = STATE_DOMAIN;
-unsigned dom = INVALID, bus = INVALID, dev = INVALID, func = INVALID, 
vslot = 0;
-char *buf2, *tok, *ptr, *end, *optkey = NULL;
-
-if ( NULL == (buf2 = ptr = strdup(str)) )
-return ERROR_NOMEM;
-
-for(tok = ptr, end = ptr + strlen(ptr) + 1; ptr < end; ptr++) {
-switch(state) {
-case STATE_DOMAIN:
-if ( *ptr == ':' ) {
-state = STATE_BUS;
-*ptr = '\0';
-if ( hex_convert(tok, , 0x) )
-goto parse_error;
-tok = ptr + 1;
-}
-break;
-case STATE_BUS:
-if ( *ptr == ':' ) {
-state = STATE_DEV;
-*ptr = '\0';
-if ( hex_convert(tok, , 0xff) )
-goto parse_error;
-tok = ptr + 1;
-}else if ( *ptr == '.' ) {
-state = STATE_FUNC;
-*ptr = '\0';
-if ( dom & ~0xff )
-goto parse_error;
-bus = dom;
-dom = 0;
-if ( hex_convert(tok, , 0xff) )
-goto parse_error;
-tok = ptr + 1;
-}
-break;
-case STATE_DEV:
-if ( *ptr == '.' ) {
-state = STATE_FUNC;
-*ptr = '\0';
-if ( hex_convert(tok, , 0xff) )
-goto parse_error;
- 

[PATCH v2 12/24] libxl: use COMPARE_PCI() macro is_pci_in_array()...

2020-11-10 Thread Paul Durrant
From: Paul Durrant 

... rather than an open-coded equivalent.

This patch tidies up the is_pci_in_array() function, making it take a single
'libxl_device_pci' argument rather than separate domain, bus, device and
function arguments. The already-available COMPARE_PCI() macro can then be
used and it is also modified to return 'bool' rather than 'int'.

The patch also modifies libxl_pci_assignable() to use is_pci_in_array() rather
than a separate open-coded equivalent, and also modifies it to return a
'bool' rather than an 'int'.

NOTE: The COMPARE_PCI() macro is also fixed to include the 'domain' in its
  comparison, which should always have been the case.

Signed-off-by: Paul Durrant 
---
Cc: Ian Jackson 
Cc: Wei Liu 
---
 tools/libs/light/libxl_internal.h |  7 +++---
 tools/libs/light/libxl_pci.c  | 38 +++
 2 files changed, 17 insertions(+), 28 deletions(-)

diff --git a/tools/libs/light/libxl_internal.h 
b/tools/libs/light/libxl_internal.h
index 3e70ff639b3c..80d798862229 100644
--- a/tools/libs/light/libxl_internal.h
+++ b/tools/libs/light/libxl_internal.h
@@ -4744,9 +4744,10 @@ void libxl__xcinfo2xlinfo(libxl_ctx *ctx,
  * devices have same identifier. */
 #define COMPARE_DEVID(a, b) ((a)->devid == (b)->devid)
 #define COMPARE_DISK(a, b) (!strcmp((a)->vdev, (b)->vdev))
-#define COMPARE_PCI(a, b) ((a)->func == (b)->func &&\
-   (a)->bus == (b)->bus &&  \
-   (a)->dev == (b)->dev)
+#define COMPARE_PCI(a, b) ((a)->domain == (b)->domain && \
+   (a)->bus == (b)->bus &&   \
+   (a)->dev == (b)->dev &&   \
+   (a)->func == (b)->func)
 #define COMPARE_USB(a, b) ((a)->ctrl == (b)->ctrl && \
(a)->port == (b)->port)
 #define COMPARE_USBCTRL(a, b) ((a)->devid == (b)->devid)
diff --git a/tools/libs/light/libxl_pci.c b/tools/libs/light/libxl_pci.c
index b87e121c4d5c..278ebd9f561b 100644
--- a/tools/libs/light/libxl_pci.c
+++ b/tools/libs/light/libxl_pci.c
@@ -317,24 +317,17 @@ retry_transaction2:
 return 0;
 }
 
-static int is_pci_in_array(libxl_device_pci *assigned, int num_assigned,
-   int dom, int bus, int dev, int func)
+static bool is_pci_in_array(libxl_device_pci *pcis, int num,
+libxl_device_pci *pci)
 {
 int i;
 
-for(i = 0; i < num_assigned; i++) {
-if ( assigned[i].domain != dom )
-continue;
-if ( assigned[i].bus != bus )
-continue;
-if ( assigned[i].dev != dev )
-continue;
-if ( assigned[i].func != func )
-continue;
-return 1;
+for (i = 0; i < num; i++) {
+if (COMPARE_PCI(pci, [i]))
+break;
 }
 
-return 0;
+return i < num;
 }
 
 /* Write the standard BDF into the sysfs path given by sysfs_path. */
@@ -1468,21 +1461,17 @@ int libxl_device_pci_add(libxl_ctx *ctx, uint32_t domid,
 return AO_INPROGRESS;
 }
 
-static int libxl_pci_assignable(libxl_ctx *ctx, libxl_device_pci *pci)
+static bool libxl_pci_assignable(libxl_ctx *ctx, libxl_device_pci *pci)
 {
 libxl_device_pci *pcis;
-int num, i;
+int num;
+bool assignable;
 
 pcis = libxl_device_pci_assignable_list(ctx, );
-for (i = 0; i < num; i++) {
-if (pcis[i].domain == pci->domain &&
-pcis[i].bus == pci->bus &&
-pcis[i].dev == pci->dev &&
-pcis[i].func == pci->func)
-break;
-}
+assignable = is_pci_in_array(pcis, num, pci);
 libxl_device_pci_assignable_list_free(pcis, num);
-return i != num;
+
+return assignable;
 }
 
 static void device_pci_add_stubdom_wait(libxl__egc *egc,
@@ -1831,8 +1820,7 @@ static void do_pci_remove(libxl__egc *egc, 
pci_remove_state *prs)
 goto out_fail;
 }
 
-attached = is_pci_in_array(pcis, num, pci->domain,
-   pci->bus, pci->dev, pci->func);
+attached = is_pci_in_array(pcis, num, pci);
 libxl_device_pci_list_free(pcis, num);
 
 rc = ERROR_INVAL;
-- 
2.20.1




[PATCH v2 15/24] docs/man: extract documentation of PCI_SPEC_STRING from the xl.cfg manpage...

2020-11-10 Thread Paul Durrant
From: Paul Durrant 

... and put it into a new xl-pci-configuration(5) manpage, akin to the
xl-network-configration(5) and xl-disk-configuration(5) manpages.

This patch moves the content of the section verbatim. A subsequent patch
will improve the documentation, once it is in its new location.

Signed-off-by: Paul Durrant 
---
Cc: Ian Jackson 
Cc: Wei Liu 
---
 docs/man/xl-pci-configuration.5.pod | 78 +
 docs/man/xl.cfg.5.pod.in| 68 +
 2 files changed, 79 insertions(+), 67 deletions(-)
 create mode 100644 docs/man/xl-pci-configuration.5.pod

diff --git a/docs/man/xl-pci-configuration.5.pod 
b/docs/man/xl-pci-configuration.5.pod
new file mode 100644
index ..72a27bd95dec
--- /dev/null
+++ b/docs/man/xl-pci-configuration.5.pod
@@ -0,0 +1,78 @@
+=encoding utf8
+
+=head1 NAME
+
+xl-pci-configuration - XL PCI Configuration Syntax
+
+=head1 SYNTAX
+
+This document specifies the format for B which is used by
+the L pci configuration option, and related L commands.
+
+Each B has the form of
+B<[:]BB:DD.F[@VSLOT],KEY=VALUE,KEY=VALUE,...> where:
+
+=over 4
+
+=item B<[:]BB:DD.F>
+
+Identifies the PCI device from the host perspective in the domain
+(B), Bus (B), Device (B) and Function (B) syntax. This is
+the same scheme as used in the output of B for the device in
+question.
+
+Note: by default B will omit the domain (B) if it
+is zero and it is optional here also. You may specify the function
+(B) as B<*> to indicate all functions.
+
+=item B<@VSLOT>
+
+Specifies the virtual slot where the guest will see this
+device. This is equivalent to the B which the guest sees. In a
+guest B and B are C<:00>.
+
+=item B
+
+By default pciback only allows PV guests to write "known safe" values
+into PCI configuration space, likewise QEMU (both qemu-xen and
+qemu-xen-traditional) imposes the same constraint on HVM guests.
+However, many devices require writes to other areas of the configuration space
+in order to operate properly.  This option tells the backend (pciback or QEMU)
+to allow all writes to the PCI configuration space of this device by this
+domain.
+
+B it gives the guest much
+more control over the device, which may have security or stability
+implications.  It is recommended to only enable this option for
+trusted VMs under administrator's control.
+
+=item B
+
+Specifies that MSI-INTx translation should be turned on for the PCI
+device. When enabled, MSI-INTx translation will always enable MSI on
+the PCI device regardless of whether the guest uses INTx or MSI. Some
+device drivers, such as NVIDIA's, detect an inconsistency and do not
+function when this option is enabled. Therefore the default is false (0).
+
+=item B
+
+Tells B to automatically attempt to re-assign a device to
+pciback if it is not already assigned.
+
+B If you set this option, B will gladly re-assign a critical
+system device, such as a network or a disk controller being used by
+dom0 without confirmation.  Please use with care.
+
+=item B
+
+B<(HVM only)> Specifies that the VM should be able to program the
+D0-D3hot power management states for the PCI device. The default is false (0).
+
+=item B
+
+B<(HVM/x86 only)> This is the same as the policy setting inside the B
+option but just specific to a given device. The default is "relaxed".
+
+Note: this would override global B option.
+
+=back
diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in
index 0532739c1fff..b00644e852f9 100644
--- a/docs/man/xl.cfg.5.pod.in
+++ b/docs/man/xl.cfg.5.pod.in
@@ -1101,73 +1101,7 @@ option is valid only when the B option is 
specified.
 =item B
 
 Specifies the host PCI devices to passthrough to this guest.
-Each B has the form of
-B<[:]BB:DD.F[@VSLOT],KEY=VALUE,KEY=VALUE,...> where:
-
-=over 4
-
-=item B<[:]BB:DD.F>
-
-Identifies the PCI device from the host perspective in the domain
-(B), Bus (B), Device (B) and Function (B) syntax. This is
-the same scheme as used in the output of B for the device in
-question.
-
-Note: by default B will omit the domain (B) if it
-is zero and it is optional here also. You may specify the function
-(B) as B<*> to indicate all functions.
-
-=item B<@VSLOT>
-
-Specifies the virtual slot where the guest will see this
-device. This is equivalent to the B which the guest sees. In a
-guest B and B are C<:00>.
-
-=item B
-
-By default pciback only allows PV guests to write "known safe" values
-into PCI configuration space, likewise QEMU (both qemu-xen and
-qemu-xen-traditional) imposes the same constraint on HVM guests.
-However, many devices require writes to other areas of the configuration space
-in order to operate properly.  This option tells the backend (pciback or QEMU)
-to allow all writes to the PCI configuration space of this device by this
-domain.
-
-B it gives the guest much
-more control over the device, which may have security or stability
-implications.  It is recommended to only enable this option for

[PATCH v2 17/24] docs/man: fix xl(1) documentation for 'pci' operations

2020-11-10 Thread Paul Durrant
From: Paul Durrant 

Currently the documentation completely fails to mention the existence of
PCI_SPEC_STRING. This patch tidies things up, specifically clarifying that
'pci-assignable-add/remove' take  arguments where as 'pci-attach/detach'
take  arguments (which will be enforced in a subsequent
patch).

Signed-off-by: Paul Durrant 
---
Cc: Ian Jackson 
Cc: Wei Liu 
---
 docs/man/xl.1.pod.in | 28 +---
 1 file changed, 17 insertions(+), 11 deletions(-)

diff --git a/docs/man/xl.1.pod.in b/docs/man/xl.1.pod.in
index f92bacfa7277..c5fbce3b5c4b 100644
--- a/docs/man/xl.1.pod.in
+++ b/docs/man/xl.1.pod.in
@@ -1597,14 +1597,18 @@ List virtual network interfaces for a domain.
 
 =item B
 
-List all the assignable PCI devices.
+List all the B of assignable PCI devices. See
+L for more information.
+
 These are devices in the system which are configured to be
 available for passthrough and are bound to a suitable PCI
 backend driver in domain 0 rather than a real driver.
 
 =item B I
 
-Make the device at PCI Bus/Device/Function BDF assignable to guests.
+Make the device at B assignable to guests. See
+L for more information.
+
 This will bind the device to the pciback driver and assign it to the
 "quarantine domain".  If it is already bound to a driver, it will
 first be unbound, and the original driver stored so that it can be
@@ -1620,8 +1624,10 @@ being used.
 
 =item B [I<-r>] I
 
-Make the device at PCI Bus/Device/Function BDF not assignable to
-guests.  This will at least unbind the device from pciback, and
+Make the device at B not assignable to guests. See
+L for more information.
+
+This will at least unbind the device from pciback, and
 re-assign it from the "quarantine domain" back to domain 0.  If the -r
 option is specified, it will also attempt to re-bind the device to its
 original driver, making it usable by Domain 0 again.  If the device is
@@ -1637,15 +1643,15 @@ As always, this should only be done if you trust the 
guest, or are
 confident that the particular device you're re-assigning to dom0 will
 cancel all in-flight DMA on FLR.
 
-=item B I I
+=item B I I
 
-Hot-plug a new pass-through pci device to the specified domain.
-B is the PCI Bus/Device/Function of the physical device to pass-through.
+Hot-plug a new pass-through pci device to the specified domain. See
+L for more information.
 
-=item B [I] I I
+=item B [I] I I
 
-Hot-unplug a previously assigned pci device from a domain. B is the PCI
-Bus/Device/Function of the physical device to be removed from the guest domain.
+Hot-unplug a pci device that was previously passed through to a domain. See
+L for more information.
 
 B
 
@@ -1660,7 +1666,7 @@ even without guest domain's collaboration.
 
 =item B I
 
-List pass-through pci devices for a domain.
+List the B of pci devices passed through to a domain.
 
 =back
 
-- 
2.20.1




[PATCH v2 23/24] docs/man: modify xl-pci-configuration(5) to add 'name' field to PCI_SPEC_STRING

2020-11-10 Thread Paul Durrant
From: Paul Durrant 

Since assignable devices can be named, a subsequent patch will support use
of a PCI_SPEC_STRING containing a 'name' parameter instead of a 'bdf'. In
this case the name will be used to look up the 'bdf' in the list of assignable
(or assigned) devices.

Signed-off-by: Paul Durrant 
---
Cc: Ian Jackson 
Cc: Wei Liu 
---
 docs/man/xl-pci-configuration.5.pod | 25 +++--
 1 file changed, 23 insertions(+), 2 deletions(-)

diff --git a/docs/man/xl-pci-configuration.5.pod 
b/docs/man/xl-pci-configuration.5.pod
index 4dd73bc498d6..db3360307cbd 100644
--- a/docs/man/xl-pci-configuration.5.pod
+++ b/docs/man/xl-pci-configuration.5.pod
@@ -51,7 +51,7 @@ is not specified, or if it is specified with an empty value 
(whether
 positionally or explicitly).
 
 B: In context of B (see L), parameters other than
-B will be ignored.
+B or B will be ignored.
 
 =head1 Positional Parameters
 
@@ -70,7 +70,11 @@ B<*> to indicate all functions of a multi-function device.
 
 =item Default Value
 
-None. This parameter is mandatory as it identifies the device.
+None. This parameter is mandatory in its positional form. As a non-positional
+parameter it is also mandatory unless a B parameter is present, in
+which case B must not be present since the B will be used to find
+the B in the list of assignable devices. See L for more information
+on naming assignable devices.
 
 =back
 
@@ -194,4 +198,21 @@ B: This overrides the global B option.
 
 =back
 
+=item B=I
+
+=over 4
+
+=item Description
+
+This is the name given when the B was made assignable. See L for
+more information on naming assignable devices.
+
+=item Default Value
+
+None. This parameter must not be present if a B parameter is present.
+If a B parameter is not present then B is mandatory as it is
+required to look up the B in the list of assignable devices.
+
+=back
+
 =back
-- 
2.20.1




[PATCH v2 01/24] xl / libxl: s/pcidev/pci and remove DEFINE_DEVICE_TYPE_STRUCT_X

2020-11-10 Thread Paul Durrant
From: Paul Durrant 

The seemingly arbitrary use of 'pci' and 'pcidev' in the code in libxl_pci.c
is confusing and also compromises use of some macros used for other device
types. Indeed it seems that DEFINE_DEVICE_TYPE_STRUCT_X exists solely because
of this duality.

This patch purges use of 'pcidev' from the libxl code, allowing evaluation of
DEFINE_DEVICE_TYPE_STRUCT_X to be replaced with DEFINE_DEVICE_TYPE_STRUCT,
hence allowing removal of the former.

For consistency the xl and libs/util code is also modified, but in this case
it is purely cosmetic.

NOTE: Some of the more gross formatting errors (such as lack of spaces after
  keywords) that came into context have been fixed in libxl_pci.c.

Signed-off-by: Paul Durrant 
---
Cc: Ian Jackson 
Cc: Wei Liu 
Cc: Anthony PERARD 
---
 tools/include/libxl.h |  17 +-
 tools/libs/light/libxl_create.c   |   6 +-
 tools/libs/light/libxl_dm.c   |  18 +-
 tools/libs/light/libxl_internal.h |  45 ++-
 tools/libs/light/libxl_pci.c  | 582 +++---
 tools/libs/light/libxl_types.idl  |   2 +-
 tools/libs/util/libxlu_pci.c  |  36 +-
 tools/xl/xl_parse.c   |  26 +-
 tools/xl/xl_pci.c |  68 ++--
 tools/xl/xl_sxp.c |  12 +-
 10 files changed, 408 insertions(+), 404 deletions(-)

diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index 1ea5b4f446e8..fbe4c81ba511 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -444,6 +444,13 @@
  */
 #define LIBXL_HAVE_DISK_SAFE_REMOVE 1
 
+/*
+ * LIBXL_HAVE_CONFIG_PCIS indicates that the 'pcidevs' and 'num_pcidevs'
+ * fields in libxl_domain_config have been renamed to 'pcis' and 'num_pcis'
+ * respectively.
+ */
+#define LIBXL_HAVE_CONFIG_PCIS 1
+
 /*
  * libxl ABI compatibility
  *
@@ -2300,15 +2307,15 @@ int libxl_device_pvcallsif_destroy(libxl_ctx *ctx, 
uint32_t domid,
 
 /* PCI Passthrough */
 int libxl_device_pci_add(libxl_ctx *ctx, uint32_t domid,
- libxl_device_pci *pcidev,
+ libxl_device_pci *pci,
  const libxl_asyncop_how *ao_how)
  LIBXL_EXTERNAL_CALLERS_ONLY;
 int libxl_device_pci_remove(libxl_ctx *ctx, uint32_t domid,
-libxl_device_pci *pcidev,
+libxl_device_pci *pci,
 const libxl_asyncop_how *ao_how)
 LIBXL_EXTERNAL_CALLERS_ONLY;
 int libxl_device_pci_destroy(libxl_ctx *ctx, uint32_t domid,
- libxl_device_pci *pcidev,
+ libxl_device_pci *pci,
  const libxl_asyncop_how *ao_how)
  LIBXL_EXTERNAL_CALLERS_ONLY;
 
@@ -2352,8 +2359,8 @@ int libxl_device_events_handler(libxl_ctx *ctx,
  * added or is not bound, the functions will emit a warning but return
  * SUCCESS.
  */
-int libxl_device_pci_assignable_add(libxl_ctx *ctx, libxl_device_pci *pcidev, 
int rebind);
-int libxl_device_pci_assignable_remove(libxl_ctx *ctx, libxl_device_pci 
*pcidev, int rebind);
+int libxl_device_pci_assignable_add(libxl_ctx *ctx, libxl_device_pci *pci, int 
rebind);
+int libxl_device_pci_assignable_remove(libxl_ctx *ctx, libxl_device_pci *pci, 
int rebind);
 libxl_device_pci *libxl_device_pci_assignable_list(libxl_ctx *ctx, int *num);
 
 /* CPUID handling */
diff --git a/tools/libs/light/libxl_create.c b/tools/libs/light/libxl_create.c
index 321a13e519b5..1f5052c52033 100644
--- a/tools/libs/light/libxl_create.c
+++ b/tools/libs/light/libxl_create.c
@@ -1100,7 +1100,7 @@ int libxl__domain_config_setdefault(libxl__gc *gc,
 goto error_out;
 }
 
-bool need_pt = d_config->num_pcidevs || d_config->num_dtdevs;
+bool need_pt = d_config->num_pcis || d_config->num_dtdevs;
 if (c_info->passthrough == LIBXL_PASSTHROUGH_DEFAULT) {
 c_info->passthrough = need_pt
 ? LIBXL_PASSTHROUGH_ENABLED : LIBXL_PASSTHROUGH_DISABLED;
@@ -1141,7 +1141,7 @@ int libxl__domain_config_setdefault(libxl__gc *gc,
  * assignment when PoD is enabled.
  */
 if (d_config->c_info.type != LIBXL_DOMAIN_TYPE_PV &&
-d_config->num_pcidevs && pod_enabled) {
+d_config->num_pcis && pod_enabled) {
 ret = ERROR_INVAL;
 LOGD(ERROR, domid,
  "PCI device assignment for HVM guest failed due to PoD enabled");
@@ -1817,7 +1817,7 @@ const libxl__device_type *device_type_tbl[] = {
 __vtpm_devtype,
 __usbctrl_devtype,
 __usbdev_devtype,
-__pcidev_devtype,
+__pci_devtype,
 __dtdev_devtype,
 __vdispl_devtype,
 __vsnd_devtype,
diff --git a/tools/libs/light/libxl_dm.c b/tools/libs/light/libxl_dm.c
index 3da83259c08e..8ebe1b60c9d7 100644
--- a/tools/libs/light/libxl_dm.c
+++ b/tools/libs/light/libxl_dm.c
@@ -442,7 +442,7 @@ int libxl__domain_device_construct_rdm(libxl__gc *gc,
 
 /* Might not expose rdm. */
 if (strategy == 

[PATCH v2 07/24] libxl: generalise 'driver_path' xenstore access functions in libxl_pci.c

2020-11-10 Thread Paul Durrant
From: Paul Durrant 

For the purposes of re-binding a device to its previous driver
libxl__device_pci_assignable_add() writes the driver path into xenstore.
This path is then read back in libxl__device_pci_assignable_remove().

The functions that support this writing to and reading from xenstore are
currently dedicated for this purpose and hence the node name 'driver_path'
is hard-coded. This patch generalizes these utility functions and passes
'driver_path' as an argument. Subsequent patches will invoke them to
access other nodes.

NOTE: Because functions will have a broader use (other than storing a
  driver path in lieu of pciback) the base xenstore path is also
  changed from '/libxl/pciback' to '/libxl/pci'.

Signed-off-by: Paul Durrant 
---
Cc: Ian Jackson 
Cc: Wei Liu 
---
 tools/libs/light/libxl_pci.c | 66 +---
 1 file changed, 32 insertions(+), 34 deletions(-)

diff --git a/tools/libs/light/libxl_pci.c b/tools/libs/light/libxl_pci.c
index c9f062fc2d8b..fdafd2c9bafb 100644
--- a/tools/libs/light/libxl_pci.c
+++ b/tools/libs/light/libxl_pci.c
@@ -718,48 +718,46 @@ static int pciback_dev_unassign(libxl__gc *gc, 
libxl_device_pci *pci)
 return 0;
 }
 
-#define PCIBACK_INFO_PATH "/libxl/pciback"
+#define PCI_INFO_PATH "/libxl/pci"
 
-static void pci_assignable_driver_path_write(libxl__gc *gc,
-libxl_device_pci *pci,
-char *driver_path)
+static char *pci_info_xs_path(libxl__gc *gc, libxl_device_pci *pci,
+  const char *node)
 {
-char *path;
+return node ?
+GCSPRINTF(PCI_INFO_PATH"/"PCI_BDF_XSPATH"/%s",
+  pci->domain, pci->bus, pci->dev, pci->func,
+  node) :
+GCSPRINTF(PCI_INFO_PATH"/"PCI_BDF_XSPATH,
+  pci->domain, pci->bus, pci->dev, pci->func);
+}
 
-path = GCSPRINTF(PCIBACK_INFO_PATH"/"PCI_BDF_XSPATH"/driver_path",
- pci->domain,
- pci->bus,
- pci->dev,
- pci->func);
-if ( libxl__xs_printf(gc, XBT_NULL, path, "%s", driver_path) < 0 ) {
-LOGE(WARN, "Write of %s to node %s failed.", driver_path, path);
+
+static void pci_info_xs_write(libxl__gc *gc, libxl_device_pci *pci,
+  const char *node, const char *val)
+{
+char *path = pci_info_xs_path(gc, pci, node);
+
+if ( libxl__xs_printf(gc, XBT_NULL, path, "%s", val) < 0 ) {
+LOGE(WARN, "Write of %s to node %s failed.", val, path);
 }
 }
 
-static char * pci_assignable_driver_path_read(libxl__gc *gc,
-  libxl_device_pci *pci)
+static char *pci_info_xs_read(libxl__gc *gc, libxl_device_pci *pci,
+  const char *node)
 {
-return libxl__xs_read(gc, XBT_NULL,
-  GCSPRINTF(
-   PCIBACK_INFO_PATH "/" PCI_BDF_XSPATH "/driver_path",
-   pci->domain,
-   pci->bus,
-   pci->dev,
-   pci->func));
+char *path = pci_info_xs_path(gc, pci, node);
+
+return libxl__xs_read(gc, XBT_NULL, path);
 }
 
-static void pci_assignable_driver_path_remove(libxl__gc *gc,
-  libxl_device_pci *pci)
+static void pci_info_xs_remove(libxl__gc *gc, libxl_device_pci *pci,
+   const char *node)
 {
+char *path = pci_info_xs_path(gc, pci, node);
 libxl_ctx *ctx = libxl__gc_owner(gc);
 
 /* Remove the xenstore entry */
-xs_rm(ctx->xsh, XBT_NULL,
-  GCSPRINTF(PCIBACK_INFO_PATH "/" PCI_BDF_XSPATH,
-pci->domain,
-pci->bus,
-pci->dev,
-pci->func) );
+xs_rm(ctx->xsh, XBT_NULL, path);
 }
 
 static int libxl__device_pci_assignable_add(libxl__gc *gc,
@@ -805,9 +803,9 @@ static int libxl__device_pci_assignable_add(libxl__gc *gc,
 /* Store driver_path for rebinding to dom0 */
 if ( rebind ) {
 if ( driver_path ) {
-pci_assignable_driver_path_write(gc, pci, driver_path);
+pci_info_xs_write(gc, pci, "driver_path", driver_path);
 } else if ( (driver_path =
- pci_assignable_driver_path_read(gc, pci)) != NULL ) {
+ pci_info_xs_read(gc, pci, "driver_path")) != NULL ) {
 LOG(INFO, PCI_BDF" not bound to a driver, will be rebound to %s",
 dom, bus, dev, func, driver_path);
 } else {
@@ -815,7 +813,7 @@ static int libxl__device_pci_assignable_add(libxl__gc *gc,
 dom, bus, dev, func);
 }
 } else {
-pci_assignable_driver_path_remove(gc, pci);
+pci_info_xs_remove(gc, pci, "driver_path");
 }
 
 if ( pciback_dev_assign(gc, pci) ) {
@@ -865,7 +863,7 @@ static int 

[PATCH v2 09/24] libxl: remove get_all_assigned_devices() from libxl_pci.c

2020-11-10 Thread Paul Durrant
From: Paul Durrant 

Use of this function is a very inefficient way to check whether a device
has already been assigned.

This patch adds code that saves the domain id in xenstore at the point of
assignment, and removes it again when the device id de-assigned (or the
domain is destroyed). It is then straightforward to check whether a device
has been assigned by checking whether a device has a saved domain id.

NOTE: To facilitate the xenstore check it is necessary to move the
  pci_info_xs_read() earlier in libxl_pci.c. To keep related functions
  together, the rest of the pci_info_xs_XXX() functions are moved too.

Signed-off-by: Paul Durrant 
---
Cc: Ian Jackson 
Cc: Wei Liu 
---
 tools/libs/light/libxl_pci.c | 149 +--
 1 file changed, 55 insertions(+), 94 deletions(-)

diff --git a/tools/libs/light/libxl_pci.c b/tools/libs/light/libxl_pci.c
index f9f8374d7d36..ff37dc5b5921 100644
--- a/tools/libs/light/libxl_pci.c
+++ b/tools/libs/light/libxl_pci.c
@@ -317,50 +317,6 @@ retry_transaction2:
 return 0;
 }
 
-static int get_all_assigned_devices(libxl__gc *gc, libxl_device_pci **list, 
int *num)
-{
-char **domlist;
-unsigned int nd = 0, i;
-
-*list = NULL;
-*num = 0;
-
-domlist = libxl__xs_directory(gc, XBT_NULL, "/local/domain", );
-for(i = 0; i < nd; i++) {
-char *path, *num_devs;
-
-path = GCSPRINTF("/local/domain/0/backend/%s/%s/0/num_devs",
- libxl__device_kind_to_string(LIBXL__DEVICE_KIND_PCI),
- domlist[i]);
-num_devs = libxl__xs_read(gc, XBT_NULL, path);
-if ( num_devs ) {
-int ndev = atoi(num_devs), j;
-char *devpath, *bdf;
-
-for(j = 0; j < ndev; j++) {
-devpath = GCSPRINTF("/local/domain/0/backend/%s/%s/0/dev-%u",
-
libxl__device_kind_to_string(LIBXL__DEVICE_KIND_PCI),
-domlist[i], j);
-bdf = libxl__xs_read(gc, XBT_NULL, devpath);
-if ( bdf ) {
-unsigned dom, bus, dev, func;
-if ( sscanf(bdf, PCI_BDF, , , , ) != 4 )
-continue;
-
-*list = realloc(*list, sizeof(libxl_device_pci) * ((*num) 
+ 1));
-if (*list == NULL)
-return ERROR_NOMEM;
-pci_struct_fill(*list + *num, dom, bus, dev, func, 0);
-(*num)++;
-}
-}
-}
-}
-libxl__ptr_add(gc, *list);
-
-return 0;
-}
-
 static int is_pci_in_array(libxl_device_pci *assigned, int num_assigned,
int dom, int bus, int dev, int func)
 {
@@ -408,19 +364,58 @@ static int sysfs_write_bdf(libxl__gc *gc, const char * 
sysfs_path,
 return 0;
 }
 
+#define PCI_INFO_PATH "/libxl/pci"
+
+static char *pci_info_xs_path(libxl__gc *gc, libxl_device_pci *pci,
+  const char *node)
+{
+return node ?
+GCSPRINTF(PCI_INFO_PATH"/"PCI_BDF_XSPATH"/%s",
+  pci->domain, pci->bus, pci->dev, pci->func,
+  node) :
+GCSPRINTF(PCI_INFO_PATH"/"PCI_BDF_XSPATH,
+  pci->domain, pci->bus, pci->dev, pci->func);
+}
+
+
+static int pci_info_xs_write(libxl__gc *gc, libxl_device_pci *pci,
+  const char *node, const char *val)
+{
+char *path = pci_info_xs_path(gc, pci, node);
+int rc = libxl__xs_printf(gc, XBT_NULL, path, "%s", val);
+
+if (rc) LOGE(WARN, "Write of %s to node %s failed.", val, path);
+
+return rc;
+}
+
+static char *pci_info_xs_read(libxl__gc *gc, libxl_device_pci *pci,
+  const char *node)
+{
+char *path = pci_info_xs_path(gc, pci, node);
+
+return libxl__xs_read(gc, XBT_NULL, path);
+}
+
+static void pci_info_xs_remove(libxl__gc *gc, libxl_device_pci *pci,
+   const char *node)
+{
+char *path = pci_info_xs_path(gc, pci, node);
+libxl_ctx *ctx = libxl__gc_owner(gc);
+
+/* Remove the xenstore entry */
+xs_rm(ctx->xsh, XBT_NULL, path);
+}
+
 libxl_device_pci *libxl_device_pci_assignable_list(libxl_ctx *ctx, int *num)
 {
 GC_INIT(ctx);
-libxl_device_pci *pcis = NULL, *new, *assigned;
+libxl_device_pci *pcis = NULL, *new;
 struct dirent *de;
 DIR *dir;
-int r, num_assigned;
 
 *num = 0;
 
-r = get_all_assigned_devices(gc, , _assigned);
-if (r) goto out;
-
 dir = opendir(SYSFS_PCIBACK_DRIVER);
 if (NULL == dir) {
 if (errno == ENOENT) {
@@ -436,9 +431,6 @@ libxl_device_pci 
*libxl_device_pci_assignable_list(libxl_ctx *ctx, int *num)
 if (sscanf(de->d_name, PCI_BDF, , , , ) != 4)
 continue;
 
-if (is_pci_in_array(assigned, num_assigned, dom, bus, dev, func))
-continue;
-
 new = realloc(pcis, ((*num) + 1) * sizeof(*new));
 if 

[PATCH v2 06/24] libxl: stop using aodev->device_config in libxl__device_pci_add()...

2020-11-10 Thread Paul Durrant
From: Paul Durrant 

... to hold a pointer to the device.

There is already a 'pci' field in 'pci_add_state' so simply use that from
the start. This also allows the 'pci' (#3) argument to be dropped from
do_pci_add().

NOTE: This patch also changes the type of the 'pci_domid' field in
  'pci_add_state' from 'int' to 'libxl_domid' which is more appropriate
  given what the field is used for.

Signed-off-by: Paul Durrant 
---
Cc: Ian Jackson 
Cc: Wei Liu 
---
 tools/libs/light/libxl_pci.c | 19 +++
 1 file changed, 7 insertions(+), 12 deletions(-)

diff --git a/tools/libs/light/libxl_pci.c b/tools/libs/light/libxl_pci.c
index 0abc679c3958..c9f062fc2d8b 100644
--- a/tools/libs/light/libxl_pci.c
+++ b/tools/libs/light/libxl_pci.c
@@ -1055,7 +1055,7 @@ typedef struct pci_add_state {
 libxl__ev_qmp qmp;
 libxl__ev_time timeout;
 libxl_device_pci *pci;
-int pci_domid;
+libxl_domid pci_domid;
 } pci_add_state;
 
 static void pci_add_qemu_trad_watch_state_cb(libxl__egc *egc,
@@ -1072,7 +1072,6 @@ static void pci_add_dm_done(libxl__egc *,
 
 static void do_pci_add(libxl__egc *egc,
libxl_domid domid,
-   libxl_device_pci *pci,
pci_add_state *pas)
 {
 STATE_AO_GC(pas->aodev->ao);
@@ -1082,7 +1081,6 @@ static void do_pci_add(libxl__egc *egc,
 /* init pci_add_state */
 libxl__xswait_init(>xswait);
 libxl__ev_qmp_init(>qmp);
-pas->pci = pci;
 pas->pci_domid = domid;
 libxl__ev_time_init(>timeout);
 
@@ -1545,13 +1543,10 @@ void libxl__device_pci_add(libxl__egc *egc, uint32_t 
domid,
 int stubdomid = 0;
 pci_add_state *pas;
 
-/* Store *pci to be used by callbacks */
-aodev->device_config = pci;
-aodev->device_type = __pci_devtype;
-
 GCNEW(pas);
 pas->aodev = aodev;
 pas->domid = domid;
+pas->pci = pci;
 pas->starting = starting;
 pas->callback = device_pci_add_stubdom_done;
 
@@ -1605,9 +1600,10 @@ void libxl__device_pci_add(libxl__egc *egc, uint32_t 
domid,
 GCNEW(pci_s);
 libxl_device_pci_init(pci_s);
 libxl_device_pci_copy(CTX, pci_s, pci);
+pas->pci = pci_s;
 pas->callback = device_pci_add_stubdom_wait;
 
-do_pci_add(egc, stubdomid, pci_s, pas); /* must be last */
+do_pci_add(egc, stubdomid, pas); /* must be last */
 return;
 }
 
@@ -1662,9 +1658,8 @@ static void device_pci_add_stubdom_done(libxl__egc *egc,
 int i;
 
 /* Convenience aliases */
-libxl__ao_device *aodev = pas->aodev;
 libxl_domid domid = pas->domid;
-libxl_device_pci *pci = aodev->device_config;
+libxl_device_pci *pci = pas->pci;
 
 if (rc) goto out;
 
@@ -1699,7 +1694,7 @@ static void device_pci_add_stubdom_done(libxl__egc *egc,
 pci->vdevfn = orig_vdev;
 }
 pas->callback = device_pci_add_done;
-do_pci_add(egc, domid, pci, pas); /* must be last */
+do_pci_add(egc, domid, pas); /* must be last */
 return;
 }
 }
@@ -1715,7 +1710,7 @@ static void device_pci_add_done(libxl__egc *egc,
 EGC_GC;
 libxl__ao_device *aodev = pas->aodev;
 libxl_domid domid = pas->domid;
-libxl_device_pci *pci = aodev->device_config;
+libxl_device_pci *pci = pas->pci;
 
 if (rc) {
 LOGD(ERROR, domid,
-- 
2.20.1




[PATCH v2 04/24] libxl: s/detatched/detached in libxl_pci.c

2020-11-10 Thread Paul Durrant
From: Paul Durrant 

Simply spelling correction. Purely cosmetic fix.

Signed-off-by: Paul Durrant 
---
Cc: Ian Jackson 
Cc: Wei Liu 
---
 tools/libs/light/libxl_pci.c | 22 +++---
 1 file changed, 11 insertions(+), 11 deletions(-)

diff --git a/tools/libs/light/libxl_pci.c b/tools/libs/light/libxl_pci.c
index 515e74fe5aae..52feac651c70 100644
--- a/tools/libs/light/libxl_pci.c
+++ b/tools/libs/light/libxl_pci.c
@@ -1861,7 +1861,7 @@ static void pci_remove_qmp_query_cb(libxl__egc *egc,
 libxl__ev_qmp *qmp, const libxl__json_object *response, int rc);
 static void pci_remove_timeout(libxl__egc *egc,
 libxl__ev_time *ev, const struct timeval *requested_abs, int rc);
-static void pci_remove_detatched(libxl__egc *egc,
+static void pci_remove_detached(libxl__egc *egc,
 pci_remove_state *prs, int rc);
 static void pci_remove_stubdom_done(libxl__egc *egc,
 libxl__ao_device *aodev);
@@ -1975,7 +1975,7 @@ skip1:
 skip_irq:
 rc = 0;
 out_fail:
-pci_remove_detatched(egc, prs, rc); /* must be last */
+pci_remove_detached(egc, prs, rc); /* must be last */
 }
 
 static void pci_remove_qemu_trad_watch_state_cb(libxl__egc *egc,
@@ -1999,7 +1999,7 @@ static void 
pci_remove_qemu_trad_watch_state_cb(libxl__egc *egc,
 rc = qemu_pci_remove_xenstore(gc, domid, pci, prs->force);
 
 out:
-pci_remove_detatched(egc, prs, rc);
+pci_remove_detached(egc, prs, rc);
 }
 
 static void pci_remove_qmp_device_del(libxl__egc *egc,
@@ -2025,7 +2025,7 @@ static void pci_remove_qmp_device_del(libxl__egc *egc,
 return;
 
 out:
-pci_remove_detatched(egc, prs, rc);
+pci_remove_detached(egc, prs, rc);
 }
 
 static void pci_remove_qmp_device_del_cb(libxl__egc *egc,
@@ -2048,7 +2048,7 @@ static void pci_remove_qmp_device_del_cb(libxl__egc *egc,
 return;
 
 out:
-pci_remove_detatched(egc, prs, rc);
+pci_remove_detached(egc, prs, rc);
 }
 
 static void pci_remove_qmp_retry_timer_cb(libxl__egc *egc, libxl__ev_time *ev,
@@ -2064,7 +2064,7 @@ static void pci_remove_qmp_retry_timer_cb(libxl__egc 
*egc, libxl__ev_time *ev,
 return;
 
 out:
-pci_remove_detatched(egc, prs, rc);
+pci_remove_detached(egc, prs, rc);
 }
 
 static void pci_remove_qmp_query_cb(libxl__egc *egc,
@@ -2124,7 +2124,7 @@ static void pci_remove_qmp_query_cb(libxl__egc *egc,
 }
 
 out:
-pci_remove_detatched(egc, prs, rc); /* must be last */
+pci_remove_detached(egc, prs, rc); /* must be last */
 }
 
 static void pci_remove_timeout(libxl__egc *egc, libxl__ev_time *ev,
@@ -2143,12 +2143,12 @@ static void pci_remove_timeout(libxl__egc *egc, 
libxl__ev_time *ev,
 /* If we timed out, we might still want to keep destroying the device
  * (when force==true), so let the next function decide what to do on
  * error */
-pci_remove_detatched(egc, prs, rc);
+pci_remove_detached(egc, prs, rc);
 }
 
-static void pci_remove_detatched(libxl__egc *egc,
- pci_remove_state *prs,
- int rc)
+static void pci_remove_detached(libxl__egc *egc,
+pci_remove_state *prs,
+int rc)
 {
 STATE_AO_GC(prs->aodev->ao);
 int stubdomid = 0;
-- 
2.20.1




[PATCH v2 03/24] libxl: use LIBXL_DEFINE_DEVICE_LIST for nic devices

2020-11-10 Thread Paul Durrant
From: Paul Durrant 

Remove open-coded definitions of libxl_device_nic_list() and
libxl_device_nic_list_free().

Signed-off-by: Paul Durrant 
---
Cc: Ian Jackson 
Cc: Wei Liu 

This patch is slightly tangential. I just happend to notice the inefficiency
while looking at code for various device types.
---
 tools/libs/light/libxl_nic.c | 19 +--
 1 file changed, 1 insertion(+), 18 deletions(-)

diff --git a/tools/libs/light/libxl_nic.c b/tools/libs/light/libxl_nic.c
index 0e5d120ae9a4..a44058f92951 100644
--- a/tools/libs/light/libxl_nic.c
+++ b/tools/libs/light/libxl_nic.c
@@ -403,24 +403,6 @@ static int libxl__nic_from_xenstore(libxl__gc *gc, const 
char *libxl_path,
 return rc;
 }
 
-libxl_device_nic *libxl_device_nic_list(libxl_ctx *ctx, uint32_t domid, int 
*num)
-{
-libxl_device_nic *r;
-
-GC_INIT(ctx);
-
-r = libxl__device_list(gc, __nic_devtype, domid, num);
-
-GC_FREE;
-
-return r;
-}
-
-void libxl_device_nic_list_free(libxl_device_nic* list, int num)
-{
-libxl__device_list_free(__nic_devtype, list, num);
-}
-
 int libxl_device_nic_getinfo(libxl_ctx *ctx, uint32_t domid,
   const libxl_device_nic *nic,
   libxl_nicinfo *nicinfo)
@@ -527,6 +509,7 @@ LIBXL_DEFINE_DEVID_TO_DEVICE(nic)
 LIBXL_DEFINE_DEVICE_ADD(nic)
 LIBXL_DEFINE_DEVICES_ADD(nic)
 LIBXL_DEFINE_DEVICE_REMOVE(nic)
+LIBXL_DEFINE_DEVICE_LIST(nic)
 
 DEFINE_DEVICE_TYPE_STRUCT(nic, VIF,
 .update_config = libxl_device_nic_update_config,
-- 
2.20.1




[ovmf test] 156606: all pass - PUSHED

2020-11-10 Thread osstest service owner
flight 156606 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/156606/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf 0af7f8e6a9253960ba820cd6ddfd8c36543d30cb
baseline version:
 ovmf 1366cd58cd4459f00b4ecf5abed13e77ac4ad06c

Last test of basis   156545  2020-11-07 20:41:45 Z2 days
Testing same since   156606  2020-11-10 00:39:48 Z0 days1 attempts


People who touched revisions under test:
  Mingyue Liang 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 pass
 test-amd64-i386-xl-qemuu-ovmf-amd64  pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   1366cd58cd..0af7f8e6a9  0af7f8e6a9253960ba820cd6ddfd8c36543d30cb -> 
xen-tested-master



[PATCH v2 02/24] libxl: use LIBXL_DEFINE_DEVICE_LIST for pci devices

2020-11-10 Thread Paul Durrant
From: Paul Durrant 

Remove open coded definition of libxl_device_pci_list().

NOTE: Using the macro also defines libxl_device_pci_list_free() so a prototype
  for it is added. Subsequent patches will make used of it.

Signed-off-by: Paul Durrant 
---
Cc: Ian Jackson 
Cc: Wei Liu 
---
 tools/include/libxl.h|  7 +++
 tools/libs/light/libxl_pci.c | 27 ++-
 2 files changed, 9 insertions(+), 25 deletions(-)

diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index fbe4c81ba511..ee52d3cf7e7e 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -451,6 +451,12 @@
  */
 #define LIBXL_HAVE_CONFIG_PCIS 1
 
+/*
+ * LIBXL_HAVE_DEVICE_PCI_LIST_FREE indicates that the
+ * libxl_device_pci_list_free() function is defined.
+ */
+#define LIBXL_HAVE_DEVICE_PCI_LIST_FREE 1
+
 /*
  * libxl ABI compatibility
  *
@@ -2321,6 +2327,7 @@ int libxl_device_pci_destroy(libxl_ctx *ctx, uint32_t 
domid,
 
 libxl_device_pci *libxl_device_pci_list(libxl_ctx *ctx, uint32_t domid,
 int *num);
+void libxl_device_pci_list_free(libxl_device_pci* list, int num);
 
 /*
  * Turns the current process into a backend device service daemon
diff --git a/tools/libs/light/libxl_pci.c b/tools/libs/light/libxl_pci.c
index 2ff1c64a3144..515e74fe5aae 100644
--- a/tools/libs/light/libxl_pci.c
+++ b/tools/libs/light/libxl_pci.c
@@ -2393,31 +2393,6 @@ static int libxl__device_pci_get_num(libxl__gc *gc, 
const char *be_path,
 return rc;
 }
 
-libxl_device_pci *libxl_device_pci_list(libxl_ctx *ctx, uint32_t domid, int 
*num)
-{
-GC_INIT(ctx);
-char *be_path;
-unsigned int n, i;
-libxl_device_pci *pcis = NULL;
-
-*num = 0;
-
-be_path = libxl__domain_device_backend_path(gc, 0, domid, 0,
-LIBXL__DEVICE_KIND_PCI);
-if (libxl__device_pci_get_num(gc, be_path, ))
-goto out;
-
-pcis = calloc(n, sizeof(libxl_device_pci));
-
-for (i = 0; i < n; i++)
-libxl__device_pci_from_xs_be(gc, be_path, i, pcis + i);
-
-*num = n;
-out:
-GC_FREE;
-return pcis;
-}
-
 void libxl__device_pci_destroy_all(libxl__egc *egc, uint32_t domid,
libxl__multidev *multidev)
 {
@@ -2492,6 +2467,8 @@ static int libxl_device_pci_compare(const 
libxl_device_pci *d1,
 return COMPARE_PCI(d1, d2);
 }
 
+LIBXL_DEFINE_DEVICE_LIST(pci)
+
 #define libxl__device_pci_update_devid NULL
 
 DEFINE_DEVICE_TYPE_STRUCT(pci, PCI,
-- 
2.20.1




[PATCH v2 08/24] libxl: remove unnecessary check from libxl__device_pci_add()

2020-11-10 Thread Paul Durrant
From: Paul Durrant 

The code currently checks explicitly whether the device is already assigned,
but this is actually unnecessary as assigned devices do not form part of
the list returned by libxl_device_pci_assignable_list() and hence the
libxl_pci_assignable() test would have already failed.

Signed-off-by: Paul Durrant 
---
Cc: Ian Jackson 
Cc: Wei Liu 
---
 tools/libs/light/libxl_pci.c | 16 +---
 1 file changed, 1 insertion(+), 15 deletions(-)

diff --git a/tools/libs/light/libxl_pci.c b/tools/libs/light/libxl_pci.c
index fdafd2c9bafb..f9f8374d7d36 100644
--- a/tools/libs/light/libxl_pci.c
+++ b/tools/libs/light/libxl_pci.c
@@ -1536,8 +1536,7 @@ void libxl__device_pci_add(libxl__egc *egc, uint32_t 
domid,
 {
 STATE_AO_GC(aodev->ao);
 libxl_ctx *ctx = libxl__gc_owner(gc);
-libxl_device_pci *assigned;
-int num_assigned, rc;
+int rc;
 int stubdomid = 0;
 pci_add_state *pas;
 
@@ -1576,19 +1575,6 @@ void libxl__device_pci_add(libxl__egc *egc, uint32_t 
domid,
 goto out;
 }
 
-rc = get_all_assigned_devices(gc, , _assigned);
-if ( rc ) {
-LOGD(ERROR, domid,
- "cannot determine if device is assigned, refusing to continue");
-goto out;
-}
-if ( is_pci_in_array(assigned, num_assigned, pci->domain,
- pci->bus, pci->dev, pci->func) ) {
-LOGD(ERROR, domid, "PCI device already attached to a domain");
-rc = ERROR_FAIL;
-goto out;
-}
-
 libxl__device_pci_reset(gc, pci->domain, pci->bus, pci->dev, pci->func);
 
 stubdomid = libxl_get_stubdom_id(ctx, domid);
-- 
2.20.1




[PATCH v2 00/24] xl / libxl: named PCI pass-through devices

2020-11-10 Thread Paul Durrant
From: Paul Durrant 

Paul Durrant (24):
  xl / libxl: s/pcidev/pci and remove DEFINE_DEVICE_TYPE_STRUCT_X
  libxl: use LIBXL_DEFINE_DEVICE_LIST for pci devices
  libxl: use LIBXL_DEFINE_DEVICE_LIST for nic devices
  libxl: s/detatched/detached in libxl_pci.c
  libxl: remove extraneous arguments to do_pci_remove() in libxl_pci.c
  libxl: stop using aodev->device_config in libxl__device_pci_add()...
  libxl: generalise 'driver_path' xenstore access functions in
libxl_pci.c
  libxl: remove unnecessary check from libxl__device_pci_add()
  libxl: remove get_all_assigned_devices() from libxl_pci.c
  libxl: make sure callers of libxl_device_pci_list() free the list
after use
  libxl: add libxl_device_pci_assignable_list_free()...
  libxl: use COMPARE_PCI() macro is_pci_in_array()...
  libxl: add/recover 'rdm_policy' to/from PCI backend in xenstore
  libxl: Make sure devices added by pci-attach are reflected in the
config
  docs/man: extract documentation of PCI_SPEC_STRING from the xl.cfg
manpage...
  docs/man: improve documentation of PCI_SPEC_STRING...
  docs/man: fix xl(1) documentation for 'pci' operations
  libxl: introduce 'libxl_pci_bdf' in the idl...
  libxlu: introduce xlu_pci_parse_spec_string()
  libxl: modify
libxl_device_pci_assignable_add/remove/list/list_free()...
  docs/man: modify xl(1) in preparation for naming of assignable devices
  xl / libxl: support naming of assignable devices
  docs/man: modify xl-pci-configuration(5) to add 'name' field to
PCI_SPEC_STRING
  xl / libxl: support 'xl pci-attach/detach' by name

 docs/man/xl-pci-configuration.5.pod  |  218 ++
 docs/man/xl.1.pod.in |   39 +-
 docs/man/xl.cfg.5.pod.in |   68 +-
 tools/golang/xenlight/helpers.gen.go |   77 +-
 tools/golang/xenlight/types.gen.go   |8 +-
 tools/include/libxl.h|   67 +-
 tools/include/libxlutil.h|8 +-
 tools/libs/light/libxl_create.c  |6 +-
 tools/libs/light/libxl_dm.c  |   18 +-
 tools/libs/light/libxl_internal.h|   53 +-
 tools/libs/light/libxl_nic.c |   19 +-
 tools/libs/light/libxl_pci.c | 1030 ++
 tools/libs/light/libxl_types.idl |   19 +-
 tools/libs/util/libxlu_pci.c |  379 +-
 tools/ocaml/libs/xl/xenlight_stubs.c |   19 +-
 tools/xl/xl_cmdtable.c   |   16 +-
 tools/xl/xl_parse.c  |   28 +-
 tools/xl/xl_pci.c|  159 ++--
 tools/xl/xl_sxp.c|   12 +-
 19 files changed, 1308 insertions(+), 935 deletions(-)
 create mode 100644 docs/man/xl-pci-configuration.5.pod
---
Cc: Anthony PERARD 
Cc: Christian Lindig 
Cc: David Scott 
Cc: George Dunlap 
Cc: Ian Jackson 
Cc: Nick Rosbrook 
Cc: Wei Liu 
-- 
2.20.1




[PATCH v2 05/24] libxl: remove extraneous arguments to do_pci_remove() in libxl_pci.c

2020-11-10 Thread Paul Durrant
From: Paul Durrant 

Both 'domid' and 'pci' are available in 'pci_remove_state' so there is no
need to also pass them as separate arguments.

Signed-off-by: Paul Durrant 
---
Cc: Ian Jackson 
Cc: Wei Liu 
---
 tools/libs/light/libxl_pci.c | 9 -
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/tools/libs/light/libxl_pci.c b/tools/libs/light/libxl_pci.c
index 52feac651c70..0abc679c3958 100644
--- a/tools/libs/light/libxl_pci.c
+++ b/tools/libs/light/libxl_pci.c
@@ -1868,14 +1868,14 @@ static void pci_remove_stubdom_done(libxl__egc *egc,
 static void pci_remove_done(libxl__egc *egc,
 pci_remove_state *prs, int rc);
 
-static void do_pci_remove(libxl__egc *egc, uint32_t domid,
-  libxl_device_pci *pci, int force,
-  pci_remove_state *prs)
+static void do_pci_remove(libxl__egc *egc, pci_remove_state *prs)
 {
 STATE_AO_GC(prs->aodev->ao);
 libxl_ctx *ctx = libxl__gc_owner(gc);
 libxl_device_pci *assigned;
+uint32_t domid = prs->domid;
 libxl_domain_type type = libxl__domain_type(gc, domid);
+libxl_device_pci *pci = prs->pci;
 int rc, num;
 uint32_t domainid = domid;
 
@@ -2272,7 +2272,6 @@ static void device_pci_remove_common_next(libxl__egc *egc,
 EGC_GC;
 
 /* Convenience aliases */
-libxl_domid domid = prs->domid;
 libxl_device_pci *const pci = prs->pci;
 libxl__ao_device *const aodev = prs->aodev;
 const unsigned int pfunc_mask = prs->pfunc_mask;
@@ -2290,7 +2289,7 @@ static void device_pci_remove_common_next(libxl__egc *egc,
 } else {
 pci->vdevfn = orig_vdev;
 }
-do_pci_remove(egc, domid, pci, prs->force, prs);
+do_pci_remove(egc, prs);
 return;
 }
 }
-- 
2.20.1




[linux-linus test] 156595: regressions - trouble: broken/fail/pass

2020-11-10 Thread osstest service owner
flight 156595 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/156595/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-xl-credit2  broken
 test-amd64-i386-qemut-rhel6hvm-intel  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 7 xen-install fail REGR. 
vs. 152332
 test-amd64-i386-xl-qemuu-ws16-amd64  7 xen-install   fail REGR. vs. 152332
 test-amd64-i386-xl-xsm7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-intel  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow 7 xen-install fail REGR. vs. 
152332
 test-amd64-i386-xl-qemut-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-examine   6 xen-install  fail REGR. vs. 152332
 test-amd64-i386-libvirt   7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-libvirt-xsm   7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-ws16-amd64  7 xen-install   fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-amd  7 xen-installfail REGR. vs. 152332
 test-amd64-i386-pair 10 xen-install/src_host fail REGR. vs. 152332
 test-amd64-i386-pair 11 xen-install/dst_host fail REGR. vs. 152332
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 
152332
 test-amd64-i386-freebsd10-amd64  7 xen-install   fail REGR. vs. 152332
 test-amd64-i386-xl-raw7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-pvshim 7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-freebsd10-i386  7 xen-installfail REGR. vs. 152332
 test-amd64-i386-xl-shadow 7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-win7-amd64  7 xen-install   fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-win7-amd64  7 xen-install   fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ovmf-amd64  7 xen-install   fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 7 xen-install fail REGR. 
vs. 152332
 test-amd64-i386-libvirt-pair 10 xen-install/src_host fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 11 xen-install/dst_host fail REGR. vs. 152332
 test-amd64-coresched-i386-xl  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-amd  7 xen-installfail REGR. vs. 152332
 test-arm64-arm64-xl   8 xen-boot fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1   8 xen-boot fail REGR. vs. 152332
 test-arm64-arm64-examine 13 examine-iommufail REGR. vs. 152332
 test-amd64-amd64-amd64-pvgrub 20 guest-stop  fail REGR. vs. 152332
 test-amd64-amd64-i386-pvgrub 20 guest-stop   fail REGR. vs. 152332
 test-armhf-armhf-libvirt  8 xen-boot fail REGR. vs. 152332
 test-armhf-armhf-xl-multivcpu  8 xen-bootfail REGR. vs. 152332
 test-armhf-armhf-xl-credit1   8 xen-boot fail REGR. vs. 152332
 test-armhf-armhf-xl-vhd   8 xen-boot fail REGR. vs. 152332
 test-armhf-armhf-libvirt-raw  8 xen-boot fail REGR. vs. 152332
 test-armhf-armhf-xl-cubietruck  8 xen-boot   fail REGR. vs. 152332
 test-armhf-armhf-examine  8 reboot   fail REGR. vs. 152332
 test-armhf-armhf-xl-credit2   8 xen-boot fail REGR. vs. 152332
 test-armhf-armhf-xl   8 xen-boot fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2   8 xen-boot   fail in 156582 REGR. vs. 152332
 test-arm64-arm64-xl-xsm 10 host-ping-check-xen fail in 156582 REGR. vs. 152332

Tests which are failing intermittently (not blocking):
 test-arm64-arm64-libvirt-xsm 10 host-ping-check-xen fail in 156582 pass in 
156595
 test-arm64-arm64-examine  8 reboot   fail in 156582 pass in 156595
 test-arm64-arm64-xl-seattle   8 xen-boot   fail pass in 156582
 test-arm64-arm64-xl-xsm   8 xen-boot   fail pass in 156582

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds  8 xen-boot fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-credit2   5 host-install(5)   broken blocked in 152332
 test-arm64-arm64-libvirt-xsm 11 leak-check/basis(11)fail blocked in 152332
 test-arm64-arm64-xl-seattle 11 leak-check/basis(11) fail in 156582 blocked in 
152332
 test-amd64-amd64-xl-qemut-win7-amd64 19 

Re: [PATCH V2 02/23] xen/ioreq: Make x86's IOREQ feature common

2020-11-10 Thread Oleksandr



On 20.10.20 10:57, Paul Durrant wrote:

Hi Paul

Sorry for the late response.


-Original Message-
From: Oleksandr Tyshchenko 
Sent: 15 October 2020 17:44
To: xen-devel@lists.xenproject.org
Cc: Oleksandr Tyshchenko ; Andrew Cooper 
;
George Dunlap ; Ian Jackson ; 
Jan Beulich
; Julien Grall ; Stefano Stabellini 
; Wei
Liu ; Roger Pau Monné ; Paul Durrant 
; Tim Deegan
; Julien Grall 
Subject: [PATCH V2 02/23] xen/ioreq: Make x86's IOREQ feature common

From: Oleksandr Tyshchenko 

As a lot of x86 code can be re-used on Arm later on, this patch
moves previously prepared x86/hvm/ioreq.c to the common code.

The common IOREQ feature is supposed to be built with IOREQ_SERVER
option enabled, which is selected for x86's config HVM for now.

In order to avoid having a gigantic patch here, the subsequent
patches will update remaining bits in the common code step by step:
- Make IOREQ related structs/materials common
- Drop the "hvm" prefixes and infixes

FWIW you could tackle the naming changes in patch #1.
Unfortunately, there are a lot of places that need touching (in order to 
replace/drop hvm prefixes), so I decided to keep that in a separate patch.
From the review comments on previous series I got that renaming could 
be done before or after the move, but within

a single series, so I chose the latter.


The 'legacy' mechanism of mapping magic pages for ioreq servers should remain 
x86 specific I think that aspect of the code needs to remain behind and not get 
moved into common code. You could do that in arch specific calls in 
hvm_ioreq_server_enable/disable() and hvm_get_ioreq_server_info().
Well, if legacy mechanism is not going to be used for other arch and 
should remain x86 specific, I will try to investigate what should be 
left in x86 code and rework the series.
As a side note, I am afraid, we won't get a 100% code movement (which I 
managed to achieve here) for the next version of this patch as we need 
arch/x86/hvm/ioreq.c.



--
Regards,

Oleksandr Tyshchenko




[xen-unstable-smoke test] 156622: tolerable all pass - PUSHED

2020-11-10 Thread osstest service owner
flight 156622 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/156622/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt 15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  16 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  3059178798a23ba870ff86ff54d442a07e6651fc
baseline version:
 xen  0a5e0ce0fb7e5a3b5dfdc936058d2c0e04e5e258

Last test of basis   156532  2020-11-06 17:01:35 Z3 days
Testing same since   156622  2020-11-10 13:01:19 Z0 days1 attempts


People who touched revisions under test:
  Andrew Cooper 
  Roger Pau Monné 

jobs:
 build-arm64-xsm  pass
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  pass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   0a5e0ce0fb..3059178798  3059178798a23ba870ff86ff54d442a07e6651fc -> smoke



[PATCH] x86/CPUID: adjust extended leaves out of range clearing

2020-11-10 Thread Jan Beulich
A maximum extended leaf input value with the high half different from
0x8000 should not be considered valid - all leaves should be cleared in
this case.

Signed-off-by: Jan Beulich 

--- a/tools/tests/cpu-policy/test-cpu-policy.c
+++ b/tools/tests/cpu-policy/test-cpu-policy.c
@@ -516,11 +516,22 @@ static void test_cpuid_out_of_range_clea
 },
 },
 {
+.name = "no extd",
+.nr_markers = 0,
+.p = {
+/* Clears all markers. */
+.extd.max_leaf = 0,
+
+.extd.vendor_ebx = 0xc2,
+.extd.raw_fms = 0xc2,
+},
+},
+{
 .name = "extd",
 .nr_markers = 1,
 .p = {
 /* Retains marker in leaf 0.  Clears others. */
-.extd.max_leaf = 0,
+.extd.max_leaf = 0x8000,
 .extd.vendor_ebx = 0xc2,
 
 .extd.raw_fms = 0xc2,
--- a/xen/lib/x86/cpuid.c
+++ b/xen/lib/x86/cpuid.c
@@ -232,7 +232,9 @@ void x86_cpuid_policy_clear_out_of_range
 ARRAY_SIZE(p->xstate.raw) - 1);
 }
 
-zero_leaves(p->extd.raw, (p->extd.max_leaf & 0x) + 1,
+zero_leaves(p->extd.raw,
+((p->extd.max_leaf >> 16) == 0x8000
+ ? (p->extd.max_leaf & 0x) + 1 : 0),
 ARRAY_SIZE(p->extd.raw) - 1);
 }
 



[xen-unstable bisection] complete test-amd64-amd64-libvirt-xsm

2020-11-10 Thread osstest service owner
branch xen-unstable
xenbranch xen-unstable
job test-amd64-amd64-libvirt-xsm
testid guest-start

Tree: libvirt git://xenbits.xen.org/libvirt.git
Tree: libvirt_keycodemapdb https://gitlab.com/keycodemap/keycodemapdb.git
Tree: linux git://xenbits.xen.org/linux-pvops.git
Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: xen git://xenbits.xen.org/xen.git

*** Found and reproduced problem changeset ***

  Bug is in tree:  xen git://xenbits.xen.org/xen.git
  Bug introduced:  e19bcb626f50a652fb1854a8b2f2c9c371687a11
  Bug not present: c3453a23f7905d24f2404787543e26ec7d02301c
  Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/156624/


  commit e19bcb626f50a652fb1854a8b2f2c9c371687a11
  Author: Juergen Gross 
  Date:   Fri Nov 6 10:48:07 2020 +0100
  
  xen/rwlock: add check_lock() handling to rwlocks
  
  Checking whether a lock is consistently used regarding interrupts on
  or off is beneficial for rwlocks, too.
  
  So add check_lock() calls to rwlock functions. For this purpose make
  check_lock() globally accessible.
  
  Signed-off-by: Juergen Gross 
  Reviewed-by: Julien Grall 
  Reviewed-by: Jan Beulich 


For bisection revision-tuple graph see:
   
http://logs.test-lab.xenproject.org/osstest/results/bisect/xen-unstable/test-amd64-amd64-libvirt-xsm.guest-start.html
Revision IDs in each graph node refer, respectively, to the Trees above.


Running cs-bisection-step 
--graph-out=/home/logs/results/bisect/xen-unstable/test-amd64-amd64-libvirt-xsm.guest-start
 --summary-out=tmp/156624.bisection-summary --basis-template=156443 
--blessings=real,real-bisect,real-retry xen-unstable 
test-amd64-amd64-libvirt-xsm guest-start
Searching for failure / basis pass:
 156588 fail [host=chardonnay0] / 156443 [host=elbling0] 156401 [host=fiano0] 
156389 [host=albana0] 156373 [host=godello0] 156354 [host=godello1] 156339 
[host=godello1] 156331 [host=huxelrebe0] 156315 [host=albana1] 156291 
[host=pinot0] 156268 [host=elbling1] 156254 [host=godello0] 156248 
[host=pinot1] 156228 [host=huxelrebe1] 156196 [host=fiano1] 156167 
[host=godello1] 156136 [host=chardonnay1] 156119 [host=albana1] 156112 
[host=godello0] 156093 ok.
Failure / basis pass flights: 156588 / 156093
(tree with no url: minios)
(tree with no url: ovmf)
(tree with no url: seabios)
Tree: libvirt git://xenbits.xen.org/libvirt.git
Tree: libvirt_keycodemapdb https://gitlab.com/keycodemap/keycodemapdb.git
Tree: linux git://xenbits.xen.org/linux-pvops.git
Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: xen git://xenbits.xen.org/xen.git
Latest 2c846fa6bcc11929c9fb857a22430fb9945654ad 
27acf0ef828bf719b2053ba398b195829413dbdd 
c3038e718a19fc596f7b1baba0f83d5146dc7784 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 
7ea428895af2840d85c524f0bd11a38aac308308 
0a5e0ce0fb7e5a3b5dfdc936058d2c0e04e5e258
Basis pass 2c846fa6bcc11929c9fb857a22430fb9945654ad 
27acf0ef828bf719b2053ba398b195829413dbdd 
c3038e718a19fc596f7b1baba0f83d5146dc7784 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 
ea6d3cd1ed79d824e605a70c3626bc437c386260 
3b49791e4cc2f38dd84bf331b75217adaef636e3
Generating revisions with ./adhoc-revtuple-generator  
git://xenbits.xen.org/libvirt.git#2c846fa6bcc11929c9fb857a22430fb9945654ad-2c846fa6bcc11929c9fb857a22430fb9945654ad
 
https://gitlab.com/keycodemap/keycodemapdb.git#27acf0ef828bf719b2053ba398b195829413dbdd-27acf0ef828bf719b2053ba398b195829413dbdd
 
git://xenbits.xen.org/linux-pvops.git#c3038e718a19fc596f7b1baba0f83d5146dc7784-c3038e718a19fc596f7b1baba0f83d5146dc7784
 
git://xenbits.xen.org/osstest/linux-firmware.git#c530a75c1e6a472b0eb9558310b518f0\
 dfcd8860-c530a75c1e6a472b0eb9558310b518f0dfcd8860 
git://xenbits.xen.org/qemu-xen-traditional.git#3d273dd05e51e5a1ffba3d98c7437ee84e8f8764-3d273dd05e51e5a1ffba3d98c7437ee84e8f8764
 
git://xenbits.xen.org/qemu-xen.git#ea6d3cd1ed79d824e605a70c3626bc437c386260-7ea428895af2840d85c524f0bd11a38aac308308
 
git://xenbits.xen.org/xen.git#3b49791e4cc2f38dd84bf331b75217adaef636e3-0a5e0ce0fb7e5a3b5dfdc936058d2c0e04e5e258
Loaded 41874 nodes in revision graph
Searching for test results:
 156050 [host=elbling0]
 156085 []
 156093 pass 2c846fa6bcc11929c9fb857a22430fb9945654ad 
27acf0ef828bf719b2053ba398b195829413dbdd 
c3038e718a19fc596f7b1baba0f83d5146dc7784 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 
ea6d3cd1ed79d824e605a70c3626bc437c386260 
3b49791e4cc2f38dd84bf331b75217adaef636e3
 156112 [host=godello0]
 156119 [host=albana1]
 156136 [host=chardonnay1]
 156167 [host=godello1]
 156196 [host=fiano1]
 156228 [host=huxelrebe1]
 156248 [host=pinot1]
 156254 

Re: [PATCH 5/5] x86/p2m: split write_p2m_entry() hook

2020-11-10 Thread Jan Beulich
On 10.11.2020 14:59, Roger Pau Monné wrote:
> On Wed, Oct 28, 2020 at 10:24:53AM +0100, Jan Beulich wrote:
>> Fair parts of the present handlers are identical; in fact
>> nestedp2m_write_p2m_entry() lacks a call to p2m_entry_modify(). Move
>> common parts right into write_p2m_entry(), splitting the hooks into a
>> "pre" one (needed just by shadow code) and a "post" one.
>>
>> For the common parts moved I think that the p2m_flush_nestedp2m() is,
>> at least from an abstract perspective, also applicable in the shadow
>> case. Hence it doesn't get a 3rd hook put in place.
>>
>> The initial comment that was in hap_write_p2m_entry() gets dropped: Its
>> placement was bogus, and looking back the the commit introducing it
>> (dd6de3ab9985 "Implement Nested-on-Nested") I can't see either what use
>> of a p2m it was meant to be associated with.
> 
> Is there any performance implication of moving from one hook to two
> hooks? Since this shouldn't be a hot path I assume it's fine.

Well, first of all just a couple of patches ago two indirect
calls were folded into one, so it's at least not getting worse
compared to where we started from. And then both HAP and nested
install just one of the two hooks.

As per the remark in an earlier patch, referred to ...

>> Signed-off-by: Jan Beulich 
>> ---
>> RFC: This is effectively the alternative to the suggestion in an earlier
>>  patch that we might do away with the hook altogether. Of course a
>>  hybrid approach would also be possible, by using direct calls here
>>  instead of splitting the hook into two.

... here, there is the option of doing away with the indirect
calls altogether, but - as said earlier - at the price of at
least coming close to a layering violation.

>> --- a/xen/arch/x86/mm/hap/nested_hap.c
>> +++ b/xen/arch/x86/mm/hap/nested_hap.c
>> @@ -71,24 +71,11 @@
>>  /*NESTED VIRT P2M FUNCTIONS */
>>  //
>>  
>> -int
>> -nestedp2m_write_p2m_entry(struct p2m_domain *p2m, unsigned long gfn,
>> -l1_pgentry_t *p, l1_pgentry_t new, unsigned int level)
>> +void
>> +nestedp2m_write_p2m_entry_post(struct p2m_domain *p2m, unsigned int oflags)
>>  {
>> -struct domain *d = p2m->domain;
>> -uint32_t old_flags;
>> -
>> -paging_lock(d);
>> -
>> -old_flags = l1e_get_flags(*p);
>> -safe_write_pte(p, new);
>> -
>> -if (old_flags & _PAGE_PRESENT)
>> -guest_flush_tlb_mask(d, p2m->dirty_cpumask);
>> -
>> -paging_unlock(d);
>> -
>> -return 0;
>> +if ( oflags & _PAGE_PRESENT )
>> +guest_flush_tlb_mask(p2m->domain, p2m->dirty_cpumask);
>>  }
> 
> This is a verbatimi copy of hap_write_p2m_entry_post. I assume there's
> a reason why we need both, but I'm missing it.

Only almost, since HAP has

if ( oflags & _PAGE_PRESENT )
guest_flush_tlb_mask(d, d->dirty_cpumask);

instead (i.e. they differ in which dirty_cpumask gets used).

>> --- a/xen/arch/x86/mm/p2m-pt.c
>> +++ b/xen/arch/x86/mm/p2m-pt.c
>> @@ -122,17 +122,55 @@ static int write_p2m_entry(struct p2m_do
>>  {
>>  struct domain *d = p2m->domain;
>>  struct vcpu *v = current;
>> -int rc = 0;
>>  
>>  if ( v->domain != d )
>>  v = d->vcpu ? d->vcpu[0] : NULL;
>>  if ( likely(v && paging_mode_enabled(d) && paging_get_hostmode(v)) ||
>>   p2m_is_nestedp2m(p2m) )
>> -rc = p2m->write_p2m_entry(p2m, gfn, p, new, level);
>> +{
>> +unsigned int oflags;
>> +mfn_t omfn;
>> +int rc;
>> +
>> +paging_lock(d);
>> +
>> +if ( p2m->write_p2m_entry_pre )
>> +p2m->write_p2m_entry_pre(d, gfn, p, new, level);
>> +
>> +oflags = l1e_get_flags(*p);
>> +omfn = l1e_get_mfn(*p);
>> +
>> +rc = p2m_entry_modify(p2m, p2m_flags_to_type(l1e_get_flags(new)),
>> +  p2m_flags_to_type(oflags), l1e_get_mfn(new),
>> +  omfn, level);
>> +if ( rc )
>> +{
>> +paging_unlock(d);
>> +return rc;
>> +}
>> +
>> +safe_write_pte(p, new);
>> +
>> +if ( p2m->write_p2m_entry_post )
>> +p2m->write_p2m_entry_post(p2m, oflags);
>> +
>> +paging_unlock(d);
>> +
>> +if ( nestedhvm_enabled(d) && !p2m_is_nestedp2m(p2m) &&
>> + (oflags & _PAGE_PRESENT) &&
>> + !p2m_get_hostp2m(d)->defer_nested_flush &&
>> + /*
>> +  * We are replacing a valid entry so we need to flush nested 
>> p2ms,
>> +  * unless the only change is an increase in access rights.
>> +  */
>> + (!mfn_eq(omfn, l1e_get_mfn(new)) ||
>> +  !perms_strictly_increased(oflags, l1e_get_flags(new))) )
>> +p2m_flush_nestedp2m(d);
> 
> It feel slightly weird to have a nested p2m hook post, and yet have
> nested specific code here.
> 
> Have you considered if the post hook could be moved outside of the
> locked region, so 

UNSUB

2020-11-10 Thread Scott Davis
UNSUB xen-devel


Re: [PATCH v3 5/7] x86: guard against straight-line speculation past RET

2020-11-10 Thread Jan Beulich
On 10.11.2020 15:08, Roger Pau Monné wrote:
> On Tue, Nov 10, 2020 at 02:19:40PM +0100, Jan Beulich wrote:
>> On 10.11.2020 12:16, Roger Pau Monné wrote:
>>> On Tue, Nov 10, 2020 at 11:06:46AM +0100, Jan Beulich wrote:
 On 10.11.2020 10:31, Roger Pau Monné wrote:
> On Fri, Oct 23, 2020 at 10:38:04AM +0200, Jan Beulich wrote:
>> Under certain conditions CPUs can speculate into the instruction stream
>> past a RET instruction. Guard against this just like 3b7dab93f240
>> ("x86/spec-ctrl: Protect against CALL/JMP straight-line speculation")
>> did - by inserting an "INT $3" insn. It's merely the mechanics of how to
>> achieve this that differ: A set of macros gets introduced to post-
>> process RET insns issued by the compiler (or living in assembly files).
>>
>> Unfortunately for clang this requires further features their built-in
>> assembler doesn't support: We need to be able to override insn mnemonics
>> produced by the compiler (which may be impossible, if internally
>> assembly mnemonics never get generated), and we want to use \(text)
>> escaping / quoting in the auxiliary macro.
>
> Could this have an option to enable/disable at build time?

 Well, a subsequent patch adds a config option for this, which in
 the worst case could be turned off. I'm afraid though I'm not
 clear about the question, because ...

> FreeBSD will drop GNU as quite soon from base, and albeit it can be
> installed as a package I would like to be able to build Xen using a
> toolchain based on LLVM exclusively.

 ... it's not clear to me what the implications here are: Are you
 saying -no-integrated-as is not going to function anymore, unless
 people explicitly install gas? If that's not what you meant to
 indicate, then I don't see how building would become impossible.
>>>
>>> I'm still inquiring about this, but I would say that when gas is
>>> removed from FreeBSD then the 'as' command would be mapped to llvm-as,
>>> and thus -no-integrated-as would hit the same issues as the integrated
>>> as. So far in Xen we have assumed that -no-integrated-as would
>>> fallback to an as capable of doing what the integrated clang as
>>> doesn't support, but that might not be the case.
>>
>> At which point Xen couldn't be built anyway, I expect. If llvm-as
>> isn't sufficiently gas-compatible, we've lost (right now at least).
>>
>>> Ideally we would have to re-run the tests with -no-integrated-as, in
>>> order to assert that the external as is really capable of what the
>>> internal one is not.
>>
>> And if it doesn't, what would we do other than failing the build
>> (which it would also if we didn't do the 2nd round of checks)?
> 
> I would always prefer a clear message (ie: your toolstack is not
> capable of building Xen) rather than a weird build time failure.

Fair point in general.

> Also we could maybe disable certain options by default if the
> toolstack doesn't have the required support to build them?

We could, but I'm afraid this will go down the route of embedding
tool chain capabilities in xen/.config, which I continue to not
consider a good idea (and the thread got stalled, as expected).

In fact (also to Andrew and Anthony), recently I've become aware
of another shortcoming of this model: Our kernel packages contain
.config files for the various architectures and specific per-
architecture flavors. It used to be easy to update them on any
system, until the tool chain capability checks got introduced.
Now, in order to update them, one has to use the precise versions
of the various tool chain parts that will be used on the build
hosts, or else an error may result (for unexpected changes to
the file), or one may unknowingly turn off options that are
expected to be enabled.

Put more generally - if I handed someone a specific .config, I'd
expect their resulting binary to contain what I did set up. Or
for them to report back that they can't build the thing. But it
should not be the case that the .config got silently changed and
certain functionality disabled just because they use a different
tool chain.

> Has anyone reported this shortcoming to upstream llvm, so they are
> aware and can work on this or maybe provide an alternative way to
> achieve the same result?

I didn't and I'm unaware of anyone else possibly having done so.
That said, I consider it sort of obvious though that the goal of
replacing the GNU tool chain implies being fully compatible (and
presumably better in certain areas).

Jan



Re: [PATCH v3 5/7] x86: guard against straight-line speculation past RET

2020-11-10 Thread Roger Pau Monné
On Tue, Nov 10, 2020 at 02:19:40PM +0100, Jan Beulich wrote:
> On 10.11.2020 12:16, Roger Pau Monné wrote:
> > On Tue, Nov 10, 2020 at 11:06:46AM +0100, Jan Beulich wrote:
> >> On 10.11.2020 10:31, Roger Pau Monné wrote:
> >>> On Fri, Oct 23, 2020 at 10:38:04AM +0200, Jan Beulich wrote:
>  Under certain conditions CPUs can speculate into the instruction stream
>  past a RET instruction. Guard against this just like 3b7dab93f240
>  ("x86/spec-ctrl: Protect against CALL/JMP straight-line speculation")
>  did - by inserting an "INT $3" insn. It's merely the mechanics of how to
>  achieve this that differ: A set of macros gets introduced to post-
>  process RET insns issued by the compiler (or living in assembly files).
> 
>  Unfortunately for clang this requires further features their built-in
>  assembler doesn't support: We need to be able to override insn mnemonics
>  produced by the compiler (which may be impossible, if internally
>  assembly mnemonics never get generated), and we want to use \(text)
>  escaping / quoting in the auxiliary macro.
> >>>
> >>> Could this have an option to enable/disable at build time?
> >>
> >> Well, a subsequent patch adds a config option for this, which in
> >> the worst case could be turned off. I'm afraid though I'm not
> >> clear about the question, because ...
> >>
> >>> FreeBSD will drop GNU as quite soon from base, and albeit it can be
> >>> installed as a package I would like to be able to build Xen using a
> >>> toolchain based on LLVM exclusively.
> >>
> >> ... it's not clear to me what the implications here are: Are you
> >> saying -no-integrated-as is not going to function anymore, unless
> >> people explicitly install gas? If that's not what you meant to
> >> indicate, then I don't see how building would become impossible.
> > 
> > I'm still inquiring about this, but I would say that when gas is
> > removed from FreeBSD then the 'as' command would be mapped to llvm-as,
> > and thus -no-integrated-as would hit the same issues as the integrated
> > as. So far in Xen we have assumed that -no-integrated-as would
> > fallback to an as capable of doing what the integrated clang as
> > doesn't support, but that might not be the case.
> 
> At which point Xen couldn't be built anyway, I expect. If llvm-as
> isn't sufficiently gas-compatible, we've lost (right now at least).
> 
> > Ideally we would have to re-run the tests with -no-integrated-as, in
> > order to assert that the external as is really capable of what the
> > internal one is not.
> 
> And if it doesn't, what would we do other than failing the build
> (which it would also if we didn't do the 2nd round of checks)?

I would always prefer a clear message (ie: your toolstack is not
capable of building Xen) rather than a weird build time failure.

Also we could maybe disable certain options by default if the
toolstack doesn't have the required support to build them?

Has anyone reported this shortcoming to upstream llvm, so they are
aware and can work on this or maybe provide an alternative way to
achieve the same result?

Thanks, Roger.



Re: [PATCH 3/5] x86/p2m: suppress audit_p2m hook when possible

2020-11-10 Thread Roger Pau Monné
On Tue, Nov 10, 2020 at 02:21:43PM +0100, Jan Beulich wrote:
> On 10.11.2020 12:30, Roger Pau Monné wrote:
> > On Wed, Oct 28, 2020 at 10:23:42AM +0100, Jan Beulich wrote:
> >> When P2M_AUDIT is false, it's unused, so instead of having a dangling
> >> NULL pointer sit there, omit the field altogether.
> >>
> >> Instead of adding "#if P2M_AUDIT && defined(CONFIG_HVM)" in even more
> >> places, fold the latter part right into the definition of P2M_AUDIT.
> >>
> >> Signed-off-by: Jan Beulich 
> > 
> > Acked-by: Roger Pau Monné 
> 
> Thanks. Since I see you keep replying to v1, are you aware of
> there being v2? For this particular patch there actually is a
> clang related fix in v2, which may be of interest to you.

Urg, no sorry. I was just going over my mail queue and didn't realize
there was a v2. Will take a look.

Roger.



Re: [PATCH 5/5] x86/p2m: split write_p2m_entry() hook

2020-11-10 Thread Roger Pau Monné
On Wed, Oct 28, 2020 at 10:24:53AM +0100, Jan Beulich wrote:
> Fair parts of the present handlers are identical; in fact
> nestedp2m_write_p2m_entry() lacks a call to p2m_entry_modify(). Move
> common parts right into write_p2m_entry(), splitting the hooks into a
> "pre" one (needed just by shadow code) and a "post" one.
> 
> For the common parts moved I think that the p2m_flush_nestedp2m() is,
> at least from an abstract perspective, also applicable in the shadow
> case. Hence it doesn't get a 3rd hook put in place.
> 
> The initial comment that was in hap_write_p2m_entry() gets dropped: Its
> placement was bogus, and looking back the the commit introducing it
> (dd6de3ab9985 "Implement Nested-on-Nested") I can't see either what use
> of a p2m it was meant to be associated with.

Is there any performance implication of moving from one hook to two
hooks? Since this shouldn't be a hot path I assume it's fine.

> Signed-off-by: Jan Beulich 
> ---
> RFC: This is effectively the alternative to the suggestion in an earlier
>  patch that we might do away with the hook altogether. Of course a
>  hybrid approach would also be possible, by using direct calls here
>  instead of splitting the hook into two.
> ---
> I'm unsure whether p2m_init_nestedp2m() zapping the "pre" hook is
> actually correct, or whether previously the sh_unshadow_for_p2m_change()
> invocation was wrongly skipped.
> 
> --- a/xen/arch/x86/mm/hap/hap.c
> +++ b/xen/arch/x86/mm/hap/hap.c
> @@ -774,55 +774,18 @@ static void hap_update_paging_modes(stru
>  put_gfn(d, cr3_gfn);
>  }
>  
> -static int
> -hap_write_p2m_entry(struct p2m_domain *p2m, unsigned long gfn, l1_pgentry_t 
> *p,
> -l1_pgentry_t new, unsigned int level)
> +static void
> +hap_write_p2m_entry_post(struct p2m_domain *p2m, unsigned int oflags)
>  {
>  struct domain *d = p2m->domain;
> -uint32_t old_flags;
> -mfn_t omfn;
> -int rc;
>  
> -/* We know always use the host p2m here, regardless if the vcpu
> - * is in host or guest mode. The vcpu can be in guest mode by
> - * a hypercall which passes a domain and chooses mostly the first
> - * vcpu. */
> -
> -paging_lock(d);
> -old_flags = l1e_get_flags(*p);
> -omfn = l1e_get_mfn(*p);
> -
> -rc = p2m_entry_modify(p2m, p2m_flags_to_type(l1e_get_flags(new)),
> -  p2m_flags_to_type(old_flags), l1e_get_mfn(new),
> -  omfn, level);
> -if ( rc )
> -{
> -paging_unlock(d);
> -return rc;
> -}
> -
> -safe_write_pte(p, new);
> -if ( old_flags & _PAGE_PRESENT )
> +if ( oflags & _PAGE_PRESENT )
>  guest_flush_tlb_mask(d, d->dirty_cpumask);
> -
> -paging_unlock(d);
> -
> -if ( nestedhvm_enabled(d) && (old_flags & _PAGE_PRESENT) &&
> - !p2m_get_hostp2m(d)->defer_nested_flush &&
> - /*
> -  * We are replacing a valid entry so we need to flush nested p2ms,
> -  * unless the only change is an increase in access rights.
> -  */
> - (!mfn_eq(omfn, l1e_get_mfn(new)) ||
> -  !perms_strictly_increased(old_flags, l1e_get_flags(new))) )
> -p2m_flush_nestedp2m(d);
> -
> -return 0;
>  }
>  
>  void hap_p2m_init(struct p2m_domain *p2m)
>  {
> -p2m->write_p2m_entry = hap_write_p2m_entry;
> +p2m->write_p2m_entry_post = hap_write_p2m_entry_post;
>  }
>  
>  static unsigned long hap_gva_to_gfn_real_mode(
> --- a/xen/arch/x86/mm/hap/nested_hap.c
> +++ b/xen/arch/x86/mm/hap/nested_hap.c
> @@ -71,24 +71,11 @@
>  /*NESTED VIRT P2M FUNCTIONS */
>  //
>  
> -int
> -nestedp2m_write_p2m_entry(struct p2m_domain *p2m, unsigned long gfn,
> -l1_pgentry_t *p, l1_pgentry_t new, unsigned int level)
> +void
> +nestedp2m_write_p2m_entry_post(struct p2m_domain *p2m, unsigned int oflags)
>  {
> -struct domain *d = p2m->domain;
> -uint32_t old_flags;
> -
> -paging_lock(d);
> -
> -old_flags = l1e_get_flags(*p);
> -safe_write_pte(p, new);
> -
> -if (old_flags & _PAGE_PRESENT)
> -guest_flush_tlb_mask(d, p2m->dirty_cpumask);
> -
> -paging_unlock(d);
> -
> -return 0;
> +if ( oflags & _PAGE_PRESENT )
> +guest_flush_tlb_mask(p2m->domain, p2m->dirty_cpumask);
>  }

This is a verbatimi copy of hap_write_p2m_entry_post. I assume there's
a reason why we need both, but I'm missing it.

>  
>  //
> --- a/xen/arch/x86/mm/p2m-pt.c
> +++ b/xen/arch/x86/mm/p2m-pt.c
> @@ -122,17 +122,55 @@ static int write_p2m_entry(struct p2m_do
>  {
>  struct domain *d = p2m->domain;
>  struct vcpu *v = current;
> -int rc = 0;
>  
>  if ( v->domain != d )
>  v = d->vcpu ? d->vcpu[0] : NULL;
>  if ( likely(v && paging_mode_enabled(d) && paging_get_hostmode(v)) ||
>   p2m_is_nestedp2m(p2m) )
> -rc = p2m->write_p2m_entry(p2m, gfn, p, new, level);
> +{
> +

Re: [PATCH 2/5] x86/p2m: collapse the two ->write_p2m_entry() hooks

2020-11-10 Thread Jan Beulich
On 10.11.2020 12:06, Roger Pau Monné wrote:
> On Wed, Oct 28, 2020 at 10:22:58AM +0100, Jan Beulich wrote:
>> @@ -1132,7 +1132,13 @@ void p2m_pt_init(struct p2m_domain *p2m)
>>  p2m->recalc = do_recalc;
>>  p2m->change_entry_type_global = p2m_pt_change_entry_type_global;
>>  p2m->change_entry_type_range = p2m_pt_change_entry_type_range;
>> -p2m->write_p2m_entry = write_p2m_entry;
>> +
>> +/* Still too early to use paging_mode_hap(). */
>> +if ( hap_enabled(p2m->domain) )
>> +hap_p2m_init(p2m);
>> +else if ( IS_ENABLED(CONFIG_SHADOW_PAGING) )
>> +shadow_p2m_init(p2m);
> 
> There's already some logic in p2m_initialise that checks for
> hap_enabled for EPT specific initialization. Do you think you could
> move this there so that it's more contained?
> 
> I think having the initialization condition sprinkled all over the
> different functions makes the logic more complicated to follow.
> 
> Also, should hap_p2m_init be limited to HAP and PT, as opposed to HAP
> and EPT which doesn't use the helper AFAICT?

It is limited to HAP and PT - we're in p2m_pt_init() here. This is
also why I don't want to move it to e.g. p2m_initialise(), as that
would be the wrong layer.

> Maybe it would be clearer to unify shadow_write_p2m_entry with
> hap_write_p2m_entry and call it p2m_pt_write_p2m_entry to match the
> rest of the p2m PT helpers?

This looks to go along the lines of what I'd put up as a post-
commit-message remark in "x86/p2m: collapse the two
->write_p2m_entry() hooks". The nested handler is perhaps the
bigger problem with such merging, plus it would feel a little like
a layering violation (which is why I did put up the question
instead of doing it right away).

Jan



[xen-4.12-testing test] 156592: tolerable FAIL - PUSHED

2020-11-10 Thread osstest service owner
flight 156592 xen-4.12-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/156592/

Failures :-/ but no regressions.

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds 13 debian-fixup fail REGR. vs. 156515

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qcow219 guest-localmigrate/x10   fail  like 156515
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop fail like 156515
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stopfail like 156515
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stopfail like 156515
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop fail like 156515
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stopfail like 156515
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stopfail like 156515
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop fail like 156515
 test-armhf-armhf-libvirt-raw 15 saverestore-support-checkfail  like 156515
 test-armhf-armhf-libvirt 16 saverestore-support-checkfail  like 156515
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop fail like 156515
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 156515
 test-amd64-i386-xl-pvshim14 guest-start  fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 15 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  15 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check 
fail never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  16 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-checkfail never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  14 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  15 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 15 migrate-support-checkfail   never pass

version targeted for testing:
 xen  46ad8841ac4da8fc2a128e49b8406966bf5d3601
baseline version:
 xen  4f9294d21c47415376215d68a0298e88582b8e7a

Last test of basis   156515  2020-11-06 05:10:43 Z4 days
Testing same since   156592  2020-11-09 18:08:09 Z0 days1 attempts


People who touched revisions under test:
  Andrew Cooper 
  Anthony PERARD 
  Bertrand Marquis 
  David Woodhouse 
  Marek Marczykowski-Górecki 
  Olaf Hering 
  Tim Deegan 
  Wei Liu 

jobs:
 build-amd64-xsm  pass
 

[PATCH] x86emul: de-duplicate scatters to the same linear address

2020-11-10 Thread Jan Beulich
The SDM specifically allows for earlier writes to fully overlapping
ranges to be dropped. If a guest did so, hvmemul_phys_mmio_access()
would crash it if varying data was written to the same address. Detect
overlaps early, as doing so in hvmemul_{linear,phys}_mmio_access() would
be quite a bit more difficult.

Note that due to cache slot use being linear address based, there's no
similar issue with multiple writes to the same physical address (mapped
through different linear addresses).

Since this requires an adjustment to the EVEX Disp8 scaling test,
correct a comment there at the same time.

Signed-off-by: Jan Beulich 
---
TBD: The SDM isn't entirely unambiguous about the faulting behavior in
 this case: If a fault would need delivering on the earlier slot
 despite the write getting squashed, we'd have to call ops->write()
 with size set to zero for the earlier write(s). However,
 hvm/emulate.c's handling of zero-byte accesses extends only to the
 virtual-to-linear address conversions (and raising of involved
 faults), so in order to also observe #PF changes to that logic
 would then also be needed. Can we live with a possible misbehavior
 here?

--- a/tools/tests/x86_emulator/evex-disp8.c
+++ b/tools/tests/x86_emulator/evex-disp8.c
@@ -647,8 +647,8 @@ static const uint16_t imm0f[16] = {
 static struct x86_emulate_ops emulops;
 
 /*
- * Access tracking (by granular) is used on the first 64 bytes of address
- * space. Instructions get encode with a raw Disp8 value of 1, which then
+ * Access tracking (byte granular) is used on the first bytes of address
+ * space. Instructions get encoded with a raw Disp8 value of 1, which then
  * gets scaled accordingly. Hence accesses below the address 
  * as well as at or above 2 *  are indications of bugs. To
  * aid diagnosis / debugging, track all accesses below 3 * .
@@ -804,6 +804,31 @@ static void test_one(const struct test *
 
 asm volatile ( "kxnorw %k1, %k1, %k1" );
 asm volatile ( "vxorps %xmm4, %xmm4, %xmm4" );
+if ( sg && (test->opc | 3) == 0xa3 )
+{
+/*
+ * Non-prefetch scatters need handling specially, due to the
+ * overlapped write elimination done by the emulator. The order of
+ * indexes chosen is somewhat arbitrary, within the constraints
+ * imposed by the various different uses.
+ */
+static const struct {
+int32_t _[16];
+} off32 = {{ 1, 0, 2, 3, 7, 6, 5, 4, 8, 10, 12, 14, 9, 11, 13, 15 }};
+
+if ( test->opc & 1 )
+{
+asm volatile ( "vpmovsxdq %0, %%zmm4" :: "m" (off32) );
+vsz >>= !evex.w;
+}
+else
+asm volatile ( "vmovdqu32 %0, %%zmm4" :: "m" (off32) );
+
+/* Scale by element size. */
+instr[6] |= (evex.w | 2) << 6;
+
+sg = false;
+}
 
 ctxt->regs->eip = (unsigned long)[0];
 ctxt->regs->edx = 0;
--- a/xen/arch/x86/x86_emulate/x86_emulate.c
+++ b/xen/arch/x86/x86_emulate/x86_emulate.c
@@ -10032,25 +10032,47 @@ x86_emulate(
 
 for ( i = 0; op_mask; ++i )
 {
-long idx = b & 1 ? index.qw[i] : index.dw[i];
+long idx = (b & 1 ? index.qw[i]
+  : index.dw[i]) * (1 << state->sib_scale);
+unsigned long offs = truncate_ea(ea.mem.off + idx);
+unsigned int j;
 
 if ( !(op_mask & (1 << i)) )
 continue;
 
-rc = ops->write(ea.mem.seg,
-truncate_ea(ea.mem.off +
-idx * (1 << state->sib_scale)),
-(void *)mmvalp + i * op_bytes, op_bytes, ctxt);
-if ( rc != X86EMUL_OKAY )
+/*
+ * hvmemul_linear_mmio_access() will find a cache slot based on
+ * linear address. hvmemul_phys_mmio_access() will crash the
+ * domain if observing varying data getting written to the same
+ * address within a cache slot. Utilize that squashing earlier
+ * writes to fully overlapping addresses is permitted by the spec.
+ */
+for ( j = i + 1; j < n; ++j )
 {
-/* See comment in gather emulation. */
-if ( rc != X86EMUL_EXCEPTION && done )
-rc = X86EMUL_RETRY;
-break;
+long idx2 = (b & 1 ? index.qw[j]
+   : index.dw[j]) * (1 << state->sib_scale);
+
+if ( (op_mask & (1 << j)) &&
+ truncate_ea(ea.mem.off + idx2) == offs )
+break;
+}
+
+if ( j >= n )
+{
+rc = ops->write(ea.mem.seg, offs,
+(void *)mmvalp + i * op_bytes, op_bytes, ctxt);
+if ( rc != X86EMUL_OKAY )
+{
+/* See comment in gather emulation. */
+if 

Re: [PATCH 3/5] x86/p2m: suppress audit_p2m hook when possible

2020-11-10 Thread Jan Beulich
On 10.11.2020 12:30, Roger Pau Monné wrote:
> On Wed, Oct 28, 2020 at 10:23:42AM +0100, Jan Beulich wrote:
>> When P2M_AUDIT is false, it's unused, so instead of having a dangling
>> NULL pointer sit there, omit the field altogether.
>>
>> Instead of adding "#if P2M_AUDIT && defined(CONFIG_HVM)" in even more
>> places, fold the latter part right into the definition of P2M_AUDIT.
>>
>> Signed-off-by: Jan Beulich 
> 
> Acked-by: Roger Pau Monné 

Thanks. Since I see you keep replying to v1, are you aware of
there being v2? For this particular patch there actually is a
clang related fix in v2, which may be of interest to you.

Jan



Re: [PATCH v3 5/7] x86: guard against straight-line speculation past RET

2020-11-10 Thread Jan Beulich
On 10.11.2020 12:16, Roger Pau Monné wrote:
> On Tue, Nov 10, 2020 at 11:06:46AM +0100, Jan Beulich wrote:
>> On 10.11.2020 10:31, Roger Pau Monné wrote:
>>> On Fri, Oct 23, 2020 at 10:38:04AM +0200, Jan Beulich wrote:
 Under certain conditions CPUs can speculate into the instruction stream
 past a RET instruction. Guard against this just like 3b7dab93f240
 ("x86/spec-ctrl: Protect against CALL/JMP straight-line speculation")
 did - by inserting an "INT $3" insn. It's merely the mechanics of how to
 achieve this that differ: A set of macros gets introduced to post-
 process RET insns issued by the compiler (or living in assembly files).

 Unfortunately for clang this requires further features their built-in
 assembler doesn't support: We need to be able to override insn mnemonics
 produced by the compiler (which may be impossible, if internally
 assembly mnemonics never get generated), and we want to use \(text)
 escaping / quoting in the auxiliary macro.
>>>
>>> Could this have an option to enable/disable at build time?
>>
>> Well, a subsequent patch adds a config option for this, which in
>> the worst case could be turned off. I'm afraid though I'm not
>> clear about the question, because ...
>>
>>> FreeBSD will drop GNU as quite soon from base, and albeit it can be
>>> installed as a package I would like to be able to build Xen using a
>>> toolchain based on LLVM exclusively.
>>
>> ... it's not clear to me what the implications here are: Are you
>> saying -no-integrated-as is not going to function anymore, unless
>> people explicitly install gas? If that's not what you meant to
>> indicate, then I don't see how building would become impossible.
> 
> I'm still inquiring about this, but I would say that when gas is
> removed from FreeBSD then the 'as' command would be mapped to llvm-as,
> and thus -no-integrated-as would hit the same issues as the integrated
> as. So far in Xen we have assumed that -no-integrated-as would
> fallback to an as capable of doing what the integrated clang as
> doesn't support, but that might not be the case.

At which point Xen couldn't be built anyway, I expect. If llvm-as
isn't sufficiently gas-compatible, we've lost (right now at least).

> Ideally we would have to re-run the tests with -no-integrated-as, in
> order to assert that the external as is really capable of what the
> internal one is not.

And if it doesn't, what would we do other than failing the build
(which it would also if we didn't do the 2nd round of checks)?

Jan



Re: [PATCH] docs: fix documentation to notice credit2 is the default

2020-11-10 Thread Jan Beulich
On 10.11.2020 14:13, Jürgen Groß wrote:
> On 10.11.20 14:07, Andrew Cooper wrote:
>> On 10/11/2020 12:49, Roger Pau Monné wrote:
>>> On Tue, Nov 10, 2020 at 12:31:14PM +0100, Jürgen Groß wrote:
 On 10.11.20 12:21, Roger Pau Monne wrote:
> Fix the command line document to account for credit2 now being the
> default scheduler.
>
> Fixes: dafd936dddbd ('Make credit2 the default scheduler')
> Signed-off-by: Roger Pau Monné 
> ---
>docs/misc/xen-command-line.pandoc | 2 +-
>1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/docs/misc/xen-command-line.pandoc 
> b/docs/misc/xen-command-line.pandoc
> index 4ae9391fcd..789aead148 100644
> --- a/docs/misc/xen-command-line.pandoc
> +++ b/docs/misc/xen-command-line.pandoc
> @@ -1876,7 +1876,7 @@ with read and write permissions.
>### sched
>> `= credit | credit2 | arinc653 | rtds | null`
> -> Default: `sched=credit`
> +> Default: `sched=credit2`
>Choose the default scheduler.
>
 Tried that before:

 https://lists.xen.org/archives/html/xen-devel/2019-01/msg01097.html

 And Andrew didn't like it...
>>> One way or another we need to get this fixed in the document. Listing
>>> credit as the still the default is wrong.
>>
>> I agree that what is there is wrong, but so is saying credit2.
>>
>> This documentation is for users, because develops know exactly how they
>> configured their schedulers, and won't actually need to refer to it.
>>
>> As a consequence, it depends heavily on what a specific
>> distro/downstream chose, config-wise.
>>
>>> I think there are several places in xen-command-line.pandoc that just
>>> contain the default values set in Kconfig, so IMO if we want to
>>> change this it should be done wholesale rather than picking on every
>>> default value change. Opinions?
>>
>> xen-command-line.pandoc wholly predates Kconfig.  We're only in this
>> mess because previous patches haven't been appropriately reviewed.
>>
>> Lets fix it up to be correct, but lets not delay fixing this to look for
>> potential other cases.
> 
> The ultimate fix would be to generate this document according to
> Kconfig settings. :-D

Except that's not suitable for putting up as a web page for
everyone to use as "cannoical" reference. Every distro could
do so, sure.

Jan



Re: [PATCH] docs: fix documentation to notice credit2 is the default

2020-11-10 Thread Jürgen Groß

On 10.11.20 14:07, Andrew Cooper wrote:

On 10/11/2020 12:49, Roger Pau Monné wrote:

On Tue, Nov 10, 2020 at 12:31:14PM +0100, Jürgen Groß wrote:

On 10.11.20 12:21, Roger Pau Monne wrote:

Fix the command line document to account for credit2 now being the
default scheduler.

Fixes: dafd936dddbd ('Make credit2 the default scheduler')
Signed-off-by: Roger Pau Monné 
---
   docs/misc/xen-command-line.pandoc | 2 +-
   1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/misc/xen-command-line.pandoc 
b/docs/misc/xen-command-line.pandoc
index 4ae9391fcd..789aead148 100644
--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -1876,7 +1876,7 @@ with read and write permissions.
   ### sched
   > `= credit | credit2 | arinc653 | rtds | null`
-> Default: `sched=credit`
+> Default: `sched=credit2`
   Choose the default scheduler.


Tried that before:

https://lists.xen.org/archives/html/xen-devel/2019-01/msg01097.html

And Andrew didn't like it...

One way or another we need to get this fixed in the document. Listing
credit as the still the default is wrong.


I agree that what is there is wrong, but so is saying credit2.

This documentation is for users, because develops know exactly how they
configured their schedulers, and won't actually need to refer to it.

As a consequence, it depends heavily on what a specific
distro/downstream chose, config-wise.


I think there are several places in xen-command-line.pandoc that just
contain the default values set in Kconfig, so IMO if we want to
change this it should be done wholesale rather than picking on every
default value change. Opinions?


xen-command-line.pandoc wholly predates Kconfig.  We're only in this
mess because previous patches haven't been appropriately reviewed.

Lets fix it up to be correct, but lets not delay fixing this to look for
potential other cases.


The ultimate fix would be to generate this document according to
Kconfig settings. :-D


Juergen


OpenPGP_0xB0DE9DD628BF132F.asc
Description: application/pgp-keys


OpenPGP_signature
Description: OpenPGP digital signature


Re: [PATCH] docs: fix documentation to notice credit2 is the default

2020-11-10 Thread Andrew Cooper
On 10/11/2020 12:49, Roger Pau Monné wrote:
> On Tue, Nov 10, 2020 at 12:31:14PM +0100, Jürgen Groß wrote:
>> On 10.11.20 12:21, Roger Pau Monne wrote:
>>> Fix the command line document to account for credit2 now being the
>>> default scheduler.
>>>
>>> Fixes: dafd936dddbd ('Make credit2 the default scheduler')
>>> Signed-off-by: Roger Pau Monné 
>>> ---
>>>   docs/misc/xen-command-line.pandoc | 2 +-
>>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/docs/misc/xen-command-line.pandoc 
>>> b/docs/misc/xen-command-line.pandoc
>>> index 4ae9391fcd..789aead148 100644
>>> --- a/docs/misc/xen-command-line.pandoc
>>> +++ b/docs/misc/xen-command-line.pandoc
>>> @@ -1876,7 +1876,7 @@ with read and write permissions.
>>>   ### sched
>>>   > `= credit | credit2 | arinc653 | rtds | null`
>>> -> Default: `sched=credit`
>>> +> Default: `sched=credit2`
>>>   Choose the default scheduler.
>>>
>> Tried that before:
>>
>> https://lists.xen.org/archives/html/xen-devel/2019-01/msg01097.html
>>
>> And Andrew didn't like it...
> One way or another we need to get this fixed in the document. Listing
> credit as the still the default is wrong.

I agree that what is there is wrong, but so is saying credit2.

This documentation is for users, because develops know exactly how they
configured their schedulers, and won't actually need to refer to it.

As a consequence, it depends heavily on what a specific
distro/downstream chose, config-wise.

> I think there are several places in xen-command-line.pandoc that just
> contain the default values set in Kconfig, so IMO if we want to
> change this it should be done wholesale rather than picking on every
> default value change. Opinions?

xen-command-line.pandoc wholly predates Kconfig.  We're only in this
mess because previous patches haven't been appropriately reviewed.

Lets fix it up to be correct, but lets not delay fixing this to look for
potential other cases.

~Andrew



Re: [PATCH] docs: fix documentation to notice credit2 is the default

2020-11-10 Thread Roger Pau Monné
On Tue, Nov 10, 2020 at 12:31:14PM +0100, Jürgen Groß wrote:
> On 10.11.20 12:21, Roger Pau Monne wrote:
> > Fix the command line document to account for credit2 now being the
> > default scheduler.
> > 
> > Fixes: dafd936dddbd ('Make credit2 the default scheduler')
> > Signed-off-by: Roger Pau Monné 
> > ---
> >   docs/misc/xen-command-line.pandoc | 2 +-
> >   1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/docs/misc/xen-command-line.pandoc 
> > b/docs/misc/xen-command-line.pandoc
> > index 4ae9391fcd..789aead148 100644
> > --- a/docs/misc/xen-command-line.pandoc
> > +++ b/docs/misc/xen-command-line.pandoc
> > @@ -1876,7 +1876,7 @@ with read and write permissions.
> >   ### sched
> >   > `= credit | credit2 | arinc653 | rtds | null`
> > -> Default: `sched=credit`
> > +> Default: `sched=credit2`
> >   Choose the default scheduler.
> > 
> 
> Tried that before:
> 
> https://lists.xen.org/archives/html/xen-devel/2019-01/msg01097.html
> 
> And Andrew didn't like it...

One way or another we need to get this fixed in the document. Listing
credit as the still the default is wrong.

I think there are several places in xen-command-line.pandoc that just
contain the default values set in Kconfig, so IMO if we want to
change this it should be done wholesale rather than picking on every
default value change. Opinions?

Thanks, Roger.



Schedule for OpenPOWER/Xen meeting

2020-11-10 Thread Olivier Lambert
Hi everyone,

We got 2 potential dates for the initial tech meeting with at least one
OpenPOWER expert, so we can discuss the effort needed to port Xen on this
architecture.

Because of time zones (on OpenPower side, there's one guy in Australia), we
got 2 possible schedules in November:

1. 3pm CT on this Thursday the 12th (! this week)
2. Or next week Thursday the 19th

I made a doodle-like so everyone can vote on their preferred schedule:
https://framadate.org/QQu5rYEOEYr4ZHc4

Note: 3pm CT would mean 9pm UTC, 10pm UTC+1 (CET). But correct me if I'm
wrong.

Reminder: the Cryptpad of the last Xen Community meeting contains the list
of people interested. If you are aware of someone interested that could
miss this email on this devel list, feel free to forward it. Cryptpad link:
https://cryptpad.fr/pad/#/2/pad/edit/k-0Aj+Sxb5SliLWrFRBwx49V/

Thank you and see you soon!

Olivier.


Re: [PATCH v2] x86/msr: fix handling of MSR_IA32_PERF_{STATUS/CTL}

2020-11-10 Thread Jan Beulich
On 10.11.2020 11:32, Andrew Cooper wrote:
> On 10/11/2020 08:03, Jan Beulich wrote:
>> On 09.11.2020 18:38, Andrew Cooper wrote:
>>> --- a/xen/include/xen/sched.h
>>> +++ b/xen/include/xen/sched.h
>>> @@ -1069,6 +1069,23 @@ extern enum cpufreq_controller {
>>>  FREQCTL_none, FREQCTL_dom0_kernel, FREQCTL_xen
>>>  } cpufreq_controller;
>>>  
>>> +static always_inline bool is_cpufreq_controller(const struct domain *d)
>>> +{
>>> +/*
>>> + * A PV dom0 can be nominated as the cpufreq controller, instead of 
>>> using
>>> + * Xen's cpufreq driver, at which point dom0 gets direct access to 
>>> certain
>>> + * MSRs.
>>> + *
>>> + * This interface only works when dom0 is identity pinned and has the 
>>> same
>>> + * number of vCPUs as pCPUs on the system.
>>> + *
>>> + * It would be far better to paravirtualise the interface.
>>> + */
>>> +return (IS_ENABLED(CONFIG_PV) &&
>>> +(cpufreq_controller == FREQCTL_dom0_kernel) &&
>>> +is_pv_domain(d) && is_hardware_domain(d));
>>> +}
>> IS_ENABLED(CONFIG_PV) is already part of is_pv_domain() and hence
>> imo shouldn't be used explicitly here.
> 
> Ah yes.  Will drop.
> 
>> Also, this being an x86 concept, is it really a good idea to put
>> in xen/sched.h? I guess this builds on Arm just because of DCE
>> from the IS_ENABLED(CONFIG_PV) (where afaict the one in
>> is_pv_domain() will still do). (But yes, I do realize that
>> cpufreq_controller itself gets declared in this file, so maybe
>> better to do some subsequent cleanup.)
> 
> I can't spot anywhere obviously better for it to live.  We don't have a
> common cpufreq.h,

Not the most obvious place for it to live at, but
xen/include/acpi/cpufreq/cpufreq.h ?

Jan



Re: [PATCH 4/5] x86/HAP: move nested-P2M flush calculations out of locked region

2020-11-10 Thread Roger Pau Monné
On Wed, Oct 28, 2020 at 10:24:12AM +0100, Jan Beulich wrote:
> By latching the old MFN into a local variable, these calculations don't
> depend on anything but local variables anymore. Hence the point in time
> when they get performed doesn't matter anymore, so they can be moved
> past the locked region.
> 
> Signed-off-by: Jan Beulich 

Acked-by: Roger Pau Monné 

Thanks, Roger.



Re: [PATCH] docs: fix documentation to notice credit2 is the default

2020-11-10 Thread Jürgen Groß

On 10.11.20 12:21, Roger Pau Monne wrote:

Fix the command line document to account for credit2 now being the
default scheduler.

Fixes: dafd936dddbd ('Make credit2 the default scheduler')
Signed-off-by: Roger Pau Monné 
---
  docs/misc/xen-command-line.pandoc | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/misc/xen-command-line.pandoc 
b/docs/misc/xen-command-line.pandoc
index 4ae9391fcd..789aead148 100644
--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -1876,7 +1876,7 @@ with read and write permissions.
  ### sched
  > `= credit | credit2 | arinc653 | rtds | null`
  
-> Default: `sched=credit`

+> Default: `sched=credit2`
  
  Choose the default scheduler.
  



Tried that before:

https://lists.xen.org/archives/html/xen-devel/2019-01/msg01097.html

And Andrew didn't like it...


Juergen


OpenPGP_0xB0DE9DD628BF132F.asc
Description: application/pgp-keys


OpenPGP_signature
Description: OpenPGP digital signature


Re: [PATCH 3/5] x86/p2m: suppress audit_p2m hook when possible

2020-11-10 Thread Roger Pau Monné
On Wed, Oct 28, 2020 at 10:23:42AM +0100, Jan Beulich wrote:
> When P2M_AUDIT is false, it's unused, so instead of having a dangling
> NULL pointer sit there, omit the field altogether.
> 
> Instead of adding "#if P2M_AUDIT && defined(CONFIG_HVM)" in even more
> places, fold the latter part right into the definition of P2M_AUDIT.
> 
> Signed-off-by: Jan Beulich 

Acked-by: Roger Pau Monné 

Thanks, Roger.



[PATCH] docs: fix documentation to notice credit2 is the default

2020-11-10 Thread Roger Pau Monne
Fix the command line document to account for credit2 now being the
default scheduler.

Fixes: dafd936dddbd ('Make credit2 the default scheduler')
Signed-off-by: Roger Pau Monné 
---
 docs/misc/xen-command-line.pandoc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/misc/xen-command-line.pandoc 
b/docs/misc/xen-command-line.pandoc
index 4ae9391fcd..789aead148 100644
--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -1876,7 +1876,7 @@ with read and write permissions.
 ### sched
 > `= credit | credit2 | arinc653 | rtds | null`
 
-> Default: `sched=credit`
+> Default: `sched=credit2`
 
 Choose the default scheduler.
 
-- 
2.29.0




Re: [PATCH v3 5/7] x86: guard against straight-line speculation past RET

2020-11-10 Thread Roger Pau Monné
On Tue, Nov 10, 2020 at 11:06:46AM +0100, Jan Beulich wrote:
> On 10.11.2020 10:31, Roger Pau Monné wrote:
> > On Fri, Oct 23, 2020 at 10:38:04AM +0200, Jan Beulich wrote:
> >> Under certain conditions CPUs can speculate into the instruction stream
> >> past a RET instruction. Guard against this just like 3b7dab93f240
> >> ("x86/spec-ctrl: Protect against CALL/JMP straight-line speculation")
> >> did - by inserting an "INT $3" insn. It's merely the mechanics of how to
> >> achieve this that differ: A set of macros gets introduced to post-
> >> process RET insns issued by the compiler (or living in assembly files).
> >>
> >> Unfortunately for clang this requires further features their built-in
> >> assembler doesn't support: We need to be able to override insn mnemonics
> >> produced by the compiler (which may be impossible, if internally
> >> assembly mnemonics never get generated), and we want to use \(text)
> >> escaping / quoting in the auxiliary macro.
> > 
> > Could this have an option to enable/disable at build time?
> 
> Well, a subsequent patch adds a config option for this, which in
> the worst case could be turned off. I'm afraid though I'm not
> clear about the question, because ...
> 
> > FreeBSD will drop GNU as quite soon from base, and albeit it can be
> > installed as a package I would like to be able to build Xen using a
> > toolchain based on LLVM exclusively.
> 
> ... it's not clear to me what the implications here are: Are you
> saying -no-integrated-as is not going to function anymore, unless
> people explicitly install gas? If that's not what you meant to
> indicate, then I don't see how building would become impossible.

I'm still inquiring about this, but I would say that when gas is
removed from FreeBSD then the 'as' command would be mapped to llvm-as,
and thus -no-integrated-as would hit the same issues as the integrated
as. So far in Xen we have assumed that -no-integrated-as would
fallback to an as capable of doing what the integrated clang as
doesn't support, but that might not be the case.

Ideally we would have to re-run the tests with -no-integrated-as, in
order to assert that the external as is really capable of what the
internal one is not.

Roger.



Re: [PATCH 2/5] x86/p2m: collapse the two ->write_p2m_entry() hooks

2020-11-10 Thread Roger Pau Monné
On Wed, Oct 28, 2020 at 10:22:58AM +0100, Jan Beulich wrote:
> The struct paging_mode instances get set to the same functions
> regardless of mode by both HAP and shadow code, hence there's no point
> having this hook there. The hook also doesn't need moving elsewhere - we
> can directly use struct p2m_domain's. This merely requires (from a
> strictly formal pov; in practice this may not even be needed) making
> sure we don't end up using safe_write_pte() for nested P2Ms.
> 
> Signed-off-by: Jan Beulich 
> ---
> Like for the possibly unnecessary p2m_is_nestedp2m() I'm not really sure
> the paging_get_hostmode() check there is still needed either. But I
> didn't want to alter more aspects than necessary here.
> 
> Of course with the p2m_is_nestedp2m() check there and with all three of
> {hap,nestedp2m,shadow}_write_p2m_entry() now globally accessible, it's
> certainly an option to do away with the indirect call there altogether.
> In fact we may even be able to go further and fold the three functions:
> They're relatively similar, and this would "seamlessly" address the
> apparent bug of nestedp2m_write_p2m_entry() not making use of
> p2m_entry_modify().
> 
> --- a/xen/arch/x86/mm/hap/hap.c
> +++ b/xen/arch/x86/mm/hap/hap.c
> @@ -823,6 +823,11 @@ hap_write_p2m_entry(struct p2m_domain *p
>  return 0;
>  }
>  
> +void hap_p2m_init(struct p2m_domain *p2m)
> +{
> +p2m->write_p2m_entry = hap_write_p2m_entry;
> +}
> +
>  static unsigned long hap_gva_to_gfn_real_mode(
>  struct vcpu *v, struct p2m_domain *p2m, unsigned long gva, uint32_t 
> *pfec)
>  {
> @@ -846,7 +851,6 @@ static const struct paging_mode hap_pagi
>  .p2m_ga_to_gfn  = hap_p2m_ga_to_gfn_real_mode,
>  .update_cr3 = hap_update_cr3,
>  .update_paging_modes= hap_update_paging_modes,
> -.write_p2m_entry= hap_write_p2m_entry,
>  .flush_tlb  = flush_tlb,
>  .guest_levels   = 1
>  };
> @@ -858,7 +862,6 @@ static const struct paging_mode hap_pagi
>  .p2m_ga_to_gfn  = hap_p2m_ga_to_gfn_2_levels,
>  .update_cr3 = hap_update_cr3,
>  .update_paging_modes= hap_update_paging_modes,
> -.write_p2m_entry= hap_write_p2m_entry,
>  .flush_tlb  = flush_tlb,
>  .guest_levels   = 2
>  };
> @@ -870,7 +873,6 @@ static const struct paging_mode hap_pagi
>  .p2m_ga_to_gfn  = hap_p2m_ga_to_gfn_3_levels,
>  .update_cr3 = hap_update_cr3,
>  .update_paging_modes= hap_update_paging_modes,
> -.write_p2m_entry= hap_write_p2m_entry,
>  .flush_tlb  = flush_tlb,
>  .guest_levels   = 3
>  };
> @@ -882,7 +884,6 @@ static const struct paging_mode hap_pagi
>  .p2m_ga_to_gfn  = hap_p2m_ga_to_gfn_4_levels,
>  .update_cr3 = hap_update_cr3,
>  .update_paging_modes= hap_update_paging_modes,
> -.write_p2m_entry= hap_write_p2m_entry,
>  .flush_tlb  = flush_tlb,
>  .guest_levels   = 4
>  };
> --- a/xen/arch/x86/mm/p2m-pt.c
> +++ b/xen/arch/x86/mm/p2m-pt.c
> @@ -126,8 +126,9 @@ static int write_p2m_entry(struct p2m_do
>  
>  if ( v->domain != d )
>  v = d->vcpu ? d->vcpu[0] : NULL;
> -if ( likely(v && paging_mode_enabled(d) && paging_get_hostmode(v)) )
> -rc = paging_get_hostmode(v)->write_p2m_entry(p2m, gfn, p, new, 
> level);
> +if ( likely(v && paging_mode_enabled(d) && paging_get_hostmode(v)) ||
> + p2m_is_nestedp2m(p2m) )
> +rc = p2m->write_p2m_entry(p2m, gfn, p, new, level);
>  else
>  safe_write_pte(p, new);
>  
> @@ -209,7 +210,7 @@ p2m_next_level(struct p2m_domain *p2m, v
>  
>  new_entry = l1e_from_mfn(mfn, P2M_BASE_FLAGS | _PAGE_RW);
>  
> -rc = p2m->write_p2m_entry(p2m, gfn, p2m_entry, new_entry, level + 1);
> +rc = write_p2m_entry(p2m, gfn, p2m_entry, new_entry, level + 1);
>  if ( rc )
>  goto error;
>  }
> @@ -251,7 +252,7 @@ p2m_next_level(struct p2m_domain *p2m, v
>  {
>  new_entry = l1e_from_pfn(pfn | (i << ((level - 1) * 
> PAGETABLE_ORDER)),
>   flags);
> -rc = p2m->write_p2m_entry(p2m, gfn, l1_entry + i, new_entry, 
> level);
> +rc = write_p2m_entry(p2m, gfn, l1_entry + i, new_entry, level);
>  if ( rc )
>  {
>  unmap_domain_page(l1_entry);
> @@ -262,8 +263,7 @@ p2m_next_level(struct p2m_domain *p2m, v
>  unmap_domain_page(l1_entry);
>  
>  new_entry = l1e_from_mfn(mfn, P2M_BASE_FLAGS | _PAGE_RW);
> -rc = p2m->write_p2m_entry(p2m, gfn, p2m_entry, new_entry,
> -  level + 1);
> +rc = write_p2m_entry(p2m, gfn, p2m_entry, new_entry, level + 1);
>  if ( rc )
>  goto error;
>  }
> @@ -335,7 +335,7 @@ static int p2m_pt_set_recalc_range(struc
>  if ( 

Re: [PATCH v2] x86/msr: fix handling of MSR_IA32_PERF_{STATUS/CTL}

2020-11-10 Thread Andrew Cooper
On 10/11/2020 08:03, Jan Beulich wrote:
> On 09.11.2020 18:38, Andrew Cooper wrote:
>> From: Roger Pau Monné 
>>
>> Currently a PV hardware domain can also be given control over the CPU
>> frequency, and such guest is allowed to write to MSR_IA32_PERF_CTL.
>> However since commit 322ec7c89f6 the default behavior has been changed
>> to reject accesses to not explicitly handled MSRs, preventing PV
>> guests that manage CPU frequency from reading
>> MSR_IA32_PERF_{STATUS/CTL}.
>>
>> Additionally some HVM guests (Windows at least) will attempt to read
>> MSR_IA32_PERF_CTL and will panic if given back a #GP fault:
>>
>>   vmx.c:3035:d8v0 RDMSR 0x0199 unimplemented
>>   d8v0 VIRIDIAN CRASH: 3b c096 f806871c1651 da0253683720 0
>>
>> Move the handling of MSR_IA32_PERF_{STATUS/CTL} to the common MSR
>> handling shared between HVM and PV guests, and add an explicit case
>> for reads to MSR_IA32_PERF_{STATUS/CTL}.
>>
>> Restore previous behavior and allow PV guests with the required
>> permissions to read the contents of the mentioned MSRs. Non privileged
>> guests will get 0 when trying to read those registers, as writes to
>> MSR_IA32_PERF_CTL by such guest will already be silently dropped.
>>
>> Fixes: 322ec7c89f6 ('x86/pv: disallow access to unknown MSRs')
>> Fixes: 84e848fd7a1 ('x86/hvm: disallow access to unknown MSRs')
>> Signed-off-by: Roger Pau Monné 
>> Signed-off-by: Andrew Cooper 
> Reviewed-by: Jan Beulich 

Thanks,

> with a nit, a minor adjustment request, and a question:
>
>> @@ -448,6 +467,21 @@ int guest_wrmsr(struct vcpu *v, uint32_t msr, uint64_t 
>> val)
>>  goto gp_fault;
>>  break;
>>  
>> +/*
>> + * This MSR are not enumerated in CPUID.  It has been around since 
>> the
> s/are/is/

Oops, yes.

>
>> --- a/xen/include/xen/sched.h
>> +++ b/xen/include/xen/sched.h
>> @@ -1069,6 +1069,23 @@ extern enum cpufreq_controller {
>>  FREQCTL_none, FREQCTL_dom0_kernel, FREQCTL_xen
>>  } cpufreq_controller;
>>  
>> +static always_inline bool is_cpufreq_controller(const struct domain *d)
>> +{
>> +/*
>> + * A PV dom0 can be nominated as the cpufreq controller, instead of 
>> using
>> + * Xen's cpufreq driver, at which point dom0 gets direct access to 
>> certain
>> + * MSRs.
>> + *
>> + * This interface only works when dom0 is identity pinned and has the 
>> same
>> + * number of vCPUs as pCPUs on the system.
>> + *
>> + * It would be far better to paravirtualise the interface.
>> + */
>> +return (IS_ENABLED(CONFIG_PV) &&
>> +(cpufreq_controller == FREQCTL_dom0_kernel) &&
>> +is_pv_domain(d) && is_hardware_domain(d));
>> +}
> IS_ENABLED(CONFIG_PV) is already part of is_pv_domain() and hence
> imo shouldn't be used explicitly here.

Ah yes.  Will drop.

> Also, this being an x86 concept, is it really a good idea to put
> in xen/sched.h? I guess this builds on Arm just because of DCE
> from the IS_ENABLED(CONFIG_PV) (where afaict the one in
> is_pv_domain() will still do). (But yes, I do realize that
> cpufreq_controller itself gets declared in this file, so maybe
> better to do some subsequent cleanup.)

I can't spot anywhere obviously better for it to live.  We don't have a
common cpufreq.h, and I'm not sure that cpuidle.h is an appropriate
place to live either.

Thanks,

~Andrew



Re: [PATCH 1/5] x86/p2m: paging_write_p2m_entry() is a private function

2020-11-10 Thread Jan Beulich
On 10.11.2020 11:27, Roger Pau Monné wrote:
> On Wed, Oct 28, 2020 at 10:22:04AM +0100, Jan Beulich wrote:
>> As it gets installed by p2m_pt_init(), it doesn't need to live in
>> paging.c. The function working in terms of l1_pgentry_t even further
>> indicates its non-paging-generic nature. Move it and drop its
>> paging_ prefix, not adding any new one now that it's static.
>>
>> This then also makes more obvious that in the EPT case we wouldn't
>> risk mistakenly calling through the NULL hook pointer.
>>
>> Signed-off-by: Jan Beulich 
> 
> Acked-by: Roger Pau Monné 

Thanks.

>> --- a/xen/arch/x86/mm/p2m-pt.c
>> +++ b/xen/arch/x86/mm/p2m-pt.c
>> @@ -108,6 +108,31 @@ static unsigned long p2m_type_to_flags(c
>>  }
>>  }
>>  
>> +/*
>> + * Atomically write a P2M entry and update the paging-assistance state
>> + * appropriately.
>> + * Arguments: the domain in question, the GFN whose mapping is being 
>> updated,
>> + * a pointer to the entry to be written, the MFN in which the entry resides,
>> + * the new contents of the entry, and the level in the p2m tree at which
>> + * we are writing.
>> + */
>> +static int write_p2m_entry(struct p2m_domain *p2m, unsigned long gfn,
>> +   l1_pgentry_t *p, l1_pgentry_t new,
>> +   unsigned int level)
>> +{
>> +struct domain *d = p2m->domain;
>> +struct vcpu *v = current;
> 
> I think you could constify both?

For v it looks like I could. For d a subsequent patch would then
need to undo it, so I'd prefer to keep it this way here.

Jan



Re: [PATCH 1/5] x86/p2m: paging_write_p2m_entry() is a private function

2020-11-10 Thread Roger Pau Monné
On Wed, Oct 28, 2020 at 10:22:04AM +0100, Jan Beulich wrote:
> As it gets installed by p2m_pt_init(), it doesn't need to live in
> paging.c. The function working in terms of l1_pgentry_t even further
> indicates its non-paging-generic nature. Move it and drop its
> paging_ prefix, not adding any new one now that it's static.
> 
> This then also makes more obvious that in the EPT case we wouldn't
> risk mistakenly calling through the NULL hook pointer.
> 
> Signed-off-by: Jan Beulich 

Acked-by: Roger Pau Monné 

> 
> --- a/xen/arch/x86/mm/p2m-pt.c
> +++ b/xen/arch/x86/mm/p2m-pt.c
> @@ -108,6 +108,31 @@ static unsigned long p2m_type_to_flags(c
>  }
>  }
>  
> +/*
> + * Atomically write a P2M entry and update the paging-assistance state
> + * appropriately.
> + * Arguments: the domain in question, the GFN whose mapping is being updated,
> + * a pointer to the entry to be written, the MFN in which the entry resides,
> + * the new contents of the entry, and the level in the p2m tree at which
> + * we are writing.
> + */
> +static int write_p2m_entry(struct p2m_domain *p2m, unsigned long gfn,
> +   l1_pgentry_t *p, l1_pgentry_t new,
> +   unsigned int level)
> +{
> +struct domain *d = p2m->domain;
> +struct vcpu *v = current;

I think you could constify both?

Thanks, Roger.



Re: [PATCH v2] x86/msr: fix handling of MSR_IA32_PERF_{STATUS/CTL}

2020-11-10 Thread Andrew Cooper
On 10/11/2020 08:04, Jan Beulich wrote:
> On 09.11.2020 19:31, Roger Pau Monné wrote:
>> On Mon, Nov 09, 2020 at 05:38:19PM +, Andrew Cooper wrote:
>>> From: Roger Pau Monné 
>>>
>>> Currently a PV hardware domain can also be given control over the CPU
>>> frequency, and such guest is allowed to write to MSR_IA32_PERF_CTL.
>>> However since commit 322ec7c89f6 the default behavior has been changed
>>> to reject accesses to not explicitly handled MSRs, preventing PV
>>> guests that manage CPU frequency from reading
>>> MSR_IA32_PERF_{STATUS/CTL}.
>>>
>>> Additionally some HVM guests (Windows at least) will attempt to read
>>> MSR_IA32_PERF_CTL and will panic if given back a #GP fault:
>>>
>>>   vmx.c:3035:d8v0 RDMSR 0x0199 unimplemented
>>>   d8v0 VIRIDIAN CRASH: 3b c096 f806871c1651 da0253683720 0
>>>
>>> Move the handling of MSR_IA32_PERF_{STATUS/CTL} to the common MSR
>>> handling shared between HVM and PV guests, and add an explicit case
>>> for reads to MSR_IA32_PERF_{STATUS/CTL}.
>>>
>>> Restore previous behavior and allow PV guests with the required
>>> permissions to read the contents of the mentioned MSRs. Non privileged
>>> guests will get 0 when trying to read those registers, as writes to
>>> MSR_IA32_PERF_CTL by such guest will already be silently dropped.
>>>
>>> Fixes: 322ec7c89f6 ('x86/pv: disallow access to unknown MSRs')
>>> Fixes: 84e848fd7a1 ('x86/hvm: disallow access to unknown MSRs')
>>> Signed-off-by: Roger Pau Monné 
>>> Signed-off-by: Andrew Cooper 
>> Reviewed-by: Roger Pau Monné 
>>
>>> ---
>>> CC: Jan Beulich 
>>> CC: Roger Pau Monné 
>>> CC: Wei Liu 
>>>
>>> v2:
>>>  * fix is_cpufreq_controller() to exclude PVH dom0, and collapse to nothing 
>>> in
>>>!CONFIG_PV builds
>>>  * Drop the cross-vendor checks.  It isn't possible to configure dom0 as
>>>cross-vendor, and anyone using is_cpufreq_controller() without an exact
>>>model match has far bigger problems.
> This already answers ...
>
>>> --- a/xen/include/xen/sched.h
>>> +++ b/xen/include/xen/sched.h
>>> @@ -1069,6 +1069,23 @@ extern enum cpufreq_controller {
>>>  FREQCTL_none, FREQCTL_dom0_kernel, FREQCTL_xen
>>>  } cpufreq_controller;
>>>  
>>> +static always_inline bool is_cpufreq_controller(const struct domain *d)
>>> +{
>>> +/*
>>> + * A PV dom0 can be nominated as the cpufreq controller, instead of 
>>> using
>>> + * Xen's cpufreq driver, at which point dom0 gets direct access to 
>>> certain
>>> + * MSRs.
>>> + *
>>> + * This interface only works when dom0 is identity pinned and has the 
>>> same
>>> + * number of vCPUs as pCPUs on the system.
>>> + *
>>> + * It would be far better to paravirtualise the interface.
>>> + */
>> Would it be useful to add an assert here that the domain cpuid vendor
>> and the BSP cpuid vendor match?
>>
>> Is it even possible to fake a different cpuid vendor for dom0?
> ... your question here.

Technically at the moment it is possible to configure a non-dom0
hardware domain as cross vendor, but anyone doing so can keep all the
resulting pieces.

~Andrew



Re: [PATCH v6 2/3] xen/evtchn: rework per event channel lock

2020-11-10 Thread Jan Beulich
On 10.11.2020 10:47, Julien Grall wrote:
> Hi,
> 
> On 10/11/2020 07:39, Jan Beulich wrote:
>> On 09.11.2020 18:46, Julien Grall wrote:
>>> Hi,
>>>
>>> On 09/11/2020 16:48, Jan Beulich wrote:
 On 09.11.2020 17:38, Juergen Gross wrote:
> Currently the lock for a single event channel needs to be taken with
> interrupts off, which causes deadlocks in some cases.
>
> Rework the per event channel lock to be non-blocking for the case of
> sending an event and removing the need for disabling interrupts for
> taking the lock.
>
> The lock is needed for avoiding races between event channel state
> changes (creation, closing, binding) against normal operations (set
> pending, [un]masking, priority changes).
>
> Use a rwlock, but with some restrictions:
>
> - Changing the state of an event channel (creation, closing, binding)
> needs to use write_lock(), with ASSERT()ing that the lock is taken as
> writer only when the state of the event channel is either before or
> after the locked region appropriate (either free or unbound).
>
> - Sending an event needs to use read_trylock() mostly, in case of not
> obtaining the lock the operation is omitted. This is needed as
> sending an event can happen with interrupts off (at least in some
> cases).
>
> - Dumping the event channel state for debug purposes is using
> read_trylock(), too, in order to avoid blocking in case the lock is
> taken as writer for a long time.
>
> - All other cases can use read_lock().
>
> Fixes: e045199c7c9c54 ("evtchn: address races with evtchn_reset()")
> Signed-off-by: Juergen Gross 
> ---
> V4:
> - switch to rwlock
> - add ASSERT() to verify correct write_lock() usage
>
> V3:
> - corrected a copy-and-paste error (Jan Beulich)
> - corrected unlocking in two cases (Jan Beulich)
> - renamed evtchn_read_trylock() (Jan Beulich)
> - added some comments and an ASSERT() for evtchn_write_lock()
> - set EVENT_WRITE_LOCK_INC to INT_MIN
>
> V2:
> - added needed barriers
>
> Signed-off-by: Juergen Gross 

 Reviewed-by: Jan Beulich 

 I'll give it overnight for others to possibly comment, but I'd
 like to get this in tomorrow if at all possible.
>>>
>>> IIRC, Citrix originally reported the issues. I would like to give them
>>> an opportunity to test the patches and confirm this effectively fixed
>>> there issues.
>>
>> I would consider waiting longer, but I'd like to get staging unblocked.
> 
> So the plan is to have the patches sitting in staging for a few days and 
> then backport?

Right, the usual thing - wait at least until a push has happened.
Unless we learn of problems with the change, I definitely want to
have this in for 4.14.1.

>> Just like with 52e1fc47abc3a0123 ("evtchn/Flask: pre-allocate node on
>> send path") we can always decide to revert / fix up afterwards. The
>> patch here surely is an improvement, even if it was to turn out it
>> doesn't fix all issues yes. I'd appreciate if you would confirm you
>> don't object to the patch going in - I'll wait a few more hours,
>> perhaps until around noon.
> I would suggest to wait until noon and if you don't get any answer, then 
> merge it.

Okay.

Jan



[xen-4.13-testing test] 156593: tolerable FAIL - PUSHED

2020-11-10 Thread osstest service owner
flight 156593 xen-4.13-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/156593/

Failures :-/ but no regressions.

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds18 guest-start/debian.repeat fail REGR. vs. 156399

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stopfail like 156399
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop fail like 156399
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop fail like 156399
 test-armhf-armhf-libvirt 16 saverestore-support-checkfail  like 156399
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 156399
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop fail like 156399
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stopfail like 156399
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stopfail like 156399
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop fail like 156399
 test-armhf-armhf-libvirt-raw 15 saverestore-support-checkfail  like 156399
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stopfail like 156399
 test-arm64-arm64-xl  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  16 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt 15 migrate-support-checkfail   never pass
 test-amd64-i386-xl-pvshim14 guest-start  fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check 
fail never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-checkfail never pass
 test-armhf-armhf-xl  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 16 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  14 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  15 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-checkfail   never pass

version targeted for testing:
 xen  971a9d14667448427ddea1cf15fd7fd409205c58
baseline version:
 xen  83115491d4b3dbcb7c8dbe74ce3e59cdfac69b03

Last test of basis   156399  2020-11-04 09:06:15 Z6 days
Testing same since   156593  2020-11-09 18:08:08 Z0 days1 attempts


People who touched revisions under test:
  Andrew Cooper 
  Anthony PERARD 
  Bertrand Marquis 
  David Woodhouse 
  Juergen Gross 
  Marek Marczykowski-Górecki 
  Olaf Hering 
  Tim Deegan 

Re: [PATCH v3 5/7] x86: guard against straight-line speculation past RET

2020-11-10 Thread Jan Beulich
On 10.11.2020 10:31, Roger Pau Monné wrote:
> On Fri, Oct 23, 2020 at 10:38:04AM +0200, Jan Beulich wrote:
>> Under certain conditions CPUs can speculate into the instruction stream
>> past a RET instruction. Guard against this just like 3b7dab93f240
>> ("x86/spec-ctrl: Protect against CALL/JMP straight-line speculation")
>> did - by inserting an "INT $3" insn. It's merely the mechanics of how to
>> achieve this that differ: A set of macros gets introduced to post-
>> process RET insns issued by the compiler (or living in assembly files).
>>
>> Unfortunately for clang this requires further features their built-in
>> assembler doesn't support: We need to be able to override insn mnemonics
>> produced by the compiler (which may be impossible, if internally
>> assembly mnemonics never get generated), and we want to use \(text)
>> escaping / quoting in the auxiliary macro.
> 
> Could this have an option to enable/disable at build time?

Well, a subsequent patch adds a config option for this, which in
the worst case could be turned off. I'm afraid though I'm not
clear about the question, because ...

> FreeBSD will drop GNU as quite soon from base, and albeit it can be
> installed as a package I would like to be able to build Xen using a
> toolchain based on LLVM exclusively.

... it's not clear to me what the implications here are: Are you
saying -no-integrated-as is not going to function anymore, unless
people explicitly install gas? If that's not what you meant to
indicate, then I don't see how building would become impossible.

Depending on the situation as a whole, we might then be in need
of a 2nd config option...

And btw, good that you pointed me back at this: The v3 change
wasn't quite complete, since we now don't need to check for
proper \(text) handling anymore in our logic to establish
CLANG_FLAGS. I've dropped that part for v4.

Jan



[xen-4.14-testing test] 156594: regressions - FAIL

2020-11-10 Thread osstest service owner
flight 156594 xen-4.14-testing real [real]
flight 156615 xen-4.14-testing real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/156594/
http://logs.test-lab.xenproject.org/osstest/logs/156615/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-xl-multivcpu 13 debian-fixupfail REGR. vs. 156525
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail REGR. 
vs. 156525

Tests which did not succeed, but are not blocking:
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop fail like 156525
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stopfail like 156525
 test-armhf-armhf-libvirt 16 saverestore-support-checkfail  like 156525
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 156525
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stopfail like 156525
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stopfail like 156525
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop fail like 156525
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop fail like 156525
 test-armhf-armhf-libvirt-raw 15 saverestore-support-checkfail  like 156525
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop fail like 156525
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stopfail like 156525
 test-amd64-i386-xl-pvshim14 guest-start  fail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 15 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  15 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check 
fail never pass
 test-arm64-arm64-xl  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  14 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  15 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  a38060ece699f22cd317219bec53c0d27279155b
baseline version:
 xen  0c96e4297da07944525729ddbe438b0131ab5b7e

Last test of basis   156525  2020-11-06 16:01:25 Z3 days
Testing same since   156594  2020-11-09 18:08:08 Z0 days1 attempts


People who touched revisions under test:
  Bertrand Marquis 
  David Woodhouse 
  Juergen Gross 
  Marek Marczykowski-Górecki 
  Olaf Hering 
  

Re: [PATCH v6 2/3] xen/evtchn: rework per event channel lock

2020-11-10 Thread Julien Grall

Hi,

On 10/11/2020 07:39, Jan Beulich wrote:

On 09.11.2020 18:46, Julien Grall wrote:

Hi,

On 09/11/2020 16:48, Jan Beulich wrote:

On 09.11.2020 17:38, Juergen Gross wrote:

Currently the lock for a single event channel needs to be taken with
interrupts off, which causes deadlocks in some cases.

Rework the per event channel lock to be non-blocking for the case of
sending an event and removing the need for disabling interrupts for
taking the lock.

The lock is needed for avoiding races between event channel state
changes (creation, closing, binding) against normal operations (set
pending, [un]masking, priority changes).

Use a rwlock, but with some restrictions:

- Changing the state of an event channel (creation, closing, binding)
needs to use write_lock(), with ASSERT()ing that the lock is taken as
writer only when the state of the event channel is either before or
after the locked region appropriate (either free or unbound).

- Sending an event needs to use read_trylock() mostly, in case of not
obtaining the lock the operation is omitted. This is needed as
sending an event can happen with interrupts off (at least in some
cases).

- Dumping the event channel state for debug purposes is using
read_trylock(), too, in order to avoid blocking in case the lock is
taken as writer for a long time.

- All other cases can use read_lock().

Fixes: e045199c7c9c54 ("evtchn: address races with evtchn_reset()")
Signed-off-by: Juergen Gross 
---
V4:
- switch to rwlock
- add ASSERT() to verify correct write_lock() usage

V3:
- corrected a copy-and-paste error (Jan Beulich)
- corrected unlocking in two cases (Jan Beulich)
- renamed evtchn_read_trylock() (Jan Beulich)
- added some comments and an ASSERT() for evtchn_write_lock()
- set EVENT_WRITE_LOCK_INC to INT_MIN

V2:
- added needed barriers

Signed-off-by: Juergen Gross 


Reviewed-by: Jan Beulich 

I'll give it overnight for others to possibly comment, but I'd
like to get this in tomorrow if at all possible.


IIRC, Citrix originally reported the issues. I would like to give them
an opportunity to test the patches and confirm this effectively fixed
there issues.


I would consider waiting longer, but I'd like to get staging unblocked.


So the plan is to have the patches sitting in staging for a few days and 
then backport?



Just like with 52e1fc47abc3a0123 ("evtchn/Flask: pre-allocate node on
send path") we can always decide to revert / fix up afterwards. The
patch here surely is an improvement, even if it was to turn out it
doesn't fix all issues yes. I'd appreciate if you would confirm you
don't object to the patch going in - I'll wait a few more hours,
perhaps until around noon.
I would suggest to wait until noon and if you don't get any answer, then 
merge it.


Cheers,

--
Julien Grall



RE: [PATCH] xen/arm: Add Cortex-A73 erratum 858921 workaround

2020-11-10 Thread Penny Zheng


> -Original Message-
> From: Julien Grall 
> Sent: Monday, November 9, 2020 8:04 PM
> To: Penny Zheng ; xen-devel@lists.xenproject.org;
> sstabell...@kernel.org
> Cc: Andre Przywara ; Bertrand Marquis
> ; Wei Chen ; Kaly Xin
> ; nd 
> Subject: Re: [PATCH] xen/arm: Add Cortex-A73 erratum 858921 workaround
> 
> Hi,
> 
> On 09/11/2020 08:21, Penny Zheng wrote:
> > CNTVCT_EL0 or CNTPCT_EL0 counter read in Cortex-A73 (all versions)
> > might return a wrong value when the counter crosses a 32bit boundary.
> >
> > Until now, there is no case for Xen itself to access CNTVCT_EL0, and
> > it also should be the Guest OS's responsibility to deal with this
> > part.
> >
> > But for CNTPCT, there exists several cases in Xen involving reading
> > CNTPCT, so a possible workaround is that performing the read twice,
> > and to return one or the other depending on whether a transition has
> > taken place.
> >
> > Signed-off-by: Penny Zheng 
> 
> Acked-by: Julien Grall 
>
Thank you. 

> On a related topic, do we need a fix similar to Linux commit 75a19a0202db
> "arm64: arch_timer: Ensure counter register reads occur with seqlock held"?
> 
Sure, I'll check this commit and talk with my teams for further work.

Cheers

--
Penny Zheng
> Cheers,
> 
> --
> Julien Grall


  1   2   >