[Xen-devel] [qemu-mainline test] 115412: regressions - FAIL

[osstest service owner]

flight 115412 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/115412/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-i3866 xen-buildfail REGR. vs. 114507
 build-amd64-xsm   6 xen-buildfail REGR. vs. 114507
 build-i386-xsm6 xen-buildfail REGR. vs. 114507
 build-amd64   6 xen-buildfail REGR. vs. 114507
 build-armhf   6 xen-buildfail REGR. vs. 114507
 build-armhf-xsm   6 xen-buildfail REGR. vs. 114507

Tests which did not succeed, but are not blocking:
 test-amd64-i386-libvirt-qcow2  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-amd64-i386-freebsd10-i386  1 build-check(1)   blocked  n/a
 test-amd64-amd64-qemuu-nested-intel  1 build-check(1)  blocked n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-armhf-armhf-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm  1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemuu-win10-i386  1 build-check(1)  blocked n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-amd64  1 build-check(1)   blocked  n/a
 test-amd64-amd64-pair 1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-credit2   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-win10-i386  1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-win7-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-pygrub   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-ws16-amd64  1 build-check(1)  blocked n/a
 test-amd64-amd64-xl-qcow2 1 build-check(1)   blocked  n/a
 test-amd64-amd64-amd64-pvgrub  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-arndale   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64  1 build-check(1) blocked n/a
 test-armhf-armhf-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl1 build-check(1)   blocked  n/a
 build-i386-libvirt1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-credit2   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-xsm1 build-check(1)   blocked  n/a
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsm  1 build-check(1)blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-raw1 build-check(1)   blocked  n/a
 test-amd64-i386-qemuu-rhel6hvm-amd  1 build-check(1)   blocked n/a
 test-amd64-amd64-i386-pvgrub  1 build-check(1)   blocked  n/a
 build-armhf-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-ws16-amd64  1 build-check(1) blocked n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvhv2-intel  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-xsm   1 build-check(1)   blocked  n/a
 test-amd64-i386-qemuu-rhel6hvm-intel  1 build-check(1) blocked n/a
 test-armhf-armhf-xl-vhd   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-win7-amd64  1 build-check(1)  blocked n/a
 test-amd64-amd64-xl   1 build-check(1)   blocked  n/a
 test-amd64-i386-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvhv2-amd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-qemuu-nested-amd  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-cubietruck  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-rtds  1 

[Xen-devel] [linux-next test] 115386: regressions - FAIL

[osstest service owner]

flight 115386 linux-next real [real]
http://logs.test-lab.xenproject.org/osstest/logs/115386/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-armhf-armhf-examine  8 reboot   fail REGR. vs. 114658
 test-armhf-armhf-xl-arndale   7 xen-boot fail REGR. vs. 114682
 test-armhf-armhf-xl-credit2   7 xen-boot fail REGR. vs. 114682
 test-armhf-armhf-xl-vhd   7 xen-boot fail REGR. vs. 114682
 build-amd64-pvops 6 kernel-build fail REGR. vs. 114682
 test-armhf-armhf-xl-xsm   7 xen-boot fail REGR. vs. 114682
 test-armhf-armhf-libvirt-xsm  7 xen-boot fail REGR. vs. 114682
 test-armhf-armhf-libvirt-raw  7 xen-boot fail REGR. vs. 114682
 test-armhf-armhf-xl   7 xen-boot fail REGR. vs. 114682
 test-armhf-armhf-xl-multivcpu  7 xen-bootfail REGR. vs. 114682
 test-armhf-armhf-libvirt  7 xen-boot fail REGR. vs. 114682
 test-armhf-armhf-xl-cubietruck  7 xen-boot   fail REGR. vs. 114682

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds  7 xen-boot fail REGR. vs. 114682

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-credit2   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-amd64-amd64-qemuu-nested-intel  1 build-check(1)  blocked n/a
 test-amd64-amd64-rumprun-amd64  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsm  1 build-check(1)blocked n/a
 test-amd64-amd64-i386-pvgrub  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemut-ws16-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-ws16-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 1 build-check(1) blocked 
n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvhv2-intel  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-xsm   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemut-win10-i386  1 build-check(1) blocked n/a
 test-amd64-amd64-pair 1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvhv2-amd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-win10-i386  1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-qemuu-nested-amd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-win7-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-pygrub   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qcow2 1 build-check(1)   blocked  n/a
 test-amd64-amd64-amd64-pvgrub  1 build-check(1)   blocked  n/a
 test-amd64-amd64-examine  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemut-win7-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemut-debianhvm-amd64  1 build-check(1)blocked n/a
 test-amd64-amd64-xl-qemut-debianhvm-amd64-xsm  1 build-check(1)blocked n/a
 test-amd64-amd64-xl-rtds  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 114682
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop fail like 114682
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qcow2 12 migrate-support-checkfail  never pass
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop  fail never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass
 test-amd64-i386-xl-qemut-win10-i386 10 windows-install fail never pass

version targeted for testing:
 linux36ef71cae353f88fd6e095e2aaa3e5953af1685d
baseline version:
 linuxebe6e90ccc6679cb01d2b280e4b61e6092d4bedb

Last test of basis  (not found) 
Failing since   (not found) 
Testing same since   114796  2017-10-20 09:26:55 Z  

[Xen-devel] [linux-linus test] 115387: regressions - FAIL

[osstest service owner]

flight 115387 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/115387/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stop   fail REGR. vs. 114682

Tests which did not succeed, but are not blocking:
 test-amd64-i386-xl-qemuu-win7-amd64 15 guest-saverestore.2fail like 114658
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 114682
 test-armhf-armhf-libvirt-xsm 14 saverestore-support-checkfail  like 114682
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 114682
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 114682
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 114682
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stopfail like 114682
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stopfail like 114682
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qcow2 12 migrate-support-checkfail  never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  14 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop  fail never pass
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop  fail never pass
 test-amd64-amd64-xl-qemut-win10-i386 10 windows-installfail never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass
 test-amd64-i386-xl-qemut-win10-i386 10 windows-install fail never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass

version targeted for testing:
 linux0b07194bb55ed836c2cc7c22e866b87a14681984
baseline version:
 linuxebe6e90ccc6679cb01d2b280e4b61e6092d4bedb

Last test of basis   114682  2017-10-18 09:54:11 Z   12 days
Failing since114781  2017-10-20 01:00:47 Z   11 days   18 attempts
Testing same since   115373  2017-10-29 22:52:46 Z1 days2 attempts


419 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-armhf-pvops   

Re: [Xen-devel] [PATCH v5 1/1] xen/time: do not decrease steal time after live migration on xen

[Dongli Zhang]

Hi Boris,

On 10/31/2017 08:58 AM, Boris Ostrovsky wrote:
> 
> 
> On 10/30/2017 08:14 PM, Dongli Zhang wrote:
>> Hi Boris,
>>
>> On 10/30/2017 09:34 PM, Boris Ostrovsky wrote:
>>> On 10/30/2017 04:03 AM, Dongli Zhang wrote:
 After guest live migration on xen, steal time in /proc/stat
 (cpustat[CPUTIME_STEAL]) might decrease because steal returned by
 xen_steal_lock() might be less than this_rq()->prev_steal_time which is
 derived from previous return value of xen_steal_clock().

 For instance, steal time of each vcpu is 335 before live migration.

 cpu  198 0 368 200064 1962 0 0 1340 0 0
 cpu0 38 0 81 50063 492 0 0 335 0 0
 cpu1 65 0 97 49763 634 0 0 335 0 0
 cpu2 38 0 81 50098 462 0 0 335 0 0
 cpu3 56 0 107 50138 374 0 0 335 0 0

 After live migration, steal time is reduced to 312.

 cpu  200 0 370 200330 1971 0 0 1248 0 0
 cpu0 38 0 82 50123 500 0 0 312 0 0
 cpu1 65 0 97 49832 634 0 0 312 0 0
 cpu2 39 0 82 50167 462 0 0 312 0 0
 cpu3 56 0 107 50207 374 0 0 312 0 0

 Since runstate times are cumulative and cleared during xen live migration
 by xen hypervisor, the idea of this patch is to accumulate runstate times
 to global percpu variables before live migration suspend. Once guest VM is
 resumed, xen_get_runstate_snapshot_cpu() would always return the sum of new
 runstate times and previously accumulated times stored in global percpu
 variables.

 Similar and more severe issue would impact prior linux 4.8-4.10 as
 discussed by Michael Las at
 https://0xstubs.org/debugging-a-flaky-cpu-steal-time-counter-on-a-paravirtualized-xen-guest,

 which would overflow steal time and lead to 100% st usage in top command
 for linux 4.8-4.10. A backport of this patch would fix that issue.

 References:
 https://0xstubs.org/debugging-a-flaky-cpu-steal-time-counter-on-a-paravirtualized-xen-guest

 Signed-off-by: Dongli Zhang 

 ---
 Changed since v1:
* relocate modification to xen_get_runstate_snapshot_cpu

 Changed since v2:
* accumulate runstate times before live migration

 Changed since v3:
* do not accumulate times in the case of guest checkpointing

 Changed since v4:
* allocate array of vcpu_runstate_info to reduce number of memory 
 allocation

 ---
   drivers/xen/manage.c |  2 ++
   drivers/xen/time.c   | 68
 ++--
   include/xen/interface/vcpu.h |  2 ++
   include/xen/xen-ops.h|  1 +
   4 files changed, 71 insertions(+), 2 deletions(-)

 diff --git a/drivers/xen/manage.c b/drivers/xen/manage.c
 index c425d03..3dc085d 100644
 --- a/drivers/xen/manage.c
 +++ b/drivers/xen/manage.c
 @@ -72,6 +72,7 @@ static int xen_suspend(void *data)
   }
 gnttab_suspend();
 +xen_accumulate_runstate_time(-1);
   xen_arch_pre_suspend();
 /*
 @@ -84,6 +85,7 @@ static int xen_suspend(void *data)
  : 0);
 xen_arch_post_suspend(si->cancelled);
 +xen_accumulate_runstate_time(si->cancelled);
>>>
>>> I am not convinced that the comment above HYPERVISOR_suspend() is
>>> correct. The call can return an error code and so if it returns -EPERM
>>> (which AFAICS it can't now but might in the future) then
>>> xen_accumulate_runstate_time() will do wrong thing.
>>
>> I would split xen_accumulate_runstate_time() into two functions to avoid the
>> -EPERM issue, as one is for saving and another is for accumulation, 
>> respectively.
>>
>> Otherwise, can you use xen_accumulate_runstate_time(2) for saving before 
>> suspend
>> and xen_accumulate_runstate_time(si->cancelled) after resume?
> 
> 
> I'd probably just say something like
> 
> si->cancelled = HYPERVISOR_suspend() ? 1 : 0;

As the call of HYPERVISOR_suspend() takes 3 lines, I would make it as below and
I think gcc would optimize it.

-   /*
-* This hypercall returns 1 if suspend was cancelled
-* or the domain was merely checkpointed, and 0 if it
-* is resuming in a new domain.
-*/
si->cancelled = HYPERVISOR_suspend(xen_pv_domain()
? virt_to_gfn(xen_start_info)
: 0);
+   si->cancelled = si->cancelled ? 1 : 0;

> 
> and keep xen_accumulate_runstate_time() as is (maybe rename it to
> xen_manage_runstate_time()). And also remove the comment above the hypercall 
> as
> it is incorrect (but please mention the reason in the commit message)
> 
>>
>>>
>>>
   gnttab_resume();
 if (!si->cancelled) {
 diff --git a/drivers/xen/time.c b/drivers/xen/time.c
 index ac5f23f..cf3afb9 100644
 --- a/drivers/xen/time.c
 +++ b/drivers/xen/time.c
 @@ -19,6 +19,9 @@
   /* runstate 

[Xen-devel] [qemu-mainline test] 115402: regressions - FAIL

[osstest service owner]

flight 115402 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/115402/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-i3866 xen-buildfail REGR. vs. 114507
 build-amd64-xsm   6 xen-buildfail REGR. vs. 114507
 build-i386-xsm6 xen-buildfail REGR. vs. 114507
 build-amd64   6 xen-buildfail REGR. vs. 114507
 build-armhf   6 xen-buildfail REGR. vs. 114507
 build-armhf-xsm   6 xen-buildfail REGR. vs. 114507

Tests which did not succeed, but are not blocking:
 test-amd64-i386-libvirt-qcow2  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-amd64-i386-freebsd10-i386  1 build-check(1)   blocked  n/a
 test-amd64-amd64-qemuu-nested-intel  1 build-check(1)  blocked n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-armhf-armhf-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm  1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemuu-win10-i386  1 build-check(1)  blocked n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-amd64  1 build-check(1)   blocked  n/a
 test-amd64-amd64-pair 1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-credit2   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-win10-i386  1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-win7-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-pygrub   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-ws16-amd64  1 build-check(1)  blocked n/a
 test-amd64-amd64-xl-qcow2 1 build-check(1)   blocked  n/a
 test-amd64-amd64-amd64-pvgrub  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-arndale   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64  1 build-check(1) blocked n/a
 test-armhf-armhf-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl1 build-check(1)   blocked  n/a
 build-i386-libvirt1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-credit2   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-xsm1 build-check(1)   blocked  n/a
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsm  1 build-check(1)blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-raw1 build-check(1)   blocked  n/a
 test-amd64-i386-qemuu-rhel6hvm-amd  1 build-check(1)   blocked n/a
 test-amd64-amd64-i386-pvgrub  1 build-check(1)   blocked  n/a
 build-armhf-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-ws16-amd64  1 build-check(1) blocked n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvhv2-intel  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-xsm   1 build-check(1)   blocked  n/a
 test-amd64-i386-qemuu-rhel6hvm-intel  1 build-check(1) blocked n/a
 test-armhf-armhf-xl-vhd   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-win7-amd64  1 build-check(1)  blocked n/a
 test-amd64-amd64-xl   1 build-check(1)   blocked  n/a
 test-amd64-i386-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvhv2-amd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-qemuu-nested-amd  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-cubietruck  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-rtds  1 

[Xen-devel] [linux-4.9 test] 115383: regressions - FAIL

[osstest service owner]

flight 115383 linux-4.9 real [real]
http://logs.test-lab.xenproject.org/osstest/logs/115383/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stop   fail REGR. vs. 114814

Tests which are failing intermittently (not blocking):
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 16 guest-localmigrate/x10 
fail in 115368 pass in 115383
 test-armhf-armhf-xl-credit2  16 guest-start/debian.repeat  fail pass in 115368

Tests which did not succeed, but are not blocking:
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 114814
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stopfail like 114814
 test-amd64-amd64-xl-pvhv2-intel 12 guest-start fail never pass
 test-amd64-amd64-xl-pvhv2-amd 12 guest-start  fail  never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qcow2 12 migrate-support-checkfail  never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-xsm  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop  fail never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop  fail never pass
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stop fail never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemut-win10-i386 10 windows-install fail never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass
 test-amd64-amd64-xl-qemut-win10-i386 10 windows-installfail never pass

version targeted for testing:
 linuxd785062ef20f9b2cd8cedcafea55ca8264f25f3e
baseline version:
 linux5d7a76acad403638f635c918cc63d1d44ffa4065

Last test of basis   114814  2017-10-20 20:51:56 Z   10 days
Failing since114845  2017-10-21 16:14:17 Z9 days   16 attempts
Testing same since   115296  2017-10-27 11:07:37 Z3 days6 attempts


People who touched revisions under test:
  Alan Stern 
  Alex Deucher 
  Alexandre Belloni 
  Andrew Morton 
  Andrey Konovalov 
  Anoob Soman 
  Arend van Spriel 
  Arnd Bergmann 
  Bart Van Assche 
  Ben Hutchings 
  Ben Skeggs 
  Bin Liu 
  Borislav Petkov 
  Brian 

Re: [Xen-devel] [PATCH v5 1/1] xen/time: do not decrease steal time after live migration on xen

[Boris Ostrovsky]

On 10/30/2017 08:14 PM, Dongli Zhang wrote:

Hi Boris,

On 10/30/2017 09:34 PM, Boris Ostrovsky wrote:

On 10/30/2017 04:03 AM, Dongli Zhang wrote:

After guest live migration on xen, steal time in /proc/stat
(cpustat[CPUTIME_STEAL]) might decrease because steal returned by
xen_steal_lock() might be less than this_rq()->prev_steal_time which is
derived from previous return value of xen_steal_clock().

For instance, steal time of each vcpu is 335 before live migration.

cpu  198 0 368 200064 1962 0 0 1340 0 0
cpu0 38 0 81 50063 492 0 0 335 0 0
cpu1 65 0 97 49763 634 0 0 335 0 0
cpu2 38 0 81 50098 462 0 0 335 0 0
cpu3 56 0 107 50138 374 0 0 335 0 0

After live migration, steal time is reduced to 312.

cpu  200 0 370 200330 1971 0 0 1248 0 0
cpu0 38 0 82 50123 500 0 0 312 0 0
cpu1 65 0 97 49832 634 0 0 312 0 0
cpu2 39 0 82 50167 462 0 0 312 0 0
cpu3 56 0 107 50207 374 0 0 312 0 0

Since runstate times are cumulative and cleared during xen live migration
by xen hypervisor, the idea of this patch is to accumulate runstate times
to global percpu variables before live migration suspend. Once guest VM is
resumed, xen_get_runstate_snapshot_cpu() would always return the sum of new
runstate times and previously accumulated times stored in global percpu
variables.

Similar and more severe issue would impact prior linux 4.8-4.10 as
discussed by Michael Las at
https://0xstubs.org/debugging-a-flaky-cpu-steal-time-counter-on-a-paravirtualized-xen-guest,
which would overflow steal time and lead to 100% st usage in top command
for linux 4.8-4.10. A backport of this patch would fix that issue.

References: 
https://0xstubs.org/debugging-a-flaky-cpu-steal-time-counter-on-a-paravirtualized-xen-guest
Signed-off-by: Dongli Zhang 

---
Changed since v1:
   * relocate modification to xen_get_runstate_snapshot_cpu

Changed since v2:
   * accumulate runstate times before live migration

Changed since v3:
   * do not accumulate times in the case of guest checkpointing

Changed since v4:
   * allocate array of vcpu_runstate_info to reduce number of memory allocation

---
  drivers/xen/manage.c |  2 ++
  drivers/xen/time.c   | 68 ++--
  include/xen/interface/vcpu.h |  2 ++
  include/xen/xen-ops.h|  1 +
  4 files changed, 71 insertions(+), 2 deletions(-)

diff --git a/drivers/xen/manage.c b/drivers/xen/manage.c
index c425d03..3dc085d 100644
--- a/drivers/xen/manage.c
+++ b/drivers/xen/manage.c
@@ -72,6 +72,7 @@ static int xen_suspend(void *data)
}
  
  	gnttab_suspend();

+   xen_accumulate_runstate_time(-1);
xen_arch_pre_suspend();
  
  	/*

@@ -84,6 +85,7 @@ static int xen_suspend(void *data)
 : 0);
  
  	xen_arch_post_suspend(si->cancelled);

+   xen_accumulate_runstate_time(si->cancelled);


I am not convinced that the comment above HYPERVISOR_suspend() is
correct. The call can return an error code and so if it returns -EPERM
(which AFAICS it can't now but might in the future) then
xen_accumulate_runstate_time() will do wrong thing.


I would split xen_accumulate_runstate_time() into two functions to avoid the
-EPERM issue, as one is for saving and another is for accumulation, 
respectively.

Otherwise, can you use xen_accumulate_runstate_time(2) for saving before suspend
and xen_accumulate_runstate_time(si->cancelled) after resume?



I'd probably just say something like

si->cancelled = HYPERVISOR_suspend() ? 1 : 0;

and keep xen_accumulate_runstate_time() as is (maybe rename it to 
xen_manage_runstate_time()). And also remove the comment above the 
hypercall as it is incorrect (but please mention the reason in the 
commit message)








gnttab_resume();
  
  	if (!si->cancelled) {

diff --git a/drivers/xen/time.c b/drivers/xen/time.c
index ac5f23f..cf3afb9 100644
--- a/drivers/xen/time.c
+++ b/drivers/xen/time.c
@@ -19,6 +19,9 @@
  /* runstate info updated by Xen */
  static DEFINE_PER_CPU(struct vcpu_runstate_info, xen_runstate);
  
+static DEFINE_PER_CPU(u64[RUNSTATE_max], old_runstate_time);

+static struct vcpu_runstate_info *runstate_delta;


I'd move this inside xen_accumulate_runstate_time() since that's the


If we split xen_accumulate_runstate_time() into two functions, we would leave
runstate_delta as global static.


only function that uses it. And why does it need to be
vcpu_runstate_info and not u64[4]?


This was suggested by Juergen to avoid the allocation and reclaim of the second
dimensional array as in v4 of this patch?

Or would you like to allocate sizeof(u64[4]) * num_possible_cpus() and emulate
the 2d array with this 1d array and move the pointer forward sizeof(u64[4]) in
each iteration?



I was thinking of

u64 **runstate_delta = (u64 **)kmalloc(sizeof(xen_runstate.time) * 
num_possible_cpus())


and then you should be able to access runstate_delta[cpu][RUNSTATE_*].






+
  /* return an consistent snapshot of 64-bit time/counter value 

Re: [Xen-devel] [PATCH v5 1/1] xen/time: do not decrease steal time after live migration on xen

[Dongli Zhang]

Hi Boris,

On 10/30/2017 09:34 PM, Boris Ostrovsky wrote:
> On 10/30/2017 04:03 AM, Dongli Zhang wrote:
>> After guest live migration on xen, steal time in /proc/stat
>> (cpustat[CPUTIME_STEAL]) might decrease because steal returned by
>> xen_steal_lock() might be less than this_rq()->prev_steal_time which is
>> derived from previous return value of xen_steal_clock().
>>
>> For instance, steal time of each vcpu is 335 before live migration.
>>
>> cpu  198 0 368 200064 1962 0 0 1340 0 0
>> cpu0 38 0 81 50063 492 0 0 335 0 0
>> cpu1 65 0 97 49763 634 0 0 335 0 0
>> cpu2 38 0 81 50098 462 0 0 335 0 0
>> cpu3 56 0 107 50138 374 0 0 335 0 0
>>
>> After live migration, steal time is reduced to 312.
>>
>> cpu  200 0 370 200330 1971 0 0 1248 0 0
>> cpu0 38 0 82 50123 500 0 0 312 0 0
>> cpu1 65 0 97 49832 634 0 0 312 0 0
>> cpu2 39 0 82 50167 462 0 0 312 0 0
>> cpu3 56 0 107 50207 374 0 0 312 0 0
>>
>> Since runstate times are cumulative and cleared during xen live migration
>> by xen hypervisor, the idea of this patch is to accumulate runstate times
>> to global percpu variables before live migration suspend. Once guest VM is
>> resumed, xen_get_runstate_snapshot_cpu() would always return the sum of new
>> runstate times and previously accumulated times stored in global percpu
>> variables.
>>
>> Similar and more severe issue would impact prior linux 4.8-4.10 as
>> discussed by Michael Las at
>> https://0xstubs.org/debugging-a-flaky-cpu-steal-time-counter-on-a-paravirtualized-xen-guest,
>> which would overflow steal time and lead to 100% st usage in top command
>> for linux 4.8-4.10. A backport of this patch would fix that issue.
>>
>> References: 
>> https://0xstubs.org/debugging-a-flaky-cpu-steal-time-counter-on-a-paravirtualized-xen-guest
>> Signed-off-by: Dongli Zhang 
>>
>> ---
>> Changed since v1:
>>   * relocate modification to xen_get_runstate_snapshot_cpu
>>
>> Changed since v2:
>>   * accumulate runstate times before live migration
>>
>> Changed since v3:
>>   * do not accumulate times in the case of guest checkpointing
>>
>> Changed since v4:
>>   * allocate array of vcpu_runstate_info to reduce number of memory 
>> allocation
>>
>> ---
>>  drivers/xen/manage.c |  2 ++
>>  drivers/xen/time.c   | 68 
>> ++--
>>  include/xen/interface/vcpu.h |  2 ++
>>  include/xen/xen-ops.h|  1 +
>>  4 files changed, 71 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/xen/manage.c b/drivers/xen/manage.c
>> index c425d03..3dc085d 100644
>> --- a/drivers/xen/manage.c
>> +++ b/drivers/xen/manage.c
>> @@ -72,6 +72,7 @@ static int xen_suspend(void *data)
>>  }
>>  
>>  gnttab_suspend();
>> +xen_accumulate_runstate_time(-1);
>>  xen_arch_pre_suspend();
>>  
>>  /*
>> @@ -84,6 +85,7 @@ static int xen_suspend(void *data)
>> : 0);
>>  
>>  xen_arch_post_suspend(si->cancelled);
>> +xen_accumulate_runstate_time(si->cancelled);
> 
> I am not convinced that the comment above HYPERVISOR_suspend() is
> correct. The call can return an error code and so if it returns -EPERM
> (which AFAICS it can't now but might in the future) then
> xen_accumulate_runstate_time() will do wrong thing.

I would split xen_accumulate_runstate_time() into two functions to avoid the
-EPERM issue, as one is for saving and another is for accumulation, 
respectively.

Otherwise, can you use xen_accumulate_runstate_time(2) for saving before suspend
and xen_accumulate_runstate_time(si->cancelled) after resume?

> 
> 
>>  gnttab_resume();
>>  
>>  if (!si->cancelled) {
>> diff --git a/drivers/xen/time.c b/drivers/xen/time.c
>> index ac5f23f..cf3afb9 100644
>> --- a/drivers/xen/time.c
>> +++ b/drivers/xen/time.c
>> @@ -19,6 +19,9 @@
>>  /* runstate info updated by Xen */
>>  static DEFINE_PER_CPU(struct vcpu_runstate_info, xen_runstate);
>>  
>> +static DEFINE_PER_CPU(u64[RUNSTATE_max], old_runstate_time);
>> +static struct vcpu_runstate_info *runstate_delta;
> 
> I'd move this inside xen_accumulate_runstate_time() since that's the

If we split xen_accumulate_runstate_time() into two functions, we would leave
runstate_delta as global static.

> only function that uses it. And why does it need to be
> vcpu_runstate_info and not u64[4]?

This was suggested by Juergen to avoid the allocation and reclaim of the second
dimensional array as in v4 of this patch?

Or would you like to allocate sizeof(u64[4]) * num_possible_cpus() and emulate
the 2d array with this 1d array and move the pointer forward sizeof(u64[4]) in
each iteration?

> 
>> +
>>  /* return an consistent snapshot of 64-bit time/counter value */
>>  static u64 get64(const u64 *p)
>>  {
>> @@ -47,8 +50,8 @@ static u64 get64(const u64 *p)
>>  return ret;
>>  }
>>  
>> -static void xen_get_runstate_snapshot_cpu(struct vcpu_runstate_info *res,
>> -  unsigned int cpu)
>> +static void 

Re: [Xen-devel] [RFC] [Draft Design] ACPI/IORT Support in Xen.

[Goel, Sameer]

On 10/12/2017 3:03 PM, Manish Jaggi wrote:
> ACPI/IORT Support in Xen.
> --
> 
> I had sent out patch series [0] to hide smmu from Dom0 IORT. Extending the 
> scope
> and including all that is required to support ACPI/IORT in Xen. Presenting 
> for review
> first _draft_ of design of ACPI/IORT support in Xen. Not complete though.
> 
> Discussed is the parsing and generation of IORT table for Dom0 and DomUs.
> It is proposed that IORT be parsed and the information in saved into xen 
> data-structure
> say host_iort_struct and is reused by all xen subsystems like ITS / SMMU etc.
> 
> Since this is first draft is open to technical comments, modifications
> and suggestions. Please be open and feel free to add any missing points / 
> additions.
> 
> 1. What is IORT. What are its components ?
> 2. Current Support in Xen
> 3. IORT for Dom0
> 4. IORT for DomU
> 5. Parsing of IORT in Xen
> 6. Generation of IORT
> 7. Future Work and TODOs
> 
> 1. What is IORT. What are its components ?
> 
> IORT refers to Input Output remapping table. It is essentially used to find
> information about the IO topology (PCIRC-SMMU-ITS) and relationships between
> devices.
> 
> A general structure of IORT is has nodes which have information about PCI RC,
> SMMU, ITS and Platform devices. Using an IORT table relationship between
> RID -> StreamID -> DeviceId can be obtained. More specifically which device is
> behind which SMMU and which interrupt controller, this topology is described 
> in
> IORT Table.
> 
> RID is a requester ID in PCI context,
> StreamID is the ID of the device in SMMU context,
> DeviceID is the ID programmed in ITS.
> 
> For a non-pci device RID could be simply an ID.
> 
> Each iort_node contains an ID map array to translate from one ID into another.
> IDmap Entry {input_range, output_range, output_node_ref, id_count}
> This array is present in PCI RC node,SMMU node, Named component node etc
> and can reference to a SMMU or ITS node.
> 
> 2. Current Support of IORT
> ---
> Currently Xen passes host IORT table to dom0 without any modifications.
> For DomU no IORT table is passed.
> 
> 3. IORT for Dom0
> -
> IORT for Dom0 is prepared by xen and it is fairly similar to the host iort.
> However few nodes could be removed removed or modified. For instance
> - host SMMU nodes should not be present
> - ITS group nodes are same as host iort but, no stage2 mapping is done for 
> them.
> - platform nodes (named components) may be selectively present depending on 
> the
> case where xen is using some. This could be controlled by  xen command line.
> - More items : TODO
> 
> 4. IORT for DomU
> ---
 
> IORT for DomU is generated by the toolstack. IORT topology is different when
> DomU supports device passthrough.
> 
> At a minimum domU IORT should include a single PCIRC and ITS Group.
> Similar PCIRC can be added in DSDT.
> Additional node can be added if platform device is assigned to domU.
> No extra node should be required for PCI device pass-through.
> 
> It is proposed that the idrange of PCIRC and ITS group be constant for domUs.
> In case if PCI PT,using a domctl toolstack can communicate
> physical RID: virtual RID, deviceID: virtual deviceID to xen.
> 
> It is assumed that domU PCI Config access would be trapped in Xen. The RID at
> which assigned device is enumerated would be the one provided by the domctl,
> domctl_set_deviceid_mapping
> 
> TODO: device assign domctl i/f.
> 
> Note: This should suffice the virtual deviceID support pointed by Andre. [4]
> We might not need this domctl if assign_device hypercall is extended to 
> provide this information.
> 
> 5. Parsing of IORT in Xen
> --
I think a Linux like approach will solve the following use cases:
1. Identify the SMMU devices and initialize the devices as needed.
2. API function to setup SMMUs in response to a discovery notification from DOM0
   - We will still need a path for non pcie devices.
   - I agree with Andre that the use cases for the named nodes in IORT should 
be treated the same as PCIe RC devices.
3. The concept of fwnode is still valid as per 4.14 and we can try reuse most 
of the parsing code.

Manish, I looked at your old patch and had a couple of questions before I 
comment more on this design. From an initial 
glance, it seems that you should be able to hide SMMUs by calling the already 
defined API functions in the iort.c implementation
(for most part :)). 

I am wondering if we really need to keep a list of parsed nodes. Or which use 
case apart from hw dom IORT mandates this?

> IORT nodes can be saved in structures so that IORT table parsing can be done
> once and is reused by all xen subsystems like ITS / SMMU etc, domain creation.
> Proposed are the structures to hold IORT information, very similar to ACPI
> structures.
> 
> iort_id_map {
>     range_t input_range;
>     range_t 

[Xen-devel] [PATCH v8 01/13] xen/pvcalls: introduce the pvcalls xenbus frontend

[Stefano Stabellini]

Introduce a xenbus frontend for the pvcalls protocol, as defined by
https://xenbits.xen.org/docs/unstable/misc/pvcalls.html.

This patch only adds the stubs, the code will be added by the following
patches.

Signed-off-by: Stefano Stabellini 
Reviewed-by: Boris Ostrovsky 
CC: boris.ostrov...@oracle.com
CC: jgr...@suse.com
---
 drivers/xen/pvcalls-front.c | 61 +
 1 file changed, 61 insertions(+)
 create mode 100644 drivers/xen/pvcalls-front.c

diff --git a/drivers/xen/pvcalls-front.c b/drivers/xen/pvcalls-front.c
new file mode 100644
index 000..a8d38c2
--- /dev/null
+++ b/drivers/xen/pvcalls-front.c
@@ -0,0 +1,61 @@
+/*
+ * (c) 2017 Stefano Stabellini 
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include 
+
+#include 
+#include 
+#include 
+#include 
+#include 
+
+static const struct xenbus_device_id pvcalls_front_ids[] = {
+   { "pvcalls" },
+   { "" }
+};
+
+static int pvcalls_front_remove(struct xenbus_device *dev)
+{
+   return 0;
+}
+
+static int pvcalls_front_probe(struct xenbus_device *dev,
+ const struct xenbus_device_id *id)
+{
+   return 0;
+}
+
+static void pvcalls_front_changed(struct xenbus_device *dev,
+   enum xenbus_state backend_state)
+{
+}
+
+static struct xenbus_driver pvcalls_front_driver = {
+   .ids = pvcalls_front_ids,
+   .probe = pvcalls_front_probe,
+   .remove = pvcalls_front_remove,
+   .otherend_changed = pvcalls_front_changed,
+};
+
+static int __init pvcalls_frontend_init(void)
+{
+   if (!xen_domain())
+   return -ENODEV;
+
+   pr_info("Initialising Xen pvcalls frontend driver\n");
+
+   return xenbus_register_frontend(_front_driver);
+}
+
+module_init(pvcalls_frontend_init);
-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v8 13/13] xen: introduce a Kconfig option to enable the pvcalls frontend

[Stefano Stabellini]

Also add pvcalls-front to the Makefile.

Signed-off-by: Stefano Stabellini 
Reviewed-by: Boris Ostrovsky 
CC: boris.ostrov...@oracle.com
CC: jgr...@suse.com
---
 drivers/xen/Kconfig  | 11 +++
 drivers/xen/Makefile |  1 +
 2 files changed, 12 insertions(+)

diff --git a/drivers/xen/Kconfig b/drivers/xen/Kconfig
index 4545561..d8dd546 100644
--- a/drivers/xen/Kconfig
+++ b/drivers/xen/Kconfig
@@ -196,6 +196,17 @@ config XEN_PCIDEV_BACKEND
 
  If in doubt, say m.
 
+config XEN_PVCALLS_FRONTEND
+   tristate "XEN PV Calls frontend driver"
+   depends on INET && XEN
+   default n
+   select XEN_XENBUS_FRONTEND
+   help
+ Experimental frontend for the Xen PV Calls protocol
+ (https://xenbits.xen.org/docs/unstable/misc/pvcalls.html). It
+ sends a small set of POSIX calls to the backend, which
+ implements them.
+
 config XEN_PVCALLS_BACKEND
bool "XEN PV Calls backend driver"
depends on INET && XEN && XEN_BACKEND
diff --git a/drivers/xen/Makefile b/drivers/xen/Makefile
index caaa15d..82466f2 100644
--- a/drivers/xen/Makefile
+++ b/drivers/xen/Makefile
@@ -36,6 +36,7 @@ obj-$(CONFIG_XEN_EFI) += efi.o
 obj-$(CONFIG_XEN_SCSI_BACKEND) += xen-scsiback.o
 obj-$(CONFIG_XEN_AUTO_XLATE)   += xlate_mmu.o
 obj-$(CONFIG_XEN_PVCALLS_BACKEND)  += pvcalls-back.o
+obj-$(CONFIG_XEN_PVCALLS_FRONTEND) += pvcalls-front.o
 xen-evtchn-y   := evtchn.o
 xen-gntdev-y   := gntdev.o
 xen-gntalloc-y := gntalloc.o
-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v8 07/13] xen/pvcalls: implement listen command

[Stefano Stabellini]

Send PVCALLS_LISTEN to the backend.

Signed-off-by: Stefano Stabellini 
Reviewed-by: Boris Ostrovsky 
CC: boris.ostrov...@oracle.com
CC: jgr...@suse.com
---
 drivers/xen/pvcalls-front.c | 57 +
 drivers/xen/pvcalls-front.h |  1 +
 2 files changed, 58 insertions(+)

diff --git a/drivers/xen/pvcalls-front.c b/drivers/xen/pvcalls-front.c
index 6e70ec3..09b2a64 100644
--- a/drivers/xen/pvcalls-front.c
+++ b/drivers/xen/pvcalls-front.c
@@ -412,6 +412,63 @@ int pvcalls_front_bind(struct socket *sock, struct 
sockaddr *addr, int addr_len)
return 0;
 }
 
+int pvcalls_front_listen(struct socket *sock, int backlog)
+{
+   struct pvcalls_bedata *bedata;
+   struct sock_mapping *map;
+   struct xen_pvcalls_request *req;
+   int notify, req_id, ret;
+
+   pvcalls_enter();
+   if (!pvcalls_front_dev) {
+   pvcalls_exit();
+   return -ENOTCONN;
+   }
+   bedata = dev_get_drvdata(_front_dev->dev);
+
+   map = (struct sock_mapping *) sock->sk->sk_send_head;
+   if (!map) {
+   pvcalls_exit();
+   return -ENOTSOCK;
+   }
+
+   if (map->passive.status != PVCALLS_STATUS_BIND) {
+   pvcalls_exit();
+   return -EOPNOTSUPP;
+   }
+
+   spin_lock(>socket_lock);
+   ret = get_request(bedata, _id);
+   if (ret < 0) {
+   spin_unlock(>socket_lock);
+   pvcalls_exit();
+   return ret;
+   }
+   req = RING_GET_REQUEST(>ring, req_id);
+   req->req_id = req_id;
+   req->cmd = PVCALLS_LISTEN;
+   req->u.listen.id = (uintptr_t) map;
+   req->u.listen.backlog = backlog;
+
+   bedata->ring.req_prod_pvt++;
+   RING_PUSH_REQUESTS_AND_CHECK_NOTIFY(>ring, notify);
+   spin_unlock(>socket_lock);
+   if (notify)
+   notify_remote_via_irq(bedata->irq);
+
+   wait_event(bedata->inflight_req,
+  READ_ONCE(bedata->rsp[req_id].req_id) == req_id);
+
+   /* read req_id, then the content */
+   smp_rmb();
+   ret = bedata->rsp[req_id].ret;
+   bedata->rsp[req_id].req_id = PVCALLS_INVALID_ID;
+
+   map->passive.status = PVCALLS_STATUS_LISTEN;
+   pvcalls_exit();
+   return ret;
+}
+
 static const struct xenbus_device_id pvcalls_front_ids[] = {
{ "pvcalls" },
{ "" }
diff --git a/drivers/xen/pvcalls-front.h b/drivers/xen/pvcalls-front.h
index 8b0a274..aa8fe10 100644
--- a/drivers/xen/pvcalls-front.h
+++ b/drivers/xen/pvcalls-front.h
@@ -9,5 +9,6 @@ int pvcalls_front_connect(struct socket *sock, struct sockaddr 
*addr,
 int pvcalls_front_bind(struct socket *sock,
   struct sockaddr *addr,
   int addr_len);
+int pvcalls_front_listen(struct socket *sock, int backlog);
 
 #endif
-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v8 11/13] xen/pvcalls: implement poll command

[Stefano Stabellini]

For active sockets, check the indexes and use the inflight_conn_req
waitqueue to wait.

For passive sockets if an accept is outstanding
(PVCALLS_FLAG_ACCEPT_INFLIGHT), check if it has been answered by looking
at bedata->rsp[req_id]. If so, return POLLIN.  Otherwise use the
inflight_accept_req waitqueue.

If no accepts are inflight, send PVCALLS_POLL to the backend. If we have
outstanding POLL requests awaiting for a response use the inflight_req
waitqueue: inflight_req is awaken when a new response is received; on
wakeup we check whether the POLL response is arrived by looking at the
PVCALLS_FLAG_POLL_RET flag. We set the flag from
pvcalls_front_event_handler, if the response was for a POLL command.

In pvcalls_front_event_handler, get the struct sock_mapping from the
poll id (we previously converted struct sock_mapping* to uintptr_t and
used it as id).

Signed-off-by: Stefano Stabellini 
Reviewed-by: Boris Ostrovsky 
CC: boris.ostrov...@oracle.com
CC: jgr...@suse.com
---
 drivers/xen/pvcalls-front.c | 144 +---
 drivers/xen/pvcalls-front.h |   3 +
 2 files changed, 138 insertions(+), 9 deletions(-)

diff --git a/drivers/xen/pvcalls-front.c b/drivers/xen/pvcalls-front.c
index 24ffdc9..c7d4251 100644
--- a/drivers/xen/pvcalls-front.c
+++ b/drivers/xen/pvcalls-front.c
@@ -83,6 +83,8 @@ struct sock_mapping {
 * Only one poll operation can be inflight for a given socket.
 */
 #define PVCALLS_FLAG_ACCEPT_INFLIGHT 0
+#define PVCALLS_FLAG_POLL_INFLIGHT   1
+#define PVCALLS_FLAG_POLL_RET2
uint8_t flags;
uint32_t inflight_req_id;
struct sock_mapping *accept_map;
@@ -154,15 +156,32 @@ static irqreturn_t pvcalls_front_event_handler(int irq, 
void *dev_id)
rsp = RING_GET_RESPONSE(>ring, bedata->ring.rsp_cons);
 
req_id = rsp->req_id;
-   dst = (uint8_t *)>rsp[req_id] + sizeof(rsp->req_id);
-   src = (uint8_t *)rsp + sizeof(rsp->req_id);
-   memcpy(dst, src, sizeof(*rsp) - sizeof(rsp->req_id));
-   /*
-* First copy the rest of the data, then req_id. It is
-* paired with the barrier when accessing bedata->rsp.
-*/
-   smp_wmb();
-   bedata->rsp[req_id].req_id = rsp->req_id;
+   if (rsp->cmd == PVCALLS_POLL) {
+   struct sock_mapping *map = (struct sock_mapping 
*)(uintptr_t)
+  rsp->u.poll.id;
+
+   clear_bit(PVCALLS_FLAG_POLL_INFLIGHT,
+ (void *)>passive.flags);
+   /*
+* clear INFLIGHT, then set RET. It pairs with
+* the checks at the beginning of
+* pvcalls_front_poll_passive.
+*/
+   smp_wmb();
+   set_bit(PVCALLS_FLAG_POLL_RET,
+   (void *)>passive.flags);
+   } else {
+   dst = (uint8_t *)>rsp[req_id] +
+ sizeof(rsp->req_id);
+   src = (uint8_t *)rsp + sizeof(rsp->req_id);
+   memcpy(dst, src, sizeof(*rsp) - sizeof(rsp->req_id));
+   /*
+* First copy the rest of the data, then req_id. It is
+* paired with the barrier when accessing bedata->rsp.
+*/
+   smp_wmb();
+   bedata->rsp[req_id].req_id = req_id;
+   }
 
done = 1;
bedata->ring.rsp_cons++;
@@ -846,6 +865,113 @@ int pvcalls_front_accept(struct socket *sock, struct 
socket *newsock, int flags)
return ret;
 }
 
+static unsigned int pvcalls_front_poll_passive(struct file *file,
+  struct pvcalls_bedata *bedata,
+  struct sock_mapping *map,
+  poll_table *wait)
+{
+   int notify, req_id, ret;
+   struct xen_pvcalls_request *req;
+
+   if (test_bit(PVCALLS_FLAG_ACCEPT_INFLIGHT,
+(void *)>passive.flags)) {
+   uint32_t req_id = READ_ONCE(map->passive.inflight_req_id);
+
+   if (req_id != PVCALLS_INVALID_ID &&
+   READ_ONCE(bedata->rsp[req_id].req_id) == req_id)
+   return POLLIN | POLLRDNORM;
+
+   poll_wait(file, >passive.inflight_accept_req, wait);
+   return 0;
+   }
+
+   if (test_and_clear_bit(PVCALLS_FLAG_POLL_RET,
+  (void *)>passive.flags))
+   return POLLIN | POLLRDNORM;
+
+   /*
+* First check RET, then INFLIGHT. No 

[Xen-devel] [PATCH v8 09/13] xen/pvcalls: implement sendmsg

[Stefano Stabellini]

Send data to an active socket by copying data to the "out" ring. Take
the active socket out_mutex so that only one function can access the
ring at any given time.

If not enough room is available on the ring, rather than returning
immediately or sleep-waiting, spin for up to 5000 cycles. This small
optimization turns out to improve performance significantly.

Signed-off-by: Stefano Stabellini 
Reviewed-by: Boris Ostrovsky 
CC: boris.ostrov...@oracle.com
CC: jgr...@suse.com
---
 drivers/xen/pvcalls-front.c | 121 
 drivers/xen/pvcalls-front.h |   3 ++
 2 files changed, 124 insertions(+)

diff --git a/drivers/xen/pvcalls-front.c b/drivers/xen/pvcalls-front.c
index d781ac4..7672578 100644
--- a/drivers/xen/pvcalls-front.c
+++ b/drivers/xen/pvcalls-front.c
@@ -29,6 +29,7 @@
 #define PVCALLS_INVALID_ID UINT_MAX
 #define PVCALLS_RING_ORDER XENBUS_MAX_RING_GRANT_ORDER
 #define PVCALLS_NR_RSP_PER_RING __CONST_RING_SIZE(xen_pvcalls, XEN_PAGE_SIZE)
+#define PVCALLS_FRONT_MAX_SPIN 5000
 
 struct pvcalls_bedata {
struct xen_pvcalls_front_ring ring;
@@ -99,6 +100,23 @@ static inline int get_request(struct pvcalls_bedata 
*bedata, int *req_id)
return 0;
 }
 
+static bool pvcalls_front_write_todo(struct sock_mapping *map)
+{
+   struct pvcalls_data_intf *intf = map->active.ring;
+   RING_IDX cons, prod, size = XEN_FLEX_RING_SIZE(PVCALLS_RING_ORDER);
+   int32_t error;
+
+   error = intf->out_error;
+   if (error == -ENOTCONN)
+   return false;
+   if (error != 0)
+   return true;
+
+   cons = intf->out_cons;
+   prod = intf->out_prod;
+   return !!(size - pvcalls_queued(prod, cons, size));
+}
+
 static irqreturn_t pvcalls_front_event_handler(int irq, void *dev_id)
 {
struct xenbus_device *dev = dev_id;
@@ -363,6 +381,109 @@ int pvcalls_front_connect(struct socket *sock, struct 
sockaddr *addr,
return ret;
 }
 
+static int __write_ring(struct pvcalls_data_intf *intf,
+   struct pvcalls_data *data,
+   struct iov_iter *msg_iter,
+   int len)
+{
+   RING_IDX cons, prod, size, masked_prod, masked_cons;
+   RING_IDX array_size = XEN_FLEX_RING_SIZE(PVCALLS_RING_ORDER);
+   int32_t error;
+
+   error = intf->out_error;
+   if (error < 0)
+   return error;
+   cons = intf->out_cons;
+   prod = intf->out_prod;
+   /* read indexes before continuing */
+   virt_mb();
+
+   size = pvcalls_queued(prod, cons, array_size);
+   if (size >= array_size)
+   return -EINVAL;
+   if (len > array_size - size)
+   len = array_size - size;
+
+   masked_prod = pvcalls_mask(prod, array_size);
+   masked_cons = pvcalls_mask(cons, array_size);
+
+   if (masked_prod < masked_cons) {
+   len = copy_from_iter(data->out + masked_prod, len, msg_iter);
+   } else {
+   if (len > array_size - masked_prod) {
+   int ret = copy_from_iter(data->out + masked_prod,
+  array_size - masked_prod, msg_iter);
+   if (ret != array_size - masked_prod) {
+   len = ret;
+   goto out;
+   }
+   len = ret + copy_from_iter(data->out, len - ret, 
msg_iter);
+   } else {
+   len = copy_from_iter(data->out + masked_prod, len, 
msg_iter);
+   }
+   }
+out:
+   /* write to ring before updating pointer */
+   virt_wmb();
+   intf->out_prod += len;
+
+   return len;
+}
+
+int pvcalls_front_sendmsg(struct socket *sock, struct msghdr *msg,
+ size_t len)
+{
+   struct pvcalls_bedata *bedata;
+   struct sock_mapping *map;
+   int sent, tot_sent = 0;
+   int count = 0, flags;
+
+   flags = msg->msg_flags;
+   if (flags & (MSG_CONFIRM|MSG_DONTROUTE|MSG_EOR|MSG_OOB))
+   return -EOPNOTSUPP;
+
+   pvcalls_enter();
+   if (!pvcalls_front_dev) {
+   pvcalls_exit();
+   return -ENOTCONN;
+   }
+   bedata = dev_get_drvdata(_front_dev->dev);
+
+   map = (struct sock_mapping *) sock->sk->sk_send_head;
+   if (!map) {
+   pvcalls_exit();
+   return -ENOTSOCK;
+   }
+
+   mutex_lock(>active.out_mutex);
+   if ((flags & MSG_DONTWAIT) && !pvcalls_front_write_todo(map)) {
+   mutex_unlock(>active.out_mutex);
+   pvcalls_exit();
+   return -EAGAIN;
+   }
+   if (len > INT_MAX)
+   len = INT_MAX;
+
+again:
+   count++;
+   sent = __write_ring(map->active.ring,
+   >active.data, >msg_iter,
+   len);
+   if (sent > 0) {
+   len -= sent;
+

[Xen-devel] [PATCH v8 12/13] xen/pvcalls: implement release command

[Stefano Stabellini]

Send PVCALLS_RELEASE to the backend and wait for a reply. Take both
in_mutex and out_mutex to avoid concurrent accesses. Then, free the
socket.

For passive sockets, check whether we have already pre-allocated an
active socket for the purpose of being accepted. If so, free that as
well.

Signed-off-by: Stefano Stabellini 
Reviewed-by: Boris Ostrovsky 
CC: boris.ostrov...@oracle.com
CC: jgr...@suse.com
---
 drivers/xen/pvcalls-front.c | 98 +
 drivers/xen/pvcalls-front.h |  1 +
 2 files changed, 99 insertions(+)

diff --git a/drivers/xen/pvcalls-front.c b/drivers/xen/pvcalls-front.c
index c7d4251..0c1ec68 100644
--- a/drivers/xen/pvcalls-front.c
+++ b/drivers/xen/pvcalls-front.c
@@ -199,6 +199,21 @@ static irqreturn_t pvcalls_front_event_handler(int irq, 
void *dev_id)
 static void pvcalls_front_free_map(struct pvcalls_bedata *bedata,
   struct sock_mapping *map)
 {
+   int i;
+
+   unbind_from_irqhandler(map->active.irq, map);
+
+   spin_lock(>socket_lock);
+   if (!list_empty(>list))
+   list_del_init(>list);
+   spin_unlock(>socket_lock);
+
+   for (i = 0; i < (1 << PVCALLS_RING_ORDER); i++)
+   gnttab_end_foreign_access(map->active.ring->ref[i], 0, 0);
+   gnttab_end_foreign_access(map->active.ref, 0, 0);
+   free_page((unsigned long)map->active.ring);
+
+   kfree(map);
 }
 
 static irqreturn_t pvcalls_front_conn_handler(int irq, void *sock_map)
@@ -972,6 +987,89 @@ unsigned int pvcalls_front_poll(struct file *file, struct 
socket *sock,
return ret;
 }
 
+int pvcalls_front_release(struct socket *sock)
+{
+   struct pvcalls_bedata *bedata;
+   struct sock_mapping *map;
+   int req_id, notify, ret;
+   struct xen_pvcalls_request *req;
+
+   if (sock->sk == NULL)
+   return 0;
+
+   pvcalls_enter();
+   if (!pvcalls_front_dev) {
+   pvcalls_exit();
+   return -EIO;
+   }
+
+   bedata = dev_get_drvdata(_front_dev->dev);
+
+   map = (struct sock_mapping *) sock->sk->sk_send_head;
+   if (map == NULL) {
+   pvcalls_exit();
+   return 0;
+   }
+
+   spin_lock(>socket_lock);
+   ret = get_request(bedata, _id);
+   if (ret < 0) {
+   spin_unlock(>socket_lock);
+   pvcalls_exit();
+   return ret;
+   }
+   sock->sk->sk_send_head = NULL;
+
+   req = RING_GET_REQUEST(>ring, req_id);
+   req->req_id = req_id;
+   req->cmd = PVCALLS_RELEASE;
+   req->u.release.id = (uintptr_t)map;
+
+   bedata->ring.req_prod_pvt++;
+   RING_PUSH_REQUESTS_AND_CHECK_NOTIFY(>ring, notify);
+   spin_unlock(>socket_lock);
+   if (notify)
+   notify_remote_via_irq(bedata->irq);
+
+   wait_event(bedata->inflight_req,
+  READ_ONCE(bedata->rsp[req_id].req_id) == req_id);
+
+   if (map->active_socket) {
+   /*
+* Set in_error and wake up inflight_conn_req to force
+* recvmsg waiters to exit.
+*/
+   map->active.ring->in_error = -EBADF;
+   wake_up_interruptible(>active.inflight_conn_req);
+
+   /*
+* Wait until there are no more waiters on the mutexes.
+* We know that no new waiters can be added because sk_send_head
+* is set to NULL -- we only need to wait for the existing
+* waiters to return.
+*/
+   while (!mutex_trylock(>active.in_mutex) ||
+  !mutex_trylock(>active.out_mutex))
+   cpu_relax();
+
+   pvcalls_front_free_map(bedata, map);
+   } else {
+   spin_lock(>socket_lock);
+   list_del(>list);
+   spin_unlock(>socket_lock);
+   if (READ_ONCE(map->passive.inflight_req_id) !=
+   PVCALLS_INVALID_ID) {
+   pvcalls_front_free_map(bedata,
+  map->passive.accept_map);
+   }
+   kfree(map);
+   }
+   WRITE_ONCE(bedata->rsp[req_id].req_id, PVCALLS_INVALID_ID);
+
+   pvcalls_exit();
+   return 0;
+}
+
 static const struct xenbus_device_id pvcalls_front_ids[] = {
{ "pvcalls" },
{ "" }
diff --git a/drivers/xen/pvcalls-front.h b/drivers/xen/pvcalls-front.h
index 25e05b8..3332978 100644
--- a/drivers/xen/pvcalls-front.h
+++ b/drivers/xen/pvcalls-front.h
@@ -23,5 +23,6 @@ int pvcalls_front_recvmsg(struct socket *sock,
 unsigned int pvcalls_front_poll(struct file *file,
struct socket *sock,
poll_table *wait);
+int pvcalls_front_release(struct socket *sock);
 
 #endif
-- 
1.9.1


___
Xen-devel mailing 

[Xen-devel] [PATCH v8 10/13] xen/pvcalls: implement recvmsg

[Stefano Stabellini]

Implement recvmsg by copying data from the "in" ring. If not enough data
is available and the recvmsg call is blocking, then wait on the
inflight_conn_req waitqueue. Take the active socket in_mutex so that
only one function can access the ring at any given time.

Signed-off-by: Stefano Stabellini 
Reviewed-by: Boris Ostrovsky 
CC: boris.ostrov...@oracle.com
CC: jgr...@suse.com
---
 drivers/xen/pvcalls-front.c | 111 
 drivers/xen/pvcalls-front.h |   4 ++
 2 files changed, 115 insertions(+)

diff --git a/drivers/xen/pvcalls-front.c b/drivers/xen/pvcalls-front.c
index 7672578..24ffdc9 100644
--- a/drivers/xen/pvcalls-front.c
+++ b/drivers/xen/pvcalls-front.c
@@ -117,6 +117,20 @@ static bool pvcalls_front_write_todo(struct sock_mapping 
*map)
return !!(size - pvcalls_queued(prod, cons, size));
 }
 
+static bool pvcalls_front_read_todo(struct sock_mapping *map)
+{
+   struct pvcalls_data_intf *intf = map->active.ring;
+   RING_IDX cons, prod;
+   int32_t error;
+
+   cons = intf->in_cons;
+   prod = intf->in_prod;
+   error = intf->in_error;
+   return (error != 0 ||
+   pvcalls_queued(prod, cons,
+  XEN_FLEX_RING_SIZE(PVCALLS_RING_ORDER)) != 0);
+}
+
 static irqreturn_t pvcalls_front_event_handler(int irq, void *dev_id)
 {
struct xenbus_device *dev = dev_id;
@@ -484,6 +498,103 @@ int pvcalls_front_sendmsg(struct socket *sock, struct 
msghdr *msg,
return tot_sent;
 }
 
+static int __read_ring(struct pvcalls_data_intf *intf,
+  struct pvcalls_data *data,
+  struct iov_iter *msg_iter,
+  size_t len, int flags)
+{
+   RING_IDX cons, prod, size, masked_prod, masked_cons;
+   RING_IDX array_size = XEN_FLEX_RING_SIZE(PVCALLS_RING_ORDER);
+   int32_t error;
+
+   cons = intf->in_cons;
+   prod = intf->in_prod;
+   error = intf->in_error;
+   /* get pointers before reading from the ring */
+   virt_rmb();
+   if (error < 0)
+   return error;
+
+   size = pvcalls_queued(prod, cons, array_size);
+   masked_prod = pvcalls_mask(prod, array_size);
+   masked_cons = pvcalls_mask(cons, array_size);
+
+   if (size == 0)
+   return 0;
+
+   if (len > size)
+   len = size;
+
+   if (masked_prod > masked_cons) {
+   len = copy_to_iter(data->in + masked_cons, len, msg_iter);
+   } else {
+   if (len > (array_size - masked_cons)) {
+   int ret = copy_to_iter(data->in + masked_cons,
+array_size - masked_cons, msg_iter);
+   if (ret != array_size - masked_cons) {
+   len = ret;
+   goto out;
+   }
+   len = ret + copy_to_iter(data->in, len - ret, msg_iter);
+   } else {
+   len = copy_to_iter(data->in + masked_cons, len, 
msg_iter);
+   }
+   }
+out:
+   /* read data from the ring before increasing the index */
+   virt_mb();
+   if (!(flags & MSG_PEEK))
+   intf->in_cons += len;
+
+   return len;
+}
+
+int pvcalls_front_recvmsg(struct socket *sock, struct msghdr *msg, size_t len,
+int flags)
+{
+   struct pvcalls_bedata *bedata;
+   int ret;
+   struct sock_mapping *map;
+
+   if (flags & (MSG_CMSG_CLOEXEC|MSG_ERRQUEUE|MSG_OOB|MSG_TRUNC))
+   return -EOPNOTSUPP;
+
+   pvcalls_enter();
+   if (!pvcalls_front_dev) {
+   pvcalls_exit();
+   return -ENOTCONN;
+   }
+   bedata = dev_get_drvdata(_front_dev->dev);
+
+   map = (struct sock_mapping *) sock->sk->sk_send_head;
+   if (!map) {
+   pvcalls_exit();
+   return -ENOTSOCK;
+   }
+
+   mutex_lock(>active.in_mutex);
+   if (len > XEN_FLEX_RING_SIZE(PVCALLS_RING_ORDER))
+   len = XEN_FLEX_RING_SIZE(PVCALLS_RING_ORDER);
+
+   while (!(flags & MSG_DONTWAIT) && !pvcalls_front_read_todo(map)) {
+   wait_event_interruptible(map->active.inflight_conn_req,
+pvcalls_front_read_todo(map));
+   }
+   ret = __read_ring(map->active.ring, >active.data,
+ >msg_iter, len, flags);
+
+   if (ret > 0)
+   notify_remote_via_irq(map->active.irq);
+   if (ret == 0)
+   ret = (flags & MSG_DONTWAIT) ? -EAGAIN : 0;
+   if (ret == -ENOTCONN)
+   ret = 0;
+
+   mutex_unlock(>active.in_mutex);
+   pvcalls_exit();
+   return ret;
+}
+
 int pvcalls_front_bind(struct socket *sock, struct sockaddr *addr, int 
addr_len)
 {
struct pvcalls_bedata *bedata;
diff --git a/drivers/xen/pvcalls-front.h 

[Xen-devel] [PATCH v8 08/13] xen/pvcalls: implement accept command

[Stefano Stabellini]

Introduce a waitqueue to allow only one outstanding accept command at
any given time and to implement polling on the passive socket. Introduce
a flags field to keep track of in-flight accept and poll commands.

Send PVCALLS_ACCEPT to the backend. Allocate a new active socket. Make
sure that only one accept command is executed at any given time by
setting PVCALLS_FLAG_ACCEPT_INFLIGHT and waiting on the
inflight_accept_req waitqueue.

Convert the new struct sock_mapping pointer into an uintptr_t and use it
as id for the new socket to pass to the backend.

Check if the accept call is non-blocking: in that case after sending the
ACCEPT command to the backend store the sock_mapping pointer of the new
struct and the inflight req_id then return -EAGAIN (which will respond
only when there is something to accept). Next time accept is called,
we'll check if the ACCEPT command has been answered, if so we'll pick up
where we left off, otherwise we return -EAGAIN again.

Note that, differently from the other commands, we can use
wait_event_interruptible (instead of wait_event) in the case of accept
as we are able to track the req_id of the ACCEPT response that we are
waiting.

Signed-off-by: Stefano Stabellini 
Reviewed-by: Boris Ostrovsky 
CC: boris.ostrov...@oracle.com
CC: jgr...@suse.com
---
 drivers/xen/pvcalls-front.c | 145 
 drivers/xen/pvcalls-front.h |   3 +
 2 files changed, 148 insertions(+)

diff --git a/drivers/xen/pvcalls-front.c b/drivers/xen/pvcalls-front.c
index 09b2a64..d781ac4 100644
--- a/drivers/xen/pvcalls-front.c
+++ b/drivers/xen/pvcalls-front.c
@@ -76,6 +76,16 @@ struct sock_mapping {
 #define PVCALLS_STATUS_BIND  1
 #define PVCALLS_STATUS_LISTEN2
uint8_t status;
+   /*
+* Internal state-machine flags.
+* Only one accept operation can be inflight for a socket.
+* Only one poll operation can be inflight for a given socket.
+*/
+#define PVCALLS_FLAG_ACCEPT_INFLIGHT 0
+   uint8_t flags;
+   uint32_t inflight_req_id;
+   struct sock_mapping *accept_map;
+   wait_queue_head_t inflight_accept_req;
} passive;
};
 };
@@ -391,6 +401,8 @@ int pvcalls_front_bind(struct socket *sock, struct sockaddr 
*addr, int addr_len)
memcpy(req->u.bind.addr, addr, sizeof(*addr));
req->u.bind.len = addr_len;
 
+   init_waitqueue_head(>passive.inflight_accept_req);
+
map->active_socket = false;
 
bedata->ring.req_prod_pvt++;
@@ -469,6 +481,139 @@ int pvcalls_front_listen(struct socket *sock, int backlog)
return ret;
 }
 
+int pvcalls_front_accept(struct socket *sock, struct socket *newsock, int 
flags)
+{
+   struct pvcalls_bedata *bedata;
+   struct sock_mapping *map;
+   struct sock_mapping *map2 = NULL;
+   struct xen_pvcalls_request *req;
+   int notify, req_id, ret, evtchn, nonblock;
+
+   pvcalls_enter();
+   if (!pvcalls_front_dev) {
+   pvcalls_exit();
+   return -ENOTCONN;
+   }
+   bedata = dev_get_drvdata(_front_dev->dev);
+
+   map = (struct sock_mapping *) sock->sk->sk_send_head;
+   if (!map) {
+   pvcalls_exit();
+   return -ENOTSOCK;
+   }
+
+   if (map->passive.status != PVCALLS_STATUS_LISTEN) {
+   pvcalls_exit();
+   return -EINVAL;
+   }
+
+   nonblock = flags & SOCK_NONBLOCK;
+   /*
+* Backend only supports 1 inflight accept request, will return
+* errors for the others
+*/
+   if (test_and_set_bit(PVCALLS_FLAG_ACCEPT_INFLIGHT,
+(void *)>passive.flags)) {
+   req_id = READ_ONCE(map->passive.inflight_req_id);
+   if (req_id != PVCALLS_INVALID_ID &&
+   READ_ONCE(bedata->rsp[req_id].req_id) == req_id) {
+   map2 = map->passive.accept_map;
+   goto received;
+   }
+   if (nonblock) {
+   pvcalls_exit();
+   return -EAGAIN;
+   }
+   if (wait_event_interruptible(map->passive.inflight_accept_req,
+   !test_and_set_bit(PVCALLS_FLAG_ACCEPT_INFLIGHT,
+ (void *)>passive.flags))) {
+   pvcalls_exit();
+   return -EINTR;
+   }
+   }
+
+   spin_lock(>socket_lock);
+   ret = get_request(bedata, _id);
+   if (ret < 0) {
+   clear_bit(PVCALLS_FLAG_ACCEPT_INFLIGHT,
+ (void *)>passive.flags);
+   spin_unlock(>socket_lock);
+   pvcalls_exit();
+   return ret;
+   }
+   map2 = kzalloc(sizeof(*map2), GFP_KERNEL);

[Xen-devel] [PATCH v8 00/13] introduce the Xen PV Calls frontend

[Stefano Stabellini]

Hi all,

this series introduces the frontend for the newly introduced PV Calls
procotol.

PV Calls is a paravirtualized protocol that allows the implementation of
a set of POSIX functions in a different domain. The PV Calls frontend
sends POSIX function calls to the backend, which implements them and
returns a value to the frontend and acts on the function call.

For more information about PV Calls, please read:

https://xenbits.xen.org/docs/unstable/misc/pvcalls.html

This patch series only implements the frontend driver. It doesn't
attempt to redirect POSIX calls to it. The functions exported in
pvcalls-front.h are meant to be used for that. A separate patch series
will be sent to use them and hook them into the system.


Changes in v8:
- cast to uintptr_t instead of uint64_t
- cast to uintptr_t before casting to struct sock_mapping*
- check return values of copy_from/to_iter


Stefano Stabellini (13):
  xen/pvcalls: introduce the pvcalls xenbus frontend
  xen/pvcalls: implement frontend disconnect
  xen/pvcalls: connect to the backend
  xen/pvcalls: implement socket command and handle events
  xen/pvcalls: implement connect command
  xen/pvcalls: implement bind command
  xen/pvcalls: implement listen command
  xen/pvcalls: implement accept command
  xen/pvcalls: implement sendmsg
  xen/pvcalls: implement recvmsg
  xen/pvcalls: implement poll command
  xen/pvcalls: implement release command
  xen: introduce a Kconfig option to enable the pvcalls frontend

 drivers/xen/Kconfig |   11 +
 drivers/xen/Makefile|1 +
 drivers/xen/pvcalls-front.c | 1277 +++
 drivers/xen/pvcalls-front.h |   28 +
 4 files changed, 1317 insertions(+)
 create mode 100644 drivers/xen/pvcalls-front.c
 create mode 100644 drivers/xen/pvcalls-front.h

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v8 05/13] xen/pvcalls: implement connect command

[Stefano Stabellini]

Send PVCALLS_CONNECT to the backend. Allocate a new ring and evtchn for
the active socket.

Introduce fields in struct sock_mapping to keep track of active sockets.
Introduce a waitqueue to allow the frontend to wait on data coming from
the backend on the active socket (recvmsg command).

Two mutexes (one of reads and one for writes) will be used to protect
the active socket in and out rings from concurrent accesses.

Signed-off-by: Stefano Stabellini 
Reviewed-by: Boris Ostrovsky 
CC: boris.ostrov...@oracle.com
CC: jgr...@suse.com
---
 drivers/xen/pvcalls-front.c | 158 
 drivers/xen/pvcalls-front.h |   2 +
 2 files changed, 160 insertions(+)

diff --git a/drivers/xen/pvcalls-front.c b/drivers/xen/pvcalls-front.c
index 326395d..8d4a43e 100644
--- a/drivers/xen/pvcalls-front.c
+++ b/drivers/xen/pvcalls-front.c
@@ -59,6 +59,18 @@ struct sock_mapping {
bool active_socket;
struct list_head list;
struct socket *sock;
+   union {
+   struct {
+   int irq;
+   grant_ref_t ref;
+   struct pvcalls_data_intf *ring;
+   struct pvcalls_data data;
+   struct mutex in_mutex;
+   struct mutex out_mutex;
+
+   wait_queue_head_t inflight_conn_req;
+   } active;
+   };
 };
 
 static inline int get_request(struct pvcalls_bedata *bedata, int *req_id)
@@ -121,6 +133,18 @@ static void pvcalls_front_free_map(struct pvcalls_bedata 
*bedata,
 {
 }
 
+static irqreturn_t pvcalls_front_conn_handler(int irq, void *sock_map)
+{
+   struct sock_mapping *map = sock_map;
+
+   if (map == NULL)
+   return IRQ_HANDLED;
+
+   wake_up_interruptible(>active.inflight_conn_req);
+
+   return IRQ_HANDLED;
+}
+
 int pvcalls_front_socket(struct socket *sock)
 {
struct pvcalls_bedata *bedata;
@@ -196,6 +220,132 @@ int pvcalls_front_socket(struct socket *sock)
return ret;
 }
 
+static int create_active(struct sock_mapping *map, int *evtchn)
+{
+   void *bytes;
+   int ret = -ENOMEM, irq = -1, i;
+
+   *evtchn = -1;
+   init_waitqueue_head(>active.inflight_conn_req);
+
+   map->active.ring = (struct pvcalls_data_intf *)
+   __get_free_page(GFP_KERNEL | __GFP_ZERO);
+   if (map->active.ring == NULL)
+   goto out_error;
+   map->active.ring->ring_order = PVCALLS_RING_ORDER;
+   bytes = (void *)__get_free_pages(GFP_KERNEL | __GFP_ZERO,
+   PVCALLS_RING_ORDER);
+   if (bytes == NULL)
+   goto out_error;
+   for (i = 0; i < (1 << PVCALLS_RING_ORDER); i++)
+   map->active.ring->ref[i] = gnttab_grant_foreign_access(
+   pvcalls_front_dev->otherend_id,
+   pfn_to_gfn(virt_to_pfn(bytes) + i), 0);
+
+   map->active.ref = gnttab_grant_foreign_access(
+   pvcalls_front_dev->otherend_id,
+   pfn_to_gfn(virt_to_pfn((void *)map->active.ring)), 0);
+
+   map->active.data.in = bytes;
+   map->active.data.out = bytes +
+   XEN_FLEX_RING_SIZE(PVCALLS_RING_ORDER);
+
+   ret = xenbus_alloc_evtchn(pvcalls_front_dev, evtchn);
+   if (ret)
+   goto out_error;
+   irq = bind_evtchn_to_irqhandler(*evtchn, pvcalls_front_conn_handler,
+   0, "pvcalls-frontend", map);
+   if (irq < 0) {
+   ret = irq;
+   goto out_error;
+   }
+
+   map->active.irq = irq;
+   map->active_socket = true;
+   mutex_init(>active.in_mutex);
+   mutex_init(>active.out_mutex);
+
+   return 0;
+
+out_error:
+   if (irq >= 0)
+   unbind_from_irqhandler(irq, map);
+   else if (*evtchn >= 0)
+   xenbus_free_evtchn(pvcalls_front_dev, *evtchn);
+   kfree(map->active.data.in);
+   kfree(map->active.ring);
+   return ret;
+}
+
+int pvcalls_front_connect(struct socket *sock, struct sockaddr *addr,
+   int addr_len, int flags)
+{
+   struct pvcalls_bedata *bedata;
+   struct sock_mapping *map = NULL;
+   struct xen_pvcalls_request *req;
+   int notify, req_id, ret, evtchn;
+
+   if (addr->sa_family != AF_INET || sock->type != SOCK_STREAM)
+   return -EOPNOTSUPP;
+
+   pvcalls_enter();
+   if (!pvcalls_front_dev) {
+   pvcalls_exit();
+   return -ENOTCONN;
+   }
+
+   bedata = dev_get_drvdata(_front_dev->dev);
+
+   map = (struct sock_mapping *)sock->sk->sk_send_head;
+   if (!map) {
+   pvcalls_exit();
+   return -ENOTSOCK;
+   }
+
+   spin_lock(>socket_lock);
+   ret = get_request(bedata, _id);
+   if (ret < 0) {
+   spin_unlock(>socket_lock);
+   

[Xen-devel] [PATCH v8 02/13] xen/pvcalls: implement frontend disconnect

[Stefano Stabellini]

Introduce a data structure named pvcalls_bedata. It contains pointers to
the command ring, the event channel, a list of active sockets and a list
of passive sockets. Lists accesses are protected by a spin_lock.

Introduce a waitqueue to allow waiting for a response on commands sent
to the backend.

Introduce an array of struct xen_pvcalls_response to store commands
responses.

Introduce a new struct sock_mapping to keep track of sockets.  In this
patch the struct sock_mapping is minimal, the fields will be added by
the next patches.

pvcalls_refcount is used to keep count of the outstanding pvcalls users.
Only remove connections once the refcount is zero.

Implement pvcalls frontend removal function. Go through the list of
active and passive sockets and free them all, one at a time.

Signed-off-by: Stefano Stabellini 
Reviewed-by: Boris Ostrovsky 
CC: boris.ostrov...@oracle.com
CC: jgr...@suse.com
---
 drivers/xen/pvcalls-front.c | 71 +
 1 file changed, 71 insertions(+)

diff --git a/drivers/xen/pvcalls-front.c b/drivers/xen/pvcalls-front.c
index a8d38c2..aae23d0 100644
--- a/drivers/xen/pvcalls-front.c
+++ b/drivers/xen/pvcalls-front.c
@@ -20,6 +20,51 @@
 #include 
 #include 
 
+#define PVCALLS_INVALID_ID UINT_MAX
+#define PVCALLS_RING_ORDER XENBUS_MAX_RING_GRANT_ORDER
+#define PVCALLS_NR_RSP_PER_RING __CONST_RING_SIZE(xen_pvcalls, XEN_PAGE_SIZE)
+
+struct pvcalls_bedata {
+   struct xen_pvcalls_front_ring ring;
+   grant_ref_t ref;
+   int irq;
+
+   struct list_head socket_mappings;
+   spinlock_t socket_lock;
+
+   wait_queue_head_t inflight_req;
+   struct xen_pvcalls_response rsp[PVCALLS_NR_RSP_PER_RING];
+};
+/* Only one front/back connection supported. */
+static struct xenbus_device *pvcalls_front_dev;
+static atomic_t pvcalls_refcount;
+
+/* first increment refcount, then proceed */
+#define pvcalls_enter() {   \
+   atomic_inc(_refcount);  \
+}
+
+/* first complete other operations, then decrement refcount */
+#define pvcalls_exit() {\
+   atomic_dec(_refcount);  \
+}
+
+struct sock_mapping {
+   bool active_socket;
+   struct list_head list;
+   struct socket *sock;
+};
+
+static irqreturn_t pvcalls_front_event_handler(int irq, void *dev_id)
+{
+   return IRQ_HANDLED;
+}
+
+static void pvcalls_front_free_map(struct pvcalls_bedata *bedata,
+  struct sock_mapping *map)
+{
+}
+
 static const struct xenbus_device_id pvcalls_front_ids[] = {
{ "pvcalls" },
{ "" }
@@ -27,6 +72,32 @@
 
 static int pvcalls_front_remove(struct xenbus_device *dev)
 {
+   struct pvcalls_bedata *bedata;
+   struct sock_mapping *map = NULL, *n;
+
+   bedata = dev_get_drvdata(_front_dev->dev);
+   dev_set_drvdata(>dev, NULL);
+   pvcalls_front_dev = NULL;
+   if (bedata->irq >= 0)
+   unbind_from_irqhandler(bedata->irq, dev);
+
+   smp_mb();
+   while (atomic_read(_refcount) > 0)
+   cpu_relax();
+   list_for_each_entry_safe(map, n, >socket_mappings, list) {
+   if (map->active_socket) {
+   /* No need to lock, refcount is 0 */
+   pvcalls_front_free_map(bedata, map);
+   } else {
+   list_del(>list);
+   kfree(map);
+   }
+   }
+   if (bedata->ref >= 0)
+   gnttab_end_foreign_access(bedata->ref, 0, 0);
+   kfree(bedata->ring.sring);
+   kfree(bedata);
+   xenbus_switch_state(dev, XenbusStateClosed);
return 0;
 }
 
-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v8 06/13] xen/pvcalls: implement bind command

[Stefano Stabellini]

Send PVCALLS_BIND to the backend. Introduce a new structure, part of
struct sock_mapping, to store information specific to passive sockets.

Introduce a status field to keep track of the status of the passive
socket.

Signed-off-by: Stefano Stabellini 
Reviewed-by: Boris Ostrovsky 
CC: boris.ostrov...@oracle.com
CC: jgr...@suse.com
---
 drivers/xen/pvcalls-front.c | 66 +
 drivers/xen/pvcalls-front.h |  3 +++
 2 files changed, 69 insertions(+)

diff --git a/drivers/xen/pvcalls-front.c b/drivers/xen/pvcalls-front.c
index 8d4a43e..6e70ec3 100644
--- a/drivers/xen/pvcalls-front.c
+++ b/drivers/xen/pvcalls-front.c
@@ -70,6 +70,13 @@ struct sock_mapping {
 
wait_queue_head_t inflight_conn_req;
} active;
+   struct {
+   /* Socket status */
+#define PVCALLS_STATUS_UNINITALIZED  0
+#define PVCALLS_STATUS_BIND  1
+#define PVCALLS_STATUS_LISTEN2
+   uint8_t status;
+   } passive;
};
 };
 
@@ -346,6 +353,65 @@ int pvcalls_front_connect(struct socket *sock, struct 
sockaddr *addr,
return ret;
 }
 
+int pvcalls_front_bind(struct socket *sock, struct sockaddr *addr, int 
addr_len)
+{
+   struct pvcalls_bedata *bedata;
+   struct sock_mapping *map = NULL;
+   struct xen_pvcalls_request *req;
+   int notify, req_id, ret;
+
+   if (addr->sa_family != AF_INET || sock->type != SOCK_STREAM)
+   return -EOPNOTSUPP;
+
+   pvcalls_enter();
+   if (!pvcalls_front_dev) {
+   pvcalls_exit();
+   return -ENOTCONN;
+   }
+   bedata = dev_get_drvdata(_front_dev->dev);
+
+   map = (struct sock_mapping *) sock->sk->sk_send_head;
+   if (map == NULL) {
+   pvcalls_exit();
+   return -ENOTSOCK;
+   }
+
+   spin_lock(>socket_lock);
+   ret = get_request(bedata, _id);
+   if (ret < 0) {
+   spin_unlock(>socket_lock);
+   pvcalls_exit();
+   return ret;
+   }
+   req = RING_GET_REQUEST(>ring, req_id);
+   req->req_id = req_id;
+   map->sock = sock;
+   req->cmd = PVCALLS_BIND;
+   req->u.bind.id = (uintptr_t)map;
+   memcpy(req->u.bind.addr, addr, sizeof(*addr));
+   req->u.bind.len = addr_len;
+
+   map->active_socket = false;
+
+   bedata->ring.req_prod_pvt++;
+   RING_PUSH_REQUESTS_AND_CHECK_NOTIFY(>ring, notify);
+   spin_unlock(>socket_lock);
+   if (notify)
+   notify_remote_via_irq(bedata->irq);
+
+   wait_event(bedata->inflight_req,
+  READ_ONCE(bedata->rsp[req_id].req_id) == req_id);
+
+   /* read req_id, then the content */
+   smp_rmb();
+   ret = bedata->rsp[req_id].ret;
+   bedata->rsp[req_id].req_id = PVCALLS_INVALID_ID;
+
+   map->passive.status = PVCALLS_STATUS_BIND;
+   pvcalls_exit();
+   return 0;
+}
+
 static const struct xenbus_device_id pvcalls_front_ids[] = {
{ "pvcalls" },
{ "" }
diff --git a/drivers/xen/pvcalls-front.h b/drivers/xen/pvcalls-front.h
index 63b0417..8b0a274 100644
--- a/drivers/xen/pvcalls-front.h
+++ b/drivers/xen/pvcalls-front.h
@@ -6,5 +6,8 @@
 int pvcalls_front_socket(struct socket *sock);
 int pvcalls_front_connect(struct socket *sock, struct sockaddr *addr,
  int addr_len, int flags);
+int pvcalls_front_bind(struct socket *sock,
+  struct sockaddr *addr,
+  int addr_len);
 
 #endif
-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v8 03/13] xen/pvcalls: connect to the backend

[Stefano Stabellini]

Implement the probe function for the pvcalls frontend. Read the
supported versions, max-page-order and function-calls nodes from
xenstore.

Only one frontend<->backend connection is supported at any given time
for a guest. Store the active frontend device to a static pointer.

Introduce a stub functions for the event handler.

Signed-off-by: Stefano Stabellini 
Reviewed-by: Boris Ostrovsky 
CC: boris.ostrov...@oracle.com
CC: jgr...@suse.com
---
 drivers/xen/pvcalls-front.c | 132 
 1 file changed, 132 insertions(+)

diff --git a/drivers/xen/pvcalls-front.c b/drivers/xen/pvcalls-front.c
index aae23d0..1618502 100644
--- a/drivers/xen/pvcalls-front.c
+++ b/drivers/xen/pvcalls-front.c
@@ -104,12 +104,144 @@ static int pvcalls_front_remove(struct xenbus_device 
*dev)
 static int pvcalls_front_probe(struct xenbus_device *dev,
  const struct xenbus_device_id *id)
 {
+   int ret = -ENOMEM, evtchn, i;
+   unsigned int max_page_order, function_calls, len;
+   char *versions;
+   grant_ref_t gref_head = 0;
+   struct xenbus_transaction xbt;
+   struct pvcalls_bedata *bedata = NULL;
+   struct xen_pvcalls_sring *sring;
+
+   if (pvcalls_front_dev != NULL) {
+   dev_err(>dev, "only one PV Calls connection supported\n");
+   return -EINVAL;
+   }
+
+   versions = xenbus_read(XBT_NIL, dev->otherend, "versions", );
+   if (!len)
+   return -EINVAL;
+   if (strcmp(versions, "1")) {
+   kfree(versions);
+   return -EINVAL;
+   }
+   kfree(versions);
+   max_page_order = xenbus_read_unsigned(dev->otherend,
+ "max-page-order", 0);
+   if (max_page_order < PVCALLS_RING_ORDER)
+   return -ENODEV;
+   function_calls = xenbus_read_unsigned(dev->otherend,
+ "function-calls", 0);
+   /* See XENBUS_FUNCTIONS_CALLS in pvcalls.h */
+   if (function_calls != 1)
+   return -ENODEV;
+   pr_info("%s max-page-order is %u\n", __func__, max_page_order);
+
+   bedata = kzalloc(sizeof(struct pvcalls_bedata), GFP_KERNEL);
+   if (!bedata)
+   return -ENOMEM;
+
+   dev_set_drvdata(>dev, bedata);
+   pvcalls_front_dev = dev;
+   init_waitqueue_head(>inflight_req);
+   INIT_LIST_HEAD(>socket_mappings);
+   spin_lock_init(>socket_lock);
+   bedata->irq = -1;
+   bedata->ref = -1;
+
+   for (i = 0; i < PVCALLS_NR_RSP_PER_RING; i++)
+   bedata->rsp[i].req_id = PVCALLS_INVALID_ID;
+
+   sring = (struct xen_pvcalls_sring *) __get_free_page(GFP_KERNEL |
+__GFP_ZERO);
+   if (!sring)
+   goto error;
+   SHARED_RING_INIT(sring);
+   FRONT_RING_INIT(>ring, sring, XEN_PAGE_SIZE);
+
+   ret = xenbus_alloc_evtchn(dev, );
+   if (ret)
+   goto error;
+
+   bedata->irq = bind_evtchn_to_irqhandler(evtchn,
+   pvcalls_front_event_handler,
+   0, "pvcalls-frontend", dev);
+   if (bedata->irq < 0) {
+   ret = bedata->irq;
+   goto error;
+   }
+
+   ret = gnttab_alloc_grant_references(1, _head);
+   if (ret < 0)
+   goto error;
+   bedata->ref = gnttab_claim_grant_reference(_head);
+   if (bedata->ref < 0) {
+   ret = bedata->ref;
+   goto error;
+   }
+   gnttab_grant_foreign_access_ref(bedata->ref, dev->otherend_id,
+   virt_to_gfn((void *)sring), 0);
+
+ again:
+   ret = xenbus_transaction_start();
+   if (ret) {
+   xenbus_dev_fatal(dev, ret, "starting transaction");
+   goto error;
+   }
+   ret = xenbus_printf(xbt, dev->nodename, "version", "%u", 1);
+   if (ret)
+   goto error_xenbus;
+   ret = xenbus_printf(xbt, dev->nodename, "ring-ref", "%d", bedata->ref);
+   if (ret)
+   goto error_xenbus;
+   ret = xenbus_printf(xbt, dev->nodename, "port", "%u",
+   evtchn);
+   if (ret)
+   goto error_xenbus;
+   ret = xenbus_transaction_end(xbt, 0);
+   if (ret) {
+   if (ret == -EAGAIN)
+   goto again;
+   xenbus_dev_fatal(dev, ret, "completing transaction");
+   goto error;
+   }
+   xenbus_switch_state(dev, XenbusStateInitialised);
+
return 0;
+
+ error_xenbus:
+   xenbus_transaction_end(xbt, 1);
+   xenbus_dev_fatal(dev, ret, "writing xenstore");
+ error:
+   pvcalls_front_remove(dev);
+   return ret;
 }
 
 static void pvcalls_front_changed(struct xenbus_device *dev,
enum 

[Xen-devel] [PATCH v8 04/13] xen/pvcalls: implement socket command and handle events

[Stefano Stabellini]

Send a PVCALLS_SOCKET command to the backend, use the masked
req_prod_pvt as req_id. This way, req_id is guaranteed to be between 0
and PVCALLS_NR_REQ_PER_RING. We already have a slot in the rsp array
ready for the response, and there cannot be two outstanding responses
with the same req_id.

Wait for the response by waiting on the inflight_req waitqueue and
check for the req_id field in rsp[req_id]. Use atomic accesses and
barriers to read the field. Note that the barriers are simple smp
barriers (as opposed to virt barriers) because they are for internal
frontend synchronization, not frontend<->backend communication.

Once a response is received, clear the corresponding rsp slot by setting
req_id to PVCALLS_INVALID_ID. Note that PVCALLS_INVALID_ID is invalid
only from the frontend point of view. It is not part of the PVCalls
protocol.

pvcalls_front_event_handler is in charge of copying responses from the
ring to the appropriate rsp slot. It is done by copying the body of the
response first, then by copying req_id atomically. After the copies,
wake up anybody waiting on waitqueue.

socket_lock protects accesses to the ring.

Convert the pointer to sock_mapping into an uintptr_t and use it as
id for the new socket to pass to the backend. The struct will be fully
initialized later on connect or bind.

sock->sk->sk_send_head is not used for ip sockets: reuse the field to
store a pointer to the struct sock_mapping corresponding to the socket.
This way, we can easily get the struct sock_mapping from the struct
socket.

Signed-off-by: Stefano Stabellini 
Reviewed-by: Boris Ostrovsky 
CC: boris.ostrov...@oracle.com
CC: jgr...@suse.com
---
 drivers/xen/pvcalls-front.c | 131 
 drivers/xen/pvcalls-front.h |   8 +++
 2 files changed, 139 insertions(+)
 create mode 100644 drivers/xen/pvcalls-front.h

diff --git a/drivers/xen/pvcalls-front.c b/drivers/xen/pvcalls-front.c
index 1618502..326395d 100644
--- a/drivers/xen/pvcalls-front.c
+++ b/drivers/xen/pvcalls-front.c
@@ -13,6 +13,10 @@
  */
 
 #include 
+#include 
+#include 
+
+#include 
 
 #include 
 #include 
@@ -20,6 +24,8 @@
 #include 
 #include 
 
+#include "pvcalls-front.h"
+
 #define PVCALLS_INVALID_ID UINT_MAX
 #define PVCALLS_RING_ORDER XENBUS_MAX_RING_GRANT_ORDER
 #define PVCALLS_NR_RSP_PER_RING __CONST_RING_SIZE(xen_pvcalls, XEN_PAGE_SIZE)
@@ -55,8 +61,58 @@ struct sock_mapping {
struct socket *sock;
 };
 
+static inline int get_request(struct pvcalls_bedata *bedata, int *req_id)
+{
+   *req_id = bedata->ring.req_prod_pvt & (RING_SIZE(>ring) - 1);
+   if (RING_FULL(>ring) ||
+   bedata->rsp[*req_id].req_id != PVCALLS_INVALID_ID)
+   return -EAGAIN;
+   return 0;
+}
+
 static irqreturn_t pvcalls_front_event_handler(int irq, void *dev_id)
 {
+   struct xenbus_device *dev = dev_id;
+   struct pvcalls_bedata *bedata;
+   struct xen_pvcalls_response *rsp;
+   uint8_t *src, *dst;
+   int req_id = 0, more = 0, done = 0;
+
+   if (dev == NULL)
+   return IRQ_HANDLED;
+
+   pvcalls_enter();
+   bedata = dev_get_drvdata(>dev);
+   if (bedata == NULL) {
+   pvcalls_exit();
+   return IRQ_HANDLED;
+   }
+
+again:
+   while (RING_HAS_UNCONSUMED_RESPONSES(>ring)) {
+   rsp = RING_GET_RESPONSE(>ring, bedata->ring.rsp_cons);
+
+   req_id = rsp->req_id;
+   dst = (uint8_t *)>rsp[req_id] + sizeof(rsp->req_id);
+   src = (uint8_t *)rsp + sizeof(rsp->req_id);
+   memcpy(dst, src, sizeof(*rsp) - sizeof(rsp->req_id));
+   /*
+* First copy the rest of the data, then req_id. It is
+* paired with the barrier when accessing bedata->rsp.
+*/
+   smp_wmb();
+   bedata->rsp[req_id].req_id = rsp->req_id;
+
+   done = 1;
+   bedata->ring.rsp_cons++;
+   }
+
+   RING_FINAL_CHECK_FOR_RESPONSES(>ring, more);
+   if (more)
+   goto again;
+   if (done)
+   wake_up(>inflight_req);
+   pvcalls_exit();
return IRQ_HANDLED;
 }
 
@@ -65,6 +121,81 @@ static void pvcalls_front_free_map(struct pvcalls_bedata 
*bedata,
 {
 }
 
+int pvcalls_front_socket(struct socket *sock)
+{
+   struct pvcalls_bedata *bedata;
+   struct sock_mapping *map = NULL;
+   struct xen_pvcalls_request *req;
+   int notify, req_id, ret;
+
+   /*
+* PVCalls only supports domain AF_INET,
+* type SOCK_STREAM and protocol 0 sockets for now.
+*
+* Check socket type here, AF_INET and protocol checks are done
+* by the caller.
+*/
+   if (sock->type != SOCK_STREAM)
+   return -EOPNOTSUPP;
+
+   pvcalls_enter();
+   if (!pvcalls_front_dev) {
+   pvcalls_exit();
+   return -EACCES;
+   }
+   

Re: [Xen-devel] [PATCH v7 13/13] xen: introduce a Kconfig option to enable the pvcalls frontend

[Stefano Stabellini]

On Mon, 30 Oct 2017, Boris Ostrovsky wrote:
> On 10/30/2017 05:42 PM, Stefano Stabellini wrote:
> > On Mon, 30 Oct 2017, Boris Ostrovsky wrote:
> >> On 10/30/2017 03:48 PM, Stefano Stabellini wrote:
> >>> On Mon, 30 Oct 2017, Boris Ostrovsky wrote:
>  Build warnings (gcc (GCC) 6.3.1 20161221 (Red Hat 6.3.1-1))
> >>> Hi Boris, I am trying to repro the warnings below. I have been
> >>> unsuccessful so far. What system are you using? Fedora? CentOS? Do have
> >>> any specific CFLAGS settings?
> >>
> >> I am on Fedora24, nothing interesting in my environment. For example,
> >>
> >> ostr@workbase> env | grep FLAGS
> >> ostr@workbase>
> >>
> >> I'll send you my config file in a separate email, just in case.  Your
> >> patches are on top of
> >>
> >> 0b07194 Linux 4.14-rc7
> >>
> > Thanks! I managed to repro and fix both the x86_64 and x86_32 issues.
> > I sent two patches:
> >
> > https://marc.info/?l=xen-devel=150939969002183=2
> > https://marc.info/?l=linux-kernel=150939962102165
> >
> > But if you prefer that I spin a new pvcalls frontend series, I could
> > also do that.
> 
> 
> I haven't pushed this anywhere so I think, given that this actually
> fixes a bug (when copy_to/from_iter() fails), it would be better to have
> a clean series in the upstream. So I'd rather have a new spin, sorry ;-(

No problem! I'll send one out shortly.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v7 13/13] xen: introduce a Kconfig option to enable the pvcalls frontend

[Boris Ostrovsky]

On 10/30/2017 05:42 PM, Stefano Stabellini wrote:
> On Mon, 30 Oct 2017, Boris Ostrovsky wrote:
>> On 10/30/2017 03:48 PM, Stefano Stabellini wrote:
>>> On Mon, 30 Oct 2017, Boris Ostrovsky wrote:
 Build warnings (gcc (GCC) 6.3.1 20161221 (Red Hat 6.3.1-1))
>>> Hi Boris, I am trying to repro the warnings below. I have been
>>> unsuccessful so far. What system are you using? Fedora? CentOS? Do have
>>> any specific CFLAGS settings?
>>
>> I am on Fedora24, nothing interesting in my environment. For example,
>>
>> ostr@workbase> env | grep FLAGS
>> ostr@workbase>
>>
>> I'll send you my config file in a separate email, just in case.  Your
>> patches are on top of
>>
>> 0b07194 Linux 4.14-rc7
>>
> Thanks! I managed to repro and fix both the x86_64 and x86_32 issues.
> I sent two patches:
>
> https://marc.info/?l=xen-devel=150939969002183=2
> https://marc.info/?l=linux-kernel=150939962102165
>
> But if you prefer that I spin a new pvcalls frontend series, I could
> also do that.


I haven't pushed this anywhere so I think, given that this actually
fixes a bug (when copy_to/from_iter() fails), it would be better to have
a clean series in the upstream. So I'd rather have a new spin, sorry ;-(


Thanks.
-borsi

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v7 13/13] xen: introduce a Kconfig option to enable the pvcalls frontend

[Stefano Stabellini]

On Mon, 30 Oct 2017, Boris Ostrovsky wrote:
> On 10/30/2017 03:48 PM, Stefano Stabellini wrote:
> > On Mon, 30 Oct 2017, Boris Ostrovsky wrote:
> >>
> >> Build warnings (gcc (GCC) 6.3.1 20161221 (Red Hat 6.3.1-1))
> > Hi Boris, I am trying to repro the warnings below. I have been
> > unsuccessful so far. What system are you using? Fedora? CentOS? Do have
> > any specific CFLAGS settings?
> 
> 
> I am on Fedora24, nothing interesting in my environment. For example,
> 
> ostr@workbase> env | grep FLAGS
> ostr@workbase>
> 
> I'll send you my config file in a separate email, just in case.  Your
> patches are on top of
> 
> 0b07194 Linux 4.14-rc7
> 

Thanks! I managed to repro and fix both the x86_64 and x86_32 issues.
I sent two patches:

https://marc.info/?l=xen-devel=150939969002183=2
https://marc.info/?l=linux-kernel=150939962102165

But if you prefer that I spin a new pvcalls frontend series, I could
also do that.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH 1/2] pvcalls: check return value from copy_from/to_iter

[Stefano Stabellini]

Check the return value of copy_from_iter in __write_ring and
copy_to_iter in __read_ring. Update "len" accordingly.

In the cases where we issue two consecutive copy_from_iter, or two
consecutive copy_to_iter, first check return, then goto out if it is not
what we expect.

Signed-off-by: Stefano Stabellini 
---
 drivers/xen/pvcalls-front.c | 30 ++
 1 file changed, 18 insertions(+), 12 deletions(-)

diff --git a/drivers/xen/pvcalls-front.c b/drivers/xen/pvcalls-front.c
index 8e7426083..f2dcac88 100644
--- a/drivers/xen/pvcalls-front.c
+++ b/drivers/xen/pvcalls-front.c
@@ -456,18 +456,21 @@ static int __write_ring(struct pvcalls_data_intf *intf,
masked_cons = pvcalls_mask(cons, array_size);
 
if (masked_prod < masked_cons) {
-   copy_from_iter(data->out + masked_prod, len, msg_iter);
+   len = copy_from_iter(data->out + masked_prod, len, msg_iter);
} else {
if (len > array_size - masked_prod) {
-   copy_from_iter(data->out + masked_prod,
+   int ret = copy_from_iter(data->out + masked_prod,
   array_size - masked_prod, msg_iter);
-   copy_from_iter(data->out,
-  len - (array_size - masked_prod),
-  msg_iter);
+   if (ret != array_size - masked_prod) {
+   len = ret;
+   goto out;
+   }
+   len = ret + copy_from_iter(data->out, len - ret, 
msg_iter);
} else {
-   copy_from_iter(data->out + masked_prod, len, msg_iter);
+   len = copy_from_iter(data->out + masked_prod, len, 
msg_iter);
}
}
+out:
/* write to ring before updating pointer */
virt_wmb();
intf->out_prod += len;
@@ -557,18 +560,21 @@ static int __read_ring(struct pvcalls_data_intf *intf,
len = size;
 
if (masked_prod > masked_cons) {
-   copy_to_iter(data->in + masked_cons, len, msg_iter);
+   len = copy_to_iter(data->in + masked_cons, len, msg_iter);
} else {
if (len > (array_size - masked_cons)) {
-   copy_to_iter(data->in + masked_cons,
+   int ret = copy_to_iter(data->in + masked_cons,
 array_size - masked_cons, msg_iter);
-   copy_to_iter(data->in,
-len - (array_size - masked_cons),
-msg_iter);
+   if (ret != array_size - masked_cons) {
+   len = ret;
+   goto out;
+   }
+   len = ret + copy_to_iter(data->in, len - ret, msg_iter);
} else {
-   copy_to_iter(data->in + masked_cons, len, msg_iter);
+   len = copy_to_iter(data->in + masked_cons, len, 
msg_iter);
}
}
+out:
/* read data from the ring before increasing the index */
virt_mb();
if (!(flags & MSG_PEEK))
-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH 2/2] pvcalls: fix casts to avoid warnings on 32 bit

[Stefano Stabellini]

Cast the map pointers to uintptr_t instead of uint64_t everywhere when
setting the socket ids.

In pvcalls_front_event_handler, first cast the poll id to uintptr_t,
then to struct sock_mapping * to avoid warnings. We know that the poll
id is fine because it is was set by the frontend initially in the poll
request.

Signed-off-by: Stefano Stabellini 
---
 drivers/xen/pvcalls-front.c | 18 +-
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/drivers/xen/pvcalls-front.c b/drivers/xen/pvcalls-front.c
index f2dcac88..0c1ec68 100644
--- a/drivers/xen/pvcalls-front.c
+++ b/drivers/xen/pvcalls-front.c
@@ -157,7 +157,7 @@ static irqreturn_t pvcalls_front_event_handler(int irq, 
void *dev_id)
 
req_id = rsp->req_id;
if (rsp->cmd == PVCALLS_POLL) {
-   struct sock_mapping *map = (struct sock_mapping *)
+   struct sock_mapping *map = (struct sock_mapping 
*)(uintptr_t)
   rsp->u.poll.id;
 
clear_bit(PVCALLS_FLAG_POLL_INFLIGHT,
@@ -280,7 +280,7 @@ int pvcalls_front_socket(struct socket *sock)
req = RING_GET_REQUEST(>ring, req_id);
req->req_id = req_id;
req->cmd = PVCALLS_SOCKET;
-   req->u.socket.id = (uint64_t) map;
+   req->u.socket.id = (uintptr_t) map;
req->u.socket.domain = AF_INET;
req->u.socket.type = SOCK_STREAM;
req->u.socket.protocol = IPPROTO_IP;
@@ -402,7 +402,7 @@ int pvcalls_front_connect(struct socket *sock, struct 
sockaddr *addr,
req = RING_GET_REQUEST(>ring, req_id);
req->req_id = req_id;
req->cmd = PVCALLS_CONNECT;
-   req->u.connect.id = (uint64_t)map;
+   req->u.connect.id = (uintptr_t)map;
req->u.connect.len = addr_len;
req->u.connect.flags = flags;
req->u.connect.ref = map->active.ref;
@@ -663,7 +663,7 @@ int pvcalls_front_bind(struct socket *sock, struct sockaddr 
*addr, int addr_len)
req->req_id = req_id;
map->sock = sock;
req->cmd = PVCALLS_BIND;
-   req->u.bind.id = (uint64_t)map;
+   req->u.bind.id = (uintptr_t)map;
memcpy(req->u.bind.addr, addr, sizeof(*addr));
req->u.bind.len = addr_len;
 
@@ -725,7 +725,7 @@ int pvcalls_front_listen(struct socket *sock, int backlog)
req = RING_GET_REQUEST(>ring, req_id);
req->req_id = req_id;
req->cmd = PVCALLS_LISTEN;
-   req->u.listen.id = (uint64_t) map;
+   req->u.listen.id = (uintptr_t) map;
req->u.listen.backlog = backlog;
 
bedata->ring.req_prod_pvt++;
@@ -829,9 +829,9 @@ int pvcalls_front_accept(struct socket *sock, struct socket 
*newsock, int flags)
req = RING_GET_REQUEST(>ring, req_id);
req->req_id = req_id;
req->cmd = PVCALLS_ACCEPT;
-   req->u.accept.id = (uint64_t) map;
+   req->u.accept.id = (uintptr_t) map;
req->u.accept.ref = map2->active.ref;
-   req->u.accept.id_new = (uint64_t) map2;
+   req->u.accept.id_new = (uintptr_t) map2;
req->u.accept.evtchn = evtchn;
map->passive.accept_map = map2;
 
@@ -925,7 +925,7 @@ static unsigned int pvcalls_front_poll_passive(struct file 
*file,
req = RING_GET_REQUEST(>ring, req_id);
req->req_id = req_id;
req->cmd = PVCALLS_POLL;
-   req->u.poll.id = (uint64_t) map;
+   req->u.poll.id = (uintptr_t) map;
 
bedata->ring.req_prod_pvt++;
RING_PUSH_REQUESTS_AND_CHECK_NOTIFY(>ring, notify);
@@ -1023,7 +1023,7 @@ int pvcalls_front_release(struct socket *sock)
req = RING_GET_REQUEST(>ring, req_id);
req->req_id = req_id;
req->cmd = PVCALLS_RELEASE;
-   req->u.release.id = (uint64_t)map;
+   req->u.release.id = (uintptr_t)map;
 
bedata->ring.req_prod_pvt++;
RING_PUSH_REQUESTS_AND_CHECK_NOTIFY(>ring, notify);
-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v7 13/13] xen: introduce a Kconfig option to enable the pvcalls frontend

[Boris Ostrovsky]

On 10/30/2017 03:48 PM, Stefano Stabellini wrote:
> On Mon, 30 Oct 2017, Boris Ostrovsky wrote:
>>
>> Build warnings (gcc (GCC) 6.3.1 20161221 (Red Hat 6.3.1-1))
> Hi Boris, I am trying to repro the warnings below. I have been
> unsuccessful so far. What system are you using? Fedora? CentOS? Do have
> any specific CFLAGS settings?


I am on Fedora24, nothing interesting in my environment. For example,

ostr@workbase> env | grep FLAGS
ostr@workbase>

I'll send you my config file in a separate email, just in case.  Your
patches are on top of

0b07194 Linux 4.14-rc7

ostr@workbase> gcc -Wp,-MD,drivers/xen/.pvcalls-front.o.d  -nostdinc
-isystem /usr/lib/gcc/x86_64-redhat-linux/6.3.1/include
-I/data/upstream/linux-xen/arch/x86/include
-I./arch/x86/include/generated  -I/data/upstream/linux-xen/include
-I./include -I/data/upstream/linux-xen/arch/x86/include/uapi
-I./arch/x86/include/generated/uapi
-I/data/upstream/linux-xen/include/uapi -I./include/generated/uapi
-include /data/upstream/linux-xen/include/linux/kconfig.h 
-I/data/upstream/linux-xen/drivers/xen -Idrivers/xen -D__KERNEL__ -Wall
-Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing
-fno-common -fshort-wchar -Werror-implicit-function-declaration
-Wno-format-security -std=gnu89 -fno-PIE -mno-sse -mno-mmx -mno-sse2
-mno-3dnow -mno-avx -m64 -falign-jumps=1 -falign-loops=1 -mno-80387
-mno-fp-ret-in-387 -mpreferred-stack-boundary=3 -mskip-rax-setup
-mtune=generic -mno-red-zone -mcmodel=kernel -funit-at-a-time
-DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1
-DCONFIG_AS_CFI_SECTIONS=1 -DCONFIG_AS_FXSAVEQ=1 -DCONFIG_AS_SSSE3=1
-DCONFIG_AS_CRC32=1 -DCONFIG_AS_AVX=1 -DCONFIG_AS_AVX2=1
-DCONFIG_AS_AVX512=1 -DCONFIG_AS_SHA1_NI=1 -DCONFIG_AS_SHA256_NI=1 -pipe
-Wno-sign-compare -fno-asynchronous-unwind-tables
-fno-delete-null-pointer-checks -Wno-frame-address -O2
--param=allow-store-data-races=0 -DCC_HAVE_ASM_GOTO
-Wframe-larger-than=2048 -fstack-protector-strong
-Wno-unused-but-set-variable -Wno-unused-const-variable
-fno-omit-frame-pointer -fno-optimize-sibling-calls
-fno-var-tracking-assignments -g -pg -mfentry -DCC_USING_FENTRY
-Wdeclaration-after-statement -Wno-pointer-sign -fno-strict-overflow
-fconserve-stack -Werror=implicit-int -Werror=strict-prototypes
-Werror=date-time -Werror=incompatible-pointer-types
-Werror=designated-init  -DMODULE  -DKBUILD_BASENAME='"pvcalls_front"' 
-DKBUILD_MODNAME='"pvcalls_front"' -c -o drivers/xen/pvcalls-front.o
/data/upstream/linux-xen/drivers/xen/pvcalls-front.c
/data/upstream/linux-xen/drivers/xen/pvcalls-front.c: In function
‘__write_ring’:
/data/upstream/linux-xen/drivers/xen/pvcalls-front.c:459:3: warning:
ignoring return value of ‘copy_from_iter’, declared with attribute
warn_unused_result [-Wunused-result]
   copy_from_iter(data->out + masked_prod, len, msg_iter);
   ^~
/data/upstream/linux-xen/drivers/xen/pvcalls-front.c:462:4: warning:
ignoring return value of ‘copy_from_iter’, declared with attribute
warn_unused_result [-Wunused-result]
copy_from_iter(data->out + masked_prod,
^~~
array_size - masked_prod, msg_iter);
~~~
/data/upstream/linux-xen/drivers/xen/pvcalls-front.c:464:4: warning:
ignoring return value of ‘copy_from_iter’, declared with attribute
warn_unused_result [-Wunused-result]
copy_from_iter(data->out,
^
len - (array_size - masked_prod),
~
msg_iter);
~
/data/upstream/linux-xen/drivers/xen/pvcalls-front.c:468:4: warning:
ignoring return value of ‘copy_from_iter’, declared with attribute
warn_unused_result [-Wunused-result]
copy_from_iter(data->out + masked_prod, len, msg_iter);
^~
/data/upstream/linux-xen/drivers/xen/pvcalls-front.c: In function
‘__read_ring’:
/data/upstream/linux-xen/drivers/xen/pvcalls-front.c:560:3: warning:
ignoring return value of ‘copy_to_iter’, declared with attribute
warn_unused_result [-Wunused-result]
   copy_to_iter(data->in + masked_cons, len, msg_iter);
   ^~~
/data/upstream/linux-xen/drivers/xen/pvcalls-front.c:563:4: warning:
ignoring return value of ‘copy_to_iter’, declared with attribute
warn_unused_result [-Wunused-result]
copy_to_iter(data->in + masked_cons,
^~~~
  array_size - masked_cons, msg_iter);
  ~~~
/data/upstream/linux-xen/drivers/xen/pvcalls-front.c:565:4: warning:
ignoring return value of ‘copy_to_iter’, declared with attribute
warn_unused_result [-Wunused-result]
copy_to_iter(data->in,
^~
  len - (array_size - masked_cons),
  ~
  msg_iter);
  ~

Re: [Xen-devel] [PATCH v7 13/13] xen: introduce a Kconfig option to enable the pvcalls frontend

[Stefano Stabellini]

On Mon, 30 Oct 2017, Boris Ostrovsky wrote:
> On 10/26/2017 04:45 PM, Boris Ostrovsky wrote:
> > On 10/26/2017 04:16 PM, Stefano Stabellini wrote:
> >> On Thu, 26 Oct 2017, Boris Ostrovsky wrote:
> >>> On 10/26/2017 03:11 PM, Stefano Stabellini wrote:
>  Also add pvcalls-front to the Makefile.
> 
>  Signed-off-by: Stefano Stabellini 
>  CC: boris.ostrov...@oracle.com
>  CC: jgr...@suse.com
> >>> Reviewed-by: Boris Ostrovsky 
> >> Thank you!!
> >>
> >> The series is fully acked now. I guess it could be added to xentip?
> >> Maybe for v4.15?
> >
> > Yes, that's the plan unless other reviews come in. I will probably
> > create the branch on Monday (assuming rc7 will be the last rc for 4.14).
> > It's later than usual but we haven't had anything for 4.15.
> >
> > -boris
> 
> Build warnings (gcc (GCC) 6.3.1 20161221 (Red Hat 6.3.1-1))

Hi Boris, I am trying to repro the warnings below. I have been
unsuccessful so far. What system are you using? Fedora? CentOS? Do have
any specific CFLAGS settings?


> /data/upstream/linux-xen/drivers/xen/pvcalls-front.c: In function
> ‘__write_ring’:
> /data/upstream/linux-xen/drivers/xen/pvcalls-front.c:459:3: warning:
> ignoring return value of ‘copy_from_iter’, declared with attribute
> warn_unused_result [-Wunused-result]
>copy_from_iter(data->out + masked_prod, len, msg_iter);
>^~
> /data/upstream/linux-xen/drivers/xen/pvcalls-front.c:462:4: warning:
> ignoring return value of ‘copy_from_iter’, declared with attribute
> warn_unused_result [-Wunused-result]
> copy_from_iter(data->out + masked_prod,
> ^~~
> array_size - masked_prod, msg_iter);
> ~~~
> /data/upstream/linux-xen/drivers/xen/pvcalls-front.c:464:4: warning:
> ignoring return value of ‘copy_from_iter’, declared with attribute
> warn_unused_result [-Wunused-result]
> copy_from_iter(data->out,
> ^
> len - (array_size - masked_prod),
> ~
> msg_iter);
> ~
> /data/upstream/linux-xen/drivers/xen/pvcalls-front.c:468:4: warning:
> ignoring return value of ‘copy_from_iter’, declared with attribute
> warn_unused_result [-Wunused-result]
> copy_from_iter(data->out + masked_prod, len, msg_iter);
> ^~
> /data/upstream/linux-xen/drivers/xen/pvcalls-front.c: In function
> ‘__read_ring’:
> /data/upstream/linux-xen/drivers/xen/pvcalls-front.c:560:3: warning:
> ignoring return value of ‘copy_to_iter’, declared with attribute
> warn_unused_result [-Wunused-result]
>copy_to_iter(data->in + masked_cons, len, msg_iter);
>^~~
> /data/upstream/linux-xen/drivers/xen/pvcalls-front.c:563:4: warning:
> ignoring return value of ‘copy_to_iter’, declared with attribute
> warn_unused_result [-Wunused-result]
> copy_to_iter(data->in + masked_cons,
> ^~~~
>   array_size - masked_cons, msg_iter);
>   ~~~
> /data/upstream/linux-xen/drivers/xen/pvcalls-front.c:565:4: warning:
> ignoring return value of ‘copy_to_iter’, declared with attribute
> warn_unused_result [-Wunused-result]
> copy_to_iter(data->in,
> ^~
>   len - (array_size - masked_cons),
>   ~
>   msg_iter);
>   ~
> /data/upstream/linux-xen/drivers/xen/pvcalls-front.c:569:4: warning:
> ignoring return value of ‘copy_to_iter’, declared with attribute
> warn_unused_result [-Wunused-result]
> copy_to_iter(data->in + masked_cons, len, msg_iter);
> ^~~
> 
> 
> 
> 
> Slightly different on 32 bit:
> 
> /data/upstream/linux-xen/drivers/xen/pvcalls-front.c: In function
> ‘pvcalls_front_event_handler’:
> /data/upstream/linux-xen/drivers/xen/pvcalls-front.c:160:31: warning:
> cast to pointer from integer of different size [-Wint-to-pointer-cast]
> struct sock_mapping *map = (struct sock_mapping *)
>^
> /data/upstream/linux-xen/drivers/xen/pvcalls-front.c: In function
> ‘pvcalls_front_socket’:
> /data/upstream/linux-xen/drivers/xen/pvcalls-front.c:283:21: warning:
> cast from pointer to integer of different size [-Wpointer-to-int-cast]
>   req->u.socket.id = (uint64_t) map;
>  ^
> /data/upstream/linux-xen/drivers/xen/pvcalls-front.c: In function
> ‘pvcalls_front_connect’:
> /data/upstream/linux-xen/drivers/xen/pvcalls-front.c:405:22: warning:
> cast from pointer to integer of different size [-Wpointer-to-int-cast]
>   req->u.connect.id = (uint64_t)map;
>   ^
> /data/upstream/linux-xen/drivers/xen/pvcalls-front.c: In function
> 

[Xen-devel] FOSDEM Virtualization and IaaS Devroom CfP (closes Dec 1) and Xen Project Stand

[Lars Kurth]

Hi all,

as we usually have a big presence at FOSDEM, here are a few relevant CfPs. To 
make it easier for people who submit talks, I collated some of the info.

== FOSDEM Virtualization and IaaS Devroom CfP ==
See https://lists.fosdem.org/pipermail/fosdem/2017-October/002632.html

Submission deadline: 01 December 2017
Acceptance notifications: 14 December 2017
Final schedule announcement: 21 December 2017
Devroom: 03 and 04 February 2018 (two days- different rooms)

== Other relevant Devroom CfP ==
* BSD (cfp closes at 2017-11-26): see 
https://lists.fosdem.org/pipermail/fosdem/2017-October/002614.html
* Embedded, Mobile and Automotive, TBD: TBD - see 
https://fosdem.org/2018/news/2017-10-04-accepted-developer-rooms/


== Xen Project Stand ==
As usual, we applied for a stand again. Acceptance notifications are not due 
until 20 November

Regards
Lars
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [examine test] 115400: regressions - FAIL

[osstest service owner]

flight 115400 examine real [real]
http://logs.test-lab.xenproject.org/osstest/logs/115400/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 examine-elbling0  2 hosts-allocate broken REGR. vs. 115392
 examine-godello0  4 memdisk-try-append   fail REGR. vs. 115392

Tests which did not succeed, but are not blocking:
 examine-godello1  4 memdisk-try-append   fail  like 115392

baseline version:
 flight   115392

jobs:
 examine-baroque0 pass
 examine-baroque1 pass
 examine-arndale-bluewaterpass
 examine-cubietruck-braquepass
 examine-chardonnay0  pass
 examine-chardonnay1  pass
 examine-elbling0 fail
 examine-elbling1 pass
 examine-fiano0   pass
 examine-fiano1   pass
 examine-cubietruck-gleizes   pass
 examine-godello0 pass
 examine-godello1 pass
 examine-huxelrebe0   pass
 examine-huxelrebe1   pass
 examine-italia0  pass
 examine-italia1  pass
 examine-arndale-lakeside pass
 examine-merlot0  pass
 examine-merlot1  pass
 examine-arndale-metrocentre  pass
 examine-cubietruck-metzinger pass
 examine-nobling0 pass
 examine-nobling1 pass
 examine-nocera0  pass
 examine-nocera1  pass
 examine-cubietruck-picasso   pass
 examine-pinot0   pass
 examine-pinot1   pass
 examine-rimava0  pass
 examine-rimava1  pass
 examine-arndale-westfieldpass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Push not applicable.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 3/6] libxl: add backend type to vkb

[Wei Liu]

On Thu, Oct 05, 2017 at 12:07:08PM +0300, Oleksandr Grytsov wrote:
> From: Oleksandr Grytsov 
> 
> New field backend_type is added to vkb device
> in order to have QEMU and user space backend
> simultaneously. Each vkb backend shall read
> appropriate XS entry and service only own
> frontends.
> 
> Signed-off-by: Oleksandr Grytsov 
> ---
>  tools/libxl/libxl_create.c  |  4 
>  tools/libxl/libxl_dm.c  |  2 ++
>  tools/libxl/libxl_types.idl |  7 +++
>  tools/libxl/libxl_vkb.c | 10 +-
>  tools/xl/xl_parse.c |  4 
>  5 files changed, 26 insertions(+), 1 deletion(-)
> 
> diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
> index f813114..7268f7f 100644
> --- a/tools/libxl/libxl_create.c
> +++ b/tools/libxl/libxl_create.c
> @@ -1349,6 +1349,7 @@ static void domcreate_launch_dm(libxl__egc *egc, 
> libxl__multidev *multidev,
>  }
>  
>  libxl_device_vkb_init();
> +vkb.backend_type = LIBXL_VKB_BACKEND_QEMU;

Hmm... See below.

>  libxl__device_add(gc, domid, __vkb_devtype, );
>  libxl_device_vkb_dispose();
>  
> @@ -1376,6 +1377,9 @@ static void domcreate_launch_dm(libxl__egc *egc, 
> libxl__multidev *multidev,
>  for (i = 0; i < d_config->num_vfbs; i++) {
>  libxl__device_add(gc, domid, __vfb_devtype,
>_config->vfbs[i]);
> +}
> +
> +for (i = 0; i < d_config->num_vkbs; i++) {
>  libxl__device_add(gc, domid, __vkb_devtype,
>_config->vkbs[i]);
>  }
> diff --git a/tools/libxl/libxl_dm.c b/tools/libxl/libxl_dm.c
> index 98f89a9..d8b0ee7 100644
> --- a/tools/libxl/libxl_dm.c
> +++ b/tools/libxl/libxl_dm.c
> @@ -1728,6 +1728,8 @@ static int 
> libxl__vfb_and_vkb_from_hvm_guest_config(libxl__gc *gc,
>  
>  vkb->backend_domid = 0;
>  vkb->devid = 0;
> +vkb->backend_type = LIBXL_VKB_BACKEND_QEMU;
> +
>  return 0;
>  }
>  
> diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
> index cd0c06f..65cd81a 100644
> --- a/tools/libxl/libxl_types.idl
> +++ b/tools/libxl/libxl_types.idl
> @@ -240,6 +240,12 @@ libxl_checkpointed_stream = 
> Enumeration("checkpointed_stream", [
>  (2, "COLO"),
>  ])
>  
> +libxl_vkb_backend = Enumeration("vkb_backend", [
> +(0, "UNKNOWN"),
> +(1, "QEMU"),
> +(2, "LINUX")
> +])

Originally this is only internal detail, but now you want to expose
this.  You need to set the default value for this; otherwise you could
break migration.

And then you also need to provide a setdefault function for
libxl_device_vkb.

Also I'm a bit confused because the LINUX type is not used in this
series.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 1/6] libxl: move vkb device to libxl_vkb.c

[Wei Liu]

On Thu, Oct 05, 2017 at 12:07:06PM +0300, Oleksandr Grytsov wrote:
> From: Oleksandr Grytsov 
> 
> Logically it is better to move vkb to
> separate file as vkb device used not only by vfb
> and console.
> 
> Signed-off-by: Oleksandr Grytsov 

Acked-by: Wei Liu 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 2/6] libxl: fix vkb XS entry and type

[Wei Liu]

On Thu, Oct 05, 2017 at 12:07:07PM +0300, Oleksandr Grytsov wrote:
> From: Oleksandr Grytsov 
> 
> vkb has vkbd name in XS.
> 
> Signed-off-by: Oleksandr Grytsov 

Acked-by: Wei Liu 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 5/5] docs: add PV sound device config

[Marek Marczykowski-Górecki]

On Mon, Oct 02, 2017 at 12:49:24PM +0300, Oleksandr Grytsov wrote:
> +=item 

Re: [Xen-devel] [PATCH 4/5] xl: add vsnd CLI commands

[Wei Liu]

On Mon, Oct 02, 2017 at 12:49:23PM +0300, Oleksandr Grytsov wrote:
> From: Oleksandr Grytsov 
> 
> Add CLI commands to attach, detach and list virtual sound devices
> 
> Signed-off-by: Oleksandr Grytsov 

Acked-by: Wei Liu 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v13 11/11] tools/libxenctrl: use new xenforeignmemory API to seed grant table

[Paul Durrant]

A previous patch added support for priv-mapping guest resources directly
(rather than having to foreign-map, which requires P2M modification for
HVM guests).

This patch makes use of the new API to seed the guest grant table unless
the underlying infrastructure (i.e. privcmd) doesn't support it, in which
case the old scheme is used.

NOTE: The call to xc_dom_gnttab_hvm_seed() in hvm_build_set_params() was
  actually unnecessary, as the grant table has already been seeded
  by a prior call to xc_dom_gnttab_init() made by libxl__build_dom().

Signed-off-by: Paul Durrant 
Acked-by: Marek Marczykowski-Górecki 
Reviewed-by: Roger Pau Monné 
Acked-by: Wei Liu 
---
Cc: Ian Jackson 

v13:
 - Re-base.

v10:
 - Use new id constant for grant table.

v4:
 - Minor cosmetic fix suggested by Roger.

v3:
 - Introduced xc_dom_set_gnttab_entry() to avoid duplicated code.
---
 tools/libxc/include/xc_dom.h|   8 +--
 tools/libxc/xc_dom_boot.c   | 114 +---
 tools/libxc/xc_sr_restore_x86_hvm.c |  10 ++--
 tools/libxc/xc_sr_restore_x86_pv.c  |   2 +-
 tools/libxl/libxl_dom.c |   1 -
 tools/python/xen/lowlevel/xc/xc.c   |   6 +-
 6 files changed, 92 insertions(+), 49 deletions(-)

diff --git a/tools/libxc/include/xc_dom.h b/tools/libxc/include/xc_dom.h
index cdcdd07d2b..45c9d676c7 100644
--- a/tools/libxc/include/xc_dom.h
+++ b/tools/libxc/include/xc_dom.h
@@ -325,12 +325,8 @@ void *xc_dom_boot_domU_map(struct xc_dom_image *dom, 
xen_pfn_t pfn,
 int xc_dom_boot_image(struct xc_dom_image *dom);
 int xc_dom_compat_check(struct xc_dom_image *dom);
 int xc_dom_gnttab_init(struct xc_dom_image *dom);
-int xc_dom_gnttab_hvm_seed(xc_interface *xch, uint32_t domid,
-   xen_pfn_t console_gmfn,
-   xen_pfn_t xenstore_gmfn,
-   uint32_t console_domid,
-   uint32_t xenstore_domid);
-int xc_dom_gnttab_seed(xc_interface *xch, uint32_t domid,
+int xc_dom_gnttab_seed(xc_interface *xch, uint32_t guest_domid,
+   bool is_hvm,
xen_pfn_t console_gmfn,
xen_pfn_t xenstore_gmfn,
uint32_t console_domid,
diff --git a/tools/libxc/xc_dom_boot.c b/tools/libxc/xc_dom_boot.c
index 40eb5185a9..01e9a1e185 100644
--- a/tools/libxc/xc_dom_boot.c
+++ b/tools/libxc/xc_dom_boot.c
@@ -282,11 +282,29 @@ static xen_pfn_t xc_dom_gnttab_setup(xc_interface *xch, 
uint32_t domid)
 return gmfn;
 }
 
-int xc_dom_gnttab_seed(xc_interface *xch, uint32_t domid,
-   xen_pfn_t console_gmfn,
-   xen_pfn_t xenstore_gmfn,
-   uint32_t console_domid,
-   uint32_t xenstore_domid)
+static void xc_dom_set_gnttab_entry(xc_interface *xch,
+grant_entry_v1_t *gnttab,
+unsigned int idx,
+uint32_t guest_domid,
+uint32_t backend_domid,
+xen_pfn_t backend_gmfn)
+{
+if ( guest_domid == backend_domid || backend_gmfn == -1)
+return;
+
+xc_dom_printf(xch, "%s: [%u] -> 0x%"PRI_xen_pfn,
+  __FUNCTION__, idx, backend_gmfn);
+
+gnttab[idx].flags = GTF_permit_access;
+gnttab[idx].domid = backend_domid;
+gnttab[idx].frame = backend_gmfn;
+}
+
+static int compat_gnttab_seed(xc_interface *xch, uint32_t domid,
+  xen_pfn_t console_gmfn,
+  xen_pfn_t xenstore_gmfn,
+  uint32_t console_domid,
+  uint32_t xenstore_domid)
 {
 
 xen_pfn_t gnttab_gmfn;
@@ -310,18 +328,10 @@ int xc_dom_gnttab_seed(xc_interface *xch, uint32_t domid,
 return -1;
 }
 
-if ( domid != console_domid  && console_gmfn != -1)
-{
-gnttab[GNTTAB_RESERVED_CONSOLE].flags = GTF_permit_access;
-gnttab[GNTTAB_RESERVED_CONSOLE].domid = console_domid;
-gnttab[GNTTAB_RESERVED_CONSOLE].frame = console_gmfn;
-}
-if ( domid != xenstore_domid && xenstore_gmfn != -1)
-{
-gnttab[GNTTAB_RESERVED_XENSTORE].flags = GTF_permit_access;
-gnttab[GNTTAB_RESERVED_XENSTORE].domid = xenstore_domid;
-gnttab[GNTTAB_RESERVED_XENSTORE].frame = xenstore_gmfn;
-}
+xc_dom_set_gnttab_entry(xch, gnttab, GNTTAB_RESERVED_CONSOLE,
+domid, console_domid, console_gmfn);
+xc_dom_set_gnttab_entry(xch, gnttab, GNTTAB_RESERVED_XENSTORE,
+domid, xenstore_domid, xenstore_gmfn);
 
 if ( munmap(gnttab, PAGE_SIZE) == -1 )
 {
@@ -339,11 +349,11 @@ int xc_dom_gnttab_seed(xc_interface *xch, uint32_t domid,
 return 0;
 }
 
-int 

[Xen-devel] [PATCH v13 10/11] common: add a new mappable resource type: XENMEM_resource_grant_table

[Paul Durrant]

This patch allows grant table frames to be mapped using the
XENMEM_acquire_resource memory op.

NOTE: This patch expands the on-stack mfn_list array in acquire_resource()
  but it is still small enough to remain on-stack.

Signed-off-by: Paul Durrant 
---
Cc: Andrew Cooper 
Cc: George Dunlap 
Cc: Ian Jackson 
Cc: Jan Beulich 
Cc: Konrad Rzeszutek Wilk 
Cc: Stefano Stabellini 
Cc: Tim Deegan 
Cc: Wei Liu 

v13:
 - Re-work the internals to avoid using the XENMAPIDX_grant_table_status
   hack.

v12:
 - Dropped limit checks as requested by Jan.

v10:
 - Addressed comments from Jan.

v8:
 - The functionality was originally incorporated into the earlier patch
   "x86/mm: add HYPERVISOR_memory_op to acquire guest resources".
---
 xen/common/grant_table.c  | 63 +--
 xen/common/memory.c   | 45 ++-
 xen/include/public/memory.h   |  6 +
 xen/include/xen/grant_table.h |  4 +++
 4 files changed, 109 insertions(+), 9 deletions(-)

diff --git a/xen/common/grant_table.c b/xen/common/grant_table.c
index 0558de9ce8..429c421cd9 100644
--- a/xen/common/grant_table.c
+++ b/xen/common/grant_table.c
@@ -3761,21 +3761,21 @@ int mem_sharing_gref_to_gfn(struct grant_table *gt, 
grant_ref_t ref,
 }
 #endif
 
-int gnttab_map_frame(struct domain *d, unsigned long idx, gfn_t gfn,
- mfn_t *mfn)
+/* Caller must hold write lock as version may change and table may grow */
+static int gnttab_get_frame(struct domain *d, bool is_status,
+unsigned long idx, mfn_t *mfn)
 {
-int rc = 0;
 struct grant_table *gt = d->grant_table;
-
-grant_write_lock(gt);
+int rc = 0;
 
 if ( gt->gt_version == 0 )
 gt->gt_version = 1;
 
-if ( gt->gt_version == 2 &&
- (idx & XENMAPIDX_grant_table_status) )
+if ( is_status )
 {
-idx &= ~XENMAPIDX_grant_table_status;
+if ( gt->gt_version != 2 )
+return -EINVAL;
+
 if ( idx < nr_status_frames(gt) )
 *mfn = _mfn(virt_to_mfn(gt->status[idx]));
 else
@@ -3792,6 +3792,25 @@ int gnttab_map_frame(struct domain *d, unsigned long 
idx, gfn_t gfn,
 rc = -EINVAL;
 }
 
+return rc;
+}
+
+int gnttab_map_frame(struct domain *d, unsigned long idx, gfn_t gfn,
+ mfn_t *mfn)
+{
+struct grant_table *gt = d->grant_table;
+bool is_status = false;
+int rc;
+
+grant_write_lock(gt);
+
+if ( idx & XENMAPIDX_grant_table_status )
+{
+is_status = true;
+idx &= ~XENMAPIDX_grant_table_status;
+}
+
+rc = gnttab_get_frame(d, is_status, idx, mfn);
 if ( !rc )
 gnttab_set_frame_gfn(gt, idx, gfn);
 
@@ -3800,6 +3819,34 @@ int gnttab_map_frame(struct domain *d, unsigned long 
idx, gfn_t gfn,
 return rc;
 }
 
+int gnttab_get_grant_frame(struct domain *d, unsigned long idx,
+   mfn_t *mfn)
+{
+struct grant_table *gt = d->grant_table;
+int rc;
+
+/* write lock required as version may change and/or table may grow */
+grant_write_lock(gt);
+rc = gnttab_get_frame(d, false, idx, mfn);
+grant_write_unlock(gt);
+
+return rc;
+}
+
+int gnttab_get_status_frame(struct domain *d, unsigned long idx,
+mfn_t *mfn)
+{
+struct grant_table *gt = d->grant_table;
+int rc;
+
+/* write lock required as version may change and/or table may grow */
+grant_write_lock(gt);
+rc = gnttab_get_frame(d, true, idx, mfn);
+grant_write_unlock(gt);
+
+return rc;
+}
+
 static void gnttab_usage_print(struct domain *rd)
 {
 int first = 1;
diff --git a/xen/common/memory.c b/xen/common/memory.c
index 1c6932fd85..8097d85be3 100644
--- a/xen/common/memory.c
+++ b/xen/common/memory.c
@@ -23,6 +23,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -965,6 +966,43 @@ static long xatp_permission_check(struct domain *d, 
unsigned int space)
 return xsm_add_to_physmap(XSM_TARGET, current->domain, d);
 }
 
+static int acquire_grant_table(struct domain *d, unsigned int id,
+   unsigned long frame,
+   unsigned int nr_frames,
+   xen_pfn_t mfn_list[])
+{
+unsigned int i = nr_frames;
+
+/* Iterate backwards in case table needs to grow */
+while ( i-- != 0 )
+{
+mfn_t mfn = INVALID_MFN;
+int rc;
+
+switch ( id )
+{
+case XENMEM_resource_grant_table_id_grant:
+rc = gnttab_get_grant_frame(d, frame + i, );
+break;
+
+case XENMEM_resource_grant_table_id_status:
+rc = gnttab_get_status_frame(d, frame + i, );
+break;
+
+

Re: [Xen-devel] [PATCH 3/5] xl: add PV sound condif parser

[Wei Liu]

On Mon, Oct 02, 2017 at 12:49:22PM +0300, Oleksandr Grytsov wrote:
> +static void parse_vsnd_card_config(const XLU_Config *config,
> +   XLU_ConfigValue *card_value,
> +   libxl_domain_config *d_config)
> +{
> +int ret;
> +
> +XLU_ConfigList *card_list;
> +
> +// get card

Please delete this.

> +ret = xlu_cfg_value_get_list(config, card_value,  _list, 0);
> +
> +if (ret) {
> +fprintf(stderr, "Failed to get vsnd card list: %s\n", strerror(ret));
> +goto out;
> +}
> +
> +libxl_device_vsnd *vsnd;
> +
> +vsnd = ARRAY_EXTEND_INIT(d_config->vsnds,
> + d_config->num_vsnds,
> + libxl_device_vsnd_init);
> +
> +const char *card_item;
> +int item = 0;
> +

Mixing code and declarations will break.

> +while ((card_item = xlu_cfg_get_listitem(card_list, item++)) != NULL) {
> +ret = parse_vsnd_item(vsnd, card_item);
> +if (ret) goto out;
> +}
> +
> +ret = 0;
> +
> +out:
> +if (ret) exit(EXIT_FAILURE);
> +}

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 2/5] libxl: add vsnd list and info

[Wei Liu]

On Mon, Oct 02, 2017 at 12:49:21PM +0300, Oleksandr Grytsov wrote:
> From: Oleksandr Grytsov 
> 
> Add getting vsnd list amd info API
> 
> Signed-off-by: Oleksandr Grytsov 

Same comments for previous patch apply here, too.

> ---
>  tools/libxl/libxl.h |  10 ++
>  tools/libxl/libxl_types.idl |  19 +++
>  tools/libxl/libxl_utils.h   |   3 +
>  tools/libxl/libxl_vsnd.c| 359 
> +++-
>  4 files changed, 388 insertions(+), 3 deletions(-)
> 
>  
[...]
> +static int libxl__sample_rates_from_string(libxl__gc *gc, const char *str,
> +   libxl_vsnd_params *params)
> +{
> +char *tmp = libxl__strdup(gc, str);
> +
> +params->num_sample_rates = 0;
> +params->sample_rates = NULL;
> +
> +char *p = strtok(tmp, " ,");
> +
> +while (p != NULL) {
> +params->sample_rates = realloc(params->sample_rates,

libxl__realloc(NOGC, ...)

[...]
> +
> +static int libxl__pcm_from_xenstore(libxl__gc *gc, const char *path,
> +libxl_vsnd_pcm *pcm)
> +{
> +libxl_ctx *ctx = libxl__gc_owner(gc);

You can use CTX throughout.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v13 02/11] x86/hvm/ioreq: simplify code and use consistent naming

[Paul Durrant]

This patch re-works much of the ioreq server initialization and teardown
code:

- The hvm_map/unmap_ioreq_gfn() functions are expanded to call through
  to hvm_alloc/free_ioreq_gfn() rather than expecting them to be called
  separately by outer functions.
- Several functions now test the validity of the hvm_ioreq_page gfn value
  to determine whether they need to act. This means can be safely called
  for the bufioreq page even when it is not used.
- hvm_add/remove_ioreq_gfn() simply return in the case of the default
  IOREQ server so callers no longer need to test before calling.
- hvm_ioreq_server_setup_pages() is renamed to hvm_ioreq_server_map_pages()
  to mirror the existing hvm_ioreq_server_unmap_pages().

All of this significantly shortens the code.

Signed-off-by: Paul Durrant 
Reviewed-by: Roger Pau Monné 
Reviewed-by: Wei Liu 
Acked-by: Jan Beulich 
---
Cc: Andrew Cooper 

v3:
 - Rebased on top of 's->is_default' to 'IS_DEFAULT(s)' changes.
 - Minor updates in response to review comments from Roger.
---
 xen/arch/x86/hvm/ioreq.c | 182 ++-
 1 file changed, 69 insertions(+), 113 deletions(-)

diff --git a/xen/arch/x86/hvm/ioreq.c b/xen/arch/x86/hvm/ioreq.c
index da31918bb1..c21fa9f280 100644
--- a/xen/arch/x86/hvm/ioreq.c
+++ b/xen/arch/x86/hvm/ioreq.c
@@ -210,63 +210,75 @@ bool handle_hvm_io_completion(struct vcpu *v)
 return true;
 }
 
-static int hvm_alloc_ioreq_gfn(struct domain *d, unsigned long *gfn)
+static unsigned long hvm_alloc_ioreq_gfn(struct hvm_ioreq_server *s)
 {
+struct domain *d = s->domain;
 unsigned int i;
-int rc;
 
-rc = -ENOMEM;
+ASSERT(!IS_DEFAULT(s));
+
 for ( i = 0; i < sizeof(d->arch.hvm_domain.ioreq_gfn.mask) * 8; i++ )
 {
 if ( test_and_clear_bit(i, >arch.hvm_domain.ioreq_gfn.mask) )
-{
-*gfn = d->arch.hvm_domain.ioreq_gfn.base + i;
-rc = 0;
-break;
-}
+return d->arch.hvm_domain.ioreq_gfn.base + i;
 }
 
-return rc;
+return gfn_x(INVALID_GFN);
 }
 
-static void hvm_free_ioreq_gfn(struct domain *d, unsigned long gfn)
+static void hvm_free_ioreq_gfn(struct hvm_ioreq_server *s,
+   unsigned long gfn)
 {
+struct domain *d = s->domain;
 unsigned int i = gfn - d->arch.hvm_domain.ioreq_gfn.base;
 
-if ( gfn != gfn_x(INVALID_GFN) )
-set_bit(i, >arch.hvm_domain.ioreq_gfn.mask);
+ASSERT(!IS_DEFAULT(s));
+ASSERT(gfn != gfn_x(INVALID_GFN));
+
+set_bit(i, >arch.hvm_domain.ioreq_gfn.mask);
 }
 
-static void hvm_unmap_ioreq_page(struct hvm_ioreq_server *s, bool buf)
+static void hvm_unmap_ioreq_gfn(struct hvm_ioreq_server *s, bool buf)
 {
 struct hvm_ioreq_page *iorp = buf ? >bufioreq : >ioreq;
 
+if ( iorp->gfn == gfn_x(INVALID_GFN) )
+return;
+
 destroy_ring_for_helper(>va, iorp->page);
+iorp->page = NULL;
+
+if ( !IS_DEFAULT(s) )
+hvm_free_ioreq_gfn(s, iorp->gfn);
+
+iorp->gfn = gfn_x(INVALID_GFN);
 }
 
-static int hvm_map_ioreq_page(
-struct hvm_ioreq_server *s, bool buf, unsigned long gfn)
+static int hvm_map_ioreq_gfn(struct hvm_ioreq_server *s, bool buf)
 {
 struct domain *d = s->domain;
 struct hvm_ioreq_page *iorp = buf ? >bufioreq : >ioreq;
-struct page_info *page;
-void *va;
 int rc;
 
-if ( (rc = prepare_ring_for_helper(d, gfn, , )) )
-return rc;
-
-if ( (iorp->va != NULL) || d->is_dying )
-{
-destroy_ring_for_helper(, page);
+if ( d->is_dying )
 return -EINVAL;
-}
 
-iorp->va = va;
-iorp->page = page;
-iorp->gfn = gfn;
+if ( IS_DEFAULT(s) )
+iorp->gfn = buf ?
+d->arch.hvm_domain.params[HVM_PARAM_BUFIOREQ_PFN] :
+d->arch.hvm_domain.params[HVM_PARAM_IOREQ_PFN];
+else
+iorp->gfn = hvm_alloc_ioreq_gfn(s);
 
-return 0;
+if ( iorp->gfn == gfn_x(INVALID_GFN) )
+return -ENOMEM;
+
+rc = prepare_ring_for_helper(d, iorp->gfn, >page, >va);
+
+if ( rc )
+hvm_unmap_ioreq_gfn(s, buf);
+
+return rc;
 }
 
 bool is_ioreq_server_page(struct domain *d, const struct page_info *page)
@@ -279,8 +291,7 @@ bool is_ioreq_server_page(struct domain *d, const struct 
page_info *page)
 
 FOR_EACH_IOREQ_SERVER(d, id, s)
 {
-if ( (s->ioreq.va && s->ioreq.page == page) ||
- (s->bufioreq.va && s->bufioreq.page == page) )
+if ( (s->ioreq.page == page) || (s->bufioreq.page == page) )
 {
 found = true;
 break;
@@ -292,20 +303,30 @@ bool is_ioreq_server_page(struct domain *d, const struct 
page_info *page)
 return found;
 }
 
-static void hvm_remove_ioreq_gfn(
-struct domain *d, struct hvm_ioreq_page *iorp)
+static void hvm_remove_ioreq_gfn(struct hvm_ioreq_server *s, bool buf)
+
 {
+

[Xen-devel] [PATCH v13 04/11] x86/hvm/ioreq: defer mapping gfns until they are actually requsted

[Paul Durrant]

A subsequent patch will introduce a new scheme to allow an emulator to
map ioreq server pages directly from Xen rather than the guest P2M.

This patch lays the groundwork for that change by deferring mapping of
gfns until their values are requested by an emulator. To that end, the
pad field of the xen_dm_op_get_ioreq_server_info structure is re-purposed
to a flags field and new flag, XEN_DMOP_no_gfns, defined which modifies the
behaviour of XEN_DMOP_get_ioreq_server_info to allow the caller to avoid
requesting the gfn values.

Signed-off-by: Paul Durrant 
Reviewed-by: Roger Pau Monné 
Acked-by: Wei Liu 
Reviewed-by: Jan Beulich 
---
Cc: Ian Jackson 
Cc: Andrew Cooper 
Cc: George Dunlap 
Cc: Konrad Rzeszutek Wilk 
Cc: Stefano Stabellini 
Cc: Tim Deegan 

v8:
 - For safety make all of the pointers passed to
   hvm_get_ioreq_server_info() optional.
 - Shrink bufioreq_handling down to a uint8_t.

v3:
 - Updated in response to review comments from Wei and Roger.
 - Added a HANDLE_BUFIOREQ macro to make the code neater.
 - This patch no longer introduces a security vulnerability since there
   is now an explicit limit on the number of ioreq servers that may be
   created for any one domain.
---
 tools/libs/devicemodel/core.c   |  8 +
 tools/libs/devicemodel/include/xendevicemodel.h |  6 ++--
 xen/arch/x86/hvm/dm.c   |  9 +++--
 xen/arch/x86/hvm/ioreq.c| 47 ++---
 xen/include/asm-x86/hvm/domain.h|  2 +-
 xen/include/public/hvm/dm_op.h  | 32 ++---
 6 files changed, 63 insertions(+), 41 deletions(-)

diff --git a/tools/libs/devicemodel/core.c b/tools/libs/devicemodel/core.c
index b66d4f9294..e684e657b6 100644
--- a/tools/libs/devicemodel/core.c
+++ b/tools/libs/devicemodel/core.c
@@ -204,6 +204,14 @@ int xendevicemodel_get_ioreq_server_info(
 
 data->id = id;
 
+/*
+ * If the caller is not requesting gfn values then instruct the
+ * hypercall not to retrieve them as this may cause them to be
+ * mapped.
+ */
+if (!ioreq_gfn && !bufioreq_gfn)
+data->flags |= XEN_DMOP_no_gfns;
+
 rc = xendevicemodel_op(dmod, domid, 1, , sizeof(op));
 if (rc)
 return rc;
diff --git a/tools/libs/devicemodel/include/xendevicemodel.h 
b/tools/libs/devicemodel/include/xendevicemodel.h
index dda0bc7695..fffee3a4a0 100644
--- a/tools/libs/devicemodel/include/xendevicemodel.h
+++ b/tools/libs/devicemodel/include/xendevicemodel.h
@@ -61,11 +61,11 @@ int xendevicemodel_create_ioreq_server(
  * @parm domid the domain id to be serviced
  * @parm id the IOREQ Server id.
  * @parm ioreq_gfn pointer to a xen_pfn_t to receive the synchronous ioreq
- *  gfn
+ *  gfn. (May be NULL if not required)
  * @parm bufioreq_gfn pointer to a xen_pfn_t to receive the buffered ioreq
- *gfn
+ *gfn. (May be NULL if not required)
  * @parm bufioreq_port pointer to a evtchn_port_t to receive the buffered
- * ioreq event channel
+ * ioreq event channel. (May be NULL if not required)
  * @return 0 on success, -1 on failure.
  */
 int xendevicemodel_get_ioreq_server_info(
diff --git a/xen/arch/x86/hvm/dm.c b/xen/arch/x86/hvm/dm.c
index 32ade9541d..4d10e91adb 100644
--- a/xen/arch/x86/hvm/dm.c
+++ b/xen/arch/x86/hvm/dm.c
@@ -416,16 +416,19 @@ static int dm_op(const struct dmop_args *op_args)
 {
 struct xen_dm_op_get_ioreq_server_info *data =
 _ioreq_server_info;
+const uint16_t valid_flags = XEN_DMOP_no_gfns;
 
 const_op = false;
 
 rc = -EINVAL;
-if ( data->pad )
+if ( data->flags & ~valid_flags )
 break;
 
 rc = hvm_get_ioreq_server_info(d, data->id,
-   >ioreq_gfn,
-   >bufioreq_gfn,
+   (data->flags & XEN_DMOP_no_gfns) ?
+   NULL : >ioreq_gfn,
+   (data->flags & XEN_DMOP_no_gfns) ?
+   NULL : >bufioreq_gfn,
>bufioreq_port);
 break;
 }
diff --git a/xen/arch/x86/hvm/ioreq.c b/xen/arch/x86/hvm/ioreq.c
index eec4e4771e..39de659ddf 100644
--- a/xen/arch/x86/hvm/ioreq.c
+++ b/xen/arch/x86/hvm/ioreq.c
@@ -350,6 +350,9 @@ static void hvm_update_ioreq_evtchn(struct hvm_ioreq_server 
*s,
 }
 }
 
+#define HANDLE_BUFIOREQ(s) \
+((s)->bufioreq_handling != HVM_IOREQSRV_BUFIOREQ_OFF)
+
 static int hvm_ioreq_server_add_vcpu(struct hvm_ioreq_server *s,
  struct vcpu *v)
 {
@@ 

[Xen-devel] [PATCH v13 05/11] x86/mm: add HYPERVISOR_memory_op to acquire guest resources

[Paul Durrant]

Certain memory resources associated with a guest are not necessarily
present in the guest P2M.

This patch adds the boilerplate for new memory op to allow such a resource
to be priv-mapped directly, by either a PV or HVM tools domain.

NOTE: Whilst the new op is not intrinsicly specific to the x86 architecture,
  I have no means to test it on an ARM platform and so cannot verify
  that it functions correctly.

Signed-off-by: Paul Durrant 
---
Cc: George Dunlap 
Cc: Jan Beulich 
Cc: Andrew Cooper 
Cc: George Dunlap 
Cc: Ian Jackson 
Cc: Konrad Rzeszutek Wilk 
Cc: Stefano Stabellini 
Cc: Tim Deegan 
Cc: Wei Liu 
Cc: Daniel De Graaf 
Cc: Julien Grall 

v13:
 - Use xen_pfn_t for mfn_list.
 - Addressed further comments from Jan and Julien.

v12:
 - Addressed more comments form Jan.
 - Removed #ifdef CONFIG_X86 from common code and instead introduced a
   stub set_foreign_p2m_entry() in asm-arm/p2m.h returning -EOPNOTSUPP.
 - Restricted mechanism for querying implementation limit on nr_frames
   and simplified compat code.

v11:
 - Addressed more comments from Jan.

v9:
 - Addressed more comments from Jan.

v8:
 - Move the code into common as requested by Jan.
 - Make the gmfn_list handle a 64-bit type to avoid limiting the MFN
   range for a 32-bit tools domain.
 - Add missing pad.
 - Add compat code.
 - Make this patch deal with purely boilerplate.
 - Drop George's A-b and Wei's R-b because the changes are non-trivial,
   and update Cc list now the boilerplate is common.

v5:
 - Switched __copy_to/from_guest_offset() to copy_to/from_guest_offset().
---
 tools/flask/policy/modules/xen.if   |  4 +-
 xen/arch/x86/mm/p2m.c   |  3 +-
 xen/common/compat/memory.c  | 95 +
 xen/common/memory.c | 93 
 xen/include/asm-arm/p2m.h   | 10 
 xen/include/asm-x86/p2m.h   |  3 ++
 xen/include/public/memory.h | 43 -
 xen/include/xlat.lst|  1 +
 xen/include/xsm/dummy.h |  6 +++
 xen/include/xsm/xsm.h   |  6 +++
 xen/xsm/dummy.c |  1 +
 xen/xsm/flask/hooks.c   |  6 +++
 xen/xsm/flask/policy/access_vectors |  2 +
 13 files changed, 269 insertions(+), 4 deletions(-)

diff --git a/tools/flask/policy/modules/xen.if 
b/tools/flask/policy/modules/xen.if
index 55437496f6..07cba8a15d 100644
--- a/tools/flask/policy/modules/xen.if
+++ b/tools/flask/policy/modules/xen.if
@@ -52,7 +52,8 @@ define(`create_domain_common', `
settime setdomainhandle getvcpucontext set_misc_info };
allow $1 $2:domain2 { set_cpuid settsc setscheduler setclaim
set_max_evtchn set_vnumainfo get_vnumainfo cacheflush
-   psr_cmt_op psr_cat_op soft_reset set_gnttab_limits };
+   psr_cmt_op psr_cat_op soft_reset set_gnttab_limits
+   resource_map };
allow $1 $2:security check_context;
allow $1 $2:shadow enable;
allow $1 $2:mmu { map_read map_write adjust memorymap physmap pinpage 
mmuext_op updatemp };
@@ -152,6 +153,7 @@ define(`device_model', `
allow $1 $2_target:domain { getdomaininfo shutdown };
allow $1 $2_target:mmu { map_read map_write adjust physmap target_hack 
};
allow $1 $2_target:hvm { getparam setparam hvmctl cacheattr dm };
+   allow $1 $2_target:domain2 resource_map;
 ')
 
 # make_device_model(priv, dm_dom, hvm_dom)
diff --git a/xen/arch/x86/mm/p2m.c b/xen/arch/x86/mm/p2m.c
index c72a3cdebb..71bb9b4f93 100644
--- a/xen/arch/x86/mm/p2m.c
+++ b/xen/arch/x86/mm/p2m.c
@@ -1132,8 +1132,7 @@ static int set_typed_p2m_entry(struct domain *d, unsigned 
long gfn_l,
 }
 
 /* Set foreign mfn in the given guest's p2m table. */
-static int set_foreign_p2m_entry(struct domain *d, unsigned long gfn,
- mfn_t mfn)
+int set_foreign_p2m_entry(struct domain *d, unsigned long gfn, mfn_t mfn)
 {
 return set_typed_p2m_entry(d, gfn, mfn, PAGE_ORDER_4K, p2m_map_foreign,
p2m_get_hostp2m(d)->default_access);
diff --git a/xen/common/compat/memory.c b/xen/common/compat/memory.c
index 35bb259808..9a7cb1a71b 100644
--- a/xen/common/compat/memory.c
+++ b/xen/common/compat/memory.c
@@ -71,6 +71,7 @@ int compat_memory_op(unsigned int cmd, 
XEN_GUEST_HANDLE_PARAM(void) compat)
 struct xen_remove_from_physmap *xrfp;
 struct xen_vnuma_topology_info *vnuma;
 struct xen_mem_access_op *mao;
+struct xen_mem_acquire_resource *mar;
 } nat;
 union {
 struct compat_memory_reservation rsrv;

[Xen-devel] [PATCH v13 09/11] tools/libxenforeignmemory: reduce xenforeignmemory_restrict code footprint

[Paul Durrant]

By using a static inline stub in private.h for OS where this functionality
is not implemented, the various duplicate stubs in the OS-specific source
modules can be avoided.

Signed-off-by: Paul Durrant 
Reviewed-by: Roger Pau Monné 
Acked-by: Wei Liu 
---
Cc: Ian Jackson 

v4:
 - Removed extraneous freebsd code.

v3:
 - Patch added in response to review comments.
---
 tools/libs/foreignmemory/freebsd.c |  7 ---
 tools/libs/foreignmemory/minios.c  |  7 ---
 tools/libs/foreignmemory/netbsd.c  |  7 ---
 tools/libs/foreignmemory/private.h | 12 +---
 tools/libs/foreignmemory/solaris.c |  7 ---
 5 files changed, 9 insertions(+), 31 deletions(-)

diff --git a/tools/libs/foreignmemory/freebsd.c 
b/tools/libs/foreignmemory/freebsd.c
index dec447485a..6e6bc4b11f 100644
--- a/tools/libs/foreignmemory/freebsd.c
+++ b/tools/libs/foreignmemory/freebsd.c
@@ -95,13 +95,6 @@ int osdep_xenforeignmemory_unmap(xenforeignmemory_handle 
*fmem,
 return munmap(addr, num << PAGE_SHIFT);
 }
 
-int osdep_xenforeignmemory_restrict(xenforeignmemory_handle *fmem,
-domid_t domid)
-{
-errno = -EOPNOTSUPP;
-return -1;
-}
-
 /*
  * Local variables:
  * mode: C
diff --git a/tools/libs/foreignmemory/minios.c 
b/tools/libs/foreignmemory/minios.c
index 75f340122e..43341ca301 100644
--- a/tools/libs/foreignmemory/minios.c
+++ b/tools/libs/foreignmemory/minios.c
@@ -58,13 +58,6 @@ int osdep_xenforeignmemory_unmap(xenforeignmemory_handle 
*fmem,
 return munmap(addr, num << PAGE_SHIFT);
 }
 
-int osdep_xenforeignmemory_restrict(xenforeignmemory_handle *fmem,
-domid_t domid)
-{
-errno = -EOPNOTSUPP;
-return -1;
-}
-
 /*
  * Local variables:
  * mode: C
diff --git a/tools/libs/foreignmemory/netbsd.c 
b/tools/libs/foreignmemory/netbsd.c
index 9bf95ef4f0..54a418ebd6 100644
--- a/tools/libs/foreignmemory/netbsd.c
+++ b/tools/libs/foreignmemory/netbsd.c
@@ -100,13 +100,6 @@ int osdep_xenforeignmemory_unmap(xenforeignmemory_handle 
*fmem,
 return munmap(addr, num*XC_PAGE_SIZE);
 }
 
-int osdep_xenforeignmemory_restrict(xenforeignmemory_handle *fmem,
-domid_t domid)
-{
-errno = -EOPNOTSUPP;
-return -1;
-}
-
 /*
  * Local variables:
  * mode: C
diff --git a/tools/libs/foreignmemory/private.h 
b/tools/libs/foreignmemory/private.h
index b191000b49..b06ce12583 100644
--- a/tools/libs/foreignmemory/private.h
+++ b/tools/libs/foreignmemory/private.h
@@ -35,9 +35,6 @@ void *osdep_xenforeignmemory_map(xenforeignmemory_handle 
*fmem,
 int osdep_xenforeignmemory_unmap(xenforeignmemory_handle *fmem,
  void *addr, size_t num);
 
-int osdep_xenforeignmemory_restrict(xenforeignmemory_handle *fmem,
-domid_t domid);
-
 #if defined(__NetBSD__) || defined(__sun__)
 /* Strictly compat for those two only only */
 void *compat_mapforeign_batch(xenforeignmem_handle *fmem, uint32_t dom,
@@ -57,6 +54,13 @@ struct xenforeignmemory_resource_handle {
 };
 
 #ifndef __linux__
+static inline int osdep_xenforeignmemory_restrict(xenforeignmemory_handle 
*fmem,
+  domid_t domid)
+{
+errno = EOPNOTSUPP;
+return -1;
+}
+
 static inline int osdep_xenforeignmemory_map_resource(
 xenforeignmemory_handle *fmem, xenforeignmemory_resource_handle *fres)
 {
@@ -70,6 +74,8 @@ static inline int osdep_xenforeignmemory_unmap_resource(
 return 0;
 }
 #else
+int osdep_xenforeignmemory_restrict(xenforeignmemory_handle *fmem,
+domid_t domid);
 int osdep_xenforeignmemory_map_resource(
 xenforeignmemory_handle *fmem, xenforeignmemory_resource_handle *fres);
 int osdep_xenforeignmemory_unmap_resource(
diff --git a/tools/libs/foreignmemory/solaris.c 
b/tools/libs/foreignmemory/solaris.c
index a33decb4ae..ee8aae4fbd 100644
--- a/tools/libs/foreignmemory/solaris.c
+++ b/tools/libs/foreignmemory/solaris.c
@@ -97,13 +97,6 @@ int osdep_xenforeignmemory_unmap(xenforeignmemory_handle 
*fmem,
 return munmap(addr, num*XC_PAGE_SIZE);
 }
 
-int osdep_xenforeignmemory_restrict(xenforeignmemory_handle *fmem,
-domid_t domid)
-{
-errno = -EOPNOTSUPP;
-return -1;
-}
-
 /*
  * Local variables:
  * mode: C
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v13 08/11] tools/libxenforeignmemory: add support for resource mapping

[Paul Durrant]

A previous patch introduced a new HYPERVISOR_memory_op to acquire guest
resources for direct priv-mapping.

This patch adds new functionality into libxenforeignmemory to make use
of a new privcmd ioctl [1] that uses the new memory op to make such
resources available via mmap(2).

[1] 
http://xenbits.xen.org/gitweb/?p=people/pauldu/linux.git;a=commit;h=ce59a05e6712

Signed-off-by: Paul Durrant 
Reviewed-by: Roger Pau Monné 
Reviewed-by: Wei Liu 
---
Cc: Ian Jackson 

v4:
 - Fixed errno and removed single-use label
 - The unmap call now returns a status
 - Use C99 initialization for ioctl struct

v2:
 - Bump minor version up to 3.
---
 tools/include/xen-sys/Linux/privcmd.h  | 11 +
 tools/libs/foreignmemory/Makefile  |  2 +-
 tools/libs/foreignmemory/core.c| 53 ++
 .../libs/foreignmemory/include/xenforeignmemory.h  | 41 +
 tools/libs/foreignmemory/libxenforeignmemory.map   |  5 ++
 tools/libs/foreignmemory/linux.c   | 45 ++
 tools/libs/foreignmemory/private.h | 31 +
 7 files changed, 187 insertions(+), 1 deletion(-)

diff --git a/tools/include/xen-sys/Linux/privcmd.h 
b/tools/include/xen-sys/Linux/privcmd.h
index 732ff7c15a..9531b728f9 100644
--- a/tools/include/xen-sys/Linux/privcmd.h
+++ b/tools/include/xen-sys/Linux/privcmd.h
@@ -86,6 +86,15 @@ typedef struct privcmd_dm_op {
const privcmd_dm_op_buf_t __user *ubufs;
 } privcmd_dm_op_t;
 
+typedef struct privcmd_mmap_resource {
+   domid_t dom;
+   __u32 type;
+   __u32 id;
+   __u32 idx;
+   __u64 num;
+   __u64 addr;
+} privcmd_mmap_resource_t;
+
 /*
  * @cmd: IOCTL_PRIVCMD_HYPERCALL
  * @arg: _hypercall_t
@@ -103,5 +112,7 @@ typedef struct privcmd_dm_op {
_IOC(_IOC_NONE, 'P', 5, sizeof(privcmd_dm_op_t))
 #define IOCTL_PRIVCMD_RESTRICT \
_IOC(_IOC_NONE, 'P', 6, sizeof(domid_t))
+#define IOCTL_PRIVCMD_MMAP_RESOURCE\
+   _IOC(_IOC_NONE, 'P', 7, sizeof(privcmd_mmap_resource_t))
 
 #endif /* __LINUX_PUBLIC_PRIVCMD_H__ */
diff --git a/tools/libs/foreignmemory/Makefile 
b/tools/libs/foreignmemory/Makefile
index cbe815fce8..ee5c3fd67e 100644
--- a/tools/libs/foreignmemory/Makefile
+++ b/tools/libs/foreignmemory/Makefile
@@ -2,7 +2,7 @@ XEN_ROOT = $(CURDIR)/../../..
 include $(XEN_ROOT)/tools/Rules.mk
 
 MAJOR= 1
-MINOR= 2
+MINOR= 3
 SHLIB_LDFLAGS += -Wl,--version-script=libxenforeignmemory.map
 
 CFLAGS   += -Werror -Wmissing-prototypes
diff --git a/tools/libs/foreignmemory/core.c b/tools/libs/foreignmemory/core.c
index 79b24d273b..efa915015c 100644
--- a/tools/libs/foreignmemory/core.c
+++ b/tools/libs/foreignmemory/core.c
@@ -17,6 +17,8 @@
 #include 
 #include 
 
+#include 
+
 #include "private.h"
 
 static int all_restrict_cb(Xentoolcore__Active_Handle *ah, domid_t domid) {
@@ -135,6 +137,57 @@ int xenforeignmemory_restrict(xenforeignmemory_handle 
*fmem,
 return osdep_xenforeignmemory_restrict(fmem, domid);
 }
 
+xenforeignmemory_resource_handle *xenforeignmemory_map_resource(
+xenforeignmemory_handle *fmem, domid_t domid, unsigned int type,
+unsigned int id, unsigned long frame, unsigned long nr_frames,
+void **paddr, int prot, int flags)
+{
+xenforeignmemory_resource_handle *fres;
+int rc;
+
+/* Check flags only contains POSIX defined values */
+if ( flags & ~(MAP_SHARED | MAP_PRIVATE) )
+{
+errno = EINVAL;
+return NULL;
+}
+
+fres = calloc(1, sizeof(*fres));
+if ( !fres )
+{
+errno = ENOMEM;
+return NULL;
+}
+
+fres->domid = domid;
+fres->type = type;
+fres->id = id;
+fres->frame = frame;
+fres->nr_frames = nr_frames;
+fres->addr = *paddr;
+fres->prot = prot;
+fres->flags = flags;
+
+rc = osdep_xenforeignmemory_map_resource(fmem, fres);
+if ( rc )
+{
+free(fres);
+fres = NULL;
+} else
+*paddr = fres->addr;
+
+return fres;
+}
+
+int xenforeignmemory_unmap_resource(
+xenforeignmemory_handle *fmem, xenforeignmemory_resource_handle *fres)
+{
+int rc = osdep_xenforeignmemory_unmap_resource(fmem, fres);
+
+free(fres);
+return rc;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/tools/libs/foreignmemory/include/xenforeignmemory.h 
b/tools/libs/foreignmemory/include/xenforeignmemory.h
index f4814c390f..d594be8df0 100644
--- a/tools/libs/foreignmemory/include/xenforeignmemory.h
+++ b/tools/libs/foreignmemory/include/xenforeignmemory.h
@@ -138,6 +138,47 @@ int xenforeignmemory_unmap(xenforeignmemory_handle *fmem,
 int xenforeignmemory_restrict(xenforeignmemory_handle *fmem,
   domid_t domid);
 
+typedef struct xenforeignmemory_resource_handle 
xenforeignmemory_resource_handle;
+
+/**
+ * This 

[Xen-devel] [PATCH v13 06/11] x86/hvm/ioreq: add a new mappable resource type...

[Paul Durrant]

... XENMEM_resource_ioreq_server

This patch adds support for a new resource type that can be mapped using
the XENMEM_acquire_resource memory op.

If an emulator makes use of this resource type then, instead of mapping
gfns, the IOREQ server will allocate pages from the heap. These pages
will never be present in the P2M of the guest at any point and so are
not vulnerable to any direct attack by the guest. They are only ever
accessible by Xen and any domain that has mapping privilege over the
guest (which may or may not be limited to the domain running the emulator).

NOTE: Use of the new resource type is not compatible with use of
  XEN_DMOP_get_ioreq_server_info unless the XEN_DMOP_no_gfns flag is
  set.

Signed-off-by: Paul Durrant 
---
Cc: George Dunlap 
Cc: Wei Liu 
Cc: Jan Beulich 
Cc: Andrew Cooper 
Cc: Ian Jackson 
Cc: Konrad Rzeszutek Wilk 
Cc: Stefano Stabellini 
Cc: Tim Deegan 
Cc: Julien Grall 

v13:
 - Introduce an arch_acquire_resource() as suggested by Julien (and have
   the ARM varient simply return -EOPNOTSUPP).
 - Check for ioreq server id truncation as requested by Jan.
 - Not added Jan's R-b due to substantive change from v12.

v12:
 - Addressed more comments from Jan.
 - Dropped George's A-b and Wei's R-b because of material change.

v11:
 - Addressed more comments from Jan.

v10:
 - Addressed comments from Jan.

v8:
 - Re-base on new boilerplate.
 - Adjust function signature of hvm_get_ioreq_server_frame(), and test
   whether the bufioreq page is present.

v5:
 - Use get_ioreq_server() function rather than indexing array directly.
 - Add more explanation into comments to state than mapping guest frames
   and allocation of pages for ioreq servers are not simultaneously
   permitted.
 - Add a comment into asm/ioreq.h stating the meaning of the index
   value passed to hvm_get_ioreq_server_frame().
---
 xen/arch/x86/hvm/ioreq.c| 156 
 xen/arch/x86/mm.c   |  49 +
 xen/common/memory.c |   3 +-
 xen/include/asm-arm/mm.h|   7 ++
 xen/include/asm-x86/hvm/ioreq.h |   2 +
 xen/include/asm-x86/mm.h|   5 ++
 xen/include/public/hvm/dm_op.h  |   4 ++
 xen/include/public/memory.h |   9 +++
 8 files changed, 234 insertions(+), 1 deletion(-)

diff --git a/xen/arch/x86/hvm/ioreq.c b/xen/arch/x86/hvm/ioreq.c
index 39de659ddf..d991ac9cdc 100644
--- a/xen/arch/x86/hvm/ioreq.c
+++ b/xen/arch/x86/hvm/ioreq.c
@@ -259,6 +259,19 @@ static int hvm_map_ioreq_gfn(struct hvm_ioreq_server *s, 
bool buf)
 struct hvm_ioreq_page *iorp = buf ? >bufioreq : >ioreq;
 int rc;
 
+if ( iorp->page )
+{
+/*
+ * If a page has already been allocated (which will happen on
+ * demand if hvm_get_ioreq_server_frame() is called), then
+ * mapping a guest frame is not permitted.
+ */
+if ( gfn_eq(iorp->gfn, INVALID_GFN) )
+return -EPERM;
+
+return 0;
+}
+
 if ( d->is_dying )
 return -EINVAL;
 
@@ -281,6 +294,70 @@ static int hvm_map_ioreq_gfn(struct hvm_ioreq_server *s, 
bool buf)
 return rc;
 }
 
+static int hvm_alloc_ioreq_mfn(struct hvm_ioreq_server *s, bool buf)
+{
+struct domain *currd = current->domain;
+struct hvm_ioreq_page *iorp = buf ? >bufioreq : >ioreq;
+
+if ( iorp->page )
+{
+/*
+ * If a guest frame has already been mapped (which may happen
+ * on demand if hvm_get_ioreq_server_info() is called), then
+ * allocating a page is not permitted.
+ */
+if ( !gfn_eq(iorp->gfn, INVALID_GFN) )
+return -EPERM;
+
+return 0;
+}
+
+/*
+ * Allocated IOREQ server pages are assigned to the emulating
+ * domain, not the target domain. This is because the emulator is
+ * likely to be destroyed after the target domain has been torn
+ * down, and we must use MEMF_no_refcount otherwise page allocation
+ * could fail if the emulating domain has already reached its
+ * maximum allocation.
+ */
+iorp->page = alloc_domheap_page(currd, MEMF_no_refcount);
+if ( !iorp->page )
+return -ENOMEM;
+
+if ( !get_page_type(iorp->page, PGT_writable_page) )
+{
+ASSERT_UNREACHABLE();
+put_page(iorp->page);
+iorp->page = NULL;
+return -ENOMEM;
+}
+
+iorp->va = __map_domain_page_global(iorp->page);
+if ( !iorp->va )
+{
+put_page_and_type(iorp->page);
+iorp->page = NULL;
+return -ENOMEM;
+}
+
+clear_page(iorp->va);
+return 0;
+}
+
+static void hvm_free_ioreq_mfn(struct hvm_ioreq_server *s, bool buf)
+{
+struct hvm_ioreq_page *iorp = buf ? >bufioreq : >ioreq;
+
+if ( !iorp->page )
+

[Xen-devel] [PATCH v13 01/11] x86/hvm/ioreq: maintain an array of ioreq servers rather than a list

[Paul Durrant]

A subsequent patch will remove the current implicit limitation on creation
of ioreq servers which is due to the allocation of gfns for the ioreq
structures and buffered ioreq ring.

It will therefore be necessary to introduce an explicit limit and, since
this limit should be small, it simplifies the code to maintain an array of
that size rather than using a list.

Also, by reserving an array slot for the default server and populating
array slots early in create, the need to pass an 'is_default' boolean
to sub-functions can be avoided.

Some function return values are changed by this patch: Specifically, in
the case where the id of the default ioreq server is passed in, -EOPNOTSUPP
is now returned rather than -ENOENT.

Signed-off-by: Paul Durrant 
Reviewed-by: Roger Pau Monné 
Reviewed-by: Jan Beulich 
---
Cc: Andrew Cooper 

v10:
 - modified FOR_EACH... macro as suggested by Jan.
 - check for NULL in IS_DEFAULT macro as suggested by Jan.

v9:
 - modified FOR_EACH... macro as requested by Andrew.

v8:
 - Addressed various comments from Jan.

v7:
 - Fixed assertion failure found in testing.

v6:
 - Updated according to comments made by Roger on v4 that I'd missed.

v5:
 - Switched GET/SET_IOREQ_SERVER() macros to get/set_ioreq_server()
   functions to avoid possible double-evaluation issues.

v4:
 - Introduced more helper macros and relocated them to the top of the
   code.

v3:
 - New patch (replacing "move is_default into struct hvm_ioreq_server") in
   response to review comments.
---
 xen/arch/x86/hvm/ioreq.c | 502 +++
 xen/include/asm-x86/hvm/domain.h |  10 +-
 2 files changed, 245 insertions(+), 267 deletions(-)

diff --git a/xen/arch/x86/hvm/ioreq.c b/xen/arch/x86/hvm/ioreq.c
index d5afe20cc8..da31918bb1 100644
--- a/xen/arch/x86/hvm/ioreq.c
+++ b/xen/arch/x86/hvm/ioreq.c
@@ -33,6 +33,37 @@
 
 #include 
 
+static void set_ioreq_server(struct domain *d, unsigned int id,
+ struct hvm_ioreq_server *s)
+{
+ASSERT(id < MAX_NR_IOREQ_SERVERS);
+ASSERT(!s || !d->arch.hvm_domain.ioreq_server.server[id]);
+
+d->arch.hvm_domain.ioreq_server.server[id] = s;
+}
+
+#define GET_IOREQ_SERVER(d, id) \
+(d)->arch.hvm_domain.ioreq_server.server[id]
+
+static struct hvm_ioreq_server *get_ioreq_server(const struct domain *d,
+ unsigned int id)
+{
+if ( id >= MAX_NR_IOREQ_SERVERS )
+return NULL;
+
+return GET_IOREQ_SERVER(d, id);
+}
+
+#define IS_DEFAULT(s) \
+((s) && (s) == GET_IOREQ_SERVER((s)->domain, DEFAULT_IOSERVID))
+
+/* Iterate over all possible ioreq servers */
+#define FOR_EACH_IOREQ_SERVER(d, id, s) \
+for ( (id) = 0; (id) < MAX_NR_IOREQ_SERVERS; (id)++ ) \
+if ( !(s = GET_IOREQ_SERVER(d, id)) ) \
+continue; \
+else
+
 static ioreq_t *get_ioreq(struct hvm_ioreq_server *s, struct vcpu *v)
 {
 shared_iopage_t *p = s->ioreq.va;
@@ -47,10 +78,9 @@ bool hvm_io_pending(struct vcpu *v)
 {
 struct domain *d = v->domain;
 struct hvm_ioreq_server *s;
+unsigned int id;
 
-list_for_each_entry ( s,
-  >arch.hvm_domain.ioreq_server.list,
-  list_entry )
+FOR_EACH_IOREQ_SERVER(d, id, s)
 {
 struct hvm_ioreq_vcpu *sv;
 
@@ -127,10 +157,9 @@ bool handle_hvm_io_completion(struct vcpu *v)
 struct hvm_vcpu_io *vio = >arch.hvm_vcpu.hvm_io;
 struct hvm_ioreq_server *s;
 enum hvm_io_completion io_completion;
+unsigned int id;
 
-  list_for_each_entry ( s,
-  >arch.hvm_domain.ioreq_server.list,
-  list_entry )
+FOR_EACH_IOREQ_SERVER(d, id, s)
 {
 struct hvm_ioreq_vcpu *sv;
 
@@ -243,13 +272,12 @@ static int hvm_map_ioreq_page(
 bool is_ioreq_server_page(struct domain *d, const struct page_info *page)
 {
 const struct hvm_ioreq_server *s;
+unsigned int id;
 bool found = false;
 
 spin_lock_recursive(>arch.hvm_domain.ioreq_server.lock);
 
-list_for_each_entry ( s,
-  >arch.hvm_domain.ioreq_server.list,
-  list_entry )
+FOR_EACH_IOREQ_SERVER(d, id, s)
 {
 if ( (s->ioreq.va && s->ioreq.page == page) ||
  (s->bufioreq.va && s->bufioreq.page == page) )
@@ -302,7 +330,7 @@ static void hvm_update_ioreq_evtchn(struct hvm_ioreq_server 
*s,
 }
 
 static int hvm_ioreq_server_add_vcpu(struct hvm_ioreq_server *s,
- bool is_default, struct vcpu *v)
+ struct vcpu *v)
 {
 struct hvm_ioreq_vcpu *sv;
 int rc;
@@ -331,7 +359,7 @@ static int hvm_ioreq_server_add_vcpu(struct 
hvm_ioreq_server *s,
 goto fail3;
 
 s->bufioreq_evtchn = rc;
-if ( is_default )
+if ( IS_DEFAULT(s) )
 

[Xen-devel] [PATCH v13 00/11] x86: guest resource mapping

[Paul Durrant]

This series introduces support for direct mapping of guest resources.
The resources are:
 - IOREQ server pages
 - Grant tables

v13:
 - Responded to more comments from Jan and Julien.
 - Build-tested using ARM cross-compilation.

v12:
 - Responded to more comments from Jan.

v11:
 - Responded to more comments from Jan.

v10:
 - Responded to comments from Jan.

v9:
 - Change to patch #1 only.

v8:
 - Re-ordered series and dropped two patches that have already been
committed.

v7:
 - Fixed assertion failure hit during domain destroy.

v6:
 - Responded to missed comments from Roger.

v5:
 - Responded to review comments from Wei.

v4:
 - Responded to further review comments from Roger.

v3:
 - Dropped original patch #1 since it is covered by Juergen's patch.
 - Added new xenforeignmemorycleanup patch (#4).
 - Replaced the patch introducing the ioreq server 'is_default' flag with
   one that changes the ioreq server list into an array (#8).
  
Paul Durrant (11):
  x86/hvm/ioreq: maintain an array of ioreq servers rather than a list
  x86/hvm/ioreq: simplify code and use consistent naming
  x86/hvm/ioreq: use gfn_t in struct hvm_ioreq_page
  x86/hvm/ioreq: defer mapping gfns until they are actually requsted
  x86/mm: add HYPERVISOR_memory_op to acquire guest resources
  x86/hvm/ioreq: add a new mappable resource type...
  x86/mm: add an extra command to HYPERVISOR_mmu_update...
  tools/libxenforeignmemory: add support for resource mapping
  tools/libxenforeignmemory: reduce xenforeignmemory_restrict code
footprint
  common: add a new mappable resource type: XENMEM_resource_grant_table
  tools/libxenctrl: use new xenforeignmemory API to seed grant table

 tools/flask/policy/modules/xen.if  |   4 +-
 tools/include/xen-sys/Linux/privcmd.h  |  11 +
 tools/libs/devicemodel/core.c  |   8 +
 tools/libs/devicemodel/include/xendevicemodel.h|   6 +-
 tools/libs/foreignmemory/Makefile  |   2 +-
 tools/libs/foreignmemory/core.c|  53 ++
 tools/libs/foreignmemory/freebsd.c |   7 -
 .../libs/foreignmemory/include/xenforeignmemory.h  |  41 +
 tools/libs/foreignmemory/libxenforeignmemory.map   |   5 +
 tools/libs/foreignmemory/linux.c   |  45 ++
 tools/libs/foreignmemory/minios.c  |   7 -
 tools/libs/foreignmemory/netbsd.c  |   7 -
 tools/libs/foreignmemory/private.h |  43 +-
 tools/libs/foreignmemory/solaris.c |   7 -
 tools/libxc/include/xc_dom.h   |   8 +-
 tools/libxc/xc_dom_boot.c  | 114 ++-
 tools/libxc/xc_sr_restore_x86_hvm.c|  10 +-
 tools/libxc/xc_sr_restore_x86_pv.c |   2 +-
 tools/libxl/libxl_dom.c|   1 -
 tools/python/xen/lowlevel/xc/xc.c  |   6 +-
 xen/arch/x86/hvm/dm.c  |   9 +-
 xen/arch/x86/hvm/ioreq.c   | 831 -
 xen/arch/x86/mm.c  |  62 +-
 xen/arch/x86/mm/p2m.c  |   3 +-
 xen/common/compat/memory.c |  95 +++
 xen/common/grant_table.c   |  63 +-
 xen/common/memory.c| 137 
 xen/include/asm-arm/mm.h   |   7 +
 xen/include/asm-arm/p2m.h  |  10 +
 xen/include/asm-x86/hvm/domain.h   |  14 +-
 xen/include/asm-x86/hvm/ioreq.h|   2 +
 xen/include/asm-x86/mm.h   |   5 +
 xen/include/asm-x86/p2m.h  |   3 +
 xen/include/public/hvm/dm_op.h |  36 +-
 xen/include/public/memory.h|  58 +-
 xen/include/public/xen.h   |  12 +-
 xen/include/xen/grant_table.h  |   4 +
 xen/include/xlat.lst   |   1 +
 xen/include/xsm/dummy.h|   6 +
 xen/include/xsm/xsm.h  |   6 +
 xen/xsm/dummy.c|   1 +
 xen/xsm/flask/hooks.c  |   6 +
 xen/xsm/flask/policy/access_vectors|   2 +
 43 files changed, 1265 insertions(+), 495 deletions(-)

---
Cc: Daniel De Graaf 
Cc: Ian Jackson 
Cc: Wei Liu 
Cc: Andrew Cooper 
Cc: George Dunlap 
Cc: Jan Beulich 
Cc: Konrad Rzeszutek Wilk 
Cc: Stefano Stabellini 
Cc: Tim Deegan 
Cc: "Marek Marczykowski-Górecki" 
Cc: Paul Durrant 
Cc: George Dunlap 
Cc: Julien Grall 

-- 
2.11.0

[Xen-devel] [PATCH v13 07/11] x86/mm: add an extra command to HYPERVISOR_mmu_update...

[Paul Durrant]

...to allow the calling domain to prevent translation of specified l1e
value.

Despite what the comment in public/xen.h might imply, specifying a
command value of MMU_NORMAL_PT_UPDATE will not simply update an l1e with
the specified value. Instead, mod_l1_entry() tests whether foreign_dom
has PG_translate set in its paging mode and, if it does, assumes that the
the pfn value in the l1e is a gfn rather than an mfn.

To allow PV tools domain to map mfn values from a previously issued
HYPERVISOR_memory_op:XENMEM_acquire_resource, there needs to be a way
to tell HYPERVISOR_mmu_update that the specific l1e value does not
require translation regardless of the paging mode of foreign_dom. This
patch therefore defines a new command value, MMU_PT_UPDATE_NO_TRANSLATE,
which has the same semantics as MMU_NORMAL_PT_UPDATE except that the
paging mode of foreign_dom is ignored and the l1e value is used verbatim.

Signed-off-by: Paul Durrant 
Reviewed-by: Jan Beulich 
---
Cc: Andrew Cooper 
Cc: George Dunlap 
Cc: Ian Jackson 
Cc: Konrad Rzeszutek Wilk 
Cc: Stefano Stabellini 
Cc: Tim Deegan 
Cc: Wei Liu 

v13:
 - Re-base.

v8:
 - New in this version, replacing "allow a privileged PV domain to map
   guest mfns".
---
 xen/arch/x86/mm.c| 13 -
 xen/include/public/xen.h | 12 +---
 2 files changed, 17 insertions(+), 8 deletions(-)

diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index a8c207b973..503bca552c 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -1724,9 +1724,10 @@ void page_unlock(struct page_info *page)
 
 /* Update the L1 entry at pl1e to new value nl1e. */
 static int mod_l1_entry(l1_pgentry_t *pl1e, l1_pgentry_t nl1e,
-unsigned long gl1mfn, int preserve_ad,
+unsigned long gl1mfn, unsigned int cmd,
 struct vcpu *pt_vcpu, struct domain *pg_dom)
 {
+bool preserve_ad = (cmd == MMU_PT_UPDATE_PRESERVE_AD);
 l1_pgentry_t ol1e;
 struct domain *pt_dom = pt_vcpu->domain;
 int rc = 0;
@@ -1748,7 +1749,8 @@ static int mod_l1_entry(l1_pgentry_t *pl1e, l1_pgentry_t 
nl1e,
 }
 
 /* Translate foreign guest address. */
-if ( paging_mode_translate(pg_dom) )
+if ( cmd != MMU_PT_UPDATE_NO_TRANSLATE &&
+ paging_mode_translate(pg_dom) )
 {
 p2m_type_t p2mt;
 p2m_query_t q = l1e_get_flags(nl1e) & _PAGE_RW ?
@@ -3438,6 +3440,7 @@ long do_mmu_update(
  */
 case MMU_NORMAL_PT_UPDATE:
 case MMU_PT_UPDATE_PRESERVE_AD:
+case MMU_PT_UPDATE_NO_TRANSLATE:
 {
 p2m_type_t p2mt;
 
@@ -3497,8 +3500,7 @@ long do_mmu_update(
 {
 case PGT_l1_page_table:
 rc = mod_l1_entry(va, l1e_from_intpte(req.val), mfn,
-  cmd == MMU_PT_UPDATE_PRESERVE_AD, v,
-  pg_owner);
+  cmd, v, pg_owner);
 break;
 case PGT_l2_page_table:
 rc = mod_l2_entry(va, l2e_from_intpte(req.val), mfn,
@@ -3773,7 +3775,8 @@ static int __do_update_va_mapping(
 goto out;
 }
 
-rc = mod_l1_entry(pl1e, val, mfn_x(gl1mfn), 0, v, pg_owner);
+rc = mod_l1_entry(pl1e, val, mfn_x(gl1mfn), MMU_NORMAL_PT_UPDATE, v,
+  pg_owner);
 
 page_unlock(gl1pg);
 put_page(gl1pg);
diff --git a/xen/include/public/xen.h b/xen/include/public/xen.h
index 308109f176..fb1df8f293 100644
--- a/xen/include/public/xen.h
+++ b/xen/include/public/xen.h
@@ -268,6 +268,10 @@ DEFINE_XEN_GUEST_HANDLE(xen_ulong_t);
  * As MMU_NORMAL_PT_UPDATE above, but A/D bits currently in the PTE are ORed
  * with those in @val.
  *
+ * ptr[1:0] == MMU_PT_UPDATE_NO_TRANSLATE:
+ * As MMU_NORMAL_PT_UPDATE above, but @val is not translated though FD
+ * page tables.
+ *
  * @val is usually the machine frame number along with some attributes.
  * The attributes by default follow the architecture defined bits. Meaning that
  * if this is a X86_64 machine and four page table layout is used, the layout
@@ -334,9 +338,11 @@ DEFINE_XEN_GUEST_HANDLE(xen_ulong_t);
  *
  * PAT (bit 7 on) --> PWT (bit 3 on) and clear bit 7.
  */
-#define MMU_NORMAL_PT_UPDATE  0 /* checked '*ptr = val'. ptr is MA.  */
-#define MMU_MACHPHYS_UPDATE   1 /* ptr = MA of frame to modify entry for */
-#define MMU_PT_UPDATE_PRESERVE_AD 2 /* atomically: *ptr = val | (*ptr&(A|D)) */
+#define MMU_NORMAL_PT_UPDATE   0 /* checked '*ptr = val'. ptr is MA.  
*/
+#define MMU_MACHPHYS_UPDATE1 /* ptr = MA of frame to modify entry for 
*/
+#define MMU_PT_UPDATE_PRESERVE_AD  2 /* atomically: *ptr = val | (*ptr&(A|D)) 
*/
+#define 

[Xen-devel] [PATCH v13 03/11] x86/hvm/ioreq: use gfn_t in struct hvm_ioreq_page

[Paul Durrant]

This patch adjusts the ioreq server code to use type-safe gfn_t values
where possible. No functional change.

Signed-off-by: Paul Durrant 
Reviewed-by: Roger Pau Monné 
Reviewed-by: Wei Liu 
Acked-by: Jan Beulich 
---
Cc: Andrew Cooper 
---
 xen/arch/x86/hvm/ioreq.c | 44 
 xen/include/asm-x86/hvm/domain.h |  2 +-
 2 files changed, 23 insertions(+), 23 deletions(-)

diff --git a/xen/arch/x86/hvm/ioreq.c b/xen/arch/x86/hvm/ioreq.c
index c21fa9f280..eec4e4771e 100644
--- a/xen/arch/x86/hvm/ioreq.c
+++ b/xen/arch/x86/hvm/ioreq.c
@@ -210,7 +210,7 @@ bool handle_hvm_io_completion(struct vcpu *v)
 return true;
 }
 
-static unsigned long hvm_alloc_ioreq_gfn(struct hvm_ioreq_server *s)
+static gfn_t hvm_alloc_ioreq_gfn(struct hvm_ioreq_server *s)
 {
 struct domain *d = s->domain;
 unsigned int i;
@@ -220,20 +220,19 @@ static unsigned long hvm_alloc_ioreq_gfn(struct 
hvm_ioreq_server *s)
 for ( i = 0; i < sizeof(d->arch.hvm_domain.ioreq_gfn.mask) * 8; i++ )
 {
 if ( test_and_clear_bit(i, >arch.hvm_domain.ioreq_gfn.mask) )
-return d->arch.hvm_domain.ioreq_gfn.base + i;
+return _gfn(d->arch.hvm_domain.ioreq_gfn.base + i);
 }
 
-return gfn_x(INVALID_GFN);
+return INVALID_GFN;
 }
 
-static void hvm_free_ioreq_gfn(struct hvm_ioreq_server *s,
-   unsigned long gfn)
+static void hvm_free_ioreq_gfn(struct hvm_ioreq_server *s, gfn_t gfn)
 {
 struct domain *d = s->domain;
-unsigned int i = gfn - d->arch.hvm_domain.ioreq_gfn.base;
+unsigned int i = gfn_x(gfn) - d->arch.hvm_domain.ioreq_gfn.base;
 
 ASSERT(!IS_DEFAULT(s));
-ASSERT(gfn != gfn_x(INVALID_GFN));
+ASSERT(!gfn_eq(gfn, INVALID_GFN));
 
 set_bit(i, >arch.hvm_domain.ioreq_gfn.mask);
 }
@@ -242,7 +241,7 @@ static void hvm_unmap_ioreq_gfn(struct hvm_ioreq_server *s, 
bool buf)
 {
 struct hvm_ioreq_page *iorp = buf ? >bufioreq : >ioreq;
 
-if ( iorp->gfn == gfn_x(INVALID_GFN) )
+if ( gfn_eq(iorp->gfn, INVALID_GFN) )
 return;
 
 destroy_ring_for_helper(>va, iorp->page);
@@ -251,7 +250,7 @@ static void hvm_unmap_ioreq_gfn(struct hvm_ioreq_server *s, 
bool buf)
 if ( !IS_DEFAULT(s) )
 hvm_free_ioreq_gfn(s, iorp->gfn);
 
-iorp->gfn = gfn_x(INVALID_GFN);
+iorp->gfn = INVALID_GFN;
 }
 
 static int hvm_map_ioreq_gfn(struct hvm_ioreq_server *s, bool buf)
@@ -264,16 +263,17 @@ static int hvm_map_ioreq_gfn(struct hvm_ioreq_server *s, 
bool buf)
 return -EINVAL;
 
 if ( IS_DEFAULT(s) )
-iorp->gfn = buf ?
-d->arch.hvm_domain.params[HVM_PARAM_BUFIOREQ_PFN] :
-d->arch.hvm_domain.params[HVM_PARAM_IOREQ_PFN];
+iorp->gfn = _gfn(buf ?
+ d->arch.hvm_domain.params[HVM_PARAM_BUFIOREQ_PFN] :
+ d->arch.hvm_domain.params[HVM_PARAM_IOREQ_PFN]);
 else
 iorp->gfn = hvm_alloc_ioreq_gfn(s);
 
-if ( iorp->gfn == gfn_x(INVALID_GFN) )
+if ( gfn_eq(iorp->gfn, INVALID_GFN) )
 return -ENOMEM;
 
-rc = prepare_ring_for_helper(d, iorp->gfn, >page, >va);
+rc = prepare_ring_for_helper(d, gfn_x(iorp->gfn), >page,
+ >va);
 
 if ( rc )
 hvm_unmap_ioreq_gfn(s, buf);
@@ -309,10 +309,10 @@ static void hvm_remove_ioreq_gfn(struct hvm_ioreq_server 
*s, bool buf)
 struct domain *d = s->domain;
 struct hvm_ioreq_page *iorp = buf ? >bufioreq : >ioreq;
 
-if ( IS_DEFAULT(s) || iorp->gfn == gfn_x(INVALID_GFN) )
+if ( IS_DEFAULT(s) || gfn_eq(iorp->gfn, INVALID_GFN) )
 return;
 
-if ( guest_physmap_remove_page(d, _gfn(iorp->gfn),
+if ( guest_physmap_remove_page(d, iorp->gfn,
_mfn(page_to_mfn(iorp->page)), 0) )
 domain_crash(d);
 clear_page(iorp->va);
@@ -324,12 +324,12 @@ static int hvm_add_ioreq_gfn(struct hvm_ioreq_server *s, 
bool buf)
 struct hvm_ioreq_page *iorp = buf ? >bufioreq : >ioreq;
 int rc;
 
-if ( IS_DEFAULT(s) || iorp->gfn == gfn_x(INVALID_GFN) )
+if ( IS_DEFAULT(s) || gfn_eq(iorp->gfn, INVALID_GFN) )
 return 0;
 
 clear_page(iorp->va);
 
-rc = guest_physmap_add_page(d, _gfn(iorp->gfn),
+rc = guest_physmap_add_page(d, iorp->gfn,
 _mfn(page_to_mfn(iorp->page)), 0);
 if ( rc == 0 )
 paging_mark_dirty(d, _mfn(page_to_mfn(iorp->page)));
@@ -590,8 +590,8 @@ static int hvm_ioreq_server_init(struct hvm_ioreq_server *s,
 INIT_LIST_HEAD(>ioreq_vcpu_list);
 spin_lock_init(>bufioreq_lock);
 
-s->ioreq.gfn = gfn_x(INVALID_GFN);
-s->bufioreq.gfn = gfn_x(INVALID_GFN);
+s->ioreq.gfn = INVALID_GFN;
+s->bufioreq.gfn = INVALID_GFN;
 
 rc = hvm_ioreq_server_alloc_rangesets(s, id);
 if ( rc )
@@ -757,11 +757,11 @@ int 

Re: [Xen-devel] [PATCH v1] x86/mm: Supresses vm_events caused by page-walks

[Tamas K Lengyel]

On Mon, Oct 30, 2017 at 11:19 AM, Razvan Cojocaru
 wrote:
> On 10/30/2017 07:07 PM, Tamas K Lengyel wrote:
>> On Mon, Oct 30, 2017 at 11:01 AM, Razvan Cojocaru
>>  wrote:
>>> On 10/30/2017 06:39 PM, Tamas K Lengyel wrote:
 On Mon, Oct 30, 2017 at 10:24 AM, Razvan Cojocaru
  wrote:
> On 30.10.2017 18:01, Tamas K Lengyel wrote:
>> On Mon, Oct 30, 2017 at 4:32 AM, Alexandru Isaila
>>  wrote:
>>> This patch is adding a way to enable/disable nested pagefault
>>> events. It introduces the xc_monitor_nested_pagefault function
>>> and adds the nested_pagefault_disabled in the monitor structure.
>>> This is needed by the introspection so it will only get gla
>>> faults and not get spammed with other faults.
>>> In p2m_set_ad_bits the v->arch.sse_pg_dirty.eip and
>>> v->arch.sse_pg_dirty.gla are used to mark that this is the
>>> second time a fault occurs and the dirty bit is set.
>>
>> Could you describe under what conditions do you get these other faults?
>
> Hey Tamas, the whole story is at page 8 of this document:
>
> https://www.researchgate.net/publication/281835515_Proposed_Processor_Extensions_for_Significant_Speedup_of_Hypervisor_Memory_Introspection

 Hi Razvan,
 thanks but I'm not sure that doc addresses my question. You
 effectively filter out npfec_kind_in_gpt and npfec_kind_unknown in
 this patch. The first, npfec_kind_in_gpt should only happen if you
 have restricted access to the gpt with ept and the processor couldn't
 walk the table. But if you don't want to get events of these types
 then why not simply not restrict access the gpt to begin with? And as
 for npfec_kind_unknown, I don't think that gets generated under any
 situation. So hence my question, what is your setup that makes this
 patch necessary?
>>>
>>> On the npfec_kind_unknown case, indeed, we were wondering when that
>>> might possibly occur when discussing this patch - it's probably reserved
>>> for the future?
>>>
>>> On why our introspection engine decides to restrict access to those
>>> specific pages, I am not intimate with its inner workings, and not sure
>>> how much could be disclosed here in any case. Is it not a worthwhile
>>> (and otherwise harmless) tool to be able to switch A/D bits-triggered
>>> EPT faults anyway, for introspection purposes?
>>
>> It changes the default behavior of mem_access events so I just wanted
>> to get some background on when that is really required. Technically
>> there is no reason why we couldn't do that filtering in Xen. I think
>> it might be better to flip the filter the other way though so the
>> default behavior remains as is (ie. change the option to enable
>> filtering instead of enabling monitoring).
>
> Wait, it shouldn't change the default behaviour at all. If nobody calls
> that function, all the EPT event kinds should be sent out - the new
> monitor flag is a "disable" flag for non-GLA event (the so-called
> "nested page fault" events).

Oh yea you are right, I completely overlooked that it is named
"nested_pagefault_disabled" =) Maybe a comment in the domctl header
would be warranted to note that this is enabled by default when
mem_access is used.

Tamas

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 1/5] libxl: add PV sound device

[Wei Liu]

On Mon, Oct 02, 2017 at 12:49:20PM +0300, Oleksandr Grytsov wrote:
> From: Oleksandr Grytsov 
> 
> Add PV sound device described in sndif.h
> 
> Signed-off-by: Oleksandr Grytsov 

[...]
>  
>  libxl__console_backend = Enumeration("console_backend", [
> diff --git a/tools/libxl/libxl_vsnd.c b/tools/libxl/libxl_vsnd.c
> new file mode 100644
> index 000..26885f9
> --- /dev/null
> +++ b/tools/libxl/libxl_vsnd.c
> @@ -0,0 +1,307 @@
> +/*
> + * Copyright (C) 2016 EPAM Systems Inc.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU Lesser General Public License as published
> + * by the Free Software Foundation; version 2.1 only. with the special
> + * exception on linking described in file LICENSE.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU Lesser General Public License for more details.
> + */
> +
> +#include "libxl_internal.h"
> +#include "xen/io/sndif.h"
> +

Use  -- this is not a local header.

> +
> +static unsigned int libxl__rates_to_str_vsnd(char *str, uint32_t 
> *sample_rates,
> + int num_sample_rates)
> +{
> +unsigned int len;
> +int i;
> +
> +len = 0;
> +
> +if (num_sample_rates == 0) {
> +return len;
> +}

Coding style.

> +
> +for (i = 0; i < num_sample_rates - 1; i++) {
> +if (str) {
> +len += sprintf([len], "%u,", sample_rates[i]);

libxl__sprintf(NOGC, ...)

> +} else {
> +len += snprintf(NULL, 0, "%u,", sample_rates[i]);
> +}
> +}
> +
> +if (str) {
> +len += sprintf([len], "%u", sample_rates[i]);
> +} else {
> +len += snprintf(NULL, 0, "%u", sample_rates[i]);
> +}
> +
> +return len;
> +}
> +
[...]
> +
> +static int libxl__set_params_vsnd(libxl__gc *gc, char *path,
> +  libxl_vsnd_params *params, flexarray_t 
> *front)
> +{
> +char *buffer;
> +int len;
> +int rc;
> +
> +if (params->sample_rates) {
> +// calculate required string size;

Coding style.

> +len = libxl__rates_to_str_vsnd(NULL, params->sample_rates,
> +   params->num_sample_rates);
> +
> +if (len) {
> +buffer = libxl__malloc(gc, len + 1);
> +
> +libxl__rates_to_str_vsnd(buffer, params->sample_rates,
> + params->num_sample_rates);
> +rc = flexarray_append_pair(front,
> +   
> GCSPRINTF("%s"XENSND_FIELD_SAMPLE_RATES,
> + path), buffer);
> +if (rc) return rc;

goto out please.

Please fix these coding style issues throughout this series.j

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v9.1 02/16] Rename PSR sysctl/domctl interfaces and xsm policy to make them be general

[Daniel De Graaf]

On 10/24/2017 05:10 AM, Yi Sun wrote:

This patch renames PSR sysctl/domctl interfaces and related xsm policy to
make them be general for all resource allocation features but not only
for CAT. Then, we can resuse the interfaces for all allocation features.

Basically, it changes 'psr_cat_op' to 'psr_alloc', and remove 'CAT_' from some
macros. E.g.:
1. psr_cat_op -> psr_alloc
2. XEN_DOMCTL_psr_cat_op -> XEN_DOMCTL_psr_alloc
3. XEN_SYSCTL_psr_cat_op -> XEN_SYSCTL_psr_alloc
4. XEN_DOMCTL_PSR_CAT_SET_L3_CBM -> XEN_DOMCTL_PSR_SET_L3_CBM
5. XEN_SYSCTL_PSR_CAT_get_l3_info -> XEN_SYSCTL_PSR_get_l3_info


Acked-by: Daniel De Graaf 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v1] x86/mm: Supresses vm_events caused by page-walks

[Razvan Cojocaru]

On 10/30/2017 07:07 PM, Tamas K Lengyel wrote:
> On Mon, Oct 30, 2017 at 11:01 AM, Razvan Cojocaru
>  wrote:
>> On 10/30/2017 06:39 PM, Tamas K Lengyel wrote:
>>> On Mon, Oct 30, 2017 at 10:24 AM, Razvan Cojocaru
>>>  wrote:
 On 30.10.2017 18:01, Tamas K Lengyel wrote:
> On Mon, Oct 30, 2017 at 4:32 AM, Alexandru Isaila
>  wrote:
>> This patch is adding a way to enable/disable nested pagefault
>> events. It introduces the xc_monitor_nested_pagefault function
>> and adds the nested_pagefault_disabled in the monitor structure.
>> This is needed by the introspection so it will only get gla
>> faults and not get spammed with other faults.
>> In p2m_set_ad_bits the v->arch.sse_pg_dirty.eip and
>> v->arch.sse_pg_dirty.gla are used to mark that this is the
>> second time a fault occurs and the dirty bit is set.
>
> Could you describe under what conditions do you get these other faults?

 Hey Tamas, the whole story is at page 8 of this document:

 https://www.researchgate.net/publication/281835515_Proposed_Processor_Extensions_for_Significant_Speedup_of_Hypervisor_Memory_Introspection
>>>
>>> Hi Razvan,
>>> thanks but I'm not sure that doc addresses my question. You
>>> effectively filter out npfec_kind_in_gpt and npfec_kind_unknown in
>>> this patch. The first, npfec_kind_in_gpt should only happen if you
>>> have restricted access to the gpt with ept and the processor couldn't
>>> walk the table. But if you don't want to get events of these types
>>> then why not simply not restrict access the gpt to begin with? And as
>>> for npfec_kind_unknown, I don't think that gets generated under any
>>> situation. So hence my question, what is your setup that makes this
>>> patch necessary?
>>
>> On the npfec_kind_unknown case, indeed, we were wondering when that
>> might possibly occur when discussing this patch - it's probably reserved
>> for the future?
>>
>> On why our introspection engine decides to restrict access to those
>> specific pages, I am not intimate with its inner workings, and not sure
>> how much could be disclosed here in any case. Is it not a worthwhile
>> (and otherwise harmless) tool to be able to switch A/D bits-triggered
>> EPT faults anyway, for introspection purposes?
> 
> It changes the default behavior of mem_access events so I just wanted
> to get some background on when that is really required. Technically
> there is no reason why we couldn't do that filtering in Xen. I think
> it might be better to flip the filter the other way though so the
> default behavior remains as is (ie. change the option to enable
> filtering instead of enabling monitoring).

Wait, it shouldn't change the default behaviour at all. If nobody calls
that function, all the EPT event kinds should be sent out - the new
monitor flag is a "disable" flag for non-GLA event (the so-called
"nested page fault" events).


Thanks,
Razvan

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v1] x86/mm: Supresses vm_events caused by page-walks

[Tamas K Lengyel]

On Mon, Oct 30, 2017 at 11:01 AM, Razvan Cojocaru
 wrote:
> On 10/30/2017 06:39 PM, Tamas K Lengyel wrote:
>> On Mon, Oct 30, 2017 at 10:24 AM, Razvan Cojocaru
>>  wrote:
>>> On 30.10.2017 18:01, Tamas K Lengyel wrote:
 On Mon, Oct 30, 2017 at 4:32 AM, Alexandru Isaila
  wrote:
> This patch is adding a way to enable/disable nested pagefault
> events. It introduces the xc_monitor_nested_pagefault function
> and adds the nested_pagefault_disabled in the monitor structure.
> This is needed by the introspection so it will only get gla
> faults and not get spammed with other faults.
> In p2m_set_ad_bits the v->arch.sse_pg_dirty.eip and
> v->arch.sse_pg_dirty.gla are used to mark that this is the
> second time a fault occurs and the dirty bit is set.

 Could you describe under what conditions do you get these other faults?
>>>
>>> Hey Tamas, the whole story is at page 8 of this document:
>>>
>>> https://www.researchgate.net/publication/281835515_Proposed_Processor_Extensions_for_Significant_Speedup_of_Hypervisor_Memory_Introspection
>>
>> Hi Razvan,
>> thanks but I'm not sure that doc addresses my question. You
>> effectively filter out npfec_kind_in_gpt and npfec_kind_unknown in
>> this patch. The first, npfec_kind_in_gpt should only happen if you
>> have restricted access to the gpt with ept and the processor couldn't
>> walk the table. But if you don't want to get events of these types
>> then why not simply not restrict access the gpt to begin with? And as
>> for npfec_kind_unknown, I don't think that gets generated under any
>> situation. So hence my question, what is your setup that makes this
>> patch necessary?
>
> On the npfec_kind_unknown case, indeed, we were wondering when that
> might possibly occur when discussing this patch - it's probably reserved
> for the future?
>
> On why our introspection engine decides to restrict access to those
> specific pages, I am not intimate with its inner workings, and not sure
> how much could be disclosed here in any case. Is it not a worthwhile
> (and otherwise harmless) tool to be able to switch A/D bits-triggered
> EPT faults anyway, for introspection purposes?

It changes the default behavior of mem_access events so I just wanted
to get some background on when that is really required. Technically
there is no reason why we couldn't do that filtering in Xen. I think
it might be better to flip the filter the other way though so the
default behavior remains as is (ie. change the option to enable
filtering instead of enabling monitoring).

Tamas

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH RFC v2 0/8] Live migration for VMs with QEMU backed local storage

[Wei Liu]

Also CC QEMU maintainers.

Is there any new QEMU patch required?

On Wed, Oct 18, 2017 at 10:26:27AM -0700, Bruno Alvisio wrote:
> I am reviving this thread about the migration of VMs with local storage. I 
> have worked on a solution to be able to migrate VMs that use QEMU as the 
> backend disk driver. I have adapted the migration flow and piggybacked on the 
> “drive-mirroring” capability already provided by QEMU.
> 
> Overview
> 1. The “xl migrate” command has an additional “-q” flag. When provided the 
> local storage of the VM is mirrored to the destination during the migration 
> process.
> 2. Internally, the modification consists on adding a new 
> libxl__stream_read_state struct to the libxl__domain_create_state structure 
> and libxl__stream_read_state structure to the libxl__domain_save_state struct.
> 3. Migration flow can now be divided into three phases:
>a. Phase One: Copies the necessary state to start a QEMU process on the 
> destination. It is started with the “-incoming defer” option.
>b. Phase Two: Disk is mirrored using the QEMU embedded NBD server.
>c. Phase Three: Once the disk is completely mirrored, virtual RAM of the 
> domain is live migrated to the destination. This phase most closely resembles 
> to the current migration flow.
> 4. If the “-q” option is not provided the migration is equivalent to the 
> current migration flow.
> 
> The new migration flow has follows the following major sequence of steps:
> 1. 1st stream copies the QEMU devices RAM from source to destination.
> 2. QEMU process is started on the destination with the option “-incoming 
> defer”. (This creates the QEMU process but it doesn’t start running the main 
> loop until “migrate incoming” command is executed)
> 3. “drive mirror” QMP command is executed so that the disk is mirrored to the 
> destination node.
> 4. An event listener waits for the QMP BLOCK_JOB_READY event sent by QEMU 
> which signals that the "disk mirror job" is complete.
> 5. 2nd Stream copies the virtual RAM from source to destination including 
> QEMU state. At this point, the VM is suspended on source.
> 6. “migrate incoming” QMP command is executed on destination.
> 7. VM is restored in destination.
> 
> This is sample configuration file that I have used to test my branch:
> 
> name="tinycore"
> disk=['/home/balvisio/tinycore.img,raw,xvda,w']
> memory=128
> builder='hvm'
> vcpus=1
> vfb = ['type=vnc']
> vif= ['bridge=xenbr0']
> boot='b'
> acpi=1
> device_model_version='qemu-xen'
> serial='pty'
> vnc=1
> xen_platform_pci=0
> 
> Notes
> 
> 1. Note that the configuration file uses "xen_platform_pci=0”. This is 
> necessary so that the block device is seen by QEMU. Further modification 
> should be made for the case "xen_platform_pci=1” if we still want to use NBD 
> mirroring capability provided by QEMU.
> 2. The current branch has still many hardcoded values. Many of the can be 
> easily removed:
>   a. Port used for disk mirroring (11000)
> b. Name of the block devices. (ide0-hd0) Currently the branch only 
> supports VM with on disk drive.
> c. Live migration memory transfer: The pages transferred by libxc is 
> hardcoded. Only a VM with 128 MB of memory is supported.
> 
> Here is a link to the branch in Github: 
> https://github.com/balvisio/xen/tree/feature/local_storage_migration
> 
> Any feedback/suggestion is appreciated.
> 

I haven't had time to review this series in detail, but I think the
series could use some clean-up and improvement to the commit message.
Please refer to

https://wiki.xenproject.org/wiki/Submitting_Xen_Project_Patches

Currently all patches are missing commit message explaining why the
change is needed as well as your SoB. In case you're not aware, we also
have the requirement each individual patch should build.

There are coding style issues in your patches, please refer to
libxl/CODING_STYLE. The same style applies to xl, too.

Furthermore, patches should be cleaned up before posting -- I see a few
debug printf's got added and removed, as well as a few TODOs with open
questions. Preferably the open questions should be mentioned in the
cover letter to catch attentions.

How this helps. Any more questions, feel free to ask.

Wei.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v1] x86/mm: Supresses vm_events caused by page-walks

[Razvan Cojocaru]

On 10/30/2017 06:39 PM, Tamas K Lengyel wrote:
> On Mon, Oct 30, 2017 at 10:24 AM, Razvan Cojocaru
>  wrote:
>> On 30.10.2017 18:01, Tamas K Lengyel wrote:
>>> On Mon, Oct 30, 2017 at 4:32 AM, Alexandru Isaila
>>>  wrote:
 This patch is adding a way to enable/disable nested pagefault
 events. It introduces the xc_monitor_nested_pagefault function
 and adds the nested_pagefault_disabled in the monitor structure.
 This is needed by the introspection so it will only get gla
 faults and not get spammed with other faults.
 In p2m_set_ad_bits the v->arch.sse_pg_dirty.eip and
 v->arch.sse_pg_dirty.gla are used to mark that this is the
 second time a fault occurs and the dirty bit is set.
>>>
>>> Could you describe under what conditions do you get these other faults?
>>
>> Hey Tamas, the whole story is at page 8 of this document:
>>
>> https://www.researchgate.net/publication/281835515_Proposed_Processor_Extensions_for_Significant_Speedup_of_Hypervisor_Memory_Introspection
> 
> Hi Razvan,
> thanks but I'm not sure that doc addresses my question. You
> effectively filter out npfec_kind_in_gpt and npfec_kind_unknown in
> this patch. The first, npfec_kind_in_gpt should only happen if you
> have restricted access to the gpt with ept and the processor couldn't
> walk the table. But if you don't want to get events of these types
> then why not simply not restrict access the gpt to begin with? And as
> for npfec_kind_unknown, I don't think that gets generated under any
> situation. So hence my question, what is your setup that makes this
> patch necessary?

On the npfec_kind_unknown case, indeed, we were wondering when that
might possibly occur when discussing this patch - it's probably reserved
for the future?

On why our introspection engine decides to restrict access to those
specific pages, I am not intimate with its inner workings, and not sure
how much could be disclosed here in any case. Is it not a worthwhile
(and otherwise harmless) tool to be able to switch A/D bits-triggered
EPT faults anyway, for introspection purposes?


Thanks,
Razvan

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH for-next 9/9] coverage: add documentation for LLVM coverage

[Roger Pau Monné]

On Mon, Oct 30, 2017 at 04:48:09PM +, Wei Liu wrote:
> On Thu, Oct 26, 2017 at 10:19:38AM +0100, Roger Pau Monne wrote:
> > Signed-off-by: Roger Pau Monné 
> > ---
> > Cc: Andrew Cooper 
> > Cc: George Dunlap 
> > Cc: Ian Jackson 
> > Cc: Jan Beulich 
> > Cc: Konrad Rzeszutek Wilk 
> > Cc: Stefano Stabellini 
> > Cc: Tim Deegan 
> > Cc: Wei Liu 
> > ---
> >  docs/misc/coverage.markdown | 47 
> > +
> >  1 file changed, 47 insertions(+)
> > 
> > diff --git a/docs/misc/coverage.markdown b/docs/misc/coverage.markdown
> > index 0a32c48f4b..565644631a 100644
> > --- a/docs/misc/coverage.markdown
> > +++ b/docs/misc/coverage.markdown
> > @@ -8,6 +8,8 @@ information. Every basic block in the code will be 
> > instrumented by the compiler
> >  to compute these statistics. It should not be used in production as it 
> > slows
> >  down your hypervisor.
> >  
> > +# GCOV (GCC coverage)
> > +
> >  ## Enable coverage
> >  
> >  Test coverage support can be turned on compiling Xen with the `coverage` 
> > option set
> > @@ -87,3 +89,48 @@ blob extracted from xencov!**
> >  * See output in a browser
> >  
> >  firefox cov/index.html
> > +
> > +# LLVM coverage
> > +
> > +## Enable coverage
> > +
> > +Coverage can be enabled using a Kconfig option, from the top-level 
> > directory
> > +use the following command to display the Kconfig menu:
> > +
> > +gmake -C xen menuconfig clang=y
> > +
> > +The LLVM coverage option can be found inside of the "Debugging Options"
> > +section. After enabling it just compile Xen as you would normally do:
> > +
> > +   gmake xen clang=y
> > +
> 
> It can be a bit confusing when make and gmake appear in the same
> document. I suggest you use make all over and add a footnote saying it
> should be gmake on FreeBSD.

This is my fault from copying the runes from my command line. I'm not
sure a footnote is even needed, everything on BSDs should use gmake
instead of make, and I don't plan to edit all the documentation files.

Thanks, Roger.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH for-next 5/9] coverage: introduce generic file

[Roger Pau Monné]

On Mon, Oct 30, 2017 at 04:48:21PM +, Wei Liu wrote:
> On Thu, Oct 26, 2017 at 10:19:34AM +0100, Roger Pau Monne wrote:
> > 
> > diff --git a/xen/common/coverage/Makefile b/xen/common/coverage/Makefile
> > index f68d050eca..0e0510679e 100644
> > --- a/xen/common/coverage/Makefile
> > +++ b/xen/common/coverage/Makefile
> > @@ -1,4 +1,4 @@
> > -obj-y += gcov_base.o gcov.o
> > +obj-y += gcov_base.o gcov.o coverage.o
> >  obj-$(CONFIG_GCOV_FORMAT_3_4) += gcc_3_4.o
> >  obj-$(CONFIG_GCOV_FORMAT_4_7) += gcc_4_7.o
> >  obj-$(CONFIG_GCOV_FORMAT_4_9) += gcc_4_9.o
> > diff --git a/xen/common/coverage/coverage.c b/xen/common/coverage/coverage.c
> > new file mode 100644
> > index 00..1dec6944be
> > --- /dev/null
> > +++ b/xen/common/coverage/coverage.c
> > @@ -0,0 +1,71 @@
> > +/*
> > + * Generic functionality for coverage analysis.
> > + *
> > + * Copyright (C) 2017 Citrix Systems R
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify it under the terms and conditions of the GNU General Public
> > + * License, version 2, as published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> > + * General Public License for more details.
> > + *
> > + * You should have received a copy of the GNU General Public
> > + * License along with this program; If not, see 
> > .
> > + */
> > +
> > +#include 
> > +#include 
> > +#include 
> > +#include 
> 
> Please sort this.

OK, I will have to include type.h in coverage.h then.

Thanks, Roger.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH for-next 7/9] coverage: introduce support for llvm profiling

[Wei Liu]

On Thu, Oct 26, 2017 at 10:19:36AM +0100, Roger Pau Monne wrote:
> Introduce the functionality in order to fill the hooks of the
> cov_sysctl_ops struct.
> 
> Signed-off-by: Roger Pau Monné 
> ---
> Cc: Andrew Cooper 
> Cc: George Dunlap 
> Cc: Ian Jackson 
> Cc: Jan Beulich 
> Cc: Konrad Rzeszutek Wilk 
> Cc: Stefano Stabellini 
> Cc: Tim Deegan 
> Cc: Wei Liu 
> Cc: llvm-...@lists.llvm.org
> ---
> Note that the file that contains the helpers is under a BSD 2-clause
> license. This is done so it can be shared with other OSes that use the
> llvm/clang compiler.

It would be helpful if you can put a reference to the original code
somewhere, either in commit message or in the comment section of the
file.

> ---
>  xen/common/coverage/Makefile |   2 +-
>  xen/common/coverage/llvm.c   | 148 
> +++
>  xen/include/public/sysctl.h  |   6 ++
>  3 files changed, 155 insertions(+), 1 deletion(-)
>  create mode 100644 xen/common/coverage/llvm.c
> 
> diff --git a/xen/common/coverage/Makefile b/xen/common/coverage/Makefile
> index e4541a1233..f2ffb2b8de 100644
> --- a/xen/common/coverage/Makefile
> +++ b/xen/common/coverage/Makefile
> @@ -11,5 +11,5 @@ obj-$(CONFIG_GCOV_FORMAT_AUTODETECT) += $(call 
> cc-ifversion,lt,0x040700, \
>   gcc_4_9.o, $(call 
> cc-ifversion,lt,0x07, \
>   gcc_5.o, gcc_7.o
>  else
> -obj-y += coverage.o
> +obj-y += llvm.o coverage.o
>  endif
> diff --git a/xen/common/coverage/llvm.c b/xen/common/coverage/llvm.c
> new file mode 100644
> index 00..c8905070a3
> --- /dev/null
> +++ b/xen/common/coverage/llvm.c
> +#include 
> +#include 
> +#include 
> +#include 
> +

Order.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH for-next 5/9] coverage: introduce generic file

[Wei Liu]

On Thu, Oct 26, 2017 at 10:19:34AM +0100, Roger Pau Monne wrote:
> 
> diff --git a/xen/common/coverage/Makefile b/xen/common/coverage/Makefile
> index f68d050eca..0e0510679e 100644
> --- a/xen/common/coverage/Makefile
> +++ b/xen/common/coverage/Makefile
> @@ -1,4 +1,4 @@
> -obj-y += gcov_base.o gcov.o
> +obj-y += gcov_base.o gcov.o coverage.o
>  obj-$(CONFIG_GCOV_FORMAT_3_4) += gcc_3_4.o
>  obj-$(CONFIG_GCOV_FORMAT_4_7) += gcc_4_7.o
>  obj-$(CONFIG_GCOV_FORMAT_4_9) += gcc_4_9.o
> diff --git a/xen/common/coverage/coverage.c b/xen/common/coverage/coverage.c
> new file mode 100644
> index 00..1dec6944be
> --- /dev/null
> +++ b/xen/common/coverage/coverage.c
> @@ -0,0 +1,71 @@
> +/*
> + * Generic functionality for coverage analysis.
> + *
> + * Copyright (C) 2017 Citrix Systems R
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms and conditions of the GNU General Public
> + * License, version 2, as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public
> + * License along with this program; If not, see 
> .
> + */
> +
> +#include 
> +#include 
> +#include 
> +#include 

Please sort this.

The rest appears to be pure code motion to me. It looks fine.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH for-next 3/9] gcov: rename sysctl and functions

[Wei Liu]

On Thu, Oct 26, 2017 at 10:19:32AM +0100, Roger Pau Monne wrote:
> Change gcov to cov.
> 
> Signed-off-by: Roger Pau Monné 

Acked-by: Wei Liu 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH for-next 2/9] gcov: rename folder and header to coverage

[Wei Liu]

> diff --git a/xen/include/xen/gcov.h b/xen/include/xen/coverage.h
> similarity index 100%
> rename from xen/include/xen/gcov.h
> rename to xen/include/xen/coverage.h

Please also rename the define guard in this file.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH for-next 9/9] coverage: add documentation for LLVM coverage

[Wei Liu]

On Thu, Oct 26, 2017 at 10:19:38AM +0100, Roger Pau Monne wrote:
> Signed-off-by: Roger Pau Monné 
> ---
> Cc: Andrew Cooper 
> Cc: George Dunlap 
> Cc: Ian Jackson 
> Cc: Jan Beulich 
> Cc: Konrad Rzeszutek Wilk 
> Cc: Stefano Stabellini 
> Cc: Tim Deegan 
> Cc: Wei Liu 
> ---
>  docs/misc/coverage.markdown | 47 
> +
>  1 file changed, 47 insertions(+)
> 
> diff --git a/docs/misc/coverage.markdown b/docs/misc/coverage.markdown
> index 0a32c48f4b..565644631a 100644
> --- a/docs/misc/coverage.markdown
> +++ b/docs/misc/coverage.markdown
> @@ -8,6 +8,8 @@ information. Every basic block in the code will be 
> instrumented by the compiler
>  to compute these statistics. It should not be used in production as it slows
>  down your hypervisor.
>  
> +# GCOV (GCC coverage)
> +
>  ## Enable coverage
>  
>  Test coverage support can be turned on compiling Xen with the `coverage` 
> option set
> @@ -87,3 +89,48 @@ blob extracted from xencov!**
>  * See output in a browser
>  
>  firefox cov/index.html
> +
> +# LLVM coverage
> +
> +## Enable coverage
> +
> +Coverage can be enabled using a Kconfig option, from the top-level directory
> +use the following command to display the Kconfig menu:
> +
> +gmake -C xen menuconfig clang=y
> +
> +The LLVM coverage option can be found inside of the "Debugging Options"
> +section. After enabling it just compile Xen as you would normally do:
> +
> +   gmake xen clang=y
> +

It can be a bit confusing when make and gmake appear in the same
document. I suggest you use make all over and add a footnote saying it
should be gmake on FreeBSD.

This footnote should apply to both gcov and llvm-cov -- I believe you
should be able to build gcov support on FreeBSD, too.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v1] x86/mm: Supresses vm_events caused by page-walks

[Tamas K Lengyel]

On Mon, Oct 30, 2017 at 10:24 AM, Razvan Cojocaru
 wrote:
> On 30.10.2017 18:01, Tamas K Lengyel wrote:
>> On Mon, Oct 30, 2017 at 4:32 AM, Alexandru Isaila
>>  wrote:
>>> This patch is adding a way to enable/disable nested pagefault
>>> events. It introduces the xc_monitor_nested_pagefault function
>>> and adds the nested_pagefault_disabled in the monitor structure.
>>> This is needed by the introspection so it will only get gla
>>> faults and not get spammed with other faults.
>>> In p2m_set_ad_bits the v->arch.sse_pg_dirty.eip and
>>> v->arch.sse_pg_dirty.gla are used to mark that this is the
>>> second time a fault occurs and the dirty bit is set.
>>
>> Could you describe under what conditions do you get these other faults?
>
> Hey Tamas, the whole story is at page 8 of this document:
>
> https://www.researchgate.net/publication/281835515_Proposed_Processor_Extensions_for_Significant_Speedup_of_Hypervisor_Memory_Introspection

Hi Razvan,
thanks but I'm not sure that doc addresses my question. You
effectively filter out npfec_kind_in_gpt and npfec_kind_unknown in
this patch. The first, npfec_kind_in_gpt should only happen if you
have restricted access to the gpt with ept and the processor couldn't
walk the table. But if you don't want to get events of these types
then why not simply not restrict access the gpt to begin with? And as
for npfec_kind_unknown, I don't think that gets generated under any
situation. So hence my question, what is your setup that makes this
patch necessary?

Thanks,
Tamas

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] tools/xenstored: Check number of strings passed to do_control()

[Wei Liu]

On Fri, Oct 27, 2017 at 04:32:15PM +, Pawel Wieczorkiewicz wrote:
> It is possible to send a zero-string message body to xenstore's
> XS_CONTROL handling function. Then the number of strings is used
> for an array allocation. This leads to a crash in strcmp() in a
> CONTROL sub-command invocation loop.
> The output of xs_count_string() should be verified and all 0 or
> negative values should be rejected with an EINVAL. At least the
> sub-command name must be specified.
> 
> The xenstore crash can only be triggered from within dom0 (there
> is a check in do_control() rejecting all non-dom0 requests with
> an EACCES).
> 
> Testing: reproduced with the following command:
> python -c 'print 16*"\x00"' | nc -U $XENSTORED_RUNDIR/socket
> 
> Signed-off-by: Pawel Wieczorkiewicz 
> Reviewed-by: Martin Pohlack 

Acked-by: Wei Liu 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [qemu-mainline test] 115390: regressions - FAIL

[osstest service owner]

flight 115390 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/115390/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-i3866 xen-buildfail REGR. vs. 114507
 build-amd64-xsm   6 xen-buildfail REGR. vs. 114507
 build-i386-xsm6 xen-buildfail REGR. vs. 114507
 build-amd64   6 xen-buildfail REGR. vs. 114507
 build-armhf   6 xen-buildfail REGR. vs. 114507
 build-armhf-xsm   6 xen-buildfail REGR. vs. 114507

Tests which did not succeed, but are not blocking:
 test-amd64-i386-libvirt-qcow2  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-amd64-i386-freebsd10-i386  1 build-check(1)   blocked  n/a
 test-amd64-amd64-qemuu-nested-intel  1 build-check(1)  blocked n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-armhf-armhf-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm  1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemuu-win10-i386  1 build-check(1)  blocked n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-amd64  1 build-check(1)   blocked  n/a
 test-amd64-amd64-pair 1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-credit2   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-win10-i386  1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-win7-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-pygrub   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-ws16-amd64  1 build-check(1)  blocked n/a
 test-amd64-amd64-xl-qcow2 1 build-check(1)   blocked  n/a
 test-amd64-amd64-amd64-pvgrub  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-arndale   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64  1 build-check(1) blocked n/a
 test-armhf-armhf-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl1 build-check(1)   blocked  n/a
 build-i386-libvirt1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-credit2   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-xsm1 build-check(1)   blocked  n/a
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsm  1 build-check(1)blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-raw1 build-check(1)   blocked  n/a
 test-amd64-i386-qemuu-rhel6hvm-amd  1 build-check(1)   blocked n/a
 test-amd64-amd64-i386-pvgrub  1 build-check(1)   blocked  n/a
 build-armhf-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-ws16-amd64  1 build-check(1) blocked n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvhv2-intel  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-xsm   1 build-check(1)   blocked  n/a
 test-amd64-i386-qemuu-rhel6hvm-intel  1 build-check(1) blocked n/a
 test-armhf-armhf-xl-vhd   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-win7-amd64  1 build-check(1)  blocked n/a
 test-amd64-amd64-xl   1 build-check(1)   blocked  n/a
 test-amd64-i386-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvhv2-amd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-qemuu-nested-amd  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-cubietruck  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-rtds  1 

Re: [Xen-devel] [PATCH v1] x86/mm: Supresses vm_events caused by page-walks

[Razvan Cojocaru]

On 30.10.2017 18:01, Tamas K Lengyel wrote:
> On Mon, Oct 30, 2017 at 4:32 AM, Alexandru Isaila
>  wrote:
>> This patch is adding a way to enable/disable nested pagefault
>> events. It introduces the xc_monitor_nested_pagefault function
>> and adds the nested_pagefault_disabled in the monitor structure.
>> This is needed by the introspection so it will only get gla
>> faults and not get spammed with other faults.
>> In p2m_set_ad_bits the v->arch.sse_pg_dirty.eip and
>> v->arch.sse_pg_dirty.gla are used to mark that this is the
>> second time a fault occurs and the dirty bit is set.
> 
> Could you describe under what conditions do you get these other faults?

Hey Tamas, the whole story is at page 8 of this document:

https://www.researchgate.net/publication/281835515_Proposed_Processor_Extensions_for_Significant_Speedup_of_Hypervisor_Memory_Introspection


Thanks,
Razvan

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v4 for-4.10] scripts: introduce a script for build test

[Wei Liu]

Signed-off-by: Ian Jackson 
Signed-off-by: Wei Liu 
---
Cc: Andrew Cooper 
Cc: George Dunlap 
Cc: Ian Jackson 
Cc: Jan Beulich 
Cc: Konrad Rzeszutek Wilk 
Cc: Stefano Stabellini 
Cc: Tim Deegan 
Cc: Wei Liu 
Cc: Julien Grall 
Cc: Anthony PERARD 

v4:
1. Check, save/restore $?.
2. Don't use trap, check exit code directly.
3. More error messages.

v3:
1. Use git-clean in default rune.
2. Print more friendly message.
3. Restore HEAD automatically.
---
 scripts/build-test.sh | 71 +++
 1 file changed, 71 insertions(+)
 create mode 100755 scripts/build-test.sh

diff --git a/scripts/build-test.sh b/scripts/build-test.sh
new file mode 100755
index 00..413540c13b
--- /dev/null
+++ b/scripts/build-test.sh
@@ -0,0 +1,71 @@
+#!/bin/sh
+
+# Run command on every commit within the range specified. If no command is
+# provided, use the default one to clean and build the whole tree.
+#
+# The default rune is rather simple. To do a cross-build, please put your usual
+# build rune in a shell script and invoke it with this script.
+
+if ! test -f xen/common/kernel.c; then
+echo "Please run this script from top-level directory"
+exit 1
+fi
+
+if test $# -lt 2 ; then
+echo "Usage: $0   [CMD]"
+exit 1
+fi
+
+status=`git status -s`
+if test -n "$status"; then
+echo "Tree is dirty, aborted"
+exit 1
+fi
+
+BASE=$1; shift
+TIP=$1; shift
+
+ORIG_BRANCH=`git symbolic-ref -q --short HEAD`
+if test $? -ne 0; then
+echo "Detached HEAD, aborted"
+exit 1
+fi
+
+git rev-list $BASE..$TIP | nl -ba | tac | \
+while read num rev; do
+echo "Testing $num $rev"
+
+git checkout $rev
+ret=$?
+if test $ret -ne 0; then
+echo "Failed to checkout $num $rev with $ret"
+exit $ret
+fi
+
+if test $# -eq 0 ; then
+git clean -fdx && ./configure && make -j4
+else
+"$@"
+fi
+ret=$?
+if test $ret -ne 0; then
+echo "Failed at $num $rev with $ret"
+exit $ret
+fi
+echo
+done
+
+ret=$?
+
+echo "Restoring original HEAD"
+git checkout $ORIG_BRANCH
+gco_ret=$?
+if test $gco_ret -ne 0; then
+echo "Failed to restore orignal HEAD. Check tree status before doing 
anything else!"
+exit $gco_ret
+fi
+
+if test $ret -eq 0; then
+echo "ok."
+fi
+exit $ret
-- 
2.11.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v1] x86/mm: Supresses vm_events caused by page-walks

[Tamas K Lengyel]

On Mon, Oct 30, 2017 at 4:32 AM, Alexandru Isaila
 wrote:
> This patch is adding a way to enable/disable nested pagefault
> events. It introduces the xc_monitor_nested_pagefault function
> and adds the nested_pagefault_disabled in the monitor structure.
> This is needed by the introspection so it will only get gla
> faults and not get spammed with other faults.
> In p2m_set_ad_bits the v->arch.sse_pg_dirty.eip and
> v->arch.sse_pg_dirty.gla are used to mark that this is the
> second time a fault occurs and the dirty bit is set.

Could you describe under what conditions do you get these other faults?

Thanks,
Tamas

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v3 for-4.10] scripts: introduce a script for build test

[Wei Liu]

On Mon, Oct 30, 2017 at 03:14:04PM +, Ian Jackson wrote:
> Wei Liu writes ("[PATCH v3 for-4.10] scripts: introduce a script for build 
> test"):
> > Signed-off-by: Ian Jackson 
> > Signed-off-by: Wei Liu 
> ...
> ...
> > +trap "echo Restoring original HEAD ; git checkout $ORIG_BRANCH" EXIT
> 
> This will smash the whole script's exit status.  I think you need to
> save/restore $?.  Be careful with your quoting.  Normally it is better
> for the argument to trap to be ''-quoted rather than "", to avoid it
> being expanded twice (and, the first time, too soon).
> 
> Also, if this fails, it leaves the failure message buried in a scrool
> of make -j4 output, where the user probably won't see it.  And it
> prints exactly the same message on success and failure.  On failure
> you should print the failing commitid, and exit nonzero.
> 
> On success you should print some reassuring `ok' message.
> 

Right. I've addressed your comments and will send out a new version
soon.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [xen-unstable test] 115378: regressions - FAIL

[osstest service owner]

flight 115378 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/115378/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stopfail REGR. vs. 114644
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stop   fail REGR. vs. 114644
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stop   fail REGR. vs. 114644

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qcow2 19 guest-start/debian.repeat fail in 115362 pass in 
115378
 test-amd64-i386-libvirt-qcow2 17 guest-start/debian.repeat fail pass in 115314
 test-amd64-i386-xl-raw   19 guest-start/debian.repeat  fail pass in 115345
 test-amd64-amd64-libvirt-vhd 17 guest-start/debian.repeat  fail pass in 115345
 test-armhf-armhf-xl-rtds  6 xen-installfail pass in 115362
 test-armhf-armhf-xl-credit2  16 guest-start/debian.repeat  fail pass in 115362
 test-armhf-armhf-xl-vhd  15 guest-start/debian.repeat  fail pass in 115362

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-examine 4 memdisk-try-append fail in 115314 blocked in 114644
 test-armhf-armhf-xl-rtds13 migrate-support-check fail in 115362 never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-check fail in 115362 never pass
 test-armhf-armhf-libvirt-xsm 14 saverestore-support-checkfail  like 114644
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 114644
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 114644
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 114644
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stopfail like 114644
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 114644
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 114644
 test-amd64-amd64-xl-pvhv2-intel 12 guest-start fail never pass
 test-amd64-amd64-xl-pvhv2-amd 12 guest-start  fail  never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qcow2 12 migrate-support-checkfail  never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop  fail never pass
 test-amd64-i386-xl-qemut-win10-i386 10 windows-install fail never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass
 test-amd64-amd64-xl-qemut-win10-i386 10 windows-installfail never pass

version targeted for testing:
 xen  bb2c1a1cc98a22e2d4c14b18421aa7be6c2adf0d
baseline version:
 xen  24fb44e971a62b345c7b6ca3c03b454a1e150abe

Last test of basis   114644  2017-10-17 10:49:11 Z   13 days
Failing since114670  2017-10-18 05:03:38 Z   12 days   19 attempts
Testing same since   115314  2017-10-28 05:53:13 Z2 days5 attempts

Re: [Xen-devel] [PATCH v7 13/13] xen: introduce a Kconfig option to enable the pvcalls frontend

[Boris Ostrovsky]

On 10/26/2017 04:45 PM, Boris Ostrovsky wrote:
> On 10/26/2017 04:16 PM, Stefano Stabellini wrote:
>> On Thu, 26 Oct 2017, Boris Ostrovsky wrote:
>>> On 10/26/2017 03:11 PM, Stefano Stabellini wrote:
 Also add pvcalls-front to the Makefile.

 Signed-off-by: Stefano Stabellini 
 CC: boris.ostrov...@oracle.com
 CC: jgr...@suse.com
>>> Reviewed-by: Boris Ostrovsky 
>> Thank you!!
>>
>> The series is fully acked now. I guess it could be added to xentip?
>> Maybe for v4.15?
>
> Yes, that's the plan unless other reviews come in. I will probably
> create the branch on Monday (assuming rc7 will be the last rc for 4.14).
> It's later than usual but we haven't had anything for 4.15.
>
> -boris

Build warnings (gcc (GCC) 6.3.1 20161221 (Red Hat 6.3.1-1))

/data/upstream/linux-xen/drivers/xen/pvcalls-front.c: In function
‘__write_ring’:
/data/upstream/linux-xen/drivers/xen/pvcalls-front.c:459:3: warning:
ignoring return value of ‘copy_from_iter’, declared with attribute
warn_unused_result [-Wunused-result]
   copy_from_iter(data->out + masked_prod, len, msg_iter);
   ^~
/data/upstream/linux-xen/drivers/xen/pvcalls-front.c:462:4: warning:
ignoring return value of ‘copy_from_iter’, declared with attribute
warn_unused_result [-Wunused-result]
copy_from_iter(data->out + masked_prod,
^~~
array_size - masked_prod, msg_iter);
~~~
/data/upstream/linux-xen/drivers/xen/pvcalls-front.c:464:4: warning:
ignoring return value of ‘copy_from_iter’, declared with attribute
warn_unused_result [-Wunused-result]
copy_from_iter(data->out,
^
len - (array_size - masked_prod),
~
msg_iter);
~
/data/upstream/linux-xen/drivers/xen/pvcalls-front.c:468:4: warning:
ignoring return value of ‘copy_from_iter’, declared with attribute
warn_unused_result [-Wunused-result]
copy_from_iter(data->out + masked_prod, len, msg_iter);
^~
/data/upstream/linux-xen/drivers/xen/pvcalls-front.c: In function
‘__read_ring’:
/data/upstream/linux-xen/drivers/xen/pvcalls-front.c:560:3: warning:
ignoring return value of ‘copy_to_iter’, declared with attribute
warn_unused_result [-Wunused-result]
   copy_to_iter(data->in + masked_cons, len, msg_iter);
   ^~~
/data/upstream/linux-xen/drivers/xen/pvcalls-front.c:563:4: warning:
ignoring return value of ‘copy_to_iter’, declared with attribute
warn_unused_result [-Wunused-result]
copy_to_iter(data->in + masked_cons,
^~~~
  array_size - masked_cons, msg_iter);
  ~~~
/data/upstream/linux-xen/drivers/xen/pvcalls-front.c:565:4: warning:
ignoring return value of ‘copy_to_iter’, declared with attribute
warn_unused_result [-Wunused-result]
copy_to_iter(data->in,
^~
  len - (array_size - masked_cons),
  ~
  msg_iter);
  ~
/data/upstream/linux-xen/drivers/xen/pvcalls-front.c:569:4: warning:
ignoring return value of ‘copy_to_iter’, declared with attribute
warn_unused_result [-Wunused-result]
copy_to_iter(data->in + masked_cons, len, msg_iter);
^~~




Slightly different on 32 bit:

/data/upstream/linux-xen/drivers/xen/pvcalls-front.c: In function
‘pvcalls_front_event_handler’:
/data/upstream/linux-xen/drivers/xen/pvcalls-front.c:160:31: warning:
cast to pointer from integer of different size [-Wint-to-pointer-cast]
struct sock_mapping *map = (struct sock_mapping *)
   ^
/data/upstream/linux-xen/drivers/xen/pvcalls-front.c: In function
‘pvcalls_front_socket’:
/data/upstream/linux-xen/drivers/xen/pvcalls-front.c:283:21: warning:
cast from pointer to integer of different size [-Wpointer-to-int-cast]
  req->u.socket.id = (uint64_t) map;
 ^
/data/upstream/linux-xen/drivers/xen/pvcalls-front.c: In function
‘pvcalls_front_connect’:
/data/upstream/linux-xen/drivers/xen/pvcalls-front.c:405:22: warning:
cast from pointer to integer of different size [-Wpointer-to-int-cast]
  req->u.connect.id = (uint64_t)map;
  ^
/data/upstream/linux-xen/drivers/xen/pvcalls-front.c: In function
‘pvcalls_front_bind’:
/data/upstream/linux-xen/drivers/xen/pvcalls-front.c:660:19: warning:
cast from pointer to integer of different size [-Wpointer-to-int-cast]
  req->u.bind.id = (uint64_t)map;
   ^
/data/upstream/linux-xen/drivers/xen/pvcalls-front.c: In function
‘pvcalls_front_listen’:
/data/upstream/linux-xen/drivers/xen/pvcalls-front.c:722:21: warning:
cast from pointer to integer of different size 

Re: [Xen-devel] [PATCH v3 for-4.10] scripts: introduce a script for build test

[Ian Jackson]

Wei Liu writes ("[PATCH v3 for-4.10] scripts: introduce a script for build 
test"):
> Signed-off-by: Ian Jackson 
> Signed-off-by: Wei Liu 
...
...
> +trap "echo Restoring original HEAD ; git checkout $ORIG_BRANCH" EXIT

This will smash the whole script's exit status.  I think you need to
save/restore $?.  Be careful with your quoting.  Normally it is better
for the argument to trap to be ''-quoted rather than "", to avoid it
being expanded twice (and, the first time, too soon).

Also, if this fails, it leaves the failure message buried in a scrool
of make -j4 output, where the user probably won't see it.  And it
prints exactly the same message on success and failure.  On failure
you should print the failing commitid, and exit nonzero.

On success you should print some reassuring `ok' message.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH for-4.10] tools/hotplug: create XEN_LOG_DIR at runtime

[Ian Jackson]

Wei Liu writes ("Re: [PATCH] tools/hotplug: create XEN_LOG_DIR at runtime"):
> On Fri, Oct 27, 2017 at 07:52:37PM +0300, Andrii Anisov wrote:
> > From: Andrii Anisov 
> > 
> > /var/log could be a tmpfs mount point, so create xen subfolder at runtime.
> > 
> > Signed-off-by: Andrii Anisov 
> > Reviewed-by: Volodymyr Babchuk 
> > Reviewed-by: Oleksandr Andrushchenko 
> 
> Acked-by: Wei Liu 
> 
> Julien I think we should apply this for 4.10.

I agree.  Subject line tag added.

Acked-by: Ian Jackson 

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v3 for-4.10] scripts: introduce a script for build test

[Wei Liu]

On Wed, Oct 25, 2017 at 05:00:21PM +0100, Wei Liu wrote:
> Signed-off-by: Ian Jackson 
> Signed-off-by: Wei Liu 
> ---
> Cc: Andrew Cooper 
> Cc: George Dunlap 
> Cc: Ian Jackson 
> Cc: Jan Beulich 
> Cc: Konrad Rzeszutek Wilk 
> Cc: Stefano Stabellini 
> Cc: Tim Deegan 
> Cc: Wei Liu 
> Cc: Julien Grall 
> Cc: Anthony PERARD 
> 
> v3:
> 1. Use git-clean in default rune.
> 2. Print more friendly message.
> 3. Restore HEAD automatically.

Ping?

> ---
>  scripts/build-test.sh | 50 ++
>  1 file changed, 50 insertions(+)
>  create mode 100755 scripts/build-test.sh
> 
> diff --git a/scripts/build-test.sh b/scripts/build-test.sh
> new file mode 100755
> index 00..f55ec5d4fa
> --- /dev/null
> +++ b/scripts/build-test.sh
> @@ -0,0 +1,50 @@
> +#!/bin/sh
> +
> +# Run command on every commit within the range specified. If no command is
> +# provided, use the default one to clean and build the whole tree.
> +#
> +# The default rune is rather simple. To do a cross-build, please put your 
> usual
> +# build rune in a shell script and invoke it with this script.
> +
> +if ! test -f xen/common/kernel.c; then
> +echo "Please run this script from top-level directory"
> +exit 1
> +fi
> +
> +if test $# -lt 2 ; then
> +echo "Usage: $0   [CMD]"
> +exit 1
> +fi
> +
> +status=`git status -s`
> +if test -n "$status"; then
> +echo "Tree is dirty, aborted"
> +exit 1
> +fi
> +
> +BASE=$1; shift
> +TIP=$1; shift
> +
> +ORIG_BRANCH=`git symbolic-ref -q --short HEAD`
> +if test $? -ne 0; then
> +echo "Detached HEAD, aborted"
> +exit 1
> +fi
> +
> +trap "echo Restoring original HEAD ; git checkout $ORIG_BRANCH" EXIT
> +
> +git rev-list $BASE..$TIP | nl -ba | tac | \
> +while read num rev; do
> +echo "Testing $num $rev"
> +git checkout $rev
> +if test $# -eq 0 ; then
> +git clean -fdx && ./configure && make -j4
> +else
> +"$@"
> +fi
> +if test $? -ne 0; then
> +echo "Failed at $num $rev"
> +exit 1
> +fi
> +echo
> +done
> -- 
> 2.11.0
> 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v5 1/1] xen/time: do not decrease steal time after live migration on xen

[Boris Ostrovsky]

On 10/30/2017 04:03 AM, Dongli Zhang wrote:
> After guest live migration on xen, steal time in /proc/stat
> (cpustat[CPUTIME_STEAL]) might decrease because steal returned by
> xen_steal_lock() might be less than this_rq()->prev_steal_time which is
> derived from previous return value of xen_steal_clock().
>
> For instance, steal time of each vcpu is 335 before live migration.
>
> cpu  198 0 368 200064 1962 0 0 1340 0 0
> cpu0 38 0 81 50063 492 0 0 335 0 0
> cpu1 65 0 97 49763 634 0 0 335 0 0
> cpu2 38 0 81 50098 462 0 0 335 0 0
> cpu3 56 0 107 50138 374 0 0 335 0 0
>
> After live migration, steal time is reduced to 312.
>
> cpu  200 0 370 200330 1971 0 0 1248 0 0
> cpu0 38 0 82 50123 500 0 0 312 0 0
> cpu1 65 0 97 49832 634 0 0 312 0 0
> cpu2 39 0 82 50167 462 0 0 312 0 0
> cpu3 56 0 107 50207 374 0 0 312 0 0
>
> Since runstate times are cumulative and cleared during xen live migration
> by xen hypervisor, the idea of this patch is to accumulate runstate times
> to global percpu variables before live migration suspend. Once guest VM is
> resumed, xen_get_runstate_snapshot_cpu() would always return the sum of new
> runstate times and previously accumulated times stored in global percpu
> variables.
>
> Similar and more severe issue would impact prior linux 4.8-4.10 as
> discussed by Michael Las at
> https://0xstubs.org/debugging-a-flaky-cpu-steal-time-counter-on-a-paravirtualized-xen-guest,
> which would overflow steal time and lead to 100% st usage in top command
> for linux 4.8-4.10. A backport of this patch would fix that issue.
>
> References: 
> https://0xstubs.org/debugging-a-flaky-cpu-steal-time-counter-on-a-paravirtualized-xen-guest
> Signed-off-by: Dongli Zhang 
>
> ---
> Changed since v1:
>   * relocate modification to xen_get_runstate_snapshot_cpu
>
> Changed since v2:
>   * accumulate runstate times before live migration
>
> Changed since v3:
>   * do not accumulate times in the case of guest checkpointing
>
> Changed since v4:
>   * allocate array of vcpu_runstate_info to reduce number of memory allocation
>
> ---
>  drivers/xen/manage.c |  2 ++
>  drivers/xen/time.c   | 68 
> ++--
>  include/xen/interface/vcpu.h |  2 ++
>  include/xen/xen-ops.h|  1 +
>  4 files changed, 71 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/xen/manage.c b/drivers/xen/manage.c
> index c425d03..3dc085d 100644
> --- a/drivers/xen/manage.c
> +++ b/drivers/xen/manage.c
> @@ -72,6 +72,7 @@ static int xen_suspend(void *data)
>   }
>  
>   gnttab_suspend();
> + xen_accumulate_runstate_time(-1);
>   xen_arch_pre_suspend();
>  
>   /*
> @@ -84,6 +85,7 @@ static int xen_suspend(void *data)
> : 0);
>  
>   xen_arch_post_suspend(si->cancelled);
> + xen_accumulate_runstate_time(si->cancelled);

I am not convinced that the comment above HYPERVISOR_suspend() is
correct. The call can return an error code and so if it returns -EPERM
(which AFAICS it can't now but might in the future) then
xen_accumulate_runstate_time() will do wrong thing.


>   gnttab_resume();
>  
>   if (!si->cancelled) {
> diff --git a/drivers/xen/time.c b/drivers/xen/time.c
> index ac5f23f..cf3afb9 100644
> --- a/drivers/xen/time.c
> +++ b/drivers/xen/time.c
> @@ -19,6 +19,9 @@
>  /* runstate info updated by Xen */
>  static DEFINE_PER_CPU(struct vcpu_runstate_info, xen_runstate);
>  
> +static DEFINE_PER_CPU(u64[RUNSTATE_max], old_runstate_time);
> +static struct vcpu_runstate_info *runstate_delta;

I'd move this inside xen_accumulate_runstate_time() since that's the
only function that uses it. And why does it need to be
vcpu_runstate_info and not u64[4]?

> +
>  /* return an consistent snapshot of 64-bit time/counter value */
>  static u64 get64(const u64 *p)
>  {
> @@ -47,8 +50,8 @@ static u64 get64(const u64 *p)
>   return ret;
>  }
>  
> -static void xen_get_runstate_snapshot_cpu(struct vcpu_runstate_info *res,
> -   unsigned int cpu)
> +static void xen_get_runstate_snapshot_cpu_delta(
> + struct vcpu_runstate_info *res, unsigned int cpu)
>  {
>   u64 state_time;
>   struct vcpu_runstate_info *state;
> @@ -66,6 +69,67 @@ static void xen_get_runstate_snapshot_cpu(struct 
> vcpu_runstate_info *res,
>(state_time & XEN_RUNSTATE_UPDATE));
>  }
>  
> +static void xen_get_runstate_snapshot_cpu(struct vcpu_runstate_info *res,
> +   unsigned int cpu)
> +{
> + int i;
> +
> + xen_get_runstate_snapshot_cpu_delta(res, cpu);
> +
> + for (i = 0; i < RUNSTATE_max; i++)
> + res->time[i] += per_cpu(old_runstate_time, cpu)[i];
> +}
> +
> +void xen_accumulate_runstate_time(int action)
> +{
> + struct vcpu_runstate_info state;
> + int cpu, i;
> +
> + switch (action) {
> + case -1: /* backup runstate time before suspend */
> + 

[Xen-devel] [examine test] 115392: tolerable all pass

[osstest service owner]

flight 115392 examine real [real]
http://logs.test-lab.xenproject.org/osstest/logs/115392/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 examine-godello1  4 memdisk-try-append  fail blocked in 115152

baseline version:
 flight   115152

jobs:
 examine-baroque0 pass
 examine-baroque1 pass
 examine-arndale-bluewaterpass
 examine-cubietruck-braquepass
 examine-chardonnay0  pass
 examine-chardonnay1  pass
 examine-elbling0 pass
 examine-elbling1 pass
 examine-fiano0   pass
 examine-fiano1   pass
 examine-cubietruck-gleizes   pass
 examine-godello0 pass
 examine-godello1 pass
 examine-huxelrebe0   pass
 examine-huxelrebe1   pass
 examine-italia0  pass
 examine-italia1  pass
 examine-arndale-lakeside pass
 examine-merlot0  pass
 examine-merlot1  pass
 examine-arndale-metrocentre  pass
 examine-cubietruck-metzinger pass
 examine-nobling0 pass
 examine-nobling1 pass
 examine-nocera0  pass
 examine-nocera1  pass
 examine-cubietruck-picasso   pass
 examine-pinot0   pass
 examine-pinot1   pass
 examine-rimava0  pass
 examine-rimava1  pass
 examine-arndale-westfieldpass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Push not applicable.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v12 05/11] x86/mm: add HYPERVISOR_memory_op to acquire guest resources

[Paul Durrant]

> -Original Message-
> From: Julien Grall [mailto:julien.gr...@linaro.org]
> Sent: 30 October 2017 12:09
> To: Paul Durrant ; Jan Beulich
> 
> Cc: Julien Grall ; Andrew Cooper
> ; Wei Liu ; George
> Dunlap ; Ian Jackson ;
> Stefano Stabellini ; xen-de...@lists.xenproject.org;
> Konrad Rzeszutek Wilk ; Daniel De Graaf
> ; Tim (Xen.org) 
> Subject: Re: [Xen-devel] [PATCH v12 05/11] x86/mm: add
> HYPERVISOR_memory_op to acquire guest resources
> 
> Hi Paul,
> 
> On 27/10/17 16:19, Paul Durrant wrote:
> >> -Original Message-
> >> From: Julien Grall [mailto:julien.gr...@linaro.org]
> >> Sent: 27 October 2017 12:46
> >> To: Jan Beulich ; Paul Durrant
> >> 
> >> Cc: Julien Grall ; Andrew Cooper
> >> ; Wei Liu ; George
> >> Dunlap ; Ian Jackson
> ;
> >> Stefano Stabellini ; xen-
> de...@lists.xenproject.org;
> >> Konrad Rzeszutek Wilk ; Daniel De Graaf
> >> ; Tim (Xen.org) 
> >> Subject: Re: [Xen-devel] [PATCH v12 05/11] x86/mm: add
> >> HYPERVISOR_memory_op to acquire guest resources
> >>
> >> Hi,
> >>
> >> On 26/10/17 16:39, Jan Beulich wrote:
> >> On 26.10.17 at 17:32,  wrote:
>  On 26/10/17 16:26, Jan Beulich wrote:
>  On 17.10.17 at 15:24,  wrote:
> >> +/* IN/OUT - If the tools domain is PV then, upon return,
> frame_list
> >> + *  will be populated with the MFNs of the resource.
> >> + *  If the tools domain is HVM then it is expected that, 
> >> on
> >> + *  entry, frame_list will be populated with a list of 
> >> GFNs
> >> + *  that will be mapped to the MFNs of the resource.
> >> + *  If -EIO is returned then the frame_list has only been
> >> + *  partially mapped and it is up to the caller to unmap 
> >> all
> >> + *  the GFNs.
> >> + *  This parameter may be NULL if nr_frames is 0.
> >> + */
> >> +XEN_GUEST_HANDLE(xen_ulong_t) frame_list;
> >
> > This is still xen_ulong_t, which I can live with, but then you shouldn't
> > copy into / out of arrays of other types in acquire_resource() (the
> > more that this is common code, and iirc xen_ulong_t and
> > unsigned long aren't the same thing on ARM32).
> 
>  xen_ulong_t is always 64-bit on Arm (32-bit and 64-bit). But shouldn't
>  we use xen_pfn_t here?
> >>>
> >>> I had put this question up earlier, but iirc Paul didn't like it.
> >>
> >> I'd like to understand why Paul doesn't like it. We should never assume
> >> that a frame fit in xen_ulong_t. xen_pfn_t was exactly introduced for
> >> that purpose.
> >
> > My reservation is whether xen_pfn_t is intended to hold either gfns or
> mfns, since this hypercall uses the same array for both. If it suitable then 
> I am
> happy to change it, but Andrew led me to believe otherwise.
> 
> Looking at the public hearders, xen_pfn_t is been used for both MFN (see
> xenpf_add_memtype) and GFN (see gnttab_setup_table).
> 
> So I think it would be fine to do the same here.

Yes, I'm going to change it in the next version.

  Cheers,

Paul

> Cheers,
> 
> --
> Julien Grall
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 0/4] libxl: use libxl__device_kind string to access device

[Oleksandr Grytsov]

On Tue, Oct 24, 2017 at 10:41 AM, Oleksandr Grytsov 
wrote:

> On Thu, Oct 5, 2017 at 12:30 PM, Oleksandr Grytsov 
> wrote:
>
>> From: Oleksandr Grytsov 
>>
>> In current implementation the path of device XS entry is created with
>> string from libxl__device_kind enum. But access to the device entry
>> usually done with hardcoded path. This is source of potential errors.
>> This patchset changes hardcoded device name in the XS path to string
>> representation of libxl__device_kind enum. Also it changes "type" field
>> in libxl__..._devtype structure to keep libxl__device_kind. It allows
>> to move some duplicated functions to macros.
>>
>> Oleksandr Grytsov (4):
>>   libxl: use libxl__device_kind to get device XS entry
>>   libxl: use libxl__device_kind in LIBXL_DEFINE_UPDATE_DEVID
>>   libxl: move libxl__device_from_ to LIBXL_DEFINE_DEVICE_FROM_TYPE
>>   libxl: move ibxl_devid_to_device_... to LIBXL_DEFINE_DEVID_TO_DEVICE
>>
>>  tools/libxl/libxl_9pfs.c  | 21 +++-
>>  tools/libxl/libxl_colo_nic.c  |  6 ++--
>>  tools/libxl/libxl_console.c   | 36 +---
>>  tools/libxl/libxl_create.c|  4 +--
>>  tools/libxl/libxl_device.c| 10 +++---
>>  tools/libxl/libxl_disk.c  | 28 +++-
>>  tools/libxl/libxl_domain.c|  2 +-
>>  tools/libxl/libxl_internal.h  | 77 ++
>> ++---
>>  tools/libxl/libxl_netbuffer.c |  6 ++--
>>  tools/libxl/libxl_nic.c   | 55 +++
>>  tools/libxl/libxl_pci.c   | 21 
>>  tools/libxl/libxl_usb.c   | 52 +++--
>>  tools/libxl/libxl_vdispl.c| 62 ++
>>  tools/libxl/libxl_vkb.c   | 61 +-
>>  tools/libxl/libxl_vsnd.c  | 62 ++
>>  tools/libxl/libxl_vtpm.c  | 33 +++
>>  16 files changed, 222 insertions(+), 314 deletions(-)
>>
>> --
>> 2.7.4
>>
>>
> ping
>
> --
> Best Regards,
> Oleksandr Grytsov.
>

ping

-- 
Best Regards,
Oleksandr Grytsov.
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 0/5] libxl: add PV sound device

[Oleksandr Grytsov]

On Tue, Oct 24, 2017 at 10:27 AM, Oleksandr Grytsov 
wrote:

> On Mon, Oct 2, 2017 at 12:49 PM, Oleksandr Grytsov 
> wrote:
>
>> From: Oleksandr Grytsov 
>>
>> This patch set adds PV sound device support to xl.cfg and xl.
>> See sndif.h for protocol implementation details.
>>
>>
>> Oleksandr Grytsov (5):
>>   libxl: add PV sound device
>>   libxl: add vsnd list and info
>>   xl: add PV sound condif parser
>>   xl: add vsnd CLI commands
>>   docs: add PV sound device config
>>
>>  docs/man/xl.cfg.pod.5.in | 150 
>>  docs/man/xl.pod.1.in |  30 ++
>>  tools/libxl/Makefile |   2 +-
>>  tools/libxl/libxl.h  |  24 ++
>>  tools/libxl/libxl_create.c   |   1 +
>>  tools/libxl/libxl_internal.h |   1 +
>>  tools/libxl/libxl_types.idl  |  83 +
>>  tools/libxl/libxl_types_internal.idl |   1 +
>>  tools/libxl/libxl_utils.h|   3 +
>>  tools/libxl/libxl_vsnd.c | 660 ++
>> +
>>  tools/xl/Makefile|   2 +-
>>  tools/xl/xl.h|   3 +
>>  tools/xl/xl_cmdtable.c   |  15 +
>>  tools/xl/xl_parse.c  | 250 +
>>  tools/xl/xl_parse.h  |   1 +
>>  tools/xl/xl_vsnd.c   | 203 +++
>>  16 files changed, 1427 insertions(+), 2 deletions(-)
>>  create mode 100644 tools/libxl/libxl_vsnd.c
>>  create mode 100644 tools/xl/xl_vsnd.c
>>
>> --
>> 2.7.4
>>
>>
> ping
>
> --
> Best Regards,
> Oleksandr Grytsov.
>

ping

-- 
Best Regards,
Oleksandr Grytsov.
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 0/6] libxl: create standalone vkb device

[Oleksandr Grytsov]

On Tue, Oct 24, 2017 at 10:40 AM, Oleksandr Grytsov 
wrote:

> On Thu, Oct 5, 2017 at 12:07 PM, Oleksandr Grytsov 
> wrote:
>
>> From: Oleksandr Grytsov 
>>
>> Currently vkb device is the part of FB and console.
>> In embedded application we use vkb protocol to communicate
>> with user space backend. For this purpose we need to have
>> possibility to enable vkb device without QEMU, FB etc.
>>
>> This particular issue was already discussed int the mail
>> thread [1]. There were few possible solutions. We've implemented
>> one suggested by Stefano: add "type" field for vkb.
>> Each backend (QEMU or user space) shall read this field and
>> serve frontend only for own type. I will provide the patch
>> for QEMU backend, once this solution is submitted to libxl.
>>
>> This patchset consist of following changes:
>>
>> * vkb related code is moved to libxl_vkb.c - as it now
>>   used not only by console and FB;
>> * add backend type support in order to support QEMU and
>>   user space backends;
>> * add getting vkb list and getting device by id in order
>>   to implement CLI commands to attach, detach and list
>>   vkb devices;
>> * add new vkb entry in xl.cfg to handle separate vkb
>>   configuration;
>> * add CLI vkb-attach, vkb-detach and vkb-list commands;
>> * update documentation accordingly.
>>
>> [1] https://marc.info/?l=qemu-devel=149219237030212=2
>>
>> Oleksandr Grytsov (6):
>>   libxl: move vkb device to libxl_vkb.c
>>   libxl: fix vkb XS entry and type
>>   libxl: add backend type to vkb
>>   libxl: vkb add list and info functions
>>   xl: add vkb config parser and CLI
>>   docs: add vkb device to xl.cfg and xl
>>
>>  docs/man/xl.cfg.pod.5.in|  24 ++
>>  docs/man/xl.pod.1.in|  22 ++
>>  tools/libxl/Makefile|   1 +
>>  tools/libxl/libxl.h |  10 +++
>>  tools/libxl/libxl_console.c |  53 -
>>  tools/libxl/libxl_create.c  |   4 +
>>  tools/libxl/libxl_dm.c  |   2 +
>>  tools/libxl/libxl_types.idl |  18 +s
>>  tools/libxl/libxl_utils.h   |   3 +
>>  tools/libxl/libxl_vkb.c | 180 ++
>> ++
>>  tools/xl/Makefile   |   2 +-
>>  tools/xl/xl.h   |   3 +
>>  tools/xl/xl_cmdtable.c  |  15 
>>  tools/xl/xl_parse.c |  77 ++-
>>  tools/xl/xl_parse.h |   2 +-
>>  tools/xl/xl_vkb.c   | 141 ++
>>  16 files changed, 501 insertions(+), 56 deletions(-)
>>  create mode 100644 tools/libxl/libxl_vkb.c
>>  create mode 100644 tools/xl/xl_vkb.c
>>
>> --
>> 2.7.4
>>
>>
> ping
>
> --
> Best Regards,
> Oleksandr Grytsov.
>

ping

-- 
Best Regards,
Oleksandr Grytsov.
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] Libvirt config converter can't handle file not ending with new line

[Wei Liu]

Hi Jim

I discover a problem when using xen_xl converter. When the file in
question doesn't end with a new line, I get the following error:

  error: configuration file syntax error: memory conf:53: expecting a value


After digging a bit (but haven't read libvirt code), it appears that the
file didn't end with a new line.

I have worked around this, but I think this is an issue that should be
fixed.

Thanks,
Wei.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v12 05/11] x86/mm: add HYPERVISOR_memory_op to acquire guest resources

[Julien Grall]

Hi Paul,

On 27/10/17 16:19, Paul Durrant wrote:

-Original Message-
From: Julien Grall [mailto:julien.gr...@linaro.org]
Sent: 27 October 2017 12:46
To: Jan Beulich ; Paul Durrant

Cc: Julien Grall ; Andrew Cooper
; Wei Liu ; George
Dunlap ; Ian Jackson ;
Stefano Stabellini ; xen-de...@lists.xenproject.org;
Konrad Rzeszutek Wilk ; Daniel De Graaf
; Tim (Xen.org) 
Subject: Re: [Xen-devel] [PATCH v12 05/11] x86/mm: add
HYPERVISOR_memory_op to acquire guest resources

Hi,

On 26/10/17 16:39, Jan Beulich wrote:

On 26.10.17 at 17:32,  wrote:

On 26/10/17 16:26, Jan Beulich wrote:

On 17.10.17 at 15:24,  wrote:

+/* IN/OUT - If the tools domain is PV then, upon return, frame_list
+ *  will be populated with the MFNs of the resource.
+ *  If the tools domain is HVM then it is expected that, on
+ *  entry, frame_list will be populated with a list of GFNs
+ *  that will be mapped to the MFNs of the resource.
+ *  If -EIO is returned then the frame_list has only been
+ *  partially mapped and it is up to the caller to unmap all
+ *  the GFNs.
+ *  This parameter may be NULL if nr_frames is 0.
+ */
+XEN_GUEST_HANDLE(xen_ulong_t) frame_list;


This is still xen_ulong_t, which I can live with, but then you shouldn't
copy into / out of arrays of other types in acquire_resource() (the
more that this is common code, and iirc xen_ulong_t and
unsigned long aren't the same thing on ARM32).


xen_ulong_t is always 64-bit on Arm (32-bit and 64-bit). But shouldn't
we use xen_pfn_t here?


I had put this question up earlier, but iirc Paul didn't like it.


I'd like to understand why Paul doesn't like it. We should never assume
that a frame fit in xen_ulong_t. xen_pfn_t was exactly introduced for
that purpose.


My reservation is whether xen_pfn_t is intended to hold either gfns or mfns, 
since this hypercall uses the same array for both. If it suitable then I am 
happy to change it, but Andrew led me to believe otherwise.


Looking at the public hearders, xen_pfn_t is been used for both MFN (see 
xenpf_add_memtype) and GFN (see gnttab_setup_table).


So I think it would be fine to do the same here.

Cheers,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v12 05/11] x86/mm: add HYPERVISOR_memory_op to acquire guest resources

[Paul Durrant]

> -Original Message-
> From: Jan Beulich [mailto:jbeul...@suse.com]
> Sent: 26 October 2017 16:27
> To: Paul Durrant 
> Cc: Julien Grall ; Andrew Cooper
> ; Wei Liu ; George
> Dunlap ; Ian Jackson ;
> Stefano Stabellini ; xen-de...@lists.xenproject.org;
> Konrad Rzeszutek Wilk ; Daniel De Graaf
> ; Tim (Xen.org) 
> Subject: Re: [PATCH v12 05/11] x86/mm: add HYPERVISOR_memory_op to
> acquire guest resources
> 
> >>> On 17.10.17 at 15:24,  wrote:
> > @@ -535,6 +588,48 @@ int compat_memory_op(unsigned int cmd,
> XEN_GUEST_HANDLE_PARAM(void) compat)
> >  rc = -EFAULT;
> >  break;
> >
> > +case XENMEM_acquire_resource:
> > +{
> > +const xen_ulong_t *xen_frame_list =
> > +(xen_ulong_t *)(nat.mar + 1);
> > +compat_ulong_t *compat_frame_list =
> > +(compat_ulong_t *)(nat.mar + 1);
> > +
> > +if ( cmp.mar.nr_frames == 0 )
> 
> Doesn't this need to be compat_handle_is_null(cmp.mar.frame_list), or
> a combination of both?

Sorry, yes this was a hang-over from the old scheme.

> 
> > +{
> > +
> DEFINE_XEN_GUEST_HANDLE(compat_mem_acquire_resource_t);
> > +
> > +if ( __copy_field_to_guest(
> > + guest_handle_cast(compat,
> > +   compat_mem_acquire_resource_t),
> > + , nr_frames) )
> > +return -EFAULT;
> > +}
> > +else
> > +{
> > +/*
> > + * NOTE: the smaller compat array overwrites the native
> > + *   array.
> > + */
> 
> I think I had already asked for a respective BUILD_BUG_ON().

You asked for the comment. I can't find where you asked for a BUILD_BUG_ON() 
but I can certainly add one.

> 
> > --- a/xen/common/memory.c
> > +++ b/xen/common/memory.c
> > @@ -965,6 +965,95 @@ static long xatp_permission_check(struct domain
> *d, unsigned int space)
> >  return xsm_add_to_physmap(XSM_TARGET, current->domain, d);
> >  }
> >
> > +static int acquire_resource(
> > +XEN_GUEST_HANDLE_PARAM(xen_mem_acquire_resource_t) arg)
> > +{
> > +struct domain *d, *currd = current->domain;
> > +xen_mem_acquire_resource_t xmar;
> > +unsigned long mfn_list[2];
> > +int rc;
> > +
> > +if ( copy_from_guest(, arg, 1) )
> > +return -EFAULT;
> > +
> > +if ( xmar.pad != 0 )
> > +return -EINVAL;
> > +
> > +if ( guest_handle_is_null(xmar.frame_list) )
> > +{
> > +/* Special case for querying implementation limit */
> > +if ( xmar.nr_frames == 0 )
> 
> Perhaps invert the condition to reduce ...
> 
> > +{
> > +xmar.nr_frames = ARRAY_SIZE(mfn_list);
> > +
> > +if ( __copy_field_to_guest(arg, , nr_frames) )
> > +return -EFAULT;
> > +
> > +return 0;
> > +}
> 
> ... overall indentation?
> 
> > +return -EINVAL;
> > +}
> > +
> > +if ( xmar.nr_frames == 0 )
> > +return -EINVAL;
> 
> Why? (Almost?) everywhere else zero counts are simply no-ops, which
> result in success returns.

Ok, I'll drop the check.

> 
> > +if ( xmar.nr_frames > ARRAY_SIZE(mfn_list) )
> > +return -E2BIG;
> > +
> > +d = rcu_lock_domain_by_any_id(xmar.domid);
> 
> This being a tools only interface, why "by_any_id" instead of
> "remote_domain_by_id"? In particular ...
> 
> > +if ( d == NULL )
> > +return -ESRCH;
> > +
> > +rc = xsm_domain_resource_map(XSM_DM_PRIV, d);
> 
> ... an unprivileged dm domain should probably not be permitted to
> invoke this on itself.

True.

> 
> > +if ( rc )
> > +goto out;
> > +
> > +switch ( xmar.type )
> > +{
> > +default:
> > +rc = -EOPNOTSUPP;
> > +break;
> > +}
> > +
> > +if ( rc )
> > +goto out;
> > +
> > +if ( !paging_mode_translate(currd) )
> > +{
> > +if ( copy_to_guest(xmar.frame_list, mfn_list, xmar.nr_frames) )
> > +rc = -EFAULT;
> > +}
> > +else
> > +{
> > +xen_pfn_t gfn_list[ARRAY_SIZE(mfn_list)];
> > +unsigned int i;
> > +
> > +rc = -EFAULT;
> > +if ( copy_from_guest(gfn_list, xmar.frame_list, xmar.nr_frames) )
> > +goto out;
> > +
> > +for ( i = 0; i < xmar.nr_frames; i++ )
> > +{
> > +rc = set_foreign_p2m_entry(currd, gfn_list[i],
> > +   _mfn(mfn_list[i]));
> > +if ( rc )
> > +{
> > +/*
> > + * Make sure rc is -EIO for any interation other than
> > + * the first.
> 
> 

[Xen-devel] [distros-debian-sid test] 72376: tolerable FAIL

[Platform Team regression test user]

flight 72376 distros-debian-sid real [real]
http://osstest.xs.citrite.net/~osstest/testlogs/logs/72376/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-i386-i386-sid-netboot-pvgrub 10 debian-di-install   fail like 72341
 test-armhf-armhf-armhf-sid-netboot-pygrub 10 debian-di-install fail like 72341
 test-amd64-i386-amd64-sid-netboot-pygrub 10 debian-di-install  fail like 72341
 test-amd64-amd64-amd64-sid-netboot-pvgrub 10 debian-di-install fail like 72341
 test-amd64-amd64-i386-sid-netboot-pygrub 10 debian-di-install  fail like 72341

baseline version:
 flight   72341

jobs:
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-amd64-sid-netboot-pvgrubfail
 test-amd64-i386-i386-sid-netboot-pvgrub  fail
 test-amd64-i386-amd64-sid-netboot-pygrub fail
 test-armhf-armhf-armhf-sid-netboot-pygrubfail
 test-amd64-amd64-i386-sid-netboot-pygrub fail



sg-report-flight on osstest.xs.citrite.net
logs: /home/osstest/logs
images: /home/osstest/images

Logs, config files, etc. are available at
http://osstest.xs.citrite.net/~osstest/testlogs/logs

Test harness code can be found at
http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary


Push not applicable.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 2/5 v2] libxl: Change the type of console_mfn to xen_pfn_t

[Wei Liu]

On Mon, Oct 30, 2017 at 02:42:57PM +0530, Bhupinder Thakur wrote:
> Hi,
> 
> On 26 October 2017 at 16:47, Andrew Cooper  wrote:
> > On 26/10/17 12:13, Wei Liu wrote:
> >> On Wed, Oct 25, 2017 at 02:57:05PM +0530, Bhupinder Thakur wrote:
> >>> Currently the type of console mfn is unsigned long in libxl. This may be
> >>> an issue for 32-bit toolstack running on 64-bit Xen, where the pfn are
> >>> 64 bit. To ensure that console_mfn can hold any valid 64-bit pfn, the
> >>> type of console_mfn is changed to xen_pfn_t.
> >>>
> >>> Signed-off-by: Bhupinder Thakur 
> >>> ---
> >>> CC: Ian Jackson 
> >>> CC: Wei Liu 
> >>> CC: Stefano Stabellini 
> >>> CC: Julien Grall 
> >>>
> >>> This patch is as per the review of commit fa1f157
> >>> libxl: Fix the bug introduced in commit "libxl: use correct type
> >>>
> >>>  tools/libxl/libxl_console.c  | 2 +-
> >>>  tools/libxl/libxl_dom.c  | 2 +-
> >>>  tools/libxl/libxl_internal.h | 2 +-
> >>>  3 files changed, 3 insertions(+), 3 deletions(-)
> >>>
> >>> diff --git a/tools/libxl/libxl_console.c b/tools/libxl/libxl_console.c
> >>> index 6bfc0e5..f2ca689 100644
> >>> --- a/tools/libxl/libxl_console.c
> >>> +++ b/tools/libxl/libxl_console.c
> >>> @@ -329,7 +329,7 @@ int libxl__device_console_add(libxl__gc *gc, uint32_t 
> >>> domid,
> >>>  flexarray_append(ro_front, "port");
> >>>  flexarray_append(ro_front, GCSPRINTF("%"PRIu32, 
> >>> state->console_port));
> >>>  flexarray_append(ro_front, "ring-ref");
> >>> -flexarray_append(ro_front, GCSPRINTF("%lu", state->console_mfn));
> >>> +flexarray_append(ro_front, GCSPRINTF("%"PRIu_xen_pfn, 
> >>> state->console_mfn));
> >> Actually, please consider changing console_mfn to console_pfn.
> >
> > If you are going to make this change, then it is a gfn, not a pfn.
> > (console_pfn would be as equally wrong for PV guests as console_mfn is
> > currently wrong for HVM guest.)
> 
> Changing console_mfn to console_gfn will require changes in many
> files. Should I go ahead and change all the files?
> 

$ cd libxl && git grep \\bconsole_mfn | wc -l
14

Not too bad, so please go ahead.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v1] x86/mm: Supresses vm_events caused by page-walks

[Alexandru Isaila]

This patch is adding a way to enable/disable nested pagefault
events. It introduces the xc_monitor_nested_pagefault function
and adds the nested_pagefault_disabled in the monitor structure.
This is needed by the introspection so it will only get gla
faults and not get spammed with other faults.
In p2m_set_ad_bits the v->arch.sse_pg_dirty.eip and
v->arch.sse_pg_dirty.gla are used to mark that this is the
second time a fault occurs and the dirty bit is set.

Signed-off-by: Alexandru Isaila 
---
 tools/libxc/include/xenctrl.h |  2 ++
 tools/libxc/xc_monitor.c  | 14 ++
 xen/arch/x86/mm/mem_access.c  | 27 +++
 xen/arch/x86/monitor.c| 13 +
 xen/include/asm-x86/domain.h  |  6 ++
 xen/include/asm-x86/monitor.h |  3 ++-
 xen/include/public/domctl.h   |  1 +
 7 files changed, 65 insertions(+), 1 deletion(-)

diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
index 666db0b..8e70714 100644
--- a/tools/libxc/include/xenctrl.h
+++ b/tools/libxc/include/xenctrl.h
@@ -2056,6 +2056,8 @@ int xc_monitor_descriptor_access(xc_interface *xch, 
uint32_t domain_id,
  bool enable);
 int xc_monitor_guest_request(xc_interface *xch, uint32_t domain_id,
  bool enable, bool sync, bool allow_userspace);
+int xc_monitor_nested_pagefault(xc_interface *xch, uint32_t domain_id,
+bool disable);
 int xc_monitor_debug_exceptions(xc_interface *xch, uint32_t domain_id,
 bool enable, bool sync);
 int xc_monitor_cpuid(xc_interface *xch, uint32_t domain_id, bool enable);
diff --git a/tools/libxc/xc_monitor.c b/tools/libxc/xc_monitor.c
index 2840f14..5aacaa8 100644
--- a/tools/libxc/xc_monitor.c
+++ b/tools/libxc/xc_monitor.c
@@ -162,6 +162,20 @@ int xc_monitor_guest_request(xc_interface *xch, uint32_t 
domain_id, bool enable,
 return do_domctl(xch, );
 }
 
+int xc_monitor_nested_pagefault(xc_interface *xch, uint32_t domain_id,
+bool disable)
+{
+DECLARE_DOMCTL;
+
+domctl.cmd = XEN_DOMCTL_monitor_op;
+domctl.domain = domain_id;
+domctl.u.monitor_op.op = disable ? XEN_DOMCTL_MONITOR_OP_ENABLE
+: XEN_DOMCTL_MONITOR_OP_DISABLE;
+domctl.u.monitor_op.event = XEN_DOMCTL_MONITOR_EVENT_NESTED_PAGEFAULT;
+
+return do_domctl(xch, );
+}
+
 int xc_monitor_emulate_each_rep(xc_interface *xch, uint32_t domain_id,
 bool enable)
 {
diff --git a/xen/arch/x86/mm/mem_access.c b/xen/arch/x86/mm/mem_access.c
index c0cd017..07a334b 100644
--- a/xen/arch/x86/mm/mem_access.c
+++ b/xen/arch/x86/mm/mem_access.c
@@ -137,6 +137,23 @@ bool p2m_mem_access_emulate_check(struct vcpu *v,
 return violation;
 }
 
+static void p2m_set_ad_bits(struct vcpu *v, paddr_t ga)
+{
+struct hvm_hw_cpu ctxt;
+uint32_t pfec = 0;
+
+hvm_funcs.save_cpu_ctxt(v, );
+
+if ( guest_cpu_user_regs()->eip == v->arch.pg_dirty.eip
+ && ga == v->arch.pg_dirty.gla )
+pfec = PFEC_write_access;
+
+paging_ga_to_gfn_cr3(v, ctxt.cr3, ga, , NULL);
+
+v->arch.pg_dirty.eip = guest_cpu_user_regs()->eip;
+v->arch.pg_dirty.gla = ga;
+}
+
 bool p2m_mem_access_check(paddr_t gpa, unsigned long gla,
   struct npfec npfec,
   vm_event_request_t **req_ptr)
@@ -208,6 +225,16 @@ bool p2m_mem_access_check(paddr_t gpa, unsigned long gla,
 }
 }
 
+if ( vm_event_check_ring(d->vm_event_monitor) &&
+ d->arch.monitor.nested_pagefault_disabled &&
+ npfec.kind != npfec_kind_with_gla ) /* don't send a mem_event */
+{
+v->arch.vm_event->emulate_flags = 0;
+p2m_set_ad_bits(v, gla);
+
+return true;
+}
+
 *req_ptr = NULL;
 req = xzalloc(vm_event_request_t);
 if ( req )
diff --git a/xen/arch/x86/monitor.c b/xen/arch/x86/monitor.c
index e59f1f5..3916e76 100644
--- a/xen/arch/x86/monitor.c
+++ b/xen/arch/x86/monitor.c
@@ -220,6 +220,19 @@ int arch_monitor_domctl_event(struct domain *d,
 break;
 }
 
+case XEN_DOMCTL_MONITOR_EVENT_NESTED_PAGEFAULT:
+{
+bool old_status = ad->monitor.nested_pagefault_disabled;
+
+if ( unlikely(old_status == requested_status) )
+return -EEXIST;
+
+domain_pause(d);
+ad->monitor.nested_pagefault_disabled = requested_status;
+domain_unpause(d);
+break;
+}
+
 case XEN_DOMCTL_MONITOR_EVENT_DESC_ACCESS:
 {
 bool old_status = ad->monitor.descriptor_access_enabled;
diff --git a/xen/include/asm-x86/domain.h b/xen/include/asm-x86/domain.h
index 4d0b77d..40a365f 100644
--- a/xen/include/asm-x86/domain.h
+++ b/xen/include/asm-x86/domain.h
@@ -408,6 +408,7 @@ struct arch_domain
 unsigned int descriptor_access_enabled : 1;
 unsigned int 

[Xen-devel] [qemu-mainline test] 115384: regressions - FAIL

[osstest service owner]

flight 115384 qemu-mainline real [real]
http://logs.test-lab.xenproject.org/osstest/logs/115384/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-i3866 xen-buildfail REGR. vs. 114507
 build-amd64-xsm   6 xen-buildfail REGR. vs. 114507
 build-i386-xsm6 xen-buildfail REGR. vs. 114507
 build-amd64   6 xen-buildfail REGR. vs. 114507
 build-armhf   6 xen-buildfail REGR. vs. 114507
 build-armhf-xsm   6 xen-buildfail REGR. vs. 114507

Tests which did not succeed, but are not blocking:
 test-amd64-i386-libvirt-qcow2  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64  1 build-check(1)blocked n/a
 test-amd64-i386-freebsd10-i386  1 build-check(1)   blocked  n/a
 test-amd64-amd64-qemuu-nested-intel  1 build-check(1)  blocked n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-armhf-armhf-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm  1 build-check(1) blocked n/a
 test-amd64-i386-xl-qemuu-win10-i386  1 build-check(1)  blocked n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-i386-freebsd10-amd64  1 build-check(1)   blocked  n/a
 test-amd64-amd64-pair 1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-credit2   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-win10-i386  1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-xl-qemuu-win7-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-pygrub   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-ws16-amd64  1 build-check(1)  blocked n/a
 test-amd64-amd64-xl-qcow2 1 build-check(1)   blocked  n/a
 test-amd64-amd64-amd64-pvgrub  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-arndale   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-debianhvm-amd64  1 build-check(1) blocked n/a
 test-armhf-armhf-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl1 build-check(1)   blocked  n/a
 build-i386-libvirt1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-ovmf-amd64  1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-credit2   1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-multivcpu  1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-xsm1 build-check(1)   blocked  n/a
 build-amd64-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsm  1 build-check(1)blocked n/a
 test-amd64-i386-xl-qemuu-ovmf-amd64  1 build-check(1)  blocked n/a
 test-amd64-i386-xl-raw1 build-check(1)   blocked  n/a
 test-amd64-i386-qemuu-rhel6hvm-amd  1 build-check(1)   blocked n/a
 test-amd64-amd64-i386-pvgrub  1 build-check(1)   blocked  n/a
 build-armhf-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-qemuu-ws16-amd64  1 build-check(1) blocked n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvhv2-intel  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl   1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-xsm   1 build-check(1)   blocked  n/a
 test-amd64-i386-qemuu-rhel6hvm-intel  1 build-check(1) blocked n/a
 test-armhf-armhf-xl-vhd   1 build-check(1)   blocked  n/a
 test-amd64-i386-xl-qemuu-win7-amd64  1 build-check(1)  blocked n/a
 test-amd64-amd64-xl   1 build-check(1)   blocked  n/a
 test-amd64-i386-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-xl-pvhv2-amd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-qemuu-nested-amd  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-cubietruck  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl-rtds  1 

Re: [Xen-devel] [PATCH v1] tools/hotplug: convert proc-xen.mount to proc-xen.service

[Wei Liu]

On Thu, Oct 26, 2017 at 04:45:38PM +0100, Andrew Cooper wrote:
> On 26/10/17 16:25, Olaf Hering wrote:
> > An upcoming change in systemd will mount xenfs right away, along with
> > all other system mounts. This improves the detection of the
> > virtualization environment, which is currently racy. Some parts of
> > systemd rely on the presence of /proc/xen/capabilities, which will only
> > exist if xenfs is mounted. Since xenfs is mounted by the proc-xen.mount
> > unit, it will be processed very late. Other units may be processed
> > earlier, and if they make use of ConditionVirtualization*= failures may
> > occour.
> >
> > Unfortunately mounting xenfs by systemd as an API filesystem will lead
> > to errors when proc-xen.mount is processed. Since that mount point
> > already exists the unit is considered as failed, and other units that
> > depend on proc-xen.mount will not start. To avoid this the existing
> > proc-xen.mount will be converted into proc-xen.service, which just
> > mounts xenfs manually. All dependencies are updated by this change.
> >
> > The existing conditionals in proc-xen.mount will prevent failures with
> > existing systemd based installations:
> > ConditionPathExists=!/proc/xen/capabilities will prevent execution with
> > a new systemd that mounts xenfs. And this conditional, in combination
> > with ConditionPathExists=/proc/xen, will trigger execution with an old
> > systemd.
> >
> > An absolute path to the mount binary has to be used. /bin/mount is
> > expected to be universally available, nowaways it is a symlink to
> > /usr/bin/mount.
> >
> > Signed-off-by: Olaf Hering 
> 
> Can't all information be obtained from /sys/hypervisor?  If not, how
> hard would it be to make happen?

It is not possible to tell from /sys/hypervisor at the moment.

There is /sys/hypervisor/properties/features but that only contains
feature bits from hypervisor. And XENFEAT_dom0 is not a reliable
indicator for Dom0.

I suppose it requires a small mount of work to make /sys/hypervisor
contain such information, but that won't get propagated to older kernel
so we need to deal with the problem at hand nonetheless.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [locking/paravirt] static_key_disable_cpuslocked(): static key 'virt_spin_lock_key+0x0/0x20' used before call to jump_label_init()

[Fengguang Wu]

On Mon, Oct 30, 2017 at 09:38:35AM +0100, Fengguang Wu wrote:

On Mon, Oct 30, 2017 at 08:47:08AM +0100, Juergen Gross wrote:

On 30/10/17 08:35, Fengguang Wu wrote:

On Sun, Oct 29, 2017 at 11:51:55PM +0100, Fengguang Wu wrote:

Hi Linus,

Up to now we see the below boot error/warnings when testing v4.14-rc6.

They hit the RC release mainly due to various imperfections in 0day's
auto bisection. So I manually list them here and CC the likely easy to
debug ones to the corresponding maintainers in the followup emails.

boot_successes: 4700
boot_failures: 247


[...]


WARNING:at_kernel/jump_label.c:#static_key_disable_cpuslocked: 7


This patch is in the tip tree only, it will be merged in 4.15. So I
don't understand why you are reporting this for 4.14-rc6.


Ah sorry, I simply checked recent bisects for that warning.
Then I'll need to carry out some new bisect on 4.14-rc6.


As Peter found, it turns out to be a bug in 0day robot -- it stored
some dmesg files to a wrong place.

So RC6 is actually free from that bug. Sorry for the noise!

Fengguang

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH V3 2/29] VIOMMU: Add vIOMMU helper functions to create, destroy vIOMMU instance

[Roger Pau Monné]

On Mon, Oct 30, 2017 at 09:41:23AM +0800, Lan Tianyu wrote:
> On 2017年10月25日 09:43, Lan Tianyu wrote:
> >> For all platforms supporting HVM, for PV I don't think it makes sense.
> >> > Since AFAIK ARM guest type is also HVM I would rather introduce this
> >> > field in the hvm_domain structure rather than the generic domain
> >> > structure.
> >> > 
> > This sounds reasonable.
> > 
> >> > You might want to wait for feedback from others regarding this issue.
> >> > 
> > I discussed with Julien before. He hoped no to add viommu code for ARM
> > first.So struct hvm_domain seems to be better place since it's arch
> > specific definition and only add struct viommu for struct hvm_domain of x86.
> 
> Hi Roger:
>   If PV guest needs PV IOMMU support, struct iommu should be put  into
> struct domain and it can be reused by full-virtualization and PV iommu.
> Malcolm Crossley sent out RFC patch of pv iommu before. I found it also
> needs to change struct domain.
> 
> https://lists.xenproject.org/archives/html/xen-devel/2016-02/msg01441.html

This patch series is from February 2016: almost two years old and
there's been no further repost.

If this can indeed be shared with a future pv-iommu work, have you
checked whether the current structure data and hooks would be
suitable for a pv-iommu implementation?

I would rather prefer to move the viommu structure from hvm_domain to
the generic domain struct when it's actually needed (ie: when pv-iommu
is implemented) rather than doing it here.

Roger.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [linux-linus test] 115373: regressions - FAIL

[osstest service owner]

flight 115373 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/115373/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stop   fail REGR. vs. 114682

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 114682
 test-armhf-armhf-libvirt-xsm 14 saverestore-support-checkfail  like 114682
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 114682
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 114682
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 114682
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 114682
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stopfail like 114682
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stopfail like 114682
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qcow2 12 migrate-support-checkfail  never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  14 saverestore-support-checkfail   never pass
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop  fail never pass
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop  fail never pass
 test-amd64-amd64-xl-qemut-win10-i386 10 windows-installfail never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass
 test-amd64-i386-xl-qemut-win10-i386 10 windows-install fail never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass

version targeted for testing:
 linux0b07194bb55ed836c2cc7c22e866b87a14681984
baseline version:
 linuxebe6e90ccc6679cb01d2b280e4b61e6092d4bedb

Last test of basis   114682  2017-10-18 09:54:11 Z   11 days
Failing since114781  2017-10-20 01:00:47 Z   10 days   17 attempts
Testing same since   115373  2017-10-29 22:52:46 Z0 days1 attempts


419 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-armhf-pvops   

Re: [Xen-devel] [PATCH 2/5 v2] libxl: Change the type of console_mfn to xen_pfn_t

[Bhupinder Thakur]

Hi,

On 26 October 2017 at 16:47, Andrew Cooper  wrote:
> On 26/10/17 12:13, Wei Liu wrote:
>> On Wed, Oct 25, 2017 at 02:57:05PM +0530, Bhupinder Thakur wrote:
>>> Currently the type of console mfn is unsigned long in libxl. This may be
>>> an issue for 32-bit toolstack running on 64-bit Xen, where the pfn are
>>> 64 bit. To ensure that console_mfn can hold any valid 64-bit pfn, the
>>> type of console_mfn is changed to xen_pfn_t.
>>>
>>> Signed-off-by: Bhupinder Thakur 
>>> ---
>>> CC: Ian Jackson 
>>> CC: Wei Liu 
>>> CC: Stefano Stabellini 
>>> CC: Julien Grall 
>>>
>>> This patch is as per the review of commit fa1f157
>>> libxl: Fix the bug introduced in commit "libxl: use correct type
>>>
>>>  tools/libxl/libxl_console.c  | 2 +-
>>>  tools/libxl/libxl_dom.c  | 2 +-
>>>  tools/libxl/libxl_internal.h | 2 +-
>>>  3 files changed, 3 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/tools/libxl/libxl_console.c b/tools/libxl/libxl_console.c
>>> index 6bfc0e5..f2ca689 100644
>>> --- a/tools/libxl/libxl_console.c
>>> +++ b/tools/libxl/libxl_console.c
>>> @@ -329,7 +329,7 @@ int libxl__device_console_add(libxl__gc *gc, uint32_t 
>>> domid,
>>>  flexarray_append(ro_front, "port");
>>>  flexarray_append(ro_front, GCSPRINTF("%"PRIu32, 
>>> state->console_port));
>>>  flexarray_append(ro_front, "ring-ref");
>>> -flexarray_append(ro_front, GCSPRINTF("%lu", state->console_mfn));
>>> +flexarray_append(ro_front, GCSPRINTF("%"PRIu_xen_pfn, 
>>> state->console_mfn));
>> Actually, please consider changing console_mfn to console_pfn.
>
> If you are going to make this change, then it is a gfn, not a pfn.
> (console_pfn would be as equally wrong for PV guests as console_mfn is
> currently wrong for HVM guest.)

Changing console_mfn to console_gfn will require changes in many
files. Should I go ahead and change all the files?

Regards,
Bhupinder

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [BUG] xen_gntdev - gntdev_vma_find_special_page - unable to handle kernel paging request

[Arthur Borsboom]

Yes, this domain has:

- Current memory allocation: 8192 MB
- Maximum memory allocation: 10240 MB.

I have changed the maximum to 8192 MB as a possible workaround; the current
memory is rarely up/down scaled anyway.
The PVH setting for dom0 has been removed.

If you don't here from me anymore, then this is the workaround. :)
Thanks in advance.

On 30 October 2017 at 09:37, Juergen Gross  wrote:

> On 30/10/17 09:05, Arthur Borsboom wrote:
> > Hi Juergen,
> >
> > I needed to wait until the VM guest crashed again, so that caused this
> > delay.
> > I have extracted the hypervisor dmesg and the dom0 dmesg which can be
> > found in the PasteBins.
> > It is always the same domain crashing, if that is of any help.
> >
> > Xen hypervisor dmesg: https://pastebin.com/Y9fv0Ey0
>
> Aah, interesting. Do you happen to configure your domains with maxmem=
> larger than memory= ?
>
> I'm asking because the error messages seem to be related to another
> problem introduced in kernel 4.13 which is fixed in 4.14. This problem
> should be avoidable by setting maxmem= to the same value as memory=
> in the domain config of HVM domains.
>
> BTW: your hypervisor parameters are containing "dom0pvh=1". You should
> remove it as PVH support has been reworked and is not yet working for
> dom0.
>
>
> Juergen
>



-- 
Arthur Borsboom
Mob: +31629089953
Email: arthurborsb...@gmail.com
[image: View Arthur's LinkedIn profile]

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Xen 4.10.0 RC1 test result

[Hao, Xudong]

Hi Andrew

Our L1 is Windows 8 and HyperV version is 6.2.9200.16384, Hardware covered 
Skylake and Broadwell.

Here I want to correct "HyperV on Xen works", maybe "L1 Windows 8 with HyperV 
installed boot up successfully" is more accurate. One progress of Xen 4.10 is 
there is one issue: L1 Windows8 booted failed for years, this issue got fixed 
from Xen d23afa63 as our monitor.
We're doing L2 installation on Windows8 HyperV and I will update result here 
and wiki.

Thanks,
-Xudong

From: Andrew Cooper [mailto:andrew.coop...@citrix.com]
Sent: Friday, October 27, 2017 5:55 PM
To: Hao, Xudong ; xen-devel@lists.xen.org
Cc: Lars Kurth ; Julien Grall 
Subject: Re: [Xen-devel] Xen 4.10.0 RC1 test result

On 27/10/17 09:28, Hao, Xudong wrote:
We performed Xen 4.10 RC1 testing on Intel Xeon Skylake, Broadwell server, 
Intel Atom Denverton platforms, verified many functional features, which 
include new features Local MCE, L2 CAT and UMIP on Xen 4.10. We'd like to share 
the result out.

Most of features passed to testing on Xen 4.10 RC1, VT-d, RAS and nested has 
some bugs.
VT-d:
[BUG] win2008 guest cannot get ip through sriov 
https://www.mail-archive.com/xen-devel@lists.xen.org/msg127433.html

RAS:
[BUG] xen-mceinj tool testing cause dom0 crash 
https://www.mail-archive.com/xen-devel@lists.xen.org/msg108671.html

Nested:
Nested status is better than Xen 4.9.0, KVM on Xen, HyperV on Xen works, while 
Xen on Xen, VMware on Xen fail. 
https://wiki.xenproject.org/wiki/Nested_Virtualization_in_Xen

Do you have any further details on your HyperV scenarios, in particular 
versions of HyperV and the hardware involved, and guests booted under HyperV?

XenServers current nested-virt testing status shows a rather bleaker picture.

More modern version of Windows Server fail to initialise the HyperV role, 
because Xen doesn't advertise Virtual NMI support to L1.  (One version, Server 
2012 R2 I believe, indicates the same, but with a BSOD instead).  Older 
versions still do actually boot successfully.

When booting windows guests under nested HyperV, old versions appear to be 
stable with a single one-vcpu guest, but unstable with multiple vcpus or 
multiple single-vcpu guests.  The instability here is a VMEntry failure trying 
to inject an NMI, and occurs because HyperV and Xen disagree on whether to use 
Virtual NMI, resulting in HyperV thinking virtual NMI is disabled, but it is 
actually enabled in hardware.

When booting windows guests under more modern nested HyperV, the guest is 
crashing because of a pagefault when trying to access the APIC page.  We 
haven't tracked down the cause of this, but I expect it is something to do with 
emulating instruction while in nested vcpu context.

Thanks,

~Andrew
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [locking/paravirt] static_key_disable_cpuslocked(): static key 'virt_spin_lock_key+0x0/0x20' used before call to jump_label_init()

[Dou Liyang]

Hi Fengguang,

There are two different warning.

At 10/30/2017 03:35 PM, Fengguang Wu wrote:

On Sun, Oct 29, 2017 at 11:51:55PM +0100, Fengguang Wu wrote:

Hi Linus,

Up to now we see the below boot error/warnings when testing v4.14-rc6.



The original warning:

WARNING:at_arch/x86/events/intel/core.c:#intel_pmu_handle_irq: 1

in v4.14-rc6, which happens rarely.


They hit the RC release mainly due to various imperfections in 0day's
auto bisection. So I manually list them here and CC the likely easy to
debug ones to the corresponding maintainers in the followup emails.

boot_successes: 4700
boot_failures: 247


[...]


WARNING:at_kernel/jump_label.c:#static_key_disable_cpuslocked: 7




This is a new warning in tip tree, and the patch has sent[1].

[1]https://lkml.org/lkml/2017/10/28/4

we may need do it in v4.14-rc6, not in tip tree directly.

Thanks,
dou.



___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [locking/paravirt] static_key_disable_cpuslocked(): static key 'virt_spin_lock_key+0x0/0x20' used before call to jump_label_init()

[Fengguang Wu]

On Mon, Oct 30, 2017 at 08:47:08AM +0100, Juergen Gross wrote:

On 30/10/17 08:35, Fengguang Wu wrote:

On Sun, Oct 29, 2017 at 11:51:55PM +0100, Fengguang Wu wrote:

Hi Linus,

Up to now we see the below boot error/warnings when testing v4.14-rc6.

They hit the RC release mainly due to various imperfections in 0day's
auto bisection. So I manually list them here and CC the likely easy to
debug ones to the corresponding maintainers in the followup emails.

boot_successes: 4700
boot_failures: 247


[...]


WARNING:at_kernel/jump_label.c:#static_key_disable_cpuslocked: 7


This patch is in the tip tree only, it will be merged in 4.15. So I
don't understand why you are reporting this for 4.14-rc6.


Ah sorry, I simply checked recent bisects for that warning.
Then I'll need to carry out some new bisect on 4.14-rc6.


There is a patch by Dou Liyang pending since 28th October addressing
that issue:

[PATCH tip v2] x86/paravirt: Make the virt_spin_lock_key setup after
jump_label_init()


Excellent!

Thanks,
Fengguang

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [BUG] xen_gntdev - gntdev_vma_find_special_page - unable to handle kernel paging request

[Juergen Gross]

On 30/10/17 09:05, Arthur Borsboom wrote:
> Hi Juergen,
> 
> I needed to wait until the VM guest crashed again, so that caused this
> delay.
> I have extracted the hypervisor dmesg and the dom0 dmesg which can be
> found in the PasteBins.
> It is always the same domain crashing, if that is of any help.
> 
> Xen hypervisor dmesg: https://pastebin.com/Y9fv0Ey0

Aah, interesting. Do you happen to configure your domains with maxmem=
larger than memory= ?

I'm asking because the error messages seem to be related to another
problem introduced in kernel 4.13 which is fixed in 4.14. This problem
should be avoidable by setting maxmem= to the same value as memory=
in the domain config of HVM domains.

BTW: your hypervisor parameters are containing "dom0pvh=1". You should
remove it as PVH support has been reworked and is not yet working for
dom0.


Juergen

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel